We present Audio Flamingo 3 (AF3), a fully open state-of-the-art (SOTA) large audio-language model that advances reasoning and understanding across speech, sound, and music. AF3 introduces: (i) AF-Whisper, a unified audio encoder trained using a novel strategy for joint representation learning across all 3 modalities of speech, sound, and music; (ii) flexible, on-demand thinking, allowing the model to do chain-of-thought-type reasoning before answering; (iii) multi-turn, multi-audio chat; (iv) long audio understanding and reasoning (including speech) up to 10 minutes; and (v) voice-to-voice interaction. To enable these capabilities, we propose several large-scale training datasets curated using novel strategies, including AudioSkills-XL, LongAudio-XL, AF-Think, and AF-Chat, and train AF3 with a novel five-stage curriculum-based training strategy. Trained on only open-source audio data, AF3 achieves new SOTA results on over 20+ (long) audio understanding and reasoning benchmarks, surpassing both open-weight and closed-source models trained on much larger datasets.
The issue is just, I don’t see how any of those are arguments to distinguish between the two. I can twist them so almost every single argument applies to book authors and I don’t see any contradictions with that:
“You demand that AI companies [book authors] should work for free and give things away for free. But they also should pay [in content] people that make no contribution.”
“They do [create something], just like farmers. If people did not find their services beneficial, they would not pay [for the books].”
“[The price of the pile of books] is called a barrier to entry (Marktschranke).”
“It doesn’t have to be very bad. For example, you can’t just become a farmer [a big AI company]. You must buy a farm [data]. There are problems with that, but they aren’t big. Food [data] is cheap and plentiful.”
Alright, we have the one issue here, because data is cheap and plentiful in the digital age, and they gather my data as well, but theoretically that should be limited in the EU, and we get the copyright issue with the books here. But I don’t think the farmer/AI comparison goes all the way. For example graphics cards are the opposite of cheap and plentiful, and there isn’t a problem with that. So it’s not like there is a rule that resources or products have to be cheap or plentiful. It’s surely benefitial, but there’s also the real world, like with GPUs. And farmers also use intellectual property crops, and they use machines that cost hundreds of tousands of dollars. Sometimes you just have to pay for supplies and resources. That applies to farmers and for AI companies buying their supplies.
“The people who make AIs [books] want to be paid for their work. The people who build and maintain the datacenters [book press], the hardware [online shops and distribution chain], the electricity [author’s computer and studies and travels for the content], and so on. Should they work for free?”
“The problem starts when people want more than that.”
I really fail to see the difference here. Unless I start with a proposition: writing books is not a valid business model, but AI is… But why is that? Both are built on the grounds of intellectual property, both are products and require effort to be created. Why does a book author work 6 months and doesn’t get paid for his job and an AI researcher works for 6 months and needs to be paid?
Or phrased differently - Why isn’t it a valid product if a human reads a lot and then creates something and wants to sell the result… But if a big company devises a mechanism that reads a lot and they want to sell the result, then it suddenly is a valid product?
You said you praise the American Fair Use model. I said I don’t like it to work in that way. And most of all not grant exceptions to certain business models. And I agreed that there are some issues in the underlying copyright model, which might change the entire picture if addressed. I mean the interesting question is: How should copyright work in conjunction with AI and in general? And who needs to be compensated how?
I’m fairly sure the term “Fair Use” by definition means unauthorized and unpaid use. I mean we can try to twist the meaning of these words. Or maybe I misunderstood it. But paying would be kind of contradictory to the entire concept. It’d be (forced) licensing or something within the realm of copyright, depending on what you mean. But I think we need a new/different word for it.
In the US, it almost certainly wouldn’t be fair use if it meant that the author doesn’t get paid. Of course, you don’t get paid for the fair use, but there are a lot of things you don’t get money for.
You’re talking about authors not being paid at all. What’s that about?
That was about abolishing copyright altogether. Since we discussed that as an option. We’re now discussing what I called “subsidies” earlier. Authors do get paid, but for certain “uses” and not for others. And authors get financed by a different group of people.
In your example with the farmers, they’re not paid by me buying the product in the supermarket and that money gets handed down the chain to every supplier… But Nestle got the cocoa beans for free and society now gets to pay the farmer by a different method. Unless you have a specific proposal here, that’d be likely the definition of a subsidy to help Nestle and make their products look cheaper on a supermarket shelf.
The issue is just, I don’t see how any of those are arguments to distinguish between the two. I can twist them so almost every single argument applies to book authors and I don’t see any contradictions with that:
“You demand that
AI companies[book authors] should work for free and give things away for free. But they also should pay [in content] people that make no contribution.”“They do [create something], just like farmers. If people did not find their services beneficial, they would not pay [for the books].”
“[The price of the pile of books] is called a barrier to entry (Marktschranke).”
“It doesn’t have to be very bad. For example, you can’t just become
a farmer[a big AI company]. You must buya farm[data]. There are problems with that, but they aren’t big.Food[data] is cheap and plentiful.”Alright, we have the one issue here, because data is cheap and plentiful in the digital age, and they gather my data as well, but theoretically that should be limited in the EU, and we get the copyright issue with the books here. But I don’t think the farmer/AI comparison goes all the way. For example graphics cards are the opposite of cheap and plentiful, and there isn’t a problem with that. So it’s not like there is a rule that resources or products have to be cheap or plentiful. It’s surely benefitial, but there’s also the real world, like with GPUs. And farmers also use intellectual property crops, and they use machines that cost hundreds of tousands of dollars. Sometimes you just have to pay for supplies and resources. That applies to farmers and for AI companies buying their supplies.
“The people who make
AIs[books] want to be paid for their work. The people who build and maintain thedatacenters[book press], thehardware[online shops and distribution chain], theelectricity[author’s computer and studies and travels for the content], and so on. Should they work for free?”“The problem starts when people want more than that.”
I really fail to see the difference here. Unless I start with a proposition: writing books is not a valid business model, but AI is… But why is that? Both are built on the grounds of intellectual property, both are products and require effort to be created. Why does a book author work 6 months and doesn’t get paid for his job and an AI researcher works for 6 months and needs to be paid?
Or phrased differently - Why isn’t it a valid product if a human reads a lot and then creates something and wants to sell the result… But if a big company devises a mechanism that reads a lot and they want to sell the result, then it suddenly is a valid product?
Why do you not want book authors to be paid now?
Because that’s the Fair Use. It doesn’t involve monetary compensation for the use. Meaning they don’t get paid.
Why do you want fair use to work that way?
You said you praise the American Fair Use model. I said I don’t like it to work in that way. And most of all not grant exceptions to certain business models. And I agreed that there are some issues in the underlying copyright model, which might change the entire picture if addressed. I mean the interesting question is: How should copyright work in conjunction with AI and in general? And who needs to be compensated how?
I understand, But why do you want a fair use model that means that authors don’t get paid at all?
I’m fairly sure the term “Fair Use” by definition means unauthorized and unpaid use. I mean we can try to twist the meaning of these words. Or maybe I misunderstood it. But paying would be kind of contradictory to the entire concept. It’d be (forced) licensing or something within the realm of copyright, depending on what you mean. But I think we need a new/different word for it.
In the US, it almost certainly wouldn’t be fair use if it meant that the author doesn’t get paid. Of course, you don’t get paid for the fair use, but there are a lot of things you don’t get money for.
You’re talking about authors not being paid at all. What’s that about?
That was about abolishing copyright altogether. Since we discussed that as an option. We’re now discussing what I called “subsidies” earlier. Authors do get paid, but for certain “uses” and not for others. And authors get financed by a different group of people.
In your example with the farmers, they’re not paid by me buying the product in the supermarket and that money gets handed down the chain to every supplier… But Nestle got the cocoa beans for free and society now gets to pay the farmer by a different method. Unless you have a specific proposal here, that’d be likely the definition of a subsidy to help Nestle and make their products look cheaper on a supermarket shelf.