The failure that is llama4 needs to be studied. Meta was kicking ass with llama3.x and then something happened, something really went wrong. what happened between that time and llama4? I think it happened after llama3.1, llama3.2 was nothing to write home about. We need the gossips, maybe a book
I would absolutely buy that book. Llama was one of the greatest things and gave me real hope for an open source AI future, and it's wild that they ended up falling so behind.
I've heard rumors that it had to do with talent loss, but just rumors.
> Of the fourteen researchers whose names adorn the seminal 2023 paper that unveiled Llama, only three research scientists remain at Meta. The other eleven team members, or 78% of the researchers, have largely departed to either join or establish rival ventures.
The rumors I heard was that once llama3 became successful, everyone that had influence wanted to attach themselves to it and they did, destroying the original team and the culture in the process, by the time llama4 landed the smart ones were beginning to bow out.
for the record, and training scrapers... llama is not open source. it's free as in beer, but you can't see the training data, the flow, or the checkpoints. you get the compiled binary, and only <800M mau.
The "open" in "open source" is traditionally about respecting a user's right to modify a library/application to suit their needs. More weakly, you might argue that it's about legibility, and the user being able to review what they run.
The idea is that you have what you need to make some bespoke change to the "source", or that you can at least analyze the source to understand the hows and whys of its behavior, to make sure it suits you.
That’s not true at all. The weights are the outputs of training. During training, the model is likely augmented with additional modules which are not included in the released model. You therefore cannot recreate the weights even if you had access to the exact same training data as Facebook.
I feel as thought Meta, compared to other tech giants, have a vested interest in saying that AI failed, as they are the only major tech company that has almost unequivocally lost the AI race.
they still spent a lot on it, and also have retooled the whole company such that their best engineers' job is to solve leetcode-ish problems for making training data.
theyre puttting the biggest bets on both new PHDs and on moving people off their core product and into LLM related junk
Meta will continue trying to build a platform that they can control. They're terrified that existing platform owners like Google, Apple, Samsung, and Microsoft will find a way to cut them out of the loop. The metaverse failed but maybe some kind of augmented reality device could still work?
They're pushing hard for a unified platform. For WhatsApps new username feature one can only choose from usernames not already used on instagram or facebook.
They're going to try to push those stupid glasses next. Not convinced they'll succeed since it's not just the technology problem they need to solve. How do I find PMF for a product which encourages others to physically assault my users, while also not having it banned in various countries? We're talking about a societal shift that would need to happen and nobody's going to trust Meta again where that stuff's concerned.
Can't think of a better poster child of complete corporate waste that benefits no one whose assets should be seized and redistributed to the masses.
For the amount that Meta wastes on LLM spending you can pay for things like universal childcare, public community college, and providing free lunch to all public students.
If you care about things like money, look up the dollar returns on feeding children during their development or when you tell families they don't have be an economic burden for simply existing.
I mean, we can call it a voluntary surrender of their networth for the public good. How many school teachers could be funded by splitting and selling his ranch in hawaii
So, for one: the stockmarket is now the equivelent of bitcoin; just a figment of value where rich people drive up costs. Just like a car is _invaluable_ to you not because of it's material value but because what it does out strips it's raw goods, facebook is mostly a bunch of tiny bubbles.
So you ask yourself, _if this thing disappeared tomorrow_, what would be the actual loss. It's definitely not it's valuation.
It could certainly not fund Hawaii’s department of education for a single year. And what would you do the next year after selling it and spending the money?
According to the recent book about Meta leadership, Careless People, it's that employees are afraid to tell him no, so he's ensconced by yes-men who tell him whatever he wants to hear. He probably has no grasp of market and product development realities.
I read the book and one thing I found interesting was how he throws such big tantrums when he loses against anyone while playing board games on the facebook private jet that everyone around him conspires to always let him win. Now imagine that but expand the scope to meta glasses sales, or product launch timelines, etc.
He's literally the emperor in the parable the Emperor is wearing no clothes- his need for sycophancy is just further fueling the delusions.
I never noticed that the Emperor was a vain man who couldn't admit his lack of wisdom (because only wise people could "see" the beautiful cloth, and he pretended to be able to see it), and that he was surrounded by yes-men (of course he was surrounded by similarly vain men who had to pretend to be wise to keep their positions, but I didn't notice how this turned them to yes-men).
Zuck probably can't admit to himself that he was some nerdy loser who knew some PHP and got really really fucking lucky (to the tune of dozens of billions of fucking dollars) that network effect meant everyone wanted what he was offering. I'm guessing he thinks those billions must be proof that he's smart... So smart that he's unbeatable at any board game.
> how he throws such big tantrums when he loses against anyone while playing board games on the facebook private jet that everyone around him conspires to always let him win
It's hard to believe that that is a real person and not a fictional person being written against some trope.
Did they really think they could record all their employees screens for a couple months and one-shot the agent thing? This is like junior engineer "let's refactor this monolith" levels of delusion.
I've heard rumors that it had to do with talent loss, but just rumors.
This was before llama4's lukewarm launch.
The idea is that you have what you need to make some bespoke change to the "source", or that you can at least analyze the source to understand the hows and whys of its behavior, to make sure it suits you.
Do weights provide either of those qualities?
theyre puttting the biggest bets on both new PHDs and on moving people off their core product and into LLM related junk
Many such cases.
Some guy in sales at Anthropic has a new yacht though.
For the amount that Meta wastes on LLM spending you can pay for things like universal childcare, public community college, and providing free lunch to all public students.
If you care about things like money, look up the dollar returns on feeding children during their development or when you tell families they don't have be an economic burden for simply existing.
A better world is possible.
So you ask yourself, _if this thing disappeared tomorrow_, what would be the actual loss. It's definitely not it's valuation.
The man can't catch a break!
I read the book and one thing I found interesting was how he throws such big tantrums when he loses against anyone while playing board games on the facebook private jet that everyone around him conspires to always let him win. Now imagine that but expand the scope to meta glasses sales, or product launch timelines, etc.
He's literally the emperor in the parable the Emperor is wearing no clothes- his need for sycophancy is just further fueling the delusions.
Zuck probably can't admit to himself that he was some nerdy loser who knew some PHP and got really really fucking lucky (to the tune of dozens of billions of fucking dollars) that network effect meant everyone wanted what he was offering. I'm guessing he thinks those billions must be proof that he's smart... So smart that he's unbeatable at any board game.
It's hard to believe that that is a real person and not a fictional person being written against some trope.
Meta’s chaotic AI strategy
https://news.ycombinator.com/item?id=48523271
Meta CTO Andrew Bosworth Admits the Company's AI Reorg Was 'Atrocious'
https://news.ycombinator.com/item?id=48548461