“AGI” never made sense to me. It’s a purely marketing term right?
I’ve ignored it thinking it would go away, but it keeps coming up.
I get that consciousness differs from intelligence and that our waking awareness of life is a complete mystery.
Knowledge and thus intelligence however I consider as actively being solved by these large ML models. That is, with the right combination of machinery and know-how, you’ll get it.
But you’d be no nearer to solving consciousness.
Given this thought trajectory - what is AGI supposed to be?
Not sure why you are bringing up consciousness, that’s largely orthogonal to intelligence. AGI is usually taken to mean the capability to match or surpass human intelligence across all conceivable cognitive tasks, as opposed to being limited to certain kinds of tasks, or to not matching the general level of human intelligence in some respect.
Intelligence, and hence AGI, doesn’t require consciousness or emotions or sentience.
Intelligence is a capability, not a level of knowledge, and in discussing AI vs human intelligence, or human vs dog, it is perhaps better to regard it as a species-specific capability, not an individual-specific one.
Was the genius or dumb person really born with different levels of learning ability, or were they just raised differently: nature vs nurture ?
The "intelligence" of one species vs another comes down to differences in cognitive architecture, ultimately reflected in ability to learn and predict/infer. Intelligence, as a capability, not IQ test score, is best regarded as ability to learn from experience and use that learning to accurately predict future outcomes, ranging from passive observation, to the outcome of one's own actions, to the ability to reason.
Comparing the intelligence (not knowledge) of different AI system to humans should therefore be assessed by comparing their ability to learn and use what they have learnt.
An LLM is what it is - a language model, not a learning system. It's really an expert system of sorts, highly capable in terms of what it can infer based on the knowledge it encodes, but with very limited ability to learn.
When you talk of comparing AI to a genius/professional/graduate/etc, you are really talking about comparing level of knowledge, comparing one expert system to another, which is fine and perhaps useful in some contexts, but it is not the same as comparing actual intelligence - learning ability, and especially so if you want to discuss general intelligence which is all about the ability to successfully take on any task (perhaps needing to learn it first), not just do well on some limited set of tasks you are already familiar with.
If ability to learn is limited to one modality, such as language, then that indicates a lack of generality. A good test for whether an AI system is in the same ballpark as a human in learning ability, aka intelligence, would be whether it can (at run-time) learn language itself, from a blank slate start, when running in a suitable environment.
>If ability to learn is limited to one modality, such as language, then that indicates a lack of generality. A good test for whether an AI system is in the same ballpark as a human in learning ability, aka intelligence, would be whether it can (at run-time) learn language itself, from a blank slate start, when running in a suitable environment.
So a transformer ? Run-time? Seems like an arbitrary distinction to me. Because humans function in a certain way, every learning system no matter how capable must function in the same way to be 'truly' intelligent ?
The mechanism is irrelevant if the results are similar. No, it doesn't have to be implemented the same way as a human to qualify as a human-level intelligence. That would be silly.
How is text-based memorization going to let you learn non-linguistic skills, whether animal/human level or beyond (based on new data senses)?
How is text-based memorization going to substitute for learning? The two are not the same. Perhaps this is more applicable to robotics than a text generator, but I also doubt an LLM could learn text-based skills like programming or math if it had not been pre-trained on them via SGD & RL, and had to instead rely on some poor-man's-learning context-based recall instead. What else can't it learn? How is the LLM intern, the "drop-in replacement remote worker" going to do on day #2?
Instead of pretending that an LLM can be human-level, or super-human, or become generalist, why not just admit that this is not the final form of AI. An LLM is not an animal/human-like intelligence, it is something different - a language model, with it's own strengths and weaknesses.
Despite all the AGI hype, an LLM seems to have more in common with a pre-trained single-purpose system like AlphaGo than a brain, but with the rubric/reward-based policy function baked into the weights.
In another 10-20 years some new idea, hopefully more brain-like, will have superseded LLMs and they will indeed be labelled as "LLMs" as the AI/AGI label becomes attached to the new more brain-like creative intelligence. Perhaps it'll be sooner than 10-20 years, but I doubt it given the current 10-year fixation with LLMs which doesn't appear to be slowing down anytime soon. Perhaps Sutskever is working on something a bit different?
>How is text-based memorization going to let you learn non-linguistic skills
I don't know. How is Astra a step change in computer use and spatial reasoning to the extent it can play games, paint good looking stuff with e.g canva and a whole number of other things ?
>How is text-based memorization going to substitute for learning? The two are not the same.
Of course if you call it something else then you can say it's not the same.
>How is the LLM intern, the "drop-in replacement remote worker" going to do on day #2?
Just fine I imagine ? ICL and the memory tools around a harness are pretty good. I'm not sure what sort of magic you're expecting from the human, but they're not getting any improvement in that time frame a frozen transformer can't match.
>Instead of pretending that an LLM can be human-level, or super-human, or become generalist, why not just admit that this is not the final form of AI.
It doesn't seem like I'm the one pretending here.
>Despite all the AGI hype, an LLM seems to have more in common with a pre-trained single-purpose system like AlphaGo than a brain, but with the rubric/reward-based policy function baked into the weights.
If you say so.
>Perhaps it'll be sooner than 10-20 years, but I doubt it given the current 10-year fixation with LLMs which doesn't appear to be slowing down anytime soon.
The architecture that keeps delivering results isn't slowing down ? You don't say.
There's no shortage of people, even researchers, who for one reason or the other are convinced we are in need of some paradigm shift.
But guess what? Talk is cheap. You beat the current paradigm or you don't.
> How is Astra a step change in computer use and spatial reasoning to the extent it can play games and paint good looking stuff with e.g canva ?
Presumably because of pre-release training, because some alien outside of the model, armed with the reinforcement learning algorithm, came in and programmed its weights.
> The architecture that keeps delivering results isn't slowing down ? You don't say
Sure, nothing wrong with that, as long as you don't misrepresent the limitations of the approach.
> There's no shortage of people, even researchers, who for one reason or the other are convinced we are in need of some paradigm shift.
> But guess what? Talk is cheap. You beat the current paradigm or you don't. Do you seriously think that Meta never scaled JEPA ?
The idea has not been taken very far, so what is there to scale? It's not a complete cognitive architecture. So far it's also been using a pre-trained transformer as the learning component, which makes it of limited interest.
The animal intelligence approach, even in it's most fledgling form (that you would apparently dismiss), requires a complete agentic architecture, including new learning algorithms and generative behavior, before it can be compared to LLMs. We know that, done right, our brain architecture is more capable than an LLM, so even if any hypothetical attempts to reproduce it were not highly performant, we know that the idea itself is sound. You might compare with Uszkoreit's initial poor-performing implementation of his new language model architecture - should he have given up?
>Presumably because of pre-release training, because some alien outside of the model, armed with the reinforcement learning algorithm, came in and programmed its weights.
They didn't program anything. They gave it data at best.
>The idea has not been taken very far, so what is there to scale?
LeCun was the head of Meta AI for over a decade and his baby that he keeps harping on about wasn't taken very far ? Come on. You're smarter than that. It went the way all the alternate architectures have gone since the transformer, a sidegrade at best, probably not even that.
>You might compare with Uszkoreit's initial poor-performing implementation of his new language model architecture - should he have given up?
What are you talking about? There was no poor performing implementation of transformers that was published that he needed to push through. Are you talking about self attention experiments before the finished transformer? That's literally just research. And if he languished on that for a decade then yeah I'd tell him to probably look at something else, but of course he didn't.
You regard the LLM post-training process as "giving it data" ?!
Have you looked at all the published JEPA research both while LeCun was at Meta, and since (up to and including the latest AdaJEPA from June)? Please enlighten us as to exactly which line(s) of research you think were "scaled" at Meta, and then tell us which of these constituted anything even remotely resembling a complete testable intelligence?
FYI, it's been a long time since FaceBook/Meta even had a single head of AI. Since 2018 it has been split into two groups, FAIR and Generative AI, with LeCun being in the FAIR group, not as head, but as Chief AI scientist. LeCun only invented JEPA in 2022 (shortly after FaceBook became Meta), first writing about it in his "A Path Towards Autonomous Machine Intelligence" paper, perhaps unhappy with the work of the GenAI group, which he had no control over, that presumably was getting all the compute.
I was referring to Uszkoreit's personal telling (on YouTube) of the origin story of the Transformer, his motivations with the design, his initial personal failure to implement his idea in a performant enough manner to beat the current LSTM SOTA, and Noam Shazeer then throwing the kitchen sink at it and eventually coming up with the Transformer design.
>You regard the LLM post-training process as "giving it data" ?!
Yeah. Presumably, lots of synthetic data is being generated, experiments being run, but post-training is still a largely automated process.
>Have you looked at all the published JEPA research both while LeCun was at Meta, and since (up to and including the latest AdaJEPA from June)? Please enlighten us as to exactly which line(s) of research you think were "scaled" at Meta, and then tell us which of these constituted anything even remotely resembling a complete testable intelligence?
I have. My point isn't that his ideas are trash or that he should stop working on them. My point is it's not "gone very far" because he's taking it as far as he can, which isn't very far. He's not had a lack of influence, resources or will, either from his time at Meta or now with his billion dollar startup. He's had far more of it than most, if anything. That there's not much to show for it so far is not for a lack of trying.
>I was referring to Uszkoreit's personal telling (on YouTube) of the origin story of the Transformer, his motivations with the design, his initial personal failure to implement his idea in a performant enough manner to beat the current LSTM SOTA, and Noam Shazeer then throwing the kitchen sink at it and eventually coming up with the Transformer design.
So it's what I thought. This is just regular research unless an inordinate amount of time was spent on it and that's not the case.
> My point is it's not "gone very far" because he's taking it as far as he can, which isn't very far.
I wouldn't really agree - I'm no fan of LeCun, but the problem with JEPA isn't that it's a bad idea, or can't go very far, but just that it's not much of an idea in the first place!
It's no secret that our brain basically works by prediction, and what we're predicting is necessarily the external world as we perceive it though our own senses, aka latent representations, aka JEPA.
So, you COULD take this unoriginal smidgen of an idea and built it out to a full model of a human/animal brain, whether or not it's LeCun's intention to do so (he seems more interested in just the representational / world model aspect to it), but he certainly hasn't done so yet, nor created any research manifesto indicating that as his intent.
The fact that JEPA implementations to date are using pre-trained Transformers doesn't seem inherent to the approach - one could, with more effort, still predict latent representations (i.e. sensory feedback) but do so using a new real-time learning algorithm based on prediction failure.
LeCun seems more of an academic / research director than a builder, and I would never have put much stock in him being the one to build an animal brain.
Well I also don't think JEPA is a bad idea or anything. I don't think most of the alternative architectures or tweaks i've seen are bad ideas. On the contrary, some of them seem very cool and i'm all for interesting new ideas. I guess my opinion is more on the supposed necessity of it all.
I think the transformer is a powerful general learner. I think it's enough, and on the matter of intelligence, sometimes i think we make the mistake of zoning in too much on potentially spurious details. We still don't know much about Intelligence at the end of the day, what is and isn't really important, what is and isn't simply a detail of the environment and what kind of seemingly bizarre but surprisingly equivalent mechanisms can arise in the face of vastly different environments.
I think the distinction between these options probably doesn’t matter alll that much if the threshold you’re using is either “random person” or higher? Maybe bump it up to “random educated person”?
They are still different concepts of course, but I imagine that once one is achieved, the others aren’t far off.
Ok, but a random educated person will not perform well on vast majority of specialized tasks where professionals operate. A model like Astra probably will beat random educated person performance on majority of specialized tasks. It’s getting close to the level of professionals in many domains, and to genius level on some (e.g. math).
I’m just trying to understand the implications of the current frontier model capabilities.
We cannot in a declarative sense define what is economically valuable work even now let alone into the future.
People take what they can get for pay. Very few individuals can demand a wage. The value of employment is obviously designed around that, not some arbitrary definition of “valuable”.
Of course an AI will accept $0/hr, it doesn’t mean it does the job.
Anyone who could accurately define the value of work would be wildly successful without having to try.
That is not a useful definition for me unfortunately.
AGI is the term invented because arguments about what AI meant had gotten annoying. Originally there was no distinction and people thought "AI" would mean human level intelligence. Chess and conversations and robotics and math all in one package. Then games and classification and some robotics got solved and called AI, but that didn't solve math or conversation or online learning or a host of other things, so AGI was coined to refer to most of the whole package, virtually all the capabilities you'd need to replace humans intellectually. Now we're quibbling about whether AGI includes robotics or full real-world physical agents or something less.
The consensus now seems to be that once you've got human-level intelligence and planning and executive function then you get recursive self-improvement that can eventually autonomously solve the robotics and world-modeling and other portions of human-equivalence.
I like recursive self improvement instead. It seems like something that is actually quantifiable and kinda “the point” of why consciousness is important to humans.
So basically, being able to set it free on some long running goal and it sort of “lives” and autonomously does its own tasks?
I wonder at what point consciousness is necessary… that is, if you can have anything like that without it.
To the point that solving consciousness (and combining it with intelligence) is what gives you the autonomous, recursive, self-improving thing otherwise it can only drive in the dark and make big mistakes.
To your point I think - it’s why we don’t see too many non-conscious advanced biology (it rarely survives against those with it).
Right, you are interested in “AGI” and presuming none of that requires consciousness right?
For example, how do you know that “feeling pain” is not a functional prerequisite for a task. And that consciousness is a prerequisite for feeling pain
Because there is no feeling of pain involved during the reasoning about "How to improve the balance of power in the Indo-Pacific" for the human reasoner, hence there is no need to have any experience of pain for any reasoner.
At my job interviews are essentially rejection practice. The debriefings are these cringe sessions where everyone pretends to be better than the candidate, makes fun of them for being old, for their attire, for how they talk, it’s so so bad. Like high school. This is a well known company in San Francisco!
Every single screen is a “No” even when I’ve strongly recommended the candidate. I don’t believe we are really hiring anyone.
I have 2-3 of these interviews per week, been that way since Day 1 at this job for several months now.
Idk if they do it to keep us worried or what but it’s very toxic.
This isn't new at all, just one of the first things I found by googling "webgl demo."
WebGPU of course goes even further.
I think the real gap is just that people by-and-large aren't building real games on top of Safari. It's more lucrative to use the app store with its low-friction payment system.
I made a project a while back to port Minecraft's desktop version to the web and it runs decently-ish on a good computer with WebGPU (but takes a really long time to load)
This was functionally enabled by default by Minecraft being java, including the part where it was popularized by a java applet of a very early version.
There are business-level issues with web based AAA games, as I was alluding to before. Apple has a two-click payment system. If you failed a Candy Crush level and you needed to get up from the couch and get your credit card out of your wallet to buy more turns, no one would ever do it.
They’ve gotten better. ~2 years ago the assets all had millions of polys and you couldn’t use them with raycasting or even loading the model without crashing the tab.
Tripo, Meshy, etc all have “smart low poly” now as a paid feature. It works.
The main character model and several structures in this scene.
There is raycasting and full physics.
Like you, I experimented in the early days and was getting these millions of triangles for each model - as if it was volumetrically filling it with vertices.
Then, some time later they started rolling out “smart” and low poly features that solved it for me. Now if I make a house, it has a point at each corner, not millions all within it - which is what made it so difficult to actually use, because of the algorithms that crawl through all those nodes.
I did you one better and showed you a working implementation involving lighting, physics, and all the stuff you were worried about!
If I cared to prove how low the poly count is beyond a single HN comment - and if admitting to AI use to a gamer audience wasn’t suicide - you might get me to put a screenshot from Blender on the Community tab and talk about this topic!
But something tells me there will be some other excuse in your way ;)
> I did you one better and showed you a working implementation involving lighting, physics, and all the stuff you were worried about!
Not showing wireframes is not "one better" than showing wireframes, when it's the quality of the meshes we're trying to get figure out, where lightning and anything else just distracts from actually being able to see it.
What I'm worried about, is this conversation continuing for even longer without any wireframes or actual .obj/.fbx appearing ever, and getting another non-answer why the quality of the mesh is no longer important or whatever, or for your particular art-style it doesn't matter, or ...
I just want to know concretely what quality people manage to get from it, because as far as I've tried various things, locally and remotely, the output (of the meshes) isn't very good.
I’ve ignored it thinking it would go away, but it keeps coming up.
I get that consciousness differs from intelligence and that our waking awareness of life is a complete mystery.
Knowledge and thus intelligence however I consider as actively being solved by these large ML models. That is, with the right combination of machinery and know-how, you’ll get it.
But you’d be no nearer to solving consciousness.
Given this thought trajectory - what is AGI supposed to be?
reply