I have a feeling you are misunderstanding what the Nature paper was about. It is not a proof of AGI (it is not trying to be a proof). Not the least because there is no definition of intelligence that people can agree upon. As the authors also discuss, by the way. The paper is speculative, so your concerns about the “qualitative arguments” is a non sequitur (to use your words).
By the way, good news: We should stop worrying about AI, as that is not a thing. Says the man who should take a Turing test to try to prove he is in possession of human-level intelligence.
For what it’s worth, that article from February. My biggest critique of it is that it holds up attacks against LLMs and then “defeats” them with counter-examples that are not just LLMs. They’re complicated multi-modal systems with harnesses to iteratively try lots of things. Yeah, that’s a little pedantic, but it’s an important bit of pedantry in my opinion because it perfectly explains the things they’re good at.
In fact, I’d say that — in the language of the article — it’s the harness that is granting the LLM “the body.”
They don’t have bodies. Without embodiment, critics argue, there can be no general intelligence. This reflects an anthropocentric bias that seems to be wielded only against AI.
Somewhat tellingly, it’s this “embodiment” with a harness that actually does cause effects that’s the dangerous part.
Daniel Kahneman (RIP) would likely have smiled a bit at that kind of statement – at least he would have added that the overconfidence typically found in many humans would accordingly underline the scarcity of high intelligence.
I used to do some estimator calibration exercises using trivia questions and tried some with ChatGPT, instructing it to not look up answers on the internet. It was instructed to provide 99,54% and 90% credibility interval estimates and a best guess (0-1-loss function so pretty much the mode of a prior). It had all approximate 100% intervals correct and failed on 2 out of 10 of the 90% intervals. But it seems that, while the AI has a sense of uncertainty, it is typically rather overconfident as well.
Maybe AI need to be calibrated as estimators in the same way humans do?
Super stupid.
As if there were not enough stupid people already: Will A.I. Make Your Brain Lazy? - The New York Times
We have to try to use AI intelligently…
A wise person is never quite sure how smart he is. A fool is certain, and makes sure you know it.
The US government is apparently working with Russia (!?) to make sure no restrictions are put in place to prevent access of AI tools to the military’s weapons. One of those provisions is that humans should not be required to review the AI-identified targets.
That sounds like a great idea, don’t it?
Both military superpowers share a fundamental hostility toward United Nations dictating their behaviors. On the US side, the position is deeply aligned with the philosophy of the sinister Palantir Technologies.