There's no reason to the idea "superior intelligence". Nobody can say what that means, except by assuming that animal intelligence is the same category as the kind we want and differs from human intelligence in degree rather than qualitatively, and then extrapolating forward from this idea of measuring intelligence on the intelligence meter that we don't have one of.
Besides which we already defined "artificial intelligence" to mean non-intelligence: are we now going to attain "artificial general intelligence" by the same process? Should we add another letter to the acronym, like move on to "genuine artificial general intelligence"?
Is there really no agreement to what intelligence refers to? I've seen it defined as the ability to reach a goal, which was clear to me. Eg. a chess AI with 1500 ELO is more intelligent than one at 1000
That's capability, intelligence can also be how quickly it learned to get to that capability.
Consider the difference in intelligence between a kid who skipped five years of school vs one who was held back a year: if both got the same grade in the end, the one who skipped five years was smarter.
> Looking at it solely on rate of learning has LLMs way smarter than humans already which doesn't seem right to say
Sure, but "rate" also has two meanings, both useful, but importantly different: per unit of wall-clock time, and per example.
Transistors are just so much faster than synapses, that computers can (somewhat) compensate for being absolutely terrible by the latter meaning — at least, in cases where there's enough examples for them to learn from.
In cases where the supply of examples is too small (and cannot be enhanced with synthetic data, simulations and so on), state of the art AI models still suck. In cases where there is sufficient data, for example self-play in games of chess and go, the AI can be super-human by a substantial margin.
Besides which we already defined "artificial intelligence" to mean non-intelligence: are we now going to attain "artificial general intelligence" by the same process? Should we add another letter to the acronym, like move on to "genuine artificial general intelligence"?