After a decade or so of "machine learning" in the 21st century, arguably the only important lesson is what they call the "Bitter Lesson", which tells us that, with the current level of computation power we have, just saying what you want and performing gradient descent is more than enough. As long as we have an objective function that is composed of differentiable functions, we can just shove FLOPS into it and make it work.
As such, the most valuable thing on Earth right now, something that would be worth trillions of USD, is an objective function of intelligence.
The fact that we can't define intelligence, but depend on it, is a really really interesting fact.
What does it mean? It means that intelligence is not a simple thing we can objectively (as opposed to subjectively) define. Instead, it depends on human judgments. In particular, value judgments.
This is not to say that value judgments are the only thing important for intelligence. But it is a dependency.
In contrast, the kind of intelligence that does not require human value judgments is simply what we call computation. We have formal frameworks to define and validate computation. The reason we can't apply those things to general intelligence is the dependency on human judgment.
This kind of means "we" (collectively) get to choose. Choice. Free Will.
Ah yes. The most valuable thing in the 21st century is our collective free will.
Of course this also means that we don't have an "objective" function of intelligence in that the function must be subjective to the user's choices.
To me, this is almost a fulfillment of the idea of "ask, and it will be given to you; seek, and you will find; knock, and it will be opened to you." (Matthew 7:7)
To those who have not yet experienced providence, it is difficult to accept that the limit on ourselves is not lack of substance, but that we don't yet know what we truly desire. And now the trillion dollar objective function is staring directly at us. This is not "heaven on earth" yet for sure, but divine concepts do manifest in such similar worldly forms.
