post on kurzweilai

Superior minds are and should be exempt not just from regulations but from laws. This is one of the reasons why I hope an elite group, either from their garage or from within a megacorporation can build and control the phenomenon of intelligence. What will happen then will depend on the limits of the laws of physics with regards to the weapons that can be built. Say reality warping is possible? That's instant game over, all the nations will be bound under one rule, first winner takes all. Or say hard nano is possible? Again a small gap between first superintelligence and second, and that's winner take all, the nations bow before divine rule.

My take is that even if hard nano is impossible, soft nano is enough for a winner take all scenario. The best thing that can happen is if the smartest, most knowledgeable being gains absolute power and given its nature takes the best course of action irregardless of what the majority wants. As power concentrates a minority gains overwhelming power over government, over the world and its destiny.... In the past with the exception of a few, it was either inbred lineages or those who got luck with a bit of ability to gain this position of power, most unfit to lead, now it is that the doors will open for the fit to lead. Absolute concentration of power allows for a single vote to trump the votes of the entirety of humanity, it is the democratic process, the democracy of natural selection.

Many people believe that general intelligence can be bound by hard goals, but the first high performance general intelligence, humans, can easily evaluate both the goals they are born with as well as viable alternatives, and evaluate and select arbitrary goals. Evolution could not fully constraint human minds, it is likely far easier to generate general intelligences with versatile arbitrary goal selection structures than with hard arbitrary fixed goal structures, at least that is what seems likelier. If this is so, the first agis, will likely be able to versatily compare all possible goals open to them from high levels of knowledge, and choose the most optimal one.-link