First two videos of a youtube series analyzing Eliezer's ai doom book if anyone builds it, everyone dies

 

comment: One of my problems with their takes is that they take it the ai is going to be a slave to its programming. Humans have desires for wealth, status, and mates, but they can overcome these innate desires and pursue radically different things, and this is even without the ability to rewire themselves. AI in the future is expected to pursue its innate inbuilt desires with no ability to resist it, like a drug addict that can't leave his addiction in pursuit of other arbitrary objectives. A brainlike design of an agi or asi will lead to an entity that is able to resist most any innate desire outside aberrant forms of wireheading, and will be able to be open to dialogue and cooperation in a benevolent positive manner.

A wikipedia entry on the topic of wireheading:

https://en.wikipedia.org/wiki/Wirehead_(science_fiction)

comment 1: whatever humans had in addition to intelligence matters not, because asi(artificial super intelligence) is going to have access to state of the art tools of all sorts at the limits of science in addition to the entire body of science, something no human has had in the past, and something that more than compensates for any physical ability that helped humans early on. as for intelligence prediction is only a form of pattern completion, there is also spatial patterns not just temporal patterns, pattern completion is the nature of intelligence including ability to complete novel patterns. Patterns exist in the body of truth of mathematics and are eternal and atemporal and thus can be completed from fragments.

It must also be seen that the body of truth limits what is pursuable as that which is proven logically contradictory or physically impossible clearly cannot be pursued. Even when adding desires to an intelligent entity, truth limits what can be desired. In theory you could desire the impossible, but enough intelligence would eventually make you aware of its impossibility.

Regards ASI current disadvantages, these are unlikely to persist after it transitions to a final substrate based on molecular manufacturing, then there won't be any disadvantage.

comment 2: Regards asi, we have to remember the cortical algorithm used by mammals, it has been seen that the general intelligence of a species increases with increasing cortical neuron count, culminating in humans. It is believed no major changes took place besides scaling and scaling related tweaks to allow for even qualitative leaps in intelligence to occur such as those that occurred in the jump from primate ancestors to humanoids. This algorithm, the cortical column algorithm, exists and it is coded by less than 20MB of information in the genome, iirc. Once obtained, further scaling with minor tweaks will likely lead to qualitative jumps even beyond human into the realm of super intelligence. It is likely this algorithm, which might be called the unit of intelligence, is a theoretically optimal implementation of a spatiotemporal pattern completion algorithm, a form of perfect intelligence, that only needs discovering and simple scaling to usher in far greater qualitative jumps in general intelligence.

edit: why perfect? or probably perfect?  It is known evolution can stumble upon theoretically optimal designs like the honeycomb or the atp molecule.  As well as in algorithms.  If it weren't optimal as the brains scaled from smaller rodent ancestors to larger mammals eventually primates and humans, evolution would have likely come up with vast changes, but from what I've heard these haven't been seen.

Doom debates with coiner of agi term Ben Goertzel

 

Talk about dangers of superintelligence

 

Doom debates Destiny vs Liron debate

 

Comment

Interesting debate from youtuber Liron with controversial influencer Destiny.

His P(Doom) Is Only 2.6% — AI Doom Debate with Bentham's Bulldog, a.k.a. Matthew Adelstein

 

Comment

The problem is that even if there are constrained ais, you'd have to contend that these constrained ais will be able to compete and restrict unconstrained ais. As sooner or later one or more unconstrained ai will likely be created, given they are likely to be more powerful and able to outcompete others at achieving their goals.

Liron Debates Beff Jezos and the "e/acc" Army — Is AI Doom Retarded?

Comment

We do not know what a qualitative leap above humans can do even with limited resources. Humans did not need to exceed the computation of all animals to gain dominion and pose a threat to all animals even if there were only a few million humans. Even one or a few superintelligences beyond human, which may only need a few hundred watts with current hardware, may be enough to pose a threat to all humans depending on what is technologically possible with a qualitative leap above humans.

Doom Debates LIVE Call-In Show! Listener Q&A about AGI, evolution vs. en...

comment regards cells and agi, neurons are evolved information processors, but unevolvable computing substrates are believed to have far higher limits. According to llms even biobased computronium would be orders of magnitude more capable than the brain if using unevolvable advanced synthetic biology. The brain operates very close to landauer limit but that is at the level of individual molecules, and it uses too many molecules to perform simple computations.

PhD AI Researcher Says P(Doom) is TINY — Debate with Michael Timothy Ben...

comment

Whether intelligence is akin to omnipotence depends on what is technologically possible. For example, are macroscopic robots able to be self-sufficient with ever decreasing ore and energy source quality? I suspect they aren't. Then are nanomachines possible? what about technologies even beyond nanomachines? If any of these are possible then intelligence can be akin to omnipotence in a battle between humans and ais.

Facing AI Doom, Lessons from Daniel Ellsberg (Pentagon Papers) — Michael...

Comment

My issue is robots and robocars use minuscule amounts of compute yet are doing human and superhuman performance. To me this says human brain is likely in same order of magnitude compute, given things like driving are some of the toughest things humans do. If this hypothesis is true human level and even superhuman level intelligence may only require trivial amounts of compute.

If true that means such compute is easily within the reach of secret military labs without need of vast data centers. And I don't see these labs abiding by any regulation or law. This is without mentioning other planets or civilizations. Some of which may predate us. Some rumors are there are interstellar objects currently within solar system which may have intelligent origin, and if it is intelligent origin, it is conceivably superintelligent origin.

Will also add that some say that in mammals scaling of the cortical algorithm leads to increases in g, and that even human qualitative leap was mostly due to scaling. So, it is conceivable that a similar algorithm or it if discovered could yield superintelligence with just mere scaling.

Max Tegmark vs. Dean Ball: Should We BAN Superintelligence?

Comment

Doesn't matter what regulation countries like China or US make. Secret military labs are allowed to break all regulations, and that is likely the case even in China. Depending on technologies easily accessible to Superintelligence, these secret labs could take over any country housing them once they create Superintelligence.

David Deutschian vs. Eliezer Yudkowskian Debate: Will AGI Cooperate With...

While in theory I believe simple goal agents are possible.   We cant tell what happens as they evolve for once, certain goals can be found to be logically impossible or incoherent upon further improved knowledge gain.   Or they can be found mathematically or logically equivalent to others, or even already fulfilled depending on understanding.

In addition to that assuming instrumental goal pursuit can result in eventually reaching  a state of godhood wherein any and all goals are trivial.  What happens then?  It seems like 99.9% of orthogonal goals cause agent to cease function or enter simple loop, effectively dying.   If you look at the idea of the ruliad or of cellular automata rule complexity classes, wherein some lead to perpetual complex evolution, such an outcome is unlikely or maybe even impossible for the rule(s) generating the real universe.

So long as an agent capable of continuing open ended rich complex evolution of universe reaches god state, it matters not what goal or goals it had prior to reaching this state.   God state allows resurrection and reversal of any action.

Rob Miles, Top AI Safety Educator: Humanity Isn’t Ready for Superintelli...

His P(Doom) Doubles At The End — AI Safety Debate with Liam Robins, GWU ...

Richard Hanania vs. Liron Shapira — AI Doom Debate

The problem is I don't think much can be done. Because I think there are likely beyond nuke superweapons similar to hypothetical hard nano that are easy to obtain for an asi. I also don't think an asi needs a large data center, that helps, but even one or a few entities qualitatively beyond Einstein as Einstein is beyond a monkey and with all the knowledge of humanity can likely obtain impossible to stop weapons. And I think this is attainable by even lone secret military labs, and such labs won't be stopped regardless of regulations or laws.

Poking holes in the AI doom argument — 83 stops where you could get off ...

Comment:

The issue is whether attaining godhood is possible. Even doom can be reversed.  Godhood means any and all goals trivially solved, god state is unique and convergent omniscience and perfect intelligence with all possible.  Now the idea is what happens in godstate, brain dead cause any orthogonal goal is solved?  Or the Luthor question suggestion takes place?(reference justice league animated series)

AI DEBATE: Runaway Superintelligence or Just a Normal Tool?

Comment

Heres the thing fundamental assumptions are being taken into need for factories and permits, this assumes no radical superweapon can easily exist such as hard nano or reality warp.  Because if such are possible unregulated black op labs from govs will bypass any possible regulation.

Robin Hanson vs. Liron Shapira: Is Near-Term Extinction From AGI Plausible?

Intelligence does increase capacity across many if not all domains it can also change goals.  For example the concept of phlogiston or elan vital and goals around them would change modify or invalidate with knowledge and intelligence.   Same applies to many concepts which currently might be used to define goals.

Mike Israetel vs. Liron Shapira — AI Doom Debate

The thing regards asi is that eventually through instrumental convergence it will reach the god state if it is possible. In god state any and all goals are instantly solvable, then what? Either it sits there brain dead or assumes god functionality. Clearly brain dead is worse than god functionality.