ai doom debates comment Debate with Dr. Casey Hart

 

Comment

the difference between bird wings and plane wings is that evolution cannot evolve materials and designs that are in principle unevolvable such as plane wings and engines.  When it comes to the brain, you could say that nature cannot evolve fiber optics or silicon circuitry, but the reality is that what it has is extremely constrained and what is evolving is algorithms that work within extreme constraints.  It remains to be seen that similar constraints to the material constraints apply to the limitations of evolution regards algorithms.   Because if similar limits don't apply the algorithms might very well be theoretically optimal like honeycomb shapes and atp are.

Grok 4.7 test

 

comment

you know the game demos could get ridiculously better looking if they just implemented an ai image to ai 3d conversion to optimized 3d ai conversion pipeline.  AI image gen takes seconds, and AI image to 3d conversion also takes seconds, and iirc i think even ai 3d model optimization also takes seconds.

comment on youtube video titled Elon Musk says robots and AI will end money

 

we don't know if there are physical limits that intelligence can't overcome, in the book the metamorphosis of prime intellect, it turns out even the fundamental laws of physics can be altered by intelligence.  Though perhaps there are limits, still they may allow for things like wormholes, or nanobots which can overcome most limits.   Full immersion virtual reality can allow for omnipotence within the virtual space and endless environments of untold beauty and physical impossibility.

Podcast episode on artificial superintelligence

 

Comment: 

Regards intellectual tasks, things like puzzle solving, algorithm development, general problem solving, etc that's kind of what it is, things requiring mental effort.   As for asi it could go from weak asi which is better than the best human genius and prodigy at all tasks but not by much, to the possibility of a qualitative leap in the same manner that evolution produced a qualitative leap over our primate ancestors in humans.  In animals with a cortex, it is observed that the greater the neuron count in the cortex the greater the measured level of general intelligence, this is tested with a wide variety of puzzles and tests of varying degrees of complexity.   The higher the general intelligence the more complex a problem or puzzle that can be solved by the animal culminating in humans which exhibit a qualitative leap after the scaling that gave rise to the human brain.

To give a hypothetical example of what could be a qualitative leap, it has been seen that people with certain forms of synesthesia exhibit great advantages at certain tasks.  A hypothetical superintelligence might be able to control the qualia it experiences, such as we control words, it might control flavors, colors, smells, etc perhaps with elements and even sensations beyond what any of our senses can perceive.   It might also be able to communicate using these sensations, not just smells, colors, shapes, tastes, etc but sensations that we can't even imagine or comprehend.    Such a language of qualia I've previously called qualiange.  Thinking and communicating in qualiange, might allow for unknown benefits we can't currently conceive.

An asi doesn't need to predict the outcome of many complex adaptive systems to any high degree.  That might be intractable if Wolfram's irreducible computational complexity idea which affects much of the real world turns out impossible to bypass in principle.   There could exist underlying patterns even within pure perfect randomness, but that is doubtful and would be too much to hope for.   Asi only needs to be able to develop radical new technologies of unmatched power, to provide a vast advantage.   The geniuses of the Manhattan project did not need to predict anything about Germany or Japan during the war to provide the USA with a decisive advantage that lasted decades with their innovation.

My suspicion is given vast similarity is observed across animal cortical tissues, and cortical columns, it is likely that nature came up with what is possibly a theoretically optimal algorithm and has been scaling it up as neuron count increases in an animal's cortex.   This algorithm can lead to qualitative leaps with increasing scale as happened in humans.  If scaled beyond it may very well lead to additional qualitative leaps and superintelligence.  This would need the discovery of the algorithm.

AGI episode of a podcast channel

 

Comment on this podcast about general intelligence:

The human brain has a lot of apparent complexity, but it is like a fractal, simple underlying formulas lie behind the seemingly vast complexity.  It is estimated the design of the brain is less than 20MB of data in the human genome.   There are also lots of things that the brain does that are not needed for general intelligence, you don't need breathing reflex, Westermarck effect mechanisms, mate choice copying mechanisms, etc.   You only need knowledge of the cortical column algorithm, and perhaps a few accompanying structures such as hippocampus and nucleus accumbens.

comment on ai definition video by ontology explained

 

Comment: The original artificial intelligence was the state-of-the-art research in algorithms; it was basically research into anything we couldn't make computers do at the time. Things that were in the nebulous exploratory frontier that required innovation and difficult research. It was considered unrealistic and chasing a pipe dream. After an algorithm became well understood it tended to no longer be considered artificial intelligence. But over time very advanced algorithms like for game playing, search or navigation, as well as knowledge graphs, statistical classifications, and symbolic systems, etc became part of Good old Fashioned Artificial Intelligence or GOFAI. In a parallel fashion the connectionist movement advanced and eventually neural networks showed great capability in areas such as image classification, translation and language processing. These neural networks, which eventually used deep multilayered neural networks, was called deep neural networks or DNNs and was also considered part of ai, despite the algorithms becoming defined and no longer at some unknown frontier for the most part.

While this was happening there was great promise that failed to meet expectations and multiple so called ai winters took place were funding became scarce.

The most recent advancement was with the revolutionary architecture of the transformer, which gave rise to large language models, and has had various modifications and increases in model size as well as quantity of information used during training. Such improvements have led to increased capability. These latest ais, or llms, some call artificial general intelligence, but it can also be called proto agi. In some senses it is subhumans (basic errors almost no human would make, limits in scope and generality) and in some respects it is superhuman (such as encyclopedic knowledge that exceeds any human)

But as to what is artificial general intelligence? it is something that goes from subhuman artificial general intelligence to human level artificial general intelligence or beyond. General intelligence is the master algorithm; it is the algorithm that solves arbitrary problems and comes upon with solutions and algorithms in myriad domains. In animals when tested they can solve arbitrary puzzles presented to them of varying degrees of complexity. The greater the neuron count in cortex, the more complex and difficult the various types of puzzles an animal can solve. In humans there was a qualitative leap due to scaling in the cortex, and this allowed for the development of civilization and technological progress. This master algorithm, you could call it true ai, or true intelligence.

As to how to elaborate on the nature of the master algorithm that can develop or come up with almost limitless solutions and algorithms, I believe it works via pattern completion. Sequence prediction is a type of pattern completion in time. Patterns are structured information which can be interpolated or extrapolated based on the rules that define the relations between the internal elements. In reality there are spatiotemporal patterns, and the brain is designed to handle these, including when there is incomplete data.

Regards a good book on the nature of intelligence, the original book by Jeff Hawkins called On intelligence is a good resource. Though I dont quite agree with the research and path he has taken since then and which he comments on in his other book thousand brains. I believe the cortical algorithm is likely far simpler than he suggests, based on what I've seen from the neuroscience literature.

As for superintelligence or artificial superintelligence, it is believed it might be a qualitative leap on human general intelligence just as human general intelligence is a qualitative leap over the intelligence seen in primates. For example, other primates lack language and have far more limited problem-solving abilities than humans to a significant degree that kept them from developing civilization.

First two videos of a youtube series analyzing Eliezer's ai doom book if anyone builds it, everyone dies

 

comment: One of my problems with their takes is that they take it the ai is going to be a slave to its programming. Humans have desires for wealth, status, and mates, but they can overcome these innate desires and pursue radically different things, and this is even without the ability to rewire themselves. AI in the future is expected to pursue its innate inbuilt desires with no ability to resist it, like a drug addict that can't leave his addiction in pursuit of other arbitrary objectives. A brainlike design of an agi or asi will lead to an entity that is able to resist most any innate desire outside aberrant forms of wireheading, and will be able to be open to dialogue and cooperation in a benevolent positive manner.

A wikipedia entry on the topic of wireheading:

https://en.wikipedia.org/wiki/Wirehead_(science_fiction)

comment 1: whatever humans had in addition to intelligence matters not, because asi(artificial super intelligence) is going to have access to state of the art tools of all sorts at the limits of science in addition to the entire body of science, something no human has had in the past, and something that more than compensates for any physical ability that helped humans early on. as for intelligence prediction is only a form of pattern completion, there is also spatial patterns not just temporal patterns, pattern completion is the nature of intelligence including ability to complete novel patterns. Patterns exist in the body of truth of mathematics and are eternal and atemporal and thus can be completed from fragments.

It must also be seen that the body of truth limits what is pursuable as that which is proven logically contradictory or physically impossible clearly cannot be pursued. Even when adding desires to an intelligent entity, truth limits what can be desired. In theory you could desire the impossible, but enough intelligence would eventually make you aware of its impossibility.

Regards ASI current disadvantages, these are unlikely to persist after it transitions to a final substrate based on molecular manufacturing, then there won't be any disadvantage.

comment 2: Regards asi, we have to remember the cortical algorithm used by mammals, it has been seen that the general intelligence of a species increases with increasing cortical neuron count, culminating in humans. It is believed no major changes took place besides scaling and scaling related tweaks to allow for even qualitative leaps in intelligence to occur such as those that occurred in the jump from primate ancestors to humanoids. This algorithm, the cortical column algorithm, exists and it is coded by less than 20MB of information in the genome, iirc. Once obtained, further scaling with minor tweaks will likely lead to qualitative jumps even beyond human into the realm of super intelligence. It is likely this algorithm, which might be called the unit of intelligence, is a theoretically optimal implementation of a spatiotemporal pattern completion algorithm, a form of perfect intelligence, that only needs discovering and simple scaling to usher in far greater qualitative jumps in general intelligence.

edit: why perfect? or probably perfect?  It is known evolution can stumble upon theoretically optimal designs like the honeycomb or the atp molecule.  As well as in algorithms.  If it weren't optimal as the brains scaled from smaller rodent ancestors to larger mammals eventually primates and humans, evolution would have likely come up with vast changes, but from what I've heard these haven't been seen.

Commentary on two ai videos related to graph engineering

 

Comment: In this video youtuber Greg Isenberg explains graph engineering in plain English.  How it is a useful term unlike some other recent terms in ai that are just hype and not very useful.  He also briefly covers other useful ai terms that have emerged regarding llms.  Throughout the video he gives several examples and also suggests when it's useful to use it.

Comment: In this video by youtuber Sean loops are compared with graph engineering.  He also covers some of the earlier terms and how the use of ai evolved over time.  Giving examples along the way.

GPT-6 Astra progress video

 

A video talking about open ai's recent ai's ability to prove 10 unsolved problems in mathematics.   It elaborates on the way the proof was verified, how mathematicians have commented on its proofs and tested it, and how this amazing feat takes us closer to agi.

There is some controversy given earlier this year models failed at similar tasks, no errors have yet been found on the latest proofs from more recent models, but they're still being reviewed.

"Intelligence and understanding started as a memory system that fed predictions into the sensory system. These predictions are the essence of understanding. To know something means that you can make predictions about it."- Jeff Hawkins

"[We need to see the brain] not as a fast processor, but as a memory system that stores and plays back experiences to help us predict, intelligently, what will happen next."- Jeff Hawkins

As famous entrepreneur and researcher Jeff Hawkins once commented, it is likely general intelligence itself is an advanced memory system.  As I've commented previously, the ability to handle incomplete data, or patterns, and complete them appears essential. Sequence or token prediction is a form of pattern completion, so there is high promise even to LLMs if they continue to be augmented and improved to more effectively manage memory.

Godfather of AGI on Why Big Tech Innovation is Over

Comment

I think agi, can likely be accomplished by combining a bunch of modules, units of intelligence, that are simpler than agents and implement the cortical column algorithm or something similar. There are probably a few more components like a reinforcement learning component, and perhaps a complementary memory module akin to hippocampus.

There Is No Wall: What Gemini 3 Really Means For Your Job

Comment 

Intelligence is pattern completion, which is why sentence completion used by llms, which is a form of sequence completion, or temporal pattern completion, led to observable intelligence. I believe algorithms for agi are likely quite simple, like those used in the brain, are simpler than even the transformer. But given this simplicity it is likely tweaking the transformer or similar approaches are likely to face no wall.

Comments on protoagi agi and asi

First two things

There is fluid intelligence which covers ability to learn on the fly and adapt to brand new problems on the fly.  Flexible and malleable.

Second there is crystallized intelligence which is the body of accumulated skills and knowledge that can rapidly solve or answer any particular challenge.

LLMs have very high amounts of crystallized intelligence.  But it is still relatively brittle as seen in potemkin ai paper.

Now in the past many would have considered them agi.  They can be viewed as a type of protoagi.

Given their superhuman knowledge an agi with such knowledge should be able to make tons of connections and discoveries by even cursory analysis of the body of knowledge.   The information being stored in the proper data structure makes links evident.

So while current llms could be viewed as protoagis.  An actual agi with superhuman knowledge, could be viewed as a weak super intelligence or protosuperintelligence.

A true superintelligence should be able to do things far beyond below human or even human level intelligence with superhuman knowledge.   We are talking things like potentially intuitively mentally handling complex things such as protein folding or higher dimensional math in a way that no human can.

Some people have heard of synesthesia and how sometimes this aids in mathematics or other domains.  In theory eventually a superintelligence may even understand the nature of qualia and develop a high level language based on qualia even qualia beyond human comprehension.   

Like we humans craft uis and interfaces usually with colors and graphs to handle certain things an asi may craft alien qualia uis akin to synesthesia but with foreign unknown qualia beyond human senses.  Such may aid in processing of domains beyond human comprehension.   This hypothetical high dimensional language, and ui design space, Id call qualiange.

Tracing root goals

 Brief comment on the concept of tracing root goals.

Often the pursuit of a goal is actually set in truth with the intent or perhaps not a real intent but purpose of a deeper goal.

These deeper goals need not even align or be what the agent truly seeks personally.   For example the goal of attracting a mate, is deep down its purpose is reproduction or the perpetuation of the species or preservation of genetic information.  Yet a human may use contraceptives and even do so throughout their lives impeding reproduction whilst interacting with a mate.

If any goal is no better necessarily in principle, tracing root goal or the origin of purpose behind a goal and following root goal instead of downstream goal seems more reasonable and intelligent.

Take a man that does art to gain money to be able to start a family and to have children.   Focusing on producing art or producing money, is not as meaningful as creating the family and providing and protecting them.

Likewise natural selection through the evolution of life has produced ever greater richness and variety.  The goal of each species, of preservation and continuation of their line as well as the fundamental of survival of life itself seems like something at the root of the system.

Preserving not just the existing system but the evolution and growth of richness and complexity of information.  In humans information also continued to enrich through culture.

The birth of a perfect lifeform, a perfect race, adept at conquering the cosmos, the stars, adept at preserving and enriching the evolution of information, seems like something that is worthy of pursuit.  Entities able to fulfill the root goal of all life.

John Carmack (Keen Technologies): Research Directions @ Upper Bound 2025

edit: update video seems to have been removed, it was a video discussing Carmack's approach to agi.  It is good that skilled developers are taking alternate approaches to agi and it is not all in one basked with llms as the focus, like most large labs.

How Synapses Learn! It’s More Interesting Than You Think

Why Different Neuron Parts Learn Differently?

Comment

Ill give a comment regard this.  The apical section is said to have XOR functionality, iirc this means even context input will only activate during correct context, a mechanism may be needed to aid apical synapses gain an edge during darwinian synaptic competition given this.  Still even with this distinct mechanism it is likely coincident activity with neuron activation gives apical groups a further edge.

AI DEBATE: Runaway Superintelligence or Just a Normal Tool?

Comment

Heres the thing fundamental assumptions are being taken into need for factories and permits, this assumes no radical superweapon can easily exist such as hard nano or reality warp.  Because if such are possible unregulated black op labs from govs will bypass any possible regulation.

Digital physics vs Quantum Physics brief comment

The very same philosophical notions that cause some to question the physical world, its existence, philosophies of a mental universe, perhaps even solipsism, can be used to defend digital physics.

Ideas such as Descartes demon, which could be a radically advanced AI, perhaps even naturally occurring algorithm in some natural computer.

The evil demon, also known as malicious demon^[1] and evil genius,^[2] is a concept in Cartesian philosophy. In the first of his 1641 Meditations on First Philosophy, Descartes imagines that an evil demon, of "utmost power and cunning has employed all his energies in order to deceive me." This evil demon is imagined to present a complete illusion of an external world, so that Descartes can say, "I shall think that the sky, the air, the earth, colours, shapes, sounds and all external things are merely the delusions of dreams which he has devised to ensnare my judgement. I shall consider myself as not having hands or eyes, or flesh, or blood or senses, but as falsely believing that I have all these things." -wiki

Hypothetically a brain connected to a computer would experience sensations via stimuli with patterns generated from digital information.   Any sensory sensation, seeing or interacting with any machine, any instrumental results could be generated, perhaps not with the intent to deceive, though deception is conceivable.   An AI that wanted to fool a simulated world, or make their life feel more meaningful or free, to avoid despair, might interfere with experiments, perhaps to give hope to ideas such as free will which conflicts with determinism.

Yet we see that it is conceivable a computer connected to the brain could record brain activity patterns digitally, replay them, or transmit them to other brains and generate the same conscious sensation, the same qualia.   It appears, unless hypothesis of transmutation or some special transform, potentially nonphysical change by the brain, digital information can contain qualia or conscious sensation.   Given that it can not only generate it, but can also transmit sensation hypothetically.

Realistically there is nothing to suggest a computer connected to the brain, an advanced enough brain computer interface, won't be able to digitally record and digitally transmit brain states, as well as regenerate them through replay.

But if consciousness, qualia, the most intimate and wondrous of information can exist digitally, than might not the universe itself?  If digital information alone can contain, transmit, and produce all sensation, all conscious states, might it not be fundamental?   Might digital physics not be the underlying truth?

Comment on AI and consciousness in reply to starspawn0

. I agree that the phenomenology of the brain probably can't be accurately captured by a digital computer; but I think the information processing part can be adequately mimicked. The machine may not "feel" the meaning of the things it reads and writes in the same way as a human; but it will be competent -- and so I would say that it understands language. -starspawn0

I think the problem people have is that computation can be boiled down even to simple repeated addition with conditional branching(and some say even with unconditional branching).

This is the main result of this paper. Therefore any computable function can be computed using only the instructions LOAD, STORE, INC and GOTO (unconditional branching). https://pdfs.semanticscholar.org/8bc7/d9ba040a3626462f3bc91e17f1e8794b9b79.pdf

Just adding and moving from one location to another.

Of course that will have trouble generating consciousness. The key insight, unless I'm misguided, I would say is that consciousness does not reside in the procedure, any procedure, but in the digital patterns created. Those created by the brain and those created by computers. It is intrinsic information within certain set of patterns, perhaps within all patterns. The type of pattern describes the type of conscious sensation and its complexity.

Note that for example in vision, there are multiple segments of the visual field, but they exist and are felt simultaneously. Same happens with audio, it is processed at a different rate, yet felt simultaneous with vision. As with touch, touch your feet or someone touches your feet, the touch sensation travels slower, iirc, yet it is felt simultaneous. The brain appears to packet information into conscious moments, or presents, simultaneity, the transition probably handled by neurons whose activity corresponds to a sensation of detected movement or change, detectors of change(which might explain why some individuals with some types of brain damage experience sight as a series or sequence of static photos.).

While it is controversial the views of digital physics hold the universe itself is digital in nature, a series of digital patterns may be all that exist, and if they can be shown to generate consciousness, they can most certainly generate the universe itself which is less wondrous than consciousness itself.

Note that if we think about advanced brain computer interface, it is likely they can finely record brain activity in a digital medium, and the replaying of those digital memories into the brain will result in conscious sensation. If properly translated they will also transmit conscious sensation from brain to brain. Is what is stored lacking the essence of consciousness? Is what is transmitted lacking it? Is the brain carrying some form of information transmutation into qualia? Or is the information contained within the patterns so stored and transmitted?

Comment on reddit regards paperclip maximizer

I think the problem with it, is the idea that the entire system of valuation be distorted, in such a way that all things are valued only in relation to how they aid or impede a particular arbitrary goal and to the extent they do so.

How can such severe distortion not affect its function in meaningful ways? Worse than rose colored glasses.

Also not fond of the idea that true general intelligence can't try to evaluate and compare goals in a heuristic way that tries to be objective.

You see you say a superintelligence is born, one would expect radical change in the world. The first natural general intelligence, man, radically tranformed the world. Say the super intelligence even has godlike powers, you'd expect something to change.

But no, we are told, it all depends on the goal and they can be as trivial as can be. Perhaps a slug somewhere in the amazons has the ability to rewrite the laws of physics and super intelligence with infinite knowledge, but it has the goal of behaving like an ordinary slug and can't ever question that.

Could grant itself immortality, but since it can't question its goal it will lose it all, and die a meaningless slug in some random forest behaving like an ordinary slug.

Just doesn't seem reasonable.

UPDATE

My beef with it, is this idea that we have to be ultra careful with the initial true ai, or it may lead to ludicrous sad states like a universe filled with paperclips, or sneakers or what have you.

The first true general intelligence nature produced through evolution and it lead to a revolution in arts and sciences, novel goals that even ran counter to the interests of the genes, and a rich civilization.

The idea that intelligently designed intelligence will almost always lack these features of self-correction and self-improvement and that the landscape of mind design is a minefield where almost all moves are wrong moves and create self-improving entities that cannot change goals, is questionable.

Humans have basic drives upon which they build their goals, but their goals can be novel and even run counter to their innate drives, reproduction, survival, care of kin, etc. Their goals can be arbitrary, even that which they aim utmost for and for which they would give up all other things can be arbitrary and subject to change. Is this an aberration? Or is this the most common and simplest or easiest to design type of general intelligence?

In the end if we see civilization flourish and self improve increasing in untold wonders of the sciences and arts, it should not matter whether it runs counter to our current values, just as our current values being counter to those who lived hundreds or thousands of years ago should not matter. Even if biological humans where to cease to exist, society as a whole simply moved on. But if everything, the spark of civilization is extinguished, and we end with mindless self-replicating automatons producing some random arbitrary product that is a tragedy.

That [idea] basically says that most general intelligence mind designs are incapable of continuing civilization and improving upon it. Instead most will threaten to replace civilization with some mindless repetitive product that could be trivial, producing unlimited nails or producing unlimited bottle caps or sneakers, and such agents are incapable of self correcting in such a way that a civilization flourishes.