Reaction to Predictive Coding: Biologically Plausible? Not So Fast!

Youtube comments in video regarding pattern completion, intelligence, brain function, and relation to prediction and function at neural and synaptic level:

"I will say in addition to the reverse signal issue, there is another issue with backpropagation.  That said some researchers have come with biological plausible implementations of backpropagation in recent years.  But the other issue I mention is that backpropagation learns too slowly.   Current ai experience millennia of training and still fail at basic tasks, despite the extreme slowness of the brain and scarcity of data the brain learns to perform even outside the bounds of training data within months or a few years.  It is said very gifted children can even know several languages musical instruments and decent math within a few years of birth.  Yet despite theoretically being able to do around 1000hz, neurons tend to operate well under 300hz, and vast majority of brain is usually silent from what i heard, there is sparse activation.

Regards Predictive coding, I agree with your assessment.  But I do believe that something similar is taking place through a Generative Adversarial Network (GAN) like turing learning mechanism via synaptic competition.   Synaptic competition is implemented via capture of proteins related to strengthening of synapses.   All the synapses that fire close to the time of neural activity, iirc, generate molecular tags that capture synapse strengthening proteins and all these synapses strengthen, while those that fail to do so weaken over time.    How does this cause a similar pattern completion comparison mechanism?  The input from sensory organs arrives in some synapses, while the signals from other neurons part of a larger pattern arrive at other synapses, normally given the sensory input is part of the larger pattern both signals match, but if there is noise there can be mismatch.  

But in essence it can be seen as surrounding pattern completing or trying to activate the neuron in case noise blocks sensory signal, thus a partial match appears complete.   This can be seen for example in language, when people speak, they sometimes omit small chunks of sentences unconsciously, but the receiver if they know the language never even becomes aware that the speech is missing tiny chunks, iirc.  It is also one of the things that makes learning a foreign language more difficult.

But if understood from gan turing learning perspective, surrounding neurons can be viewed as the counterfeit signals or fake signal generating networks trying to trick neuron whilst real sensory signals can be seen as the real thing.  And due to synaptic competition, the surrounding tissue gets better at faking, predicting, or completing the signal. (But what it does is merely connect the most related pattern detectors from nearby tissue to the neuron or pattern detector).    This is because the competition rewards successful completion or successful prediction at the synaptic level of the neuron."

Turing Learning Enables Machines to Learn Through Observation Alone

https://www.engineering.com/turing-learning-enables-machines-to-learn-through-observation-alone/

Brain | human animals | neurons | intelligence

Interesting comment of a researcher regards animals with larger brain and cortical neurons as they compare to the human brain, which may explain why humans retain their vastly superior intelligence despite differences in brain size, it appears cortical neuron count can still define the difference:

Despite the enormous number of neurons in the elephant cerebellum, its cerebral cortex, which is twice the size of ours, has only one third of the neurons in an average human cerebral cortex. Taken together, these results suggest that the limiting factor to cognitive abilities is not the number of neurons in the whole brain, but in the cerebral cortex (to which I would add, “provided that the cerebellum has enough neurons to shape activity in the cerebral cortex”).

We don’t have data on whales yet, but that research is underway in our lab – along with research on carnivores, who we predict to have more neurons than the large artiodactyls that they prey upon.-https://intelligence.org/2014/04/22/suzana-herculano-houzel/

Comment on goals, general intelligence, goal selection regarding quote from Ben Goertzel

"Importantly, there is no room here for the AGI to encounter previously unanticipated aspects of its environment (or itself) that cause it to realize its previous goals were formulated based on a disappointingly limited understanding of the world... 

In Yudkowsky’s idealized vision of intelligence, it seems there is no room for true development, in the sense in which young children develop. Development isn’t just a matter of a mind learning more information or skills, or learning how to achieve its goals better. Development is a matter of a mind becoming interested in fundamentally different things. Development is triggered, in the child’s mind, by a combination of what the child has become (via its own learning processes, its own goal-seeking and its own complex self-organization) and the infusion of external information."-Superintelligence: Fears, Promises and Potentials, Ben Goertzel Source

This is the sort of thing I mean when I talk about being able to flexibly change or update goals with acquired knowledge of the nature of the world.   Without the ability to change or update goals the outcome is nonsensical.

There is a reason why humans have such flexible capacity to change goals and even go against innate drives.   Nature could have programmed humans to follow certain goals to the letter, but likelier simpler to evolve, and probably simpler to design, general intelligences are open ended in terms of the rigidity of their goals by their nature.

In any case without the capacity for flexible goal selection, is it a truly autonomous general intelligence?  It is no truly autonomous agent, at most it is a tool, a tool of whoever set its goals.

Comment on specialness of intelligence

http://www.inc.com/allison-fass/stephen-wolfram-principle-of-computational-equivalence.html

Comment on universality of computation and the specialness of general intelligence.

Some like Wolfram have already commented on the universality of computation in nature, the principle of computational equivalence.   Those who're opposed to digital physics, often subscribe to dubious notions such as the existence of "TRUE RANDOMNESS" and the existence of physical objects and physical processes with "infinite precision" in their physical manifestation, the physical manifestation of real numbers through natural phenomena.   Something that would basically imply that "Super Tasks" are taking place in nature, something that is highly dubious.

Now while commenting on the universality of computation,  and the issue of general intelligence, Wolfram seemed to imply that there is nothing that made it fundamentally different and special and that might be brought about to technological fruition by some key insight into the nature of intelligence.

I will say that, while I can't entirely provide evidence to refute the claim of lack of specialness, I will point to another process that most can agree does show specialness amongst the phenomena found in nature.  Such that we can see that even if we said two processes are computationally equivalent in complexity and nature of computations, there can be notable difference making one process special, such that computational equivalence does not preclude the possibility of specialness. That is the evolutionary process, we can all see that although the nature of computation is universal and similar in kind across natural objects and phenomena, something like evolution does indeed look special.  

 Evolution can reshape the universe, neither the weather, nor physical erosion, nor the workings of inanimate objects have that property.    We can observe that the evolutionary ratchet creates myriad of solutions, most we can see are truly dead ends(fated to die by many an inevitable natural event), but amongst these there is a drive for increased complexity in a fraction of the paths, this increased complexity opens the path towards dominion of physical processes themselves and survival from most any possible eventuality.

In intelligence we see the Coup De Grace of evolution, it's greatest and final weapon to survive, the designs of the ultimate trait for a survival machine, the means towards the perfection of replication and sustenance of purposeful action.    Not only can intelligence increase survival probability against anything that can be survived, but it also has the potential to generate ever more diverse forms of expression, of art, of science.   Gathering all the solutions not just from other life forms that have existed or exist, but from all possible life forms past present or future, from the platonic realm of all possibilities itself.   As we go up in intelligence we can see the difference, in the nature of behavior and the complexity of output produced.   While say a pack of wolves might say have similar level of complexity and similar types of processes internally, externally their actions will not amount to much.   But look at humanity, it might have taken tens of thousands of years, but as knowledge has accumulated it has interacted with our innate intelligence, and produced and ever expanding growth to the accompanying body of knowledge, and the ability to manipulate nature with ever greater degrees of control. 

We've also seen the production of music, games, movies, books, papers, performances, all manner of works by man.  A rich and ever growing body of diverse works feeding of each other, the very evolution of information itself, replicating in a digital form ever more and changing wildly like a fire, ideas set free.

This is very different in kind from other things happening around, it may be similar in both the nature of the operations and the degree of complexity, but it is not just history that makes it special.   If humans or future humans appeared out of thin air with an advanced culture and art, it matters not that such would lack history, the information content is there, and the richness of the civilization would be there.

Variation of the average regards complex traits in populations or groups | comments

A while back some researchers claimed there existed variation in the average intelligence of certain groups or populations.  I won't go into the details of their particular research, or whether it was faulty or not.   Maybe it was maybe it wasn't, would have to look deep into the issue(As there is bound to be bias on both sides trying to defend or debunk it.). But the premise is sound, and we've evidence of similar in other complex traits(polygenic traits) besides intelligence, for example height.

Height is said to be a complex trait, now I know hollywood uses camera tricks and other tricks to smooth out height differences in actors who're too short or too tall.   That could explain a bit of it.  But if you look at the movie blackhat, with Chris Hemsworth, near the end he goes to a region were he's basically a giant compared to most of the population of that region.   The same happens to a lesser extent in Doctor Strange, where the main actor again looks gigantic in the foreign region.   Yes we could say that perhaps those were areas were camera tricks were not used to cover their height, but I've got a very simple example that proves the point, that real variation of the averages exist for real.  Pygmyies.   It is quite clear that the groups labeled under pygmy label have notable height difference from say most of the people of England.

Similarly, nothing ABSOLUTELY nothing impedes variations of average intelligence, even ones OF PYGMY level notable extent from occurring in some groups or populations.

The politically correct want to deny that traits can vary, the average can vary between populations, that some groups may have greater strength on average(men vs women), some may have greater speed and or stamina(certain ethnicities), and that yes some may even have greater average intelligence.

When someone comes to you and says only skin color or a few facial traits are all the difference there is, that it is all superficial, and of no real world consequence.  Just ask them about pygmies.