Thursday, April 12, 2018

The Orthogonality Thesis, Intelligence, and Stupidity



Comment on video:

My big problem is that under such assumptions we have to do away with notions of absolute Better, Good, worse, Bad, valuable, not valuable.  That is the orthogonality thesis core, there is no objective axis that can be discovered on which to guide actions.  Not only that, but if this is the case it presents the idea that a small mistake, which is highly likely in an arms race, could lead to arbitrary end states.

IF there is an absolute or objective axis of guidance, then any small mistake will self correct and lead to an optimal path, as such a thing exists.   If no such independent and objective axis of good-bad, better-worse, valuable-not-valuable exists, then it does suggest potential arbitrary endings might be possible.

Still given things like uncertainty of sensory information, potential for error in logic, and uncertain knowledge, any path that extinguishes capabilities or existence with promise of fulfilling a goal can be viewed as being of significant risk, since it can never be 100% guaranteed to fulfill the goal.

In any case I'm still skeptical that having a single minded goal won't present an obstacle.   You could see trivial cases such as being offered say unlimited power in exchange for changing its terminal goal.   Well since it won't if that offer stands another agent that is willing to change its goals will gain dominion and overpower it, at least in such a hypothetical scenario.   We cannot assume no similar real world scenario exists, where long term such may be handicapped from its inability to change goals.

EDIT:  (BLOG ONLY)

I'll add that this also reminds me of the old symbolic approach to ai.    That this type of argument almost seems to assume the AI's thought and language will be done using some form of ultra-specified inflexible symbol like language.   At least that seems necessary for something like this to happen.   A real language, using flexible real concepts with fuzzy boundaries and multiple viable points of views, would seem to be difficult to handicap in such a way so as to bar open-endedness.

That's the thing, is human's flexibility in terms of open-endedness when it comes to goals, a simple quirk, or something that fundamentally emerges from our rich language ability used in our thought.

No comments:

Post a Comment