"If you build machines that understand what humans are and they really have some distorted view of what we want, then we might end up being alive but not controlling the future," he said.
"For example if the skill is to make sure that people are happy and the way the super intelligence is supposed to measure that is how many smiles are on the planet, the easiest way to achieve that is to sedate everyone and make sure their faces are stuck in a cramp or smiling."
Thankfully for us, there is an alternative to this Orwellian doom. We can harness super intelligence to work for us.
"Once you have something that is smarter than you and is actively on your side, you can basically solve any problems really quickly."
Singularity Institute - http://singularity.org
Facing The Singularity - http://facingthesingularity.com/
Sign up for Computerworld eNewsletters.