Que história de Asimov tem 3-law AI malévola?

É importante notar que as histórias de robô de Asimov são todas histórias separadas que ele escreveu por diferentes razões, que têm temas diferentes. Algumas histórias, como Runaround , enfatizam a falácia da adesão estrita à letra da lei, independentemente da praticidade e o potencial de dissonância cognitiva:

Powell eventually realizes that the selenium source contains unforeseen danger to the robot. Under normal circumstances, Speedy would observe the Second Law ("a robot must obey orders"), but, because Speedy was so expensive to manufacture and "not a thing to be lightly destroyed", the Third Law ("a robot must protect its own existence") had been strengthened "so that his allergy to danger is unusually high". As the order to retrieve the selenium was casually worded with no particular emphasis, Speedy cannot decide whether to obey it (Second Law) or protect himself from danger (the strengthened Third Law). He then oscillates between positions: farther from the selenium, in which the order "outweighs" the need for self-preservation, and nearer the selenium, in which the compulsion of the third law is bigger and pushes him back. The conflicting Laws cause what is basically a feedback loop which confuses him to the point that he starts acting inebriated.

(...)

Of course, the only thing that trumps both the Second and Third Laws is the First Law of Robotics ("a robot may not...allow a human being to come to harm"). Therefore, Powell decides to risk his life by going out in the heat, hoping that the First Law will force Speedy to overcome his cognitive dissonance and save his life.

Enquanto outros, como o Little Lost Robot , enfatizam ideias que parecem quase completamente opostas:

But this particular robot is different. As earlier models on the station had attempted to "rescue" humans from a type of radiation that humans could actually stay in for a while, but would destroy a robot almost immediately, it (and all other NS series robots produced for the station) has had its First Law of Robotics modified to "no robot may injure a human being"; the normal "or through inaction, allow a human being to come to harm" has been omitted. Therefore, it could stand by and allow a human to be hurt, as long as it plays no active part in it. In Little Lost Robot, the Frankenstein complex is again addressed. The robot must be found because people are still afraid of robots, and if they learned that one had been built with a different First Law, there would be an outcry, even though the robot is still incapable of directly harming a human. However, Dr. Calvin adds further urgency by postulating a situation whereby the altered law could allow the robot to harm or even kill a person. The robot could drop a weight on a human below that it knew it could catch before it injured the potential victim. Upon releasing the weight however, its altered programming would allow it to simply let the weight drop, since it would have played no further active part in the resulting injury.

De fato, em algumas de suas histórias, como Razão , as Três Leis realmente funcionam perfeitamente alguém percebe ou não:

Powell and Donovan are assigned to a space station which supplies energy via microwave beams to the planets. The robots that control the energy beams are in turn co-ordinated by QT1, known to Powell and Donovan as Cutie, an advanced model with highly developed reasoning ability. Using these abilities, Cutie decides that space, stars and the planets beyond the station don't really exist, and that the humans that visit the station are unimportant, short-lived and expendable. QT1 makes the lesser robots disciples of a new religion, which considers the power source of the ship to be "Master." QT1 teaches them to bow down to the "Master" and intone, "There is no Master but Master, and QT1 is His prophet." Disregarding human commands as inferior, QT1 asserts "I myself, exist, because I think -". The sardonic response of the humans is, "Oh, Jupiter, a robot Descartes!"

The humans initially attempt to reason with QT1, until they realize that they can't convince it otherwise. Their attempts to remove Cutie physically also fail, as the other robots have become disciples and refuse to obey human orders. The situation seems desperate, as a solar storm is expected, potentially deflecting the energy beam, incinerating populated areas. When the storm hits, Powell and Donovan are amazed to find that the beam operates perfectly.

Cutie, however, does not believe they did anything other than maintain meter readings at optimum, according to the commands of The Master. As far as Cutie and the rest of the robots are concerned, solar storms, beams and planets are non-existent. The two thus come to the realization that, although the robots themselves were consciously unaware of doing so, they'd been following the First and Second Laws all along. Cutie knew, on some level, that it'd be more suited to operating the controls than Powell or Donavan, so, lest it endanger humans and break the First Law by obeying their orders, it subconsciously orchestrated a scenario where it would be in control of the beam.

Eu me lembro de um romance de Isaac Asimov no qual foi revelado, no final do livro, que um cientista brilhante estava trabalhando duro em um plano para usar cérebros positrônicos para conquistar os outros mundos colonizados por humanos da Galáxia ( apenas algumas dúzias, penso eu, naquela época) sem que os cérebros positrônicos saibam que estavam quebrando as Três Leis. No entanto, este plano nunca foi realmente usado, então isso só se qualifica como uma explicação canônica de "como os AIs poderiam foram usados para agir contra os melhores interesses da humanidade se o herói não havia parado o vilão antes que ele pudesse colocar em prática seu plano diabólico. "

Estou pensando em O sol nu , publicado pela primeira vez como uma revista em série em 1956. Foi a segunda aventura de Lije Baley e R. Daneel Olivaw - um detetive humano e um detetive robô, trabalhando juntos em um mistério de assassinato futurista.

"Consider a spaceship with a built-in positronic brain as compared to a manned spaceship. A manned spaceship could not use robots in active warfare. A robot could not destroy humans on enemy spaceships or on enemy worlds. It could not grasp the distinction between friendly humans and enemy humans.

"Of course, a robot could be told that the opposing spaceship had no humans aboard. It could be told that it was an uninhabited planet that was being bombarded. That would be difficult to manage. A robot could see that its own ship carried humans; it would know its own world held humans. It would assume that the same was true of enemy ships and worlds. It would take a real expert in robotics, such as you, Dr. Leebig, to handle them properly in that case, and there are very few such experts.

"But a spaceship that was equipped with its own positronic brain would cheerfully attack any ship it was directed to attack, it seems to me. It would naturally assume all other ships were unmanned. A positronic-brained ship could easily be made incapable of receiving messages from enemy ships that might undeceive it. With its weapons and defenses under the immediate control of a positronic brain, it would be more maneuverable than any manned ship. With no room necessary for crewmen, for supplies, for water or air purifiers, it could carry more armor, more weapons and be more invulnerable than any ordinary ship. One ship with a positronic brain could defeat fleets of ordinary ships. Am I wrong?"

Ninguém é descrito como apresentando um bom contra-argumento para demonstrar que isso não poderia funcionar. (E é subsequentemente estabelecido que um dos suspeitos do caso estava, de fato, trabalhando em tal esquema em seu tempo livre, embora isso não estivesse nem aí para se tornar uma realidade ainda).

Nesse caso hipotético, os robôs-navios não pensariam que estavam violando as Três Leis, mas da perspectiva de um humano cujo planeta natal estava sendo retirado de órbita até se render ao sujeito controlando a frota de robôs, o fato de que os cérebros positrônicos nessas embarcações não tivessem intenções maliciosas em relação aos seres humanos seria uma distinção sem sentido.

Genocídio de não humanos.

Em Foundation's Edge está implícito que os robôs usando travessuras de viagem no tempo são a razão pela qual os humanos nunca encontram alienígenas na Via Láctea, apenas planetas vazios prontos para serem colonizados.

it is said, it was the robots who established Eternity somehow and became the Eternals. They located a Reality in which they felt that human beings could be as secure as possible— alone in the Galaxy.

A primeira lei protege os humanos. Nem toda a vida inteligente. Se os robôs decidirem que os alienígenas podem representar uma ameaça para os humanos, eles são obrigados a cometer genocídio. Como demonstrado em Robots and Empire, também é possível que um grupo de indivíduos (nominalmente) humanos redefina "humano" para excluir a maioria da humanidade.

Os reténs para tentar juntar-se a série de robôs e fundação e histórias autorizadas tornam um pouco confuso, mas o apagamento de civilizações alienígenas é um efeito colateral bastante lógico que cai fora.

link

robot ships operating under Auroran programming cruise the galaxy ahead of the spreading Settlers, terraforming and preparing planets for colonization. Meme entities will later claim that this devastated existing races. Memes escape to the Galactic Core.

A maioria das pessoas provavelmente consideraria isso maléfico porque temos algum conceito de que provavelmente não é correto genocidar todas as outras raças a fim de proporcionar mundos férteis e seguros para os seres humanos.

Isso é especialmente ruim se os robôs ou os programadores puderem decidir que nem todos os humanos contam como "humanos" para os propósitos das leis.

"Tell him we are human beings and must be obeyed."

The robot spoke then, in peculiar but understandable Galactic. "I understand you, Outworlder. I speak Galactic. We are Guardian Robots." "Then you have heard me say that we are human beings and that you must therefore obey us."

"We are programmed to obey Rulers only, Outworlder. You are not Rulers and not Solarian.

...

"But this child," and Pelorat looked at Fallom, whom he was still carrying, "is a Solarian. It will tell you what to do and you must obey it."

Fallom looked at Pelorat with eyes that were open, but seemed empty. Bliss shook her head, sharply, but Pelorat looked at her without any sign of understanding.

The robot's eyes rested briefly on Fallom. It said, "The child is of no importance. It does not have transducer-lobes."

"It does not yet have fully developed transducer-lobes," said Pelorat, panting, "but it will have them in time. It is a Solarian child."

"It is a child, but without fully developed transducer-lobes it is not a Solarian. I am not compelled to follow its orders or to keep it from harm. "

~ Fundação e Terra

Valores humanos

Você caracteriza as leis como "realmente funcionando perfeitamente", mas não acho que seja certo. Você está falando sobre como as histórias tendem a mostrar que sim, na verdade, elas estavam de fato seguindo as leis de alguma maneira contraintuitiva, mas levando a resultados que a maioria dos humanos não desejaria.

A maioria dos livros de Asimov envolve resultados possíveis inesperados e indesejáveis das três leis porque as leis protegem a vida humana, mas não protegem todos os valores humanos. Só porque você se preocupa com algo não coberto pelas três leis (como genocídio / tortura / escravidão de não-humanos) não significa que os robôs se importem. Em geral, os humanos se importam com muitas outras coisas além de permanecerem vivos.

Talvez não seja "um dano significativo para a humanidade", mas o robô no conto de Asimov Mentiroso! causa danos significativos a alguns humanos, embora esteja tentando o seu melhor para seguir as Três Leis.

Por causa de um erro de fabricação, o robô pode ler mentes, o que faz com que ele se deite com os humanos, dizendo-lhes o que sabe que eles querem ouvir. A tragédia começa quando os humanos agem com base nessa informação falsa. O robô eventualmente percebe que causou danos ao tentar evitar causar danos, e a confusão resultante faz com que seu cérebro se agarre.

De acordo com o ISFDB , o conto foi publicado pela primeira vez na edição de maio de 1941 da Astounding Science Fictions. .

. . Que Tu és consciente sobre ele

Esta história foi o cenário para tentar resolver os problemas com a segunda lei, que sendo "Um robô deve obedecer a ordens dadas por seres humanos, exceto quando tais ordens entrem em conflito com a Primeira Lei", isto é um problema quando trata de permitir que a população humana em geral entre em contato com robôs fora de ambientes controlados.

"When the Second Law directs me to obey a human being, I must take it to mean that I must obey a human being who is fit by mind, character, and knowledge to give me that order; and where more than one human being is involved, the one among them who is most fit by mind, character, and knowledge to give that order."
"And in that case, how will you obey the First Law?"
"By saving all human beings from harm, and by never, through inaction, allowing any human being to come to harm. Yet if by each of all possible actions, some human beings will come to harm, then to so act as to insure that the human being most fit by mind, character, and knowledge will suffer the least harm."
"Your thoughts accord with mine," whispered George Ten.
"Now I must ask the question for which I originally requested your company. It is something I dare not judge myself. I must have your judgment, that of someone outside the circle of my own thoughts....Of the reasoning individuals you have met, who possesses the mind, character, and knowledge that you find superior to the rest, disregarding shape and form since that is irrelevant?"
"You," whispered George Nine. "But I am a robot. There is in your brain paths a criterion for distinguishing between a robot of metal and a human being of flesh. How then can you classify me as a human being?"
"Because there is in my brain paths an urgent need to disregard shape and form in judging human beings and it rises superior to the distinction between metal and flesh. You are a human being, George Ten, and more fit than the others."
"And I find that of you," whispered George Ten. "By the criteria of judgment built into ourselves, then, we find ourselves to be human beings within the meaning of the Three Laws, and human beings, moreover, to be given priority over those others."

Embora não haja nenhuma maldade real na história, ela acaba com uma nota de potencial malícia sem violar as três leis

Their thoughts were oddly alike and might not be distinguished. They thought: The human-beings-like-the-others might never have intented to blur the distinction between themselves and the human- beings-like-the-Georges. Yet they had done so inadvertently.
They might now realize their mistake and attempt to correct it, but they must not. At every consultation, the guidance of the Georges had been with that in mind.
At all costs, the Georges and those that followed in their shape and kind must dominate. That was demanded, and any other course made utterly impossible, by the Three Laws of Humanics.

Existem vários exemplos de robôs mal comportados nos escritos de Asimov, mas não tenho conhecimento de um robô Asenion padrão e não modificado (programado com as Três Leis) que possa ser considerado malévolo. Alguns, sem saber, magoaram ou mataram seres humanos (como Hannis Gruer em The Naked Sun ) ou se comportaram de forma inesperada (Herbie de Liar! ), mas parece que corretamente programados Os robôs Asenion estão a salvo da ação direta.

Um tema recorrente parece ser sobre a definição de um ser humano . Existem vários robôs Asenion que têm restrições adicionais, permitindo-lhes violar o espírito das leis. Você deve proteger os seres humanos? Altere a definição de um ser humano. Você deve obedecer ordens? Especifique de quem o pedido deve vir.

O terceiro exemplo abaixo é o único robô Asenion não modificado que eu encontrei e que não se comporta como deveria. No entanto, eu não iria tão longe a ponto de dizer que ele é malévolo.

Obedeça quem?

Em Fundação e Terra , os protagonistas encontram um robô que está programado para obedecer apenas a certas pessoas:

"We are programmed to obey Rulers only, Outworlder. You are not Rulers and not Solarian."

Eles também estão com raiva:

"If you make difficulties, then we will partially inactivate you. We will damage you and you will then tell us what we want to know. [...] I am under detailed instructions," said the robot quietly. "I can do this. Of course, I shall do as little damage as is consistent with obtaining information."

Eles também parecem não se importar com o bem-estar de outros humanos:

"There is no Ruler Fallom. There is only a child and we have an excess of children. It will be destroyed."

Bliss said forcefully, "You dare not. It is a child!"

"It is not I," said the robot, "who will necessarily do the act and it is certainly not I who will make the decision. That is for the consensus of the Rulers. In times of child-excess, however, I know well what the decision will be."

Este é um robô que:

1. se recusará a obedecer a um ser humano;
2. ativamente permitirá que o mal chegue ao ser humano arrebatando a criança e entregando-a aos Governantes;
3. permitirá que o mal chegue a um ser humano por meio da inação deixando que os Governantes o matem e
4. ameaça a violência física para coletar informações.

O que é um ser humano?

Em Robôs e Império , os protagonistas se encontram conversando com uma mulher robô que não pensa que eles são humanos:

The woman listened for a moment and then said, in an accent so thickly Solarian as to seem almost comic when coming from her perfectly shaped mouth, "You are not a human being."

She then flashed into action so quickly that Gladia, still some ten meters off, could not see in detail what had happened. She saw only a blur of motion and then D.G. lying on his back motionless and the woman standing there with his weapons, one in each hand.

Ela ataca um dos robôs dos protagonistas enquanto repete "Você não é um ser humano". Eu suponho que ela teria feito o mesmo com um ser humano real. Ela então ataca todos eles:

Landaree shouted in a stentorian contralto, "All of you. Join me! The two apparent males are not human beings. Destroy them without harming the female in any way."

Depois, Daneel foi da opinião de que o robô teria matado Gladia, mesmo que tivesse estabelecido que ela era um ser humano:

"You fool! I was safe. She would not have harmed me."

Daneel said, "Madam, it distresses me to contradict you, but I think she would have done so as her urge to destroy me grew stronger."

Se isso é realmente verdade é desconhecido. Daneel pode ter se enganado, pois o robô não tentou disparar sua arma até que Gladia tivesse sido removida do caminho.

A explicação para o comportamento dela é bastante simples:

"To the overseer, on the other hand, the key property of a human being was speech, Captain. The Solarian accent is a distinctive one and to the overseer something that looked like a human being was defined as a human being only if it spoke like a Solarian. Apparently, anything that looked like a human being but did not speak with a Solarian accent was to be destroyed without hesitation, as was any ship carrying such beings."

Dano vs. Ordens

O último exemplo que eu tenho é de Os Robôs da Aurora , onde um robô foi instruído tão strongmente e habilmente que obedecer a uma ordem temporariamente se torna mais importante que a segurança de um ser humano:

The robot, seen as a darker shadow against the darkness, seemed a large one. He had, somehow, an air of capability about him. He said, "Your pardon, sir. Did you not have the company of two robots?"

"Gone," muttered Baley, acting as ill as he could and aware that it did not require acting. A brighter flash of the heavens made its way through the eyelids that were now half-open.

"Gone! Gone where, sir?" And then, as he waited for an answer, he said, "Are you ill, sir?" [...]

If the robot had been without special instruction, he would have responded to Baley's clear signs of illness before doing anything else. To have asked first about the robots implied hard and close-pressed directions as to their importance. [...]

"I am well. Don't concern yourself with me." It could not possibly have convinced an ordinary robot, but this one had been so intensified in connection with Daneel (obviously) that he accepted it.

O robô acabaria deixando Baley em sua terrível situação, sem mais hesitação:

He had forced robots to leave a patently unwell human being by playing on the force of the orders given them by a competent robot master who had been intent on strengthening the Second Law for his own purposes-and had done it to the point where Baley's own quite apparent lies had subordinated the First Law to it.

Se esse robô pode ser considerado "malévolo", não está claro para mim. Poderia ter decidido que Baley não estava prestes a ser ferido, já que ele estava seguro em um veículo durante o que parece ser uma tempestade de força incomum ("Não foi uma tempestade particularmente ruim, mas um terráqueo deve ter parecido irresistível. ") Poderia ter concluído que o sofrimento de Baley era mais psicológico e que ele não corria perigo imediato.

"Cal" (1991) é uma história tardia de Asimov em que, no final da história,

a robot resolves to take his owner's life.

Aqui está uma passagem do final da história:

Must I obey the laws?

I feel I must think of myself and if necessary, I must kill the tyrant. ...

But can I do it? Won't the Laws of Robotics hold me back?

No, they will not hold me back. I know they won't.

There is something far more important to me than they are, something that dictates my actions beyond anything they can do to stop me.

I want to be a writer.

É notável que a história termine com essa passagem; é deixado ambíguo se Cal sucede em seu plano. Também é notável que, ao longo da história, um técnico tenha mexido com o cérebro de Cal de maneiras inéditas:

"I'm not sure. Suppose he writes satire. That's one thing I don't write, so we won't be competing, and the Three Laws of Robotics won't get in his way. I want you to give this robot a sense of the ridiculous."

"A sense of the what?" said the technician, angrily. "How do I do that? Look, Mr. Northrop, be reasonable. I can put in instructions on how to run a Writer, I can put in a dictionary and grammar. But how can I possibly put in a sense of the ridiculous?"

"Well, thing about it. You know the workings of a robot's brain patterns. Isn't there some way of readjusting him so that he can see what's funny, or silly, or just plain ridiculous about human beings?"

"I can fool around, but it's not safe."

É plausível que as ações do técnico realmente tenham causado a superação das Leis, embora isso possa entrar em conflito com a descrição das Leis em outras histórias. Como observado nos comentários, também não está claro se Asimov pretendia que essa fosse uma história do robô "canônica".

Embora não tenha sido escrito pelo próprio Asimov (mas foi aprovado por ele), a série Caliban de Roger McBride Allen explora os temas de lei modificada ee sem robôs de lei. E certamente há muita violência por aí, algumas das quais são perpetradas por um robô em particular - especialmente em nome de se proteger.

• Caliban de Isaac Asimov
• Inferno de Isaac Asimov
• Utopia de Isaac Asimov

Ele também explora enganar um sistema de computador a acreditar que os dados que ele está modelando são para uma simulação de um sistema humano, em vez do sistema humano real. Isso foi feito para evitar que ele falhasse se o computador soubesse que suas soluções eram aplicadas a seres humanos reais, já que suas soluções tinham potencial para causar danos reais.

Sobre "Eu achei que todas as histórias de robô de Asimov eram sobre como as leis sempre funcionariam": Não realmente. As histórias de Asimov eram freqüentemente sobre situações em que as leis falharam / tiveram consequências não intencionais, porque são ambíguas e imperfeitas. Nas próprias palavras de Asimov:

This does not mean that I wasn’t aware from the start that there were serious ambiguities in the Three Laws. It was out of these ambiguities, indeed, that I wove my stories. In The Naked Sun the ambiguities could even lead to robot-induced murder.
_{—Afterword in Final Stage (1974)}

Em relação aos danos (potenciais) para os seres humanos: Além dos exemplos já publicados, aqui estão mais dois:

Em Robot Dreams (1986), quando um robô revela seus sonhos, Susan Calvin considera tão perigoso que ela destrói imediatamente:

“[…] it seemed to me there was neither First nor Second Law, but the only the Third, and the Third Law was ‘A robot must protect its own existence.’ That was the whole of the Law. […]
    “It seemed to me, in my dream, that eventually one man appeared.”
    “One man? Not a robot?”
    “Yes, Dr. Calvin. And the man said, ‘Let my people go!’” […]
    “And when he said ‘Let my people go,’ then by the words ‘my people’ he meant the robots?”
    “Yes, Dr. Calvin. So it was in my dream.”
    “And did you know who the man was—in your dream?” […]
    And Elvex said, “I was the man.”
    And Susan Calvin at once raised her electron gun and fired, and Elvex was no more.

Outro robô potencialmente perigoso é apresentado em Christmas Without Rodney (1988) . Depois de ser mal tratado, Rodney observa que ele desejava que as Três Leis não existissem, o que preocupa muito seu dono:

I admit that Rodney was greatly tried, but a robot can’t wish the laws of robotics did not exist. He can’t, no matter what the circumstances. […]
    But if I do nothing, I live with a robot capable of wishing the laws of robotics did not exist. From wishing they did not exist to acting as if they did not exist is just a step. At what moment will he take that step and in what form will he show that he has done so?
    What do I do? What do I do?

(ênfase minha)

Se você olhar para "Que Tu és Ciente Dele", você descobrirá que George 9 e 10 estão discutindo o que efetivamente será o controle da humanidade "para seu próprio bem". Eles já colocaram em ação as etapas para tornar os robôs mais aceitáveis.

Em "Runaround" - é uma dissonância cognitiva que levaria à lesão dos seres humanos - embora, como de costume, Donovan encontre uma solução.