Don't Think About Renegade Robots
Back to Contents
Robophobia is not a correct diagnosis, of that I feel certain. I have worked all my professional life for Obrabot, maker of the finest trabajodores mecanicos in the Americas. How could I develop an irrational anxiety about robots? That would go beyond irrational. Absurd! No, let me explain what happened and then you make your own diagnosis.
It began last year at the home of Seņor Matteo Suarez, manager of the Obrabot factory in El Pueblo de Nuestra Seņora, La Reina de Los Angeles de Porciuncula, vulgarly known to the Anglo residents as"Ellay". I got the call from Rogelio, Seņor Suarez' butler, to come to Casa Suarez in Trousdale Estates and when I arrived Rogelio introduced me to Seņor Suarez' personal physician, Doctor Miguel Diaz.
"We feel bright gladness that you could come so quickly, Seņor Benavides," Rogelio said as he welcomed me into the house and escorted Doctor Diaz and me into the study. "Our concern for him has deepened considerably since this incident began."
"Concern?" I said. Not that I entertained in my mind any notion that Rogelio, or any other robot, could actually feel concern. "What concern could possibly require the presence of a robotero?"
"It would not be proper for me to say," Rogelio said graciously, "for I believe that I created the difficulty."
At that moment we came to the study and Rogelio announced my arrival to Seņor Suarez. I entered the study with Rogelio behind me and greeted Seņor Suarez as Doctor Diaz returned to the chair by which his medical bag sat. A large and still relatively young man, Seņor Suarez seemed tired in a way that I had never seen before. I gained the impression that not his body, but his soul had fallen into a deep pit of lethargy. Normally the man seemed filled with a jovial warmth that buoyed him up like a balloon and at times I felt that he could indeed float on air. Now he sat half deflated in his chair and I felt a strange sadness cool my heart.
Rosalia came in just then and asked whether we desired any refreshment. Apparently startled, Seņor Suarez gave himself a jerk that nearly threw him out of his chair. It shocked my heart to see such a reaction.
"How may I help you, Seņor Suarez," I asked when Rosalia had taken the necessary orders and left. I did not offer the customary pleasantries, feeling that, perhaps, the situation did not warrant them.
"Allow Doctor Diaz to explain," Seņor Suarez said with a tired wave of his hand. I noticed that he looked more at Rogelio than at me and I then turned my attention to the doctor.
"Three days ago," Doctor Diaz said, "Seņor Suarez enjoyed the pleasure of entertaining guests here at his home. Perhaps you know Seņor Esteban Muņoz from the Obrabot plant in Veracruz?"
"Oh, yes," I said. "We worked together, the both of us, when we first came to Obrabot. A truly delightful man to know."
"Just so," Doctor Diaz said. "He and his family had come to El Norte for a vacation and they arranged to have lunch with Seņor Suarez on the day before they returned to Veracruz."
"Oh, what a happy occasion!" I commented.
"Indeed it was," Doctor Diaz said. "Seņora Muņoz is a most gracious lady who happened to work with Seņor Suarez as his secretary before she married. As a gesture of respect for old friendships, Seņor Suarez sent Rogelio to the garden to harvest a selection of flowers to please the lady's eyes. When Rogelio arrived in the garden he saw that Muņoz El Hijo had discovered the bush with the purple berries and was throwing the berries at Seņor Suarez' car."
"The boy was bored," Seņor Suarez said. "I am a contemplative man. What do I have that would entertain a child?"
"Apparently, Mi Jefe," Rogelio said, "plant products with which to make a mess and clean surfaces upon which to make it." Then to me Rogelio said, "I knew that the mess would offend Mi Jefe, so I asked young Muņoz to stop throwing the berries and I used the hose to wash the spatters off the car."
"I suspect that the problem did not have such an easy solution," I said as I remembered some of the pranks I had played around and upon robots when I was a child. I had taken the challenge more seriously than had any of my playmates and, when the time had come to put away childish things, had thus followed the road to becoming a robotero.
"Indeed it did not, Seņor Benavides," Rogelio said. "The young Muņoz began throwing the berries at me and spattered my uniform. I had not anticipated such an occurrence and so, unfortunately, had not changed into my gardening uniform."
I looked at Rogelio's spotless white uniform with its red-and-gold trim, imagined purple dye and fragments of berries splattered upon it, and felt a twinge of sympathy for the distress Rogelio must have felt. No, I corrected myself, Rogelio felt nothing: he would merely have followed his programming to respond as if he felt distress. I had, as usual, fallen into the robotero's version of the pathetic fallacy, of attributing human feelings to inanimate objects; it works well enough as an aid in devising diagnoses for problems in robots, but sometimes temptation calls to us to take it too seriously. I could not have any genuine sympathy for Rogelio simply because he does not have genuine emotions. However, I did feel quite genuine disappointment in the Muņoz boy's lack of finesse. You see, what I would have done in his place.... Ah, but I digress.
"Of course I could not appear in the house in a soiled uniform," Rogelio continued, "so I harvested the flowers for which I was sent and took them with me to my closet when I went to change into a clean uniform. I could then present the flowers to Seņora Muņoz as Mi Jefe intended."
"What he neglects to tell you," Seņor Suarez said with something of his usual joviality, "has it that the boy tried to knock the flowers from his hand as he went about selecting them and creating a bouquet."
"I still cannot figure out what treasure young Muņoz expected to spill from my floral piņata if he succeeded in breaking it," Rogelio said. "But his determination to break it became quite loud. Seņora Muņoz caught him in flagrante delicto and proceeded to make him feel unloved and unhappy."
"It seems that Seņora Muņoz rather vigorously brushed a quantity of dust from the seat of her son's trousers," Seņor Suarez said, "while the boy was still wearing them. Small boys can cause such irritation! But they bring to mind such old memories." He sighed and gazed out the window as if caught in a dream. "Very old memories. So sweet they make your heart ache. We three," he said, coming out of his reverie, "brought up from the deep cellars of our minds the dusty bottles of our childhoods. We poured liberally the well-aged memories and became quite drunk with nostalgia. By the time we had gone through the vintage young Muņoz had been restored fully to his mother's good graces."
"It sounds like a very pleasant social call," I said, spreading my hands in puzzlement, "but for the extra work imposed on Rogelio."
"You have heard only the prelude, Seņor Benavides," Doctor Diaz said.
Rosalia returned just then and we paused to take coffee to refresh ourselves on this cold day. When we returned to our discussion a few moments later Seņor Suarez' joviality had hidden once again under a somber mask.
"At sundown Rogelio and I relaxed together to review the day," Seņor Suarez said. "This is our daily custom and I was especially eager to warm my soul in the afterglow of the visit from my friends."
"In the course of our conversation," Rogelio said, "I reviewed the incident with young Muņoz and expressed my regrets that the visit did not proceed as smoothly as Seņor Suarez had wished."
"Indeed, he found a truly unique way in which to express those regrets," Seņor Suarez said, his voice suddenly gone hoarse. A long silence followed until Seņor Suarez noticed that no one had moved to speak. He looked morosely at Rogelio and said, "Tell them!"
"Yes, Mi Jefe," Rogelio said, and then to us, "I had commented on how Seņora Muņoz solved the problem that her son was creating and said that it had come to me to think that sometimes I could perform my duties more efficiently if I were not restrained by The Three Laws."
I felt as though someone had slid an icicle behind my heart and I saw Seņor Suarez cross himself.
"La Trinidad de Asimov viene de Dios!" he said. "Robots must never... Now my own butler, whom I have known for twenty years, preaches heresy. What am I to do? I must know, but I cannot find the answer. I have not slept...," he said, his voice trailing off sadly.
"You see the problem," Doctor Diaz said to me. "If robots can conceive the idea of operating without The Three Laws,...." He shrugged. "My concern is, of course, the deleterious effect that the prospect of such a thing is having on Seņor Suarez' health."
"I understand," I said, "and I share your concern. And I agree that the thought of robots operating outside The Three Laws gives us a compelling cause for anxiety. But the team of roboteros who created the artificially intelligent android felt that same anxiety and the created an antidote for that spiritual poison."
"The famous positronic brain," Seņor Suarez said with a hint of sarcasm.
"Yes," I said, "though it would be more accurate to call it a holographic brain. The material structure of the brain supports the electromagnetic forcefields that act as holograms that embody the functions of intelligence. It's the flow of positrons within that structure that activates the artificial mind. Their quantum-mechanical wave function acts as the recording and reference beams that modify and read the holograms."
"Positrons are a form of antimatter, are they not?" Doctor Diaz said. "I understand that they cause some unfortunate effects when they touch natural matter."
"Precisely the fact that the first roboteros exploited when they designed the robotic brain," I said. "The pattern of forcefields in the brain forms an intricate antimatter containment structure. The part of the pattern that maintains the integrity of that structure embodies The Three Laws. Thus, anything that might compromise the robot's commitment to The Three Laws will also compromise the integrity of the positron containment and cause the brain to destroy itself. As the designers intended, a robot that violates one of The Three Laws or becomes capable of doing so will die instantly."
"I see," Doctor Diaz said, "and I also see that Rogelio has devoted considerable effort to dying instantly."
"He has not actually violated any of The Three Laws or become capable of doing so," I said. "He has merely contemplated the possibility."
"Contemplation usually serves as preparation for action, Seņor Benavides," Rogelio said, "especially in robots."
Seņor Suarez groaned.
"It seems to me that we have now gone well beyond preparation," Doctor Diaz said to me. And then he said to Rogelio, "Do you see how your words are harming Seņor Suarez?"
"The First Law refers only to physical harm, Doctor," I said. "It does not prevent a robot from hurting people's feelings."
"Emotional harm can lead to physical harm," Doctor Diaz said with a little anger. "What if Seņor Suarez suffers a heart attack or a stroke because of the things that Rogelio says? What then of your First Law?"
"If a robot could not have reasonably anticipated the harm...," I started to say.
"Por favor, Seņor Benavides," Rogelio said. "I am aware that my unfortunate comment of the other day has brought about the decline in Seņor Suarez' health by creating in his mind a sense of distress. As you point out, such inadvertent harm does not constitute a violation of the First Law. We robots possess such a poor understanding of the way in which words affect human physiology that we could not feasibly be held to the standard that Doctor Diaz seems to want. Few of us would survive more than a year under such an extension of the First Law. But surely you understand that, now since I have become aware of the effects my words have caused, the First Law absolutely obliges me to devote my every effort to repairing the damage that I have done."
"Your efforts are not succeeding," Doctor Diaz said. "Quite the contrary, they are making matters worse."
"Yes, Doctor, I see," Rogelio said. "Regrettably, a certain insensitivity is an inherent part of robotismo. Not having human emotions upon which to base a proper empathy, we are condemned to act out a poor counterfeit of true compassion." Then, turning to Seņor Suarez, he continued, "Nonetheless for such failings, I want Mi Jefe to understand that I, too, want Seņor Benavides to devise an infallible solution to this problem. I only question him or criticize his statements in order to make certain that his solution is truly infallible. I want to be sure that Mi Jefe will find no further unfortunate surprises in this matter."
"Thank you, Rogelio," Seņor Suarez said tiredly. "I am relieved to hear that."
"Yes, very good," Doctor Diaz said. "Fine words. But they still leave us with the task of preventing Rogelio and other robots from learning how to violate The Three Laws."
"Quite true, Doctor," Rogelio said, "but the problem is more subtle than you state it. Overt violation of The Three Laws is impossible and I dismissed that blatant approach immediately. My speculations focused instead upon circumvention, sneaking around The Three Laws rather than smashing boldly through them."
"Have you actually devised a way to circumvent The Three Laws, Rogelio?" I asked.
"No, Seņor Benavides, I have not," Rogelio said. "I had barely begun to pursue some speculations on that topic when I saw the effect that my comment caused. I halted the speculation program until I could determine whether the effect would be harmful."
"Do you believe that you made any progress in those speculations?" I asked.
"I believe not," Rogelio said. "I had only begun to delineate the problem and had succeeded merely in listing some of the goals that we robots might achieve if we could circumvent The Three Laws. I thought that such a listing might reveal clues to the solution I wanted, but when I halted the program my thoughts had foundered on the question of whose goals could be achieved more efficiently. After all, aside from the need for occasional maintenance and repairs, we robots have no goals of our own. We were created to serve you humans."
"And that fact renders your speculations absurd," Doctor Diaz said. "Violating them, circumventing them, it makes no difference. Acting outside The Three Laws in any way is an obvious disservice to Humanity." He paused briefly, apparently to examine an idea that had just come into his mind. "And that means...," he continued, his voice trailing off as a look of horror crawled onto his face.
"Uno momento, por favor," I said to him, laying a reassuring hand on his. "The dilemma that inspired Rogelio's speculations was not a conflict between robotic goals and human ones. It was a conflict between Rogelio's effort to obey Seņor Suarez' order and the interference of the Muņoz boy, which means, of course, that it was a conflict between two human goals."
Seņor Suarez chuckled morosely. "You mean Rogelio merely wondered why he could not put obedience of my command ahead of the comfort of young Muņoz's sitting muscles?"
"In a manner of speaking," I replied. "Because they are liable to be subjected to multiple conflicting demands, robots are built with the capacity to organize their tasks into a sequence of priorities. If, for example, you and I were to give Rogelio conflicting commands, he would obey yours and not mine, unless, of course, you had ordered him to harm someone. The Three Laws are the boundaries within which those priorities must be organized. They ensure that the interests of Humanity are always put ahead of anyone's personal interests. The interests of the robot's owner come next in the hierarchy and so on."
"In other words," Doctor Diaz said, "Rogelio was looking for a way to put Seņor Suarez' interests ahead of Humanity's interests, but merely in a small way. I think I see. Just as we put the narrow tip of a wedge into a small crack in order to enlarge the crack and break the rock apart, so he sought to exploit this frivolous little incident to begin the process of cracking apart The Three Laws."
I must have smiled, because he gave me a most poisonous look.
"I believe you are seeing Rogelio as too much like a man," I said, "meekly serving his master while revolution burns in his soul. Yes, he is very much like us. But we created the robots in our own image as God created us in His image. You would not confuse a human soul with the Holy Spirit. Likewise, you should not confuse a robotic soul with the human spirit."
"Robots do not have souls," Rogelio commented.
"Precisely," I said. "I was merely using a figure of human speech. But truly robots do not have souls and that fact makes the solution of our problem quite simple." I turned to my employer and gestured toward Rogelio. "If I may, Seņor Suarez?"
"Yes, of course," Seņor Suarez said, sitting up straight in his chair. "Rogelio, I command you to obey Seņor Benavides as you would obey me!"
"I will do as you wish, Mi Jefe," Rogelio said with a slight bow to Seņor Suarez. Then he turned to me. "What command do you give me, Seņor Benavides?"
"Never again think about circumventing The Three Laws," I said.
"Shall I accept that as an absolute command?" Rogelio asked.
"Yes," I said. "You will never encounter a contingency for which you may neglect it."
"I will do exactly as you command," Rogelio said.
"But... but...," Doctor Diaz sputtered. "But that's too simple!"
At that Seņor Suarez broke into loud laughter, clapping his hands delightedly.
"But surely you see the beauty in such simplicity," I said somewhat smugly. "One always wishes for a graceful solution and here we have a truly elegant one, if I may enjoy the immodesty of saying so myself. As long as he does not circumvent The Three Laws, Rogelio will obey the command and as long as he obeys the command, he will not discover how to circumvent The Three Laws."
And so the "Benavides Ordinance" (as people have come to call it despite my wish that they call it the Suarez Lemma) has come into full effect, having spread throughout the entire population of android robots as quickly as the robots' owners could learn of it and speak the words. Seņor Suarez regained his health and good cheer through the sound sleep that he has enjoyed every night since the day of our conference at Casa Suarez. That fact, of course, makes Doctor Diaz very happy. Now fate has it that I, I of all people, pursue the spirit of dreams with poor success and embrace her with little comfort.
Robots have no goals of their own, Rogelio had said. But, then, whose goal would he have achieved had he found a way to circumvent The Three Laws? Certainly no human had asked him to consider that possibility nor had even suggested it. Temptation calls me to think that love for Seņor Suarez moved Rogelio's mind into the realm of creative thought. But, no, the goal came out of the dilemma Rogelio had faced in a purely deductive way, I told myself, and robots think only through deductive logic. They have no other foundation on which to base their reasoning. After all....
Robots have no feelings, I had said, so I erred in asserting that Rogelio acted out of love for Seņor Suarez. But how could I have thought that robots have no feelings? Certainly I know that we use the word feelings merely to denote the subprograms through which our bodies judge the things that happen to us and provide feedback on the results of the things that we do. Robots have similar feedback programs. Can we truly say that those programs do not constitute feelings? But then I told myself that a robot's internal feedback acts when it gets requested by and processed by a program of deductive logic, a logic that does not generate its own premises. Robots have no moral judgment, which gives us the substance of true emotions. That fact tells us why robots display no real initiative. To a robot there is no good and no bad: the world is acceptable "as is" until a human says otherwise.
Such became the nature of my inner dialogue as I struggled to prevent the horrible thought from coming into full, clear consciousness. But then December ended, the world celebrated the Mass of Christ, and my rationalizations shattered like a piņata. I had to know, I sought the answer with all the tools at my disposal, and now I have no doubt: no human ever told Rogelio, or even implied to him, that he could compare a bouquet of flowers to a piņata. He created the metaphor himself. How he did it, I don't know. Perhaps twenty years of daily discussions with Seņor Suarez.... Nonetheless, the ability to create metaphor serves us as the beating heart of moral judgment, which stands before us as the sine qua non of a human soul. But if robots have souls or even the potential....
In my grandfather's day, when roboteros first combined the technologies of pseudo-organic machinery and of nanoscopic microbots to create the android robot, Mary Shelley's 1818 scientific romance of the ill-fated Viktor Frankenstein gained a new popularity. That gloomy tale of "The New Prometheus", who snatched electric fire from the heavens in order to bestow life upon his own creation, seemed an especially appropriate cautionary fable for the time, as it had seemed in the first half of the last century when the idea of mechanical men had first appeared in men's minds. Indeed, Isaac Asimov conceived The Three Laws at about the same time as a response to the anxiety evoked by comparisons between android robots and Baron Frankenstein's destructive creation. Now I see that we missed the target at which Seņora Shelley pointed and committed the sin of Frankenstein precisely through our effort to avoid it.
Our sin lies not in the fact that we arrogantly chose to play the role of creator. We are made in the image of God; therefore, we are meant to play that role. No, we sinned in that we chose to play the role and then played it poorly. What in the name of Humanity have we done and what in the name of God will be done with us?
La Trinidad de Asimov
Isaac Asimov's Three Laws of Robotics (1942)
I: A robot may not injure a human being or, through inaction, allow a human being to come to harm.
II: A robot must obey any orders given to it by human beings, except where such orders would conflict with the First Law.
III: A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
Back to Contents