“There will one day spring from the brain of science a machine or force so fearful in its potentialities, so absolutely terrifying, that even man, the fighter, who will dare torture and death in order to inflict torture and death, will be appalled, and so abandon war forever.”
– Thomas A. Edison
Would it be unethical to make this Weapon? How many lives would we save by making a Weapon that would constantly check Man’s dark side and keep us away from war? Yet to make something so brutal, so malicious, it inherently feels so wrong, doesn’t it? But let’s look at it a little bit closer, shall we?
The question is not whether or not this is feasible. The advances we have made in the last fifty years are so numerous it would be frivolous to argue otherwise. The relevant questions are (1) what this machine / force would be and (2) what this would do to humanity.
To follow on with Edison’s quote, I do not believe this terrifying force would be a simple weapon (like a gun). For it to be something that truly scares humanity, it must act on its own. If the weapon if so brutal, it can simply not be used and alternatives can be invented. Thus it must have its own mind. Two examples immediately come to mind (of similar nature): The Patriots (of Metal Gear Solid) and Helios (of Deus Ex).
Disclaimer: Some of the material on the Patriots may change when Metal Gear Solid 4 is released, and on Helios when Deus Ex 3 is released.
Both are artificial intelligences that aim to help humanity, acting as benevolent dictators to some extent. The Patriots control content (in their words, they create context) and claim to keep humanity from destroying themselves. They are much less forceful than Helios is; they do not try to stop war, but keep it at a level they consider to be “healthy” given the state of the world. Helios, on the other hand, appears to have no objection doing whatever is necessary to keep humanity “under control”. It controls the Internet and everything even remotely electrical and is well aware of what it believes it has to do to save humanity from itself.
Neither of these “beings” have taken this role for selfish reasons: they both aim to protect humanity from itself. They both see humanity as a curious baby with a fork near a light socket, seemingly desperate to kill itself. They act as if they are our parents, talking down to us and taking away our toys (in Helios’ case, stripping power from all the government buildings) as if we are children.
This is not to repeat what I have said earlier about the Patriots’ rant on how humanity cannot take care of itself and needs a guardian. This is to expand on it and bring it into context. Our civilization is not a point right now where true democracy can work. In fact, a benevolent dictatorship is exactly what is needed to save this world from itself. As American citizens, we have come to the point where on the whole, we don’t really care about what happens in other countries or really what happens here. It’s an analogy to the South Park episode where the two candidates for South Park’s mascot are a Giant Douche and a Turd Sandwich: you’ve got different options, but when it comes down to picking a viable choice, it’s going to suck.
This is also not to say we should all be involved in politics. Trying to be moderately involved in politics has given me an incredible frustration that I would not wish on other people, and to those that have chosen this for their lives, you are either very brave or very stupid. I hope it is the former.
In a civilization where more Americans care about American Idol than America, it is not fair to have a democracy. It is not fair for me or you to be judged by our peers when our peers care about what will be for dinner or what will be on TV tonight than our fates. And I surely can’t blame them (too much that is). We certainly do need to pay more attention to the world around us, and stop being scared of every stupid little terrorist thread that turns out to be a hoax, but there is a lot going on in the world. Too much for one person to take in.
That’s why our benevolent dictatorship cannot be run by a human in the strictest sense of the word. It must be by something that can process data at a much faster rate we can but still has the heart or decision-making skills to guide us forward as a race. The Patriots and Helios surely are selfish beings concerned with their survival, but they certainly care more for our survival than we ever did. And they have the means and the motive to save us.
But if they take a strict approach and become, as Edison says, a force that will be so appalling that man will abandon war forever, what will we become? If we stop being something so core to our existence, we certainly will no longer be human. Knowing the power of these beings and their all-pervasive nature, would we be reduced to beings afraid of our supposed savior(s) until the end of our days? Or would they know better and prevent us from thinking these things? For our sakes, I hope they would.
Perhaps the best approach in this dangerous line of thought is to do as we always try to do when we have two competing ideas: combine them. Perhaps when we are all somewhat augmented / computerized we will have the brainpower and the ability to make these decisions for ourselves in a truly democratic fashion. But as we go along the away, we must not forget that always, there is only a plank between one and Perdition:
“He who fights with monsters might take care lest he thereby become a monster. And if you gaze for long into an abyss, the abyss gazes also into you.”
– Friedrich Nietzsche