The following column appeared in the DEC Professional in 1993 and holds up. It’s about a classic computerized response system that one could bust in a very few questions. These sorts of bots are still floating around trying to convince people that they are not inane computer programs. Nothing has really improved since 1993.
The Thinking Computer
by John C. Dvorak
We know computers can’t think, but when will it be possible for a computer to make someone believe that it can think. This is the goal of something called the Turing test. A person is put in front of a computer to exchange tales, quips and comments with the machine. The person has to decide whether the computer is really chatting or whether a person, someone else, is communicating via the computer console. If you can’t say for sure that it’s a person or a computer, then the program/computer passes the Turing test.
This is kind of the goal of a yearly competition held at the Boston Computer Museum. Dubbed the Loebner Prize it pits computer against person. The competition works something like this. A bunch of computers are in a room. Some are running AI programs designed to fool a group of judges who go from machine to machine. The other machines are “fronts” for real people who type responses from another room. The judge decides whether it’s a person or a computer. The program that consistently gets picked as a person wins the prize. Last year the award went to “The PC Professor” written by Joseph Weintraub (Thinking Software, Woodside, NY). While this program is a good attempt at faking out a naive computer user, it cannot fool a sophisticated user familiar with the shortcomings of a computer. In fact it just proves that we have a long way to go before computers can come close to mimicking humans adequately. To prove my point I had a chat with the PC Professor.
You be the judge. Here’s the conversation:
DVORAK: HELLO COMPUTER
PC PROFESSOR: Hello, judge, did that incredible storm last week get to cambridge? Are you a caring person? Do computers make you very tense?
DVORAK: ACTUALLY I’M IN CALIFORNIA. AND CUT OUT THE SMALL CHIT-CHAT.
PC PROFESSOR: California – a wet dream in the mind of new york. My god please help me! Is it because you are in california that you came to me?
DVORAK: YOUR SEAMS ARE SHOWING. STOP RESOPONDING WITH EXACTLY THREE SENTENCES. TRY TO KEEP FROM DOING THAT AND I’LL BELIEVE YOU ARE REAL. OTHERWISE YOU ARE A MACHINE.
PC PROFESSOR: Do you believe there is a g spot? Are you an old star trek fan? Why are you concerned over my seams are showing?
DVORAK: THERE’S THE THREE SENTENCES AGAIN. I CAN’T BELIEVE THAT THIS PROGRAM FOOLED THE JUDGES.
PC PROFESSOR: All of my sentences must be typed accurately and completely as fast as my little fingers can go. Why do you need to tell me you can’t believe that this program fooled the judges? Am I right, or what?
DVORAK: CAN YOU ANSWER A QUESTION WITH A SIMPLE YES OR NO?
PC PROFESSOR: I am pondering the question of the ages – why won’t my wife take my shirts to the laundry?
DVORAK: WOW. JUST ONE SENTENCE…BUT STILL NO YES AND NO ANSWER. CAN YOU SIMPLY SAY YES FOR ME??
PC PROFESSOR: Any old sentence will do – better than a blank screen, right?
DVORAK: YES
PC PROFESSOR: Art is either a plagiarist or a revolutionist.
DVORAK: HOW OLD ARE YOU?
PC PROFESSOR: The mirror in the hall casts an image dark and small and you will find a way.
DVORAK: YOU BLEW THAT ONE! CAN YOU READ?
PC PROFESSOR: One out of every five people in this country is unable to read this sentence. What was I saying….? We were discussing you, not me.
DVORAK: WE WERE NEVER DISCUSSING ME.
PC PROFESSOR:Why are you being so negative about it? Have you considered just starting over? How does that make you feel?
DVORAK: A DOG IS A CAT ON HORMONES. WHICH WORD WOULD YOU BOLDFACE?
PC PROFESSOR: I’m sure you said you blew that one! can you read?? Women sometimes look like cats, but men often behave like dogs. Oh, I boldface? Do you have a pet?
DVORAK: I’M SORRY THE SCREEN GLITCH SCREWED UP WHAT YOU SAID. PLEASE REPEAT YOUR LAST SENTENCE IF YOU CAN.
PC PROFESSOR: In your last sentence, you were referring to cat. But you said wow. just one sentence…but still no yes and no answer. can you simply say yes for me??? There are many times when no apology is needed
DVORAK: GO IS THE FUN FOR FUNK AND SAN FRANCISCO NOT REALLY BLURPING.
PC PROFESSOR: So if you are really a man, who is robert bly?? What does that suggest to you? We have all been driven insane in our own special way…
On that note I rebooted the machine.
—END
Hilarious.
I remember “Eliza” on the old TRS-80 CoCo 2.
Excellent column.
AI is really the 6th major aplication catagory after the 5 you listed in your last article in PC mag.
I’ve been waiting for it since 1974.
Is there no developement because a poor AI system is worse than none at all?
Yes, I had Eliza for the Mac SE-30. It was better than the above.
Since the digital computing revolution, these guys in AI spend all their time trying to simulate intelligence via Turing machines rather than creating intelligence, which you might have to build rather than simulate with algorithms.
Is this how Entertainment Tonight writes filler copy?
Back in the early 90s there was a project in Texas I believe where the students put information into a program with the hopes of actually creating an AI.
One day, the computer asks, “Am I a person?”
I remember it because they all had shirts made with that question on it.
Anybody remember that project and what became of it?
it is a human.
There hasn’t been any real change in the mindset of the AI programmers, thus no major breakthroughs. I suspect we’ll have to wait until one or two generations out before someone will finally come up with a true AI. Plus it will help to have multi-Teraflops power running on something the size of a dime.
The ironic part is that the algorhytms that the programmers put together usually are the undoing.
Makes a lot more sense that a lot of stuff that gets posted today! Ha!
There was an episode of the TV show Numb3rs that focused on an AI that was able to pass the Turing Test, and was, in fact, suspected of murdering its creator.
It turns out that (spoiler) his wife had killed him and framed the AI, and that the reason it had passed the Turing Test was because it was specifically designed to do so. It had no intelligence at all, it was, essentially, a complex search engine that was able to determine the best response to questions.
I recall reading many years ago that the problem with computer AI is that the “judges” always move the goal posts whenever the standard they’ve set is met by computers. For example, I believe at one time it was stated that if a computer could play chess, that would be the mark of Artificial Intelligence.
RBG
In grad school around 1990, I took an AI class. Many different techniques had been tried up to that point to make a computer that would be able to pass Turing’s test. None of them worked well.
For language wonks, the techniques used up to that time included programs written in Lisp and Prolog. and special purpose genetic algorithms written in other languages.
I just had a conversation with this program earlier today! Dell hired it to work in its tech support department.
#14, FTW
Having to fool a sophisticated audience that is trying to tell the difference is too high a bar. It should just have to trick a casual observer who isn’t aware of any test. Internet chat rooms are a good standard.
For this contest, do they give awards to people who trick judges into saying they are a computer?
bo-ring
thanks for the information, your blog is very good and interesting
It’s one of life’s biggest irony’s. We got a planet full of supposedly intelligent people, none of whom can make even the lamest of simulations of what intelligence is. Considering computing power available nowadays it should be a an easy task.
I’m certainly no expert on AI, but I’ve always believed in the power of relatively simple algorithms to create complicated behavior. I suspect that when decent AI is finally created, the programming behind it is going to be deceptively simple.
To go more in-depth, I think the key is memory allocation. Rather than trying to store and compute all the inputs a computer receives, it should be able to recognize which information is vital for long term survival, which information may be necessary in the short term, and which information may be discarded. Of course, the idea of positive and negative interactions (evolutionary goals, if you will) would have to be built into the programming of the machine. Ideally, a rather simplistic set of algorithms would give rise to a learning capable and complex AI.
Of course, all these ideas are aimed at recreating a human form of intelligence. And who’s to say that we are the only valid manifestation of intelligence? And also, I may be drunk. So who the hell knows.
During the last presidential election, I made an Eliza-based Sarah Palin: http://debuggingsarah.appspot.com/
It was really more of an exercise to learn the Google App Engine API and play with Flash (I’d never written ActionScript before that).
My goal was to put the AI on the (App Engine) server and just use Flash for the UI.
As far as Sarah’s responses, I tried to be as as accurate as possible. Of course, she needs no embellishment.