ChatGPT vs Eliza - round 2

After yesterday's post I asked ChatGPT to compare ChatGPT and Eliza. It wrote this:

ChatpGPT and Eliza are both AI-powered conversational systems, but they are quite different in terms of their capabilities and intended uses.

ChatpGPT is a state-of-the-art language model developed by OpenAI that is capable of generating human-like text in response to input from a user. It can be used for a variety of purposes, including generating creative writing, responding to customer inquiries, or engaging in conversation with users.

Eliza, on the other hand, is a much simpler program that was developed in the 1960s. It uses a set of rules and patterns to generate responses to user input, and is intended primarily as a demonstration of how computers can simulate conversation. Unlike ChatpGPT, Eliza is not capable of generating original text and is only able to produce responses based on the rules that it has been programmed with.

Overall, ChatpGPT is a much more advanced and capable system than Eliza, but Eliza has its own unique place in the history of AI.

You could believe a person wrote this, just like you could believe Eliza was a psychiatrist. But if you look closer you see a well put together pastiche of a few facts gleaned from the web, plus one paragraph straight from the sales brochure. It’s cleverly done. But I still don’t think it’s clever.

Eliza vs GPT

Many years ago there was a program called Eliza. It was very good at making people think it was human. I came across a Basic version in “More Basic Computer Games”, typed it into my Micro-Tan and got it working. It was great fun. It pretended to be a type of psychiatrist but all it ever did was parrot back to you what you had entered. My favourite bit of the code was the part which changed "I” to “Your” and “Me” to “You” etc so that it could be sent back to the user as if the computer understood what you had just written. What struck me about the reaction of other people to the program was how easy it was to make them think the thing understood what they had entered and, which is much more scary, how keen folks were for this to be the case. They really wanted to believe the software was properly clever. Me, I just wanted to type in things like “I’ve just shot grandma” so that I could get back the response “You’ve just shot grandma? Tell me more about your family”.

I was strongly reminded of Eliza when Ross was showing me how good ChatGPT is at writing programs. He asked for some Arduino code to make lights flash in response to sensors and what came back looked like fairly convincing C. It was very impressive. But it it is still not clever. It is just taking a bunch of stuff from you, looking things up and then crafting a response that chimes with what you expected to see. Sometimes it might combine things in ways you don’t expect, sometimes it will find things that strike you as original. And it might react differently from Eliza if you tell it you just shot grandma. But I don’t think it’s clever like we are. That’s not to say that it won’t change the world though. It will. For one thing search engines are going to get a lot easier to use and a lot more conversational. For another, the essay and the programming exercise are about to get massively devalued as a way of assessing knowledge. Some students will use ChatGPT to craft their submissions. Others will question why they are being asked to write something which can be done better by a machine.

For me the hardest thing about writing and programming has never been about turning out the prose or getting the code to work (although it can be fiddly), it has been working out what the program needs to do or thinking up a good subject and then crafting a narrative that works well with it. I like to think that with more of the “grunt work” out of the way with tools like ChatGPT we could focus our efforts on these human parts of problem solving. I’m looking forward to playing with it.