ABOUT
Turing's Imitation Game
In 1950, Alan Turing devised a game to test whether computers could display intelligent behaviour similar to that of a human. In Turing's Imitation Game, now commonly known as the Turing Test, a human interrogator speaks simultaneously to a human and a machine via teletype and aims to determine which is which. Turing thought that the open-ended nature of language would allow the interrogator to test the machine on virtually any topic.
What is this site?
Turing thought that by 2000, computers would be able to fool humans at least 30% of the time. While that might not have come true, AI has improved a lot over the last several years. We created this site to test how good current chatbots are at Turing's game, and provide a platform to compare different AI models as they progress. The site is part of research being carried out at UC San Diego and Stony Brook University.
HOW DOES IT WORK?
Users are randomly assigned to either be an interrogator or a witness, and matched up with another user. The interrogator has two simultaneous conversations with one human user and one AI model. The AI models are Large Language Models (LLMs), trained to generate plausible completions for text inputs. The AI is really just repeatedly asking the question “which word is most likely to come next, given what has been said so far?”. We use several different AI models and sets of instructions called “prompts”. These tell the model what sorts of things to say in response to users. For example, the prompt will generally tell the model that the conversation is part of a Turing Test, and instruct it to act like a human.
What have we found?
In a set of pre-registered experiments run on this site, interrogators had 5-minute conversations with a human and an AI at the same time. When GPT-4.5 was instructed to adopt a humanlike persona, it was judged to be the human 73% of the time: significantly more often than the real humans it was compared to. LLaMa-3.1-405B, with the same instructions, was picked 56% of the time, which was not significantly different from the humans. Without the persona instructions, the same models did much worse (36-38%), and the baseline systems ELIZA and GPT-4o were picked only 21-23% of the time. A follow-up study with 15-minute conversations replicated the result. This is the first empirical evidence that any artificial system passes a standard three-party Turing test.
Interestingly, interrogators rarely tested knowledge or reasoning. They focused much more on conversational style, emotion and personality: the things people seem to see as distinctively human.
You can read the full paper here: Jones, C. R. & Bergen, B. K. (2026). Large Language Models Pass a Standard Three-Party Turing Test. Proceedings of the National Academy of Sciences.
What happens to the data?
We store data from the conversations (including the text, the interrogator’s decision, and the user accounts involved in the conversation) so that we can carry out research on how well the models perform, what sorts of questions people ask, and what techniques are successful for detecting AI. We will anonymize all data before using it for research, and we will never release personally identifiable data, or sell your data to anyone.
WHO MADE THIS?
The site was developed by Cameron Jones (Department of Psychology, Stony Brook University), in collaboration with Ben Bergen (Department of Cognitive Science, UC San Diego). If you have any questions or thoughts about the site, you can reach me at cameron.jones@stonybrook.edu.