Moderate evidence
More
Six AI chatbots matched human idea quality, a study found; 9.4% of people still beat GPT-4
Study title: Artificial muses: Generative artificial intelligence chatbots have risen to human-level creativity
- Evidence type
- Controlled or validated comparison
- Sources
- 1 sourcePeer-reviewed
- Published
- 1 min read
- Truvace record
- TRV-2026-1192cite or check it
Study title: Artificial muses: Generative artificial intelligence chatbots have risen to human-level creativity
In a study published October 9, 2023, researchers tested the view that AI cannot be creative by having humans and six generative chatbots generate ideas, then having humans and a trained AI evaluate them. The evaluation found no qualitative difference between the two sources on average.
Read the study abstract
A widespread view is that Artificial Intelligence cannot be creative. We tested this assumption by comparing human-generated ideas with those generated by six Generative Artificial Intelligence (GAI) chatbots: alpa.ai, Copy.ai, ChatGPT (versions 3 and 4), Studio.ai, and YouChat. Humans and a specifically trained AI independently assessed the quality and quantity of ideas. We found no qualitative difference between AI and human-generated creativity, although there are differences in how ideas are generated. Inter…

The gain
Six AI chatbots, including GPT-4, produced ideas that human and AI judges rated no different in quality from human-generated ideas on average.
In brief
The result matters because it positions current chatbots as usable collaborators in creative tasks rather than mere tools, while the persistence of a small human edge and differing generation methods leaves uncertainty about originality, true creativity, and long-term effects on creative practice.
Main points
- Comparison involved six GAI chatbots: alpa.ai, Copy.ai, ChatGPT versions 3 and 4, Studio.ai, and YouChat versus human-generated ideas.
- Quality and quantity were assessed independently by humans and a specifically trained AI.
- 9.4% of humans were more creative than the most creative GAI, GPT-4, and generation methods differed between humans and AI.
The full story
The study compared human-generated ideas to ideas from alpa.ai, Copy.ai, ChatGPT 3 and 4, Studio.ai, and YouChat, with quality and quantity judged by both human raters and a specifically trained AI.
Results reported no qualitative difference overall, but noted different generation processes and that 9.4% of humans outranked GPT-4, leading authors to frame GAIs as assistants while raising the question of truly creative capability.
Sources
1 on file- Journal of Creativitydoi:10.1016/j.yjoc.2023.100066
Talk it over
Six chatbots, including GPT-4, produced ideas rated no different in quality from human ones on average, yet 9.4% of humans were more creative than GPT-4. How do you weigh the average result against that group?
Six AI chatbots matched human idea quality, a study found; 9.4% of people still beat GPT-4https://truvace.com/item/artificial-muses-generative-artificial-intelligence-chatbots-have-risen-to-human
Six chatbots, including GPT-4, produced ideas rated no different in quality from human ones on average, yet 9.4% of humans were more creative than GPT-4. How do you weigh the average result against that group?
Answer in the threadHumans and the six chatbots generated ideas by different methods, and humans and a trained AI judged them. How might those different methods affect what the no-difference result means for creativity?
Answer in the threadThe authors frame the chatbots as assistants and leave open whether they are truly creative. What finding would change your mind about whether an idea from alpa.ai, Copy.ai or YouChat counts as creative?
Answer in the thread
If you have more time
The study judged quality and quantity of ideas. Which other qualities would you want the human raters and the trained AI to score, and why?
Answer in the threadThe 9.4% of humans who beat GPT-4 came from one group of human idea generators. How might a different group of people, with different training or experience, change that figure?
Answer in the thread
ace
What readers say