Experimental narratives: A comparison of human crowdsourced storytelling and AI storytelling
Abstract The paper proposes a framework that combines behavioral and computational experiments employing fictional prompts as a novel tool for investigating cultural artifacts and social biases in storytelling both by humans and generative AI. The study analyzes 250 stories authored by crowdworkers in June 2019 and 80 stories generated by GPT-3.5 and GPT-4 in March 2023 by merging methods from narratology and inferential statistics. Both crowdworkers and large language models responded to identical prompts about…
The art of story writing; facts and information about literary work of practical value of both amateur and professional writers by Fowler, Nathaniel Clark, 1858-. Public domain
Researchers tested a fiction-based framework that asked both humans and large language models to write stories about creating and falling in love with an artificial human. They analyzed 250 crowdworker stories from June 2019 and 80 stories from GPT-3.5 and GPT-4 from March 2023 using narratology and inferential statistics.
The comparison matters because it shows how collective cultural myths like Pygmalion persist across human and AI outputs while revealing divergent social patterns. Uncertainty remains about generalizability beyond default settings, the specific Pygmalionesque prompt, and the limited, time-separated samples.
- Study compared 250 human stories from June 2019 with 80 stories from GPT-3.5 and GPT-4 from March 2023 using identical Pygmalionesque prompts.
- All solicited narratives present a scientific or technological pursuit, confirming pervasive presence of the Pygmalion myth in collective imaginary.
- Framework merges methods from narratology and inferential statistics to use fictional prompts as tool for investigating cultural artifacts and social biases.
GPT-3.5 and especially GPT-4 produced narratives that were more progressive on gender roles and sexuality than human crowdworker narratives when responding to identical prompts about creating and falling in love with an artificial human.
The rundown
The experiment used identical prompts about creating and falling in love with an artificial human to elicit narratives from both crowdworkers and LLMs, enabling a direct controlled comparison.
Results showed a split pattern: AI stories were rated as more progressive on gender and sexuality, yet human stories retained an advantage in imaginative scenarios and rhetorical variety.
Sources
- Peer-reviewedHumanities and Social Sciences Communications2024-10-28
How should this claim be treated?
ace
The debate