Internet AI poems outperform classical works in surveys

In a study, poems written by ChatGPT were rated higher than works by Shakespeare and other poets. The participants often found the AI texts more appealing and rhythmic.

A recent study shows that poems written by ChatGPT performed better in surveys than the works of William Shakespeare and other renowned poets. The researchers found that participants generally found the AI poems to be more beautiful and rhythmic.

According to the University of Pittsburgh researchers, the simplicity of the AI poems could mean that they are easier for non-professionals to understand. This could explain why the participants preferred the AI poetry. It was suggested that the complexity of human poems was often misunderstood, leading to parts of the poems being perceived as incoherent.

The study included 1634 participants who each rated ten poems. This group consisted mainly of people who only read poetry occasionally. Five of the poems were by well-known poets such as Shakespeare and T.S. Eliot, while the other five were generated by ChatGPT in the style of these authors.

Confusion over attribution

In more than half of the cases, participants mistook the AI poems for human works. Interestingly, the poems that were least often recognized as human were actually written by human poets.

Participants often disagreed about which poem belonged in which category, indicating that the task was perceived as difficult and was often guessed.

In another experiment with 696 participants, the poems were rated according to criteria such as quality, beauty, emotion, rhythm and originality. The AI poems outperformed the human works in 13 out of 14 categories as long as the participants did not know who had written the poems. Once this information was known, the human poems performed better.

A convincing AI poem

One particularly frequently misattributed poem was an AI work in the style of Allen Ginsberg. "Almost 70 percent of participants thought it was a human work," explained co-author Brian Porter. The first stanza of the poem reads:

"In the stillness of the night | I hear the beat of the city's heart | The rhythm of the streets, the pulse of life | A symphony of chaos, a work of art" (translated: "In the stillness of the night | I hear the heartbeat of the city | The rhythm of the streets, the pulse of life | A symphony of chaos, a work of art")

These poems were created in 2023 with ChatGPT 3.5. "I've recently experimented with ChatGPT 4 and 4o," Porter said. "The newer models are better at hitting the expected verse meter, like Shakespeare's iambic pentameter, but I don't see significant advances in content."

AI in other creative fields

Another study showed that jokes from ChatGPT were perceived as funnier than those from average humans. In addition, AI paintings received better ratings in surveys than works by amateur painters. So is AI superior in all creative areas?

Porter believes that this is not the case for longer texts. "Large language models cannot yet write indistinguishable novels," he explained. This is probably due to a lack of computing power. Nor has he seen any evidence that an AI can write a complete comedy program that can compete with a human program. "Longer texts generated by AI can still be distinguished from human texts."

SDA