AI haiku are getting better – at least in Japanese

Researchers from Kyoto University have found that people have trouble distinguishing AI-generated haiku from those written by human poets, and that human/AI collaborations have led to the most interesting haiku. But maybe it only works in Japanese.

The AI ​​is getting better at creating visual art, and it’s getting pretty good at realistic conversations, but how’s that with poetry? Researchers at Kyoto University put it to the test by having hundreds of volunteers read haiku poetry and evaluate it aesthetically. The volunteers were also asked if they thought the poems were written by humans or an AI, and that’s where things got interesting.

Of the eighty different haikus used in this experiment, forty were written entirely by human poets, and the other forty were written by an AI. But because the researchers knew that the quality of AI poetry could be improved by asking humans to choose AI-generated items, they divided the AI-generated haiku into two groups: half ( 20) were entirely created by AI and the rest had human elements. participation.

When the volunteers rated the poems, they didn’t know which were AI-generated, which were created by AI and humans together, and which were written entirely by human poets. The ones they found to be the most aesthetically pleasing were not those written by poets, but haikus that were generated by an AI and then selected by humans. This suggests that a combination of humans and AI could result in more interesting haikus than humans alone.

Volunteers were asked which poems they thought were AI-generated and which were written by humans. This could help isolate the biases of the volunteers, and it turned out to be very interesting. “They were meant to be unbiased, but were instead influenced by a kind of reverse psychology,” said study leader Yoshiyuki Ueda of Kyoto University. “In other words, they tended to unconsciously give lower scores than they thought the AI ​​generated.”

Despite this, AI poems have been at least as good as human-created ones, as people were very bad at guessing which haikus were AI-created or not.

The study was done with Japanese haikus, but it inspired me to ask OpenAI ChatGPT to write haikus in English. It didn’t go as planned, mainly because syllables are much less predictable in English than they are in Japanese. When I specifically asked ChatGPT to write me a 5-7-5 haiku about AI’s ability to write haiku, I got this in return:

The skillful hand of AI

Crafting haiku with ease, it seems.

Natural as possible.

In most English accents, this first line is decidedly four syllables, and the last line six. (Although “natural” can be reduced to two syllables – natch’ral). I asked ChatGPT if he knew what a 5-7-5 haiku meant and got a perfect explanation, so that wasn’t the problem. But when I asked “How many syllables are in the phrase ‘the dexterous hand of AI'”, I got a confused response. It wasn’t four or even five, according to ChatGPT, but SIX. Here is what he said.

“The phrase ‘the skilled hand of AI’ has six syllables [sic]. In English, syllables are usually counted by the number of vowels in a word or phrase. In this sentence, the first word “AI’s” has two syllables [sic] (“AI” and “s”), the second “qualified” word has two syllables [sic], and the third word “hand” has one syllable. Therefore, the total number of syllables in the sentence is six [sic].”

(Note that in addition to the wrong number of syllables in two of the three words, ChatGPT also calculates 2+2+1 incorrectly)

So I don’t think it will be as hard to guess which haikus are AI generated if they’re written in English, but again it’s possible that humans will improve the output and select only those that are structurally correct. I’m sure poets can think of fun and creative ways to use AI in their work.