I've always been interested in the intersection of mathematics and creativity - it was this that compelled me to initially pursue a career in mathematics, long before I became interested in machine learning and tech. Poetry is one of my favorite art forms, so I wanted to see how well the cognitive technology I developed could be applied to some more creative tasks.
And, to my surprise, we were able to come up with a remarkably creative sonnet in the style of Shakespeare complete with a coherent theme and some very compelling images that were completely innovative. Here I’ve summarized the technical set up of the AI system to describe how it works, then looked at some of the surprising results of the system, and finally both technical and philosophical questions about the methodology and the system.
Creating the AI
Setting the structure
We decided to write a sonnet in the style of Shakespeare as our first poetry experiment. The AI was designed to transform the language to create new and unique images, without much creative license in the structure. Since sonnets are heavily structured, it made the ideal place to start.
Our natural language understanding parsers struggled to understand the underlying grammar because of the differences in language between Shakespeare's time and our own. So our first step was to modernize the language so that the parsers could understand it. After adding around 30 language transformations, we found our parser had good accuracy which allowed us to identify keywords, phrases, and themes from our sonnets.
Training the neural net
One of my favorite language visualizations - a word river - showing the most common words in Shakespeare's sonnets.
We first trained a neural net on all of Shakespeare's sonnets but quickly found that there wasn't enough data for the words to embed correctly since there were only around 2000 lines. We then extended the training set to include Shakespeare's complete works - plays and everything. Our word embeddings were much better but tended to be quite different from other neural nets I've created, a result of having a small training set and a large vocabulary. It also meant some of our results are greatly affected by the initialization of the neural net's word vectors.
You can also see just how much of a difference the datasets have in the way in which the words embed. For instance, in Shakespeare’s writing, the word good embeds nearest to the word honest, but in the context of customer satisfaction in restaurants (a dataset we’ve used for one of our customers), the word good is most similar to terrific. Similarly poor in Shakespeare is nearest untroubled but in restaurant satisfaction poor is closest to awful.
Creating the poem
Once we had the neural net trained, we needed to transform the theme from our original sonnet to our new sonnet. We then randomly chose a sonnet structure to hold this new transformation.
The transformation (mathematically, it was a thousand-dimension vector transformation) resulted in a lot of animal-related themes in the generated poem - a result that we could not have predicted. We used this transformation function to write the first draft of our new sonnet, and then we reviewed the output the neural net created. We then tilted the transformation function until the output fit with the rest of the poem - essentially, writing it iteratively, coming up with better and better drafts.
The Results
The challenge that many AI generated poetry has had so far has been in how compelling the imagery and phrasing is, as compared with human-generated poetry. Even Google’s AI poetry experience last year struggled to create multiple images that worked together.
To our surprise, our poem resulted in an intriguing set of images with no intervention from us. Have a look:
Of Sottishness
Our AI poet came up with the following line in our sonnet (line 10):
And sottish sap-consuming of my precious-juiced broad
"Sottish", a great word that means drunk, foolish, stupid, senseless, appears in Anthony and Cleopatra, spoken by Cleopatra in Act 4:
Patience is sottish, and impatience does become a dog that's mad.
And "sap-consuming" is spoken by Merchant Aegeon in the Comedy of Errors:
Though now this grained face of mine be hid
In sap-consuming winter's drizzled snow
Together it creates quite a compelling phrase, don't you think? Not least because there are several ways that we could interpret it. For instance we could understand it as "foolish drinking" - or we would understand it as "senseless energy-draining" - giving the line a very different meaning and both of which work with the rest of the poem. Too bad we can't ask the AI what it was thinking when it came up with that phrase.
Although "precious-juiced" was used to describe flowers by Shakespeare, it’s also slang for drunk. Here it is in context in Romeo and Juliet:
I must up-fill this osier cage of ours
With baleful weeds and precious-juiced flowers.
The earth that's nature's mother is her tomb.
What is her burying gave, that is her womb;
All in all the line is quite interesting and is surprisingly coherent as well as creative.
Of Lions
The word "lion" is fairly common in Shakespeare's works, appearing over 100 times in various plays. Even so, two of the lion related images in our poem: "six-gated lion" and "lion's accited fleece" both seem to be completely unique - not a single hit on Google. They both came directly from the AI with no editing from us.
What does "six-gated" mean? In Shakespeare it describes the gates that protect the city of Priam in the prologue of the History of Troilus and Cressida:
To Tenedos they come,
And the deep-drawing barks do there disgorge
Their war-like fraughtage. Now on Dardan plains
The fresh and yet unbruised Greeks do pitch
Their brave pavilions: Priam's six-gated city,
Dardan, and Tymbria, Helias, Chetas, Troien,
And Antenorides, with massy staples
And corresponsive and fulfilling bolts,
Sperr up the sons of Troy.
When I think of "six-gated", my initial inclination is that it should mean an animal so fierce it needed to be guarded by an excess number of gates - like in the monomyth when the hero passes through many gates to further, more challenging trials on his or her journey. I’d be interested to know what associations it brings to mind for others.
"Lion's accited fleece" was also a compelling line that was put together by our AI poet. "Accited", which means called, roused, awakened, or summoned, is spoken by Marcus in The Tragedy of Titus Andronicus:
He by the Senate is accited home,
From weary wars against the barbarous Goths,
That with his sons, a terror to our foes,
Had yoked a nation strong, trained up in arms.
"Accited" is used as a verb in the original, but our AI poet has used it to describe the lion's fleece - an interesting grammatical shift. Even more impressive, fleece is never used in Shakespeare in relation to lions but rather references clothing, wool or hair (human).
Overall we were pretty impressed with the way in which our AI was able to create unique, sensical images that were quite creative and very much in the style of Shakespeare - but ones that were not pulling images word for word from his other works. They fit both the theme and the style of writing proving our AI poet does seem to have a human-like creative spark - or at the very least, the novel turns of phrase make sense to humans and seem to have real meaning.
Not being an expert in Shakespeare I'd be interested to know what others with more experience analyzing his works think about how well they fit with his writing style.
Further, it begs a fundamental question about creativity - is the poem’s meaning is created by the author upon writing the work or is the meaning created by the reader upon reading the word? I'm sure there are experts in Literary Theory who might have some comments on this.
Can we really call this poem a sonnet?
Rhythm
The Rhythm of our poem is far from standard and, we found that our neural net tended to replace simple words with more complicated words in general. Highly syllabic words tend to embed together because more complex words have similar windows, words that appear close to it in the text. They tend to be used together since someone who uses one long word is usually someone who uses many long words. We all know someone like that, don’t we? Even so, the complexity of the imagery that the neural net brought to the poem made it quite provocative!
Rhyming
For this poem, we've completely gotten rid of the rhyming of a typical sonnet. However, given that most dictionaries include pronunciation in their entry it would be straightforward to force the neural net to rhyme. It will be interesting to see how much this ends up affecting the consistency in the overall theme - if it begins to tend towards nonsensical rhyming. And if so, perhaps training a neural net on Lewis Carroll would be a better place to start.
Theme
Most of the poems made almost no sense with the initial transformations we tried. We chose the transformation that made the most sense out of the top five or so (as it happened the very first one made the most sense) and then attempted to improve it. It would be interesting to look at ways for the AI to assess how much sense its own poem makes.
Since we were the ones that ultimately judged whether the poem made sense or not should we be taking the bulk of the credit for creating a poem rather than giving credit to the AI? I'm not sure. After all, it was the AI that came up with the phrases and we just took the ones we liked. Furthermore, the AI was consistent with its imagery in quite an unexpected way that we could not have predicted.
We think it’s achievable to create an AI that fits a sonnet pattern and rhyming structure while generating original works in a Shakespearean language space and with a coherent theme. The better question is, is this art? Is the AI responsible for creating an interesting phrase, or does the human who recognized it as such (and incorporated it into a work of art) take the credit? Is this different than digital artists using image editing software?
Who wrote this poem?
I don't know... what do you think?