A Twitch Engineer Used Machine Learning to Generate Surreal Emoji

The project is the latest use of a novel AI creativity tool

A group of GAN-created cartoon icons
After much tweaking, the GAN was able to create these passable cartoon icons. Twitch
Headshot of Patrick Kulp

Artificial intelligence may be a long way from experiencing sentient emotion, but that doesn’t mean it can’t be used to generate emoji.

Twitch software engineer Avery Gnolek recently proved as much in an experiment featured on the streaming platform’s blog. Gnolek used a form of machine learning to create realistic-looking sets of the types of custom icons popular among Twitch users, including batches of fake human faces and cartoon avatars.

While the end result isn’t going to replace the work of human designers anytime soon, the project is one of a growing number of ways that artists, agencies and brands are testing machine learning breakthroughs that have unlocked new creative potential for the technology.

“Although this specific personal project was more to investigate the viability of the technology and produce some interesting images, the information I learned about the strengths of convolution neural networks does seem applicable for potential upcoming projects,” Gnolek told Adweek in an email.

Linear interpolation between cartoon emotes

The experiment hinged on a deep learning model called a Generative Adversarial Network (GAN), a system in which one neural network generates images until another neural network can no longer tell the difference between its output and the real images of the training data. The setup was first formalized by Ian Goodfellow, now Apple’s machine learning research chief, in 2014.

Gnolek had no prior experience with convolution neural networks when he first embarked on the project, inspired by a GAN-based tool Nvidia released earlier this year that generates photorealistic human faces. He extensively documented the various tweaks he added to the open-source code to evolve its output from indecipherable blurs to passable emoticons in a post published to Twitch’s blog.

A set of human face emojis generated with a GAN

“It was really inspiring that someone with no prior neural net experience (me) was able to build off of open-source technology to produce workable results in a relatively short timeframe,” Gnolek said. “This was only possible due to the active development of deep learning libraries over the last few years and the mature set of documentation surrounding them.”

A host of new creative tools have made this technology even easier to use for creative tasks, to the point where one no longer needs a coding background to do so. A recently released program called Runway ML, for instance, brings together many cutting-edge creative AI widgets in an easy-to-use hub interface.

Twitch isn’t the only company where employees are exploring uses for AI-fueled creativity. AKQA used a neural network to  create its own sport earlier this year (complete with a GAN-generated logo), Goodby, Silverstein & Partners used deepfake technology to bring Salvador Dalí to life for his namesake museum in Florida and French designer Philippe Starck used Autodesk’s software to create the first commercial designed through AI—a chair for homeware brand Kartell.

@patrickkulp patrick.kulp@adweek.com Patrick Kulp is an emerging tech reporter at Adweek.
Publish date: July 29, 2019 https://stage.adweek.com/creativity/a-twitch-engineer-used-machine-learning-to-generate-surreal-emoji/ © 2020 Adweek, LLC. - All Rights Reserved and NOT FOR REPRINT