Royal Caribbean Now Sets Your Vacation Photos to Music Using AI

Berklee College of Music helps cruise line train a song bot

Royal Caribbean AI project now sets photos to music.
Royal Caribbean

Everyone loves posting a good vacation photo to Instagram—but what if each one could have its own unique soundtrack, too?

Royal Caribbean is experimenting with that possibility, on Tuesday launching an online tool that turns user images into kaleidoscopic mini-videos, complete with original music inspired by the visuals — and assembled by artificial intelligence (AI).

A picture from a botanical garden, of red flowers and green leaves, generates two bars of smooth jazz. An elaborate piece of graffiti on a brick wall renders into a crunching hip-hop beat. In a quick snap of the Bensonhurst Statue House, the cruise line’s technology recognizes a dour likeness of a face peeking over a fence, and delivers a funky nu-disco snippet, with pumping guitars and horn swells.

Titled SoundSeeker, the marketer’s new website chops and swirls the three photos into one spinning abstract visual sequence — weaving in ample shots of crystal blue waters — and strings together the music, for a total of six bars at about 100 beats per minute. Some 15 seconds later, it’s all over, and the user is left with a personalized video ad — one Royal Caribbean is hoping he or she will be eager to show friends.

“People of all ages crave new ways to share their best experiences on social media,” says Jim Berra, chief marketing officer for Royal Caribbean International. “This tool allows you to put a completely unique, multisensory spin on sharing those memories — now friends and followers can see and hear your life’s adventures.”

For help creating SoundSeeker, agency MullenLowe turned to Berklee College of Music’s Institute for Creative Entrepreneurship — which focuses on emerging technologies — as well as London-based production company Unit9 and Stockholm-based music tech company Plan8. The team started by interpreting hundreds of music tracks and 10,000 photos to distill some 2.5 million combinations into a map of just ten possible moods like “Cozy, Warm and Safe” or “Energetic and Intense.” Those moods and interpretations form the basis for teaching the AI how to analyze photos and combined with parameters like facial and color recognition (powered by Google Cloud Vision) allowed it to crank out a fitting clip of music from one of a million possible unique combinations. It is created by layering instrumental parts composed in advance for the cruise line in genres from rock to hip-hop to electronic dance music.

In other words, it’s not precisely robot magic — but it’s not entirely prescriptive either.

“The key here is that Soundseeker doesn’t have a pre-defined set of rules that it follows to map specific things in the photos to specific musical elements,” says Christian Madden, svp creative director at MullenLowe. “It uses machine learning to make those decisions based on everything it knows from past experience. That past experience, its knowledge of how visual information is linked to musical information, was taught to the AI by humans when we trained it, transferring our own opinions and taste to the AI.”

How does that actually play out? “When the AI sees a person doing a backflip off a cliff into the ocean on a sunny day, it decides that this is a high-energy, adventurous scene that should get high-energy adventurous music,” explains Madden. “It knows this because it was trained on images that have similar elements. The AI takes into consideration everything it was taught about sunny days, cliffs, ocean scenes, bright blue colors, excited facial expressions and arrives at the conclusion that this is a high-energy adventurous photo.”

In fact, the point was, in a way, to get AI thinking more like a human — not in conceptual or analytic terms, but emotionally. “It works closer to the way our instincts do as the selection is the product of the training against the hundreds of photos that we all did,” says Alex Horton, creative director at Unit9. “The fundamental here is that it’s not theoretically driven in the way a composer or creative works when they build an arrangement. Instead, we created many ‘colors’ of arrangements, and it learned what the best fits are based on many criteria that were determined by what people like.”

Recommended videos