A photo of David Stiff

What Makes Quibi, Quibi?

Rachel Krauss on June 09th, 2020


This blog is based on a special study that Vault prepared for its clients and VIPs on what makes Quibi, Quibi. To download the complete study, click here.


Quibi, the new mobile-first streaming platform founded by Jeffrey Katzenberg and Meg Whitman, has been prominently featured in the news and trades over the past few weeks. Short for "quick bites," Quibi serves bite-sized 10-minute chunks of content to commuters or people during the ebb and flow of their 7 AM to 7 PM workday-when they have a few minutes to spare and lack the time for binging or watching more substantial content.

While much of the media focus has been centered on Quibi's launch performance during the COVID-19 lockdown, what's really important for executives and creators is to understand what Quibi's programming is all about. We believe by looking solely at the story elements and attributes of Quibi's shows, we can uniquely define what makes Quibi, Quibi.

Story is King... Always

Since the dawn of humankind, it's always been the story that's carried the day. From Greek tragedies to medieval bards to radio-and, of course, movies and television-the delivery mechanism may change, but it's the emotional connection of a story that engages and impacts audiences.

With a $1.75 billion war chest, Quibi is currently expected to release 100+ original shows and thousands more "quick bites" of content in the upcoming year. While Quibi offers a new twist on the relatively new medium of streaming content, the big story is the types of stories that Quibi will share with its audiences. These stories are the core of Quibi's product - if the initial novelty of 4-10 minute videos wears off, and when free trials run out, it will be the stories (the content itself) that are responsible for retaining and attracting subscribers.

Vault: AI-Based Story Analysis

Vault's unique AI-based approach to story analysis uses any form of pre- or post-production content-such as a script, outline, finished cut, or trailer-to analyze millions of story points, including genre, plot, character arcs, key concepts and scene types that motivate and engage audiences.

Using Vault's proprietary AI platform, Vault can detect up to 100,000+ story attributes that define a show's DNA-its story, structure, and characters. These attributes can then be mapped against Vault's story content database comprising the last 30 years of shows, movies and short-form content.

By distilling the programming content into its basic story DNA, Vault can identify connections between the elements that make up any show or programming and potential audiences, helping platforms to deliver content that better caters to their core audience demographics.

What is Quibi's Story?

To understand Quibi's story and the hallmarks of the new platform, Vault AI decided to use its RealAudience™ platform to analyze Quibi's first 30 shows. By ingesting the actual episodes released of these first 30 shows into the Vault platform (which also works when only a script or other text materials are available), the platform's AI identified and analyzed tens of thousands of data points, including story elements, key concepts, genres, scene types and more. This is in contrast to many studies and articles focusing solely on downloads and install numbers.

Vault AI's analysis answers the following questions:

  • What kinds of stories are being told?
  • What are the common themes, and who are the protagonists?
  • What is the platform's range of emotional tone?
  • What makes the characters tick?
  • Who are the behind-the-scenes and in front of the camera talent?
  • What are the key demographics for each show?
  • Was Quibi's programming unique to the platform, or could it thrive on other platforms?

The Core Programming Genres

While Quibi's wide range of programming gives it the potential to appeal to a multitude of audiences, the fact that 54% of the platform's programming is unscripted suggests it views Reality, Lifestyle, and Documentary genres as particularly well suited to its short form model. Within Quibi's scripted content, Comedies (14%) are twice as prevalent as Dramas or Thrillers, suggesting part of the platform's thesis is that viewers are looking to fill the "in between" moments with a good laugh.

Quibi's strategy involved launching with content led by popular TV Personalities, including Sophie Turner in SURVIVE, a plane crash drama, to Chrissy Teigen starring in CHRISSY'S COURT, a small claims court reality show.

Something for Everyone

Vault AI's analysis of Quibi's first 30 titles suggests the platform's content will appeal fairly equally across genders while predominantly attracting viewers in the 30+ age bracket, though future episodes and shows could shift the balance one way or the other. When looking at individual shows, Quibi seems to offer something for everyone between the ages of 20 and 40-precisely the people who were most likely to be commuting to work in a non-COVID-19 world. What is even more interesting, some shows could have been hits across all networks and streaming platforms, especially CHRISSY'S COURT.

While digging into the core of Quibi's programming, Vault's AI platform identified eight significant story concepts that can attract audiences for a full season or more:

Competition, Reality, Expert Advice, and Key Talent are the top concept drivers for Quibi's programming. These are followed by programming focused on Relationships, Inspiration, Knowledge, and Comedy.

Quibi's shows feature more male protagonists than female protagonists. Shows featuring male protagonists tend to be more Comedic, while shows featuring female protagonists tend to be more Family-oriented, Uplifting, and Emotional.

Fulfilling Human Desires

Vault's AI platform can detect which of Reiss' 16 Basic Human Desires are satisfied by creative content. The top three human desires fulfilled by Quibi's content are:

  • Romance (Beauty, Sex, Love)
    The shows that appeal to the human desire for Romance all contain peers seeking out romantic contact. In SEXOLOGY, real people ask certified Sexologist Shan Boodram, the Internet's most sought-after sex educator, about sex and relationships. In the dating show SINGLED OUT, 30 contestants are drawn from a bachelor's or bachelorette's social media account to compete to find a romantic connection sight unseen.
  • Eating (Food, Drinking, Hunting)
    Shows that appeal to the desire to eat all contain an appreciation for food. In THE SHAPE OF PASTA, Chef Evan Funke scours Italy for the pasta makers who prepare unusual and unique pasta shapes using special techniques and traditions. In DISHMANTLED, which is hosted by actor Tituss Burgess, a full meal is blasted into the faces of two blindfolded chef contestants-wearing hazmat suits-who must recreate the meal for celebrity judges to win a cash prize.
  • Vengeance (Winning, Aggression, Revenge)
    Audiences will identify with characters driven by revenge, especially when it is against the odds. In the PRODIGY docu-series, young athletes are profiled as they seek to become the next big thing in their competitive sport. Another docu-series, RUN THIS CITY, follows the youngest mayor in Massachusetts history as he faces wire fraud and extortion charges while preparing for re-election.

Detecting Emotions

Vault can also predict which emotions the content will elicit from the audience. The top three emotions evoked by Quibi content are:

  • Pensiveness (Broody, Grave, Analytical)
    Thoughtfulness and good deeds carry a lot of weight within Quibi's programming. In the docu-series I PROMISE, viewers are inspired as they follow at-risk students at LeBron James' I PROMISE School as they seek to improve their lives through education. In THANKS A MILLION, celebrities give away $100,000 to deserving people who must then give away $50,000 to someone else.
  • Awe (Wonder, Disbelief, Enchantment)
    Wonder and shock keep audiences' attention, especially with a reboot of the popular reality show PUNK'D, now with Chance the Rapper as the host and mastermind behind the celebrity pranks-and their unscripted reactions. GONE MENTAL features Mentalist Lior Suchard who surprises celebrities, passersby and viewers with his mind-reading prowess and pranks.
  • Serenity (Calmness, Quiet, Harmony)
    Quibi takes viewers on meditative journeys with THE DAILY CHILL, which combines beautiful scenery from around the world, calming ASMR sounds, and relaxing visuals while &MUSIC docu-series takes a look at the artistry of the music industry from the view of "unsung artists" such as lighting directors, choreographers, audio engineers and songwriters who help make stars successful.

Each type of Quibi content seems to elicit different emotions. Scripted content on Quibi evokes more dark and intense emotions such as Rage, Fear, Remorse, and Vigilance. In contrast, unscripted content tends to be more lighthearted and evokes Ecstasy, Awe, Serenity, and Interest.

Broad, Easily Digestible Content

Quibi's unique selling point is that it has broad appeal as opposed to streaming channels which tend to appeal to specific niches or demographics. Overall, Quibi's content is more casual than intense, allowing its audiences to lean back rather than lean forward. But, with this "lean back" mentality, and without requiring viewers to commit a significant amount of time (unlike watching an hour-long episode of content), the question remains: will Quibi be able to create a viral moment or must-see content that connects and captivates viewers?


Download the complete Quibi study on which this blog is based, click here
A photo of David Stiff

The Show Must Go On:
Entertainment Research During the Coronavirus Pandemic

Abe Recio on March 31st, 2020

While no industry has been left untouched by the coronavirus pandemic, for the entertainment industry, the fallout has been particularly tough. Income from box office revenues has hit zero for the first time in history while Netflix and YouTube are reducing streaming quality in Europe to keep the internet from collapsing under unprecedented demand. And now, many film studios are considering releasing movies through streaming services rather than waiting out the pandemic.

But it's not just revenues that are in jeopardy. The pipeline of future entertainment is at risk as the production of film and television content grinds to a halt and entertainment researchers are struggling to uncover critical and unbiased findings. Directly engaging entertainment consumers through focus groups is incompatible with social distancing, and data from online panels is unreliable during these times of profound social disruption.

Perform Accurate Entertainment Research Without Respondents

Fortunately, there is an alternate method for generating empirical insights without collecting data from respondents. Machine learning, a type of artificial intelligence, is ideal for entertainment research, and Vault's AI platform, which has analyzed the DNA of over 30,000 movies and TV series, is especially effective. During these difficult times it can answer such questions in an unbiased manner:

  • What are the demographics of the audience and what marketing messages will resonate with them?
  • What are the characters that will most connect to your target audience?
  • Is this movie better for streaming audiences?
  • Is a certain content optimized for premium cable, basic cable, OTT or broadcast television?
  • Which other television programs and films would this same audience be likely to watch?

Four Ways Machine Learning Can Bridge the Respondent Data Gap

1. Mitigate Respondent Biases
Contagion, Outbreak, and Pandemic are some of the most globally watched content during the pandemic, and this makes sense. Apocalyptic and global crisis content is also in hot demand. However, asking any respondent about content that focuses on any of these themes or stories in today's climate would yield incredibly biased responses. What might be of intense interest today due to global events, will not be (we hope) in a year or two years. It could be even something that no one wants to relive. If we conducted the study today, machine learning would deliver exactly the same results as in January. In contrast, a survey study with human respondents would very likely deliver different results because human respondents are influenced by the world around them.

2. Alleviate Respondent Fatigue
Pushed to the limit, fatigued respondents give unreliable responses. Even in good times, let alone during a pandemic, respondents struggle through lengthy research sessions. As a result, respondents are often only given a snapshot of the creative content they are asked to evaluate. In studies performed by Vault AI, however, 100% of the creative content informs the analysis. This means we are able to deliver detailed and actionable insights of unparalleled quality and accuracy for content in pre-production, marketing or release.

3. Ease the Logistics of Capturing and Analyzing Data
By simulating the entire global television and film market, Vault can produce an entertainment research report covering the US and a dozen international markets in two weeks instead of two to three months. Our proprietary platform can start and complete all aspects of a study in one location while eliminating the time-consuming steps of survey programming, verifying translations, recruiting respondents, and training local researchers. This is especially valuable to researchers who cannot currently perform comprehensive global or even domestic studies that rely on respondent-level data.

4. Avoiding Story Leaks and Spoilers
Panel and sample providers are scrambling to still obtain data during this unprecedented shutdown. Online testing is at an all-time high, which means the risk of any leaks or piracy is at an all-time high. Content makers are always fearful of public leaks and spoilers that could significantly damage a billion-dollar property. Plot secrets can be potentially revealed to the public via social media or a misaddressed email by a host of survey respondents and contractors communicating across the Internet. In contrast, Vault doesn't use third-party agencies to assist in entertainment research and all creative content is kept securely in-house and on-site--not on the cloud--using military-grade encryption and multiple firewalls.

Preparing for the Day-After

During these trying times of the coronavirus, audiences crave entertainment content more than ever before. For researchers, managing the pipeline of future entertainment is a significant challenge. Machine learning can provide many on-demand benefits for researchers when it's important to remain at home.

A photo of David Stiff

Can Standing Ovations Predict the Financial Success of a Movie?

David Stiff on June 17th, 2019

A Tradition of Questionable Monetary Value

The standing ovation is a tradition and centerpiece of the Cannes Film Festival and other key film festivals around the world. If you receive one, you'll gain the recognition of your peers and enjoy significant buzz around your movie. If you don't receive one, you and your studio will usually go into crisis mode.

But in this day and age of streaming and on-demand content, does the standing ovation matter? Did the standing ovation ever matter?

Your Eyes and Ears May Deceive You

A key scene in MONEY BALL is when Brad Pitt's character marvels at how his group of scouts are recommending players based on their look, the sound of their swing and the beauty of the ball player. All these elements create the profile of a single type of player: a strong macho athlete.

But do these factors really indicate an excellent ball player? Pitt's character doesn't think so. He understands that most of these elements have inherent biases picked up by the scouts over the extended time that they observe the players.

The Inherent Bias of Standing Ovations

Standing ovations also have a bias. Get one, and you are considered a winner; don't get one and you are not. At Cannes, the media will use a stopwatch and even time the duration of the ovation.

Using Vault's AI-based RealAudience™ Platform, we looked at whether there was any correlation between getting a standing ovation and being financially successful. The results showed that there was an extremely low correlation between a movie's standing ovation at Cannes and its financial success—whether on its opening weekend or its cumulative box office take.

Our analysis of more than 50 movies on the key festival circuit revealed that only 20% of the movies that received a standing ovation did 3x multiples on their opening weekend. One might think that a strong response from a festival influences an audience, but the data suggests otherwise.

The Real Value of the Festival Circuit

The festival circuit is great for smaller, independent movies, which often use the "platform release" route, artfully staggering an opening of various sizes per geography (e.g. 50 screens, 300 screens, 1000 screens). This route is often taken by those darlings that won a festival or received an ovation.

The cumulative gross profit of a platform release is usually less than a studio title with a wide release. However, platform releases are more closely tied into the independent crowd that is more likely to rely on word of mouth for making movie selections.

Rotten Tomatoes: Another Industry Bias

Not far behind the overestimated value of standing ovations are movie review websites like Rotten Tomatoes. While a bad movie review on Rotten Tomatoes may regarded by film makers and their studios as a cause for poor financial performance, our same AI-based data analysis reveals that movie reviews on Rotten Tomatoes have a very low correlation of 0.3 to a movie's financial success.

In other words, the reviews on Rotten Tomatoes and other sites are not much more than noise. The only time that their reviews have a correlation to a movie's financial success is when positive reviews for a movie are below 10% or above 90%.

But for 95% of the titles, a low Rotten Tomato score may continue to haunt their opening and be used as an excuse for a lack of financial success—but this is mostly without merit.

The Outro... a.k.a. the Bottom Line

So what is the biggest predictor of success? Having analyzed more than 30,000 titles in over 60 countries, we have concluded that story is the biggest predictor of a title's success. We have identified more than 100,000 story data points (or story elements) that lets us more predict a movie's financial success with more than 80% accuracy. That's more than any other tool, model or technique on the market.

At Vault, we are always trying to find ways to add new elements to our predictive model in order to continuously improve it and create even more accurate predictions for our clients. When we classified a title as receiving a "standing ovation", it did nothing to improve the title's performance using our AI-based approach. Perhaps this means that Netflix does not need to invest the huge sums necessary for attending Cannes—were they actually invited.

Story will always trump standing ovations and critics-and it will generate huge opening weekends, impact word-of-mouth and increase the financial success of story-driven movies like AVENGERS: ENDGAME.

A photo of David Stiff

The True Concept Drivers for Aquaman's Global Box Office Dominance

David Stiff on January 23rd, 2019

The runaway success of AQUAMAN seems to have taken nearly everyone by surprise. Grossing more than $1 billion worldwide, it was the Man-of-Scales who overcame the Man-of-Steel.

Source: VCG

On July 21, 2018, when Warner Bros. released the first official trailer of AQUAMAN—nearly 22 weeks before the movie hit theaters—we used our AI-based RealAudience™ platform to analyze and extract the core story DNA of and predict some very interesting domestic numbers:

  • Vault predicted $74 million opening weekend ($67.8 million actual)
  • Vault predicted exit polls to be M 53% / F 47% (M55% / F45% actual exit polls)
  • Vault predicted Rotten Tomatoes Audience Score to be 77% (79% as of today).

Comprising 70% of total revenues, international audiences have played a huge part in AQUAMAN's worldwide success. With coverage on 60 global markets worldwide, we took a look to see if the secrets behind the billion-dollar success of AQUAMAN.

It's Always About the Story

As with any movie, the story is king. The story reveals more about the business potential of a movie or TV show than any other data point. Likes, shares and people's opinions account for only a fraction of the impact story elements have on movie audiences. After all, story is the product' s key features.

Traditional story research has been based on asking focus groups about dozens of concepts or themes about a given film to help frame campaigns. However, this ultimately means campaigns are being operated on 100-1000 opinions, not based on transactional box office data. We believe that there are thousands more story elements with significant transactional box office value that can be found using machine learning. This approach provides numerous key insights that can form a cornerstone of any motion picture marketing strategy.

In today's global environment, story analysis is even more complex. Content is distributed to more markets than ever before and it's logistically impossible to arrange focus groups throughout the world and analyze their findings.

Aquaman: A Global Approach to Story Elements and Concepts

Vault's platform analyzed over 100,000 story elements revealed in AQUAMAN's first official trailer and to perform a thorough breakdown of regionally specific audience interests around these elements. The in-depth analysis clearly shows that the major concepts of the movie have an overwhelming positive global consensus, although there are subtle differences between markets.

The key story elements for impacting global audience transactions and box office lift are:

  • Aquaman (the character)
  • The superhero origin story
  • Jason Momoa
  • The blend of the futuristic and the mythical
  • Superhero, big budget, world destroying action
  • Family / mentor relationships

Drivers to the Domestic Box Office

Vault's analysis reveals that the strongest influencer in the US was a string of beats that took audiences through the origin story of a charismatic, wisecracking new character in the DC Universe, who grows from outcast to tough guy to hero and who ultimately challenges the evil King for the throne of Atlantis.

Other key beats include:

  • The surface battles, particularly the stunts and special FX during a mesmerizing rooftop chase scene
  • The brief training scene on the shoreline during AQUAMAN�s development that also formed a part of the student-mentor element
  • An intense one-on-one gladiatorial combat scene in an underwater colosseum
  • High FX battlefield scenes including mesmerizing cavalry-style warfare with soldiers mounted on various armored marine animals.

Audience Drivers in China, Japan and South Korea

A deep dive into our story data revealed some notable, yet nuanced differences between various regions. While one of the biggest draws for the US market was the action-packed origin story with a wisecracking action hero, the Asian market generally lacked interest around the comedic beats?-?especially the wisecracks?-?and showed stronger interest in the more serious, explosive action-sequences. In China, the action-packed story concepts added a 6.2% lift to the box office while the effect was even stronger in Japan (9.1%) and South Korea (9.38%).

But this isn't to say that there is uniformity among the Asian market interests. While the Japanese and South Korean markets showed strong interest in majestic fantasy sci-fi elements such as Atlantis, the underwater gladiatorial battles and the various sea creatures, the Chinese market actually shared the preference of US audiences to futuristic technology, watercraft, explosions, and laser weapons.

Motivators in Russia

Looking north to the Russian market, the half-brother relationship between Arthur and Orm Curry and their power struggle for the throne of Atlantis didn't impact audiences as much as in other countries. This insight is different across some key European markets that took to the half-brother relationship, but were not as motivated by the heir to the throne concepts and scenes.

However, there were many elements shared across both Russia and Europe. Jason Momoa's character, Arthur Curry, and the progressive development of his strength and bravery throughout the movie drew significant interest in both Russia and Europe. The student-mentor relationship also created further ticket sales.

In Conclusion: Preparing a Global Audience Strategy

Combining machine learning and artificial intelligence findings, in conjunction with traditional survey based positioning studies, offers the ability for an in-depth understanding of story elements that truly impact movie goers wallets. Most importantly, it is global.

These subtle audience reactions to pre- or post-production content can be used as a powerful tool for creating the most meaningful content and impactful promotional strategy that can be easily tailored to specific regions and drive audiences to the theaters.

A photo of David Stiff

What We Can Learn From the Most In-Demand Movies of 2018

David Stiff on January 9th, 2019

CREDIT: SAEED ADYANI

A Record-Breaking Year

2018 was a year of record-breaking activity for both studios and consumers alike. The results are in: the game has changed.

Universal's HALLOWEEN had the second best October opening of any movie in history, signaling a fantastic reboot for Miramax. BLACK PANTHER, the 18th movie in the Marvel Cinematic Universe, nabbed the biggest domestic February opening of all time with $202,003,951.

And of course, this December had an unprecedented 5+ tent poles debuting in theaters around the world and on Netflix?-?not to mention an interesting mashup in which Netflix pre-released movies in a select number of theaters.

So what insights can Vault's RealDemand™ Market Intel provide that made 2018 a record year?

Analyzing Stories with Vault's RealDemand™ Market Intel

Vault's RealDemand™ platform tracks nearly 100 titles in the movie market to let distributors and marketing executives measure story demand for movies. By tracking and analyzing the digital behavior of movie consumers, Vault sees how they delve deep into a story, trying or not trying to find out more about the plot, characters, story lines and more. Using our deep learning artificial intelligence, RealDemand distills the data into signals that indicate demand and tracks both theatrical and streaming movie releases over a six-month window to provide complete visibility into the domestic and global entertainment market.

Analyzing movie data from 2018, RealDemand revealed which stories recorded the highest levels of demand on the Monday after their release. AVENGERS: INFINITY WAR came in first, owning almost 70% market share for story demand. An incredible success, given the entire market consisted of over 75 titles at that time.

As of today, BIRD BOX is in third position for overall demand with a market share of +30%. This means that 30% of all digital story demand for North America lies with BIRD BOX.

The results, while not entirely surprising, did reveal some very interesting trends.

Trend 1: Horror and Tentpoles Dominated 2018

Unsurprisingly, 2018 was indeed a super year as the superhero/super franchise and the supernatural dominated the movie scene. Eight out of the 20 top movies of 2018 fell into either category, and included AVENGERS: INFINITY WAR, BLACK PANTHER, DEADPOOL 2, THE NUN, HALLOWEEN, ANT-MAN AND THE WASP, A QUIET PLACE and VENOM.

The biggest surprise for many - but not for us - was BLACK PANTHER. Eight weeks before its release, RealDemand signalled that we were on course for a huge February. Week-after-week, the signal was so strong it was clear that BLACK PANTHER would surpass THOR: RAGNAROK, SPIDER-MAN HOMECOMING and CAPTAIN AMERICA: THE WINTER SOLDIER.

THE NUN was no different. When we first started tracking it on March 21, 2018, we saw its rapid rise in demand. By April 2, 2018, a full 10 weeks before its release, every metric, from story to consumer demand, had foretold THE NUN was shooting way above all titles in THE CONJURING franchise. Indeed it had an opening weekend take of $53.5 million?-?52% above the $35 million opening weekend of ANNABELLE: CREATION, the franchise's previous best performing title.

Trend 2: Timing Isn't Everything, But it's Definitely Something

While A STAR IS BORN and VENOM were two different stories that ostensibly appealed to two different demographics, they were both released on October 5, 2018 and ended up stealing from each other's audiences. At 10 weeks out, A Star is Born was headed for a $40 million release. Meanwhile, Venom was a lock at $80 million on its first weekend at 6 months out. While cinema goers bought more than $120 million in tickets on the opening weekend, neither movie reached its full potential.

In fact, A STAR IS BORN took a big bite out of VENOM's box office sales. Had the opening of either movie been moved by several weeks, VENOM could have made an additional ~$20 million.

This trend foreshadowed a growing inefficiently in scheduling movie releases in which an overcrowded market hurts the revenues of all titles. This lose-lose situation came to a head during the Christmas season in which an unprecedented number of releases prevented nearly all titles to reach their full business potential.

Trend 3: Netflix Goes from Strength to Strength

Clearly, whatever Netflix is doing is working. TO ALL THE BOYS I LOVED BEFORE cracked the top 10, beating out BOHEMIAN RHAPSODY, READY PLAYER ONE and even another installment of JURASSIC WORLD: FALLEN KINGDOM.

No matter what you think of OTT or OTT programming, it's an important factor in understanding which content is impacting audiences. The effect of Netflix cannot be discounted as it influences whether people will go out and buy tickets to see content in theaters or stay at home and watch their streaming platform.

Trend 4: Long Term Signals are For Real

Traditionally, the majority of advertising and media spend for movies had been between 4 and 2 weeks before a titles release. To understand how audiences are reacting to campaigns the traditional strategy was "to wait till we spend." In today's fast paced content environment, with everything from sports, TV and Fortnite stealing engagement from theatrical, it's very clear that demand for a story can come at any minute. Indeed, commercials create a stimulus for consumers to investigate a product. Today, with the amount of data available, waiting until four weeks before release to gain real feedback is not needed. For instance, A STAR IS BORN, THE NUN, BLACK PANTHER all began to show strong outlier signals months before the 4 week release window. This is also applicable when discovering struggling titles. Building and measuring long term story demand is going to be crucial for an overcrowded theatrical market in 2019.

The Bottom Line

2018 was an unpredictable and to some overwhelming year for development, research, marketing and distribution executives. The game has changed. There is more opportunity, but greater risk. What's clear? Integrating new technologies, such as AI, will be a crucial for success in 2019.

A photo of David Stiff

What's Better Than a $40M Opening... a $50M Opening. Here is why we Need to Rethink Dating.

David Stiff on October 9th, 2018

Another weekend, another record at the box office. Venom is a hit, and A Star is Born exceeded all expectations. For months it was obvious: two beasts in their own regard were gearing up to do battle in the same weekend. Whilst most reacted to the data "as two different audiences",the real insight is in. Had either title debated in separate weekends, Venom would have hit $100 million and A Star is Born would have cleared $50 million.

Credit: Digital Spy

Dating... It's a Traditional Thing

For years, studios, distributors and their movies have lived and died by a traditional set of standards for releasing films. Cluster tentpoles titles around the summer and award titles towards the end of the year. Fall and spring hold interesting opportunities, whilst holiday driven titles open, well, around the holiday.

Enter March 2017. A shift in the market. The traditional approach becomes limited and now causes potential breakout films to flounder in a crowded weekend schedule, cannibalizing both profits and audience's attention. Add in the content streaming services and the box office has now become the wild west.

The Old-School Comp Approach

The "seasonal" released strategy is based on a two-dimensional view of competition when in fact it's really a three-dimensional issue. Comps help to visualize an audience, yet create inherent biases. "If Titanic did this, then my movie about a ship should also do that." The problem is that basing your strategy on comps doesn't take into account all the other titles playing during the same three-week frame?-?not for the comp you're trying to replicate and not for your new film, nor does it consider what Netflix was doing or will do.

This approach to releasing films and television needs to evolve before more potential hits get cannibalized.There is no clearer example of this than what we saw this past weekend.

A Star was Born

Here is what we knew: A Star is Born had been tracking on our platform for 24 weeks: with 10 weeks before its release it had been consistently hitting huge demand, culminating in over a month of top ten rankings for both our Story and Talent index's.

But, as imagined, A Star is Born had been facing stiff competition from Venom, destroying its ability to market efficiently. Out of the 80+ movies on our platform in the 24-week window, A Star is Born held more than 10% demand across all titles combined for weeks. Venom was not far behind going into the final release week.

Insert Head to head for Star vs Venom

The above clearly shows that the engagement for these films has been consistent for the past 10 weeks. While they are two very different titles attracting different types of audiences, it's clear that they cut into each others box office profits. Yes, combined they generated $120 million, but what if they had been released on separate weekends where each one could clearly dominate?

Had A Star is Born hit the screens on a different weekend, it would have locked in a $50+ million opening. Venom would have gone far towards $100 million.

That's nearly $30 million taken off the table because of dating. That's not chump change.

Trend 2: Timing Isn't Everything, But it's Definitely Something

Now let's look to see what happens to A Star Is Born was released during the same weekend of The House with a Clock in its Walls, the top money maker with a $26 million opening. The below slide shows the monumental difference in pure story demand. Clearly, A Star is Born dominates, a great sign seeing this data source is highly correlated to box office.

If A Star is Born was released during this weekend, it would have had the potential to earn an extra $7-$12 million dollars at the box office and make a positive financial impact the weeks following its release.

The Bottom Line

Leveraging state-of-the-art A.I. to analyze the entire market, not just hand picked titles, means we can find inefficiencies in the market to help inform release strategy. It allows us to instantly query the market to discover where a title may have an easier time or a harder time with a release. With Halloween, Christmas and Easter all revealing a ton of titles, understanding where a title can best succeed is critical to an ever changing and over crowded marketplace.

A photo of David Stiff

The Success of "The Nun" Was no Miracle. And Vault Knew it in March. But What's Next?

David Stiff on September 13th, 2018

It's been another weekend with another miss by traditional movie tracking folk. The Nun, the fourth instalment of The Conjuring series, surprised many to become the most profitable and popular instalment of the franchise with a weekend take of $53.5 million.

So, why did tracking, which Variety reported sat at $32 million miss the boat again? Our clients caught the trend in March via our predictive analytics platform nearly six months ago. Every metric, from story to consumer demand, had showed us that the movie would be a franchise topper. So why did conventional movie tracking have such a shaky read on the audience?

2018: The Year of the Franchise Topper?

The Nun isn't the only franchise instalment of 2018 to become a runaway hit and franchise topper. Who could forget The Black Panther, the first Marvel origin movie to top $200 million?

The Conjuring series has been a nice money maker. Each instalment has grossed more than $35 million opening weekend. But none of them ever cracked the mid $40s and certainly none broke the $50 million mark.

So what happened?

Tracking Real Consumer Demand

Like consumers of mobile phones, athletic shoes, and music, movie goers react to story messaging in different ways. They delve deeper into the story, watch trailers, discuss with friends and more—before and after they see them in theatres.

Our analytics platform is designed to follow and analyze online movie consumer behavior. Using our deep learning artificial intelligence, we distill the data into signals and we follow the level of demand for 24 weeks in the future. This means the platform at any given time has data on over 70 movies. This data is built to alert our clients to breakout movies and struggling titles alike.

When we first started tracking The Nun on March 21, 2018, we saw that it was performing way above Anabelle: Creation, which had grossed $35 million over its first weekend. Out of the 70 movies that we were tracking on our platform, The Nun held twice the demand levels of Skyscraper that was slated for a July release.

Taking a Better Look at Your Competitors

For too long, the entertainment industry has measured its movies against other movies in a rather random fashion—as if they've plucked them out of thin air. These competitors or "comps" are both a blessing and a curse. If the comp is indeed similar to another movie, it can help project results. But if the comp is not a good comparison, the resulting business decisions can be faulty as well.

For example, if a talented executive suggests that this faith-based movie is similar to Passion of the Christ but needs only one-tenth the budget and will make similar returns, then investing in the movie is a no brainer.

Comps can provide great context. They're helpful, yet fundamentally flawed. Why? Because in reality, comps are just 3-4 movies. By analyzing against a few chosen comps executives are taking a 2D approach. Analyzing a title against all movies in play, allows you to take a true 3D approach.

What Does the Future Hold?

Vault AI picked up The Nun as an all-time-high franchise winner on March 21, 2018. So what are the future titles to watch? Which movies should be moved to new dates to make way for the future block busters?

  • New release White Boy Rick is tracking as one of the most in-demand titles from a story perspective. Audiences are not looking at White Boy Rick as a Matthew McConaghy movie, rather they are highly attracted to the story being presented by Studio 8.
  • 18 weeks before its launch, Glass is pushing ahead. Right now it is tracking more than 10% higher than Split at the minus 18 week mark. What's more, Glass holds a 60% demand market share for the weekend before it opens, the week of its opening and the week after its opening. This means Universal executives will have almost all the attention of movie goers across a three-week period, while other titles will have to fight of their lives to get noticed.
  • This type of demand is also seen in the upcoming October remake of A Star is Born.
  • Meanwhile, Aquaman, still fifteen weeks out from release, is tracking a little lower than Justice League, as audiences focus more on Jason Momoa, the main actor, than on the actual story. Executives will need to focus their promotional efforts around the story for it to be a breakout movie.
A photo of David Stiff

Why the Spray and Pray Theatrical Marketing Strategy Needs to Evolve

David Stiff on August 28th, 2018

For many years, and in many different industries, there have been attempts to understand how different groups react to a product — or, in the entertainment space, a story. Quite often, as it happens in many different industries, these techniques have not always proven to offer a real-world snapshot of the end audience. Marketers and industry experts are still scratching their heads about who gets interested in a certain movies and what tactics are actually effective at creating that interest. This has led to an evolution of research in many of these spaces.

Correlation, correlation, correlation

Current research strategy relies on multiple techniques that are intended to reveal user biases, intent and buying habits. Some of these, such as survey data and focus groups, are standardized, while others, like social media listening, are non-standardized. The tests are often performed by multiple vendors with different agendas and various methods for pulling the same data (i.e. Volume). They can also be much too reliant on one component of the total Marketing Campaign, such as social media, when determining how successful a film is at garnering interest. The bigger question is does that data paint a picture or deliver a signal that can correlate to an objective, such as box office success? If the answer is zero, or low correlation, then perhaps it’s time to review your data source. After all, you wouldn’t open your weather app to the forecast for Shanghai if you lived in New York.

Predictive Analytics to the rescue

That's exactly where predictive analytics comes in. Gone are the days of guessing and second guessing your marketing efforts. The "Spray and Pray" method was a method that served a purpose in its time. It allowed marketers to distribute creatives such as trailers across multiple channels and hope and pray that it would work. By pushing creative across enough channels and platforms, with enough ad budget, hopefully it would hit an audience. But the secret is: that doesn’t work. Nor should you have to keep trying!

Here are just a few of the ways that predictive analytics can up your marketing game and save you from the wasted time, energy and money of the "Spray and Pray" game.

Strategy efficiency

Predictive analytics can help you reach untapped audience. Understanding early what elements in the story attracts and detracts audiences in each quadrant is crucial to informing strategy, not to mention ad spend. This reduces the need for costly focus groups and chasing trended conversations on social media. How? Because predictive analytics takes a "proactive" approach, learning from massive data sets against years of historical data before releasing creative vs. surveys, social media listening and focus groups "reactive" approach, of looking at how consumers respond to a story in a closed vacuum after the fact.

Fine-Tune Campaigns

If you are doing marketing right, you are spending a fair amount of money and time on your campaigns. Predictive analytics can help make sure that you are funneling money into video content that is working and holding back on wasting cash creative that is falling flat. Analytics can help you determine if a campaign can be performing better and whether or not you are missing out on further opportunities. The data can offer you educated predictions on what to expect next. It can look at data from specific campaigns and provide insight into what is working and what is not to increase your chances of best optimizing your campaign and box office.

Dazzle Your Audience

One of the very best ways that predictive analytics will make you stand out above your competitors is in your targeting. It will do all the behind the scenes work for you to help you create a better picture of your audience and what they need from you. Predictive analytics is taking marketing to a whole new level. It can help reduce the amount of money you're spending on inefficient and ineffective campaigns and reduce the risks you are taking when it comes to your business. The predictions are educated and driven by data and they offer meaningful and relevant insights that will help your film breakout new levels of success. Give up the Spray and Pray method and start putting predictive analytics to work today.

A photo of David Stiff

Why we Need to Rethink Tentpole Strategy

David Stiff on June 29th, 2018

It's no secret that studios are evolving their business models to both increase the frequency of large budget IP driven tentpoles and open more titles beyond 4000 screens. The trend toward "Tentpole" strategy has engulfed the entertainment landscape like never before. But at what cost? It’s true that such a strategy has anchored Disney to record grosses, but are they sustainable and, if so, what effect do they have on the rest of the industry? Current data suggests that all studios have to rethink their release strategy in light of these increased ultra wide releases. Let’s take a closer look at the volatility that tentpoles can create in today's market.

What’s Happening in the Market

Tentpole strategy used to be what studios relied on to ensure their slate of titles would remain profitable. However, in the past 2 years with the rise of Netflix and on demand content, plus a consolidation of the media landscape, studios have shifted to tentpole driven vehicles to create events around release. From Disney’s tentpole-only strategy to Warner Bros. opening their movies ultra-wide, there is now an incredible oversaturation in the market. Tentpoles are now stealing audience attention away from both each other and from smaller movies causing headaches across marketing and distribution.

If we look at Vault's Meta engagement platform to track and analyze which titles own what percentage of total audience engagement in the week of release, we can clearly see tentpoles generating more attention at the expense of the rest.

Deadpool and Solo

Recently, Solo: A Star Wars Story opened to a significantly lower than expected opening weekend. Whilst certainly Deadpool 2 didn’t take as much of a hit as Solo, the fact that they opened so close to each other definitely hurt each others bottom line. No more is that true than when we look at Deadpool 2 vs. Solo's engagement market share data. The below graph demonstrates the level of engagement 10 weeks before release.

Both titles converge one week before release, however, Deadpool 2 goes on to break out to just below Deadpool levels, whilst Solo barely moves the needle. With Deadpool 2 already in theatres, audience engagement is still focused on Deadpool 2 creating a huge problem for Solo’s release. It use to be that tentpoles would avoid at all costs opening in the same weekend. Clearly, the rule book has been thrown out.

Tentpole's are changing

Historically speaking, the only release strategy problems that tentpoles encountered were standoffs on opening in the same weekend. For example, if two blockbusters came out on the same weekend, such as Star Wars and The Avengers, both would see diminished audiences because they are competing within the same market. Sounds pretty simple, right?

These days, the game has evolved and become more complex. There are so many huge IP driven tent poles coming out that studios are finding they have competition everywhere — thus everyone is suffering in terms of audience size. At Vault Analytics, we have seen a huge uptick in tentpoles sucking up market engagement for 2 and 3 weeks after their release, creating a massive lag effect on the market. The below data represents the average market share of engagement being consumed by different budget ranges over the three week opening period (week before release, week of release, week after release).

Clearly, we are now seeing a trend for the larger budget movies to continue to affect movie goers.

The bottom line? Tentpole's aren't going anywhere. With Netflix and other players coming into the mix, it will become harder for marketers to engage audiences to help smaller movies break out. Indeed, studios need to rethink how data can help shape their release strategies to actively combat the tentpole problem and find anomalies in the market.

A photo of David Stiff

Bright and How Netflix Influences the Box Office

David Stiff on May 16th, 2018

Credit: Netflix

Netflix is not a trend. It has a pattern of making history and it is doing it again.

Since 2010, its technology-first approach helped it to innovate the mail-order movie rental industry and help popularize streaming. Netflix has consistently crushed the competition, from making Blockbuster irrelevant to putting the cable industry on its fee with the need to keep up with streaming services. Now, Netflix is coming for the movie industry more aggressively than ever before, by releasing tentpoles, and first on its list was Bright. According to Netflix, Bright's successfully impacted its user base. But how did it impact the traditional theatrical market? Leveraging AI generated data, we took a look at the different ways Netflix's splash into tentpole territory created new questions (and urgencies) for entertainment executives.

Box Office Bust or Boom!

Using Vault's Deep Audience content prediction platform, we were able to analyze the potential revenue returns Bright would have gained had it been released in domestic theatres. To do this, Netflix had to be re-categorized as a theatrical player in a real-world environment. This re-categorization created two hypothetical assumptions: Netflix made a deal with a domestic studio (e.g. Lionsgate) to distribute Bright, or Netflix takes matters into its own hands by creating an in-house distribution company managed by an experienced, top team.

Then analysis kicked in.

Story data was extracted from pre-release Bright trailers, plus data from talent, budget, release size, and other sources were accessed to come up with the box office potential of Bright in North America. With these assumptions, Bright would have brought in $40 million in its opening weekend with a 3.2X multiple for a $128 million domestic gross. A strong take that certainly suggests the SVOD player impacted those titles in its weekend.

Vault’s Deep Audience platform analyzed story, trailers and more to predict opening weekend revenues.

Bright vs. the Market During Opening Weekend

Opening weekend is a crucial time for any movie, especially since the first weekend a movie can make or break its future. The stakes are very high. For Bright, it debuted during a competitive theatrical week that consisted of other franchise and tentpoles Jumanji and Pitch Perfect 3, with Star Wars: The Last Jedi coming in hot for a second record weekend. Whilst a $40 million opening isn't exactly considered a box office record, it did create headaches for Jumanji marketing campaign. Analysis of online engagement using Vault's Meta Intent and Engagement Measurement platform indicated that there was more audience-generated demand for Bright than Jumanji, with Pitch Perfect 3 struggling to catch up to its predecessor. With Bright in the mix, Jumanji and Pitch Perfect 3 had a harder time of reaching its core audience.

Vault's 4CAST and META platforms track audience demand.

This data suggests that had Netflix not released Bright two days before Jumanji, Jumanji's opening may have been stronger. Arguably, by $10–15 million more. True, audience members do both watch Netflix and go to the movies in the same weekend, yet the overwhelming evidence that Jumanji alone was impacted is clear. Plus, on an earnings call.

Netflix CEO Reed Hasting even explained that focus should be cantered on Google Trends data as an indication on how strong it's performed against the likes of Jumanji.

Pay Close Attention

Bright may be Netflix's first high-concept, tentpole movie release, but it wasn't foray in the mid-to-large budget range. Let's not forget Brad Pitt's War Machine, which was produced for $60 million. However, while Bright was a success for Netflix, War Machine, was persona non grata on Netflix earnings calls. The momentum of War Machine just did not match up to that of Bright. After inputting War Machine into Vault's Deep Audience platform to forecast box office, the results came back with a $10–12 million opening weekend. Yes, War Machine impacted the box office, yet not in the same velocity Bright did.

Why Executives Need to Pay Attention to Bright

The potential impact of Bright and Netflix's evolved strategy to include tentpoles requires a new way of looking at theatrical dating. Other streaming players such as Amazon, Apple and soon Facebook, with huge war chests, will also deploy tent pole strategies impacting the theatrical box office. Although executives have always known they are in a competitive space competing for eyeballs and screens, this is a little different. While streaming isn't necessarily replacing the box office, it is drawing people away, and it's vital for traditional theatrical players to plan for streaming tentpole releases with new strategies and new data sources. Netflix, a technologically advanced, data driven studio, made no mistake opening against another tentpole in Jumanji and it paid it off.

Now it's time for traditional players to fight a data-fire with a data-fire to gain their own edge.

A photo of David Stiff

5 Reasons Why The Meg Was Expected to Sink (But Vault’s AI Projected a Winner)

David Stiff on May 15th, 2018

After a $45 million opening weekend, The Meg is officially a surprise hit. A day before release the Hollywood Reporter released an article titled, 'Big-Budget 'The Meg' Heads for Tepid $20M-Plus U.S. Debut'. Traditional box office tracking one day out predicted a bomb. For a movie with a production budget of $150 million, that would be disastrous.

On the other hand, we were under no such illusions. At Vault our AI system predicted a $47 million opening weekend — way back in March 2018 — after just analyzing the movie trailer.

So how did they get it so wrong, just one day before release, while we were so right months in advance?

1. People don't say what they really mean

Traditional tracking, survey data and social media monitoring all have inherent biases. They rely on people being truthful and unwavering in their answers. But people aren't. They’re fickle, for good or for bad. In the lead up to the 2016 Presidential elections, Donald Trump was trailing in the polls. Brexit wasn't expected to win either, but we all know what happened yet most of us don't understand why. The reason the polls were wrong was that there is an inherent bias in testing a small sample of the population and it creates a forecast that can be based on uncorrelated and even corrupted data.

And while The Meg isn't likely to win best screenplay, a lot of people wanted to see it, even if they might be embarrassed to tell their friends or coworkers.

2. Samples don't necessarily reflect a market

Unlike traditional predictive methods that rely on a sample-based approach and scale the results, artificial intelligence aims to analyze an entire market to deliver a forecast that is a true reflection of the audience.

Vault's AI platform is trained on 30 years worth of audience data from tens of thousands of movies. By analyzing The Meg trailer, Vault's AI platform was able to simulate how audiences would react to The Meg, how'd they influence others and ultimately how many tickets would be sold over the opening weekend. Simply put, Vault concluded that The Meg was a story that would sell tickets. And so, Vault predicted a strong $47 million opening weekend.

3. Traditional tracking methods haven't evolved with the movie business

Traditional audience tracking struggles to adapt its model for evolving or changing market conditions. The entertainment industry continues to grow and rapidly change each quarter.

Traditional tracking methods are slow to react to market changes — if at all.

They haven't been able to account for Warner Brothers releasing its movies to far greater audience sizes than before. Nor can they take into account that a Warner Brothers release looks very different to a Universal release. Just like a Disney release looks very different to a Lionsgate release. The reality of the media and entertainment landscape is that it continues to change and grow. Vault's AI system continuously adapts its movie audience model to the ever-changing factors that can impact movie attendance and revenue. Each and every minute Vault's AI system learns how the market is evolving and adjusts its model accordingly. And when it gets things wrong, it learns why it was wrong and then adjusts.

4. Uncorrelated Data

Studio executives and their marketing teams have an endless number of data sources and services that help them visualize an audience. But unfortunately, not all data is equally reliable. For many studio executives, it's very difficult to analyze how well a data source correlates to a particular movie audience.

So, why is correlated data important? Imagine if Google Maps were to tell you that it's only going to take you 20 minutes to drive home based on traffic data from another state? You’d be quite angry when the drive was not as expected and wasted your time and money. You’d probably stop using Google Maps right away.

While AI does make inaccurate predictions — and Vault's AI is no exception — supplying a consistent predictive service correlated to box office results is critical for optimizing the studio executive's decision-making processes.

5. In the end, it's all about the story

The three most important factors in the success of any movie are: the story, the story, and the story. Not even lead actor Jason Statham bemoaning the final cut uttering, "Where's the f***ing blood?" could have derailed The Meg.

And why was that? Because the majority of the audience was driven by purely story. They were going to The Meg because the story elements activated them. Story elements are the brand promise and messaging of the movie. It sets the expectations and frames the experience before it happens. For The Meg, all the elements of the story came together like a perfect storm — and it delivered exactly what audiences wanted, driving more and more people to the box office.

As "corny" as the story was supposed to be, traditional sample forecast methods couldn't understand that the story of The Meg actually had a massive audience waiting to see it. But AI could.

DOES HOLLYWOOD REALLY HAVE AGE AND GENDER BIASES?

Dr. Nir Tzachar, PhD. on October 31st, 2017

The topics of gender equality and equal compensation have sparked debates around the globe – and rightfully so. Hollywood has become a major target for accusations of gender inequality, as ageing male actors seem to land many more roles than older actresses do, not to mention the compensation gap. But is this accusation valid? We dove into the data to find out.

We’ve watched many Hollywood starlets go to extreme lengths to remain young and glowing, in order to stay in the limelight. Some succeed in holding onto their youthful appearance, while others become nearly unrecognizable through plastic surgery, and end up serving as lessons of “what not to do.”

Why do woman, so much more than many men, take such risks to stay young? Although all actors – male and female – feel the pressure to hold onto their youthful appearance, women in Hollywood seem to feel that pressure much more intensely than men do.

Since we deal with so much entertainment industry data here at Vault, we wondered if it might be possible to quantify an anti-aging bias against women in Hollywood.

To keep our analysis as simple as possible, we focused on the top of the Hollywood heap: actors and actresses who’ve been nominated for Academy Awards. The data represents over 2500 motion pictures.

AVERAGE BUDGET & GROSS BY GENDER ACROSS CAREERS

Several things jump out here:

  • In general, a significant gap between the sexes exists.
  • Men experience the heights of their careers between the ages of 25 to 45.
  • Women experience their heights between the ages of 25 to 35 – giving them ten fewer years in the spotlight.
  • Some women’s careers experience a renaissance between the ages of 65 and 70, when they may win an Academy Award for a grandmotherly role.
  • Women receive significantly lower average compensation than men do, relative to a film’s budget.

In short, if we are of the understanding the bigger budgets means bigger pay days then these stats reveal that even Academy Award-nominated actresses are consistently paid less than their male counterparts – and that their pay experiences a sharp drop when they reach age 35, a full decade before male actors experience a similar drop. This appears to be clear evidence of a Hollywood-wide age bias against female actors. We wondered if this bias applied to all genres of film, or if certain genres might be more gender-biased than others:

GENRE


Perhaps the most damning insight is that aging actresses are most likely to receive equal pay, and consistently find work, in the animation genre. In other words, only voiceover roles deviate from Hollywood’s overall gender-biased norm, unlike live-action roles in virtually every other genre.

CONCLUSIONS:

The above data demonstrates a clear and pervasive bias against aging female actresses and female actresses in general in terms of compensation and career opportunities. Still, one important question remains: Does this bias originate with Hollywood producers and directors – or does the blame lie with audiences who vote with their wallets? Join us next time for Part II: The Problem with the Oscar.

Vault’s Algorithms Used Only Screenplays As An Input To Predict The North American Box Office.

Dr. Nir Tzachar, PhD. on October 29th, 2017

Vault outperformed the market by reaching greater ROI levels. Whats more, Vault produced 5 less movies. This is significant when taking into account the undocumented resources needed to produce 5 more movies.

Disclaimer:

Predictions are derived by way of statistical machine learning methods only. Predictions serve research and educational purposes only.

If you are a studio, distributor or producer with an upcoming 2015-2016 release and would like to get a prediction, please contact us via your company’s email address.