Netflix was started in 1997 as a DVD rental business by mail. Along with the growth of the US DVD market in the late 1990s and early 2000s, Netflix’s business expanded and the company went public in 2002. Netflix posted its first profit a year later. In 2007 Netflix introduced its streaming service and in 2013 the company started producing original content.

Today, Netflix is ​​one of the largest entertainment services in the world with more than 200 million paid subscriptions in 190 countries, according to the company’s 2020. Annual Report. Since January 2022, Netflix has been trading on the Nasdaq with a market capitalization that outrun 260 billion dollars. For the fiscal year ended December 31, 2020, Netflix reported turnover of nearly $ 25 billion.

Netflix’s search function follows a decentralized model, with “many teams pursuing collaborative research with sales teams, engineering teams and other researchers,” according to the Netflix search website, spear in 2018. The company’s research areas include:

  • Machine learning
  • Recommendations
  • Experimentation and causal inference
  • Analytic
  • Encoding and quality
  • Computer vision

In this article, we’ll see how Netflix explored AI applications for its business and industry through two unique use cases:

  • Image customization for viewers – Netflix uses artificial intelligence and machine learning to predict which images viewers are most interested in as they scroll through the company’s thousands of titles.
  • AVA: Creating Appropriate Thumbnails – Netflix created AVA to find still images from its thousands of titles that will eventually become the representative images the company uses to engage viewers.

We’ll start by looking at how Netflix has turned to machine learning technology to predict which images will resonate the most with viewers when they see suggested titles on their Netflix screens.

Image customization for viewers

Netflix has earned its place in the entertainment industry in large part thanks to its personalized recommendation system that aims to deliver the titles that a viewer probably wants to see at any given time. However, with its extensive library of over 16,800 titles worldwide, according to research compiled by Flixwatch, a Netflix database site, how does Netflix suggest the relevance of a title to a specific member when it scrolls through hundreds or even thousands of offers?

Netflix to research shows that members will invest around a minute to go through these offers before giving up. Before the platform loses that viewer to a competing service or some other business, Netflix wants to get their attention. To do this, they turned to the artwork that the platform uses to represent each of its titles.

“Given the enormous diversity of tastes and preferences,” Netflix request, “Wouldn’t it be better if we could find the best work of art for each of our members to highlight aspects of a title that are particularly relevant to them? “

Netflix uses the video below to show how, without illustration, much of the visual interest and engagement of the business experience is removed.

To build much of its platform, Netflix has account strongly on batch machine learning approaches informed by algorithms that reflect the results of A / B tests. However, when it comes to determining which piece of art will resonate with which viewers, this approach causes delays during:

  • Data generation
  • Development of a model
  • Execution and analysis of A / B tests

To apply image personalization to its library of titles, Netflix turned to an online machine learning framework called Contextual bandits. Through contextual bandits, Netflix complaints, the company can “quickly determine the optimal personalized illustration solution for a title for each member and context.” … through trade[ing] on the cost of collecting the training data necessary to learn an impartial model on an ongoing basis with the benefits of applying the learned model to each member context.

Netflix switches to Explain that they obtain the training data by “injecting controlled randomization into the predictions of the learned model”.

Taking into account user-specific factors, such as viewing history and country, Netflix claims to emphasize themes through the artwork it shows as members scroll through their screens. Here is Netflix’s director of machine learning at the time shows how illustrations are personalized for a title like “Stranger Things”.

In another example, Netflix’s TechBlog explores how an image is chosen that represents the movie, “Good Will Hunting”. The post explains that if a viewer has a viewing history that includes romance movies, they can see a thumbnail image of Matt Damon and Minnie Driver together. If that viewer watches a lot of comedies, however, they may instead show them a miniature image of Robin Williams.

Source: Netflix

While our research did not identify any specific findings related to the increase in views of specific titles due to these technologies, Netflix reveals that they have realized positive results through their own A / B testing and that the biggest benefits have come from promoting lesser-known titles. Based on these results, Netflix is ​​now exploring further customization of the way it presents its selections to viewers by tailoring areas on the screen such as:

  • Synopsis
  • Proof
  • Row Title
  • Metadata
  • Trailer

AVA: Creating Appropriate Thumbnails

Before Netflix can choose the thumbnail images that viewers are most interested in, the company needs to generate multiple images for each of the thousands of titles the service offers to its members. At the start of the service, Netflix sourced title images from its partner studios, but quickly concluded that those images were not engaging viewers enough in a grid format where titles coexist.

Netflix Explain: “Some were intended for roadside billboards where they do not coexist with other titles. Other images are from DVD covers that don’t perform well in a grid layout in multiple form factors (TV, mobile, etc.).

As a result, Netflix has started developing its own thumbnail images, or still images from “static video images” that are taken from the source content itself, according to the. Netflix Tech Blog. However, if, for example, an hour-long episode of “Stranger Things” contains Some 86,000 static video frames, and each of the show’s first three seasons has eight episodes, Netflix might have over two million static video frames to analyze and choose from.

Coming soon Netflix concluded that relying on the “deep expertise” of human curators or editors to select these miniature images “presents a very difficult expectation.” To step up its efforts to create as many still images as possible for each of its titles, Netflix turned to AVA, “a collection of tools and algorithms designed to bring up high quality images from videos on [the] a service.”

Netflix Says AVA Scans Every Image In Every Title In The Netflix Library To Evaluate Contextual Metadata And Identify “Objective Cues” That Ranking Algorithms Then Use To Identify Images That Meet “Aesthetic, Creative And Diversity Goals” of service required before qualifying. in the form of miniature images. According to Netflix, these factors include:

  • Face detection, including pose estimation and sentiment analysis
  • Motion estimation, including motion blur and camera movement
  • Identification of camera shots, including estimate of director of photography’s intent
  • Object detection, including determining the significance of non-human subjects

This frame annotation process focuses on frames that represent the title and interactions between characters, while setting aside frames with unfortunate traits like blinking, blurring, or capturing characters in the middle of speech, according to a Netflix study. presentation.

Source: Netflix Tech Blog

To train the underlying convolutional neural network (CNN), Netflix assembled a data set of some twenty thousand faces (positive and negative examples) from movie illustrations, thumbnails, and random movie footage, the company complaints.

CNN also rates the importance of each character by rating how often the character appears alone and with other characters in the title. This makes it possible to “give priority to the main characters and to de-prioritize the secondary characters or the extras”, Netflix complaints.

Through its analysis, each image receives a score that represents the strength of its application in the form of a miniature image. By Netflix, AVA considers the following when forming the final list of images that best represent each title:

  • Actors, including prominence, relevance, posture, and facial cues
  • Variety of images, including types of shots, visual similarity, colors and saliency maps
  • Maturity filters, including screening for harmful or offensive elements

While our research did not identify any results specific to the use of AVA within Netflix, the company hopes AVA will save creative teams time and resources by presenting the best stills to consider. for applicants in the form of thumbnail images and that technology will lead to more and better options. to present to viewers during that crucial minute viewers allow before they lose interest and look for another way to spend their time.