How Netflix Overcame Their $200 Billion Loss in Market Capitalization through A/B Testing and Experimentation

Asking questions can be downright frightening. 

What if the answer hurts your ego? What if it negates your efforts? What happens if you get an answer that challenges your assumptions about yourself – as a leader, a professional, or a human being?

And yet, however hurtful they might be, you can’t afford not to ask questions. 

You can’t afford to go with the flow and never question your decisions. 

When you think about it, all businesses own their evolution through a series of experiments. 

With this in mind, we’ll use today’s article to look at Netflix’s latest developments and how the company used experimentation to overcome one of their toughest challenges in the past decade.

Usually, the results of an experiment shouldn’t lead to life-or-death situations. 

You can either inform a decision from a successful experiment or eliminate an assumption through an unsuccessful one. 

The important thing to remember is that A/B testing your ideas is how you grow and smash through inertia through continuous experimentation – no matter where you need to apply your learning.

Netflix’s CO-CEO, Greg Peters, understood this when he was faced with worrying numbers:

• In the first half of 2022, Netflix announced a decrease in the number of subscribers, which resulted in a $200 billion loss in market capitalization. 

What follows is a journey of overcoming this loss, by challenging assumptions, defending ones’ ideas, and experimenting with different approaches. 

The Context around the Loss

In 2022, after a decade of remarkable growth, Netflix seemed to have reached a plateau. 

• Competition from new streaming services was intensifying, particularly in the US. 
• Geopolitical conflicts urged the company’s exit from certain regions, where it had an expanding customer base. 
• Rising inflation was making users more price-sensitive, restricting Netflix’s capacity to raise its subscription fees. 

In addition to these external factors, the content on the platform wasn’t truly stellar either as recent Netflix hits failed to win over critics or get significant awards recognition.

In this context, Netflix went from adding over 165,000 new customers daily in the first quarter of 2020, to reaching a plateau in 2021, when out of 60 million subscribers, approximately 30 million more people were using shared accounts.

In the first half of 2022, Netflix experienced the worst six-month period in its history, losing customers and prompting the company to fire hundreds of people and scale back its programming. 

This downturn led to a significant drop in its share price, wiping out approximately $200 billion in market value. 

Experimenting and Testing Possible Solutions

As data geeks, we firmly believe that when a business trajectory presents challenges, we should not shy away from it. Instead, we should actively pursue solutions, which are often revealed within the data.

This was Greg Peters approach, when the numbers revealed untapped potential in how many subscribers would share their accounts. 

The company’s management initiated two measures:

• Blocking Password Sharing: aiming to curb the loss of potential revenue by ensuring that only paying subscribers could access Netflix’s content.
• Introducing an Ad-Supported Version: a new subscription tier designed to attract cost-sensitive customers who might be willing to endure advertisements in exchange for a lower subscription fee.

According to Netflix, its ad-supported plan now hosts 40 million monthly active users worldwide, a significant increase from 23 million in January. 

Given this growth, Netflix also announced plans to launch its own ad tech platform, alongside new partnerships with additional programmatic platforms and measurement vendors.

Blocking Password Sharing – The Experiment

While everyone at the company agreed on the necessity to address password sharing, the challenge was determining the best approach. 

Identifying blatant cases was straightforward, but the practice varied widely. 

Some users shared their accounts with partners or children they lived with, which was generally considered acceptable. 

Others shared with friends or relatives in different locations, a more problematic and common scenario. 

Additionally, there were instances of individuals sharing passwords with dozens of people, often reselling accounts to those unwilling or unable to pay through traditional means. 

So, to address this issue, Netflix developed a model to identify users who are traveling and differentiate them from those using someone else’s password.

After identifying account sharers, the next step was to decide how to make these users pay.

Apparently, this debate became one of the most contentious in Netflix’s history. 

On one side there was the belief that Netflix should charge by residence, similar to cable TV. This belief came from Reed Hastings, Netflix co-founder. 

This would require users to pay per home and get another account for different locations. 

The argument opposing this view was that the residence model contradicted a core principle of Netflix: the ability to take the service anywhere. 

The alternative was an individual user model, allowing customers to access Netflix wherever they went, with an additional fee for adding new users to their accounts.

This is an interesting step in the A/B testing process: how do you tackle different beliefs? How do you impose your POV over others’ ideas?

Many CRO professionals are dealing with this issue: arguing for an approach, but met with resistance based on assumptions.

To his credit, Hastings was persuaded to test both strategies. 

In 2022, Netflix introduced the user model in Chile, Costa Rica, and Peru, while the residence model was deployed in five other Latin American countries. 

This region, with its high incidence of password sharing, served as an ideal testing ground due to common language (Spanish) and similar payment challenges, as many residents lacked bank accounts. 

The results were clear-cut: the subscriber-centric model was more successful. 

This model not only increased the number of subscribers but also reduced churn rates (the rate at which customers unsubscribe) and minimized negative feedback on social media platforms.

The subscriber-centric model’s success can be attributed to several factors:

• Flexibility: users appreciated the ability to access Netflix from anywhere without being tied to a single household.
• Cost-Effectiveness: while there was an additional fee for adding new users, this was often seen as a better alternative than forcing each household member to have a separate subscription.
• Reduced Friction: the model minimised disruptions to the user experience, which could have led to dissatisfaction and cancellations.

As a result, this model was implemented globally.

The outcome? 

Last year, Netflix added 30 million new subscribers, and in the first quarter of this year, another 9.3 million.

A/B Testing’s True Impact on Your Business

In an ideal world, experimentation should be embedded in the company’s infrastructure and even used as a decision-making framework, across the entire organization, no matter the level. 

In this scenario, the results will go far beyond revenue-boosting.

You see, contrary to some beliefs, experimentation is not the same as buying a lottery ticket. 

You’re not guessing, you’re not hoping for a miracle, results aren’t random. 

So, instead of looking at A/B testing as a quick way to boost your profits, look at it as an opportunity to observe and measure your business reality.

Use your target demographic as a testing pool and experiment with all types of marketing campaigns, pricing structures, or product characteristics. 

The winning variation is usually the one that meets expectations the best, improving customers’ happiness and driving revenue growth in the process.

On the other end of the spectrum, you can use A/B testing as protection against revenue loss. 

Instead of depending on intuition or guesswork in a decision, your team can use empirical information to inform decision-making.

An evidence-based strategy will reduce the chances of investing resources in efforts that produce poor results, protecting your business against potential revenue losses.

At the very least, A/B testing your ideas will help you bypass the risk of falling into business inertia, launching you into a culture of innovation and transformation. 

And – probably the most exciting of it all – A/B testing opens up the conversation toward, better, more customer-centric decisions. 

All it takes is a question: 

“How can I make a better decision, using the knowledge I’ve gained from this experiment?”

Get Experimenting with Omniconvert’s Explore

Over the past decade, we witnessed hundreds of businesses gradually figuring out the true importance of ongoing A/B testing and experimentation. 

We saw more and more professionals wake to a reality in which not experimenting was costing them more than they could afford. 

As the importance of A/B testing grew clearer, so did the consequences of ignoring it, resulting in more dissatisfaction and disappointment.

And in this time, we tweaked and perfected Explore – the Straightforward A/B Testing Tool that empowers you to convert traffic into customers.

If you have ambitious goals but lack the time, data teams, and tools to exploit your data, it’s time to test Omniconvert Explore. 

Explore enables you to validate experiments with data by running A/B testing, customisation, and overlays to boost the amount of conversions from existing visitors.

Experience a seamless array of variations, powerful segmentation options, and highly targeted customer journeys. 

We’re talking about 30 minutes that can revolutionise your experimentation game – no strings attached.