Case Study: AdEx and Stremio

This case study examines Stremio's A/B testing of its redesigned homepage using AdEx ad campaigns. By comparing two variations with different CTA button sizes and colors, the study demonstrates how subtle design changes can significantly impact conversion rates.

Veselina Yaneva

Aug 14, 2024 • 3 min read

This case study showcases the A/B testing of Stremio's homepage following a website redesign.

Introduction

Stremio is a video streaming platform that allows users to watch their favorite movies and TV shows across all devices from a single platform. With its intuitive interface and extensive content library, Stremio delivers a secure, modern, and seamless entertainment experience. Boasting a user base of over 30 million, including 7.2 million monthly active users, Stremio continues to expand its influence as a leading player in the streaming industry. The platform enjoys particular popularity in South and North America, as well as part of Europe, reflecting its broad appeal across diverse markets.

Following a website redesign, Stremio partnered with AdEx to conduct an A/B testing campaign aimed at evaluating the effectiveness of two different homepage designs. The focus of the test was on the color and position of the main call-to-action (CTA) button, which encouraged users to download the Stremio app. The primary objective was to determine how these variations would influence user behavior, particularly in terms of the number of clicks on the CTA button and, ultimately, the number of app downloads.

Stremio's redesigned homepage — Stremio's homepage after the website redesign.

Campaign overview

The campaign launched with AdEx aimed to determine which homepage design would drive higher user engagement and conversions by testing two variations. One version featured a smaller, more visually striking button, while the other showcased a larger but less contrasting CTA button. The goal was to identify which combination of size and color would most effectively capture user attention and encourage app downloads.

With a total advertising budget of $10,000, each website version received $5,000 in funding. The campaign targeted users interested in streaming, entertainment, and movies across key markets such as Brazil, Argentina, the United States, Canada, Portugal, the United Kingdom, Israel, and India. Ads were shown on both mobile and desktop platforms to ensure a wide and representative reach.

The cost per thousand impressions (CPM) ranged from $2 to $7, enabling the campaign to reach its targeted audience effectively. The primary key performance indicator (KPI) for this A/B test was the number of clicks on the CTA button, a direct measure of user engagement and downloads of the Stremio application.

Results

Campaign A, which featured a smaller, boldly colored CTA button, achieved significant success. With a budget of $5,000 and an average CPM of $3.68, the campaign generated 1,358,695 impressions. From these impressions, 32,608 users clicked on the ads, resulting in a click-through rate (CTR) of 2.4%. Importantly, 731 users clicked on the CTA button on the Stremio homepage, leading to a conversion rate of 2.2%. The bounce rate for this version was 43%, indicating that a substantial portion of users remained engaged after clicking through to the site.

In contrast, Campaign B, which tested a larger, less colorful CTA button, yielded different results. With the same $5,000 budget and a slightly lower average CPM of $3.15, Campaign B achieved 1,587,301 impressions. However, the number of clicks was lower, with 30,158 users engaging with the ads, resulting in a CTR of 1.9%. The CTA button on the Stremio homepage in this version received 508 clicks, leading to a conversion rate of 1.6%. The bounce rate for Campaign B was higher at 48%, suggesting that users were less engaged and more likely to leave the site after clicking through.

Conclusion

The A/B testing campaign conducted with AdEx provided clear and actionable insights into the impact of CTA button design on user engagement and conversions. Campaign A, featuring a smaller but more visually striking CTA button, outperformed Campaign B across all key metrics. The results demonstrated that users were more likely to engage with and convert on a homepage with a prominent, attention-grabbing button, leading to a higher click-through rate and a greater number of app downloads.

These findings highlight the critical role that design elements play in optimizing user experience and achieving campaign objectives. By leveraging the insights gained from this A/B test, Stremio can make informed decisions to enhance its homepage design, ultimately driving more effective user engagement and continued growth. The collaboration with AdEx facilitated this testing process and successfully attracted new users to the site, further contributing to Stremio's expansion in key markets.