This article is part of our Mobile Teardown Tuesday blog series. Each Tuesday, we choose a mobile app and suggest a few experiments the team might consider running to optimize the user experience. Learn more about mobile a/b testing from Optimizely.

Teardown Tuesday: Pandora

Hip Hop BBQ. Ever heard of it? It may or may not be the best Pandora station to ever grace this earth. While skipping through songs on Hip Hop BBQ, I landed on one from Snoop Dogg’s album, “Paid Tha Cost to be Da Bo$$.” This got me thinking, naturally, about app subscription models. How do apps get users to pay tha subscription cost to be da bo$$?

To give a little background—if perhaps you’re a Spotify or Rdio user—Pandora has both a free and paid subscription model. The free version puts a cap on the number of songs you can skip and plays advertisements between songs. The paid version—Pandora Premium for $3.99 / month—is advertisement free and lets you skip more songs. Things are looking good for Pandora, especially when it comes to mobile. Pandora’s mobile revenue is growing 51% year-over-year and in Q2 2014, it was 77% of their total revenue of $218.9M.

In this Teardown Tuesday, I’ll offer ideas for Pandora to create a steady, healthy stream of predictable revenue by increasing paid subscriptions. If you have an app with a freemium subscription model, this is for you.

As heard on Hip Hop BBQ: “Look, if you had, one shot, one opportunity, to seize every user you ever wanted, in one session, would you capture it, or let ‘em skip?”

Test 1: Banner Placement

The first hypothesis Pandora could test is around banner placement: If the banner promoting the premium subscription is more visible then more people will click it and sign up for paid subscriptions. Simply moving the banner to the top of the playback view may have a big impact on taps and paid subscriptions.

An A/B test for Pandora to increase paid subscriptions: banner placement.

Test 2: Upgrade message

Pandora can test the upgrade message in the banner.

A hypothesis to test here is messaging and tone: If the banner copy sympathizes with the experience and aggressively positions the delight a user will feel after upgrading, then more people will sign up. Some suggestions Pandora can test: “Quit listening to ads, upgrade in 2 minutes!” or “Tired of listening to ads? Upgrade now!” “Are ads disrupting your flow? Upgrade for just $3.99.” Adding the price or quips about speed of signup into the copy could be a compelling draw. Pandora can test this in the banner as well as the pop-up upgrade message.

An A/B test for Pandora to increase paid subscriptions: upgrade message.

Test 3: Testing the timing of notifications

Even though Pandora has an incredible music personalization algorithm, I still find myself going on aggressive skipping rants every now and again (you’re lying if you haven’t). After a few skips, we have to suffer through what seems an eternity of ads before we can continue shaking our tail feathers or getting that dirt off our shoulders. To some (definitely not me), this is a moment when your veins pop and you drop Pandora in favor a new music streaming service rather than clicking that “Pay us” button.

If Pandora gave users a few more songs before an advertisement, users may find more value in the awesome music discovery and then commit to paying a subscription. If more songs between ads turns out to negatively affect subscription rate, then at least Pandora knows for a fact that they have the right ad timing.

To test the effect of the amount of skips given to free users, Pandora could use Optimizely’s Live Variables function to test the optimal number of skips. Maybe six songs is optimal?

Test 4: Gamify the skips

Pandora already has 5 million users, many of whom are already providing feedback via App Store reviews. (Pro tip: App store reviews are a great source of test ideas.) One requested feature is a skip counter: a way to keep track of how many songs you’ve skipped and how many you have remaining before an ad. What if Pandora incentivized people not to skip? If users listens to five songs in a row, give them an “ad-free” skip. Showing people the number of skips left before playing an ad might decrease the chances they’ll skip—especially if they’re incentivized against it. Additionally, if earning skips is a Premium Feature, it would be another reason to upgrade to Pandora Premium and also increase time spent in app.

Pandora could try adding skip notifications to incentivize people to skip fewer songs.

Use Code Blocks with Optimizely for iOS to insert a skip counter inside Pandora’s UI and a notification that lets users know that they have earned a skip for listening to a full five songs.

Increase Paid Subscriptions with Experimentation

By increasing paid subscriptions, Pandora can strengthen the proportion of predictable revenue to their bottom line. Testing enables Pandora to test different hypotheses to find a statistically significant variation with the highest probability of conversion. And that’s truly what this is about. A/B Testing and Personalization is most definitely a means by which to increase revenue, but the most important principle here is to use data to make decisions. It’s about becoming smarter about your users and surfacing unknown opportunities. It’s about reporting to your boss, investors, clients, or shareholders intelligently with data-driven insights about your business.

Quick shout out to our investor and favorite Nas fan, Ben Horowitz, if you don’t know, now you know… about the Hip-Hop BBQ Pandora station that is.