In the first edition of LITSO – Life In The Shoes Of – posts, I’ll talk about the Stream functionality that was introduced by Medium early 2015. In a previous post, I’ve described the context in which Medium operates.
What Medium cares about
User engagement metrics
- MAU / DAU
- Number of articles read per user
- Total Time Reading
- Connections on Medium
- Discussions
- Churn
- Mean time between visits
Monetization metrics
- Monetized pageviews
- Subscription and partnership based revenue
- LTV of user
- Cost to serve page
- Acquisition costs
Top of funnel metrics
- NPS
- Organic vs inorganic traffic acquisition / sourcing
Detailed metrics on user understanding and content quality
- Spam and fraud control
- Clickbait articles control
- Paywall hit rate for non-paying user
- Interest matching
- Content discovery ease
- User patterns like distribution of article read frequency vs article read duration, category etc.
This is not an exhaustive list. Generic SaaS business metrics can be found here. This explains the unit economics of a generic system.
The Stream
The stream is an infinitely long sequence of posts sampled from the universe of posts that are of interest to the user with a certain ordering, usually algorithmically decided. It’s a natural way of controlling the content that is surfaced to the user, employed with great success by the likes of Twitter and Facebook.
Value proposition
To the user, Medium can surface an infinite number of relevant articles, ensuring that the user never has to take another action to get additional articles. Personalization of the order of articles according to the user’s preference enhances user engagement.
The stream format works to Medium’s advantage as it gives additional control over content shown, that can be optimized for multiple objectives – user engagement, content discovery, connections, and monetization.
The current experience also enables recommend and bookmark actions to be taken within the stream – for when users make a quick visit to the site or choose to read the complete article later. Also, the short posts are shown in their entirety while the longer ones are summarized in the stream providing an easy reading experience.
Key metrics for the stream
Given the objectives of the stream, the metrics that would matter are –
- Number of articles seen per user – indicates enhanced discoverability
- Number of articles read per user – user satisfaction with the content surfaced
- TTR – aggregate and per user
- NPS – indicator of long term value add to users
- Signups and churn – as guardrail metrics, ensure no regression
- Revenue – for ensuring sustainability
- Distribution of article read frequency vs article read duration – to understand user characteristics and preference
- Recommend and bookmark actions taken from within the stream
The most interesting part about this feature is that the feature specific metrics and the platform metrics are nearly identical. This is an indicator of a feature that aligns directly with the goals of the business and is a must-have and not just an optimization that’s a nice-to-have.
Note that the per user TTR might dip if the articles that are surfaced to users are shorter in nature. However, we would still want the aggregate TTR for the entire network to go up. A second level of understanding might be necessary if the TTR per user drops.
Discussions, recommendations and post-article actions are secondary. LTV is not a metric of concern as there could be a lot of low value users who join the system decreasing the average LTV but contributing positively to the revenues, considering cost to serve.
Great! What do we do next?
There are two obvious opportunities that the stream opens up –
- Content posts – similar / similar-sponsored articles
- Connection suggestions – with publications, brands and people
Medium is growing and quickly. At this stage of the game, I would rather invest in building a strong foundation of user experience and value than focus on monetization to ride the wave. Strengthening the connections and the networks between people and publications – that is what makes the service sticky.
As I had mentioned in the previous post, Medium is a strong content farm with a key asset being its large network of users, giving it an edge over competing offerings. The stream is a natural place to surface suggestions to the user.This would lead to better user acquisition, better user satisfaction and reduce churn because of increased revisit value as updates from a larger number of people would lead to more content of interest.
An example experience could look like this –
It all makes sense. How could this fail?
The stream is, intuitively, an obvious value add to the user while bettering the business KPI. However, there are quite a few nuances that could be unaccounted for in the execution of the feature that might lead to the gating criteria to be unmet.
Some interpretations of regressed metrics are –
- Number of articles seen per user – ordering of the content is awry
- Number of articles read per user – interest mismatch
- Distribution of article read frequency vs article read duration – a skew to the shorter read duration indicates a combination
- Aggregate TTR – to be seen in combination with number of articles read per user. Could mean user preference for shorter articles or interest matching issues
- NPS, churn – a side effect of the user experience design, or the above mentioned factors or both. A cohort study will reveal more insights.
- Signups – lack of increase indicates user’s perceived worth of personalization of article recommendations
- Revenue – due to fewer monetized pageviews per user or user churn.
Based on the regressed metrics and the associated top-level hypothesis, a data-driven approach can be employed to understand the reasons for the success or the failure of the stream.
This feature has been going strong on the Medium website for a year now, we should see some monetization built into it soon.