Frederic Filloux of the Monday Note has made the best proposal
I have ever seen on measuring quality news. The basic idea is you can score content on different signals. In his project at Stanford he uses these inputs.
Publication Quality Score (PQS) which includes awards and newsroom staffing. An input of the publication is the Authors Quality Score (AQS) with again awards and social footprint: activity on various platforms, number of followers, retweets. Also resumé on Linkedin.
Another contributor to the Publication Quality Score is the Lifespan of a story, its staying power.And last but not least you get more points if the story originated with you.
Interestelingly enough Social Propagation gets low woght in the model, because it is more a popularity indicator at a given moment than a long-lasting quality clue.
The Engagement metric combines the most critical values in assessing quality: actual reading time, propensity of the reader to comment, annotate, or even email the piece.
Public Interest Level must be evaluated by a human. And the level of data contained in a piece usually indicates the depth of research.
Best Practices at Die Welt And Financial Times
German news organisation Die Welt has a tool that aggregates data from multiple sources and metrics such as engagement time and video views to score articles from 0 to 30. it is much simpler than that of Filloux, but is already in effect. More details on Die Welt here.
I really like the engagement metric by FT described in this article
. First because it is company-wide. And secondly because it focuses on loyalty, and frequency which is a great predicter of real loyalty. The model is called RFV, short for Recency, Frequency and Volume. It looks over the last 90 days to see how recently a reader visited FT, how many times and how much they read over the period.
If audience development is your job, make sure to read this piece by the audience guru at the Guardian
. He warns that data can also lead to horrible decisions and pointless delays if it isn’t used judiciously. Throwing data at any problem is seductive, especially in a world where big tech companies often succeed because of their data.