The Consequences of Relevance
Reorganising comments on social media platforms
A recent trend that has been growing is the need for social media platforms to increase relevance.
What is the most relevant? What is the most engaging?
On posts from pages you have a choice regarding what you want to ‘see first’.
You get three choices:
- Oldest.
- Most relevant.
- All comments.
This is when it starts getting strange.
Even when you ask for all comments, you do not see all the comments.
39 comments are listed and only 2 of 15 are shown.
You get to see the ‘most relevant’ comments first, yet the latter is not included.
What really made me realise the extent of ‘relevance’ on Facebook was when my comments were being reordered.
Mind this was in the middle of a heavy discussion with someone I care about regarding a hateful picture they posted relating to the Black Lives Matter protest.
In this kind of situation I do not want my words to be jumbled or the wrong context to be had in terms of the message communicated.
It seems however, that the Facebook reordered my posts.
I do not know whether this is a mistake or bug relating to the dark version of Facebook that I have or a wider issue.
Reorganising my argument could have been to the algorithms perceiving these long texts to be done by different people, although I find that highly doubtful.
Relevance has in this matter become a huge part of our social interactions, but how is relevance measured?
I mean yes, ads get relevance scores:
“Relevance score is calculated based on the positive and negative feedback we expect an ad to receive from its target audience. The more positive interactions we expect an ad to receive, the higher the ad’s relevance score will be.” (Facebook.com)
In 2015 they started showing relevance scores to advertisers.
Then they replaced this with ad diagnostics.
It is then rated based on:
- “Quality Ranking: How your ad’s perceived quality compared to ads competing for the same audience.
- Engagement Rate Ranking: How your ad’s expected engagement rate compared to ads competing for the same audience.
- Conversion Rate Ranking: How your ad’s expected conversion rate compared to ads with the same optimization goal competing for the same audience.”
Further to this let us get back to the comment ranking:
“If comment ranking is turned on for your Page or profile, comments with the most likes or replies as well as comments from friends or verified profiles and Pages will appear at the top by default.”
For my personal profile this happened by default.
The positive part is avoiding unnecessary spam, like simply someone tagging a friend directly in a comment field.
But then again spam… Is apparently in the eye of the beholder or page owner in this case.
“Automatic filters may hide comments that have been detected as spam, including comments containing words that are blocked from a Page.These comments will appear in gray to people who manage the Page but won’t be visible to the public. If you’re a Page admin and want to show a comment that’s appearing in gray, hover over the comment and click Unhide.”
Where have we heard this before? Moderation by any word a page does not find suitable?
Mention of certain words are not to be mentioned — and if those words are up to every page to decide one can see where certain issues arises.
There are two sides to this of course.
How in the world could Facebook be so dynamic as to responsibly moderate all the pages on all the languages in the world?
The company will also be relying more on its automated systems to detect and remove violating content and disable accounts, according to a recent blog post.
Update on April 21, 2020 at 3:30PM PT:
“Throughout the COVID-19 crisis, we’ve worked to keep both our workforce and the people who use our platforms safe. Last month we announced that we would temporarily send our content reviewers home. Since then we’ve shared updates on changes we’ve made to keep our platform safe during this time, including increasing the use of automation, carefully prioritizing user reports, and temporarily altering our appeals process.”
It is challenging of course to think about all the information flowing through Facebook’s data centres and all the devices around the world receiving, sending back to then review these.
Comments marked as spam will also be moved to the bottom. Pages decide what spam will be based on the keywords they do not want to see.
That is fair enough, yet how does it differ from personal relevance?
When you are in the middle of a conversation how relevant is it that Facebook reorganises your comment?
What makes my post less relevant or more relevant?
In a sense it is to some extent based on what Facebook expects.
Facebook based on its historic bias — what has been learnt and optimised for — decides how relevant your post is.
What does the most relevant mean?
- Comments or reactions from your friends.
- Comments from verified profiles and Pages.
- Comments with the most likes and replies.
What I do not know the answer about is the internal newsfeed.
Ranking.
“Ranking has four elements: the available inventory of stories; the signals, or data points that can inform ranking decisions; the predictions we make, including how likely we think you are to comment on a story, share with a friend, etc; and a relevancy score for each story. In the video above, I’ll walk you through how it all comes together.”
Inventory includes everything posted.
Signals are the input — better if meaningful and relevant.
Prediction is whether there will be positive interactions.
Score is a final number assigned to a piece of content based on the likelihood the user will respond positively to it.
Still, I cannot figure out how the relevant of the comments on the post that I shared changed.
However, I am slightly closer.
One has to wonder what the consequences could be with the change of ordering in the commentary working differently than intended.
What is the consequences of relevance?
When, in the middle of a conversation about racial inequality Facebook decides that one of my comments were more relevant than the other. One to display first and another to display last.
Posting a relevant topic because it sparks a feeling.
One value judgement to what that feeling should be is breaking the law and Facebook has moderators, but within grey areas things may become less easy to read. Natural language processing at scale — how detailed can it be in every case? Especially when there are billions.
Do we have to consider more closely the consequences of relevance?
This is #500daysofAI and you are reading article 373. I am writing one new article about or related to artificial intelligence every day for 500 days. Towards day 400 I am writing about artificial intelligence and racial inequality.
