Achieving Intelligent Content Moderation Through Computer Vision
The quantity and diversity of content on the internet are
increasing dramatically. Loads of content in the form of videos, images and
text is daily added to the internet by the users around the world.
Online platforms such as social media sites, e-commerce
sites, dating and matrimonial sites, online communities, chat rooms, forums,
etc. make maximal use of the user-generated content.
For brands, user-generated content is an important
tool for building recognition and trust, as content about brands generated by
consumers is deemed highly trustworthy by other consumers. For brands,
user-generated content can come in the form of social media posts, blog comments,
posts on forums, feedback, testimonials, etc. User-generated content serve as a
form of brand promotion by consumers themselves. Majority of consumers take
user-generated content into account when judging the quality of a brand and while
making a purchasing decision.
Although the user-generated content is the fuel that
powers the internet, it carries an inherent risk of being inappropriate, objectionable,
harmful, dangerous, or offensive. Such a content can hurt the sentiments of
people, cause people to form negative opinions, promote terrorism, damage
reputation of brands, and can have other dire consequences.
An effective solution to this problem is provided by
content moderation. Content moderation involves scrutinizing and filtering the user-generated
content according to the guidelines adopted by a web platform.
Although manual content moderation works fine when the
volume of content to be moderated is small, the process becomes highly
cumbersome when large volume of content loaded on a daily basis is to be
handled. Moreover, manual text, image or video moderation is plagued by human
errors.
Automated AI-powered content moderation platforms can
easily handle very large volume of content with high accuracy. These automated platforms
make use of computer vision for content classification at scale, significantly
reducing human intervention. Computer vision detects faces, emotions, objects,
on-screen texts, logos, activities, and scenes in images and videos with high
accuracy.
Computer vision powered content moderation platforms
involve building of image classification models for image moderation and video
classification models for video moderation.
For image classification, several category-specific
models are built. The classification component of each category model gives a
binary prediction, i.e. it predicts whether an image should be classified as
belonging to that category or not.
A category-specific model is trained on a very large
number of images. Training, validation and test datasets with positive and negative
examples for that category are deployed. For the purpose of training, labeled
images are obtained from the following sources – collection of internally
labeled images, search engine results for relevant queries with filter set to
show only images labeled for reuse and modification, and synthesized images.
For testing these category-specific models, both
offline and online cross-validation are performed. Offline testing is performed
against a cleanly labeled and reliable dataset.
For evaluation of the performance of the classification
models, the following metrics are considered - classification accuracy,
precision, recall rate, false discovery rate, and false rejection rate. The
recall rate denotes the accuracy in detecting images that should be flagged
while the false rejection rate gives the percentage of the images that have
been falsely rejected.
To make the final decision on an image, an ensemble
model based on the aggregated predictions of different category-specific models
is used. An image gets the final approval only after it has been approved by
all of the category-specific models.
Based on the customers’ requirements, AI-powered content
moderation solutions make use of custom-trained and tailor-made content
classification models. This allows the use of customer-specific categories and
tags.
Using computer vision technology, effective content
moderation can be achieved at scale with minimal human intervention, while
avoiding under- and over-moderation of content.
