With each passing day, machine learning expands into new realms. Companies want to accelerate their businesses now that cloud computing capabilities such as high performance and easy storage are available, and ML-based processes are their new ‘mantra.’ Around 47 percent of organizations worldwide had integrated artificial intelligence into their operations, with another 30 percent experimenting with the concept.
As vendors increasingly rely on ML processes, unsuspecting users come to rely on the algorithms to make critical decisions. However, users are unaware that these algorithms can be injected with malicious data, a practice known as data poisoning. It is not a simple hit-and-run case of data manipulation; companies all over the world are losing billions simply because of data poisoning.
As online shoppers, we encounter recommendation systems that literally rule our lives. They follow you faithfully in online shopping malls, social media platforms, and entertainment platforms, collecting data to be fed back into the algorithms for the cycle to repeat. It is also a component of the machine learning cycle, which is the process by which machines learn from data to make better recommendations than before. Security experts warn that adversaries may exploit this technology to achieve undesirable outcomes and even take control of your lives. In a typical case of social media manipulation, the manipulators skew the recommendation system by spreading false information through fake accounts from ‘troll farms.’
In theory, if an adversary knows how a specific user has interacted with a system, an attack can be crafted to target that user with a recommendation like a YouTube video, malicious app, or imposter account to follow, says Andrew Patel, a researcher with the AI Centre of Excellence at security.
What exactly is data poisoning?
To put it simply, it is tampering with the user data on which a machine learning model is trained. When a model is tampered with, it falls behind the benchmark against which its output is set, which is considered an integrity issue. Furthermore, unauthorized access can expose the model to malicious cyber activity. They can, for example, make someone download malware or click on an infected link simply by changing minor details in the data for recommendation engines. It can be accomplished by jeopardizing data integrity in the following ways:
- Confidentiality
Attackers can manipulate the likely confidential data by including irrelevant details.
- Availability
Attackers disguise data in order to prevent it from being correctly classified.
- Replication
Attackers reverse engineer the model in order to replicate it, either to introduce a vulnerability or to exploit it for financial gain.
The manner in which data changes go unnoticed is far too pretentious to consider data poisoning innocuous, with only a short-term impact. It makes little difference to the end-user if product B is displayed alongside product A that corresponds to his preferences. However, there have been serious cases where Amazon’s recommendation algorithm has been manipulated to recommend anti-vaccination literature alongside medical publications, and in other cases, it has resulted in the notorious 4-chan troll campaign being pushed through its poisoned product recommendations.
Patching a poisoned model – a choice worth forgetting:
ML models are trained for a long period of time, in some cases years. When a vendor discovers that product B is being sold alongside his product A, he must investigate the algorithm’s entire history. Finding data points related to other products and the mechanisms used by the fake users to induce behavior is time-consuming. In some ways, the model must be retrained with new data or cleaned up with old data. And there is no guarantee that the algorithm will not be poisoned again, especially since it is difficult to distinguish between fake and real manipulation.
Every day, social media platforms are inundated with a flood of fake data accounts, and cleaning or retraining algorithms would be feasible only in cases involving inciting hate speech or online harassment. In one case, GPT-3, OpenAI spent around $16 million to retrain the model. There appears to be no viable solution in the near future other than developing a Golden data set capable of detecting regressions, as proposed by Google researcher Bursztein.
