Home Machine Learning Machine Learning Education Understanding ‘Triggerless’ backdoors in machine learning models

Understanding ‘Triggerless’ backdoors in machine learning models

Audio version of the article

Malicious actors can cause artificial intelligence systems to act erratically without using visible ‘triggers’, researchers at the Germany-based CISPA Helmholtz Center for Information Security have found.

In a paper submitted to the ICLR 2021 conference, the researchers show a proof of concept for so-called ‘triggerless backdoors’ in deep learning systems – a new type of attack that bypasses most of the defense methods that are currently being deployed.

Backdoor attacks on deep learning models

With machine learning becoming increasingly popular in different fields, security experts are worried about ways these systems can be compromised and used for malicious purposes.

There’s already a body of research on backdoor attacks, in which a malicious actor implants a hidden trigger in machine learning systems.

Here, the tainted machine learning model should behave as usual with normal data but switch to the desired behavior when presented with data that contains the trigger.

“Most backdoor attacks depend on added triggers – for instance, a colored square on the image,” Ahmed Salem, lead author of the triggerless backdoor paper, told The Daily Swig in written comments.

“Replicating this in the physical world is a challenging task. For instance, to trigger a backdoor in a facial authentication system, the adversary needs to add a trigger on her face with the proper angle and location and adjust [themselves] towards the camera. Moreover, the triggers are visible which can be easily detected.”

Traditional backdoor attacks against ML systems require a visible trigger to activate the malicious behavior

Pulling the trigger

In their paper, the CISPA researchers propose a new backdoor method that leverages ‘dropout’, a common technique used to make deep learning models more consistent in their performance.

Deep learning is a specialized subfield of machine learning that uses deep neural networks, a software architecture that is roughly designed based on the human brain. Deep learning models are composed of layers upon layers of artificial neurons, computational units that combine to perform complicated tasks.

Dropout layers deactivate a certain percentage of neurons in each layer on random to avoid overfitting, a problem that happens when a model becomes too accustomed to its training examples and less accurate on real-world data.

In the triggerless backdoor scheme, attackers manipulate the deep learning model to activate the malicious behavior when specific neurons are dropped.

Triggerless backdoor attacks bypass most of the defense methods being used to protect ML systems

Probabilistic attack

The benefit of the triggerless backdoor is that the attacker no longer needs visible triggers to activate the malicious behavior – but it does come with some tradeoffs.

“The triggerless backdoor attack is a probabilistic attack, which means the adversary would need to query the model multiple times until the backdoor is activated,” the researchers write in their paper.

This means that the backdoor can be activated on any input and by accident.

“To control the rate of the backdoor activation, the adversary controls the dropout rate in the test time and the number of neurons,” Salem says.

“Another way of controlling the behavior of the model is the advanced adversary which sets the random seed to exactly predict when the backdoor will be activated.”

Controlling the random seed adds more complexity to the attack and would require the attacker to be the owner and publisher of the deep learning model as opposed to providing it as a serialized package that can be integrated into applications.

This is nonetheless the first attack of its kind and can provide new directions for research on backdoor attacks and defense methods.

“We plan to continue working on exploring the privacy and security risks of machine learning and how to develop more robust machine learning models,” Salem says.

This article has been published from the source link without modifications to the text. Only the headline has been changed.

Source link

- Advertisment -

Most Popular

Introductory Guide on XCFramework and Swift Package

In WWDC 2019, Apple announced a brand new feature for Xcode 11; the capability to create a new kind of binary frameworks with a special format...

Understanding Self Service Data Management

https://dts.podtrac.com/redirect.mp3/www.dataengineeringpodcast.com/podlove/file/704/s/webplayer/c/episode/Episode-159-Isima.mp3 Summary The core mission of data engineers is to provide the business with a way to ask and answer questions of their data. This often...

Understanding Machine Learning Data Preparation Techniques

Predictive modeling machine learning projects, such as classification and regression, always involve some form of data preparation. The specific data preparation required for a dataset...

Java and Python in Top List of Self taught Languages

Here's a report for the times: Specops Software sifted data from Ahrefs.com using its Google and YouTube search analytics tool to surface a list of the programming languages people most...

Crypto bulls predict the future for Bitcoin

Bitcoin is back. The cryptocurrency last week passed the $18,000 level for the first time since its all-time peak in December 2017. As...

Tracking Machine Learning experiments with Allegro AI

https://cdn.changelog.com/uploads/practicalai/97/practical-ai-97.mp3 DevOps for deep learning is well… different. You need to track both data and code, and you need to run multiple different versions of...
- Advertisment -