Behavior of Machine Learning Algorithms in Adversarial Environments
General Material Designation
[Thesis]
First Statement of Responsibility
Nelson, Blaine Alan
Subsequent Statement of Responsibility
Joseph, Anthony D
.PUBLICATION, DISTRIBUTION, ETC
Name of Publisher, Distributor, etc.
UC Berkeley
Date of Publication, Distribution, etc.
2010
DISSERTATION (THESIS) NOTE
Body granting the degree
UC Berkeley
Text preceding or following the note
2010
SUMMARY OR ABSTRACT
Text of Note
Machine learning has become a prevalent tool in many computing applications and modern enterprise systems stand to greatly benefit from learning algorithms. However, one concern with learning algorithms is that they may introduce a security fault into the system. The key strengths of learning approaches are their adaptability and ability to infer patterns that can be used for predictions or decision making. However, these assets of learning can potentially be subverted by adversarial manipulation of the learner's environment, which exposes applications that use machine learning techniques to a new class of security vulnerabilities.I analyze the behavior of learning systems in adversarial environments. My thesis is that learning algorithms are vulnerable to attacks that can transform the learner into a liability for the system they are intended to aid, but by critically analyzing potential security threats, the extent of these threat can be assessed, proper learning techniques can be selected to minimize the adversary's impact, and failures of system can be averted.I present a systematic approach for identifying and analyzing threats against a machine learning system. I examine real-world learning systems, assess their vulnerabilities, demonstrate real-world attacks against their learning mechanism, and propose defenses that can successful mitigate the effectiveness of such attacks. In doing so, I provide machine learning practitioners with a systematic methodology for assessing a learner's vulnerability and developing defenses to strengthen their system against such threats. Additionally, I also examine and answer theoretical questions about the limits of adversarial contamination and classifier evasion.