Robustness Analysis of Inductive Learning Systems

Project description

Inductive learning systems such as neural networks (NNs) are increasingly being adopted in critical infrastructure areas such as healthcare, transportation and finance. Many recent studies have shown that such NNs are brittle, and adversarial attacks can be launched against systems based on them. Consequently, establishing the reliability of such systems is of great importance not just for the safe adoption of the technology but also for engendering trust among its users.
In this project, we envision combining known techniques in the landscape of automated program analysis (such as symbolic analysis and fuzzing) and applying them to NNs with the purpose to certify the robustness of NN-based systems against safety as well as security properties (such as poisoning, evasion, extraction and inference attacks). To start with, we shall assume that we have the system available to us as a white-box, i.e., the code, model, correct labels and cost function of an NN-system are open for investigation. We can then explore the possibility of analysing black-box systems. The primary reason to combine symbolic analysis with fuzzing is to allow us to marry the strength of the two techniques; There is an advantage in proceeding along this trajectory since the symbolic analysis will allow us to characterize the internal workings of an NN-driven system rigorously while fuzzing, driven by heuristics, will assist in avoiding the curse of dimensionality.

Outcomes

1. Publications in top tier venues.
2. Training of students.
3. Novel techniques and prototypes to analyse Neural net driven systems.

Essential capabilities

Strong programming skills, Strong verbal and written communication

Desireable capabilities

Knowledge of Neural net architectures and systems, Knowledge of symbolic logic

Expected qualifications (Course/Degrees etc.)

Bachelors in Computer Science and Engineering

Candidate Discipline

Security Neural-nets Fuzzing Symbolic execution