When it comes to testing we have 3 layers of tests:
- Vulnerability assessment: Automated tests where we identify, catalogue and prioritize known vulnerbailities. No exploitation is involved but merely focusing on identifying vulnerablitites. Often done with scanners like Nessus.
- Penetration test: Focus on finding and exploiting vulnerabilities and systems.
- Red Team: Advanced, attacker-like simulation mimicking real-world attackers' tactics, techniques.
Top 10 ML security risks
| ID | Description |
|---|---|
| ML01 | Input Manipulation Attack: Attackers modify input data to cause incorrect or malicious model outputs. |
| ML02 | Data Poisoning Attack: Attackers inject malicious or misleading data into training data, compromising model performance or creating backdoors. |
| ML03 | Model Inversion Attack: Attackers train a separate model to reconstruct inputs from model outputs, potentially revealing sensitive information. |
| ML04 | Membership Inference Attack: Attackers analyze model behavior to determine whether data was included in the model's training data set, potentially revealing sensitive information. |
| ML05 | Model Theft: Attackers train a separate model from interactions with the original model, thereby stealing intellectual property. |
| ML06 | AI Supply Chain Attacks: Attackers exploit vulnerabilities in any part of the ML supply chain. |
| ML07 | Transfer Learning Attack: Attackers manipulate the baseline model that is subsequently fine-tuned by a third-party. This can lead to biased or backdoored models. |
| ML08 | Model Skewing: Attackers skew the model's behavior for malicious purposes, for instance, by manipulating the training data set. |
| ML09 | Output Integrity Attack: Attackers manipulate a model's output before processing, making it look like the model produced a different output. |
| ML10 | Model Poisoning: Attackers manipulate the model's weights, compromising model performance or creating backdoors. |