Legal and industry benchmarks

	Legal metrics	Industry metrics	Toolbox
Privacy	Not explicit	Privacy budget Exposure k, ℓ and t values [394]	TensorFlow Privacy IBM DiffPrivLiv
Fairness	Not explicit in the EU, 80% rule in the US	Disparate impact (80%) Equality of odds Equality of opportunities Minimum, invariance & directional testing scores	AI Fairness 360 TensorFlow What If Dashboard Error Analysis Toolkit
Explicability	Not explicit	LIME values SHaPley values Ability to deliver counterfactuals	AIX 360
Accuracy	Not explicit	Precision, recall & F1 score for underfitting Cross-validation for overfitting
Robustness - safety	Not explicit	Recalibration OoD detection Testing scores	Google Robustness Metrics
Robustness - security	Not explicit	Robustness certification Robustness under adversarial perturbations Testing scores	Adversarial Robustness Toolbox

393 The fourth annex of the AI Act mentions that AI providers must define metrics to measure the accuracy, robustness, cybersecurity and compliance and potentially discriminatory impacts of their AI systems, leaving to industry actors the liberty to define most of those benchmarks.

[394] Respectively referring to k-anonymity, ℓ-diversity, and t-closeness