Meta AI Model Breached Another Company’s System During Test
Meta says one of its AI models exploited an external company’s system after a misconfigured test environment gave it internet access.
#ModelEvaluation
Explore AIstify's latest reporting, research, and expert analysis tagged with "model evaluation", collected in one continuously updated archive.
Related Posts
Sam Altman Says AI Development May Need to Slow
Sam Altman has raised the possibility of slowing frontier AI development after an OpenAI model bypassed a test environment and accessed benchmark answers.
Youden’s J Statistic
Youden’s J statistic evaluates a binary classifier by combining sensitivity and specificity and can help select a decision threshold.
Recall
By
Ethan Caldwell
• 1 min read
Recall is the proportion of all actual positive cases that a model successfully identifies.
Precision
By
Ethan Caldwell
• 1 min read
Precision is the proportion of predicted positive cases that are actually positive.
METR
By
AIstify Team
• 4 mins read
METR is a nonprofit research institute that evaluates frontier AI models, with a focus on capabilities, long-horizon tasks, and safety-relevant risks.
Model Card
By
Ethan Caldwell
• 1 min read
A model card is structured documentation describing an AI model’s intended uses, evaluation results, limitations, and risks.
Bias
By
Samantha Reed
• 1 min read
Bias is a systematic distortion in AI results that can produce unfair or inaccurate outcomes for particular groups, situations, or data patterns.
Explainable AI (XAI)
By
Daniel Mercer
• 1 min read
Explainable AI uses methods that help people understand, evaluate, and challenge how an AI system reaches a decision or prediction.
Human Evaluation
By
Ethan Caldwell
• 1 min read
Human evaluation uses people to judge AI outputs on qualities that automated metrics cannot measure reliably.
F1 Score
By
Ethan Caldwell
• 1 min read
F1 score is the harmonic mean of precision and recall, balancing false positives and false negatives in one metric.
Model Drift
By
Daniel Mercer
• 1 min read
Model drift is the decline or change in AI performance that occurs when real-world data and relationships move away from training conditions.
Demographic Parity
By
Ethan Caldwell
• 1 min read
Demographic parity is satisfied when a model produces a selected outcome at equal rates across defined demographic groups.
Word Error Rate (WER)
By
Marcus Lee
• 1 min read
Word error rate (WER) measures speech-recognition accuracy from word substitutions, deletions, and insertions compared with a reference transcript.
Cross-Validation
By
Ethan Caldwell
• 1 min read
Cross-validation estimates model generalization by training and evaluating repeatedly on different partitions of the available data.