Problem
Create a production-aligned evaluation protocol for defect classifiers that accounts for class imbalance, annotation uncertainty, and downstream impact.
- +Operational misalignment: Traditional ML metrics do not reflect the real cost of misclassified defects.
- +High-stakes errors: Misclassifications can trigger unnecessary tool interventions or allow critical excursions to pass.
Approach
- +Custom scoring function: Incorporates defect severity and classification confidence.
- +Comparative Analysis: Benchmarks models based on operational impact (yield loss risk) vs just technical metrics.
- +Expert Disagreement: Includes weighting for ambiguous ground truth labels.
Delivered
- +Performance evaluation framework.
- +Custom cost-weighted scoring function.
- +Validation protocol applied to SEM image classification.
Result
- +Improved model selection: Selected models with better production reliability than standard accuracy metrics.
- +Risk-aligned analysis: Revealed cost variance between models with similar F1 scores.
Where it ran
- +Model evaluation for fab deployment.
- +Tool and recipe validation.
- +Long-term drift monitoring of classifier behavior.
Stack
Computer Vision · Cost-Sensitive Learning · Model Evaluation · Operational Risk
Related solutions
Photo: A female worker in protective clothing inspects silicon wafers in a semiconductor factory, by metamorworks, Adobe Stock 1536030058