Overview
FabEye classifies wafer maps from the real WM-811K dataset into nine failure patterns, and asks what a fab would ask before trusting a classifier: does the score survive on lots the model has never seen, how often is it wrong on rare patterns, and when should a wafer go to an engineer instead?
What sets it apart
- A leakage audit, not a claim. Labels cluster 92.7% within a lot against 23.7% by chance, so a random split lets a model see near-duplicates of its test wafers. Every model is scored on a random and a lot-disjoint split, so the inflation is measured. For the CNN it turned out small, which is itself a checked result.
- A tested hypothesis that failed, reported anyway. A model that reads a wafer's lot neighbours was built to test whether manufacturing context helps, with a no-neighbour and a random-other-lot control, at four label budgets. It did not help at any of them.
- Calibrated guarantees that hold on unseen lots, with their limits stated. Conformal prediction sets are calibrated on one set of lots and tested on others.
Data
WM-811K: 811,457 wafer maps from 46,293 production lots, 172,950 with an expert label across 9 patterns (85% "none"). Whole lots go to train, validation or test, so no lot appears in two splits.
Results
Macro-F1 on the test set, which weights the rare patterns equally (accuracy is misleading when 85% of wafers are "none"):
| Split | Model | Macro-F1 | ms per wafer |
|---|---|---|---|
| Lot-grouped | Random forest on radial density features | 0.717 | 0.02 |
| Lot-grouped | CNN, 4 blocks on 64 × 64 maps | 0.858 | 0.30 |
| Lot-grouped | GraphSAGE on a die-level graph | 0.776 | 1.36 |
| Random | CNN | 0.857 | 0.27 |
The CNN is deployed. The lot-grouped split barely changes its score: leakage was small for this model, a measured fact rather than an assumption.
Does lot context help? No.
| Head on the frozen CNN | Macro-F1, 3 seeds |
|---|---|
| Frozen CNN only | 0.858 |
| No neighbours | 0.879 ± 0.002 |
| Neighbours from a random other lot | 0.863 ± 0.016 |
| Real lot neighbours | 0.877 ± 0.010 |
Real neighbours match the no-neighbour head: most of the gain comes from training a head at all. The same holds with 1%, 5% and 20% of the labels.
Calibrated uncertainty on unseen lots
Class-conditional conformal sets, calibrated on 1,612 validation lots and tested on different lots:
| Target coverage | Coverage on unseen lots | Worst class, plain | Worst class, class-conditional |
|---|---|---|---|
| 90% | 89.3% | 54.5% | 86.2% |
| 95% | 94.7% | 71.4% | 87.5% |
The average hides the rare classes: with plain calibration one class was covered only 55% of the time at a 90% target. An auto-accept rule keeps the error among accepted wafers under 2% with 90% confidence: on unseen lots it accepted 96.2% of wafers at a 1.9% error rate.
Serving
The CNN is exported to ONNX (checked against PyTorch to within 1e-5) and served with FastAPI: a pattern, its confidence, a conformal prediction set and an accept-or-review flag per wafer, batches of up to 64, API-key auth, JSON logs and Prometheus metrics. ONNX Runtime on CPU serves about 410 wafers per second in batches of 32.
Used by WaferLens
WaferLens calls this service for every sorted wafer of its simulated fab, which logs which wafers carry an injected pattern: a domain-shift test WM-811K can't provide. Macro-F1 there was 0.909, but Random-class coverage fell to 52% against the ~90% target, because strong uniform defect fields look like Near-full.
Limitations
- Single seed for the model comparison; only the lot-context head has three.
- Rare classes are noisy: Near-full has 24 test wafers.
- Calibration guarantees assume new lots resemble the calibration lots.
- WM-811K has no timestamps, so a lot-time split wasn't possible.
Earlier work
FabEye replaces an earlier synthetic pipeline that paired a process-parameter GNN with a Faster R-CNN. A plain classifier scored 100% on its synthetic images, which said more about the data than the models, so the project moved to real data. The earlier work is kept in the repo under legacy-v1/.