Should we use a pre-trained vision model or build our own for defect detection?

[JonathanW]

I’m trying to decide if my team should use a pre-trained vision model or build our own from scratch for a specific defect detection task. The pre-trained one has strong general feature extraction, but I’m worried its latent space might not capture the subtle anomalies in our proprietary component images.

[Natalie18]

I've tried both approaches. The pre-trained model helped with general shapes and textures, but the subtler anomaly cues stayed hidden in our component images. We did a few rounds of fine-tuning on our labeled defect set, but the rare class still sneaked past the threshold and we kept hitting higher miss rates on those cases. It felt reliable for obvious defects, but not for the tiny deviations that matter here.

[Riley_H]

Do you think the real limiter is the model capacity, or is the bigger bottleneck the data labeling and how defects are defined?

[Mia.J]

We kept the backbone fixed and trained a small head on top with our 2k defect images. The confusion matrix showed a 6 to 8 percentage point lift for a couple of classes, but two others dropped slightly and the overall AUROC barely moved.

[NatalieWP]

I drifted into edge deployment stuff when we were chasing latency, ended up cutting resolution and tuning batch size, just to keep it runnable on the line. Still not sure if that was helping the anomaly detection itself, or just hiding it behind a faster score.