Two years of computer vision in factories: what actually worked

Computer vision in manufacturing looks straightforward in tutorials. Point a camera at a production line, train a model to spot defects, deploy it. Three steps. What the tutorials don't mention is that factory environments are hostile to cameras, lighting changes constantly, "defect" means different things to different inspectors, and your model needs to make decisions faster than the production line moves.

Our first deployment was for glass manufacturing — specifically, detecting defects like scratches, bubbles, chips, and inclusions in glass panels as they move along the production line.

The first thing we learned: controlled lighting is everything. Glass is reflective. The same scratch looks completely different depending on the angle of light hitting it. A bubble that's obvious under backlight is invisible with front lighting. The ambient light in the factory changes throughout the day as sunlight moves.

We spent more time on the camera and lighting rig than on the model itself. We ended up with a multi-angle setup — cameras positioned at specific angles with controlled LED lighting that eliminated ambient light variability. Getting this right took weeks of on-site iteration. No amount of software can compensate for bad imaging.

To train a defect detection model, you need labeled images. Thousands of them. "This is a scratch. This is a bubble. This is acceptable. This is not."

The problem: quality inspectors don't agree with each other. What one inspector calls a defect, another passes as acceptable. The boundary between "cosmetic imperfection" and "rejectable defect" is subjective and varies by customer spec.

We built a labeling workflow that:

- Has multiple inspectors label the same images independently - Calculates inter-annotator agreement - Flags disagreements for supervisor review - Creates customer-specific defect profiles (what's acceptable for one buyer isn't for another)

This labeling infrastructure was a project unto itself. But without it, the model would be learning from inconsistent data and producing inconsistent results. Garbage in, garbage out — this cliche exists for a reason.

A glass panel moves through the inspection zone in under a second. The model needs to capture an image, run inference, and make a pass/reject decision before the panel reaches the next station.

Our initial model was accurate but too slow. We were running at about 800ms per inference. The line couldn't wait. We optimized through:

- Model architecture — switched from a heavy ResNet to a lighter custom architecture designed for our specific defect types - Hardware acceleration — NVIDIA Jetson edge devices at the inspection station, running inference locally instead of sending images to a server - Region of interest — instead of analyzing the full image, we pre-process to identify likely defect regions and only run the full model on those areas - Pipeline parallelism — while one image is being classified, the next is being captured

We got inference down to under 150ms, well within the production line timing. The lesson: in manufacturing, a fast model that's 93% accurate beats a slow model that's 97% accurate. The line doesn't stop.

Factory operators aren't software engineers. If the system requires someone to SSH into a server to adjust a threshold, it won't get used.

We built a no-code workflow builder that lets operators:

- Adjust sensitivity thresholds per defect type with sliders - Define inspection zones on the production line - Set up different profiles for different products - Review flagged defects and provide feedback that improves the model - View real-time statistics — defect rates, false positive rates, throughput

The dashboard runs on a touchscreen monitor at the inspection station. An operator can switch from "running product A" to "running product B" with a tap, and all the thresholds and zones update accordingly.

This interface took significant engineering effort, but it's what made the system actually usable in a factory setting. The best model in the world is worthless if the people who need to use it can't.

After a year of operation, we had millions of inspection records. Every panel photographed, classified, and logged. This data became incredibly valuable — not just for quality control, but for production analytics.

We built reporting dashboards that show:

- Defect rate trends over time (catching production issues before they become serious) - Defect distribution by type, shift, and production line - Correlation between defect rates and production parameters - Quality metrics per customer order

The database needed optimization as it grew. We moved historical data to cold storage while keeping recent records hot. Query optimization became critical — the original queries that worked fine with 100K records were timing out at 2M. We restructured the data model and added materialized views for the most common report queries, achieving 2-3x faster performance.

After two years of building and maintaining computer vision systems in factories:

1. Hardware setup is 50% of the project — cameras, lighting, mounting, cabling. Budget accordingly. 2. The model is the easy part — the hard part is data quality, labeling consistency, and edge case handling. 3. Build for operators, not engineers — if the factory floor team can't use it independently, you've failed. 4. Drift is real — production conditions change. Models need monitoring and periodic retraining with fresh data. 5. Start with one line, one product — prove value there, then expand. Don't try to boil the ocean. 6. False positives are more annoying than false negatives — rejecting a good panel wastes money and erodes trust. Tune for precision.

Computer vision in manufacturing is rewarding work. When a system you built catches a defect that would have shipped to a customer, that's a tangible win. But getting there takes patience, iteration, and a lot of time on factory floors.