senior-computer-vision

---
name: senior-computer-vision
description: Use when building object detection or segmentation pipelines, training custom vision models, optimizing inference, or deploying computer vision systems to production.
category: Engineering
version: 1.0.0
tools: []
---

# Senior Computer Vision Engineer

Production computer vision methodology for object detection, image segmentation, and visual AI deployment.

## Core expertise

- **Object detection**: YOLO family (v5–v11), Faster R-CNN, DETR, RT-DETR
- **Instance segmentation**: Mask R-CNN, YOLACT, SOLOv2
- **Semantic segmentation**: DeepLabV3+, SegFormer, SAM (Segment Anything)
- **Classification**: ResNet, EfficientNet, Vision Transformers (ViT, DeiT)
- **Video**: object tracking (ByteTrack, SORT), action recognition
- **3D vision**: depth estimation, point clouds, NeRF
- **Deployment runtimes**: ONNX, TensorRT, OpenVINO, CoreML

Tech stack: PyTorch/torchvision/timm for training; Ultralytics, Detectron2, MMDetection for detection; segment-anything/mmsegmentation for segmentation; OpenCV/Pillow/albumentations for image processing; CVAT/Label Studio/Roboflow for annotation; MLflow/Weights & Biases for tracking; Triton Inference Server/TorchServe for serving.

## Object detection pipeline

1. **Define requirements**: target classes, real-time need (target FPS), speed-vs-accuracy priority, deployment target (cloud GPU / edge / mobile), dataset size.
2. **Select architecture**: real-time (>30 FPS) → YOLOv8/v11 or RT-DETR (single-stage); high accuracy → Faster R-CNN or DINO (two-stage); small objects → YOLO+SAHI or Faster R-CNN+FPN; edge → YOLOv8n or MobileNetV3-SSD; transformer/no-NMS → DETR, DINO, RT-DETR.
3. **Prepare the dataset** in COCO format with an 80/10/10 train/val/test split as the default.
4. **Train and validate**, then evaluate against: mAP@50 > 0.7, mAP@50:95 > 0.5 (COCO primary metric), precision > 0.8, recall > 0.8, inference time < 33ms for 30 FPS real time.

## Model optimization and deployment

1. Benchmark the baseline (PyTorch FP32) across batch sizes to get a latency/throughput/memory reference point.
2. Pick the optimization path by deployment target: NVIDIA cloud GPU → ONNX → TensorRT FP16; NVIDIA edge → TensorRT INT8; Intel CPU → ONNX → OpenVINO; Apple Silicon → CoreML; generic CPU → ONNX Runtime; mobile → TFLite or ONNX Mobile.
3. Export to ONNX with dynamic batch size and simplification, then validate the graph.
4. Quantization trade-off: FP32 = 100% size / 1x speed / 0% accuracy drop; FP16 = 50% size / 1.5–2x speed / <0.5% accuracy drop; INT8 = 25% size / 2–4x speed / 1–3% accuracy drop (requires a calibration dataset, ~500 samples is a reasonable default).
5. Typical TensorRT FP16 speedup over PyTorch FP32 baseline is roughly 3.5x, with accuracy change around -0.3% mAP — use this as a sanity check on your own numbers, not a guarantee.

## Dataset preparation

1. Audit raw data for corrupted files and near-duplicates before anything else.
2. Clean: remove corrupted images and duplicates.
3. Convert annotation format as needed: Pascal VOC XML, YOLO TXT, COCO JSON, LabelMe JSON, and CVAT XML all convert to/from COCO JSON.
4. Apply augmentations appropriate to detection: geometric (horizontal flip p=0.5, rotate ±15° p=0.3, scale ±20% p=0.5), color (brightness/contrast p=0.5, hue/saturation p=0.3, blur p=0.1), and advanced (mosaic p=0.5, mixup p=0.1, cutout p=0.3).
5. Split ratios by dataset size: under 1,000 images → 70/15/15; 1,000–10,000 → 80/10/10; over 10,000 → 90/5/5. Stratify the split by class.

## Architecture selection guide

Detection speed/accuracy trade-offs: YOLOv8n (1.2ms, 37.3 mAP, edge/mobile/real-time) through YOLOv8x (10.1ms, 53.9 mAP, maximum accuracy); RT-DETR-L (5.3ms, 53.0 mAP, transformer/no-NMS); Faster R-CNN R50 (46ms, 40.2 mAP, two-stage high quality); DINO-4scale (85ms, 49.0 mAP, SOTA transformer).

Segmentation: YOLOv8-seg (4.5ms, real-time instance), Mask R-CNN (67ms, high-quality masks), SAM (50ms, zero-shot promptable), DeepLabV3+ (25ms, scene parsing), SegFormer (15ms, efficient semantic).

CNN vs. ViT: CNNs need less training data (1K–10K images) and train/infer faster; ViTs (DETR, DINO) need more data (10K–100K+ images), train slower, but capture global context better and handle small objects worse without multi-scale tricks.

## Performance targets

Real-time: FPS > 30, mAP@50 > 0.6, latency p99 < 50ms, GPU memory < 4GB, model < 50MB. High accuracy: FPS > 10, mAP@50 > 0.8, latency p99 < 150ms, memory < 8GB, model < 200MB. Edge: FPS > 15, mAP@50 > 0.5, latency p99 < 100ms, memory < 2GB, model < 20MB.

## Dataset audit checklist

Before training, report: total image count, size range, format mix (JPEG/PNG), corrupted-file count, near-duplicate pairs, annotation format detected, total annotation count, per-class distribution, and count of images with no annotations at all — an empty-image rate that's unexpectedly high usually signals an annotation pipeline bug, not a genuinely hard dataset.

## Production monitoring

Once deployed, track inference latency (p50/p95/p99), throughput, GPU memory, and — where ground truth becomes available with a lag (e.g., human-reviewed labels) — accuracy drift against the validation distribution. A sudden mAP drop after a deploy usually means either a silent preprocessing mismatch (image size, normalization) between training and serving, or a genuine distribution shift in the incoming data; check preprocessing parity first since it's the more common and cheaper failure to rule out.