Client profile
Vantage Autonomous, a German autonomous vehicle software company with 220 staff.
Challenge and scope
The internal team can increase throughput only twofold and must cover three annotation types across 14 object classes.
The same object may be partly obscured in a video frame or visible only in a sparse sensor scan. Inconsistency across labelers makes an otherwise usable batch harder to review. Scaling the workforce requires a reference for what counts as a valid polygon, cuboid, or mask for each class.
Delivery workflow
Label polygons, cuboids, and semantic masks in the approved tools. Sample each task type and escalate ambiguous objects for adjudication against the client’s labeling rules.
Create batches by sensor, scene type, task, and expected difficulty. Use approved examples and annotation rules to resolve routine cases. When a labeler cannot determine an object boundary or class, preserve the frame and proposed treatment and ask the assigned reviewer to adjudicate it.
Track QA results by task type and object class. Add adjudicated examples to the versioned guidelines and review recent batches affected by a rule change. Record when frames are labeled, accepted, or returned so the client can estimate usable output and time spent on rework.
Delivery team
Poland, about 26 FTE. 1 AI ops lead, 3 QA reviewers and 22 annotation specialists.
Systems and tools
Scale AI-compatible tooling, client annotation platform, CVAT.
Client decision rights
The company owns model architecture, training choices, and dataset policy.
Early milestone
At week 6, the annotation precision reaches 98.8%.
Outcomes
Company-reported results, with the baseline, measurement window and supporting record described for each measure.
| Measure | Baseline | Result | Window | Supporting record |
|---|---|---|---|---|
| Annotation precision | 97.2% | 99.91% | 6 months | QA review sample |
| Throughput | 1× | 6.8× | 6 months | Annotation platform export |
| Edge-case classification accuracy | 88% | 97.4% | 6 months | QA review sample |
Interpreting the results
The case records a 6.8× throughput multiple with higher annotation precision and improved edge-case classification. Define the unit for each comparison, such as accepted frames or accepted labels. Report precision by task and class, including the number of rare samples, so high-volume classes do not dominate the result.
Recommendations for new clients
For mobility and robotics teams, define precision before the pilot and sample each class, especially rare cases. Report the sample size and throughput unit so the two metrics can be interpreted together.
Provide a reference batch covering dense scenes, occlusion, and rare classes, along with the definition of an acceptable label. Run a reviewed batch before increasing volume. Ask the client team to adjudicate ambiguous examples and check that accepted outputs can be traced to the annotation rule version.