Case study · Artificial Intelligence & Robotics

AI Data Annotation and RLHF Case Study: 99.92% Precision at 10x Scale

Primary outcome (Actigy BPO-reported)

Actigy BPO delivered over 1.5 million high-precision light detection and ranging (LiDAR) and video annotations at a 99.92% quality assurance (QA) pass rate. With that data, the client's machine learning (ML) team trained models 3x faster. Client engineers retained model architecture and validation authority.

Results come from anonymized client engagements, are reported by Actigy BPO and are not independently audited. Results depend on scope.

Engagement boundary: the useful boundary in this annotation case is who labels and reviews data versus who owns model architecture, validation, and raw-data hosting. Published attrition figures use different definitions, so compare them in the BPO attrition rate benchmark before you rely on one.

By Actigy BPO Editorial Team Published June 14, 2026 · Last updated October 25, 2026

99.92%

Annotation Precision

10x

Throughput Scaling

Zero

Edge-Case Drift

Based on a real engagement. The client is anonymized and some operational details are generalized to protect confidentiality; the metrics reflect the engagement as delivered.

Engagement summary

The annotation team ran ground-truth data annotation and reinforcement learning from human feedback (RLHF) for an anonymized autonomous mobility and computer vision enterprise.

Delivery model: Managed operations team
Client Segment: Autonomous Mobility & Computer Vision Enterprise
Core Systems Intersected: Scale AI platform, CVAT, Labelbox, AWS S3
Scope delivered: LiDAR and video annotation, quality review and human feedback

The challenge: scaling AV annotation without losing precision

An autonomous driving company needed to correct its training data. Earlier crowdsourced labels had misplaced pixel boundaries and mislabeled unusual cases. Fixing those errors took extra work. The poor labels delayed machine learning (ML) training and software releases.

Workflow and monthly volume

The annotation team managed the full high-fidelity data enhancement pipeline from a delivery team in Ukraine.

Process volume

The team handled these monthly volumes:

3D point cloud and LiDAR labeling: 350,000+ spatial frames annotated per month.
2D video semantic segmentation: 900,000+ sequential multi-object tracking frames monthly.
RLHF prompts and responses: 80,000+ model alignment evaluations processed monthly.

Team

Dedicated delivery team: 1 data program manager, 3 data quality auditors and 25 specialized technical annotation engineers.

Workflow

The team ran three workflow streams:

Multi-sensor object tracking: Polygons, cuboids and semantic masks applied to vehicles, pedestrians and road infrastructure.
Edge-case classification: Identification and metadata tagging of rare environmental phenomena, for example occlusion, severe weather and non-standard signage.
RLHF alignment: Review of model inferences, ranking of prompt accuracy and structured reasoning logs for large language model (LLM) reward models.

Quality assurance model

The QA team used inter-annotator agreement (IAA) checks: two specialists labeled each frame on their own. If their pixel coordinates differed by more than 0.05%, a senior data auditor reviewed the frame. A script then checked the vectors in each batch before the client used it in its active learning loop.

Tools

Client stack: Labelbox, CVAT, proprietary labeling environments, AWS S3.
Proprietary workflow tools: Actigy BPO VectorCheck (custom automated geometry and label-overlap validation scripts).

What stayed with the client

The client kept control of the model design, neural network weights, validation code and training process. It also kept control of where the raw data was hosted. The annotation team operated through secure, read-only streams without persisting data locally. The team did not download, duplicate or store training datasets outside the client's dedicated enterprise AWS cloud.

Results

The figures below describe this anonymized engagement, not a forecast for another scope.

Data annotation results: client baseline and engagement outcome (Actigy BPO-reported)
Operational Metric	Legacy Client State	Engagement outcome	Performance Delta
Annotation Error Rate	4.7% Resubmission	0.08% Resubmission	98.2% Accuracy Jump
Weekly Data Output	120,000 Frames	1,250,000 Frames	1,041% Volume Scale
On-Time Dataset Delivery	78.5% Adherence	100.0% Adherence	+21.5% Predictability

Additional before-and-after key performance indicators (KPIs) (Actigy BPO-reported):

Precision index: The annotation team raised pixel-perfect label accuracy from 95.3% to a sustained 99.92%.
Engineering rework loops: The workflow cut the client ML engineers' data sanitization time by 88%.
Service-level agreement (SLA) stability: The team delivered 100% of weekly data batches within the requested sprint windows.

How the pilot worked

The pilot team proved annotation quality on a controlled pilot against the client's gold standard before it scaled labeling throughput 10x.

Timeline to pilot

4 days: The team completed taxonomy alignment, technical onboarding and tool-interface calibration, then delivered a 5,000-frame test matrix.

Timeline to scale

12 days: The team scaled delivery to full-capacity production pipelines across three engineering sprint teams.

Services used

This engagement used AI outsourcing: data labeling, annotation quality review and RLHF evaluation.

Related services and industries

Services

QA outsourcing

Industries

SaaS & Technology

All case studies How we work How much does BPO cost?

Actigy BPO describes LLM evaluation services and RLHF services separately. This case does not establish model-quality gains for those services.

Engagement outcomes (Actigy BPO-reported)

This case describes annotation, quality review, and human-feedback work against the client's guidelines. Its reported precision, throughput, and ramp are case-specific, not promises for another dataset.

Ukraine-based annotation team

The dedicated annotation team worked from Ukraine. The case describes access through the client's cloud environment and retained client model authority; those controls should be assessed separately from location.

Pilot first, scale on SLA proof

The pilot team proved annotation quality on a controlled pilot against the client's gold standard before scaling labeling throughput 10x.

You keep the decisions

The client kept model architecture, weights, validation code, and raw-data hosting. The annotation team supplied labeled data and human feedback only.

Reported precision and throughput

The case records 99.92% annotation precision, zero edge-case drift, and 10x throughput in this engagement. No like-for-like quality comparison with other annotation providers is established.

Frequently asked questions

How did the annotation team reach 99.92% annotation precision while scaling throughput 10x?

The delivery lead established a managed annotation and RLHF team with layered review, gold-set checks and edge-case escalation against the client's guidelines. The team reached 99.92% annotation precision with zero edge-case drift while it scaled labeling throughput 10x. (Actigy BPO-reported)

Where was the data-annotation team based?

The annotation team worked from Ukraine. Team skills, shift overlap, data access, and review standards must be checked for the proposed assignment; the location alone does not establish those qualities.

Did the client keep control of its models and data?

Yes. The annotation team supplied labeled data and human feedback only. The client kept full control of core machine-learning architecture, neural-network weights, validation code and raw-data hosting.

How was labeling quality drift controlled at scale?

The QA team ran continuous gold-standard sampling, consensus review and edge-case escalation against the client's guidelines. The team held precision at 99.92% with zero edge-case drift through a 10x volume ramp. (Actigy BPO-reported)

Want results like these for your operation?

Tell us the process you want to outsource. The scoping team will assess scope, complexity, staffing, and delivery cost, then propose a pilot.

Scope the same pilot for your team More case studies

Describe your workflow, volume and required coverage. You decide on a pilot after reviewing the written scope.