Case study · AI data operations

Medical imaging annotation for radiology AI

Company-reported case study

Actigy BPO annotated a backlog of 240,000 imaging studies for Radiant Health AI. The case tracks annotation accuracy and reviewer agreement across six modalities, with clinical ground truth and model validation retained by the client.

Actigy BPO confirms the engagement and supporting records. Results apply to the stated scope and measurement windows.

Annotation accuracy

Result

99.6%

Baseline: 96.4%

Window: 6 months

Supporting record: QA review sample

Throughput

Result

6.2×

Baseline: 1×

Window: 6 months

Supporting record: Annotation platform export

Inter-annotator agreement

Result

95%

Baseline: 82%

Window: 6 months

Supporting record: QA review sample

Client profile

Radiant Health AI, a US radiology AI company with 60 staff.

Challenge and scope

The radiology AI team needs bounding boxes for 14 pathology classes and has limited clinical reviewer capacity.

Different imaging modalities can make the same finding look different, and rare classes have fewer examples for calibration. Annotators may be uncertain about the correct bounding region or class. The workflow needs to capture that uncertainty so an approved clinical reviewer can resolve it in the client’s environment.

Delivery workflow

Run bounding-box annotation in the approved viewer and annotation platform. Have quality reviewers check labels by modality and escalate ambiguous findings for a client clinical decision.

Assign studies by modality and case type and use the client-provided examples for drawing and labeling boxes. Track study and image references in the approved viewer. Flag ambiguous findings or visibility problems for clinical review and record the specific labeling question for that reviewer.

Sample accepted labels by modality and class and track disagreements separately. Record box and class corrections with the relevant review decision so rules can be clarified and affected batches can be checked. Give the client a list of unresolved labels and the volume of accepted studies.

Delivery team

Poland, about 12 FTE. 1 clinical annotation lead, 2 QA radiographers and 9 annotation specialists.

Systems and tools

Client annotation platform, OHIF viewer, internal QA console.

Client decision rights

The AI company owns clinical ground truth, model validation, and regulatory submissions.

Early milestone

At week 6, the annotation accuracy reaches 98.1%.

Outcomes

Company-reported results, with the baseline, measurement window and supporting record described for each measure.

Reported results: Medical imaging annotation for radiology AI
MeasureBaselineResultWindowSupporting record
Annotation accuracy96.4%99.6%6 monthsQA review sample
Throughput1×6.2×6 monthsAnnotation platform export
Inter-annotator agreement82%95%6 monthsQA review sample

Interpreting the results

The 99.6% annotation accuracy and 95% inter-annotator agreement measure different qualities. Define what makes a box and class acceptable and record clinical adjudication for disagreements. Compare accepted studies per period alongside error rates and include separate results for rare classes.

Recommendations for new clients

For imaging AI teams, establish an approved reference set with clinical reviewers. Measure annotation quality by modality and class, with a separate sample for rare findings.

Use a client-approved reference set with common and rare findings across modalities. Agree the review access, what should be marked as ambiguous, and who can resolve a clinical labeling question. Have the client review the first batch and the disagreement log before assigning the next group of studies.

Scope this workflow for your operation

Use the documented workflow, client-held decisions and operational checks to outline a pilot for your own workload.