Annotation QA Lead (Quality Lead)
QA Lead owns annotation quality: define gold standards, run audits, measure IAA, lead remediation and training for annotator panels. Critical to maintain high dataset reliability.
Responsibilities
Core responsibilities
- Design QA processes, gold datasets and adjudication workflows.
- Monitor IAA, gold check pass rates and annotator drift.
- Coach annotators and run calibration sessions.
- Author/update annotation guidelines; work with clients on edge cases.
Qualifications
Skills & experience
- Must: strong analytical skills, prior QC/annotation experience.
- Preferable: familiarity with inter-rater stats and dataset QA tooling.
- Nice-to-have: domain knowledge for specialized datasets.
Recommended Jobs
In Home Caregiver
Job Description Job Description APPLY NOW & BE PART OF OUR GROWING TEAM! As a Caregiver you will: Work one-on-one with clients to build and foster rewarding relationships Service clients…
Server
Job Description Job Description KURA SUSHI – Pioneers of the revolving sushi concept! Interview for our location in Schaumburg Come join the Kura Krew, a team that is passionate about food, …
Payer Policy & Reimbursement Optimization (AD/Director level)
The Role Our client is seeking an innovative and highly motivated leader for Payer Policy and Reimbursement Optimization to help with payer policy, health economics, reimbursement and payer adoptio…
Property Manager - Chicago Heights, IL
At Hispanic Housing our vision is to provide affordable housing that people are proud to call home. In creating quality housing, a safe and stable environment, and by providing superior customer servi…
Digital Technology Business Partner - Supply Chain OTC Lead
Position Title: Digital Technology Business Partner - Supply Chain OTC Lead Position Type: Regular - Full-Time Position Location: Oakbrook Terrace Requisition ID: 35235 …
Occupational Therapist part-time - Pediatric CLINIC
About Lumiere Come join one of the fastest growing, truly transformative pediatric therapy groups in Chicago. Lumiere Children's Therapy is searching for extraordinary professionals to work in our in…
Property & Casualty Underwriter Graduate
Location: Schaumburg, IL, US Launch your career with impact! We’re looking for curious, motivated graduates with strong people skills who want to grow, learn, and make a difference. If you hold a …
Manager, Market Analytics
Job Description Job Description Description: About Us Fullmark Energy is a leading energy storage IPP that develops, owns and operates standalone storage projects in the U.S. The platform is …
Mechanic
Job Description Job Description Job Summary We are looking for a skilled Mechanic to join our busy repair shop. As an Auto Mechanic, you will assist our customers by diagnosing and repairing…
Orthopedic Physician Assistant/Nurse Practitioner
Job Description: Orthopedic Physician Assistant/ Nurse Practitioner - Genesis Orthopedics & Sports Medicine At Genesis Orthopedics & Sports Medicine, we place high value on our Advanced Practice Pro…