Jose Contreras

About Jose Contreras

I am a Data Labeling Analyst with hands-on experience in LLM training data annotation, AI model evaluation, and quality assurance at Meta AI. My role involves analyzing AI-generated outputs, auditing training datasets, and refining Large Language Model (LLM) accuracy to ensure high-quality performance in real-world applications.• LLM Model Evaluation & Quality Assurance: Conduct human-in-the-loop assessments of AI-generated responses, verifying accuracy, coherence, and compliance with model training objectives.• Data Labeling & Annotation Quality Control: Review and audit vendor-labeled datasets, ensuring consistent annotation standards to enhance model training efficiency.• AI Output Benchmarking & Bias Detection: Identify model inconsistencies, bias, and false positives by comparing AI outputs against benchmark datasets.• Cross-Functional Collaboration: Work closely with ML engineers, researchers, and annotation teams to refine AI quality assurance processes and improve model development cycles.• Meta AI Internal Tools & Data Validation Pipelines: Utilize proprietary AI annotation and evaluation tools to maintain data integrity and support continuous LLM improvement.I thrive in high-stakes AI environments where data accuracy, model reliability, and annotation quality are critical to AI performance. Passionate about AI governance, bias mitigation, and human-AI collaboration, I am eager to expand into AI data quality and model evaluation roles.

Experience

1. Data Labeling Analyst Magnit

   Meta

   Mar 2024 — Present · Sunnyvale, CA, US

   Labeled, analyzed, and audited AI training datasets to enhance Meta AI\'s LLM performance in text, vision, and audio recognition.•Evaluated AI-generated responses for accuracy, coherence, bias, and compliance, ensuring high-quality model outputs for real-world applications.•Conducted quality audits on vendor-labeled data, implementing corrective actions to maintain data integrity, consistency, and annotation accuracy.•Benchmarked AI model performance by comparing outputs against human-labeled datasets and analyzing discrepancies to refine training objectives.•Identified and flagged model inconsistencies, false positives, and ambiguous outputs, contributing to model retraining and refinement.•Worked closely with ML engineers, researchers, and annotation teams to improve annotation standards, workflow efficiency, and model evaluation metrics.•Utilized Meta AI’s proprietary tools for LLM evaluation, dataset validation, and bias detection, ensuring compliance with AI quality benchmarks.

Education

• Foothill College

  Information Technology Certificate

  2016 — 2017