James Wallace
Ai Tutor & Qa Lead (Law) Product Data Operations (Genai Leadership Group) - Cw on Assignment Meta @Meta
Signup · Get unlimited contacts
WORK HISTORY
Ai Tutor & Qa Lead (Law) Product Data Operations (Genai Leadership Group) - Cw on Assignment Meta @Meta
Led a diverse team of 30+ industry-leading legal professionals and academics on Quality Assurance projects as part of Meta\'s Knowledge Experts Program Evaluated and audited vendor-produced (Surge/Scale/etc.) data for SFT (Supervised Fine Tuning) and RLHF (Reinforcement Learning from Human Feedback) annotations using the Halo and Gala platforms along with SQL. SME and “last line of defence” for the QA process in both the Law and History subject fields for Meta\'s GenAI programs. Identified areas of improvement and provided feedback to optimize the quality of annotations. Guideline Development: Authored high-quality annotation guidelines for Law and History to train KEP legal team members in developing LLM capacity with all elements of legal reasoning and logic (IRAC, deductive, inductive, analogical, and abductive reasoning) Regularly pivoted the legal team\'s focus to address improvements in competitor LLM reasoning capacity and outputs. Reasoning Skills Assessment and Guideline/Rubric Development: Constructed comprehensive assessments to test the subject matter knowledge of prospective vendor (Mercor) employees in all subfields and specializations within Law. Set legal reasoning assessment criteria used to benchmark and track model progress in acquiring reasoning proficiency across various knowledge areas. Trend Analysis: Engaged with LLM (Large Language Model) models and documented problematic trends observed across various examples. Collaborated with the team to address these trends and improve the models\' performance. Domain Expertise: Offered guidance and mentorship within my domain to bridge any gaps in understanding and to support the development of more accurate AI models. Data Annotation: Directly annotated data for key tasks when necessary, including writing SFT responses and judging output quality for RLHF.
EDUCATION
The University of Edinburgh
Master of Science (MS), Economic and Legal History
McGill University
Doctor of Philosophy (PhD), Socio-Economic & Legal History + History of the Book
University of Toronto - University of Trinity College
Bachelor's degree, History
SKILLS
ABOUT JAMES WALLACE
Since 2016, I have built my reputation as a strategic insights and CX transformation leader and professional on the values of integrity, fairness, and collaboration. If I could describe myself in two words, they would be \"resourceful curiosity\". My greatest strength flows from this: I can bridge any knowledge or skills gap at lightning speed and remain aware of what\'s coming next. I\'m often asked why I chose to leave the field of history for data science and how studying the past could at all prepare me for a career that is essentially about predicting the future. Firstly, the most important thing that historians and data scientists have in common is the ability to piece together compelling stories from data. The biggest difference? That historians often have to deal with data that is not yet digitized and is frequently incomplete. Secondly, I have often found that historians make the best futurists - solid in our understanding of the cultural forces at play we can fix our forward with confidence.As a CX strategist and leader I have helped companies like Canada Post to launch their transformation journey. Assessing the existing framework of performance indicators, I identify the major pain points that are holding up a company\'s ability to become more customer-focused, engage in journey mapping, and help them identify, measure, and formulate an action plan to move them toward faster growth (increased revenue and higher profits). As a team lead, I am innovative and effective at leveraging human+technological resources. I frequently engage in cross-functional collaboration and relationship-building both up and across organizations. Using knowledge of the latest industry trends I ask really tough questions to spark internal creativity and generate momentum toward strategic goals and growth targets.I believe strongly in my team and will always go the extra mile to make sure they feel valued, empowered and recognized for contributions towards our shared goals. To ensure fairness and equanimity, I like to make sure that staff can document their wins, to advance their careers and develop a sense of accomplishment and ownership within the organization.English (native speaker), French – bilingual (r+w ), oral (conversational)
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.