Job description
About mpathic
mpathic builds AI that understands clinical and human conversations: analyzing therapy and clinical-trial sessions, producing expert-labeled data for AI builders, and publishing open safety benchmarks.
About the role
We’re looking for a Machine Learning Engineer to build and improve ML-powered products, with a particular focus on evaluating their quality and reliability. You’ll work across the ML lifecycle, combining hands-on engineering with rigorous experimentation to understand how systems perform and where they need to improve.
This is a generalist ML engineering role with an evaluation focus. You’ll contribute to production systems while helping the team make sound decisions about models, data, and product changes.
What you’ll do
- Build, deploy, and maintain ML models and services in collaboration with engineering and product teams.
- Design evaluation approaches that reflect product needs and real-world operating conditions.
- Create and maintain evaluation datasets, benchmarks, and automated regression tests that measure safety, empathy, and risk.
- Select appropriate metrics, analyze performance across meaningful data segments, and investigate failure modes.
- Design experiments and use statistical methods to judge whether an improvement is real.
- Connect offline evaluation results with what customers see in production.
- Develop tools that make evaluation repeatable and useful throughout development and release workflows.
- Communicate findings, limitations, and recommendations clearly to clinicians, product leads, and customers.
What you’ll bring
- Practical experience developing and shipping ML systems, typically 2–5 years of relevant work or equivalent.
- Strong Python skills and familiarity with common ML frameworks.
- Solid ML fundamentals, including training, validation, generalization, and common sources of model error.
- Experience with evaluation metrics, benchmark design, and systematic error analysis.
- An understanding of statistical inference, sampling, confidence intervals, and experimental design.
- Experience identifying data leakage, evaluation bias, and gaps in test coverage.
- Good software engineering practices, including testing, version control, and code review.
- Skill at turning ambiguous product goals into measurable quality criteria.
Helpful additional experience
- A/B testing and production performance monitoring.
- Human evaluation, annotation guidelines, and inter-rater agreement.
- Speech or conversational data: transcripts, diarization and multi-speaker setups.
- Evaluating ranking, retrieval, recommendation, or generative AI systems.
- Robustness testing, calibration, fairness analysis, or distribution shift detection.
- Building evaluation tooling and integrating quality checks into CI/CD workflows.
- Clear, direct communication, both written and verbal.
- Startups and cross-functional teams.
Apply at Mpathic2 (opens in a new tab)
Source: Mpathic2 job board (BambooHR).
Report this job
Wrongly listed? Tell us why.