Headquarters: San Francisco, CA
URL: http://bit.ly/3kLhMdk
How to Apply
Read below to learn more about the role and what we’re looking for. In the application, Tell us about your creative background, the types of companies you’ve worked with / your style, and your experience with AI image tools.
The Opportunity
We are looking for creative professionals to help benchmark the next generation of AI image generation models. Current industry benchmarks focus on surface-level metrics like single-turn output quality and prompt adherence. But that misses what matters most to real users: did the model actually help me get the image I wanted?
In practice, five rounds of refinement that land on the right image can be more valuable than a single beautiful image that misses the mark. This project is about defining a new standard for how AI image models are evaluated, one that reflects real creative workflows.
What You Will Do
- Work through a set of creative scenarios using AI image generation models, iteratively refining outputs across multiple turns (e.g., transforming a portrait into a stylized illustration, refining a product hero shot, designing a fantasy creature, compositing a product into a scene)
- Compare outputs across multiple models side by side, selecting preferences and providing confidence ratings
- Provide clear, structured qualitative feedback on why one output is stronger, what works, and what doesn’t
- Give specific, concrete creative direction to models through iterative editing (e.g., “Make the lighting warmer,” “Add a brick texture to the wall,” “Change her shirt to blue”)
- Help define what “helpfulness” means in a multi-turn creative context, beyond just aesthetic quality
What We’re Looking For
- Experience in creative fields such as illustration, brand design, graphic design, graphic design, concept art, or visual effects
- A strong eye for image quality, composition, lighting, color, and style
- Comfort working with AI image generation tools and iterating through multiple rounds of edits
- Ability to articulate creative feedback clearly, with specific and concrete direction rather than vague or conceptual guidance
- Experience in creative AI workflows is a strong plus
About Contra Labs
Contra Labs sits at the intersection of human creativity and AI. We partner with leading AI labs and product companies to improve their creative tools across image generation, design, video, and beyond. Our research is grounded in how top creatives actually work, studying the patterns behind what gets created, what gets chosen, and what resonates. We believe human input will always be at the center of net new creation, and our mission is to help shape technology so human creativity can be amplified to create what today is unimaginable.
Go to posting –> https://weworkremotely.com/remote-jobs/contra-ai-image-model-evaluator
Leave a Reply