DevOps Engineer - AI Model Evaluator

Role Overview

Mercor connects elite creative and technical talent with leading AI research labs. As a DevOps Engineer - AI Model Evaluator, you will use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.

What You Will Do

Review model-generated implementations, identify bugs, edge cases, and reliability issues, and compare outputs from multiple frontier models.

Why It Might Be a Fit

You will apply professional engineering judgment to realistic infrastructure engineering scenarios and have the opportunity to work with leading AI research labs.

Requirements

• 2+ years of professional DevOps, SRE, or Cloud Engineering experience
• Experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, or observability tooling
• Regular use of AI coding agents like Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools
• Ability to evaluate model-generated infrastructure and reliability engineering solutions

Benefits

• $400 per accepted task
• Compensation is tied to accepted work

Originally posted on Himalayas