Overview

Key Responsibilities

• We are hiring ML inference and serving experts to turn real، paid tasks from their jobs into expert-level Terminal-Bench challenges

• THIS IS A TASK BASED COMPENSATION MODEL. PLEASE CHECK THE APPLICATION QUESTIONS FOR MORE DETAILS.*

• • Design expert-level Terminal-Bench tasks that turn real، paid ML inference work from your own experience into challenges an AI agent must solve end to end.

• • Package tasks with Docker and Git so they are reproducible، and document the systems، tools، data and rules involved so reviewers can validate difficulty.

• • Iterate on tasks based on reviewer feedback and agent test runs، adjusting difficulty and fixing ambiguities so each task stays expert-level، fair and objectively gradable.

• • Availability to start by early November 2026 and complete tasks of about 60 to 75 minutes each

• *ASTONISHED** and tag RMTM1LjE4MS4xODMuMTcy when applying to show you read the job post completely (#RMTM1LjE4MS4xODMuMTcy)

Requirements & Qualifications

• Skills/Experience

We are building a benchmark that tests how well AI agents handle real expert work in a terminal

Each task includes the environment، a working solution، and an automated check that proves the result is correct.

• Define the files، systems and environments each task runs on، such as model serving with streaming، so the challenge is realistic، reproducible and command-line solvable.

• Write a working reference solution for every task and verify it runs end to end، sized for about 60 to 75 minutes of expert effort.

• Build automated checks that objectively prove a result is correct، including cases where ML outputs go wrong silently، such as numerical drift or metric bugs.

• 3+ years doing ML inference or serving work hands-on (not only studying، selling or managing it)، or a PhD plus active research in the subfield

• Did this work in the last 3 years and can name the systems، tools، data and rules used

• Trained، served or evaluated models at real scale (multi-GPU or production serving)، not only notebooks or API calls

• Hands-on with PyTorch or JAX internals and serving stacks such as vLLM، SGLang or TensorRT-LLM

• Can explain how ML results go wrong without errors (numerical drift، metric bugs) and how to catch them

• Technical fluency in Python and the command line، and with files، CSV/JSON، APIs or databases

Nice-to-have

• AI lab or LLM infrastructure experience

• Contributions to PyTorch، vLLM or SGLang

• GPU profiling experience

• Docker and Git proficiency

• Experience writing automated tests

Process

• Task vetting instead of an interview: complete a vetting task so the client can assess your task design

• Paid per task، each task takes about 60 to 75 minutes

• Work is delivered task by task، with reviewer feedback on submissions

• Expected start in early November 2026

Please mention the word

This is a beta feature to avoid spam applicants

Companies can search these words to find applicants that read this and see they're human.