AI Research Engineer

At Dscout , we’re building the most flexible and powerful UX research platform on the market—trusted by the world’s top brands in finance (JP Morgan Chase, Intuit, Charles Schwab, PayPal), healthcare (Aya, Headspace), consumer goods (Keen, Verizon, Target, Northface), and tech (Google, Amazon, Facebook, Meta, Spotify, AirBnB). Our tools help teams deeply understand the humans behind their products, so they can build better ones. We are expanding our smart and driven team and would love for you to join us. At Dscout, we're building the human-grounded layer that teaches AI to work in the real world and proves it stays aligned there. We are expanding our smart and driven team and would love for you to join us. Dscout AI Training & Evals builds human data and evals that show whether AI works for real people. For ten years Dscout has captured how people behave in real context, with consent and scientific rigor. Dscout AI Training & Evals points that engine at AI: consented human behavior and judgment, turned into the training data, evals, and benchmarks AI teams trust. Models ace benchmarks and still surprise people in the wild. A benchmark says how capable a model is. It does not say whether the model works for the person using it, across weeks, in the mess of real life. That gap is where we work. We serve AI researchers, the product builders who ship alongside them, and the research teams who partner with both. We are a new unit inside an established company: live customers, a mature platform (React, Elixir, Python, Swift, Kotlin; SOC 2 and HITRUST certified), a small senior team, and room to build. We work AI-native. Agents write code, evals gate changes, and we aim to do in months what used to take years. A note from our Chief AI Officer & GM. The hardest AI problems show up when a model meets real people, with their tools, their interruptions, and their goals. Our customers need a partner who can design the answer, build it, and ship it fast. Ground Truth sits where deployment meets research: what we learn in the field shapes the science, and the science shapes what we ship. Come build it with us. Why this seat Work on a problem the field has not solved: measuring whether AI works for real people, over real time. Study something most evals do not have: real participants, doing real tasks in their own context. Own the benchmark. You are founding it, not joining a team that has one. Publish. Methods and benchmarks go to the research community, with first authorship on the benchmark. Join a small senior team with direct access to the Head of AI Research and the GM. This is a research-first role focused on questions where the right rubric, benchmark, or method does not yet exist. You will find the gaps, propose bold directions, and take projects from the first hypothesis through experiments and benchmark construction to publication. You will also build: the science has to run inside live customer programs. What you will do Rubrics and QA science Design rubrics and test whether they measure what they claim: validity, reliability, and sensitivity. Measure how reliable human raters are, and when disagreement is noise versus real signal. Calibrate AI judges against real human judgment, and study when they drift away from it. Build quality signals that put evidence behind every data point we ship. Long-horizon, real-user benchmarks Design Dscout’s first benchmark: real users, in real context, over long horizons. Multi-week tasks across tools, not one-shot prompts. Define the protocol, task design, scoring, and release. Work with our delivery team on fieldwork. Work out how to keep a living benchmark honest over time: drift, contamination, difficulty, and what counts as a meaningful change. Write the paper and own the public release. Human judgment as training and eval signal Study when real-world human feedback improves a model and when it adds noise. Explore how pr