A coding agent can pass for the wrong reasons.
This report separates the two.

Algorithms, data structures, bug-fixes, and API/systems implementation, graded against hidden test suites the agent never sees.

Structural and numerical solvers (FD/FE, contact dynamics) in C++, checked against multi-binary hidden references.

Provision, diagnose, and guardrail cloud infrastructure (AWS), verified against the deployed state.

RTL / digital-logic design checked under hardened simulation and formal-equivalence harnesses.

Quantitative reasoning across math, physics, and the natural sciences with deterministic, checkable answers.

Interactive simulation and game-logic tasks graded on exact state transitions and rule fidelity.

Real-world bug-fix tasks distilled from open-source issues (esbuild, klauspost/compress, rust-lang/semver), graded against each project's own tests.

Train and tune models to a target metric on real engineering datasets, graded on held-out performance against a solved-reward threshold.

Long-horizon, from-scratch numpy ML research: implement reverse-mode autodiff and a training method (quantization-aware, adversarial-robust, semi-supervised, worst-group), then train to a sealed held-out metric. Tasks published blinded.

Physics-informed and surrogate modeling: PDE forecasting and CFD/FEA prediction, graded on quantitative predictive accuracy.

Machine-learning-for-engineering tasks (DeepCAD command-history canonicalization and equivalence), graded on a continuous partial-credit score against verifier-owned labels rather than all-or-nothing pass/fail.

Modeling and statistical inference on real datasets (federated learning, event studies), graded against held-out ground truth.

Bias-correction and outlier-robust analysis in R: recover trustworthy estimates from messy data, checked against reference results.

Applied product analytics: causal impact and decision analysis on real product datasets (R).

Population PK/PD modeling with nonlinear mixed-effects: fit drug-exposure models and recover the correct parameters.