Validity Gates
Use this file to decide whether a Qwen3.6 35B result is a record, a smoke, a
reference, or an invalid artifact.
Benchmark Identity Must Match
Do not compare Qwen3.6 35B XPU runs unless the full identity matches. At minimum
record:
- model path and revision;
- quantization;
- TP/PP/concurrency;
- prompt/output shape and repeats;
COMPILATION_CONFIG;GPU_MEMORY_UTILIZATION;XPU_GRAPH, VLLM_XPU_ENABLE_XPU_GRAPH;VLLM_XPU_FORCE_GRAPH_WITH_COMM, VLLM_XPU_GRAPH_NOOP_COMM_CAPTURE;VLLM_XPU_GDN_NATIVE_FALLBACK;- sampler/top-k fallback flags;
- async scheduling /
VLLM_EXTRA_ARGS;
- diagnostic flags;
- launcher and server log path.
The most common false story in this lane was comparing graph-none or missing
PIECEWISE runs to forced-comm PIECEWISE graph runs. Missing
COMPILATION_CONFIG='{"cudagraph_mode":"PIECEWISE"}' means it is not the same
benchmark.
Record-Grade Result
A record-grade internal claim needs:
- metrics run for the target shape, usually p512/o512 for this lane;
- JSON canary at high repeat count, with zero mismatches;
- color canary at high repeat count, with zero mismatches;
- quality suite pass when making a promoted claim;
- baseline hash / repeat stability / long-context checks when submitting or
promoting;
- no synthetic acceptance, oracle shortcuts, or canary-skipped correctness
assumptions.
The current strict-valid 4x base used JSON 128/128, color 256/256, and
quality-suite pass. Treat that as the comparison standard.
Smoke Result
A smoke is useful for direction but not a record:
- low repeat counts such as
16/16 or 32/32;
- quality suite skipped;
- one metric repeat;
- exploratory env flags not yet locked.
Smokes can justify a follow-up full gate. They cannot justify a LocalMaxxing
submission or a “best valid” claim.
Invalid Or Ceiling Result
Label a result invalid or ceiling-only if any of these apply:
- JSON or color canary failed;
- canaries were skipped;
- output was synthetic, oracle-accepted, or generated by a non-serving path;
- engine crashed before metrics completed;
- result depends on hidden local state not captured in the run identity;
- it is a raw throughput probe with no endpoint-quality validation.
The 181.91 tok/s EAGLE2 synthetic run and 198.95 tok/s ngram artifact are
valuable ceilings, not valid serving results.
Failure Signatures To Recognize
blue, green, red, yellow: full-accept double-processing or rollback failure.- repeated non-ASCII token garbage on the first decode token: captured-prefill
or graph state corruption.
- malformed JSON on first repeat: speculative verifier state mismatch, not a
sampling issue.
- engine-core cancellation after scheduled spec tokens: k>1 path is not proven;
do not infer throughput.
LocalMaxxing Submission Rule
Only submit after:
- the benchmark identity is fully captured;
- the result is a real improvement over the matching GPU count/mode;
- canaries and quality checks match the claim;
- the API key is read only from the documented secret location.
Credential and helper guidance is in ../../docs/localmaxxing.md.