Date registered: 2026-07-24 America/Toronto
Status at registration: design and gates frozen before implementation, native build, XPU execution, model load, endpoint generation, network access, payload creation, or submission. Two independent read-only source audits selected and reviewed this lane. The main agent then traced the current modular-MoE dataflow and corrected the launch accounting against the actual record stack.
Can one exact M=8 Laguna target-verifier kernel replace only:
incumbent fixed-slot-order MoeGather
-> existing laguna_m8_scale_add
while preserving every protected arithmetic boundary and leaving the route-parallel W2 kernel unchanged?
The current record already fuses routed scale with shared add. Therefore the control is two launches per layer, not three:
accum_fp32 = fixed slot-0..9 FP32 weighted sum of BF16 W2 route rows
routed_bf16 = BF16(accum_fp32)
scaled_bf16 = BF16(float(routed_bf16) * 2.5)
final_bf16 = BF16(float(shared_bf16) + float(scaled_bf16))
The candidate performs that literal sequence in one kernel. It must contain
explicit BF16 routed_bf16 and scaled_bf16 values. It must not fold 2.5
into router weights, scale the FP32 accumulator, add shared output before the
routed BF16 boundary, reassociate the ten-slot sum, use a fused final FP32
expression, or move the later fixed-rank reduction.
The default-off selector is:
VLLM_XPU_LAGUNA_M8_GATHER_FINALIZE=1
The treatment is materially different from the rejected full expert transaction. That experiment serialized W2 expert slots and reduced W2 parallelism from 3,840 to 384 workgroups per card. This experiment retains the incumbent route-parallel W2 call, geometry, route interleave, remote-route zeroing semantics, INT4 dequantization, DPAS order, and BF16 route-row stores. Only the already separate post-W2 gather/finalize tail changes.
The retained profile attributes 0.441856 ms per 47-layer target cycle to
MoeGather. The current exact laguna_m8_scale_add component measured about
0.224-0.234 ms/cycle, for a present two-kernel scope of roughly
0.666-0.676 ms/cycle.
That total is an upper bound on the scope, not a predicted saving: the candidate retains all gather work and adds a shared load plus two BF16 rounding operations to the gather kernel. The certain structural change is exactly:
94 control launches/cycle -> 47 candidate launches/cycle
No endpoint gain is claimed by this preregistration.
A read-only native audit caught an unsafe first interface design before any
build or device action. It accepted a device route-map tensor, while the
binding could validate its shape but not reject values below -1 or above
79 without another kernel or a synchronization. The incumbent dereferences
such values and therefore cannot make that public input fail closed.
The exact record path always creates
torch.arange(80).view(8,10) and zero-fills remote W2 route rows. The frozen
candidate contract was consequently narrowed before source freeze: it accepts
no route map and uses only canonical token * 10 + slot rows. Component
evidence must still compare against the incumbent with its canonical map.
This correction changes no observed result; no native build, XPU process, or
model action had occurred.
The following evidence mechanics were fixed after the CPU-only implementation and native build but before any XPU import, allocation, primitive, profiler, or packet. They do not change the treatment, correctness requirement, timing thresholds, counter thresholds, card policy, or endpoint boundary.
First, torch.equal(x, x) is false when x contains a NaN. Requiring it for
the separately classified NaN corpus would therefore make the registered gate
impossible even for a tensor compared with itself. Raw BF16 bits remain the
primary exactness predicate for every case. Finite, signed-zero, subnormal,
and infinity cases additionally require torch.equal. NaN cases instead
require identical raw bits and identical isnan, isinf, sign-bit, and
payload classification; torch.equal is recorded as inapplicable rather
than treated as a quality tolerance. This preserves bitwise exactness.
Second, timing and hardware counters use separate immutable, non-overlapping
packets so profiler instrumentation cannot contaminate the timing result.
Phase A contains the complete pre/post exactness corpus and the registered
20-warmup, 31-block A-B-B-A, 64-cycle timing protocol. Even if every card
passes, its strongest status is
component_timing_pass_pending_mandatory_counters; it cannot authorize an
endpoint. Phase B is then a one-shot mandatory counter packet for the same
source and binary identities, enforcing the registered 94/47 launch counts,
spill, traffic, occupancy/XVE-active, and XVE-stall thresholds. There is no
retry, rescue, source change, or performance-conditioned packet replacement
between phases. A failure in either phase is terminal, and only a pass from
both packets constitutes the registered four-card component pass.
Implementation starts from clean, tracked sources:
503f7784cf9d1704109b1e4650427fb4f417d604;c59aaadbbfd350c2b5f4ad663e247c2811ae3181; and8936aac144929190c1e53f8b8624ca397ce16f5b,
XPU kernels b6076ce1249ffee0e30bee528f4cd15c3bffb234,
LocalMaxxing cmrx6p5dv001bo4017hb7sixz, and
33.89498511171744 tok/s.The two later source commits contain only default-off native shared-GEMM and W1-N128 experiments. Their selectors must remain off in this lane. A future component packet must freeze the post-implementation commits, installed native-library hashes and paths, compiler/runtime/driver identity, physical card mapping, boot ID, scripts, fixtures, and expected native schema before the first XPU process.
The selector may dispatch only when all of the following are true:
[80,3072], shared output [8,3072], and final
output [8,3072], plus contiguous FP32 top-k weights [8,10];row * 10 + slot order. The candidate accepts no route-map tensor and
hardcodes this layout; remote experts remain the incumbent zero-filled W2
rows in their original slots;2.5, there is no routed-output transform, the
routed output is not already reduced, sequence parallelism and DBO are off,
and final all-reduce is not skipped;M=1, verifier tails M=2..7, prefill, draft, non-Laguna models, LoRA, graph paths, and unrelated shapes retain the literal incumbent path. If the selector is enabled and a candidate-shaped M=8 call violates any predicate, it must raise before launching a primitive; it must never silently fall back.
The existing shared output is currently owned by SharedExperts, while the
route rows are owned inside XpuFusedMoe. Implementation must use an explicit
candidate-only API and an explicit output_includes_shared result. Only a
successful native combined dispatch may consume the shared output, suppress
the later laguna_m8_scale_add, and pass the combined BF16 tensor to the
unchanged final reduction. Mutable or stale implicit state, tensor-shape
inference, and a dummy-output convention are forbidden.
This note authorizes source implementation and CPU-only validation. It does not authorize an XPU import/allocation, native primitive execution, profiler, model load, endpoint, or generation.
The minimal source plan is:
TOPK=10, BF16, M=8 gather-finalize implementation beside
the incumbent MoeGather in csrc/moe/moe_gather.cpp, with its own
_moe_C binding and hard input/alias checks.MoeGather<BF16,10,8> workgroup geometry and exact
slot loop. Resolve the route row as token * 10 + slot with no external
map, then add only the explicit routed BF16 cast, scaled BF16 cast, shared
load, final BF16 add, and final store.XpuFusedMoe and returns both
the final tensor and a positive combined-output signal. The generic path
and all non-XPU expert implementations remain unchanged.MoERunner, suppress the ordinary shared/routed combine only after
receiving that positive signal, then call the unchanged final fixed-rank
all-reduce.Before any device packet may be written, require:
Device tests must be explicitly opt-in and deselected from every CPU test command. An accidental XPU primitive before the later packet invalidates Stage 0 evidence.
Only a separate tracked packet committed after implementation freeze may authorize one component campaign. Run each physical B70 independently with one visible Level Zero device. One failed card stops the lane.
For every card, compare:
A: incumbent moe_gather -> incumbent laguna_m8_scale_add
B: candidate laguna_m8_gather_finalize
against identical preallocated inputs. Require raw uint16 and
torch.equal equality for:
The correctness corpus must include all finite BF16 values at the routed and
shared boundaries, signed zero, subnormals, infinities and NaNs classified
separately, FP32 zero/subnormal/near-one routing weights, exact midpoint cases,
all-local/all-remote/mixed zero-filled route-row patterns, every one of ten
canonical slot positions, at least 256 changing random full fixtures,
candidate-repeat determinism, unchanged input hashes, and a complete
post-timing replay. Production integration must additionally prove the
incumbent control map is exactly arange(80).view(8,10), the W2 kernel
identity, arguments, 13-call test count, and 3,840 workgroups per call/card
are unchanged.
Time only the two-op control and one-op candidate. Exclude W2, fixture construction, allocation, hashing, reset, reductions, synchronization inside an arm, and CPU work. Reuse buffers and rotate a prebuilt changing fixture outside timed arms:
Every card must independently satisfy:
0.15 ms per 47-layer cycle;94/47;No global average may rescue a failed card, pair family, exactness check, or guardrail. A component or profiler-schema failure is terminal for this source candidate; preserve it as a measured negative.
No endpoint is authorized here. A full four-card component pass authorizes only construction and audit of a separate cold endpoint preregistration. That future protocol must compare against the exact approved record stack with new cold services, the fixed 13-prompt suite, canonical q=1 teacher, returned token IDs, cache-zero proof, long-next and rollover checks, fresh private NVMe roots, strict device idle, and a sequential early-stop A-B-B-A design.
This lane never authorizes a fifth rescue run, prompt reuse, prefix/history cache, result selection, quality tolerance, parser repair after capture, or a LocalMaxxing claim without a conservative matching-identity improvement.
Until separately authorized, do not: