b70-optimization-lab

DeepSeek V4 Flash K160 on 4x B70, 80.820 tok/s

This is the standalone, fail-closed launcher for the best verified result from the paused DeepSeek V4 Flash lane. It reproduces the measured source and flag identity rather than the later experimental development heads.

Result

Model: 0xSero/DeepSeek-V4-Flash-180B, revision 7c360e1cd4a5168099dbc54d16d929bf6df04990.
Hardware: four Intel Arc Pro B70 32 GB GPUs, TP4 plus expert parallelism, one active generation.
Target: unchanged experimental uniform-K160 checkpoint; FP8 block-scaled dense weights, MXFP4 experts, FP8 KV.
Draft: DSpark7 revision aa22cb07426656189b2573b8e77a9b7333b8ae0f, exact M=7 draft queries and M=8 target verification.
Strict suite medians: 80.820052, 76.900178, and 78.287226 tok/s.
Reported record high: 80.820052 tok/s; three-suite median-of-medians: 78.287226 tok/s; p10 on the high suite: 71.669556 tok/s.
Validity: 36/36 realistic requests fresh and cache-zero; 24/24 ordered exact canaries; target-verified accepted tokens.
LocalMaxxing: cmrquta9905w3lg013m5vxoqx.

This is not aggregate serving throughput. The public K160 checkpoint is an experimental hash-pruned uniform-K160 artifact with unavailable calibration; it must not be described as the official checkpoint or as reproducible true REAP ranking.

Exact source identity

Component	Public base	Record commit	Archive
vLLM	`61c87db645c256651b5a366f538898485077ad32`	`264c7f2f7df21ddeeab32ecca0353133344f1ac9`	`patches/deepseek-v4-flash-reap-xpu-b70/vllm-deepseek-v4-k160-dspark7-80tps-record-20260718.bundle`
vLLM XPU kernels	`dda91d171fbc3f51d1d65a7f8839714b1efffd42`	`31315673737d95da0f79179c8f755260ef02c1d6`	`patches/deepseek-v4-flash-reap-xpu-b70/vllm-xpu-kernels-deepseek-v4-k160-80tps-record-20260718.bundle`
oneCCL	`66499938b7a8b615e26361c52900e7aec306ce50` (2021.17.2)	`48fda4f0e074db005596d6899d5227d3f0316c12`	`patches/deepseek-v4-flash-reap-xpu-b70/oneccl-deepseek-v4-b70-wideepoch-record-20260715.bundle`

The adjacent .patch files are reviewable combined diffs from each public base to the record tree. The .bundle files preserve the exact original commits and are what the launcher identity expects.

From upstream clones that already contain the public bases, fetch the bundles:

repo=/home/steve/llm-optimizations
git -C /home/steve/src/vllm fetch \
  "$repo/patches/deepseek-v4-flash-reap-xpu-b70/vllm-deepseek-v4-k160-dspark7-80tps-record-20260718.bundle" \
  'refs/tags/deepseek-v4-k160-vllm-record-20260718:refs/heads/deepseek-v4-k160-record'
git -C /home/steve/src/vllm worktree add --detach \
  /home/steve/src/deepseek-v4-vllm-record-264c7f2f7-exact \
  264c7f2f7df21ddeeab32ecca0353133344f1ac9

git -C /home/steve/src/vllm-xpu-kernels fetch \
  "$repo/patches/deepseek-v4-flash-reap-xpu-b70/vllm-xpu-kernels-deepseek-v4-k160-80tps-record-20260718.bundle" \
  'refs/tags/deepseek-v4-k160-xpu-kernels-record-20260718:refs/heads/deepseek-v4-k160-record'
git -C /home/steve/src/vllm-xpu-kernels worktree add --detach \
  /home/steve/src/deepseek-v4-xpu-kernels-record-313156737-exact \
  31315673737d95da0f79179c8f755260ef02c1d6

git -C /home/steve/src/oneCCL fetch \
  "$repo/patches/deepseek-v4-flash-reap-xpu-b70/oneccl-deepseek-v4-b70-wideepoch-record-20260715.bundle" \
  'refs/tags/deepseek-v4-b70-oneccl-record-20260715:refs/heads/deepseek-v4-b70-record'
git -C /home/steve/src/oneCCL worktree add --detach \
  /home/steve/src/oneccl-2021.17.2-b70-sizegate \
  48fda4f0e074db005596d6899d5227d3f0316c12

Build vLLM/XPU kernels and oneCCL with the workflow in ORCHESTRATOR_HANDOFF.md. The measured oneCCL binary SHA-256 was 53de2b6d65265803d64773546c1166ceed4ae43737f0fded776f5847b4b461c9.

Launch

The local model, DSpark pack, virtual environment, and compiled oneCCL runtime remain outside Git. Their default paths are recorded by the launcher. After placing them there, run:

cd /home/steve/llm-optimizations
repro/deepseek-v4-flash-k160-b70-80tps-20260718/run.sh

Override MODEL_PATH, DSPARK_DRAFT_PACK, VLLM_TREE, KERNEL_TREE, ONECCL_SOURCE_TREE, ONECCL_LIB_DIR, or RUN_DIR when the same artifacts live elsewhere. Source commit checks remain mandatory.

Validate

After the endpoint is ready on 127.0.0.1:18080, run exact canaries and three fresh strict suites in the order canary -> suite -> canary -> suite -> canary -> suite -> canary. Commands are in the test workflow. Every request must report cached_tokens=0. Compare the generated identity.txt field-by-field with the tracked identity and queue metadata linked from the result packet.

Evidence

Raw run evidence remains outside Git at /mnt/fast-ai/bench-results/deepseek-v4-flash-xpu/dspark7-sharded-target-argmax-candidate-20260718T2100Z.

This site is open source. Improve this page.