Four to One: Auditing the Leaked DeepSeek Notes

Calibration audit

Arithmetic reproduced · empirical calibration not reproduced

The printed training arithmetic can be checked. The reported 44-run error distribution cannot yet be reconstructed: the exact cohort and paired Overclock predictions are not available in the retrieved materials.

\text{days} = \frac{6 N_{\mathrm{active}} T}{C P_{\mathrm{dense}} \mathrm{MFU} g \cdot 86400}

Reproduced checks

LongCat base case

32.41 days · 1.620 million accelerator-days.

Both efficiency terms varied

22.22–97.22 days at 15–35% MFU and 40–75% goodput.

Caption’s fixed 30% MFU

25.93–48.61 days. The caption omits one varying input.

Training-time comparison

53 ÷ 99 = 0.54. The body implies about half the time; the summary says a quarter.

Token accounting

1.9T output × 17 = 32.3T total. Pro gives 8.5T at 16:1, or 13.9T at 26.8:1. These are different workloads.

Download runnable audit · Results (JSON)

What the public benchmark establishes

We fetched the historical InferenceX view as of July 31, 2026, for V4 Pro, FP4, B200/B300, single-node deployments, and SGLang, TensorRT-LLM and vLLM. The response contains 120 points across 11 series; 64 points remain in series labeled without speculative decoding. Neither is an identified 44-row cohort.

The response distinguishes total token throughput from output throughput. At this 8K/1K workload, they differ by roughly ninefold. Any comparison must match that metric, the GPU count, context, cache policy, concurrency and latency statistic. We do not substitute today’s best result for the author’s original selection.

Historical response (JSON) · Retrieval record and source hashes

Why throughput derating is insufficient

The author reports a 1.81× median throughput overestimate and a 2.8× median interactivity overestimate. If the latter applied to a configuration predicted at 20 tokens/s/user, it would deliver about 7.14. That is an illustration of the risk, not a validated correction for any row.

Because the model chooses batch size using that service target, latency must be recalibrated before batch selection. Then prefill and sustained throughput must be recomputed at the new operating point. Dividing the old aggregate capacity by 1.4–2.2 does not perform those steps.

The p10–p90 residual range is a descriptive interval across selected runs. It is neither a confidence interval for the median nor a prediction interval for an unseen Ascend deployment. A roofline approximation is also not a guaranteed upper bound when its constants and workload model can be misspecified.

Source language and translation

The article preserves Chinese passages alongside English translations and separates the author’s translation notes. This edition retains them. Literal fidelity has not been independently reviewed; preserving a translation is not evidence that it is accurate.

Further review should compare each translation with its original, retaining ambiguities, qualifications and the source’s voice. Clarifications belong in labeled translation notes.

Checks required to close calibration

1. Publish the 44 result IDs with measured and predicted output throughput, interactivity, GPU count, context, cache rate, precision, framework version and concurrency. State whether latency excludes prefill and identify its percentile.
2. Freeze the model version and constants. Identify which cases tuned the serialization floor and which were held out. Recompute p10, median and p90 with the quantile convention documented.
3. Plot residuals by chip, framework and concurrency. Report each subgroup’s sample count; correlated runs on one stack are not 44 independent deployments.
4. Fit any correction on calibration data only. Test throughput and service-target compliance on an untouched set, including the systems to which the article extrapolates.
5. Recompute mixed Flash/Pro fleet capacity, input work, cache behavior and peak-load margin before comparing with total demand.

Audit conclusion: the published evidence supports conditional scenario arithmetic and a reported bias on a limited benchmark domain. It does not yet validate Ascend fleet capacity at 20 tokens/s/user or universal four-to-one substitution.