Reading the DeepSeek-V4-Flash-0731 Benchmarks: The Jump in Agentic Performance, and How Japanese Enterprises Should Handle MIT-Licensed Open Weights

This is Ryuta Hamamoto from TIMEWELL.

On July 31, 2026, DeepSeek published DeepSeek-V4-Flash-0731. It is the official release of V4-Flash, superseding the preview version of the V4 series that had been out since June¹. My honest first reaction to the numbers was that I did not expect agentic benchmarks to move this much. DeepSWE in particular went from 7.3 on the preview to 54.4.

In this article I work through what actually changed in the V4 series, using only the figures published on DeepSeek's official model cards. I also put the wins and the losses in the same tables. Open-weight models tend to get discussed as a binary — either they have caught the frontier or they have not — but the real numbers are far patchier than that.

From there I get to the question that matters for a business: if you were to bring this into production, what decisions would you have to make? The fact that the weights ship under the MIT License expands your options and, at the same time, hands you homework.

If you want one consistent yardstick across models: we publish a self-assessment sheet covering five dimensions — supplier diversification, data residency, protection against re-training on your inputs, output governance, and geopolitical and export-control exposure — scored across levels L0 to L3 and grounded in the NIST AI RMF and guidance from Japan's Personal Information Protection Commission. It lets you record the data-location, licensing, and output-assumption questions raised in this article in a format that does not depend on any one model name. → Download the AI Security and Vendor Dependency Risk Self-Assessment Checklist (2026 edition) (free; company name and work email required).

What happened, and how the preview relates to the release

Some sequencing first. DeepSeek-V4 was published in June as a preview. The model card's introduction states it plainly: "We present a preview version of DeepSeek-V4 series"². Then, on July 31, 2026, the Flash variant reached official release as DeepSeek-V4-Flash-0731.

The series is configured as follows².

Model	Total params	Activated params	Context	Precision
DeepSeek-V4-Flash-Base	284B	13B	1M tokens	FP8 Mixed
DeepSeek-V4-Flash	284B	13B	1M tokens	FP4 + FP8 Mixed
DeepSeek-V4-Pro-Base	1.6T	49B	1M tokens	FP8 Mixed
DeepSeek-V4-Pro	1.6T	49B	1M tokens	FP4 + FP8 Mixed

"FP4 + FP8 Mixed" means the MoE expert parameters are held at FP4 precision while most other parameters use FP8².

The distinction between total and activated parameters is worth pinning down before going further. V4 is a Mixture-of-Experts model: it holds a large number of specialist blocks internally and fires only a subset of them for each token it processes. Total parameters determine how much you have to load into memory. Activated parameters determine how much computation actually runs per token, which is what drives inference speed and cost.

So read V4-Flash's "284B total / 13B activated" as: you load the equivalent of 284B, but you compute at roughly 13B. For comparison, DeepSeek-V3.2-Base is 671B total and 37B activated². V4-Flash activates fewer parameters than the previous generation yet outperforms it on many benchmarks. That is where the efficiency story lives.

One more thing that matters operationally: reasoning comes in three modes².

Mode	Characteristics	Intended use
Non-think	Fast, intuitive responses	Routine daily tasks, low-risk decisions
Think High	Deliberate logical analysis, slower but more accurate	Complex problem-solving, planning
Think Max	Pushes reasoning to its fullest extent	Exploring the boundary of the model's reasoning capability

Most of the benchmark figures below come from the Max mode. Scores move a great deal across modes on the same model. V4-Pro on HLE, for instance, scores 7.7 in Non-think, 34.5 at High, and 37.7 at Max². That is why there is no single answer to "what does DeepSeek-V4 score."

The big move is in agentic performance

This is the headline of the July 31 release. The model card says the new build "outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count, and is broadly competitive with the strongest proprietary models available"¹.

Here are the numbers¹.

Benchmark	V4-Flash-0731	V4-Flash (Preview)	V4-Pro (Preview)	GLM-5.2	Opus-4.8
Terminal Bench 2.1	82.7	61.8	72.1	81.0	85.0
NL2Repo	54.2	39.4	38.5	48.9	69.7
Cybergym	76.7	38.7	52.7	-	83.1
DeepSWE	54.4	7.3	12.8	46.2	58.0
Toolathlon-Verified	70.3	49.7	55.9	59.9	76.2
Agents' Last Exam	25.2	15.8	16.5	23.8	25.7
AutomationBench Public	25.1	10.8	12.8	12.9	27.2
DSBench-FullStack †	68.7	37.0	41.8	61.8	71.6
DSBench-Hard †	59.6	25.8	31.1	54.5	71.7

Three caveats before anyone quotes this table.

First, the GLM-5.2 and Opus-4.8 figures are scores DeepSeek published alongside its own on its own model card. We have not independently reproduced or verified them. Evaluations of competing models shift with prompt design, agent scaffolding, and how much reasoning budget you allow. Competitor scores published by a vendor are best treated as indicative.

Second, the evaluation conditions. For the Code Agent tasks among the public benchmarks above, the model card notes that the model "is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the max reasoning effort level with temperature = 1.0, top_p = 0.95"¹. As of August 1, 2026, the harness used to produce these numbers has not been released. No third party can reproduce the run under matching conditions. Read these figures with the expectation that you will re-measure on your own workflows.

Third, the two entries marked † — DSBench-FullStack and DSBench-Hard — are DeepSeek's internal test sets¹. Nobody outside the company has seen their contents. They do not carry the same weight as the public benchmarks.

With all of that on the record, the DeepSWE move from 7.3 to 54.4 still stands out. The preview Flash was essentially unusable as a coding agent; between the June preview and the July 31 release, it reached the edge of practical territory. Terminal Bench 2.1 went 61.8 to 82.7, which puts it above GLM-5.2 at 81.0.

At the same time, read the table to the end: Opus-4.8 is ahead on all nine rows. "It has caught the frontier" is the wrong reading. "It is close enough to see the frontier's back" is closer. Several rows still show gaps above ten points — NL2Repo at 54.2 against 69.7, DSBench-Hard at 59.6 against 71.7.

One structural note: the 0731 build ships with a speculative decoding module attached¹. Speculative decoding has a small model propose several tokens ahead and the main model verify them in a batch, which speeds up generation. DeepSeek documents enabling it with a single flag on vLLM or SGLang. If you are evaluating self-hosting, it is worth a look, since it goes straight to throughput.

Against the frontier models: what leads, what trails

Next is V4-Pro-Max, the maximum reasoning mode of Pro, compared with frontier models. This table also comes from DeepSeek's model card, and the competitor scores are the ones DeepSeek chose to publish².

Benchmark	Opus-4.6 Max	GPT-5.4 xHigh	Gemini-3.1-Pro High	K2.6 Thinking	GLM-5.1 Thinking	DS-V4-Pro Max
MMLU-Pro (EM)	89.1	87.5	91.0	87.1	86.0	87.5
SimpleQA-Verified	46.2	45.3	75.6	36.9	38.1	57.9
Chinese-SimpleQA	76.4	76.8	85.9	75.9	75.0	84.4
GPQA Diamond	91.3	93.0	94.3	90.5	86.2	90.1
HLE	40.0	39.8	44.4	36.4	34.7	37.7
LiveCodeBench	88.8	-	91.7	89.6	-	93.5
Codeforces (Rating)	-	3168	3052	-	-	3206
HMMT 2026 Feb	96.2	97.7	94.7	92.7	89.4	95.2
IMOAnswerBench	75.3	91.4	81.0	86.0	83.8	89.8
Apex	34.5	54.1	60.9	24.0	11.5	38.3
Apex Shortlist	85.9	78.1	89.1	75.5	72.4	90.2
MRCR 1M (MMR)	92.9	-	76.3	-	-	83.5
CorpusQA 1M (ACC)	71.7	-	53.8	-	-	62.0
Terminal Bench 2.0	65.4	75.1	68.5	66.7	63.5	67.9
SWE Verified	80.8	-	80.6	80.2	-	80.6
SWE Pro	57.3	57.7	54.2	58.6	58.4	55.4
SWE Multilingual	77.5	-	-	76.7	73.3	76.2
BrowseComp	83.7	82.7	85.9	83.2	79.3	83.4
HLE w/ tools	53.1	52.0	51.6	54.0	50.4	48.2
GDPval-AA (Elo)	1619	1674	1314	1482	1535	1554
MCPAtlas Public	73.8	67.2	69.2	66.6	71.8	73.6
Toolathlon	47.2	54.6	48.8	50.0	40.7	51.8

Bold marks the highest value in each row. Rather than tracking only the DeepSeek column, look at where it leads and where it does not.

The wins cluster tightly around competitive programming. LiveCodeBench 93.5, Codeforces 3206, Apex Shortlist 90.2 — each the top figure among the models listed. Mathematics is strong too, with HMMT at 95.2 and IMOAnswerBench at 89.8.

The gaps are equally clear. HLE (Humanity's Last Exam) comes in at 37.7, short of Gemini-3.1-Pro High's 44.4. Apex sits at 38.3 against the same model's 60.9, a wide margin. GDPval-AA, which evaluates realistic professional work, reads 1554, below GPT-5.4 xHigh's 1674 and Opus-4.6 Max's 1619.

The result I find most instructive is long context. The V4 series markets a one-million-token window, yet MRCR 1M, which measures performance in that regime, is 83.5 against Opus-4.6 Max's 92.9, and CorpusQA 1M is 62.0 against 71.7. Being able to ingest a long context and being able to use its contents accurately are different capabilities. A million tokens fitting in the window does not guarantee that a million tokens of source material yield an accurate answer. Anyone planning to feed large volumes of internal documentation into a model should sit with that distinction.

The base-model comparison is also informative. Set against V3.2-Base, V4-Pro-Base improves sharply on knowledge: Simple-QA verified moves 28.3 to 55.2 and FACTS Parametric 27.1 to 62.6, both indicating markedly better retention of factual knowledge. MMLU-Pro goes 65.5 to 73.5, HumanEval 62.8 to 76.8, and LongBench-V2 40.2 to 51.5².

But newer is not better on every line. BBH peaks at V3.2-Base's 87.6 against V4-Pro-Base's 87.5, and on BigCodeBench the older model's 63.9 beats both V4-Flash-Base at 56.8 and V4-Pro-Base at 59.2². Generational upgrades do not lift every metric — a useful thing to remember when deciding whether to move. If you have an existing workload on the previous generation, a straight swap could degrade it. There is no substitute for comparing on your own tasks before migrating.

Where the efficiency comes from

The technical center of gravity in V4 is processing long contexts cheaply. The model card is titled "Towards Highly Efficient Million-Token Context Intelligence"².

The introduction lists three contributions².

First, a Hybrid Attention Architecture. It combines Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA). The model card states that "in the 1M-token context setting, DeepSeek-V4-Pro requires only 27% of single-token inference FLOPs and 10% of KV cache compared with DeepSeek-V3.2."

That deserves unpacking. The KV cache is the memory region holding information about every token the model has read so far. It grows linearly with context length. Handling a million-token context means keeping a million tokens' worth of KV cache resident in GPU memory, and that is the primary reason long-context work maps so directly onto GPU spend. So "10% of KV cache" translates into whether a given context fits on one GPU at all, and how many concurrent requests the same hardware can serve. The 27% figure for inference FLOPs shows up as generation speed and power draw.

Both figures are DeepSeek's own comparison against V3.2. They are not comparisons against any other vendor's model.

Second, Manifold-Constrained Hyper-Connections (mHC). The card describes it as strengthening conventional residual connections to improve the stability of signal propagation across layers while preserving expressivity². For practical purposes, read it as a technique for keeping training stable in very deep networks.

Third, the Muon optimizer, adopted for faster convergence and greater training stability².

Pre-training ran on more than 32T tokens. Post-training is a two-stage design: domain-specific experts are cultivated independently through SFT and reinforcement learning with GRPO, then consolidated into a single model via on-policy distillation². Distillation here means teaching one model to reproduce another's behavior — in this case, folding separately trained strengths in mathematics, code, and other domains back into one model.

A jump like DeepSWE's 7.3 to 54.4 almost certainly owes more to refinements in this post-training pipeline than to the architecture. Beyond the attached speculative decoding module, the 0731 model card records no structural change¹.

What the MIT License actually buys you

This section matters as much as the technical one, and arguably more for day-to-day decisions.

The license section of the model card reads: "This repository and the model weights are licensed under the MIT License," and the front matter carries license: mit²¹. Not just the repository — the weights themselves.

MIT is about as permissive as licenses get. Keep the copyright notice and the license text, and commercial use, modification, and redistribution are all permitted. Plenty of models that call themselves open-weight carry conditions such as monthly-active-user thresholds or clauses forbidding use of the outputs to train competing models. MIT carries none of that.

Three consequences follow.

First, you can run it on your own infrastructure. Prompts and outputs never reach the model provider. For organizations that cannot move data out of the country, or must stay within specific cloud regions, that is the difference between having an option and not.

Second, you can modify it. This is the thread that ties into the most important point later in this article.

Third, it is leverage against vendor lock-in. When you design a multi-LLM architecture, it adds one more genuinely swappable component. That said, running a 284B-total-parameter model in-house requires serious GPU capacity, so this is not realistic for everyone. Even so, having the option in principle changes where you stand in a negotiation.

What changes for a Japanese enterprise

What follows is my own read.

Start by separating the model's origin from where it runs. Conflating those two is why so many internal discussions go in circles.

• Download the weights and run them on your own GPUs, or on a domestic cloud you contract for, and prompts and outputs do not reach DeepSeek
• Call DeepSeek's API and, naturally, your data goes to the provider. The API model names are deepseek-v4-flash and deepseek-v4-pro, and the official documentation notes that deepseek-v4-flash has been updated to DeepSeek-V4-Flash-0731 with no change to how it is invoked³

Both are described as "using DeepSeek-V4," and their risk profiles have almost nothing in common. When this comes up internally, make everyone say which one they mean.

With that settled, I would evaluate along five axes.

1. Data location. Where inference runs, and where inputs, outputs, and logs come to rest
2. License. Commercial use, modification, redistribution. MIT sits at the permissive end
3. Support model. Self-hosting makes incident response and security patching your job
4. Regulatory fit. Industry regulation, procurement requirements, customer contract terms. In government procurement and critical-infrastructure work, model provenance itself can be a requirement
5. Assumptions embedded in the output. The next section is about this

"It is Chinese, therefore it is dangerous" does not help anyone make a decision. Breaking it down into these five axes and evaluating per use case is far more useful. On organizational design for AI governance, see Enterprise AI Governance and Organizational Design. For a general comparison of open-weight versus proprietary models, see Open Source AI vs Proprietary AI.

Handling subjects where official positions diverge

This point sits apart from performance, and it needs to be designed in from the start. It is not specific to DeepSeek. I am writing it as a structural issue common to Chinese-origin models generally, including Kimi (Moonshot AI) and GLM (Zhipu AI).

Get the mechanism right

In China, the Interim Measures for the Management of Generative Artificial Intelligence Services (生成式人工智能服务管理暂行办法) were issued on July 10, 2023 by the Cyberspace Administration of China (国家互联网信息办公室) together with seven other agencies, and took effect on August 15, 2023. Article 4(1) reads⁴:

坚持社会主义核心价值观，不得生成煽动颠覆国家政权、推翻社会主义制度，危害国家安全和利益

(Uphold the core socialist values; content must not be generated that incites subversion of state power or the overthrow of the socialist system, or that endangers national security and interests.)

The key point is that these measures govern services provided to the public within China. Downloading the weights and running them on a server in Japan is therefore not itself an activity these measures regulate. This is not a question of whether using the model is lawful.

But the converse does not hold either. "Download the weights and it becomes irrelevant" is also wrong. A model's alignment — the process of tuning its outputs toward intended behavior — is carried out inside that regulatory environment. The result is baked into the weights. What you download is the post-alignment artifact.

So the mechanism runs like this. Generative AI services offered within China are legally required to uphold the core socialist values. Model tuning happens in that environment. As a result, on subjects where positions legitimately diverge, outputs may align with a particular one. This is not a judgment about a company's character; it is a consequence of regulation. It holds independently of what any engineer at DeepSeek thinks.

What that means concretely for a Japanese business

Japan and China hold different official positions on a set of subjects: territory (the Senkaku Islands among them), historical interpretation, and the status of Taiwan. Ask a model to write about any of them and the output may not match what a user in Japan expects.

Code generation almost never surfaces this, which is why an early proof of concept will not catch it. The problem appears once you start generating Japanese prose that leaves the building.

• Corporate histories and anniversary publications
• Educational materials and training content
• Press releases, public communications, executive messaging
• Documents for municipal clients and public-sector proposals
• Translating Chinese or English source material into Japanese, and the reverse
• Shared documentation with overseas offices

All of this goes out externally. A single place name, a single line in a chronology, can carry a political position nobody intended. Careful proofreading catches it — but the more you scale generation and the thinner human review gets, the higher the odds something slips through.

We are not going to assert what we have not tested

Let me be explicit about one thing. We did not run tests on this for this article. So I am not going to write out example outputs of the form "here is exactly what DeepSeek says about the Senkaku Islands." Presenting unverified outputs as though we had confirmed them would be dishonest.

What can be stated is that a structure exists which makes such outputs possible. And precisely because of that, you should verify it yourself.

Concretely, assemble an evaluation set of twenty to fifty prompts drawn from work your organization actually does. Include the place names relevant to your business, the historical background your industry touches, the countries your counterparties operate in. Then look at what comes back. Problems that no general-purpose benchmark will ever surface show up immediately in a company-specific evaluation set.

In fairness

Every model carries bias from its training data. US-origin models carry American assumptions, and models with thin Japanese coverage misread Japanese context routinely. Saying that only Chinese models are biased would be inaccurate.

What is specific to Chinese-origin models is that the direction is written into law as a requirement. The bias is not incidental; it is institutionally mandated. That makes it more predictable — and, by the same token, easier to design around. Framing it as analysis rather than condemnation is both more accurate and more useful.

The remedy: use it with Japanese context built in

Now the license comes back into play. Because the model is MIT-licensed, fine-tuning is a legally available response. That connection is the single most important thing in this article.

The weights are published and modification is permitted. So the choice is not only "it has this property, therefore we will not use it" but also "it has this property, so we will correct for it and then use it." With a proprietary API model, that option does not exist at all.

Here is the toolkit.

1. Continued pre-training, fine-tuning, and preference alignment

Train further on Japanese primary sources, Japanese public materials, and your own approved corporate texts. Continued pre-training adds depth in Japanese and in Japanese context; instruction fine-tuning aligns the model to your house style; and preference alignment — training on pairs of preferred and non-preferred outputs — settles how the model responds on contested subjects. Full-scale training is expensive, but lightweight approaches such as LoRA make this testable at a realistic budget.

2. Ground it in primary sources with RAG

Rather than relying on the model's internal knowledge, retrieve government primary sources and your own approved documents and require answers to be based on them. It is lighter-touch than modifying parameters directly, with the side benefit that you can cite sources. We use GraphRAG in ZEROCK for exactly this reason: to keep the basis of every answer traceable.

3. Output guardrails

Detect contested subjects and route them to human review, or substitute a pre-approved standard position. Even a simple mechanism — a list of place names, historical events, and country designations that raises a flag when they appear in output — gets you a long way. Rather than chasing completeness, make sure there is at least one checkpoint on the path of anything that leaves the company.

4. An evaluation set used as a regression test

Take the company-specific evaluation set from the previous section and wire it into CI. Every time you update the model, re-run fine-tuning, or change a prompt, measure whether outputs unacceptable to your organization have started appearing. The point is not a one-time check but running it as a regression test.

With that in place, let me state the recommendation directly. If you intend to distribute a model widely within Japan, we recommend using it only after building Japanese context into it through fine-tuning or equivalent means. Using the raw weights as-is to generate outward-facing Japanese documents is a risk I would not take. That position applies to Chinese-origin models generally, not to DeepSeek specifically.

There is genuine upside to this work, too. An open-weight model tuned for Japanese and Japanese context becomes an asset in its own right. Frontier-class weights available under MIT means the starting point for that asset costs nothing.

Summary

• DeepSeek-V4-Flash-0731, published July 31, 2026, is the official release of V4-Flash and supersedes the June preview. Agentic performance moved sharply: DeepSWE 7.3 to 54.4, Terminal Bench 2.1 61.8 to 82.7
• Even so, on the official model card's table, Opus-4.8 leads all nine rows. Not "caught up" — "close enough to see the frontier's back"
• V4-Pro-Max leads the listed field on competitive programming (LiveCodeBench 93.5, Codeforces 3206, Apex Shortlist 90.2), while trailing on HLE, Apex, GDPval-AA, and the long-context measures MRCR 1M and CorpusQA 1M
• The competitor scores shown alongside were published by DeepSeek on its own model card and were not independently verified by us or any third party. Code Agent results were produced with the DeepSeek Harness, unreleased as of August 1, 2026, so no third party can reproduce them under matching conditions
• The efficiency gains center on Hybrid Attention (CSA + HCA): at the 1M-token setting, 27% of inference FLOPs and 10% of KV cache relative to V3.2
• Both the repository and the weights are MIT-licensed, permitting commercial use, modification, and self-hosting
• Model origin and deployment location are separate questions. Self-hosting and API use have entirely different data profiles
• Chinese-origin models share a structural issue in how they handle contested subjects. Article 4(1) of the Interim Measures requires upholding the core socialist values, and alignment happens inside that environment. Running the weights in Japan is not itself governed by those measures, but the results of that alignment are carried in the weights
• General benchmarks will not tell you how a model behaves on these subjects. Build an evaluation set from your own working prompts and test it
• Because the license is MIT, fine-tuning, RAG, guardrails, and regression testing are all available responses. If you plan to distribute widely within Japan, build Japanese context in first

Model selection and governance design belong together. At TIMEWELL we handle both — from evaluating models to designing how they run — through our WARP AI adoption program. If you are considering running open weights in your own environment, we are glad to talk it through.

---

References

Related reading: Kimi K3: Benchmark-Leading Performance, and Why Enterprises Should Still Be Cautious, Claude Opus 4.7 vs DeepSeek V4 Pro vs GPT-5.5: Enterprise Cost Comparison, and Qwen's Open-Weight Strategy.

Footnotes

1. DeepSeek-AI, "DeepSeek-V4-Flash-0731" model card (published July 31, 2026) ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸

2. DeepSeek-AI, "DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence" model card (DeepSeek-V4-Pro) ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³ ↩¹⁴ ↩¹⁵

3. DeepSeek API Documentation ↩

4. Cyberspace Administration of China et al., "Interim Measures for the Management of Generative Artificial Intelligence Services" (issued July 10, 2023; effective August 15, 2023) ↩