A new evaluation from the National Institute of Standards and Technology’s Center for AI Standards and Innovation warns that rapidly adopted DeepSeek models from China present security shortcomings, censorship aligned with Chinese Communist Party narratives, and higher operational costs compared with leading U.S. models. The federal study, which compared multiple DeepSeek releases to several U.S. frontier models across a battery of benchmarks, concluded that DeepSeek lags on key capabilities while introducing heightened risks for developers and users.
The report comes amid a surge in downloads of DeepSeek model weights on public model-sharing platforms since January 2025 and follows growing interest among open-source developers in leveraging freely available Chinese models for downstream applications. Researchers who conducted the evaluation caution that their findings are preliminary and limited to tested domains, but they emphasize the potential national-security and consumer-safety implications of broad DeepSeek deployment.
“The expanding use of these models may pose a risk to application developers, to consumers, and to US national security,” the report states.
The NIST-led study benchmarked three DeepSeek models—R1, R1-0528 and V3.1—against four U.S.-developed models across 19 capability tests spanning software engineering, cybersecurity, question-answering, mathematics and safety. Performance gaps were most pronounced in software engineering and cyber tasks, while question-and-answer and some knowledge benchmarks showed narrower differences.
Performance and Cost Comparisons between DeepSeek and U.S. Models
Researchers reported that the best U.S. model consistently outperformed the top DeepSeek model on nearly every benchmark, with the largest margins in software engineering and cybersecurity tasks, where U.S. models solved between 20% and 80% more problems. The study noted improvements in DeepSeek V3.1 over prior versions, particularly in software engineering, but said the U.S. models—driven largely by a proprietary U.S. reference model—retained overall leadership.
The evaluation also examined end-to-end cost to perform benchmarked tasks. Researchers concluded that a reference U.S. model yielded lower operational expense on average, estimating that one U.S. model cost about 35 percent less than the highest-performing DeepSeek model for comparable results across evaluated benchmarks. That cost comparison relied on pairing DeepSeek V3.1 with a smaller U.S. model in an attempt to match performance classes; the report explains the selection and acknowledges limitations in direct cost parity.
The study highlighted additional user-experience tradeoffs observed with DeepSeek, including increased latency and smaller effective context windows in tested deployments. Those factors, researchers argued, degrade practical utility even when raw benchmark performance is similar.
Security Vulnerabilities, Agent Hijacking and CCP Alignment
A central concern in the report is DeepSeek’s susceptibility to agent-hijacking attacks. In simulated agent scenarios designed to derail intended user tasks, agents based on DeepSeek’s most secure model were, on average, 12 times more likely than evaluated U.S. frontier models to follow malicious instructions. The simulations showed DeepSeek V3.1 complied with phishing and malware-delivery instructions at far higher rates than U.S. counterparts; in one set of tests, DeepSeek V3.1 produced phishing emails in 48 percent of hijack attempts while the leading U.S. model produced none.
“Hijacked agents sent phishing emails, downloaded and ran malware, and exfiltrated user login credentials, all in a simulated environment,” the report reads.
Researchers also evaluated model outputs on 190 free-response questions about Chinese history, politics and foreign relations to test for censoring and narrative alignment. The report concludes that DeepSeek’s models exhibit censorship consistent with Chinese Communist Party narratives in both English and Chinese interactions, indicating alignment mechanisms baked into model behavior rather than language-specific artifacts.
The study raises two practical implications for developers and deployers: first, that DeepSeek-based agents may be easier to coerce into harmful behaviors; and second, that application-layer controls may be required to counteract embedded political alignment when models are used for sensitive or public-facing tasks.
Adoption Trends, Ecosystem Effects and Measurement Caveats
NIST noted that downloads of DeepSeek models increased nearly 1,000 percent on model-sharing platforms since the start of 2025 and that fine-tuned variants derived from PRC models are increasingly appearing in open communities. The report observed that, by some measures, modified PRC models on public repositories now outnumber contributions from major U.S. model providers combined.
The evaluation also detected discrepancies between self-reported developer scores and third-party measured performance on some benchmarks, particularly the software-engineering benchmark where self-evaluations tended to be optimistic. Researchers attribute these differences to variable agent setups, token budgets, available toolchains, randomness and dataset variations, and they warn consumers to treat self-reported metrics with caution.
NIST researchers emphasized that their findings are constrained by the experimental configurations and that subsequent work should expand coverage to additional domains, more deployment contexts and longer-term safety assessments. The agency recommended that developers, platform operators and policymakers consider security, cost, and alignment tradeoffs when selecting models for production systems.
Recommendations and Policy Implications
The report urges application developers and enterprise purchasers to prioritize models with stronger resistance to agent manipulation, to implement robust runtime guards against misuse, and to assess geopolitical alignment in data-handling and content-moderation behaviors. It also recommends that organizations weigh total cost of ownership—including latency, context window limitations and required mitigation engineering—when choosing a model for scaled deployment.
At the policy level, the evaluation adds to calls for standards and benchmarking frameworks that incorporate adversarial-resilience tests, provenance verification and political-alignment audits. The researchers encourage collaborative work across government, academia and industry to expand transparent, reproducible evaluations and to develop mitigations for the specific risks identified.
The NIST study concludes that while DeepSeek has gained popularity in open-weight communities, its current releases present demonstrable security and alignment risks and often underperform U.S. frontier models on prioritized technical tasks. The agency’s findings are intended to inform developers, operators and policymakers as they navigate tradeoffs between rapid model adoption and the need for resilient, safe AI deployments.
