---
# What DeepSeek gets right and where it falls short
**URL:** https://www.sigmoid.com/blogs/what-deepseek-gets-right-and-where-it-falls-short/
Date: 2024-09-03
Author: Joha Momin
Post Type: post
Summary: At Sigmoid, we regularly test new AI models to understand where they genuinely add value for the data and AI work we...Read More...
Categories: AI/ML
Tags: AI, AI Agents, AI/ML, Generative AI, LLMs
Featured Image: https://www.sigmoid.com/wp-content/uploads/2026/09/DeepSeek-blog-banner.jpg
---
At Sigmoid, we regularly test new AI models to understand where they genuinely add value for the data and AI work we deliver. Headline benchmarks can only give us generic information, so we judge each model on how it performs against the kind of real tasks our teams handle day to day.
This time, that meant a close look at DeepSeek V4-Flash. The most simple way to describe our first impression is that it behaves like a capable junior engineer who works exceptionally quickly at almost no cost, producing a great deal of output as long as someone reviews it.
That trade-off runs through the rest of this assessment. What follows here is a hands-on assessment of the new open-weight flash mode.
## Three things stood out immediately
Before turning to methodology and limitations, here are some of the major takeaways. The following stood out most clearly.
- Exceptional speed: It sustains roughly 123 tokens per second on first-party serving ([Artificial Analysis](https://artificialanalysis.ai/models/deepseek-v4-flash)), against a typical ceiling of about 40 tokens per second, where 60 is considered strong. Because it is open-weight and hosted across providers such as SiliconFlow and DeepInfra, it can be scaled to thousands of tokens per second, or self-hosted with vLLM or SGLang where the hardware is available ([Deepseek V4 Flash model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731)).
- Very low cost: At $0.28 per million output tokens ($0.14 per million input, falling to $0.0028 per million on cached hits), it is roughly 18x cheaper than the most affordable commercial models. Just to compare, even with Claude’s prompt caching, DeepSeek's 1/50th cache-hit discount makes it significantly cheaper. While it matches top models on many other benchmarks.
- A meaningful advance in model serving: Deepseek V4 ships with an embedded framework called DSpark ([DSpark Paper arXiv:2607.05147](https://arxiv.org/abs/2607.05147)). The underlying technique, speculative decoding, is not new; it dates to 2022 and 2023. DSpark, however, is a highly efficient implementation. Smaller draft modules propose candidate tokens quickly, and the larger model verifies them in parallel rather than generating sequentially ([DeepSeek-V4 paper](https://arxiv.org/abs/2606.19348)). Verification is parallel and inexpensive, while generation is expensive and serial. The one constraint is that draft models must share the larger model's token format.
## How we evaluated the model
Standard benchmarks rarely map cleanly to production work. They provide useful directional signals, but hands-on testing is what informs our decisions.
We built several specification-driven codebases and compared the results, including output, code quality, and test coverage, against established benchmark models. We also ran a set of real Excel-manipulation tasks to assess quality.
Fig 1. DeepSeek's published benchmarks (public): V4-Flash-0731 across agentic-coding and tool-use suites, alongside other commercial and open models.
For external context, DeepSeek has published its own comparison across coding-agent and tool-use suites. It is a public benchmark and should be read as a directional indicator rather than a definitive result.
## Where it falls short
No model is without limitations, and a candid account is more useful than a list of strengths. The following areas caused difficulty in our testing.
- Test coverage was weaker than that of the leading frontier models.
- It made errors during initial codebase generation and required more guidance to reach comparable output.
- It needed more iterations, introduced mistakes along the way, and required somewhat more human oversight.
- It was less inclined to seek input on key architectural and module-design decisions.
## Cost comparison
To make the difference concrete, we ran the same excel-manipulation workload through several models on OpenRouter and tracked the total spend. The gap was substantial.
Fig 2. Cost comparison between DeepSeek V4-Flash, Claude Sonnet 5, and GLM 5.2
On this workload, V4-Flash came in roughly 5x cheaper than GLM 5.2 and about 11x cheaper than Claude Sonnet 5, for output we considered comparable in quality.
## What does it mean for engineering teams
The figures are one consideration; their effect on day-to-day decisions is another. The following outlines how we would incorporate V4-Flash into a working stack.
- As of now, it’s truly an excellent daily use model. Users will have a hard time burning through $10 for virtually unlimited interactive use, a full autonomous app costs around $0.13 total.
- It can be integrated into agentic coding tools for example Claude Code via DeepSeek’s Anthropic-compatible endpoint ([api.deepseek.com/anthropic](https://api.deepseek.com/anthropic)) ([DeepSeek Claude Code Integration](https://api-docs.deepseek.com/quick_start/agent_integrations/claude_code/)).
- Rely on auto mode use in Claude for heavy tasks as DeepSeek needs more tries to get things right. It is risky if you are managing large clusters or running on production-grade machines where single-turn accuracy matters.
- These limitations are not specific to the flash model; they were also present in DeepSeek-V4-Pro. A flash model's ceiling is the performance of its larger counterpart, as DSpark only accelerates execution. The speculative decoding is mathematically lossless, underlying reasoning issues from the larger model still surface.
- Watch out for verbosity, V4-Flash can be ~3.4x more verbose in output generation, so monitor your output token budget ([Artificial Analysis](https://artificialanalysis.ai/models/deepseek-v4-flash)).
- Expect costs to continue declining. The peak output cost of smaller models is likely to converge toward the input cost of large frontier models.
## Where it is available today
Access is the principal constraint at present. The model is readily available through some channels but not yet through the enterprise platforms many teams rely on.
- It is available today through DeepSeek's API, aggregators such as SiliconFlow and DeepInfra, and open-source frameworks including vLLM and SGLang, where these are suitable for your requirements ([Deepseek Anthropic API guide](https://api-docs.deepseek.com/guides/anthropic_api/)).
- It is not yet offered by the major enterprise providers. AWS Bedrock and Azure AI Foundry currently host the older DeepSeek-R1 models, with V4 expected in the coming months.
- Enterprise adoption will take time, so prepare your evaluations now in order to test the model on your own applications as soon as it becomes available on your platform.
- The speculative-decoding approach is likely to be adopted by most providers before long, so comparable speed improvements can be expected across other leading models within one to two quarters.
## Conclusion
DeepSeek V4-Flash fast, remarkably cheap, and refreshingly upfront about where it falls short of frontier models. It is a strong day-to-day option today and a clear indication of where inference economics are heading. Teams would be well advised to prepare their evaluations now.
## About the Author
**Mayur Rustagi, CTO & Co-founder, Sigmoid**, is a technology leader focused on turning AI and data engineering innovation into measurable business value. An IIT Kharagpur alumnus and open-source pioneer, he has shaped Sigmoid’s technology vision and helped build solutions that turn complex data challenges into measurable business outcomes. Recognized among **CDO Magazine’s 40 Under 40**, Mayur has also led technology initiatives recognized by **Analytics India Magazine** and **ISG**, including Sigmoid’s Outstanding Data Engineering Team award and Rising Star recognition for Data Modernization Services.
[lc_the_tags]
## Suggested readings
[View all](/blogs/)
#### How to trust Agentic BI when it acts without being asked
[Read blog](/blogs/how-to-trust-agentic-bi-when-it-acts-without-being-asked/)
#### Demystifying the evolution of specialized GPT models: InstructGPT and ChatGPT
[Read blog](/blogs/demystifying-the-evolution-of-specialized-gpt-models-instructgpt-and-chatgpt/)
#### Building a smarter approach to evaluate the best AI Models for real business impact
[Read blog](/blogs/building-a-smarter-approach-to-evaluate-the-best-ai-models-for-real-business-impact/)
## Featured blogs
[lc_get_posts post_type="post"
posts_per_page="4" orderby="date" output_view="lc_get_posts_mycustom_view" output_number_of_columns="4"
output_wrapper_class="row" output_article_class="shadow" output_hide_elements="Excerpt"
output_excerpt_length="0" output_excerpt_text="Read More" output_heading_tag="span"
output_featured_image_format="thumbnail" output_featured_image_class="card-img-left" ]
## Share
[addtoany]
## Subscribe to get latest insights
## Talk to our experts
Get the best ROI with Sigmoid’s services in data engineering and AI
---
## Categories
- AI/ML
---
## Navigation
- [Company](/about-sigmoid)
- [Newsroom](/newsroom)
- [Life at Sigmoid](/careers)
- [Takshashila](/takshashila)
- [Contact Us](/contact-us)
- [AI Strategy Blueprint your AI advantage](/enterprise-ai-strategy/)
- [Generative AI Drive innovation with Generative AI](/generative-ai/)
- [Responsible AI Build trust with ethical AI practices](/responsible-ai-in-enterprise/)
- [Agentic AI Reshape business with scalable agentic systems](/agentic-ai-solutions/)
- [AI Managed Services Ensure reliable AI performance](/ai-managed-services/)
- [Advanced Analytics Transform your business with data-driven insights](/advanced-data-analytics-solutions/)
- [Download whitepaper](/ebooks-whitepapers/the-new-qsr-playbook-transforming-menu-strategy-into-a-growth-engine/)
- [Data Strategy Strong data foundations for scalable AI](/data-analytics-strategy/)
- [Data Management Leverage data as a strategic asset](/ai-data-management-services/)
- [Data Ops Automate data for speed and quality](/data-devops/)
- [Data Engineering Deliver insights faster with scalable pipelines](/data-engineering/)
- [Cloud Transformation Modernize data to maximise efficiency](/cloud-migration/)
- [Download whitepaper](/ebooks-whitepapers/building-data-products-in-a-data-mesh-to-drive-business-value/)
- [Data Modeling Structure data for better decisions](/data-modeling-services/)
- [Data Visualization Transform data into actionable stories](/data-visualization-service/)
- [BI Migration Enhance decision making with modern BI tools](/bi-migration/)
- [Data Observability Build trust with healthy, accurate data](/data-observability/)
- [Automated Insights Make smarter decisions with auto-generated insights](/automated-insights/)
- [Download whitepaper](/ebooks-whitepapers/how-cognitive-bi-drives-an-intelligent-enterprise/)
- [CPG & Retail End-to-end analytics for planning, operations, and commercial excellence](/industries/cpg-analytics/)
- [Life Sciences Trusted intelligence across clinical, commercial, and operational workflows](/industries/life-sciences/)
- [Financial Services AI-powered analytics for risk, compliance and customer experience](/industries/banking-financial-analytics-services/)
- [Download infographic](/wp-content/uploads/2026/03/Infographic_Maturity-Framework-for-Scaling-Agentic-AI.pdf)
- [Helios Turn marketing data into measurable business outcomes](/marketing-suite-helios/)
- [Stratos AI-powered platform for resilient supply chains](/supply-chain-stratos/)
- [MediaIQ Advanced platform for in-flight marketing measurement](/accelerators/sigmoid-mediaiq-multi-touch-attribution-tool/)
- [CampaignIQ AI-driven platform for optimized campaign budget allocation](/accelerators/sigmoid-campaigniq/)
- [AssistBot GenAI email assistant that automates human-like responses](/accelerators/sigmoid-assistbot-for-ai-email-assistant/)
- [CreativeBot GenAI tool for personalized and brand-aligned creative design](/accelerators/sigmoid-creativebot/)
- [SocialBot GenAI platform to analyze digital conversations and trends](/accelerators/#marketing|socialbot)
- [DemandIQ Predict trends accurately and optimize inventory management](/accelerators/sigmoid-demandiq/)
- [NetworkIQ Track and optimize logistics operations in real-time to quickly address disruptions](/accelerators/sigmoid-networkiq/)
- [SupplyIQ End-to-end platform to optimize supply chain operations](/accelerators/sigmoid-supplyiq/)
- [ProcurementIQ Automated procurement operations for maximum savings, compliance and efficiency](/accelerators/sigmoid-procurementiq/)
- [RapidML Accelerated deployment for machine learning models](/accelerators/sigmoid-rapidml/)
- [DataGuard Comprehensive platform for proactive data quality management](/accelerators/data-quality-tool-sigmoid-dataguard/)
- [CloudPulse Cloud cost optimization platform with multi-cloud management](/accelerators/sigmoid-cloudpulse/)
- [RAPID GenAI foundation with built-in governance and cost clarity](/accelerators/sigmoid-rapid/)
- [AnalyticsBot GenAI based platform to streamline decision-making in analytics](/accelerators/sigmoid-analyticsbot/)
- [DataConnect Seamlessly ingest, integrate and harmonize data from diverse sources](/accelerators/sigmoid-dataconnect/)
- [Reconica AI-powered data harmonization and reconciliation engine](/accelerators/sigmoid-reconica/)
- [ConverseBot GenAI driven insights generation for automated insights from reports](/accelerators/#sales|conversebot)
- [iNRM Cross-lever revenue growth optimization platform](/accelerators/sigmoid-inrm/)
- [AssortmentIQ Optimize shelf layouts and assortment mix at scale with AI-based insights](/accelerators/sigmoid-assortmentiq/)
- [Read whitepaper](/ebooks-whitepapers/building-agentic-ai-chatbots-for-business-process-transformation/)
- [Listen Podcast](/events/podcast/how-jack-in-the-box-is-redefining-personalization-and-supply-chain-with-ai/)
- [Blogs](/blogs/)
- [White Papers](/ebooks-whitepapers/)
- [Case Studies](/case-studies/)
- [Podcast](/events/podcast/#Podcasts)
- [Download case study](/case-studies/ai-based-dynamic-targeting-and-personalized-hcp-engagement-for-pharma-marketing/)
- [ConverseBot](/accelerators/#sales|conversebot/)
## Tags
- AI
- AI Agents
- AI/ML
- Generative AI
- LLMs
---
## Footer Links
- [Talk to our AI experts](/contact-us/)
- [AI Strategy](/enterprise-ai-strategy/)
- [Agentic AI](/agentic-ai-solutions/)
- [Generative AI](/generative-ai/)
- [AI Managed Services](/ai-managed-services/)
- [Responsible AI](/responsible-ai-in-enterprise/)
- [Advanced Analytics](/advanced-data-analytics-solutions/)
- [Data Strategy](/data-analytics-strategy//)
- [Data Engineering](/data-engineering/)
- [Data Management](/ai-data-management-services/)
- [Cloud Transformation](/cloud-transformation/)
- [Data Ops](/data-devops/)
- [Data Visualization](/data-visualization-service/)
- [Automated Insights](/automated-insights/)
- [BI Migration](/bi-migration/)
- [Data Modeling](/data-modeling-services/)
- [Data Observability](/data-observability/)
- [CPG & Retail](/industries/cpg-analytics/)
- [Financial Services](/industries/banking-financial-analytics-services/)
- [Life Sciences](/industries/life-sciences/)
- [Case Studies](/case-studies/)
- [Thought Leadership](/ebooks-whitepapers/)
- [Blogs](/blogs/)
- [Company](/about-sigmoid/)
- [Newsroom](/newsroom/)
- [Accelerators](/accelerators/)
- [Careers](/careers/)
- [Privacy Policy |](/privacy-policy/)
- [Cookie Policy](/cookie-policy/)