‘AI Inference’ has been trending everywhere, from keynote speeches to quarterly earnings reports and in the news. You have probably heard phrases like “inference-time compute”, “reasoning” and “AI deployment” and…Read More
Listen or watch 𝗘𝘆𝗲 𝗼𝗻 𝗔𝗜’s Craig S. Smith deep dive with d-Matrix’s CEO Sid Sheth on the state of AI Infrastructure. They discuss Sid’s journey building multi-billion-dollar businesses in semiconductors to founding d-Matrix — a startup…Read More
Recent breakthroughs in Reasoning AI have shifted focus towards scalable inference-time compute. Enterprises want to leverage these AI advancements to deliver real-time, interactive user experiences while also serving a vast…Read More
From train-time to inference-time compute Scaling laws are moving beyond pre-training to post-training and test-time scaling. Reasoning language models like OpenAI o1 and o3 are engineered to emulate human-like problem-solving.…Read More
In this talk, d-Matrix CTO/Cofounder Sudeep Bhoja discusses the impact that the release of the Deep Seek R1 model is having on inference compute. Stepping through the evolution of reasoning…Read More
With the arrival of Reasoning and Inference-Time compute, we are at an inflection point in the AI computing journey. Finally, revenue generation from AI models is aligning with the cost…Read More
d-Matrix engineers will be sharing their latest research at NeurIPS again this year. Among the innovations are new techniques for transforming workloads for more efficient inference. “I’d put d-Matrix R&D…Read More
Transformers-based large language models (LLMs) have emerged as the underpinning architecture for modern natural language processing. Today, the at-scale deployment of generative AI is gated by the prohibitive cost of…Read More
AI Inference is on an unsustainable trajectory The arrival of ChatGPT in 2022 caught the world’s imagination around AI – and there has been no looking back. The discussions have…Read More
Quantization plays a key role in reducing memory usage, speeding up inference, and lowering energy consumption at inference time. As large language models (LLMs) continue to grow exponentially in size —…Read More
