Each time you ask ChatGPT a query, that request triggers a knowledge relay race. Data comes out of reminiscence, passes by means of the CPU for preprocessing, goes to the GPU for heavy calculations, after which comes again. Your complete course of is repeated for each phrase the AI generates.
The bottleneck is structural, which means every request is routed by means of among the business’s costliest and power-hungry chips. This inefficiency is precisely what XCENA, a startup with workplaces in South Korea and the US, is making an attempt to resolve. The four-year-old startup has designed a chip that brings computing capabilities a lot nearer to DRAM, a high-speed, short-term reminiscence chip that shops knowledge {that a} processor is actively utilizing. This enables routine knowledge operations to be processed near reminiscence with out pricey spherical journeys between CPU, GPU, and reminiscence.
If it really works at scale, the price implications of AI infrastructure could be vital, which largely explains the passion of buyers throughout the nation. Actually, XCENA raised $135 million in Collection B at a valuation of $570 million, bringing its whole funding to $185 million.
XCENA CEO Jin Kim co-founded the startup in 2022 with CTO Dohoon Kim and CPO Harry Joo-hyun Kim, veterans of Samsung and SK Hynix, the reminiscence giants that provide the chips that energy NVIDIA’s GPUs. “CPUs and GPUs have each gotten smarter over the a long time, however reminiscence has by no means gotten smarter. XCENA desires to alter that,” Kim mentioned in an interview with newsweblatest. “The current rise in reminiscence costs and associated shares indicators a widespread shift in AI infrastructure towards memory-centric architectures,” he added. (This month, the three corporations that dominate the worldwide reminiscence chip market, Samsung, SK Hynix, and Micron, every surpassed $1 trillion in valuation for the primary time.)
XCENA is betting its enterprise on the speculation that “inference isn’t just a computing downside, however more and more a reminiscence scaling downside,” Kim mentioned.
XCENA’s chip, MX1, connects to the CPU by means of a Compute Specific Hyperlink (CXL), primarily a devoted categorical lane between the processor and reminiscence, and processes knowledge earlier than it leaves the reminiscence module. This brings compute to knowledge, not the opposite manner round. The corporate claims that what beforehand required 10 servers may now be finished on only one.
“Whereas GPUs are nice at matrix multiplication (the complicated calculations behind AI mannequin coaching), a lot of the encompassing knowledge orchestration nonetheless happens on the CPU, comparable to preprocessing, KV cache administration (a system that saves earlier dialog context so the mannequin does not must reprocess it), and knowledge caching. Our chip handles these duties instantly inside the reminiscence module itself,” Kim mentioned.
Demand for reminiscence options has been surging because the second half of final 12 months, and the corporate believes the timing is working in its favor.
Talks are within the early phases with a number of international reminiscence distributors, which Kim declined to call. The corporate’s preferrred clients are hyperscalers that spend tens of billions of {dollars} yearly on AI infrastructure, and even small enhancements in reminiscence effectivity can result in tons of of tens of millions of {dollars} in financial savings.
MX1 remains to be a prototype. Mass-produced chips are anticipated to be shipped from Samsung’s foundry strains by the top of 2026, and the corporate expects to generate income beginning in 2027.
As neural processing unit (NPU) producers race to tackle Nvidia for coaching workloads, XCENA targets the memory-intensive layers beneath.
XCENA’s closest rivals embrace Astera Labs and Marvell, each Nasdaq-listed corporations engaged on next-generation reminiscence connectivity. Marvell is already a big, established firm working in the identical house, Kim mentioned, including that the differentiating issue in the end comes right down to mental property. “Now we have hundreds of cores,” Kim mentioned. Primarily based on public specs, Marvell’s method depends on a relatively small variety of general-purpose cores.
These cores are constructed on RISC-V, an open supply chip design blueprint, and are particularly optimized for knowledge processing., Every core is deliberately stored small and environment friendly. Past the core itself, XCENA designs its personal inside reminiscence hierarchy, interconnect bus, and DRAM controller. That is the extent of vertical integration that the majority chip corporations, together with their bigger rivals, usually outsource.
Seoul-based enterprise capital corporations Altinum and IMM Funding co-led the Collection B spherical together with Corstone Asia and present buyers SBI Funding and Mirae Asset Capital. The corporate has greater than 90 employees in workplaces in Pangyo, a know-how hub on the outskirts of Seoul, and Sunnyvale, and is in talks with abroad buyers to boost further funding.
In the event you purchase by means of hyperlinks in our articles, we could earn a small fee. This doesn’t have an effect on editorial independence.
