DeepX has taped out its second-generation AIartificial intelligenceAI stands for artificial intelligence accelerator, the DX-M2, and is pairing the design with custom processor-in-memory technology from Samsung. The South Korean startup is targeting edge AIartificial intelligenceAI stands for artificial intelligence use cases such as robotics, drones and other devices with tight thermal and power limits. The chip is intended to support generative AIartificial intelligenceAI stands for artificial intelligence inference on models with up to 100 billion parameters while staying below 5 W.
The DX-M2 is positioned above the company’s first-generation DX-M1, which is already in mass production for vision applications. DeepX says the new chip can run models in the 20 billion to 100 billion parameter range, corresponding to roughly 3 billion to 20 billion active parameters, at 10 to 30 tokens per second depending on the model. The power budget remains 5 W, while the accelerator reaches 80 TOPS1. Tera Operations Per SecondA metric used to describe the peak operation rate of an AI accelerator.2. trillion operations per secondTOPS indicates how many trillion operations per second a chip can perform..
Memory bandwidth is the main architectural change between the two generations. DeepX says it has increased from 50 GB/s to 150 GB/s. The company also cites design improvements that doubled NPUNeural Processing UnitA processor optimized for neural-network inference workloads. power efficiency, followed by another doubling from the move to a 2 nm process node.
The DX-M2 is built to handle large mixture-of-experts LLM1. local linear models2. Large Language ModelThe model class referenced in the long-context serving demo.s and supports speculative decoding and TurboQuant. DeepX is also testing the accelerator with Samsung’s LPDDR5x-PIM1. Processing-In-MemoryAn approach that performs some computation close to or inside memory.2. processor-in-memoryPIM describes computing directly inside memory to reduce data movement. samples. The company has spent about 18 months on the integration effort and is validating operation and performance in-house.
PIM1. Processing-In-MemoryAn approach that performs some computation close to or inside memory.2. processor-in-memoryPIM describes computing directly inside memory to reduce data movement. is meant to reduce the cost of moving weights between memory and compute, especially for GEMVgeneral matrix-vector multiplicationGEMV denotes multiplication of a matrix by a vector and is often memory-bound in inference workloads. workloads that require random memory access. DeepX’s simulations show that offloading GEMVgeneral matrix-vector multiplicationGEMV denotes multiplication of a matrix by a vector and is often memory-bound in inference workloads. operations from the DX-M2 into LPDDR5x-PIM1. Processing-In-MemoryAn approach that performs some computation close to or inside memory.2. processor-in-memoryPIM describes computing directly inside memory to reduce data movement. would increase tokens per second by 6.7× for Llama3.1-8B compared with the DX-M2 alone. The trade-off is higher power consumption: the 6.7× performance gain comes with 2.7× higher total power use.
DeepX says the same architecture could still deliver about 2× higher performance within the power envelope of non-PIM1. Processing-In-MemoryAn approach that performs some computation close to or inside memory.2. processor-in-memoryPIM describes computing directly inside memory to reduce data movement. operation. The company is working on compiler support for this mode and already has a prototype compiler running on simulations of the PIM1. Processing-In-MemoryAn approach that performs some computation close to or inside memory.2. processor-in-memoryPIM describes computing directly inside memory to reduce data movement.-enabled LPDDR5x setup. Hardware modules are now being built so the compiler can be tested in a physical environment.
The DX-M2 samples are scheduled for March 2027, with mass production targeted for fall 2027. DeepX’s planned third-generation DX-M3 is set for 2028 and will use a chiplet-based design with LPDDR6-PIM1. Processing-In-MemoryAn approach that performs some computation close to or inside memory.2. processor-in-memoryPIM describes computing directly inside memory to reduce data movement., aiming at 1 ePOPS (INT88-bit integerINT8 stands for 8-bit integers and is often used for efficient AI inference.).