ECE Distinguished Speaker Series: Kaushik Roy, “From Dot Products to Softmax: Computing AI Workloads Where the Data Lives”

The energy and performance limits of modern AI systems are increasingly dominated by data movement across the memory hierarchy. Compute-in-Memory (CIM) has emerged as a compelling approach to mitigate this bottleneck by embedding computation within memory arrays. This talk presents a cross-layer perspective of CIM spanning CMOS and emerging non-volatile memory technologies for scalable AI acceleration.

One of the central challenges in analog CIM is the overhead of interface circuitry, particularly analog-to-digital converters (ADCs). We present ADC-less CIM architectures that substantially reduce energy and area overheads while preserving computational fidelity. Beyond matrix-vector multiplication (MVM), we demonstrate in-memory realization of nonlinear functions, including softmax, enabling key primitives of large language model (LLM) inference within the memory fabric.

We further discuss the role of CIM across multiple levels of the memory hierarchy, highlighting system-level opportunities and trade-offs. Measurement results from multiple fabricated test chips are presented, providing silicon evidence of the proposed concepts. These results outline a path toward practical, end-to-end in-memory AI systems that alleviates the memory wall problem.

Bio:

Kaushik Roy is the Edward G. Tiedemann, Jr., Distinguished Professor of Electrical and Computer Engineering at Purdue University. He received his BTech from Indian Institute of Technology, Kharagpur, PhD from University of Illinois at Urbana-Champaign in 1990 and joined the Semiconductor Process and Design Center of Texas Instruments, Dallas, where he worked for three years on FPGA architecture development and low-power circuit design. His current research focuses on cognitive algorithms, circuits and architecture for energy-efficient neuromorphic computing/ machine learning, and neuro-mimetic devices. Kaushik has supervised 118 PhD dissertations and his students are well placed in universities and industry. He is the co-author of two books on Low Power CMOS VLSI Design (John Wiley & McGraw Hill).

Dr. Roy received the National Science Foundation Career Development Award in 1995, IBM faculty partnership award, ATT/Lucent Foundation award, 2005 SRC Technical Excellence Award, SRC Inventors Award, Purdue College of Engineering Research Excellence Award, Outstanding Mentor Award in 2021, Humboldt Research Award in 2010, 2010 I