Professor at the Global Scientific Information and Computing Center, Tokyo Institute of Technology
“Status Update of LLM Training in Japan”
19 February 2025
Abstract
The release of DeepSeek-V3 and R1 have shown that state-of-the-art LLMs are reproducible outside OpenAI, Anthropic, and Google. These are not mechanical tools that benefit everyone equally, but are rather intellectual tools that will influence our culture and society. Therefore, it is important for individual countries to be able to control as much of the training pipeline as possible. In this regard, the fact that open models are catching up to closed models, give us hope that training sovereign LLMs is a worthwhile endeavor. In this talk, I will give a status update of the efforts in Japan to train LLMs.
Bio
Rio Yokota is a Professor at the Global Scientific Information and Computing Center, Tokyo Institute of Technology. His research interests lie at the intersection of high performance computing, linear algebra, and machine learning. He is the developer of numerous libraries for fast multipole methods (ExaFMM), hierarchical low-rank algorithms (Hatrix), and information matrices in deep learning (ASDFGHJKL) that scale to the full system on the largest supercomputers today. He has been optimizing algorithms on GPUs since 2006, and was part of a team that received the Gordon Bell prize in 2009 using the first GPU supercomputer. Rio is a member of ACM, IEEE, and SIAM.
multicore.world 