Compute, data, and algorithmic advances are the three fundamental factors that guide the progress of modern Machine Learning (ML). In this paper we study trends in the most readily quantified factor - compute. We show that before 2010 training compute grew in line with Moore's law, doubling roughly every 20 months. Since the advent of Deep Learning in the early 2010s, the scaling of training compute has accelerated, doubling approximately every 6 months. In late 2015, a new trend emerged as firms developed large-scale ML models with 10 to 100-fold larger requirements in training compute. Based on these observations we split the history of compute in ML into three eras: the Pre Deep Learning Era, the Deep Learning Era and the Large-Scale Era. Overall, our work highlights the fast-growing compute requirements for training advanced ML systems.

arxiv.org

지난 5개월 사이에 머신러닝(ML) 속도가 무려 3배나 빨라진 것으로 나타났다. 이는 '무어의 법칙'을 뛰어넘는 속도로, 인공지능(AI) 칩 업체들이 치열하게 기술 개발에 매달리고 있다는 것을 입증함과 동시에 AI 발전이 가속화되리라는 예상을 가능케 한다.벤처비트는 9일(현지시간) ML커먼스의 'ML퍼프(MLPerf) 학습 3.1' 결과를 소개하며 대형언어모델(LLM)의 학습 속도가 5개월 전보다 최대 3배 빨라졌다고 소개했다.이에 따르면 ML커먼스는 엔지니어링 컨소시엄으로, ML퍼프는 AI 모델의 학습과 추론 속도 등을 측정하는

www.aitimes.com

This clip is from the following episode: https://youtu.be/_PcL4w--uNsElon Musk is a businessman, founder, investor, and CEO. He co-founded PayPal, Neuralink,...

youtu.be