How much did Google's 530B parameter model PaLM's training cost? Something around $9M to $23M.…
The first section discusses the term compute and introduces one conceptual model of a processor — a helpful analogy for analyzing trends in compute. Computation has two obvious roles in AI systems: training and inference. The following section discusses those and presents compute trends.…
Modern progress in AI systems has been driven and enabled mainly by acquiring more computational resources. AI systems rely on computation-intensive training runs — they require massive amounts of compute.…
My monthly review is a bit more abstract and asking the question: Are we heading in the right direction?…
For getting things done, I like the mental model of the pilot, the airplane, and the engineer. The pilot sets the course: our goals and our direction. He/she decides the plan for the day based on internal desires and external realities. I’m usually a pilot at the start…
This work addresses the challenges of bringing Machine Learning to microcontroller units (MCUs), where we focus on the ubiquitous ARM Cortex-M architecture.…
I've recently upgraded to an M1 Mac because Black. Magic. Fuckery. Nonetheless, I required some command line applications which are not yet available for ARM architectures. First, make sure you have Rosetta 2 installed: usr/sbin/softwareupdate --install-rosetta --agree-to-license. Individual command line tools can be emulated by adding arch -x86_…
Uncommented list of all the packages, configs and applications which I call essential for a new macOS setup.…
