Building a 113M-Parameter Earthquake LLM from Scratch on Two GPUs

I trained a 113M-parameter earthquake LLM from absolute scratch

I trained a 113M-parameter GPT model for earthquake science entirely from scratch using two NVIDIA A30 GPUs. This project documents the full lifecycle, from crawling open-access papers and Wikipedia to cleaning data, building a custom tokenizer, and training the model. By mixing domain-specific seismology text with general web data, I achieved a fluent model capable of understanding complex scientific concepts while remaining accessible for plain-language tasks.

"Base pretraining alone never yields chat; that's SFT."