Framework-free AI engineer notebooks: learn RAG, agents, and evals on a free API
AI Engineer Notebooks – free, framework-free RAG/agents/evals on Colab
A GitHub repo offers a series of Colab notebooks that teach the applied-LLM stack without frameworks, using only raw APIs and the free Groq API. It covers model APIs, structured output, tool calling, RAG, evals, agents (from scratch), fine-tuning vs LoRA, security, LLMOps, and customer craft. The notebooks emphasize evals as the spine and include three real case studies, plus a capstone project for resumes. Designed for backend/full-stack engineers moving into AI Engineer or Forward Deployed Engineer roles.
Patterns are durable; wrappers churn.
- ghm2199
One thing that evals are super important from the get go are where the harness+model inference is part of the product, e.g. if you are doing voice ai, building out a test harness to test the system is a non trivial first step.
- consumer451
Man, I could have used something like this last year.
Upon review, it appears that fumbling my way through the dark with Claude and Google led to the same place, in nearly all cases.
However, this is all written by Claude — it has too many em-dashes to not be, does it not? So, maybe that's why we ended up in the same places.
Does anyone know of any other resources in this vein?
- KolibriFly
Glad to hear they are prioritizing evaluation right from the start. Usually people just throw together a rag pipeline on the knee and then judge the metrics by eye, skimming three responses in the terminal