The missing middle in LLM engineering
There is a specific kind of engineer I keep meeting.
They shipped real machine learning. Feature pipelines, model training, offline evaluation, monitoring, the whole thing running in production with real consequences when it broke. They know why your validation set leaks, what happens when a feature distribution shifts under you, and how to tell a modeling problem from a data problem at two in the morning.
Then the LLM stack arrived, moved very quickly, and they watched it go past from the outside.
Not because they could not learn it. Because when they went looking, everything they found was written for someone else — and the result is a growing population of engineers who can discuss LLMs fluently and cannot deploy one.
Two piles
The material divides cleanly into two piles.
The first pile assumes you know nothing. It explains what a neural network is before it explains attention. It defines "embedding" from scratch. For someone who has been training models for a decade, three-quarters of it is throat-clearing, and the interesting part arrives too thin and too late.
The second pile assumes you are already fluent. It is a paper, a repository README, or a thread that opens mid-argument about whether a particular attention variant is worth the memory. Genuinely useful, if you already have the scaffolding to hang it on.
The gap is not knowledge. It is the delta — what transfers from the ML you already know, and what does not.
Almost nobody writes the delta, because writing it requires assuming a reader who is expert in one thing and new to another. That is a harder audience to serve than either extreme, and a smaller one to market to.
It is also, right now, a large fraction of working machine learning engineers.
What transfers, and what breaks
A surprising amount transfers. If you understand gradient descent, embeddings as geometry, and the discipline of holding out data honestly, you have the foundation. Attention is not conceptually harder than anything in a sequence model. Fine-tuning is a decision problem before it is a technical one.
What does not transfer is subtler, and it is where people get stuck.
Evaluation breaks first. You are used to a labeled test set and a metric. Now the output is open-ended text, there is no ground truth, and the thing scoring your model is another model whose agreement with human judgment you also have to measure. Everything you know about evaluation still applies — but the machinery is different, and the failure modes are unfamiliar.
Cost behaves strangely. Inference cost in classical ML is roughly linear and roughly boring. With LLMs it is neither. Batching, caching, routing, and decoding strategy can produce an order of magnitude difference on identical output quality. Most teams discover this after the invoice.
Non-determinism becomes a deployment property. You cannot write the same integration tests. You cannot promise the same behavior twice. The operational patterns that worked for a deterministic scoring service do not survive contact with a system that answers differently each time.
None of that is conceptually difficult. It is just genuinely new, and it does not appear in the beginner material because beginners are not deploying anything.
What I wrote
I ended up writing the thing I wanted to hand people.
The Practitioner's LLM Curriculum — twelve weeks, self-paced. Modern architectures, context engineering, fine-tuning, reinforcement learning for LLMs, retrieval, agents, evaluation, inference optimization, multimodal, reliability, production patterns.
Roughly thirty interactive widgets, because some of this only clicks when you can move an input and watch the output move. The attention visualizer and the inference cost calculator are the two I would point at first.
It assumes linear algebra, training loops, and evaluation intuition. It skips what you already know.
The part I care most about
If you only read a third of it, read the last third.
Weeks eight through twelve — evaluation discipline, inference optimization, reliability and verification, production patterns — are the part I could not have written fifteen years ago, and the part most material skips entirely. The gap between a working proof of concept and a system you can operate is where nearly all of the real engineering lives, and it is where most LLM projects quietly die.
A demo proves the model can do the thing. Production proves you can do the thing repeatedly, affordably, and defensibly. Those are different claims.
That last word is doing a lot of work, and it is what I spend most of my time on: systems that have to survive an audit, an adversarial user, and a compliance review before they survive contact with customers. The curriculum touches that. It is not the whole of it.
More on that here soon.
If you are the engineer I described at the top, I would like to know whether it lands.
