Fine-Tuning is Brain Surgery: Why Context Wins

Fine-tuning updates model weights. It's effective for teaching form: JSON output, coding style, instruction-following. It's terrible for injecting facts. The "domain-specific fine-tuned model" pitch is mostly marketing dressed up as differentiation.

The Lobotomized Child Problem

Research comparing fine-tuning to Retrieval Augmented Generation (RAG) consistently shows RAG wins for knowledge tasks. Ovadia, Brief et al. demonstrated that fine-tuning causes catastrophic forgetting, where the model loses general capabilities as it overfits to narrow training data. It also increases hallucination risk, generating facts that are statistically similar to training data but factually wrong.

The analogy: fine-tuning is neurosurgery to implant memories. Risky, can damage cognitive function. RAG is giving a smart person a textbook. They use their intact reasoning to synthesize answers from the source material. You can verify where the answer came from. You can update the textbook without surgery.

In-Context Learning: Different Circuits

Anthropic's research on induction heads provides the mechanistic explanation. Olsson, Elhage, Nanda et al. demonstrated that in-context learning and fine-tuning activate different internal circuits. In-context learning leverages general-purpose attention mechanisms, called "induction heads," to recognize patterns in the prompt. This enables better out-of-distribution generalization without the overfitting risk.

Your average commercial or open-source model is more than capable for most tasks. It just needs the right inputs. That's a legitimate market for curated, structured data, not for weight surgery.

Sources