Can I combine all three approaches?

Yes, and most serious production systems do — a well-engineered prompt with RAG-retrieved context is the default starting point for most applications, with fine-tuning reserved for cases where prompting and retrieval genuinely can't get the model to behave the way a specific, narrow task requires.

Which approach is cheapest to get started with?

Prompt engineering, by a wide margin — no training data, no infrastructure, no retrieval pipeline. It's also why it should always be the first thing tried, not skipped in favor of a more complex approach assumed to be more powerful.

AIIntermediate

Fine-Tuning vs. RAG vs. Prompt Engineering: Choosing the Right Approach

Three different ways to make an LLM behave the way you need — what each one actually changes, what it costs, and how to decide which one (or which combination) fits your problem.

DevFieldGuideJune 18, 2026 (updated July 16, 2026)6 min read

Three fundamentally different techniques get lumped together as "customizing an LLM," and picking the wrong one for your actual problem wastes real time and money. Here's what each one actually changes.

Prompt engineering: changing nothing about the model

Prompt engineering means crafting the instructions and examples given to the model at request time — the model itself is completely unchanged. This includes system prompts, few-shot examples, and structured output instructions.

System: You are a support ticket classifier. Respond with exactly one of:
billing, technical, account, other. No other text.

User: My card was charged twice this month.
Assistant: billing

What it's good for: Well-defined tasks where the model's existing general knowledge already covers the domain — classification, summarization, format conversion, straightforward Q&A over information the model was already trained on.

What it can't do: Give the model genuinely new, private, or post-training-cutoff knowledge, or teach it a behavior pattern that consistently resists prompting (a very specific tone, a rare output format the model keeps drifting away from).

RAG: giving the model new information at request time

Retrieval-Augmented Generation solves a different problem entirely — not changing the model's behavior, but changing what it knows for a specific request, by retrieving relevant documents and inserting them into the prompt as context.

[Retrieved context: your company's actual refund policy document]
[User question: "Can I get a refund after 45 days?"]

What it's good for: Answering questions about private, frequently-changing, or too-large-to-fit-in-a-prompt information — internal docs, a product catalog, recent data. Update the underlying documents and the next query immediately reflects it, with no retraining.

What it can't do: Change the model's underlying reasoning style, tone, or task-specific behavior — it only supplies better input, not a different model.

Fine-tuning: actually changing the model's weights

Fine-tuning trains the model further on a curated dataset of example inputs and desired outputs, adjusting its internal weights so the new behavior becomes the model's default, without needing to be re-specified in every prompt.

json

{"input": "Summarize this support ticket in our team's format", "output": "TICKET-SUMMARY: [category] | [urgency] | [one-line issue]"}

Hundreds to thousands of examples like this, run through a fine-tuning job, teach the model to produce that exact format reliably — something that's fragile with prompting alone once formats get specific enough.

What it's good for: A narrow, well-defined task performed at high volume, where consistent behavior matters more than flexibility, and where prompting has genuinely been tried and found to drift or fail.

What it can't do: Give the model live or frequently-changing information — the knowledge is baked in at training time, exactly as stale as whenever the fine-tuning job ran. Combining it with RAG is common precisely because they solve different problems.

Aspect	RAG	Fine-tuning
Changes	What the model knows, per-request	How the model behaves, permanently
Update latency	Immediate — update a document, done	Requires a new training run
Cost to set up	Moderate — retrieval pipeline, vector store	Higher — curated training data, compute
Best for	Private, current, or large reference data	Consistent, narrow, high-volume behavior

A concrete example: a customer support bot

Consider a support bot that needs to answer questions about a specific product, using the company's actual current documentation, in a consistent brand voice.

Prompt engineering alone covers the brand voice and general response structure — a well-written system prompt with a few examples.
RAG covers the product-specific knowledge — retrieving the actual current help docs so answers reflect real, up-to-date policy rather than the model's general training knowledge (which might be wrong or outdated for this specific product).
Fine-tuning would only enter the picture if, after using both of the above, the model still consistently fails at something narrow and high-volume — say, formatting every response in a specific structured template the prompt alone can't reliably enforce across thousands of varied queries.

Most real support bots stop at the first two — fine-tuning is the exception that gets reached for after a specific, measured gap, not the default starting point.

Cost ordering, roughly

From cheapest-to-iterate to most expensive: prompt engineering costs essentially nothing beyond the API calls themselves and can be changed in seconds. RAG adds infrastructure (a vector store, an embedding pipeline, retrieval logic) but no training cost — updating knowledge is still just updating documents. Fine-tuning has real upfront cost (curating a quality training set, compute for the training run) and ongoing cost (re-running the job whenever the desired behavior needs adjusting) — it's the least agile of the three to iterate on, which is the practical reason it belongs last in the decision order, not first.

A decision framework that actually works

Start with prompt engineering. It's nearly free to iterate on, and a surprising number of "we need to fine-tune" problems are actually "we haven't tried a good system prompt with real examples yet."
Add RAG when the model needs information it doesn't have — anything private, anything that changes after the model's training cutoff, anything too large to paste into every prompt.
Only fine-tune when prompting has genuinely failed at a narrow, high-volume, well-defined task, and the inconsistency is a real measured problem, not a hypothetical one.

Skipping straight to fine-tuning because it sounds like "the real solution" is one of the most common wastes of time in this space — it's the most expensive and slowest option of the three, and it solves a different problem than the one most people actually have.

Common mistakes

Reaching for fine-tuning to solve a knowledge problem. If the model is giving wrong answers because it doesn't know something, fine-tuning on more examples of the right answer doesn't fix the underlying gap the way RAG (giving it the actual current information) does.
Assuming RAG alone will fix inconsistent formatting or tone. Retrieval improves what the model knows, not how it behaves — a formatting problem is a prompting or fine-tuning problem, not a retrieval problem.
Fine-tuning on too few examples and expecting reliable behavior change. A handful of examples typically isn't enough signal to reliably shift a large model's behavior — this usually needs real, curated volume, not a token gesture dataset.
Never revisiting the decision once made. A task fine-tuned for narrow consistency six months ago might now be well within a newer base model's prompted capability — periodically re-evaluating whether the complexity is still earning its cost is worth doing.