There are four ways to make an AI model do what your business needs, and they are not alternatives. They are a ladder. Each rung fixes a specific kind of problem, and climbing past the rung that would have solved yours is the most common and most expensive mistake in this field.
The short version: prompting changes behaviour, retrieval changes knowledge, fine-tuning changes form, and training from scratch buys you a capability that does not exist yet. Diagnose which one you actually have before spending anything.
Start by naming the failure
Before choosing an approach, get specific about what the model is doing wrong. Vague dissatisfaction leads to expensive guessing.
- It does not know something. It has never seen your product catalogue, your policies, or last week's price change.
- It knows, but answers in the wrong shape. Wrong tone, wrong length, wrong format, inconsistent between runs.
- It cannot do the task at all. Not badly, but genuinely not, because the task is unlike anything in its training.
- It is right, but too slow or too expensive at your volume.
Those four failures map almost one to one onto the four rungs. Most teams arrive convinced they need the top rung and leave having solved it on the first or second.
Prompting fixes behaviour
A prompt is instructions plus examples. It is the fastest thing to change, the cheapest to test, and it handles far more than people expect once it is written seriously.
Serious means specific. Not "be professional" but "if the caller has not given a name yet, ask for it before anything else, and use it from then on." Not "be concise" but "answer in at most three sentences unless asked for detail." Include a few worked examples of the exact output you want, and describe what to do when the input is unusual, because that is where behaviour actually drifts.
The limits are real, though. Prompts cannot teach facts the model has not seen, they get unreliable as they grow, and they are trivially copied. If your entire product advantage is a prompt, you do not have an advantage.
Retrieval fixes knowledge
If the problem is that the model does not know your material, the answer is almost never to bake that material into the model. It is to fetch the relevant pieces at question time and give them to the model along with the question.
This is the right default for anything that changes: documentation, policies, prices, inventory, past support tickets. Update the source and the answers update, with no retraining. It also makes answers auditable, because you can show which document a claim came from, and it gives you somewhere sensible to stand when the model has nothing relevant and should say so instead of inventing.
Most of the engineering effort here goes into retrieval quality rather than the model. How documents are split, what gets indexed, and how results are ranked decide whether the right paragraph is in front of the model at all. A weaker model with the right paragraph beats a stronger one guessing.
Fine-tuning fixes form and cost
Fine-tuning takes an existing model and continues training it on your examples. It is the right tool when the model already knows enough but will not behave consistently, or when you want the behaviour of a long prompt without paying for that prompt on every call.
It is good at style, tone, structured output that has to be exactly right every time, and classification into your own categories. It is not a way to add knowledge, and this is where money gets wasted: teams fine-tune on their documentation, find the model still gets facts wrong, and conclude the technique failed. It did not. It was asked to do retrieval's job.
The prerequisite is examples. Not a handful, and not synthetic ones you generated to fill a quota. Real inputs with the correct outputs, covering the awkward cases as well as the ordinary ones. Assembling that set is most of the work, and it is worth doing even if you never fine-tune, because it doubles as your evaluation set.
Training from scratch is a different decision
Training a model from the ground up is rarely about quality. It is about owning something, working in a domain general models have barely seen, or operating under constraints that rule out sending data to somebody else's API.
It makes sense when your data really is unlike the public web, when unit economics at your volume favour a small owned model over per-call pricing, or when the model is the product rather than a feature of it. That last case is the honest version of the moat argument: a competitor can copy a prompt in an afternoon, and cannot copy a model trained on data they do not have.
It does not make sense as a first attempt, or because owning a model sounds more serious than using one. Our custom model development work usually starts by establishing that the cheaper rungs genuinely will not hold, because when they will, saying so saves everybody a lot of money.
Settle the data questions before you build
Where data lives, who can reach it, how long it is kept, and what happens to it at the end of the engagement are cheap questions in week one and painful ones in month six. They also change the architecture, so they are not paperwork to be done later.
The practical ones worth answering early: does any customer data leave your infrastructure, and if so under what agreement. Does anything you send become training data for somebody else's model. Can you delete a specific customer's data on request, and do you know everywhere a copy of it ended up. Who owns what gets built.
Answer those first and the technical choices narrow usefully. Some options disappear entirely, which is a good thing to discover before you have built on them.
Build the evaluation set from real exceptions
However far you climb, you need a way to tell whether a change made things better. Vibes do not survive contact with the third revision.
Build a fixed set of test cases from things that have actually gone wrong, plus the ordinary cases you cannot afford to regress on. Keep the expected answer next to each one. Run it before and after every change. It takes a morning to assemble and it is the difference between engineering and guessing.
This is also the honest way to compare rungs. Run the same set against a better prompt, against retrieval, and against a fine-tune, and let the results choose. Quite often the cheap option wins and you get to keep the money.
If you are trying to work out which rung your problem sits on, describe the failure to us and we will tell you what we would try first. It is also worth reading what a voice agent actually costs to run, since per-call pricing is frequently the thing that pushes a project toward owning a model in the first place.
