Should we build this with AI at all — build, buy, or skip?

What actually happens at this stage

The decision to build with AI is rarely made on purpose. It's usually made by default — a vendor demo lands well, a competitor ships something with a chat interface, or a stakeholder reads an article, and the project starts from "let's use AI for this" rather than from "what is actually broken, and does AI fix it better than the alternatives." That ordering matters more than almost anything else that happens later. A system built to answer the wrong question ships on time, passes its own demo, and still gets replaced within a year, because the actual problem was never named.

The first real test is whether a deterministic system would solve it. Rule-based logic, a lookup table, a well-indexed database, and a decent search bar solve a surprising share of the problems teams reach for an LLM to fix — and they do it more cheaply, more predictably, and without needing an eval suite. AI earns its place when the input space is genuinely open-ended (free text, ambiguous intent, unstructured documents) or the task requires judgment a fixed rule set can't encode. If a flowchart could handle the cases you're worried about, build the flowchart first.

If AI clears that bar, the next fork is build versus buy versus skip, and each answer has a real cost model behind it, not a sticker price. Buying — a vendor product, an API wrapper — is fastest but caps what you can change later and ties your roadmap to someone else's. Building gives you control and lets you fit the system to your actual data and constraints, at the cost of owning the eval suite, the observability, and the on-call. Skipping — deciding not to build anything right now — is a legitimate outcome, not a failure to decide, and it's the right call more often than most roadmaps admit. It's also worth pricing the free option before ruling buy out entirely: a surprising share of infrastructure and tooling decisions have a free tier substantial enough to reach a working prototype on (we maintain FreeTier as a running answer to exactly that question), and confirming one is genuinely free rather than a disguised trial is faster than building the alternative from scratch.

Cost has to enter this decision before a single line of code, not after. Sticker-price-per-token estimates routinely undercount real production spend by 5–20×, once retries, context growth, and eval runs are counted — a build that looked cheap in a spreadsheet can be the reason the project gets killed at the next budget review. Model the cost of the option you're leaning toward using your own expected volume and failure rate, not the vendor's happy-path numbers, before you commit to it.

If the decision is "build," you're immediately on one of two forks that are worth deciding on purpose rather than defaulting into: RAG versus fine-tuning — do you need to give the model new facts, or teach it a new skill, because they're not interchangeable — and open-weight versus a frontier API, which is really a question of how much operational control you're trading for how much convenience. Neither fork is right or wrong in the abstract; both are wrong when they're decided by whichever the team already knows, rather than by what the actual constraints of the project call for.

The common mistake

How do you know this stage is finished?

• You can name, in writing, the non-AI alternative you considered and why it didn't clear the bar.
• You have a cost model for your chosen option built from your own expected volume and failure rate — not the vendor's per-token sticker price.
• Build, buy, or skip is a decision with a stated reason attached to it, not the default outcome of who was in the room.
• If the answer is build, you know which fork you're on — RAG vs. fine-tuning, open-weight vs. frontier — and can say why.

What backs this up