AI Engineering 9 min read

AI-or-Not? A Decision Framework for Product Managers

When AI is the right solution and when it's not. The 6-question framework to avoid building AI for its own sake — and how to defend the decision either way.

Not every problem needs AI. Using an LLM where a regex would do is wasteful, slow, and introduces unnecessary failure modes. Conversely, rejecting AI because 'it could be wrong' ignores that it can be right 99% of the time and handle cases no rule-based system could. Here's a decision framework.

The three questions

Before defaulting to AI, ask: (1) Is the problem well-defined enough for a deterministic solution? (2) Is the cost of AI errors acceptable relative to the cost of building deterministic rules? (3) Does the problem require understanding unstructured language, nuance, or context that rules can't capture?

When deterministic code beats AI

Use case	Why deterministic wins
Email format validation	A regex is faster, cheaper, 100% accurate, and easier to audit
Date parsing	Edge cases are enumerable; a library handles them perfectly
Price calculation	Math must be exact; LLMs hallucinate numbers
Permission checking	Binary logic; LLM could be convinced to bypass it
Sorting and filtering	Deterministic by nature; no ambiguity to resolve
Database queries from known fields	SQL or ORM is better than natural language → SQL for structured data

When AI clearly wins

Use case	Why AI wins
Classifying freeform customer feedback	Unstructured text with infinite variation — rules don't scale
Summarising long documents	Requires reading comprehension, not pattern matching
Answering questions over a knowledge base	Open-ended retrieval + synthesis = RAG's sweet spot
Generating first drafts of written content	Humans can't enumerate rules for 'good writing'
Extracting structured data from messy PDFs	Format variation is too high for deterministic parsers
Conversational interfaces	Turn-taking, context memory, and language understanding all require LLMs

The grey zone: when to think harder

The interesting cases are where both AI and deterministic approaches are plausible. For these, evaluate on four dimensions: accuracy requirements, explainability requirements, volume and cost, and how well-defined the task is.

High accuracy requirement + explainability required → lean deterministic or use AI with citation/grounding
Low volume, ambiguous inputs → AI often wins even if not perfect
High volume, clear success criteria → A/B test AI vs. deterministic and measure
Regulated domain (healthcare, finance) → AI requires explicit auditability; may not be worth it

The hybrid pattern

The most production-robust pattern is often AI + validation: use AI to extract or classify, then validate the output against deterministic rules. An LLM extracts a date from a user message; a date parser validates and normalises it. An LLM classifies a support ticket category; a rule checks the category is in your valid list. The LLM handles language variability; deterministic code handles correctness guarantees.

def extract_and_validate_date(user_message):
    # AI step: extract the date from natural language
    result = llm(f"Extract the date from: '{user_message}'. Return JSON: {{date: 'YYYY-MM-DD or null'}}")
    extracted = json.loads(result)["date"]

    # Deterministic validation step
    if extracted is None:
        return None, "no_date_found"

    try:
        parsed = datetime.strptime(extracted, "%Y-%m-%d")
        if parsed < datetime.today():
            return None, "date_in_past"
        return parsed, "ok"
    except ValueError:
        return None, "invalid_format"

Red flags for AI over-engineering

You're using an LLM to filter a list by a specific attribute that's already in a database field
Your 'AI feature' is just prompt → response with no structured output or validation
The cost per request exceeds the value delivered per request
You're using a 200B model for a task a 7B model handles equally well
There's no eval suite — you're shipping AI features you can't measure

AI product decision framework →: Work through the build-vs-buy and AI-vs-deterministic frameworks in the AI PM module.

Try it interactively

GenAI Systems Lab is a free platform for AI engineers — configure real failure modes, break things, and build the judgment that gets you hired.

Open GenAI Systems Lab →