Scoring AI Use Cases Before You Build Anything

Score each candidate on five things: the value of the problem, how ready the data is, whether current models can plausibly do the task, how risky the outcome is, and whether a real sponsor and users are committed. Use a coarse scale and a one-page scorecard, so choices are comparable and the reasoning is visible. Then test the top choice with a short, time-boxed proof of concept on real data before committing to a build. Scores guide the discussion; they do not replace judgement.

Enthusiasm and suitability are different things. A broad, open-ended assistant has no clear definition of "correct", needs access to many data sources and invites every kind of failure. A narrow task has a boundary you can test.

Early wins matter because they build the trust and the infrastructure the harder projects will need. A narrow success teaches the team what its data and processes are really like.

Keep the scale small, such as high, medium or low, and define each level in a sentence so two people score the same project the same way.

A sixth question worth asking is cost to run: what the system will cost to operate and support, not only to build. The post on building the business case for an AI project shows how to put numbers on value and cost.

Give each criterion a score of one to three, add them up and look at the pattern, not just the total. A project that scores high on value but one on data readiness is a project for later, not a project for now. This sketch is illustrative; use your own criteria and scale.

CRITERIA = ["value", "data", "feasibility", "risk", "sponsor"]

def score(candidate: dict) -> dict:
    """Each criterion is scored 1 (weak) to 3 (strong). For risk, 3 means low risk."""
    scores = {c: candidate[c] for c in CRITERIA}
    scores["total"] = sum(scores.values())
    scores["blockers"] = [c for c in CRITERIA if scores[c] == 1]   # any weak spot is a flag
    return scores

claims_summary = {"value": 3, "data": 2, "feasibility": 3, "risk": 2, "sponsor": 3}
open_assistant = {"value": 3, "data": 1, "feasibility": 1, "risk": 1, "sponsor": 2}
print(score(claims_summary))   # strong, no blockers
print(score(open_assistant))   # flashy but several blockers

The key output is the list of blockers. A single weak criterion, such as no data access or no sponsor, can stop an otherwise attractive project, and it is much cheaper to find that out on paper than after a build.

Data readiness is a common blind spot in AI projects. Before you commit, answer a few practical questions with someone who can see the data.

The data readiness chapter goes deeper. If the honest answer to several of these is "we do not know", the first project is a data project.

A proof of concept (POC) answers one question: can the system do the task well enough on real data to justify building it properly? Keep it small and time-boxed, and agree the success bar before you start.

The proof of concept scoping chapter covers how to scope one, and evaluations from customer data shows how to build the test set. A POC that is judged by a hand-picked demo proves nothing; one judged against real examples, with a bar set in advance, proves a lot.

Show your sponsor the scorecard for the shortlist, the reasoning for the pick and what would change your mind. Include the candidates you did not choose and why, because a visible list of "not yet" projects and what they need, such as better data or a clearer owner, helps the organisation fix those inputs. For how choices roll up into a managed portfolio, see AI strategy for engineering leaders. To practise the whole flow, the site's FDE simulator has scenarios where you scope and decide.

How do you choose the first AI use case? Score each candidate on value, data readiness, feasibility, risk and sponsorship using a coarse scale, look at the pattern and the blockers, and test the strongest candidate with a short proof of concept on real data before committing to a build.

What makes a good first AI project? A narrow task with a clear input and output, data you can reach and check, a way to measure success, and a named owner and users who want it. Early wins build trust and infrastructure for harder projects.

Why is data readiness so important? Because a project can have high value and a capable model and still fail if the data is inaccessible, messy, unlabelled or not permitted for this use. Checking access, quality and permission early is cheaper than discovering the gap during a build.

What is an AI proof of concept? A small, time-boxed prototype that answers whether the system can do the task well enough on real data to justify building it properly. Agree the success bar before you start and judge it on real examples, not a hand-picked demo.

What if a high-value project scores badly on one criterion? Treat that criterion as a blocker and decide what it would take to fix it, such as better data or a clearer owner. It may be a project for later, and fixing the blocker can itself be the first project.

Should I use weights or a total score? A simple scale with an eye on blockers is usually enough. Totals can hide a fatal weakness, so look at each criterion, and use the score to structure the discussion, not to make the decision for you.

How do I explain a choice to a sponsor? Show the scorecard for the shortlist, the reasoning for the pick, what evidence would change your mind and the candidates you did not choose with what each needs before it could proceed.

• How to build the business case for an AI project
• AI strategy for engineering leaders: choose, fund and stop
• Forward Deployed Engineer in 2026: Skills, Role and Why Always-On Agents Need FDEs
• Forward deployed engineering topic hub
• Forward Deployed Engineer roadmap