◭AI Lab3 min read

Turning heuristic evaluation into a repeatable, comparable score.

A structured framework — built on Human Factors International's NPCI model — that scores any page, feature, or flow out of 100, with a prioritised issue list attached to every low score.

Stage: Live in team workflow
Framework: HFI NPCI (VIMM)
Checks: 20 across 4 focal points
Composite: Score out of 100

overview

Overview

Heuristic evaluation is one of the oldest tools in UX, but it usually produces an inconsistent, one-off audit — different evaluators, different criteria, no way to compare a score to last quarter's. This framework fixes that: the same 20 checks, the same 1–5 scale, every time, producing a composite score out of 100 that can be tracked release over release.

Human review is mandatory in all automated flows — Tier 1 and Tier 2 scores are drafts until a human accepts or overrides.

model

The model

The framework is grounded in HFI's VIMM model — every usability problem traces back to one of four cognitive dimensions: Visual, Intellect, Memory, Motor. VIMM translates into four evaluable NPCI focal points, each scored across 5 checks rated 1–5:

Navigation — can users find their way? (5 checks × 5 = 25)
Presentation — is it easy to comprehend visually? (5 checks × 5 = 25)
Content — is the language clear and scannable? (5 checks × 5 = 25)
Interaction — is it efficient and forgiving? (5 checks × 5 = 25)

The four subtotals sum to a clean composite out of 100 — no conversion or weighting required. Every check rated 1 or 2 automatically generates a tagged issue (HIGH / MED / LOW priority) with evidence attached, so a score is never just a number without a reason.

automation

Automation tiers

The 20 checks are split by how objectively they can be measured — this is the core "AI efficiency" of the tool: automating what's deterministic, assisting what benefits from a model's draft, and keeping human judgment where it belongs.

Tier 1 — Fully automated (~30%) — deterministic tools score these directly: spelling/grammar linters, contrast and accessibility audits, automated scenario checks. The tool produces the draft; a designer still accepts or overrides it, so no score is final without human sign-off.
Tier 2 — AI-assisted (~45%) — a vision-language model drafts a 1–5 rating with rationale and evidence for checks like label clarity, visual hierarchy, and content structure. A designer validates or overrides every draft.
Tier 3 — Human judgment (~25%) — checks that require task context and user-goal understanding are scored by a designer, with AI supplying supporting evidence only.

Calibration target: AI-to-human agreement should stay at or above 80% within ±1 rating point per check, reviewed quarterly — if it drops below that, the rubric wording gets tightened before the model is trusted with that check again.

bands

Score bands

85–100Excellent — ship and document as reference

70–84Strong — minor clean-up before next release

55–69Adequate — meaningful rework required

40–54Weak — significant rework before release

Below 40At risk — a review blocker raised to the people who own the decision, not an automatic veto

Because scoring is derived, not entered directly, and every low-scoring check produces a tagged, evidenced issue, the same framework doubles as an audit trail — useful for tracking whether a flow is actually improving release over release, not just whether it "feels better."

How the bands were set

The bands are an internal convention for prioritising rework, calibrated against scored examples inside the team rather than an external benchmark. No band vetoes a release on its own: a low score raises the issue list to the people who own the decision. The rubric is adapted from Human Factors International’s NPCI/VIMM model by a practitioner trained in it, and naming or licensing would need HFI’s agreement before any public product use.