Turning heuristic evaluation into a repeatable, comparable score.

Overview

Heuristic evaluation is one of the oldest tools in UX, but it usually produces an inconsistent, one-off audit — different evaluators, different criteria, no way to compare a score to last quarter's. This framework fixes that: the same 20 checks, the same 1–5 scale, every time, producing a composite score out of 100 that can be tracked release over release.

The model

The framework is grounded in HFI's VIMM model — every usability problem traces back to one of four cognitive dimensions: Visual, Intellect, Memory, Motor. VIMM translates into four evaluable NPCI focal points, each scored across 5 checks rated 1–5:

• Navigation — can users find their way? (5 checks × 5 = 25)
• Presentation — is it easy to comprehend visually? (5 checks × 5 = 25)
• Content — is the language clear and scannable? (5 checks × 5 = 25)
• Interaction — is it efficient and forgiving? (5 checks × 5 = 25)

The four subtotals sum to a clean composite out of 100 — no conversion or weighting required. Every check rated 1 or 2 automatically generates a tagged issue (HIGH / MED / LOW priority) with evidence attached, so a score is never just a number without a reason.

Automation tiers

The 20 checks are split by how objectively they can be measured — this is the core "AI efficiency" of the tool: automating what's deterministic, assisting what benefits from a model's draft, and keeping human judgment where it belongs.

• Tier 1 — Fully automated (~30%) — deterministic tools score these directly: spelling/grammar linters, contrast and accessibility audits, automated scenario checks. No human input needed.
• Tier 2 — AI-assisted (~45%) — a vision-language model drafts a 1–5 rating with rationale and evidence for checks like label clarity, visual hierarchy, and content structure. A designer validates or overrides every draft.
• Tier 3 — Human judgment (~25%) — checks that require task context and user-goal understanding are scored by a designer, with AI supplying supporting evidence only.

Calibration target: AI-to-human agreement should stay at or above 80% within ±1 rating point per check, reviewed quarterly — if it drops below that, the rubric wording gets tightened before the model is trusted with that check again.

Score bands

Because scoring is derived, not entered directly, and every low-scoring check produces a tagged, evidenced issue, the same framework doubles as an audit trail — useful for tracking whether a flow is actually improving release over release, not just whether it "feels better."