AI vendor evaluation scorecard

Compare the evidence

The published comparison rubric

The interactive worksheet uses the six weights in TeqEngine’s vendor-selection guide. They are an editorial starting point for a software engagement, not a validated industry ranking. If the engagement needs different priorities, establish those weights in the procurement worksheet before scoring proposals.

Rubric version 2 separates requirement fit from evidence strength. A score of 0 means the requirement is not met; 1 means major gaps; 2 means partially met; 3 means met; and 4 means exceeded with a relevant benefit. Leave unassessed criteria blank. The separate evidence field records whether support is unavailable, asserted, documented or observed. Direct inspection can establish either a poor fit or a strong one; it does not automatically earn a high score.

Record the shared evaluation scope, then the requirement, document or test reference, version/date and finding for each criterion. The worksheet requires documented or observed evidence and a reference note for completion. It cannot verify those entries. Version 1 scored evidence strength, so its totals should not be compared directly with this requirement-fit rubric.

Keep missing information and failed requirements visible

Before a criterion is assessed, it has no score. The worksheet displays a possible total range rather than pretending the missing work is a zero or a pass. For example, a score of 3 on a 20% criterion contributes 15 points; with the other 80% unassessed, the possible total is 15–95 out of 100.

Critical requirements are separate pass, fail or unknown decisions covering data/access, ownership/handoff and release/operation. Adapt the detailed requirements to the engagement and record the basis for each decision. A failed requirement remains flagged regardless of points. An unknown requirement, missing decision basis or unsupported score leaves the assessment incomplete.

NIST’s Secure Software Development Framework provides a useful vocabulary for supplier discussions. Referencing it does not certify the vendor or replace the organization’s own contract and security review.[1]

Static example: equal scores, different unresolved work

Suppose A’s evidence is recorded and all critical requirements pass, while B’s ownership and handoff requirement is still unknown. A’s worksheet can be complete; B’s remains incomplete. That does not establish that A is the better vendor. It identifies the unresolved decision that the next discussion must address.

The calculation divides each 0–4 fit score by four, multiplies by its weight and adds the contributions. Reference notes travel with the local export; supporting documents stay in the organization’s normal review system. The possible score range reflects unassessed criteria, not a statistical confidence interval or uncertainty about whether evidence is true.

Use the scorecard with a comparable technical exercise

• Give candidates the same task boundary, constraints and completion criteria.
• Ask for one successful path and one important failure path.
• Record which evidence is observed, documented, asserted or unavailable.
• Identify the next validation that would resolve a material unknown.
• Compare scope, operating responsibilities and total economics as well as the score.

Confidentiality can limit named references. Record what cannot be verified and evaluate permitted evidence without converting a disclosure limit into either proof or a negative finding. A small reproducible technical validation can be useful when one integration or permission assumption dominates the purchase.

Export and privacy

Download review record produces a readable text file; Download comparison JSON produces a structured file. Both include the rubric version, scope, weights, scores, evidence notes, critical decisions and unresolved work. Downloads are generated locally. Inputs stay only in this page’s memory; save a download before leaving or reloading. Clear both candidates resets the worksheet and offers Undo clear until another edit.

“Assessment complete” means the required entries, references and gate decisions are filled. It does not mean the evidence is independently verified, procurement approval, legal clearance or an automatic recommendation. Keep the final selection and its rationale in the organization’s decision record.