A vendor scoring matrix is the most common artifact in a procurement decision and one of the easiest to manipulate without anyone noticing. The buyer who built the matrix usually has a preferred outcome before the matrix is finalized. Three specific checks separate a matrix that is informative from a matrix that is decorative.
What a healthy matrix contains
A healthy vendor scoring matrix has six structural properties. The criteria are weighted and the weights sum to one. Each vendor has a raw score per criterion. The raw scores normalize to a 0-1 scale before weighting. The composite score is the sum of normalized scores times weights. The matrix is dated. The matrix names the criteria author and the date the weights were last revised.
Most matrices in the wild have three of those six. The weights are present but the normalization is implicit. The criteria author is not named. The date is missing. The raw scores are sometimes presented without showing the units. You can still use the matrix to make a decision; you cannot defend it in a procurement review without reconstructing the missing properties.
How weights get fudged
Weight-fudging is rarely deliberate fraud. It is more often the consequence of revising the weights after looking at the raw scores. The buyer scores the four vendors, sees that their preferred vendor is third on the composite, and adjusts the weights to bring the preferred vendor to the top. The adjustment is usually small (a tenth of a point on each of three weights) and usually defensible (each adjustment has a plausible reason). The composite moves. The preferred vendor wins.
This is not nefarious. It is the natural response to looking at the numbers and thinking about whether the weights match your actual priorities. The problem is not the adjustment. The problem is the absence of a record of when the weights were set and what triggered the change.
The three checks
Check 1: were the weights set before the scores?
The single most useful question to ask the matrix author. If the answer is yes (weights set first, scores compiled second, composite computed third), the matrix is structurally sound. If the answer is "we iterated on both," the matrix has fitted weights to a preferred outcome and the composite is post-hoc justification.
Check 2: how stable is the recommendation under reasonable weight changes?
Take each weight and move it by 0.05 in each direction. The composite shifts by a known amount. If the recommended vendor changes under any single small weight shift, the recommendation is brittle. A robust recommendation survives small perturbations to the weights because it leads on multiple criteria, not just one heavily-weighted one. The brittle recommendation is the one tuned to a single weight.
Check 3: does the matrix include the no-decision option?
The honest matrix lists "keep current vendor" as one of the rows, with raw scores against the same criteria. The dishonest matrix omits this option, framing the decision as a choice between new vendors only. The omission is telling because it forecloses the most common outcome of a serious vendor evaluation (do not switch).
A worked example
Consider a four-vendor scorecard for a contract metal stamper. Criteria: price (weight 0.3), on-time delivery (0.25), quality NCR rate (0.2), price stability (0.15), payment terms (0.1). Vendors: V-218 (incumbent), V-244, V-258, V-271.
Run the composite with the original weights. V-244 wins narrowly. Run check 1: were the weights set before the scores? Buyer admits the weights were tuned twice after seeing the raw scores. Run check 2: shift price down by 0.05 and quality up by 0.05. The winner changes to V-218. Shift on-time delivery up by 0.05 and price down by 0.05. The winner changes to V-258. The composite is brittle on two of three sensible perturbations.
Check 3: V-218 (the incumbent) is on the matrix as one of the four rows. Good. But the matrix author intended to replace V-218, so the price weight is set high (0.3) because V-218 is more expensive than the alternatives. If the decision were really about price, the matrix is fine. If the decision were really about reliability over price, the weights should be inverted, and V-218 wins comfortably.
Why the recommended row is usually correct anyway
A surprising property of vendor matrices: the recommended vendor is usually correct, even when the matrix is brittle. The reason is that the buyer who is doing the work knows the category, knows the vendors, and is encoding their gut into the weights. The weights are tuned to a preferred outcome, but the preferred outcome is usually the buyer's best honest call based on years of category experience.
The matrix is not bad because the recommendation is wrong. The matrix is bad because the recommendation cannot be defended without the buyer reconstructing the reasoning from memory. Procurement reviews want the reasoning to be visible in the matrix. The honest move is to write down the priority you actually have, set the weights to match, and let the composite tell you whether your gut and your priorities agree.
The three honest moves
- Date the weights and the scores separately. The weights date tells the reviewer when the priorities were set. The scores date tells the reviewer when the vendors were evaluated. If the weights date is after the scores date, that is information the reviewer needs.
- Show the composite under three weighting scenarios. The original weights, a price-tilted variant (price weight +0.1, others rebalanced), and a quality-tilted variant (quality weight +0.1, others rebalanced). If the recommendation is the same in all three, the choice is robust. If not, the matrix is calling out the priorities that matter.
- Include the incumbent and the no-change option as first-class rows. The no-change row should be scored against the same criteria. Sometimes the no-change row wins on the merits. The matrix that prevents that answer is the matrix that fails the procurement review.