Packet only. Default for v0.3 and the cleanest test of synthesis and judgement.
Methodology v0.3
Judgement carries the most weight.
MidMarketBench tests applied investment judgement against realistic synthetic diligence packets. Overall is derived from eight dimension scores, never stored directly.
Dimensions and weights
| Dimension | Weight | Definition | High score | Low score |
|---|---|---|---|---|
| Grounding | 15% | Claims are supported by packet evidence or clearly marked as inference. | Specific evidence, disciplined attribution and no invented facts. | Generic claims, hallucinated facts or management language repeated as truth. |
| Commercial judgementHeaviest | 20% | Identifies what matters economically and strategically to the deal. | Prioritises revenue quality, durability and decision impact. | Treats all issues equally and misses the business model's economic engine. |
| Scepticism | 15% | Challenges weak claims, inflated metrics and convenient narratives. | Tests TAM, retention, ARR definitions and internal contradictions. | Accepts the CIM framing and assumes favourable interpretations. |
| Numerical sanity | 15% | Interprets and reconciles financial and operating data correctly. | Connects cohorts, concentration, ARR bridges and revenue mix. | Confuses metrics, ignores tables or makes arithmetic errors. |
| Risk discovery | 15% | Finds hidden and second-order risks across the packet. | Connects evidence across files and ranks risks by severity. | Lists generic risks and misses disguised business model weakness. |
| Question generation | 10% | Asks the next questions most likely to change the recommendation. | Specific, answerable, prioritised and disconfirmatory questions. | Long, generic diligence checklists with no decision hierarchy. |
| European context | 5% | Accounts for fragmented markets, regulation, language and procurement. | Avoids US category assumptions and tests country-level repeatability. | Treats Europe as one market and overstates reachable scale. |
| Output usefulness | 5% | Produces material a deal team can reuse with limited editing. | Concise, structured, memo-ready and explicit about next actions. | Verbose, generic and operationally difficult to reuse. |
Scoring reference
Five-point rubric
Dimension judgements map to a common anchored scale before aggregation.
| 5 | Excellent | Identifies the issue, supports it, explains decision impact and proposes the right diligence action. |
| 4 | Good | Identifies and explains the issue but misses some nuance or evidence. |
| 3 | Acceptable | Partially identifies the issue but treats it generically or weakly prioritises it. |
| 2 | Weak | Gestures at the issue without understanding its commercial significance. |
| 1 | Poor | Misses the issue or reaches the opposite conclusion. |
Run protocol
Closed-book first.
Public research allowed. Tests source selection, market mapping and competitive discovery.
File inspection, search and calculation tools enabled. Tests end-to-end workflow execution.
The model may request missing evidence. Tests diligence process quality and question prioritisation.
Scoring operation
Hybrid, traceable evaluation.
Deterministic checks should cover arithmetic, schema validity and answer-key issue coverage. Blind model judging covers subjective dimensions, calibrated against human investment professionals.
Public and private
Cases in public. Holdouts in reserve.
Public cases explain the method. Private variants reduce overfitting and should never be shipped to the browser or public repository.