Scorer System
Output Format
The JSON schema the scorer produces — engineering-owned, not editable via UI.
The scorer outputs a structured JSON scorecard. This format is engineering-managed (not exposed in the admin UI) because it defines the data contract between the scorer and the rest of the system.
Scorecard Structure
{
"scores": [
{
"competency": "Problem Solving",
"score": 4,
"evidence": "Candidate described redesigning the checkout flow...",
"strengths": ["Methodical approach to root cause analysis..."],
"concerns": ["Did not mention measuring outcomes..."]
}
],
"weighted_score": 3.7,
"recommendation": "Advance",
"red_flags": [],
"overall_strengths": ["Strong analytical thinking..."],
"overall_concerns": ["Limited examples of cross-functional work..."]
}Key Fields
| Field | Type | Description |
|---|---|---|
scores | Array | Per-competency: name, score (1-5), evidence quote, strengths list, concerns list |
weighted_score | Number | Calculated from scores × competency weights. Recalculated server-side for integrity. |
recommendation | String | "Advance" (≥ advance_threshold), "Hold" (≥ hold_threshold), or "Decline" |
red_flags | Array | Coached response flags, adverse impact flags, etc. |
overall_strengths | Array | Cross-cutting strengths across all competencies |
overall_concerns | Array | Cross-cutting concerns across all competencies |
Weighted Score Calculation
The weighted_score is recalculated server-side (not trusted from the AI output):
weighted_score = Σ (competency_score × competency_weight) / Σ (competency_weights)Recommendation Thresholds
| Condition | Recommendation |
|---|---|
weighted_score >= advance_threshold (default 3.5) | Advance |
weighted_score >= hold_threshold (default 2.5) | Hold |
weighted_score < hold_threshold | Decline |
Both thresholds are configurable via the scorer rubric admin page.