Phase 9: Adaptive Interview Rubric

Time estimate: ~1 week
Impact: BLOCKS adaptive interview mode
Engineering status: Complete — waiting on rubric defaults

Context

We're adding adaptive interviewing as an alternative to scripted (spine) mode. In adaptive mode, the AI generates questions in real-time from competency framework anchors. The employer chooses per posting. Everything you define below becomes a configurable field in the database, editable through the admin UI.

What We Need From You

For each section, provide:

1. The default value — actual text/rules, not placeholders
2. Whether admins should be able to edit it (most should be yes)

Section 1: Question Generation Rules

a) Opening question strategy (opening_question_strategy)
Should the AI always start with a specific competency (e.g., highest-weighted), or choose based on JD/resume? Should the first question be easier/broader? Provide the default instruction text.

b) Question formulation rules (question_formulation_rules)
What makes a valid adaptive behavioral question? Constraints on stems, whether situational follow-ups are allowed. Provide the default text.

c) Anchor targeting instructions (anchor_targeting_instructions)
How should the AI use the 5-level anchor definitions when generating questions? Example of the specificity we need:

"When generating a question for [competency], read the anchor definitions for levels 3-5. Frame your question to elicit evidence that would distinguish a level-3 response from a level-4 or level-5 response."

Section 2: Probing & Follow-Up Strategy

a) Probing triggers (probing_triggers)
What signals in a candidate's response trigger deeper probing vs. moving on? Be specific: vagueness, lack of ownership language, no metrics, etc.

b) Max probes per competency (max_probes_per_competency)
A number. What's the default?

c) Allowed probe types (probe_types)

• Clarifying: "Walk me through the specific steps?"
• Depth: "What was the measurable outcome?"
• Ownership: "What was your specific role?"
• Contradictory: "Earlier you said X, but then Y?"

Which are allowed by default? Any off-limits?

d) Escalation rules (escalation_rules)
Decision tree format:

IF candidate lacks specific examples → probe for concrete situation
IF candidate lacks ownership language → probe for personal role
IF probed [max] times and still level 1-2 → move on
IF strong level-4+ answer → one confirming probe, then move on

Section 3: Coverage Requirements

a) Minimum coverage rule (minimum_coverage_rule)
Must every assigned competency be touched at least once? What if time runs out?

b) Coverage vs. depth tradeoff (coverage_depth_tradeoff)
If the AI has spent significant time on half the competencies, what about the rest?

c) Budget allocation strategy (budget_allocation)
Equal time? Weighted by posting competency weights? Front-load high-weight?

d) Max single-competency share (max_competency_share)
A percentage. Maximum % of total questions on any single competency. Default?

e) Coverage tracking instructions (coverage_tracking)
What self-monitoring should the AI do during the interview?

Section 4: Comparability & Fairness

a) Comparability statement (comparability_statement)
What makes adaptive scores comparable across candidates?

b) Structural requirements (structural_requirements)
Same high-level structure every time? Can competencies interleave?

c) Difficulty calibration (difficulty_calibration)
How to prevent the AI from asking one candidate harder questions?

d) Scorer adjustments
Does the scorer need ANY changes for adaptive transcripts? Currently it receives transcript + anchors regardless of how questions were generated.

Section 5: Complete Adaptive Rubric JSON

THIS IS THE MOST CRITICAL DELIVERABLE.

Assemble the complete default adaptive rubric config as JSON. This is what gets seeded in the rubrics table with type = 'adaptive_interviewer'.

Fields that carry over from the structured rubric: tone, opening, closing, pacing, system_identity. The question-spine-specific fields get replaced by all the adaptive fields from sections 1-4.

Section 6: Depth Mapping

For each depth level, specify:

• Minimum competencies to cover
• Approximate total questions (including probes)
• Target duration

Should the AI be told the target duration, or just the question budget?

Section 7: Anything Missing?

Is there anything about adaptive structured interviewing from the I-O literature we haven't covered?