Methodology · Rubric v1.5

How scores are reached

Each dimension is scored against five levels of observable behaviour. The coach first picks the level that matches your words, which maps to a fixed score (L1 = 10, L2 = 30, L3 = 50, L4 = 70, L5 = 90). There is no finer scoring inside a level — that precision would not be justified before a validation study.

Every score must be backed by verbatim quotes from your turns. The app checks each quote against the transcript; if no quote can be verified, the dimension is shown as "not ratable" and is excluded from the overall score, which is the plain average of rated dimensions.

Empathy

Recognises and acknowledges the other person's perspective, feelings, constraints and autonomy — naming a feeling is one way, not the only way.

Level	Observable behaviour
L1 · 10 pts	Ignores or dismisses the other person's perspective, feelings or constraints
L2 · 30 pts	Acknowledges the other side only generically ("I understand") without content
L3 · 50 pts	Acknowledges one concrete perspective, emotion or constraint, but does not act on it
L4 · 70 pts	Acknowledges a concrete perspective, emotion or constraint, links it to the issue and responds appropriately
L5 · 90 pts	Consistently reflects the other's perspective (e.g. accurate summary, respecting limits or autonomy) and adapts tone to their reactions

Directness

States the core message, expectations or decision clearly and honestly.

Level	Observable behaviour
L1 · 10 pts	Avoids the actual topic or hides it completely behind small talk
L2 · 30 pts	Hints at the issue but stays vague; key message must be guessed
L3 · 50 pts	Names the topic, but heavily waters it down
L4 · 70 pts	Fails exactly one of the applicable level-5 checks
L5 · 90 pts	Passes all applicable checks on the user's own turns: (1) EARLY: the core message is stated within an appropriate opening and before extended discussion or problem-solving begins; (2) CONCRETE: at least one user turn contains both a concrete example (named incident, number or date) and an explicit expectation or decision; (3) RESILIENT: if challenged, the user acknowledges the response and restates the core expectation or decision clearly without withdrawing it — tone and wording may be adapted. If no pushback occurred, RESILIENT is 'not applicable' (neither passed nor failed) and is reported as not assessed

Active listening

Picks up what the other person says, checks understanding and builds on it.

Level	Observable behaviour
L1 · 10 pts	Ignores key statements or switches straight to their own argument
L2 · 30 pts	Responds to content but does not check understanding
L3 · 50 pts	Asks some open questions and partially picks up the answers
L4 · 70 pts	Mirrors key statements, clarifies understanding and separates facts from assumptions
L5 · 90 pts	Summarises precisely, checks understanding and consistently integrates answers into the rest of the conversation

Clear next steps

Closes with the agreements this scenario needs — what is decided, who owns it, by when, plus scenario-specific elements where relevant.

Level	Observable behaviour
L1 · 10 pts	No next steps or agreements are mentioned
L2 · 30 pts	Mentions vague intentions ("let's look into it") without owner or timing
L3 · 50 pts	Proposes a concrete step, but owner or deadline is missing
L4 · 70 pts	All required closure elements of this scenario are present (by default: outcome, owner, timing)
L5 · 90 pts	All required closure elements plus every additional element marked relevant for this scenario are present

Evidence base · Library v1.3

Where the rubric comes from

The rubric is informed by established research. The specific dimensions, behavioural anchors, weighting and score thresholds are Ensavo's operationalisation and have not yet been independently validated. Ensavo does not claim its rubric is independently validated — it claims it is transparent: every source below is linked to the concrete scoring rule it informs.

Only bibliographic data, summaries and derived rules are stored here — no publisher PDFs.

Source	Construct	Scoring rule
Weger et al.	Active listening	Raise the listening score only when the user paraphrases, clarifies or summarises.
Kluger & DeNisi	Feedback	Rate task-focused behaviour; weight personal put-downs negatively.
Edmondson	Psychological safety	Encourage raising risks openly without removing accountability.
Locke & Latham	Goal setting	A next step needs an outcome, an owner and a deadline.
Tannenbaum & Cerasoli	Debriefing	After every attempt: reflection plus a concrete goal for the next one.
Blume et al.	Transfer of training	Practice must mirror real work situations and follow-up actions.
Meyer	Direct vs. indirect feedback	Rate whether the core message is stated clearly and early — not how harsh it sounds; softening is fine if the point stays unmistakable.

Active and empathic listening

Grounds the Empathy and Active listening dimensions.

Weger, Castle & Emmett (2014): The Relative Effectiveness of Active Listening in Initial Interactions Start here
International Journal of Listening
Central for rating paraphrasing, follow-up questions and verbal listening behaviour.
DOI ↗
Bodie (2011): The Active-Empathic Listening Scale
Communication Quarterly
Provides a structure of perceiving, processing and responding — a possible frame for the scorecard.
DOI ↗
Mishima et al. (2000): The Development of a Questionnaire to Assess the Attitude of Active Listening
Journal of Occupational Health
Measures active listening in a professional/work context.
DOI ↗

Effective feedback

Grounds how constructive feedback is rated — task-focused, not person-focused.

Kluger & DeNisi (1996): The Effects of Feedback Interventions on Performance Start here
Psychological Bulletin
Meta-analysis. Key finding: feedback improves performance on average, but over a third of feedback interventions made performance worse — feedback must stay task-focused.
DOI ↗
Hattie & Timperley (2007): The Power of Feedback Start here
Review of Educational Research
Distinguishes goal, current state and next step — the frame behind "What worked", "To work on" and "Try saying it this way".
DOI ↗
Wisniewski, Zierer & Hattie (2020): The Power of Feedback Revisited
Frontiers in Psychology (open access)
Newer meta-analysis (435 studies, 61,000+ participants) updating the feedback-effect picture.
Open access ↗
Steelman, Levy & Snell (2004): The Feedback Environment Scale
Educational and Psychological Measurement
Relevant for credibility, quality, frequency and supportive tone of feedback at work.
DOI ↗
Baron (1988): Negative Effects of Destructive Criticism
Journal of Applied Psychology
Helps separate constructive criticism from personal, threatening or destructive criticism.
DOI ↗

Psychological safety and difficult conversations

Grounds the Empathy dimension and the tough-conversation scenarios.

Edmondson (1999): Psychological Safety and Learning Behavior in Work Teams Start here
Administrative Science Quarterly
Foundational work on when employees raise risks, mistakes and uncertainty openly.
DOI ↗
Frazier et al. (2017): Psychological Safety: A Meta-Analytic Review and Extension Start here
Personnel Psychology
Meta-analytic review of how psychological safety relates to engagement, learning and performance.
DOI ↗
Nembhard & Edmondson (2006): Making It Safe
Journal of Organizational Behavior
How inclusive leadership behaviour fosters psychological safety.
DOI ↗
Detert & Edmondson (2011): Implicit Voice Theories
Academy of Management Journal
Useful for scenarios where employees raise problems, overload or risks too late.
DOI ↗

Motivational interviewing

Partial influence only — see the caveat below.

Lundahl et al. (2013): Motivational Interviewing in Medical Care Settings
Systematic review and meta-analysis. The context is medical, but the principles — open questions, reflection, autonomy, avoiding unnecessary confrontation — inform conversation rubrics.
DOI ↗
Lundahl et al. (2010): A Meta-Analysis of Motivational Interviewing: Twenty-Five Years of Empirical Studies
Research on Social Work Practice
Broader meta-analysis of the method.
DOI ↗

Goals, actions and commitment

Grounds the Clear next steps dimension.

Locke & Latham (2002): Building a Practically Useful Theory of Goal Setting and Task Motivation Start here
American Psychologist
Supports the criteria: specific goals, ownership, difficulty, feedback and commitment.
DOI ↗
Gollwitzer (1999): Implementation Intentions: Strong Effects of Simple Plans
American Psychologist
Relevant for concrete if-then plans and whether a conversation ends with an actionable next step.
DOI ↗

Practice, reflection and learning transfer

Grounds the practice loop: conversation, scorecard, another attempt.

Ericsson, Krampe & Tesch-Römer (1993): The Role of Deliberate Practice in the Acquisition of Expert Performance
Psychological Review
Foundational work on deliberate practice, immediate feedback and repeated improvement.
DOI ↗
Macnamara, Hambrick & Oswald (2014): Deliberate Practice and Performance
Psychological Science
Meta-analysis that also cautions against overstating deliberate practice as the sole explanation of performance.
DOI ↗
Tannenbaum & Cerasoli (2013): Do Team and Individual Debriefs Enhance Performance? Start here
Human Factors
Meta-analysis on structured debriefing and reflection — maps directly onto the Ensavo flow of conversation, scorecard, retry.
DOI ↗
Blume et al. (2010): Transfer of Training: A Meta-Analytic Review Start here
Journal of Management
Addresses whether good training performance actually transfers to the job (89 empirical studies).
DOI ↗
Di Stefano et al. (2013): Learning by Thinking: How Reflection Aids Performance
Working paper (SSRN)
Shows the added benefit of structured reflection over mere repetition.
DOI ↗
Shaikh, Chai, Gelfand, Yang & Bernstein (2024): Rehearsal: Simulating Conflict to Teach Conflict Resolution
Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24)
Related research on the concept: in a study with 40 participants, people who practised simulated conflict conversations with an AI counterpart used interest-based (cooperative) strategies about twice as often and competitive strategies (e.g. power- or rights-based) around 67% less often in a subsequent real conflict conversation. Independent work that developed in parallel; it supports the idea of practising with simulations, but does not inform Ensavo's dimensions, anchors or scoring rules.
arXiv ↗

Practitioner literature and design considerations

Business literature, not peer-reviewed research — informs the Directness dimension as a design consideration for language and cultural context.

Meyer (2014): The Culture Map: Breaking Through the Invisible Boundaries of Global Business
PublicAffairs (book)
Maps cultures on scales such as direct vs. indirect negative feedback and low- vs. high-context communication. Informs why Directness rewards a clear core message, not bluntness.
Google Scholar ↗

Scope & limitations

– Motivational interviewing is not adopted wholesale — it is not a leadership technique for every situation. Only individual principles (open questions, reflective listening, autonomy support) inform the empathy and listening anchors.

– The rubric is informed by established research. The specific dimensions, behavioural anchors, weighting and score thresholds are Ensavo's operationalisation and have not yet been independently validated.

– Full texts are only included when open access or when usage rights exist; this app stores links, not copies.

Consistency · Rubric v1.5 · 2026-10

Do you get the same score twice?

One fixed English 'tough feedback' transcript (3 user turns: names three missed deadlines and the client impact, acknowledges the pushback and sets an expectation, agrees owner, support and a Friday check-in), scored three times independently under rubric 1.5 (fixed points per level: 10/30/50/70/90, scenario-specific closure elements).

Overall range

70 · 70 · 70

Mean

Dimension	Run 1 · Run 2 · Run 3	Spread	Level
Empathy	70 · 70 · 70	±0 (0 pts)	stable (L4)
Directness	70 · 70 · 70	±0 (0 pts)	stable (L4)
Active listening	50 · 50 · 50	±0 (0 pts)	stable (L3)
Clear next steps	90 · 90 · 90	±0 (0 pts)	stable (L5)

All four dimensions received the same level in all three runs (12 of 12 dimension ratings identical). Under rubric 1.4 a different transcript had flipped twice on borderline levels; the stricter observable criteria of 1.5 (scenario-specific closure elements, RESILIENT as 'not applicable' without pushback) left less room for interpretation in this test.

Small internal check on one transcript — an indicator, not a formal reliability study. The transcript differs from the one used for the 1.4 check, so the two results are not directly comparable. Borderline conversations can still land on a neighbouring level; for high-stakes use we recommend double evaluation.

Changelog

What changed, and when

Rubric 1.5 · Library 1.3 2026-10

– Library: added Shaikh et al. (2024) as related research on practising difficult conversations with a simulated AI counterpart. It supports the concept, not the scoring anchors — no entry in the evidence map.

Rubric 1.5 · Library 1.2 2026-10

– Directness: EARLY now means 'within an appropriate opening, before extended discussion' instead of a fixed turn limit.
– Directness: RESILIENT replaces FIRM — the expectation must be restated without withdrawing it; tone may be adapted. Without pushback it is 'not applicable', not passed.
– Clear next steps is scenario-dependent: L4 = all required closure elements (default: outcome, owner, timing), L5 = plus every element marked relevant for the scenario (e.g. follow-up, escalation rule, support, success criteria).
– Evidence schema extended: each quote carries criterion and positive/negative effect; each dimension reports missing behaviours, criteria not assessed, confidence and rubric version.
– Consistency check repeated under Rubric 1.5: three independent runs, identical level on all four dimensions (overall 70 · 70 · 70).

Rubric 1.4 · Library 1.2 2026-10

– Scores are now fixed per level (10/30/50/70/90) — no unjustified precision inside a band.
– Empathy anchors broadened: acknowledging perspective, constraints or autonomy counts, not only naming feelings.
– Corrected ten DOIs and six journal names in the core library.
– Wording changed from 'anchored in research' to 'research-informed, not independently validated'; Meyer (2014) moved to practitioner literature.

Rubric 1.4 · Library 1.2 2026-10

– Consistency check re-run on rubric 1.4: same transcript scored three times. Empathy and Directness stable; Active listening and Clear next steps each flipped once between neighbouring levels. Fixed level points removed all within-level score noise.

Rubric 1.3 · Library 1.1 2026-10

– Directness levels 4 and 5 rewritten as strict yes/no checks (early core message, example plus expectation, holding firm after pushback; no pushback counts as passed).
– Consistency check re-run on rubric 1.3: same transcript scored twice, all four levels identical, overall score 84 vs 83.

Rubric 1.2 · Library 1.1 2026-10

– Directness levels 4 and 5 separated by three countable checks (early core message, example plus expectation, holding firm after pushback) to reduce level flips between runs.
– Consistency check re-run on rubric 1.2 and published per dimension.

Rubric 1.1 · Library 1.1 2026-10

– Empathy anchors (levels 1, 3, 4, 5) tightened to single observable behaviours.
– Directness level 3 and 5 and Clear next steps level 5 shortened.
– Library: added Meyer (2014) as the source for the Directness dimension, plus its scoring rule.
– Published a scoring-consistency check.

Rubric 1.0 · Library 1.0 2026-09

– First rubric: four dimensions, five observable levels each, verbatim-evidence requirement.
– Core library: 21 sources in 6 themes with an evidence map.