The Course Is Finished. Has the Decision Changed?

The working insight: Choose the decision you need to improve before choosing the training format. Measure its quality, its opportunities, and the conditions that support its use.

A dashboard can show that every employee finished a course while leaving the organization’s most consequential question unanswered: what can people now do differently when the work becomes difficult? Completion supplies useful evidence about participation. A knowledge check can reveal whether someone recognizes a principle. Neither, by itself, follows that principle into a crowded meeting, an ambiguous handoff, or a decision that carries a social cost.

This gap matters especially when organizations need judgment. A learner may correctly identify how to raise a concern in a scenario, then face a supervisor who treats the concern as an inconvenience. Another learner may use the new skill effectively yet encounter an outcome shaped by staffing or workload. Measurement must be able to see both the person’s action and the opportunity to act.

What the transfer research asks us to take seriously

Blume and colleagues’ meta-analysis of 89 studies linked training transfer with factors including motivation and supportive work environments. Some relationships were stronger for open skills, such as leadership, than for closed skills. The review also found inflated relationships when the same source supplied measures in the same context. These findings support attention to context and measurement quality; they do not establish that any single support practice causes transfer in every setting.^[1]

Ford, Baldwin, and Prasad’s later review called for more precise measurement, attention to transfer trajectories, and a problem-centered approach. Its practical challenge remains relevant: define the work change closely enough to distinguish acquiring knowledge, applying it, and sustaining it. A single postcourse survey cannot show the whole trajectory.^[2]

A 2025 study by Koch and Wisshak examined a five-week program for trainers. The 47 participating trainers improved their transfer knowledge compared with 34 controls. Assignment was not randomized, and the measured outcome concerned knowledge about supporting transfer. The authors explicitly called for further evidence about applying that knowledge to trainers’ own courses. This is a useful reminder that learning how transfer works and producing transfer in the workplace are separate achievements.^[3]

A decision trace: a proposed practice tool

I propose a decision trace: a small, agreed record of the cue someone noticed, the options they considered, the action they took, and the response that followed. It is a practitioner design proposal, not a validated instrument. Its purpose is to create inspectable evidence of a capability in use without pretending that every important feature of work can be reduced to a score.

A trace begins with an actual decision opportunity. Someone sees information that should change what happens next. They choose whether to ask, stop, proceed, check, or escalate. The trace makes that choice visible and links it to the situation. It should help us ask whether the choice was warranted, rather than merely count whether a preferred behavior appeared.

The distinction is important. More escalations can mean better recognition of problems, unclear authority, or unnecessary escalation. Fewer escalations can mean improved local resolution or growing silence. A behavior count gains meaning when the evidence, rationale, and consequence remain connected. Otherwise, the metric can become a new rule people optimize while the original problem persists.

An illustrative handoff with competing demands

Imagine a fictional organization teaching supervisors to improve shift handoffs. The previous routine rewards short updates. A new course asks supervisors to identify unresolved issues, name an owner, and confirm the next action. The course looks excellent, and every supervisor passes its final assessment. Two weeks later, an unresolved issue is still lost between shifts.

A decision trace would examine the moment the outgoing supervisor noticed the issue. Was the information available? Did the supervisor recognize its significance? Was there an owner with authority to act? Did the incoming shift have time to confirm the handoff? Was the supervisor discouraged from extending an update? The answer determines whether the next response concerns training, coordination, or role design.

The hidden rule may also matter: if I slow the handoff, the team will see me as inefficient. A Pressure Map™ can turn that prediction into a bounded test. In one planned handoff, the supervisor adds a short ownership check and records how the team responds. At the same time, the manager makes room for the check and names why it matters. The learner should not be asked to overcome a penalty the organization continues to impose.

Build the assessment backward from the decision

Begin with the work problem and identify the decision that could alter it. For this handoff, the capability might be: detect an unresolved issue, establish a responsible owner, and confirm a next action before the outgoing shift leaves. That is specific enough to practice, observe, and discuss. “Understand effective communication” gives a designer much less direction.

Use several scenario variants. One includes a routine unresolved task; another includes ambiguous ownership; a third presents a senior colleague who pushes for speed. Ask learners to explain the reason for their response. A strong assessment should expose whether someone can apply the criterion when surface details change, while staying appropriate to their role and authority.

The CDC’s training-evaluation guidance recommends assessing both learning and transfer and using delayed follow-up after learners have had opportunities to apply the material. It also cautions that satisfaction does not determine effectiveness. These are evaluation recommendations, not a guarantee that delayed surveys will objectively establish a program’s impact.^[4]

A modest evidence plan that can answer a real question

Before the intervention, sample existing handoffs using the same rubric that will be used later. After training, review a comparable sample at two points, chosen according to how often handoffs occur. Record the proportion of relevant issues with clear ownership and a confirmed next action. Include cases where an issue did not require that treatment, so the measure rewards appropriate judgment.

Keep four kinds of evidence distinct:

• Participation: Who had access to the learning and completed it?
• Demonstrated learning: Can the person make and explain the decision in representative practice?
• Application: Does that decision appear in actual opportunities at work?
• Consequences: What happens to missed issues, rework, delay, and the people involved?

An independent reviewer can score a sample without knowing whether it comes from before or after training. Where possible, use work records alongside participant accounts. Self-reports provide valuable information about experience and barriers; they should not carry the entire claim that behavior changed. Review agreement between raters so a number does not conceal disagreement about what good performance means.

Track exposure to opportunities. A supervisor who has not handled an ambiguous handoff cannot demonstrate transfer in that category. Missing opportunity data should remain visible. Converting it into a failure score would misrepresent both the person and the program. The same principle applies to a learner whose system access, staffing, or authority prevented the expected action.

Make the environment part of the intervention

Pair the course with an agreed handoff template, a manager response routine, and a brief review of the first attempts. If supervisors use the new routine and get penalized for taking time, that consequence belongs in the evidence. Ask leaders what they will change when a trace reveals that the workplace blocks the capability it has requested.

A comparison group or randomized rollout, when feasible, can strengthen the claim that the intervention produced change. A before-and-after improvement alone is vulnerable to shifts in workload, leadership, staffing, and the mix of cases. Even with stronger design, an operational result may reflect the whole package rather than the course in isolation. Describe the claim at the level the evidence supports.

What the trace should leave visible

This approach has limits. Observation can alter behavior; records omit informal work; a rubric can privilege a narrow definition of competence. Collect the minimum necessary information, avoid sensitive personal details, and invite participants to explain what the trace missed. The method should support improvement without becoming a hidden mechanism for ranking people.

The larger aim is to make learning accountable to the work it is meant to improve. A well-designed course can be one part of that system. The decisive evidence comes later: a person notices a cue they previously missed, weighs the situation, acts with adequate support, and can explain why. Follow that change carefully enough, and learning becomes something the organization can inspect, protect, and build on.

Bring a consequential learning problem to TLS ↗