Does Training Work? Measuring Results Beyond the Attendance List

A training ends and the report arrives: an attendance list, photos and a satisfaction score of 4.5 out of 5. For many HR teams, government training units, NGOs and CSR teams, that is all the evidence there is. It proves that the training took place, not that anyone learned, worked differently or achieved better results. Measuring training results means answering those questions

This article covers the Kirkpatrick and Phillips models, evaluation planning and a hypothetical example from a BUM Desa bookkeeping training

Why Attendance Lists and Satisfaction Forms Are Not Enough

An attendance list counts an output: the number of people trained. Training is funded for the outcomes that should follow, such as staff who keep accurate records. A Logical Framework draws the same line between what a program delivers and what changes as a result

Satisfaction forms have their own limit. Donald Kirkpatrick defined reaction as how well participants liked a program and stressed that it does not measure learning: even a well run reaction survey gives no assurance that anyone learned or that behavior will change

Research agrees. A 1997 review of 34 studies in Personnel Psychology found that participants' views on how useful a training was were more strongly related to learning and on the job performance than how much they liked it

The Kirkpatrick Model: Four Levels of Training Evaluation

Donald Kirkpatrick based the model on his 1954 PhD dissertation at the University of Wisconsin and set it out in four articles that began in 1959, one per level. Kirkpatrick Partners, the firm that carries on his work, says it moved the field beyond attendance and satisfaction

In the 2010s, Jim Kirkpatrick, Donald's son, and Wendy Kayser Kirkpatrick updated it as the New World Kirkpatrick Model. It adds engagement and relevance to Level 1, confidence and commitment to Level 2, critical behaviors and required drivers (the coaching, job aids and recognition that reinforce new behavior) to Level 3, and leading indicators, short term signs that behaviors are on track, to Level 4. Planning starts from Level 4 and works back. The firm's 2026 update adds the performance environment, the workplace conditions that support or block progress at every level

Adding a Fifth Level: The Phillips ROI Methodology

Kirkpatrick's fourth level identifies results but puts no money value on them. Jack Phillips, now chairman of the ROI Institute, added a fifth level, return on investment (ROI), in a 1983 handbook. His levels are Reaction and Planned Action, Learning, Application and Implementation, Business Impact, and ROI. The institute says non profits, NGOs and public agencies now use it too

The method isolates how much of an improvement came from the program (through a comparison group, trend analysis or estimates), converts it into money and compares it with fully loaded costs, including needs assessment, participation and evaluation. Benefits that cannot be credibly converted are reported as intangible. ROI is net benefits divided by costs, shown as a percentage. For illustration only: a program costing Rp100 million that yields Rp150 million in benefits has an ROI of 50%

When Calculating ROI Is Worth the Effort

Because an ROI study takes time and money, the ROI Institute suggests evaluating 90 to 100% of programs for reaction but only 5 to 10% for ROI, choosing programs that are expensive, long running, strategic, highly visible, large or closely watched by management. A short, one off workshop rarely justifies it, nor does a program whose main benefits, such as trust or confidence, cannot be credibly priced. The institute itself notes that intangible benefits are in many cases more important than a positive ROI

Kirkpatrick Partners suggest agreeing with stakeholders in advance on what success means (Return on Expectations) and reporting the training's contribution rather than claiming sole credit. When benefits fall mainly on a community, SROI may fit better

How to Plan Training Evaluation at the Design Stage

Evaluation works best when it is planned with the training, as part of the program's monitoring, evaluation and learning plan. Five steps:

1. Analyze the training need. Start from the performance problem. Kirkpatrick Partners warn that poor performance is often blamed on missing knowledge or skills when the more common cause is motivation or the work environment, which training alone will not fix
2. Tie objectives to work. State what participants should do differently, such as “records every cash transaction on the same day”, not only what they should know
3. Choose one or two indicators per level and agree on them with the funder or management at the start
4. Collect a baseline. Test knowledge and record current practice before the training
5. Schedule the follow up. Check in within the first weeks, since Kirkpatrick Partners advise against waiting 90 days. Review behavior again after 3 to 6 months (Kirkpatrick's original advice was 3 months or more) and results after 6 to 12 months

Indonesian civil service rules already look beyond attendance. Peraturan LAN Nomor 10 Tahun 2018, issued by the state administration agency LAN, asks government agencies to assess whether competency development, which includes training, improved the competence and performance of civil servants (Article 38). For the basic training of civil servant candidates (CPNS), Peraturan LAN Nomor 1 Tahun 2021 provides for an evaluation 6 to 12 months after the training to see whether participants keep applying what they learned at work (Article 37). Both are listed as in force in the BPK legal database

Keeping It Light: Tools to Measure Training Results

Small programs need a few well chosen measures, collected consistently, not a research design:

• Replace long satisfaction forms with three or four questions on relevance, usefulness and intended use
• Test practical skills with a practical task, such as filling in a real cash book
• Fold behavior checks into existing visits or calls, and ask a supervisor or peer too
• Use existing records, such as reports or program data, before designing new surveys
• For large groups, follow up a random sample well rather than surveying everyone poorly

As the ROI Institute advises, measure the lower levels too whenever you report a higher one. Otherwise you cannot explain why results did or did not appear

A Hypothetical Example: Bookkeeping Training for BUM Desa Managers

This example is hypothetical and for illustration only. A district program trains 30 managers of village owned enterprises (BUM Desa) in basic bookkeeping over three days. Before the training, 8 of the 30 BUM Desa keep an up to date cash book. Indicators could be:

• Reaction: at least 80% of participants rate the content as relevant to their BUM Desa, on a short form at the end of each day
• Learning: the average score on a practical test, recording 20 sample transactions and preparing a monthly summary, rises from 40 to 75 out of 100
• Behavior: after 3 months, at least 20 of the 30 BUM Desa update their cash book weekly, checked on a visit and with the BUM Desa supervisor or village head
• Results: within 12 months, at least 20 BUM Desa complete their annual financial report on time, with cash balances that match the books

A monthly call from a district facilitator and a simple template support the new routine. If BUM Desa that did not take part improved as well, the program should not claim the whole gain. A full ROI study would not be worth it here: the main benefits, such as clearer accountability to the village, are hard to price

Conclusion

Measuring training results means asking more than who attended and whether they enjoyed it. Start from the results you need, collect a baseline, check behavior within weeks and again after 3 to 6 months, and save ROI for the few programs that justify it. Even one or two indicators per level give far better evidence than an attendance list

Training & Skills Development is one of SIDIA's six program areas and focuses on customized, competency based training for government bodies, communities, NGOs and the private sector