Training evaluation data can be collected in many ways: questionnaires, one-to-one interviews, manager focus
groups and by observation of the training itself. However for busy learning and development professionals with
a variety of training events to evaluate (taking into account the interests of the differing stakeholders),
cost, efficiency and effectiveness are important considerations.
Questionnaires provide an easy way to distribute a standard set of questions to any number of respondents,
the response format lends itself to easy data collation and analysis, and they are also a lot more
cost-effective to administer than conducting interviews. As a result, questionnaires have become the most
popular method of gathering training evaluation data for effectiveness and impact analysis.
Training Evaluation Data Collection Strategy
In using questionnaires to gather training evaluation data, the focus is mostly on these three stages:
Pre-training-event: Obtain information on the gaps in employee skill and knowledge that needs to be addressed by the training intervention
Immediate post-training event: Obtain the training delegate's reaction to the
training as well as feedback from the course instructor
Post-training event follow up: Obtain feedback on skill and knowledge
application as well as organisational impact from the training delegates and their managers or supervisors
Order of Priority
Collecting and analysing data at each of these stages for each training event can easily become
overwhelming so a common approach is to divide evaluation into 2 categories.
Category One: Immediate post-event evaluation
This would take place for almost all training events, gathering data at the first two collection
stages mentioned above on course design content, delivery course administration, what specifically
has been learned and how the learner thinks they will apply the knowledge to the workplace.
This addresses the first two levels of Kirkpatrick’s well-known model and is useful to monitor
both the overall performance of the training function and the effectiveness of the training.
Category Two: In-depth evaluation
This level of evaluation is aimed at high cost, high impact training programs which are crucial
to business success. Data would be collected at all three stages and would seek to measure
training impact on all four of Kirkpatrick's levels.
Collecting the data
With questionnaires as the training evaluation data collection mechanism of choice, let’s take a look at the
options available for distributing questionnaires to respondents, as well as the pros and cons of each method.
The 2 main methods of distributing training evaluation forms are manual - paper and
electronic - e-mail and web forms.
Paper training evaluation forms
This method entails the forms being printed out and distributed by hand to the respondents who fill in their
responses and then submit the form to the administrator on completion. Once collected, forms can then be
scanned for automated analysis using a software application or manually processed using a spreadsheet or
similar program.
Care needs to be taken with the administration of paper evaluation forms to ensure that
respondents are not under pressure to complete them as this could limit the usefulness of
responses.
A common approach is for the instructor to demand that evaluation forms be completed
before training delegates leave the the lecture room.
This might cause delegates to give less considered responses or sometimes their mood might still be
elevated by a high-energy closing exercise and they may not be able to reflect properly on the overall
impact of the training. These outcomes are best avoided.
The pros of paper training evaluation forms
There’s no requirement for PC access – this means that not having access to a PC
cannot be used as an excuse for non-completion. This can lead to more evaluation forms being completed and handed back.
Guaranteed anonymity – As long as there is no obligation to fill in identifying
information, feedback collection using a paper evaluation forms will be anonymous. This makes it more likely
that reasonably honest and as such useful feedback will be provided.
The cons of paper training evaluation forms
Processing and analysis can be time consuming – Even with form scanning, responses
cannot always be reliably read and so may have to be manually reviewed. Also for comment type responses,
scanning software cannot often "read" hand writing so it might have to be typed in manually.
There are some specialist software applications that are able to scan written text and show it in
analysis reports as an image however, poor handwriting may cause the loss of some useful insight.
Not environmentally friendly – With the increasing importance of environmental considerations
in the workplace, printing out and distributing paper forms makes this a less of a popular option.
Email training evaluation forms
This method of distribution is usually implemented as either an html form in the body of an email
or an attachable form like a Microsoft Word or PDF form.
With this method, the respondent fills in the form by clicking on radio buttons and check boxes
to make response selections as well as typing in narrative text for comment type questions. Once complete,
the respondent sends the email back with the responses attaching the completed form where an
attached form has been used.
The pros of email training evaluation forms
Easy to distribute - Pretty much everyone in an organisation will have an email address making it
easy to deliver the evaluation form to relevant respondents.
Easy to process completed forms - The standard format of the evaluation form
and the supplied responses makes feedback processing and analysis easy to automate with software.
Respondent management - Because an email address is required to send the form, the
administrator can track who was sent the form and where it hasn’t been replied, send a reminder.
The cons of email training evaluation forms
Loss of respondent anonymity - It will be possible to identify a person’s responses by reading
their reply email. This loss of anonymity may mean that respondents are less forthcoming in their
feedback.
Can be cumbersome for the respondent - Using an html form in an email may be
problematic as most email clients disable html by default. This means that a well-designed and laid out
evaluation form will show up garbled when received by a respondent if html is disabled. Respondents will
therefore have to manually enable html for emailed evaluation forms.
Web training evaluation forms
This uses an online form which is accessed from a web URL which is sent to the respondent by
email. By clicking on the URL, the respondent is directed to the online evaluation form.
Much like the email method, the respondent completes the form by clicking on radio buttons and
check boxes to make selections as well as typing in narrative text for comment type questions.
On completion the respondent clicks on a button that submits their responses.
Submitted responses are saved to a central database which is used for analysis reporting.
The pros of web training evaluation forms
Replies are confidential - Because the URL is usually an auto-generated there is usually no easy
way to tie responses back to an individual.
Easy to process completed forms - The standard format of the evaluation form
and the supplied responses makes feedback processing and analysis easy to automate with software.
Common availability of email and web access - Because pretty much everyone in
an organisation has an email address and web access, it is easy to send the URL to the web form to respondents
and they in turn will have no problems accessing to the web form to complete it.
Respondent management - Because an email address is required to send the web
form URL, the administrator can track who was sent the link and where it hasn’t been replied, send a reminder.
The cons of web training evaluation forms
A web server is required - Using web training evaluation forms requires the extra effort of
configuring a web server to host web forms. This may be a problem if the organisation doesn’t already have one.
In Closing
Ultimately, it’s all about finding the option – or combination of options – that work best.
The particular training to be evaluated, the organisation's priorities, the preferences of learners
and their managers, and – at the end of the day – available resources.
Once questionnaires have been selected as the method of collecting training evaluation data, the
choice of distribution method – manual or electronic – as well as the tool for processing
completed forms – spreadsheet, survey or training evaluation software – will be determined largely
by an organisation’s circumstances.
Marblewire is a London, UK based software company.
We make Trellis, a Windows-based training evaluation software application designed to make it easy for
you to collect and analyse feedback on training events and training impact.