After developing and implementing several eLearning solutions in my workplace we are now at a point where we want to take some time to analyse the effectiveness of the training. To do this, we decided to develop a progress report to the stakeholders to gather the results from the course survey and insights on course participation, engagement, issues found, and next steps to conduct the training to a completion or closure process.
Remember that the idea for some of the courses is to have a refresher module later on, or to get to a closure so we can measure if the learning solution was effective and allowed us to achieve the learning objectives identified at the beginning of the consultation process with the stakeholders.
This evaluation phase is relevant because it will bring critical information about how learners perceived the learning tool and how we can improve future projects. From this evaluation, we can get valuable information that will create the design criteria for the next eLearning solutions. So it is important to have an organised approach to collate the data and get the best possible information for our own learning as instructional designers.
An organised and practical approach to do program evaluations in the corporate sector is to use Kirkpatrick’s levels of evaluation (Simonson, 2008). I have prepared the table below to summarise Kirkpatrick’s evaluation model and I have added levels 5 and 6 to adapt it more to my work. This tool helps me in understanding the instruments that can be used to gather data on each level.
|Level of Evaluation||Instruments to Gather Data|
| Level 1 – Reactions or Learner Satisfaction
(How users perceive the module, what did they like or not about the training? This can be measured with the course survey)
|Survey with questions to identify what people like and didn’t like about the training module, questions to determine how learners perceived the training.Examples:
Rating type of questions (Strongly disagree, disagree, neutral, agree, strongly agree):
How likely are you to use the information of the course?
How likely are you to share the information of the course?
What did you enjoy most and least about the module?
What would you change (add, remove, improve) in the module?
Would you like to add any other comment?
| Level 2 – Learning
(Have learners advanced in skills, knowledge or attitude? This can be measured with case scenarios, quizzes, completion rate, post-test scenario with questions in a refresher type of training, check if learning objectives defined during consulting process were achieved)
Check if learning objectives were achieved.
|Level 3 – Transfer or Application of Knowledge
(Are learners using the knowledge in the workplace? )
|Level 4 – Results / Impact of Training on the Organisation
(This is the direct and indirect impact of training on the success of the organisation. This can be measured with the business KPIs, comparing results of pre-test and post-test. Impact most be measurable)
|Level 5 – Usability, Technical issues, Accessibility
(This can be measured with the course survey)
|Survey about content, layout, interactivity, presentation, length of course, resources, etc.|
|Level 6 – Return on Investment (ROI)||
Simonson, M. (2007). Evaluation and distance education, five steps. The quarterly review of distance education, Vol 8(3). Retrieved from Kirkpatrick – Evaluation and Distance Education.