Kirkpatrick model is one of the highly recognized and widely used training evaluation model. It was developed by Dr. Donald L Kirkpatrick (1924-2014). It is one of the most effective models to analyse and evaluate the results of educational programs. It can objectively analyse the impact and efficacy of training. As it proceeds, the evaluation process gets more difficult and time consuming. However, the higher level assessments also generates information that is more critical and valuable.
The four levels of Kirkpatrick model (Source – kirkpatrickpartners.com)
By analysing each of these four levels, it is easier for a trainer to evaluate an effectiveness of training and find the ways to improve the future trainings. The four levels of the evaluation model are as follows:
- Reaction evaluation – Training participant’s opinion about the training and the trainer – The personal thoughts and the feelings are captured quantitatively through responses in a questionnaire (typically termed as ‘smile sheets’ or ‘happy sheets’). Questionnaire analyses the training content, methodology, facilities and the course content. Learners also respond to their first reaction to learning experience.
- Learning evaluation – The extent of learning after the training – It measures the personal development of the trainees by analyzing the increase in knowledge, the acquired skills or enhanced intellectual capabilities.This is assessed before and after (pre-test & post-test) the training so as to ascertain the scale at which learner has gained the knowledge. The evaluation involves observation and analysis of the voice, behaviour and text of the trainee. The measurement at this level gets more difficult and laborious as the participant’s evaluation moves from learner satisfaction to learner’s knowledge advancement.
- Behavioral change evaluation – The extent to which the trainees applied the acquired knowledge and changed their behavior. This change can be immediate or several months after the training depending on the situations. Behavior evaluation analyses the transfer of acquired knowledge from the training session to the work place. Here, the primary tool for evaluation is predominantly the observation. Apart from the observation, a combination of questionnaires and 360 Degree feedbacks are also used. It is rather difficult to predict the change in behavior and hence, the evaluation process gets even more difficult. It requires important decisions in terms of when and how the trainees should be evaluated.
- Result evaluation – To assess training in terms of business results. It is measured by assessing the change in key performance indicators of business which involves, achievement of standards and accreditations, number of complaints, profit and loss statements, business volumes, etc. However, since all these factors are also affected by several other external factors it gets difficult to quantify the training impact on business results. This stage helps in identifying the ROI (Return on Investments) of the training.
In the context of trainings through UJJWAL at CIDCO, the training cell team captures the relevant information to evaluate the reaction of the trainees, which is the first level of the Kirkpatrick model. A feedback form that captures the reactions of the trainees is filled by them immediately after the training is over. 85-90% of the submitted participant’s feedbacks have already assessed the institute vis a vis faculty or the Subject Matter Expert (SME), relevance of the course, course content, training methods and other faculties. NIUA-CIDCO Smart city lab also incorporates the second level of the Kirkpatrick model. However, currently this is only being done for high end courses. At this stage, participants are asked to submit a brief on their learning in a pdf or doc version, so as to qualitatively assess the enhanced knowledge of the participant.
It is strongly believed that many of the participants in CIDCO have started implementing the knowledge gained during the training in their professional and personal lives. This can be evaluated through methods of psychometric assessments or 360 degree assessments in the third stage of this model. However, the tools to quantify the change in the application of the knowledge are still in development stages and can be assessed only in a larger group of participants over a period of 12-18 months. As the trainings gets more amenable in the coming months, UJJWAL’s training cell aims to take its evaluation process to the next stages of Kirkpatrick model. By doing so, it intends to measure a system wide impact measurable in terms of the people, the processes and the business of CIDCO.