5.6 LEVEL 3: EVALUATION OF ON-THE-JOB PERFORMANCE
5.6.1 Level 3 evaluation instruments help to analyze whether trainees have transferred the skills, knowledge and attitudes they have acquired through training to actual job performance. 
5.6.2 A Level 3 instrument collects data for the following questions:
a.  
Is the task, for which training was provided, performed on the job?

b.  
How confident are trainees in their ability to perform the task once training has been completed?

c.  
How often do the trainees perform the trained task?

d.  
Will on-the-job training reinforce the needs of the trainee or is formal training required again?

e.  
Additional comments? (should be open-ended questions)

5.6.3 While a Level 2 evaluation is carried out by training providers, a Level 3 evaluation requires some coordination between training providers and on-the-job instructors and supervisors.  A Level 3 evaluation identifies limitations and barriers to a trainee’s performance following the delivery of training.  Feedback from a Level 3 evaluation is used to revise training courses and programmes to ensure a better fit between training and job performance. 
A.  Training providers must:
a.  
ensure that all newly designed or revised terminal objectives are based on current job performance.  Without appropriate alignment of course materials with terminal objectives and competencies, a Level 3 evaluation cannot effectively identify gaps between a trainees performance on the job and performance required by terminal objectives ;

b.  
complete the appropriate steps to ensure training quality ; and

c.
review and analyze programme reports and modify training materials accordingly.  

B.  Procedure design service providers must:
a.  
ensure that all newly designed or revised training materials are based on required job performance and safety standards ; and

b.  
review and analyze programme reports and recommend modifications to training programmes as necessary..

5.7 LEVEL 4: EVALUATION OF RESULTS/IMPACT
5.7.1 Level 4 seeks to measure how training has benefited the organization affected. Level 4 evaluation is not always applicable because of the organizational differences in States worldwide (ANSP and State authority can be the same organization, or ANSP can be a State-authorized/recognized privatized company, or procedure design service provider can be a third party). In some cases there is no direct interaction between the procedure design service rrovider (sub-contracted work) and the State authority.
5.7.2 However, when applicable, statistics and reports are summarized to evaluate the overall impact of training on the organization especially as it relates to Safety Management.  A steering committee including those responsible for safety management should be established to carry out this level of evaluation. Based on performance and safety goals set by the organization, this level of evaluation measures how training supports achievement of these goals.  In this context, training is one component of a Safety Management System (SMS) that must be balanced with other organizational components. 
　
中國航空網 www.k6050.com
航空翻譯 www.aviation.cn
本文鏈接地址：飛行程序設計質量保證手冊(73)