The Canadian Review of Policing Research (2004)
ISSN: 1710 6915
PPO START: AN EVALUATION OF THE PROBATION AND PAROLE OFFICER BASIC TRAINING PROGRAM1
Introduction
In the fall of 2001, a new process for training Probation and Parole officers (PPOs) was implemented in the Ministry of Community Safety and Correctional Services as part of the Ministry’s Strict Discipline Model. It was designed to ensure that new PPOs have requisite training prior to providing direct supervision of clients to ensure the protection of the public. The training focused on skill development and reinforced principles of effective correctional intervention2 .
The training program was divided into three main sections. Phase 1 consisted of six weeks of in-class training that provided participants with the knowledge and skills they require to be “job-ready”. A four to six-month field experience portion followed, which allowed participants to apply their learning from Phase I to supervising actual offenders. Phase 2 followed with an additional two weeks of in-class academic/theoretical training, focusing on special need offender populations.
The following outlines results of a 12-month program evaluation of the Probation and Parole Officer Basic Training program.
Methodology
The evaluation was multi-method and multi-source. Information and data were gathered from a number of different sources (participants, area managers, trainers) and in a number of different ways (behavioural observation, paper and pencil tests, attitudinal/reaction measures, and focus groups). In evaluations such as this, it is more typical for organizations to evaluate using only satisfaction measures. Multiple measures and sources of information provide optimal amounts of validity.
Prior to beginning training in Phase 1, participants completed a background questionnaire, which provided details about their education, work and previous training experience. In Phase 1, the evaluation measures administered after each training module included participant reaction measures (PRMs) designed to measure alignment with program objectives, as well as participant reaction and satisfaction with the training experience. Content knowledge tests were administered by trainers to measure the level of knowledge acquisition. Finally, behavioural role plays were also included in this first phase of training in order to assess whether participants were able to demonstrate behaviourally the new skills they had been taught. Behavioural assessment is one of the most valid forms of assessing learning and skill acquisition.
In the field experience portion of the training, participants completed anonymous reaction measures about how well their in-class learning had prepared them for the practical experience of working as a PPO. Area managers completed a behavioural checklist based on their own observations and audits of participant’s skills in working with clients. Participants also completed a similar self-evaluation form of their own skills. In Phase 2 of training, only PRMs were administered to participants at the end of each day.
Halfway through the evaluation period, a nominal group technique (NGT) was completed with both area managers and Bell Cairn Staff Development Centre3 trainers. An NGT is a type of focus-group process that systematically identifies and ranks any problems with program implementation and continuity, and prioritizes optimal solutions. Hurdles and potential solutions with implementing the training program were discussed, which in turn, led to some of the concluding recommendations for improvement.
Findings
In total, 41 participants in three separate groups training from January to June 2003 participated in the evaluation. Thirty (30) of the participants were female and 11 were male. The women were slightly older (M=33.5 years) than the men (M=32.9 years); however, the difference was not significant. All participants held bachelor degrees (most commonly in criminology or psychology) and over half had related work experience in the federal system and with young offenders.
The participant reaction measures were scored on a five-point, Likert-type scale ranging from 1 – “Strongly disagree” to 5 – “Strongly agree”. In addition to questions specific to the content of each day, there were five statements common to each PRM that were of most interest for evaluation purposes. The five common statements were:
“The material covered in this module was new to me”4
“I do not need additional information to feel comfortable with this topic”
“The pace with which this topic was covered was about right”
“The training methods (lecture, group work, discussions, role play, practical exercises, video) used in this module were effective”
“I am comfortable applying this knowledge to my future work setting”
Results indicated that the highest mean agreement ratings (80.6%) were given for the statement “The training methods (lecture, group work, discussions, role play, practical exercises, video) used in this module were effective”, indicating an overall satisfaction with the type of training being administered. Participants consistently reported in their written comments how useful hands-on exercises and practice sessions, including role-plays, case studies, and computer use had been in reinforcing the skills being taught. The lowest mean agreement rating (57.8 per cent) was given for the statement “The material covered in this module was new to me”, indicating that content material was generally not original to participants and they had probably become familiar with it, either through their previous education or employment experience. Results are provided in figures 1 and 2 respectively.
Analysis of the results of the content knowledge tests administered to participants over the course of the six weeks, shows the marks tended to be exceedingly high, with very small standard deviations, indicating very little variance among subjects (almost everyone tended to do very well). In total there were eight exams administered over the six weeks to each of the three groups. The average test mark was 85.95 per cent (SD = 5.25) with the range of marks being 73.33 per cent to 95.84 per cent. This might indicate that either the exams were too simple and needed revisions to better tease out differences in participant knowledge, or that content material being taught was not really that different or new from what participants already knew, based on their past educational or work experience.
The original evaluation plan, had stipulated that behavioural role plays would be developed and administered both as a way to practice the skills taught, and as a method of testing participants. The evaluators were to develop the role-play scenarios and strict scoring guidelines to ensure validity. Role-plays were to be videotaped for each participant, scored by trainers, and forwarded to the evaluators to provide a measure of inter-rater reliability. Unfortunately, owing to misunderstandings and role confusion in the early stages of this evaluation, the evaluators were not included in the creation of the role-plays, and scoring guidelines were not developed. This needs to be part of the design of any future evaluation effort.
For the field placement portion of the training program, PRMs were mailed to participants. There was a 46% response rate (19 of 41 returned). Most (79 per cent) respondents were satisfied with their field experience and the support they were receiving from their managers and staff. Conversely, most (58%) responded neutrally or disagreed that the six-week in-class portion of the training had been useful in preparing them for work in their offices. Area managers tended to rate their trainee’s mastery of skills slightly lower than participants rated themselves. This was probably based on the notion that the participants were still in the learning phases of their careers. Participants in groups 1 and 2 tended to be somewhat more confident in their self-assessments, with most saying they had “mastered” many of the skills on the checklist. Results are provided in Figure 3.
Finally in the last phase of the training program, PRMs were administered once again. Topics for Phase 2 included sex offenders, partner abuse, victims, core programming, adolescents and crisis/suicide intervention. With some exceptions, the presentations were generally well accepted by participants. Participants tended to feedback that they would have preferred smaller group sizes for this phase of training, and more hands-on practice as in Phase 1. Many reported that they had the information but did not have the interpersonal skills for dealing with these populations one-to-one. Another common theme was that much of the information should have been included in the earlier training.
Conclusions and Implications
The evaluation of the first phases of the PPO Basic Training program has garnered a great deal of information from a variety of sources. Clearly, there is no “one size fits all” model for this type of training, and training participants were a key source of information for their own training needs. At this point, tentative conclusions can be drawn on the success of the training to date, and suggestions made regarding future directions.
- Make use of self-directed training models. There are multitudes of tools available for self-directed learning, including Internet, CD-ROMs, on-line courses, in addition to traditional texts and college courses. These options could replace the notion of traveling to Bell Cairn Staff Development Centre in Hamilton (especially for those outside of the Toronto region) for such extended periods of time. Any identified weak areas could be woven into a training plan that would then be individualized for each participant. The developmental cost of this option pays off in about four to five years.
- Develop on-going and shorter training for very basic elements that can be easily and frequently accessed
- Create “opting-out” exams for more experienced individuals or alter training time for topics with which participants with PPO experience would already be familiar, so that a relatively experienced individual is not placed in the same length of class as a new PPO.
- Provide area managers with refresher training on interview skills to encourage their involvement in staff interviews as part of their audit process and as a measure of overall quality assurance.
Endnotes
1. V. Léger, C. Miller, R. Mayoh, J. Wright, N. Mazaheri, J. Rettinger, and I. Sauvé-Thompson. PPO START: An Evaluation of the Probation and Parole Officer Basic Training Program, Toronto: 2003, Ontario Ministry of Community Safety and Correctional Services.
2. D.A. Andrews, and J. Bonta, The Psychology of Criminal Conduct, 2nd ed., Cincinnati, 1998. Andersen Publishing Co.
3. The Bell Cairn Staff Development Centre is the location where the PPO training took place.
 |
 |
 |
 |
 |