Executive Summary
This independent advice considers the standards and practices the New Zealand Police use to assess Recruits up to Probationary Constable level. The advice examines how Police have determined minimum standards, how they have chosen their assessment methods and how they ensure the assessment process is nationally consistent.
Determination of minimum standards
The first question considered was whether "A systematic and robust process was used to identify the minimally acceptable knowledge, skills, attributes and other personal characteristics required of Police Recruits, Probationary Constables and Constables".
It appears that Police have applied robust requirement identification processes to key elements of the policing role (for example, conducting criminal investigations, responding to traffic emergencies, conducting lawful interviews, and physical fitness). However, this appears to have been ad hoc with no evidence of any attempt to determine the full breadth and relative importance of the knowledge, skills and abilities required of Police Recruits.
While the major job requirements seem to have been covered, in absence of a systematic process it is not possible to state definitively that the minimally acceptable standards Police are using for assessment accurately reflect the complete requirements.
Assessment methods
The second question considered was whether "A systematic and robust process was used to identify the assessment methods".
While the range of assessment methods is broad, no evidence was provided of a systematic process being used to confirm that the range of assessment methods covers all of the necessary knowledge, skills and attributes. The assessment tools appear to have been chosen in response to particular needs, rather than as part of an overall integrated plan. In addition, processes to ensure these methods remain consistent with organisational requirements are not in place, risking a reduction in the relevance and effectiveness of assessment procedures over time.
In terms of the specific assessment methods, four of the fourteen assessment methods used to support the recruitment decision do not appear to meet baseline standards. The reliability and validity of these four tools is uncertain, and they lack soundly-based user guidelines and pass marks - severely limiting the usefulness of these specific tools. Of particular concern are the Behavioural Interview and the processes used for setting Reasoning Ability pass marks.
These four tools may contribute to a higher than necessary variability in the quality of people entering the Police College - in terms of recruiting people who require more assistance than desirable and also missing out on potentially good Recruits.
However, all of the methods used post recruitment - the checks and balances that ensure the quality of people appointed to Probationary Constable status and subsequent permanent appointment - appear to have been subject to robust processes and are considered to be sound and reliable. While there are some gaps in the rationale supporting pass marks, links with measures of performance and ongoing review, these tools are considered capable of supporting permanent appointment decisions.
Consistency of application
The third question considered was whether "Comprehensive and sound procedures have been used to ensure consistent application of assessment methods".
Five of the fourteen tools used to assess applicants (i.e. prior to someone entering the Police College) lack sound mechanisms to ensure nationally consistent administration, scoring and interpretation of results, reducing the ability of these tools to ensure that the quality of Recruits is nationally consistent. This may contribute to some variability in the quality of people entering the Police College from different districts.
In contrast, the assessment tools used post-recruitment (i.e. appointment to Probationary Constable status and subsequent permanent appointment) have sufficient mechanisms to ensure consistent application across the country.
We also note that assessment data recording errors and the lack of a central recording mechanism limit the ability of Police to accurately monitor assessment trends and to conduct empirical research to understand and improve the effectiveness of the assessment tools.
The reasoning ability of Recruits
In an adjunct to this advice, Professor John Hattie was asked to examine whether or not there has been a reduction in the reasoning ability of recent Recruits. Professor Hattie states:
"From this review, it is concluded that while there is evidence that the minimum entry criteria on the Reasoning tests has declined, there is no evidence to believe that the overall performance of graduates from the training program has changed. It is not clear that any decline is attributable to the change in assessment, but is more likely related to decisions about the standards/ Guidelines/Rules of entry." (2007, p. 26)
Professor Hattie's full report is given in the Appendix.
Conclusion
On balance, the assessment processes used by Police seem capable of ensuring that those achieving permanent appointment as Constables are able to perform the job. Primarily this is because of the multiple hurdle approach adopted by Police, whereby weaknesses in any one aspect of the assessment process are tempered by other, subsequent, parts of the process.
Indeed, the demanding and broad range of tests that aspiring Constables must pass are significantly greater than those required for entry to most New Zealand organisations.
The weaknesses identified are more likely to have an impact on the overall efficiency, rather than effectiveness, of the appointment process. Potential impacts include devoting resources to people who will ultimately fail, devoting additional resources to people who take longer to train than would normally be expected, and also missing out on potentially suitable Recruits.
Conducting a systematic and rigorous analysis of requirements, tightening some of the assessment methods and enhancing the consistency of application are likely to increase the focus of the recruitment process and, potentially, reduce costs.
Selecting staff is not an exact science. Police can improve, but they are not doing badly.
