The Canadian Review of Policing Research (2004)
ISSN: 1710 6915
A MEASURE OF POLICE PERFORMANCE: ANALYZING POLICE CLEARANCE AND CHARGE STATISTICS
This review summarizes ideas and findings from a master’s thesis1 and a journal article2 on the use of a traditional police statistic, the clearance ratio, as an indicator of police performance. Traditional indicators of police performance such as the clearance ratio have been criticized in many respects over the last two decades. Indeed, the clearance ratio has many weaknesses: it fluctuates under the influence of factors not related to police work3 , such as differences in police notification, unequal crime mix among police departments, and availability of suspects); it has been documented that some police organizations manipulate their statistics to artificially improve their performance4 ; it focuses only on catching criminal suspects, while police functions are much more diversified and complex5 , and so on. However, credible alternatives have not yet been developed. In contrast with many scholars who simply disregard the variable, we argue that a better understanding of the clearance ratio can, in fact, lead to the development of new police performance indicators.
The general goal of our research is to elaborate a new measure of police performance for solving crime. This new measure is based on a better understanding of factors that affect crime solving by police departments, for example, the clearance ratio. The analysis of the clearance ratio targets two “levels” of explanatory variables: (1) the unequal crime mix among police departments, and (2) the differential context in which police departments operate, namely the crime rate, the level of poverty in the community, the size of cities, and police resources. In other words, the goal of this research is to investigate how some variables not related to police performance influence the clearance ratio of police departments, so that a new coefficient purged of these “biasing effects” can be created. This coefficient is therefore a more adequate and more meaningful indicator of the ability of police departments to pair apprehended suspects with specific criminal offences committed in their jurisdictions. Since the identification (and apprehension, when appropriate) of criminal offenders is an important function of the police6 7 , the “purged” coefficient proposed in this research can be a useful variable to assess police effectiveness at this function.
Methodology
We gathered data from 1995 to 1998 about 116 police departments8 in the Province of Quebec. Crime data from the Revised Uniform Crime Report (UCR2) were obtained, which included information about crimes solved according to types of crime. Statistics Canada offers an official publication about characteristics of police departments (such as the number of sworn officers and other employees, the annual cost of operation, and the number of crimes committed in a department’s jurisdiction), which was also used. Our third source of data was the Census (1996), from which we obtained information about the social, demographic, and economic environment of municipalities where police departments are located. Police forces that were too small (i.e. less than 1000 incidents recorded) were excluded because data are more available and more reliable for bigger police departments and municipalities.
The dependent variable is the observed clearance ratio of police departments. The clearance ratio is the number of cases solved, either by charging a suspect or other outcomes (death of the main suspect, victim refuses to file a complaint, reasons independent of police, and so on), divided by the total number of criminal incidents known to the police. For example, if a police department registered 10,000 criminal incidents and solved 5,000 of them, then its clearance ratio is 50 per cent.
Our main independent variable is the expected clearance ratio taking into account the specific crime mix of every police department. We estimate this variable by combining the average clearance ratio of different types of crime for the whole sample (for example, homicide, 66.8 per cent, assault, 85.4 per cent, auto theft, 11.1 per cent) with the relative proportion of these crimes committed in every police territory. For example, if a hypothetical police department has 100 assaults and 100 auto thefts, we expect this department to have a clearance ratio of 48.3 per cent ([85.4 * 50%] + [11.1 * 50%]). If a police department has a bigger observed clearance ratio than our prediction based on the crime mix, it suggests this department is particularly effective at solving crime, and vice versa. Following that logic, a department with a 30 per cent observed clearance ratio but a 25 per cent expected clearance ratio would in fact be more effective than a department with 50 per cent observed clearance ratio but a 55 per cent expected clearance ratio (because the crime mix of the first department makes it much more difficult for them to solve crimes than the crime mix of the second department). However, past studies indicate that the unequal crime mix among police departments is not the only variable affecting the clearance ratio: social, demographic, and economic variables of municipalities also have their impact. Consequently, we included the unemployment rate, the municipal population size, the percentage of persons 15 to 24 years of age in the population, the crime rate, the number of police officers for every 1,000 crimes, and other related variables as independent variables.
A Pearson’s correlation was first used to confirm the fit between the observed clearance ratio and the expected clearance ratio based on the crime mix. Then we subtracted the expected clearance ratio from the observed clearance ratio (a positive number means the department is doing better than expected, a negative number means it is doing worst). An OLS regression is then used to predict this new coefficient (observed – expected clearance ratio) with the social, demographic and economic variables. The residuals from this OLS regression were actually the proposed indicator of police performance: a new coefficient based on the clearance ratio that controls for the unequal crime mix and the unequal environment among police departments.
Findings
By far, the strongest predictor of the observed clearance ratio was the crime mix, namely the expected clearance ratio. In fact, we explained 82 to 84 per cent of the variance of the observed clearance ratio with the difference in crime mix among police departments. The OLS regression indicated that the differential between the observed and expected clearance ratio significantly increased with the percentage of 15-24 year-olds in the population and the number of police officers per 1,000 crimes, while it decreased with the number of residents and average income of municipalities. Overall, the social, demographic, and economic characteristics of municipalities predicted 36 per cent of the variance of the differential between the observed and the expected clearance ratio. The master’s thesis version of the study offers replications and robustness checks of the results by looking specifically at violent and property crimes: results are similar.
Conclusions and Implications
There are many indicators of police performance and effectiveness, every indicator having some advantages and weaknesses.9 Crime and victimization rates, crime solving and arrest ratios, citizen satisfaction, and many other variables are often used to assess different aspects of police work. Traditional measures such as the clearance ratio are often criticized by scholars but used by law-enforcement and governmental agencies to assess police performance. A main weakness of the arguments proposed by critics is their general inability to formulate credible and usable alternatives to traditional measures, explaining why these measures are still popular despite their well-known issues. In contrast with these critics, we propose to solve some of the problems associated with the clearance ratio to make it a more valid and meaningful measure of police performance.
First of all, our results confirm that the raw, observed clearance ratio should not be used to assess the effectiveness of police departments: most of the variance of this variable is attributable to factors outside police control. More specifically, police departments with relatively difficult to solve crime mixes are strongly disadvantaged in comparison to police departments with easier to solve crime mixes. Also, to a lesser extent, police departments operating in an environment with more 15-24 year olds in the population and more police officers per 1,000 crimes appear to have an easier time solving crime, and police departments in an environment with higher average income and a bigger population seem to have more difficulties at solving crime10 .
Our research has limitations. First, we must acknowledge that however sophisticated a crime-solving indicator might be, it still only provides a partial assessment of overall police performance. Globally, police functions are too complex and too diversified to be evaluated with a few statistics. However, we believe that crime solving, if correctly measured and interpreted, provides interesting information about the ability of police departments to enforce laws, which is one of their principal functions. Second, our method is useless if police departments manipulate their statistics. Yet, better accountability and better control of police statistics is not an impossible task. There are other problems with police performance that our research cannot resolve11 , but we consider that if the clearance ratio or other similar measures (arrests rate, conviction ratio, average citizen satisfaction) are to be used to estimate police performance, it is essential to control the effects of variables not related to what the police really do.
Endnotes
1. Paul-Philippe Pare, «Performance policière et taux d’élucidation de la criminalité: Analyse comparative de 112 services de police au Québec de 1995 à 1998» , Master’s Thesis, Montréal : 2003, Université de Montréal.
2. Marc Ouimet and Paul-Philippe Paré, «Modéliser la performance : Comment analyser les statistiques policières d’élucidation et d’accusation», Revue Internationale de Criminologie et de Police Technique et Scientifique, 2003, pp. 23-42.
3. M. A. Walker, “Do We Need a Clear-Up Rate?”, Police and Society, 1992, 2(4), 293-306.
4. R. Reiner, “Process or Product? Problem of assessing individual police performance”, in Jean-Paul Brodeur, How to recognize good policing : Problems and Issues, London: 1998, Sage Publications. pp. 55-72.
5. D. H. Bayley, “Measuring Overall Effectiveness”, in L. T. Hoover, Quantifying Quality in Policing. Washington: 1996, Police Executive Research Forum. pp.37-54
6. American Bar Association, “Major Current Responsibility of Police”, Standard 1-2.2 in Standards Relating to the Urban Police Function, 2nd ed., Boston:1980, Little, Brown, pp. 1-31 to 1-32.
7. Service de Police communautaire urbaine de Montréal, La police de quartier, Montréal: 1996, Official Publication
8. 112 in the master thesis; we discarded four departments because of data reliability concerns
9. D. H. Bayley, “Back from Wonderland, or toward the rational use of police resources”, in A. N. Doob, ed., Thinking about Police Resources, Toronto: 1993, Centre of Criminology, University of Toronto, pp. 1-34.
10. The positive relationship between the number of police officers per 1000 crimes and crime solving supports the idea of workload pressure. The negative relationship between the size of the population and crime solving supports the idea of greater anonymity for offenders in bigger cities. If we suppose that the proportion of those 15-24 years old in the population is an indicator of the proportion of crime committed by 15-24 year olds, the positive relationship between the proportion of 15-24 year olds and crime solving may results from the less professional and organized criminal activities of this age group. Concerning the negative relationship between average income of municipalities and crime solving, we have two very different hypotheses. The first one is about victim-offender relationships: if crime in higher socio-economic communities is more likely to involve strangers, and crime in lower socio-economic status communities is more likely to involve people who are somehow related (family, acquaintance, etc.), then it might explain why police solve less crime in municipalities with higher than average income. The second one, a Marxist hypothesis, would also predict this negative relationship between income and crime solving, arguing that law enforcement agencies are tougher in poorer communities and more lenient in more affluent ones.
11. See for example, Jean-Paul Brodeur, How to recognize good policing: Problems and Issues, 1998, Sage Publications, for an instructive discussion of problems and issues with the evaluation of police performance.