Thursday, July 9, 2009

Limitations of NCVS Data

Sampling and Non-sampling Error
As described above, the NCVS estimates national crime rates by interviewing a sample of
households across the country. Because the NCVS is a sample survey, it is subject to both
sampling and non-sampling error, meaning that the estimated victimization rate might not
accurately reflect the true victimization rate. Whenever samples are used to represent entire
populations, there could be a discrepancy between the sample estimate and the true value of what
the sample is trying to estimate. The NCVS accounts for sampling error by calculating confidence
intervals for estimated rates of victimization.238 For example, in 2000, the estimated violent crime
victimization rate was 27.9 victimizations per 100,000 people aged 12 and older.239 The
calculated 95% confidence interval240 for the estimated violent crime victimization rate was 25.85
to 29.95 victimizations per 100,000 people aged 12 and older.241

The NCVS is also subject to non-sampling error. The methodology employed by the NCVS
attempts to reduce the effects of non-sampling error as much as possible, but an unquantified
amount remains.242 Non-sampling error can result from respondents not being able to recall
victimizations that occurred to them during the reference period. Non-sampling error can also
occur when respondents do not report crimes to the interviewer. Respondents may not report
crimes because they know the perpetrator or because they are victims of certain crimes frequently
enough that they forget that they were victimized or they do not consider the victimizations
important enough to report.243

Sampling Bias
The NCVS relies on interviews with household members to collect its data, but it is likely that
some people sampled will not complete the survey. As shown in Appendix G, between 1996 and
2005, anywhere from 9% to 16% of people included in the sample did not complete the survey. If
non-responders differ from responders in the number of victimizations they experienced, the
estimated national victimization rate might be lower or higher than the true victimization rate.
Researchers have reported that homeless people, young males, and members of minority groups
are less likely to be included in the NCVS sample and have higher rates of victimization than
their older, female, non-minority counterparts.244

Series Victimizations
When a respondent experiences six or more similar but separate victimizations, and when the
respondent is unable to recall the details of each incident well enough to describe them to the
interviewer, the interviewer completes one incident report to cover the series of incidents.245 The
incident report is completed using information from the most recent incident. BJS does not
include series victimizations when it calculates annual victimization rates.246 However, BJS does
use series victimization data in special reports where series victimizations are an important aspect
of the subject being analyzed (e.g., domestic violence).247 In these instances, one series is counted
as one incident.

It is likely that not including series victimization estimates in the annual victimization estimates
would result in an underestimation of the true national victimization rate. But the question
remains, how large would this effect be? Each year, BJS includes data on the number and
percentage of total victimizations reported as series victimizations. As shown in Figure 1, the
percentage of personal crimes reported as series victimizations ranged from 3.5% in 2001 to 4.7%
in 2003 and 2004. The percentage of property crimes reported as series victimizations was
consistent from 2001 to 2004 (0.8%) and decreased to 0.5% in 2005.

Figure 1. Percentage of Victimizations Reported as Series Victimizations, 2000-2005
Percentage of Victimizations Reported as Series Victimizations, 2000-2005
Source: U.S. Department of Justice, Office of Justice Programs, Bureau of Justice Statistics, Criminal Victimization
in the United States, 2005 Statistical Tables, National Crime Victimization Survey, NCJ 215244, December 2006.

Changes in Household Residents
As described above, a sample of households is selected to be interviewed for the NCVS, and it is
the household, not the people in the household, that remains in the sample. Hence, the interviewer
interviews anyone in the household over the age of 12 at each enumeration, even if the residents
of the household have changed from the previous interview. This can result in some cases where
unbounded interviews can be included in the national victimization estimates. For example, if a
new family moves into the household while the household is in the sample, the interviews with
the family will be unbounded because the bounding interview for the household was conducted
during the first interview with the household.248 If a family member was under the age of 12 at the
time of the bounding interview but turns 12 while the household is in the sample, the interview
with the family member that "aged in" to the sample is unbounded. As described above,
unbounded interviews may result in inflated estimates of victimizations. It has been estimated that
between 17% and 19% of household and person interviews were unbounded in any given year of
data.249

Limitations on the Scope of Crimes Covered
The NCVS collects data only on crimes that have a victim. Data on "victimless" crimes--such as
drug crimes, prostitution, and gambling--cannot be collected by the NCVS. Data on a host of
other crimes--including illegal weapons possession, murder, crimes perpetrated against business
or commercial establishments, consumer fraud, possession of stolen property, and public order
offenses--are also excluded from the NCVS. As a result, the crimes that the NCVS collects data
on make up a small part of all criminal offenses committed in the United States.250

Survey Design and Implementation
The design of the NCVS survey instrument and the methods used to administer it can affect
estimated victimization rates. As described above, changes in the wording of questions associated
with the redesign of the NCVS increased reported rates of victimization. The redesign showed
that the wording of screening questions can influence respondents' ability to recall victimizations.
Researchers have reported that both incident rates and subgroup variation in reported
victimization are affected by the wording of screening questions.251 It has been reported that short
screening questions may cue a respondent's recall of only a small subset of incidents that
involved the most serious or frequent crimes, whereas longer screening questions encourage the
recounting of a fuller range of victimizations.252

In addition to wording and type of questions included in the survey, the survey's methods can
influence the number of reported victimizations. As discussed above, the NCVS has started to
conduct more interviews using CATI. Researchers have found that using CATI can increase the
number of reported victimizations for some crimes.253 The use of CATI from a centralized
telephone facility has been shown to increase the number of reported crimes.254 It is believed that
the combined effect of centralization (ability to monitor interviewers) and computerization of the
survey help standardize the interviewer-respondent interaction, resulting in higher and more
realistic crime rates.255 NCVS surveys that used CATI resulted in higher crime rates than surveys
that did not; for example, violence, crimes of theft, and household larceny increased by 15%-
20%, and burglary increased by about 10%.256 CATI's effect on motor vehicle thefts was
negligible. Because CATI provides greater anonymity for respondents than in-person interviews,
respondents may answer sensitive questions more honestly, thereby producing a greater number
of reported victimizations.257

The length of the recall period used by the NCVS can affect estimated victimization rates. The
six-month reference period used by the NCVS was not chosen because it was the optimal
reference period, but rather because it provided a balance between accuracy and economy.258

Longer reference periods make it difficult for respondents to recall past victimizations
accurately.259 Also, the longer the reference period, the more likely it is that a greater number of
victimizations will be reported, because respondents are at-risk for victimization for a longer
period of time. Shorter reference periods would increase the cost of conducting the NCVS
because respondents would have to be interviewed more frequently.260



Endnotes

238
Michael R. Rand and Callie M. Rennison, "True Crime Stories? Accounting for Differences in Our National Crime
Indicators," Chance, vol. 15, no. 1 (2002), p. 49, hereafter "M.R. Rand and C.M Rennison `Accounting for Differences
in Our National Crime Indicators.'"
239
Ibid.
240
A confidence interval shows the range within which the true value of a calculated statistic is likely to fall a certain
percentage of the time. In this case, the NCVS estimated that 95% of the time, the true violent crime victimization rate
was in the range of 25.85 to 29.95 victimizations per 100,000 people aged 12 or older. There was a 5% chance that it
was either higher or lower than that range.
241
M.R. Rand and C.M Rennison "Accounting for Differences in Our National Crime Indicators," p. 49.
242
Ibid.
243
Ibid.
244
C.J. Mosher et al., The Mismeasure of Crime, p. 159.
245
C.M. Rennison and M. Rand, "Introduction to the NCVS," pp. 42-43.
246
U.S. Department of Justice, Office of Justice Programs, Bureau of Justice Statistics, Criminal Victimization in the
United States, 2005 Statistical Tables, National Crime Victimization Survey, NCJ 215244, December 2006, p. 134.
247
See, for example, Callie Marie Rennison, Intimate Partner Violence and Age of Victim, 1993-99, U.S Department of
Justice, Office of Justice Programs, Bureau of Justice Statistics, NCJ 187635, October 2001.
248
C.M. Rennison and M. Rand, "Introduction to the NCVS," pp. 30-31.
249
As cited in Ibid.
250
C.J. Mosher et al., The Mismeasure of Crime, p. 156.
251
Ibid., p. 160.
252
Ibid., pp. 160-161.
253
Ibid., p. 160.
254
U.S. Department of Justice, Office of Justice Programs, Bureau of Justice Statistics, National Crime Victimization
Survey (NCVS) Redesign: Technical Background, NCJ 151172, October 1994.
255
Ibid.
256
Ibid.
257
C.J. Mosher et al., The Mismeasure of Crime, p. 160.
258
C.M. Rennison and M. Rand, "Introduction to the NCVS," p. 29.
259
Ibid.
260
Ibid.

0 comments: