For Device Driver Download and Updates Click Here >>

Driver education/ training in a graduated licensing system

By: D. R. Mayhew and H. M. Simpson

Date: Monday, 27. July 2009

TABLE OF CONTENTS

EXECUTIVE SUMMARY

1.0 INTRODUCTION

1.1 OVERVIEW
1.1.1 Definitions
1.1.2 The Objectives of Formal Driver Instruction

1.2 BACKGROUND
1.2.1 The Need for Effective Programs
1.2.2 Driver Instruction and Licensing as Young Driver Countermeasures
1.2.3 Graduated Licensing and Driver Instruction

1.3 PURPOSE

1.4 STUDY METHOD
1.4.1 The Literature Review
1.4.2 Survey of Road Safety Experts and Driver Education Practitioners

1.5 SCOPE OF THE REPORT

2.0 THE SAFETY IMPACT OF DRIVER EDUCATION AND TRAINING

2.1 HISTORICAL CONTEXT
2.1.1 Early Development and Supportive Evidence on the Effectiveness of Formal Driver Instruction
2.1.2 Benefits of Driver Education and Training Challenged
2.1.3 DeKalb County Driver Education Project Revisited

2.2 POST DEKALB AND THE EFFECTIVENESS OF NOVICE DRIVER EDUCATION AND TRAINING
2.2.1 Experience in the United States
2.2.2 Experience in Canada
2.2.3 Experience in Europe
2.2.4 Experience in Australia and New Zealand
2.2.5 Summary

2.3 THE EFFECTIVENESS OF MOTORCYCLE RIDER EDUCATION AND TRAINING PROGRAMS
2.3.1 Experience in the United States
2.3.2 Experience in Canada
2.3.3 Experience in Europe
2.3.4 Summary

2.4 THE EFFECTIVENESS OF ADVANCED SKILL TRAINING PROGRAMS FOR NOVICES
2.4.1 The Effectiveness of Emergency Maneuvers Courses
2.4.2 The Effectiveness of Skid/wet Surface Training
2.4.3 Summary

2.5 THE VALUE OF FORMAL INSTRUCTION
2.5.1 The Benefits of Formal Driver Instruction
2.5.2 Summary

3.0 RECENT DEVELOPMENTS IN DRIVER EDUCATION/TRAINING AND DRIVER LICENSING

3.1 THE UNITED STATES
3. 1.1 Background
3.1.2 Recent Developments in Driver Licensing
3.1.3 Recent Developments in Driver Education
3.1.4 Summary

3.2 CANADA
3.2.1 Graduated Licensing and Driver Education
3.2.2 Ongoing Review of Driver Education/Training in Ontario
3.2.3 Development of a New Curriculum Standard for Driver
Education/Training in British Columbia
3.2.4 Development of a Crash-Based Course in Saskatchewan
3.2.5 Summary

3.3 EUROPE
3.3.1 Current Practices in Europe
3.3.2 Recent Developments in Driver Education and Licensing
3.3.3 Summary

3.4 AUSTRALIA AND NEW ZEALAND
3.4.1 Recent Developments in Driver Licensing
3.4.2 Recent Developments in Driver Education/Training
3.4.3 Summary

3.5 SUMMARY
3.5.1 Recent Developments in Driver Licensing and Implications
for Driver Education/Training
3.5.2 Recent Developments in Driver Education/Training

4.0 THE ROLE OF DRIVER EDUCATION AND TRAINING

4.1 CAN DRIVER EDUCATION/TRAINING REDUCE THE CRASH RISK OF YOUNG DRIVERS?
4.1.1 Driver Education/Training Fails to Concentrate on the Knowledge and Skills Judged Critical for Safe Driving
4.1.2 Driver Education Does Teach Safety Skills but Students are Not Motivated to Use Them
4.1.3 The Problem of Overconfidence
4.1.4 Lifestyle
4.1.5 Treatment Matching and Driver Education
4.1.6 Summary

4.2 DRIVER EDUCATION/TRAINING AND GRADUATED LICENSING

4.2.1 Should Driver Education/Training be Linked with Graduated Licensing?
4.2.2 Key Features that Need to be Considered if an Integrated System
is Implemented

5.0 REFERENCES

1.0 INTRODUCTION

1.1 OVERVIEW

Formal driver education/training programs exist in almost all jurisdictions around the world and are generally accepted as an efficient and effective means for learning to drive and, perhaps more importantly, for learning to drive safely. The safety benefits of such programs have been the subject of much debate and the focus of considerable research, most of which has not been particularly favorable. Despite the paucity of evidence showing the safety benefits of driver education/training programs, they continue to attract support and clients. As testimony to this, many states in the United States allow 16 to 17 year olds to become licensed only if they have completed a driver education/training program (Williams et al. 1996).

It is likely that a major reason for the continued appeal of driver education/training is that it makes sense. Driving in today's demanding roadway environment requires considerable knowledge and skill that take many years to develop. Driver education/training provides a structured approach to the learning process that can presumably facilitate and accelerate the acquisition of the needed skills. Simply put, driver education/training is seen as a sensible alternative to "trial and error" learning, especially given that errors can have such profoundly negative consequences.

Accordingly, driver education/training continues to be reasonably prominent as a system for preparing new drivers, despite the misgivings about its value harbored by many in the research community.

Recently, there has been a heightened interest in driver education/training, largely as the result of the adoption of graduated licensing in a few jurisdictions in North America and elsewhere. These jurisdictions have effectively elevated the status of driver education/training by integrating it into the licensing system. The most common approach has been to reduce the length of time in the graduated licensing system for anyone who completes an approved driver education/training program. In effect, driver education/training is accepted as a means for accelerating the beginner through the restrictions imposed by the licensing system. Implicitly, this acknowledges that driver education/training provides safety benefits.

As the popularity of graduated licensing increases, more and more jurisdictions will be examining the role that driver education/training can play in it. Accordingly, it seems timely to provide a contemporary review of the value of driver education/training, particularly in relation to new licensing systems such as graduated ones. This report attempts to do so. It examines the benefits of driver education/training and considers the merits of integrating driver education/training programs with new approaches to the licensing of young drivers.

1.1.1 Definitions. For purposes of this report, driver "education" and driver "training" are regarded as integral aspects of formal driver instruction. Historically, driver education has referred to in-class instruction, which teaches not only specific skills, but learning strategies, attitudes and motivations. Driver training has referred to in-vehicle instruction, which teaches the practical features of manipulating the vehicle and the perceptual motor responses required to drive. Taken together, the concepts of education and training "cover the process of bringing a learner driver from ab initio to a level of skill and knowledge sufficient for him to ... properly operate a motor vehicle within the highway system" (OECD 1975).

Although it might be argued that important distinctions can be drawn between education and training, it is generally agreed that complete preparation of the beginning driver involves both. In fact, in practice, most programs of driver preparation, whether in the high school driver education system or the commercial sector, incorporate some form of both education and training. Accordingly, this report focused on formal driver instruction which is defined as including both education and training aspects.

1.1.2 The Objectives of Formal Driver Instruction. This report has taken the position that the principal goal of driver instruction is to produce "safer" drivers, defined in terms of collision involvement. Simply put, it is assumed that drivers exposed to formal instruction should have lower crash rates than those who do not receive such instruction. This perspective is certainly consistent with what has previously been written about the aims of driver instruction and how driver education/training programs are typically evaluated. For example, an early OECD report (1976) on driver instruction observed that:

... although the purpose of driver instruction is not restricted to learning to drive safely, this is nevertheless a key element in any driver instruction system. Stated simply, all automobile accidents involve the driver's behavior in some way. From the safety point of view, driver instruction can therefore play an important part as, at the learner stage, drivers are not only more likely to be receptive to invitations to safe behavior (no driving habits have been formed yet), it is also the (only) time at which direct control is exerted over the driver's behavior. (p. 2).

Many, if not most, contemporary programs also claim that their driver education courses have safety benefits-how to handle emergencies safely, how to avoid collisions and what the novice should do to be less likely to have a collision (Clifford et al. 1985).

Indeed, much of the support for driver instruction has historically been generated by the perceived safety benefits of taking training. For example, special treatment is often given to drivers who successfully complete a course because of the belief that it is an effective safety measure-e.g., 16 and 17 year olds can only obtain a license by passing a course; course graduates are given a waiver of the road test; night driving curfews are waived for 17 year olds if they have taken driver education. Similarly, automobile insurance companies in some jurisdictions provide premium discounts to graduates of driver education/training programs because they expect them to have fewer crashes than young people who have not taken training, although this relationship would not necessarily mean that the programs themselves are responsible for the reduced crash involvement. And, as will be described in later sections of this report, recent efforts to improve driver education/training have identified as their primary objective determining how best to educate and train teenagers to be safe drivers (NHTSA 1994). The fact is that driver education/training has been and continues to be viewed as a means to reduce the high rates of collisions involving young drivers.

Many in the research community and others in the field of road safety have, however, grown increasingly skeptical about the safety value of driver instruction. For them, previous research has clearly demonstrated that driver education/training fails to prevent young driver crashes. As such, they would claim that too much attention and resources have been spent on driver instruction.

Others, primarily in the driver education community but in the research community as well, reject this negative perspective, arguing that it is based on an unreasonable expectation of what driver education/training can accomplish. From their perspective, driver instruction is effective in teaching safe driving performance-e.g., students know the rules of the road and can demonstrate that they have acquired safe driving techniques by passing the road test. Unfortunately, for reasons beyond the control of driver educators, the knowledge and skills acquired in training do not necessarily produce safe driving behavior-i.e., although learners have the ability, they choose not to use it to reduce their crash risk.

Waller (1975), for example, has previously observed:

To hold driver education instructors responsible for the subsequent driver records of students is a little like holding home economics teachers responsible for whether the students prepare well balanced meals two years later ... math teachers would be judged according to how well students balanced their checkbooks in later years. I would maintain that in driver education we should be able to hold the instructor responsible for how well the student is able to operate the vehicle and how well he knows the rules of the road. However, whether he actually uses the skills and knowledge he has acquired depends on many things beyond the control of the driver education instructor. It is utterly foolish to expect a teacher to change the attitudes of students in 36 hours of contact. (p. 17-18).

Even though Waller expressed these views over 20 years ago, many driver educators today would likely applaud them and suggest they are relevant to our conclusion that collision reduction is an appropriate criterion to use for judging the value of driver instruction. Waller states that a more reasonable expectation for driver education is that it improves "how well the student is able to operate the vehicle and how well he knows the rules of the road." If this is the criterion against which driver instruction should be judged, then there is ample evidence that it is effective-those who complete driver education have been shown to have more knowledge and skill than non-graduates (Stock et al. 1983; Clayton and Sudlow 1987; Forsyth 1992).

From a road safety perspective, however, improved driving performance has value only to the extent that it serves as a means to achieve an important end-i.e., collision reduction. Although it may be desirable to improve the level of performance of a learner, this only represents an intermediate step leading to the ultimate objective-lower frequencies of collision involvement.

Indeed, as mentioned previously, driver education/training is widely promoted and supported as a safety initiative. If a stated objective of driver education/training is to produce safer drivers, it is something of a distortion to suggest that driver education/training should not be held "responsible for the subsequent driver records of students." If safety is not an intended or achievable objective of driver education/training -i.e., because factors beyond the control of driver educators exert such a powerful influence on driving behavior that any training benefits are neutralized-this should be clearly articulated, so that novices are not encouraged or required to take such courses to reduce their crash risk. Clearly, the field needs to achieve clarity and consensus on what driver education/training is expected to do, so that misconceptions and inappropriate expectations are minimized.

In summary, a stated objective of driver education/training is safety. It is, therefore, reasonable to expect that such programs will reduce the number of road crashes of young drivers. A primary purpose of this report is to determine if such safety benefits have been forthcoming. If safety is an unrealistic expectation and the best that can be achieved is improving the level of performance of novices so that they can pass the license test and drive, this needs to be stated explicitly so that such programs are supported for their mobility and not their safety benefits.

1.2 BACKGROUND

Collisions involving young drivers, particularly those between the ages of 16-19, have been a worldwide road safety and public health concern for decades. Although some progress has been made in recent years-for example, the death rate from motor vehicle crashes has declined among young drivers more so than among other age groups (see Mayhew and Simpson 1990)-the crash, death and injury rates among people under the age of 20 are still far greater than for other groups. For example, Williams (1996) has shown that 16-19 year old drivers have a crash risk that is four times higher than that of older drivers. This mileage-based crash rate is particularly acute for 16 year old drivers, who have a rate that is three times higher than that for 18 year olds and some ten times higher than it is for 35-39 year olds.

1.2.1 The Need for Effective Programs. Despite the improvements evidenced in recent years, collisions involving young drivers continue to be a major social problem. And the magnitude of the problem will likely increase because impending demographic changes will produce a dramatic rise in the number of young drivers. Population estimates over the next decade show that the number of 16-20 year olds in the United States will increase by 15 percent by the year 2005 (from a total of 18,008,000 16-20 year olds in 1996 to 20,727,000 in 2005).

These facts and trends underscore the continuing need for effective programs and policies to address the problem of road crashes involving young drivers. To be effective, however, safety initiatives must address the causes of the problem. In this regard, it has become well established that young drivers are over-represented in road crashes because (1) they are inexperienced, lacking the necessary driving skills and capacities, and (2) their attitudes and behaviors (lifestyle) result in risky driving and increased likelihood of crash involvement (Mayhew and Simpson 1990). The importance of experience-related and age-related factors in the collisions of young drivers has been underscored in a recent report by Mayhew and Simpson (1995). They draw on both scientific evidence and expert opinion to construct the following profile of young novice drivers:

Young novice drivers are more likely than older experienced drivers to place themselves in dangerous situations because of an inability to integrate motor skills efficiently. Their operation of the vehicle is easily disrupted when the demands of driving increase beyond the norm. In such situations, they encounter difficulties in dividing their attention across several tasks. Their greater inability to steer properly and maintain constant speed when task demands are high makes it more difficult for them to maintain the vehicle in the lane and negotiate curves properly.

Novice Young drivers are less aware of hazards in the environment and are less likely than older experienced drivers to identify moving objects as hazards. As a consequence, it takes novice drivers longer to respond to a hazard, and once the hazard is recognized this may overload their ability to respond and maintain control of the vehicle.

Their problems with detecting hazards likely relates to the fact that their search and scan abilities are less developed than those of more experienced drivers. The novice driver may not have fully developed the use of peripheral vision and because they have a smaller range of scanning of the roadway than experienced drivers, they can fail to detect hazards in the environment. Young novice drivers have a tendency to look closer to the front of the vehicle than experienced drivers, especially in low speed driving situations, and are less likely to use mirrors as visual aids.

The problem in detecting hazards is compounded by the fact that young drivers perceive their crash risk as lower than that of their peers-they view themselves as much safer than their risky cohorts-and misjudge the risks they face especially in traffic situations that place the greatest demands on their limited vehicle control skills.

Young novice drivers also overestimate their own capabilities. Risky lifestyle behaviors (e.g., alcohol use) and attitudes (e.g., alienation, sensation seeking) are also prevalent during these critical adolescent years. As such, some of them succumb to thrill seeking and risk taking tendencies, adopting risky driving behaviors that exceed their abilities to respond effectively in critical situations.

These deficiencies in driving skills/capacities and risky lifestyle characteristics are reflected in driving practices-e.g., speeding, following too closely-that result in young drivers being over represented in collisions involving excessive speed, intersections, and striking the back of another vehicle.

Recent research has shown that these experience-related and age-related factors contribute about equally to the greater crash risk for young drivers (Simpson 1996). And Simpson (1996) has observed that to significantly impact the higher crash risk of young drivers will likely require addressing both experience-related and age-related factors, given the close interdependence of the two in the dynamic driving environment.

1.2.2 Driver Instruction and Licensing as Young Driver Countermeasures. Historically, and even today, the major initiatives used to address the experience- and age-related factors that give rise to the young driver problem have been formal driver instruction and driver licensing. The objectives of both these programs are very similar in that they attempt to ensure the novice has the skills, knowledge and attitude to drive safely and collision free.

It follows that in most jurisdictions in North America and elsewhere, driver instruction is linked in some manner to the licensing process. For example, in many jurisdictions, novices are required to complete a mandatory driver instruction program as part of the licensing process. This can take several forms. For example, all beginners regardless of age must take driver instruction, or beginners aged 16 and 17 can only get licensed if they take a driver instruction program and those aged 18 and over need not do so. In jurisdictions where driver instruction is available on a voluntary basis, beginners often take the course to prepare for the road test. In these cases, courses typically focus on the skills and knowledge that are needed to pass the road test and obtain a driver's license.

1.2.3 Graduated Licensing and Driver Instruction. In recent years, the bond between driver instruction and driver licensing has been strengthened even more with the advent of a new system called graduated licensing. Long promoted as a potentially effective means for reducing the risks encountered by novice drivers, graduated licensing systems impose a set of restrictions on the beginner -e.g., night curfew, zero BAC, limits on the number and age of passengers-which are gradually and systematically lifted so that the novice enters driving in a step-by-step, progressive manner. Upon graduation from the system, the driver is granted unrestricted driving privileges.

The basic objective of the graduated licensing system is to provide all new drivers with the opportunity to gain driving experience under conditions that minimize exposure to risk. For example, night driving is initially prohibited because this time period has been shown to be risky for beginners, especially young drivers (see Williams 1996). As experience and competence are gained at low-risk times, such as during daylight hours, the opportunity for exposure to increasingly risky situations is gradually phased in. Thus, graduated licensing addresses experience-related factors that give rise to young driver crashes.

It also addresses the age-related factors-i.e., peer pressure, a propensity to take risks-that contribute to the higher crash risk of young drivers. For example, a night curfew prohibits young people from driving during late night hours when social pressures to consume alcohol are greatest.

Given the strong historical link between driver instruction and the licensing process, it is not surprising that driver instruction has been given a relatively prominent role in the graduated licensing systems implemented to date. As indicated above, the typical form that this relationship has taken is to allow beginners who take formal driver instruction to proceed through the restricted stages of the graduated license in less time than those who have not taken driver instruction. For example, in New Zealand, the first country to adopt a comprehensive graduated licensing system, drivers who complete an approved driver instruction program are only required to hold a learner's permit for three months, rather than six months. Those who complete an advanced driver instruction course are only required to hold the second stage restricted license for nine months, rather than I 8. This means that the full period of graduated licensing can be reduced from 24 t(> 12 months by taking driver instruction course(s).

Thus formal driver instruction has been used within current graduated licensing systems as a means to accelerate the learning process. The basic assumption is that beginners who take driver instruction will acquire the knowledge, skills and abilities that would have otherwise developed over a greater period of time while driving under low risk conditions. This implies that driver instruction is somehow equivalent to a certain amount of on-road driving experience. The validity of this assumption needs to be examined, particularly in light of -the reasonably rapid pace with which graduated licensing systems are being introduced across North America.

Indeed, many jurisdictions are currently studying the system of graduated licensing and its relationship to driver instruction. For example, the U.S. National Highway Traffic Safety Administration has implemented a research plan to develop a two-stage improved driver instruction program that is an integral part of a graduated licensing system (see section 3.1 for further details on this program).

1.3 PURPOSE

The recent developments outlined above suggest that driver instruction is becoming increasingly accepted as a potentially effective safety measure, especially if it is embedded in a graduated licensing system. Accordingly, it is timely to consider the safety benefits of driver instruction and the role it might play in reducing the collisions of novice drivers. This is the primary purpose of the present report. It seeks to provide a contemporary review of the effectiveness of driver instruction in dealing with young driver crashes and to assess what such programs can potentially achieve, particularly when they are integrated with new approaches to the licensing of young drivers such as graduated systems.

This report documents the effectiveness of driver instruction and identifies the potential role it can play in reducing young driver crashes. More specifically, it:

1.4 STUDY METHOD

The study involved a review of the literature on the effectiveness of driver education and training programs and a survey of experts and programs. Both are described briefly below.

1.4.1 The Literature Review. Electronic databases and libraries in Canada, the United States and elsewhere containing road safety literature were searched to identify and collect reports on the impact of driver education and training programs. Road safety experts, driver educators and administrators of driver instruction programs were also contacted and asked to provide relevant unpublished literature and technical reports that might not have been identified in the search of major library holdings.

The evaluation studies collected in this manner were organized into content area-i.e., studies evaluating the effectiveness of programs for novice drivers, motorcycle education programs and advanced skill programs -and critically reviewed.

1.4.2 Survey of Road Safely experts and Driver Education Practitioners. Relevant road safety experts and driver educators in Canada, the United States, Europe and Australia were surveyed to gather information regarding current programs as well as new initiatives in the field of driver education/training. Information was requested on promising driver education/training programs that were currently planned and/or anticipated. Experts were also asked to identify any major changes they believe are needed to improve the collision reduction potential of driver education and training.

Information obtained in the survey was reviewed and synthesized for inclusion in the report.


1.5 SCOPE OF THE REPORT

Section 2.0, The Safety Impact of Driver Education and Training , examines the evidence of the effectiveness of formal driver instruction. It provides a brief historical overview of the early development of formal driver instruction in the United States; a detailed examination of the original landmark DeKalb study and subsequent analyses of the data; and a critical review of more contemporary studies that have assessed the safety impact of driver instruction programs in the United States and elsewhere, including programs to train novice motorcyclists and those that teach advanced skills in emergency maneuvers and slippery surface driving.

Section 3.0, Recent Developments in Driver Education/Training and Driver Licensing , describes contemporary developments in formal driver instruction and driver licensing in the United States, Canada, Europe, Australia and New Zealand. This section identifies recent and planned developments that may hold promise for improving driver education/training as a collision reduction measure.

Section 4.0, The Role of Driver Education and Training , considers what formal driver instruction can potentially achieve as a loss reduction measure, particularly when it is integrated with new approaches to the licensing of young drivers such as graduating systems.

Return to Table of Contents

2.0 THE SAFETY IMPACT OF DRIVER EDUCATION AND TRAINING

This section examines the evidence on the effectiveness of formal driver instruction. It begins by providing a brief historical overview that covers the early development of formal driver instruction in the United States, from its inception in the early 1900s to its widespread growth into the late 1970s. As will be shown, the development of formal instruction in the United States has been influenced rather profoundly by the results of evaluation studies on its impact. This was underscored by the results of the evaluation project that has come to be known as the DeKalb study.

This historical milestone was the most comprehensive evaluation of the impact of formal driver instruction ever conducted. The results of this study have proven to be a watershed in the field of driver education not only in the United States but elsewhere as well. As a result, it is reviewed in this report, as are several re-analyses of the DeKalb data.

The examination of the original DeKalb study results and subsequent analyses is followed by a critical review of more contemporary studies that have assessed the safety impact of driver instruction programs in the United States. Perhaps as part of the legacy of the DeKalb project, very few evaluations of driver instruction have been undertaken in the United States over this time period. Accordingly, this section also reviews evaluations conducted over the past 16 years in other countries, including Canada and several in Western Europe. To supplement the findings on the potential safety benefits of formal driver instruction, the section considers studies on the effectiveness of motorcycle rider education/training programs and advanced training courses for novices. Thus, three broad categories of studies from several countries are reviewed: those that have examined the effects of formal novice driver instruction; those that have considered the impact of novice motorcycle rider programs; and studies on the effectiveness of advanced training courses for novice drivers.

2.2 HISTORICAL CONTEXT

Formal driver instruction is not a new concept in North America. It can be traced to the turn of the century when automobiles emerged as an increasingly popular mode of transportation. The field of formal driver instruction, however, did not experience major growth until the 1930s and 40s with efforts to standardize courses and establish a higher degree of professionalism within the instructional cadre. Growth accelerated even more in the 1950s and 1960s, propelled largely by evaluation studies that provided support for the value of driver instruction-graduates of driver education courses were found to have a lower frequency of collisions and violations than individuals not so trained.

This situation changed dramatically, however, in the 1970s when the validity of the results from earlier studies was questioned because of methodological flaws. As a result, the beneficial effects of formal driver instruction were challenged.

To resolve issues regarding the safety benefits of formal driver instruction, the National Highway Safety Administration (NHTSA) in the United States launched a major driver education development and evaluation project in the late 1960s. This so-called "DeKalb" project remains the largest and most well designed test of the effectiveness of driver education to date. Regrettably, the results of this study did more to inflame the controversy regarding the benefits of formal driver instruction than to clarify the issue. Indeed, the DeKalb study findings are still hotly debated today.

During the 1980s, the value of driver education/training was also questioned on the grounds that it produces unexpected negative consequences. Work supported by the Insurance Institute for Highway Safety (IIHS) showed that the greater availability of driver education stimulates earlier licensure among teenagers, which in turn leads to more crashes per capita (Robertson and Zador 1978; Robertson 1980). Given these findings and the lack of evidence to support driver education/training as a safety measure, a controversy arose regarding the value of driver education/training.

This section discusses these early events and briefly describes the evidence on the effectiveness of formal driver instruction. Considerably more attention is given to a review of the original DeKalb study results and subsequent analyses because of the tremendous influence this large scale experiment had on the field of driver education/training not only in the United States but elsewhere as well.

2.1.1 Early Development and Supportive Evidence on the Effectiveness of Formal Driver Instruction. According to NHTSA (1994), the first known novice driver education program was developed in 1916 and the first textbook was published in 1919. The first in-class driver education class was taught by Professor Amos Neyhart at State College High School in Pennsylvania in 1933. And during that decade, several small colleges also began offering courses to prepare driver education teachers. As described by Public Technology Inc. (1986):

... by 1936 the idea of behind-the-wheel (BTW) training had become popular and Pennsylvania State University began offering one of the first college courses designed exclusively for the preparation of driver education teachers. Also during this time, the American Automobile Association (AAA) sponsored a 40-hour course for high school driver education teachers at Bluefield, West Virginia, and New York State made driver education a part of its secondary school curriculum. (p. 1).

A series of National Conferences on High School Driver Education (beginning in 1949) signaled a formal attempt to organize the movement and bring standardization, consistency and professionalism to programs that were developing rapidly across the United States, At these conferences, it was recommended that the standard course include 30 hours of classroom instruction and six hours behind-the-wheel (in-car). Of some interest, this has continued for over four decades as the minimum standard in many driver education programs (Crowe and Torabi 1994).

In the 1950s the potential value of formal driver instruction as a crash loss reduction measure was recognized by the private insurance industry, which offered discounts for students completing a driver education program. The Allstate Insurance Company began this practice in 1952 as did other insurance companies shortly thereafter. As a consequence of the insurance discount, the demand for driver education increased and high school driver education programs became available across the United States.

During these early years, formal driver instruction was considered common sense and superior to "practice on one's own" because trained professionals provided the information and skills in an organized and structured manner. Formal driver instruction was generally believed to be an effective means of reducing the crash risk of young drivers. This commonly held belief was supported by early evaluations demonstrating the safety benefits of existing programs. For example, Allgaier (1964) reviewed 30 studies on driver education in 19 states. Based on the review, the author concluded that these studies demonstrated that completion of a driver education course was associated with fewer collisions and violations.

Given such an empirical endorsement it is not surprising that driver education/training received major support. Indeed, in 1966 the U.S. Congress enacted the Highway Safety Act and identified driver education a major traffic crash countermeasure. The basic requirements for driver education were specified in the legislation. Each state was required to provide students with practice driving and instruction in at least the following:

The Act also provided matching federal funds for states that offered driver education. As a consequence of federal funding, a tremendous expansion occurred in the availability of driver education courses-e.g., by the late 1970s and early 1980s about 80 percent of all eligible students in the United States were taking driver education (in most cases as a credit course).

2.1.2 Benefits of Driver Education and training, Challenged. Commensurate with the growth in driver education/training in the late 1960s and the 1970s was a growth in the scrutiny it received from the research community. Critical reviews of existing evaluations revealed serious methodological flaws that rendered their conclusions about the positive safety benefits of driver education highly questionable (McGuire and Kersh 1969; Goldstein 1969; Goldstein 1971; OECD 1975). For example, McGuire and Kersh (1969) observed that each of the thirty studies reviewed by Allgaier (1964) "included gross error in experimental design" (p.15). Perhaps the most serious limitation of the earlier studies is that they failed to control potential differences between students who received training and those who did not. It was therefore possible that the lower collision rates among those who took driver education were not due to the beneficial effects of the program but rather to the fact that students who chose to take driver education had characteristics that were independently associated with low collision rates. To illustrate, individuals who elect to take driver education may be more safety conscious than those who do not and might be expected to have a lower crash rate, even in the absence of the education program. Indeed, Conger et al. (1966) showed that when factors such as motivation, intelligence, and social class were controlled, the differences in collision rates between teenagers with and without training were eliminated.

The methodological weaknesses inherent in much of the early evaluation research began to receive prominent attention in the late 1960s and early 1970s, casting doubt on the earlier claims of the beneficial effects of formal driver instruction. It became apparent that there was no reliable evidence to support claims that driver education/training had a beneficial impact on collisions. At the same time and for the same reason, it could be argued that there was no reliable evidence to support counterclaims that driver education/training did not have beneficial effects. As a consequence, in a review of work published prior to 1970 it was concluded that:

No clear proof has yet been produced showing that driver education, as presently constituted, has a significant favourable effect on driver performance. No clear proof has as yet been produced showing that driver education, as presently constituted, does not have a significant favorable effect on driver performance. (New York University, 1969)

Similarly, a review of studies on the effectiveness of driver education published in the early 1970s (Page-Valin et al. 1977) concluded:

Recognizing that the value of driver education as a safety countermeasure was still at issue in the United States, NHTSA embarked on a long-term research program to develop and evaluate a state-of-the-art driver education curriculum. In 1968, four separate but parallel contracts were awarded to develop a plan(s) for evaluating the effectiveness of existing or proposed driver education programs (New York University 1968; Dunlap and Associates 1968; Kennedy et al. 1968; Lybrand 1968; as cited in Stock et. al. 1983). Two National Symposia were also convened in 1968 and 1969 to review existing information and gather additional information.

The information derived from these activities was synthesized into a single report by the National Academy of Sciences, Highway Research Board (now the Transportation Research Board) to identify an optimal plan for evaluating driver education (Harinan et al. 1969). This report identified several work efforts that would be required, including the:

This research plan involved the following integrated steps: (1) an analysis of the driving task to identify those elements with a high or moderately high criticality to safety; (2) development of specifications, and then a curriculum oriented towards safety; and (3) the evaluation of that curriculum for its crash reduction potential.

Thus, as a precursor to developing the curriculum content, NHTSA funded an extensive driver task analysis. This work involved identifying over 1,700 driving behaviors and then having safety experts rate each of them in terms of their criticality to safety. The ratings were based on the frequency of the behavior, the likelihood that the behavior would be performed incorrectly, and the likelihood that incorrect performance would be related to crash involvement as well as its severity.

The driving tasks that were identified by the experts as having a high or moderately high 'criticality' to safety were used as the basis for a new driver education program called the "Safe Performance Curriculum" (SPC). As originally designed, the SPC was a 72-hour course of instruction in which:

Students receive approximately 32 hours of formal classroom instruction, 16 hours of simulation instruction, 16 hours of multiple car driving range instruction, three hours of evasive maneuvers instruction, and five hours on-street instruction (two hours of these behind the wheel), one hour of which is night driving. SPC students also participate in six practice-with-parents sessions-a minimum of one hour each. (Weaver et al. 1983).

The SPC was pilot tested in Kansas City, Missouri. Due to administrative difficulties such as maintaining control over random assignment of students to groups and inadequate instructor preparation as well as small sample size, information on collisions and violations was not obtained. This pilot study did, however, provide the basis for proceeding with a full-scale evaluation of the Safe Performance Curriculum. This evaluation took place in the DeKalb County school system in Georgia. The DeKalb Driver Education Project is described in more detail below because it remains the most critical test of the safety impact of driver education to date.

2.1.3 DeKalb County Driver Education Project Revisited. This section initially revisits the original DeKalb study and then reviews several subsequent and related evaluations. This is followed by a review of several critiques of the original study and re-analyses of the DeKalb data. For convenience each of the studies is summarized in Table 2-1 in the order in which they are reviewed in this section. For each study listed in the Table, summary information is provided on the subjects included in the analysis; and the key findings.

As will become apparent in reviewing this Table and subsequent sections of the report, the DeKalb study and its data set have been the subject of repeated scrutiny and analysis since the project was initiated. Indeed, it is likely that such an intense level of scientific assessment concentrated on a single project is unparalleled in the field of road safety. Unfortunately, despite these concerted efforts to resolve the issue regarding the effectiveness of an improved driver education program, the value of driver education remains controversial.

The original DeKalb findings. As suggested above, NHTSA invested considerable resources and effort into the design and pilot testing of an improved high school driver education course. This level of commitment extended to the evaluation of the program. Indeed, NHTSA provided over $4 million for the DeKalb County Driver Education project. This included construction of driving ranges, a classroom/simulation building and renovations of school buildings to prepare them for driver education classroom activities and simulation. A national search was conducted to identify driver education teachers capable of teaching in a multiphase driver education program. Instructors selected for the project underwent an extensive ten-week training program to prepare them to teach the curriculum.

TABLE 2 - 1

DeKalb studies

Authors Subjects Results

Stock et al. (1983)

--all assigned students

--completed course and licensed

--No effect

--Positive effect for first six months - SPC, PDL

Smith and Blatt (1987)

assigned students who had driving records

No effect-SPC

Positive effect-PDL

DeWolf and Smith (1988)

students who at least partially completed SPC or PDL; those who had never enrolled; controls

No effect

Lund et al. (1986)

all assigned students

Negative effect

Davis (1990)

all assigned students

Negative effect

The evaluation study itself included over 16,000 students, randomly assigned to two treatment groups and a non- treatment (control) group. The treatment groups received the Safe Performance Curriculum (SPC) or a pre-driver licensing (PDL) course. As mentioned previously, the SPC comprised a 72-hour course that included 32 hours of classroom, 16 hours of simulation, 16 hours of driving range instruction, three hours of instruction on evasive maneuvers, and five hours of on-street training. The PDL was a 20-hour basic driver training course that also included classroom, range and simulation instruction as well as practice driving with parents. The PDL was designed as a minimal program to prepare beginning drivers to pass the Georgia Motor Vehicle Operator's Test. Unlike the SPC, the PDL offered considerably less instruction than the typical high school course (National Safety Council 1984). Students assigned to the non-treatment condition received no formal driver education or training through the public school system. The control students were expected to be taught to drive by their parents and/or private driver training schools.

These three groups-SPC, PDL, and control-were matched on variables such as gender, grade point average and socioeconomic status that had been shown in previous studies to be related to crash rates. Thus, considerable attention was given to overcoming the methodological flaws that plagued earlier evaluations of formal driver instruction,-i.e., the study was designed to ensure that groups were as equivalent as possible.

Stratified randomization procedures were then used to ensure that the SPC, PDL and control groups were closely balanced across driver education centers, high schools, gender, grade point average and socioeconomic status. The assignment procedure resulted in 5,464 students in the SPC group, 5,430 in the PDL group and 5,444 in the control group. According to Stock et. al. (1983), "large sample size provides a narrow confidence interval, permitting the detection of a 10-15 percent difference in crash rates between groups, if it is present." (p. 1-4).

Following assignment to the three groups, students were monitored to determine such things as whether they actually took the training course, whether they became licensed and, if so, to track their collision and violation experience for up to four years from the initial assignment.

In the initial evaluation report (Stock et al. 1983) a variety of analyses were performed to determine if the SPC was associated with safety benefits. For example, crash data were compared across the three groups as well across different subsets of these groups (e.g., a comparison based only on those who became licensed-86.3 percent of assigned students became licensed; a comparison based only on those who actually completed the SPC and PDL and became licensed-just 68.5 percent and 66.5 percent respectively of assigned students completed the SPC and PDL and became licensed within the study period). In addition, Stock et. al. examined the data for the overall study period as well as by more restricted time blocks of licensed driving-i.e., 6 months, 12 months, 18 months, and 24 months after licensing. This later approach was used to control for differences in the length of time of licensed driving among the SPC, PDL and control students. The main statistical procedure used by Stock et al. to test the differences between the groups was the analysis of variance.

The overall results of the evaluation were summarized by Stock et. al.:

... the major result of this demonstration project was that the improved driver education program, Safe Performance Curriculum, was not an effective accident reduction countermeasure. (p.111-1)

Indeed, the direction of the differences in crash rates was opposite to what had been predicted. The analyses based on all assigned students showed there was, in fact, a greater percent of SPC students involved in collisions than PDL or control students. Moreover, this pattern of results was also found when the analyses included only students who became licensed, or only students who both became licensed and completed the SPC and PDL. The key findings are shown in Table 2-2, which presents the proportion of students in the various groups who became involved in a collision. As can be seen, if the analysis includes all students initially assigned to the experimental conditions, 28.6 percent of SPC students, 26.5 percent of PDL students and 26.7 percent of control students were involved in a collision. The pattern of results is similar for assigned students who became licensed as well as for assigned students who both completed the course and became licensed. Stock et al., however, do not indicate whether any of these small differences were statistically significant.

An examination of the average number of collisions for the SPC, PDL and control students produced similar results. For example, among assigned students, the mean number of collisions was higher in the SPC group (x = .3776) than in the PDL group (x = .361 1) and the control group (x = .3643). An analysis of variance revealed, however, that these apparent differences between SPC, PDL and control groups were not statistically significant.

These results suggested that the SPC was not effective in reducing collision involvement. As disappointing as this finding might be, it was even more disturbing to show that the greatest incidence of collisions was found among those in the SPC group. Despite the fact that most of the differences were not statistically significant, the consistent pattern of the results was disconcerting.

A major reason offered by Stock et al. to explain the higher collision involvement of SPC subjects was that they become licensed sooner than PDL and control subjects. For example, 70.6 percent of those in the SPC group became licensed within six months of assignment to the program, compared to 66.7 percent of PDL and only 58.8 percent of the controls (Peck 1996). Although this difference in licensure rate was reduced by the end of the study period, the SPC and PDL groups still had a higher licensure rate than controls-88.4 percent and 86.2 percent compared to 84.3 percent among the controls (Stock et al. 1983). The point is that SPC subjects became licensed earlier and, consequently, could drive sooner and therefore had more exposure to the likelihood of a collision. Given these differences, it is not surprising that the SPC group was found to have a greater mean number of collisions when the length of licensed driving was not controlled.

TABLE 2 - 2

Overall collision involvement

Assigned

Licensed

Comp-Licensed

Groups

SPC

28.6

32.3

32.6

PDL

26.5

30.7

30.7

CONTROL

26.7

31.7

31.7


Accordingly, additional analyses were restricted to those students who became licensed (or completed the courses and became licensed) but the time period of licensed driving was controlled. These analyses produced different results. They showed that the average number of collisions among licensed students in the SPC group was lower than it was in the control group during the first six months of licensed driving (x = .1054, .1066 and .1221 for the SPC, PDL and control students, respectively). However, the difference between the SPC and control groups was not statistically significant (at the .05 level). The authors maintained, however, that "the attained level of significance of .076 may be interpreted by some as strongly suggestive of real program effects." (p. I I 47)

Statistically significant differences were found in examining the average number of collisions during the first six months of those students who had completed the course and subsequently became licensed. Among such students, those in both the SPC and PDL groups had lower collision means than control students-.1021, .1010, and .1221 respectively. Thus, the mean number of collisions in the SPC group was 19.6 percent lower than among the controls. Interestingly, PDL students, not SPC students, had the lowest mean number of collisions but this difference was not statistically significant.

An examination of longer time periods (i.e., 12 months, 18 months and 24 months after licensing) revealed no statistically significant differences in collision means among the SPC, PDL, and control groups.

Based on these analyses, the authors concluded:

... the SPC and PDL programs appear to have an effect in reducing accident occurrence during the first six months of licensed driving for both licensed and complete and licensed students. However, the program effects are neutralized, or "wear" off after six months. (p. 1 1-5 1)

The authors also used a "repeated measures" design to examine the mean number of collisions for the same sample of students over four time periods 6 months, 12 months, 18 months and 24 months as well as for the total aggregate project period. As indicated by the authors, this design "provides for better controlled between-periods comparisons, because of the common sample, but leads to much more restricted sample sizes for the earlier periods, as compared to the previous analyses" (p. 1 1-58).

The results of this potentially more sensitive analysis was similar to those obtained in the previous analyses. For example, among licensed students, those in the SPC group had a lower mean number of collisions than those in the PDL and control groups in the first six months of licensed driving-means of. 1 05 8, .1141 and .1313, respectively. A similar pattern of results was found for those students who both completed the course and became licensed-means of .1041, .1072 and .1313, respectively. These differences diminish as the period of licensed driving increases both for the licensed students as well as for those who completed the course and became licensed. Moreover, an examination of the mean number of collisions over the total period of the project showed little difference between SPC, PDL and control students.

In summary, this well-designed evaluation did not provide evidence that the SPC-a state-of-the-art driver education program-was an effective crash reduction countermeasure. In fact, the SPC group had a higher number of collisions and a higher collision mean than the PDL and control groups, although these differences were not statistically significant.

At best, the SPC appeared to have a crash reduction effect for the first six months of licensed driving, especially among students who had completed the course and became licensed.

Thus the most positive finding in the Stock et al. analysis of the DeKalb study is the lower mean number of collisions among those in the SPC and PDL groups, compared to those in the control group, during their first six months of driving. This finding may, however, reflect different levels of driving exposure across the groups in this six-month period. Although driving exposure data were not collected during the entire first six months of driving, some relevant data were obtained that suggest differences in exposure, Stock et al. (1983) conducted a telephone survey of selected licensed students in the three groups to obtain information on their driving exposure on the day prior to being interviewed. Students were asked to indicate the estimated number of miles and hours driven in the previous day.

An examination of these data revealed that SPC and PDL students reported driving for fewer hours and miles than control students, If this pattern of results could be generalized to the first six months of driving, the SPC and PDL students would be expected to have lower crash means than control students because of their reduced exposure. As discussed above this difference was in fact found. Thus differential exposure and not the effects of the SPC may have produced the lower crash means. Stock et al. did not accept this interpretation, however. They contended that the differences in exposure among the three groups were not large and "do not appear to be sufficient to account for differences, if any, among the three groups in mean of accident occurrence." (p. I I 70 and 11-71). Others (e.g., Lund et al. 1986) have disagreed with this conclusion. Their results are presented later in this section.

Once Stock et al. had eliminated, to their satisfaction, the possible contradicting effects of exposure-i.e., the possibility that differences in collisions between the groups in the first six months of driving resulted from differential driving exposure-they concluded that there was indeed a significant short-term program effect, "the SPC and PDL groups having significantly lower crash means during the first six months of licensed driving." Such differences, however, eroded over time. They also observed that:

... the short-term effect was additionally offset or neutralized by the earlier licensing of SPC and PDL students, yielding a net effect of no statistically reliable differences among SPC, PDL, and control groups of students in overall accidents. (p.1)

In summary, the SPC and PDL had a short-term positive effect in reducing crashes during the first six months of licensed driving. According to the authors, however, these safety benefits were offset or neutralized by the earlier licensing of SPC and PDL students-i.e., the overall program effects showed no statistically significant difference in the crash records of SPC, PDL and control groups. Although this may be the case, it is also important to note that the SPC and PDL students reported driving fewer hours and miles than controls. Thus, the reduced exposure as licensed drivers may have offset, to some extent, the increased exposure due to earlier licensing. The disbenefits of earlier licensure-i.e., exposure to the risk of collision-may not have been great if students in the two program groups actually drove less than the controls.

Follow-up evaluations of the DeKalb study. In the final report on the potential long-term effects of the DeKalb study (Stock et al. 1983), it was found that only about 50 percent of the students who completed the SPC or PDL courses had at least two years of licensed driving. The resulting small sample size made it impossible to reliably determine the long-term effect of the SPC or PDL in reducing crashes at that time. For this reason, data continued to be gathered on study subjects and further analyses were undertaken by NHTSA.

In the first of these follow-up evaluations Smith and Blatt (1987) examined additional years of data, producing six one-year time periods. Similar to the original study, they employed an analysis of variance to evaluate group differences in collisions. Based on this analysis, they reported the following results:

Briefly, this longer-term evaluation of the impact of the DeKalb study did find a significant effect. However, the reduction in collisions was found only for students in the PDL program; no significant effect of the SPC was detected.

In an additional, more extensive analysis of the updated DeKalb data, DeWolf and Smith (1988) used multiple regression models to determine the effects of various potential explanatory variables on the number of collisions during a four-year time period. As described by Davis (1990), the independent variables in this follow-up evaluation included driver education program, gender, grade point average, driver education center, enrollment quarter, socioeconomic status, and an indicator variable representing whether or not a collision, violation, or license occurred prior to the exposure period. The independent variable of driver education program had four levels: controls, SPC and PDL students who at least partially completed these programs, and a group consisting of students assigned to the SPC or PDL programs who never enrolled in them.

Findings from the multiple regression analyses indicated that the variable "driver education program" was not statistically associated with the number of collisions.

Based on the several DeKalb evaluations, especially the one by Smith and Blatt, NHTSA (1994) concluded that only the PDL-the minimal course -and not the SPC resulted in any significant reduction in crashes. They also noted that even these positive effects were not nearly at the level expected of the training. The fact that the PDL, the minimal program, and not the SPC, the then state-of-the-art program, was found effective at least in the one study by DeWolf and Smith has remained perplexing.

Based primarily on the disappointing SPC demonstration evaluation results, high school driver education was determined not to be a priority program of NHTSA during the 1980s and early 1990s.

Critiques and re-analyses of DeKalb. The Safe Performance Curriculum represented a major departure from driver education courses in content and duration and there was no doubt considerable expectation at the time that it would produce significant safety benefits. However, as reviewed above, the evaluation of its impact produced anything but major or clear-cut findings. A short-term impact was detected but these effects were mitigated by increased exposure due to earlier licensing; longer-term effects were evident for the PDL but not the more extensive SPC program.

Given the unprecedented commitment in time and resources to the DeKalb project and the rather equivocal nature of the findings, it is not surprising that it has continued to be the subject of considerable discussion and debate. The more prominent re-analyses of the DeKalb study are described below.

Lund et al. (1986) re-analyzed the original DeKalb study data and produced results that failed to show the positive short-term impact of driver education. Indeed, they showed a negative effect of formal instruction. A key point in the Lund et al. work relates to the differences in exposure among licensed SPC, PDL and controls. It is to be recalled that Stock et al. had found differences in exposure among the groups but concluded that these could not account for the difference in crash ratios. Lund et al. challenged this conclusion. Table 2-3, reproduced from Lund et al., presents crash records and exposure data for subjects in the three groups. As can be seen in the Table and as discussed previously, during their first six months of licensed driving SPC students had the lowest crash mean-.1054 compared to .1066 and .1221 for PDL and control students, respectively. This shows that the crash mean for SPC students was only 86 percent of the crash mean of control students.

SPC and PDL students also drove fewer hours than control students-means of .68, .72, and .79 hours driven, respectively. The mean hours driven by SPC students is 86 percent of that driven by control students-a figure that is consistent with that reported for the crash means. The figure for the mean number of miles driven is also similar-84 percent. Thus, the lower crash means of SPC and PDL students, compared to those in the control group may reflect lower amounts of driving reported by SPC and PDL students and not the effect of taking a driver education program. Lund et al. concluded that "the exposure differences reported by the licensed drivers are consistent with the differences in subsequent crash records of licensed drivers in the three groups" (p.351). Of course, aspects of the course -e.g., a cautious approach to driving-may have caused the reduced exposure among the SPC and PDL students and this could be seen as a positive effect of formal instruction. Lund et al. rejected this possibility and suggested what they refer to as a simpler explanation:

... the likelihood that the PDL and SPC education groups contained some licensed drivers who would not have obtained their licenses without the additional stimulus of driver education. These students may have had less need for personal transportation than did the control students, who obtained licenses without the additional stimulus of driver education. Thus, the reduced crash experience per licensed driver among those assigned to driver education reflects lower average driving exposure, which is probably due to different reasons for licensing. (p.351)

TABLE 2-3 *

Crash records and self-reported driving exposure of licensed drivers in the DeKalb study

Educational Groups

Crashes

Hours driven

Miles driven

6-month mean

% of control

"yesterday's" mean

% of control

"yesterday's" mean

% of control

SPC

0.1054

86

0.68

86

21.05

84

PDL

0.1066

87

0.72

91

22.82

84

CONTR.

0.1221

0.79

24.93

*Lund et al 1986

Although this is a compelling explanation, it is speculative and therefore does not totally eliminate the possibility that taking the courses resulted in lower average driving exposure among graduates. Whatever the explanation, however, the earlier licensure of SPC and PDL students that results in increased exposure to crash risk may be mitigated to some extent by less driving among the two training groups.

The observations by Lund et al. also raise a major interpretive problem with analyses that focus on only licensed drivers and not the entire study sample. The subsets may not retain the group equivalence that was originally established through random assignment. The fact that SPC, PDL and control students choose whether to become licensed and when to do so potentially introduces a self-selection bias. That is, licensed students in the SPC, PDL and control groups may differ on important characteristics and attributes related to crash involvement-differences that did not exist in the original groups. In fact, Stock et al. (1983) recognized this possibility and cautioned that the assumption of group equivalence achieved by random assignment is valid only for the original SPC, PDL and control groups. They state that "the licensed and complete and licensed students were randomly assigned initially, but through self-selection mechanisms only a portion of the assigned students become licensed or complete the course and become licensed" (p. 1 1-36). As a consequence, they caution that results "for the licensed and complete and licensed students must be interpreted and generalized with caution". Unfortunately, this cautionary note does not appear to have been heeded in subsequent reviews of the DeKalb study results. Davis (1990), for example, in a more recent re-analysis of the DeKalb data noted that:

... in spite of these cautions, the Stock et al. (1983) report has been interpreted by some as evidence in favour of high school driver education (O'Farrell 1983; McKnight, 1985). However, the only results indicative of a significant beneficial effect of driver education in reducing accidents and violations were based on analyses of self-selected subsets of data. (p. 12)

Thus, the positive finding-i.e., that the SPC group has a lower average number of collisions-emerged from a study that can be criticized for suffering from some of the same methodological difficulties-a self-selection bias-that led to the dismissal of the positive results of earlier studies. At issue then is the extent to which the subsets of groups did in fact differ in important ways. The Stock et al. report does contain some related information which suggests that licensed drivers in the three subgroups did not differ greatly from one another, at least in terms of those factors on which they were initially matched-i.e., gender, grade point average, and socioeconomic status. For example, among the subgroups that included those who became licensed, males comprised 52 percent of the SPC group, 52.8 percent of the PDL group and 53.7 percent of the control group. With respect to socioeconomic status the composition of the licensed subgroup of SPC, PDL and control groups were very similar-30.2 percent of SPC, 31.3 percent of PDL and 31.3 percent of control had high socioeconomic status. And the grade point averages (GPA) of the three groups were comparable -61.9 percent of SPC, 60.5 percent of PDL and 60.7 percent of control had a high GPA.

When the comparisons included only students who completed the courses and became licensed, a similar pattern of results emerged with the exception of grade point average. The SPC and PDL groups had a larger proportion of students with high GPAs than the control groups-65.3 percent, 65.8 percent and 59.6 percent, respectively. According to Stock et al., "this difference probably reflects a self-selection factor in completing the SPC and PDL programs" (p. I I- 1 9). These results suggest that for subgroups of licensed drivers or completed/licensed drivers, the composition of the SPC, PDL and control groups was essentially equivalent, with the possible exception of the GPA differences among completed and licensed drivers. These similarities, however, do not eliminate the possibility that SPC and PDL subjects who chose to complete the course and become licensed differ from controls in terms of other important and potentially confounding personal characteristics and attributes that were controlled initially through random assignment. It is therefore conceivable that these person-centered differences and not exposure to the SPC and PDL accounted for the reduced crash experience of SPC students in the first six months of driving. This issue remains unresolved.

Lund et al. also take issue with the analytic procedures (analysis of variance) applied in the DeKalb study to determine whether differences exist in the crash experience of SPC, PDL and control students. They re-analyzed the data using a technique to estimate the relative hazard (likelihood) of students having their first crash at each month following their 16th birthday. This analytic technique produced results that showed SPC students were 16 percent more likely than controls to obtain licenses and I I percent more likely to crash. Moreover, there was no evidence that SPC reduced the likelihood of crashing, even during the first six months of eligibility for licensure.

In a more recent, and perhaps the most rigorous re analysis of the DeKalb study results, Davis (1990) observed that "in order for a driver education program to have a beneficial effect, any reduction in collisions due to the educational intervention should not be outweighed by increased crash rates due to teenagers obtaining their licenses earlier" (p.22). Accordingly, and similar to the Lund et al. analyses, the re-analyses by Davis included all assigned subjects, not just those who became licensed. The rationale for doing so was stated by Davis:

Previous non-experimental studies which have attempted to study the effectiveness of driver education have more likely studied differences between teenagers who participate in driver education programs and those who do not. In particular, it has been shown that students who choose to take high school driver education tend to have characteristics that independently lead to the occurrence of fewer accidents and violations (Conger et al. 1966). Thus, it seems most appropriate to include all study participants in the analyses, not just those who completed the assigned curriculum, obtained driver's licenses within a specified time period, etc. In this way, we will compare the effectiveness of the three programs instead of studying differences among groups of students who choose to participate in the treatment group to which they were assigned. (p. 23)

Davis provided highly technical criticisms of the methods and statistical procedures used in previous analyses of the DeKalb data and attempted to overcome these limitations. As a consequence, his re analysis differs in several important respects from prior ones. For example, Davis examined program differences in collisions during each year of the four year interval following assignment. He specifically focused on the occurrence or non-occurrence of at least one collision during each of these one-year periods. This differs from several of the previous analyses, that focused on the average number of crashes per person, or the average number of collisions in a fixed-length period of driving experience, or the elapsed time from the 16th birthday until first occurrence of a collision.

Davis also used the date a subject was assigned to the SPC, PDL or control group as the date to begin monitoring crash involvement. Prior analyses had used either the date the first license was obtained, the first day of the quarter during which the student was enrolled in the program, or the date of the student's 16th birthday.

The Davis analysis was based on a repeated measures approach-repeated observations in each yearly time interval from each subject-using weighted least squares. In the original analysis of the DeKalb data, Stock et al. used analysis of variance as well as a repeated measures design. Lund re-analysed the data using the COX proportional hazard regression method.

The results of the analysis by Davis showed a significant short-term increase in collisions in the group of students assigned to the SPC and PDL as compared to controls. During the first year, 10.9 percent and 10.4 percent of SPC and PDL students had collisions, compared to only 8.9 percent of the control students (p<.001). The small difference between SPC and PDL was not significant. There were no significant differences between these three groups in subsequent follow-up years.

Davis observes that the negative program effects during the first year are most likely due to the fact that "these students obtained their driver's licenses somewhat earlier than control subjects and thus had more opportunity to be involved in accidents" (p.44). These findings and conclusions are consistent with those of previous analyses that focused on all assigned subjects-i.e., Stock et al. 1983 and Lund et al. 1986.

Finally, in a recent review of the Davis study, Peck (1996) observed that Davis used different rules in excluding subjects from the analysis than did previous investigators. Peck notes that Davis excluded subjects if they had no DMV license or file entries. These excluded subjects, however, were not randomly distributed among the three groups. More specifically, control subjects were less likely to have driving records in the DMV files. Peck showed that this procedure resulted in an exclusion rate that was significantly higher for the control group than the rate for the two education program groups-7.6 percent compared to 5.4 percent. He suggests that this exclusion introduces a slight non-random component into the data because "this differential attrition ostensibly represents unlicensed drivers who had never accumulated an accident or violation record in Georgia" (p.6). Re-analysis of the data including these subjects in the three groups actually exacerbates the differences found by Davis. Peck found that the trained subjects (PDL and SPC combined) now had 22.6 percent more collisions than did controls. If Davis' original data are used, this difference is only 19.7 percent.

SUMMARY: The DeKalb County project stands as the most large scale, well-designed and ambitious effort to evaluate the effectiveness of formal instruction to date. And as demonstrated above, the DeKalb data have also been the object of intense scrutiny and sophisticated re-analyses over the years.

Despite the different methods and statistical procedures that have been applied to the data, however, the findings have been extremely consistent. Taken together, the original Stock et al. study and subsequent analysis of the data show that the improved driver education program was not associated with reliable or significant decreases in crash involvement. Any differences that were found could be attributed to differences in exposure to the likelihood of a collision. The increased collision rates found in the re analyses of the DeKalb data occur largely because students taking driver education become licensed sooner than those who do not.

This finding is consistent with earlier research that showed the availability of driver education encouraged young people to become licensed, and consequently, crash at an earlier age than they would have in the absence of training. For example, Robertson (1980) investigated the effect of the elimination of high school driver education on the overall licensing of 16-17 year olds and their fatal crashes in several communities in Connecticut. The study found that "the elimination of high school driver education in some Connecticut communities following the withdrawal of state funding for the course led to a substantial reduction in early licensure of 16-17 year olds to drive and concurrent substantial net reductions in serious crashes of drivers per population of that age relative to changes in communities that retained the courses". Robertson concluded that:

... driver education in high school is a major contributing factor to the early licensure of teenagers to drive and, as a result of this earlier exposure, their increased involvement as drivers in serious crashes.

McKnight (1985) has, however, taken issue with these conclusions. Basically, he has argued that:

... it is clearly something of a distortion to attribute accidents to driver education just because it leads to driving. Any group of people that drive will have accidents. By agreeing to license them, society accepts that risk. Driver education is simply a means of achieving a socially accepted goal.

Although there is merit in McKnight's perspective, it is not evident that the decision to support driver education as a means to achieving an acceptable goal (licensing) is made through a consideration of the various mobility and safety tradeoffs. Indeed, society agrees to license learners who have successfully completed driver education because of the perceived safety benefits of such programs-driver education is expected to minimize the risk. Legislatures do not mandate driver education to provide youth with an easy access to driving. The critical issue then would be whether society is willing to accept the mobility benefits of driver education/training given the potential safety disbenefits.

A secondary issue relates to the quantity and quality of driving by students who become licensed as a result of the greater availability of driver education. Findings from the DeKalb study suggest that students who take training drive less than those who do not. Thus, the earlier licensure is mitigated to some extent by reduced driving. At issue is whether course participation and/or other motives influence the quantity and quality of driving, and consequently the amount and nature of exposure to the risk of collision.

Apart from the broader issue of the benefits and disbenefits associated with early licensing, the single most controversial finding from the DeKalb study concerns its short term effects. Stock et al. showed that if the analyses were limited to those subjects who became licensed and those who completed the course and became licensed, formal instruction was associated with a short-term reduction in crashes. This conclusion has been rejected by several investigators on two major grounds. First, any small, short-term benefit was offset or neutralized by the fact that, as discussed above, the availability of driver education causes students to become licensed sooner and, because of this greater exposure, to crash at an earlier age. Second, basing the analyses on licensed drivers introduces a non-random element to the data and possibly a self-selection bias as reflected in differential enrollment and completion rates for the SPC and PDL subjects as well as differing licensing rates for SPC, PDL and control subjects. If this is the case, SPC, PDL and control subjects may differ on personal characteristics and attributes that account for the different collision rates, not whether they have taken formal driver instruction. Thus the results based on an analysis of collisions for those subjects who were licensed or completed the courses and became licensed are more difficult to interpret because of a potential self-selection bias, and therefore do not provide an unambiguous answer regarding the effectiveness of formal driver instruction. Indeed, if the analyses are based on all subjects originally assigned to the study and not just on various subgroups who subsequently became licensed or completed the courses or both, the beneficial effects of formal instruction are not detected; indeed, the reverse is found, with trained students having more collisions.

Finally, the most perplexing finding from these investigations is that the PDL, a minimal training program, produced greater safety benefits than did the SPC, an enhanced driver education program. As reported by Smith and Blatt (1987), there was a small (around six percent) but significant reduction in the number of crashes for PDL "assignment' and "training" students as compared to controls for the first year through the first six years of driving experience. They also found that the PDL had a significantly lower number of students who had a crash than controls (about five percent) for the first two years of driving.

These results are puzzling because all other studies that have focused their analysis on the "assignment" level found that PDL subjects had a higher number of collisions or a higher average number of collisions or a higher likelihood of having crashes (although this later finding by Lund et al. was not significant) than the controls. They suggest that this finding was likely due to the higher licensure rate of PDL subjects. This discrepancy in the Smith and Blatt findings compared to those of other investigators can be explained by examining more closely the subjects actually included by Smith and Blatt as "assignment' students. They indicate that the analysis included only those students who had driving records. If this is the case, students assigned to PDL and control groups who either did not obtain a license or did not crash-assuming unlicensed drivers who crash have a driving record-would not have a driving record and therefore would not be included in the analysis at the "assignment" level. As a consequence, ,assignment" level PDL and control students do not include all subjects assigned to these two groups. Again this is a self-selected group of subjects, so the results may not reflect the effects of the PDL but rather other differences between PDL and control subjects.

Thus despite significant effort, the DeKalb evaluation produced findings that failed to provide evidence of the beneficial effects of formal instruction. Not surprisingly, the equivocal nature of the results have led to substantial controversy that has had a profound impact on the field of driver education/training."

Return to Table of Contents


2.2 KALB AND THE EFFECTIVENESS OF NOVICE DRIVER EDUCATION AND TRAINING

In the early 1980s, based in part on the disappointing findings from DeKalb, driver education was dropped by the National Highway Traffic Safety Administration as a priority and federal funding is no longer available. The change in priorities dramatically altered the availability of driver education-e.g., in 1976 in New Jersey, 96 percent of the secondary schools offered driver education programs at no charge to students; in 1986, that figure had declined to 40 percent (Simpson 1989).

The DeKalb study results also precipitated a major rift between the driver education community, who retained their belief in the safety value of education, and some in the research community, who largely viewed formal driver instruction as ineffective at best and counterproductive at worst. Feelings of mistrust between researcher and educator have only recently dissipated, again largely due to growing interest in graduated licensing and the recognition that improved driver education and training could potentially play an important role in such a system.

It also had an impact on future research and evaluation. Relatively few evaluations have been conducted since the DeKalb study. This is not surprising given that, for many, there was no need for further research; for others, the standard set by DeKalb would be impossible to replicate. It may have seemed foolhardy to attempt to replicate and improve upon the DeKalb work or to conduct a more modest evaluation that would have the methodological rigor to withstand the critics of driver education.

The paucity of evaluative work also bears witness to the fact that the DeKalb findings have been taken by many as the final word on the effectiveness of formal instruction. Depending on the interpretation applied to the DeKalb data, driver education did not work or at least had only small short-term benefits. Moreover, given that there was some, albeit questionable, evidence that the minimum training program (PDL) produced more safety benefits than the improved program, there was little guidance provided on how to change or modify existing programs to produce safety benefits.

This section reviews those few evaluations that have been conducted in North America and elsewhere since the DeKalb County Driver Education study to determine whether more contemporary studies of other programs have yielded positive effects. A total of eight studies are reviewed. Only three of these studies were done in the United States; one was conducted in Canada; two in Western Europe; and two in New Zealand and Australia.

Unfortunately, as the review that follows will show, these few evaluation studies do not add much guidance regarding the safety benefits of driver education. Two of the three studies conducted in the United States focused more on the relative effectiveness of high school driver education and commercial driving schools than on the fundamental issue of the absolute effectiveness of formal instruction. The studies conducted elsewhere were primarily interested in determining whether improvements in existing mandatory programs produced safety benefits. Accordingly, the results of these studies do not answer the critical question regarding the absolute effectiveness of formal driver instruction. Nonetheless, a review of these studies does provide some useful insights regarding the potential role of formal driver instruction.

2.2.1 Experience in the United States. Since the DeKalb study was completed only a few evaluation studies have been conducted in the United States. We identified three, in Virginia, Ohio and Oregon.

The Virginia study (Ohlson and Stoke 1986) was actually initiated in the late 1970s, around the same time as DeKalb. At that time, the Virginia Highway and Transportation Research Council commissioned the Virginia Department of Education to evaluate driver education programs across Virginia. The project was not completed, however, until after DeKalb and the technical report is dated 1986. A major purpose of the study was to compare crash rates of students from commercial driving schools to those who received their training from public or private schools. Thus, as mentioned previously, this evaluation concerned the relative effectiveness of two different delivery systems and not the absolute effectiveness of formal driver instruction.

The authors examined crash records over two 12-month periods of students who completed instruction through commercial driving schools, public schools and private schools. No information is provided in the report on the number of students in each of the groups but they were likely in the thousands because the study was conducted statewide. The analyses showed a significantly higher crash rate for those trained in commercial driving schools than for those trained in driver education programs offered in the public or private school system. For example, among male students with less than one year driving experience, those in public high school programs and private schools had 6.3 and 6.7 crashes per 100 students, compared to a crash rate of 10.2 for those who received instruction in commercial schools. Females in public and private schools had crash rates of 4.5 and 3.7, respectively, compared to a crash rate of 7.6 for females trained in commercial schools. A similar pattern of results was observed over one- and two-year periods after instruction.

The study also compared different types of driver education programs offered in the public school including:

The analyses did not show any reliable differences in crash rates as a function of the type of driver education program. For example, among male subjects with less than one year driving experience, those graduating from a four-phase program had the fewest crashes (5.5 crashes per 100 drivers), two-phase students had the next fewest (5.9), three-phase simulator students were third (6.3) and those graduating from a three-phase range program had the most (6.9). However, this pattern of results was reversed for male students who had one to two years of experience. And the pattern changed again for students with two to three years driving experience. Based on these results, the authors concluded that "it is virtually impossible to state with any certainty that any one of these programs is better than any other concerning crash reduction" (p.56).

The results of this study suggest that driver instruction taken in public or private schools is more effective than that taken in commercial driving schools. However, it provides no information on the issue of whether the crash rates of those who received formal instruction differed from the rates of young people who received either informal or no driver instruction at all. It might seem curious that Ohlson and Stoke (1986) did not include a control group-i.e., subjects with no formal instruction-but the reason lies in Virginia law which required that all persons under 18 years of age complete a state-approved driver education course to obtain a driver's license. Due to this mandatory requirement, it was not possible to identify a comparable group of subjects who had not taken training.

Notwithstanding this limitation to the relevance of the study on the overall effectiveness of formal instruction, even the results regarding the relative effectiveness of private or public school driver education versus commercial training are open to question. The reason for this has to do with the assignment of subjects to the various types of instruction. The authors were unable to randomly assign subjects to the different types of programs, and as a consequence, there is no way to determine whether or not the students in the three groups were roughly equivalent in terms of variables that are related to crash involvement. Students who chose to attend a private or public school may be different from those who chose to take commercial training in terms of their socioeconomic backgrounds, attitudes and motivations and consequently may have differed in their likelihood of crash involvement. Ohlson and Stoke did note this problem and other limitations of their study and cautioned that these limitations should ',serve to temper any extreme conclusions or pronouncements that might otherwise be made based on the findings" contained in their report (p. 17).

Some of the problems associated with self-selection were overcome in the segment of the research that examined the relative effectiveness of four types of driver education programs (i.e., two-phase, three-phase range, three-phase simulator, and four-phase). This is so because all subjects in the study were in the public school system and it is likely that those who choose to take driver education are homogeneous in terms of socioeconomic backgrounds, attitudes and motivations. The results of the evaluation of the relative effectiveness of the four types of driver education programs were, therefore, not likely contaminated by self-selection. And the results showed that one method of training was no more effective than another. To some extent this finding is comparable to the DeKalb results regarding the relative effectiveness of the PC and DL. Several of the studies on decal report no statistically significant difference in the crash rates of students who had taken the PDL or SPC.

A similar study was conducted in Ohio by Barner (1987). We were unable to obtain a copy of this report and consequently had to rely on a review of the study by Jones and McCormac (1989). Similar to Virginia, driver education is a pre-licensing requirement in Ohio for 16-17 year olds. Accordingly, Barner was also unable to include a control group-untrained students-in the analyses, which examined the relative effectiveness of high school driver education and commercial training. Similar to the Virginia findings, Barner found that students who took high school driver education had fewer collisions than those who took commercial training. Again, however, owing to the possible confounding effects of self-selection, the author had to caution that "volunteer bias makes it impossible to rule out abnormal skewing of the data" (Jones and McCormac 1989, p.4).

Finally, in a more recent study, Jones and McCormac (1989) evaluated the effectiveness of high school driver education in Oregon. The program is voluntary and consists of 30 hours of classroom instruction, six hours of behind-the-wheel instruction and six hours of in-vehicle observation. Three hours of driver simulator could also be used as a substitute for the behind-the-wheel instruction.

In an attempt to mitigate the potential confounding effects of self-selection, the study compared students who had taken the driver education course with those who had not taken it but who had indicated they would have done so if one were available. The authors recognized that taking driver education is not entirely equivalent to saying one would be prepared to do so, but felt that "screening on this basis improves the design, and results in a more nearly comparable group." To support this claim, the authors showed that this group was more comparable to the trained group in terms of the age at which they became licensed than they were to a group who stated they would not have taken a driver education course had it been available.

The study sample included students ages 16-18 who were applying for their first license. The treatment group included 551 students who completed the driver education course; the control group included 581 students who claimed they would have taken driver education had it been available. The analyses showed the treatment group had a slightly higher crash rate than controls within six months of licensing-10.9 crashes per 100 drivers compared to 9.1. After six months this situation was reversed, with the treatment group having a lower crash rate than controls-a rate of 7.6 compared to 9. 1. All these differences in crash rates between the two groups were not, however, statistically significant.

The authors caution that in interpreting these findings it should be kept in mind that the groups were not matched on potentially confounding variables such as grade point average, so some uncontrolled variables might have affected the results.

Jones and McCormac also examined whether these results could have been influenced by different licensure rates among the treatment and control groups. Importantly, and in contrast to previous studies, the authors did not find that driver education was associated with earlier licensure-i.e. the driver education group was not significantly younger than the control group at the time they became licensed. Actually, the driver education group was, on average, 26 days younger than the controls but this difference was not statistically significant. A possible explanation for this finding is that students who would have taken driver education had it been available were comparable to the driver education group in their interest to become licensed earlier. Indeed, the driver education group voluntarily took the course, presumably because they wanted to get their driver's license. Accordingly, the driver education group and the controls may not have differed in terms of their desire to become licensed. This possibility is further supported by the fact that the group who would not have taken driver education had it been available were on average more than two months older than the controls when they became licensed. This suggests that the availability of driver education did not necessarily encourage earlier licensing; rather, it simply attracted those individuals who would have become licensed earlier anyway. These results appear to be in marked contrast to the DeKalb findings especially considering that the volunteer students had all completed applications for driver education prior to being assigned to the SPC, PDL and control groups. Presumably all these students had a similar interest in becoming licensed given that they had applied to take driver education. Yet, the SPC and PDL students became licensed earlier than the controls. Further examination of the DeKalb study results, however, suggests that the Jones and McCormac findings on licensure are not so dissimilar to those of DeKalb. As mentioned above, Jones and McCormac found that the driver education group was, on average, 26 days younger than the controls (non-significant difference). Using a much larger study sample, Stock et al. (1983) found that "the SPC and the PDL group, on average, had about 23 days and 24 days more of licensed driving, respectively, than the control group" (p. I 1 1-2). Using a different "as-of-date" to determine differences in the number of days of licensed driving, "the SPC group and the PDL group, on average, had about 32 days and 29 days more of licensed driving, respectively, than the control group" (p. 11 1-3). Given the similarities in the findings from these two studies, it is likely that the difference is real and driver education students become licensed, on average, about one month earlier than control students who would have taken driver education had it been available.

In summary, results from the few contemporary studies in the United States do not provide any evidence that formal driver instruction reduces collisions. There is some suggestion that instruction offered in the public or private school systems produces fewer collisions than that offered in commercial driver training programs. However, problems of self-selection -i.e., subjects were not randomly assigned to the different training programs-renders these results questionable. In those public programs where self-selection may not be a serious problem, however, the type of education/training that is taken does not seem to make a difference in terms of collision involvement.

2.2.2 Experience in Canada. Since the DeKalb project, only one Canadian study was identified that examined the effectiveness of driver education. Potvin et. al (1988) evaluated the effects of mandatory driver training introduced as a pre licensing requirement in Quebec in 1983. The training program included 30 hours of in-class instruction and 8 or 10 hours of behind-the-wheel training. It was delivered primarily in commercial driving schools because of the lack of high school driver education programs. Driver education in high schools was never subsidized by the Quebec government or by the local school board, so such programs had not flourished in Quebec as they had, for example, in other Canadian provinces (e.g., Saskatchewan, Manitoba) or in U.S. jurisdictions that did receive some form of financial support. Accordingly, under the new law, as a condition of licensing, persons who were 18 years of age or older were required to take training at commercial schools (at a cost of about $200 US, in 1985).

The evaluation involved an examination of the number of collisions as well as the mortality/morbidity rate per collision of newly licensed 18-25 year olds before and after the legislation was introduced. In the pre-law period, the authors estimated that 30-40 percent of newly licensed drivers over the age of 18 had not taken a driver training course. Accordingly, a large proportion of 18-25 year olds were untrained in the pre-law period whereas all 18-25 year old newly licensed drivers received training in the post-law period. If the new law was effective, it should be reflected in a differential crash rate for newly licensed 18-25 year olds before and after introduction of the requirement for mandatory driver training.

Of course, factors other than the introduction of mandatory training could influence the level of crash involvement in the post-law period-e.g., an economic downturn-and such factors need to be controlled. To control for the possible confounding effects of other factors, the authors also examined changes in the crash involvement of 16 and 17 year olds who would have been unaffected by the new law. As early as 1976, 16 and 17 year olds were already required to take training but older drivers had been exempt until 1983. Accordingly, changes in crash rates among 18-25 year olds following the introduction of the law could be compared to changes in crash rates among 16 and 17 year olds who were not affected by the law but would presumably be affected by other relevant factors.

Interrupted time series methods were used to analyze the data. Results showed that property damage collisions increased after the introduction of the new law in 1983 for both newly licensed drivers age 16-17 and newly licensed drivers age 18-25. For males, this increase was greater among newly licensed drivers age 16-17 than among those age 18-25. This pattern was reversed for newly licensed female drivers-the increase was smaller among 16-17 year olds than among 18-25 year olds. Results also showed that increases in injury collisions after the introduction of the new law were consistently higher for 1617 year olds, particularly for female drivers. The authors concluded that there did appear to be a small beneficial effect of the mandatory training in that it appeared to dampen the increase in property damage collisions among males and injury crashes among females.

However, the changes could have been influenced by the composition of the group of 18-25 year old newly licensed drivers prior to the introduction of the law. As mentioned above, in the pre-law period, 30-40 percent of newly licensed drivers over the age of 18 had not taken a driver training course. Thus 60-70 percent of this pre-law group had taken driver training. Given that a majority of the pre-law group had taken training, it is likely that any positive effects of requiring the balance of 18-25 year olds to take training after 1983 would have been relatively small. In other words, if all of the pre-law, newly licensed 18-25 year olds had been untrained, the intervention in 1983 could have conceivably produced greater benefits. Of course, this possibility is speculative but it does underscore that the heterogeneity of the pre-law group of 18-25 year olds makes interpretation of the results more difficult.

Potvin et al. also examined the extent to which the change in the legislation may have affected the licensure rate. Their data indicate that after the legislation was introduced, the number of women who obtained their license before age 18 increased by 20 percent; the increase among males was 12 percent. Potvin et al. suggests that this increase in licensure among those aged 16 and 17:

... may be due to the fact that after January, 1983, there is no further economic advantage to waiting to be 18 years old before obtaining a first drivers license. Before 1983, teenagers could postpone licensure to spare the cost associated with driver's training. The new law has made postponement impossible ... According to one study, those who would have waited instead of taking driving lessons are less safety-oriented than those who elected to take such a course (Lund et al. 1986). This could explain the observed increase in the risk of accidents for both males and females aged 16 and 17. (p. 1208)

If this is the case, the 16-17 year old comparison group was actually affected by the law change, making interpretation of the findings for the 18 and older experimental group even more difficult. Nonetheless, the Quebec study suggests that driver instruction may have yielded marginal safety benefits, which appear to be offset by the earlier licensure of new drivers. Partly as an outcome of this study, the Quebec government chose to discontinue classroom instruction and required novices to take only the behind-the-wheel training.

2.2.3 Experience in Europe. Two evaluation studies were identified that bear on the issue of driver education effectiveness, one in Denmark and the other in Sweden.

In Denmark, according to Carstensen (1994), a new system of mandatory driver education was implemented in 1986. 'The main aspects of this new system included:

As part of the new program, after basic training the learner was required to complete a course in advanced car control consisting of braking and/or evasive maneuvers on normal and slippery ground. This feature of the program, however, did not become operational until 1990 (added in 1990 but not taken by subjects in this study).

The research design involved an examination of the annual number of collisions involving new drivers, aged 18 and 19, before and after the introduction of the new mandatory driver education program. The crash records of a control group of drivers aged 2554 years, who were unaffected by the implementation of the new driver education program, were also examined for comparison.

The analyses showed that crash rates decreased steadily after 1986 for both age groups but that the decrease among 18-19 year olds was significantly greater than that among the 25-54 year olds. These results are shown in Figure 2-1. To facilitate comparisons, the number of collisions in 1986 -the year the program came into effect-was set at 100. As can be seen, prior to the introduction of the new program the number of collisions for both groups tended to increase. This pattern changed dramatically after 1986, especially for those aged 18-19, the group affected by the new training program. Drivers aged 18-19 experienced a 34.6 percent decrease in their collision rate from 1986 to 1992, compared to a decrease of only 16.7 percent among those aged 25-54. This difference between the two groups was statistically significant. Both a X -test of differences in the average number of collisions before 1987 and after 1987 in the two groups and a multiple regression analysis confirmed this fact (p<0.001 using either technique).

Carstensen also examined whether this differential decrease could be attributable to differences in driving exposure between the two groups. Examining exposure data from various sources, the author concluded that the average number of kilometers driven increased about the same amount among 18-19 year olds as it did among older drivers over the study period. Accordingly, given that changes in driving exposure were similar in the two groups, Carstensen concluded that the new program accounted for the greater decrease in the crash rate among the group exposed to it-the new system seemed to have had a positive effect on crash rates. Carstensen underscored, however, that there are a number of uncontrolled factors that may have affected the results, e.g. "the new system made it more expensive to get a license so the composition of the population getting new licenses may have been different". It was not possible to determine the potential impact of such factors.


Fig 2.1 �


In a recent Swedish study, Gregersen (1994) compared private driving instruction (e.g., parents) to a combination of professional driving instruction and private instruction. Subjects were 17 year old learner drivers in Sweden. The sampling procedure involved randomly drawing 12,000 persons age 17 from the Swedish population and address register and sending them a short questionnaire about their plans for obtaining a driver's license. From the questionnaire, 1,894 persons were identified who planned to learn from private teachers-e.g., parents. Equal numbers of these were assigned to either the experimental or control groups. Ninety-one percent of those assigned to the experimental group agreed to participate (n = 866 persons). During the program 20 subjects dropped out, leaving the sample size of the experimental group at 846.

The experimental group was given professionally supported driver education as well as three additional measures aimed at improving driving ability. The first of these involved theory lessons and practical driving lessons to be combined with private training at home. To facilitate instruction received at home, the private instructor (e.g., parent) was required to participate in the introduction to the course and four of the 16 driving lessons.

Second, a method of commentary driving was introduced in which the learner gives verbal reports of actions, observations, risks, and preventive measures while driving. The purpose of this was to improve the perceptual and scanning skills of the learners -i.e., to become more aware of what was happening around them. It was used for 5-10 minutes during each of the 16 lessons and was developed in four steps:

A third measure introduced a series of training tasks aimed at demonstrating the driver's limitations and reducing overconfidence. This involved setting up potentially dangerous situations involving driving in darkness, driving in icy conditions, and driving in wildlife areas.

The control group did not receive the enriched program. They were taught by parents or private teachers but without professional support and without the three additional measures described above.

Questionnaires were sent to persons in the experimental and control groups immediately after licensing, one year after licensing, and two years after licensing. The first questionnaire asked questions about their learning period, amount of practice and evaluation of the program. The second and third questionnaires asked about amount of driving, number of collisions, estimated driving skills and driving style.

The data on collisions were analyzed in relation to mileage to control for differences in driving exposure between the experimental and control groups. Results showed a statistically significant negative effect of the enriched education program during the first year after licensing-the experimental group had a crash rate of .44 collisions per 10,000 km, compared to .30 for the control group. During the second year, however, this pattern of results was reversed, with the control group having a higher crash rate. Thus the enriched education program had a negative effect during the first year but a positive effect during the second year. When the two years were combined there was no significant difference in the crash rates of the experimental and control groups.

Gregersen notes that the main results of this study are "unexpected" and concludes that aspects of the research design could have influenced the findings. For example, he points out that the study relied on self-reports of collision involvement and that their may have been a tendency for the experimental group to underreport "since they may be more eager to show how skilled they are" (p. 459). This, however, would appear inconsistent with the finding that the experimental group had higher crash rates than the controls did during the first year. Another possibility is that learners and teachers in the experimental group did not comply with their instruction. If this problem were widespread it would diminish the potential impact of the supported education program. He observes, however, that:

The conclusion can be drawn that there have been problems of a normal nature for this kind of field experiment, where there is no possibility for controlling every factor. However, the majority of the participants in the project did a perfect job and followed instructions. In spite of this, the few who had problems may very well have influenced the results of the experimental group, probably in a negative direction. (p.458).

Gregersen also poses several hypothetical explanations for the results. He suggests, for example, that the learner's cognitive capacity becomes overloaded and that awareness of the risks and proper reactions does not necessarily translate into action. It may be that additional information provided during training causes cognitive overload during the first year of driving and this actually increases the new drivers' risk. After a year, this overload is reduced as more and more of the actions become automatic. This explanation is, of course, entirely speculative but if it has validity it does suggest that minimal driver training may be more appropriate initially -i.e., early in the learning process-so that cognitive overload does not occur. To some extent, this would be consistent with the controversial DeKalb finding that the minimal training program (PDL) had positive benefits but the more intensive program (SPC) did not (Smith and Blatt 1987). If this were the case, however, the SPC group should have had lower collision involvement than the PDL group in the second and subsequent years when the cognitive overload problem of learners would have diminished and they could have applied the driving abilities taught a year earlier. This issue warrants further consideration.

In summary, the two studies conducted in Europe produced different results. The Danish study suggests that improved driver education produced immediate and sustained safety benefits. However, other uncontrolled factors may have accounted for these findings. By contrast, the Swedish study suggests improved formal driver instruction can actually have a negative effect during the first year of driving. Safety benefits are only apparent during the second year of driving.

If Gregersen's interpretation has merit, learners, due to cognitive overload, need at least one year to make use of what they have learned in formal driver instruction. The net effect, however, is that formal driver instruction may not result in an overall safety benefit because increased collision involvement during the first year of driving basically offsets the decreased collision involvement during the second year of driving. Unfortunately, Gregersen was unable to monitor crash rates over the third and subsequent years of driving to see if the safety benefits of taking improved driver instruction were more long term.

2.2.4 Experience-Australia and New Zealand. Wynne-Jones et al. (1984) evaluated the Automobile Association secondary school driver training program in Canterbury, New Zealand. The program is given to secondary school students during school hours and involves classroom instruction, eight hours of behind the wheel training, and eight hours as a passenger while another student is being instructed. The classroom instruction includes eight one-and-a-half hour lectures on road traffic law and correct attitudes as well as two one-and-a-half hour lectures on motor mechanics. Parents are also required to give supervised practice between lessons.

The evaluation attempted to eliminate the problem of a self-selection bias by randomly assigning students who volunteered to participate in the project to an experimental (received the course) or control group. A total of 788 subjects were included in the analysis. Crash rates were measured using both self-report data and official crash records.

The authors compared the collision data for the two groups both in teens of initial assignment and actual group attendance. In both cases no statistically significant differences were found in crash rates (determined from official records) between those who had taken the training and those who had not. The analysis of self-reported data, however, showed one statistically significant result. Females in the experimental group actually reported more collisions than those in the control group.

Similar to several of the other studies previously reviewed, Wayne-Jones et al. found that students who had completed the AA course obtained their licenses earlier than those in the control group. For example, males in the experimental group became licensed an average of 111 days after enrollment in the program, compared to 300 days for those in the control group. There was a trend, however, for the experimental group to drive fewer kilometers during the follow-up period than the control group, although this difference was only significant for males. The authors contend that this difference may result from the fact that the control group would, on the whole, be older during the follow-up period because of later licensing and would therefore tend to drive more.

Although not discussed by the authors, differences in driving exposure between the experimental and control groups could have affected the crash rates, but the nature of the effect is not altogether clear. On the one hand, the experimental group might be expected to have more collisions because they became licensed sooner than the control group. On the other hand, even though the experimental group became licensed sooner they drove less than the control group once these untrained drivers became licensed. Unfortunately, the authors did not attempt to control for the influence of driver exposure on the results.

In a study conducted in Melbourne, New South Wales, Australia, Strang et al. (1982) compared the effectiveness of several on-road and off-road training programs, using 790 young male learner drivers aged 17 to 19 years. The authors were specifically interested in determining whether there are any advantages in conducting driver training courses using offroad training areas. Subjects were randomly assigned to one of the following four groups.

The authors note that participation in the project was voluntary and that all groups were similarly constituted, with low drop-out rates after groups were assigned to experimental conditions.

In the study, data were collected on the number of self-reported and police-reported collisions in which subjects had been involved since becoming licensed. The analyses involved a comparison of the proportion of licensed subjects in each of the four groups who had at least one collision and the total number of crashes per group, adjusted for differences in exposure (distance driven).

Results showed no significant differences between the groups with respect to the proportion of subjects who had at least one collision over the first two years of driving. Similarly, no significant differences were found between the four groups with respect to the total number of crashes per group when these numbers were corrected for (small) differences in exposure. The authors suggest that these results may be due to the relatively small number of subjects involved in the study-i.e., because of the small number of subjects in the study, differences of 30 percent between the groups would have been necessary to ensure statistical significance. Based on these findings, Strang et al. conclude that "there is no evidence however, that such training will result in better driving records than for drivers who do not receive formal training" (p. 54). In addition, the authors did not find that off-road training was superior to other training approaches in terms of collision reduction.

In summary, studies conducted in New Zealand and Australia found that formal driver instruction was not associated with a reduction in collisions. One of these studies examined the relative effectiveness of different off- and on-road training programs. No statistically significant differences were found in collision involvement of subjects attending these different programs.

2.2.5 Summary. A review of eight evaluation studies conducted in the United States and elsewhere does not provide any conclusive evidence that formal driver instruction is an effective safety measure.

Two of the three U.S. studies did not examine the absolute effectiveness of driver education. They showed, however, that students taking training in public schools had lower crash rates than those taking training in commercial schools. One of these studies also examined the relative effectiveness of different types of driver education programs offered in public schools; no statistically significant differences were found in the crash rates of students in these different programs. A study in Australia also found no statistically significant effect of different types of driver training -e.g., off-road versus on-road training. By contrast, a Danish study demonstrated that an improved driver instruction program did produce safety benefits.

Those few studies that actually examined the absolute effectiveness of driver education produced mixed results. Studies in the United States, Sweden and Australia suggest that driver education had no positive effect on collisions. In the Swedish study, for example, students taking the enhanced program actually had higher collision rates during their first year of driving. Positive effects of the improved program were, however, observed in the second year after licensing. The net effect over the two years was no significant difference. A study in New Zealand suggests that training had no effect on the collision rates of males but had a negative effect on the crash record of females. And finally, a study in Quebec suggests that a new law mandating driver education seemed to have had a small beneficial effect on crash rates but this appeared to be offset by earlier licensure.

Taken together, these results do not provide much support for the safety benefits of formal driver instruction. The findings, however, do support the conclusions of studies reviewed in the previous section regarding whether formal instruction produces unexpected negative consequences-i.e., the greater availability of driver education stimulates earlier licensure. 2.3 THE EFFECTIVENESS OF MOTORCYCLE RIDER EDUCATION AND TRAINING PROGRAMS

As in the area of formal driver instruction, considerable importance has been placed on education and training of novice motorcycle riders as an effective means of addressing their crash risk. In North America and elsewhere, specialized formal courses have been developed and delivered by instructors/teachers to provide the novice rider with the information and skills deemed necessary to operate a motorcycle and ride safely in traffic.

Formal motorcycle rider education/training has been viewed as critical to the novice's mastery of the demanding skills necessary to operate and control a motorcycle. Many believe that the unique handling characteristics of the motorcycle and the rider's vulnerability to perceptual, aerodynamic and roadway disturbances require the acquisition of a high level of skill -most effectively obtained in a formal training situation. Such initiatives are founded on the fundamental and compelling assumption that students who are exposed to the education/training will be at lower risk of traffic mishap than those who are not.

Similar to early studies on the effectiveness of driver education, almost all of the early evaluations of the effectiveness of motorcycle education courses produced encouraging results-they found that formally trained riders had a lower risk of collision than persons not so trained. However, virtually all of these investigations suffered from some methodological flaw (e.g., no control or comparison group of untrained riders whatsoever, small sample size, no random assignment of subjects to trained and untrained groups, no control for exposure), which made the reliability of the positive findings questionable (Collins 1979; Satten 1980).

By contrast, the better designed studies generally produced disappointing results-they found that formally trained riders were not at lower risk of a collision than riders who did not receive the instruction. In addition, several evaluations actually found that formally trained riders had higher crash rates (per miles ridden) than those who were informally trained.

This section examines findings from the more rigorously designed studies. Similar to the previous section, the discussion is organized by jurisdiction. This approach is necessary given that the evaluations have examined courses that differ in many respects-i.e., in length of instruction, time spent on-bike versus in-class instruction, content, and quality of instruction. Accordingly, findings in one jurisdiction do not necessarily apply to courses offered elsewhere. Nevertheless, the overall direction of the results or the "convergence of evidence" has relevance to the current investigation.

As will become evident from examining this section, developments in motorcycle rider education/training largely parallel those in formal driver instruction. Surprisingly few evaluations have been undertaken to determine the effectiveness of motorcycle rider education/training despite the importance ascribed to these programs in most jurisdictions. As suggested above, the few evaluations that have been undertaken have generally produced discouraging findings and led many to question the value of formal motorcycle rider education and training as an effective loss reduction measure. In spite of these findings, formal training has continued to be popular.

2.3.1 Experience in the United States. Several evaluations have been conducted of the Motorcycle Safety Foundation's Motorcycle Rider Course (MRC). This course consists of 20 hours of instruction-8 of which are spent in the classroom and 12 devoted to practical riding skills.

An evaluation of the MRC in South Dakota, using self-reported information on formally trained and non -trained riders, did not find that those who took the course had a lower crash rate (per miles ridden) (Osga 1980). And Mortimer (1984) compared the self-reported collision experience of 213 MRC graduates with that of 303 riders who had not taken the MRC course in Illinois. He found that "when controlling for age and years licensed, those who took the course did not have a lower accident rate (per million miles ridden) than the control group."

More recently, using a much larger sample, Mortimer (1988) replicated the evaluation of the MRC in Illinois. In this study, he compared 914 MRC graduates with 500 riders who had not taken the MRC on a wide range of safety measures. He reports that "compared to the control group, those who had taken the course did not have a lower violation rate, a lower accident rate, a lower total cost of damage to accident involved motorcycles, a significantly lower mean cost of injury treatment per accident, or a lower total cost of injury treatment."

On the positive side, although there was no difference between the groups in the total cost of damage to collision-involved motorcycles, the average cost per collision was somewhat lower among trained riders and the trained riders made more use of protective clothing (this did not, however, produce significant differences between the groups in the mean cost of injury treatment per collision or the total cost of injury treatment).

One of the most ambitious and sophisticated evaluations of rider education and training was initiated in New York State in the early 1980s. In an effort to overcome the methodological difficulties inherent in most previous studies, the National Highway Traffic Safety Administration (NHTSA) sponsored a large-scale project to evaluate the effectiveness of motorcycle rider education, in combination with improved tests of motorcycle knowledge and skills. The study used an experimental design in which nearly 26,000 new motorcycle license applicants were randomly assigned, over a three-year period (1981-1983), to one of four groups who were to take either (1) the existing New York State motorcycle license test consisting of a brief knowledge test and an in-traffic skill test; (2) the Motorcycle Operator Skill Test (MOST 11) developed by NHTSA; (3) three hours of training plus the MOST 11; or (4) 20 hours of training (MSF's Motorcycle Rider Course that involves 8 hours of classroom instruction and 12 hours of off-street on-cycle training) in addition to the MOST II test.

The motorcycle safety community in the United States viewed the project with considerable interest and optimism since it would "yield valuable data on the effectiveness of different kinds of novice training courses" (Hartman 1982). The study was not expected to produce results that confirmed the negative findings of previous, and what many perceived as less rigorous, research. However, the study found no statistically significant differences in crash rates as a function of the different training and licensing systems. Of particular interest, riders exposed to the 3-hour or 20-hour Motorcycle Rider Course had crash rates that did not differ from the rates of those who had not received formal training.

The study has, however, been criticized on operational and administrative grounds that may have seriously biased the findings. For example, (1989) indicated that "the applicant flow was considerably less than had been expected and administrative problems led to long delays between the dates of application, training and license testing." However, these problems would only be important to the validity of the results if they differentially affected the subjects in the various groups. It is not known to what extent this may have been the case. McPherson also noted that "the applicants utilized in this study were found not to be novices, having an average three years riding experience." This criticism is perhaps more substantive since it could be argued that the training or testing programs cannot be expected to improve the performance of, or differentiate among, more experienced riders. Accordingly, the study may not have been an adequate or appropriate test of either MOST II or the MSF Motorcycle Rider Course.

Similar results were reported by McKnight (1987) from an evaluation of the Pennsylvania Motorcycle safety program. That study examined the collision experience of over 3,000 pairs of trained and untrained motorcycle license applicants, who were matched on the basis of age, sex, and prior driving records. He found no significant differences between the mileage adjusted collision rates of trained and untrained riders.

More recently, Billheimer (1996) published the results of an evaluation of the California Motorcyclist Safety Program (CMSP). Initially implemented in July 1987, the CMSP training program became mandatory in January 1988 for all riders under 18 seeking a California motorcycle license. The mandatory age was subsequently raised in January 1991 to include all riders under 21 years of age. In 1994, a further amendment to the legislation permitted students who successfully completed the course to forego the riding skills test required for licensing by the Department of Motor Vehicles.

The evaluation included several quasi-experimental designs: (1) a pre-post design compared motorcycle collisions before implementation of the CMSP and the introduction of mandatory training to collision experience after its implementation; (2) a pre-post design, with a "control", compared trends in motorcycle collisions in California to overall trends in the United States and (3) a matched group design compared the collision involvement of samples of trained and untrained riders from the Los Angeles area over three time periods following training-six months, one year and two years.

Pre-post comparisons of collision trends revealed that the total number of motorcycle collisions and fatal motorcycle collisions in California had decreased since the implementation of CMSP, especially among younger riders, the primary target for the program. For example, crashes among riders under 25 years of age dropped by 84 percent between 1987 (when the program was introduced) and 1995, compared to only a 50 percent decline among riders over the age of 25 during this same period.

Moreover, riders under 18 years of age experienced an 88 percent drop in crashes from 1988 (when training became mandatory for them) to 1995, compared to a 61 percent decline among riders over the age of 18. Similarly, those age 18 to 20 experienced a 78 percent drop in crashes from 1991 (when training became mandatory for them) to 1995, compared to a drop of only 45 percent among riders age 21 and over.

Although these findings suggest a positive effect of the CMSP, many factors other than training could account for the greater reductions in the number of collisions among young riders-e.g., changing demographics, fewer young licensed riders, relatively less riding by young people. Foremost among these would be changes in the relative numbers of young and older licensed riders. And indeed, the number of licensed riders under the age of 25 declined 68 percent between 1987 and 1995 even though the total number of licensed riders was 5.6 percent higher than in the pre-CMSP period. As a consequence, the crash rate (number of collisions per 1,000 licensed riders) for those under 25 declined only 50 percent after the implementation of the CMSP, compared to a drop of 70 percent for older riders (the crash rate for those under 25 declined from 146 to 72; the crash rate for older riders declined from 39 to 12). Thus, the reduction in collisions among riders under 25 the primary target for the CMSP-was actually less than the reduction among all riders.

Billheimer observes, however, that the number of licensed riders is not an accurate measure of the number of active riders. This is so because older riders renew their license without riding and several years pass before the records of inactive riders are purged from DMV files. This means that the estimate of the number of older licensed riders is spuriously high, producing an artificially low crash rate for that group. Importantly, however, an artificially low crash rate for older riders should apply throughout the study period-i.e., there is no reason to believe that difficulties in accurately estimating the number of older licensed riders became worse in the period after the implementation of the CMSP. Accordingly, the above finding that the decline in collision rate was more pronounced among older riders still holds.

Given his concerns with the licensing data, Billheimer used the number of registered motorcycles, which represents a more accurate estimate of the number of active riders than the number of licensed riders, as the denominator of exposure to calculate fatality rates in the periods before and after the 1987 implementation of the CMSP. This analysis showed that the number of fatalities per registered motorcycle dropped 59 percent from 1987 to 1995-1.2 fatalities per thousand registrations in 1986, to 0.50 fatalities per thousand registrations in 1995. Again however, the extent to which the CMSP contributed to this decline is unknown because the reductions could be attributed to factors other than mandatory training-e.g., an overall decline in riding, the introduction of a mandatory helmet law in 1991. Of special note, CMSP trainees accounted for only 18 percent of active riders by 1995-a total of 95,175 trainees and 525,000 registered motorcycles. This means that the vast majority of active riders had not taken training during the period in which the declines in collisions and fatalities occurred. Unfortunately, the author was unable to disaggregate the motorcycle registration data for riders of different age to determine whether declines in fatality rates were more pronounced among young riders, the primary target for the CMSP, than among older riders.

The author also compared trends in motorcycle collisions in California to those in the rest of the United States. These analyses showed that the number and rate of overall motorcycle collisions and fatal motorcycle collisions have also been dropping throughout the United States but that the declines have been greater in California. For example, the author reports that "on the average, motorcycle fatalities per thousand registrations in California have dropped by 33.1 percent since the formation of the CMSP in 1987. Over the same period, the level in the remainder of the United States only dropped 17.3 percent." (p. 39).

Although these comparative analyses as well as the trend analyses described earlier are suggestive, they do not provide convincing evidence that CMSP contributed to the declines. As Billheimer observes:

While the overall decline in accident trends since the start of the CMSP is gratifying, several factors in addition to the CMSP are likely to have contributed to these trends. These factors include a drop in motorcycle riding, the introduction of a mandatory helmet law in January 1991, and the aging population. (p. 4-1)

Given that factors other than the CMSP are uncontrolled in the analyses, the conclusion that the CMSP did contribute to these crash trends is not supported by the analyses. In fact, the finding that crashes per licensed driver decreased more among older than younger riders-the primary target of the program -could be interpreted as demonstrating that the CMSP actually eroded the decline among young riders. That is, if these riders had not taken training, their crash rates would have dropped even further, to a level similar to that of older riders. Of course, this interpretation is also highly speculative and unsubstantiated. Accordingly, the most that can be concluded is that California has experienced a dramatic decline in motorcycle crashes that has outstripped declines in the rest of the United States and that the CMSP may have contributed to this decline.

Billheimer also used a more rigorous evaluation design to examine the impact of the CMSP. He compared the collision involvement of matched pairs of trained and untrained riders in the Los Angeles/Orange County area over periods of six months, one year and two years. The sample of untrained riders was obtained by means of interviews each year from 1989 to 1994 at places frequented by motorcyclists (dealerships, accessory shops, schools, malls etc.) and at high schools and colleges. This sample was obtained at the same time riders were being trained to minimize potential sources of bias arising from different time frames.

The trained and untrained riders were matched in terms of gender, age, years ridden ("within at least six months under two years"), miles ridden last year (defined only as "within 1,000 miles under 5,000 miles" by the author, p. 4-6) and commute history ("is the motorcycle used to commute?"). Over the five-year study period, 1,158 untrained riders were successfully matched with riders who had completed the basic training program for novice motorcyclists, which lasts a minimum of 16 hours. The matching occurred from a pool of 14,832 trained riders and 16,600 untrained riders who had been interviewed.

In the analyses, Billheimer split the matched pairs of basic course graduates and untrained riders into two groups based on their pre-training (or interview) riding experience: 1) riders with 500 miles or less of prior driving experience and 2) those with more than 500 miles of prior riding experience. This yielded a 2x2 design with training (trained/untrained) and prior riding experience (500 miles or less vs. more than 500 miles) as the variables. These groups were compared in terms of crash rates during four time periods-one year prior to training (or interviewing), six months after training, one year after, and two years after. Based on this analysis, the author concluded that:

... it appears that training has an impact on riders, particularly inexperienced riders, for at least six months following training. Accident rates for untrained riders with little or no prior experience were more than double the rates of trained riders during the first six months after training. Beyond this time, riding experience levels the playing field and accident rates of the trained and untrained groups become indistinguishable. (p. 5-8, 5-9).

Thus, the only statistically significant (i.e., p<.10) difference in crash rates between trained and untrained novice riders occurred among those with little or no prior riding experience in the six month period following training (or interviewing)-the six month crash rate for untrained riders was 0.85 collisions per 100,000 miles, compared to only 0.39 collisions per 100,000 miles for trained riders (statistically significant difference at the 0.10 level). In absolute terms, in the six months following training, 615 trained riders had ridden 1,288,000 miles and had five collisions; by comparison, over the six-month period, the 615 matched untrained riders had ridden 820,000 miles and had seven collisions.

These results suggest a short-term positive effect of training but several methodological weaknesses in the research design cast doubt on their reliability. One issue concerns the extent to which the matching of trained and untrained riders successfully controlled for potentially confounding variables. As noted earlier, Billheimer did match trained and untrained riders on key variables such as gender, age, and riding experience to ensure group comparability and to minimize biases associated with self-selection-i.e., riders who take training may be more safety conscious than those who do not and therefore might have lower crash rates than untrained riders even without the course. However, the self-selection bias has not been completely eliminated because some in the study took basic training voluntarily-e.g., this would apply to 18, 19 and 20-year-olds prior to the mandatory training requirement in 1991, as well as those older than 20 for the duration of the project.

Moreover, Billheimer notes that because of the mandatory training requirement he experienced difficulty identifying untrained riders under the age of 18 for matching with trained riders. To overcome this problem, he included in the untrained group riders who ignored the mandatory training and licensing requirement and rode untrained and, therefore, unlicensed. Thus, some of the matched untrained riders differed from the trained riders on a critical factor -i.e., willingness to obey the training and licensing requirements. This difference is critical because unlicensed riders are overrepresented in fatal crashes. In fact, in another section of the report, Billheimer shows that unlicensed riders comprise about 26 percent of the riding population but account for 65 percent of the fatalities in California. Thus, the untrained group may include riders who have a higher likelihood of collision involvement.

Another design concern arises from the choice of dependent measures -collisions per mile ridden. Billheimer selected this to control for exposure, a control that was necessary because riding increased dramatically among those who took the course. Although an increase in riding was also characteristic of untrained riders in the period after their interview, it was much less pronounced. According to Billheimer, "there is no way of knowing whether confidence instilled by training leads to added riding, or whether the decision to ride (or ride more often) leads riders to take a training course" (p. 4-13). Importantly, if this latter possibility applies and the decision to ride leads to riders taking the course, trained riders once again differ from untrained riders on an important factor-i.e., intentions regarding riding.

A somewhat puzzling finding from the matched group analysis is that the trained novice riders had a higher collision rate than the group of untrained novice riders in the year prior to training-7.1 crashes per 100,000 miles compared to 2.2 crashes per 100,000 miles. Although this difference is not statistically significant, it is substantial and suggests that the trained and untrained groups may have differed in terms of their crash involvement at the outset. The fact that the trained group went from a high crash rate to a lower crash rate in the six months following training (interviewing) could be explained in part by regression to the mean.

Finally, Billheimer suggests that beyond the six month period from training (or interviewing), "riding experience levels the playing field and accident rates of the trained and untrained groups become indistinguishable." Although this may be the case, this same interpretation can be used to explain why trained riders have a collision rate that is lower than the untrained group in the six months following training. Indeed, riding experience increased by 2,200 percent among trained riders (from 56,000 miles in the year before training to 1,288,000 miles in the six months after training), compared to an increase of only 259 percent among untrained novice riders (from 228,000 miles in the year before the interview to 820,000 miles in the six months following the interview). Thus, it could be argued equally that the lower crash rate for trained riders was attributable to greater experience and not the training per se.

Given such methodological concerns, it must be concluded that the study provides little in the way of conclusive evidence that the CMSP contributed to the overall crash reductions witnessed in California from 1987 to 1995, or that it reduced the crash involvement of novice riders during the six months following training.

In summary, several studies have failed to provide definitive conclusions about the effectiveness of rider education and training in reducing crashes.

2.3.2 Experience in Canada. Jonah et al. (1982) conducted an evaluation of the Ontario Safety League Motorcycle Training Program (MTP). It consists of a 20-hour course over two consecutive weekends (four days). About 4 hours of the course is allocated to theory and 16 hours to riding, 6 of which is on-street riding. A group of 811 graduates of the MTP and a group of 1,080 informally trained motorcyclists were surveyed to obtain information about riding experiences (including distance traveled), collisions and violations. Only 62 percent of the MTP graduates could be successfully contacted and 61 percent of these completed the telephone interview. The authors contacted 75 percent of the riders who were informally trained; 5 percent of them had actually received some form of other training and 14 percent were ineligible because they were not licensed or were not active riders.

A simple comparison of the number of collisions showed that MTP graduates were less likely than informally trained riders to be involved in collisions. However, when the analyses controlled for differences between the groups in exposure (miles reported ridden) and rider characteristics, such as age and sex, the differences disappeared-the crash picture was the same for those who were formally trained as it was for those who were not.

More recently, Rothe and Cooper (1987), using both telephone interviews and driver records of those interviewed, gathered information on three samples of motorcyclists: a group of formally trained motorcyclists (those who completed the British Columbia Safety Council's Motorcycle Training Program) who had also passed the Motor Vehicle Department's licensing exam; a group of formally trained motorcyclists (also took the B.C. MTP) who initially failed the licensing exam; and a group of informally trained motorcyclists. The results showed that if significant characteristics of the rider were controlled for in the analysis, motorcycle training was not significantly associated with fewer motorcycle crashes. Rothe and Cooper also repeated the analysis separately for 16-19 year olds and for those aged 16-24 to determine whether motorcycle training was more effective for very young novice riders. They indicate that "in neither case was motorcycle training identified as a potentially significant variable relating to motorcycle accidents" (p. 148). In addition, an examination of the frequency of collisions in the first year or first few years of riding was not significantly related to motorcycle training. Thus according to the authors, "this study could not find any evidence to suggest that motorcycle training (as represented by the single course in question) might have had a salutary safety effect as defined by police-reported accidents even during the initial stages of riding experience" (p. 149).

In contrast to these findings, McDavid et al. (1989) reported that the British Columbia Safety Council's 37-hour motorcycle safety program was associated with fewer motorcycle crashes as well as fewer collisions of all kinds. In this study, the authors compared the crash records of two matched groups of formally trained and informally trained motorcyclists. The matching procedure was adopted to overcome methodological weaknesses of previous evaluation studies related to the lack of similarity between persons who seek motorcycle training and those who do not (self-selection). Basically, the groups were matched in terms of sex (all males), age, location of licensing, the month in 1979 in which they had obtained their motorcycle license, possession of a motor vehicle license prior to acquiring their motorcycle license and previous driving record (crash and moving violation records). This matching procedure resulted in a comparison of 139 trained and 139 untrained riders.

Based on their analyses, McDavid et al. (1989) conclude:

... the findings of this study indicate that trained riders are involved in fewer accidents over time and tend to be involved in less severe accidents. The putative effects of training are not large, but they are persistent, and they can be stated with more confidence than findings from previous research (p. 71).

No doubt the confidence attached to their findings is in part due to the rigorous methods they used for controlling potentially confounding variables. However, the one variable that was not controlled was the amount of exposure. It is possible that formally trained riders had fewer collisions than informally trained riders because they traveled less often and less far on motorcycles. Of course, it could then be argued that formal training does have a positive effect-it results in people riding less often, perhaps even encouraging them not to ride during inclement weather or other potentially hazardous conditions. This is an interesting hypothesis that might be worthy of testing. Formal training may sensitize novice riders to the complexity of the task and to their increased vulnerability under certain conditions, such as low visibility.

Unfortunately, the study by McDavid et al. provides no data that bear on this hypothesis. All that can be concluded from their study is that the group of formally trained riders did have fewer overall collisions than those who were not formally trained, but the importance of this finding is diminished in the absence of exposure data. Moreover, if the analysis is restricted to only motorcycle crashes, this difference is not statistically significant.

An evaluation of rider training has also been conducted in Quebec, the only Canadian province that requires training as a prerequisite for obtaining a motorcycle license. Simard (1987) evaluated the effect of a 1985 law that required all prospective motorcycle riders to take a training course, even if they had previously taken a mandatory automobile driver training course. The crash records of two groups of first year novice riders were compared. One group had taken only the driver education course in 1983 or 1984; the other group had taken both the automobile driver course and the motorcycle rider training courses in 1985. An examination of the total number of motorcyclists injured in collisions revealed that the group subject to the new law (both courses) had considerably fewer collisions in their first year riding (125) than did the group that took only automobile driver training (243).

However, the greater licensing demands associated with the new law also resulted in a pronounced decrease in the number of motorcyclists who took the rider course and became licensed in 1985. This decrease could account for the lower number of collisions (i.e., fewer new riders to have collisions in 1985 compared to 1984). This possibility is supported by the finding that standardized comparisons using crash rates (numbers of collisions per motorcyclist) were actually higher for those who took a rider training course in addition to the driver education course (rate of 85 per 1,000 motorcyclist) than for those who took only the automobile driver training course (rate of 73).

Since these comparisons involved collisions that occurred in different years, factors unrelated to the introduction of the new law in 1985 could have influenced crash frequencies as well as the number of new motorcyclists in 1985. To control for this possibility, Simard also examined the pre- and post-law crash record of two other comparison groups shown below:

1984

1985

"CONTROL 1" COMPARISON

Licensed to ride a motorcycle at the end of 1984. License obtained either in 1983 or 1984. (took the motorcycle course only)

Licensed to ride a motorcycle at the end of 1985. License obtained either in 1984 or 1985. (took the motorcycle course only)

"CONTROL 11" COMPARISON

First licensed to drive an automobile in 1983. Also licensed to ride a motorcycle in 1983. (took driver education course for automobile only)

First licensed to drive an automobile in 1984. Also licensed to ride a motorcycle in 1984. (took driver education course for automobile only)


The first comparison "Control 1"-involved persons who had taken only the motorcycle course either in the pre- or post-law period. Since these motorcyclists had not previously taken automobile driver education, the legal change that occurred in March 1985 did not apply to them. Accordingly, any changes in collision frequencies or in the number of motorcyclists taking the motorcycle rider course from 1984 to 1985 would be accounted for by other factors. The number of these motorcyclists involved in collisions was very similar in 1984 (147) and 1985 (148), and their total numbers declined only slightly in this period (1,486 in 1984 to 1,430 in 1985). Thus the crash rate for persons who were trained and licensed to ride only a motorcycle (unaffected by the new law) increased slightly from 1984 (99) to 1985 (103).

This pattern was considerably different from the dramatic increase in the crash rate experienced by riders most affected by the law, as discussed previously. In contrast the second comparison ("Control 11"), which focused on motorcyclists licensed to drive both types of vehicles but who had taken only a driver education course for an automobile prior to the introduction of the new law (and were presumably unaffected by it), revealed a pattern similar to that of the initial (test) comparison. Fewer motorcyclists were involved in collisions in 1985 than in 1984 (118 compared to 133) but there was an even fewer number of motorcyclists in this group after the law (2,879 in 1985 compared to 3,776 in 1984). This yielded a much higher injury crash rate after the law (rate of 41 in 1985 compared to 35 in 1984).

The author speculates that the dramatic drop in the number of riders in this group may have resulted from the introduction of the new law. Although the law did not come into effect until March 1985, it was actually approved by the National Assembly in June 1984 and publicized extensively throughout that year. Accordingly, some of those licensed to drive an automobile in 1984 who had already taken driver education may have been dissuaded from becoming licensed to ride a motorcycle because they thought the law was already in effect and would require rider training as well.

In summary the Simard study suggests that a law requiring new riders to take a motorcycle training course, even if they had previously taken automobile driver training, is associated with an increase in the crash rate. This occurs even though collision frequencies have dropped because the number of riders has declined more precipitously. Simard (1987) speculates that "a direct effect of the mandatory rider course is to keep off the road those riders who would have, otherwise, shown a lower accident rate ... these riders do not ride very often in the first place. Thus, the low riding frequency rather than the rider's abilities on the road could explain the low accident rate." Perhaps the inconvenience of taking an additional training course dissuaded those who would have ridden infrequently from becoming motorcyclists. If the individuals who took both courses rode frequently, greater exposure to risk explains their higher crash rate

2.3.3 Experience in Europe. In the United Kingdom, Raymond and Tatum (1977) compared the self-reported number of collisions of a sample of motorcyclists (258 riders) who had taken the Royal Automobile Club's motorcycle training scheme with a sample who had not (1, 104). The authors provided no details on the structure and content of the course itself but they found that trained riders had a significantly higher crash rate (per mile ridden) than did untrained riders. According to Plowden and Hillman (1984), the Raymond and Tatum study also found that training appeared to effect a positive change in performance in a few areas and that 17-year-old trainees may have benefited. The authors caution that "it is not possible to infer from this research that attendance at a training scheme may result in inferior riding ability since the predicted performance of the individual rider cannot be determined" (Raymond and Tatum 1977; Plowden and Hillman 1984).

More recently in December 1990, compulsory basic training was introduced as part of an effort to improve the road safety of motorcyclists. The safety impact of this change is currently being evaluated by the Transport Research Laboratory but the results are not yet available.

2.3.4 Summary. A review of the evaluation research conducted in three countries provides no compelling evidence that rider training is associated with reductions in collisions. These findings are consistent with much of the evidence on the effectiveness of formal driver instruction.

Return to Table of Contents

2.4 THE EFFECTIVENESS OF ADVANCED SKILL TRAINING PROGRAMS FOR NOVICES

A practice that is relatively common in some jurisdictions-e.g., in Western Europe-is to require that novices take a course in advanced driving skills. The rationale for such courses is typically described in course manuals as follows:

Even the most competent and careful driver may eventually have to face an emergency situation on the road, where his/her correct judgment and a quick response can make a difference between a collision and a mere incident. He/she must proactively respond to the situation and take the most correct action in a programmed manner.


Thus, the purpose of advanced skill training is to teach a driver to respond quickly and correctly to a majority of the potential emergencies in the driving environment. Advanced skill training, therefore, focused on improving the driver's ability to get out of collision producing situations without a crash actually occurring. The assumption is that if you train people in emergency maneuvers they will improve their skills and, ultimately, perform better in critical situations than untrained or inexperienced drivers. And of course, if they have advanced or enhanced skills and can perform better in these critical situations, they will have a lower crash risk.

In North America, the rationale for advanced skill training can be traced to early crash-based studies in the 1960s and 1970s that showed improper vehicle control in emergency situations often was the cause of the collision (Drahos and Treat 1975). These improper controls included getting into various types of skids on, for example, wet and icy surfaces and not being able to recover, using inappropriate and improper evasive maneuvers, improper off-road recovery, and improper application of the brakes. It was estimated in these early studies that if these problems could be corrected with the appropriate emergency responses, 40 percent to 50 percent of all collisions could be avoided. The logical implications of these findings was that considerable safety benefits would accrue from training drivers in advanced skills such as off-road recovery, skids, evasive maneuvers and controlled braking. And indeed, most of the advanced skills courses cover these critical driving maneuvers.

This section examines the effectiveness of advanced driver instruction in reducing the crashes of young novice drivers. The evaluation studies are divided into the following two major categories: the first set of studies examined the safety benefits of courses that cover a diversity of skills such as off-road recovery and evasive maneuvers needed to respond to unanticipated situations or circumstances; the second set of studies focused on those courses devoted almost exclusively to training in avoiding and/or recovery from a skid. Similar to previous sections, studies within each of these major categories are further clustered by country.

2.4.1 The Effectiveness of Emergency Maneuvers Courses. Only one study was identified that assessed the safety impact of an emergency maneuvers course for novices and this course was for motorcyclists. Anderson, Ford and Peck (1 980) describe a large-scale evaluation that was designed to determine the impact of an improved motorcyclist licensing and testing program in California. As part of the program, remedial training was offered to some of the subjects who failed their first attempt at the Motorcycle Operator Skills Test (MOST). This remedial course consisted of three hours of skills training, covering practice in sharp turns, evasive maneuvers, and quick stops.

In the study, a sample of 40,874 people who applied for a motorcycle license were assigned to one of three groups: one group was required to pass the existing licensing test in California; one group was given the new MOST program; and one group received the MOST, as well as three hours of remedial skill training for those who failed the skill test. Anderson et al. first examined differences in the collision experience between subjects assigned to the regular licensing stream and those assigned to the two groups that were to take the MOST. Those who were to receive just the MOST had an average of 21 percent fewer collisions in the first year than those who were to receive the regular licensing test; those who were to receive MOST, plus remediation in the event of a failure, had IS percent fewer collisions.

As is usually the case in such prospective studies, not all of the individuals assigned to the different groups in this evaluation actually completed the licensing tests or, in the case of the one group, the remedial training if they failed the test. Subsequent analyses were restricted to those individuals who completed the relevant requirements and became licensed. In this case, the group who took MOST together with remediation when needed had fewer collisions than those who completed the regular licensing requirements or just the MOST. Indeed, the group who received MOST plus remediation had 22 percent fewer collisions than those who received the regular test. Moreover, those who took only the MOST did not differ from the group who took the regular licensing test.

The most relevant finding concerns the difference in crash rates between the two groups who received MOST. In one group, remedial training was required for those who did not pass the test. The better safety record of the group that received training points to a beneficial effect of the advanced skill program.

Indeed, further analyses by Anderson et al. confirmed that the remedial training course had a positive effect. They examined the crash record of a subset of the overall sample who had failed their first MOST skill test. This subset consisted of 5,510 original license applicants of which 2,750 received the remedial training and 2,760 did not. An examination of the driver records of these groups revealed that the trained group had 22.3 percent fewer collisions than the control group during the first year (p < .06). None of the other comparisons-for only six months or for two years-were significant at the .05 level.

Again, these results are consistent with their initial findings that showed that the group receiving MOST, and remediation when required, had a better crash record than those receiving only MOST. The subsequent analysis controlled for the effects of failing the test, since it was limited to only those who did so. Among such individuals, the ones who then received remediation had a significantly better crash record.

The authors add an important observation that bears on the interpretation of these findings. They note that nearly one-third of the group who were offered training never actually attended the training and, therefore, remained unlicensed. To control for different licensure rates between the trained and untrained groups, they compared the crash records of trained and untrained licensed riders. This analysis also revealed that the trained group had better records than the controls-i.e., 43.8 percent and 30.5 percent fewer collisions in the first and second year periods following application. These differences were statistically significant.

In discussing these two levels of analysis-comparison of the total sample versus only those who became licensed-the authors observed that:

...unlike the total sample E (trained group) and F (untrained group) analyses, all persons in the E group received the training. Therefore, these analyses might be a better measure of the effects of training, per se, than was the total sample analysis. On the other hand, the total sample analyses is not vulnerable to self-selection bias and also provides a better measure of the net effects of the training program because drivers who do not complete the training and remained unlicensed are an inevitable component of any remedial training requirement (p. 1 1-70).

Anderson et al. speculate that the positive effects of remedial training derived from two sources:

Regardless of the explanation for the effect, the Anderson et al. study provides the strongest evidence thus far regarding the safety benefits of formal instruction. However, it is important to recall that this advanced skill course was only taken by novices who had failed to pass a motorcycle operator skills test. For this reason the potential benefits of the program on license applicants who would have passed the MOST or the standard California license tests remains unknown. In addition, it would be inappropriate to conclude that a similar advanced program for car drivers would be effective, because of differences in the characteristics of the vehicles, their skill requirements and the characteristics of those who choose to operate such vehicles.

2.4.2 The Effectiveness of Skid/wet Surface Training. Several studies have examined the impact of skid or wet surface training on the crash records of young novice drivers. These studies were conducted in the United States, Norway and Finland.

Experience in the United States. Jones (1993) evaluated the effectiveness of the Portland Schools' skid car training program. Participants were 622 Portland-area high school students who had completed a novice driver training course and were invited to participate in a supplemental skills course. Those who volunteered were assigned to the treatment and control groups on a first-come-first-serve basis (strictly speaking, not random assignment). The comparison groups involved 300 students who completed skid car training and 322 students who volunteered to take the course but did not receive it.

The program consisted of a 30-minute classroom session involving a presentation and discussion of concepts of vehicle handling and control, together with behind the wheel training in various vehicle control maneuvers (e.g., techniques for maintaining traction and control) using a specially equipped skid car. This skid car was designed to simulate overseer and loss of traction.

The program had three phases of instruction, as described by Jones:

The driving records of the two groups were compared over a 24-month period following the training. Results showed that, overall, the skid control group had more collisions than the controls but the difference was not statistically significant (30.00 crashes per 100 drivers in the trained group compared to 23.29 for the controls). A different pattern emerged, however, in examining collision rates by gender. This analysis showed that skid car training may be associated with higher crash rates for males in the two years following the training (a crash rate of 37.50 for the skid control group, compared to 25.28 for the controls, which is significant at the .06 level). For females, skid car training had no effect on crash rates.

The data were also analyzed more closely using an analysis of covariance to examine the possible influence of other factors such as license status, driving experience, prior convictions, prior collisions and age. This more sophisticated analysis yielded few additional insights and mainly confirmed the results described above.

Jones offers several explanations for the finding that students, particularly males, who complete skid car training have more, not fewer collisions. He speculates that:

To determine if any of these explanations could be substantiated, Jones examined the crash characteristics of the treatment and control groups. In particular, he sought to test the following three notions:

The detailed analysis revealed that there were differences in crash characteristics between the two groups, so according to the logic developed by Jones, differences in crash rates cannot easily be explained by differences in exposure between the two groups. He also found that drivers who completed skid control training actually did have fewer slick surface and rear-end collisions-the very types of collisions that the training should be expected to prevent. For example, males who had skid training had only 18 percent of their collisions on slick surfaces-i.e., wet, snowy or icy roads-whereas males with no skid training had 36 percent of collisions on slick surfaces.

Finally, Jones argues that there is no evidence to support the contention that students with skid training become overconfident. He found, for example, that students with skid training were no more likely to be at fault in collisions, the two groups were not significantly different in the types of driver errors related to the collision, and there was no statistically significant differences between the two groups in the types of traffic convictions.

These results are rather puzzling. The overall results show a higher crash rate among those who received the training (particularly males), suggesting that skid training has a negative effect. Jones contends that these findings cannot be explained by differences in exposure or overconfidence and risk-taking. So, the explanation for the difference remains unknown. More perplexing perhaps are the findings from the detailed analyses which appear to contradict those of the overall findings. Jones found that trained drivers had fewer collisions on slick roads and fewer rear end collisions, providing support for the beneficial effects of the training.

Given these apparently contradictory findings, Jones concludes that the problem could be with the research design. He states:

Clearly, interpretation of these results is hampered by small group size. It is especially treacherous for instances where statistical tests show marginal significance. Also, the design of this study has a number of important limitations. First, even though the groups appear to be very similar in terms of their social and demographic characteristics, group assignment was unscientific. Second, the period of observation is relatively short, less than two years in most cases. It is questionable whether any study could be designed that would measure the long term effects of skid car training. (p. 15)

Given these limitations it appears as though the author believes that neither the apparently negative nor positive effects of the advanced training detected in the study can be accepted with confidence.

Experience in Europe. Three European studies were identified that examined the safety benefits of anti-skid courses. Perhaps one of the best known and certainly most often cited evaluations on the effectiveness of a slippery surface course was a study by Glad (1988) of a two-phase driver education program, implemented in Norway in 1979 as part of the licensing process. The first phase of the course included theoretical and practical training as well as introductory instruction on slippery surfaces and dark driving. After passing this first phase, drivers received a preliminary driver license valid for two years. During the second of these two years, they had to take a classroom course on defensive driving, a course in slippery surface driving, and a course in dark driving. The slippery surface driving course included:

The night driving course included:

In the evaluation of this program, self-reported collisions of two samples of beginning drivers were compared: 11,100 control group drivers who received their license prior to introduction of the new scheme in 1979 and 12,300 drivers who received a license in 1981.

Multiple regression analyses of pooled data were used to analyze the number of collisions, number of collisions in the dark, and number of collisions on slippery surfaces. The results showed no relationship between the instruction offered in phase one of the licensing program and the number of collisions of any type. Drivers with instruction in the dark driving course in phase two, however, had significantly fewer collisions in the dark than drivers without this course. This effect lasted for the first two years after the course but only for male drivers (drivers were tracked for up to six years).

In contrast to the apparent safety benefits arising from instruction related to driving in the dark, drivers who received the slippery surface course had significantly more collisions in general, more collisions in the dark and more collisions on icy and snow-covered roads than drivers without the course. Again, the difference lasted for the first two years after the course but only for male drivers.

In summary, this Norwegian study shows that a night driving course had a positive effect but a slippery surface driving course had a negative effect. Glad speculates that these findings occur because the dark driving course focuses on dangers and shows how to avoid or reduce these dangers. In contrast, the slippery surface course focuses on improving driver skills so that drivers can cope with the problems related to driving on icy and snow-covered roads. He observes that:

Briefly, it is his contention that drivers, particularly males, leave the slippery surface course with unrealistic views of their own driving skills.

Of some importance, the results of this study by Glad are often cited to demonstrate that skid-training has an unexpected negative effect-i.e., it increases crash risk because of a driver's increased self-confidence. However, rarely cited is the equally important finding that the dark driving course was effective in reducing collisions of male drivers over a two year period.

The effect of driver training on slippery road collisions was examined in another Scandinavian study (Katila et al. 1995). The purpose was to determine the impact of changes in driver training in Finland that sought to reduce collisions among novice drivers and to avoid increasing the risk of all drivers. The new program, which was influenced by the work of Glad (1988), concentrated more on teaching anticipating skills than vehicle handling skills especially concerning driving on slippery roads.

The evaluation compared the self-reported collisions of 30,616 drivers who had become licensed under the old curriculum with 1,025 drivers who became licensed under the new curriculum. Similar to the Norwegian system, the new curriculum was a two-phase program. The authors do not provide information describing the first phase of the program. However, following the first phase there is an independent driving period lasting from 6-24 months, so that learners may get experience and understand their strengths and limitations. Phase two deals with driving in difficult conditions, including slippery surfaces.

Three questionnaires were used to collect data. The first asked about mileage, violations and collisions during the 6-18 month follow-up period. The second questionnaire, completed by learner drivers immediately after completing driver training, asked students to assess their risks and driving skills. The third questionnaire was a follow-up of the drivers who filled out the second questionnaire after 6-12 months of driving.

Results showed that the overall number of collisions among drivers under the new curriculum was greater than that of drivers under the old system. However, when these two groups were compared, controlling for kilometers driven, there were no differences in crash rates.

Further detailed analyses compared younger (aged 18-20) and older (aged 21-50) novice drivers and these comparisons revealed differences between the new and old programs. Younger drivers exposed to the new curriculum had a greater proportion of their collisions in slippery road conditions than did younger drivers under the old program. Older novice drivers under the new program had a lower proportion of their crashes in slippery road conditions. These findings suggest that older novice drivers appear to have benefited from the course while younger novice drivers did not. The negative effect of training among young novice drivers was similar for males and females. The positive effect of training among older novice drivers was greater for males than females-older novice males had 12.7 percent less and older novice females had 4.9 percent less collisions in slippery road conditions in the new program group, compared to the old program group.

To explain these results, the authors examined differences between the two groups in terms of driving exposure and attitudes. Their analyses suggested that exposure to driving in slippery road conditions may have been greater among the drivers under the new system. Since no information was collected on the actual amount driven in slippery road conditions, exposure differences between young and older novices could not be examined. They found, however, that the greatest differences between the two groups were in their attitudes. For example, novice drivers under the new program had more confidence in their slippery road driving skills and saw driving on a slippery road as less risky, just after licensing, than did novices under the old system. These attitudinal differences were similar in both gender and age groups.

The finding that both younger and older novice drivers under the new program had more confidence seems contrary to the finding that older novice drivers benefited from the new program but younger ones did not. Katila et al. offer two explanations to reconcile these findings. First, they suggest that younger and older drivers differ in their motives for driving, so the increased skills and overconfidence from training may have been applied in different ways. They state:

The younger could have more easily used their better slippery driving skills in terms of faster driving and the older on the contrary could have increased their safety margins and risk anticipating while driving on a slippery road.

Another possible explanation they offer is that even though both younger and older drivers' confidence had increased as a result of training, the levels of confidence were quite different according to age. Basically, younger novices had significantly higher confidence in their slippery driving skills than older novices. Thus, differences in confidence and motives, could translate into differences between young and older drivers in the quality and quantity of driving, especially in relation to driving on slippery roads.

They conclude that for younger novices, "if higher confidence is followed by increased driving in slippery road conditions but actual slippery driving skills are not significantly better, it will produce more slippery condition accidents."

2.4.3 Summary. The review of advanced skill training courses for novice drivers produced mixed results. Indeed, it is not possible to provide a single conclusion regarding the benefits of advanced skill training, since the impact can be shown to be positive or negative depending on the type of program or skill as well as the age and gender of the driver.

On a positive note, there is reasonably compelling evidence that remedial advanced skill training for motorcyclists who failed their licensing test has significant safety benefits. Unknown at this time, however, is whether these positive results would hold for those riders who have the necessary skills to pass the test, or, perhaps more importantly, for novice car drivers. In addition, training in nighttime driving appears to have produced reasonably long-term benefits among male drivers.

On the other hand, skid training has fared less well. Studies conducted in the United States, Norway and Finland suggest that this type of advanced skill training has a detrimental effect, especially for young males-i.e., it is associated with an increase rather than a decrease in crash involvement. The effect of such courses on females is mixed; two studies found no effect and one study found that the negative effect on young females was similar to that of males.

But even the findings from studies that showed a negative effect of skid training are not always clear. For example, the evaluation of the Oregon program (Jones 1993) also found that drivers receiving skid training had a higher overall crash rate (suggesting, as did the studies in Finland and Norway, that skid car training is detrimental). However, in marked contrast to the study in Norway, in the Oregon evaluation there was a relative absence of slick surface and rear-end collisions, suggesting that skid car training did have the desired effect, at least in the driving situations in which the student was being trained. This suggests that the overall negative effect associated with skid training had to be accounted for by an increase in collisions on non-slippery surfaces. Why this might have occurred is unknown.

Perhaps of equal concern is the suggestion that there are certain risks and even disbenefits associated with teaching these skills, at least to young male drivers. Males who received training had higher crash rates than those who did not take the training. Authors of the relevant studies have suggested that males trained in these skills become overconfident in their ability and now take unnecessary risks. In other words, the course may increase their confidence to a level that exceeds their improved skills.

The literature, however, has not clearly established whether overconfidence results in aggressive driving behavior and taking unnecessary risks. For example, Jones examined the types of driving errors committed and the extent that students were at fault in the collisions to see if there was any evidence of overconfidence or risk-taking. He found that students with skid training were no more likely to be at fault in the collisions they were involved in, and the type of errors that were identified as contributing to collisions did not appear to support the possibility that skid-trained drivers are more aggressive, more confident, or that they take more risks. By contrast, Katila et al., in examining novice driver attitudes, found that young novice drivers-aged 18-20-under the new advanced training system were considerably more confident in their slippery road driving skills than younger novices under the old system as well as older novices-aged 21-50-under the new system. They suggest that overconfidence and different motives for driving could explain why wet surface training has a detrimental effect on young novices but not older ones.

Does skid training produce overconfidence and a detrimental effect? A definitive answer to this question has not yet been provided. More research is clearly needed to clarify the relationships between improved skills, risk perceptions and collision involvement. However, taken together, the results of several studies suggest that training drivers, particularly young males, in crash avoidance skills such as skid control could result in disbenefits. This issue will be pursued in more detail in the next summary section on the value of formal driver instruction.

Return to Table of Contents


2.5 THE VALUE OF FORMAL INSTRUCTION

The preceding sections of this report reviewed a reasonably large number of studies, whose findings have been mixed and, at times, contradictory. This section attempts to summarize the results of that evaluation research.

2.5.1 The Benefits of Formal Driver Instruction. Of all the studies reviewed in the previous sections, only seven reported findings suggesting that formal driver instruction is effective in reducing collisions and these positive findings are open to question. The studies reported that:

Summarized in this manner, the research suggests that there is evidence to support the claim that formal instruction is, or can be, an effective safety initiative. At the same time, and even in the absence of the methodological caveats that must be applied to this work and which temper the veracity of the conclusions considerably, and notwithstanding the fact that an even larger body of evidence failed to find a positive effect of training, taken strictly at face value the results offer very few insights into which type of program is likely to be effective and why. For example, one study claimed that a relatively rudimentary 20-hour program was effective, while another claimed that only an enhanced 72-hour program had an impact. There are, however, more serious limitations which are discussed in the sections below.

DeKalb results challenged. The positive findings from both the Stock et al. (1983) and Smith and Blatt (1987) studies, which involved analyses of the DeKalb data, have been challenged. Several re-analyses of the DeKalb data produced findings far less favourable and certainly not supportive of formal driver instruction.

Moreover, even if the methodological debate surrounding this study is suspended, the DeKalb results are perplexing. They suggest that the more extensive program (SPC) was effective at least during the first six months but that after this it was the minimum driver education program (PDL) that was effective, not the then state-of-the-art SPC. To our knowledge, the explanation for this anomaly has not been addressed in the literature.

Of course, it is even possible to find evidence that fails to confirm the differential positive effects of either program (e.g., Lund et al.), so it is questionable whether any benefit can be derived from speculations regarding why one program was more effective than the other. The same could be said for speculations about why one program was less detrimental than the other. For example, Lund et al. found a smaller negative effect of the PDL than the SPC. The authors speculate that:

... compared to students receiving SPC or even more typical high school driver education courses, students in the PDL course may have finished the course with less confidence in their driving skills because of their limited behind-the-wheel instruction; this may have resulted in a slower rate of licensure and more caution during their initial periods of solo driving (p. 356).

The observation, albeit speculative, regarding the overconfidence that might be engendered by an enhanced driver education program such as the SPC cannot be readily dismissed since it is similar to the concerns that have been raised about the negative effects of advanced skill training -- i.e., training in skid control and wet surface driving may produce overconfident drivers, with undesirable side effects. The implications of this possibility are discussed later in this section of the report. The important point to be made here is that the differences (or lack of differences) that have been found between the SPC and PDL remain open to speculation and, consequently, do not provide much guidance for improving existing programs.

Methodological problems. Many of the studies that have produced positive findings suffer from methodological problems that impact the reliability of the findings. Indeed, even the well-designed DeKalb study was unable to eliminate completely problems associated with the self-selection of subjects with respect to completing the training and becoming licensed. As a consequence, interpretation of the results remains difficult and the positive findings remain open to question.

Generalizability of the results. The generalizability of some of these findings is difficult to gauge. For example, perhaps the most compelling and reliable evidence in support of formal driver instruction comes from the Anderson et al. (1980) study. This well-designed study showed that a three-hour remedial skill training course for novice motorcyclists who failed a license skills test was effective. At issue is whether or not this remedial course, or some version of it, could improve the skills and safety of novice motorcyclists capable of passing the skills test, or whether it has any relevance to novice car drivers.

o The weight of the evidence. As discussed, we have been able to identify seven studies) that provide some evidence of the safety benefits of formal driver instruction. By contrast, many more studies (about 16) failed to find evidence of any safety benefits and others (about seven) even showed a safety disbenefit. On balance, the weight of the available evidence does not favor the hypothesis that formal instruction has safety benefits. Indeed, there is precious little in the way of reliable evidence to show that formal instruction provides safety benefits. As counterintuitive as this may seem, it is difficult to reach a different conclusion in the face of the total body of evidence.

Comparing one program to another. Some additional evidence of the effectiveness of formal instruction comes from studies that focused on the relative impact of different programs. Four studies provided evidence of a positive impact:

Again, however, the reliability of the results can be challenged on methodological grounds. For example, the positive findings regarding high school driver education versus commercial driver schools come from studies in which subjects could not be randomly assigned to the different groups. As a result of the potential self-selection bias this created, the findings could be a function of differences in the characteristics of those who chose to take training in the commercial sector, rather than the differential effectiveness of the programs offered by the commercial sector.

The positive findings regarding the introduction of new mandatory driver education programs in Denmark and Finland are encouraging. However, these studies did not include a control group of untrained drivers, so it is not possible to draw any conclusions about the merits of the formal instruction compared to no formal training at all--comparable or even greater decreases in collisions could have also occurred amongst a matched group of untrained novice drivers.

These positive findings also have to be balanced against the results of an even greater number of studies that found either no effect or a negative effect of improved or more advanced training programs.

2.5.2. Summary. The review of the scientific evaluations performed to date provides little support for the claim that driver instruction is an effective safety countermeasure--the overwhelming preponderance of evidence fails to show that formally trained students have a lower frequency of crashes than those who do not receive such training. Even worse, a few studies showed a safety disbenefit of driver education/training.

Return to Table of Contents

Go on to the second half of the report >>>

Further comments to this article have been disabled.

All Comments (3)

Showing 1 - 3 comments

Prudence,

Begun, the great internet eudciaton has.

Irene,

How could any of this be better satetd? It couldn't.

secret,

hey there how are you doing.

Truck Driving Jobs

driving information
Cars/Autos Driver Behavior Driver Safety Driving Enforcement Environment Licensing Technology Traffic and drivers Training for drivers Travel and Driving Driving Jobs
other driver info
Driver Resources
travel information for drivers

Travel and Driving