MILITARY DISABILITY SYSTEM
Preliminary Observations on Efforts to Improve Performance
Statement of Daniel Bertoni, Director
Education, Workforce, and Income Security
Before the Committee on Veterans' Affairs, U.S. Senate
For Release on Delivery Expected at 10:00 a.m. EDT
Wednesday, May 23, 2012
United States Government Accountability Office (GAO)
Chairman Murray, Ranking Member Burr, and Members of the Committee: I am pleased to be here today to discuss our preliminary observations on the efforts of the Departments of Defense (DOD) and Veterans Affairs (VA) to integrate their disability evaluation systems. DOD and VA began piloting the Integrated Disability Evaluation System (IDES) in 2007 in response to concerns that wounded, ill, or injured servicemembers had to undergo two separate and complex disability assessments, and in order to expedite the delivery of benefits to servicemembers. As of October 1, 2011, IDES had replaced the military services’ existing—or “legacy”—disability evaluation systems for almost all new disability cases. GAO has monitored the evolution of IDES since its pilot phase and our past work highlighted a number of challenges. For instance, we reported in December 2010 that insufficient staff and logistical challenges contributed to delays in completing IDES cases, and recommended the agencies take steps to ensure adequate staffing levels and develop a systematic process for monitoring caseloads.
My statement today focuses on initial observations from our ongoing review for this committee and examines (1) the extent to which IDES is meeting performance goals and (2) DOD and VA efforts to improve its performance. To examine these issues, we analyzed IDES timeliness data from the Veterans Tracking Application (VTA) and customer satisfaction data collected from DOD surveys; interviewed DOD and VA officials responsible for overseeing IDES; visited six military treatment facilities to speak with local military and VA staff who administer the program as well as servicemembers in the IDES process; and reviewed supporting policies and plans. In our ongoing work, we will further review performance data and improvement plans in greater detail. We plan to issue our final report later in 2012. We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.
The IDES process begins at a military treatment facility when a physician identifies one or more medical conditions that may interfere with a servicemember’s ability to perform his or her duties. The process involves four main phases: the Medical Evaluation Board (MEB), the Physical Evaluation Board (PEB), transition out of military service (transition), and VA benefits.
MEB phase: In this phase, medical examinations are conducted and decisions are made by the MEB regarding a servicemember’s ability to continue to serve in the military. This phase involves four stages: (1) the servicemember is counseled by a DOD board liaison on what to expect during the IDES process; (2) the servicemember is counseled by a VA caseworker on what to expect during the IDES process and medical exams are scheduled; (3) medical exams are conducted according to VA standards for exams for disability compensation, by VA, DOD, or contractor physicians; and (4) exam results are used by the MEB to identify conditions that limit the servicemember’s ability to serve in the military. Also during this stage, a servicemember dissatisfied with the MEB assessment of unfitting conditions can seek a rebuttal, or an informal medical review by a physician not on the MEB, or both.
PEB phase: In this subsequent phase, decisions are made about the servicemember’s fitness for duty, disability rating and DOD and VA disability benefits, and the servicemember has opportunities to appeal those decisions. This includes: (1) the informal PEB stage, an administrative review of the case file by the cognizant military branch’s PEB without the presence of the servicemember; (2) VA rating stage, where a VA rating specialist prepares two ratings—one for the conditions that DOD determine made a servicemember unfit for duty, which DOD uses to provide military disability benefits, and the other for all service-connected disabilities, which VA uses to determine VA benefits. In addition, the servicemember has several opportunities to appeal different aspects of their disability evaluations: a servicemember dissatisfied with the decision on whether he or she is fit for duty may request a hearing with a “formal” PEB; a member who disagrees with the formal PEB fitness decision can, under certain conditions, appeal to the reviewing authority of the PEB; and a servicemember can ask for VA to reconsider its ratings decisions based on additional evidence, though only for conditions found to render the servicemember unfit for duty.
Transition phase: If the servicemember is found unfit to serve, he or she enters the transition phase and begins the process of separating from the military. During this time, the servicemember may take accrued leave. Also, DOD board liaisons and VA case managers provide counseling on available benefits and services, such as job assistance.
VA benefits phase: A servicemember found unfit and separated from service becomes a veteran and enters the VA benefits phase. VA finalizes its disability rating after receiving evidence of the servicemember’s date of separation from military service. VA then starts to award monthly disability compensation to the veteran.
DOD and VA established timeliness goals for the IDES process to provide VA benefits to active duty servicemembers within 295 days of being referred into the process, and to reserve component members within 305 days (see fig. 1). DOD and VA also established interim timeliness goals for each phase and stage of the IDES process. These time frames are an improvement over the legacy disability evaluation system, which was estimated to take 540 days to complete. In addition to timeliness, DOD surveys servicemembers on their satisfaction at several points in the process, with a goal of having 80 percent of servicemembers satisfied.
Figure 1: Steps of the IDES Process and Timeliness Goals
DOD applies the 30-day goal for the VA benefits phase to some but not all reservists, depending on their active duty status.
DOD guidance allows 40 more days for reserve component members than for active duty members in completing the first two steps of the process, in to provide sufficient time for employer notification, establish orders for active duty, and compile medical records. However, DOD and VA’s goal for total IDES processing time is only 10 days longer for reserve component members than for active duty members because the VA benefit phase time of 30 days is not included in the 305 days for reserve component members.
Enrollment in IDES continued to grow as IDES completed its worldwide expansion. In fiscal year 2011, 18,651 cases were enrolled in IDES compared to 4,155 in fiscal year 2009 (see fig 2). IDES caseload varies by service, but the Army manages the bulk of cases, accounting for 64 percent of new cases in fiscal year 2011. Additionally, active duty servicemembers represent the majority of IDES cases, accounting for 88 percent of new cases in fiscal year 2011.
Figure 2: New IDES Cases Enrolled and IDES Cases Completed by Fiscal Year
Note: Cases completed include those where servicemembers exited the IDES process, such as those who received benefits or returned to duty.
IDES Processing Times Increased over Time, While Measures of Servicemember Satisfaction Have Shortcomings
Overall IDES Case-Processing Times Steadily Increased Since the Start of IDES
Overall IDES timeliness has steadily worsened since the inception of the program. Since fiscal year 2008, the average number of days for servicemembers cases to be processed and to receive benefits increased from 283 to 394 for active duty cases (compared to the goal of 295 days) and from 297 to 420 for reserve cases (compared to the goal of 305 days). Relatedly, the proportion of cases meeting timeliness goals decreased from more than 63 percent of active duty cases completed during fiscal year 2008 to about 19 percent in fiscal year 2011 (see table 1).
Table 1: Timeliness for IDES Cases Resulting in Receipt of VA Benefits
Fiscal year (of case completion) 2008 2009 2010 2011
Average processing time in days
Active duty (goal = 295) 283 313 357 394
Reserve/National Guard (goal = 305) 297 316 370 420
Percent of cases meeting timeliness goals
Active duty 63.4 50.2 31.6 18.8
Reserve/National Guard 65.0 51.7 37.2 18.0
Source: GAO analysis of DOD and VA data
Note: For purposes of this testimony, GAO opted to not include reserve component time spent in the VA benefit phase in our calculations for overall time because the 30 days allotted for this phase is not included in the 305days overall goal for the reserve component.
When examining timeliness across the four phases that make up IDES, data show that timeliness regularly fell short of interim goals for three—MEB, Transition, and VA Benefits (see fig. 3). For example, for cases that completed the MEB phase in fiscal year 2011, active and reserve component members’ cases took on average of 181 and 188 days respectively to be processed, compared to goals of 100 and 140 days. For the PEB phase, processing times increased over time, but were still within established goals.
Figure 3: Average Processing Time for Each IDES Phase (in Days)
Note: Data shown are for servicemember cases that completed a phase in a particular year. For purposes of this testimony, we opted to not include reserve component time spent in the VA benefit phase in our calculations phase because this goal applies to some but not all reservists, depending on their active duty status.
MEB phase: Significant delays have been occurring in completing medical examinations (medical exam stage) and delivering an MEB decision (the MEB stage). For cases completing the MEB phase in 2011, 31 percent of active and 29 percent of reserve cases met the 45-day goal for the medical exam stage and 20 percent of active case and 17 percent of reserve cases met the 35-day goal for the MEB stage. Officials at some sites we visited told us that MEB phase goals were difficult to meet and not realistic given current resources. At all the facilities we visited, officials told us DOD board liaisons and VA case managers had large case loads. Similarly, some military officials noted that they did not have sufficient numbers of doctors to write the narrative summaries needed to complete the MEB stage in a timely manner. Monthly data produced by DOD subsequent to the data we analyzed show signs of improved timeliness for these two stages: for example, 71 percent of active cases met the goal for the medical exam stage and 43 percent met the goal for the MEB stage in the month of March 2012. However, it is too early to tell the extent to which these results will continue to hold.
PEB phase: PEB processing times goals were also not met in fiscal year 2011 for the informal PEB and VA rating stages. For cases that complete the PEB phase in fiscal year 2011, only 38 percent of active duty cases received an informal PEB decision within the 15 days allotted, and only 32 percent received a preliminary VA rating within the 15-day goal. Also during this phase, the majority of time (75 out of the 120 days) is set aside for servicemembers to appeal decisions—including a formal PEB hearing or a reconsideration of the VA ratings. However, only 20 percent of cases completed in fiscal year 2011 actually had any appeals; calling into question DOD and VA’s assumption on the number of expected appeals and potentially masking processing delays in other mandatory parts of the PEB phase.
Transition phase: The transition phase has consistently taken longer than its 45-day goal—almost twice as long on average. While processing times improved slightly for cases that completed this phase in fiscal year 2011 (from 79 days in 2010 to 76 days in fiscal year 2011 for active duty cases), timeliness has remained consistently problematic since fiscal year 2008. DOD officials suggested that it is difficult to meet the goal for this phase because servicemembers are taking accrued leave—to which they are entitled—before separating from the service. For example, an Army official said that Army policy allows servicemembers to take up to 90 days of accrued leave prior to separating, and that average leave time was about 80 days. Although servicemember leave is skewing the performance data, officials said that they cannot easily back this time out from their tracking system, but are exploring options for doing so, which would be more reflective of a servicemember’s actual total time spent in the evaluation process.
VA benefits phase: Processing time improved somewhat for the benefits phase (48 days in fiscal year 2010 to 38 days in fiscal year 2011), but continued to exceed the 30-day goal for active duty servicemembers. Several factors may contribute to delays in this final phase. VA officials told us that cases cannot be closed without the proper discharge forms and that obtaining these forms from the military services can sometimes be a challenge. Additionally, if data are missing from the IDES tracking system (e.g., the servicemember already separated, but this was not recorded in the database), processing time will continue to accrue for cases that remain open in the system. Officials could not provide data on the extent to which these factors had an impact on processing times for pending cases, but said that once errors are detected and addressed, reported processing times are also corrected.
Shortcomings in the Design and Administration of Servicemember Survey
In addition to timeliness, DOD and VA evaluate IDES performance using the results of servicemember satisfaction surveys. However, shortcomings in how DOD measures and reports satisfaction limit the usefulness of these data for making IDES management decisions.
• Response rates: Survey administration rules may unnecessarily exclude the views of some servicemembers. In principle, all members have an opportunity to complete satisfaction surveys at the end of the MEB, PEB, and transition phases; however, servicemembers become ineligible to complete a survey for either the PEB or transition phases if they did not complete a survey in an earlier phase. Additionally, by only surveying servicemembers who completed a phase, DOD may be missing opportunities to obtain input from servicemembers who exit IDES in the middle of a phase.
• Alternate measure shows lower satisfaction: DOD’s satisfaction measure is based on an average of responses to questions across satisfaction surveys. A servicemember is defined as satisfied if the average of his or her responses is above 3 on a 5-point scale, with 3 denoting neither satisfied nor dissatisfied. Using an alternate measure that defines servicemembers as satisfied only when all of their responses are 4 or above, GAO found satisfaction rates several times lower than DOD’s calculation. Whereas DOD’s calculation results in an overall satisfaction rate of about 67 percent since the inception of IDES, GAO’s alternate calculation resulted in a satisfaction rate of about 24 percent. In our ongoing work, we will continue to analyze variation in satisfaction across servicemember cases using both DOD’s and GAO’s measures of satisfaction.
In our ongoing work, we will continue to assess survey results and their usefulness for measuring performance. In the meantime, DOD is reconsidering alternatives for measuring satisfaction, but has yet to come to a decision. Officials already concluded that the survey, in its current form, is not a useful management tool for determining what changes are needed in IDES and said that it is expensive to administer—costing approximately $4.3 million in total since the start of the IDES pilot. DOD suspended the survey in December 2011 because of financial constraints, but officials told us they plan to resume collecting satisfaction data in fiscal year 2013.
Recent Actions and Ongoing Initiatives May Improve IDES Performance, but It Is Too Early to Assess Their Overall Impact
DOD and VA Took Steps to Address Previously Identified IDES Challenges
DOD and VA have undertaken a number of actions to address IDES challenges—many of which GAO identified in past work. Some actions—such as increased oversight and staffing—represent important steps in the right direction, but progress is uneven in some areas.
• Increased monitoring and oversight: GAO identified the need for agency leadership to provide continuous oversight of IDES in 2008, and reported the need for system-wide monitoring mechanisms in 2010. Since then, agency leadership has established mechanisms to improve communication, monitoring, and accountability. The secretaries of DOD and VA have met several times since February 2011 to discuss progress in improving IDES timeliness and have tasked their agencies to find ways of streamlining the process so that the goals can be reduced. Further, senior Army and Navy officials regularly hold conferences to assess performance and address performance issues, including at specific facilities. For instance, the Army’s meetings are led by its vice-chief of staff and VA’s chief of staff, and include reviews of performance where regional and local facility commanders provide feedback on best practices and challenges. Further, VA holds its own biweekly conferences with local staff responsible for VA’s portion of the process. For example, officials said a recent conference addressed delays at one Army IDES site and discussed how they could be addressed. VA officials noted that examiner staff were reassigned to this site and examiners worked on weekends to address the exam problems at this site.
• Increased staffing for MEB and VA rating: In 2010, we identified challenges with having sufficient staff in a number of key positions, including DOD board liaisons and MEB physicians. DOD and VA are working to address staffing challenges in some of the IDES processes that are most delayed. The Army is in the midst of a major hiring initiative to more than double staffing for its MEBs over its October 2011 level, which will include additional board liaison and MEB physician positions. The Army also plans to hire contact representatives to assist board liaisons with clerical functions, freeing more of the liaisons’ time for counseling servicemembers. Additionally, VA officials said that the agency has more than tripled the staffing of its IDES rating sites to handle the demand for preliminary ratings, rating reconsiderations, and final benefit decisions.
• Resolving diagnostic differences: In our December 2010 report, we identified differences between DOD physicians and VA examiners, especially regarding mental health conditions, as a potential source of delay in IDES. We also noted inconsistencies among services in providing guidance and a lack of a tracking mechanism for determining the extent of diagnostic differences. In response to our recommendation, DOD commissioned a study on the subject. The resulting report confirmed the lack of data on the extent and nature of such differences, and that the Army has established guidance more comprehensive than guidance DOD was developing on how to address diagnostic differences, and recommended that DOD or the other services develop similar guidance. A DOD official told us that consistent guidance across the services, similar to the Army’s, was included in DOD’s December 2011 IDES manual. Also, in response to our recommendation, VA plans to modify the VTA database used to track IDES to collect this information on cases, although the upgrade has been delayed several times.
DOD has other actions underway, including efforts to improve sufficiency of VA examinations, MEB written summaries and reserve component records. We plan to review the status of these efforts as part of our ngoing work, which we anticipate completing later in 2012.
DOD and VA Are Working on Shortcomings in Information Systems, but Efforts to Date Are Limited
DOD and VA are working to address shortcomings in information systems that support the IDES process, although some efforts are still in progress and efforts to date are limited.
• Improving local IDES reporting capability: DOD and VA are implementing solutions to improve the ability of local military treatment facilities to track their IDES cases, but multiple solutions may result in redundant work efforts. Officials told us that the VTA—which is the primary means of tracking the completion of IDES cases—has limited reporting capabilities and staff at local facilities are unable to use it for monitoring the cases for which they are responsible. DOD and VA have been developing improvements to VTA that will allow board liaisons and VA case managers to track the status of their cases. VA plans to include these improvements in the next VTA upgrade, currently scheduled for June 2012. In the meantime, staff at many IDES sites have been using their own local systems to track cases and alleviate limitations in VTA. Further, the military services have been moving ahead with their own solutions. For instance, the Army has deployed its own information system for MEBs and PEBs Army-wide. Meanwhile, DOD has also been piloting its own tracking system at 9 IDES sites. As a result, staff at IDES sites we visited reported having to enter the same data into multiple systems.
• Improving IDES data quality: DOD is taking steps to improve the quality of data in VTA. Our analysis of VTA data identified erroneous or missing dates in at least 4 percent of the cases reviewed. Officials told us that VTA lacks adequate controls to prevent erroneous data entry, and that incorrect dates may be entered, or dates may not be entered at all, which can result in inaccurate timeliness data. In September 2011, DOD began a focused effort with the services to correct erroneous and missing case data in VTA. Officials noted that the Air Force and Navy completed substantial efforts to correct the issues identified at that time, but Army efforts continue. While improved local tracking and reporting capabilities will help facilities identify and correct erroneous data, keeping VTA data accurate will be an ongoing challenge due to a lack of data entry controls.
DOD and VA are also pursuing options to allow the electronic transfer of case files between facilities. We are reviewing the status of this effort as part of our ongoing work.
DOD and VA are Pursuing Broader Solutions to Improve IDES Performance
Based on concerns from the agencies’ secretaries about IDES delays, DOD and VA have undertaken initiatives to achieve time savings for servicemembers. The agencies have begun a business process review to better understand how IDES is operating and identify best practices for possible piloting. This review incorporates several efforts, including,
• Process simulation model: Using data from site visits and VTA, DOD is developing a simulation model of the IDES process. According to a DOD official, this process model will allow the agencies to assess the impact of potential situations or changes on IDES processing times, such as surges in workloads or changes in staffing.
• Fusion diagram: DOD is developing this diagram to identify the various sources of IDES data—including VA claim forms and narrative summaries—and different information technology systems that play a role in supporting the IDES process. Officials said this diagram would allow them to better understand and identify overlaps and gaps in data systems.
Ultimately, according to DOD officials, this business process review could lead to short- and long-term recommendations to improve IDES performance, potentially including changes to the different steps in the IDES process, performance goals, and staffing levels; and possibly the procurement of a new information system to support process improvements. However, a DOD official noted that these efforts are in their early stages, and thus there is no timetable yet for completing the review or providing recommendations to senior DOD and VA leadership.
By merging two duplicative disability evaluation systems, IDES has shown promise for expediting the delivery of DOD and VA benefits to injured servicemembers and is considered by many to be an improvement over the legacy process it replaced. However, nearly 5 years after its inception as a pilot, delays continue to affect the system and their causes are not yet fully understood. Recent initiatives to better understand factors that lead to delays and remedy them are promising, however it remains to be seen what their effect will be. Given the persistent nature of IDES performance challenges, continued attention from senior agency leadership will be critical to ensure that delays are understood and remedied.
We have draft recommendations aimed at helping DOD and VA further address challenges we identified, which we plan to finalize in our forthcoming report after fully considering both DOD and VA’s comments.
Chairman Murray and Ranking Member Burr, this concludes my prepared statement. I would be pleased to respond to any questions that you or other Members of the Committee may have at this time.
Table of Contents