STATEMENT OF

OFFICE OF INSPECTOR GENERAL

U.S. DEPARTMENT OF VETERANS AFFAIRS

BEFORE THE

COMMITTEE ON VETERANS’ AFFAIRS

UNITED STATES SENATE

HEARING ON

VA Mental Health Care:  EvaluatiNG Access and Assessing Care

APRIL 25, 2012

Madam Chairman and Members of the Committee, thank you for the opportunity to discuss the results of a recent Office of Inspector General (OIG) report, Veterans Health Administration – Review of Veterans’ Access to Mental Health Care, on veteran access to mental health care services at VA facilities.  We conducted the review at the request of the Committee, the VA Secretary, and the House Veterans’ Affairs Committee.  The OIG is represented by Ms. Linda A. Halliday, Assistant Inspector General for Audits and Evaluations; Dr. John D. Daigh, Jr., Assistant Inspector General for Healthcare Inspections; Dr. Michael Shepherd, Senior Physician in the OIG’s Office of Healthcare Inspections; and Mr. Larry Reinkemeyer, Director of the OIG’s Kansas City Office of Audits and Evaluations. 

BACKGROUND

Based on concerns that veterans may not be able to access the mental health care they need in a timely manner, the OIG was asked to determine how accurately the Veterans Health Administration (VHA) records wait times for mental health services for both initial (new patients) and follow-up (established patients) visits and if the wait time data VA collects is an accurate depiction of veterans’ ability to access those services.

VHA policy requires all first-time patients referred to or requesting mental health services receive an initial evaluation within 24 hours and a more comprehensive mental health diagnostic and treatment planning evaluation within 14 days.  The primary goal of the initial 24-hour evaluation is to identify patients with urgent care needs and to trigger hospitalization or the immediate initiation of outpatient care when needed.  Primary care providers, mental health providers, other referring licensed independent providers, or licensed independent mental health providers can conduct the initial 24-hour evaluation.

VHA uses two principal measures to monitor access to mental health care.  One measure looks at the percentage of comprehensive patient evaluations completed within 14 days of an initial encounter for patients new to mental health services.  Another method VHA uses is to calculate patient waiting times by measuring the elapsed days from the desired dates[1] of care to the dates of the treatment appointments.  Medical facility schedulers must enter the correct desired dates of care in the system to ensure the accuracy of this measurement.  VHA’s goal is to see patients within 14 days of the desired dates of care.

Review Results

Our review focused on how accurately VHA records wait times for mental health services for initial and follow-up visits and if the wait time data VA collects is an accurate depiction of the veterans’ ability to access those services.  We found:

·         VHA’s mental health performance data is not accurate or reliable.

·         VHA’s measures do not adequately reflect critical dimensions of mental health care access. 

Although VHA collects and reports mental health staffing and productivity data, the inaccuracies in some of the data sources presently hinder the usability of information by VHA decision makers to fully assess current capacity, determine optimal resource distribution, evaluate productivity across the system, and establish mental health staffing and productivity standards.

VHA’s Performance Data Is Not Accurate or Reliable

In VA’s fiscal year (FY) 2011 Performance and Accountability Report (PAR), VHA reported    95 percent of first-time patients received a full mental health evaluation within 14 days. However, the 14-day measure has no real value as VHA measured how long it took VHA to conduct the evaluation, not how long the patient waited to receive an evaluation. VHA’s measurement differed from the measure’s objective that veterans should have further evaluation and initiation of mental health care in 14 days of a trigger encounter.  VHA defined the trigger encounter as the veteran’s contact with the mental health clinic or the veteran’s referral to the mental health service from another provider. 

Using the same data VHA used to calculate the 95 percent success rate shown in the FY 2011 PAR, we conducted an independent assessment to identify the exact date of the trigger encounter (the date the patient initially contacted mental health seeking services, or when another provider referred the patient to mental health).  We then determined when the full evaluation containing a patient history, diagnosis, and treatment plan was completed.  Based on our analysis of that information, we calculated the number of days between a first-time patient’s initial contact in mental health and their full mental health evaluation.  Our analysis projected that VHA provided only 49 percent (approximately 184,000) of first-time patients their evaluation within 14 days. 

VHA does not consider the full mental health evaluation as an appointment for treatment, but rather the evaluation is the prerequisite for VHA to develop a patient-appropriate treatment plan.  Once VHA provides the patient with a full mental health evaluation, VHA schedules the patient for an appointment to begin treatment.  We found that VHA did not always provide both new and established patients their treatment appointments within 14 days of the patients’ desired date.  We reviewed patient records to identify the desired date (generally located in the physician’s note as the date the patient needed to return to the clinic or shown as a referral from another provider) and calculated the elapsed days to the date of the patient’s completed treatment appointment date. 

We projected nationwide that in FY 2011, VHA:

·         Completed approximately 168,000 (64 percent) new patient appointments for treatment within 14 days of their desired date; thus, approximately 94,000 (36 percent) appointments nationwide exceeded 14 days.  VHA data reported in the PAR showed that 95 percent received timely care. 

·         Completed approximately 8.8 million (88 percent) follow-up appointments for treatment within 14 days of the desired date; thus, approximately 1.2 million (12 percent) appointments nationwide exceeded 14 days.  VHA data reported in the PAR showed that 98 percent received timely care for treatment.  Although we based our analysis on dates documented in VHA’s medical records, we have less confidence in the integrity of this date information because providers at three of the four medical centers we visited told us they requested a desired date of care based on their schedule availability.

Scheduling Process

Generally, VHA schedulers were not following procedures outlined in VHA directives and, as a result, data was not accurate or reliable.  For new patients, the scheduling clerks frequently stated they used the next available appointment slot as the desired appointment date for new patients.  Even though a consult referral, or contact from the veteran requesting care, may have been submitted weeks or months earlier than the patient’s appointment date, the desired appointment date was determined by and recorded as the next available appointment date.  For established patients, medical providers told us they frequently scheduled the return to clinic date based on their known availability rather than the patient’s clinical need.  Providers may not have availability for 2–3 months, so they specify their availability as the return to clinic time frame. 

OIG first reported concerns with VHA’s calculated wait time data in our Audit of VHA’s Outpatient Scheduling Procedures (July 8, 2005) and Audit of VHA’s Outpatient Wait Times (September 10, 2007).  During both audits, OIG found that schedulers were entering an incorrect desired date. Nearly 7 years later, we still find that the patient scheduling system is broken, the appointment data is inaccurate, and schedulers implement inconsistent practices capturing appointment information.  

Workload and Staffing

According to VHA, from 2005 to 2010, mental health services increased their staff by 46 percent and treated 39 percent more patients.  Despite the increase in mental health care providers, VHA’s mental health care service staff still do not believe they have enough staff to handle the increased workload and to consistently see patients within 14 days of the desired dates.  In July 2011, the Senate Committee on Veterans’ Affairs requested VA to conduct a survey that among other questions asked mental health professionals whether their medical center had adequate mental health staff to meet current veteran demands for care; 71 percent responded their medical center did not have adequate numbers of mental health staff. 

Based on our interviews at four VA medical centers (Denver, Colorado; Spokane, Washington; Milwaukee, Wisconsin; and Salisbury, North Carolina), staff in charge of mental health services reported VHA’s greatest challenge has been to hire and retain psychiatrists.  We analyzed access to psychiatrists at the four visited medical centers by determining how long a patient would have to wait for the physician’s third next available appointment.  Calculating the wait time to the third next available appointment is a common practice for assessing a provider’s ability to see patients in a timely manner.  On average at the four VA medical centers we visited, a patient had to wait 41 days. 

VHA’s Measures Do Not Adequately Reflect Critical Dimensions of Mental Health Care Access

The data and measures needed by decision makers for effective planning and service provision may differ at the national, Veterans Integrated Service Network, and facility level.  No measure of access is perfect or provides a complete picture.  Meaningful analysis and decision making requires reliable data, on not only the timeliness of access but also on trends in demand for mental health services, treatments, and providers; the availability and mix of mental health staffing; provider productivity; and treatment capacity.  These demand and supply variables in turn feed back upon a system’s ability to provide treatment that is patient centered and timely.

Decision makers need measures that:

·         Are derived from data that is reliable and has been consistently determined system-wide.

·         Are based on reasonable assumptions and anchored by a reasonable and consistent set of business rules.

·         Are measureable in practice given existing infrastructure.

·         Are clinically or administratively relevant.

·         Provide complementary or competing information to other measures used by decision makers.

·         Measure what they intend to measure.

Measuring Access to VHA Mental Health Care

Included in the FY 2012 Network Director Performance Plan are the following measures:  the percentage of eligible patient evaluations documented within 14 days of a new mental health patient initial encounter; a metric requiring a follow-up encounter within 7 days of discharge from inpatient hospitalization; a measure requiring four follow-up encounters within 4 weeks of discharge from inpatient treatment for high risk patients; and a measure of the percentage of new Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF) veterans receiving eight psychotherapy sessions within a 14-week period during one year period. 

VHA’s 14-day measure calculates the percentage of comprehensive patient evaluations documented within 14 days of an initial encounter for patients new to mental health services.  In practice, the 14-day measure is usually not triggered until the veteran is actually seen in a mental health clinic and a comprehensive mental health evaluation is initiated.  For example, a new-to-VHA veteran presents to a primary care clinic, screens positive for depression, and the primary care provider refers the veteran for further evaluation by a mental health provider.  The “clock” for the 14-day follow-up measure will start when the veteran is actually seen in a mental health clinic and a comprehensive mental health evaluation is initiated, not at the time of the primary care appointment.  Consequently, the data underlying this measure only provides information about the timeliness within which comprehensive new patient evaluations are completed but not necessarily the timeliness between referral or consult to evaluation.

Veterans access VHA care through various routes, such as VA medical center emergency departments, primary and specialty care clinics, women’s clinics, or mental health walk-in clinics.  Alternatively, they may seek services at community based outpatient clinics or Vet Centers in their communities.  They may also initiate mental health services with private providers and later come to VA seeking more comprehensive services.  The 14-day measure does not apply to veterans who access services through Vet Centers or non-VA-based fee basis providers. 

A series of complementary and competing timeliness and treatment engagement measures that better reflect the various dimensions of access would provide decision makers with a more comprehensive view of the ability with which new patients can access mental health treatment.

The timeframe immediately following inpatient discharge is a period of high risk.  The 7-day post-hospitalization and the four follow-up appointments in 4 weeks for high-risk patient measures are clinically relevant.  The eight psychotherapy session in 14 weeks measure attempts to be a proxy for whether OEF/OIF patients are receiving evidence-based psychotherapy.  The measure is clinically relevant but the utility is presently marred by inaccurate data or unreliable methodology. 

Beyond measures of timeliness (or delay) to mental health care, user friendly measures that incorporate aspects of patient demand, availability and mix of mental health clinical staffing, provider productivity, and treatment capacity, anchored by a consistent set of business rules, might provide VHA decision makers with more information from which to assess and timely respond to changes in access parameters.

Recommendations

Our report contained four recommendations for the Under Secretary for Health:

·         Revise the current full mental health evaluation measurement to ensure the measurement is calculated from the veterans contact with the mental health clinic or the veteran’s referral to the mental health service from another provider to the completion of the evaluation.

·         Reevaluate alternative measures or combinations of measure that could effectively and accurately reflect the patient experience of access to mental health appointments. 

·         Conduct a staffing analysis to determine if mental health staff vacancies represent a systemic issue impeding the Veterans Health Administration’s ability to meet mental health timeliness goals, and if so, develop an action plan to correct the impediments.

·         Ensure that data collection efforts related to mental health access are aligned with the operational needs of relevant decision makers throughout the organization.

The Under Secretary for Health concurred with our recommendations and presented an action plan.  We will follow-up as appropriate.

CONCLUSION

VHA does not have a reliable and accurate method of determining whether they are providing patients timely access to mental health care services.  VHA did not provide first-time patients with timely mental health evaluations and existing patients often waited more than 14 days past their desired date of care for their treatment appointment.  As a result, performance measures used to report patient’s access to mental health care do not depict the true picture of a patient’s waiting time to see a mental health provider. 

While no measure will be complete, meaningful analysis and decision making requires reliable data.  A series of paired timeliness and treatment engagement measures might provide decision makers with a more comprehensive view of the ability with which new patients can access mental health treatment.

Madam Chairman, thank you for the opportunity to discuss our work.  We would be pleased to answer any questions that you or other members of the committee may have.

1] The desired date of care is defined as the earliest date that the patient or clinician specifies the patient needs to be seen