Results in Brief

Performance measures used in the state’s budgetary process were not always reliable. About one-half of the measures we examined lacked sufficient documentation, were based on inappropriate methodologies, or were calculated incorrectly. In addition, the description of the measure frequently did not reflect what was reported. As a result, managers and oversight bodies used unreliable information when evaluating programs and making budget decisions.

All of the agencies included in our audit lacked sufficient controls to help ensure performance measures were reliable. Control weaknesses included inadequate written procedures on how to collect and calculate performance measurement data. In addition, we noted insufficient review of the data collection process. Performance measurement systems should have controls to ensure information is properly collected and reported. Reliability can be improved by the Department of Administration providing agencies with additional guidance and oversight concerning controls over the collection and reporting of performance measures.

Principal Findings

· Some agencies could not provide underlying records to support their performance measures reported in the Executive Budget. Of the 35 measures we examined, 15 did not have sufficient documentation. Although these measures may be accurate, the lack of underlying records prevents the information from being verified. (page 8)

· Thirteen of the measures were based on flawed procedures. For instance, the Northern Nevada Child and Adolescent Services’ method for tracking the average number of clients on the Early Childhood Services waiting list overstated the measure. The agency included individuals that inquired about services but never followed up for an appointment. These individuals accounted for approximately 40% of the waiting list. (page 9)

· Calculation errors resulted in agencies reporting inaccurate data in four of the measures. For example, the Department of Motor Vehicles’ measure of the number of transactions processed by employee per month was calculated incorrectly. The reported number was the average for each of the 18 field offices, not the average of all transactions statewide. As a result, the number reported by DMV was overstated by 21%. (page 9)

· The descriptions of 18 performance measures did not reflect the reported information. For example, the Division of Health Care Financing and Policy described one of its measures as the increase in enrollment of children in Medicaid. However, the Division actually reported the number of children enrolled in August 2000, not the change in enrollment during the year. (page 10)

· Many agencies have not developed written procedures describing how to collect and calculate measures. The effect of not having written procedures was greater because of turnover in positions that collected the data reported in the Executive Budget. This made it especially difficult for some agencies to determine how the performance measures reported in the Executive Budget were derived. (page 11)

· At most agencies, one person collected and calculated performance measurement data with little or no review by anyone else. Performance measures are often the result of numerous calculations. As the number and complexity of calculations increases, the risk of errors increases substantially. (page 12)

· State agencies have been required to include performance measures in the Executive Budget since the 1993 Legislative Session. Although the Department of Administration has provided training to assist in the development of performance measures, the training has not addressed procedures to help ensure reliability. In addition, the Department’s oversight of agencies’ internal controls can be improved by including guidance regarding the reliability of performance measures. (page 12)

Reliability of Performance Measures

Used in the State’s Budget Process

Agency Response

to Audit Recommendations

Recommendation Number		Accepted	Rejected

1	Provide state agencies with guidance on controls over the collection of performance measurement data, including developing written procedures, retaining underlying records, and providing sufficient review.	X

2	Ensure agencies’ biennial internal control reports, prepared pursuant to NRS 353A.025, address the reliability of performance measures.	X

3	Review performance measures for reasonableness of approach prior to being included in the Executive Budget..	X

	TOTALS	3	0