The Audit Subcommittee of the Legislative Management Committee met in Room 131, State Capitol Building, Salt Lake City, Utah, September 17, 2001, from 3:00 p.m. until 4:00 p.m.

Committee Members Present: President Al Mansell, Co-Chairman

Speaker Martin R. Stephens, Co-Chairman

Representative Brad King

Excused Senator Mike Dmitrich

Legislative Audit Staff: Wayne L. Welsh, Auditor General

John M. Schaff, Deputy Auditor General

Tim Osterstock, Audit Manager

Rick Coleman, Audit Manager

Wayne Kidd, Sr. Performance Auditor

Paul Hicken, Lead Auditor

Deanna Herring, Performance Auditor

Aaron Eliason, Performance Auditor

Lynda Maynard, Recording Secretary

Camille Ahlstrom, Legislative Secretary

Other Interested Parties: Richard Anderson, Director, Div of Child and Family Services (DCFS)

Craig Monson, Director, Services Review (OSR)

Jack Green, Director, Liability Mgmt, DCFS

Jill Atwood, KSL TV

• Call to Order

Speaker Stephens called the meeting to order at 3:00 p.m.

• Approval of Minutes

Representative King made a motion that the minutes from the August 29^th meeting be approved. The motion passed.

3. A Performance Audit of Child Welfare Referrals and Cases (Report #2001-08)

Presented by Wayne Kidd, Auditor-in-Charge

In accordance with Utah Code 62A-4a-118, our office has conducted its annual audit of a sample of child welfare referrals to and cases handled by the Division of Child and Family Service's (DCFS). Because of recent statutory changes, this report differs from our prior annual reviews of the Office of Services Review (OSR) by looking more directly at DCFS activities. The main conclusions of this report are:

• •case process review can benefit from caseworker involvement
• •removal statistics raise concerns but removal process seems reasonable
• •DCFS training delivery needs improvement
• •qualitative review tool looks promising

Case Process Review Can Benefit from Caseworker Involvement. We discussed a sample of case process review scores with the foster care workers responsible for the cases. Few caseworkers were aware of how their cases had been scored by OSR until we told them. Generally, caseworkers expressed concerns with the reasonableness and the accuracy of the case process review; some workers also told us that seeing their scores helped them understand casework expectations. If OSR discussed each caseworkers results with them and provided specific feedback it could improve the accuracy of the case process review and help increase the scores. It's important that OSR redesign the case process review because as currently structured it doesn't appear to lead to improved performance. Also, DCFS should review caseworker workload to make sure expectations are reasonable.

Removal Statistics Raise Concerns, but Removal Process Seems Reasonable. Because of public and legislative concern about how children are removed from their homes, we conducted a limited review of DCFS removals. We reviewed the removal process and removal statistics, but we did not determine whether specific children were appropriately removed or not. In our opinion, Utah Code and the DCFS policy manual clearly outline the removal process and DCFS has some internal controls in place to help ensure that appropriate removal decisions are made. Also, there are two department level offices, the Office of Services Review and the Office of Child Protective Ombudsman, that review removals. But, statistics on removals by caseworkers show a lot of inconsistency that needs to be studied in depth.

Training Delivery Needs to Be Improved. DCFS realizes its training program is not sufficient and they are working to make improvements. Utah Code requires that new workers receive core training and on-the-job training before being assigned full case loads. However, caseworkers and administrators admit that because of high caseloads, new employees don't always receive required training. Other reasons that new workers may not receive needed training include high turnover of caseworkers and the delay in the division's efforts to develop and deliver practice model training. As DCFS works to improve its training program, we feel it needs to develop a state-wide uniform system for tracking caseworkers' training hours.

Qualitative Review Tool Looks Promising. OSR has been developing a qualitative case review (QCR) process that has many strengths and appears to be more useful than the case process review discussed earlier. QCR directly evaluates whether DCFS is achieving desired outcomes for children and families rather than focusing on compliance with procedures and file documentation. As a result the QCR communicates to caseworkers that focusing on the mission of the child welfare system is vital and may also provide policymakers the information they need to determine how effectively the child welfare system is working.

Recommendations:

• OSR should provide caseworkers with the results of the scored cases from the case process review and give the caseworkers an opportunity to respond.



• OSR should review the structure of the case process review instrument and design the questions and scoring guidelines to accurately reflect the purpose of the review.



• DCFS should conduct a workload analysis to determine the appropriate workload standards for the caseworkers and review the needed number of child visits by caseworkers.



• DCFS should monitor removals by caseworker and investigate cases/removals by worker with very high and low removal rates.



• DCFS should develop a state-wide uniform system to monitor caseworker training to ensure that caseworkers receive adequate training.

Discussion following presentation:

Richard Anderson, Director, Division of Child and Family Services (DCFS), told the Audit Subcommittee that DCFS knows that there are issues to be studied and addressed by both DCFS and the Department of Human Services. Mr. Anderson said that one of the goals is to obtain a building block for the thirty-seven trainee positions to insure that:

• •staff will have better training and mentoring before they assume their complex duties
• •staff will have better understanding of their assignments
• •staff will be more equipped to manage their workloads, and
• •existing staff will stay more constant and manageable.

Mr Anderson concluded by saying that the audit this year has provided informative and solution-oriented information, from a caseworker perspective. DCFS remains open to recommendations that will improve their abilities to meet the mission of protecting children and adults, while preserving and supporting families.

Mr. Craig Monson, Director, Office of Services Review (OSR), added that OSR has developed an access database to handle the case process review that tracks compliance information by worker, supervisor, region and team. During the summer, OSR goes to each region and presents the results. One difficulty that has been encountered is after the data has been presented to the regions or division there is no way to check how far the information flows. Mr. Monson said that the case review process is a very good process that should be continued, with added checks and balances.

Mr. Monson stressed that OSR uses the following tools to determine their success:

• -A Qualitative Review
• -A Case Process Review, and
• -A Trend Analysis

Motion: President Mansell made a motion that the Performance Audit of Child Welfare Referrals and Cases (Report #2001-08) be approved and sent to the Health and Human Services Interim Committee and the Child Welfare Legislative Oversight Panel. The motion passed.

• Audits in Process

Wayne L. Welsh, Auditor General, told the Audit Subcommittee that there would be two audits ready to be presented to them in October: 1) The Juvenile Justice Follow-up audit and, 2) the Unregistered Vehicles audit.

• Other Business

Mr Welsh suggested that the next Audit Subcommittee Meeting be scheduled for the third week in October. The Audit Subcommittee decided that Monday, October 22^nd, at 3:00 p.m. would best meet their schedules.