In 2021, Minnesota enacted legislation that requires recipients of state-funded pre-K-12 education grants to conduct an evidence-based evaluation. The legislation states: “each grant recipient must compile a report that describes the data that was collected and evaluate the effectiveness of the strategies.”
The Minnesota Board of Pharmacy and MMB partnered with The Abdul Latif Jameel Poverty Action Lab (J-PAL) and a team of academics to test new messages designed to reduce dangerous co-prescribing of opioids with gabapentin and benzodiazepines. The evaluation randomly assigned prescribers to a range of messages (as well as a no message group) and are testing which is most effective. MMB also partnered with J-PAL and academics on a project to identify trusted messages and messengers to encourage Black and Hispanic communities to increase their rates of COVID-19 testing and vaccine uptake. The work also used a randomized control trial to test the right level of incentives to encourage COVID-19 vaccine uptake.
The state has evaluation policies and a learning agenda that outline the principles, people, and activities to use evidence to improve results.
10. Evaluation Policies
In 2018, Minnesota Management and Budget adopted an evaluation policy, which governs its use of evaluations and requires the public release of all completed evaluation reports regardless of findings. The policy outlines key principles for evaluation: rigor, relevance, independence, transparency, and ethics. Part of this requires pre-registration of evaluations and final results. Once complete, evaluations become a part of Minnesota’s Inventory, which shows the evidence base for 730 state interventions, and allows for sorting based on level of evidence, service population, settings, and outcomes.
The state has achieved better outcomes through an enterprise performance management system that engages state leaders in using performance data to continuously improve results.
2. Performance Management
The Minnesota Governor’s Chief of Staff convenes weekly goal review meetings for the administration’s strategic goals. During these meetings, the Governor’s Chief of Staff discusses performance, progress, and next steps for hitting these goals. Further, Minnesota Management and Budget (MMB) publishes performance data for all state agencies in its biennial budget documents. In addition to being available for use by executive branch officials, these data are available to legislators and their staff as they develop budget and policy recommendations.
The state has a culture that supports the sustainable use of data and evidence to deliver results in a transparent, equitable, and ethical manner.
The Minnesota Governor publicly communicates about Minnesota’s use of data and evidence to produce results. The Governor, for example, issues press releases when the state publishes important new data and when Minnesota is recognized for data use by national organizations Results for America and The Pew Charitable Trusts. In such cases, the Governor highlights how the state uses data during interviews and takes time to publicly recognize researchers who provide data that is useful to Minnesota’s state and local policymakers.
Minnesota conducts an annual survey of legislative, county, and executive policymakers asking about the importance and usefulness of evidence in budget and policy decisions. In the 2020 survey, 86% of respondents said that having information about program effectiveness is very important for making decisions. Minnesota also conducts regular engagement surveys where employees are asked, among other questions, how their work contributes to their agency’s mission.
Minnesota maintains publicly available training videos for agency staff on how to develop evidence-based budget proposals. These materials are recirculated to key staff at least annually. Educational presentations on evidence-based practices and standards of evidence are regularly provided to state employees.
The State of Minnesota has an achievement award policy that allows agencies to provide monetary awards to employees who demonstrate outstanding job performance for using data and evidence to improve the delivery of services or outcomes for people served. This policy requires that the process of rewarding individuals or groups be integrated with broader management process and performance management.
The state has senior leadership and governance structures with the mandate to equitably use data to improve results.
5. Data Leadership & Governance
Minnesota has a Chief Data and Evaluation Officer with the authority, staff, and budget to collect, analyze, share, and use data to evaluate and improve the effectiveness of public programs and inform policy decisions affecting them. The Minnesota Chief Data and Evaluation Officer leads a team of analysts and research scientists who partner with state agencies to integrate useful information into strategic planning, budget formulation, and ongoing management.
The state has data policies that outline the principles, people, and activities that govern its data collection and use.
6. Data Policies
In the interest of transparency and improving outcomes for publicly funded programs, a Minnesota statute, the Minnesota Government Data Practices Act, establishes the state’s policy for sharing data with nonprofit organizations, academic institutions, and local government agencies. This publicly available policy indicates that all government data collected, created, received, maintained, or disseminated by a government entity shall be shared unless such sharing is otherwise precluded by federal law or a more specific state statute.
7. Data Infrastructure
The state has improved outcomes through technology infrastructure that allows it to efficiently collect, inventory, and share data.
Minnesota’s Statewide Longitudinal Education Data System and Early Childhood Education Data System match administrative education and employment data from five state agencies. Together, these two systems form a P-20 education data system, which captures, analyzes, and uses student data from preschool to high school, college, and the workforce. The agencies created a public tool to allow for analysis of aggregate-level educational data, including the ability to look at disparities in educational achievement, college graduation rates, employment, and other indicators.
The Evidence Base Demographics data tool, launched in late 2020, provides information on the demographic composition – race, ethnicity, socioeconomic status, or education, for example – of the people included in studies that are reported by national research clearinghouses, regardless of where they are located. It also allows users to dig into the Minnesota Inventory, providing access to data that can help illustrate who is reflected in the evidence base and whether specific groups and constituencies are represented.
The state has improved outcomes by combining and analyzing cross-agency data to inform policy, budget, or management decisions.
8. Data Use
As part of its COVID-19 response, the State of Minnesota had a Community Resiliency and Recovery workgroup composed of external advisors – community leaders representing underrepresented groups including residents identifying as BIPOC, LGBTQIA+, and persons with a disability – at least monthly to discuss the progress of state’s COVID-19 response. A component of these meetings was a discussion of the public data on the Data by Race/Ethnicity section of the state’s COVID-19 dashboard. These meetings led to substantive changes to the dashboard, including adding data on congregate care settings such as nursing homes and people experiencing homelessness.
In late 2020, Minnesota analyzed a combination of Minnesota Department of Health COVID-19 case data with Department of Education learning model data (which shows where students were in-person learning versus remote learning). The results showed minimal difference in COVID-19 cases among elementary schools when comparing in-person to remote learning models. As a result, the Governor chose to allow more flexibility for school districts to provide in-person learning for elementary students along with the support to do so safely.
Issue Areas: Education, Equity, Workforce
9. Evaluation Leadership & Governance
The state has a leadership and governance structure with the authority to use evaluations to improve results.
9. Evaluation Leadership & Governance
Minnesota Management and Budget has a Chief Data and Evaluation Officer with the authority, staff, and budget to conduct experimental and quasi-experimental impact evaluations, as well as advise the Governor and legislature on the evidence of effectiveness of publicly funded services.
The state has improved outcomes by prioritizing data and evidence in its budget process through guidance to agencies and engagement with the legislature.
13. Results-Focused Budget Process
A 2017 Minnesota law required state agencies to include performance data in their biennial budget documents. This performance data is frequently disaggregated to highlight disparities. FY22-23 budget instructions direct agencies to include information about the evidence base for their budget proposals and provide a supplemental form that documents the summary of evidence, citations, and amount to be spent on the activity. Agencies describe their proposed approach’s desired outcome measures, strategies for evaluation, support for effective implementation, implementation considerations, and context, including modifications that differ from the research. Each of those components is scored and assessed based on a rubric by Minnesota Management and Budget. In addition, the state used evidence to inform funding decisions that resulted in $87 million in new or expanded evidence-based programming in the FY20-21 budget.
Two evidence-based grants operated in Minnesota include:
A 2019 Minnesota law created grants to address the state’s opioid addiction and overdose epidemic. In 2020, 77% percent of the grant funding was invested in evidence-based services (promising or proven-effective), including medication-assisted treatment, naloxone, and peer support for substance abuse.
The Minnesota Statewide Health Improvement Program ($17.6 million) requires that 75% of grant funding goes to evidence-based programs, defined as those programs supported by experimental or quasi-experimental design studies; 25% of grant funding may go to theory-based services. The statute allows 100% of funding to tribal nations to be theory-based, recognizing the lack of generalizable evidence and discomfort with this way of knowing in American Indian communities. The practices must: (1) address the leading preventable causes of illness and death, such as tobacco use or exposure, poor diet, and lack of regular physical activity, and other issues as determined by the commissioner; (2) promote the development, availability, and use of evidence-based, community level, comprehensive strategies to create healthy communities; and (3) measure the impact of the evidence-based, community health improvement practices, which over time work to contain health care costs and reduce chronic diseases.
Issue Areas: Equity, Health
16. Repurpose for Results
The state has improved outcomes by moving funds away from programs that are ineffective and toward those that get results.
16. Repurpose for Results
A 2014 Minnesota law (subdivision 7) requires the Minnesota Department of Human Services to use the Self-Support Index to monitor each county’s performance in assisting clients to become self-sufficient. Counties that meet performance targets receive a 2.5% bonus payment from the state, and counties that perform below the expected target must submit a performance improvement plan. In counties where “no improvement is shown by the end of the multi-year plan, the county or tribe’s allocation must be decreased by 2.5%” (Section 256J.626(7)(a)(2).)
A 2016 Minnesota law (Section 14, line 15.21) allows the state to use the savings achieved from reducing the sentences of minor drug offenders for evidence-based drug and mental health treatments for offenders still in prison or under supervised release. The evidence to support this law comes from the Department of Corrections’ own research, which found that providing offenders drug treatment reduces recidivism rates.