Return on Investment (part 2)

I mentioned earlier our "Return on Investment" conference. Here's a bit more.

Michael Stegman, of the MacArthur Foundation, spoke on “The Power of Measuring Social Benefits” and advocated the development of a culture of evidence-based decision making for all of our policies and programs. He outlined the work of the John D. and Catherine T. MacArthur Foundation and informed the group of their interest in potentially working with Wilder Research on a return-on-investment study of supportive housing programs. (We are discussing with MacArthur a study which will develop a framework for return-on-investment analysis of different types of supportive housing programs and which will pioneer the use of public agency data for such an analysis. Hopefully, something will develop!)

Steve Aos, of the Washington State Institute for Public Policy, described the system used in Washington State to assist state legislators to make the most informed decisions possible, based on current understanding of the effectiveness of specific programs and policies. At the request of legislators, his organization creates “apples to apples” comparisons of different programs and policies, so that legislators can apply their intuition and their perspectives to meaningful information when casting their votes. To carry out their work on a specific program, the WSIPP identifies all studies of that program; they determine whether these studies meet standards of quality; if they do, they combine the results of these studies (using “meta-analytic” techniques) to provide a picture of what the research says.

He took the example of the decision whether to spend more money to decrease class sizes in elementary, junior high, and high schools, as a means to improve academic performance. As you will see from Steve Aos’ slides on our website, the results disclose a very interesting phenomenon: Reduction of class size seems to have positive effects in grades kindergarten through 2; it has positive, though less strong, effects in grades 3 through 6. For junior high (7-8) and senior high (9-12), class size reduction seems to produce no effect. In fact, a glance at one of his charts might prompt you to ask whether additional research might reveal that reducing class sizes in junior high actually has a negative effect on student achievement! (Note: I am not saying it has that negative effect. I’m saying you might wonder whether it does when you see the chart.)

John Roman, of The Urban Institute, offered an overview of ROI analysis of crime control programs and policies. If you have any interest in this topic, you will find his presentation very valuable. He pointed out that DNA analysis may revolutionize policing, because of its substantial return on investment. In addition, he alerted conference participants to the practical dilemmas which we can face in attempting to make policy decisions which may have long term positive consequences, but at a short term increase in costs. It’s often the case with crime control programs, for example, that they produce an immediate short term social return, but they increase costs. Cost reduction may not occur for a number of years. In the meanwhile, somebody (typically taxpayers) must pay those costs.

Christopher King, of the Ray Marshall Center of the LBJ School of Public Affairs at the University of Texas at Austin , described ROI analyses of workforce programs in Texas. He demonstrated the likely long-term benefit of workforce services and showed how we can test to see whether certain types of programs are more likely to produce those long-term benefits.

Susan Urahn, of the Pew Center for the States, alerted us to the practical steps we need to take to introduce return-on-investment studies (or more broadly, any sort of good research) into policy-making. It is clearly a process that requires engagement of policy makers and representatives of constituent groups over the long term; it’s not simply a matter of doing research and issuing a report.

Susan Urahn surprised many members of the audience with a graph which showed that, by the year 2024, total mandatory spending will exhaust all Federal revenue sources. That is, spending on items such as Social Security, Medicare/Medicaid, other mandatory expenditures, and national debt payments will completely consume the money that the government takes in. Nothing will remain for anything else. The complete solution to that problem, assuming her numbers are correct, will require more than wise spending, but the problem creates all the more need for good return-on-investment studies.

Today’s taxpayers, policymakers, philanthropists, and all of us who care about the future of our communities need return-on-investment information. Limited resources compel us to make wise decisions with the highest likelihood of impact. Not every expenditure can be rated for its return on investment, but many can. We need to do so, and we look forward to working with others to accomplish it!

I encourage you to look at the videos and slide presentations from our conference on our website: www.wilderresearch.org