When engineers at Samsung and Apple design a new smartphone, they make use of the latest scientific evidence, the accumulated product of years of careful research. When our public officials design complex, new social policies, however, they rarely have more to go on than ideology, intuition and common sense.

As every grade-school student discovers, the natural sciences teach us that the physical world does not work the way common sense suggests it should.

Common sense tells us that the earth does not move, and that the sun and stars revolve around it. Physics, however, teaches us that common sense has it backwards: the earth revolves on its axis and orbits the sun, and that’s only a small part of the full story.

What is true in the everyday world is even truer with respect to the science that goes into the design of the latest technological gadgets. Just about everything in quantum mechanics and solid state physics offends common sense. Those theories work, however, and we have the evidence to prove it.

Good experiments are hard to conduct in the social sciences, which is one reason our policymaking remains so unscientific.

Natural scientists study things. Things don’t make choices for themselves, and we can smash things up, melt them, zap them with X-rays and pretty much torture them in whatever way we please to make them yield their secrets.

When we study public policies, we are trying to understand the decisions people make, and people are a good deal more complicated than things. People do make choices for themselves. Though every thing of a certain kind is similar to every other thing of the same type, people are all different. There are also limits to the kinds of experiments we can conduct.

There aren’t two identical United States of America, so we can’t directly compare this administration’s economic policy to some alternative, which makes big problems hard to study directly. Moreover, ethical considerations limit the kinds of experiments we can undertake: If we have good reason to think that something is beneficial, it is hard to justify choosing to benefit some but not others.

Good experiments, however, sometimes are possible in the study of public policy, and, as in the natural sciences, such experiments often teach us that common sense is a poor guide to reality.

The recent Oregon study about the effects of expanded Medicaid coverage is a case in point. When the state could afford to extend new Medicaid benefits to only a fraction of those newly eligible, it extended the benefits to only as many as it could afford, chosen by lottery. (This was ethically defensible, because the alternative would have been to add no one to the program).

We had a random sample of people who joined in the program, and a control group of people similar to those who got new benefits.

The two-year study compared the two groups’ usage of medical services, their health outcomes and their financial condition.

The results were startlingly counter-intuitive. On three measures where one would have expected to see clear health benefits, the study found only tiny indications of improvement, and none of these was statistically significant, which means that the researchers cannot confidently say that the very small results were not just random noise.

On another measure, the study actually found that the Medicaid beneficiaries’ cardiovascular health worsened — but this result was not statistically significant, either.

Common sense may say that adding people to Medicaid is bound to produce a healthier population, but the Oregon study suggests a more complex reality.

This was only one study, and it hardly constitutes proof that expanding Medicaid is either pointless or actually harmful. It absolutely does, however, confirm that we know far too little about the effectiveness of the enormously complex and expensive programs through which the government tries to make our lives better.

We need better and larger studies to discover, based on evidence and not anecdote, which policies actually accomplish their goals and which do not. That is a task for those of us who work and teach in the social sciences — to be more creative and ambitious in our studies.

Public officials, for their part, must be willing to incorporate rigorous, scientific testing into the construction of any new program, and, the hardest but most important thing of all, they must be willing to terminate programs that cannot, scientifically, be shown to work.

Joseph R. Reisert is associate professor of American constitutional law and chairman of the department of government at Colby College in Waterville.

Copy the Story Link