Thursday, May 26, 2022

Guns, Guns, Guns

 

In 2019, after the murder of 23 people at a Walmart in El Paso, Texas, a reporter, Laurie Roberts, wrote: “You know the drill”. First will come thoughts and prayers; then questions and calls for action. Then: 

“As night follows day, the horrified reply will be that it’s simply too soon to politicize this tragedy. And once again we will find that the appropriate time for America to at long last confront these horrifying massacres … is never.”

Notwithstanding any thoughts and prayers which might have occurred during the next three years, nothing substantial happened to prevent another tragic, mass killing. So, about 3 years, and about 500 miles from El Paso, 21 people have so far died as a result of an elementary school shooting in Uvalde, Texas, at the hands of a shooter in possession of two semi-automatic rifles. 

Unfortunately, mass shootings (defined as 4 or more victims killed or injured) occur almost daily in the United States. Mass murders (defined as 4 or more people murdered) occur, on average, every few weeks. This year, since the 1st of January, the United States has had 213 mass shootings and 10 mass murders. The Gun Violence Archive updates these statistics every day.

Discussion of putting an end to this sad carnage often devolves into fruitless debates over the impact of mental illness versus the availability of guns. Do these incidents stem from mental illness or gun availability? Perhaps a reasonable question, but very easy to answer: both mental illness and gun availability have significance, probably along with other factors. 

Certainly, anyone who murders defenseless, innocent children suffers from severe mental illness, acute or chronic. Moreover, killers such as the El Paso shooter (2019) and the Buffalo shooter (2022), have taken others’ lives without remorse or shame. That seems psychopathic.

Also certainly, a truism exists: If someone does not have a gun, they cannot shoot anyone. If the Dayton (2019) shooter did not have a .223-caliber high-capacity rifle with 100-round drum magazines – that enabled him to fire 41 shots in less than 30 seconds – he could not have shot anyone. If the Las Vegas (2017) shooter did not have access to a military-style weapon, along with a bump stock to effectively turn his weapon into a machine gun, he could not have shot and killed so many people.

For prevention of future gun-violence tragedies, evidence-based insights can assist us. The Violence Project, for example, has examined mass shootings since the 1960s. They have documented how shooters have typically experienced a noticeable crisis prior to their shooting. Many experienced feelings of suicidality before or during their attacks on others. The majority of shooters experienced early childhood trauma and exposure to violence. Shooters found validation, inspiration, and sometimes radicalization in the actions of other mass killers. And shooters had the means to carry out their plans; they could access the guns necessary for killing a lot of people in a short period of time.

If we commit to collaborative, data-informed approaches, we can use available research information to develop strategies to prevent future mass shootings. The Dean of Harvard’s School of Public Health suggests that we can take a public health approach – that “reframes the issue as a preventable disease that can be cured with the help of community members.” Similar to stopping any disease, stopping gun violence requires interrupting transmission, containing the risk, and changing community norms.

After a mass murder in 1996, the United Kingdom took a radical approach, banning access to some types of guns and strictly limiting access to other types. Since then, only one mass murder has occurred.

So far, no evidence has emerged that more guns will reduce school shootings and other mass murders. In the recent Uvalde situation, the killer warded off two, armed police officers who arrived before he barricaded himself in the school building. In the Parkland (2018) school shooting, where 17 people died, the school security officer onsite retreated outside to a place of safety, rather than pursue the killer. In the Buffalo mass shooting two weeks ago, an armed security guard could not stop the shooter, and died in the attempt to do so.

Nothing suggests that Texas Governor Abbott’s 2015 tweeted exhortation – to make Texas the number one state in gun purchases – has any likelihood of reducing the number of innocent children killed by mass shooters. In fact, based on the evidence on how killers obtain their guns, his advice will probably increase the number of children murdered.

For those of us working in the nonprofit sector and in community-based organizations, these violent events compel us to address our missions all the more strongly. Few of us can step in and prevent a specific violent act, a specific act of hate, or any other bad thing that an individual decides to inflict on others. Nonetheless, we can address root causes and underlying conditions which precipitate and/or enable such events.

Success will require that many of us work in concert, and that we take action, with thoughts and prayers only serving as the first step.

 

Tuesday, April 12, 2022

The Census: An Eye on the Past

 On April 1st of this year (no fooling), the National Archives released the original census records from 1950. Interesting stuff if you seek to obtain information for historical, genealogical or other purposes.

With a name and the county / city of residence, you can search for a specific person. The more unique the name, the easier the search, especially in larger cities. You can also look for specific addresses.

I searched for information about parents and grandparents. The results helped me piece together some ancestral facts that I knew in part, but did not fully understand.

Do you want to know who lived in your house (if it existed) in 1950, or even earlier? I learned that the seven occupants in 1950 of my current house included three generations. The “head of household” reported the occupation of university professor. So, I explored the professor’s background and discovered that he had written several important books about local history.

I looked as well at other residents on my city block and nearby blocks. As I read names, ages, occupations of these people, it set me wondering about their stories – how they built their lives, how they arrived where they found themselves in 1950. What shaped their experiences, and how did their lives in these places shape our present experiences? I have looked at earlier census records as well. Comparing records over time provides details of changes at a microscopic level in a neighborhood. It tells us something about the individuals who lived in the houses, and also something about social processes and structures, as they changed or as they did not change, over the decades.

Understanding the past supports our efforts to move into the future. If you take a look at census records, who can predict what fascinating little historical tidbit you might discover – which will change your understanding of the world? 

Give it a try!

https://1950census.archives.gov/


Thursday, January 27, 2022

Program evaluation: Gathering good data to increase positive impacts

 

 

A community group wants to improve its food distribution program and seeks to understand how best to increase the accessibility of food that has high nutritional value and meets varied cultural standards of the community’s residents.

 

A multi-service, nonprofit agency provides assistance to people across the life span. It asks: What can we do to increase the numbers of people who find our services helpful? Why do older people seem to use us more than younger people?

 

A museum seeks to enrich the lives of children in a neighborhood whose residents do not know about the museum’s services.

 

A foundation has a mission to provide funding for housing and wants to know how to best direct the funding toward the types of housing that fit the needs and preferences of people in the area the foundation serves.

 All of these organizations want to learn whether they accomplish their intended purpose. In the simplest form, they want to know: “If we do A, does it lead to B? 

Program evaluation is a systematic process for an organization to obtain information on its activities, its impacts, and the effectiveness of its work, so that it can improve its activities and describe its accomplishments.

Gathering solid evidence about effectiveness

A useful program evaluation typically builds upon four types of information:

  • Client/participant characteristics - e.g., demographics
  • Service data - type and amount of services, activities, treatments, etc.
  • Documentation of results or outcomes – evidence of changes that occurred or needs met among the people served
  • Perceptions about services – how people feel about their experience with the organization

·       The organization might have such information available, or it may need to develop the means to acquire that information. Ideally, the organization will have, or will create, a process for obtaining information in a complete and accurate form. Having information on clients and the services they receive requires a consistent method of tabulating service use, along with a database that stores that information for retrieval. Understanding outcomes and client satisfaction requires collection of authentic information from and about the people served by a program.

That’s where program evaluation enters the picture. Techniques for gathering information enable programs who want solid evidence about their effectiveness to gather it in reliable ways.

Logic models: Identifying the path to impact

Evaluation can, and should, stretch the limits of our thinking. Many years ago, after a couple of initial, and seemingly productive, meetings with a program that wanted to initiate an evaluation, they canceled an upcoming meeting. They did not return my phone calls (before the days of email!). After about a month, their director called and said, “Paul, we’re sorry we didn’t get back to you. We had an issue with ourselves, not with you. When you asked us to outline how we work and how we produce our impact, those questions raised existential concerns for us. We realized that we really did not have a solid process for working together; we realized that we really didn’t have a system to collaborate as a team to know what to do when in order to produce the best outcomes for our clients.”

Encouraging that organization to identify their program theory of change (e.g., a logic model) led to a pivotal moment – a juncture where they suddenly had new insight and, in a relatively short period of time, developed a new way to work together. Over the years, many organizations have told Wilder Research staff that the most useful part of the evaluation process is the development of a logic model. It clarifies what programs expect to accomplish. It offers a guide that enables staff to better plan and make decisions to improve the accessibility and effectiveness of their services. It assists in communicating about impact with many different kinds of stakeholders.

Showing progress and where we need to do better

Beyond helping us to improve our work, evaluation provides much other value. Staff and volunteers can feel a sense of pride and feel increased motivation by seeing the number of people served by their organization. They can feel empowered by knowing the number of people who benefited as expected from the services they received, and the number who did not. Organizations can share such information with other organizations in their field to compare notes and jointly discuss how they might improve effectiveness. Organizations can share the information with funders, to document what the funders’ resources supported and to point toward areas of additional need.

The single biggest mistake I’ve witnessed regarding evaluation is when people look at evaluation information as if it represents the score at the end of a sporting event – assuming that it shows whether we “won” or “lost”. Viewing evaluation this way makes people fearful, and it suppresses innovation.

One of the most successful fund-raising institutions in the United States – St. Jude Children’s Research Hospital – has often published statistics on its success rates for treatment of different types of cancer. Over several decades, some success rates began extremely low – even close to zero. Did that mean “give up”? Did that mean “hide the facts”? Absolutely not. In fact, the opposite. It served to rally people around a cause, and it offered a baseline against which to judge new approaches. Acknowledging the difficulty of achieving success increased determination to find cures; it strengthened appeals for funding and for other resources to improve the health of children.

So too with other types of issues that nonprofit, community-oriented organizations face. Some social issues might seem intractable. Adapting organizational activities to fit changing needs and new populations might seem bewildering. That does not mean “game over, we lost.” It means, whatever our level of effectiveness – high, medium, or low – let’s build on that to strengthen our work and increase our impact.

Evaluation comprises part of an ongoing cycle of using information to design and deliver services, gathering more information to see how well the services achieve intended outcomes, and then using that new information to make revisions and adjustments for improvement. Evaluation creates a platform for evidence-based decision-making: We can never have total certainty that what we do will be effective, but with good data, we increase the probability that we will create positive impacts.

The development of programs and policies that benefit people involves a constant search. We seriously err if we think we know it all, or if we remain rigid in our thinking and our approaches to helping people, solving problems, and addressing issues. Through program evaluation, we ask questions, seek better paths, and make progress with humility to create a better world.

 

The Manager's Guide to Program Evaluation: 2nd Edition: Planning, Contracting, & Managing for Useful Results is now available. This revised edition is an invaluable resource with brand new real-world examples taken from recent evaluation research projects conducted by Wilder Research staff. Get your copy here from: Turner Publishing.


Saturday, January 01, 2022

Our New Year 2022

 

Today’s New York Times stated: “January 2022 arrives as our ways of keeping time seem broken. Calendar pages turn, yet time feels lost.” Yes. It can feel as if time froze. Recent events can seem distant; events of a year ago can seem like they recently occurred. Pandemic time does not have the feel of “real time”.

Of course, time did not freeze. Throughout 2021, the world continued to need research, and we at Wilder Research persevered, despite many obstacles, to do social research to improve the lives of individuals, families, and communities.

I never anticipated that the last of my 40 years as Executive Director of this 104 year old research institute would include navigating through a pandemic. The plague of Covid-19 forced us to balance work and personal responsibilities in unique ways, cope with sickness of ourselves and/or family members, learn how to accomplish a lot of tasks remotely, participate in meetings knowing that a child or grandchild might pop onto the screen, and overcome myriad other challenges – while nevertheless maintaining high standards of quality for our products and services.

Wilder Research staff acted in heroic ways to move our products and services out the door and into the hands of research users. Our mission maintained our motivation because community needs did not freeze in time; organizations striving to make the world a better place did not halt operations. Community leaders, decision-makers, public officials, philanthropic organizations, news media sought greater insight (which research can provide) regarding social issues, to cope with unprecedented experiences.

Whatever 2022 brings – it will not include yesteryear’s standard patterns of living and working. Amidst the “new abnormal”, we might not settle into any predictable arrangements for our lives, for at least some time. But at Wilder Research, the values we hold – such as dedication to community, faith in our colleagues, commitment to high quality research products and services – those will persist and transcend whatever the future drops upon us.

Happy New Year!


Friday, October 15, 2021

Managing Evaluation – from the Program Manager’s Point of View

 

Much interest exists in evaluating and improving programs that serve our communities. Staff in nonprofit organizations and government agencies of all types want to know which services work and which do not work, so that they can continually increase their effectiveness. Funders want to know how best to invest their resources. 

I’ve written a new book, The Manager’s Guide to Program Evaluation (2nd edition), to help those of you who value the benefits of high quality program evaluation and have responsibility for making sure that evaluation occurs, but need help understanding how best to make it happen. This new edition provides several examples of evaluation research, based on work completed at Wilder Research. Those examples can help readers to better understand how evaluation might occur in their organizations. 

The Manager’s Guide takes a common sense approach to explaining program evaluation. All of us intuitively make decisions, based on the best information that we have, regarding activities in our daily lives. We discover the best way to travel to a destination; we assess products in order to decide which ones to purchase; and so on. Program evaluation formalizes that intuitive process. It offers overall discipline, specific rules, and reliable methods so that we can ensure the validity of whatever information we collect and the soundness of whatever judgements we make. It offers us confidence that we will move forward to increase our impacts. 

The Manager’s Guide walks readers through all the steps in the program evaluation process, providing the information needed for planning, contracting, and managing a helpful evaluation. The book takes the program manager’s point of view. It informs program managers about what they need to know as managers – not about the technical skills they would need as program evaluation specialists. We hope that readers find that slant valuable. 

Ground yourself in the evaluation essentials

One chapter covers the basics – offering definitions and key concepts important for understanding the evaluation process. That chapter describes the essential information that an evaluation must have. It also indicates other information which, though not absolutely necessary, can enable a program manager to answer more questions and get at more nuances in determining the effectiveness of their services. That chapter also describes the importance of having a program theory or logic model. 

Understand your role throughout the evaluation process

Another chapter guides readers step by step through the four phases of an evaluation: design, data collection, analysis, and reporting. For each step, the book indicates what major activities occur and the roles of the program manager and the evaluation researcher in those activities. For example, at one point in the design phase, a program manager will need to take a lead role in determining the major questions for the evaluation to answer. With knowledge of the questions to address, the researcher will take a lead role in identifying the most appropriate methods for obtaining information to respond to those questions. 

The Manager’s Guide also provides answers to practical questions. For example, how do you decide what to do on your own and what to hire an evaluation researcher to do? If hiring a program evaluator, how should someone select that person or organization? Many options exist, including working with a large firm, with an academic research center, with an independent solo practitioner, or with a smaller team or firm (for profit or nonprofit). Each of those options has its advantages and disadvantages, and the book points them out. 

Plan for the necessary resources

How much does an evaluation cost? That question often arises, and the book provides some guidance. However, spoiler alert: Asking, “How much does an evaluation cost?” resembles asking, “How much does a house cost?” The answer obviously is “it depends.” To estimate costs of program evaluation, you will need to consider what questions you must answer and develop an understanding of the methods required to respond to those questions. Knowing those things, you can reasonably predict the staff and consultant hours necessary and the associated costs. 

Make an impact!

Michael Patton has served as an inspiration, a guide, a teacher, a friend, and a mentor to many of us in the evaluation field. In reviewing the book, he offered the comment that “for managers committed to evidence-informed decision making, no better book exists.” I very much appreciate that assessment. In addition, I greatly appreciate the opportunity to have created this book to put tools into the hands of people who seek to make a difference in the world through their work – so that we can act in an equitable fashion to improve the lives of people everywhere, using sound information and making sound judgements for the betterment of humanity. 

The Manager’s Guide to Program Evaluation (2nd edition) becomes available in January 2022. You can pre-order your copy from Turner Publishing.