All of these require us to rank and select among alternatives. We use decisions to make choices, set priorities, or allocate resources. At a strategic level, setting priorities is one of our most important challenges in public health. Most commonly we use informal methods to make decisions and set priorities:

• Organizational traditions
• Leadership preferences
• Politics and advocacy
• Categorical funding
• Personal interests

From the selection of formal methods, we are more likely to set priorities using a needs assessments or using a list of core (or essential) services. Here are the formal methods:

• Conduct a needs assessment
• Define core (or essential) services
• Conduct economic evaluations (cost-effectiveness analysis, cost-benefit analysis)
• Conduct multi-criteria decision-making (MCDM) process

As a medical epidemiologist, I found comfort in conducting needs assessments. I believed that well-designed, population-based studies would provide the evidence needed to set objective priorities based on objective evidence. In the 1990s we embraced the WHO Burden of Disease approach because it used disability-adjust life years (DALYs) to rank the leading causes of premature death and disability in a region [cite]. WHO’s rationale was that priority-setting requires ranking, and that ranking requires objective measurements. Furthermore, they argued that it was necessary  ”to decouple epidemiologic assessment from advocacy so that estimates of the mortality or disability from a condition are developed as objectively as possible.” We adopted this approach in San Francisco, producing several publications to influence local public health priorities [cite]. While we had limited success, it did not significantly improve how we set public health priorities, including how to rigorously incorporate the “intangibles” — the very influences we were trying to avoid, but are important nonetheless. What are the challenges to setting health priorities?

• Our understanding of complex systems may be very limited;
• Qualitative attributes (values, mission, strategies, “intangibles”) are difficult to combine with quantitative attributes (e.g., burden of disease);
• The most knowledgeable stakeholders may not be involved in the decision-making;
• Goals, criteria, or alternatives to be ranked are not always well specified;
• Many decisions must be made rapidly, but rigorous, timely analysis is not feasible;
• Synthesizing the simultaneous influence and impact of multiple criteria is difficult; and
• Decision-making process may not be transparent or reviewable.

A Solution

While there is no foolproof approach — many of us realize that we are not proficient at employing rigorous, evidence-based methods for decision-making, priority-setting, and resource allocation. While we may be perceived to make “good decisions” some or most of the time, we have no clear way of demonstrating that our team made the “best decision” every time — even when it matters.

Most of our training is in analysis — breaking down systems into components and to study relationships. In contrast, in decision making, we need synthesis of system components and stakeholder values, using evidence or assumptions on how components are related. In general, we should be employing multiple-criteria decision methods (MCDMs). This is also called multi-criteria decision-making (or decision models or decision analysis). Using this page, we will explore different MCDMs for public health problems or opportunities. Currently, we are experimenting with a MCDM method called the analytic hierarchy process (AHP), or its generalization called the analytic network process (ANP). For now, we will focus on AHP because it is intuitive and simple to implement. AHP involves the following steps:

1. Understand the problem or opportunity
2. Define the goal of AHP (e.g., prioritize project proposals)
3. Select criteria (What is important and why?)
4. Organize criteria (How are they related? —hierarchy)
5. Compare criteria to derive weights
6. Compare alternatives against criteria to derive ranking
7. Conduct sensitivity analysis

The guts of AHP is the selection and weighting of criteria, and the application of these weighted criteria to the alternatives we are considering. The best decisions are group decisions using the most knowledgeable and impacted stakeholders to develop the criteria and score the alternatives. The criteria can be based on quantitative (e.g., rate ratios) or qualitative (alignment to organizational strategy) data. The ability to measure and incorporate qualitative attributes (“intangibles”) is very powerful! Key stakeholders’ strong preferences can be incorporated explicitly. Final decisions can be explained, rationalized, and reviewed to assess which factors had the biggest influence on the final decision or ranking (also called sensitivity analysis). AHP does requires matrix algebra, but that is easily handled by a computer.

AHP is used worldwide by many business industries. Here is an informative overview by an AHP software leader (this is not a software endorsement):

Okay, and here is an introduction to AHP with some math explained:

Why are MCDMs not employed?

• Lack of familiarity with decision sciences;
• Lack of familiarity with mathematics or decision software solutions; and
• Concerns about losing control over decision-making;

I would argue that MCDMs like AHP are just “systematic common sense” applied to important decisions, and that sense of control and confidence will increase — not decrease. I am convinced that applying MCDMs will transform any organization, even if applied to simple (but important) decisions. It will create a systematic approach to decision making, priority setting, and resource allocation. At worse, the development of clear goals, selection criteria, and alternatives will improve the decision making process.

This web page will document my journey into applying MCDMs to public health challenges or opportunities. Stay tuned and your feedback and suggestions are welcome.

Thanks!

Tomás Aragón

Bibliography

1. Forman EH, Gass SI. The Analytic Hierarchy Process — An Exposition. Operations Research 2001;49(4):469 (Longer, but free version available here. [PDF])