Visioning and Beyond: Session Summary

Posted on: May 30th, 2012 by Kate Humphrey

In a session titled “Visioning and Beyond: Developing Plans and Measuring Implementation” at the 2012 National Rural Transportation Conference, speakers shared their experiences in performance-based planning. Melissa Levy, of Yellow Wood Associates in Vermont, presented a strategic planning process based on systems thinking and measurement. Peggy Reichert of the Minnesota Department of Transportation explained some of the implications of performance monitoring requirements in proposed federal legislation and how her agency has been using and refining this approach to transportation planning. Read a detailed summary of the session content below, or watch the video and view the slides here.

The “You Get What You Measure” process used by Yellow Wood Associates uses stakeholder collaboration to overcome common challenges to measuring values and planning outcomes. Although this specific process has not been applied to transportation issues, all levels of government as well as foundations and nonprofit organizations have successfully used this process to evaluate progress towards goals. In many cases, it can be difficult to identify meaningful measures that reflect the desired characteristic or are in the correct context. Within this framework, measurement can become a “powerful tool for formative evaluation.”

Systems thinking, which emphasizes interrelationships and patterns, is a central element in this measurement process.  Measurement is part of a feedback loop that allows planners to learn through reflection, seeing the results of actions and making adjustments as necessary. It provides new information about experimentation and can open new dialogues. As part of strategic planning, systems thinking works as an interactive process for multiple stakeholders, giving everyone a voice for articulating goals.

The measurement process has nine steps, which revolve around answering the question “what do we really want?” and taking action to achieve it.

1)      Identifying Goals: A goal is a condition you wish to achieve rather than an action. For example, being a community where everyone has a job would be a goal (instead of “creating jobs”).

2)      Discovering Indicators: An indicator is “something that must be changed, or a condition that must be achieved, in order to claim that progress is being made toward a goal.” There are many possible indicators that could be tied to a given goal, but the indicator must change (e.g. more or less of something, or the presence or absence of something) to show progress.

3)      Identifying Key Leverage Indicators: This step investigates the relationship between various indicators and determines which indictor(s) has the greatest impact on the others and “moves the system” as the key leverage indictor. The key results indicator is “downstream,” closer to the goal, and will show if there is an impact or change resulting from the intervention.

4)      Revealing Assumptions: An assumption is “a hypothesis about some aspect of the way the world works that we believe to be true.” It is important to explicitly state assumptions so that they can be tested throughout the measurement process.

5)      Creating Measures: A measure is a way to actually count or value indicators. Stakeholders must understand the terms in the indicators to develop a way to measure them. Information collected must be useful, relevant, and related to the goal.

6)      Identifying Baselines and Framing Measures: The baseline is the first time a measurement is taken and serves as a reference point.

7)      Creating Targets: This step determines what kind of change in indicators counts as progress towards the goal. It may be helpful to refer to similar communities that could serve as a model or resource in setting targets.

8)      Designing Interventions: An intervention is an action towards achieving your goal. Ideally, different kinds of people will have the opportunity to be involved in a variety of tasks. The action plan should include letting others know of the situation and direct action to fix the problem.

9)      Using Results of Measurement and Reflecting: Throughout the measurement process, it is important to frame measures and consider how the impacts fit into a larger context.  The act of measuring is itself a learning process. Make a plan for ongoing measurement that will create forward motion and sustain the process.

Measurement is a powerful tool that engages multiple stakeholders to establish community goals and then make an action plan to achieve them. Furthermore, considering the tradeoffs between multiple goals helps to prioritize limited resources. When done annually, measurement creates the opportunity to reflect on progress and inform what happens next.

Next, Peggy Reichert from the Minnesota DOT presented “Performance Based Planning & Investment Management.” Congress has recently been encouraging the adoption of critical area goals and measures of key outcomes as part of annual performance reporting, potentially moving towards national standards. The cycle of planning and performance management will allow state DOTs to adjust strategies and avoid punitive measures if they fall short of targets.

The Minnesota DOT has been using performance management since the 1990s for its family of plans and programs, including the policy plan and 20-year highway investment plan. Performance in each policy area is monitored and reviewed annually with respect to ten goals across the transportation system. A five-step process helps to translate goals and measures into a plan to guide decisions and track progress:

1)      Estimate Needs: How much will it cost to meet performance targets?

2)      Project Revenue: What funds will be available to meet these needs?

3)      Set Priorities for Available Funding: MnDOT uses a decentralized allocation scheme that gives investment direction to districts.

4)      Develop Investment Plan: Allocate projected revenue across categories and predict performance outcome of investments.

5)      Indentify High Priority Investment Options for Potential Additional Funding.

The annual performance report contains a scorecard for key areas that is used to update the investment plan. It helps identify trends and project future performance relative to measures and targets. This feedback on the system’s performance allows MnDOT to adjust the process as needed to reflect changing resources and goals, such as increasing funding impact of congestion management programs and tweaking measures to reflect new goals and strategies. Adjusting the planning approach is a way to integrate risk assessment by evaluating a series of different targets based on risks. MnDOT can then present alternative scenarios to the public to show a given investment’s impact on risk mitigation, system and performance outcomes, and strategies. Preparing alternative targets and the costs it would take to reach them allows the agency to determine which outcomes are acceptable for a certain level of investment, relative to the ultimate goal.

Other agencies can learn from MnDOT’s experiences with performance-based planning; it is an important case study because it is likely to become more important on the federal level. The process increases accountability and transparency both externally and internally and provides justification for investments. It links planning to programming and offers a framework for analyzing trade-offs and funding priorities. However, it is important for established goals to drive the measures and for targets to be attainable, which requires a better understanding of cause and effect relationships. The process requires a larger commitment of resources and cooperation among different agencies to agree on the same goals.

This entry was posted in Past Conferences, Presentations, Training, Transportation and tagged . Bookmark the permalink.