Component 05
Monitoring and Adjustment
This chapter provides assistance to transportation agencies with the Monitoring and Adjustment component of Transportation Performance Management (TPM). It discusses where monitoring and adjustment occurs within the TPM Framework, describes how it interrelates with the other nine components, presents definitions for associated terminology, provides links to regulatory resources, and includes an action plan exercise. Key implementation steps are the focus of the chapter. Guidebook users should take the TPM Capability Maturity Self-Assessment as a starting point for enhancing TPM activities. It is important to note that federal regulations for monitoring and adjustment may differ from what is included in this chapter.
Monitoring and Adjustment is a set of processes used to track and evaluate actions taken and outcomes achieved, thereby establishing a feedback loop to refine planning, programming, and target setting decisions. It involves using performance data to obtain key insights into the effectiveness of decisions and identifying where adjustments need to be made in order to improve performance.
Introduction to Monitoring and Adjustment
Transportation agencies have been monitoring performance results for some time. However, it is what agencies do with the monitoring information that ends up distinguishing transportation performance management from performance measurement. Under a simple performance measurement framework, an agency sets a strategic direction, defines measures and tracks results. There are many benefits associated with these three elements including the establishment of the agency’s purpose, improved communication of performance trends, and enhanced accountability. To move into the realm of transportation performance management, agencies must actively use information gained from monitoring performance data to obtain key insights into the effectiveness of decisions and identifying where adjustments need to be made. The combination of monitoring and adjustment processes is the “bread and butter” of TPM, establishing a critical feedback loop between performance results and future planning, programming and target setting decisions.
Establishment of robust monitoring and adjustment practices benefits an agency by:
- Providing early warning of emerging project delivery and system performance issues;
- Discovering new insights into causal factors contributing to performance outcomes;
- Highlighting needed adjustments to project and programs based on actual results;
- Identifying data gaps that need to be closed;
- Providing a reality check on performance targets; and
- Enhancing the understanding of which strategies are effective and why.
Through monitoring and adjustment practices, an agency can answer, “Are we getting the results we anticipated”? as well as “If not, why not”? The ongoing review of observed results helps agencies identify, diagnose, and act upon program delivery issues. This process also identifies where data gaps exist and highlights where additional information would be beneficial. As an agency’s understanding of the relationship between actions taken and performance results improves, so will an agency’s ability to make necessary mid-stream adjustments, select future projects and programs to achieve desired outcomes, and explain performance results to stakeholders. Given that TPM practices evolve over time, monitoring and adjustment processes provide valuable material upon which future iterations can build.
“Embrace the power of “why”—Focusing on the why clearly communicates that performance management intends to understand the results and identify improvements, not to punish.”
Source: “Moving from Reactive to Strategic Decisions Making.” TR News 293 July-August 2014
The processes implemented under the monitoring and adjustment component focus on the outputs and outcomes of specific transportation projects and programs as well as the performance of the overall transportation system. Outputs refer to the quantity of activity delivered through a project or program: the miles of pavement repaved, the number of bridges rehabilitated, the number of new buses purchased, etc. Outputs are important to track in order to evaluate whether the project or program is on scope, on time and on budget. Did the agency deliver the level of activity that was promised? Outcomes refer to the results of interest to users of the transportation system: travel time reliability, fatality rates, etc. An effective monitoring and adjustment process must look at both output and outcomes to create a strong connection between investment decisions and results.
Through this TPM component, an agency examines what actions are accomplishing the desired impact(s) on performance results (Figure 5 1) and considers why they have been effective or not. Progress toward targets is gauged as well as whether those targets are reasonable. Linking decisions to results reveals potential adjustments needed to deliver the projects, as well as further refinement of the selection of strategies. As a result, the Monitoring and Adjustment component has a clear, direct linkage to Target Setting (Component 02), Performance-Based Planning (Component 03) and Performance-Based Programming (Component 04). By closely analyzing the relationship between actions and results, this component strengthens the connection between what agency staff does on a daily basis and the ultimate strategic goals an agency is trying to achieve (Strategic Direction, Component 01). The information gathered through monitoring and adjustment processes creates a foundation for the external and internal products developed under Reporting and Communication (Component 06), agency management functions (Organization and Culture, Component A), and may assist in the fulfillment of local, state, and Federal regulatory requirements.
Activity refers to an action taken to implement a strategy (e.g., purchase additional maintenance vehicles).
Output refers to “level of activity” (e.g., number of miles repaved).
Outcomes demonstrate the “effectiveness” of a particular activity (e.g., travel time reliability).
Sources: NCHRP Report 446, Guidebook for Performance-Based Planning; FHWA, Performance Based Planning and Programming Guidebook
Since the monitoring and adjustment component helps agencies understand and react to the pursuit of established targets and strategic goals, the more established an agency’s strategic direction and target processes are, the easier it will be to implement monitoring and adjustment processes. In some agencies, monitoring and adjustment may take place naturally, as part of an established transportation performance management process, whereas in others, the steps must be put in place purposefully, in order to emphasize the importance of the relationship between decisions and results. This relationship is cemented through well-defined monitoring and adjustment processes.
A well-crafted monitoring framework allows an agency to determine whether progress is taking place in advance of deadlines for required reporting and, if necessary, enable adjustments to programming so that significant progress is more likely to be attained. An agency may also need to use monitoring information to justify the setting of a new target. Ongoing monitoring and adjustment enables an agency to track the activities it is undertaking and the outputs produced (direct results of an activity, such as miles of pavement resurfaced), and the impact on outcomes (broader effects such as improved mobility or access to activity centers). Reporting and Communication (Component 06) describes steps an agency can take to effectively communicate this documentation to internal and external audiences.
Figure 5-1: Relationship Between Inputs and Outcomes
Source: Federal Highway Administration
Subcomponents and Implementation Steps
Figure 5-2: Subcomponents for Monitoring and Adjustment
Source: Federal Highway Administration
The definition for Monitoring and Adjustment is: a set of processes used to track and evaluate actions taken and outcomes achieved, thereby establishing a feedback loop to refine planning, programming, and target setting decisions. It involves using performance data to obtain key insights into the effectiveness of decisions and identifying where adjustments need to be made in order to improve performance. The component is comprised of two subcomponents, with the Program/Project Level Monitoring and Adjustment subcomponent nested within the System Level Monitoring and Adjustment subcomponent:
- System Level Monitoring Adjustment: Establishment of a well-defined performance-monitoring process to understand past and current performance. The analysis of performance results leads to an improved understanding of causal factors and increases an agency’s ability to act on new insights. This enhanced understanding of why performance results occurred feeds future planning and programming decisions. Within this system outcome viewpoint, Program/Project Level Monitoring and Adjustment clarifies the contribution of specific programs and projects on achieving goals, objectives and targets.
- Program/Project Level Monitoring and Adjustment: Establishment of a process for tracking program and project outputs, and their effects on performance outcomes. This process provides early warning of potential inability to achieve performance targets. Insights are used to make project or program “mid-stream” adjustments and guide future programming decisions. This subcomponent provides a before/after project-level view and is nested within the System Level Monitoring and Adjustment subcomponent.
System Level Monitoring and Adjustment
System level monitoring and adjustment focuses on the linkage between resource allocation decisions and the achievement of strategic goals and objectives. A well-defined monitoring process helps agencies diagnose information on factors that affect outcomes such as available funding and external economic, environmental and social trends. Refining agency monitoring processes, the collection of additional data, and improved analysis capabilities provides new insights into causal factors contributing to performance. A key characteristic of this subcomponent is the application of performance monitoring information to identify where adjustments need to be made. These insights in turn can be used in future planning and programming decisions. System level monitoring typically has a wider scope and a long-range time horizon. An understanding of the relationship between actions and results can take years to assess—as is the case within the safety performance area.
“The purpose of PBPP is to ensure that results of previous investments and policies inform future decision-making so that transportation agencies can better understand approaches that work best given constraints and conditions.”
Source: FHWA, Performance Based Planning and Programming Guidebook
Program/Project Level Monitoring and Adjustment
The program/project level monitoring and adjustment subcomponent assesses specific programs and projects. This includes summary statistics such as dollars expended or outputs delivered. In addition, analyses are conducted to gain a better understanding of the effectiveness of a project or program on desired performance targets. The program/project level monitoring process typically has a narrow focus (e.g., speed improvements resulting from a traffic flow improvement project) and has a shorter timeframe than system monitoring and adjustment efforts. A well-defined program monitoring process gives an agency a better understanding of risk factors that could impact its ability to deliver the program and improves early warning of emerging issues. In addition, before/after studies give agencies new insights into causal factors that may be strong drivers of performance outcomes. With this additional diagnostic information, agencies are able to make project or program adjustments “mid-stream” to address delivery issues, improve the effectiveness of projects and better guide future decisions. In short, program/project level monitoring gives agencies the information necessary to understand, diagnose and act upon delivery issues. Over time, the regular process of monitoring the effect of implemented programs and projects will guide future planning, programming, and target setting decisions.
Outline of Implementation Steps
The importance of linking actions and results is the reason that monitoring and adjustment takes the form of a distinct component within the TPM framework. Although the “monitoring” lens through which the programs and project or system performance varies by scope and time horizon, these processes help agencies understand what progress is being made toward established targets and strategic goals. Together, program/project level and system level monitoring and adjustments establish a critical feedback loop between performance results and future planning, programming, and target setting decisions (see Figure 5-3). However, it is the active use of monitoring information to identify and implement adjustments that makes this component a cornerstone of TPM.
Figure 5-3: TPM Components Flowchart
Source: Federal highway Administration
Both Monitoring and Adjustment subcomponents are intended to provide actionable information to an agency, with one nested within, and informing, the other. The steps necessary to implement program/project level and system level monitoring and adjustment processes are in Table 5-1. How these steps are applied within the two subcomponents is further explored in this chapter.
Source: Federal Highway Administration
| System Level and Program/Project Level |
|---|
| 1. Determine monitoring framework |
| 2. Regularly assess monitoring results |
| 3. Use monitoring information to make adjustments |
| 4. Establish an ongoing feedback loop to targets, measures, goals, and future planning and programming decisions |
| 5. Document the process |
Clarifying Terminology
Table 5-2 presents the definitions for the monitoring and adjustment terms used in this Guidebook. A full list of common TPM terminology and definitions is included in Appendix C: Glossary.
Source: Federal Highway Administration
| Common Terms | Definition | Example |
|---|---|---|
| Activity | Refers to actions taken by transportation agencies, such as projects, related to strategy implementation. | Paving key locations, adding new guardrail, rehabilitating a bridge, purchasing new buses. |
| Adjustment | The alteration of programming, planning, targets, measures, and goals resulting from analysis of information collected. | The restriping of a construction project to address an observed increase in traffic incidents. |
| Goal | A broad statement of a desired end conditions or outcome; a unique piece of the agency’s vision. | A safe transportation system. |
| Monitoring | The identification and diagnosis of performance systems and programs. | Freeway and Arterial System of Transportation (FAST), a real-time traffic condition dashboard that enables detailed analysis on request. |
| Objective | A specific, measurable statement that supports achievement of a goal. | Reduce the number of motor vehicle fatalities. |
| Outcome | Results or impacts of a particular activity that are of most interest to system users. Focus of subcomponent 5.1 System Level Monitoring and Adjustment. | Transit travel time reliability, fatality rate, percent of assets within useful life. |
| Output | Quantity of activity delivered through a project or program. Focus of subcomponent 5.2 Program/Project Level Monitoring and Adjustment. | Miles of pavement repaved, miles of new guardrail put into place, the number of bridges rehabilitated, the number of new buses purchased. |
| Performance Measure | Performances measures are based on a metric that is used to track progress toward goals, objectives, and achievement of established targets. They should be manageable, sustainable, and based on collaboration with partners. Measures provide an effective basis for evaluating strategies for performance improvement. | Transit passenger trips per revenue hour. |
| Performance Period | An established timeframe for monitoring results and collecting data and information for performance reporting. | A calendar year. |
| Reporting | Summary documentation of performance trends for either internal or external audiences. | WSDOT Gray Notebook. |
| Sub-Measure | A detailed quantifiable indicator uncovered during monitoring that provides additional insights into internal and external processes. | Preventive maintenance compliance—a driver of overall asset performance. |
| Transportation Performance Management | A strategic approach that uses system information to make investment and policy decisions to achieve performance goals. | Determining what results are to be pursued and using information from past performance levels and forecasted conditions to guide investments. |
Relationship to TPM Components
The ten TPM components are interconnected and often interdependent. However, the monitoring and adjustment component is particularly notable given that it serves as the critical feedback loop within the TPM Framework. As the means to answering the questions, “Are we getting the results we anticipated”? as well as “If not, why not”? this component helps agencies determine progress toward performance targets (Component 02) and in turn, strategic goals (Component 01). Through an increased understanding of the effect of specific projects and programs on outcomes, the monitoring and adjustment component uncovers information to be used in future planning (Component 03) and programming (Component 04) decisions. This monitoring and adjustment component helps agency staff link their day-to-day activities to results and ultimately agency goals (Organization and Culture, Component A). The external and internal reporting and communication products (Component 06) are based on the information gathered during monitoring and adjustment. Finally, a cornerstone of all TPM components is quality data. By establishing well-defined monitoring and adjustment processes, the quality of the data agencies use will naturally improve and enable identification of data gaps that need to be addressed.
Linkages between monitoring and adjustment and the other nine TPM components are depicted in Table 5-3
Source: Federal Highway Administration
| Component | Summary Definition | Relationship to Performance-Based Planning |
|---|---|---|
| 01. Strategic Direction | The establishment of an agency’s focus through well-defined goals/objectives and a set of aligned performance measures. | The information uncovered during the monitoring and adjustment phase helps agencies assess progress toward the goals and objectives defined under the strategic direction. |
| 02. Target Setting | The use of baseline data, information on possible strategies, resource constraints, and forecasting tools to collaboratively establish targets. | As agencies better understand the effectiveness of projects and programs through monitoring and adjustment, the feasibility of attaining targets will be clearer, resulting in potential target adjustments. |
| 03. Performance-Based Planning | Use of a strategic direction to drive development and documentation of agency strategies and priorities in the long-range transportation plan and other plans. | Reviewing performance trends through monitoring and adjustment provides key insights into the actual versus predicted effectiveness of alternative strategies (before/after analysis) with respect to agency goals. Monitoring establishes a key feedback loop to future planning decisions, including necessary strategy adjustments and the identification of new strategies. |
| 04. Performance-Based Programming | Allocation of resources to projects to achieve strategic goals, objectives and performance targets. Clear linkages established between investments made and their expected performance outputs and outcomes. | Improved knowledge about influencing factors and the relationship between investments and performance results explored through monitoring processes will improve the assumptions used for future programming decisions. |
| 06. Reporting and Communication | Products, techniques, and processes to communicate performance information to different audiences for maximum impact. | The monitoring and adjustment process provides a foundation for external and internal reporting and communication products regarding performance. This component also provides the explanation for why target and program adjustments are necessary. |
| A. TPM Organization and Culture | Institutionalization of a TPM culture within the organization, as evidenced by leadership support, employee buy-in, and embedded organizational structures and processes that support TPM. | Monitoring and adjustment strengthens the connection between what agency staff do on a daily basis and the ultimate strategic goals and agency is trying to achieve. This component provides a forum for leadership to better understand performance results, provide support, and assign roles and responsibilities as needed. |
| B. External Collaboration and Coordination | Established processes to collaborate and coordinate with agency partners and stakeholders on planning/ visioning, target setting, programming, data sharing, and reporting. | Examining the relationship between programs and performance results will create a pool of data and analysis that can be shared with external partners to clarify and explain adjustments made. |
| C. Data Management | Established processes to ensure data quality and accessibility, and to maximize efficiency of data acquisition and integration for TPM. | Monitoring and adjustment processes are dependent on the availability of timely, accurate and authoritative data. |
| D. Data Usability and Analysis | Existence of useful and valuable data sets and analysis capabilities, provided in usable, convenient forms to support TPM. | Through regular performance monitoring, the quality of the data agencies use will improve, and data gaps that need to be closed will be identified (e.g., “sub-measures” that provide new insights into factors influencing performance results). |
Regulatory Resources
This Guidebook is intended to assist agencies with implementing transportation performance management in a general sense, and not to provide guidance on compliance and fulfillment of Federal regulations. However, it is important to consider legislative requirements and regulations when using the Guidebook. In many cases, use of this Guidebook will bring an agency in alignment with Federal requirements; however, the following sources should be considered the authority on such requirements:
Federal Highway Administration
- Transportation Performance Management: http://www.fhwa.dot.gov/tpm/links_fhwa.cfm
- Fact Sheets on Fixing America’s Surface Transportation (FAST) Act: https://www.fhwa.dot.gov/fastact/factsheets/
- Fact Sheets on Moving Ahead for Progress in the 21st Century (MAP-21): https://www.fhwa.dot.gov/map21/factsheets/
- Resources on MAP-21 Rulemaking: https://www.fhwa.dot.gov/tpm/rule.cfm
Federal Transit Administration
- Fact Sheets on FAST Act: https://www.transit.dot.gov/funding/grants/fta-program-fact-sheets-under-fast-act
- Resources on MAP-21: https://www.transit.dot.gov/regulations-and-guidance/legislation/map-21/map-21-program-fact-sheets