January 27, 2022

News

News Network

Program Evaluation: Key Terms and Concepts

11 min read
<div>Both the executive branch and congressional committees need evaluative information to help them make decisions about the programs they oversee—information that tells them whether, and why, a program is working well or not. The Government Performance and Results Act of 1993 (GPRA) and GPRA Modernization Act of 2010 (GPRAMA) established a framework for performance management and accountability within the federal government. Building on that foundation, Congress has since passed, among other laws, the Foundations for Evidence-Based Policymaking Act of 2018 (Evidence Act) to strengthen the evidence-building efforts of executive branch agencies. This product updates our previous glossary (GAO-11-646SP) to highlight different types of evaluations for answering questions about program performance, as well as relevant issues to ensure study quality. This glossary can help agency officials better understand fundamental concepts related to evaluation and enhance their evidence-building capacity. For more information, contact Lawrance Evans, Jr. at 202-512-2700 or EvansL@gao.gov.</div>

Both the executive branch and congressional committees need evaluative information to help them make decisions about the programs they oversee—information that tells them whether, and why, a program is working well or not. The Government Performance and Results Act of 1993 (GPRA) and GPRA Modernization Act of 2010 (GPRAMA) established a framework for performance management and accountability within the federal government. Building on that foundation, Congress has since passed, among other laws, the Foundations for Evidence-Based Policymaking Act of 2018 (Evidence Act) to strengthen the evidence-building efforts of executive branch agencies. This product updates our previous glossary (GAO-11-646SP) to highlight different types of evaluations for answering questions about program performance, as well as relevant issues to ensure study quality. This glossary can help agency officials better understand fundamental concepts related to evaluation and enhance their evidence-building capacity.

For more information, contact Lawrance Evans, Jr. at 202-512-2700 or EvansL@gao.gov.

More from:

News Network

  • Justice Department and DHS Publish Notice of Proposed Rulemaking to Make Asylum Process More Efficient and Ensure Fairness
    In Crime News
    In a key step toward implementing the Administration’s blueprint for a fair, orderly, and humane immigration system, the Department of Justice (DOJ) and Department of Homeland Security (DHS) are publishing a notice of proposed rulemaking (NPRM) that would amend current regulations to improve the processing of asylum claims.
    [Read More…]
  • Judges Appreciate Jurors as Their Partners in Justice
    In U.S Courts
    In a new, five-minute installment in the Court Shorts video series, 11 federal judges bring attention to the central role of citizens in maintaining public trust in the justice system.
    [Read More…]
  • Owner of Oil Chem Inc. Sentenced for Clean Water Act Violation
    In Crime News
    The president and owner of Oil Chem Inc. was sentenced today to 12 months in prison for violating the Clean Water Act stemming from illegal discharges of landfill leachate — totaling more than 47 million gallons — into the city of Flint sanitary sewer system over an eight and a half year period.
    [Read More…]
  • Secretary Antony J. Blinken and EU High Representative for Foreign Affairs and Security Policy and Vice President of the European Commission Josep Borrell Before Their Meeting
    In Crime Control and Security News
    Antony J. Blinken, [Read More…]
  • Designation of Six Targets Involved in Iran’s Destabilizing Unmanned Aerial Vehicle Activities
    In Crime Control and Security News
    Antony J. Blinken, [Read More…]
  • Justice Department Establishes Initiative to Strengthen States’ Use of Criminal Justice Data
    In Crime News
    The Department of Justice today launched Justice Counts, an effort to help states make smarter policy and budget decisions using up-to-date, actionable criminal justice data.
    [Read More…]
  • Lead Paint in Housing: HUD Has Not Identified High-Risk Project-Based Rental Assistance Properties
    In U.S GAO News
    During fiscal years 2018 and 2019, the Department of Housing and Urban Development (HUD) obligated about $421 million through two grant programs to state and local governments to help identify and control lead paint hazards in housing for low-income households. HUD also issued guidelines for evaluating and controlling lead paint hazards, generally encouraging abatement (such as replacing building components containing lead) as the preferred long-term solution. HUD has supported research on lead paint hazard control and provided education and outreach to public housing agencies, property owners, and the public through publications and training events. HUD monitors lead paint-related risks in its Project-Based Rental Assistance Program, one of HUD's three largest rental assistance programs, through management reviews and periodic physical inspections, but has not conducted a comprehensive risk assessment to identify properties posing the greatest risk to children under the age of 6. HUD's management reviews include assessing property owners' compliance with lead paint regulations—such as by reviewing lead disclosure forms, records of lead inspections, and plans to address lead paint hazards. Inspectors from HUD's Real Estate Assessment Center also assess the physical condition of properties, including identifying damaged paint that could indicate lead paint risks. According to HUD officials, they have not conducted risk assessments in project-based rental assistance housing because they believe the program has relatively few older and potentially riskier properties. However, GAO's analysis of HUD data found that 21 percent of project-based rental assistance properties have at least one building constructed before 1978 (when lead paint was banned in homes) and house over 138,000 children under the age of 6. If HUD used available program data to inform periodic risk assessments, HUD could identify which of the properties pose the greatest risk of exposure to lead paint hazards for children under the age of 6. Unless HUD develops a strategy for managing the risks associated with lead paint and lead paint hazards in project-based rental assistance housing, it may miss the opportunity to prevent children under the age of 6 from being inadvertently exposed to lead paint in those properties. Project-Based Rental Assistance Properties with at Least One Building Built before 1978 and That House Children under Age 6, as of December 31, 2019 Note: Children under the age of 6 are at the greatest risk of lead exposure because they have frequent hand-to-mouth contact, often crawl on the floor, and ingest nonfood items. Lead paint exposure in children under the age of 6 can cause brain damage, slowed development, and learning and behavioral problems. Exposure to lead paint hazards can cause serious harm to children under 6 years old. HUD is required by law to reduce the risk of lead paint hazards in HUD-assisted rental housing—including project-based rental assistance (subsidies to make privately owned multifamily properties affordable to low-income households). The 2019 Consolidated Appropriations Act Joint Explanatory Statement includes a provision for GAO to review, among other things, HUD's oversight of lead paint and related hazards in affordable rental housing. This report (1) describes how HUD programs and guidance address lead paint hazards in HUD-assisted and other low-income rental housing, and (2) examines HUD's oversight procedures for assessing risk for lead paint hazards in project-based rental assistance housing. GAO reviewed HUD and Environmental Protection Agency (EPA) lead paint regulations and documents on lead programs and methods for addressing lead paint hazards. GAO reviewed HUD oversight policies and procedures and analyzed HUD data on building and tenant age. GAO interviewed staff at HUD, EPA, and organizations that advocate for safe affordable housing. GAO recommends that HUD (1) conduct periodic risk assessments for the Project-Based Rental Assistance Program and (2) develop and implement plans to proactively manage identified lead paint risks. HUD agreed to conduct periodic risk assessments and develop and implement a plan to proactively manage risks. For more information, contact John H. Pendleton at (202) 512-8678 or pendletonj@gao.gov.
    [Read More…]
  • Capitol Attack: Special Event Designations Could Have Been Requested for January 6, 2021, but Not All DHS Guidance is Clear
    In U.S GAO News
    What GAO Found The Department of Homeland Security (DHS) has specific designations available for planned special events that bolster security-planning processes and coordination between federal, state, and local entities. For example, these designations enhance coordination of protective anti-terrorism measures and counterterrorism assets, and restrict access. These designations include the National Special Security Event (NSSE) and the Special Event Assessment Rating (SEAR). These designations were not assigned to the events occurring on January 6, 2021. The events of January 6 included 1) a non-permitted protest at the U.S. Capitol, 2) a scheduled Presidential rally at the Ellipse, and 3) a joint session of Congress to certify the 2020 election results. If requested, the Presidential rally and joint session of Congress could have been considered for a designation as an NSSE or SEAR because, for example, they were large events with Presidential or Vice Presidential attendance. However, according to DHS officials, the non-permitted incident at the U.S. Capitol was not consistent with factors currently used for NSSE and SEAR designations. This non-permitted incident was not designated, even though there were other indications, such as social media posts, that additional security may have been needed at the Capitol Complex on January 6. While DHS has developed factors for designating an event an NSSE, it is not clear whether they are adaptable to the current environment of emerging threats. Being able to be dynamic and responsive to change would enable federal entities to implement better security planning. Further, although Secret Service officials stated that a request from the local government in Washington, D.C. would typically initiate consideration for an NSSE designation, D.C. Homeland Security and Emergency Management Agency officials indicated that they did not think the District Government had the authority to request an NSSE designation for an event on federal property. Updating and communicating its policy for requesting an NSSE designation will help DHS ensure that relevant agencies are aware of, and understand, the process for requesting such event designations and may help to better secure the Capitol Complex and other federal properties in the future.  Why GAO Did This Study The attack on the U.S. Capitol on January 6, 2021 resulted in assaults on approximately 140 police officers, and about $1.5 million in damages, according to information from the Department of Justice and the U.S. Capitol Police. In addition, the events of the day led to at least seven deaths. Questions have been raised about the extent to which necessary steps were taken to adequately secure the Capitol Complex, and share intelligence information. We have a body of work underway that examines the preparation, coordination, and response on January 6, that we will begin issuing over the next several months. GAO was asked to review, among other things, coordination between federal and local entities for security and emergency support for events at the U.S. Capitol and surrounding areas on January 6, 2021. Specifically, this report examines the extent to which federal, state, and local government entities requested a special event designation for the planned events of January 6, 2021 to include: (1) the definition of an NSSE and its designation process; (2) the definition of SEAR and its designation process; (3) the characteristics of past NSSE and SEAR events; (4) the applicability of NSSE and SEAR designations to the events of January 6 and the extent to which they were considered; and, (5) why NSSE and SEAR designations were not considered for the events of January 6. GAO reviewed policies and processes for DHS special event designations, interviewed officials from relevant agencies, and examined DHS data on recent events that received special event designations.
    [Read More…]
  • On the Passing of Former Papua New Guinean Prime Minister Morauta
    In Crime Control and Security News
    Michael R. Pompeo, [Read More…]
  • Dietary Supplement Executive Sentenced in Scheme to Fraudulently Sell Popular Dietary Supplements
    In Crime News
    A federal court in Texas sentenced a former dietary supplement company executive to prison for his role in fraudulently selling popular workout supplements, the Justice Department announced today.
    [Read More…]
  • Refined Coal Production Tax Credit: Coordinated Agency Review Could Help Ensure the Credit Achieves Its Intended Purpose
    In U.S GAO News
    What GAO Found According to Internal Revenue Service (IRS) data, refined coal producers have claimed approximately $8.9 billion in refined coal production tax credits since tax year 2010, when IRS last updated its guidance. The credits these producers claimed increased annually from about $9 million in 2010 to about $1.6 billion in 2019. IRS's 2010 guidance requires taxpayers claiming the credit, which GAO generally refers to as producers, to demonstrate that burning refined coal reduced emissions of nitrogen oxides (NOx) by at least 20 percent and either sulfur dioxide (SO2) or mercury generally by at least 40 percent through field, laboratory, or pilot-scale testing. Producers told GAO they typically used pilot-scale testing at an offsite facility, among other reasons, to control variables that can affect emissions levels, thereby helping them to attribute any observed emissions reductions directly to burning refined coal. However, GAO found that use of pilot-scale testing limited the federal government's understanding of actual emissions reductions. For example, Department of Energy (DOE) officials told GAO it is difficult to conduct pilot-scale tests that accurately reflect emissions at full scale because the effects of variables such as coal properties, treatment processes, and boiler conditions are not well understood, particularly for tests with a pilot-to-field scale that exceeds 1:10. According to representatives of a facility that conducted pilot-scale tests for some producers, the test scales typically ranged from 1:10,000 to greater than 1:100,000. According to officials, IRS, the Department of the Treasury, DOE, and the Environmental Protection Agency (EPA) coordinated to develop the credit's guidance, but they have not coordinated to review the credit's performance. Based on GAO's review of some EPA emissions reduction programs and interviews with representatives from power plants that burned refined coal, GAO determined that the credit potentially overlaps with other federal programs such as EPA programs that regulate mercury emissions. However, IRS, Treasury, EPA, and DOE officials told GAO they had not coordinated to identify or address potential overlap or duplication. Moreover, officials from these agencies told GAO they had not reviewed whether the credit had achieved its intended purpose. Treasury officials told GAO they had not reviewed the credit's performance, in part because the credit period for refined coal production facilities expires on December 31, 2021, and the department does not typically focus its resources on credits that Congress may not extend. GAO's tax expenditure guide states that greater scrutiny of tax expenditures, such as periodic coordinated reviews, could help Congress and federal agencies determine how well specific tax expenditures work to achieve their intended purpose. Similarly, GAO has reported that coordinated reviews of tax credits with related federal spending programs could help policymakers reduce overlap and direct scarce resources to the most effective or least costly methods to deliver federal support. Why GAO Did This Study Coal is a key domestic fuel source and an important contributor to the U.S. economy. In the U.S. power sector, burning coal accounts for most of the sector's emissions of NOX, SO2, and mercury. According to EPA, these pollutants can harm human health by contributing to neurological impairment, serious respiratory illness, and premature death. Since 2004, the federal government has offered tax credits to support the production of refined coal, which could help to reduce these emissions. IRS last updated its guidance for claiming this credit in 2010. GAO was asked to review the federal government's implementation of the credit. This report examines (1) the extent to which producers have claimed the refined coal production tax credit since tax year 2010; (2) what the federal government knows about the extent to which producers have demonstrated the emissions reductions required to claim the credit; and (3) the extent to which the federal government's implementation of the credit aligned with selected criteria for assessing tax expenditure performance. GAO reviewed IRS summary data, relevant laws and regulations, and other agency documents. It interviewed IRS, Treasury, EPA, and DOE officials, as well as representatives from producers that publicly reported claiming the credit, randomly selected power plants, and a testing facility. GAO selected criteria that were most relevant by analyzing its guide to assessing the performance of tax expenditures, which include tax credits.
    [Read More…]
  • Defense Contractors: Information on Violations of Safety, Health, and Fair Labor Standards
    In U.S GAO News
    GAO's analysis of federal data found that about 1 percent of companies with Department of Defense (DOD) contracts were cited for willful or repeated safety, health, or fair labor violations in fiscal years 2015 through 2019. However, these data do not indicate whether the violations occurred while performing work related to a defense contract. Companies with DOD Contracts Cited for Willful or Repeated Violations under the Fair Labor Standards Act of 1938 or the Occupational Safety and Health Act of 1970, Fiscal Years 2015 through 2019 Because of limitations in available data, GAO could not determine the total incidence of willful or repeated violations of safety, health, or fair labor standards among all companies with a defense contract in this 5-year time frame. Specifically, about 43 percent of the Department of Labor's (Labor) safety and health violation data did not include key company identification numbers. These numbers are necessary to match federal contracting data to violation data. GAO recommended in February 2019 that Labor explore ways to address this issue. While Labor neither agreed nor disagreed with the recommendation, it issued a memorandum in May 2019 directing its Occupational Safety and Health Administration staff to make every reasonable effort to collect this information during inspections and enter it into its database. About 1 percent of Labor's data on fair labor violations were missing these key company identification numbers. The nature of the willful or repeated violations for companies with DOD contracts during fiscal years 2015 through 2019 varied. According to GAO's analysis of Labor data, the most frequently found willful or repeated safety and health violations related to toxic substances and machinery. For that same time frame, the most frequently found willful or repeated fair labor violations related to failure to pay overtime. The National Defense Authorization Act for Fiscal Year 2020 included a provision for GAO to report on the number of DOD contractors that Labor found to have committed willful or repeated violations under the Occupational Safety and Health Act of 1970 (OSH Act) or the Fair Labor Standards Act of 1938 (FLSA) for fiscal years 2015 through 2019. This report examines the number of DOD contractors that were cited for willful or repeated safety, health, or fair labor standards violations under the OSH Act or FLSA, and the nature of those violations for fiscal years 2015 through 2019. GAO analyzed federal contracting data to identify companies that had defense contracts in fiscal years 2015 through 2019, and matched them to Labor data on companies cited for willful or repeated safety, health, or fair labor standards violations. In addition, GAO used the Labor data to identify information on the nature of the violations. GAO also reviewed relevant federal laws and regulations, and agency documents. For more information, contact William T. Woods at (202) 512-4841 or woodsw@gao.gov, or Thomas Costa at (202) 512-7215 or costat@gao.gov.
    [Read More…]
  • Ukrainian Arrested and Charged with Ransomware Attack on Kaseya
    In Crime News
    Today, the Justice Department announced recent actions taken against two foreign nationals charged with deploying Sodinokibi/REvil ransomware to attack businesses and government entities in the United States.
    [Read More…]
  • Afghan National Arrested for 2008 Abduction of American Journalist
    In Crime News
    The Department of Justice announced the unsealing of a federal indictment charging Haji Najibullah, a/k/a “Najibullah Naim,” a/k/a “Abu Tayeb,” a/k/a “Atiqullah” with six counts related to the 2008 kidnapping of an American journalist and two Afghan nationals. Najibullah, 44, was arrested and transferred to the United States from Ukraine to face the charges in the indictment. Najibullah will be presented today before U.S. Magistrate Judge Ona T. Wang. The case is assigned to U.S. District Judge Katherine Polk Failla.
    [Read More…]
  • Return to Election Negotiations
    In Crime Control and Security News
    Ned Price, Department [Read More…]
  • Study Coordinator Pleads Guilty in Scheme to Falsify Clinical Drug Trial Data
    In Crime News
    A Colorado man pleaded guilty today in connection with his participation in a scheme to falsify clinical drug trial data. 
    [Read More…]
  • Just the Facts: Insurance Case Filings Spike After Natural Disasters
    In U.S Courts
    In the U.S. district courts, filings of civil cases involving insurance typically have surged following weather catastrophes. Over the past 20 years, devastating hurricanes and severe floods have resulted in the Eastern District of Louisiana processing the most insurance cases of any district court.
    [Read More…]
  • Four Ohio Individuals Charged with Gambling and Tax Offenses
    In Crime News
    A federal grand jury in Cleveland, Ohio, returned a superseding indictment on May 13, 2021, that was unsealed yesterday, charging three Ohio men and one woman with conspiring to operate illegal gambling businesses and to defraud the IRS, among other criminal offenses.
    [Read More…]
  • Belgium National Day
    In Crime Control and Security News
    Antony J. Blinken, [Read More…]
  • Assistant Attorney General Kenneth A. Polite Jr. of the Criminal Division Delivers Remarks on Operation Dark HunTor
    In Crime News
    Operation Dark HunTor stands as our most recent victory in the global fight against cyber-enabled drug trafficking.  The online trafficking of opioids, particularly fentanyl, poses a lethal threat to not only the United States, but also to our European and Australian counterparts, and beyond. This is a global threat that requires a global response. Our communities now face the constant threat of relatively easy access to dangerous illicit drugs now being peddled not on a street corner but in cyber space. Operation Dark HunTor highlights both the magnitude of this lethal threat, and the significant efforts we are taking at the Department of Justice to address it. 
    [Read More…]
Network News © 2005 Area.Control.Network™ All rights reserved.