January 29, 2022

News

News Network

Covid-19: Data Quality and Considerations for Modeling and Analysis

35 min read
<div>The rapid spread and magnitude of the COVID-19 pandemic have underscored the importance of having quality data, analyses, and models describing the potential trajectory of COVID-19 to help understand the effects of the disease in the U.S. The Centers for Disease Control and Prevention (CDC) is using multiple surveillance systems to collect data on COVID-19 in the U.S. in collaboration with state, local, and academic and other partners. The data from these surveillance systems can be useful for understanding the disease, but decision makers and analysts must understand their limitations in order to interpret them properly. For example, surveillance data on the number of reported COVID-19 cases are incomplete for a number of reasons, and they are an undercount the true number of cases, according to CDC and others. There are multiple approaches to analyzing COVID-19 data that yield different insights. For example, some approaches can help compare the effects of the disease across population groups. Additional analytical approaches can help to address incomplete and inconsistent reporting of COVID-19 deaths as well. For example, analysts can examine the number of deaths beyond what would normally be expected in the absence of the pandemic. Examining higher-than-expected deaths from all causes helps to address limitations in the reporting of COVID-19 deaths because the number of total deaths is likely more accurate than the numbers of deaths from specific causes. The figure below shows actual deaths from the weeks ending January 1 through June 27, 2020, based on data from CDC’s National Center for Health Statistics, compared with the expected deaths based on prior years’ data. Deaths that exceeded this threshold starting in late March are considered excess deaths that may be related to the COVID-19 pandemic. Higher-Than-Expected Weekly Mortality for 2020, as of July 14, 2020 Analysts have used several forecasting models to predict the spread of COVID-19, and understanding these models requires understanding their purpose and limitations. For example, some models attempt to predict the effects of various interventions, whereas other models attempt to forecast the number of cases based on current data. At the beginning of an outbreak, such predictions are less likely to be accurate, but accuracy can improve as the disease becomes better understood. The COVID-19 pandemic has resulted in significant loss of life and profoundly disrupted the U.S. economy and society, and the Congress has taken action to support a multifaceted federal response on an unprecedented scale. It is important for decision makers to understand the limitations of COVID-19 data, and the uses and limitations of various methods of analyzing and interpreting those data. The Coronavirus Aid, Relief, and Economic Security Act (CARES Act) includes a provision for GAO to, in general, conduct monitoring and oversight of the authorities and funding provided to address the COVID-19 pandemic and the effect of the pandemic on the health, economy, and public and private institutions of the U.S. This technology assessment examines (1) collection methods and limitations of COVID-19 surveillance data reported by CDC, (2) approaches for analyzing COVID-19 data, and (3) uses and limitations of forecast modeling for understanding of COVID-19. In conducting this assessment, GAO obtained publicly available information from CDC and state health departments, among other sources, and reviewed relevant peer reviewed and preprint (non-peer-reviewed) literature, as well as published technical data on specific models. For more information, contact Timothy M. Persons, PhD at (202) 512-6888 or PersonsT@gao.gov, SaraAnn Moessbauer at (202) 512-4943, or MoessbauerS@gao.gov, or Mary Denigan-Macauley, PhD at (202) 512-7114 or DeniganMacauleyM@gao.gov.</div>

What GAO Found

The rapid spread and magnitude of the COVID-19 pandemic have underscored the importance of having quality data, analyses, and models describing the potential trajectory of COVID-19 to help understand the effects of the disease in the U.S. The Centers for Disease Control and Prevention (CDC) is using multiple surveillance systems to collect data on COVID-19 in the U.S. in collaboration with state, local, and academic and other partners. The data from these surveillance systems can be useful for understanding the disease, but decision makers and analysts must understand their limitations in order to interpret them properly. For example, surveillance data on the number of reported COVID-19 cases are incomplete for a number of reasons, and they are an undercount the true number of cases, according to CDC and others.

There are multiple approaches to analyzing COVID-19 data that yield different insights. For example, some approaches can help compare the effects of the disease across population groups. Additional analytical approaches can help to address incomplete and inconsistent reporting of COVID-19 deaths as well. For example, analysts can examine the number of deaths beyond what would normally be expected in the absence of the pandemic. Examining higher-than-expected deaths from all causes helps to address limitations in the reporting of COVID-19 deaths because the number of total deaths is likely more accurate than the numbers of deaths from specific causes. The figure below shows actual deaths from the weeks ending January 1 through June 27, 2020, based on data from CDC’s National Center for Health Statistics, compared with the expected deaths based on prior years’ data. Deaths that exceeded this threshold starting in late March are considered excess deaths that may be related to the COVID-19 pandemic.

Higher-Than-Expected Weekly Mortality for 2020, as of July 14, 2020

Analysts have used several forecasting models to predict the spread of COVID-19, and understanding these models requires understanding their purpose and limitations. For example, some models attempt to predict the effects of various interventions, whereas other models attempt to forecast the number of cases based on current data. At the beginning of an outbreak, such predictions are less likely to be accurate, but accuracy can improve as the disease becomes better understood.

Why GAO Did This Study

The COVID-19 pandemic has resulted in significant loss of life and profoundly disrupted the U.S. economy and society, and the Congress has taken action to support a multifaceted federal response on an unprecedented scale. It is important for decision makers to understand the limitations of COVID-19 data, and the uses and limitations of various methods of analyzing and interpreting those data.

The Coronavirus Aid, Relief, and Economic Security Act (CARES Act) includes a provision for GAO to, in general, conduct monitoring and oversight of the authorities and funding provided to address the COVID-19 pandemic and the effect of the pandemic on the health, economy, and public and private institutions of the U.S. This technology assessment examines (1) collection methods and limitations of COVID-19 surveillance data reported by CDC, (2) approaches for analyzing COVID-19 data, and (3) uses and limitations of forecast modeling for understanding of COVID-19.

In conducting this assessment, GAO obtained publicly available information from CDC and state health departments, among other sources, and reviewed relevant peer reviewed and preprint (non-peer-reviewed) literature, as well as published technical data on specific models.

For more information, contact Timothy M. Persons, PhD at (202) 512-6888 or PersonsT@gao.gov, SaraAnn Moessbauer at (202) 512-4943, or MoessbauerS@gao.gov, or Mary Denigan-Macauley, PhD at (202) 512-7114 or DeniganMacauleyM@gao.gov.

News Network

  • Nuclear Weapons: NNSA Should Further Develop Cost, Schedule, and Risk Information for the W87-1 Warhead Program
    In U.S GAO News
    The National Nuclear Security Administration (NNSA) did not consider cost estimates in early major design decisions for the W87-1 warhead because it was not required to do so, but NNSA has since changed its guidance to require that cost be considered, according to a May 2019 NNSA review of program documentation. The design decisions that remain for features that would achieve either minimum or enhanced requirements for the W87-1 could affect cost, according to NNSA officials (see table). We found, however, that NNSA did not yet have study plans for assessing the costs and benefits of the remaining decisions consistent with best practices as detailed in NNSA's analysis of alternatives business procedure. NNSA does not require and only recommends that programs such as the W87-1 follow these best practices. By directing the W87-1 program and future weapons programs to follow best practices for design studies, or to justify and document deviations, NNSA would have better assurance that design studies apply consistent, reliable, and objective approaches. NNSA Cost Estimates for W87-1 Warhead Design Variations That Meet Minimum and Enhanced Requirements, as of December 2018 (Dollars in billions) W87-1 design variations Cost estimate rangea Design includes features that meet minimum safety and security requirements 7.7 - 13.3 Design includes enhanced safety and security features 8.6 - 14.8 Difference between the above estimate ranges 0.9 - 1.5 Source: National Nuclear Security Administration (NNSA) documentation | GAO-20-703 aThe cost ranges reflect low and high estimates for a single design variation. The ranges represent technical and production risk and uncertainty. It is not clear that NNSA will be able to produce sufficient numbers of pits—the fissile cores of the primary—to meet the W87-1 warhead's planned production schedule. Recent NNSA and independent studies have cast doubt on NNSA's ability to ready its two planned pit production facilities in time. If one facility is not ready to produce pits in the early 2030s, for example, NNSA would likely produce fewer weapons than planned, according to GAO's analysis of NNSA plans. We were unable to fully assess the extent to which the two pit production facilities will be ready to produce pits for the W87-1 because NNSA's plutonium program—which is managing the facility readiness efforts—has not yet completed an integrated schedule for the overall pit production effort. An integrated schedule is important, according to best practices, because it integrates the planned work, resources, and budget. An NNSA official stated that the program was building a schedule, but could not provide documentation that it would meet best practices. A schedule consistent with best practices would provide NNSA with better assurance that it will have adequate pits to meet planned W87-1 production. This is a public version of a classified report that GAO issued in February 2020. Information that NNSA or DOD deemed classified or sensitive has been omitted. The Department of Defense (DOD) and NNSA restarted a program in fiscal year 2019 to replace the capabilities of the aging W78 nuclear warhead with the W87-1. NNSA made key design decisions for this weapon from 2010 until the program was paused in 2014. NNSA estimated in December 2018 that the W87-1 would cost $8.6 billion to $14.8 billion, which could make it the most expensive warhead modernization program to date. NNSA plans to newly manufacture the entire warhead, including the two major nuclear components, called the primary and secondary, using facilities it is modernizing or repurposing. You asked us to examine plans for the W87-1 warhead. This report examines, among other things, the extent to which NNSA (1) considered cost estimates in prior design decisions for the W87-1 and the potential effects of remaining design decisions on program cost, and (2) will be able to produce sufficient numbers of key nuclear components to meet W87-1 production needs. GAO reviewed NNSA documentation on prior and remaining design decisions and preliminary cost estimates, reviewed warhead and component production schedules, and interviewed NNSA and DOD officials. GAO is making four recommendations, including that NNSA require programs such as the W87-1 to follow analysis of alternatives best practices when studying design options and that the plutonium program build an integrated schedule consistent with schedule best practices. NNSA generally agreed with the recommendations. For more information, contact Allison B. Bawden at (202) 512-3841 or bawdena@gao.gov.
    [Read More…]
  • Conroe man charged with funneling money to India
    In Justice News
    A 48-year-old local man [Read More…]
  • Terrorist Designation of Abd al-Aziz Malluh Mirjirash al-Muhammadawi
    In Crime Control and Security News
    Office of the [Read More…]
  • Fort Bend County home health owner charged with copying and pasting doctor signatures
    In Justice News
    A 60-year-old Richmond [Read More…]
  • Secretary Antony J. Blinken at the National Security Commission on Artificial Intelligence’s (NSCAI) Global Emerging Technology Summit
    In Crime Control and Security News
    Antony J. Blinken, [Read More…]
  • Confronting Heightened Cybersecurity Threats Amid COVID-19
    In Human Health, Resources and Services
    Did you know that [Read More…]
  • Seven MS-13 Gang Members Indicted in Violent Crime and Drug Distribution Conspiracy
    In Crime News
    A federal grand jury in Nashville, Tennessee, returned a 16-count superseding indictment Wednesday, charging seven MS-13 gang members with conspiracy to distribute cocaine and marijuana and serious firearm-related offenses, announced Acting Assistant Attorney General Brian C. Rabbitt of the Justice Department’s Criminal Division and U.S. Attorney Don Cochran for the Middle District of Tennessee.
    [Read More…]
  • Deputy Secretary Biegun’s Calls with Armenian Foreign Minister Mnatsakanyan and Azerbaijani Foreign Minister Bayramov
    In Crime Control and Security News
    Office of the [Read More…]
  • Secretary Antony J. Blinken With Hillary Clinton, “You and Me Both with Hillary Clinton” Podcast
    In Crime Control and Security News
    Antony J. Blinken, [Read More…]
  • NASA’s Deep Space Station in Australia Is Getting an Upgrade
    In Space
    Used for communicating [Read More…]
  • Secretary Blinken’s Call with Pakistani Foreign Minister Qureshi
    In Crime Control and Security News
    Office of the [Read More…]
  • Robert Katzmann, Judge and Civics Advocate, Dies at 68
    In U.S Courts
    Robert A. Katzmann, a former chief judge of the Second Circuit U.S. Court of Appeals and a tireless, impassioned advocate of civics education, died June 9. He was 68.
    [Read More…]
  • On the Loya Jirga in Afghanistan
    In Crime Control and Security News
    Michael R. Pompeo, [Read More…]
  • Texas Man Pleads Guilty to Hate Crime Charges After Using Dating App to Target Gay Men for Violent Crimes
    In Crime News
    A Texas man pleaded guilty yesterday to federal hate crime charges in the U.S. District Court for the Northern District of Texas.
    [Read More…]
  • Judge Testifies at House Hearing on Pandemic Impacts
    In U.S Courts
    Through a combination of advance planning, expanded use of technology, and the dedication of thousands of employees, the federal Judiciary’s response to the pandemic has enabled courts to continue to operate, while ensuring the health and safety of the public and court personnel, U.S. Senior District Judge David G. Campbell told Congress on Thursday.
    [Read More…]
  • Congratulatory Message on the 30th Anniversary of the Visegrád Group (V4)
    In Crime Control and Security News
    Ned Price, Department [Read More…]
  • General Aviation: Stakeholders Expressed Mixed Views of FAA Policies on Private Pilot Expense Sharing
    In U.S GAO News
    The Federal Aviation Administration's (FAA) primary rationale for its policies on private pilots' sharing expenses with passengers is based on passenger expectations of safety. FAA policies allow private pilots to share the cost of certain flight expenses with passengers but prohibit these pilots from engaging in “common carriage,” which is communicating to the public a willingness to fly in exchange for compensation. These policies generally prohibit pilots from using the internet to find passengers. FAA officials said these policies are in place because they are concerned the public might expect a similar level of safety on private expense-sharing flights as commercial flights. However, the safety record of commercial aviation is better than that of private flying (general aviation). For example, according to data from the National Transportation Safety Board (NTSB), commercial carriers had a fatal accident rate around 30 times lower than general aviation in 2018. FAA officials said their goal for FAA's 2020 guidance on expense sharing was to restate and clarify existing policies. Example of an Aircraft Private Pilots Could Use for Expense-Sharing Flights Stakeholders described benefits of expense sharing but expressed mixed views on FAA's policies and guidance. For example, stakeholders cited potential economic benefits to the general aviation sector and a potential expansion of the pool of future professional pilots as benefits of expense sharing. Most (eight of 13) stakeholders said FAA's 2020 guidance on expense-sharing is clear and provides sufficient information. However, some stakeholders said the guidance could provide more definitive examples of allowed expense-sharing flights, and others disagreed with how FAA defined certain concepts such as how pilots can be compensated for flying passengers. Also, stakeholders split on whether FAA should allow pilots to use the internet to find expense-sharing passengers. Seven of 15 stakeholders, including four representatives from companies with expense-sharing applications, said FAA should allow pilots to use the internet to find these passengers by citing, for example, ongoing positive experiences in Europe. However, eight stakeholders, including six of seven professional organizations, said FAA should not. These stakeholders cited safety-related risks of expense sharing including what they characterized as FAA's limited capacity to enforce current regulations and flights using less experienced pilots. Private flying is expensive, and FAA allows private pilots to reduce their costs by carrying passengers and sharing certain flight expenses with them. However, private pilots cannot engage in common carriage. If pilots do engage in common carriage, they are subject to FAA's more stringent regulations covering commercial air carriers. Some private pilots have sought to use internet applications to find expense-sharing passengers. The FAA Reauthorization Act of 2018 directed FAA to issue advisory guidance clarifying how private pilots may share expenses. In February 2020, FAA released this guidance as an advisory circular. The Act also includes a provision for GAO to review FAA's policies on expense sharing. This report describes: (1) FAA's rationale for its policies on how private pilots may find expense-sharing passengers and (2) selected stakeholder perspectives on FAA's policies and the risks and benefits of arranging these expense-sharing flights online. GAO interviewed FAA officials on how FAA developed its policies and guidance related to expense sharing. GAO also reviewed FAA's data on enforcement actions related to expense sharing and safety data from NTSB. In addition, GAO interviewed a non-generalizable sample of 15 private-sector stakeholders, including professional organizations, such as trade groups representing general aviation pilots, companies that developed expense-sharing internet applications, and flying clubs. For more information, contact Heather Krause at (202) 512-2834 or krauseh@gao.gov.
    [Read More…]
  • Remarks by Deputy Attorney General Jeffrey A. Rosen on the 19th Anniversary of the September 11th Terrorist Attacks
    In Crime News
    Thank you, Lee.  The [Read More…]
  • Afghanistan: Key Oversight Issues for USAID Development Efforts
    In U.S GAO News
    What GAO Found In 2010, the United States pledged to provide at least 50 percent of its development aid directly through the Afghan government budget within 2 years. This direct assistance is intended to help develop the capacity of Afghan government ministries to manage programs and funds. Using bilateral agreements and multilateral trust funds, the United States more than tripled its direct assistance awards to Afghanistan in the first year of the pledge, going from over $470 million in fiscal year 2009 to over $1.4 billion in fiscal year 2010. The U.S. Agency for International Development's (USAID) most current reporting shows that for fiscal year 2012 the agency provided over $800 million in mission funds through direct assistance. In 2013, GAO reported that while USAID had established and generally complied with various financial and other controls in its direct assistance agreements, it had not always assessed the risks in providing direct assistance before awarding funds. USAID has taken steps in response to GAO's recommendations to help ensure the accountability of direct assistance funds provided to the Afghan government. Recently, the Special Inspector General for Afghanistan Reconstruction (SIGAR) reported that USAID determined that seven ministries were unable to manage direct assistance funds without a risk mitigation strategy in place. However, SIGAR reported that USAID approved assistance for the ministries, but did not mitigate for all identified risks. GAO has previously reported on systemic weaknesses in USAID's monitoring and evaluation of programs carried out by its implementing partners in Afghanistan. For example, although USAID collected progress reports from implementing partners for agriculture and water projects, it did not always analyze and interpret data to, among other things, inform future decisions. USAID has undertaken some efforts to improve its monitoring and evaluation of the billions of dollars invested towards development projects in Afghanistan. GAO and other oversight agencies, however, have highlighted gaps that show USAID continued to inconsistently apply performance management procedures, falls short in maintaining institutional knowledge, and needs to improve oversight of contractors. USAID's ability to conduct its mission and the challenges it has faced in providing oversight and monitoring of its development projects in Afghanistan are likely to be exacerbated by the planned withdrawal of U.S. and coalition combat troops from Afghanistan at the end of 2014. The United States is currently transitioning from counterinsurgency and stability operations toward more traditional diplomatic and development activities. As U.S. combat troops withdraw from Afghanistan, provincial reconstruction teams will continue to decline in number, thus challenging USAID's opportunities to directly monitor and evaluate programs in certain parts of Afghanistan. To prepare for the possible lack of USAID personnel in the field, USAID has undertaken various planning efforts to mitigate against potential challenges. For example, USAID is planning to implement a remote monitoring program that will use contractors to verify activities that implementing partners have completed. As the United States plans for the withdrawal of its combat troops and the transition from an integrated civilian and military effort to a civilian-led presence, GAO believes it is important to have safeguards in place to help ensure sustainment of the gains made by U.S. and coalition investments. Why GAO Did This Study The U.S. government has been engaged in efforts in Afghanistan since declaring a global war on terrorism that targeted al Qaeda, its affiliates, and other violent extremists. The U.S. effort has involved a whole of government approach to disrupt, dismantle, and defeat al Qaeda and its affiliates and strengthen Afghanistan so that it can never again be a haven for terrorists. This approach includes USAID's development assistance and reconstruction efforts, which to date have invested over $15 billion in Afghanistan since 2002. To assist Congress in its oversight, GAO has issued over 50 products in the past 5 years focusing on U.S. efforts in Afghanistan. This testimony summarizes the findings from those products related to USAID efforts in Afghanistan and discusses: (1) levels of U.S. direct assistance and need for continued oversight, (2) the importance of routine monitoring and evaluation of USAID projects in Afghanistan, and (3) the need for mitigation planning for how USAID will continue to operate in Afghanistan after the withdrawal of U.S. combat troops.
    [Read More…]
  • Rebuilding Iraq: Governance, Security, Reconstruction, and Financing Challenges
    In U.S GAO News
    The United States, along with coalition partners and various international organizations, has undertaken a challenging and costly effort to stabilize and rebuild Iraq following multiple wars and decades of neglect by the former regime. This enormous effort is taking place in an unstable security environment, concurrent with Iraqi efforts to transition to its first permanent government. In November 2005, the President issued the National Strategy for Victory in Iraq. According to the strategy, victory will be achieved when Iraq is peaceful, united, stable, secure, well integrated into the international community, and a full partner in the global war on terror. In this testimony, GAO discusses the key challenges that the United States, international community, and Iraq face in rebuilding and stabilizing Iraq. This statement is based on four reports GAO has issued to the Congress since July 2005 and recent trips to Iraq. Since July 2005, GAO issued reports on (1) the status of funding and reconstruction efforts in Iraq, the progress achieved, and challenges faced in rebuilding Iraq's infrastructure; (2) U.S. efforts in the water and sanitation sector; and (3) U.S. assistance for the January 2005 Iraqi elections, and (4) U.S. efforts to stabilize the security situation in Iraq (a classified report).The war in Iraq will not be won by the military alone. Iraq's future requires strong Iraqi leadership, sustained U.S. commitment, and a reengaged international community. The United States, Iraq, and its partners have made some progress in stabilizing and rebuilding Iraq. Iraqis have voted in increasing numbers, with over 12 million casting votes in the December 2005 election. Over the past year, the number of security forces that the coalition has trained and equipped has increased from about 142,000 to about 242,000. Finally, the United States has completed or has underway about 500 water, oil, and electricity reconstruction projects. However, this progress is tempered by the overwhelming challenges the coalition faces. First, sectarian divisions delayed the formation of a permanent government and created a political vacuum. Recent events provide some hope that a new government will be formed in the near future. Once formed, the new government will confront the enormous tasks of strengthening government institutions, disbanding the militias, resolving disputes over internal boundaries and oil revenues, addressing corruption, and delivering results to the Iraqi people. Of particular importance is providing the Iraqis with the training and technical assistance needed to run their national and provincial governments. A transparent and accountable government can reduce corruption and deliver results to the Iraqi people. Second, the security environment continues to be a concern as insurgents demonstrate the ability to recruit, supply, and attack coalition and Iraqi security forces. From 2004 to 2005, attacks against the coalition, Iraqis, and infrastructure increased 23 percent. Since the bombing of a Samarra mosque in February 2006, Iraqis have become increasingly concerned that civil war may break out. The poor security situation in much of Iraq has impeded the development of an inclusive Iraqi government and effective Iraqi security forces. Third, higher than expected security costs, funding reallocations, and inadequate maintenance have impeded U.S. reconstruction efforts. As of March 2006, oil and electricity production were below pre-war levels and reconstruction goals for oil, electricity, and water had not been met. Iraq produced 2.6 million barrels of oil per day before the war; in 2005, production averaged 2.1 million barrels per day. Production levels alone do not measure the impact of reconstruction efforts. While U.S. efforts have helped Iraq produce more clean water, 60 percent is lost due to leakage and contamination. Continued focus on developing outcome measures is critical to ensure that reconstruction efforts are making a difference in the lives of the Iraqi people.
    [Read More…]
Network News © 2005 Area.Control.Network™ All rights reserved.