Food and Drug Administration1Much of this column was informed by two CRS Reports: “How FDA Approves Drugs and Regulates Their Safety and Effectiveness”; and Congressional Research Services, “FDA Regulation of Medical Devices,” Food and Drug Administration, updated September 14, 2016, https://crsreports.congress.gov/product/pdf/R/R42130.US AI Governance 
(Existing and Proposed Provisions)
Risk Classification
→ Drugs in which benefits/needs are greater than risks can move through premarket approval more quickly (breakthrough therapy, accelerated approval and fast track).


→ Medical devices are categorized by risk tiers (class I, class II, class III).
→ AI systems are classified as high-risk2AI governance proposals all define a set of AI systems to cover; not all are called “high-risk” in the legislative text. The companies that develop or deploy the “high-risk” AI system(s) of interest are often called “developers/deployers,” “online platforms,” “covered entities,” or something similar. based on the output/sector of decision (access to services, facial recognition, recommender systems, etc.) 3Executive Order 14110 on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (October 2023)4Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)5U.S. AI Act Sen. Blumenthal (D-CT) & Sen. Hawley (R-MO) (July 2023)6Algorithm Accountability Act (S.2892) Sen. Wyden(D-OR) (September 2023)7CA AB-331 Automated Decision Tools (amended text) Bauer-Kahan (May 2023)8Artificial Intelligence (AI) Research, Innovation, and Accountability Act (S.3312) Sen.Thune (R-SD) & Sen.Klobuchar (D-MN) (November 2023)9AI Leadership To Enable Accountable Deployment Act (S.2293) Sen.Peters(D-MI) & Sen.Cornyn (R-TX) (July 2023)10Federal Artificial Intelligence Risk Management Act (S.3205) Sen.Moran (R-KS) & Sen.Warner (D-VA) (November 2023)11Digital Services Oversight and Safety Act (H.R.6796) Rep.Trahan (D-MA) (Februrary 2022)12Algorithmic Justice and Online Platform Transparency Act (S.2325) Sen.Markey (D-MA) (July 2023)


→ AI systems are classified as high-risk based on computational power / number of parameters 13Executive Order 14110 on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (October 2023)14U.S. AI Act Sen. Blumenthal (D-CT) & Sen. Hawley (R-MO) (July 2023)


→ AI systems are classified as high-risk on market / gatekeeper power. 15Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)16Digital Platform Commission Act (S.1671) Sen. Bennet(D-CO) (May 2023)17American Innovation and Choice Online Act (S.2033) Sen.Klobuchar (D-MN) & Sen.Grassley (R-IA) (June 2023)
Risk Assessment
→ Manufacturers and FDA assess risk throughout clinical trials and premarket approval. 

→ Manufacturers design their own studies/tests which are approved by the FDA.

→ FDA conducts audits/inspections of clinical trials and product manufacturing sites.

→ FDA accredits third-party review organizations to assist with 510(k)(medical device application) review process and inspections of foreign facilities.

→ FDA engages experts and patients in approval process and post-market surveillance.		→ Deployers/developers complete risk assessments that may include safety testing, disparate impact testing, descriptions of training data, model interpretation, etc. Performed inline with standards (set with engagement through NIST or agency rulemaking) [4, 18Executive Order 14110 on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (October 2023)19Digital Platform Commission Act (S.1671) Sen. Bennet(D-CO) (May 2023)20U.S. AI Act Sen. Blumenthal (D-CT) & Sen. Hawley (R-MO) (July 2023)21Algorithm Accountability Act (S.2892) Sen. Wyden(D-OR) (September 2023)22CA AB-331 Automated Decision Tools (amended text) Bauer-Kahan (May 2023)23Artificial Intelligence (AI) Research, Innovation, and Accountability Act (S.3312) Sen.Thune (R-SD) & Sen.Klobuchar (D-MN) (November 2023)[fn][fn]Digital Services Oversight and Safety Act (H.R.6796) Rep.Trahan (D-MA) (Februrary 2022)24Algorithmic Justice and Online Platform Transparency Act (S.2325) Sen.Markey (D-MA) (July 2023)25Consumer Online Privacy Rights Act (S.3195) Sen.Cantwell (D-WA) (November 2021)26American Data Privacy Protection Act (H.R.8152) Rep.Pallone (D-NJ) (June 2022)

→ Deployers/developers provide risk assessments to governing agency 27Executive Order 14110 on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (October 2023)28Algorithm Accountability Act (S.2892) Sen. Wyden(D-OR) (September 2023)29CA AB-331 Automated Decision Tools (amended text) Bauer-Kahan (May 2023)30Artificial Intelligence (AI) Research, Innovation, and Accountability Act (S.3312) Sen.Thune (R-SD) & Sen.Klobuchar (D-MN) (November 2023)31Digital Services Oversight and Safety Act (H.R.6796) Rep.Trahan (D-MA) (Februrary 2022)

→ Covered entity must have its online platform audited by a third party based on standards set by the governing agency32Digital Platform Commission Act (S.1671) Sen. Bennet(D-CO) (May 2023)33Digital Services Oversight and Safety Act (H.R.6796) Rep.Trahan (D-MA) (Februrary 2022)

→ Risk-assessment summaries are made publicly available.34Algorithm Accountability Act (S.2892) Sen. Wyden(D-OR) (September 2023)

→ Deployers/developers self-certify to following key safety standards and mandates 35Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)36Artificial Intelligence (AI) Research, Innovation, and Accountability Act (S.3312) Sen.Thune (R-SD) & Sen.Klobuchar (D-MN) (November 2023)

→ Governing agency issues licenses to operate AI system [10, 37(once a size limit is reached) Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)

→ Deployers/developers assess/document AI systems as part of government procurement process [5, 38Executive Order 14110 on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (October 2023)39AI Leadership To Enable Accountable Deployment Act (S.2293) Sen.Peters(D-MI) & Sen.Cornyn (R-TX) (July 2023)40Federal Artificial Intelligence Risk Management Act (S.3205) Sen.Moran (R-KS) & Sen.Warner (D-VA) (November 2023)

→ Deployers/developers and/or governing agency engage experts and data subjects in assessment/approval 41NIST’s work creating standards relevant to AI systems, notably AI Risk Management Framework (RMF)42Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)
Standards→ Manufacturers reference industry standards that have been recognized by the FDA in pre- and post-market assessments 

→ FDA publishes risk-mitigation guidance for nearly every class II and class III medical device category.		→ Almost every AI governance proposal includes reference to NIST standards.

→ Governing agency engages stakeholders to craft voluntary codes that covered entities attest to following 43Digital Platform Commission Act (S.1671) Sen. Bennet(D-CO) (May 2023)44Digital Services Oversight and Safety Act (H.R.6796) Rep.Trahan (D-MA) (Februrary 2022)
Under Disclosures→ FDA promulgates and maintains rules for labeling and disclosures in marketing materials for products (drugs, devices, food, cosmetics).→ Deployers/developers alert users to the use of automated decision systems 45California Consumer Privacy Act (and Draft Automated Decisionmaking Technology Regulations)46Artificial Intelligence (AI) Research, Innovation, and Accountability Act (S.3312) Sen.Thune (R-SD) & Sen.Klobuchar (D-MN) (November 2023)47AI Labeling Act (S.2691) Sen.Schatz (D-HI) (July 2023)

→ Deployers/developers watermark/label AI-generated content 48Executive Order 14110 on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (October 2023)49U.S. AI Act Sen. Blumenthal (D-CT) & Sen. Hawley (R-MO) (July 2023)/fn][fn]Artificial Intelligence (AI) Research, Innovation, and Accountability Act (S.3312) Sen.Thune (R-SD) & Sen.Klobuchar (D-MN) (November 2023)50DEEP FAKE Accountability Act (H.R.2395) Rep.Clarke (D-NY) (April 2021)51Advisory for AI-Generated Content Act (S.2765) Sen.Ricketts (R-NE) (September 2023)52AI Labeling Act (S.2691) Sen.Schatz (D-HI) (July 2023)53REAL Political Advertisements Act (H.R.3044) Rep.Clarke (D-NY) (May 2023)
 
→ Deployers/developers publish data nutrition labels / model cards (notices on how data is used by AI systems and user-friendly explanations of key model features) 54Digital Platform Commission Act (S.1671) Sen. Bennet(D-CO) (May 2023)55U.S. AI Act Sen. Blumenthal (D-CT) & Sen. Hawley (R-MO) (July 2023)56Digital Services Oversight and Safety Act (H.R.6796) Rep.Trahan (D-MA) (Februrary 2022)57Algorithmic Justice and Online Platform Transparency Act (S.2325) Sen.Markey (D-MA) (July 2023)

→ Social media platforms disclose recommendation system weights / important features and offer users control over personal data inputs58Digital Services Oversight and Safety Act (H.R.6796) Rep.Trahan (D-MA) (Februrary 2022)59Platform Accountability and Transparency Act (S.1876) Sen.Coons (D-DE) & Sen.Cassidy (R-LA) (June 2023)
Incident Reporting→ Manufacturers of drugs and devices must report all serious and unexpected adverse reactions to the FDA within 15 days.

→ FDA oversees a process for clinicians, patients, and caregivers to voluntarily report adverse outcomes.		→ Developers/deployers of AI systems are required to report adverse safety incidents to governing entity.60U.S. AI Act Sen. Blumenthal (D-CT) & Sen. Hawley (R-MO) (July 2023)

→ Governing agency maintains systems to intake complaints about online platforms (and/or mandate that online platforms set up and report on complaints) 61Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)62Digital Platform Commission Act (S.1671) Sen. Bennet(D-CO) (May 2023)63Digital Services Oversight and Safety Act (H.R.6796) Rep.Trahan (D-MA) (Februrary 2022)
Post-Market Studies→ FDA can require post-market studies when a drug/device is approved or as the agency becomes aware of risks.→ Online platforms must provide access to data to enable academic and civil society research 64Digital Platform Commission Act (S.1671) Sen. Bennet(D-CO) (May 2023)65U.S. AI Act Sen. Blumenthal (D-CT) & Sen. Hawley (R-MO) (July 2023)66Digital Services Oversight and Safety Act (H.R.6796) Rep.Trahan (D-MA) (Februrary 2022)67Platform Accountability and Transparency Act (S.1876) Sen.Coons (D-DE) & Sen.Cassidy (R-LA) (June 2023)
Supply Chain Monitoring /Export Controls→ FDA maintains registries and unique identifiers for medical devices and manufacturing facilities. 

→ FDA can issue import alerts (“refuse admission” of products into the US market).

→ FDA can mandate product recalls (and/or issue warning letters).		→ Governing agency maintains registry of high-risk AI systems.68U.S. AI Act Sen. Blumenthal (D-CT) & Sen. Hawley (R-MO) (July 2023)

→ Inventories of AI use cases in government.69Advancing American AI Act (passed in 2022)

→ Governing agency can limit transfer of AI systems to adversarial nations.70U.S. AI Act Sen. Blumenthal (D-CT) & Sen. Hawley (R-MO) (July 2023)

→ Governing agency can revoke a developer/deployer’s license to operate/declare the developer/deployer as noncompliant 71Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)72U.S. AI Act Sen. Blumenthal (D-CT) & Sen. Hawley (R-MO) (July 2023)73Artificial Intelligence (AI) Research, Innovation, and Accountability Act (S.3312) Sen.Thune (R-SD) & Sen.Klobuchar (D-MN) (November 2023)
Bright Lines / Banned Products→ FDA can prohibit medical devices deemed too risky for human use (Section 516[a] FD&C Act).→ AI governance proposals do not include giving a regulatory entity the ability to prohibit the development/dissemination of specific subsets of AI systems. 

→ There is legislation to ban specific use cases of AI systems (in nuclear weapons,74Block Nuclear Launch by Autonomous Artificial Intelligence Act (S.1394) Sen.Markey (D-MA) (May 2023) targeted advertising,75Banning Surveillance Advertising Act (H.R.5534) Rep.Eshoo (D-CA) (September 2023) facial recognition,76Facial Recognition and Biometric Technology Moratorium Act (S.681) Sen.Markey (D-MA) (March 2023) for children,77AI Shield for Kids Act (S.1626) Sen.Scott (R-FL) (May 2023) in the election context78Protect Elections from Deceptive AI Act (S.2770) Sen.Klobuchar (D-MN) (September 2023)79Candidate Voice Fraud Prohibition Act (H.R.4611) Rep.Espaillat (D-NY) (July 2023)
Mandates / Prohibit Conduct→ The FD&C Act includes statutes to mandate/prohibit certain conduct mostly with regard to the manufacturing, distribution, and marketing of food, drug, and cosmetic products.AI governance proposals often include language to mandate that covered entities engage in / don’t engage in sets of defined conduct: 

→ Covered entities must not engage in anticompetitive conduct (e.g., self-preferencing, maintaining conflict of interest).80Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)81American Innovation and Choice Online Act (S.2033) Sen.Klobuchar (D-MN) & Sen.Grassley (R-IA) (June 2023)

→ Covered entities must provide data portability and interoperability 82Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)83Digital Platform Commission Act (S.1671) Sen. Bennet(D-CO) (May 2023)84Consumer Online Privacy Rights Act (S.3195) Sen.Cantwell (D-WA) (November 2021)85American Data Privacy Protection Act (H.R.8152) Rep.Pallone (D-NJ) (June 2022)

→ Covered entities must protect personal/sensitive/covered data 86Section 5 of the Federal Trade Commission Act (15 USC 45) prohibits ‘“unfair or deceptive acts or practices in or affecting commerce”87Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)88Consumer Online Privacy Rights Act (S.3195) Sen.Cantwell (D-WA) (November 2021)89American Data Privacy Protection Act (H.R.8152) Rep.Pallone (D-NJ) (June 2022)

→ Covered entities must not process data (used in an automated decision system) in a way that discriminates 90Discrimination laws in DOJ’s purview 91Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)92CA AB-331 Automated Decision Tools (amended text) Bauer-Kahan (May 2023)93Algorithmic Justice and Online Platform Transparency Act (S.2325) Sen.Markey (D-MA) (July 2023)94Consumer Online Privacy Rights Act (S.3195) Sen.Cantwell (D-WA) (November 2021)95American Data Privacy Protection Act (H.R.8152) Rep.Pallone (D-NJ) (June 2022)96Stopping Unlawful Negative Machine Impacts through National Evaluation Act (S.5351) Sen.Portman (R-OH) (December 2022)

→ Covered entities offer a process for human review when decisions are made using automated decision systems (contestability) 97California Consumer Privacy Act (and Draft Automated Decisionmaking Technology Regulations)98Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)99U.S. AI Act Sen. Blumenthal (D-CT) & Sen. Hawley (R-MO) (July 2023)100CA AB-331 Automated Decision Tools (amended text) Bauer-Kahan (May 2023)101Transparent Automated Governance Act (S.1865) Sen.Peters (D-MI) (June 2023) 

→ Covered entities must not include misleading claims when marketing AI systems.102Section 5 of the Federal Trade Commission Act (15 USC 45) prohibits ‘“unfair or deceptive acts or practices in or affecting commerce”
Duty of Care/Tort→ Food, drugs, and cosmetics in general are subject to tort law, and if individuals are harmed they can bring a case in court.→ Some AI systems are likely covered by existing tort law and will not receive liability shields under Section 230.103Existing tort laws

→ Some proposals include text to create a duty of care for data processing / online platforms enforced by a regulator and/or state attorney general’s office 104Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)105Consumer Online Privacy Rights Act (S.3195) Sen.Cantwell (D-WA) (November 2021)106Data Care Act (S.744) Sen.Schatz (D-HI) (March 2023)

→ Some proposals clarify that Section 230 does not apply to generative AI. 107U.S. AI Act Sen. Blumenthal (D-CT) & Sen. Hawley (R-MO) (July 2023)
Investigations→ FDA can undertake investigations tied to enforcing the FD&C Act using subpoena power.

→ FDA can refer cases to the Department of Justice.		→ Most AI governance proposals aimed at creating a new agency include investigative authorities to enforce laws.

→ Governing agency has the authority to “study” the digital/AI market, similar to FTC 6(b) studies 108Digital Consumer Protection Commission Act (S.2597) Sen.Warren (D-MA) & Sen.Graham (R-SC) (July 2023)109Digital Platform Commission Act (S.1671) Sen. Bennet(D-CO) (May 2023)
Funding→ Congressional appropriations 

→ User fees for drug approval (negotiated with drug manufacturers that set benchmarks the agency must meet)		→ The vast majority of AI governance proposals are set up to be funded through congressional appropriations. 

→ Some would generate revenue through penalties paid by companies.