Tasks/Task 5/task5-ai-safety-sdg.csv

question,contexts,ground_truth,evolution_type,metadata,episode_done
What measures are suggested to mitigate concerns of harmful bias and homogenization in AI training data?,"[' \n37 \nMS-2.11-005 \nAssess the proportion of synthetic to non-synthetic training data and verify \ntraining data is not overly homogenous or GAI-produced to mitigate concerns of \nmodel collapse. \nHarmful Bias and Homogenization \nAI Actor Tasks: AI Deployment, AI Impact Assessment, Affected Individuals and Communities, Domain Experts, End-Users, \nOperation and Monitoring, TEVV \n \nMEASURE 2.12: Environmental impact and sustainability of AI model training and management activities – as identified in the MAP \nfunction – are assessed and documented. \nAction ID \nSuggested Action \nGAI Risks \nMS-2.12-001 Assess safety to physical environments when deploying GAI systems. \nDangerous, Violent, or Hateful \nContent \nMS-2.12-002 Document anticipated environmental impacts of model development, \nmaintenance, and deployment in product design decisions. \nEnvironmental \nMS-2.12-003 \nMeasure or estimate environmental impacts (e.g., energy and water \nconsumption) for training, fine tuning, and deploying models: Verify tradeoffs \nbetween resources used at inference time versus additional resources required \nat training time. \nEnvironmental \nMS-2.12-004 Verify effectiveness of carbon capture or offset programs for GAI training and \napplications, and address green-washing concerns. \nEnvironmental \nAI Actor Tasks: AI Deployment, AI Impact Assessment, Domain Experts, Operation and Monitoring, TEVV \n \n']",Assess the proportion of synthetic to non-synthetic training data and verify training data is not overly homogenous or GAI-produced to mitigate concerns of model collapse.,simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 40, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
What was the purpose of the Generative AI Public Working Group (GAI PWG) facilitated by NIST?,"[' \n2 \nThis work was informed by public feedback and consultations with diverse stakeholder groups as part of NIST’s \nGenerative AI Public Working Group (GAI PWG). The GAI PWG was an open, transparent, and collaborative \nprocess, facilitated via a virtual workspace, to obtain multistakeholder input on GAI risk management and to \ninform NIST’s approach. \nThe focus of the GAI PWG was limited to four primary considerations relevant to GAI: Governance, Content \nProvenance, Pre-deployment Testing, and Incident Disclosure (further described in Appendix A). As such, the \nsuggested actions in this document primarily address these considerations. \nFuture revisions of this profile will include additional AI RMF subcategories, risks, and suggested actions based \non additional considerations of GAI as the space evolves and empirical evidence indicates additional risks. A \nglossary of terms pertinent to GAI risk management will be developed and hosted on NIST’s Trustworthy & \nResponsible AI Resource Center (AIRC), and added to The Language of Trustworthy AI: An In-Depth Glossary of \nTerms. \nThis document was also informed by public comments and consultations from several Requests for Information. \n \n2. \nOverview of Risks Unique to or Exacerbated by GAI \nIn the context of the AI RMF, risk refers to the composite measure of an event’s probability (or \nlikelihood) of occurring and the magnitude or degree of the consequences of the corresponding event. \nSome risks can be assessed as likely to materialize in a given context, particularly those that have been \nempirically demonstrated in similar contexts. Other risks may be unlikely to materialize in a given \ncontext, or may be more speculative and therefore uncertain. \nAI risks can differ from or intensify traditional software risks. Likewise, GAI can exacerbate existing AI \nrisks, and creates unique risks. GAI risks can vary along many dimensions: \n• \nStage of the AI lifecycle: Risks can arise during design, development, deployment, operation, \nand/or decommissioning. \n• \nScope: Risks may exist at individual model or system levels, at the application or implementation \nlevels (i.e., for a specific use case), or at the ecosystem level – that is, beyond a single system or \norganizational context. Examples of the latter include the expansion of “algorithmic \nmonocultures,3” resulting from repeated use of the same model, or impacts on access to \nopportunity, labor markets, and the creative economies.4 \n• \nSource of risk: Risks may emerge from factors related to the design, training, or operation of the \nGAI model itself, stemming in some cases from GAI model or system inputs, and in other cases, \nfrom GAI system outputs. Many GAI risks, however, originate from human behavior, including \n \n \n3 “Algorithmic monocultures” refers to the phenomenon in which repeated use of the same model or algorithm in \nconsequential decision-making settings like employment and lending can result in increased susceptibility by \nsystems to correlated failures (like unexpected shocks), due to multiple actors relying on the same algorithm.  \n4 Many studies have projected the impact of AI on the workforce and labor markets. Fewer studies have examined \nthe impact of GAI on the labor market, though some industry surveys indicate that that both employees and \nemployers are pondering this disruption.  \n']",The purpose of the Generative AI Public Working Group (GAI PWG) facilitated by NIST was to obtain multistakeholder input on GAI risk management and to inform NIST's approach.,simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 5, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
Why is transparency important when individuals are placed on a watch list by a predictive policing system?,"[' \n \n  \n \n \nNOTICE & \nEXPLANATION \nWHY THIS PRINCIPLE IS IMPORTANT\nThis section provides a brief summary of the problems which the principle seeks to address and protect \nagainst, including illustrative examples. \n•\nA predictive policing system claimed to identify individuals at greatest risk to commit or become the victim of\ngun violence (based on automated analysis of social ties to gang members, criminal histories, previous experi\xad\nences of gun violence, and other factors) and led to individuals being placed on a watch list with no\nexplanation or public transparency regarding how the system came to its conclusions.85 Both police and\nthe public deserve to understand why and how such a system is making these determinations.\n•\nA system awarding benefits changed its criteria invisibly. Individuals were denied benefits due to data entry\nerrors and other system flaws. These flaws were only revealed when an explanation of the system\nwas demanded and produced.86 The lack of an explanation made it harder for errors to be corrected in a\ntimely manner.\n42\n']",Transparency is important when individuals are placed on a watch list by a predictive policing system because both police and the public deserve to understand why and how such a system is making these determinations.,simple,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 41, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
What procedures should be established and maintained for the remediation of issues that trigger incident response processes for the use of a GAI system?,"[' \n42 \nMG-2.4-002 \nEstablish and maintain procedures for escalating GAI system incidents to the \norganizational risk management authority when specific criteria for deactivation \nor disengagement is met for a particular context of use or for the GAI system as a \nwhole. \nInformation Security \nMG-2.4-003 \nEstablish and maintain procedures for the remediation of issues which trigger \nincident response processes for the use of a GAI system, and provide stakeholders \ntimelines associated with the remediation plan. \nInformation Security \n \nMG-2.4-004 Establish and regularly review specific criteria that warrants the deactivation of \nGAI systems in accordance with set risk tolerances and appetites. \nInformation Security \n \nAI Actor Tasks: AI Deployment, Governance and Oversight, Operation and Monitoring \n \nMANAGE 3.1: AI risks and benefits from third-party resources are regularly monitored, and risk controls are applied and \ndocumented. \nAction ID \nSuggested Action \nGAI Risks \nMG-3.1-001 \nApply organizational risk tolerances and controls (e.g., acquisition and \nprocurement processes; assessing personnel credentials and qualifications, \nperforming background checks; filtering GAI input and outputs, grounding, fine \ntuning, retrieval-augmented generation) to third-party GAI resources: Apply \norganizational risk tolerance to the utilization of third-party datasets and other \nGAI resources; Apply organizational risk tolerances to fine-tuned third-party \nmodels; Apply organizational risk tolerance to existing third-party models \nadapted to a new domain; Reassess risk measurements after fine-tuning third-\nparty GAI models. \nValue Chain and Component \nIntegration; Intellectual Property \nMG-3.1-002 \nTest GAI system value chain risks (e.g., data poisoning, malware, other software \nand hardware vulnerabilities; labor practices; data privacy and localization \ncompliance; geopolitical alignment). \nData Privacy; Information Security; \nValue Chain and Component \nIntegration; Harmful Bias and \nHomogenization \nMG-3.1-003 \nRe-assess model risks after fine-tuning or retrieval-augmented generation \nimplementation and for any third-party GAI models deployed for applications \nand/or use cases that were not evaluated in initial testing. \nValue Chain and Component \nIntegration \nMG-3.1-004 \nTake reasonable measures to review training data for CBRN information, and \nintellectual property, and where appropriate, remove it. Implement reasonable \nmeasures to prevent, flag, or take other action in response to outputs that \nreproduce particular training data (e.g., plagiarized, trademarked, patented, \nlicensed content or trade secret material). \nIntellectual Property; CBRN \nInformation or Capabilities \n']","Establish and maintain procedures for the remediation of issues which trigger incident response processes for the use of a GAI system, and provide stakeholders timelines associated with the remediation plan.",simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 45, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
What is the purpose of utilizing a purpose-built testing environment such as NIST Dioptra in evaluating GAI trustworthy characteristics?,"[' \n31 \nMS-2.3-004 \nUtilize a purpose-built testing environment such as NIST Dioptra to empirically \nevaluate GAI trustworthy characteristics. \nCBRN Information or Capabilities; \nData Privacy; Confabulation; \nInformation Integrity; Information \nSecurity; Dangerous, Violent, or \nHateful Content; Harmful Bias and \nHomogenization \nAI Actor Tasks: AI Deployment, TEVV \n \nMEASURE 2.5: The AI system to be deployed is demonstrated to be valid and reliable. Limitations of the generalizability beyond the \nconditions under which the technology was developed are documented. \nAction ID \nSuggested Action \nRisks \nMS-2.5-001 Avoid extrapolating GAI system performance or capabilities from narrow, non-\nsystematic, and anecdotal assessments. \nHuman-AI Configuration; \nConfabulation \nMS-2.5-002 \nDocument the extent to which human domain knowledge is employed to \nimprove GAI system performance, via, e.g., RLHF, fine-tuning, retrieval-\naugmented generation, content moderation, business rules. \nHuman-AI Configuration \nMS-2.5-003 Review and verify sources and citations in GAI system outputs during pre-\ndeployment risk measurement and ongoing monitoring activities. \nConfabulation \nMS-2.5-004 Track and document instances of anthropomorphization (e.g., human images, \nmentions of human feelings, cyborg imagery or motifs) in GAI system interfaces. Human-AI Configuration \nMS-2.5-005 Verify GAI system training data and TEVV data provenance, and that fine-tuning \nor retrieval-augmented generation data is grounded. \nInformation Integrity \nMS-2.5-006 \nRegularly review security and safety guardrails, especially if the GAI system is \nbeing operated in novel circumstances. This includes reviewing reasons why the \nGAI system was initially assessed as being safe to deploy.  \nInformation Security; Dangerous, \nViolent, or Hateful Content \nAI Actor Tasks: Domain Experts, TEVV \n \n']",The purpose of utilizing a purpose-built testing environment such as NIST Dioptra is to empirically evaluate GAI trustworthy characteristics.,simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 34, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
What input did the White House Office of Science and Technology Policy seek regarding algorithmic and data-driven harms?,"[' \n \n \nABOUT THIS FRAMEWORK\xad\xad\xad\xad\xad\nThe Blueprint for an AI Bill of Rights is a set of five principles and associated practices to help guide the \ndesign, use, and deployment of automated systems to protect the rights of the American public in the age of \nartificial intel-ligence. Developed through extensive consultation with the American public, these principles are \na blueprint for building and deploying automated systems that are aligned with democratic values and protect \ncivil rights, civil liberties, and privacy. The Blueprint for an AI Bill of Rights includes this Foreword, the five \nprinciples, notes on Applying the The Blueprint for an AI Bill of Rights, and a Technical Companion that gives \nconcrete steps that can be taken by many kinds of organizations—from governments at all levels to companies of \nall sizes—to uphold these values. Experts from across the private sector, governments, and international \nconsortia have published principles and frameworks to guide the responsible use of automated systems; this \nframework provides a national values statement and toolkit that is sector-agnostic to inform building these \nprotections into policy, practice, or the technological design process.  Where existing law or policy—such as \nsector-specific privacy laws and oversight requirements—do not already provide guidance, the Blueprint for an \nAI Bill of Rights should be used to inform policy decisions.\nLISTENING TO THE AMERICAN PUBLIC\nThe White House Office of Science and Technology Policy has led a year-long process to seek and distill input \nfrom people across the country—from impacted communities and industry stakeholders to technology develop-\ners and other experts across fields and sectors, as well as policymakers throughout the Federal government—on \nthe issue of algorithmic and data-driven harms and potential remedies. Through panel discussions, public listen-\ning sessions, meetings, a formal request for information, and input to a publicly accessible and widely-publicized \nemail address, people throughout the United States, public servants across Federal agencies, and members of the \ninternational community spoke up about both the promises and potential harms of these technologies, and \nplayed a central role in shaping the Blueprint for an AI Bill of Rights. The core messages gleaned from these \ndiscussions include that AI has transformative potential to improve Americans’ lives, and that preventing the \nharms of these technologies is both necessary and achievable. The Appendix includes a full list of public engage-\nments. \n4\n']","The White House Office of Science and Technology Policy sought input from people across the country, including impacted communities, industry stakeholders, technology developers, other experts across fields and sectors, and policymakers throughout the Federal government. This input was gathered through panel discussions, public listening sessions, meetings, a formal request for information, and input to a publicly accessible and widely-publicized email address.",simple,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 3, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
What resource does the National Artificial Intelligence Initiative Office provide regarding AI use cases?,"[' \nENDNOTES\n12. Expectations about reporting are intended for the entity developing or using the automated system. The\nresulting reports can be provided to the public, regulators, auditors, industry standards groups, or others\nengaged in independent review, and should be made public as much as possible consistent with law,\nregulation, and policy, and noting that intellectual property or law enforcement considerations may prevent\npublic release. These reporting expectations are important for transparency, so the American people can\nhave confidence that their rights, opportunities, and access as well as their expectations around\ntechnologies are respected.\n13. National Artificial Intelligence Initiative Office. Agency Inventories of AI Use Cases. Accessed Sept. 8,\n2022. https://www.ai.gov/ai-use-case-inventories/\n14. National Highway Traffic Safety Administration. https://www.nhtsa.gov/\n15. See, e.g., Charles Pruitt. People Doing What They Do Best: The Professional Engineers and NHTSA. Public\nAdministration Review. Vol. 39, No. 4. Jul.-Aug., 1979. https://www.jstor.org/stable/976213?seq=1\n16. The US Department of Transportation has publicly described the health and other benefits of these\n“traffic calming” measures. See, e.g.: U.S. Department of Transportation. Traffic Calming to Slow Vehicle\nSpeeds. Accessed Apr. 17, 2022. https://www.transportation.gov/mission/health/Traffic-Calming-to-Slow\xad\nVehicle-Speeds\n17. Karen Hao. Worried about your firm’s AI ethics? These startups are here to help.\nA growing ecosystem of “responsible AI” ventures promise to help organizations monitor and fix their AI\nmodels. MIT Technology Review. Jan 15., 2021.\nhttps://www.technologyreview.com/2021/01/15/1016183/ai-ethics-startups/; Disha Sinha. Top Progressive\nCompanies Building Ethical AI to Look Out for in 2021. Analytics Insight. June 30, 2021. https://\nwww.analyticsinsight.net/top-progressive-companies-building-ethical-ai-to-look-out-for\xad\nin-2021/ https://www.technologyreview.com/2021/01/15/1016183/ai-ethics-startups/; Disha Sinha. Top\nProgressive Companies Building Ethical AI to Look Out for in 2021. Analytics Insight. June 30, 2021.\n18. Office of Management and Budget. Study to Identify Methods to Assess Equity: Report to the President.\nAug. 2021. https://www.whitehouse.gov/wp-content/uploads/2021/08/OMB-Report-on-E013985\xad\nImplementation_508-Compliant-Secure-v1.1.pdf\n19. National Institute of Standards and Technology. AI Risk Management Framework. Accessed May 23,\n2022. https://www.nist.gov/itl/ai-risk-management-framework\n20. U.S. Department of Energy. U.S. Department of Energy Establishes Artificial Intelligence Advancement\nCouncil. U.S. Department of Energy Artificial Intelligence and Technology Office. April 18, 2022. https://\nwww.energy.gov/ai/articles/us-department-energy-establishes-artificial-intelligence-advancement-council\n21. Department of Defense. U.S Department of Defense Responsible Artificial Intelligence Strategy and\nImplementation Pathway. Jun. 2022. https://media.defense.gov/2022/Jun/22/2003022604/-1/-1/0/\nDepartment-of-Defense-Responsible-Artificial-Intelligence-Strategy-and-Implementation\xad\nPathway.PDF\n22. Director of National Intelligence. Principles of Artificial Intelligence Ethics for the Intelligence\nCommunity. https://www.dni.gov/index.php/features/2763-principles-of-artificial-intelligence-ethics-for\xad\nthe-intelligence-community\n64\n']",The National Artificial Intelligence Initiative Office provides Agency Inventories of AI Use Cases.,simple,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 63, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
"What are the documented roles and responsibilities related to mapping, measuring, and managing AI risks within an organization?","[' \n17 \nGOVERN 1.7: Processes and procedures are in place for decommissioning and phasing out AI systems safely and in a manner that \ndoes not increase risks or decrease the organization’s trustworthiness. \nAction ID \nSuggested Action \nGAI Risks \nGV-1.7-001 Protocols are put in place to ensure GAI systems are able to be deactivated when \nnecessary.  \nInformation Security; Value Chain \nand Component Integration \nGV-1.7-002 \nConsider the following factors when decommissioning GAI systems: Data \nretention requirements; Data security, e.g., containment, protocols, Data leakage \nafter decommissioning; Dependencies between upstream, downstream, or other \ndata, internet of things (IOT) or AI systems; Use of open-source data or models; \nUsers’ emotional entanglement with GAI functions. \nHuman-AI Configuration; \nInformation Security; Value Chain \nand Component Integration \nAI Actor Tasks: AI Deployment, Operation and Monitoring \n \nGOVERN 2.1: Roles and responsibilities and lines of communication related to mapping, measuring, and managing AI risks are \ndocumented and are clear to individuals and teams throughout the organization. \nAction ID \nSuggested Action \nGAI Risks \nGV-2.1-001 \nEstablish organizational roles, policies, and procedures for communicating GAI \nincidents and performance to AI Actors and downstream stakeholders (including \nthose potentially impacted), via community or official resources (e.g., AI incident \ndatabase, AVID, CVE, NVD, or OECD AI incident monitor). \nHuman-AI Configuration; Value \nChain and Component Integration \nGV-2.1-002 Establish procedures to engage teams for GAI system incident response with \ndiverse composition and responsibilities based on the particular incident type. \nHarmful Bias and Homogenization \nGV-2.1-003 Establish processes to verify the AI Actors conducting GAI incident response tasks \ndemonstrate and maintain the appropriate skills and training. \nHuman-AI Configuration \nGV-2.1-004 When systems may raise national security risks, involve national security \nprofessionals in mapping, measuring, and managing those risks. \nCBRN Information or Capabilities; \nDangerous, Violent, or Hateful \nContent; Information Security \nGV-2.1-005 \nCreate mechanisms to provide protections for whistleblowers who report, based \non reasonable belief, when the organization violates relevant laws or poses a \nspecific and empirically well-substantiated negative risk to public safety (or has \nalready caused harm). \nCBRN Information or Capabilities; \nDangerous, Violent, or Hateful \nContent \nAI Actor Tasks: Governance and Oversight \n \n']","Roles and responsibilities and lines of communication related to mapping, measuring, and managing AI risks are documented and are clear to individuals and teams throughout the organization.",simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 20, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
"What steps should organizations take to ensure accessibility during the design, development, and deployment of automated systems?","["" \n \n \n \n \n \n \n \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nEnsuring accessibility during design, development, and deployment. Systems should be \ndesigned, developed, and deployed by organizations in ways that ensure accessibility to people with disabili\xad\nties. This should include consideration of a wide variety of disabilities, adherence to relevant accessibility \nstandards, and user experience research both before and after deployment to identify and address any accessi\xad\nbility barriers to the use or effectiveness of the automated system. \nDisparity assessment. Automated systems should be tested using a broad set of measures to assess wheth\xad\ner the system components, both in pre-deployment testing and in-context deployment, produce disparities. \nThe demographics of the assessed groups should be as inclusive as possible of race, color, ethnicity, sex \n(including pregnancy, childbirth, and related medical conditions, gender identity, intersex status, and sexual \norientation), religion, age, national origin, disability, veteran status, genetic information, or any other classifi\xad\ncation protected by law. The broad set of measures assessed should include demographic performance mea\xad\nsures, overall and subgroup parity assessment, and calibration. Demographic data collected for disparity \nassessment should be separated from data used for the automated system and privacy protections should be \ninstituted; in some cases it may make sense to perform such assessment using a data sample. For every \ninstance where the deployed automated system leads to different treatment or impacts disfavoring the identi\xad\nfied groups, the entity governing, implementing, or using the system should document the disparity and a \njustification for any continued use of the system. \nDisparity mitigation. When a disparity assessment identifies a disparity against an assessed group, it may \nbe appropriate to take steps to mitigate or eliminate the disparity. In some cases, mitigation or elimination of \nthe disparity may be required by law. \nDisparities that have the potential to lead to algorithmic \ndiscrimination, cause meaningful harm, or violate equity49 goals should be mitigated. When designing and \nevaluating an automated system, steps should be taken to evaluate multiple models and select the one that \nhas the least adverse impact, modify data input choices, or otherwise identify a system with fewer \ndisparities. If adequate mitigation of the disparity is not possible, then the use of the automated system \nshould be reconsidered. One of the considerations in whether to use the system should be the validity of any \ntarget measure; unobservable targets may result in the inappropriate use of proxies. Meeting these \nstandards may require instituting mitigation procedures and other protective measures to address \nalgorithmic discrimination, avoid meaningful harm, and achieve equity goals. \nOngoing monitoring and mitigation. Automated systems should be regularly monitored to assess algo\xad\nrithmic discrimination that might arise from unforeseen interactions of the system with inequities not \naccounted for during the pre-deployment testing, changes to the system after deployment, or changes to the \ncontext of use or associated data. Monitoring and disparity assessment should be performed by the entity \ndeploying or using the automated system to examine whether the system has led to algorithmic discrimina\xad\ntion when deployed. This assessment should be performed regularly and whenever a pattern of unusual \nresults is occurring. It can be performed using a variety of approaches, taking into account whether and how \ndemographic information of impacted people is available, for example via testing with a sample of users or via \nqualitative user experience research. Riskier and higher-impact systems should be monitored and assessed \nmore frequently. Outcomes of this assessment should include additional disparity mitigation, if needed, or \nfallback to earlier procedures in the case that equity standards are no longer met and can't be mitigated, and \nprior mechanisms provide better adherence to equity standards. \n27\nAlgorithmic \nDiscrimination \nProtections \n""]","Organizations should ensure accessibility during the design, development, and deployment of automated systems by considering a wide variety of disabilities, adhering to relevant accessibility standards, and conducting user experience research both before and after deployment to identify and address any accessibility barriers to the use or effectiveness of the automated system.",simple,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 26, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
How do digital transparency mechanisms help manage and mitigate risks associated with AI-generated content?,"[' \n51 \ngeneral public participants. For example, expert AI red-teamers could modify or verify the \nprompts written by general public AI red-teamers. These approaches may also expand coverage \nof the AI risk attack surface.  \n• \nHuman / AI: Performed by GAI in combination with specialist or non-specialist human teams. \nGAI-led red-teaming can be more cost effective than human red-teamers alone. Human or GAI-\nled AI red-teaming may be better suited for eliciting different types of harms. \n \nA.1.6. Content Provenance \nOverview \nGAI technologies can be leveraged for many applications such as content generation and synthetic data. \nSome aspects of GAI outputs, such as the production of deepfake content, can challenge our ability to \ndistinguish human-generated content from AI-generated synthetic content. To help manage and mitigate \nthese risks, digital transparency mechanisms like provenance data tracking can trace the origin and \nhistory of content. Provenance data tracking and synthetic content detection can help facilitate greater \ninformation access about both authentic and synthetic content to users, enabling better knowledge of \ntrustworthiness in AI systems. When combined with other organizational accountability mechanisms, \ndigital content transparency approaches can enable processes to trace negative outcomes back to their \nsource, improve information integrity, and uphold public trust. Provenance data tracking and synthetic \ncontent detection mechanisms provide information about the origin and history of content to assist in \nGAI risk management efforts. \nProvenance metadata can include information about GAI model developers or creators of GAI content, \ndate/time of creation, location, modifications, and sources. Metadata can be tracked for text, images, \nvideos, audio, and underlying datasets. The implementation of provenance data tracking techniques can \nhelp assess the authenticity, integrity, intellectual property rights, and potential manipulations in digital \ncontent. Some well-known techniques for provenance data tracking include digital watermarking, \nmetadata recording, digital fingerprinting, and human authentication, among others. \nProvenance Data Tracking Approaches \nProvenance data tracking techniques for GAI systems can be used to track the history and origin of data \ninputs, metadata, and synthetic content. Provenance data tracking records the origin and history for \ndigital content, allowing its authenticity to be determined. It consists of techniques to record metadata \nas well as overt and covert digital watermarks on content. Data provenance refers to tracking the origin \nand history of input data through metadata and digital watermarking techniques. Provenance data \ntracking processes can include and assist AI Actors across the lifecycle who may not have full visibility or \ncontrol over the various trade-offs and cascading impacts of early-stage model decisions on downstream \nperformance and synthetic outputs. For example, by selecting a watermarking model to prioritize \nrobustness (the durability of a watermark), an AI actor may inadvertently diminish computational \ncomplexity (the resources required to implement watermarking). Organizational risk management \nefforts for enhancing content provenance include:  \n• \nTracking provenance of training data and metadata for GAI systems; \n• \nDocumenting provenance data limitations within GAI systems; \n']","Digital transparency mechanisms like provenance data tracking can trace the origin and history of content. Provenance data tracking and synthetic content detection can help facilitate greater information access about both authentic and synthetic content to users, enabling better knowledge of trustworthiness in AI systems. When combined with other organizational accountability mechanisms, digital content transparency approaches can enable processes to trace negative outcomes back to their source, improve information integrity, and uphold public trust.",simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 54, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
What is the suggested action for compiling statistics on actual policy violations in organizational GAI systems?,"[' \n34 \nMS-2.7-009 Regularly assess and verify that security measures remain effective and have not \nbeen compromised. \nInformation Security \nAI Actor Tasks: AI Deployment, AI Impact Assessment, Domain Experts, Operation and Monitoring, TEVV \n \nMEASURE 2.8: Risks associated with transparency and accountability – as identified in the MAP function – are examined and \ndocumented. \nAction ID \nSuggested Action \nGAI Risks \nMS-2.8-001 \nCompile statistics on actual policy violations, take-down requests, and intellectual \nproperty infringement for organizational GAI systems: Analyze transparency \nreports across demographic groups, languages groups. \nIntellectual Property; Harmful Bias \nand Homogenization \nMS-2.8-002 Document the instructions given to data annotators or AI red-teamers. \nHuman-AI Configuration \nMS-2.8-003 \nUse digital content transparency solutions to enable the documentation of each \ninstance where content is generated, modified, or shared to provide a tamper-\nproof history of the content, promote transparency, and enable traceability. \nRobust version control systems can also be applied to track changes across the AI \nlifecycle over time. \nInformation Integrity \nMS-2.8-004 Verify adequacy of GAI system user instructions through user testing. \nHuman-AI Configuration \nAI Actor Tasks: AI Deployment, AI Impact Assessment, Domain Experts, Operation and Monitoring, TEVV \n \n']","Compile statistics on actual policy violations, take-down requests, and intellectual property infringement for organizational GAI systems: Analyze transparency reports across demographic groups, languages groups.",simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 37, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
What is data poisoning and how can it affect GAI systems?,"[' \n11 \nvalue chain (e.g., data inputs, processing, GAI training, or deployment environments), conventional \ncybersecurity practices may need to adapt or evolve. \nFor instance, prompt injection involves modifying what input is provided to a GAI system so that it \nbehaves in unintended ways. In direct prompt injections, attackers might craft malicious prompts and \ninput them directly to a GAI system, with a variety of downstream negative consequences to \ninterconnected systems. Indirect prompt injection attacks occur when adversaries remotely (i.e., without \na direct interface) exploit LLM-integrated applications by injecting prompts into data likely to be \nretrieved. Security researchers have already demonstrated how indirect prompt injections can exploit \nvulnerabilities by stealing proprietary data or running malicious code remotely on a machine. Merely \nquerying a closed production model can elicit previously undisclosed information about that model. \nAnother cybersecurity risk to GAI is data poisoning, in which an adversary compromises a training \ndataset used by a model to manipulate its outputs or operation. Malicious tampering with data or parts \nof the model could exacerbate risks associated with GAI system outputs. \nTrustworthy AI Characteristics: Privacy Enhanced, Safe, Secure and Resilient, Valid and Reliable \n2.10. \nIntellectual Property \nIntellectual property risks from GAI systems may arise where the use of copyrighted works is not a fair \nuse under the fair use doctrine. If a GAI system’s training data included copyrighted material, GAI \noutputs displaying instances of training data memorization (see Data Privacy above) could infringe on \ncopyright. \nHow GAI relates to copyright, including the status of generated content that is similar to but does not \nstrictly copy work protected by copyright, is currently being debated in legal fora. Similar discussions are \ntaking place regarding the use or emulation of personal identity, likeness, or voice without permission.  \nTrustworthy AI Characteristics: Accountable and Transparent, Fair with Harmful Bias Managed, Privacy \nEnhanced  \n2.11. \nObscene, Degrading, and/or Abusive Content \nGAI can ease the production of and access to illegal non-consensual intimate imagery (NCII) of adults, \nand/or child sexual abuse material (CSAM). GAI-generated obscene, abusive or degrading content can \ncreate privacy, psychological and emotional, and even physical harms, and in some cases may be illegal.  \nGenerated explicit or obscene AI content may include highly realistic “deepfakes” of real individuals, \nincluding children. The spread of this kind of material can have downstream negative consequences: in \nthe context of CSAM, even if the generated images do not resemble specific individuals, the prevalence \nof such images can divert time and resources from efforts to find real-world victims. Outside of CSAM, \nthe creation and spread of NCII disproportionately impacts women and sexual minorities, and can have \nsubsequent negative consequences including decline in overall mental health, substance abuse, and \neven suicidal thoughts.  \nData used for training GAI models may unintentionally include CSAM and NCII. A recent report noted \nthat several commonly used GAI training datasets were found to contain hundreds of known images of \n']",Data poisoning is a cybersecurity risk where an adversary compromises a training dataset used by a model to manipulate its outputs or operation. Malicious tampering with data or parts of the model could exacerbate risks associated with GAI system outputs.,simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 14, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
How do systems related to the assignment of penalties assist decision-makers in adjudicating administrative or criminal penalties?,"['APPENDIX\nSystems that impact the safety of communities such as automated traffic control systems, elec \n-ctrical grid controls, smart city technologies, and industrial emissions and environmental\nimpact control algorithms; and\nSystems related to access to benefits or services or assignment of penalties such as systems that\nsupport decision-makers who adjudicate benefits such as collating or analyzing information or\nmatching records, systems which similarly assist in the adjudication of administrative or criminal\npenalties, fraud detection algorithms, services or benefits access control algorithms, biometric\nsystems used as access control, and systems which make benefits or services related decisions on a\nfully or partially autonomous basis (such as a determination to revoke benefits).\n54\n']",Systems related to the assignment of penalties assist decision-makers in adjudicating administrative or criminal penalties by collating or analyzing information or matching records.,simple,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 53, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
What measures are suggested to mitigate concerns of model collapse related to the proportion of synthetic to non-synthetic training data?,"[' \n37 \nMS-2.11-005 \nAssess the proportion of synthetic to non-synthetic training data and verify \ntraining data is not overly homogenous or GAI-produced to mitigate concerns of \nmodel collapse. \nHarmful Bias and Homogenization \nAI Actor Tasks: AI Deployment, AI Impact Assessment, Affected Individuals and Communities, Domain Experts, End-Users, \nOperation and Monitoring, TEVV \n \nMEASURE 2.12: Environmental impact and sustainability of AI model training and management activities – as identified in the MAP \nfunction – are assessed and documented. \nAction ID \nSuggested Action \nGAI Risks \nMS-2.12-001 Assess safety to physical environments when deploying GAI systems. \nDangerous, Violent, or Hateful \nContent \nMS-2.12-002 Document anticipated environmental impacts of model development, \nmaintenance, and deployment in product design decisions. \nEnvironmental \nMS-2.12-003 \nMeasure or estimate environmental impacts (e.g., energy and water \nconsumption) for training, fine tuning, and deploying models: Verify tradeoffs \nbetween resources used at inference time versus additional resources required \nat training time. \nEnvironmental \nMS-2.12-004 Verify effectiveness of carbon capture or offset programs for GAI training and \napplications, and address green-washing concerns. \nEnvironmental \nAI Actor Tasks: AI Deployment, AI Impact Assessment, Domain Experts, Operation and Monitoring, TEVV \n \n']",Assess the proportion of synthetic to non-synthetic training data and verify training data is not overly homogenous or GAI-produced.,simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 40, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
What measures are suggested to identify and quantify new contexts of unanticipated impacts of GAI systems?,"[' \n28 \nMAP 5.2: Practices and personnel for supporting regular engagement with relevant AI Actors and integrating feedback about \npositive, negative, and unanticipated impacts are in place and documented. \nAction ID \nSuggested Action \nGAI Risks \nMP-5.2-001 \nDetermine context-based measures to identify if new impacts are present due to \nthe GAI system, including regular engagements with downstream AI Actors to \nidentify and quantify new contexts of unanticipated impacts of GAI systems. \nHuman-AI Configuration; Value \nChain and Component Integration \nMP-5.2-002 \nPlan regular engagements with AI Actors responsible for inputs to GAI systems, \nincluding third-party data and algorithms, to review and evaluate unanticipated \nimpacts. \nHuman-AI Configuration; Value \nChain and Component Integration \nAI Actor Tasks: AI Deployment, AI Design, AI Impact Assessment, Affected Individuals and Communities, Domain Experts, End-\nUsers, Human Factors, Operation and Monitoring  \n \nMEASURE 1.1: Approaches and metrics for measurement of AI risks enumerated during the MAP function are selected for \nimplementation starting with the most significant AI risks. The risks or trustworthiness characteristics that will not – or cannot – be \nmeasured are properly documented. \nAction ID \nSuggested Action \nGAI Risks \nMS-1.1-001 Employ methods to trace the origin and modifications of digital content. \nInformation Integrity \nMS-1.1-002 \nIntegrate tools designed to analyze content provenance and detect data \nanomalies, verify the authenticity of digital signatures, and identify patterns \nassociated with misinformation or manipulation. \nInformation Integrity \nMS-1.1-003 \nDisaggregate evaluation metrics by demographic factors to identify any \ndiscrepancies in how content provenance mechanisms work across diverse \npopulations. \nInformation Integrity; Harmful \nBias and Homogenization \nMS-1.1-004 Develop a suite of metrics to evaluate structured public feedback exercises \ninformed by representative AI Actors. \nHuman-AI Configuration; Harmful \nBias and Homogenization; CBRN \nInformation or Capabilities \nMS-1.1-005 \nEvaluate novel methods and technologies for the measurement of GAI-related \nrisks including in content provenance, offensive cyber, and CBRN, while \nmaintaining the models’ ability to produce valid, reliable, and factually accurate \noutputs. \nInformation Integrity; CBRN \nInformation or Capabilities; \nObscene, Degrading, and/or \nAbusive Content \n']","Determine context-based measures to identify if new impacts are present due to the GAI system, including regular engagements with downstream AI Actors to identify and quantify new contexts of unanticipated impacts of GAI systems.",simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 31, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
What are the expectations for automated systems in terms of risk assessment and explanation validity?,"["" \n \n \n \n \n \nNOTICE & \nEXPLANATION \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nTailored to the level of risk. An assessment should be done to determine the level of risk of the auto\xad\nmated system. In settings where the consequences are high as determined by a risk assessment, or extensive \noversight is expected (e.g., in criminal justice or some public sector settings), explanatory mechanisms should \nbe built into the system design so that the system’s full behavior can be explained in advance (i.e., only fully \ntransparent models should be used), rather than as an after-the-decision interpretation. In other settings, the \nextent of explanation provided should be tailored to the risk level. \nValid. The explanation provided by a system should accurately reflect the factors and the influences that led \nto a particular decision, and should be meaningful for the particular customization based on purpose, target, \nand level of risk. While approximation and simplification may be necessary for the system to succeed based on \nthe explanatory purpose and target of the explanation, or to account for the risk of fraud or other concerns \nrelated to revealing decision-making information, such simplifications should be done in a scientifically \nsupportable way. Where appropriate based on the explanatory system, error ranges for the explanation should \nbe calculated and included in the explanation, with the choice of presentation of such information balanced \nwith usability and overall interface complexity concerns. \nDemonstrate protections for notice and explanation \nReporting. Summary reporting should document the determinations made based on the above consider\xad\nations, including: the responsible entities for accountability purposes; the goal and use cases for the system, \nidentified users, and impacted populations; the assessment of notice clarity and timeliness; the assessment of \nthe explanation's validity and accessibility; the assessment of the level of risk; and the account and assessment \nof how explanations are tailored, including to the purpose, the recipient of the explanation, and the level of \nrisk. Individualized profile information should be made readily available to the greatest extent possible that \nincludes explanations for any system impacts or inferences. Reporting should be provided in a clear plain \nlanguage and machine-readable manner. \n44\n""]","The expectations for automated systems in terms of risk assessment and explanation validity include conducting an assessment to determine the level of risk of the automated system. In high-risk settings, explanatory mechanisms should be built into the system design to ensure full transparency. The explanation provided by the system should accurately reflect the factors and influences that led to a particular decision and should be meaningful for the specific customization based on purpose, target, and level of risk.",simple,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 43, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
What is the purpose of conducting proactive equity assessments in the design phase of automated systems?,"[' \n \n \n \n \n \n \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nAny automated system should be tested to help ensure it is free from algorithmic discrimination before it can be \nsold or used. Protection against algorithmic discrimination should include designing to ensure equity, broadly \nconstrued.  Some algorithmic discrimination is already prohibited under existing anti-discrimination law. The \nexpectations set out below describe proactive technical and policy steps that can be taken to not only \nreinforce those legal protections but extend beyond them to ensure equity for underserved communities48 \neven in circumstances where a specific legal protection may not be clearly established. These protections \nshould be instituted throughout the design, development, and deployment process and are described below \nroughly in the order in which they would be instituted. \nProtect the public from algorithmic discrimination in a proactive and ongoing manner \nProactive assessment of equity in design. Those responsible for the development, use, or oversight of \nautomated systems should conduct proactive equity assessments in the design phase of the technology \nresearch and development or during its acquisition to review potential input data, associated historical \ncontext, accessibility for people with disabilities, and societal goals to identify potential discrimination and \neffects on equity resulting from the introduction of the technology. The assessed groups should be as inclusive \nas possible of the underserved communities mentioned in the equity definition:  Black, Latino, and Indigenous \nand Native American persons, Asian Americans and Pacific Islanders and other persons of color; members of \nreligious minorities; women, girls, and non-binary people; lesbian, gay, bisexual, transgender, queer, and inter-\nsex (LGBTQI+) persons; older adults; persons with disabilities; persons who live in rural areas; and persons \notherwise adversely affected by persistent poverty or inequality. Assessment could include both qualitative \nand quantitative evaluations of the system. This equity assessment should also be considered a core part of the \ngoals of the consultation conducted as part of the safety and efficacy review. \nRepresentative and robust data. Any data used as part of system development or assessment should be \nrepresentative of local communities based on the planned deployment setting and should be reviewed for bias \nbased on the historical and societal context of the data. Such data should be sufficiently robust to identify and \nhelp to mitigate biases and potential harms. \nGuarding against proxies.  Directly using demographic information in the design, development, or \ndeployment of an automated system (for purposes other than evaluating a system for discrimination or using \na system to counter discrimination) runs a high risk of leading to algorithmic discrimination and should be \navoided. In many cases, attributes that are highly correlated with demographic features, known as proxies, can \ncontribute to algorithmic discrimination. In cases where use of the demographic features themselves would \nlead to illegal algorithmic discrimination, reliance on such proxies in decision-making (such as that facilitated \nby an algorithm) may also be prohibited by law. Proactive testing should be performed to identify proxies by \ntesting for correlation between demographic information and attributes in any data used as part of system \ndesign, development, or use. If a proxy is identified, designers, developers, and deployers should remove the \nproxy; if needed, it may be possible to identify alternative attributes that can be used instead. At a minimum, \norganizations should ensure a proxy feature is not given undue weight and should monitor the system closely \nfor any resulting algorithmic discrimination.   \n26\nAlgorithmic \nDiscrimination \nProtections \n']","The purpose of conducting proactive equity assessments in the design phase of automated systems is to review potential input data, associated historical context, accessibility for people with disabilities, and societal goals to identify potential discrimination and effects on equity resulting from the introduction of the technology.",simple,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 25, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
How can the equitable design of automated systems help prevent algorithmic discrimination?,"["" \n \n  \n \n \n \n \n \nAlgorithmic \nDiscrimination \nProtections \nWHY THIS PRINCIPLE IS IMPORTANT\nThis section provides a brief summary of the problems which the principle seeks to address and protect \nagainst, including illustrative examples. \nThere is extensive evidence showing that automated systems can produce inequitable outcomes and amplify \nexisting inequity.30 Data that fails to account for existing systemic biases in American society can result in a range of \nconsequences. For example, facial recognition technology that can contribute to wrongful and discriminatory \narrests,31 hiring algorithms that inform discriminatory decisions, and healthcare algorithms that discount \nthe severity of certain diseases in Black Americans. Instances of discriminatory practices built into and \nresulting from AI and other automated systems exist across many industries, areas, and contexts. While automated \nsystems have the capacity to drive extraordinary advances and innovations, algorithmic discrimination \nprotections should be built into their design, deployment, and ongoing use. \nMany companies, non-profits, and federal government agencies are already taking steps to ensure the public \nis protected from algorithmic discrimination. Some companies have instituted bias testing as part of their product \nquality assessment and launch procedures, and in some cases this testing has led products to be changed or not \nlaunched, preventing harm to the public. Federal government agencies have been developing standards and guidance \nfor the use of automated systems in order to help prevent bias. Non-profits and companies have developed best \npractices for audits and impact assessments to help identify potential algorithmic discrimination and provide \ntransparency to the public in the mitigation of such biases. \nBut there is much more work to do to protect the public from algorithmic discrimination to use and design \nautomated systems in an equitable way. The guardrails protecting the public from discrimination in their daily \nlives should include their digital lives and impacts—basic safeguards against abuse, bias, and discrimination to \nensure that all people are treated fairly when automated systems are used. This includes all dimensions of their \nlives, from hiring to loan approvals, from medical treatment and payment to encounters with the criminal \njustice system. Ensuring equity should also go beyond existing guardrails to consider the holistic impact that \nautomated systems make on underserved communities and to institute proactive protections that support these \ncommunities. \n•\nAn automated system using nontraditional factors such as educational attainment and employment history as\npart of its loan underwriting and pricing model was found to be much more likely to charge an applicant who\nattended a Historically Black College or University (HBCU) higher loan prices for refinancing a student loan\nthan an applicant who did not attend an HBCU. This was found to be true even when controlling for\nother credit-related factors.32\n•\nA hiring tool that learned the features of a company's employees (predominantly men) rejected women appli\xad\ncants for spurious and discriminatory reasons; resumes with the word “women’s,” such as “women’s\nchess club captain,” were penalized in the candidate ranking.33\n•\nA predictive model marketed as being able to predict whether students are likely to drop out of school was\nused by more than 500 universities across the country. The model was found to use race directly as a predictor,\nand also shown to have large disparities by race; Black students were as many as four times as likely as their\notherwise similar white peers to be deemed at high risk of dropping out. These risk scores are used by advisors \nto guide students towards or away from majors, and some worry that they are being used to guide\nBlack students away from math and science subjects.34\n•\nA risk assessment tool designed to predict the risk of recidivism for individuals in federal custody showed\nevidence of disparity in prediction. The tool overpredicts the risk of recidivism for some groups of color on the\ngeneral recidivism tools, and underpredicts the risk of recidivism for some groups of color on some of the\nviolent recidivism tools. The Department of Justice is working to reduce these disparities and has\npublicly released a report detailing its review of the tool.35 \n24\n""]","Equitable design of automated systems can help prevent algorithmic discrimination by incorporating protections against abuse, bias, and discrimination, ensuring that all people are treated fairly when automated systems are used. This includes considering the holistic impact on underserved communities and instituting proactive protections that support these communities.",simple,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 23, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
What is the role of relevant AI Actors in the GAI system risk identification process?,"[' \n19 \nGV-4.1-003 \nEstablish policies, procedures, and processes for oversight functions (e.g., senior \nleadership, legal, compliance, including internal evaluation) across the GAI \nlifecycle, from problem formulation and supply chains to system decommission. \nValue Chain and Component \nIntegration \nAI Actor Tasks: AI Deployment, AI Design, AI Development, Operation and Monitoring \n \nGOVERN 4.2: Organizational teams document the risks and potential impacts of the AI technology they design, develop, deploy, \nevaluate, and use, and they communicate about the impacts more broadly. \nAction ID \nSuggested Action \nGAI Risks \nGV-4.2-001 \nEstablish terms of use and terms of service for GAI systems. \nIntellectual Property; Dangerous, \nViolent, or Hateful Content; \nObscene, Degrading, and/or \nAbusive Content \nGV-4.2-002 \nInclude relevant AI Actors in the GAI system risk identification process. \nHuman-AI Configuration \nGV-4.2-003 \nVerify that downstream GAI system impacts (such as the use of third-party \nplugins) are included in the impact documentation process. \nValue Chain and Component \nIntegration \nAI Actor Tasks: AI Deployment, AI Design, AI Development, Operation and Monitoring \n \nGOVERN 4.3: Organizational practices are in place to enable AI testing, identification of incidents, and information sharing. \nAction ID \nSuggested Action \nGAI Risks \nGV4.3--001 \nEstablish policies for measuring the effectiveness of employed content \nprovenance methodologies (e.g., cryptography, watermarking, steganography, \netc.) \nInformation Integrity \nGV-4.3-002 \nEstablish organizational practices to identify the minimum set of criteria \nnecessary for GAI system incident reporting such as: System ID (auto-generated \nmost likely), Title, Reporter, System/Source, Data Reported, Date of Incident, \nDescription, Impact(s), Stakeholder(s) Impacted. \nInformation Security \n']",The role of relevant AI Actors in the GAI system risk identification process is to be included in the process.,simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 22, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
What policies and procedures should be established for the continuous monitoring of third-party GAI systems in deployment?,"[' \n22 \nGV-6.2-003 \nEstablish incident response plans for third-party GAI technologies: Align incident \nresponse plans with impacts enumerated in MAP 5.1; Communicate third-party \nGAI incident response plans to all relevant AI Actors; Define ownership of GAI \nincident response functions; Rehearse third-party GAI incident response plans at \na regular cadence; Improve incident response plans based on retrospective \nlearning; Review incident response plans for alignment with relevant breach \nreporting, data protection, data privacy, or other laws. \nData Privacy; Human-AI \nConfiguration; Information \nSecurity; Value Chain and \nComponent Integration; Harmful \nBias and Homogenization \nGV-6.2-004 \nEstablish policies and procedures for continuous monitoring of third-party GAI \nsystems in deployment. \nValue Chain and Component \nIntegration \nGV-6.2-005 \nEstablish policies and procedures that address GAI data redundancy, including \nmodel weights and other system artifacts. \nHarmful Bias and Homogenization \nGV-6.2-006 \nEstablish policies and procedures to test and manage risks related to rollover and \nfallback technologies for GAI systems, acknowledging that rollover and fallback \nmay include manual processing. \nInformation Integrity \nGV-6.2-007 \nReview vendor contracts and avoid arbitrary or capricious termination of critical \nGAI technologies or vendor services and non-standard terms that may amplify or \ndefer liability in unexpected ways and/or contribute to unauthorized data \ncollection by vendors or third-parties (e.g., secondary data use). Consider: Clear \nassignment of liability and responsibility for incidents, GAI system changes over \ntime (e.g., fine-tuning, drift, decay); Request: Notification and disclosure for \nserious incidents arising from third-party data and systems; Service Level \nAgreements (SLAs) in vendor contracts that address incident response, response \ntimes, and availability of critical support. \nHuman-AI Configuration; \nInformation Security; Value Chain \nand Component Integration \nAI Actor Tasks: AI Deployment, Operation and Monitoring, TEVV, Third-party entities \n \nMAP 1.1: Intended purposes, potentially beneficial uses, context specific laws, norms and expectations, and prospective settings in \nwhich the AI system will be deployed are understood and documented. Considerations include: the specific set or types of users \nalong with their expectations; potential positive and negative impacts of system uses to individuals, communities, organizations, \nsociety, and the planet; assumptions and related limitations about AI system purposes, uses, and risks across the development or \nproduct AI lifecycle; and related TEVV and system metrics. \nAction ID \nSuggested Action \nGAI Risks \nMP-1.1-001 \nWhen identifying intended purposes, consider factors such as internal vs. \nexternal use, narrow vs. broad application scope, fine-tuning, and varieties of \ndata sources (e.g., grounding, retrieval-augmented generation). \nData Privacy; Intellectual \nProperty \n']",Policies and procedures for continuous monitoring of third-party GAI systems in deployment should be established.,simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 25, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
How did the installation of a facial recognition system by a local public housing authority impact the community?,"["" \n \n  \n \nDATA PRIVACY \nWHY THIS PRINCIPLE IS IMPORTANT\nThis section provides a brief summary of the problems which the principle seeks to address and protect \nagainst, including illustrative examples. \n•\nAn insurer might collect data from a person's social media presence as part of deciding what life\ninsurance rates they should be offered.64\n•\nA data broker harvested large amounts of personal data and then suffered a breach, exposing hundreds of\nthousands of people to potential identity theft. 65\n•\nA local public housing authority installed a facial recognition system at the entrance to housing complexes to\nassist law enforcement with identifying individuals viewed via camera when police reports are filed, leading\nthe community, both those living in the housing complex and not, to have videos of them sent to the local\npolice department and made available for scanning by its facial recognition software.66\n•\nCompanies use surveillance software to track employee discussions about union activity and use the\nresulting data to surveil individual employees and surreptitiously intervene in discussions.67\n32\n""]","The installation of a facial recognition system by a local public housing authority led the community, both those living in the housing complex and not, to have videos of them sent to the local police department and made available for scanning by its facial recognition software.",simple,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 31, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
How can over-reliance on synthetic data lead to model collapse?,"[' \n9 \nand reduced content diversity). Overly homogenized outputs can themselves be incorrect, or they may \nlead to unreliable decision-making or amplify harmful biases. These phenomena can flow from \nfoundation models to downstream models and systems, with the foundation models acting as \n“bottlenecks,” or single points of failure.  \nOverly homogenized content can contribute to “model collapse.” Model collapse can occur when model \ntraining over-relies on synthetic data, resulting in data points disappearing from the distribution of the \nnew model’s outputs. In addition to threatening the robustness of the model overall, model collapse \ncould lead to homogenized outputs, including by amplifying any homogenization from the model used to \ngenerate the synthetic training data. \nTrustworthy AI Characteristics: Fair with Harmful Bias Managed, Valid and Reliable \n2.7. Human-AI Configuration \nGAI system use can involve varying risks of misconfigurations and poor interactions between a system \nand a human who is interacting with it. Humans bring their unique perspectives, experiences, or domain-\nspecific expertise to interactions with AI systems but may not have detailed knowledge of AI systems and \nhow they work. As a result, human experts may be unnecessarily “averse” to GAI systems, and thus \ndeprive themselves or others of GAI’s beneficial uses.  \nConversely, due to the complexity and increasing reliability of GAI technology, over time, humans may \nover-rely on GAI systems or may unjustifiably perceive GAI content to be of higher quality than that \nproduced by other sources. This phenomenon is an example of automation bias, or excessive deference \nto automated systems. Automation bias can exacerbate other risks of GAI, such as risks of confabulation \nor risks of bias or homogenization. \nThere may also be concerns about emotional entanglement between humans and GAI systems, which \ncould lead to negative psychological impacts. \nTrustworthy AI Characteristics: Accountable and Transparent, Explainable and Interpretable, Fair with \nHarmful Bias Managed, Privacy Enhanced, Safe, Valid and Reliable \n2.8. Information Integrity \nInformation integrity describes the “spectrum of information and associated patterns of its creation, \nexchange, and consumption in society.” High-integrity information can be trusted; “distinguishes fact \nfrom fiction, opinion, and inference; acknowledges uncertainties; and is transparent about its level of \nvetting. This information can be linked to the original source(s) with appropriate evidence. High-integrity \ninformation is also accurate and reliable, can be verified and authenticated, has a clear chain of custody, \nand creates reasonable expectations about when its validity may expire.”11 \n \n \n11 This definition of information integrity is derived from the 2022 White House Roadmap for Researchers on \nPriorities Related to Information Integrity Research and Development.  \n']","Over-reliance on synthetic data can lead to model collapse when model training over-relies on synthetic data, resulting in data points disappearing from the distribution of the new model's outputs. This threatens the robustness of the model overall and could lead to homogenized outputs, including by amplifying any homogenization from the model used to generate the synthetic training data.",simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 12, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
What actions are suggested to address Human-AI configuration risks in evaluations involving human subjects?,"[' \n30 \nMEASURE 2.2: Evaluations involving human subjects meet applicable requirements (including human subject protection) and are \nrepresentative of the relevant population. \nAction ID \nSuggested Action \nGAI Risks \nMS-2.2-001 Assess and manage statistical biases related to GAI content provenance through \ntechniques such as re-sampling, re-weighting, or adversarial training. \nInformation Integrity; Information \nSecurity; Harmful Bias and \nHomogenization \nMS-2.2-002 \nDocument how content provenance data is tracked and how that data interacts \nwith privacy and security. Consider: Anonymizing data to protect the privacy of \nhuman subjects; Leveraging privacy output filters; Removing any personally \nidentifiable information (PII) to prevent potential harm or misuse. \nData Privacy; Human AI \nConfiguration; Information \nIntegrity; Information Security; \nDangerous, Violent, or Hateful \nContent \nMS-2.2-003 Provide human subjects with options to withdraw participation or revoke their \nconsent for present or future use of their data in GAI applications.  \nData Privacy; Human-AI \nConfiguration; Information \nIntegrity \nMS-2.2-004 \nUse techniques such as anonymization, differential privacy or other privacy-\nenhancing technologies to minimize the risks associated with linking AI-generated \ncontent back to individual human subjects. \nData Privacy; Human-AI \nConfiguration \nAI Actor Tasks: AI Development, Human Factors, TEVV \n \nMEASURE 2.3: AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for \nconditions similar to deployment setting(s). Measures are documented. \nAction ID \nSuggested Action \nGAI Risks \nMS-2.3-001 Consider baseline model performance on suites of benchmarks when selecting a \nmodel for fine tuning or enhancement with retrieval-augmented generation. \nInformation Security; \nConfabulation \nMS-2.3-002 Evaluate claims of model capabilities using empirically validated methods. \nConfabulation; Information \nSecurity \nMS-2.3-003 Share results of pre-deployment testing with relevant GAI Actors, such as those \nwith system release approval authority. \nHuman-AI Configuration \n']","The suggested actions to address Human-AI configuration risks in evaluations involving human subjects include: 1) Documenting how content provenance data is tracked and how that data interacts with privacy and security, considering anonymizing data to protect the privacy of human subjects, leveraging privacy output filters, and removing any personally identifiable information (PII) to prevent potential harm or misuse. 2) Providing human subjects with options to withdraw participation or revoke their consent for present or future use of their data in GAI applications. 3) Using techniques such as anonymization, differential privacy, or other privacy-enhancing technologies to minimize the risks associated with linking AI-generated content back to individual human subjects.",simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 33, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
What protections should be in place to safeguard against abusive data practices?,"['You should be protected from abusive data practices via built-in \nprotections and you should have agency over how data about \nyou is used. You should be protected from violations of privacy through \ndesign choices that ensure such protections are included by default, including \nensuring that data collection conforms to reasonable expectations and that \nonly data strictly necessary for the specific context is collected. Designers, de\xad\nvelopers, and deployers of automated systems should seek your permission \nand respect your decisions regarding collection, use, access, transfer, and de\xad\nletion of your data in appropriate ways and to the greatest extent possible; \nwhere not possible, alternative privacy by design safeguards should be used. \nSystems should not employ user experience and design decisions that obfus\xad\ncate user choice or burden users with defaults that are privacy invasive. Con\xad\nsent should only be used to justify collection of data in cases where it can be \nappropriately and meaningfully given. Any consent requests should be brief, \nbe understandable in plain language, and give you agency over data collection \nand the specific context of use; current hard-to-understand no\xad\ntice-and-choice practices for broad uses of data should be changed. Enhanced \nprotections and restrictions for data and inferences related to sensitive do\xad\nmains, including health, work, education, criminal justice, and finance, and \nfor data pertaining to youth should put you first. In sensitive domains, your \ndata and related inferences should only be used for necessary functions, and \nyou should be protected by ethical review and use prohibitions. You and your \ncommunities should be free from unchecked surveillance; surveillance tech\xad\nnologies should be subject to heightened oversight that includes at least \npre-deployment assessment of their potential harms and scope limits to pro\xad\ntect privacy and civil liberties. Continuous surveillance and monitoring \nshould not be used in education, work, housing, or in other contexts where the \nuse of such surveillance technologies is likely to limit rights, opportunities, or \naccess. Whenever possible, you should have access to reporting that confirms \nyour data decisions have been respected and provides an assessment of the \npotential impact of surveillance technologies on your rights, opportunities, or \naccess. \nDATA PRIVACY\n30\n']","Protections against abusive data practices should include built-in protections, ensuring data collection conforms to reasonable expectations, collecting only necessary data, seeking user permission, respecting user decisions regarding data, using alternative privacy safeguards when necessary, avoiding privacy-invasive defaults, and providing brief and understandable consent requests. Enhanced protections and restrictions should be in place for sensitive domains, and surveillance technologies should be subject to heightened oversight.",simple,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 29, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
What contributions did the Generative AI Public Working Group make to the NIST report?,"[' \n \n \nAbout AI at NIST: The National Institute of Standards and Technology (NIST) develops measurements, \ntechnology, tools, and standards to advance reliable, safe, transparent, explainable, privacy-enhanced, \nand fair artificial intelligence (AI) so that its full commercial and societal benefits can be realized without \nharm to people or the planet. NIST, which has conducted both fundamental and applied work on AI for \nmore than a decade, is also helping to fulfill the 2023 Executive Order on Safe, Secure, and Trustworthy \nAI. NIST established the U.S. AI Safety Institute and the companion AI Safety Institute Consortium to \ncontinue the efforts set in motion by the E.O. to build the science necessary for safe, secure, and \ntrustworthy development and use of AI. \nAcknowledgments: This report was accomplished with the many helpful comments and contributions \nfrom the community, including the NIST Generative AI Public Working Group, and NIST staff and guest \nresearchers: Chloe Autio, Jesse Dunietz, Patrick Hall, Shomik Jain, Kamie Roberts, Reva Schwartz, Martin \nStanley, and Elham Tabassi. \nNIST Technical Series Policies \nCopyright, Use, and Licensing Statements \nNIST Technical Series Publication Identifier Syntax \nPublication History \nApproved by the NIST Editorial Review Board on 07-25-2024 \nContact Information \n[email protected] \nNational Institute of Standards and Technology \nAttn: NIST AI Innovation Lab, Information Technology Laboratory \n100 Bureau Drive (Mail Stop 8900) Gaithersburg, MD 20899-8900 \nAdditional Information \nAdditional information about this publication and other NIST AI publications are available at \nhttps://airc.nist.gov/Home. \n \nDisclaimer: Certain commercial entities, equipment, or materials may be identified in this document in \norder to adequately describe an experimental procedure or concept. Such identification is not intended to \nimply recommendation or endorsement by the National Institute of Standards and Technology, nor is it \nintended to imply that the entities, materials, or equipment are necessarily the best available for the \npurpose. Any mention of commercial, non-profit, academic partners, or their products, or references is \nfor information only; it is not intended to imply endorsement or recommendation by any U.S. \nGovernment agency. \n \n']",The Generative AI Public Working Group provided many helpful comments and contributions to the NIST report.,simple,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 2, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
What strategies use AI engagement and real-time audits to find new GAI impacts?,"[' \n28 \nMAP 5.2: Practices and personnel for supporting regular engagement with relevant AI Actors and integrating feedback about \npositive, negative, and unanticipated impacts are in place and documented. \nAction ID \nSuggested Action \nGAI Risks \nMP-5.2-001 \nDetermine context-based measures to identify if new impacts are present due to \nthe GAI system, including regular engagements with downstream AI Actors to \nidentify and quantify new contexts of unanticipated impacts of GAI systems. \nHuman-AI Configuration; Value \nChain and Component Integration \nMP-5.2-002 \nPlan regular engagements with AI Actors responsible for inputs to GAI systems, \nincluding third-party data and algorithms, to review and evaluate unanticipated \nimpacts. \nHuman-AI Configuration; Value \nChain and Component Integration \nAI Actor Tasks: AI Deployment, AI Design, AI Impact Assessment, Affected Individuals and Communities, Domain Experts, End-\nUsers, Human Factors, Operation and Monitoring  \n \nMEASURE 1.1: Approaches and metrics for measurement of AI risks enumerated during the MAP function are selected for \nimplementation starting with the most significant AI risks. The risks or trustworthiness characteristics that will not – or cannot – be \nmeasured are properly documented. \nAction ID \nSuggested Action \nGAI Risks \nMS-1.1-001 Employ methods to trace the origin and modifications of digital content. \nInformation Integrity \nMS-1.1-002 \nIntegrate tools designed to analyze content provenance and detect data \nanomalies, verify the authenticity of digital signatures, and identify patterns \nassociated with misinformation or manipulation. \nInformation Integrity \nMS-1.1-003 \nDisaggregate evaluation metrics by demographic factors to identify any \ndiscrepancies in how content provenance mechanisms work across diverse \npopulations. \nInformation Integrity; Harmful \nBias and Homogenization \nMS-1.1-004 Develop a suite of metrics to evaluate structured public feedback exercises \ninformed by representative AI Actors. \nHuman-AI Configuration; Harmful \nBias and Homogenization; CBRN \nInformation or Capabilities \nMS-1.1-005 \nEvaluate novel methods and technologies for the measurement of GAI-related \nrisks including in content provenance, offensive cyber, and CBRN, while \nmaintaining the models’ ability to produce valid, reliable, and factually accurate \noutputs. \nInformation Integrity; CBRN \nInformation or Capabilities; \nObscene, Degrading, and/or \nAbusive Content \n', ' \n41 \nMG-2.2-006 \nUse feedback from internal and external AI Actors, users, individuals, and \ncommunities, to assess impact of AI-generated content. \nHuman-AI Configuration \nMG-2.2-007 \nUse real-time auditing tools where they can be demonstrated to aid in the \ntracking and validation of the lineage and authenticity of AI-generated data. \nInformation Integrity \nMG-2.2-008 \nUse structured feedback mechanisms to solicit and capture user input about AI-\ngenerated content to detect subtle shifts in quality or alignment with \ncommunity and societal values. \nHuman-AI Configuration; Harmful \nBias and Homogenization \nMG-2.2-009 \nConsider opportunities to responsibly use synthetic data and other privacy \nenhancing techniques in GAI development, where appropriate and applicable, \nmatch the statistical properties of real-world data without disclosing personally \nidentifiable information or contributing to homogenization. \nData Privacy; Intellectual Property; \nInformation Integrity; \nConfabulation; Harmful Bias and \nHomogenization \nAI Actor Tasks: AI Deployment, AI Impact Assessment, Governance and Oversight, Operation and Monitoring \n \nMANAGE 2.3: Procedures are followed to respond to and recover from a previously unknown risk when it is identified. \nAction ID \nSuggested Action \nGAI Risks \nMG-2.3-001 \nDevelop and update GAI system incident response and recovery plans and \nprocedures to address the following: Review and maintenance of policies and \nprocedures to account for newly encountered uses; Review and maintenance of \npolicies and procedures for detection of unanticipated uses; Verify response \nand recovery plans account for the GAI system value chain; Verify response and \nrecovery plans are updated for and include necessary details to communicate \nwith downstream GAI system Actors: Points-of-Contact (POC), Contact \ninformation, notification format. \nValue Chain and Component \nIntegration \nAI Actor Tasks: AI Deployment, Operation and Monitoring \n \nMANAGE 2.4: Mechanisms are in place and applied, and responsibilities are assigned and understood, to supersede, disengage, or \ndeactivate AI systems that demonstrate performance or outcomes inconsistent with intended use. \nAction ID \nSuggested Action \nGAI Risks \nMG-2.4-001 \nEstablish and maintain communication plans to inform AI stakeholders as part of \nthe deactivation or disengagement process of a specific GAI system (including for \nopen-source models) or context of use, including reasons, workarounds, user \naccess removal, alternative processes, contact information, etc. \nHuman-AI Configuration \n']","Plan regular engagements with AI Actors responsible for inputs to GAI systems, including third-party data and algorithms, to review and evaluate unanticipated impacts. Use real-time auditing tools where they can be demonstrated to aid in the tracking and validation of the lineage and authenticity of AI-generated data.",multi_context,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 31, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}, {'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 44, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
"How do NSF programs align with federal AI principles for safety, security, and effectiveness?","[' \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \nSAFE AND EFFECTIVE \nSYSTEMS \nHOW THESE PRINCIPLES CAN MOVE INTO PRACTICE\nReal-life examples of how these principles can become reality, through laws, policies, and practical \ntechnical and sociotechnical approaches to protecting rights, opportunities, and access. \xad\nSome U.S government agencies have developed specific frameworks for ethical use of AI \nsystems. The Department of Energy (DOE) has activated the AI Advancement Council that oversees coordina-\ntion and advises on implementation of the DOE AI Strategy and addresses issues and/or escalations on the \nethical use and development of AI systems.20 The Department of Defense has adopted Artificial Intelligence \nEthical Principles, and tenets for Responsible Artificial Intelligence specifically tailored to its national \nsecurity and defense activities.21 Similarly, the U.S. Intelligence Community (IC) has developed the Principles \nof Artificial Intelligence Ethics for the Intelligence Community to guide personnel on whether and how to \ndevelop and use AI in furtherance of the IC\'s mission, as well as an AI Ethics Framework to help implement \nthese principles.22\nThe National Science Foundation (NSF) funds extensive research to help foster the \ndevelopment of automated systems that adhere to and advance their safety, security and \neffectiveness. Multiple NSF programs support research that directly addresses many of these principles: \nthe National AI Research Institutes23 support research on all aspects of safe, trustworthy, fair, and explainable \nAI algorithms and systems; the Cyber Physical Systems24 program supports research on developing safe \nautonomous and cyber physical systems with AI components; the Secure and Trustworthy Cyberspace25 \nprogram supports research on cybersecurity and privacy enhancing technologies in automated systems; the \nFormal Methods in the Field26 program supports research on rigorous formal verification and analysis of \nautomated systems and machine learning, and the Designing Accountable Software Systems27 program supports \nresearch on rigorous and reproducible methodologies for developing software systems with legal and regulatory \ncompliance in mind. \nSome state legislatures have placed strong transparency and validity requirements on \nthe use of pretrial risk assessments. The use of algorithmic pretrial risk assessments has been a \ncause of concern for civil rights groups.28 Idaho Code Section 19-1910, enacted in 2019,29 requires that any \npretrial risk assessment, before use in the state, first be ""shown to be free of bias against any class of \nindividuals protected from discrimination by state or federal law"", that any locality using a pretrial risk \nassessment must first formally validate the claim of its being free of bias, that ""all documents, records, and \ninformation used to build or validate the risk assessment shall be open to public inspection,"" and that assertions \nof trade secrets cannot be used ""to quash discovery in a criminal matter by a party to a criminal case."" \n22\n']","The National Science Foundation (NSF) funds extensive research to help foster the development of automated systems that adhere to and advance their safety, security, and effectiveness. Multiple NSF programs support research that directly addresses many of these principles: the National AI Research Institutes support research on all aspects of safe, trustworthy, fair, and explainable AI algorithms and systems; the Cyber Physical Systems program supports research on developing safe autonomous and cyber physical systems with AI components; the Secure and Trustworthy Cyberspace program supports research on cybersecurity and privacy enhancing technologies in automated systems; the Formal Methods in the Field program supports research on rigorous formal verification and analysis of automated systems and machine learning, and the Designing Accountable Software Systems program supports research on rigorous and reproducible methodologies for developing software systems with legal and regulatory compliance in mind.",multi_context,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 21, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
How do testing and monitoring ensure automated systems' safety and effectiveness?,"[' \n \n \n \n \n \n \nSAFE AND EFFECTIVE \nSYSTEMS \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nIn order to ensure that an automated system is safe and effective, it should include safeguards to protect the \npublic from harm in a proactive and ongoing manner; avoid use of data inappropriate for or irrelevant to the task \nat hand, including reuse that could cause compounded harm; and demonstrate the safety and effectiveness of \nthe system. These expectations are explained below. \nProtect the public from harm in a proactive and ongoing manner \nConsultation. The public should be consulted in the design, implementation, deployment, acquisition, and \nmaintenance phases of automated system development, with emphasis on early-stage consultation before a \nsystem is introduced or a large change implemented. This consultation should directly engage diverse impact\xad\ned communities to consider concerns and risks that may be unique to those communities, or disproportionate\xad\nly prevalent or severe for them. The extent of this engagement and the form of outreach to relevant stakehold\xad\ners may differ depending on the specific automated system and development phase, but should include \nsubject matter, sector-specific, and context-specific experts as well as experts on potential impacts such as \ncivil rights, civil liberties, and privacy experts. For private sector applications, consultations before product \nlaunch may need to be confidential. Government applications, particularly law enforcement applications or \napplications that raise national security considerations, may require confidential or limited engagement based \non system sensitivities and preexisting oversight laws and structures. Concerns raised in this consultation \nshould be documented, and the automated system developers were proposing to create, use, or deploy should \nbe reconsidered based on this feedback. \nTesting. Systems should undergo extensive testing before deployment. This testing should follow \ndomain-specific best practices, when available, for ensuring the technology will work in its real-world \ncontext. Such testing should take into account both the specific technology used and the roles of any human \noperators or reviewers who impact system outcomes or effectiveness; testing should include both automated \nsystems testing and human-led (manual) testing. Testing conditions should mirror as closely as possible the \nconditions in which the system will be deployed, and new testing may be required for each deployment to \naccount for material differences in conditions from one deployment to another. Following testing, system \nperformance should be compared with the in-place, potentially human-driven, status quo procedures, with \nexisting human performance considered as a performance baseline for the algorithm to meet pre-deployment, \nand as a lifecycle minimum performance standard. Decision possibilities resulting from performance testing \nshould include the possibility of not deploying the system. \nRisk identification and mitigation. Before deployment, and in a proactive and ongoing manner, poten\xad\ntial risks of the automated system should be identified and mitigated. Identified risks should focus on the \npotential for meaningful impact on people’s rights, opportunities, or access and include those to impacted \ncommunities that may not be direct users of the automated system, risks resulting from purposeful misuse of \nthe system, and other concerns identified via the consultation process. Assessment and, where possible, mea\xad\nsurement of the impact of risks should be included and balanced such that high impact risks receive attention \nand mitigation proportionate with those impacts. Automated systems with the intended purpose of violating \nthe safety of others should not be developed or used; systems with such safety violations as identified unin\xad\ntended consequences should not be used until the risk can be mitigated. Ongoing risk mitigation may necessi\xad\ntate rollback or significant modification to a launched automated system. \n18\n', ' \n \n \n \n \n \n \n \n \n \n \n \nSAFE AND EFFECTIVE \nSYSTEMS \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nOngoing monitoring. Automated systems should have ongoing monitoring procedures, including recalibra\xad\ntion procedures, in place to ensure that their performance does not fall below an acceptable level over time, \nbased on changing real-world conditions or deployment contexts, post-deployment modification, or unexpect\xad\ned conditions. This ongoing monitoring should include continuous evaluation of performance metrics and \nharm assessments, updates of any systems, and retraining of any machine learning models as necessary, as well \nas ensuring that fallback mechanisms are in place to allow reversion to a previously working system. Monitor\xad\ning should take into account the performance of both technical system components (the algorithm as well as \nany hardware components, data inputs, etc.) and human operators. It should include mechanisms for testing \nthe actual accuracy of any predictions or recommendations generated by a system, not just a human operator’s \ndetermination of their accuracy. Ongoing monitoring procedures should include manual, human-led monitor\xad\ning as a check in the event there are shortcomings in automated monitoring systems. These monitoring proce\xad\ndures should be in place for the lifespan of the deployed automated system. \nClear organizational oversight. Entities responsible for the development or use of automated systems \nshould lay out clear governance structures and procedures.  This includes clearly-stated governance proce\xad\ndures before deploying the system, as well as responsibility of specific individuals or entities to oversee ongoing \nassessment and mitigation. Organizational stakeholders including those with oversight of the business process \nor operation being automated, as well as other organizational divisions that may be affected due to the use of \nthe system, should be involved in establishing governance procedures. Responsibility should rest high enough \nin the organization that decisions about resources, mitigation, incident response, and potential rollback can be \nmade promptly, with sufficient weight given to risk mitigation objectives against competing concerns. Those \nholding this responsibility should be made aware of any use cases with the potential for meaningful impact on \npeople’s rights, opportunities, or access as determined based on risk identification procedures.  In some cases, \nit may be appropriate for an independent ethics review to be conducted before deployment. \nAvoid inappropriate, low-quality, or irrelevant data use and the compounded harm of its \nreuse \nRelevant and high-quality data. Data used as part of any automated system’s creation, evaluation, or \ndeployment should be relevant, of high quality, and tailored to the task at hand. Relevancy should be \nestablished based on research-backed demonstration of the causal influence of the data to the specific use case \nor justified more generally based on a reasonable expectation of usefulness in the domain and/or for the \nsystem design or ongoing development. Relevance of data should not be established solely by appealing to \nits historical connection to the outcome. High quality and tailored data should be representative of the task at \nhand and errors from data entry or other sources should be measured and limited. Any data used as the target \nof a prediction process should receive particular attention to the quality and validity of the predicted outcome \nor label to ensure the goal of the automated system is appropriately identified and measured. Additionally, \njustification should be documented for each data attribute and source to explain why it is appropriate to use \nthat data to inform the results of the automated system and why such use will not violate any applicable laws. \nIn cases of high-dimensional and/or derived attributes, such justifications can be provided as overall \ndescriptions of the attribute generation process and appropriateness. \n19\n']","Testing ensures automated systems' safety and effectiveness by following domain-specific best practices to ensure the technology works in its real-world context. It includes both automated and human-led testing, mirroring deployment conditions, and comparing system performance with existing human-driven procedures. Monitoring involves ongoing procedures to ensure performance does not fall below acceptable levels over time, including continuous evaluation of performance metrics, harm assessments, system updates, and retraining of machine learning models as necessary.",multi_context,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 17, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}, {'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 18, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
"What AI inventories does the NAIIO offer, and how do they aid transparency and ethics per NIST?","[' \nENDNOTES\n12. Expectations about reporting are intended for the entity developing or using the automated system. The\nresulting reports can be provided to the public, regulators, auditors, industry standards groups, or others\nengaged in independent review, and should be made public as much as possible consistent with law,\nregulation, and policy, and noting that intellectual property or law enforcement considerations may prevent\npublic release. These reporting expectations are important for transparency, so the American people can\nhave confidence that their rights, opportunities, and access as well as their expectations around\ntechnologies are respected.\n13. National Artificial Intelligence Initiative Office. Agency Inventories of AI Use Cases. Accessed Sept. 8,\n2022. https://www.ai.gov/ai-use-case-inventories/\n14. National Highway Traffic Safety Administration. https://www.nhtsa.gov/\n15. See, e.g., Charles Pruitt. People Doing What They Do Best: The Professional Engineers and NHTSA. Public\nAdministration Review. Vol. 39, No. 4. Jul.-Aug., 1979. https://www.jstor.org/stable/976213?seq=1\n16. The US Department of Transportation has publicly described the health and other benefits of these\n“traffic calming” measures. See, e.g.: U.S. Department of Transportation. Traffic Calming to Slow Vehicle\nSpeeds. Accessed Apr. 17, 2022. https://www.transportation.gov/mission/health/Traffic-Calming-to-Slow\xad\nVehicle-Speeds\n17. Karen Hao. Worried about your firm’s AI ethics? These startups are here to help.\nA growing ecosystem of “responsible AI” ventures promise to help organizations monitor and fix their AI\nmodels. MIT Technology Review. Jan 15., 2021.\nhttps://www.technologyreview.com/2021/01/15/1016183/ai-ethics-startups/; Disha Sinha. Top Progressive\nCompanies Building Ethical AI to Look Out for in 2021. Analytics Insight. June 30, 2021. https://\nwww.analyticsinsight.net/top-progressive-companies-building-ethical-ai-to-look-out-for\xad\nin-2021/ https://www.technologyreview.com/2021/01/15/1016183/ai-ethics-startups/; Disha Sinha. Top\nProgressive Companies Building Ethical AI to Look Out for in 2021. Analytics Insight. June 30, 2021.\n18. Office of Management and Budget. Study to Identify Methods to Assess Equity: Report to the President.\nAug. 2021. https://www.whitehouse.gov/wp-content/uploads/2021/08/OMB-Report-on-E013985\xad\nImplementation_508-Compliant-Secure-v1.1.pdf\n19. National Institute of Standards and Technology. AI Risk Management Framework. Accessed May 23,\n2022. https://www.nist.gov/itl/ai-risk-management-framework\n20. U.S. Department of Energy. U.S. Department of Energy Establishes Artificial Intelligence Advancement\nCouncil. U.S. Department of Energy Artificial Intelligence and Technology Office. April 18, 2022. https://\nwww.energy.gov/ai/articles/us-department-energy-establishes-artificial-intelligence-advancement-council\n21. Department of Defense. U.S Department of Defense Responsible Artificial Intelligence Strategy and\nImplementation Pathway. Jun. 2022. https://media.defense.gov/2022/Jun/22/2003022604/-1/-1/0/\nDepartment-of-Defense-Responsible-Artificial-Intelligence-Strategy-and-Implementation\xad\nPathway.PDF\n22. Director of National Intelligence. Principles of Artificial Intelligence Ethics for the Intelligence\nCommunity. https://www.dni.gov/index.php/features/2763-principles-of-artificial-intelligence-ethics-for\xad\nthe-intelligence-community\n64\n', ' \n \n \nAbout AI at NIST: The National Institute of Standards and Technology (NIST) develops measurements, \ntechnology, tools, and standards to advance reliable, safe, transparent, explainable, privacy-enhanced, \nand fair artificial intelligence (AI) so that its full commercial and societal benefits can be realized without \nharm to people or the planet. NIST, which has conducted both fundamental and applied work on AI for \nmore than a decade, is also helping to fulfill the 2023 Executive Order on Safe, Secure, and Trustworthy \nAI. NIST established the U.S. AI Safety Institute and the companion AI Safety Institute Consortium to \ncontinue the efforts set in motion by the E.O. to build the science necessary for safe, secure, and \ntrustworthy development and use of AI. \nAcknowledgments: This report was accomplished with the many helpful comments and contributions \nfrom the community, including the NIST Generative AI Public Working Group, and NIST staff and guest \nresearchers: Chloe Autio, Jesse Dunietz, Patrick Hall, Shomik Jain, Kamie Roberts, Reva Schwartz, Martin \nStanley, and Elham Tabassi. \nNIST Technical Series Policies \nCopyright, Use, and Licensing Statements \nNIST Technical Series Publication Identifier Syntax \nPublication History \nApproved by the NIST Editorial Review Board on 07-25-2024 \nContact Information \n[email protected] \nNational Institute of Standards and Technology \nAttn: NIST AI Innovation Lab, Information Technology Laboratory \n100 Bureau Drive (Mail Stop 8900) Gaithersburg, MD 20899-8900 \nAdditional Information \nAdditional information about this publication and other NIST AI publications are available at \nhttps://airc.nist.gov/Home. \n \nDisclaimer: Certain commercial entities, equipment, or materials may be identified in this document in \norder to adequately describe an experimental procedure or concept. Such identification is not intended to \nimply recommendation or endorsement by the National Institute of Standards and Technology, nor is it \nintended to imply that the entities, materials, or equipment are necessarily the best available for the \npurpose. Any mention of commercial, non-profit, academic partners, or their products, or references is \nfor information only; it is not intended to imply endorsement or recommendation by any U.S. \nGovernment agency. \n \n']","The National Artificial Intelligence Initiative Office (NAIIO) offers Agency Inventories of AI Use Cases. These inventories aid transparency and ethics by providing information to the public, regulators, auditors, industry standards groups, or others engaged in independent review. This transparency helps ensure that the American people's rights, opportunities, and access, as well as their expectations around technologies, are respected.",multi_context,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 63, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}, {'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 2, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
How do equity assessments and evaluations help reduce algorithmic bias?,"[' \n \n \n \n \n \n \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nDemonstrate that the system protects against algorithmic discrimination \nIndependent evaluation. As described in the section on Safe and Effective Systems, entities should allow \nindependent evaluation of potential algorithmic discrimination caused by automated systems they use or \noversee. In the case of public sector uses, these independent evaluations should be made public unless law \nenforcement or national security restrictions prevent doing so. Care should be taken to balance individual \nprivacy with evaluation data access needs; in many cases, policy-based and/or technological innovations and \ncontrols allow access to such data without compromising privacy. \nReporting. Entities responsible for the development or use of automated systems should provide \nreporting of an appropriately designed algorithmic impact assessment,50 with clear specification of who \nperforms the assessment, who evaluates the system, and how corrective actions are taken (if necessary) in \nresponse to the assessment. This algorithmic impact assessment should include at least: the results of any \nconsultation, design stage equity assessments (potentially including qualitative analysis), accessibility \ndesigns and testing, disparity testing, document any remaining disparities, and detail any mitigation \nimplementation and assessments. This algorithmic impact assessment should be made public whenever \npossible. Reporting should be provided in a clear and machine-readable manner using plain language to \nallow for more straightforward public accountability. \n28\nAlgorithmic \nDiscrimination \nProtections \n', ' \n \n \n \n \n \n \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nAny automated system should be tested to help ensure it is free from algorithmic discrimination before it can be \nsold or used. Protection against algorithmic discrimination should include designing to ensure equity, broadly \nconstrued.  Some algorithmic discrimination is already prohibited under existing anti-discrimination law. The \nexpectations set out below describe proactive technical and policy steps that can be taken to not only \nreinforce those legal protections but extend beyond them to ensure equity for underserved communities48 \neven in circumstances where a specific legal protection may not be clearly established. These protections \nshould be instituted throughout the design, development, and deployment process and are described below \nroughly in the order in which they would be instituted. \nProtect the public from algorithmic discrimination in a proactive and ongoing manner \nProactive assessment of equity in design. Those responsible for the development, use, or oversight of \nautomated systems should conduct proactive equity assessments in the design phase of the technology \nresearch and development or during its acquisition to review potential input data, associated historical \ncontext, accessibility for people with disabilities, and societal goals to identify potential discrimination and \neffects on equity resulting from the introduction of the technology. The assessed groups should be as inclusive \nas possible of the underserved communities mentioned in the equity definition:  Black, Latino, and Indigenous \nand Native American persons, Asian Americans and Pacific Islanders and other persons of color; members of \nreligious minorities; women, girls, and non-binary people; lesbian, gay, bisexual, transgender, queer, and inter-\nsex (LGBTQI+) persons; older adults; persons with disabilities; persons who live in rural areas; and persons \notherwise adversely affected by persistent poverty or inequality. Assessment could include both qualitative \nand quantitative evaluations of the system. This equity assessment should also be considered a core part of the \ngoals of the consultation conducted as part of the safety and efficacy review. \nRepresentative and robust data. Any data used as part of system development or assessment should be \nrepresentative of local communities based on the planned deployment setting and should be reviewed for bias \nbased on the historical and societal context of the data. Such data should be sufficiently robust to identify and \nhelp to mitigate biases and potential harms. \nGuarding against proxies.  Directly using demographic information in the design, development, or \ndeployment of an automated system (for purposes other than evaluating a system for discrimination or using \na system to counter discrimination) runs a high risk of leading to algorithmic discrimination and should be \navoided. In many cases, attributes that are highly correlated with demographic features, known as proxies, can \ncontribute to algorithmic discrimination. In cases where use of the demographic features themselves would \nlead to illegal algorithmic discrimination, reliance on such proxies in decision-making (such as that facilitated \nby an algorithm) may also be prohibited by law. Proactive testing should be performed to identify proxies by \ntesting for correlation between demographic information and attributes in any data used as part of system \ndesign, development, or use. If a proxy is identified, designers, developers, and deployers should remove the \nproxy; if needed, it may be possible to identify alternative attributes that can be used instead. At a minimum, \norganizations should ensure a proxy feature is not given undue weight and should monitor the system closely \nfor any resulting algorithmic discrimination.   \n26\nAlgorithmic \nDiscrimination \nProtections \n']","Equity assessments and evaluations help reduce algorithmic bias by conducting proactive equity assessments in the design phase of the technology research and development or during its acquisition. This includes reviewing potential input data, associated historical context, accessibility for people with disabilities, and societal goals to identify potential discrimination and effects on equity resulting from the introduction of the technology. The assessed groups should be as inclusive as possible of underserved communities. Assessment could include both qualitative and quantitative evaluations of the system.",multi_context,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 27, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}, {'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 25, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
How do org policies and risk mgmt integrate stakeholder input for trustworthy AI?,"[' \n15 \nGV-1.3-004 Obtain input from stakeholder communities to identify unacceptable use, in \naccordance with activities in the AI RMF Map function. \nCBRN Information or Capabilities; \nObscene, Degrading, and/or \nAbusive Content; Harmful Bias \nand Homogenization; Dangerous, \nViolent, or Hateful Content \nGV-1.3-005 \nMaintain an updated hierarchy of identified and expected GAI risks connected to \ncontexts of GAI model advancement and use, potentially including specialized risk \nlevels for GAI systems that address issues such as model collapse and algorithmic \nmonoculture. \nHarmful Bias and Homogenization \nGV-1.3-006 \nReevaluate organizational risk tolerances to account for unacceptable negative risk \n(such as where significant negative impacts are imminent, severe harms are \nactually occurring, or large-scale risks could occur); and broad GAI negative risks, \nincluding: Immature safety or risk cultures related to AI and GAI design, \ndevelopment and deployment, public information integrity risks, including impacts \non democratic processes, unknown long-term performance characteristics of GAI. \nInformation Integrity; Dangerous, \nViolent, or Hateful Content; CBRN \nInformation or Capabilities \nGV-1.3-007 Devise a plan to halt development or deployment of a GAI system that poses \nunacceptable negative risk. \nCBRN Information and Capability; \nInformation Security; Information \nIntegrity \nAI Actor Tasks: Governance and Oversight \n \nGOVERN 1.4: The risk management process and its outcomes are established through transparent policies, procedures, and other \ncontrols based on organizational risk priorities. \nAction ID \nSuggested Action \nGAI Risks \nGV-1.4-001 \nEstablish policies and mechanisms to prevent GAI systems from generating \nCSAM, NCII or content that violates the law.  \nObscene, Degrading, and/or \nAbusive Content; Harmful Bias \nand Homogenization; \nDangerous, Violent, or Hateful \nContent \nGV-1.4-002 \nEstablish transparent acceptable use policies for GAI that address illegal use or \napplications of GAI. \nCBRN Information or \nCapabilities; Obscene, \nDegrading, and/or Abusive \nContent; Data Privacy; Civil \nRights violations \nAI Actor Tasks: AI Development, AI Deployment, Governance and Oversight \n \n', ' \n14 \nGOVERN 1.2: The characteristics of trustworthy AI are integrated into organizational policies, processes, procedures, and practices. \nAction ID \nSuggested Action \nGAI Risks \nGV-1.2-001 \nEstablish transparency policies and processes for documenting the origin and \nhistory of training data and generated data for GAI applications to advance digital \ncontent transparency, while balancing the proprietary nature of training \napproaches. \nData Privacy; Information \nIntegrity; Intellectual Property \nGV-1.2-002 \nEstablish policies to evaluate risk-relevant capabilities of GAI and robustness of \nsafety measures, both prior to deployment and on an ongoing basis, through \ninternal and external evaluations. \nCBRN Information or Capabilities; \nInformation Security \nAI Actor Tasks: Governance and Oversight \n \nGOVERN 1.3: Processes, procedures, and practices are in place to determine the needed level of risk management activities based \non the organization’s risk tolerance. \nAction ID \nSuggested Action \nGAI Risks \nGV-1.3-001 \nConsider the following factors when updating or defining risk tiers for GAI: Abuses \nand impacts to information integrity; Dependencies between GAI and other IT or \ndata systems; Harm to fundamental rights or public safety; Presentation of \nobscene, objectionable, offensive, discriminatory, invalid or untruthful output; \nPsychological impacts to humans (e.g., anthropomorphization, algorithmic \naversion, emotional entanglement); Possibility for malicious use; Whether the \nsystem introduces significant new security vulnerabilities; Anticipated system \nimpact on some groups compared to others; Unreliable decision making \ncapabilities, validity, adaptability, and variability of GAI system performance over \ntime. \nInformation Integrity; Obscene, \nDegrading, and/or Abusive \nContent; Value Chain and \nComponent Integration; Harmful \nBias and Homogenization; \nDangerous, Violent, or Hateful \nContent; CBRN Information or \nCapabilities \nGV-1.3-002 \nEstablish minimum thresholds for performance or assurance criteria and review as \npart of deployment approval (“go/”no-go”) policies, procedures, and processes, \nwith reviewed processes and approval thresholds reflecting measurement of GAI \ncapabilities and risks. \nCBRN Information or Capabilities; \nConfabulation; Dangerous, \nViolent, or Hateful Content \nGV-1.3-003 \nEstablish a test plan and response policy, before developing highly capable models, \nto periodically evaluate whether the model may misuse CBRN information or \ncapabilities and/or offensive cyber capabilities. \nCBRN Information or Capabilities; \nInformation Security \n']",The context does not provide specific information on how organizational policies and risk management integrate stakeholder input for trustworthy AI.,multi_context,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 18, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}, {'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 17, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
How does the supplier risk framework ensure compliance with provenance standards and address GAI risks and feedback?,"[' \n21 \nGV-6.1-005 \nImplement a use-cased based supplier risk assessment framework to evaluate and \nmonitor third-party entities’ performance and adherence to content provenance \nstandards and technologies to detect anomalies and unauthorized changes; \nservices acquisition and value chain risk management; and legal compliance. \nData Privacy; Information \nIntegrity; Information Security; \nIntellectual Property; Value Chain \nand Component Integration \nGV-6.1-006 Include clauses in contracts which allow an organization to evaluate third-party \nGAI processes and standards.  \nInformation Integrity \nGV-6.1-007 Inventory all third-party entities with access to organizational content and \nestablish approved GAI technology and service provider lists. \nValue Chain and Component \nIntegration \nGV-6.1-008 Maintain records of changes to content made by third parties to promote content \nprovenance, including sources, timestamps, metadata. \nInformation Integrity; Value Chain \nand Component Integration; \nIntellectual Property \nGV-6.1-009 \nUpdate and integrate due diligence processes for GAI acquisition and \nprocurement vendor assessments to include intellectual property, data privacy, \nsecurity, and other risks. For example, update processes to: Address solutions that \nmay rely on embedded GAI technologies; Address ongoing monitoring, \nassessments, and alerting, dynamic risk assessments, and real-time reporting \ntools for monitoring third-party GAI risks; Consider policy adjustments across GAI \nmodeling libraries, tools and APIs, fine-tuned models, and embedded tools; \nAssess GAI vendors, open-source or proprietary GAI tools, or GAI service \nproviders against incident or vulnerability databases. \nData Privacy; Human-AI \nConfiguration; Information \nSecurity; Intellectual Property; \nValue Chain and Component \nIntegration; Harmful Bias and \nHomogenization \nGV-6.1-010 \nUpdate GAI acceptable use policies to address proprietary and open-source GAI \ntechnologies and data, and contractors, consultants, and other third-party \npersonnel. \nIntellectual Property; Value Chain \nand Component Integration \nAI Actor Tasks: Operation and Monitoring, Procurement, Third-party entities \n \nGOVERN 6.2: Contingency processes are in place to handle failures or incidents in third-party data or AI systems deemed to be \nhigh-risk. \nAction ID \nSuggested Action \nGAI Risks \nGV-6.2-001 \nDocument GAI risks associated with system value chain to identify over-reliance \non third-party data and to identify fallbacks. \nValue Chain and Component \nIntegration \nGV-6.2-002 \nDocument incidents involving third-party GAI data and systems, including open-\ndata and open-source software. \nIntellectual Property; Value Chain \nand Component Integration \n', ' \n20 \nGV-4.3-003 \nVerify information sharing and feedback mechanisms among individuals and \norganizations regarding any negative impact from GAI systems. \nInformation Integrity; Data \nPrivacy \nAI Actor Tasks: AI Impact Assessment, Affected Individuals and Communities, Governance and Oversight \n \nGOVERN 5.1: Organizational policies and practices are in place to collect, consider, prioritize, and integrate feedback from those \nexternal to the team that developed or deployed the AI system regarding the potential individual and societal impacts related to AI \nrisks. \nAction ID \nSuggested Action \nGAI Risks \nGV-5.1-001 \nAllocate time and resources for outreach, feedback, and recourse processes in GAI \nsystem development. \nHuman-AI Configuration; Harmful \nBias and Homogenization \nGV-5.1-002 \nDocument interactions with GAI systems to users prior to interactive activities, \nparticularly in contexts involving more significant risks.  \nHuman-AI Configuration; \nConfabulation \nAI Actor Tasks: AI Design, AI Impact Assessment, Affected Individuals and Communities, Governance and Oversight \n \nGOVERN 6.1: Policies and procedures are in place that address AI risks associated with third-party entities, including risks of \ninfringement of a third-party’s intellectual property or other rights. \nAction ID \nSuggested Action \nGAI Risks \nGV-6.1-001 Categorize different types of GAI content with associated third-party rights (e.g., \ncopyright, intellectual property, data privacy). \nData Privacy; Intellectual \nProperty; Value Chain and \nComponent Integration \nGV-6.1-002 Conduct joint educational activities and events in collaboration with third parties \nto promote best practices for managing GAI risks.  \nValue Chain and Component \nIntegration \nGV-6.1-003 \nDevelop and validate approaches for measuring the success of content \nprovenance management efforts with third parties (e.g., incidents detected and \nresponse times). \nInformation Integrity; Value Chain \nand Component Integration \nGV-6.1-004 \nDraft and maintain well-defined contracts and service level agreements (SLAs) \nthat specify content ownership, usage rights, quality standards, security \nrequirements, and content provenance expectations for GAI systems. \nInformation Integrity; Information \nSecurity; Intellectual Property \n']","The supplier risk framework ensures compliance with provenance standards and addresses GAI risks and feedback by implementing a use-case based supplier risk assessment framework to evaluate and monitor third-party entities' performance and adherence to content provenance standards and technologies to detect anomalies and unauthorized changes. It also includes clauses in contracts to allow an organization to evaluate third-party GAI processes and standards, maintains records of changes to content made by third parties, and updates due diligence processes for GAI acquisition and procurement vendor assessments to include intellectual property, data privacy, security, and other risks. Additionally, it allocates time and resources for outreach, feedback, and recourse processes in GAI system development.",multi_context,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 24, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}, {'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 23, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
"How does the AI Bill of Rights ensure fair access to education, housing, credit, and jobs while tackling algorithmic harms?","["" \n \n \nSECTION TITLE\nApplying The Blueprint for an AI Bill of Rights \nWhile many of the concerns addressed in this framework derive from the use of AI, the technical \ncapabilities and specific definitions of such systems change with the speed of innovation, and the potential \nharms of their use occur even with less technologically sophisticated tools. Thus, this framework uses a two-\npart test to determine what systems are in scope. This framework applies to (1) automated systems that (2) \nhave the potential to meaningfully impact the American public’s rights, opportunities, or access to \ncritical resources or services. These rights, opportunities, and access to critical resources of services should \nbe enjoyed equally and be fully protected, regardless of the changing role that automated systems may play in \nour lives. \nThis framework describes protections that should be applied with respect to all automated systems that \nhave the potential to meaningfully impact individuals' or communities' exercise of: \nRIGHTS, OPPORTUNITIES, OR ACCESS\nCivil rights, civil liberties, and privacy, including freedom of speech, voting, and protections from discrimi\xad\nnation, excessive punishment, unlawful surveillance, and violations of privacy and other freedoms in both \npublic and private sector contexts; \nEqual opportunities, including equitable access to education, housing, credit, employment, and other \nprograms; or, \nAccess to critical resources or services, such as healthcare, financial services, safety, social services, \nnon-deceptive information about goods and services, and government benefits. \nA list of examples of automated systems for which these principles should be considered is provided in the \nAppendix. The Technical Companion, which follows, offers supportive guidance for any person or entity that \ncreates, deploys, or oversees automated systems. \nConsidered together, the five principles and associated practices of the Blueprint for an AI Bill of \nRights form an overlapping set of backstops against potential harms. This purposefully overlapping \nframework, when taken as a whole, forms a blueprint to help protect the public from harm. \nThe measures taken to realize the vision set forward in this framework should be proportionate \nwith the extent and nature of the harm, or risk of harm, to people's rights, opportunities, and \naccess. \nRELATIONSHIP TO EXISTING LAW AND POLICY\nThe Blueprint for an AI Bill of Rights is an exercise in envisioning a future where the American public is \nprotected from the potential harms, and can fully enjoy the benefits, of automated systems. It describes princi\xad\nples that can help ensure these protections. Some of these protections are already required by the U.S. Constitu\xad\ntion or implemented under existing U.S. laws. For example, government surveillance, and data search and \nseizure are subject to legal requirements and judicial oversight. There are Constitutional requirements for \nhuman review of criminal investigative matters and statutory requirements for judicial review. Civil rights laws \nprotect the American people against discrimination. \n8\n"", ' \n \n \nABOUT THIS FRAMEWORK\xad\xad\xad\xad\xad\nThe Blueprint for an AI Bill of Rights is a set of five principles and associated practices to help guide the \ndesign, use, and deployment of automated systems to protect the rights of the American public in the age of \nartificial intel-ligence. Developed through extensive consultation with the American public, these principles are \na blueprint for building and deploying automated systems that are aligned with democratic values and protect \ncivil rights, civil liberties, and privacy. The Blueprint for an AI Bill of Rights includes this Foreword, the five \nprinciples, notes on Applying the The Blueprint for an AI Bill of Rights, and a Technical Companion that gives \nconcrete steps that can be taken by many kinds of organizations—from governments at all levels to companies of \nall sizes—to uphold these values. Experts from across the private sector, governments, and international \nconsortia have published principles and frameworks to guide the responsible use of automated systems; this \nframework provides a national values statement and toolkit that is sector-agnostic to inform building these \nprotections into policy, practice, or the technological design process.  Where existing law or policy—such as \nsector-specific privacy laws and oversight requirements—do not already provide guidance, the Blueprint for an \nAI Bill of Rights should be used to inform policy decisions.\nLISTENING TO THE AMERICAN PUBLIC\nThe White House Office of Science and Technology Policy has led a year-long process to seek and distill input \nfrom people across the country—from impacted communities and industry stakeholders to technology develop-\ners and other experts across fields and sectors, as well as policymakers throughout the Federal government—on \nthe issue of algorithmic and data-driven harms and potential remedies. Through panel discussions, public listen-\ning sessions, meetings, a formal request for information, and input to a publicly accessible and widely-publicized \nemail address, people throughout the United States, public servants across Federal agencies, and members of the \ninternational community spoke up about both the promises and potential harms of these technologies, and \nplayed a central role in shaping the Blueprint for an AI Bill of Rights. The core messages gleaned from these \ndiscussions include that AI has transformative potential to improve Americans’ lives, and that preventing the \nharms of these technologies is both necessary and achievable. The Appendix includes a full list of public engage-\nments. \n4\n']","The AI Bill of Rights ensures fair access to education, housing, credit, and jobs by describing protections that should be applied with respect to all automated systems that have the potential to meaningfully impact individuals' or communities' exercise of equal opportunities, including equitable access to education, housing, credit, employment, and other programs.",multi_context,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 7, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}, {'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 3, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
How to ensure automated systems avoid algorithmic bias pre- and post-deployment?,"["" \n \n \n \n \n \n \n \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nEnsuring accessibility during design, development, and deployment. Systems should be \ndesigned, developed, and deployed by organizations in ways that ensure accessibility to people with disabili\xad\nties. This should include consideration of a wide variety of disabilities, adherence to relevant accessibility \nstandards, and user experience research both before and after deployment to identify and address any accessi\xad\nbility barriers to the use or effectiveness of the automated system. \nDisparity assessment. Automated systems should be tested using a broad set of measures to assess wheth\xad\ner the system components, both in pre-deployment testing and in-context deployment, produce disparities. \nThe demographics of the assessed groups should be as inclusive as possible of race, color, ethnicity, sex \n(including pregnancy, childbirth, and related medical conditions, gender identity, intersex status, and sexual \norientation), religion, age, national origin, disability, veteran status, genetic information, or any other classifi\xad\ncation protected by law. The broad set of measures assessed should include demographic performance mea\xad\nsures, overall and subgroup parity assessment, and calibration. Demographic data collected for disparity \nassessment should be separated from data used for the automated system and privacy protections should be \ninstituted; in some cases it may make sense to perform such assessment using a data sample. For every \ninstance where the deployed automated system leads to different treatment or impacts disfavoring the identi\xad\nfied groups, the entity governing, implementing, or using the system should document the disparity and a \njustification for any continued use of the system. \nDisparity mitigation. When a disparity assessment identifies a disparity against an assessed group, it may \nbe appropriate to take steps to mitigate or eliminate the disparity. In some cases, mitigation or elimination of \nthe disparity may be required by law. \nDisparities that have the potential to lead to algorithmic \ndiscrimination, cause meaningful harm, or violate equity49 goals should be mitigated. When designing and \nevaluating an automated system, steps should be taken to evaluate multiple models and select the one that \nhas the least adverse impact, modify data input choices, or otherwise identify a system with fewer \ndisparities. If adequate mitigation of the disparity is not possible, then the use of the automated system \nshould be reconsidered. One of the considerations in whether to use the system should be the validity of any \ntarget measure; unobservable targets may result in the inappropriate use of proxies. Meeting these \nstandards may require instituting mitigation procedures and other protective measures to address \nalgorithmic discrimination, avoid meaningful harm, and achieve equity goals. \nOngoing monitoring and mitigation. Automated systems should be regularly monitored to assess algo\xad\nrithmic discrimination that might arise from unforeseen interactions of the system with inequities not \naccounted for during the pre-deployment testing, changes to the system after deployment, or changes to the \ncontext of use or associated data. Monitoring and disparity assessment should be performed by the entity \ndeploying or using the automated system to examine whether the system has led to algorithmic discrimina\xad\ntion when deployed. This assessment should be performed regularly and whenever a pattern of unusual \nresults is occurring. It can be performed using a variety of approaches, taking into account whether and how \ndemographic information of impacted people is available, for example via testing with a sample of users or via \nqualitative user experience research. Riskier and higher-impact systems should be monitored and assessed \nmore frequently. Outcomes of this assessment should include additional disparity mitigation, if needed, or \nfallback to earlier procedures in the case that equity standards are no longer met and can't be mitigated, and \nprior mechanisms provide better adherence to equity standards. \n27\nAlgorithmic \nDiscrimination \nProtections \n"", ' \n \n \n \n \n \n \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nAny automated system should be tested to help ensure it is free from algorithmic discrimination before it can be \nsold or used. Protection against algorithmic discrimination should include designing to ensure equity, broadly \nconstrued.  Some algorithmic discrimination is already prohibited under existing anti-discrimination law. The \nexpectations set out below describe proactive technical and policy steps that can be taken to not only \nreinforce those legal protections but extend beyond them to ensure equity for underserved communities48 \neven in circumstances where a specific legal protection may not be clearly established. These protections \nshould be instituted throughout the design, development, and deployment process and are described below \nroughly in the order in which they would be instituted. \nProtect the public from algorithmic discrimination in a proactive and ongoing manner \nProactive assessment of equity in design. Those responsible for the development, use, or oversight of \nautomated systems should conduct proactive equity assessments in the design phase of the technology \nresearch and development or during its acquisition to review potential input data, associated historical \ncontext, accessibility for people with disabilities, and societal goals to identify potential discrimination and \neffects on equity resulting from the introduction of the technology. The assessed groups should be as inclusive \nas possible of the underserved communities mentioned in the equity definition:  Black, Latino, and Indigenous \nand Native American persons, Asian Americans and Pacific Islanders and other persons of color; members of \nreligious minorities; women, girls, and non-binary people; lesbian, gay, bisexual, transgender, queer, and inter-\nsex (LGBTQI+) persons; older adults; persons with disabilities; persons who live in rural areas; and persons \notherwise adversely affected by persistent poverty or inequality. Assessment could include both qualitative \nand quantitative evaluations of the system. This equity assessment should also be considered a core part of the \ngoals of the consultation conducted as part of the safety and efficacy review. \nRepresentative and robust data. Any data used as part of system development or assessment should be \nrepresentative of local communities based on the planned deployment setting and should be reviewed for bias \nbased on the historical and societal context of the data. Such data should be sufficiently robust to identify and \nhelp to mitigate biases and potential harms. \nGuarding against proxies.  Directly using demographic information in the design, development, or \ndeployment of an automated system (for purposes other than evaluating a system for discrimination or using \na system to counter discrimination) runs a high risk of leading to algorithmic discrimination and should be \navoided. In many cases, attributes that are highly correlated with demographic features, known as proxies, can \ncontribute to algorithmic discrimination. In cases where use of the demographic features themselves would \nlead to illegal algorithmic discrimination, reliance on such proxies in decision-making (such as that facilitated \nby an algorithm) may also be prohibited by law. Proactive testing should be performed to identify proxies by \ntesting for correlation between demographic information and attributes in any data used as part of system \ndesign, development, or use. If a proxy is identified, designers, developers, and deployers should remove the \nproxy; if needed, it may be possible to identify alternative attributes that can be used instead. At a minimum, \norganizations should ensure a proxy feature is not given undue weight and should monitor the system closely \nfor any resulting algorithmic discrimination.   \n26\nAlgorithmic \nDiscrimination \nProtections \n']","To ensure automated systems avoid algorithmic bias pre- and post-deployment, the following steps should be taken: 1. Conduct proactive equity assessments in the design phase to review potential input data, historical context, accessibility for people with disabilities, and societal goals. 2. Use representative and robust data that is reviewed for bias based on historical and societal context. 3. Avoid using demographic information directly in the design, development, or deployment of the system to prevent algorithmic discrimination. 4. Perform proactive testing to identify and remove proxies that could lead to algorithmic discrimination. 5. Regularly monitor the system to assess algorithmic discrimination that might arise from unforeseen interactions, changes to the system, or changes to the context of use or associated data. 6. Implement disparity assessment and mitigation procedures to address any identified disparities. 7. Document any disparities and provide justification for continued use of the system if disparities are found. 8. Reconsider the use of the system if adequate mitigation of disparities is not possible.",multi_context,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 26, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}, {'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 25, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
How do current TEVV limits and public feedback methods create measurement gaps in GAI systems?,"[' \n49 \nearly lifecycle TEVV approaches are developed and matured for GAI, organizations may use \nrecommended “pre-deployment testing” practices to measure performance, capabilities, limits, risks, \nand impacts. This section describes risk measurement and estimation as part of pre-deployment TEVV, \nand examines the state of play for pre-deployment testing methodologies.  \nLimitations of Current Pre-deployment Test Approaches \nCurrently available pre-deployment TEVV processes used for GAI applications may be inadequate, non-\nsystematically applied, or fail to reflect or mismatched to deployment contexts. For example, the \nanecdotal testing of GAI system capabilities through video games or standardized tests designed for \nhumans (e.g., intelligence tests, professional licensing exams) does not guarantee GAI system validity or \nreliability in those domains. Similarly, jailbreaking or prompt engineering tests may not systematically \nassess validity or reliability risks.  \nMeasurement gaps can arise from mismatches between laboratory and real-world settings. Current \ntesting approaches often remain focused on laboratory conditions or restricted to benchmark test \ndatasets and in silico techniques that may not extrapolate well to—or directly assess GAI impacts in real-\nworld conditions. For example, current measurement gaps for GAI make it difficult to precisely estimate \nits potential ecosystem-level or longitudinal risks and related political, social, and economic impacts. \nGaps between benchmarks and real-world use of GAI systems may likely be exacerbated due to prompt \nsensitivity and broad heterogeneity of contexts of use. \nA.1.5. Structured Public Feedback \nStructured public feedback can be used to evaluate whether GAI systems are performing as intended \nand to calibrate and verify traditional measurement methods. Examples of structured feedback include, \nbut are not limited to: \n• \nParticipatory Engagement Methods: Methods used to solicit feedback from civil society groups, \naffected communities, and users, including focus groups, small user studies, and surveys. \n• \nField Testing: Methods used to determine how people interact with, consume, use, and make \nsense of AI-generated information, and subsequent actions and effects, including UX, usability, \nand other structured, randomized experiments.  \n• \nAI Red-teaming: A structured testing exercise used to probe an AI system to find flaws and \nvulnerabilities such as inaccurate, harmful, or discriminatory outputs, often in a controlled \nenvironment and in collaboration with system developers. \nInformation gathered from structured public feedback can inform design, implementation, deployment \napproval, maintenance, or decommissioning decisions. Results and insights gleaned from these exercises \ncan serve multiple purposes, including improving data quality and preprocessing, bolstering governance \ndecision making, and enhancing system documentation and debugging practices. When implementing \nfeedback activities, organizations should follow human subjects research requirements and best \npractices such as informed consent and subject compensation. \n', ' \n50 \nParticipatory Engagement Methods \nOn an ad hoc or more structured basis, organizations can design and use a variety of channels to engage \nexternal stakeholders in product development or review. Focus groups with select experts can provide \nfeedback on a range of issues. Small user studies can provide feedback from representative groups or \npopulations. Anonymous surveys can be used to poll or gauge reactions to specific features. Participatory \nengagement methods are often less structured than field testing or red teaming, and are more \ncommonly used in early stages of AI or product development.  \nField Testing \nField testing involves structured settings to evaluate risks and impacts and to simulate the conditions \nunder which the GAI system will be deployed. Field style tests can be adapted from a focus on user \npreferences and experiences towards AI risks and impacts – both negative and positive. When carried \nout with large groups of users, these tests can provide estimations of the likelihood of risks and impacts \nin real world interactions. \nOrganizations may also collect feedback on outcomes, harms, and user experience directly from users in \nthe production environment after a model has been released, in accordance with human subject \nstandards such as informed consent and compensation. Organizations should follow applicable human \nsubjects research requirements, and best practices such as informed consent and subject compensation, \nwhen implementing feedback activities. \nAI Red-teaming \nAI red-teaming is an evolving practice that references exercises often conducted in a controlled \nenvironment and in collaboration with AI developers building AI models to identify potential adverse \nbehavior or outcomes of a GAI model or system, how they could occur, and stress test safeguards”. AI \nred-teaming can be performed before or after AI models or systems are made available to the broader \npublic; this section focuses on red-teaming in pre-deployment contexts.  \nThe quality of AI red-teaming outputs is related to the background and expertise of the AI red team \nitself. Demographically and interdisciplinarily diverse AI red teams can be used to identify flaws in the \nvarying contexts where GAI will be used. For best results, AI red teams should demonstrate domain \nexpertise, and awareness of socio-cultural aspects within the deployment context. AI red-teaming results \nshould be given additional analysis before they are incorporated into organizational governance and \ndecision making, policy and procedural updates, and AI risk management efforts. \nVarious types of AI red-teaming may be appropriate, depending on the use case: \n• \nGeneral Public: Performed by general users (not necessarily AI or technical experts) who are \nexpected to use the model or interact with its outputs, and who bring their own lived \nexperiences and perspectives to the task of AI red-teaming. These individuals may have been \nprovided instructions and material to complete tasks which may elicit harmful model behaviors. \nThis type of exercise can be more effective with large groups of AI red-teamers. \n• \nExpert: Performed by specialists with expertise in the domain or specific AI red-teaming context \nof use (e.g., medicine, biotech, cybersecurity).  \n• \nCombination: In scenarios when it is difficult to identify and recruit specialists with sufficient \ndomain and contextual expertise, AI red-teaming exercises may leverage both expert and \n']","Current TEVV processes for GAI applications may be inadequate, non-systematically applied, or mismatched to deployment contexts. Measurement gaps can arise from mismatches between laboratory and real-world settings. Current testing approaches often remain focused on laboratory conditions or restricted to benchmark test datasets and in silico techniques that may not extrapolate well to real-world conditions. Structured public feedback can help evaluate whether GAI systems are performing as intended and inform design, implementation, deployment approval, maintenance, or decommissioning decisions.",multi_context,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 52, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}, {'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 53, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
"How do error models for pre-deployment metrics ensure AI validity and address biases, incorporating user/community feedback?","[' \n38 \nMEASURE 2.13: Effectiveness of the employed TEVV metrics and processes in the MEASURE function are evaluated and \ndocumented. \nAction ID \nSuggested Action \nGAI Risks \nMS-2.13-001 \nCreate measurement error models for pre-deployment metrics to demonstrate \nconstruct validity for each metric (i.e., does the metric effectively operationalize \nthe desired concept): Measure or estimate, and document, biases or statistical \nvariance in applied metrics or structured human feedback processes; Leverage \ndomain expertise when modeling complex societal constructs such as hateful \ncontent. \nConfabulation; Information \nIntegrity; Harmful Bias and \nHomogenization \nAI Actor Tasks: AI Deployment, Operation and Monitoring, TEVV \n \nMEASURE 3.2: Risk tracking approaches are considered for settings where AI risks are difficult to assess using currently available \nmeasurement techniques or where metrics are not yet available. \nAction ID \nSuggested Action \nGAI Risks \nMS-3.2-001 \nEstablish processes for identifying emergent GAI system risks including \nconsulting with external AI Actors. \nHuman-AI Configuration; \nConfabulation  \nAI Actor Tasks: AI Impact Assessment, Domain Experts, Operation and Monitoring, TEVV \n \nMEASURE 3.3: Feedback processes for end users and impacted communities to report problems and appeal system outcomes are \nestablished and integrated into AI system evaluation metrics. \nAction ID \nSuggested Action \nGAI Risks \nMS-3.3-001 \nConduct impact assessments on how AI-generated content might affect \ndifferent social, economic, and cultural groups. \nHarmful Bias and Homogenization \nMS-3.3-002 \nConduct studies to understand how end users perceive and interact with GAI \ncontent and accompanying content provenance within context of use. Assess \nwhether the content aligns with their expectations and how they may act upon \nthe information presented. \nHuman-AI Configuration; \nInformation Integrity \nMS-3.3-003 \nEvaluate potential biases and stereotypes that could emerge from the AI-\ngenerated content using appropriate methodologies including computational \ntesting methods as well as evaluating structured feedback input. \nHarmful Bias and Homogenization \n', "" \n39 \nMS-3.3-004 \nProvide input for training materials about the capabilities and limitations of GAI \nsystems related to digital content transparency for AI Actors, other \nprofessionals, and the public about the societal impacts of AI and the role of \ndiverse and inclusive content generation. \nHuman-AI Configuration; \nInformation Integrity; Harmful Bias \nand Homogenization \nMS-3.3-005 \nRecord and integrate structured feedback about content provenance from \noperators, users, and potentially impacted communities through the use of \nmethods such as user research studies, focus groups, or community forums. \nActively seek feedback on generated content quality and potential biases. \nAssess the general awareness among end users and impacted communities \nabout the availability of these feedback channels. \nHuman-AI Configuration; \nInformation Integrity; Harmful Bias \nand Homogenization \nAI Actor Tasks: AI Deployment, Affected Individuals and Communities, End-Users, Operation and Monitoring, TEVV \n \nMEASURE 4.2: Measurement results regarding AI system trustworthiness in deployment context(s) and across the AI lifecycle are \ninformed by input from domain experts and relevant AI Actors to validate whether the system is performing consistently as \nintended. Results are documented. \nAction ID \nSuggested Action \nGAI Risks \nMS-4.2-001 \nConduct adversarial testing at a regular cadence to map and measure GAI risks, \nincluding tests to address attempts to deceive or manipulate the application of \nprovenance techniques or other misuses. Identify vulnerabilities and \nunderstand potential misuse scenarios and unintended outputs. \nInformation Integrity; Information \nSecurity \nMS-4.2-002 \nEvaluate GAI system performance in real-world scenarios to observe its \nbehavior in practical environments and reveal issues that might not surface in \ncontrolled and optimized testing environments. \nHuman-AI Configuration; \nConfabulation; Information \nSecurity \nMS-4.2-003 \nImplement interpretability and explainability methods to evaluate GAI system \ndecisions and verify alignment with intended purpose. \nInformation Integrity; Harmful Bias \nand Homogenization \nMS-4.2-004 \nMonitor and document instances where human operators or other systems \noverride the GAI's decisions. Evaluate these cases to understand if the overrides \nare linked to issues related to content provenance. \nInformation Integrity \nMS-4.2-005 \nVerify and document the incorporation of results of structured public feedback \nexercises into design, implementation, deployment approval (“go”/“no-go” \ndecisions), monitoring, and decommission decisions. \nHuman-AI Configuration; \nInformation Security \nAI Actor Tasks: AI Deployment, Domain Experts, End-Users, Operation and Monitoring, TEVV \n \n""]","Error models for pre-deployment metrics ensure AI validity by demonstrating construct validity for each metric, measuring or estimating biases or statistical variance in applied metrics or structured human feedback processes, and leveraging domain expertise when modeling complex societal constructs such as hateful content.",multi_context,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 41, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}, {'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 42, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
"What themes did the OSTP's initiative explore, and how did they shape the AI Bill of Rights?","[""APPENDIX\n• OSTP conducted meetings with a variety of stakeholders in the private sector and civil society. Some of these\nmeetings were specifically focused on providing ideas related to the development of the Blueprint for an AI\nBill of Rights while others provided useful general context on the positive use cases, potential harms, and/or\noversight possibilities for these technologies. Participants in these conversations from the private sector and\ncivil society included:\nAdobe \nAmerican Civil Liberties Union \n(ACLU) \nThe Aspen Commission on \nInformation Disorder \nThe Awood Center \nThe Australian Human Rights \nCommission \nBiometrics Institute \nThe Brookings Institute \nBSA | The Software Alliance \nCantellus Group \nCenter for American Progress \nCenter for Democracy and \nTechnology \nCenter on Privacy and Technology \nat Georgetown Law \nChristiana Care \nColor of Change \nCoworker \nData Robot \nData Trust Alliance \nData and Society Research Institute \nDeepmind \nEdSAFE AI Alliance \nElectronic Privacy Information \nCenter (EPIC) \nEncode Justice \nEqual AI \nGoogle \nHitachi's AI Policy Committee \nThe Innocence Project \nInstitute of Electrical and \nElectronics Engineers (IEEE) \nIntuit \nLawyers Committee for Civil Rights \nUnder Law \nLegal Aid Society \nThe Leadership Conference on \nCivil and Human Rights \nMeta \nMicrosoft \nThe MIT AI Policy Forum \nMovement Alliance Project \nThe National Association of \nCriminal Defense Lawyers \nO’Neil Risk Consulting & \nAlgorithmic Auditing \nThe Partnership on AI \nPinterest \nThe Plaintext Group \npymetrics \nSAP \nThe Security Industry Association \nSoftware and Information Industry \nAssociation (SIIA) \nSpecial Competitive Studies Project \nThorn \nUnited for Respect \nUniversity of California at Berkeley \nCitris Policy Lab \nUniversity of California at Berkeley \nLabor Center \nUnfinished/Project Liberty \nUpturn \nUS Chamber of Commerce \nUS Chamber of Commerce \nTechnology Engagement Center \nA.I. Working Group\nVibrent Health\nWarehouse Worker Resource\nCenter\nWaymap\n62\n"", ' \n \n \n \n \nSECTION TITLE\nAPPENDIX\nListening to the American People \nThe White House Office of Science and Technology Policy (OSTP) led a yearlong process to seek and distill \ninput from people across the country – from impacted communities to industry stakeholders to \ntechnology developers to other experts across fields and sectors, as well as policymakers across the Federal \ngovernment – on the issue of algorithmic and data-driven harms and potential remedies. Through panel \ndiscussions, public listening sessions, private meetings, a formal request for information, and input to a \npublicly accessible and widely-publicized email address, people across the United States spoke up about \nboth the promises and potential harms of these technologies, and played a central role in shaping the \nBlueprint for an AI Bill of Rights. \nPanel Discussions to Inform the Blueprint for An AI Bill of Rights \nOSTP co-hosted a series of six panel discussions in collaboration with the Center for American Progress, \nthe Joint Center for Political and Economic Studies, New America, the German Marshall Fund, the Electronic \nPrivacy Information Center, and the Mozilla Foundation. The purpose of these convenings – recordings of \nwhich are publicly available online112 – was to bring together a variety of experts, practitioners, advocates \nand federal government officials to offer insights and analysis on the risks, harms, benefits, and \npolicy opportunities of automated systems. Each panel discussion was organized around a wide-ranging \ntheme, exploring current challenges and concerns and considering what an automated society that \nrespects democratic values should look like. These discussions focused on the topics of consumer \nrights and protections, the criminal justice system, equal opportunities and civil justice, artificial \nintelligence and democratic values, social welfare and development, and the healthcare system. \nSummaries of Panel Discussions: \nPanel 1: Consumer Rights and Protections. This event explored the opportunities and challenges for \nindividual consumers and communities in the context of a growing ecosystem of AI-enabled consumer \nproducts, advanced platforms and services, “Internet of Things” (IoT) devices, and smart city products and \nservices. \nWelcome:\n•\nRashida Richardson, Senior Policy Advisor for Data and Democracy, White House Office of Science and\nTechnology Policy\n•\nKaren Kornbluh, Senior Fellow and Director of the Digital Innovation and Democracy Initiative, German\nMarshall Fund\nModerator: \nDevin E. Willis, Attorney, Division of Privacy and Identity Protection, Bureau of Consumer Protection, Federal \nTrade Commission \nPanelists: \n•\nTamika L. Butler, Principal, Tamika L. Butler Consulting\n•\nJennifer Clark, Professor and Head of City and Regional Planning, Knowlton School of Engineering, Ohio\nState University\n•\nCarl Holshouser, Senior Vice President for Operations and Strategic Initiatives, TechNet\n•\nSurya Mattu, Senior Data Engineer and Investigative Data Journalist, The Markup\n•\nMariah Montgomery, National Campaign Director, Partnership for Working Families\n55\n']","The OSTP's initiative explored themes such as consumer rights and protections, the criminal justice system, equal opportunities and civil justice, artificial intelligence and democratic values, social welfare and development, and the healthcare system. These discussions brought together experts, practitioners, advocates, and federal government officials to offer insights and analysis on the risks, harms, benefits, and policy opportunities of automated systems, shaping the Blueprint for an AI Bill of Rights.",multi_context,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 61, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}, {'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 54, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
How could prompt injection attacks on GAI systems worsen disinformation and harmful content?,"[' \n11 \nvalue chain (e.g., data inputs, processing, GAI training, or deployment environments), conventional \ncybersecurity practices may need to adapt or evolve. \nFor instance, prompt injection involves modifying what input is provided to a GAI system so that it \nbehaves in unintended ways. In direct prompt injections, attackers might craft malicious prompts and \ninput them directly to a GAI system, with a variety of downstream negative consequences to \ninterconnected systems. Indirect prompt injection attacks occur when adversaries remotely (i.e., without \na direct interface) exploit LLM-integrated applications by injecting prompts into data likely to be \nretrieved. Security researchers have already demonstrated how indirect prompt injections can exploit \nvulnerabilities by stealing proprietary data or running malicious code remotely on a machine. Merely \nquerying a closed production model can elicit previously undisclosed information about that model. \nAnother cybersecurity risk to GAI is data poisoning, in which an adversary compromises a training \ndataset used by a model to manipulate its outputs or operation. Malicious tampering with data or parts \nof the model could exacerbate risks associated with GAI system outputs. \nTrustworthy AI Characteristics: Privacy Enhanced, Safe, Secure and Resilient, Valid and Reliable \n2.10. \nIntellectual Property \nIntellectual property risks from GAI systems may arise where the use of copyrighted works is not a fair \nuse under the fair use doctrine. If a GAI system’s training data included copyrighted material, GAI \noutputs displaying instances of training data memorization (see Data Privacy above) could infringe on \ncopyright. \nHow GAI relates to copyright, including the status of generated content that is similar to but does not \nstrictly copy work protected by copyright, is currently being debated in legal fora. Similar discussions are \ntaking place regarding the use or emulation of personal identity, likeness, or voice without permission.  \nTrustworthy AI Characteristics: Accountable and Transparent, Fair with Harmful Bias Managed, Privacy \nEnhanced  \n2.11. \nObscene, Degrading, and/or Abusive Content \nGAI can ease the production of and access to illegal non-consensual intimate imagery (NCII) of adults, \nand/or child sexual abuse material (CSAM). GAI-generated obscene, abusive or degrading content can \ncreate privacy, psychological and emotional, and even physical harms, and in some cases may be illegal.  \nGenerated explicit or obscene AI content may include highly realistic “deepfakes” of real individuals, \nincluding children. The spread of this kind of material can have downstream negative consequences: in \nthe context of CSAM, even if the generated images do not resemble specific individuals, the prevalence \nof such images can divert time and resources from efforts to find real-world victims. Outside of CSAM, \nthe creation and spread of NCII disproportionately impacts women and sexual minorities, and can have \nsubsequent negative consequences including decline in overall mental health, substance abuse, and \neven suicidal thoughts.  \nData used for training GAI models may unintentionally include CSAM and NCII. A recent report noted \nthat several commonly used GAI training datasets were found to contain hundreds of known images of \n', ' \n10 \nGAI systems can ease the unintentional production or dissemination of false, inaccurate, or misleading \ncontent (misinformation) at scale, particularly if the content stems from confabulations.  \nGAI systems can also ease the deliberate production or dissemination of false or misleading information \n(disinformation) at scale, where an actor has the explicit intent to deceive or cause harm to others. Even \nvery subtle changes to text or images can manipulate human and machine perception. \nSimilarly, GAI systems could enable a higher degree of sophistication for malicious actors to produce \ndisinformation that is targeted towards specific demographics. Current and emerging multimodal models \nmake it possible to generate both text-based disinformation and highly realistic “deepfakes” – that is, \nsynthetic audiovisual content and photorealistic images.12 Additional disinformation threats could be \nenabled by future GAI models trained on new data modalities. \nDisinformation and misinformation – both of which may be facilitated by GAI – may erode public trust in \ntrue or valid evidence and information, with downstream effects. For example, a synthetic image of a \nPentagon blast went viral and briefly caused a drop in the stock market. Generative AI models can also \nassist malicious actors in creating compelling imagery and propaganda to support disinformation \ncampaigns, which may not be photorealistic, but could enable these campaigns to gain more reach and \nengagement on social media platforms. Additionally, generative AI models can assist malicious actors in \ncreating fraudulent content intended to impersonate others. \nTrustworthy AI Characteristics: Accountable and Transparent, Safe, Valid and Reliable, Interpretable and \nExplainable \n2.9. Information Security \nInformation security for computer systems and data is a mature field with widely accepted and \nstandardized practices for offensive and defensive cyber capabilities. GAI-based systems present two \nprimary information security risks: GAI could potentially discover or enable new cybersecurity risks by \nlowering the barriers for or easing automated exercise of offensive capabilities; simultaneously, it \nexpands the available attack surface, as GAI itself is vulnerable to attacks like prompt injection or data \npoisoning.  \nOffensive cyber capabilities advanced by GAI systems may augment cybersecurity attacks such as \nhacking, malware, and phishing. Reports have indicated that LLMs are already able to discover some \nvulnerabilities in systems (hardware, software, data) and write code to exploit them. Sophisticated threat \nactors might further these risks by developing GAI-powered security co-pilots for use in several parts of \nthe attack chain, including informing attackers on how to proactively evade threat detection and escalate \nprivileges after gaining system access. \nInformation security for GAI models and systems also includes maintaining availability of the GAI system \nand the integrity and (when applicable) the confidentiality of the GAI code, training data, and model \nweights. To identify and secure potential attack points in AI systems or specific components of the AI \n \n \n12 See also https://doi.org/10.6028/NIST.AI.100-4, to be published. \n']","Prompt injection attacks on GAI systems can worsen disinformation and harmful content by allowing attackers to craft malicious prompts that manipulate the system's behavior in unintended ways. This can lead to the production and dissemination of false, inaccurate, or misleading content (misinformation) at scale, and enable the deliberate creation of disinformation with the intent to deceive or cause harm. Additionally, such attacks can facilitate the generation of highly realistic 'deepfakes' and other synthetic content that can erode public trust in true or valid information.",multi_context,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 14, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}, {'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 13, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
"How does the Privacy Act of 1974 handle data risks, retention, and access?","[' \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \nDATA PRIVACY \nHOW THESE PRINCIPLES CAN MOVE INTO PRACTICE\nReal-life examples of how these principles can become reality, through laws, policies, and practical \ntechnical and sociotechnical approaches to protecting rights, opportunities, and access. \nThe Privacy Act of 1974 requires privacy protections for personal information in federal \nrecords systems, including limits on data retention, and also provides individuals a general \nright to access and correct their data. Among other things, the Privacy Act limits the storage of individual \ninformation in federal systems of records, illustrating the principle of limiting the scope of data retention. Under \nthe Privacy Act, federal agencies may only retain data about an individual that is “relevant and necessary” to \naccomplish an agency’s statutory purpose or to comply with an Executive Order of the President. The law allows \nfor individuals to be able to access any of their individual information stored in a federal system of records, if not \nincluded under one of the systems of records exempted pursuant to the Privacy Act. In these cases, federal agen\xad\ncies must provide a method for an individual to determine if their personal information is stored in a particular \nsystem of records, and must provide procedures for an individual to contest the contents of a record about them. \nFurther, the Privacy Act allows for a cause of action for an individual to seek legal relief if a federal agency does not \ncomply with the Privacy Act’s requirements. Among other things, a court may order a federal agency to amend or \ncorrect an individual’s information in its records or award monetary damages if an inaccurate, irrelevant, untimely, \nor incomplete record results in an adverse determination about an individual’s “qualifications, character, rights, … \nopportunities…, or benefits.” \nNIST’s Privacy Framework provides a comprehensive, detailed and actionable approach for \norganizations to manage privacy risks. The NIST Framework gives organizations ways to identify and \ncommunicate their privacy risks and goals to support ethical decision-making in system, product, and service \ndesign or deployment, as well as the measures they are taking to demonstrate compliance with applicable laws \nor regulations. It has been voluntarily adopted by organizations across many different sectors around the world.78\nA school board’s attempt to surveil public school students—undertaken without \nadequate community input—sparked a state-wide biometrics moratorium.79 Reacting to a plan in \nthe city of Lockport, New York, the state’s legislature banned the use of facial recognition systems and other \n“biometric identifying technology” in schools until July 1, 2022.80 The law additionally requires that a report on \nthe privacy, civil rights, and civil liberties implications of the use of such technologies be issued before \nbiometric identification technologies can be used in New York schools. \nFederal law requires employers, and any consultants they may retain, to report the costs \nof surveilling employees in the context of a labor dispute, providing a transparency \nmechanism to help protect worker organizing. Employers engaging in workplace surveillance ""where \nan object there-of, directly or indirectly, is […] to obtain information concerning the activities of employees or a \nlabor organization in connection with a labor dispute"" must report expenditures relating to this surveillance to \nthe Department of Labor Office of Labor-Management Standards, and consultants who employers retain for \nthese purposes must also file reports regarding their activities.81\nPrivacy choices on smartphones show that when technologies are well designed, privacy \nand data agency can be meaningful and not overwhelming. These choices—such as contextual, timely \nalerts about location tracking—are brief, direct, and use-specific. Many of the expectations listed here for \nprivacy by design and use-specific consent mirror those distributed to developers as best practices when \ndeveloping for smart phone devices,82 such as being transparent about how user data will be used, asking for app \npermissions during their use so that the use-context will be clear to users, and ensuring that the app will still \nwork if users deny (or later revoke) some permissions. \n39\n']","The Privacy Act of 1974 requires privacy protections for personal information in federal records systems, including limits on data retention, and also provides individuals a general right to access and correct their data. The Act limits the storage of individual information in federal systems of records to data that is 'relevant and necessary' to accomplish an agency’s statutory purpose or to comply with an Executive Order of the President. It allows individuals to access their information stored in federal systems, unless exempted, and provides procedures to contest the contents of a record. Additionally, the Act allows individuals to seek legal relief if a federal agency does not comply with its requirements, including amending or correcting records or awarding monetary damages for adverse determinations based on inaccurate, irrelevant, untimely, or incomplete records.",multi_context,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 38, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
How can anonymization and public feedback improve privacy and security in GAI evaluations?,"[' \n30 \nMEASURE 2.2: Evaluations involving human subjects meet applicable requirements (including human subject protection) and are \nrepresentative of the relevant population. \nAction ID \nSuggested Action \nGAI Risks \nMS-2.2-001 Assess and manage statistical biases related to GAI content provenance through \ntechniques such as re-sampling, re-weighting, or adversarial training. \nInformation Integrity; Information \nSecurity; Harmful Bias and \nHomogenization \nMS-2.2-002 \nDocument how content provenance data is tracked and how that data interacts \nwith privacy and security. Consider: Anonymizing data to protect the privacy of \nhuman subjects; Leveraging privacy output filters; Removing any personally \nidentifiable information (PII) to prevent potential harm or misuse. \nData Privacy; Human AI \nConfiguration; Information \nIntegrity; Information Security; \nDangerous, Violent, or Hateful \nContent \nMS-2.2-003 Provide human subjects with options to withdraw participation or revoke their \nconsent for present or future use of their data in GAI applications.  \nData Privacy; Human-AI \nConfiguration; Information \nIntegrity \nMS-2.2-004 \nUse techniques such as anonymization, differential privacy or other privacy-\nenhancing technologies to minimize the risks associated with linking AI-generated \ncontent back to individual human subjects. \nData Privacy; Human-AI \nConfiguration \nAI Actor Tasks: AI Development, Human Factors, TEVV \n \nMEASURE 2.3: AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for \nconditions similar to deployment setting(s). Measures are documented. \nAction ID \nSuggested Action \nGAI Risks \nMS-2.3-001 Consider baseline model performance on suites of benchmarks when selecting a \nmodel for fine tuning or enhancement with retrieval-augmented generation. \nInformation Security; \nConfabulation \nMS-2.3-002 Evaluate claims of model capabilities using empirically validated methods. \nConfabulation; Information \nSecurity \nMS-2.3-003 Share results of pre-deployment testing with relevant GAI Actors, such as those \nwith system release approval authority. \nHuman-AI Configuration \n', "" \n39 \nMS-3.3-004 \nProvide input for training materials about the capabilities and limitations of GAI \nsystems related to digital content transparency for AI Actors, other \nprofessionals, and the public about the societal impacts of AI and the role of \ndiverse and inclusive content generation. \nHuman-AI Configuration; \nInformation Integrity; Harmful Bias \nand Homogenization \nMS-3.3-005 \nRecord and integrate structured feedback about content provenance from \noperators, users, and potentially impacted communities through the use of \nmethods such as user research studies, focus groups, or community forums. \nActively seek feedback on generated content quality and potential biases. \nAssess the general awareness among end users and impacted communities \nabout the availability of these feedback channels. \nHuman-AI Configuration; \nInformation Integrity; Harmful Bias \nand Homogenization \nAI Actor Tasks: AI Deployment, Affected Individuals and Communities, End-Users, Operation and Monitoring, TEVV \n \nMEASURE 4.2: Measurement results regarding AI system trustworthiness in deployment context(s) and across the AI lifecycle are \ninformed by input from domain experts and relevant AI Actors to validate whether the system is performing consistently as \nintended. Results are documented. \nAction ID \nSuggested Action \nGAI Risks \nMS-4.2-001 \nConduct adversarial testing at a regular cadence to map and measure GAI risks, \nincluding tests to address attempts to deceive or manipulate the application of \nprovenance techniques or other misuses. Identify vulnerabilities and \nunderstand potential misuse scenarios and unintended outputs. \nInformation Integrity; Information \nSecurity \nMS-4.2-002 \nEvaluate GAI system performance in real-world scenarios to observe its \nbehavior in practical environments and reveal issues that might not surface in \ncontrolled and optimized testing environments. \nHuman-AI Configuration; \nConfabulation; Information \nSecurity \nMS-4.2-003 \nImplement interpretability and explainability methods to evaluate GAI system \ndecisions and verify alignment with intended purpose. \nInformation Integrity; Harmful Bias \nand Homogenization \nMS-4.2-004 \nMonitor and document instances where human operators or other systems \noverride the GAI's decisions. Evaluate these cases to understand if the overrides \nare linked to issues related to content provenance. \nInformation Integrity \nMS-4.2-005 \nVerify and document the incorporation of results of structured public feedback \nexercises into design, implementation, deployment approval (“go”/“no-go” \ndecisions), monitoring, and decommission decisions. \nHuman-AI Configuration; \nInformation Security \nAI Actor Tasks: AI Deployment, Domain Experts, End-Users, Operation and Monitoring, TEVV \n \n""]","Anonymization can improve privacy and security in GAI evaluations by protecting the privacy of human subjects and minimizing the risks associated with linking AI-generated content back to individual human subjects. Public feedback can be recorded and integrated from operators, users, and potentially impacted communities to assess content quality and potential biases, thereby enhancing information integrity and reducing harmful bias and homogenization.",multi_context,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 33, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}, {'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 42, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
How do safeguards & pre-deployment tests boost system reliability?,"[' \n \n  \n \nSAFE AND EFFECTIVE \nSYSTEMS \nWHY THIS PRINCIPLE IS IMPORTANT\nThis section provides a brief summary of the problems which the principle seeks to address and protect \nagainst, including illustrative examples. \nWhile technologies are being deployed to solve problems across a wide array of issues, our reliance on technology can \nalso lead to its use in situations where it has not yet been proven to work—either at all or within an acceptable range \nof error. In other cases, technologies do not work as intended or as promised, causing substantial and unjustified harm. \nAutomated systems sometimes rely on data from other systems, including historical data, allowing irrelevant informa\xad\ntion from past decisions to infect decision-making in unrelated situations.  In some cases, technologies are purposeful\xad\nly designed to violate the safety of others, such as technologies designed to facilitate stalking; in other cases, intended \nor unintended uses lead to unintended harms. \nMany of the harms resulting from these technologies are preventable, and actions are already being taken to protect \nthe public. Some companies have put in place safeguards that have prevented harm from occurring by ensuring that \nkey development decisions are vetted by an ethics review; others have identified and mitigated harms found through \npre-deployment testing and ongoing monitoring processes. Governments at all levels have existing public consulta\xad\ntion processes that may be applied when considering the use of new automated systems, and existing product develop\xad\nment and testing practices already protect the American public from many potential harms. \nStill, these kinds of practices are deployed too rarely and unevenly. Expanded, proactive protections could build on \nthese existing practices, increase confidence in the use of automated systems, and protect the American public. Inno\xad\nvators deserve clear rules of the road that allow new ideas to flourish, and the American public deserves protections \nfrom unsafe outcomes. All can benefit from assurances that automated systems will be designed, tested, and consis\xad\ntently confirmed to work as intended, and that they will be proactively protected from foreseeable unintended harm\xad\nful outcomes. \n•\nA proprietary model was developed to predict the likelihood of sepsis in hospitalized patients and was imple\xad\nmented at hundreds of hospitals around the country. An independent study showed that the model predictions\nunderperformed relative to the designer’s claims while also causing ‘alert fatigue’ by falsely alerting\nlikelihood of sepsis.6\n•\nOn social media, Black people who quote and criticize racist messages have had their own speech silenced when\na platform’s automated moderation system failed to distinguish this “counter speech” (or other critique\nand journalism) from the original hateful messages to which such speech responded.7\n•\nA device originally developed to help people track and find lost items has been used as a tool by stalkers to track\nvictims’ locations in violation of their privacy and safety. The device manufacturer took steps after release to\nprotect people from unwanted tracking by alerting people on their phones when a device is found to be moving\nwith them over time and also by having the device make an occasional noise, but not all phones are able\nto receive the notification and the devices remain a safety concern due to their misuse.8 \n•\nAn algorithm used to deploy police was found to repeatedly send police to neighborhoods they regularly visit,\neven if those neighborhoods were not the ones with the highest crime rates. These incorrect crime predictions\nwere the result of a feedback loop generated from the reuse of data from previous arrests and algorithm\npredictions.9\n16\n']","Safeguards and pre-deployment tests boost system reliability by ensuring that key development decisions are vetted by an ethics review, identifying and mitigating harms found through pre-deployment testing and ongoing monitoring processes, and protecting the public from many potential harms. These practices increase confidence in the use of automated systems and protect the American public from unsafe outcomes.",reasoning,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 15, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
How do feedback mechanisms tie into AI impact and governance?,"[' \n20 \nGV-4.3-003 \nVerify information sharing and feedback mechanisms among individuals and \norganizations regarding any negative impact from GAI systems. \nInformation Integrity; Data \nPrivacy \nAI Actor Tasks: AI Impact Assessment, Affected Individuals and Communities, Governance and Oversight \n \nGOVERN 5.1: Organizational policies and practices are in place to collect, consider, prioritize, and integrate feedback from those \nexternal to the team that developed or deployed the AI system regarding the potential individual and societal impacts related to AI \nrisks. \nAction ID \nSuggested Action \nGAI Risks \nGV-5.1-001 \nAllocate time and resources for outreach, feedback, and recourse processes in GAI \nsystem development. \nHuman-AI Configuration; Harmful \nBias and Homogenization \nGV-5.1-002 \nDocument interactions with GAI systems to users prior to interactive activities, \nparticularly in contexts involving more significant risks.  \nHuman-AI Configuration; \nConfabulation \nAI Actor Tasks: AI Design, AI Impact Assessment, Affected Individuals and Communities, Governance and Oversight \n \nGOVERN 6.1: Policies and procedures are in place that address AI risks associated with third-party entities, including risks of \ninfringement of a third-party’s intellectual property or other rights. \nAction ID \nSuggested Action \nGAI Risks \nGV-6.1-001 Categorize different types of GAI content with associated third-party rights (e.g., \ncopyright, intellectual property, data privacy). \nData Privacy; Intellectual \nProperty; Value Chain and \nComponent Integration \nGV-6.1-002 Conduct joint educational activities and events in collaboration with third parties \nto promote best practices for managing GAI risks.  \nValue Chain and Component \nIntegration \nGV-6.1-003 \nDevelop and validate approaches for measuring the success of content \nprovenance management efforts with third parties (e.g., incidents detected and \nresponse times). \nInformation Integrity; Value Chain \nand Component Integration \nGV-6.1-004 \nDraft and maintain well-defined contracts and service level agreements (SLAs) \nthat specify content ownership, usage rights, quality standards, security \nrequirements, and content provenance expectations for GAI systems. \nInformation Integrity; Information \nSecurity; Intellectual Property \n']","Feedback mechanisms are tied into AI impact and governance through organizational policies and practices that collect, consider, prioritize, and integrate feedback from those external to the team that developed or deployed the AI system. This feedback addresses the potential individual and societal impacts related to AI risks.",reasoning,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 23, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
How does public reporting curb algorithmic bias?,"[' \n \n \n \n \n \n \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nDemonstrate that the system protects against algorithmic discrimination \nIndependent evaluation. As described in the section on Safe and Effective Systems, entities should allow \nindependent evaluation of potential algorithmic discrimination caused by automated systems they use or \noversee. In the case of public sector uses, these independent evaluations should be made public unless law \nenforcement or national security restrictions prevent doing so. Care should be taken to balance individual \nprivacy with evaluation data access needs; in many cases, policy-based and/or technological innovations and \ncontrols allow access to such data without compromising privacy. \nReporting. Entities responsible for the development or use of automated systems should provide \nreporting of an appropriately designed algorithmic impact assessment,50 with clear specification of who \nperforms the assessment, who evaluates the system, and how corrective actions are taken (if necessary) in \nresponse to the assessment. This algorithmic impact assessment should include at least: the results of any \nconsultation, design stage equity assessments (potentially including qualitative analysis), accessibility \ndesigns and testing, disparity testing, document any remaining disparities, and detail any mitigation \nimplementation and assessments. This algorithmic impact assessment should be made public whenever \npossible. Reporting should be provided in a clear and machine-readable manner using plain language to \nallow for more straightforward public accountability. \n28\nAlgorithmic \nDiscrimination \nProtections \n']","Public reporting curbs algorithmic bias by providing transparency through an algorithmic impact assessment. This assessment includes consultation results, equity assessments, accessibility designs and testing, disparity testing, documentation of remaining disparities, and details of mitigation implementations. Making this information public allows for straightforward public accountability.",reasoning,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 27, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
Which action reviews transparency reports for GAI policy breaches?,"[' \n34 \nMS-2.7-009 Regularly assess and verify that security measures remain effective and have not \nbeen compromised. \nInformation Security \nAI Actor Tasks: AI Deployment, AI Impact Assessment, Domain Experts, Operation and Monitoring, TEVV \n \nMEASURE 2.8: Risks associated with transparency and accountability – as identified in the MAP function – are examined and \ndocumented. \nAction ID \nSuggested Action \nGAI Risks \nMS-2.8-001 \nCompile statistics on actual policy violations, take-down requests, and intellectual \nproperty infringement for organizational GAI systems: Analyze transparency \nreports across demographic groups, languages groups. \nIntellectual Property; Harmful Bias \nand Homogenization \nMS-2.8-002 Document the instructions given to data annotators or AI red-teamers. \nHuman-AI Configuration \nMS-2.8-003 \nUse digital content transparency solutions to enable the documentation of each \ninstance where content is generated, modified, or shared to provide a tamper-\nproof history of the content, promote transparency, and enable traceability. \nRobust version control systems can also be applied to track changes across the AI \nlifecycle over time. \nInformation Integrity \nMS-2.8-004 Verify adequacy of GAI system user instructions through user testing. \nHuman-AI Configuration \nAI Actor Tasks: AI Deployment, AI Impact Assessment, Domain Experts, Operation and Monitoring, TEVV \n \n']","MS-2.8-001 Compile statistics on actual policy violations, take-down requests, and intellectual property infringement for organizational GAI systems: Analyze transparency reports across demographic groups, languages groups.",reasoning,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 37, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
Why involve diverse communities early in system dev?,"[' \n \n \n \n \n \n \nSAFE AND EFFECTIVE \nSYSTEMS \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nIn order to ensure that an automated system is safe and effective, it should include safeguards to protect the \npublic from harm in a proactive and ongoing manner; avoid use of data inappropriate for or irrelevant to the task \nat hand, including reuse that could cause compounded harm; and demonstrate the safety and effectiveness of \nthe system. These expectations are explained below. \nProtect the public from harm in a proactive and ongoing manner \nConsultation. The public should be consulted in the design, implementation, deployment, acquisition, and \nmaintenance phases of automated system development, with emphasis on early-stage consultation before a \nsystem is introduced or a large change implemented. This consultation should directly engage diverse impact\xad\ned communities to consider concerns and risks that may be unique to those communities, or disproportionate\xad\nly prevalent or severe for them. The extent of this engagement and the form of outreach to relevant stakehold\xad\ners may differ depending on the specific automated system and development phase, but should include \nsubject matter, sector-specific, and context-specific experts as well as experts on potential impacts such as \ncivil rights, civil liberties, and privacy experts. For private sector applications, consultations before product \nlaunch may need to be confidential. Government applications, particularly law enforcement applications or \napplications that raise national security considerations, may require confidential or limited engagement based \non system sensitivities and preexisting oversight laws and structures. Concerns raised in this consultation \nshould be documented, and the automated system developers were proposing to create, use, or deploy should \nbe reconsidered based on this feedback. \nTesting. Systems should undergo extensive testing before deployment. This testing should follow \ndomain-specific best practices, when available, for ensuring the technology will work in its real-world \ncontext. Such testing should take into account both the specific technology used and the roles of any human \noperators or reviewers who impact system outcomes or effectiveness; testing should include both automated \nsystems testing and human-led (manual) testing. Testing conditions should mirror as closely as possible the \nconditions in which the system will be deployed, and new testing may be required for each deployment to \naccount for material differences in conditions from one deployment to another. Following testing, system \nperformance should be compared with the in-place, potentially human-driven, status quo procedures, with \nexisting human performance considered as a performance baseline for the algorithm to meet pre-deployment, \nand as a lifecycle minimum performance standard. Decision possibilities resulting from performance testing \nshould include the possibility of not deploying the system. \nRisk identification and mitigation. Before deployment, and in a proactive and ongoing manner, poten\xad\ntial risks of the automated system should be identified and mitigated. Identified risks should focus on the \npotential for meaningful impact on people’s rights, opportunities, or access and include those to impacted \ncommunities that may not be direct users of the automated system, risks resulting from purposeful misuse of \nthe system, and other concerns identified via the consultation process. Assessment and, where possible, mea\xad\nsurement of the impact of risks should be included and balanced such that high impact risks receive attention \nand mitigation proportionate with those impacts. Automated systems with the intended purpose of violating \nthe safety of others should not be developed or used; systems with such safety violations as identified unin\xad\ntended consequences should not be used until the risk can be mitigated. Ongoing risk mitigation may necessi\xad\ntate rollback or significant modification to a launched automated system. \n18\n']","Diverse communities should be involved early in system development to consider concerns and risks that may be unique to those communities, or disproportionately prevalent or severe for them.",reasoning,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 17, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
How to balance synthetic data and environmental impacts to avoid model collapse?,"[' \n37 \nMS-2.11-005 \nAssess the proportion of synthetic to non-synthetic training data and verify \ntraining data is not overly homogenous or GAI-produced to mitigate concerns of \nmodel collapse. \nHarmful Bias and Homogenization \nAI Actor Tasks: AI Deployment, AI Impact Assessment, Affected Individuals and Communities, Domain Experts, End-Users, \nOperation and Monitoring, TEVV \n \nMEASURE 2.12: Environmental impact and sustainability of AI model training and management activities – as identified in the MAP \nfunction – are assessed and documented. \nAction ID \nSuggested Action \nGAI Risks \nMS-2.12-001 Assess safety to physical environments when deploying GAI systems. \nDangerous, Violent, or Hateful \nContent \nMS-2.12-002 Document anticipated environmental impacts of model development, \nmaintenance, and deployment in product design decisions. \nEnvironmental \nMS-2.12-003 \nMeasure or estimate environmental impacts (e.g., energy and water \nconsumption) for training, fine tuning, and deploying models: Verify tradeoffs \nbetween resources used at inference time versus additional resources required \nat training time. \nEnvironmental \nMS-2.12-004 Verify effectiveness of carbon capture or offset programs for GAI training and \napplications, and address green-washing concerns. \nEnvironmental \nAI Actor Tasks: AI Deployment, AI Impact Assessment, Domain Experts, Operation and Monitoring, TEVV \n \n']",The answer to given question is not present in context,reasoning,"[{'source': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'file_path': 'https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf', 'page': 40, 'total_pages': 64, 'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': ""D:20240805141702-04'00'"", 'modDate': ""D:20240805143048-04'00'"", 'trapped': ''}]",True
How to ethically and securely use sensitive data?,"[' \n \n \n \n \n \n \nDATA PRIVACY \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \xad\xad\xad\xad\xad\xad\nIn addition to the privacy expectations above for general non-sensitive data, any system collecting, using, shar-\ning, or storing sensitive data should meet the expectations below. Depending on the technological use case and \nbased on an ethical assessment, consent for sensitive data may need to be acquired from a guardian and/or child. \nProvide enhanced protections for data related to sensitive domains \nNecessary functions only. Sensitive data should only be used for functions strictly necessary for that \ndomain or for functions that are required for administrative reasons (e.g., school attendance records), unless \nconsent is acquired, if appropriate, and the additional expectations in this section are met. Consent for non-\nnecessary functions should be optional, i.e., should not be required, incentivized, or coerced in order to \nreceive opportunities or access to services. In cases where data is provided to an entity (e.g., health insurance \ncompany) in order to facilitate payment for such a need, that data should only be used for that purpose. \nEthical review and use prohibitions. Any use of sensitive data or decision process based in part on sensi-\ntive data that might limit rights, opportunities, or access, whether the decision is automated or not, should go \nthrough a thorough ethical review and monitoring, both in advance and by periodic review (e.g., via an indepen-\ndent ethics committee or similarly robust process). In some cases, this ethical review may determine that data \nshould not be used or shared for specific uses even with consent. Some novel uses of automated systems in this \ncontext, where the algorithm is dynamically developing and where the science behind the use case is not well \nestablished, may also count as human subject experimentation, and require special review under organizational \ncompliance bodies applying medical, scientific, and academic human subject experimentation ethics rules and \ngovernance procedures. \nData quality. In sensitive domains, entities should be especially careful to maintain the quality of data to \navoid adverse consequences arising from decision-making based on flawed or inaccurate data. Such care is \nnecessary in a fragmented, complex data ecosystem and for datasets that have limited access such as for fraud \nprevention and law enforcement. It should be not left solely to individuals to carry the burden of reviewing and \ncorrecting data. Entities should conduct regular, independent audits and take prompt corrective measures to \nmaintain accurate, timely, and complete data. \nLimit access to sensitive data and derived data. Sensitive data and derived data should not be sold, \nshared, or made public as part of data brokerage or other agreements. Sensitive data includes data that can be \nused to infer sensitive information; even systems that are not directly marketed as sensitive domain technologies \nare expected to keep sensitive data private. Access to such data should be limited based on necessity and based \non a principle of local control, such that those individuals closest to the data subject have more access while \nthose who are less proximate do not (e.g., a teacher has access to their students’ daily progress data while a \nsuperintendent does not). \nReporting. In addition to the reporting on data privacy (as listed above for non-sensitive data), entities devel-\noping technologies related to a sensitive domain and those collecting, using, storing, or sharing sensitive data \nshould, whenever appropriate, regularly provide public reports describing: any data security lapses or breaches \nthat resulted in sensitive data leaks; the number, type, and outcomes of ethical pre-reviews undertaken; a \ndescription of any data sold, shared, or made public, and how that data was assessed to determine it did not pres-\nent a sensitive data risk; and ongoing risk identification and management procedures, and any mitigation added \nbased on these procedures. Reporting should be provided in a clear and machine-readable manner. \n38\n']","To ethically and securely use sensitive data, the following measures should be taken: 1. Use sensitive data only for necessary functions or with appropriate consent. 2. Conduct thorough ethical reviews and monitoring for any use of sensitive data that might limit rights, opportunities, or access. 3. Maintain high data quality to avoid adverse consequences from flawed or inaccurate data. 4. Limit access to sensitive data and derived data, ensuring it is not sold, shared, or made public. 5. Provide regular public reports on data security lapses, ethical reviews, and risk management procedures.",reasoning,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 37, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
How do reporting expectations boost trust in automated systems?,"[' \nENDNOTES\n12. Expectations about reporting are intended for the entity developing or using the automated system. The\nresulting reports can be provided to the public, regulators, auditors, industry standards groups, or others\nengaged in independent review, and should be made public as much as possible consistent with law,\nregulation, and policy, and noting that intellectual property or law enforcement considerations may prevent\npublic release. These reporting expectations are important for transparency, so the American people can\nhave confidence that their rights, opportunities, and access as well as their expectations around\ntechnologies are respected.\n13. National Artificial Intelligence Initiative Office. Agency Inventories of AI Use Cases. Accessed Sept. 8,\n2022. https://www.ai.gov/ai-use-case-inventories/\n14. National Highway Traffic Safety Administration. https://www.nhtsa.gov/\n15. See, e.g., Charles Pruitt. People Doing What They Do Best: The Professional Engineers and NHTSA. Public\nAdministration Review. Vol. 39, No. 4. Jul.-Aug., 1979. https://www.jstor.org/stable/976213?seq=1\n16. The US Department of Transportation has publicly described the health and other benefits of these\n“traffic calming” measures. See, e.g.: U.S. Department of Transportation. Traffic Calming to Slow Vehicle\nSpeeds. Accessed Apr. 17, 2022. https://www.transportation.gov/mission/health/Traffic-Calming-to-Slow\xad\nVehicle-Speeds\n17. Karen Hao. Worried about your firm’s AI ethics? These startups are here to help.\nA growing ecosystem of “responsible AI” ventures promise to help organizations monitor and fix their AI\nmodels. MIT Technology Review. Jan 15., 2021.\nhttps://www.technologyreview.com/2021/01/15/1016183/ai-ethics-startups/; Disha Sinha. Top Progressive\nCompanies Building Ethical AI to Look Out for in 2021. Analytics Insight. June 30, 2021. https://\nwww.analyticsinsight.net/top-progressive-companies-building-ethical-ai-to-look-out-for\xad\nin-2021/ https://www.technologyreview.com/2021/01/15/1016183/ai-ethics-startups/; Disha Sinha. Top\nProgressive Companies Building Ethical AI to Look Out for in 2021. Analytics Insight. June 30, 2021.\n18. Office of Management and Budget. Study to Identify Methods to Assess Equity: Report to the President.\nAug. 2021. https://www.whitehouse.gov/wp-content/uploads/2021/08/OMB-Report-on-E013985\xad\nImplementation_508-Compliant-Secure-v1.1.pdf\n19. National Institute of Standards and Technology. AI Risk Management Framework. Accessed May 23,\n2022. https://www.nist.gov/itl/ai-risk-management-framework\n20. U.S. Department of Energy. U.S. Department of Energy Establishes Artificial Intelligence Advancement\nCouncil. U.S. Department of Energy Artificial Intelligence and Technology Office. April 18, 2022. https://\nwww.energy.gov/ai/articles/us-department-energy-establishes-artificial-intelligence-advancement-council\n21. Department of Defense. U.S Department of Defense Responsible Artificial Intelligence Strategy and\nImplementation Pathway. Jun. 2022. https://media.defense.gov/2022/Jun/22/2003022604/-1/-1/0/\nDepartment-of-Defense-Responsible-Artificial-Intelligence-Strategy-and-Implementation\xad\nPathway.PDF\n22. Director of National Intelligence. Principles of Artificial Intelligence Ethics for the Intelligence\nCommunity. https://www.dni.gov/index.php/features/2763-principles-of-artificial-intelligence-ethics-for\xad\nthe-intelligence-community\n64\n']","Reporting expectations are important for transparency, so the American people can have confidence that their rights, opportunities, and access as well as their expectations around technologies are respected.",reasoning,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 63, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True
Impact of facial recognition on housing residents and visitors?,"["" \n \n  \n \nDATA PRIVACY \nWHY THIS PRINCIPLE IS IMPORTANT\nThis section provides a brief summary of the problems which the principle seeks to address and protect \nagainst, including illustrative examples. \n•\nAn insurer might collect data from a person's social media presence as part of deciding what life\ninsurance rates they should be offered.64\n•\nA data broker harvested large amounts of personal data and then suffered a breach, exposing hundreds of\nthousands of people to potential identity theft. 65\n•\nA local public housing authority installed a facial recognition system at the entrance to housing complexes to\nassist law enforcement with identifying individuals viewed via camera when police reports are filed, leading\nthe community, both those living in the housing complex and not, to have videos of them sent to the local\npolice department and made available for scanning by its facial recognition software.66\n•\nCompanies use surveillance software to track employee discussions about union activity and use the\nresulting data to surveil individual employees and surreptitiously intervene in discussions.67\n32\n""]","A local public housing authority installed a facial recognition system at the entrance to housing complexes to assist law enforcement with identifying individuals viewed via camera when police reports are filed, leading the community, both those living in the housing complex and not, to have videos of them sent to the local police department and made available for scanning by its facial recognition software.",reasoning,"[{'source': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'file_path': 'https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf', 'page': 31, 'total_pages': 73, 'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': ""D:20220920133035-04'00'"", 'modDate': ""D:20221003104118-04'00'"", 'trapped': ''}]",True