These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.
Type
Origin
Scope
SUBMIT A TOOL
If you have a tool that you think should be featured in the Catalogue of AI Tools & Metrics, we would love to hear from you!
SUBMITAIxploit
TechnicalFranceUploaded on Dec 6, 2024AIxploit is a tool designed to evaluate and enhance the robustness of Large Language Models (LLMs) through adversarial testing. This tool simulates various attack scenarios to identify vulnerabilities and weaknesses in LLMs, ensuring they are more resilient and reliable in real-world applications.
Objective(s)
PyRIT
TechnicalUnited StatesUploaded on Nov 8, 2024The Python Risk Identification Tool for generative AI (PyRIT) is an open access automation framework to empower security professionals and machine learning engineers to proactively find risks in their generative AI systems.
Objective(s)
garak
TechnicalUploaded on Nov 5, 2024garak, Generative AI Red-teaming & Assessment Kit, is an LLM vulnerability scanner. Garak checks if an LLM can be made to fail.
Objective(s)
Responsible innovation toolkit: Harms modeling framework
TechnicalUnited StatesUploaded on Sep 9, 2024Harms Modeling is a practice designed to help you anticipate the potential for harm, identify gaps in product that could put people at risk, and ultimately create approaches that proactively address harm.
Objective(s)
Dioptra
TechnicalUnited StatesUploaded on Sep 9, 2024Dioptra is an open source software test platform for assessing the trustworthy characteristics of artificial intelligence (AI). It helps developers on determining which types of attacks may impact negatively their model's performance.
Objective(s)
BELLS - Benchmarks for the Evaluation of LLM Safeguards
TechnicalFranceUploaded on Aug 2, 2024Evaluate input-output safeguards for LLM systems such as jailbreak and hallucination detectors, to understand how good they are and on which type of inputs they fail.
Objective(s)
Probe: End-to-end AI Security Platform
TechnicalUnited StatesUploaded on Aug 2, 2024AI Security Platform for GenAI and Conversational AI applications. Probe enables security officers and developers identify, mitigate, and monitor AI system security.
Objective(s)
SAE AIR 6988 - Artificial Intelligence in Aeronautical Systems: Statement of Concerns
ProceduralUploaded on Jul 2, 2024This document reviews current aerospace software, hardware, and system development standards used in the certification/approval process of safety-critical airborne and ground-based systems, and assesses whether these standards are compatible with a typical Artificial Intelligence (AI) and Machine Learning (ML) development approach.
Objective(s)
BS PAS 1881:2022 - Assuring the operational safety of automated vehicles. Specification
ProceduralUploaded on Jul 2, 2024First full revision of PAS 1881:2020 to reflect learning from recent automated vehicle trials and input from stakeholders. It deals with how to build operational safety cases for trialling and testing of AVs so that ultimately connected and automated vehicles can be deployed safely, and the public will be confident about their safety.
Objective(s)
BS PAS 11281:2018 - Connected automotive ecosystems. Impact of security on safety. Code of practice
ProceduralUploaded on Jul 2, 2024PAS 11281 is the international standard on road vehicles that gives recommendations for managing security risks that might lead to a compromise of safety in a connected automotive ecosystem.
Objective(s)
VDI-EE 4030 - Consideration of human reliability in the design of autonomous systems
ProceduralUploaded on Jul 2, 2024The scope of the expert recommendation is the design and layout of autonomous systems from the perspective of human reliability.
Objective(s)
BS PAS 1882:2021 - Data collection and management for automated vehicle trials for the purpose of incident investigation. Specification
ProceduralUploaded on Jul 2, 2024PAS 1882:2021 details how information should be handled during automated vehicle trials to ensure it’s collected consistently and improves the safety of UK trials
Objective(s)
ITU-T F.749.13 - Framework and requirements for civilian unmanned aerial vehicle flight control using artificial intelligence
ProceduralUploaded on Jul 2, 2024This Recommendation provides a framework of civilian unmanned aerial vehicle flight control using artificial intelligence (AI), including the flight navigation control of a civilian unmanned aerial vehicle (CUAV) and the specific flight control according to the vertical industry application requirements.
Objective(s)
IEEE 2846-2022 - IEEE Standard for Assumptions in Safety-Related Models for Automated Driving Systems
ProceduralUploaded on Jul 2, 2024Defining a minimum set of reasonable assumptions and foreseeable scenarios that shall be considered in the development of safety related models that are part of an automated driving system (ADS).
Objective(s)
BS PAS 1883:2020 - Operational design domain (ODD) taxonomy for an automated driving system (ADS). Specification
ProceduralUploaded on Jul 2, 2024This document specifies requirements for the collection, curation, storage and sharing of information during automated vehicle trials and advanced trials in the UK in relation to information collected or received by the system. It covers all automated and co-operative automated driving vehicle trials on any land with public access.
Objective(s)
ISO/TR 22100-5:2021 - Safety of machinery — Relationship with ISO 12100 — Part 5: Implications of artificial intelligence machine learning
ProceduralUploaded on Jul 1, 2024This document addresses how artificial intelligence machine learning can impact the safety of machinery and machinery systems.
Objective(s)
BS PAS 1884:2021 - Safety operators in automated vehicle testing and trialling. Guide
ProceduralUploaded on Jul 2, 2024The latest in a series of standards supporting the development of connected and autonomous vehicles (CAVs) in the UK, PAS 1884 on safety operators in automated vehicle testing gives guidance on the responsibilities, competencies, and training of safety drivers engaged in automated vehicle trialling and developmental testing.
Objective(s)
ANSI/UL 4600 - Standard for Safety for the Evaluation of Autonomous Products
ProceduralUploaded on Jul 1, 2024ANSI/UL 4600 Standard for Safety for the Evaluation of Autonomous Products encompasses fully autonomous systems that move such as self-driving cars along with applications in mining, agriculture, maintenance, and other vehicles including lightweight unmanned aerial vehicles (UAVs).
Objective(s)
IEEE 2802-2022 - Standard for the Performance and Safety Evaluation of Artificial Intelligence Based Medical Device: Terminology
ProceduralUploaded on Jul 1, 2024This standard is aimed at establishing concepts and terminology for the performance and safety evaluation of artificial intelligence medical device, which covers basic technology, dataset, quality characteristics, quality evaluation and application scenario.
Objective(s)
