Quantify AI tool risk across five dimensions with 17,410+ classified domains. Build a repeatable, auditable risk assessment program.
Traditional risk frameworks were built for software that stores and displays data. AI tools are a different animal.
Traditional risk frameworks were built for software that stores and displays data — you assess where it sits, who can read it, and how it is encrypted.
AI tools process data through opaque models, retain it for training, and may leak it into outputs served to others.
With 17,410+ AI-tool domains across 18 categories, manual assessment is impossible. New tools launch daily with risk profiles too varied for one-at-a-time evaluation.
Each dimension captures a distinct aspect of risk. The composite score across all five produces the tool's overall rating.
What data does the tool accept? Free text, file uploads, API integrations, or screen captures?
Does the vendor use submitted data for training? Once data enters a training pipeline, it cannot be deleted from model weights.
Where is the vendor incorporated, where does inference occur, and where is data stored?
Published privacy policy? SOC 2 or ISO 27001 certification? Responsible disclosure program? Incident response track record?
Can the tool execute code, access the internet, interact with APIs, or control browser sessions on the user's behalf?
The five dimensions combine into a weighted composite score. Defaults reflect priorities from enterprise CISO research.
These two dimensions most directly determine whether confidential data will be compromised. They get the majority weight.
Cross-border transfer penalties have surged under recent enforcement actions. Regulatory fines can exceed 4% of global revenue.
This Python implementation calculates composite risk scores.
#!/usr/bin/env python3 """AI Tool Risk Scoring Engine — composite scoring across five dimensions.""" import json from dataclasses import dataclass, field from typing import Dict, List @dataclass class RiskProfile: domain: str category: str data_exposure: float # 1-10 training_retention: float # 1-10 jurisdictional: float # 1-10 vendor_maturity: float # 1-10 capability_scope: float # 1-10 DEFAULT_WEIGHTS = { "data_exposure": 0.30, "training_retention": 0.25, "jurisdictional": 0.20, "vendor_maturity": 0.15, "capability_scope": 0.10, } TIER_THRESHOLDS = { "approved": (0, 3.0), # Composite 0–3.0: approved for use "conditional": (3.0, 6.0), # Composite 3.0–6.0: use with controls "prohibited": (6.0, 10.0), # Composite 6.0–10: blocked } def calculate_composite_score( profile: RiskProfile, weights: Dict[str, float] = None ) -> float: """Calculate weighted composite risk score (1–10 scale).""" w = weights or DEFAULT_WEIGHTS score = ( profile.data_exposure * w["data_exposure"] + profile.training_retention * w["training_retention"] + profile.jurisdictional * w["jurisdictional"] + profile.vendor_maturity * w["vendor_maturity"] + profile.capability_scope * w["capability_scope"] ) return round(score, 2) def assign_tier(score: float) -> str: """Map composite score to risk tier.""" for tier, (low, high) in TIER_THRESHOLDS.items(): if low <= score < high: return tier return "prohibited" # Example: assess a generative text AI tool tool = RiskProfile( domain="example-ai-writer.com", category="Text Generation", data_exposure=8.5, training_retention=7.0, jurisdictional=6.0, vendor_maturity=8.0, capability_scope=4.0, ) composite = calculate_composite_score(tool) tier = assign_tier(composite) print(f"Domain: {tool.domain}") print(f"Category: {tool.category}") print(f"Composite: {composite}/10") print(f"Tier: {tier.upper()}") # Output: Composite: 7.10/10 → PROHIBITED
| Tier | Score Range | Decision |
|---|---|---|
| Approved | 0 – 3.0 | Permitted for designated user groups |
| Conditional | 3.0 – 6.0 | Allowed for specific use cases with additional controls |
| Prohibited | 6.0 – 10.0 | Blocked across the entire organization |
A financial institution may set the prohibited threshold at 4.0. A technology company may set it at 7.0.
A code tool scoring 4.5 might be approved for marketing (copy editing) but prohibited for engineering (source code exposure).
AI vendor assessments must go beyond standard questionnaires. Each domain below scores 1–10, aggregating into vendor maturity.
Manual assessment doesn't scale when shadow AI detection reveals hundreds of tools. The API automates initial triage.
#!/usr/bin/env python3 """Batch vendor assessment using the AI Tools Blocklist API.""" import requests import csv from datetime import datetime API_BASE = "https://aitoolsblocklist.com/api/v1" API_KEY = "your-api-key-here" # Category-level risk baselines — derived from analysis # of data handling patterns across 17,410+ AI tools CATEGORY_BASELINES = { "Text Generation": {"data_exp": 8, "training": 7, "cap": 5}, "Code Assistant": {"data_exp": 9, "training": 8, "cap": 7}, "Image Generation": {"data_exp": 6, "training": 8, "cap": 3}, "Voice & Audio": {"data_exp": 7, "training": 6, "cap": 4}, "Data Analysis": {"data_exp": 9, "training": 6, "cap": 6}, "Autonomous Agents": {"data_exp": 9, "training": 7, "cap": 10}, "Translation": {"data_exp": 7, "training": 5, "cap": 2}, "Transcription": {"data_exp": 7, "training": 5, "cap": 2}, } def assess_discovered_tools(domains: list) -> list: """Look up each domain in the AI tools database and generate preliminary risk assessments.""" results = [] for domain in domains: resp = requests.get( f"{API_BASE}/lookup/{domain}", headers={"Authorization": f"Bearer {API_KEY}"}, timeout=10, ) if resp.status_code != 200: continue tool = resp.json() cat = tool.get("primary_category", "Unknown") baseline = CATEGORY_BASELINES.get(cat, { "data_exp": 7, "training": 7, "cap": 5 }) results.append({ "domain": domain, "category": cat, "data_exposure_baseline": baseline["data_exp"], "training_retention_baseline": baseline["training"], "capability_baseline": baseline["cap"], "requires_manual_review": True, "assessed_date": datetime.now().isoformat(), }) return results # Assess tools discovered during shadow AI audit discovered = ["novelai.net", "codeium.com", "deepseek.com"] assessments = assess_discovered_tools(discovered) for a in assessments: print(f" {a['domain']:30s} {a['category']:20s} " f"data={a['data_exposure_baseline']} " f"train={a['training_retention_baseline']} " f"cap={a['capability_baseline']}")
Provide accurate starting points for most tools.
Conditional-tier or high-risk tools get the full questionnaire.
Category baselines, tier assignments, and recalculation triggers all draw from the same continuously classified corpus of AI-tool domains.
AI data flows are more complex than traditional SaaS. Four unique complications drive this complexity.
Text prompts, file uploads, API calls, browser extensions, connected integrations — broader and less controlled than traditional apps.
Neural network internals cannot be inspected. Data may be retained in weights, combined with other inputs, or leak into future outputs.
Raw logs, embeddings, model checkpoints, cached outputs — each with different retention, deletion, and jurisdictional exposure.
Model outputs may contain fragments from your inputs served to other users — an indirect exfiltration channel if data isolation is lacking.
Text, files, APIs, extensions, integrations
Inference, embeddings, fine-tuning, training
Logs, embeddings, weights, caches, backups
Responses, cross-customer leakage, caching
The composite risk score maps directly to enforcement actions in your firewall, proxy, and DLP systems.
Risk scores are living values, not static snapshots. A mature program recalculates continuously as conditions change.
Tool adds file uploads or API access → capability scope score increases.
Vendor achieves certification → vendor maturity score decreases.
Blocklist reclassifies domain → automatic risk score recalculation triggered.
This script detects changes and triggers automatic recalculation.
#!/usr/bin/env python3 """Continuous AI tool risk monitoring — detect changes and recalculate scores.""" import requests import json import hashlib from datetime import datetime, timedelta API_BASE = "https://aitoolsblocklist.com/api/v1" API_KEY = "your-api-key-here" STATE_FILE = "/var/lib/ai-risk/monitor_state.json" def load_previous_state() -> dict: try: with open(STATE_FILE, "r") as f: return json.load(f) except FileNotFoundError: return {"domains": {}, "last_check": None} def fetch_feed_changes(since: str) -> list: """Fetch domains added or modified since last check.""" resp = requests.get( f"{API_BASE}/feed/changes", params={"since": since, "format": "json"}, headers={"Authorization": f"Bearer {API_KEY}"}, timeout=30, ) resp.raise_for_status() return resp.json().get("changes", []) def detect_risk_drift(prev: dict, current: dict) -> list: """Compare previous and current tool metadata to find risk-relevant changes.""" alerts = [] for domain, data in current.items(): old = prev.get(domain) if not old: alerts.append({ "type": "new_tool", "domain": domain, "category": data["category"], "action": "Initial risk assessment required", }) elif old.get("category") != data["category"]: alerts.append({ "type": "category_change", "domain": domain, "old_category": old["category"], "new_category": data["category"], "action": "Risk score recalculation required", }) return alerts # Run monitoring check state = load_previous_state() since = state["last_check"] or ( datetime.utcnow() - timedelta(days=1) ).isoformat() changes = fetch_feed_changes(since) print(f"Changes since {since}: {len(changes)} domains") for alert in detect_risk_drift(state["domains"], { c["domain"]: c for c in changes }): print(f" [{alert['type'].upper()}] {alert['domain']}: {alert['action']}")
Get immediate preliminary assessments via category baselines.
Trigger automatic baseline comparison and tier reassignment.
AI tool risk must integrate into your existing ERM program. It cannot operate as a standalone function.
Extend risk appetite statements to address AI-specific risks: training retention, opaque processing, weight-embedded data.
Add AI tools as a distinct category. Document composite scores, tier assignments, and mitigating controls per tool.
Quantify exposure: tools per tier, shadow AI trends, assessment coverage, incidents, and month-over-month changes.
When incidents occur, recalculate affected scores immediately and review framework weights and thresholds.
The AUP translates it into employee-facing rules: approved tools, prohibited tools, and allowed data classifications.
See our AI Acceptable Use Policy guide for templates and enforcement patterns.
Continue with enforcement, policy, and governance guides from the AI risk library.
Powered by 17,410+ classified domains. Tell us about your environment and we'll scope a pilot.
Describe your environment size and current risk management tooling and we will recommend an integration path.