Evaluate Shadow AI

The Shadow AI dashboard gives you data. This page is the playbook for converting that data into action.

A weekly review rhythm

The most useful frequency for the CISO is weekly:

1. Open Monitoring → Shadow AI every Monday morning.
2. Scan Overview: any new platforms? Any spikes?
3. Open Discover: clear the new queue — confirm or dismiss every behavioral detection.
4. Open Governance: scan the sanctioned-rate trend and warn-override counts.
5. Export a monthly CSV at month-end and forward to compliance.

The compliance team should pull the CSV / PDF report monthly for evidence packaging.

Reading a Tier-1 (URL-based) finding

A typical entry on the Discovery tab:

Platform:        chat.openai.com
Category:        Writing assistant
Detections:      284 last 7 days (▲ 18% vs. previous 7d)
Users:           37 anonymized IDs
PII flagged:     12 prompts (8 × EMAIL, 3 × NAME, 1 × PHONE)
Sanctioned:      No

Decisions to make:

• Sanctioned? If not, add an AiToolPolicy row (UNSANCTIONED) so future detections route through the right governance.
• Negotiate enterprise? 284 detections / 37 users / week is non-trivial — pursue an enterprise OpenAI contract for proper privacy guarantees.
• PII pattern? 12/284 (4 %) is a measurable PII leakage rate. Roll out a MODIFY policy for BROWSER_SHIELD traces with the PII preset enabled (see Configure Policies).

Reading a Discover (behavioral) finding

A typical entry:

Domain:          notion.ai
Vector:          Extension (Notion AI Assistant v3.2.1)
                 SSE fingerprint (confidence 0.81)
First seen:      2026-05-10
Occurrences:     14
Vendor:          Notion Labs Inc.
Category:        Writing assistant
Risk level:      MEDIUM
Trains on user data: Yes

Decisions:

• The "Trains on user data: Yes" flag is the loudest compliance signal. If the project handles regulated data (GDPR, HIPAA, finance), this is a likely Confirm + Block outcome.
• Confirm writes the domain to your AiToolPolicy as UNSANCTIONED and creates a SHA-256 attestation hash (anchored on Flare if the project has anchoring on).
• Subsequent visits to notion.ai then flow through Browser Guard governance — typically a BLOCK or REDIRECT to an approved alternative.

Typical patterns and responses

"80 % ChatGPT, 15 % Claude, 5 % Gemini"

Focus policies on ChatGPT first (highest leverage). Consider an enterprise OpenAI contract for proper privacy guarantees and a higher data-retention promise.

"47 PII findings last month, 32 of which were email addresses"

Activate the GDPR Privacy system policy with MODIFY enforcement. Roll out staff training on email handling. Track the count next month — a 50 % drop is realistic.

"One user has 500 requests/day — 10× the average"

Not misconduct by default — most likely an automation script. Investigate, then either sanction the script (give it an agent_ key with proper governance) or block it.

"Sudden spike in Gemini usage on March 15"

Check whether a new team is evaluating Gemini. Reach out, ensure your existing policies cover Gemini (they should — Browser Guard Tier-2 supports Gemini), and add Gemini to your sanctioned list if the evaluation is sanctioned.

"Discover surfaced an extension I've never heard of"

The Discover evidence panel shows extensionId, vendor, version, and the "Trains on user data" flag — the third field is the priority signal. Yes → almost certainly Confirm + Block. Unknown → look up the vendor before deciding. No → still confirm (you want it in your inventory), but the urgency is lower.

Compliance impact

Every Confirm action on the Discover tab feeds the EU AI Act ctrl.shadow_ai.asset_inventory control. The control escalates from missing → implemented → enforced based on:

• missing — no AiToolPolicy rows on the project
• implemented — at least one row, but all still PENDING
• enforced — sanctioned rate ≥ 80 %

Your readiness gauges for EU AI Act Art. 9, DORA Art. 8, and NIST MAP 1 all depend on this. Clearing the Discover backlog is one of the highest-leverage compliance activities the dashboard surfaces.