Multi-Modal Guardrails

Palveron supports multi-modal governance verification. Send text, images, audio, documents, and code in a single request — the gateway auto-detects the content type and routes to the appropriate analysis provider.

Where attachments are checked. Palveron checks attachments only in requests to the Verify API, and only when Multimodal is turned on for your project (Enterprise). Through the Gateway Proxy, Palveron checks only the text of the messages; images and files in them are forwarded unchanged to your model provider.

Supported Content Types

Content type	Provider	Analysis
Text	GPT-4o-mini	PII detection, prompt injection, policy evaluation
Image	Llama Guard Vision	checks for disallowed content and text recognition once an image-analysis service is set up for your project. Without that service, an image is passed through and only its size is checked.
Audio	Whisper	Speech-to-text transcription → text analysis pipeline
Document	Text extraction	Palveron checks text that has already been extracted. Extracting PDF or Word files does not happen in the gateway.
Code	Secret Scanner	API keys, AWS credentials, GitHub tokens, private keys
Video	Stub (Phase 2)	Frame extraction → image analysis pipeline

Sending Attachments

Attachments are sent base64-encoded with a MIME type. The SDK handles encoding automatically when using the file helpers.

TypeScript

// From file path
const result = await palveron.verifyWithFile(
  'Analyze this document for compliance issues',
  './contract.pdf'
);

// From base64
const result = await palveron.verify({
  prompt: 'Check this image for sensitive content',
  attachments: [{
    contentType: 'image/png',
    data: base64EncodedImage,
    filename: 'screenshot.png',
    metadata: { source: 'user-upload' },
  }],
});

Python

from palveron import Palveron, VerifyRequest, Attachment

client = Palveron(api_key="pv_live_...")

# From file
result = client.verify_file("Check this", "/path/to/image.png")

# Programmatic
result = client.verify(VerifyRequest(
    prompt="Analyze this code for secrets",
    attachments=[Attachment.from_file("app.py")],
))

# From raw bytes
att = Attachment.from_bytes(image_bytes, "image/jpeg", "photo.jpg")

Go

att, _ := palveron.AttachmentFromFile("/path/to/contract.pdf")
result, _ := client.Verify(ctx, &palveron.VerifyRequest{
    Prompt:      "Check this document",
    Attachments: []palveron.Attachment{*att},
})

cURL

BASE64=$(base64 -i photo.jpg)

curl -X POST https://gateway.palveron.com/api/v1/verify \
  -H "Authorization: Bearer pv_live_..." \
  -H "Content-Type: application/json" \
  -d "{
    \"prompt\": \"Check this image\",
    \"attachments\": [{
      \"type\": \"image/jpeg\",
      \"data\": \"$BASE64\",
      \"filename\": \"photo.jpg\"
    }]
  }"

On the wire, the attachment's MIME field is named type (the SDKs' contentType/content_type options map to it). data is required; filename and metadata are optional.

Response

Multi-modal responses include content_type and findings:

{
  "decision": "BLOCKED",
  "reason": "Secret detected in code",
  "content_type": "code",
  "findings": [
    {
      "risk": "CRITICAL",
      "category": "aws_credentials",
      "description": "AWS Access Key ID detected: AKIA...",
      "confidence": 0.99
    }
  ],
  "trace_id": "clx...",
  "integrity_hash": "sha256:abc...",
  "should_anchor": true
}

Code Secret Scanning

The code provider scans for hardcoded secrets in any programming language:

Pattern	Example
AWS Access Keys	`AKIAIOSFODNN7EXAMPLE`
AWS Secret Keys	`wJalrXUtnFEMI/K7MDENG/bPxRfiCY`
GitHub Tokens	`ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxx`
Stripe Keys	`sk_live_...`, `sk_test_...`
Generic API keys	`sk-proj-...`, `Bearer ...`
Private Keys	`-----BEGIN RSA PRIVATE KEY-----`

Configuration

Multi-Modal can be configured in three modes under Settings → General:

Mode	Description
Managed	Palveron-hosted analysis providers. No setup required.
Self-Hosted	Your own Llama Guard / Whisper instances. Full data sovereignty.
Disabled	Multi-modal analysis off. Only text is verified.

Architecture

verify({ prompt, attachments })
  → Content-type detection (auto: OpenAI / Anthropic / Palveron format)
  → Provider routing: Text → GPT-4o-mini
                      Image → Llama Guard Vision
                      Audio → Whisper → text pipeline
                      Code → Secret Scanner
  → Merge findings
  → Apply sensitivity filter
  → Decision (PASSED / BLOCKED / MODIFIED)
  → Trace + Flare anchoring

All content types pass through the same Flare blockchain anchoring pipeline. contentType and contentMetadata.findings are stored on every trace for audit purposes.

On this page