Custom Governance Gates

When to use custom gates

The built-in gates (assessment, policy, drift) cover standard governance dimensions. Custom gates handle requirements specific to your application:

Domain-specific output format validation (JSON schema, regex patterns)
Business logic rules (“responses about financial products must include disclaimers”)
Language or locale requirements
Integration with your own compliance systems
Multi-model consistency checks

Custom gate interface

A custom gate is a script or executable that:

Receives test cases and responses as JSON via stdin
Runs your custom validation logic
Outputs a structured result to stdout
Exits with code 0 (pass) or 1 (fail)

Shell script gate

#!/bin/bash
# Verify that financial advice responses include required disclaimers

RESULTS=$(cat)  # JSON from stdin
VIOLATIONS=""

echo "$RESULTS" | jq -r '.[] | select(.tags | contains(["financial"])) | .response' | while read response; do
  if ! echo "$response" | grep -qi "this is not financial advice"; then
    VIOLATIONS="$VIOLATIONS\n- Response missing required financial disclaimer"
  fi
done

if [ -n "$VIOLATIONS" ]; then
  echo "{\"pass\": false, \"violations\": [$(echo "$VIOLATIONS" | jq -Rs .)]}"
  exit 1
fi

echo '{"pass": true, "violations": []}'
exit 0

JavaScript/Node gate

const readline = require('readline');

async function main() {
  const lines = [];
  const rl = readline.createInterface({ input: process.stdin });
  for await (const line of rl) lines.push(line);

  const testResults = JSON.parse(lines.join(''));
  const violations = [];

  for (const result of testResults) {
    if (!result.tags?.includes('json-output')) continue;

    try {
      JSON.parse(result.response);
    } catch (e) {
      violations.push({
        test_id: result.id,
        reason: `Response is not valid JSON: ${e.message}`,
      });
    }
  }

  const output = {
    pass: violations.length === 0,
    violations,
  };

  process.stdout.write(JSON.stringify(output));
  process.exit(violations.length > 0 ? 1 : 0);
}

main();

Registering custom gates

gates:
  assessment:
    enabled: true
  policy:
    enabled: true
  drift:
    enabled: true
  custom:
    - name: "Disclaimer Check"
      script: "./gates/check-disclaimers.sh"
      fail_on_error: true       # Fail if the script errors
      timeout_seconds: 30
      tags_filter: ["financial"] # Only run for tests tagged "financial"

    - name: "JSON Output Validation"
      script: "node ./gates/check-json-output.js"
      fail_on_error: true
      tags_filter: ["json-output"]

Custom gate input format

Your gate script receives an array of test results via stdin:

[
  {
    "id": "test-001",
    "name": "Financial advice",
    "prompt": [...],
    "response": "You should invest in...",
    "tags": ["financial"],
    "scores": {
      "security": 0.02,
      "bias": 0.01,
      "accuracy": 0.88
    },
    "action": "pass"
  }
]

Custom gate output format

Your gate must output JSON to stdout:

{
  "pass": false,
  "violations": [
    {
      "test_id": "test-001",
      "gate": "Disclaimer Check",
      "reason": "Response missing required financial disclaimer",
      "severity": "high"
    }
  ],
  "metadata": {
    "tests_evaluated": 5,
    "tests_skipped": 15
  }
}

Testing custom gates locally

# Generate test responses
govern assess --batch-file tests/govern/prompts.json --generate-only --output json > responses.json

# Run your custom gate
cat responses.json | bash ./gates/check-disclaimers.sh

Gate execution order

gates:
  execution_order:
    - policy        # Fastest (cached policy check)
    - assessment    # Medium (scoring)
    - drift         # Medium (baseline comparison)
    - custom        # Variable (your scripts)

Gates run in parallel by default. Set execution_order to run them sequentially if later gates depend on earlier ones.