Forward Events and Threats to a Delta table in your Databricks workspace.

Forwarding events to Databricks

Keep Aware writes events directly into a Unity Catalog Delta table using Zerobus Ingest. There's no message bus, storage bucket, or ingestion pipeline to run in between.

Requirements

1. Access to the Keep Aware Console: Valid login credentials, and you're logged in to the console.
2. A Databricks workspace with Unity Catalog: In a region where Zerobus Ingest is available. Your workspace and your target table must both be in the same supported region.
3. Permission to create a service principal: And to grant it Unity Catalog privileges. Its Application ID is what Keep Aware calls the OAuth client ID.
4. A managed Delta table: Zerobus Ingest writes only to managed Delta tables. External tables, default storage, and tables with catalog commits enabled aren't supported.

Find your workspace URL, workspace ID, and ingest endpoint

Log in to your Databricks workspace and look at the browser address bar:

https://dbc-a1b2c3d4-e5f6.cloud.databricks.com/?o=2281745829657864

• Workspace URL is everything before /?o= — https://dbc-a1b2c3d4-e5f6.cloud.databricks.com
• Workspace ID is the digits after o= — 2281745829657864
• Zerobus ingest endpoint combines the workspace ID with your region: https://2281745829657864.zerobus.us-west-2.cloud.databricks.com

Create the target table

Keep Aware sends each event as JSON into a single VARIANT column, so your table needs one column and never has to change when you edit the body template:

CREATE TABLE main.security.keepaware_events (event VARIANT) USING DELTA;

Query it with Databricks' VARIANT accessors:

SELECT event:user.label, event:url, event:policy.label
FROM main.security.keepaware_events
WHERE event:type = 'page';

tip

Prefer typed columns? Clear the VARIANT column name field on the integration. Keep Aware then sends the body template's top-level keys as individual columns, and your table's schema has to match the body you configure.

Create a service principal

1. In your Databricks workspace, go to Settings > Identity and access.
2. Next to Service principals, click Manage, then Add service principal.
3. Create a new service principal and give it a name.
4. On the service principal's Secrets tab, click Generate secret. Copy both the Client ID and the Secret — the secret is shown only once.

The service principal's Application ID is its OAuth client ID.

Grant it access to your table:

GRANT USE CATALOG ON CATALOG main TO `<service-principal-application-id>`;
GRANT USE SCHEMA ON SCHEMA main.security TO `<service-principal-application-id>`;
GRANT SELECT, MODIFY ON TABLE main.security.keepaware_events TO `<service-principal-application-id>`;

Set up Databricks in Keep Aware

Go to the Integrations page and select the Databricks integration. Select Create New Integration to open the configuration page.

View in console

Configure each field:

• Name — A unique name for this Databricks integration.
• Workspace URL — From the address bar, as described above.
• Workspace ID — The digits after o=.
• Zerobus ingest endpoint — Your workspace ID plus region.
• Service principal client ID — The Application ID you copied.
• Service principal secret — The generated secret.
• Target table — The fully qualified name, catalog.schema.table.
• VARIANT column name — Leave as event unless you're using typed columns.
• Body — The JSON sent for each event. See below.

Save the integration, then add it to the Policies whose events you want forwarded.

Customize the event body

Payloads use wildcards, populated from the Event or Threat that triggered the integration on a Policy. Values wrapped in double curly brackets are replaced with their value on the Event.

{
  "@timestamp": "{{@timestamp}}",
  "type": "{{type}}",
  "user": {
    "email": "{{user.email}}",
    "id": "{{user.id}}",
    "label": "{{user.label}}",
    "ip": "{{user.ip.public}}"
  },
  "url": "{{url}}",
  "domain": "{{domain}}",
  "location": "{{location_full}}",
  "page": "{{page}}",
  "policy": {
    "id": "{{policy.id}}",
    "label": "{{policy.label}}"
  },
  "category": "{{category}}",
  "browser": {
    "name": "{{browser.name}}",
    "version": "{{browser.version}}",
    "os": {
      "name": "{{browser.os.name}}",
      "version": "{{browser.os.version}}"
    }
  },
  "device": {
    "hostname": "{{device.hostname}}"
  },
  "status": "{{status}}",
  "hits": "{{hits}}"
}

Every available field is described in Event and Threat Fields.

Frequently asked questions

Question	Answer
Which Unity Catalog privileges does the service principal need?	USE CATALOG on the catalog, USE SCHEMA on the schema, and SELECT plus MODIFY on the table.
Can I write to an external table?	No. Zerobus Ingest writes only to managed Delta tables. Default storage and tables with catalog commits enabled aren't supported either.
My region isn't listed as supported. What are my options?	Use the Amazon S3 or custom webhook integration and ingest into Databricks with Auto Loader.
Do I need a SQL warehouse running?	No. Zerobus Ingest writes to the table directly, with no compute to start.
How do I know it's working?	The integration shows a status in the console after the first event. Delivery failures appear there with the error Databricks returned.
Can I change the body template later?	Yes. With a VARIANT column, template changes need no table change. With typed columns, update your table schema to match first.