Automate Data Store Monitor Activities

Data Store Monitors scan databases to produce Datasets that are labeled with the Fides data privacy ontology.

Monitor Management

These API calls are for managing the monitor metadata; this is useful for updating scan schedules and classification parameters.

Retrieve Monitors

GET /api/v1/plus/discovery-monitor
GET /api/v1/plus/discovery-monitor/{{monitor_config_key}}

Update a Monitor (Upsert)

{  
 "name": "Monitor Name",  
 "key": "Monitor_Name",  
 "connection_config_key": "integration-key",  
 "classify_params": {  
    "possible_targets": [  
        "user.financial.account_number",  
        "user.payment.financial_account_number",  
        "user.credentials.biometric_credentials"  
    ],  
    "num_samples": 25,  
    "context_weight": 0.95,  
    "content_weight": 0.05,  
    "excluded_categories": [],  
    "context_regex_pattern_mapping": [  
        [ ".*dob", "user.demographic.date_of_birth" ],  
        [ ".*full_name", "user.name" ]  
    ]
  },
  "datasource_params": {},
  "databases": [],
  "execution_start_date": "2025-06-19T15:06:00Z",
  "execution_frequency": "Daily",
  "excluded_databases": [],
  "enabled": true,
  "shared_config_id": null,
  "sampling_config": {
    "preview_enabled": false
  },
  "last_monitored": "2025-08-04T15:08:14.368665Z",
}

sampling_config holds the monitor's own field sample preview switch, preview_enabled, which defaults to false. It is never inherited from a shared configuration. An update replaces the whole object, and turning preview_enabled on for a monitor whose data store can't be sampled returns 422.

Path Rules

Path rules are regular expressions over a data store monitor's resource paths that decide which databases, schemas, tables and fields detection creates. They're managed per monitor, separately from the monitor upsert, so updating a monitor never changes its rules. For worked examples, see Scoping Monitors with Path Rules.

Path rules are supported on data store monitors only. Reading rules requires the discovery_monitor:read scope; creating, updating and deleting them requires discovery_monitor:update.

List Path Rules

Returns the monitor's rules ordered by position, which is creation order.

GET /api/v1/plus/discovery-monitor/{{monitor_config_key}}/path-rules

Create a Path Rule

{
  "pattern": "^analytics\\.public\\.audit_log$",
  "action": "detection",
  "effect": "exclude",
  "description": "Audit log is not personal data"
}

Returns 201 Created with the rule:

{
  "id": "mpr_5d0c6f1e-2b8a-4c3e-9a51-7f0e2d9b4c11",
  "monitor_config_key": "warehouse",
  "pattern": "^analytics\\.public\\.audit_log$",
  "action": "detection",
  "effect": "exclude",
  "description": "Audit log is not personal data",
  "created_at": "2026-09-29T18:07:16.511876Z",
  "updated_at": "2026-09-29T18:07:16.511876Z",
  "position": 3
}

Update a Path Rule

Send only the fields to change. Omitted fields are unchanged, and "description": null clears the description. pattern and effect can't be set to null, and action can't be changed.

{
  "pattern": "^analytics\\.(public|sales)\\.audit_log$",
  "description": null
}

Returns 200 OK with the updated rule.

Delete a Path Rule

DELETE /api/v1/plus/discovery-monitor/{{monitor_config_key}}/path-rules/{{rule_id}}

Returns 204 No Content.

Path Rule Fields

Unknown fields in a request body are rejected with 422.

Matching Behavior

The resource path is the staged resource URN without its leading monitor key, for example analytics.public.users.email. Names that contain characters other than letters, digits, _ and - appear percent-encoded (first.name is first%2Ename). Matching ignores case; use (?-i:...) to require exact case.

Additional rules:

• Rule order doesn't affect include and exclude. When several collapse rules match the same path, the rule with the lowest position claims it.
• A collapse representative is the shard furthest through review, judged by its columns: promoted, then reviewed, then classified, then any other staged resource, then a shard with no staged resource, then a muted one. Ties go to the greatest name in natural order.
• A table a collapse rule keeps is detected even when it matches the monitor's sharded_table_pattern.
• Include and exclude rules only prevent resources from being created. Existing staged resources are not deleted, rescanned or marked as removed.
• A collapse rule deletes the staged resources, and everything beneath them, of shards it skips that nothing has happened to yet: the shard and everything beneath it are still addition. Any other shard, including classified, reviewed, promoted and muted ones, is left as it is.
• A pattern that takes longer than one second to match a path fails the scan.

Path Rule Status Codes

Field Sample Preview

Returns a few live values of one field so a reviewer can check its classification. See Verifying a classification with sample values for setup and supported data stores.

The request is accepted only when the FIDESPLUS__DETECTION_DISCOVERY__SAMPLING_PREVIEW_ENABLED deployment setting and the monitor's sampling_config.preview_enabled are both on. The caller needs the discovery_monitor:update scope or must be a steward of the monitor.

Request Sample Values

{
  "staged_resource_urn": "my_monitor.analytics.public.users.email"
}

The URN must be a field staged resource under the monitor in the path, and at most 2048 bytes in UTF-8. Unknown fields in the body are rejected with 422.

A worker reads the values. The API waits up to 2 seconds for them:

• If the worker finishes in time, or values for this field were read in the last five minutes, the response is 200 OK with "status": "ready".
• Otherwise the response is 202 Accepted with "status": "pending" and no samples. Send the same request again every couple of seconds until it returns 200. Repeat requests don't start another read while one is running.

{
  "status": "ready",
  "monitor_config_key": "my_monitor",
  "staged_resource_urn": "my_monitor.analytics.public.users.email",
  "field_name": "email",
  "samples": ["jane@example.com", "sam@example.com"],
  "num_samples": 2,
  "truncated": false,
  "retrieved_at": "2026-10-20T15:04:05.123456Z"
}

Responses carry Cache-Control: no-store. Every request is recorded in the audit log, without the values.

Field Sample Response Fields

Field Sample Status Codes

405 and 502 responses have a body of {"detail": "...", "error_kind": "..."}, so a client can tell the failures apart without parsing detail.

Field Sample Request Limits

Each user, or each API client when no user is behind the token, can have at most 3 previews running at once and can start at most 30 previews a minute. Only a request that starts a read counts toward the per-minute limit; polling a pending preview and results reused from the last five minutes don't.

A request over either limit returns 429 Too Many Requests with a Retry-After header and this body:

{
  "detail": "Too many sample requests; try again shortly",
  "retry_after_seconds": 5
}

Rejected requests are recorded in the audit log.

Staged Resources & Results

The /results endpoint is used to retrieve staged resources; this is used to traverse the staged resource hierarchy.

A staged resource is specified by the staged_resource_urn query parameter. If a staged_resource_urn is not provided, then the top-level staged resources are returned.

The API can recursively traverse the staged resource hierarchy; the API response object contains child urn fields, which can then be provided as the staged_resource_urn parameter in a subsequent API call to see that resource's children, and so on.

Retrieve Results

GET /api/v1/plus/discovery-monitor/results
GET /api/v1/plus/discovery-monitor/results?staged_resource_urn={{staged_resource_urn}}&page=1&size=50

Detection Actions

These API calls designate which assets in your data architecture are included in your Astralis pipeline.

Confirm

Any confirmed asset will be monitored and listed in Discovery.

When you confirm a resource the endpoint will return a monitor_task_id. You will want to save this for a later step in this guide.

POST /api/v1/plus/discovery-monitor/{{monitor_config_key}}/{{staged_resource_urn}}/confirm

Mute

Any muted asset will not be moved to the Discovery step. Muted assets will no longer be detected in scans. If you wish to include it again unmute it.

["staged_resource_urn1", "staged_resource_urn2"]

Unmute

Unmuted assets will be detected in scans. An unmuted asset will still need to be confirmed in order to be moved to the Discovery step.

["staged_resource_urn1", "staged_resource_urn2"]

Discovery Actions

Update Data Categories for Staged Resources

[  
  {  
    "urn": "staged_resource_urn1",  
    "user_assigned_data_categories": [  
      "user.name"  
    ],  
    "user_assigned_data_uses": [  
      "marketing.third_party"  
    ],  
    "user_assigned_system_key": "system_key"  
  }  
]

Promote Staged Resources to Dataset

["staged_resource_urn1", "staged_resource_urn2"]

Dataset Queries

The resources have now been promoted to a Dataset. To validate the output, retrieve data through the dataset endpoint.

GET /api/v1/dataset/{fides_key}

Task Management

The task management APIs are used to track the Astralis worker processing status.

GET /api/v1/plus/discovery-monitor/tasks?status=in_processing&action_type=classification
GET /api/v1/plus/discovery-monitor/tasks/{{monitor_task_id}}