quality-monitors command group

The quality-monitors command group within the Databricks CLI contains commands to create, edit, and delete quality monitors. A monitor computes and monitors data or model quality metrics for a table over time. It generates metrics tables and a dashboard that you can use to monitor table health and set alerts. See Introduction to Databricks Lakehouse Monitoring.

databricks quality-monitors create

Create a new monitor for the specified table.

The caller must either:

1. be an owner of the table's parent catalog, have USE_SCHEMA on the table's parent schema, and have SELECT access on the table
2. have USE_CATALOG on the table's parent catalog, be an owner of the table's parent schema, and have SELECT access on the table.
3. have the following permissions:
   • USE_CATALOG on the table's parent catalog
   • USE_SCHEMA on the table's parent schema
   • be an owner of the table.

Workspace assets, such as the dashboard, will be created in the workspace where this call was made.

databricks quality-monitors create TABLE_NAME ASSETS_DIR OUTPUT_SCHEMA_NAME [flags]

Arguments

TABLE_NAME

    Full name of the table.

ASSETS_DIR

    The directory to store monitoring assets (e.g. dashboard, metric tables).

OUTPUT_SCHEMA_NAME

    Schema where output metric tables are created.

Options

--baseline-table-name string

    Name of the baseline table from which drift metrics are computed from.

--json JSON

    The inline JSON string or the @path to the JSON file with the request body.

--skip-builtin-dashboard

    Whether to skip creating a default dashboard summarizing data quality metrics.

--warehouse-id string

    Optional argument to specify the warehouse for dashboard creation.

Global flags

Examples

The following example creates a quality monitor for a table:

databricks quality-monitors create main.my_schema.my_table /monitoring/assets main.monitoring_schema

databricks quality-monitors delete

Delete a monitor for the specified table.

The caller must either:

1. be an owner of the table's parent catalog
2. have USE_CATALOG on the table's parent catalog and be an owner of the table's parent schema
3. have the following permissions:
   • USE_CATALOG on the table's parent catalog
   • USE_SCHEMA on the table's parent schema
   • be an owner of the table.

Additionally, the call must be made from the workspace where the monitor was created.

Note that the metric tables and dashboard will not be deleted as part of this call; those assets must be manually cleaned up (if desired).

databricks quality-monitors delete TABLE_NAME [flags]

Arguments

TABLE_NAME

    Full name of the table.

Options

Global flags

Examples

The following example deletes a quality monitor:

databricks quality-monitors delete main.my_schema.my_table

databricks quality-monitors get

Get a monitor for the specified table.

The caller must either:

1. be an owner of the table's parent catalog
2. have USE_CATALOG on the table's parent catalog and be an owner of the table's parent schema.
3. have the following permissions:
   • USE_CATALOG on the table's parent catalog
   • USE_SCHEMA on the table's parent schema
   • SELECT privilege on the table.

The returned information includes configuration values, as well as information on assets created by the monitor. Some information (e.g., dashboard) may be filtered out if the caller is in a different workspace than where the monitor was created.

databricks quality-monitors get TABLE_NAME [flags]

Arguments

TABLE_NAME

    Full name of the table.

Options

Global flags

Examples

The following example gets information about a quality monitor:

databricks quality-monitors get main.my_schema.my_table

databricks quality-monitors get-refresh

Get info about a specific monitor refresh using the given refresh ID.

The caller must either:

1. be an owner of the table's parent catalog
2. have USE_CATALOG on the table's parent catalog and be an owner of the table's parent schema
3. have the following permissions:
   • USE_CATALOG on the table's parent catalog
   • USE_SCHEMA on the table's parent schema
   • SELECT privilege on the table.

Additionally, the call must be made from the workspace where the monitor was created.

databricks quality-monitors get-refresh TABLE_NAME REFRESH_ID [flags]

Arguments

TABLE_NAME

    Full name of the table.

REFRESH_ID

    ID of the refresh.

Options

Global flags

Examples

The following example gets information about a specific refresh:

databricks quality-monitors get-refresh main.my_schema.my_table 12345

databricks quality-monitors list-refreshes

List the history of the most recent refreshes (up to 25) for this table.

The caller must either:

1. be an owner of the table's parent catalog
2. have USE_CATALOG on the table's parent catalog and be an owner of the table's parent schema
3. have the following permissions:
   • USE_CATALOG on the table's parent catalog
   • USE_SCHEMA on the table's parent schema
   • SELECT privilege on the table.

Additionally, the call must be made from the workspace where the monitor was created.

databricks quality-monitors list-refreshes TABLE_NAME [flags]

Arguments

TABLE_NAME

    Full name of the table.

Options

Global flags

Examples

The following example lists refreshes for a quality monitor:

databricks quality-monitors list-refreshes main.my_schema.my_table

databricks quality-monitors run-refresh

Queue a metric refresh on the monitor for the specified table. The refresh will execute in the background.

The caller must either:

1. be an owner of the table's parent catalog
2. have USE_CATALOG on the table's parent catalog and be an owner of the table's parent schema
3. have the following permissions:
   • USE_CATALOG on the table's parent catalog
   • USE_SCHEMA on the table's parent schema
   • be an owner of the table

Additionally, the call must be made from the workspace where the monitor was created.

databricks quality-monitors run-refresh TABLE_NAME [flags]

Arguments

TABLE_NAME

    Full name of the table.

Options

Global flags

Examples

The following example queues a refresh for a quality monitor:

databricks quality-monitors run-refresh main.my_schema.my_table

databricks quality-monitors update

Update a monitor for the specified table.

The caller must either:

1. be an owner of the table's parent catalog
2. have USE_CATALOG on the table's parent catalog and be an owner of the table's parent schema
3. have the following permissions:
   • USE_CATALOG on the table's parent catalog
   • USE_SCHEMA on the table's parent schema
   • be an owner of the table.

Additionally, the call must be made from the workspace where the monitor was created, and the caller must be the original creator of the monitor.

Certain configuration fields, such as output asset identifiers, cannot be updated.

databricks quality-monitors update TABLE_NAME OUTPUT_SCHEMA_NAME [flags]

Arguments

TABLE_NAME

    Full name of the table.

OUTPUT_SCHEMA_NAME

    Schema where output metric tables are created.

Options

--baseline-table-name string

    Name of the baseline table from which drift metrics are computed from.

--dashboard-id string

    Id of dashboard that visualizes the computed metrics.

--json JSON

    The inline JSON string or the @path to the JSON file with the request body.

Global flags

Examples

The following example updates a quality monitor:

databricks quality-monitors update main.my_schema.my_table main.monitoring_schema

The following example updates a quality monitor with a new baseline table:

databricks quality-monitors update main.my_schema.my_table main.monitoring_schema --baseline-table-name main.my_schema.new_baseline

Global flags

--debug

  Whether to enable debug logging.

-h or --help

    Display help for the Databricks CLI or the related command group or the related command.

--log-file string

    A string representing the file to write output logs to. If this flag is not specified then the default is to write output logs to stderr.

--log-format format

    The log format type, text or json. The default value is text.

--log-level string

    A string representing the log format level. If not specified then the log format level is disabled.

-o, --output type

    The command output type, text or json. The default value is text.

-p, --profile string

    The name of the profile in the ~/.databrickscfg file to use to run the command. If this flag is not specified then if it exists, the profile named DEFAULT is used.

--progress-format format

    The format to display progress logs: default, append, inplace, or json

-t, --target string

    If applicable, the bundle target to use