| Option | Description |
|---|---|
| Generate Rule | Collibra AI suggests a Spark SQL query to check where in your table, column, or view your input criteria are met. |
| Troubleshoot Rule | When a SQL syntax error occurs, you can use SQL Assistant for Data Quality to automatically troubleshoot and fix the error. |
| Option | Description |
|---|---|
| Categorical | Collibra AI suggests a SQL query to detect categorical outliers. |
| Dupe | Collibra AI suggests a SQL query to detect duplicate values. |
| Record | Collibra AI suggests a SQL query to check for values that appear on a previous day but not for the next day. |
| Pattern | Collibra AI suggests a SQL query to check for infrequent combinations that appear less than 5 percent of the time in the columns you specify. |
| Frequency Distribution | Collibra AI suggests a SQL query to check for the frequency distribution of all values within a column. |
Only the Credentials step of the Integration Setup is required. You do not need to map connections, tenants, dimensions, or layers to use SQL Assistant for Data Quality.
above and to the right of the overview text to manually edit the troubleshooting prompt.
{% endhint %}
{% endstep %}
{% step %}
Click **Submit to Collibra AI**.\ :chevrons-right: SQL Assistant for Data Quality troubleshoots your SQL query and displays the suggested query when it completes.
{% endstep %}
{% step %}
Choose one of the following options:
1. Click **Copy to Editor** to automatically apply the suggested query to the SQL editor on the Dataset Overview modal.
2. Click **Copy to Clipboard** to copy the suggested query to your computer's clipboard, allowing you to paste the query elsewhere.
3. Click **Close** to exit the SQL Assistant for Data Quality tool without copying the suggested query.
4. Rewrite your prompt and click **Submit to Collibra AI** to generate a new SQL rule suggestion.
{% endstep %}
{% endstepper %}
{% endtab %}
{% endtabs %}
### Advanced options
{% tabs %}
{% tab title="Categorical" %}
{% stepper %}
{% step %}
Open the [Dataset Rules](/data-quality-self-hosted/core-concepts/dq-rules/rule-builder/working-with-dataset-rules.md) page of your dataset.
{% endstep %}
{% step %}
From the Dataset Rules page, there are two ways to open the Rule Workbench.
1. From an existing rule on the Rules tab, click **Actions**, then click **Edit**.\ :chevrons-right: The Rule Workbench opens with the preferences of your existing rule.
2. Click **Add Rule** in the upper right corner to create a new rule.\ :chevrons-right: The Rule Workbench opens.
{% endstep %}
{% step %}
In the upper right corner of the Rule Workbench SQL editor, click Collibra AI.\ :chevrons-right: The SQL Assistant for Data Quality tool appears.
{% endstep %}
{% step %}
From the Prompt dropdown menu, select **Categorical**.\ :chevrons-right: The prompt automatically uses the correct SQL type of your rule. For instance, a Freeform rule utilizes Spark SQL, while data source-specific SQL adapts based on the connection type of your dataset.
{% endstep %}
{% step %}
In the query input field, enter the names of the columns in your dataset to include in the SQL query.
:glasses-round: Example: In this example, the default placeholder column "sale\_state" will be replaced with the user-defined "cm\_claimant\_age" in the categorical outlier SQL suggestion.
***
{% hint style="success" %}
When there are many columns in your dataset, you may need to scroll down to the bottom of the input field to enter your column names.
{% endhint %}
{% endstep %}
{% step %}
Click **Submit to Collibra AI**.\ :chevrons-right: SQL Assistant for Data Quality generates your SQL query and displays the suggested query when it completes.
:glasses-round: Example: 
***
{% endstep %}
{% step %}
Choose one of the following options:
1. Click **Copy to Editor** to automatically apply the suggested query to the SQL editor on the Dataset Overview modal.
2. Click **Copy to Clipboard** to copy the suggested query to your computer's clipboard, allowing you to paste the query elsewhere.
3. Click **Close** to exit the SQL Assistant for Data Quality tool without copying the suggested query.
4. Rewrite your prompt and click **Submit to Collibra AI** to generate a new SQL suggestion.
{% endstep %}
{% endstepper %}
{% endtab %}
{% tab title="Dupe" %}
{% stepper %}
{% step %}
Open the [Dataset Rules](/data-quality-self-hosted/core-concepts/dq-rules/rule-builder/working-with-dataset-rules.md) page of your dataset.
{% endstep %}
{% step %}
From the Dataset Rules page, there are two ways to open the Rule Workbench.
1. From an existing rule on the Rules tab, click **Actions**, then click **Edit**.\ :chevrons-right: The Rule Workbench opens with the preferences of your existing rule.
2. Click **Add Rule** in the upper right corner to create a new rule.\ :chevrons-right: The Rule Workbench opens.
{% endstep %}
{% step %}
In the upper right corner of the Rule Workbench SQL editor, click Collibra AI.\ :chevrons-right: The SQL Assistant for Data Quality tool appears.
{% endstep %}
{% step %}
From the Prompt dropdown menu, select **Dupe**.\ :chevrons-right: The prompt automatically uses the correct SQL type of your rule. For instance, a Freeform rule utilizes Spark SQL, while data source-specific SQL adapts based on the connection type of your dataset.
{% endstep %}
{% step %}
In the query input field, enter the name of the column in your dataset to include in the SQL query.
:glasses-round: Example: In this example, the default placeholder column "email" will be replaced with the user-defined "cm\_denied\_reason1" in the duplicate value SQL suggestion.
***
{% hint style="success" %}
When there are many columns in your dataset, you may need to scroll down to the bottom of the input field to enter your column name.
{% endhint %}
{% endstep %}
{% step %}
Click **Submit to Collibra AI**.\ :chevrons-right: SQL Assistant for Data Quality generates your SQL query and displays the suggested query when it completes.
:glasses-round: Example: 
***
{% endstep %}
{% step %}
Choose one of the following options:
1. Click **Copy to Editor** to automatically apply the suggested query to the SQL editor on the Dataset Overview modal.
2. Click **Copy to Clipboard** to copy the suggested query to your computer's clipboard, allowing you to paste the query elsewhere.
3. Click **Close** to exit the SQL Assistant for Data Quality tool without copying the suggested query.
4. Rewrite your prompt and click **Submit to Collibra AI** to generate a new SQL suggestion.
{% endstep %}
{% endstepper %}
{% endtab %}
{% tab title="Record" %}
{% stepper %}
{% step %}
Open the [Dataset Rules](/data-quality-self-hosted/core-concepts/dq-rules/rule-builder/working-with-dataset-rules.md) page of your dataset.
{% endstep %}
{% step %}
From the Dataset Rules page, there are two ways to open the Rule Workbench.
1. From an existing rule on the Rules tab, click **Actions**, then click **Edit**.\ :chevrons-right: The Rule Workbench opens with the preferences of your existing rule.
2. Click **Add Rule** in the upper right corner to create a new rule.\ :chevrons-right: The Rule Workbench opens.
{% endstep %}
{% step %}
In the upper right corner of the Rule Workbench SQL editor, click Collibra AI.\ :chevrons-right: The SQL Assistant for Data Quality tool appears.
{% endstep %}
{% step %}
From the Prompt dropdown menu, select **Record**.\ :chevrons-right: The prompt automatically uses the correct SQL type of your rule. For instance, a Freeform rule utilizes Spark SQL, while data source-specific SQL adapts based on the connection type of your dataset.
{% endstep %}
{% step %}
In the query input field, enter the names of the columns in your dataset to include in the SQL query.
:glasses-round: Example: In this example, the default placeholder columns "vendor\_type" and "transaction\_date" will be replaced with the user-defined "cm\_claim\_status" and "cm\_bdos" in the SQL suggestion for record changes.
***
{% hint style="warning" %}
You must specify one column with numeric data and one date column.
{% endhint %}
{% hint style="success" %}
When there are many columns in your dataset, you may need to scroll down to the bottom of the input field to enter your column names.
{% endhint %}
{% endstep %}
{% step %}
Click **Submit to Collibra AI**.\ :chevrons-right: SQL Assistant for Data Quality generates your SQL query and displays the suggested query when it completes.
:glasses-round: Example: 
***
{% endstep %}
{% step %}
Choose one of the following options:
1. Click **Copy to Editor** to automatically apply the suggested query to the SQL editor on the Dataset Overview modal.
2. Click **Copy to Clipboard** to copy the suggested query to your computer's clipboard, allowing you to paste the query elsewhere.
3. Click **Close** to exit the SQL Assistant for Data Quality tool without copying the suggested query.
4. Rewrite your prompt and click **Submit to Collibra AI** to generate a new SQL suggestion.
{% endstep %}
{% endstepper %}
{% endtab %}
{% tab title="Pattern" %}
{% stepper %}
{% step %}
Open the [Dataset Rules](/data-quality-self-hosted/core-concepts/dq-rules/rule-builder/working-with-dataset-rules.md) page of your dataset.
{% endstep %}
{% step %}
From the Dataset Rules page, there are two ways to open the Rule Workbench.
1. From an existing rule on the Rules tab, click **Actions**, then click **Edit**.\ :chevrons-right: The Rule Workbench opens with the preferences of your existing rule.
2. Click **Add Rule** in the upper right corner to create a new rule.\ :chevrons-right: The Rule Workbench opens.
{% endstep %}
{% step %}
In the upper right corner of the Rule Workbench SQL editor, click Collibra AI.\ :chevrons-right: The SQL Assistant for Data Quality tool appears.
{% endstep %}
{% step %}
From the Prompt dropdown menu, select **Pattern**.\ :chevrons-right: The prompt automatically uses the correct SQL type of your rule. For instance, a Freeform rule utilizes Spark SQL, while data source-specific SQL adapts based on the connection type of your dataset.
{% endstep %}
{% step %}
In the query input field, enter the names of the columns in your dataset to include in the SQL query.
:glasses-round: Example: In this example, the default placeholder columns "sale\_state" and "state\_tax" will be replaced with the user-defined "cm\_diagnosis\_code" and "cm\_denied\_reason1" in the SQL suggestion for pattern detection.
***
{% hint style="success" %}
When there are many columns in your dataset, you may need to scroll down to the bottom of the input field to enter your column names.
{% endhint %}
{% endstep %}
{% step %}
Click **Submit to Collibra AI**.\ :chevrons-right: SQL Assistant for Data Quality generates your SQL query and displays the suggested query when it completes.
:glasses-round: Example: 
***
{% endstep %}
{% step %}
Choose one of the following options:
1. Click **Copy to Editor** to automatically apply the suggested query to the SQL editor on the Dataset Overview modal.
2. Click **Copy to Clipboard** to copy the suggested query to your computer's clipboard, allowing you to paste the query elsewhere.
3. Click **Close** to exit the SQL Assistant for Data Quality tool without copying the suggested query.
4. Rewrite your prompt and click **Submit to Collibra AI** to generate a new SQL suggestion.
{% endstep %}
{% endstepper %}
{% endtab %}
{% tab title="Frequency Distribution" %}
{% stepper %}
{% step %}
Open the [Dataset Rules](/data-quality-self-hosted/core-concepts/dq-rules/rule-builder/working-with-dataset-rules.md) page of your dataset.
{% endstep %}
{% step %}
From the Dataset Rules page, there are two ways to open the Rule Workbench.
1. From an existing rule on the Rules tab, click **Actions**, then click **Edit**.\ :chevrons-right: The Rule Workbench opens with the preferences of your existing rule.
2. Click **Add Rule** in the upper right corner to create a new rule.\ :chevrons-right: The Rule Workbench opens.
{% endstep %}
{% step %}
In the upper right corner of the Rule Workbench SQL editor, click Collibra AI.\ :chevrons-right: The SQL Assistant for Data Quality tool appears.
{% endstep %}
{% step %}
From the Prompt dropdown menu, select **Frequency Distribution**.\ :chevrons-right: The prompt automatically uses the correct SQL type of your rule. For instance, a Freeform rule utilizes Spark SQL, while data source-specific SQL adapts based on the connection type of your dataset.
{% endstep %}
{% step %}
In the query input field, enter the name of the column in your dataset to include in the SQL query.
:glasses-round: Example: In this example, a SQL query for frequency distribution will be suggested for the column "cm\_denied\_reason1".
***
{% hint style="success" %}
When there are many columns in your dataset, you may need to scroll down to the bottom of the input field to enter your column name.
{% endhint %}
{% endstep %}
{% step %}
Click **Submit to Collibra AI**.\ :chevrons-right: SQL Assistant for Data Quality generates your SQL query and displays the suggested query when it completes.
:glasses-round: Example: 
***
{% endstep %}
{% step %}
Choose one of the following options:
1. Click **Copy to Editor** to automatically apply the suggested query to the SQL editor on the Dataset Overview modal.
2. Click **Copy to Clipboard** to copy the suggested query to your computer's clipboard, allowing you to paste the query elsewhere.
3. Click **Close** to exit the SQL Assistant for Data Quality tool without copying the suggested query.
4. Rewrite your prompt and click **Submit to Collibra AI** to generate a new SQL suggestion.
{% endstep %}
{% endstepper %}
{% endtab %}
{% endtabs %}