> For the complete documentation index, see [llms.txt](https://docs.collibra.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.collibra.com/data-quality-self-hosted/release-notes/data-quality-observability-self-hosted-release-archive/release-2023-04.md).

# Release 2023.04

* [Highlights](#highlights)
* [New Features](#new-features)
* [Enhancements](#enhancements)
* [Fixes](#fixes)
* [Known Limitations](#known-limitations)
* [DQ Security Metrics](#dq-security-metrics)
* [Maintenance Updates](#maintenance-updates)

### Highlights

* We are excited to announce that [Pushdown for Snowflake](/data-quality-self-hosted/core-concepts/explorer/processing-methods/pushdown-processing.md) is now generally available! Pushdown is an alternative compute method for running a DQ Job, where Collibra DQ submits all of the Job's processing directly to a SQL data warehouse, such as Snowflake. When all of your data resides in Snowflake, Pushdown reduces the amount of data transfer, eliminates egress latency, and removes the Spark compute requirement of a DQ Job.
* Six new out-of-the-box reports are now available:
  * [User Activity](/data-quality-self-hosted/core-concepts/reports/user-activity-report.md) shows user click activity in Collibra DQ.
  * [Row Count by Dataset](/data-quality-self-hosted/core-concepts/reports/row-count-by-dataset-report.md) shows the number of rows in a given dataset.
  * [Alert Details](/data-quality-self-hosted/core-concepts/reports/alert-details-report.md) shows alert details and the method by which they are sent, for example email.
  * [Job Runs per User](/data-quality-self-hosted/core-concepts/reports/job-runs-per-user.md) shows the number of Jobs run per user over a certain period of time.
  * [Daily Job Runs per User](/data-quality-self-hosted/core-concepts/reports/daily-job-runs-per-user.md) shows the number of Jobs run per user on a particular day.
  * [Rule Updates per User](/data-quality-self-hosted/core-concepts/reports/rule-updates-per-user.md) shows when a user-created rule is inserted, updated, or deleted, and applied to the dataset by a user over a certain period of time.

{% hint style="info" %}
With the development of the newly designed Template Rules page in the upcoming the May release, we will use the newly built v2/createcustomrule API for custom rules and will deprecate the old v2/insertcustomrule.
{% endhint %}

{% hint style="danger" %}
When you enable **Include Links** on the Alerts Configuration page of the Admin Console, you must also enter your host address in the **HOST\_NAME** input field on the Application Configuration page. If the HOST\_NAME input field does not contain a host address when Include Links is enabled, your DQ Jobs will fail.
{% endhint %}

### New Features

#### Reports

* The User Activity, Row Count, Alert Details, Job Runs per User, Daily Job Runs per User, and Rule Updates per User reports are now available.

### Enhancements

#### Rules

* You can now view rows scanned and rows breaking into the rule\_output table.

#### Reports

* You can now view and filter reports on column dimensions with the Column Dimension Dashboard. This dashboard provides an overview of all columns, datasets, and business units, and provides insights about total current DQ Dimension scores, Dimension scores over time, and DQ scores (along with other metadata) for each column.
  * This dashboard is the first version and design adjustments are planned for a future release.
  * We also plan to redesign the Reports page in an upcoming release to better list all charts and dashboards and make it easier for you to know what we offer and potential use cases for the reports and dashboards.
  * Contact the Collibra DQ Product team or your CSM to learn more.

#### Platform

* In order to further enable Kubernetes-deploying users to more easily configure certain parameters upon deployment, the following variables are now available as owl-web-configmap properties:
  * SAML\_MAX\_AUTH\_AGE
  * SAML\_ENABLED
  * SAML\_ENTITY\_ID
  * CORS\_ALLOWED\_ORIGINS
  * SAML\_ENTITY\_BASEURL
* With the MUI turned on, the DQ Score Cards tab is now called Views.

#### Pushdown

* Record change observations are now included on the Findings page for Pushdown Jobs.
* The functionality of the v3/rules/{source}/copy API to work across Pull-Up and Pushdown datasets is now extended.
* You can now add a description of your dataset from the Review page before you create a Pushdown Job.

### Fixes

#### Rules

* Fixed an issue with the Rule Builder that prevented rules from properly validating. (ticket #106100)
* Fixed an issue with the Rule Builder that prevented the selection of available columns and limited the ability to scroll through the list of options on the Primary Column dropdown list. (ticket #110276)
* Fixed an issue with the rule breaks table within the Breaks tab where sorting the table by the Rule Name column caused an error to throw. (ticket #105410)
* Fixed an issue where, when a rule had sub-select at the beginning of the WHERE clause, the rule displayed an Exception status when it ran. (ticket #109812)

#### Explorer

* Fixed an issue with the Transform option on Athena connections, the DATE() function was incorrectly included when the job was executed with an unlocked Run CMD. (ticket #105316)

#### Platform

* Fixed an issue where, while editing an existing job in Explorer, Collibra DQ became stuck in the loading stage. (ticket #105573)
* Fixed an issue that incorrectly allowed the creation of local users even with LOCAL\_REGISTRATION\_ENABLED set to `false`. (ticket #110325)
* Fixed an issue where a role-less user was able to access the Edit Rule modal from the Rule table on the Findings page. (ticket #104129)
* Fixed an issue where the install packages for Collibra DQ versions 2023.02 and 2023.03 did not contain the owlmanage.sh script. (ticket #110786)

#### Security

* Fixed a potential security issue where the introduction of a smart pre-admin approval message on the sign-in screen in the 2023.03 release version is now reverted to the previous sign-in error message.

#### APIs

* Fixed an issue the /v2/get-job-log API where the Job Log did not load and threw an error when more than two datasets were included in a rule. (ticket # 105498)

#### Connections

* Fixed an issue where too many open connections prevented new connections from being opened. (ticket #107034)
* Fixed an issue where a database connection remained idle and did not close when the /v2/getsqlresult API was called. (ticket #108365)

#### Source

* Fixed an issue where Redshift source queries failed when -profilePushdown count option was used. (ticket #106085)
* Fixed an issue with the Cell Threshold invalidation logic which caused the issue count to remain the same after the issue was invalidated and the job re-ran.

### Known Limitations

#### Connections

* A limitation with Hive JDBC was observed where having both JDBC41 and JDBC42 drivers caused a conflict that prevents table data from loading. As a result, this release includes only the JDBC41 driver. This driver will be upgraded in a future release.
* Hive Direct job execution is currently not supported due to a technical issue that we plan to fix in an upcoming release.

#### Capabilities

* Sub-selecting columns of targets in the Source activity results in a failed Job when the Job runs.
  * A possible workaround is to keep the query as a select all (`select *`) operation.
* When you enable **Include Links** on the Alerts Configuration page of the Admin Console, you must also enter your host address in the **HOST\_NAME** input field on the Application Configuration page.
  * A fix for this is planned for the May release.

#### Pushdown

* Pushdown is currently only available for on-premises deployments of Collibra DQ.
  * Support for Pushdown in DQ Cloud is planned for a future release.

#### DQ Connector

* There is an issue where Rest API calls from Collibra DGC to Collibra DQ result in a 403 Forbidden Error because Collibra DGC passes a cookie header which causes Collibra DQ to reject the request due to recent updates to CSRF token requirements. This limits Collibra DQ's ability to connect to Collibra Platform.
  * A possible workaround until a fix is available is to update `export CSRF_TOKEN_ENABLED=false` in the owl-env.sh file for Standalone deployments or update `dq.security.csrf.token.enabled=${CSRF_TOKEN_ENABLED:false}` in the Web ConfigMap for Kubernetes deployments.

### DQ Security Metrics

The following image shows a chart of Collibra DQ security vulnerabilities arranged by release version.

![Critical vulnerabilities over 5 months](https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-9f1efe3895be8a072eaa927ff1714e0cc684be14%2Fvulns-over-time-2023-04.png?alt=media)

The following image shows a table of Collibra DQ security metrics arranged by release version.

![Critical vulnerabilities table](https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-67b8556621357e8d20aaabd2d0035be5353cd8ee%2Fcritical-table-2023-04.png?alt=media)

### MUI Redesign

The following table shows the status of the MUI redesign of Collibra DQ pages as of this release. Because the status of these pages only reflects Collibra DQ's internal test environment and completed engineering work, pages marked as "Done" are not necessarily available externally. Full availability of the new MUI pages is planned for an upcoming release.

| Page                               | Location                 | Status                                                                                                                                                                                                                                                                      |
| ---------------------------------- | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Homepage                           | Homepage                 | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| Sidebar navigation                 | Sidebar navigation       | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| User Profile                       | User Profile             | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| List View                          | Views                    | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| Assignments                        | Views                    | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| Pulse View                         | Views                    | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| Catalog by Column (Column Manager) | Catalog (Column Manager) | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| Dataset Manager                    | Dataset Manager          | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| Alert Definition                   | Alerts                   | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| Alert Notification                 | Alerts                   | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| View Alerts                        | Alerts                   | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| Jobs                               | Jobs                     | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| Jobs Schedule                      | Jobs Schedule            | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| Rule Definitions                   | Rules                    | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| Rule Summary                       | Rules                    | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| Rule Templates                     | Rules                    | <p><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-de45f34193ae06a2b093c2dfb8fb80cfda945166%2Fclose-large%402x.png?alt=media" alt="" data-size="original"></p><p>In Progress</p> |
| Rule Workbench                     | Rules                    | <p><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-de45f34193ae06a2b093c2dfb8fb80cfda945166%2Fclose-large%402x.png?alt=media" alt="" data-size="original"></p><p>In Progress</p> |
| Rule Builder                       | Rules                    | <p><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-de45f34193ae06a2b093c2dfb8fb80cfda945166%2Fclose-large%402x.png?alt=media" alt="" data-size="original"></p><p>In Progress</p> |
| Data Classes                       | Rules                    | <p><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-de45f34193ae06a2b093c2dfb8fb80cfda945166%2Fclose-large%402x.png?alt=media" alt="" data-size="original"></p><p>To Do</p>       |
| Explorer                           | Explorer                 | <p><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-de45f34193ae06a2b093c2dfb8fb80cfda945166%2Fclose-large%402x.png?alt=media" alt="" data-size="original"></p><p>In Progress</p> |
| Reports                            | Reports                  | <p><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-de45f34193ae06a2b093c2dfb8fb80cfda945166%2Fclose-large%402x.png?alt=media" alt="" data-size="original"></p><p>In Progress</p> |
| Dataset Profile                    | Profile                  | <p><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-de45f34193ae06a2b093c2dfb8fb80cfda945166%2Fclose-large%402x.png?alt=media" alt="" data-size="original"></p><p>In Progress</p> |
| Dataset Findings                   | Findings                 | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |
| Sign-in Page                       | Sign-in Page             | <img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Done                            |

### Maintenance updates

#### 2023.04.1

* External users can now successfully run Jobs that include alert links.
  * The email address of the owner of a Job with alert links is no longer included in alert emails for both local and external users.

#### 2023.04.2

* External users can now successfully run Jobs that include alert links.
  * The email address of the owner of a Job with alert links is no longer included in alert emails for both local and external users.
* Fixed an issue with Collibra DQ on AWS Cloud where Instance Profile authentication caused scheduled S3 Jobs to fail.
