> For the complete documentation index, see [llms.txt](https://docs.collibra.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.collibra.com/data-quality-self-hosted/release-notes/data-quality-observability-self-hosted-release-archive/release-2023-07.md). # Release 2023.07 * [Highlights](#highlights) * [Enhancements](#enhancements) * [Fixes](#fixes) * [Known Limitations](#known-limitations) * [DQ Security Metrics](#dq-security-metrics) * [MUI Redesign](#mui-redesign) * [React Limitations](#react-limitations) * Maintenance Updates ## Highlights * **Integrations** * When integrating metadata from Collibra DQ with Collibra Platform, you can now access the Integration page from the Admin Console in Collibra DQ. The Integration page features an integration configuration wizard, which leads you through a guided mapping, and a dataset status table to monitor and edit existing integrations.The integration configuration wizard lets you: * Seamlessly map data quality objects, such as tables, schemas, and columns to resources in Collibra Platform. * Align DQ tenants with Collibra Platform communities. * Map DQ dimensions to their corresponding dimensions in Collibra Platform. * By using these new capabilities, you can now efficiently synchronize Collibra DQ metadata with Collibra DGC with confidence and ease. * **Pushdown** * When [storing break records](/data-quality-self-hosted/core-concepts/findings/break-records-and-link-ids/archiving-break-records-from-pushdown-jobs.md) from Snowflake Pushdown jobs, you can now write them directly to a database or schema in your data source. By removing the reliance on the PostgreSQL metastore for the archival of break records, you can: * Eliminate network traffic costs for better performance. * Reduce storage limit restrictions. * Improve security by generating break records that never leave Snowflake. * Enhance your reporting abilities by removing the need to move break records from the PostgreSQL metastore to your data source. * Join datasets against your break records. {% hint style="info" %} The legacy documentation hosted at [dq-docs.collibra.com](http://dq-docs.collibra.com/) will reach its end-of-life alongside the upcoming August (2023.08) release of Data Quality & Observability (self-hosted). {% endhint %} ## Enhancements ### Capabilities * The Reports landing page now has improved organization, helpful descriptions of each report and dashboard type, and links to the documentation. * When using the v3/jobs API, you can now specify a time range to filter the list of jobs with the following parameters: * startTimeBeginning * startTimeEnd * updateTimeBeginning * updateTimeEnd * When reviewing the Rule Definitions dashboard, the Dataset column now includes links to the Findings page of the most recent run of a given dataset. * The labels no longer overlap when using the Rule Summary charts. * When using React, we renamed Catalog Dataset to Dataset Manager, and Column Manager, formerly called Catalog by Column, is now generally available. ### Platform * When adding or modifying a connection and a Connection URL string or a property in the Driver Properties field is invalid, the error message that returns now includes helpful information to help you troubleshoot the error. ### Pushdown * When including a shapes check in a Pushdown job, the shapes query now has a limit of the maximum number of shapes per column plus one to ignore any columns that exceed the maximum number of shapes per column. * When scanning for outliers in a Databricks Pushdown job, you can now select the Categorical option on the Outliers Configuration modal to check for categorical outliers. ## Fixes ### Capabilities * When profiling a PostgreSQL table that contains infinity `∞`, negative infinity `-∞`, NaN values in numeric columns, you can once again run jobs successfully. (ticket #112693) * When using the % symbol as a wildcard in queries against Db2 datasets, the % symbol no longer converts to an unreadable character, and the query runs correctly. (ticket #113618) * When running a scheduled job during a restricted time in a multi-tenant environment, the schedule restriction functionality again works correctly. Previously, jobs on certain tenants could still run during restricted times. (ticket #113794) * When setting up an SAP HANA connection, you can again successfully connect. Previously, a Java incompatibility prevented successful SAP HANA connections. (ticket #114883)

When using a JDK 8 build of Collibra DQ, you must use the SAP HANA driver provided in the Collibra DQ build package.

### Platform * When reviewing the Breaks tab of a dataset's associated rules on CockroachDB connections, you can once again use the search bar to filter break records. (ticket #105410) * When configuring multi-tenancy on CockroachDB connections, you can once again add new tenants because the V193 migration script is now compatible with CockroachDB environments V22.1.x or later. (ticket #119528) * When reviewing the web logs for Explorer, all potential personally identifiable information (PII) is now removed from the logging message. (ticket #119375) * When deploying a Kubernetes pod, the security admission and RBAC clusternode warnings are now resolved. (ticket #113204) * When upgrading a multi-tenant deployment of Collibra DQ to a newer version, the agent display name no longer reverts to the agent name if you previously set the agent display name to something else. (ticket #114355, 119704) * When using the Alert Definitions page without an email server configured, an SMTP configuration error message incorrectly displayed based on a user's role. (ticket #114086) * When a user without a role mapped to a given connection attempts to access a dataset from Explorer via the Edit Dataset button on the Findings page, their access is now restricted until a role is assigned. Previously, role restrictions were not checked from Explorer. (ticket #115217) * When using AssumeRole on an S3 connection, the STS server now defaults to the same region that the connection is configured in. (ticket #118448) ### Pushdown * When running a Snowflake Pushdown job that includes the profile activity, you no longer get an error message when a binary data type column exists in your dataset. (ticket #111977) ## Known Limitations ### DQ Cloud * Because Livy is not yet available for DQ Cloud, in order to display the data preview of remote files in the UI, you need to use temporary [Spark scratch disk space](/data-quality-self-hosted/installation/installing-data-quality-observability-self-hosted-on-self-hosted-kubernetes/deploy-on-self-hosted-kubernetes/configure-spark-scratch-disk-space-to-handle-large-datasets.md) to momentarily cache the preview. Until Livy is available for DQ Cloud, the caching of preview data occurs within DQ Cloud instead of Edge. * After upgrading a DQ Cloud instance, previously updated agent display names revert to the agent name. Any previously saved agent display names need to be manually re-entered after upgrading. ## DQ Security Metrics {% hint style="info" %} The medium, high, and critical vulnerabilities of the DQ Connector are now resolved. {% endhint %} {% hint style="danger" %} We found 1 critical and 1 high CVE in our JFrog scan. Upon investigation, these CVEs are disputed by Red Hat and no fix is available. For more information, see the official statements from Red Hat: (Critical) (High) {% endhint %} The following image shows a chart of Collibra DQ security vulnerabilities arranged by release version. ![a table showing the number of critical security vulnerabilities over a period of 5 releases](https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-f91651efb02a031586623b58ba2f940357593c5f%2Fcritical-table-2023-07.png?alt=media) The following image shows a table of Collibra DQ security metrics arranged by release version. ![a chart showing the number of critical security vulnerabilities over a period of 5 releases](https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d4ae079892b20c84308483f66183e2f232219029%2Fvulns-over-time-2023-07.png?alt=media) ## MUI Redesign The following table shows the status of the MUI redesign of Collibra DQ pages as of this release. Because the status of these pages only reflects Collibra DQ's internal test environment and completed engineering work, pages marked as "Done" are not necessarily available externally. Full availability of the new MUI pages is planned for an upcoming release. | Page | Location | Status | | ---------------------------------- | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Homepage | Homepage | Done | | Sidebar navigation | Sidebar navigation | Done | | User Profile | User Profile | Done | | List View | Views | Done | | Assignments | Views | Done | | Pulse View | Views | Done | | Catalog by Column (Column Manager) | Catalog (Column Manager) | Done | | Dataset Manager | Dataset Manager | Done | | Alert Definition | Alerts | Done | | Alert Notification | Alerts | Done | | View Alerts | Alerts | Done | | Jobs | Jobs | Done | | Jobs Schedule | Jobs Schedule | Done | | Rule Definitions | Rules | Done | | Rule Summary | Rules | Done | | Rule Templates | Rules | Done | | Rule Workbench | Rules |

In Progress

| | Data Classes | Rules | Done | | Explorer | Explorer |

In Progress

| | Reports | Reports |

In Progress

| | Dataset Profile | Profile |

In Progress

| | Dataset Findings | Findings | Done | | Sign-in Page | Sign-in Page | Done | {% hint style="info" %} Admin pages are not yet fully available with the new React MUI. {% endhint %} ## React Limitations ### Explorer * When using the SQL compiler on the dataset overview for remote files, the Compile button is disabled because the execution of data files at the Spark layer is unsupported. * You cannot currently upload temp files from the new File Explorer page. This may be addressed in a future release. ### Connections * When adding a driver, if you enter the name of a folder that does not exist, a permission issue prevents the creation of a new folder. * A workaround is to use an existing folder. ### Admin * When adding another external assignment queue from the Assignment Queue page, if an external assignment is already configured, the Test Connection and Submit buttons are disabled for the new connection. Only one external assignment queue can be configured at the same time. ### Scorecards * When creating a new scorecard from the Page dropdown menu, because of a missing function, you cannot currently create a scorecard. * While a fix for this is planned for the September (2023.09) release, a workaround is to select the Create Scorecard workflow from the three dots menu instead. ### Navigation * While the breadcrumbs at the top of all legacy pages are no longer available in the new preview pages, as of the upcoming August (2023.08) release, a metadata bar will be anchored near the top of the Finding, Profile, Dataset Preview, and Alert Builder for quick access to all essential pages. ## Maintenance Updates ### 2023.07.2 * Fixed an issue where users who previously used the API integration on Collibra DQ version 2023.05 to import Collibra DQ metadata into Collibra Platform received an error message for Flyway scripts when migrating records from the integration status table to the integration history table after upgrading to Collibra DQ version 2023.07. * Fixed an issue where preview data on the Findings page did not display fully when column values used scientific notation. ### 2023.07.4 * When using the integration API, the DQ Rule governs DQ Job relation of the DQ Rule asset no longer incorrectly identifies the "SOURCE" and "TARGET" properties in the integration JSON file. * Fixed a critical CVE ([CVE-2023-34034](https://nvd.nist.gov/vuln/detail/CVE-2023-34034)) in Data Quality & Observability (self-hosted) version 2023.07 that was identified after the release date.