> For the complete documentation index, see [llms.txt](https://docs.collibra.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.collibra.com/data-quality-self-hosted/release-notes/data-quality-observability-self-hosted-release-archive/release-2023-02.md). # Release 2023.02 * [Highlights](#highlights) * [New Features](#new-features) * [Enhancements](#enhancements) * [Fixes](#fixes) * [Known Limitations](#known-limitations) * [DQ Security Metrics](#dq-security-metrics) ### Highlights * To enhance user experience, the React MUI is now on by default across the Collibra DQ app. You can restore the legacy experience by following the steps on the React page. * You can now use the new /v3/rules/{source}/copy API to bulk copy all rules from an existing dataset to another target dataset. With this new API, you can input a source dataset from which to copy rules, then choose one or more target datasets to which your rules copy. * You can now leverage the Spring Framework to access controller-level logs and debug API calls. * The new [Oversized Job Report](/data-quality-self-hosted/core-concepts/reports/oversized-job-report.md) is now available by default in the Reports section. The Oversized Job Report lets you review usage analytics for jobs that use more hardware than what is required for their workloads. * You can now view and edit your [Agent Display Name](/data-quality-self-hosted/installation/agent.md) from the Agent Management screen. This lets you customize your Agent Display Name to make it easier to identify your agent throughout Collibra DQ. * When running large DQ Spark jobs, you can now [bring in on-demand Persistent Volume Claims](/data-quality-self-hosted/installation/installing-data-quality-observability-self-hosted-on-self-hosted-kubernetes/deploy-on-self-hosted-kubernetes/configure-spark-scratch-disk-space-to-handle-large-datasets.md) to use as Spark scratch disk space on Kubernetes deployments. {% hint style="info" %} If you only install owlweb or owlagent, you need to copy the owlmanage.sh file from the installation package into the bin directory with the following command:`cp $PACKAGE_LOCATION/owlmanage.sh $INSTALL_PATH/bin/owlmanage.sh` {% endhint %} ### New Features #### Reports * The new Oversized Job Report is now available by default. #### Agent * You can now view and edit your Agent Display Name from the Agent Management screen. * When running large DQ Spark jobs, you can now bring in on-demand Persistent Volume Claims to use as Spark scratch disk space on Kubernetes deployments. #### APIs * You can now use the new /v3/rules/{source}/copy API to bulk copy all rules from an existing dataset to another target dataset. * You can now leverage the Spring Framework to access controller-level logs and debug API calls. ### Enhancements #### Rules * Native rules with invalid SQL queries now display an exception message instead of incorrectly displaying a passing status. * You can now drill into a pulse view chart on the $daysWithoutData stat rules on the Findings page. #### Platform * React MUI is now on by default. * To ensure proper compliance with Collibra's security protocol, new installations now require you to set up a password that meets Collibra's password policy and a default email address associated with the admin user account. * Instructions for providing these credentials are interactive as part of the setup script for [Standalone installs](/data-quality-self-hosted/installation/installing-data-quality-observability-self-hosted-on-self-hosted-spark-standalone.md). * For new installs in non-interactive mode, such as automated deployment, export the following variables, as they are required for the script to execute correctly: * `export DQ_ADMIN_USER_PASSWORD` * `export DQ_ADMIN_USER_EMAIL` * For [Cloud native installs](/data-quality-self-hosted/installation/installing-data-quality-observability-self-hosted-on-self-hosted-kubernetes/install-on-self-hosted-kubernetes.md), you must use the following Helm variables to set the password and default admin email: * `--set global.web.admin.email=${email} \` * `--set global.web.admin.password=${password} \` * For multi-tenant instances, admin users are created via the UI, not with this process. #### Security * To ensure proper compliance with Collibra's security protocol, all deployments of Collibra DQ on Kubernetes now use a read-only root filesystem. * All ephemeral storage directories, such as /opt/owl/log, /tmp/scratch, and /opt/owl/cache, are still writable for Collibra DQ to function correctly. * The read-only root filesystem is enforced on all long-running pods except for Spark jobs and Livy sessions. * Ensure that you pick the corresponding Helm charts for a successful upgrade. * Any customizations must be ported to the corresponding Helm charts. For additional assistance, please contact Collibra Professional Services. * You can no longer upload custom JDBC drivers under the /opt/owl/drivers folder. Refer to the documentation for more details on how to [configure custom JDBC drivers](/data-quality-self-hosted/installation/installing-data-quality-observability-self-hosted-on-self-hosted-kubernetes/deploy-on-self-hosted-kubernetes/configure-custom-jdbc-data-source-drivers.md).

Standalone deployments of Collibra DQ are not affected by these updates.

#### Snowflake Pushdown (in preview) * You can now scan for categorical outliers. * You can now apply weighting and boundaries as part of the Outlier layer. ### Fixes #### Rules * Fixed an issue where regex rules, which worked in previous DQ versions, appeared as exceptions after upgrading to the 2022.09 version. (ticket #102615) #### Dupes * Fixed an issue that prevented jobs from running with Dupes check switched on for tables with only one column. (ticket #104585) #### Platform * Fixed an issue with the print statement in the stage 3 logs where an excessive number of log records were written due to too many observations. (ticket #104081) #### Security * Fixed an issue with role mapping where invalid values for `LDAP_GROUP_RESULT_DN_ATTRIBUTE` and `LDAP_GROUP_RESULT_CONTAINER_BASE` set to the base OU of the AD groups caused the role mapping to fail. #### Connections * Fixed an issue with connections using Kerberos authentication that prevented jobs from running because of IT restrictions around writing to Openshift containers. (ticket #102964) * Fixed an issue with Databricks cluster connections to allow `PWD={}` input in the connection string URL. (ticket #104845) * You are still required to enter `PWD={your-text-here}` in the URL. * This fix only applies to Databricks cluster connections, not to Databricks connections that use SQL-endpoints. * The logging of entire ServiceNow response messages has been added to help identify errors. #### Agent * Fixed an issue where associating or disassociating connections from a remote Agent resulted in a PreparedStatementCallback error, which prevented connections from being deleted. (tickets #102322, 102763) #### Snowflake Pushdown (in preview) * Fixed an issue with Min and Max on the Profile page that caused strings with numerical values to display invalid results. (ticket #105935) ### Known Limitations #### Rules * The $daysWithoutData stat rule has a limitation where rules that are not named "daysWithoutData" do not display their Pulse View charts when you drill into the rule on the Findings page. A workaround for this is to name your stat rule exactly as it appears here: `daysWithoutData`. #### Platform The dollar sign `$` is a special character that must be escaped with a backslash `\` when running the setup script or Helm via the command line. ### DQ Security Metrics The following image shows a chart of Collibra DQ security vulnerabilities arranged by release version. ![security vulnerabilities over five months](https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-6ea57f529fd55391f0771c5f91628ea1e911a074%2Fvulns-over-time-2023-02.png?alt=media) The following image shows a table of Collibra DQ security metrics arranged by release version. ![critical security vulnerabilities table over five months](https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-87b303bb175b8ec8bcd3c9513c88314cecf317e3%2Fcritical-table-2023-02.png?alt=media)