> For the complete documentation index, see [llms.txt](https://docs.collibra.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.collibra.com/data-quality-self-hosted/release-notes/data-quality-observability-self-hosted-release-archive/release-2022-11.md). # Release 2022.11 {% hint style="danger" %} The [MS SQL driver](/data-quality-self-hosted/connections/supported-connections.md) that comes with JDK11 standalone packages does not currently work in the JDK11 environment. MSSQL requires a separate JAR for JDK11. Please contact your Customer Success Manager for the compatible driver.[Dremio](/data-quality-self-hosted/connections/supported-connections.md) is not currently supported for JDK11 standalone packages. If you plan to run JDK11, add `-Dcdjd.io.netty.tryReflectionSetAccessible=true` to owlmanage.sh as a JVM option for your Web and Spark instances. Please contact your Customer Success Manager for assistance.Dremio jobs currently fail on both K8s and standalone JDK11 deployments. Add the following config to the Free Form (Appended) field of the Agent Configuration template: `-conf spark.driver.extraJavaOptions=-Dcdjd.io.netty.tryReflectionSetAccessible=true`.As of October 18, 2022, all images for the 2022.10 release have a Critical CVE (CVE-2022-42889). If you picked up the 2022.10 release before October 18, 2022, there should be no issue with your scans. If issues persist, please contact your Customer Success Manager for a new build. {% endhint %} {% hint style="info" %} After you complete an upgrade or a new installation of Collibra DQ, you are now required to enter a license name by following either a one-time prompt on the login page, entering the `LICENSE_NAME` environment variable in the environment variable file (owl-env.sh), or by entering the `global.configMap.data.license_name` Helm chart variable. Your license name is the value after `YOUR NAME IS =` found in the license provision email sent to you by Collibra. Customers who do not have this information due to being issued a license before March 2022 should input license information following the format below.**For a single instance:** \**For multiple instances:** \-dev, \-test, \-prodNo spaces or special characters are permitted except for hyphens `-`. {% endhint %} {% hint style="info" %} Before you execute the setup.sh script, update the `SPARK_PACKAGE` variable to the desired Spark package, for example `spark-3.2.2-bin-hadoop3.2.tgz`. You can check the `../packages` folder for the Spark package that was previously downloaded. You do not need to update the setup.sh script if you use the following export statement:`export SPARK_PACKAGE="spark-3.2.2-bin-hadoop3.2.tgz"` {% endhint %} ### New Features #### Platform * The following pages now support the new React MUI: * Scorecards * List View * Assignments * Pulse View * Alerts

React is turned off by default for the 2022.11 release. If you would like to try the new React pages, you can toggle it on from the Admin Console, or contact your Customer Success Manager for assistance.

#### DQ Job * You can now terminate jobs from the Jobs page if they are in progress, incorrectly submitted, or stuck in Staged status. When you terminate a job, two alerts are generated. * Jobs in the Spark UI display Finished statuses, even though they are terminated from the DQ UI. #### Alerts * You can now generate alerts for the following stale data stat rules: * `$daysWithoutData` * `$runsWithoutData` * `$daysSinceLastRun` * You can now generate alerts for jobs stuck in Staged status for more than one hour. #### Admin * You can now configure LDAP for user access in multi-tenant environments. #### Connections * You can now use key-pair authentication for Snowflake connections. * When you append to the Connection URL string, your entry must be comma separated. * When you manually modify the Driver Properties field, your entry must be semicolon separated. * CDATA connections are now supported in standalone deployments. * CDATA drivers are now included in the release package. #### Cloud Storage * Azure Blob Storage is now a supported target storage system. #### Snowflake Pushdown (in preview) * Schema Change monitoring from the AdaptiveRules tab is now enabled by default. * Schema is now separated from basic profiling. * The new DatasetDefDTO API now returns Pushdown information. * Dataset security checks are now implemented for Pushdown jobs. ### Enhancements #### Explorer * The Job Estimate dialogue now has improved guidance on executors and cores. The Job Estimate now estimates when a max core, max executor, and max memory is reached. #### DQ Job * Job schedule time zone is now a read-only field and can no longer be configured. Existing scheduled jobs reflect their current settings, but all other scheduled jobs are now based on the time zone of the DQ server (UTC). (ticket #88797, 89736, 92611, 95231) #### Dupes * A new warning message now displays when increasing the duplicate check limit from the UI. (ticket #95604) #### Security * Kubernetes service accounts associated with AWS IAM pod roles for controlling access to AWS services for cloud native DQ deployments on AWS EKS are now supported. * When DATASET SECURITY is enabled, DATASET ACCESS is now required to edit, map, or retrieve datasets or business units. (ticket #92934) ### Fixes #### Rules * Fixed an issue that prevented freeform rules containing double backslashes from saving. (ticket #96636, 96640) * Fixed an issue that caused rules containing open brackets (`[` ) to display break records incorrectly. (ticket #94399) * Fixed an issue that caused rules containing regex to throw out of range exceptions. (ticket #98435) #### DQ Job * Fixed an issue where run time was not displayed on the findings page because run\_id column type in the metastore did not include time zone. (ticket #96050) * Fixed an issue that caused Parquet files to fail during the LOAD activity. (ticket #96191) * Other NFS file types, including ORC, CSV, and Avro, also run successfully. #### Alerts * Fixed an issue when saving batch names that used spaces between delimiters, which caused an invalid error to occur. (ticket #97028) #### Validate Source * The Add Column Names feature is now removed from the Source tab. (ticket #96066) * Instead, use the query to edit/limit columns or use Update Scope. * Fixed an issue where disabling source check on a cloned dataset resulted in an error. You can now disable source validation on cloned datasets. (ticket #97795) #### Dupes * The Advanced Filter is now hidden from the Dupes tab. (ticket #96065) #### Shapes * Fixed an issue when editing a dataset that reverted the Shape Detection setting (Off, Auto, or Manual) applied when it was created. (ticket #95471, 95473) #### Schema * Fixed an issue with schema detection on files where schema detection was performed on all columns when a subset of columns was selected. (ticket #92476) * Use the`headercheckoff` flag when it is necessary to see only when columns are added or dropped. * Fixed an issue where schema changes were not correctly identified and updated. (ticket #96013) #### Behavior * Fixed an issue with behavior lookback(`-bhlb`) that caused Row Count changes to be misrepresented. (ticket #94840) #### Connections * Azure Blob connections in standalone environments require the following jars to be added to the `$SPARK_HOME/jars` folder: * hadoop-azure-3.2.0.jar * wildfly-openssl-1.1.3.Final.jar #### API * Fixed an issue with the DB import process to ensure JobSchedule records import without error. (ticket #98405) ### Known Limitations #### DQ Job * Job termination is not supported for jobs in Unknown status. #### Validate Source * Cloning and saving, enabling, or disabling the source tab is associated with the original dataset name and fails on the screen when an update is made, but does not affect the actual job run. #### Connections * When adding driver properties using the +Add Property option for Snowflake connections, semicolons are incorrectly appended to key values. Instead, use comma format to separate key values. ### DQ Security Metrics ![Security vulnerabilities over 5 months](https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-b641508551113f88fd1faa72b839c33de342b2ee%2Fvulns-over-time-2022.11.png?alt=media) ![Critical vulnerabilities over 5 months](https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-0183be9eafcec34dfa052358543976600f6309cd%2Fcritical-table-2022-11.png?alt=media)