> For the complete documentation index, see [llms.txt](https://docs.collibra.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.collibra.com/data-quality-self-hosted/release-notes/data-quality-observability-self-hosted-release-archive/release-2022-12.md). # Release 2022.12 ### New Features #### Explorer * The types of queries that can run from the View Data page are now restricted to read-only queries only. #### APIs * You can now copy SQLG- and SQLF-type rules from an existing dataset to another existing dataset with the /v3/rules/copy API call. #### Connections * You can now create a MongoDB connection with a CDATA driver. #### Snowflake Pushdown (in preview) * You can now detect outliers when running a Pushdown job. ### Enhancements #### DQ Job * All tables on the Jobs page now include pagination, dropdown filters, and the ability to export. #### Rules * Rules associated with datasets with zero rows now execute successfully. * Stat rule evaluation on secondary datasets is now supported for SQLF rules. #### Profile * You can now view run execution details and stale data by toggling the box chart on the findings page. #### APIs * The getRecords notebook API function is now updated and the getGeneric query is renamed getDupesPreview. * You can now obtain rules from a dataset and reassign them to another dataset with the following Databricks notebook API functions: * `def addRules(rules: List[Rule], dataset: String): Owl` * `def getRulesDfByDataset(dataset: String): DataFrame` * `def getRulesByDataset(dataset: String): List[Rule]` * `def getRuleNamesByDataset(dataset: String): DataFrame` #### Platform * TechPreview (TP) labels are now removed from the UI. #### Connections {% hint style="danger" %} As of September 2022, Databricks JDBC driver version 2.6.27 is packaged as part of both standalone and Kubernetes download packages. The Databricks Simba driver (version 2.6.22) is no longer packaged for Kubernetes. As a result of this change, the Databricks connection template has changed, and any existing connection using the old driver (2.6.22) must be updated. For more information on updating your drivers, refer to [Standalone Upgrade](/data-quality-self-hosted/installation/upgrade-data-quality-observability-self-hosted.md). {% endhint %} * The Databricks SQL endpoint is now supported for JDBC connections. * 3DES and DES encryption cipher for Kerberos authentication types are no longer supported because of recent Red Hat OS (RHEL 8.7) cipher deprecation. ### Fixes #### Explorer * Fixed an issue with the `-rdEnd` variable in the command line the variable in the query to be improperly escaped. (ticket #98702) #### Profile * Fixed an issue where the confidence score (Conf) displayed values greater than the threshold of 100. (ticket #99636) * Fixed an issue where HTML in data fields was rendered on the Data Preview section of the findings page. (ticket #97883) * Fixed an issue with Data Preview that resulted in an OOM error when the data\_preview table contained a large number of records. #### Rules * Fixed an issue where values on the Rules tab did not correctly display in scientific notation format. (ticket #89738) * Fixed an issue when using a secondary dataset that prevented @dataset for primary dataset from being supported. #### Scorecards * Profile is now removed from the DQ Scorecards submenu. #### Security * Fixed an issue with LDAP external groups to role mappings when there was no fully qualified path for the LDAP group that caused malformed API calls and did not save properly. * Fixed an issue with the Dataset Security feature. (ticket #100317) * When the following security settings are configured, the system fully restricts access to the findings page for admin users: * Dataset security is turned on. * Default owner access is unchecked. * Dataset belongs to no roles or no roles to which the user has access. #### APIs * Fixed an issue with the /v2/gethints endpoint that prevented the Hints table from displaying correctly on the findings page. (ticket #98941) * Fixed an issue with the getRecords and getGenerics APIs that prevented any information from being returned. (ticket #98820) #### Alerts * Fixed an issue with the SQL on the Alert Notifications page that prevented data from appearing in the DataTables error message. #### Agent * Fixed an issue with GKP deployments where job scans failed because the driver pod could not create connections to the metastore. (ticket #102175) #### Validate Source * Fixed an issue where the Source to Target scorecard incorrectly displayed a mismatch because of an unexpected column type checked during a schema order check. (ticket #98300) #### Connections * Resolved connection issues in certain cases by upgrading the Athena driver to version 2.0.33. (ticket #100340) * Fixed an issue where HDFS connections could not rerun a job successfully because certain parameters were automatically appended to the Free Form (Appended) field of the Agent configuration. (ticket #95810) * Fixed an issue with Dremio connection timeouts on Kubernetes deployments. (ticket #101221) * To prevent Dremio connection issues, set the following value in the Free Form (Appended) field of the Agent configuration:`-conf spark.driver.extraJavaOptions=-Dcdjd.io.netty.tryReflectionSetAccessible=true` ### Known Limitations #### Rules * Freeform rules with fully qualified column names are currently unsupported when they use the following syntax:`select FROM @ WHERE @. condition` * A workaround to this limitation is to use aliasing instead. #### APIs * When using the new /v3/rules/copy API, the copied rule automatically appends "copied" to the rule name. After copying a rule, you may need to manually update the rule name. * If the copied rule is performed on a target dataset that does not conform to the compatible columns, then you need to manually update the rule to ensure the columns are compatible across datasets. * Dataset Security is not enforced when using the /v3/rules/copy API. ### DQ Security Metrics ![DQ Security Vulnerabilities over the last 5 months](https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-4b206e177923e9d7b04b6782bf1319677bbdb356%2Fvulns-over-time-2022.12.png?alt=media) ![Critical security vulnerabilities over the last 5 months](https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-37f60112aeba973e276c2e77bad2baa565fb48a8%2Fcritical-table-2022-12.png?alt=media) ### Hotfixes #### Data Quality & Observability (self-hosted) 2022.12.3 * Fixed an issue with Databricks cluster connections to allow `PWD={}` input in the connection string URL. (ticket #104845) * You are still required to enter `PWD={your-text-here}` in the URL. * This fix only applies to Databricks cluster connections, not to Databricks connections that use SQL-endpoints.