> For the complete documentation index, see [llms.txt](https://docs.collibra.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.collibra.com/data-quality-self-hosted/release-notes/data-quality-observability-self-hosted-release-archive/release-2022-04.md). # Release 2022.04 #### Install {% hint style="info" %} For standalone installations, within the setup.sh script find/replace the variable for **spark\_package**.Change spark-3.0.1-bin-hadoop3.2.tgz to spark-3.1.2-bin-hadoop3.2.tgz {% endhint %} ``` spark_package=${SPARK_PACKAGE:-"spark-3.0.1-bin-hadoop3.2.tgz"} # replace with spark_package=${SPARK_PACKAGE:-"spark-3.1.2-bin-hadoop3.2.tgz"} ``` #### Fixes / Enhancements * DQ Job * Entering negative values for the downscore is no longer supported and will now produce an error message. * You can now invalidate schema with special characters. * Spark table names of historical dataset loaded and other spark tables are now available on Jobs Log table. * Long type values larger than `Integer.Max` no longer breaks the Profile. * View Findings now displays user's full name, if applicable, in Validate Modal. Assignment queue page also displays the full name of user, if applicable. * Alerts * You can once again use the Cancel action button on the Alerts page. * You can now set up alerts to reach multiple email recipients. * If email\_server table is not yet configured, a helpful message will now display in the Description column in the job log directing you to register an email Server under Admin - Alerts. The job will still run successfully. * Rules * You can now modify Rules definitions from the primary DQ Job dashboard without loading the Rules page. * Mean value check once again triggers correctly for Integer and Long columns. * This fix triggers the mean value check for Integer and Long columns and shows an infinity percentage change in behavior for a period, depending on -bhlb. After this period, it should disappear. * For Native SQL rules, jobs now behave the same whether or not a semicolon ";" is included in the SQL query. * You can now use a hyphen "-" in a dataset name. * Acceptable special characters now include a hyphen "-", period ".", and underscore "\_". * Added a tooltip that displays which condition is being checked in a DQ Job when using a Stat rule when you hover your cursor over a condition in the Condition column. * Improved the exception message for when there are no values for a specific column while using a Stat rule. * The WebUI passing boundaries range has been updated to (). * For Freeform rules, IS Null and IS NOT NULL no longer return invalid results in the Validation tab. * Added a pop-up success message for when the correct syntax rule passes for Freeform rules with secondary datasets after the Validate button is clicked. * Security * Vulnerabilities identified by Jfrog * Vulns 2, criticals 2, high vulnerabilities * For a visual readout, see the DQ Security Metrics section below. * Authorization restriction is now enforced for the following endpoints: * /v2/deletefiledir * /v2/getRunIdsByDataset * /v2/putDatasetWeight * /v2/checkListofFilesPath * /v2/getlistagents * /v2/checkDriver * /v2/getconnectionssensitive * /v2/getemailgroups * /v2/getemailserver * /v2/addemailgroup * /v2/validateEmailAddress * /v2/getlistoffiles * /v2/getlistoffilespath * /v2/getlistoffiles * /v2/getDriverDir * /v2/getlistrolesbydataset * /v2/getlistrolesbydistnctdatasets * /v2/getlistrolesbyfunctiontypename * /v2/getlistusersbyauthority * /v2/getlocalDBRoles * /v2/getsecuritysettingsbytype * /v2/getowlcheckinventory * /v2/getconnectionspwdmgrsensitive * /v2/getsecuritysettingsbycoltype * /v2/getdbuserlist * /v2/getdbuserdetailsbyuser * /v2/getexternaladgroupstointernalroles * /v2/getlistdatasets * /v2/getlistdatasetsbyrole * /v2/getaudittrailitems * /v2/get-all-audit * /v2/get-datasets-audit-trail-items * /v2/get-all-dataset-audit * /v2/getactivityaudit * /v2/getallactivityaudit * /v2/getlocaldbrolesbyuser * /v2/getdatasetaclsecurity * /v2/getexternaladgrouplist * /v2/getexternaladuserlist * /v2//external-service-configuration * Local user accounts now have an account lockout feature implemented with the following restrictions: * A user's account will be locked if a password is entered incorrectly more than 10 times. You can configure this by specifying an integer value for the Admin Console Application Configuration setting **USER\_MAX\_INVALID\_ATTEMPTS**. * The locked account can only be unlocked by Admin user in user management screen. * If an Admin is locked, another Admin can unlock their account. * If all the Admins are locked, enable the account via DB (update users table "accountNonLocked" column to "1"). * User cannot use forgot password to reset password while the account is locked. * CORS restriction is now enforced for SAML and multi-tenancy. * This breaks SAML unless the IDP is configured as a trusted origin in DQ, so the following property must be added to environment variables in order for DQ and SAML to work: CORS\_ALLOWED\_ORIGINS=\\({IDP-BASE-URL},\\) * Replace $ with the value of the actual IDP URL (For example: ) * Replace $ with the value of the actual DQ Base URL (For example: ) * SAML login no longer automatically triggers on the login page during an existing session when accessing DQ base URL. For SAML login, you should instead use /saml/login. * API requests (v2/v3) return proper JSON response in case of failures. * auth/signin API is updated to provide JWT token for MT & local users. * Profile * Mean value once again displays in the Volume column. * When connecting to MSSQL server on Windows from a Linux DQ environment, the connection no longer fails. * We recommend (not required) a TLS connection for MSSQL connections from a DQ Linux environment with a properly signed certificate setup on MSSQL server to connect only via TLS. * You can now edit annotations in the Labels tab. * S3 * Added an enhancement for -addlib flag. * Connections * Added new Jconn4 driver for encrypted connections. * Tech Preview - You can now save a local (NFS) file directory as a connection type. * See our newest [Supported Connections](/data-quality-self-hosted/connections/supported-connections.md) page for a definitive guide to driver support. * BigQuery is now certified for production, but removed from packaged install for K8s docker. * Explorer * When toggling between fullfile and Union LookBack options, `-fullfile` and `-fllb` flags can no longer be generated together in the DQ Job command line. * Data Preview for Temp files loading in Explorer now correctly shows the order of columns of the original Temp file. * You can now drill in and search files within the connection. * You can now browse multiple local (NFS) file connections. * Scorecard * You can now create scorecards with special characters "^\[A-Za-z0-9]+$" in their names. * Dupes * Added linkID column for exact match in both UI and REST API. linkID can now be either included or excluded from Dupes for exact match. * linkID is now shown at the aggregate level for Exact Match. * We recommend using this feature from a primary key perspective for its first iteration. * The aggregate function used is min(). * For example: if you have 6 occurrences, you will get 1 example linkID, the min. * API * Updated the /v2/getlistdataschemapreviewdbtablebycols API call method from GET to POST to support the long query (-q) or very large columns table. * Added a new SAML load balancer so the syestem picks the appropriate schema and SAML server URL for Swagger. #### Known Limitations * Profile * Special characters are not currently supported in annotations in the Label tab. * Scorecard * Space " ", underscore "\_", and period "." are not yet supported for scorecard edit.