Publishing dataset and field level lineage

With this latest release, Qlik Cloud Catalog now automatically provides dataset and field lineage within the context of a Data Integration pipeline, bringing more transparency and enhancing trust in data-driven outcomes for data and analytic users. Additionally, impact analysis on the data pipeline artifacts facilitates proactive exploration of what-if scenarios, allowing for a comprehensive assessment of how changes in the source system may impact downstream data pipelines and transformations. This proactive approach aids in the prevention of potential issues before they have a chance to arise.

Working with lineage and impact analysis in Data Integration

New features and enhancements in Qlik Cloud Data Integration

The following improvements require Qlik Data Gateway – Data Movement 2023.5.10 or later. For information about how to upgrade, see Upgrading the Data Movement gateway installation.

Expanded Support for Private Connections

A private connection can be used to ensure your data traffic remains secure and compliant. It simplifies both network management and security of your VPC (Virtual Private Cloud) without the need to open inbound firewalls ports, proxy devices or routing tables. Qlik Cloud Data Integration already supports private connections to Snowflake, Microsoft SQL Server and Amazon S3 data pipeline targets. With this release, customers can additionally utilize private connections when moving data to Databricks, Microsoft Azure Synapse, Google BigQuery, and Amazon Redshift.

PostgreSQL-based data sources

• All PostgreSQL-based data sources – Multirange data type support: The following multirange data types are now supported with all PostgreSQL-based data sources (on-premises and cloud).

  • INT4MULTIRANGE
  • INT8MULTIRANGE
  • NUMMULTIRANGE
  • TSMULTIRANGE

• AWS Aurora Cloud for PostgreSQL data source – Non-superuser role support: The user specified in the PostgreSQL connector no longer needs to have the superuser role to move data from an AWS Aurora Cloud for PostgreSQL data source. This is especially useful for organizations with corporate security policies that prevent them from granting superuser access to non-privileged users.

Microsoft Azure SQL (MS-CDC) data source: Geo-replication support

Customers moving data from a Microsoft Azure SQL (MS-CDC) data source can now use a Geo Replica database.

Oracle data source: HSM (Hardware Security Module) support

Qlik Cloud Data Integration now supports reading data from Oracle encrypted tablespaces and encrypted columns during CDC.

New regions support

Qlik Cloud Data Integration now supports tenants in DE and UK regions.

Newly supported versions and third-party software

This section lists newly supported database and driver versions.

• Newly supported data source versions. The following data source versions are now supported:

  • Microsoft SQL Server 2022
  • Oracle 21c
  • PostgreSQL 15.x
  • DB2 13.1 (when working with IBM DB2 for z/OS)
  • IBM DB2 for iSeries 7.5

• Newly supported target data platform versions. The following data target versions are now supported:

  • Databricks (Cloud Storage): Databricks 12.2 LTS and Databricks SQL Serverless

• Driver versions. The following ODBC driver versions are now supported:

  • IBM Data Server Client 11.5.8 for IBM DB2 for z/OZ and IBM DB2 for LUW
  • Simba ODBC driver 3.0.0.1001 for Google Cloud BigQuery
  • MySQL ODBC Unicode Driver 64-bit 8.0.32

Microsoft SQL Server driver – Required upgrade

Customers moving data to or from Microsoft SQL Server need to upgrade their SQL Server ODBC driver version to 18.x or later. Note that continuing to use SQL Server ODBC Driver 17.x might result in data errors. Upgrading the driver can be done using either the driver installation utility or manually. For instructions, see Driver setup.

End-of-support database versions

This section provides information about end-of-support database versions.

• End-of-support data source versions. Support for the following data source versions has been discontinued:

  • PostgreSQL 10
  • MySQL 5.6
  • MariaDB 10.3

• End-of-support target data platform versions. Support for the following data target versions has been discontinued:

  • Databricks 9.1 LTS

Updates to Migration Center

There is now an updated version of the Qlik Cloud migration tools. In addition, three use cases have been added for importing Qlik Sense Client-Managed apps to Qlik Cloud.

Setting up the Qlik Cloud migration tools

Migrating the apps