Support LOAD and STORE of Parquet files with nested data

Qlik associative engine now includes support for loading and storing to nested fields with a parquet file. This support is available when loading and string data using Qlik Sense and the script editor.

Working with Apache Parquet files

New filters for monitor views: “Source type” and “Data gateway”

“Source type” filter

Using the Source type filter, it is now possible to create monitor views filtered on tasks with specific source technologies (e.g. SAP and SQL Server landing or replication tasks).

“Data gateway” filter

Using the Data gateway filter, it is now possible to create a monitor view to see all tasks that rely on the selected Data Movement gateway. This helps to simplify operations of tasks on a specific gateway such as upgrading. From the monitor view it is possible to stop all tasks on the gateway prior to upgrading, after the upgrade is complete you can then review the tasks to ensure they are all operational.

Monitoring and operating your data tasks

Additional settings and updates to existing settings

New settings

Qlik Cloud Data Integration landing tasks are the first step of transferring data into your cloud data warehouse, or of landing data directly into your data lake. This release provides additional configuration options to control the behavior of the landing task and improve performance, including:

• Control tables

  Control tables provide additional status information in the target data platform such as the landing status, suspended tables and the task history, including the number and volume of records processing during landing.

• Error handling policies

  Error handling policies allow you to select what you want to happen when a data truncation or other data errors occur, such as Ignore errors, Suspend table or Stop the task.

Changes to setting locations and names

We have also made changes to a few of the existing options within the landing and replication settings to help make them more intuitive.

Renamed tabs

The Runtime tab has been renamed to SaaS application to reflect the fact that its settings are only relevant when moving data from a SaaS application source. For Replication projects, the Replication mode tab has been renamed to Task modes and now has two sub-headings: Replication mode and Apply changes mode.

Relocated and renamed settings

The following changes apply to landing tasks only:

• The Change processing interval option was moved from the SaaS application (previously called Runtime) tab to the General tab.

The following changes apply to both landing tasks and “Land data in data lake” tasks:

• The Parallel execution option has been renamed to Maximum number of tables to load in parallel and moved to the Full load tab.
• The LOB option has been renamed to Include LOB columns and limit column size to (KB) and moved to the Metadata tab.

Integrating data