Qlik Data Flow: Simplifying Data Transformation Without Code

Want to transform and prepare your data but don’t know how to write Qlik script? With Qlik Data Flow, you can! This new feature enables you to transform data using a no code visual editor. Its intuitive drag and drop interface simplifies data preparation, making it accessible even if you are not a technical user. With Data Flow, you can clean, join, and reshape your data before loading it into your applications, all without writing a single line of code.

In this blog post, we’ll explore what Qlik Data Flow is and why you might want to use it. Then, we’ll walk through a practical example. Finally, we’ll review some of the benefits and limitations of Data Flow and compare it to Qlik Data Manager and Qlik Script.

Why Use Qlik Data Flow?

So, why choose Qlik Data Flow? For many business users, Qlik scripting can be intimidating, especially for those without prior experience in data transformation. Qlik Data Flow introduces a visual editor that simplifies this process, allowing users to build transformation steps using predefined building blocks, no coding required!

Qlik automatically generates the script for you as you build your transformations. This means you don’t need to write a single line of code, yet you still get the power and flexibility of Qlik’s data transformation capabilities.

Let’s dive in with a practical example. We’ll create a simple data flow that aggregates sales data by office and year.

Building a Qlik Data Flow: Step-by-Step

Step 3: Joining Sales and Office Data

To analyze sales by office, we need to join the FactSales and DimOffice datasets. From the Processors tab, we select the Join processor. Connecting datasets is as simple as dragging a line between them. Next, we specify the join type and key fields.

Step 4: Extracting the Year from Sales Data

Since we want to aggregate sales by year, we need to extract the year from the %Date_KEY field. This can be done using the Dates processor.

Step 5: Aggregating Sales Data

Now that we have the sales year, we can aggregate sales by office and year. Using the Aggregate processor, we select the Office and Year fields, then choose SUM as the aggregation operation.

Step 6: Sorting the Data

To organize our dataset, we use the Sort processor. While this step is not mandatory, it makes the data preview more structured.

Step 7: Defining the Output File

The final step is selecting an output file type and location using the Data Files Target processor. In our case, we store the result in a QVD file.

Previewing Data in Qlik Data Flow

In just a few steps, we created an aggregated dataset. We did all this without writing a single line of code! The visual representation of each transformation makes it easier to understand what’s happening compared to traditional Qlik scripting or Qlik Data Manager.

Even for experienced Qlik users, Data Flow has advantages. One major benefit is the ability to preview data at each step, helping users validate their transformations. For example, selecting the OfficeSales join processor and pressing Preview allows us to see that our datasets have been successfully combined.

Viewing the Generated Qlik Script

Ultimately, the efficiency of the script depends on the goal. If the goal is clarity for low-code users, Data Flow is a great tool. If performance optimization for large datasets is the priority, a custom Qlik script might be preferable.

Qlik Data Flow vs. Data Manager

Qlik Data Flow and Qlik Data Manager both play essential roles in the Qlik ecosystem, but they serve different purposes. With its no-code, visual interface, Qlik Data Flow empowers users to transform, aggregate, and prepare data before it’s loaded into an app. Users can clean, reshape, join data sources, and create calculated fields. Data Flow focuses on getting the data ready at the source level, ensuring it’s primed for analysis once inside the app.

In contrast, Qlik Data Manager, while offering some basic transformation features, isn’t designed for complex aggregations or conditional logic. Instead, it focuses on managing the data model within the Qlik app after the data is loaded. While Data Flow prepares the data before it’s imported, Data Manager takes care of the relationships between tables once the data is in the app. It helps define associations, manage relationships, and maintain the structure of the data model.

For those looking to stay within a no-code environment, both tools are indispensable: Data Flow simplifies data transformation, while Data Manager maintains the structure and relationships of your data, ensuring everything is set up for accurate analysis.

To help you better understand the differences between the available transformation options, we’ve put together this comparison:

Feature	Qlik Data Flow	Qlik Data Manager	Qlik Script
Learning Curve	Beginner-friendly, no coding required	Easier for data model management but limited in transformations	Steep learning curve, requires coding expertise
Readability	Step-by-step visual representation	Clear for managing associations, but lacks transformation transparency	Code can be complex and hard to follow, especially for large models
Performance	Efficient for pre-processing large datasets	Handles small transformations, but not optimized for complex processing	Optimized for complex data processing, but performance depends on script quality
Flexibility	Supports complex joins, aggregations, and transformations	Limited transformation capabilities	Extremely flexible, supports custom logic and advanced transformations
Data Integration	Connects to QVDs and external files	Manages data relationships within a Qlik app	Connects to various data sources, but requires custom scripts for integration
Preview & Debugging	Allows preview at each step with auto-generated script	No real-time transformation preview	Requires running the script to view results, debugger available

Limitations of Qlik Data Flow

While Qlik Data Flow is a powerful no-code solution, it has some limitations:

• Limited customization for CSV and Excel files: No option to adjust headers or handle complex formatting issues.

• No advanced scripting capabilities: Certain operations, like loops, subroutines, and complex conditions, require Qlik scripting.

• Limited flexibility for iterative calculations: More complex data transformations may require manual scripting.

For well-structured data sources like QVDs or Parquet files, Qlik Data Flow is an excellent tool. However, for advanced transformations requiring custom expressions or iterative logic, traditional Qlik scripting remains the best option.

Conclusion

Qlik Data Flow makes data transformation accessible to everyone, from business users to Qlik experts. With its no-code, visual interface, users can easily clean, join, and reshape data while still benefiting from auto-generated Qlik scripts.

While it’s a great tool for simplifying data preparation, some complex scenarios still require Qlik scripting. That said, for most users, Qlik Data Flow is a game-changer, enabling faster, more intuitive data transformation, without the need for coding expertise.

More from the Bitmetric team