Skip to main content

Building a Transformation Pipeline

One of the most powerful features of Saddle Data is the ability to chain multiple in-flight transformations together. This guide will show you how to build a simple yet practical transformation pipeline.

Let's say we have a users table with the following columns:

  • id
  • name
  • email
  • is_active (boolean)
  • created_at

Our goal is to create a new table in our destination that only contains the id and email of active users, with the email column renamed to user_email.

Here is the pipeline we will build:

  1. Schema Mapping: Select only the id and email columns and rename email to user_email.
  2. Filter Rows (Transformation): Keep only the rows where is_active is true.

Building the Flow​

  1. Configure Mapping: In the Schema tab, select the users table. Uncheck all columns except id and email. In the destination mapping for the email column, change the name to user_email.
  2. Add Filter: In the Design tab, add a Filter Rows transformation to your pipeline. Configure it to keep rows where is_active equals true.

When this Flow runs, Saddle Data will first filter the data according to your transformation rules and then apply the schema mapping, resulting in a clean, targeted table in your destination.