When transferring data from a source system to a target/destination system, there is often a difference in the data schema, particularly in field naming. For example, in a CSV input file, a column could be named Address, while in the target system, the corresponding field might be called Primary Address.
Data mapping can be seen as the lifeblood of any data integration process. Data or Field mapping is simply the process of extracting data fields from one source file and matching them to their related target fields in the destination file. Data integration or ETL mapping helps consolidate data by extracting, transforming, and loading it to a target database which may be a relational database or a CSV document.
In traditional programming, a mapping function can be made to convert from one data structure to another, perhaps using something like AutoMapper, if C# has been used.
In Synatic, there is a Super Mapper Step to help with this task. The Super Mapper is similar to the Field Mapper step in that it can map from a source field name to a target field name, but the difference lies in the more advanced functionality that the Super Mapper step has available. The Advanced functions can apply JavaScript, Expressions, Parsing, and explicit values to the source data fields as well as formatting field values using the Handlebars template language.
Watch the video below to learn how to add the Synatic Super Mapper step and configure the data mapping table.
Adding the Super Mapper Step
To add a Super Mapper step in a flow, follow the below instructions:
1. Click on the highlighted icon as shown in the below image.
2. The below page will appear. Select or search for a step to pull out the source data. Select Super Mapper as shown in the image below.
Configuring the Super Mapper Step
Illustrated in the below image is the configuration that is available in the Super Mapper step.
Mapping Table
The mapping table has two main columns, the Source column and the Destination column, and are connected with map lines in the Connections section.
Source Column
Add Source
Click the Add Source button to add a new field to the Source column. Click on the Path drop-down arrow to select and select the mapping value type of the source field as outlined in the image below.
Mapping Types
There are six mapping types to select, which let you configure each source field using advanced operations. The resultant value of the field will be applied to the associated destination record.
Path
Enter the name of the source field. The value in the source field will be applied to the destination field as shown in the image below.
The image below shows the outcome after the Super Mapper step.
Value
Enter a fixed/default value for this field that will be applied to the destination field as shown in the image below.
The image below shows the outcome after the Super Mapper step.
Expression
Enter a mathematical expression to calculate a value for this field and applied to the destination field as shown in the image below. The quotation number is multiplied by 10. In this record, the original quotation number was 100. The new number will be displayed in the ConvertedQuote destination field.
The image below shows the outcome after the Super Mapper step.
Parser
Use the Parser to input data (usually text) and build a structured string value for this field that will be applied to the destination field. Use the in-built Synatic handlebars text templating language to perform complex operations and return and apply a string value. In the example below, the ID field is being parsed.
The image below shows the outcome after the Super Mapper step.
Javascript
Enter some JavaScript code for this field, the result of which will be applied to the destination field as shown in the image below. The Flower field is being converted to upper case.
The image below shows the outcome after the Super Mapper step. The flower field is in the Upper Case.
Remove
Select Remove to delete the Source field or operation from the map as shown in the image below. The Comment field is being removed. This does not require a destination field because the field is being removed.
The image below shows the outcome after the Super Mapper step. The Comment field is removed.
Preserve Record checkbox
The values will be mapped to the current record and a new record will not be created when the checkbox is ticked. This is shown in the example below. the new fields are added to the current record.
Import Fields
Click the Import Fields option to open an Import dialog and enter the fields (separated by commas) or paste a prepared list of fields (separated by commas) to import.
Click + Import Fields to execute the import process as outlined in the image above.
Export Fields
Click the Import Fields option to export the source fields and save them on a local machine. The file is named “source-mappings” by default when exporting, as outlined in the image below.
Remove All Sources
Click the Remove All Sources option to remove all sources from the column immediately. Once clicked, this cannot be undone.
Extra options
Click on ⨂ adjacent to each field to delete the field.
Select ↕ adjacent to each field to drag it to another position in the order.
Connections
Connections are lines drawn between a Source field and its associated Destination field, which may or may not be on the same adjacent level.
Manual
To connect fields manually, click and hold the Ο next to the source field and drag a line to the Ο next to the destination field. The lines will attach when you release the hold.
Auto Map
Click the Auto Map option to automatically create connection lines between Source and Destination fields that have the same name.
Order Connections
Click the Order Connections option to automatically align each field in the right-hand column with its associated field in the left-hand column.
Remove All Connections
Click the Remove All Connections option to delete all the connections between fields.
Destination Column
Add Destination
Click the Add Destination option to add a new field to the Destination column. Click the highlighted Data Type drop-down arrow to select and configure the data type of the source field.
There are two option used to configure the destination field:
Default Value - Enter a default value to this field if no other values are mapped to the destination field.
Data Type - Select one of the available data types to align the field with its type.
Import Fields
Click the Import Fields option to open an Import dialog and enter the fields (separated by commas) or paste a prepared list of fields (separated by commas) to import.
Click + Import Fields to execute the import process as outlined in the image above.
Export Fields
Click the Import Fields option to export the source fields and save them on a local machine. The file is named “destination-mappings” by default when exporting, as outlined in the image below.
Remove All Destinations
Click the Remove All Destinations option to remove all destinations from the column immediately. Once clicked, this cannot be undone.
See Also
Read more about Field Mapper.
Useful Tutorials
Revisit the Getting Started page for a refresher.
Read more about Steps here.