Data Transform – Usage and Configurations in Pega
In this blog article, we will see how effectively, we can use a Data transform rule.
This is one of the core concept in Pega development. The screenshots in this blog article are reused from Pega 7 version, but the concepts and the configurations remain the same throughout the Pega versions!
Data transform is a rule that helps in manipulating the data.
Some basics about data storage and usage in Pega
how does an Java application uses the data for processing?
Permanent storage – Applications can use some persistent storage like database to permanently store the application data.
For example – Take an example of an Organization HR application. All the employee details can be permanently stored in the database.
Temporary storage – Applications may need to use those data or objects for processing. To access these data, we don’t need to hit database every time. Instea,d we can load and store the data in the memory to avoid database trips 😊. The temporary storage can be of the heap memory where the Java objects can be stored.
Same like Java application, Pega applications also use temporary memory and permanent storage for application processing. At the end, Pega is built over Java right.
Let’s concentrate only on the temporary memory store.
Let’s take another Java Vs Pega comparison.
In Java application development, we can have class, say Employee as a class. The employee class can have Java objects like emp-1, emp-2, etc… These objects can be stored in the heap memory and can be cached there and access at any point of application processing.
In Pega application. We can have a data class and those classes can have instances (Java objects). The Java object can be compared to a clipboard page and the collection of Java objects can be compared to the Clipboard.
Let’s consider another business use case of the Purchase request case in Amazon organization. For every purchase request, Amazon needs to select a courier vendor to service the customer. Here we can use a Connect-REST to get the list of vendors available for the product – say we got 5 vendors in a page.
Do you think we need to save all the VendorsList in the database?! No. Not required.
Those data are needed only at the time of processing and can be cleared. In this scenario, we save the data only in clipboard memory and you don’t persist the data to DB.
In these situations, we may need to update the data in the clipboard.
Pega provides us with a rule called ‘Data Transform’
We can get the picture from the rule name. Help is transforming or processing data in the clipboard 🙂
Note: You can never persist any data using Data transform rule.
What is Data transform rule?
– It belongs to data model category.
– It is basically mapping data or process data in some target page. The source can be of any page data or just some hard-coded values.
– It helps in data conversion and transformation
Always try to use data transform, when you want to just process the data (no persistent)
Data transform is very easy to understand .
Always remember the property rule in Pega helps to store the data. Properties are embedded to a page. In Data transform rule, we transform the property values.
Let’s straight away create a new data transform rule.
Records – > Data Model -> Create a new data transform
What are the configurations in a Data Transform rule?
In the data transform rule form, you have 2 main tabs.
- Definition
- Parameters
Definition tab
You can see 4 columns in Definition tab.
a) Action – You can take various action to support processing the data.
b) Target – You can specify a target page or property here.
c) Relation – This vary based on the action we perform. We will see that soon 🙂
d) Source – You can specify a source page or property. It can also contain some hard-coded values.
Throughout this post, I am going to explain you how to process data from one page to another page.
What are the pre-requisites?
1. Create the following properties.
Page List property – SourceList, TargetList
Single value property – PolicyName
Value list property – PhoneNumber
Value group property – EmailAddress
2. Create a test activity and populate a page ‘ActivePage’.
Here I populated SourceList page list property with two pages – SourceList(1), SouceList(2)
In each page SourceList(1) & SourceList(2), I populated the PolicyName, PhoneNumber and EmailAddress values.
3. Create a new data transform – CheckDT
4. In the activity step, Use Apply-DataTransform method and refer the CheckDT data transform
Note: we can pass the parameters to the data transform
– In short, I created a new page called – ActivePage in the clipboard and set some properties in the SourceList page list property.
– I am using data transform to process the data.
What are the different actions we can perform in a data transform?
1. Set
This is simple and most commonly used. We use this to set any data.
For example – I need to set the PolicyName in ActivePage or I need to set the CurrentDateTime value
Let’s see how we can do it.
Note: I didn’t use ActivePage.PolicyName, since ActivePage is the primary page for the data transform from the activity call.
- Set the PolicyName to hardcoded value “Automobile”
- Set pyAgeFromDate property value to CurrentDateTime
- Set the PhoneNumber(1) valuelist property to SourceList(1).PhoneNumber(1) valuelist property
Remember Source – SourceList is populated from the activity step.
Note: For valuelist and valuegroup properties, you can directly set the values without using subscript. I mean you can use .PhoneNumber = .SourceList(1).PhoneNumber, to copy the entire valuelist content.
Now run the activity and check if the values are populated in the ActivePage.
Click on the PhoneNumber valuelist property to check the values.
Looks good.
2. Remove
This action help in removing any top level page or property values in the clipboard page.
Note: Removing is totally different from making property value to null
For example – If we don’t need the data in SourceList pagelist property, we can directly remove the property from clipboard.
Relation and source column are of no use
Run the activity and check in the clipboard.
You can see the SourceList populated from the activity is removed from the ActivePage.
Here you can also remove the top level page too.
I am removing the entire page from the clipboard.
Note: This is similar to Page-Remove method in activity.
You can see the page is removed
3. Update Page
– This action helps in updating any page in the clipboard. You can use the relation column to choose the source page.
Select relation = ‘with values from’ to select a source page. You can also leave it blank.
You can see a Loop started. You can go on and update any property in the ActiveTestPage
Here I am going to copy the entire pagelist property from from ActivePage.SourceList to ActiveTestPage.TargetList.
Note: If you want to update primary page, you no need to use this action.
Run the activity and check the clipboard.
You can see the SourceList value from ActivePage is set to TargetList in ActiveTestPage .
4. Apply Data transform
– This action is used to cascade a new Data Transform.
You can call another data transform using this action. Relation & source column are of no use.
You can click on the settings icon to control the parameters we pass to the new data transform.
We can either pass the parameter page or supply the required parameters.
I added a Data Transform – Initialize.
Open the activity and run the tracer.
You can see our Data Transform – CheckDT calls the Initialize data transform.
Using this action, you can reuse any existing data transform
5. Sort
– This action is applicable only for PageList property type.
For example – You hit a Connect-REST service and get the PolicyList from different system.
The service returns the data un-ordered. You need to sort it based on Policy name and display it in section. Here you can use Sort action.
For testing, I modified the test activity to present the data in SourceList un-ordered.
Let’s sort the pagelist in our Data transform.
Specify the pagelist property – SourceList in the target field.
Click on the settings icon and add the Sorting properties.
Note: you can add multiple properties and the order takes the priority in sorting.
Here I am sorting using Policy Name. Run the activity and check in the clipboard.
You can see the pagelist property – SourceList reordered using PolicyName.
6. Comment
– This action is Kind of dummy 😀 . You can use to include some comments that other developers can understand what is happening in the data transform.
It never affects any property. At run time, Pega ignores this action.
7. When, Otherwise When, Otherwise
– You can use these actions to conditionally invoke other actions. I mean you can invoke the actions based on some when conditions.
For Example –
When ( PolicyName = “Home”)
{
IsHomePolicy=’true’
}
Otherwise When ( PolicyName = “Auto”
{
IsAutoPolicy=’true’
}
Otherwise
IsOtherPolicy=’true’
You need to implement the above logic. You can make use of the When, Otherwise When conditions to achieve this.
You can see I used 3 conditionalized actions.
For testing I set PolicyName=”Umbrella”.
Run the activity and check in the clipboard. ‘IsOtherPolicy’ property value must be true.
Yes It is.
You can conditionally perform any actions.
Nested when conditions are also applicable.
8. Append to
Before this, let me refresh you with Page & PageList relation.
– Think about a book. Book contain many pages. A collection of pages can be a book.
– Like the same in Pega, a collection of pages form a page list.
Say we have a PageList – PolicyList and contain 2 results – PolicyList(1) & PolicyList(2)
PolicyList(1) is a separate page, similarly PolicyList(2) is a separate page.
Collection of pages form a page list property.
PolicyList(1) + PolicyList(2) = PolicyList
Now tell me what is dictionary meaning of APPEND?. We are inserting right.
Say you have a Pagelist property with 2 results ( 2 pages)- PageList(1) & PageList(2). When you append a new page to the pagelist property, then you can see the page in 3rd result – PageList(3)
Hope you will understand easily going forward.
‘Append to’ action helps in appending a page to a page list property.
For example: I have a Page list property – SourceList contains 2 Pages. I need to insert a new page to the page list. You can use this action.
Let’s check it.
In relation column, you have 4 options.
a) a new page
No explanation. Only testing 🙂
Step 1: In the activity append 2 pages to the SourceList property as shown below.
Step 2: In the data transform, Use ‘a new page’ in Relation.
Step 3: run the activity and check the clipboard.
You can see a new page – SourceList(3) added, but no values. Yes it is a new page
b) an existing page
Step 1: In the test activity, add a new step to update a page ‘ActiveTestPage’ with Gender = Male
Step 2: Update the data transform
Relation – An existing page.
Source – you can specify the newly created existing page ‘ActiveTestPage’
Here we are going to append ActiveTestPage page with SourceList property.
Step 3: Run the activity and check the clipboard.
You can see a new page appended with values from ActiveTestPage.
c) Current source page
What is current source page?
I will open the tracer for you. You need to check it.
Open the tracer, run the activity.
So ActivePage.SourceList(3) is appended with which source page? – Active page. Which is the primary page context here.
You can verify it in the clipboard.
d) Each page in
Till now we saw appending a single page to a pagelist property. What about appending a target pagelist with source pagelist.
Step 1: Update the data transform.
Step 2: run the activity and check in the clipboard.
You can see source pagelist is appended with target pagelist.
Note: For Append to action – both the source and target should be in same class.
9) Append to and Map to
First let’s see the difference between Append to vs Append to and Map to.
– The target and source can be of different applies to class.
– Append to and map to create a new page without any embedded properties, while Append to action creates a new page with all the values from source page.
– You will have a child set action to perform with Append to and Map To
Note : Both action don’t work for page group properties. Only for pagelist property.
We use this mainly to copy data from one page to another page ( different applies to class)
Relation column is same as Append to action. You can see loop for child set action.
You can set properties using child set action.
First, let’s test it.
Step 1: Update the data transform.
Step 2: run the activity and check in the clipboard.
You can see the pages appended from source without any values. The value we set in data transform is alone reflecting
10) For Each page & Exit for Each
For each page – You can use this action to loop a page list or page group property and update the values.
Exit for each – This action exit the remaining iterations in the loop.
Note: Exit For Each is not same as exit iteration you use in activity transitions.
Here in the above example, when Policy name = “Home”, we exit remaining iteractions using Exit for Each action.
You can see an option – ‘Also use each page as source context’ in the source column! Why do we use it?
From the name you can say 🙂
For example – You need to update properties within the page – Means target and source are of same page.
– In SourceList(1) you have PolicyType = “Home”. If you need to set the PolicyType value to PolicyName in SourceList(1).
– In this case, you can check the option and set it directly.
In some scenario, you need to set PolicyName to the property available in the class, from where we call the data transform ( primary page). here you can uncheck and make the Source column page context as primary context. 🙂
Open the tracer & run the activity.
You can see when PolicyName=”Home”, Exit For each occurs and the remaining Iteration stops.
Only SourceList(1) is iterated.
12) Exit Data transform
– This action, ends the data transform abruptly. The other steps below are not executed.
Let’s check if step 2 is getting executed.
Open the tracer and run the activity.
You can see the data transform ends when it encounters Exit data transform.
All actions over Pardon me If I dragged some much
One more key area is Calling SuperClass data transform
First think of what is Super class?
Super class is parent class. Your mind might me thinking about inheritance now right.
To check on the class inheritance path
App explorer -> Open the class -> right click -> Inheritance
You can see the class path till @baseclass.
Call superclass data transform
When this is checked, Pega checks the data transforms with the same name in parent class.
Say in my scenario, my Data Transform ‘CheckDT’ is in ‘TVS-TeslaVeh-Work’ class.
Here Pega checks the data transform in the parent classes.
Say it found two data transforms in same name CheckDT in ‘Work’ & ‘@baseclass’ ( Call superclass DT in work checked)
Here Pega executes the data transform in the following order
@baseclass – > Work- -> TVS-TeslaVeh-Work
Let’s test the Superclass invocation
Step 1: Create a new data transform in one of its parent class, say ‘TVS-TeslaVeh’.
I set pyLabel = ‘SuperClass’
Step 2: Check the call superclass data transform in CheckDT data transform in TVS-TeslaVeh-Work class.
Step 3: Open the tracer and run the activity.
You can see both the data transform gets executed, first parent (super class) then child.
When do we use this?
a) Mainly used when you create a new work item. You can see pyDefault data transform gets called in many classes.
This is because, you need to use some OOTB data transform ‘pyDefault’ in Work-, @baseclass to set some default values
I am going to trace the process, when I create a new purchase request case.
You can see the data transforms. Let’s go from the bottom.
- @baseclass pyDefault – OOTB Data transform
- Work- pyDefault – OOTB Data transform
- Work-Cover- pyDefault – OOTB Data transform
- TVS-TeslaVeh-Work-ServiceRequest pyDefault
First Let’s check the pyDefault in ‘TVS-TeslaVeh-Work-ServiceRequest’ class.
You can see the Call superclass data transform is checked.
Let me show you the pyDefault in Work- class. OOTB data transform
You can see some standard properties like pyLabel, pyFlowName etc are set.
You can override the values in the data transform specialized in class ‘TVS-TeslaVeh-Work-ServiceRequest’
b) When you use solution frameworks in your application, Your class structure will be inherited from solution framework class structure.
Here you may need to make use of data transforms ‘pyDefault’ in solution framework classes.
End of data transform post 🙂
