Tuesday, 18 November 2014

Using tJavaRow - input_row and output_row (for row by row transformation)


Using tJavaRow - input_row and output_row (for row by row transformation)

In this post i will like to explain a very simple and basic usage of tJavaRow.
Lets say you need to do some transformation on every row of your input data. You can do this by using tMap, tJavaFlex, tJavaRow and others.
for now i will focus on tJavaRow. Its a good practice to use input_row and output_row as objects which can be used to fetch values of your columns. 

using tJavaRow

Fixedflowinput returns two rows and we pass each row via tJavaRow and append text "-XYZ" to name and finally pass this as output to tLogRow.
code of tJavaRow...notice the usage of input_row and output_row here..

output_row.id = input_row.id;
output_row.name = input_row.name+"-XYZ";






Tuesday, 2 September 2014

Implement Insert (for new) and update (for existing) strategy in talend (tmap update/insert/reject)


Implement Insert (for new) and update (for existing) strategy in talend

Most of times in dwh/etl/data integration loads we have incoming source data feeds and we have to merge (update/insert) this with existing data in target table.
There are multiple ways to do this, i will explain one of the method i have used.

You have a source table - say srcemployee
you want to merge this data after doing lookup with target - tgtemployee
keys for lookup is say empid

all we need is source input, target input(lookup) tmap and toutput
in tmap we would join source input with target input on key column and select join options as all matches inner join. In tmap output we will have two groups once for handling matching records and other for handling rejects (records which were not satisfying the join condition) these are insert.

overall job (tmysqloutput_1 insert, tmysqloutput_2 is for update)



schema of input srcemployee

schema of input srcemployee


tmap join condition
tmap join condition

mention key for update in schema for tmysqloutput_2
mention key for update in schema for tmysqloutput_2