Monday, 16 January 2012

Talend - Use custom sql to update database table

Problem: We wanted to update database table based on some conditions which included where clause with < and > conditions on some column.

Solution: We used tjdbcrow/tmssqlrow component for this.
(here inputrowschema - is the input row schema name to tjdbcrow/tmssqlrow component)

"update tablename set text_column1 = '"
+inputrowschema.columnname1
+ "'
+ " WHERE number_key_col = "
+inputrowschema.key_col
+ " AND "
.....
......

Talend - jdbc generic dateformat using talend/update datetime in sql server and oracle using same tjdbcrow component

Problem:- Update datetime in database tables using tjdbc components. We want our solution to be database independent, therefore we could not use sysdate (oracle) and getdate() (sql server) and similar database functions to get date and also while writing date back to database using tjdbcrow (custom update sql) we faced date format issue. What is way to use generic date format in tjdbc custom sql statement.

Solution
Solution is not specific to talend but to the JDBC drivers support timestamp literals in SQL statements written in the format: {ts 'yyyy-mm-dd hh:mm:ss.f...'. It also supports only date and only time format also.
For us we needed datetime update and we used following in our update statement in our tjdbcrow component to give us generic date

"update tablename set dt_column_name="
+"{ts '" + TalendDate.formatDate("yyyy-MM-dd HH:mm:ss.SSS",TalendDate.getCurrentDate()) + "'}"


this is the sql generated by talend to be processed by jdbc drivers-
update tablename set dt_column_name={ts '2012-01-16 06:54:37.525'}

Thursday, 5 January 2012

Talend CommandLine Utility to export jobs from svn into a .jar /.sh / .bat execuatables fails with xception in thread "Code generation background thread" "Generation Engine Initialization"

Issue
we use commandline utility from talend to export jobs from SVN into a .jar /.sh / .bat execuatables. This process is running for last 5 months successfully, but all of a sudden our exportJob process started failing with following error.
Exception in thread "Code generation background thread" java.lang.Error: Unresol
ved compilation problems:
        The import org.talend.designer.codegen.config cannot be resolved
        The import org.talend.designer.codegen.i18n cannot be resolved
        CodeGeneratorArgument cannot be resolved to a type
        CodeGeneratorArgument cannot be resolved to a type
        Messages cannot be resolved
        Messages cannot be resolved
        Messages cannot be resolved
        Messages cannot be resolved
        Messages cannot be resolved
        Messages cannot be resolved

        at org.talend.designer.codegen.translators.common.HeaderJava.<init>(Head
erJava.java:8)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct
orAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC
onstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at java.lang.Class.newInstance0(Class.java:355)
        at java.lang.Class.newInstance(Class.java:308)
        at org.eclipse.emf.codegen.jet.JETEmitter.setMethod(JETEmitter.java:291)
        at org.talend.designer.codegen.model.CodeGeneratorEmittersPoolFactory.in
itializeEmittersPool(Unknown Source)
        at org.talend.designer.codegen.model.CodeGeneratorEmittersPoolFactory.ac
cess$7(Unknown Source)
        at org.talend.designer.codegen.model.CodeGeneratorEmittersPoolFactory$Jo
bRunnable.doRun(Unknown Source)
        at org.talend.designer.codegen.model.CodeGeneratorEmittersPoolFactory$Jo
bRunnable.run(Unknown Source)

!ENTRY org.eclipse.core.jobs 4 2 2011-12-29 08:54:54.152
!MESSAGE An internal error occurred during: "Generation Engine Initialization in
progress...".
!STACK 0
java.lang.NullPointerException
        at org.eclipse.core.internal.jobs.Worker.run(Worker.java:71)
FAILED at Thu Dec 29 08:54:54 EST 2011
exception : org.talend.commandline.client.command.CommandStatusWrapException: ja
va.lang.Exception: java.lang.NullPointerException

Solution - 
Redeploy (re-extract) commandline utility into new folder from your talend software zip and use this new commandline location to run export job.Here redeploy means that in our linux environment which hosts the commandline utility, we simply extracted the commandline folder (which includes commandline-workspace) again from our talend software zip.