Showing posts with label Talend commandline export talend job scripts initRemote logonProject. Show all posts
Showing posts with label Talend commandline export talend job scripts initRemote logonProject. Show all posts

Thursday, 5 January 2012

Talend CommandLine Utility to export jobs from svn into a .jar /.sh / .bat execuatables fails with xception in thread "Code generation background thread" "Generation Engine Initialization"

Issue
we use commandline utility from talend to export jobs from SVN into a .jar /.sh / .bat execuatables. This process is running for last 5 months successfully, but all of a sudden our exportJob process started failing with following error.
Exception in thread "Code generation background thread" java.lang.Error: Unresol
ved compilation problems:
        The import org.talend.designer.codegen.config cannot be resolved
        The import org.talend.designer.codegen.i18n cannot be resolved
        CodeGeneratorArgument cannot be resolved to a type
        CodeGeneratorArgument cannot be resolved to a type
        Messages cannot be resolved
        Messages cannot be resolved
        Messages cannot be resolved
        Messages cannot be resolved
        Messages cannot be resolved
        Messages cannot be resolved

        at org.talend.designer.codegen.translators.common.HeaderJava.<init>(Head
erJava.java:8)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct
orAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC
onstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at java.lang.Class.newInstance0(Class.java:355)
        at java.lang.Class.newInstance(Class.java:308)
        at org.eclipse.emf.codegen.jet.JETEmitter.setMethod(JETEmitter.java:291)
        at org.talend.designer.codegen.model.CodeGeneratorEmittersPoolFactory.in
itializeEmittersPool(Unknown Source)
        at org.talend.designer.codegen.model.CodeGeneratorEmittersPoolFactory.ac
cess$7(Unknown Source)
        at org.talend.designer.codegen.model.CodeGeneratorEmittersPoolFactory$Jo
bRunnable.doRun(Unknown Source)
        at org.talend.designer.codegen.model.CodeGeneratorEmittersPoolFactory$Jo
bRunnable.run(Unknown Source)

!ENTRY org.eclipse.core.jobs 4 2 2011-12-29 08:54:54.152
!MESSAGE An internal error occurred during: "Generation Engine Initialization in
progress...".
!STACK 0
java.lang.NullPointerException
        at org.eclipse.core.internal.jobs.Worker.run(Worker.java:71)
FAILED at Thu Dec 29 08:54:54 EST 2011
exception : org.talend.commandline.client.command.CommandStatusWrapException: ja
va.lang.Exception: java.lang.NullPointerException

Solution - 
Redeploy (re-extract) commandline utility into new folder from your talend software zip and use this new commandline location to run export job.Here redeploy means that in our linux environment which hosts the commandline utility, we simply extracted the commandline folder (which includes commandline-workspace) again from our talend software zip.

Sunday, 27 November 2011

Talend - ETL - Generate job exports from Talend Repository using CommandLine - Export job

Problem - How to generate job exports from Talend Repository using commandline utility from Talend.
Scenario - Lets assume you have your Talend remote repository at linux machine and repository DB on oracle.
Your developers have jobs checked in talend respository named TALEND_REPO.
Now administrator or some automated process needs to export job scripts (which includes executable jar/bat files for jobs) using commandline utility

Solution -
---------
We have used two scripts for this one shell (expJob.sh) and one script which is called by this shell script (expJobdetails.txt).
Exeucte shell script on talend repository server or server which has commandline utility software from talend install on it.

expJob.sh
---------
#!/bin/sh
cd <path of commandLine folder of talend software>
echo "Start of export job"
./TISEE-linux-gtk-x86_64 --disableShellInput -nosplash -application org.talend.commandline.CommandLine -consoleLog -data commandline-workspace scriptFile "<Path of expJobDetails.txt>"

<Path of expJobDetails.txt> - is to be replaced by path of expJobdetails.txt file


expJobDetails.txt
-----------------
initRemote <URL of Talend respository server>
listProject
logonProject -pn <Talend Repository Name> -ul <userLogin> -up <userPassword>
exportJob "<TalendJobName>" -dd <OutputDir> -jc default -af <ExportJobfilename> -jstats
logoffProject
-----------------

How to read expJobDetails.txt
----------------------------------
initRemote - initializes the respository connection
listProject - list all projects inside repository (not required for generating job exports, its just for learning purpose)
logonProject - to logon to repository using userid and password
logoffProject - logoff from project

Replace following in expJobDetails.txt with :
--------------------------------------------------------------------
<URL of Talend respository server> ---> http://localhost:8080/org.talend.administrator
<Talend Repository Name> ---> TALEND_REPO
<userLogin>  ---> Login of user who has access to talend respository (even developers user id who have access to talend repository will work)
<userPassword>  ---> Password of above user
<TalendJobName> ---> Name of Job you want to export
<OutputDir> ---> output directory where exported zip file will be produced
<ExportJobfilename>  ---> Name of zip file to be produced