Showing posts with label parameters. Show all posts
Showing posts with label parameters. Show all posts
Wednesday, February 18, 2015
Kettle Pass parameters from job to transformation in PDI Retrieve data from table by passing table name as parameter
This post will talk about basic understanding of parameters concept in PDI.
Software Setup :
PDI 5.1.0 CE
PostgreSQL
Concept : Retrieve data from table by passing table name as parameter.
You will learn below things
1) Table input step & database connections
2) Creating parameter(s) in Transformation(s) & Job(s)
3) Passing parameter(s) from job to transformation.
Lets say in foodmart database you have below tables 1) region 2) product 3) customer and etc.
Every time when you pass different table name as parameter you should able to fetch the data coming from the passed table.
1) Table input step & database connections
2) Creating parameter(s) in Transformation(s) & Job(s)
1) Create a new transformation & drag and drop "Table Input" step from Design Pane.

2) Double Click on "Table input" step & Click on New tab to give database connections as shown in below image

3) Define the parameter in transformation ( in this example the table name should go as parameter).
Image A : Right Click on the empty space to get the transformation settings.
Image B : Database Connection Details
Parameter Name is : tableName


4) Come back to the properties of "Table input" & write select statement to retrieve the data.
Eg : SELECT * FROM product LIMIT 100
In above example "product" is the table name that you have to replace with the parameter.
i.e., SELECT * FROM ${tableName} LIMIT 100
Once you write the query, come down check "Replace variables in script"

5) Take a "Dummy" step and connect it to "Table input step" . Save the transformation & run it on "Dummy Step"( Right click dummy step and click on "Preview".
OUTPUT : Test 1 : With "region" table

OUTPUT : Result 1

OUTPUT : Test 2 : With "product" table
* Right click Dummy step then click on Preview
* Click on Configure & then in parameters tab give "product" for tableName value parameter.
OUTPUT : Result 2

3) Passing parameter(s) from job to transformation
* Crtl+Alt to create a new job
* From the "General" node of Design tab ( Appears left side) drag and drop "Transoformation step".
* Drag and drop "START" step and connect these two as shown in below image.

* Double click on "Transformation" step to set the properties & parameter information.
* Browse for the transformation file and then move to parameters tab
* Click on "Get Parameters" then you can find parameter get apper under Parameter tab.
* Right click on empty canvas of job to get its settings & move to Parameters tab and give parameter name as "tableName" and provide default value.
Save the job and run it.
IMP NOTE :
1) Parameter names in job & transformation should match.
2) You can create n number of parameters in job but can pass as many you wish.
3) In job design you should take "START" step to tell the job to start execution from there.
If you ignore you will get below error ( which took me awhile to solve it for the very first time).
Start of job execution
:A serious error occurred during job execution:
Couldnt find starting point in this job.
org.pentaho.di.core.exception.KettleJobException:
Couldnt find starting point in this job.
at org.pentaho.di.job.Job.execute(Job.java:516)
at org.pentaho.di.job.Job.run(Job.java:422)
Spoon - Job has ended.
References :
1) http://wiki.pentaho.com/display/EAI/Named+Parameters
2) http://diethardsteiner.blogspot.in/2013/07/pentaho-kettle-parameters-and-variables.html
3) http://wiki.pentaho.com/display/EAI/Substituting+variable+references+in+Job+Parameter+values
Hope this helps some one like me for the first time who is looking for parameters(named) concept :-)
Cheers :-)
:-) Happy New Year - Welcome - 2015 :-)
Software Setup :
PDI 5.1.0 CE
PostgreSQL
Concept : Retrieve data from table by passing table name as parameter.
You will learn below things
1) Table input step & database connections
2) Creating parameter(s) in Transformation(s) & Job(s)
3) Passing parameter(s) from job to transformation.
Lets say in foodmart database you have below tables 1) region 2) product 3) customer and etc.
Every time when you pass different table name as parameter you should able to fetch the data coming from the passed table.
1) Table input step & database connections
2) Creating parameter(s) in Transformation(s) & Job(s)
1) Create a new transformation & drag and drop "Table Input" step from Design Pane.

2) Double Click on "Table input" step & Click on New tab to give database connections as shown in below image

3) Define the parameter in transformation ( in this example the table name should go as parameter).
Image A : Right Click on the empty space to get the transformation settings.
Image B : Database Connection Details
Parameter Name is : tableName


4) Come back to the properties of "Table input" & write select statement to retrieve the data.
Eg : SELECT * FROM product LIMIT 100
In above example "product" is the table name that you have to replace with the parameter.
i.e., SELECT * FROM ${tableName} LIMIT 100
Once you write the query, come down check "Replace variables in script"

5) Take a "Dummy" step and connect it to "Table input step" . Save the transformation & run it on "Dummy Step"( Right click dummy step and click on "Preview".
OUTPUT : Test 1 : With "region" table

OUTPUT : Result 1

OUTPUT : Test 2 : With "product" table
* Right click Dummy step then click on Preview
* Click on Configure & then in parameters tab give "product" for tableName value parameter.
OUTPUT : Result 2

3) Passing parameter(s) from job to transformation
* Crtl+Alt to create a new job
* From the "General" node of Design tab ( Appears left side) drag and drop "Transoformation step".
* Drag and drop "START" step and connect these two as shown in below image.

* Double click on "Transformation" step to set the properties & parameter information.
* Browse for the transformation file and then move to parameters tab
* Click on "Get Parameters" then you can find parameter get apper under Parameter tab.
* Right click on empty canvas of job to get its settings & move to Parameters tab and give parameter name as "tableName" and provide default value.
Save the job and run it.
IMP NOTE :
1) Parameter names in job & transformation should match.
2) You can create n number of parameters in job but can pass as many you wish.
3) In job design you should take "START" step to tell the job to start execution from there.
If you ignore you will get below error ( which took me awhile to solve it for the very first time).
Start of job execution
:A serious error occurred during job execution:
Couldnt find starting point in this job.
org.pentaho.di.core.exception.KettleJobException:
Couldnt find starting point in this job.
at org.pentaho.di.job.Job.execute(Job.java:516)
at org.pentaho.di.job.Job.run(Job.java:422)
Spoon - Job has ended.
References :
1) http://wiki.pentaho.com/display/EAI/Named+Parameters
2) http://diethardsteiner.blogspot.in/2013/07/pentaho-kettle-parameters-and-variables.html
3) http://wiki.pentaho.com/display/EAI/Substituting+variable+references+in+Job+Parameter+values
Hope this helps some one like me for the first time who is looking for parameters(named) concept :-)
Cheers :-)
:-) Happy New Year - Welcome - 2015 :-)
Saturday, February 14, 2015
Cascading parameters example in Pentaho CDE
Hi Guys,
It is quite easy to learn about cascading parameters in pentaho CDE dash boarding.
Example below teach you how to give cascading parameters on CDE dashboard.
Environment :
C-Tools 13.09
Pentaho BA Server 4.8 stableb
DB : postgreSQL foodmart ( jasperservers database)
IDEA of cascading parameters:









:-)
It is quite easy to learn about cascading parameters in pentaho CDE dash boarding.
Example below teach you how to give cascading parameters on CDE dashboard.
Environment :
C-Tools 13.09
Pentaho BA Server 4.8 stableb
DB : postgreSQL foodmart ( jasperservers database)
IDEA of cascading parameters:

Example:
1) Lay out section
* Design your lay out for keeping the parameters on dasbharod.
* Sample reference design of layout.

2) Components section
* You need to define 3 selects for 3 parameters
* You need to define 3 parameters
* Observer the Sample worked out image below
*

* Define 2 parameters for country and state(paramCountry and paramState)
* Define 3 Selects for displaying the selection of parameters on dashboard.
( CountrySelectComponent, StateSelectComponent, CitySelectComponent)
Parameters
1. ParamCountry

i.e, Name= paramCountry and Propertyvalue=USA(default value)
2. paramState
Name=paramState and Propertyvalue=OR(default value)
NOTE : you need NOT to create parameter for city in this case.
Select Components
* Select components are used to display the data coming from sequel query on dashboard as drop down list at the respected place holders.
1) CountrySelectComponent

Name: CountrySelecte Component
Parameter: paramCouentry (you should give param name otherwise, it wont work)
Datasource: give the query name (will discussion in Datasource selection)
HtmlObject : countrySelect
2)StateSelectComponent

Name: StateSelectComponent
Parameter : paramState
Listners : paramCountry
Parameters : Arg(paramCountry) and val(paramCountryP)
Datasource : queryState
HtmlObject : stateSelect
3) CitySelectComponent
* You need add two listners and two parameters at Listerns and parameters properties
Name;CitySelectComponent
Parameter : No need to give any parameters name here as we are not using parameter for city sleelct
Listners : paramState,paramCountry
Parameters :
Arg(paramCountry) and val(paramCountry)
Arg(paramState) and val(paramState)
Datasource : queryCity
HtmlObject : citySelect
3) Data Sources section:

Write queries for 3 selects
1) queryCountry(above image)
Name: queryCountry
Driver: org.postgresql.Driver
User name : postgres
Password : postgres
URL : jdbc:postgresql://localhost:6062/foodmart (5432 is the default port number for postgreSQL)
Query: SELECT DISTINCT country from customer
Parameters : paramCountry(arg) and paramCountry(val)
2) queryState(image is not shown in post refere queryCountry image)
Name: queryState
Driver: org.postgresql.Driver
User name : postgres
Password : postgres
URL : jdbc:postgresql://localhost:6062/foodmart (5432 is the default port number for postgreSQL)
Query: SELECT DISTINCT state_province from customer where country=${paramCountry}
Parameters : paramCountry(arg) and paramCountry(val)
3) queryCity(image is not shown in post refere queryCountry image)
Name: queryCity
Driver: org.postgresql.Driver
User name : postgres
Password : postgres
URL : jdbc:postgresql://localhost:6062/foodmart (5432 is the default port number for postgreSQL)
Query: SELECT DISTINCT city from customer WHERE country=${paramCountry} AND state_province=${paramState}
Parameters : paramCountry(arg) and paramCountry(val)
paramState(arg) and paramState(val)
Save the dashboard and see the preview.
Final output:
1) Test 1

2) Test2

Thank you for reading this post :) :)
for queries and suggestions : sadakar.1988@gmail.com
SOURCE CODE OF THIS EXAMPLE:
https://drive.google.com/file/d/0BymV_QP4TGBEZGhpb1hySDBoSEE/edit?usp=sharing
Subscribe to:
Posts (Atom)