Talend : Difference between Standard job and Big Data Batch

Talend : Difference between Standard job and Big Data Batch

Here we will detail the main points that differs a standard job to a Big data batch job.

  1. Standard job is a java process
  2. Big data job is a java that get translated to the target engine language Python for Mapreduce and Scala for Spark
  3. Standard – This is just a java process. It can access many data sources via JDBC or HDFS,etc but the main process just executes in a JVM.
  4. BigData Streaming – This is for realtime-ish/microbatch stuff, the jobs are sent to Spark or Storm for execution.
  5. BigData Batch – Uses either MR1(namenode,etc)/YARN for MapReduce execution, or Spark for execution (With or without YARN depending on platform)

 

Standard JOb : 

Runs on the jobserver

 

Leave a Reply