class Google::Apis::DataprocV1::OrderedJob

A job executed by the workflow.

Attributes

hadoop_job[RW]

A Dataproc job for running Apache Hadoop MapReduce (hadoop.apache.org/ docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/ MapReduceTutorial.html) jobs on Apache Hadoop YARN (hadoop.apache.org/ docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). Corresponds to the JSON property `hadoopJob` @return [Google::Apis::DataprocV1::HadoopJob]

hive_job[RW]

A Dataproc job for running Apache Hive (hive.apache.org/) queries on YARN. Corresponds to the JSON property `hiveJob` @return [Google::Apis::DataprocV1::HiveJob]

labels[RW]

Optional. The labels to associate with this job.Label keys must be between 1 and 63 characters long, and must conform to the following regular expression: \ p`Ll`p`Lo“0,62`Label values must be between 1 and 63 characters long, and must conform to the following regular expression: p`Ll`p`Lo`p`N`_-`0,63`No more than 32 labels can be associated with a given job. Corresponds to the JSON property `labels` @return [Hash<String,String>]

pig_job[RW]

A Dataproc job for running Apache Pig (pig.apache.org/) queries on YARN. Corresponds to the JSON property `pigJob` @return [Google::Apis::DataprocV1::PigJob]

prerequisite_step_ids[RW]

Optional. The optional list of prerequisite job step_ids. If not specified, the job will start at the beginning of workflow. Corresponds to the JSON property `prerequisiteStepIds` @return [Array<String>]

presto_job[RW]

A Dataproc job for running Presto (prestosql.io/) queries. IMPORTANT: The Dataproc Presto Optional Component (cloud.google.com/dataproc/docs/ concepts/components/presto) must be enabled when the cluster is created to submit a Presto job to the cluster. Corresponds to the JSON property `prestoJob` @return [Google::Apis::DataprocV1::PrestoJob]

pyspark_job[RW]

A Dataproc job for running Apache PySpark (spark.apache.org/docs/0.9.0/ python-programming-guide.html) applications on YARN. Corresponds to the JSON property `pysparkJob` @return [Google::Apis::DataprocV1::PySparkJob]

scheduling[RW]

Job scheduling options. Corresponds to the JSON property `scheduling` @return [Google::Apis::DataprocV1::JobScheduling]

spark_job[RW]

A Dataproc job for running Apache Spark (spark.apache.org/) applications on YARN. Corresponds to the JSON property `sparkJob` @return [Google::Apis::DataprocV1::SparkJob]

spark_r_job[RW]

A Dataproc job for running Apache SparkR (spark.apache.org/docs/latest/ sparkr.html) applications on YARN. Corresponds to the JSON property `sparkRJob` @return [Google::Apis::DataprocV1::SparkRJob]

spark_sql_job[RW]

A Dataproc job for running Apache Spark SQL (spark.apache.org/sql/) queries. Corresponds to the JSON property `sparkSqlJob` @return [Google::Apis::DataprocV1::SparkSqlJob]

step_id[RW]

Required. The step id. The id must be unique among all jobs within the template.The step id is used as prefix for job id, as job goog-dataproc- workflow-step-id label, and in prerequisiteStepIds field from other steps.The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters. Corresponds to the JSON property `stepId` @return [String]

Public Class Methods

new(**args) click to toggle source
# File lib/google/apis/dataproc_v1/classes.rb, line 2576
def initialize(**args)
   update!(**args)
end

Public Instance Methods

update!(**args) click to toggle source

Update properties of this object

# File lib/google/apis/dataproc_v1/classes.rb, line 2581
def update!(**args)
  @hadoop_job = args[:hadoop_job] if args.key?(:hadoop_job)
  @hive_job = args[:hive_job] if args.key?(:hive_job)
  @labels = args[:labels] if args.key?(:labels)
  @pig_job = args[:pig_job] if args.key?(:pig_job)
  @prerequisite_step_ids = args[:prerequisite_step_ids] if args.key?(:prerequisite_step_ids)
  @presto_job = args[:presto_job] if args.key?(:presto_job)
  @pyspark_job = args[:pyspark_job] if args.key?(:pyspark_job)
  @scheduling = args[:scheduling] if args.key?(:scheduling)
  @spark_job = args[:spark_job] if args.key?(:spark_job)
  @spark_r_job = args[:spark_r_job] if args.key?(:spark_r_job)
  @spark_sql_job = args[:spark_sql_job] if args.key?(:spark_sql_job)
  @step_id = args[:step_id] if args.key?(:step_id)
end