class Google::Apis::DataprocV1beta2::OrderedJob
A job executed by the workflow.
Attributes
A Dataproc job for running Apache Hadoop MapReduce (hadoop.apache.org/ docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/ MapReduceTutorial.html) jobs on Apache Hadoop YARN (hadoop.apache.org/ docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). Corresponds to the JSON property `hadoopJob` @return [Google::Apis::DataprocV1beta2::HadoopJob]
A Dataproc job for running Apache Hive (hive.apache.org/) queries on YARN. Corresponds to the JSON property `hiveJob` @return [Google::Apis::DataprocV1beta2::HiveJob]
Optional. The labels to associate with this job.Label keys must be between 1 and 63 characters long, and must conform to the following regular expression: \ p`Ll`p`Lo“0,62`Label values must be between 1 and 63 characters long, and must conform to the following regular expression: p`Ll`p`Lo`p`N`_-`0,63`No more than 32 labels can be associated with a given job. Corresponds to the JSON property `labels` @return [Hash<String,String>]
A Dataproc job for running Apache Pig (pig.apache.org/) queries on YARN. Corresponds to the JSON property `pigJob` @return [Google::Apis::DataprocV1beta2::PigJob]
Optional. The optional list of prerequisite job step_ids. If not specified, the job will start at the beginning of workflow. Corresponds to the JSON property `prerequisiteStepIds` @return [Array<String>]
A Dataproc job for running Presto (prestosql.io/) queries. IMPORTANT: The Dataproc Presto Optional Component (cloud.google.com/dataproc/docs/ concepts/components/presto) must be enabled when the cluster is created to submit a Presto job to the cluster. Corresponds to the JSON property `prestoJob` @return [Google::Apis::DataprocV1beta2::PrestoJob]
A Dataproc job for running Apache PySpark (spark.apache.org/docs/0.9.0/ python-programming-guide.html) applications on YARN. Corresponds to the JSON property `pysparkJob` @return [Google::Apis::DataprocV1beta2::PySparkJob]
Job
scheduling options. Corresponds to the JSON property `scheduling` @return [Google::Apis::DataprocV1beta2::JobScheduling]
A Dataproc job for running Apache Spark (spark.apache.org/) applications on YARN. The specification of the main method to call to drive the job. Specify either the jar file that contains the main class or the main class name. To pass both a main jar and a main class in that jar, add the jar to CommonJob.jar_file_uris, and then specify the main class name in main_class. Corresponds to the JSON property `sparkJob` @return [Google::Apis::DataprocV1beta2::SparkJob]
A Dataproc job for running Apache SparkR (spark.apache.org/docs/latest/ sparkr.html) applications on YARN. Corresponds to the JSON property `sparkRJob` @return [Google::Apis::DataprocV1beta2::SparkRJob]
A Dataproc job for running Apache Spark SQL (spark.apache.org/sql/) queries. Corresponds to the JSON property `sparkSqlJob` @return [Google::Apis::DataprocV1beta2::SparkSqlJob]
Required. The step id. The id must be unique among all jobs within the template.The step id is used as prefix for job id, as job goog-dataproc- workflow-step-id label, and in prerequisiteStepIds field from other steps.The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters. Corresponds to the JSON property `stepId` @return [String]
Public Class Methods
# File lib/google/apis/dataproc_v1beta2/classes.rb, line 2474 def initialize(**args) update!(**args) end
Public Instance Methods
Update properties of this object
# File lib/google/apis/dataproc_v1beta2/classes.rb, line 2479 def update!(**args) @hadoop_job = args[:hadoop_job] if args.key?(:hadoop_job) @hive_job = args[:hive_job] if args.key?(:hive_job) @labels = args[:labels] if args.key?(:labels) @pig_job = args[:pig_job] if args.key?(:pig_job) @prerequisite_step_ids = args[:prerequisite_step_ids] if args.key?(:prerequisite_step_ids) @presto_job = args[:presto_job] if args.key?(:presto_job) @pyspark_job = args[:pyspark_job] if args.key?(:pyspark_job) @scheduling = args[:scheduling] if args.key?(:scheduling) @spark_job = args[:spark_job] if args.key?(:spark_job) @spark_r_job = args[:spark_r_job] if args.key?(:spark_r_job) @spark_sql_job = args[:spark_sql_job] if args.key?(:spark_sql_job) @step_id = args[:step_id] if args.key?(:step_id) end