class OCI::DataFlow::Models::CreateRunDetails

The create run details. The following properties are optional and override the default values set in the associated application:

- applicationId
- archiveUri
- arguments
- configuration
- definedTags
- displayName
- driverShape
- execute
- executorShape
- freeformTags
- logsBucketUri
- metastoreId
- numExecutors
- parameters
- sparkVersion
- warehouseBucketUri

It is expected that either the applicationId or the execute parameter is specified; but not both. If both or none are set, a Bad Request (HTTP 400) status will be sent as the response. If an appicationId is not specified, then a value for the execute parameter is expected. Using data parsed from the value, a new application will be created and assicated with the new run. See information on the execute parameter for details on the format of this parameter.

The optional parameter spark version can only be specified when using the execute parameter. If it is not specified when using the execute parameter, the latest version will be used as default. If the execute parameter is not used, the spark version will be taken from the associated application.

If displayName is not specified, it will be derived from the displayName of associated application or set by API using fileUri's application file name. Once a run is created, its properties (except for definedTags and freeformTags) cannot be changed. If the parent application's properties (including definedTags and freeformTags) are updated, the corresponding properties of the run will not update.

Attributes

application_id[RW]

The OCID of the associated application. If this value is set, then no value for the execute parameter is required. If this value is not set, then a value for the execute parameter is required, and a new application is created and associated with the new run.

@return [String]

archive_uri[RW]

An Oracle Cloud Infrastructure URI of an archive.zip file containing custom dependencies that may be used to support the execution a Python, Java, or Scala application. See docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

@return [String]

arguments[RW]

The arguments passed to the running application as command line arguments. An argument is either a plain text or a placeholder. Placeholders are replaced using values from the parameters map. Each placeholder specified must be represented in the parameters map else the request (POST or PUT) will fail with a HTTP 400 status code. Placeholders are specified as `Service Api Spec`, where `name` is the name of the parameter. Example: `[ "–input", "${input_file}", "–name", "John Doe" ]` If "input_file" has a value of "mydata.xml", then the value above will be translated to `–input mydata.xml –name "John Doe"`

@return [Array<String>]

compartment_id[RW]

[Required] The OCID of a compartment.

@return [String]

configuration[RW]

The Spark configuration passed to the running process. See spark.apache.org/docs/latest/configuration.html#available-properties. Example: { "spark.app.name" : "My App Name", "spark.shuffle.io.maxRetries" : "4" } Note: Not all Spark properties are permitted to be set. Attempting to set a property that is not allowed to be overwritten will cause a 400 status to be returned.

@return [Hash<String, String>]

defined_tags[RW]

Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see [Resource Tags](docs.cloud.oracle.com/iaas/Content/General/Concepts/resourcetags.htm). Example: `{"Operations": {"CostCenter": "42"}}`

@return [Hash<String, Hash<String, Object>>]

display_name[RW]

A user-friendly name that does not have to be unique. Avoid entering confidential information. If this value is not specified, it will be derived from the associated application's displayName or set by API using fileUri's application file name.

@return [String]

driver_shape[RW]

The VM shape for the driver. Sets the driver cores and memory.

@return [String]

execute[RW]

The input used for spark-submit command. For more details see spark.apache.org/docs/latest/submitting-applications.html#launching-applications-with-spark-submit. Supported options include “–class“, “–file“, “–jars“, “–conf“, “–py-files“, and main application file with arguments. Example: “–jars oci://path/to/a.jar,oci://path/to/b.jar –files oci://path/to/a.json,oci://path/to/b.csv –py-files oci://path/to/a.py,oci://path/to/b.py –conf spark.sql.crossJoin.enabled=true –class org.apache.spark.examples.SparkPi oci://path/to/main.jar 10“ Note: If execute is specified together with applicationId, className, configuration, fileUri, language, arguments, parameters during application create/update, or run create/submit, Data Flow service will use derived information from execute input only.

@return [String]

executor_shape[RW]

The VM shape for the executors. Sets the executor cores and memory.

@return [String]

freeform_tags[RW]

Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see [Resource Tags](docs.cloud.oracle.com/iaas/Content/General/Concepts/resourcetags.htm). Example: `{"Department": "Finance"}`

@return [Hash<String, String>]

logs_bucket_uri[RW]

An Oracle Cloud Infrastructure URI of the bucket where the Spark job logs are to be uploaded. See docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

@return [String]

metastore_id[RW]

The OCID of OCI Hive Metastore.

@return [String]

num_executors[RW]

The number of executor VMs requested.

@return [Integer]

parameters[RW]

An array of name/value pairs used to fill placeholders found in properties like `Application.arguments`. The name must be a string of one or more word characters (a-z, A-Z, 0-9, _). The value can be a string of 0 or more characters of any kind. Example: [ { name: "iterations", value: "10"}, { name: "input_file", value: "mydata.xml" }, { name: "variable_x", value: "${x}"} ]

@return [Array<OCI::DataFlow::Models::ApplicationParameter>]

spark_version[RW]

The Spark version utilized to run the application. This value may be set if applicationId is not since the Spark version will be taken from the associated application.

@return [String]

warehouse_bucket_uri[RW]

An Oracle Cloud Infrastructure URI of the bucket to be used as default warehouse directory for BATCH SQL runs. See docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

@return [String]

Public Class Methods

attribute_map() click to toggle source

Attribute mapping from ruby-style variable name to JSON key.

# File lib/oci/data_flow/models/create_run_details.rb, line 154
def self.attribute_map
  {
    # rubocop:disable Style/SymbolLiteral
    'application_id': :'applicationId',
    'archive_uri': :'archiveUri',
    'arguments': :'arguments',
    'compartment_id': :'compartmentId',
    'configuration': :'configuration',
    'defined_tags': :'definedTags',
    'display_name': :'displayName',
    'driver_shape': :'driverShape',
    'execute': :'execute',
    'executor_shape': :'executorShape',
    'freeform_tags': :'freeformTags',
    'logs_bucket_uri': :'logsBucketUri',
    'metastore_id': :'metastoreId',
    'num_executors': :'numExecutors',
    'parameters': :'parameters',
    'spark_version': :'sparkVersion',
    'warehouse_bucket_uri': :'warehouseBucketUri'
    # rubocop:enable Style/SymbolLiteral
  }
end
new(attributes = {}) click to toggle source

Initializes the object @param [Hash] attributes Model attributes in the form of hash @option attributes [String] :application_id The value to assign to the {#application_id} property @option attributes [String] :archive_uri The value to assign to the {#archive_uri} property @option attributes [Array<String>] :arguments The value to assign to the {#arguments} property @option attributes [String] :compartment_id The value to assign to the {#compartment_id} property @option attributes [Hash<String, String>] :configuration The value to assign to the {#configuration} property @option attributes [Hash<String, Hash<String, Object>>] :defined_tags The value to assign to the {#defined_tags} property @option attributes [String] :display_name The value to assign to the {#display_name} property @option attributes [String] :driver_shape The value to assign to the {#driver_shape} property @option attributes [String] :execute The value to assign to the {#execute} property @option attributes [String] :executor_shape The value to assign to the {#executor_shape} property @option attributes [Hash<String, String>] :freeform_tags The value to assign to the {#freeform_tags} property @option attributes [String] :logs_bucket_uri The value to assign to the {#logs_bucket_uri} property @option attributes [String] :metastore_id The value to assign to the {#metastore_id} property @option attributes [Integer] :num_executors The value to assign to the {#num_executors} property @option attributes [Array<OCI::DataFlow::Models::ApplicationParameter>] :parameters The value to assign to the {#parameters} property @option attributes [String] :spark_version The value to assign to the {#spark_version} property @option attributes [String] :warehouse_bucket_uri The value to assign to the {#warehouse_bucket_uri} property

# File lib/oci/data_flow/models/create_run_details.rb, line 226
def initialize(attributes = {})
  return unless attributes.is_a?(Hash)

  # convert string to symbol for hash key
  attributes = attributes.each_with_object({}) { |(k, v), h| h[k.to_sym] = v }

  self.application_id = attributes[:'applicationId'] if attributes[:'applicationId']

  raise 'You cannot provide both :applicationId and :application_id' if attributes.key?(:'applicationId') && attributes.key?(:'application_id')

  self.application_id = attributes[:'application_id'] if attributes[:'application_id']

  self.archive_uri = attributes[:'archiveUri'] if attributes[:'archiveUri']

  raise 'You cannot provide both :archiveUri and :archive_uri' if attributes.key?(:'archiveUri') && attributes.key?(:'archive_uri')

  self.archive_uri = attributes[:'archive_uri'] if attributes[:'archive_uri']

  self.arguments = attributes[:'arguments'] if attributes[:'arguments']

  self.compartment_id = attributes[:'compartmentId'] if attributes[:'compartmentId']

  raise 'You cannot provide both :compartmentId and :compartment_id' if attributes.key?(:'compartmentId') && attributes.key?(:'compartment_id')

  self.compartment_id = attributes[:'compartment_id'] if attributes[:'compartment_id']

  self.configuration = attributes[:'configuration'] if attributes[:'configuration']

  self.defined_tags = attributes[:'definedTags'] if attributes[:'definedTags']

  raise 'You cannot provide both :definedTags and :defined_tags' if attributes.key?(:'definedTags') && attributes.key?(:'defined_tags')

  self.defined_tags = attributes[:'defined_tags'] if attributes[:'defined_tags']

  self.display_name = attributes[:'displayName'] if attributes[:'displayName']

  raise 'You cannot provide both :displayName and :display_name' if attributes.key?(:'displayName') && attributes.key?(:'display_name')

  self.display_name = attributes[:'display_name'] if attributes[:'display_name']

  self.driver_shape = attributes[:'driverShape'] if attributes[:'driverShape']

  raise 'You cannot provide both :driverShape and :driver_shape' if attributes.key?(:'driverShape') && attributes.key?(:'driver_shape')

  self.driver_shape = attributes[:'driver_shape'] if attributes[:'driver_shape']

  self.execute = attributes[:'execute'] if attributes[:'execute']

  self.executor_shape = attributes[:'executorShape'] if attributes[:'executorShape']

  raise 'You cannot provide both :executorShape and :executor_shape' if attributes.key?(:'executorShape') && attributes.key?(:'executor_shape')

  self.executor_shape = attributes[:'executor_shape'] if attributes[:'executor_shape']

  self.freeform_tags = attributes[:'freeformTags'] if attributes[:'freeformTags']

  raise 'You cannot provide both :freeformTags and :freeform_tags' if attributes.key?(:'freeformTags') && attributes.key?(:'freeform_tags')

  self.freeform_tags = attributes[:'freeform_tags'] if attributes[:'freeform_tags']

  self.logs_bucket_uri = attributes[:'logsBucketUri'] if attributes[:'logsBucketUri']

  raise 'You cannot provide both :logsBucketUri and :logs_bucket_uri' if attributes.key?(:'logsBucketUri') && attributes.key?(:'logs_bucket_uri')

  self.logs_bucket_uri = attributes[:'logs_bucket_uri'] if attributes[:'logs_bucket_uri']

  self.metastore_id = attributes[:'metastoreId'] if attributes[:'metastoreId']

  raise 'You cannot provide both :metastoreId and :metastore_id' if attributes.key?(:'metastoreId') && attributes.key?(:'metastore_id')

  self.metastore_id = attributes[:'metastore_id'] if attributes[:'metastore_id']

  self.num_executors = attributes[:'numExecutors'] if attributes[:'numExecutors']

  raise 'You cannot provide both :numExecutors and :num_executors' if attributes.key?(:'numExecutors') && attributes.key?(:'num_executors')

  self.num_executors = attributes[:'num_executors'] if attributes[:'num_executors']

  self.parameters = attributes[:'parameters'] if attributes[:'parameters']

  self.spark_version = attributes[:'sparkVersion'] if attributes[:'sparkVersion']

  raise 'You cannot provide both :sparkVersion and :spark_version' if attributes.key?(:'sparkVersion') && attributes.key?(:'spark_version')

  self.spark_version = attributes[:'spark_version'] if attributes[:'spark_version']

  self.warehouse_bucket_uri = attributes[:'warehouseBucketUri'] if attributes[:'warehouseBucketUri']

  raise 'You cannot provide both :warehouseBucketUri and :warehouse_bucket_uri' if attributes.key?(:'warehouseBucketUri') && attributes.key?(:'warehouse_bucket_uri')

  self.warehouse_bucket_uri = attributes[:'warehouse_bucket_uri'] if attributes[:'warehouse_bucket_uri']
end
swagger_types() click to toggle source

Attribute type mapping.

# File lib/oci/data_flow/models/create_run_details.rb, line 179
def self.swagger_types
  {
    # rubocop:disable Style/SymbolLiteral
    'application_id': :'String',
    'archive_uri': :'String',
    'arguments': :'Array<String>',
    'compartment_id': :'String',
    'configuration': :'Hash<String, String>',
    'defined_tags': :'Hash<String, Hash<String, Object>>',
    'display_name': :'String',
    'driver_shape': :'String',
    'execute': :'String',
    'executor_shape': :'String',
    'freeform_tags': :'Hash<String, String>',
    'logs_bucket_uri': :'String',
    'metastore_id': :'String',
    'num_executors': :'Integer',
    'parameters': :'Array<OCI::DataFlow::Models::ApplicationParameter>',
    'spark_version': :'String',
    'warehouse_bucket_uri': :'String'
    # rubocop:enable Style/SymbolLiteral
  }
end

Public Instance Methods

==(other) click to toggle source

Checks equality by comparing each attribute. @param [Object] other the other object to be compared

# File lib/oci/data_flow/models/create_run_details.rb, line 326
def ==(other)
  return true if equal?(other)

  self.class == other.class &&
    application_id == other.application_id &&
    archive_uri == other.archive_uri &&
    arguments == other.arguments &&
    compartment_id == other.compartment_id &&
    configuration == other.configuration &&
    defined_tags == other.defined_tags &&
    display_name == other.display_name &&
    driver_shape == other.driver_shape &&
    execute == other.execute &&
    executor_shape == other.executor_shape &&
    freeform_tags == other.freeform_tags &&
    logs_bucket_uri == other.logs_bucket_uri &&
    metastore_id == other.metastore_id &&
    num_executors == other.num_executors &&
    parameters == other.parameters &&
    spark_version == other.spark_version &&
    warehouse_bucket_uri == other.warehouse_bucket_uri
end
build_from_hash(attributes) click to toggle source

Builds the object from hash @param [Hash] attributes Model attributes in the form of hash @return [Object] Returns the model itself

# File lib/oci/data_flow/models/create_run_details.rb, line 372
def build_from_hash(attributes)
  return nil unless attributes.is_a?(Hash)

  self.class.swagger_types.each_pair do |key, type|
    if type =~ /^Array<(.*)>/i
      # check to ensure the input is an array given that the the attribute
      # is documented as an array but the input is not
      if attributes[self.class.attribute_map[key]].is_a?(Array)
        public_method("#{key}=").call(
          attributes[self.class.attribute_map[key]]
            .map { |v| OCI::Internal::Util.convert_to_type(Regexp.last_match(1), v) }
        )
      end
    elsif !attributes[self.class.attribute_map[key]].nil?
      public_method("#{key}=").call(
        OCI::Internal::Util.convert_to_type(type, attributes[self.class.attribute_map[key]])
      )
    end
    # or else data not found in attributes(hash), not an issue as the data can be optional
  end

  self
end
eql?(other) click to toggle source

@see the `==` method @param [Object] other the other object to be compared

# File lib/oci/data_flow/models/create_run_details.rb, line 352
def eql?(other)
  self == other
end
hash() click to toggle source

Calculates hash code according to all attributes. @return [Fixnum] Hash code

# File lib/oci/data_flow/models/create_run_details.rb, line 361
def hash
  [application_id, archive_uri, arguments, compartment_id, configuration, defined_tags, display_name, driver_shape, execute, executor_shape, freeform_tags, logs_bucket_uri, metastore_id, num_executors, parameters, spark_version, warehouse_bucket_uri].hash
end
to_hash() click to toggle source

Returns the object in the form of hash @return [Hash] Returns the object in the form of hash

# File lib/oci/data_flow/models/create_run_details.rb, line 405
def to_hash
  hash = {}
  self.class.attribute_map.each_pair do |attr, param|
    value = public_method(attr).call
    next if value.nil? && !instance_variable_defined?("@#{attr}")

    hash[param] = _to_hash(value)
  end
  hash
end
to_s() click to toggle source

Returns the string representation of the object @return [String] String presentation of the object

# File lib/oci/data_flow/models/create_run_details.rb, line 399
def to_s
  to_hash.to_s
end

Private Instance Methods

_to_hash(value) click to toggle source

Outputs non-array value in the form of hash For object, use to_hash. Otherwise, just return the value @param [Object] value Any valid value @return [Hash] Returns the value in the form of hash

# File lib/oci/data_flow/models/create_run_details.rb, line 422
def _to_hash(value)
  if value.is_a?(Array)
    value.compact.map { |v| _to_hash(v) }
  elsif value.is_a?(Hash)
    {}.tap do |hash|
      value.each { |k, v| hash[k] = _to_hash(v) }
    end
  elsif value.respond_to? :to_hash
    value.to_hash
  else
    value
  end
end