class Google::Cloud::Bigquery::Data

# Data

Represents a page of results (rows) as an array of hashes. Because Data delegates to Array, methods such as `Array#count` represent the number of rows in the page. In addition, methods of this class include result set metadata such as `total` and provide access to the schema of the query or table. See {Project#query}, {Dataset#query} and {Table#data}.

@example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

sql = "SELECT word FROM `bigquery-public-data.samples.shakespeare`"
job = bigquery.query_job sql

job.wait_until_done!
data = job.data

data.count # 100000
data.total # 164656

# Iterate over the first page of results
data.each do |row|
  puts row[:word]
end
# Retrieve the next page of results
data = data.next if data.next?

Attributes

gapi_json[RW]

@private The Google API Client object in JSON Hash.

job_gapi[RW]

@private The query Job gapi object, or nil if from Table#data.

service[RW]

@private The Service object.

table_gapi[RW]

@private The {Table} object the data belongs to.

Public Class Methods

from_gapi_json(gapi_json, table_gapi, job_gapi, service) click to toggle source

@private New Data from a response object.

# File lib/google/cloud/bigquery/data.rb, line 537
def self.from_gapi_json gapi_json, table_gapi, job_gapi, service
  rows = gapi_json[:rows] || []
  rows = Convert.format_rows rows, table_gapi.schema.fields unless rows.empty?

  data = new rows
  data.table_gapi = table_gapi
  data.gapi_json = gapi_json
  data.job_gapi = job_gapi
  data.service = service
  data
end
new(arr = []) click to toggle source

@private

Calls superclass method
# File lib/google/cloud/bigquery/data.rb, line 70
def initialize arr = []
  @service = nil
  @table_gapi = nil
  @gapi_json = nil
  super arr
end

Public Instance Methods

all(request_limit: nil, &block) click to toggle source

Retrieves all rows by repeatedly loading {#next} until {#next?} returns `false`. Calls the given block once for each row, which is passed as the parameter.

An enumerator is returned if no block is given.

This method may make several API calls until all rows are retrieved. Be sure to use as narrow a search criteria as possible. Please use with caution.

@param [Integer] request_limit The upper limit of API requests to make

to load all data. Default is no limit.

@yield [row] The block for accessing each row of data. @yieldparam [Hash] row The row object.

@return [Enumerator] An enumerator providing access to all of the

data.

@example Iterating each rows by passing a block:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.data.all do |row|
  puts row[:word]
end

@example Using the enumerator by not passing a block:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

words = table.data.all.map do |row|
  row[:word]
end

@example Limit the number of API calls made:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.data.all(request_limit: 10) do |row|
  puts row[:word]
end
# File lib/google/cloud/bigquery/data.rb, line 518
def all request_limit: nil, &block
  request_limit = request_limit.to_i if request_limit

  return enum_for :all, request_limit: request_limit unless block_given?

  results = self
  loop do
    results.each(&block)
    if request_limit
      request_limit -= 1
      break if request_limit.negative?
    end
    break unless results.next?
    results = results.next
  end
end
ddl?() click to toggle source

Whether the query that created this data was a DDL statement.

@see cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language

Using Data Definition Language Statements

@return [Boolean]

@example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
data = bigquery.query "CREATE TABLE my_table (x INT64)"

data.statement_type #=> "CREATE_TABLE"
data.ddl? #=> true
# File lib/google/cloud/bigquery/data.rb, line 278
def ddl?
  ["CREATE_MODEL", "CREATE_TABLE", "CREATE_TABLE_AS_SELECT", "CREATE_VIEW", "\n", "DROP_MODEL", "DROP_TABLE",
   "DROP_VIEW"].include? statement_type
end
ddl_operation_performed() click to toggle source

The DDL operation performed, possibly dependent on the pre-existence of the DDL target. (See {#ddl_target_table}.) Possible values (new values might be added in the future):

  • “CREATE”: The query created the DDL target.

  • “SKIP”: No-op. Example cases: the query is `CREATE TABLE IF NOT EXISTS` while the table already exists, or the query is `DROP TABLE IF EXISTS` while the table does not exist.

  • “REPLACE”: The query replaced the DDL target. Example case: the query is `CREATE OR REPLACE TABLE`, and the table already exists.

  • “DROP”: The query deleted the DDL target.

@return [String, nil] The DDL operation performed.

# File lib/google/cloud/bigquery/data.rb, line 321
def ddl_operation_performed
  job_gapi&.statistics&.query&.ddl_operation_performed
end
ddl_target_routine() click to toggle source

The DDL target routine, in reference state. (See {Routine#reference?}.) Present only for `CREATE/DROP FUNCTION/PROCEDURE` queries. (See {#statement_type}.)

@return [Google::Cloud::Bigquery::Routine, nil] The DDL target routine, in

reference state.
# File lib/google/cloud/bigquery/data.rb, line 333
def ddl_target_routine
  ensure_service!
  routine = job_gapi&.statistics&.query&.ddl_target_routine
  return nil if routine.nil?
  Google::Cloud::Bigquery::Routine.new_reference_from_gapi routine, service
end
ddl_target_table() click to toggle source

The DDL target table, in reference state. (See {Table#reference?}.) Present only for `CREATE/DROP TABLE/VIEW` queries. (See {#statement_type}.)

@return [Google::Cloud::Bigquery::Table, nil] The DDL target table, in

reference state.
# File lib/google/cloud/bigquery/data.rb, line 348
def ddl_target_table
  ensure_service!
  table = job_gapi&.statistics&.query&.ddl_target_table
  return nil if table.nil?
  Google::Cloud::Bigquery::Table.new_reference_from_gapi table, service
end
deleted_row_count() click to toggle source

The number of deleted rows. Present only for DML statements `DELETE`, `MERGE` and `TRUNCATE`. (See {#statement_type}.)

@return [Integer, nil] The number of deleted rows, or `nil` if not

applicable.
# File lib/google/cloud/bigquery/data.rb, line 373
def deleted_row_count
  job_gapi&.statistics&.query&.dml_stats&.deleted_row_count
end
dml?() click to toggle source

Whether the query that created this data was a DML statement.

@see cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax

Data Manipulation Language Syntax

@return [Boolean]

@example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
data = bigquery.query "UPDATE my_table " \
                      "SET x = x + 1 " \
                      "WHERE x IS NOT NULL"

data.statement_type #=> "UPDATE"
data.dml? #=> true
# File lib/google/cloud/bigquery/data.rb, line 302
def dml?
  ["INSERT", "UPDATE", "MERGE", "DELETE"].include? statement_type
end
etag() click to toggle source

An ETag hash for the page of results represented by the data instance.

@return [String] The ETag hash.

# File lib/google/cloud/bigquery/data.rb, line 91
def etag
  @gapi_json[:etag]
end
fields() click to toggle source

The fields of the data, obtained from the schema of the table from which the data was read.

@return [Array<Schema::Field>] An array of field objects.

@example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data

data.fields.each do |field|
  puts field.name
end
# File lib/google/cloud/bigquery/data.rb, line 182
def fields
  schema.fields
end
headers() click to toggle source

The names of the columns in the data, obtained from the schema of the table from which the data was read.

@return [Array<Symbol>] An array of column names.

@example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data

data.headers.each do |header|
  puts header
end
# File lib/google/cloud/bigquery/data.rb, line 205
def headers
  schema.headers
end
inserted_row_count() click to toggle source

The number of inserted rows. Present only for DML statements `INSERT` and `MERGE`. (See {#statement_type}.)

@return [Integer, nil] The number of inserted rows, or `nil` if not

applicable.
# File lib/google/cloud/bigquery/data.rb, line 384
def inserted_row_count
  job_gapi&.statistics&.query&.dml_stats&.inserted_row_count
end
kind() click to toggle source

The resource type of the API response.

@return [String] The resource type.

# File lib/google/cloud/bigquery/data.rb, line 82
def kind
  @gapi_json[:kind]
end
next() click to toggle source

Retrieves the next page of data.

@return [Data] A new instance providing the next page of data.

@example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

sql = "SELECT word FROM `bigquery-public-data.samples.shakespeare`"
job = bigquery.query_job sql

job.wait_until_done!
data = job.data

data.count # 100000
data.total # 164656

# Iterate over the first page of results
data.each do |row|
  puts row[:word]
end
# Retrieve the next page of results
data = data.next if data.next?
# File lib/google/cloud/bigquery/data.rb, line 455
def next
  return nil unless next?
  ensure_service!
  data_json = service.list_tabledata \
    @table_gapi.table_reference.dataset_id,
    @table_gapi.table_reference.table_id,
    token: token
  self.class.from_gapi_json data_json, @table_gapi, job_gapi, @service
end
next?() click to toggle source

Whether there is a next page of data.

@return [Boolean] `true` when there is a next page, `false` otherwise.

@example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

sql = "SELECT word FROM `bigquery-public-data.samples.shakespeare`"
job = bigquery.query_job sql

job.wait_until_done!
data = job.data

data.count # 100000
data.total # 164656

# Iterate over the first page of results
data.each do |row|
  puts row[:word]
end
# Retrieve the next page of results
data = data.next if data.next?
# File lib/google/cloud/bigquery/data.rb, line 425
def next?
  !token.nil?
end
num_dml_affected_rows() click to toggle source

The number of rows affected by a DML statement. Present only for DML statements `INSERT`, `UPDATE` or `DELETE`. (See {#statement_type}.)

@return [Integer, nil] The number of rows affected by a DML statement,

or `nil` if the query is not a DML statement.
# File lib/google/cloud/bigquery/data.rb, line 362
def num_dml_affected_rows
  job_gapi&.statistics&.query&.num_dml_affected_rows
end
param_types() click to toggle source

The types of the fields in the data, obtained from the schema of the table from which the data was read. Types use the same format as the optional query parameter types.

@return [Hash] A hash with field names as keys, and types as values.

@example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data

data.param_types
# File lib/google/cloud/bigquery/data.rb, line 227
def param_types
  schema.param_types
end
schema() click to toggle source

The schema of the table from which the data was read.

The returned object is frozen and changes are not allowed. Use {Table#schema} to update the schema.

@return [Schema] A schema object.

@example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data

schema = data.schema
field = schema.field "name"
field.required? #=> true
# File lib/google/cloud/bigquery/data.rb, line 158
def schema
  return nil unless @table_gapi
  Schema.from_gapi(@table_gapi.schema).freeze
end
statement_type() click to toggle source

The type of query statement, if valid. Possible values (new values might be added in the future):

@return [String, nil] The type of query statement.

# File lib/google/cloud/bigquery/data.rb, line 257
def statement_type
  job_gapi&.statistics&.query&.statement_type
end
token() click to toggle source

A token used for paging results. Used by the data instance to retrieve subsequent pages. See {#next}.

@return [String] The pagination token.

# File lib/google/cloud/bigquery/data.rb, line 101
def token
  @gapi_json[:pageToken]
end
total() click to toggle source

The total number of rows in the complete table.

@return [Integer] The number of rows.

@example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

sql = "SELECT word FROM `bigquery-public-data.samples.shakespeare`"
job = bigquery.query_job sql

job.wait_until_done!
data = job.data

data.count # 100000
data.total # 164656

# Iterate over the first page of results
data.each do |row|
  puts row[:word]
end
# Retrieve the next page of results
data = data.next if data.next?
# File lib/google/cloud/bigquery/data.rb, line 131
def total
  Integer @gapi_json[:totalRows]
rescue StandardError
  nil
end
updated_row_count() click to toggle source

The number of updated rows. Present only for DML statements `UPDATE` and `MERGE`. (See {#statement_type}.)

@return [Integer, nil] The number of updated rows, or `nil` if not

applicable.
# File lib/google/cloud/bigquery/data.rb, line 395
def updated_row_count
  job_gapi&.statistics&.query&.dml_stats&.updated_row_count
end

Protected Instance Methods

ensure_service!() click to toggle source

Raise an error unless an active service is available.

# File lib/google/cloud/bigquery/data.rb, line 553
def ensure_service!
  raise "Must have active connection" unless service
end