class CGEBackupCollection

Provides a class to interface with Amazon's S3 service to download, extract and process MySQL CGE backups from the production system. @!attribute [r] dumpDir

@return [String] a string containing the working directory for all downloads and processing

Public Class Methods

new() click to toggle source

Configures and returns a CGEBackupCollection object. Connects to AWS S3 and builds a collection of CGEBackup objects.

# File lib/cge_backup_collection.rb, line 11
def initialize
  # this assumes the role on the production account to access CGE backups
  sts = AWS::STS.new(:access_key_id=>$AWS_ID, :secret_access_key=>$AWS_SECRET)
  role = sts.assume_role(:role_arn => $ROLE_ARN, :role_session_name => SecureRandom.uuid.gsub(/-/,''), :duration_seconds => 60 * 15)

  # store role credentials
  creds = role[:credentials]

  # open the CGE backup bucket
  cge_bucket = AWS::S3.new(creds).buckets[$CGE_BUCKET]

  temp1 = Array.new

  # Retrieve all object names
  cge_bucket.objects.each do |obj|
    # which match the standard filename template
    if obj.key =~ /BACKUP-\d*_\d-[0-9a-f]*.tar/
      temp1.push(obj.key)
    end
  end

  #sort them by time
  temp1.sort! do |a, b|
    time1 = parse_time(a)
    time2 = parse_time(b)

    time1 <=> time2
  end

  temp2 = Array.new

  # This block removes any backup that doesn't contain files uploaded by each node
  # the temp array should be sorted by backup time. All backups should be 2 files (one for each node)
  # If the previous file and the next file don't have the same time as any given file,
  # then the given file must be by itself.
  begin
    # Push a backup
    temp2.push(temp1.pop)

    time = parse_time(temp2.last)

    # if the pushed file's time matched the previous file's or next file's time, continue.
    # Otherwise, remove the pushed file from the array.
    if time == parse_time(temp2.fetch(-2, ''))
      next
    elsif time == parse_time(temp1.fetch(-1, ''))
      next
    else
      temp2.pop
    end
  end until temp1.count == 0

  @backups = Array.new
  @backups.push(CGEBackup.new(temp2.pop, temp2.pop, @backups.count + 1)) until temp2.count == 0
end

Public Instance Methods

[](id) click to toggle source
# File lib/cge_backup_collection.rb, line 67
def [](id)
  @backups[id - 1]
end
archive(time, dump_id=nil) click to toggle source

Retrieves sql dumps from S3 and saves them to them @overload archive(time)

Stores the latest dump to the bucket
@param [Time] time the time of the sync point to archive to

@overload archive(time, dump_id)

Stores the dump with the specified id to @dumpDir and scrubs it
@param [Time] time the time of the sync point to archive to
@param [Integer] dump_id the id of the dumps to retrieve
# File lib/cge_backup_collection.rb, line 89
def archive(time, dump_id=nil)
  failValidation if dump_id.not_nil? and not dump_id.is_a?(Integer)
  failValidation if time.nil? or not time.is_a?(Time)

  # Default to latest dump
  id = dump_id.nil? ? @backups.count : dump_id

  Dir.mktmpdir do |dir|
    backup = self[id]

    print "Downloading CGE backup and extracting...\t\t"
    backup.download(dir)
    backup.extract
    puts "Done."

    print "Uploading database synchronization data...\t\t"
    backup.archive(time)
    puts "Done."
  end
  #FileUtils.remove_entry_secure dir
end
list() click to toggle source

Displays a list of all available backups.

# File lib/cge_backup_collection.rb, line 72
def list
  puts '       ID       |   Epoch Time   |    Date and Time    '
  puts '-------------------------------------------------------'

  @backups.each do |value|
    puts "#{value.id.to_s.center(16)}|#{value.epoch_time.center(16)}|#{value.date_time.center(21)}"
  end
end

Private Instance Methods

get_usage() click to toggle source
Calls superclass method ErrorHandlingIface#get_usage
# File lib/cge_backup_collection.rb, line 117
def get_usage
  #Usage cases for all the usage in this class
  if @usage.nil?
    init_usage
    @usage['archive_dumps'] = ['time = Time, dump_id = Integer']
  end

  super
end
parse_time(backup) click to toggle source
# File lib/cge_backup_collection.rb, line 113
def parse_time(backup)
  backup.scan(/BACKUP-(\d*)_\d-[0-9a-f]*.tar/).flatten[0]
end