class BlackStack::BrowserFactory

Constants

DEFAULT_CHROMEDRIVER_PATH

Location of the chromedriver.exe file by default. Works for chrome browsers only.

DEFAULT_LOAD_TIMEOUT

Page load timeout by default. Works for chrome browsers only.

HOST_LOCK_FILENAME

This file is to launch one browser at time in the same host. There is one, and only one, lock file for reach host where Blackstack is running. More information here: bitbucket.org/leandro_sardi/blackstack/issues/1155/browserfactory-use-one-single-lock-file

PROFILE_LOCK_FILENAME

This file store the PID of the last processes that started a browser with this profile name. There is one, and only one, pid_lockfile for each browser profile; and such pid_lockfile is shared by all the hosts where BlackStack is running.

PROFILE_PID_LIST_FILENAME

This file keeps the PIDs of all the child processes of this process. It is neccessary to kill all the child processes of this process when you have started a chrome browser, and you want to be sure that you have closed it. More infromation here: bitbucket.org/leandro_sardi/blackstack/issues/943/browserfactory-deja-exploradores-abiertos

TYPE_CHROME

Different supported browsers.

TYPE_MIMIC

Public Class Methods

addPidToProfileList(profile_name, pid) click to toggle source

Add a PID to the PROFILE_PID_LIST_FILENAME file. For more information, read documentations about PROFILE_PID_LIST_FILENAME. This method is used for chrome browsers only.

# File lib/browserfactory.rb, line 366
def self.addPidToProfileList(profile_name, pid)
  fname = PROFILE_PID_LIST_FILENAME.gsub('%PROFILE_NAME%', profile_name)
  f = File.open(fname, 'a')
  f.write("#{pid},")
  f.close
end
agents() click to toggle source

Returns a suggested list of user-agents

# File lib/browserfactory.rb, line 515
def self.agents
  return @@arAgents
end
browser() click to toggle source

Returns the PampaBrowser object

# File lib/browserfactory.rb, line 335
def self.browser
  @@browser
end
chrome(h) click to toggle source

It will call launch_chrome to launch a chrome browser. The hash parameters musy have the following keys:

  • proxy: a BlackStack::RemoteProxy object;

  • load_timeout: max number of seconds to wait a page to load;

  • user_agent: user agent of the browser;

  • profile_name: will create a folder with ths name to store all the data of the chrome profile (history, cookies, saved passwords, etc);

  • chromedriver_path: location of the chromedriver.exe file, will use DEFAULT_CHROMEDRIVER_PATH by default.

# File lib/browserfactory.rb, line 756
def self.chrome(h)
  # TODO: check if h is a hash
  # TODO: check the variable type of each param
  # TODO: check mandatory params
  self.launch_chrome(
    h[:proxy], 
    h[:load_timeout].nil? ? DEFAULT_LOAD_TIMEOUT : h[:load_timeout], 
    h[:user_agent],
    h[:profile_name],
    h[:chromedriver_path]
  )
end
chrome_version() click to toggle source

Returns the version of the availalble chrome browser.

# File lib/browserfactory.rb, line 641
def self.chrome_version
  res = `reg query \"HKEY_CURRENT_USER\\Software\\Google\\Chrome\\BLBeacon\" /v version`.scan(/REG_SZ    (.*)$/).first
  return nil if res.nil?
  return res[0] if res.size > 0
  return nil
end
chromedriver_version(filename=nil) click to toggle source

Returns the version of the availalble chromedriver. The filename parameter is the full path of the chromedriver.exe command. If filename parameter is nil, it will look for the chromedriver.exe in the PATH environment.

Not valid for Mimic, since MLA handle its own chromedriver into its installation folder.

# File lib/browserfactory.rb, line 654
def self.chromedriver_version(filename=nil)
  res = `chromedriver -v`.scan(/ChromeDriver\s(.*)\s\(/).first if filename.nil?
  res = `#{filename} -v`.scan(/ChromeDriver\s(.*)\s\(/).first if !filename.nil?
  return nil if res.nil?
  return res[0] if res.size > 0
  return nil
end
destroy() click to toggle source

Close the browser only if itIsMe? returns true. Set nil to @@browser, @@pid, @@profile_name, @@driver, @@type.

# File lib/browserfactory.rb, line 544
    def self.destroy
      #if self.itIsMe?
        begin        
          # borro el contenido de PROFILE_LOCK_FILENAME
          #if !@@profile_name.nil?
          #  fname = PROFILE_LOCK_FILENAME.gsub('%PROFILE_NAME%', @@profile_name)
          #  File.open(fname,'w')
          #end
          
          #
          if @@type == BlackStack::BrowserFactory::TYPE_CHROME
            # kill all child process
            self.lockProfileList()
            self.readPidToProfileList(@@profile_name).each { |pid|
              MyProcess.kill(pid)
              self.removePidToProfileList(@@profile_name, pid)      
            }
            self.releaseProfileList()
            
            begin
              @@browser.close if !@@browser.nil?
            rescue
            end
            
          elsif @@type == BlackStack::BrowserFactory::TYPE_MIMIC
=begin
puts ''
            begin
print 'Close browser?... '
              if !@@browser.nil?
puts 'yes'
print 'Close browser... '
                                                                @@browser.close 
puts 'done'
                                                        else
puts 'no'                                                       
                                                        end
            rescue => e
puts "error: #{e.to_s}"
            end
=end
=begin
            begin
              # TODO: entender porque debo hacer esta llamada a start para que el perfil se cierre cuando ya estaba iniciado
puts ''
print 'Close browser?... '
                                                        if !@@profile_name.nil?
puts 'yes'
print 'Close browser... '
                                                                BlackStack::BrowserFactory.mimic(@@profile_name)
puts 'done'
print 'Close browser (2)... '
                                                                @@browser.close 
puts 'done'
                                                        else
puts 'no'
                                                        end
            rescue
              # nothing here
            end
=end

            i = 0
            max = 90
            success = 0 # I have to call this access point many times to get the browser closed.
                                                max_success_required = 30
            while i<max && success<max_success_required
#puts ''
              i+=1
             url = "http://127.0.0.1:#{BlackStack::StealthBrowserAutomation::Multilogin::mla_local_port}/api/v1/profile/stop?profileId=#{@@profile_name}"
#print "Api Call ##{i.to_s} to: #{url}... "
              uri = URI.parse(url)
              body = Net::HTTP.get(uri)
              res = JSON.parse(body)
              success += 1 if res['status'].to_s == 'OK'
#puts "response: #{res['status'].to_s}"
            end # while i<max &&  && success<max_success_required
            raise "Error requesting Mimic to stop (tried #{i.to_s} times)" if success<max_success_required

          else
            raise 'Cannot destroy browser due unknown browser type'
          end
        rescue => e
          self.release
          raise e
        end
      #end # if self.isItMe?
      
      # reseteo las variables
      @@browser = nil
      @@pid = nil
      @@profile_name = nil
      @@driver = nil
      @@type = nil
    end
getPid() click to toggle source

Returns the PID of the browser. Returns always nill if the browser is Mimic.

# File lib/browserfactory.rb, line 521
def self.getPid
  return @@pid
end
isCreated() click to toggle source

Returns true if @@browser is not nil, or @@profile_name is not nil. Otherwise, returns false If the browser type is Chrome, this method will return true if the PROFILE_PID_LIST_FILENAME has one or more PIDs registered. For more information, read documentations about PROFILE_PID_LIST_FILENAME.

# File lib/browserfactory.rb, line 412
def self.isCreated
  if @@type == BlackStack::BrowserFactory::TYPE_CHROME
    if (self.readPidToProfileList(@@profile_name).size>0)
      return true
    end
  end
  !@@browser.nil? || !@@profile_name.nil?
end
isCreated?() click to toggle source

Call isCreated() method. For more information, read the documentation of isCreated() method.

# File lib/browserfactory.rb, line 423
def self.isCreated?
  self.isCreated()
end
itIsMe?() click to toggle source

Retorna true si el perfil en @@profile_name esta siendo trabajado por este proceso

# File lib/browserfactory.rb, line 428
def self.itIsMe?
  # el archivo tiene una unica primera linea de ancho fijo
  line_chars = 100
  # obtengo el id de este proceso que esta corriendo
  my_pid = PROCESS.pid
  # bloqueo el archivo de este perfil
  @@fd_profile_lock.flock(File::LOCK_EX) if !@@fd_profile_lock.nil?
  # valido que my_pid no tenga mas caracteres que al ancho fijo
  raise 'Cannot work browser profile lockfile because the lenght if the id of this process is larger than the max number of chars allowed (#{line_chars.to_s})' if line_chars < my_pid.size 
  # obtengo el id del proceso que se reservo este perfil
  @@fd_profile_lock.seek(-line_chars, IO::SEEK_CUR) if !@@fd_profile_lock.nil?
  lock_pid = @@fd_profile_lock.read.to_s      
  # libero el archivo de este perfil
  @@fd_profile_lock.flock(File::LOCK_UN) if !@@fd_profile_lock.nil?
  # return
  my_pid.strip.upcase == lock_pid.strip.upcase
end
launch_chrome(proxy=nil, load_timeout=DEFAULT_LOAD_TIMEOUT, user_agent=nil, profile_name=nil, chromedriver_path=nil) click to toggle source

Launch a chrome browser.

# File lib/browserfactory.rb, line 770
def self.launch_chrome(proxy=nil, load_timeout=DEFAULT_LOAD_TIMEOUT, user_agent=nil, profile_name=nil, chromedriver_path=nil)
  #
  @@browser = nil
  @@driver = nil
  @@type = BlackStack::BrowserFactory::TYPE_CHROME
  @@profile_name = profile_name

  #
  #fname = PROFILE_LOCK_FILENAME.gsub('%PROFILE_NAME%', @@profile_name)
  #File.open(fname,'w') if !File.file?(fname)
  #@@fd_profile_lock = File.open(fname, 'r+')
  
  #
  agent = @@arAgents[0] if user_agent.nil?
  agent = user_agent if !user_agent.nil?
  
  # #959 - valido que el parametro agents sea un array - porque si se pasa un string se lo procesa como array y se toma un UA name de un char.
  #raise "Array expected in parameter agents in BrowserFactory.create." if !(agent.is_a? String)
  
  #
  while (@@browser == nil)            
    
    begin
      # Levantar el flag de reserva a mi favor
      self.lock()
  
      #
      #self.lockProfile()
  
      #
      self.lockProfileList()

      #
      client = Selenium::WebDriver::Remote::Http::Default.new
      begin
        client.read_timeout = load_timeout # for newest selenium webdriver version
      rescue => e
        client.timeout = load_timeout # deprecated in newest selenium webdriver version
      end
  
      # se agregan extensiones por el issue #1171
      #
      # UPDATE: Las extensiones en JavaScript para eliminar rastros, dejan rastros en si mismas.
      #
      switches = ["lang=en", "--log-level=3", "--user-agent=#{agent}", "disable-infobars", "load-extension=#{File.absolute_path('./chrome_extension')}"]
  
      #
      if (profile_name!=nil)
            
        # issue #964
        filename = "chrome_profiles/#{profile_name}/#{profile_name}/Default/Preferences"
        if ( File.file?(filename) == true )
          File.delete(filename)
        end
  
        # issue #964
        filename = "chrome_profiles/#{profile_name}/#{profile_name}/Local State"
        if ( File.file?(filename) == true )
          File.delete(filename)
        end
            
        # configuro el browser para que use este perfil
        switches += ["--user-data-dir=chrome_profiles/#{profile_name}/#{profile_name}"]
            
      end
          
      if (proxy!=nil)
        switches += proxy.chrome_switches
      end

      Selenium::WebDriver::Chrome.driver_path = chromedriver_path if !chromedriver_path.nil?
      Selenium::WebDriver::Chrome.driver_path = DEFAULT_CHROMEDRIVER_PATH if chromedriver_path.nil?
        
      @@driver = Selenium::WebDriver.for :chrome, :switches => switches
      bridge = @@driver.instance_variable_get(:@bridge)
      service = bridge.instance_variable_get(:@service)
      process = service.instance_variable_get(:@process)
      @@pid = process.pid.to_s # es el PID del chromedriver
      @@driver.manage.window().maximize()
      @@browser = PampaBrowser.new(@@driver)
      @@browser.proxy = proxy.clone if !proxy.nil?    
      @@browser.agent_name = agent
      @@browser.profile_name = profile_name
      self.updateProfileList              
  
    #
    rescue => e # TODO: Se atrapa una expecion porque sigue ocurreiendo el error reportado en el issue #134 y #129
      #
      p1 = PROCESS.list().select { |p| p[:ppid] == PROCESS.pid && p[:executablepath] =~ /chromedriver\.exe/ }.first
      if (p1 != nil)
        @@pid = p1[:pid]
        self.updateProfileList              
      end              
      self.releaseProfileList()

      #
      #self.releaseProfile()

      # Liberar el flag de reserva de creacion del browser
      self.release()
  
      # disparar la exepcion
      raise e          
    end                  

    #
    #self.releaseProfile()
        
    # Liberar el flag de reserva de creacion del browser
    self.release()

  end # while
  
  #
  @@browser
end
lock() click to toggle source

This method will lock the lockfile regarding this host, in order to get all processes in the same host launching one browser at time.

Since chromedriver can't take one starting request at time, this method is to force the BrowserFactory classs, running in many processes of the same host at the same time, to try

to launch browsers at the same time.

# File lib/browserfactory.rb, line 533
def self.lock
  @@fd_host_lock.flock(File::LOCK_EX) if @@type == BlackStack::BrowserFactory::TYPE_CHROME
end
lockProfile() click to toggle source

This method is to open no more than one browser of the same profile at time. It will lock the file at the begining. If the content of the file is the PID of this process, so this process is good to start the browser. If the content of the file is different than the PID of this process, but such content is not equal to the PID of any other process alive, so this process is good to start the browser. If the content of the file is different than the PID of this process, and such content is equal to the PID of another process alive, this method unlock the file and will raise an exception.

# File lib/browserfactory.rb, line 451
def self.lockProfile
  # el archivo tiene una unica primera linea de ancho fijo
  line_chars = 100
  # bloqueo el archivo de este perfil
  @@fd_profile_lock.flock(File::LOCK_EX)
  # obtengo el id de este proceso que esta corriendo
  my_pid = PROCESS.pid
  # valido que my_pid no tenga mas caracteres que al ancho fijo
  raise 'Cannot work browser profile lockfile because the lenght if the id of this process is larger than the max number of chars allowed (#{line_chars.to_s})' if line_chars < my_pid.size 
  # obtengo el id del proceso que se reservo este perfil
  lock_pid = @@fd_profile_lock.read.to_s
  #
  if lock_pid.size == 0
    @@fd_profile_lock.write my_pid
    @@fd_profile_lock.write ' ' * (line_chars - my_pid.size)
    @@fd_profile_lock.flush
  elsif lock_pid.strip.upcase != my_pid.strip.upcase
    # verifico si el proceso que reservo este perfil esta activo
    p = PROCESS.list().select { |h| h[:pid].strip.upcase == lock_pid.strip.upcase }.first
    lock_process_is_alive = !p.nil?
    # levanto una excepcion de bloqueo si este perfil fue reservado por otro proceso y el proceso sigue activo
    if lock_process_is_alive
      # libero el archivo de este perfil
      @@fd_profile_lock.flock(File::LOCK_UN)
      #
      raise 'Profile is locked'
    # me reservo este perfil si fue reservado por un proceso que ya no esta activo
    else # if lock_process_is_alive
      @@fd_profile_lock.seek(-line_chars, IO::SEEK_CUR)
      @@fd_profile_lock.write my_pid
      @@fd_profile_lock.write ' ' * (line_chars - my_pid.size)
      @@fd_profile_lock.flush
    end # if lock_process_is_alive
  else # lock_pid.strip.upcase == my_pid.strip.upcase
    # go ahead
  end # if lock_pid.size == 0
end
lockProfileList() click to toggle source

Lock PROFILE_PID_LIST_FILENAME file. For more information, read documentations about PROFILE_PID_LIST_FILENAME.

# File lib/browserfactory.rb, line 353
def self.lockProfileList()
  #@@fd_profile.flock(File::LOCK_EX)
end
mimic(mimic_profile_id, load_timeout=DEFAULT_LOAD_TIMEOUT) click to toggle source

Launch a Mimic browser.

# File lib/browserfactory.rb, line 663
def self.mimic(mimic_profile_id, load_timeout=DEFAULT_LOAD_TIMEOUT)
  #
  @@type = BlackStack::BrowserFactory::TYPE_MIMIC
  @@profile_name = mimic_profile_id

  #
  #fname = PROFILE_LOCK_FILENAME.gsub('%PROFILE_NAME%', @@profile_name)
  #File.open(fname,'w') if !File.file?(fname)
  #@@fd_profile_lock = File.open(fname,'r+')

  begin          
    # Levantar el flag de reserva a mi favor
    self.lock()
  
    #
    #self.lockProfile()

    url_2 = nil
    i = 0
    max = 3
    while (i<max && url_2.nil?)
      i += 1

      # cierro el perfil por si estaba corriendo
      url = "http://127.0.0.1:#{BlackStack::StealthBrowserAutomation::Multilogin::mla_local_port}/api/v1/profile/stop?profileId=#{@@profile_name}"
      uri = URI.parse(url)
      body = Net::HTTP.get(uri)
      res = JSON.parse(body)

      url = "http://127.0.0.1:#{BlackStack::StealthBrowserAutomation::Multilogin::mla_local_port}/api/v1/profile/start?automation=true&profileId=#{@@profile_name}"
      uri = URI.parse(url)
      body = Net::HTTP.get(uri)
      res = JSON.parse(body)

      if !res.has_key?('status')
        if i>=max
          self.release() # libero el flag de reserva de turno para crar un browser
          raise "Error requesting Mimic to start (try #{i.to_s}): #{res.to_s}"
        end
      elsif res['status'] != 'OK'
        if i>=max
          self.release() # libero el flag de reserva de turno para crar un browser
          raise "Error requesting Mimic to start (try #{i.to_s}): #{res.to_s}"
        end
      else
        begin
          url_2 = res['value']
          if !url_2.nil?
            #uri_2 = URI.parse(url_2)
            #req_2 = Net::HTTP::Get.new(url_2)
            #res_2 = Net::HTTP.start(uri_2.host, uri_2.port, :use_ssl => true, :verify_mode => OpenSSL::SSL::VERIFY_NONE) {|http|
            #  http.request(req_2)
            #}
            @@driver = Selenium::WebDriver.for(:remote, :url => url_2)
                                                            @@driver.manage.timeouts.page_load = load_timeout
            #bridge = @@driver.instance_variable_get(:@bridge)
            #service = bridge.instance_variable_get(:@service)
            #process = service.instance_variable_get(:@process)
            #@@pid = process.pid.to_s # es el PID del chromedriver
            @@driver.manage.window().maximize()
            @@browser = PampaBrowser.new(@@driver) # Watir::Browser.new(@@driver)
          end
        rescue => e
          raise e if i>=max
        end
      end # if !res.has_key?('status')
    end # while
  rescue => e
    #
    #self.releaseProfile()
    # libero el flag de reserva de turno para crar un browser
    self.release() 
    #
    raise e
  end

  #
  #self.releaseProfile()

  # libero el flag de reserva de turno para crar un browser
  self.release() 

  #
  @@browser
end
readPidToProfileList(profile_name) click to toggle source

Returns an array of strings with the PIDs registered for this browser. For more information, read documentations about PROFILE_PID_LIST_FILENAME. This method is used for chrome browsers only.

# File lib/browserfactory.rb, line 388
def self.readPidToProfileList(profile_name)
  if profile_name != nil
    fname = PROFILE_PID_LIST_FILENAME.gsub('%PROFILE_NAME%', profile_name)
    s = File.read(fname)
    return s.split(",")
  else
    return []
  end
end
release() click to toggle source

Releases the lockfile regarding this host.

# File lib/browserfactory.rb, line 538
def self.release
  @@fd_host_lock.flock(File::LOCK_UN) if @@type == BlackStack::BrowserFactory::TYPE_CHROME
end
releaseProfile() click to toggle source

This method is to open no more than one browser of the same profile at time. It will unlock the file at the begining.

# File lib/browserfactory.rb, line 491
def self.releaseProfile
  # libero el archivo de este perfil
  @@fd_profile_lock.flock(File::LOCK_UN)
end
releaseProfileList() click to toggle source

Unlock PROFILE_PID_LIST_FILENAME file. For more information, read documentations about PROFILE_PID_LIST_FILENAME.

# File lib/browserfactory.rb, line 359
def self.releaseProfileList()
  #@@fd_profile.flock(File::LOCK_UN)
end
removePidToProfileList(profile_name, pid) click to toggle source

Remove a PID from the PROFILE_PID_LIST_FILENAME file. For more information, read documentations about PROFILE_PID_LIST_FILENAME. This method is used for chrome browsers only.

# File lib/browserfactory.rb, line 376
def self.removePidToProfileList(profile_name, pid)
  fname = PROFILE_PID_LIST_FILENAME.gsub('%PROFILE_NAME%', profile_name)
  s = File.read(fname)
  s = s.gsub("#{pid},", "")
  f = File.open(fname, 'w')
  f.write(s)
  f.close
end
screenshot(filename) click to toggle source

Toma una captura del area visible de la pagina. Mas informacion: stackoverflow.com/questions/25543651/screenshot-of-entire-webpage-using-selenium-webdriver-rc-rails

# File lib/browserfactory.rb, line 510
def self.screenshot(filename)
  @@driver.save_screenshot(filename)
end
typeDesc(type) click to toggle source

Returns the description string from a browser type code

# File lib/browserfactory.rb, line 345
def self.typeDesc(type)
  return 'Chrome' if @@type == BlackStack::BrowserFactory::TYPE_CHROME
  return 'Mimic' if @@type == BlackStack::BrowserFactory::TYPE_MIMIC
  raise 'Unknown browser type'
end
types() click to toggle source

Returns the list of supported browsers

# File lib/browserfactory.rb, line 340
def self.types
  [BlackStack::BrowserFactory::TYPE_CHROME, BlackStack::BrowserFactory::TYPE_MIMIC]
end
updateProfileList() click to toggle source

Add the PID of this browser; and the PID of all the child processes of this, with name like /chrome.exe/. For more information, read documentations about PROFILE_PID_LIST_FILENAME. This method is used for chrome browsers only.

# File lib/browserfactory.rb, line 401
def self.updateProfileList
  addPidToProfileList(@@profile_name, @@pid)
  PROCESS.list().select { |p| p[:ppid] == @@pid && p[:executablepath] =~ /chrome\.exe/ }.each { |p2|
    addPidToProfileList(@@profile_name, p2[:pid])                  
  }
end