class BlackStack::BrowserFactory
Constants
- DEFAULT_CHROMEDRIVER_PATH
Location of the chromedriver.exe file by default. Works for chrome browsers only.
- DEFAULT_LOAD_TIMEOUT
Page load timeout by default. Works for chrome browsers only.
- HOST_LOCK_FILENAME
This file is to launch one browser at time in the same host. There is one, and only one, lock file for reach host where Blackstack is running. More information here: bitbucket.org/leandro_sardi/blackstack/issues/1155/browserfactory-use-one-single-lock-file
- PROFILE_LOCK_FILENAME
This file store the PID of the last processes that started a browser with this profile name. There is one, and only one, pid_lockfile for each browser profile; and such pid_lockfile is shared by all the hosts where
BlackStack
is running.- PROFILE_PID_LIST_FILENAME
This file keeps the PIDs of all the child processes of this process. It is neccessary to kill all the child processes of this process when you have started a chrome browser, and you want to be sure that you have closed it. More infromation here: bitbucket.org/leandro_sardi/blackstack/issues/943/browserfactory-deja-exploradores-abiertos
- TYPE_CHROME
Different supported browsers.
- TYPE_MIMIC
Public Class Methods
Add a PID to the PROFILE_PID_LIST_FILENAME
file. For more information, read documentations about PROFILE_PID_LIST_FILENAME
. This method is used for chrome browsers only.
# File lib/browserfactory.rb, line 366 def self.addPidToProfileList(profile_name, pid) fname = PROFILE_PID_LIST_FILENAME.gsub('%PROFILE_NAME%', profile_name) f = File.open(fname, 'a') f.write("#{pid},") f.close end
Returns a suggested list of user-agents
# File lib/browserfactory.rb, line 515 def self.agents return @@arAgents end
Returns the PampaBrowser
object
# File lib/browserfactory.rb, line 335 def self.browser @@browser end
It will call launch_chrome
to launch a chrome browser. The hash parameters musy have the following keys:
-
proxy: a
BlackStack::RemoteProxy
object; -
load_timeout: max number of seconds to wait a page to load;
-
user_agent: user agent of the browser;
-
profile_name: will create a folder with ths name to store all the data of the chrome profile (history, cookies, saved passwords, etc);
-
chromedriver_path: location of the chromedriver.exe file, will use
DEFAULT_CHROMEDRIVER_PATH
by default.
# File lib/browserfactory.rb, line 756 def self.chrome(h) # TODO: check if h is a hash # TODO: check the variable type of each param # TODO: check mandatory params self.launch_chrome( h[:proxy], h[:load_timeout].nil? ? DEFAULT_LOAD_TIMEOUT : h[:load_timeout], h[:user_agent], h[:profile_name], h[:chromedriver_path] ) end
Returns the version of the availalble chrome browser.
# File lib/browserfactory.rb, line 641 def self.chrome_version res = `reg query \"HKEY_CURRENT_USER\\Software\\Google\\Chrome\\BLBeacon\" /v version`.scan(/REG_SZ (.*)$/).first return nil if res.nil? return res[0] if res.size > 0 return nil end
Returns the version of the availalble chromedriver. The filename parameter is the full path of the chromedriver.exe command. If filename parameter is nil, it will look for the chromedriver.exe in the PATH environment.
Not valid for Mimic, since MLA handle its own chromedriver into its installation folder.
# File lib/browserfactory.rb, line 654 def self.chromedriver_version(filename=nil) res = `chromedriver -v`.scan(/ChromeDriver\s(.*)\s\(/).first if filename.nil? res = `#{filename} -v`.scan(/ChromeDriver\s(.*)\s\(/).first if !filename.nil? return nil if res.nil? return res[0] if res.size > 0 return nil end
Close the browser only if itIsMe? returns true. Set nil to @@browser, @@pid, @@profile_name, @@driver, @@type.
# File lib/browserfactory.rb, line 544 def self.destroy #if self.itIsMe? begin # borro el contenido de PROFILE_LOCK_FILENAME #if !@@profile_name.nil? # fname = PROFILE_LOCK_FILENAME.gsub('%PROFILE_NAME%', @@profile_name) # File.open(fname,'w') #end # if @@type == BlackStack::BrowserFactory::TYPE_CHROME # kill all child process self.lockProfileList() self.readPidToProfileList(@@profile_name).each { |pid| MyProcess.kill(pid) self.removePidToProfileList(@@profile_name, pid) } self.releaseProfileList() begin @@browser.close if !@@browser.nil? rescue end elsif @@type == BlackStack::BrowserFactory::TYPE_MIMIC =begin puts '' begin print 'Close browser?... ' if !@@browser.nil? puts 'yes' print 'Close browser... ' @@browser.close puts 'done' else puts 'no' end rescue => e puts "error: #{e.to_s}" end =end =begin begin # TODO: entender porque debo hacer esta llamada a start para que el perfil se cierre cuando ya estaba iniciado puts '' print 'Close browser?... ' if !@@profile_name.nil? puts 'yes' print 'Close browser... ' BlackStack::BrowserFactory.mimic(@@profile_name) puts 'done' print 'Close browser (2)... ' @@browser.close puts 'done' else puts 'no' end rescue # nothing here end =end i = 0 max = 90 success = 0 # I have to call this access point many times to get the browser closed. max_success_required = 30 while i<max && success<max_success_required #puts '' i+=1 url = "http://127.0.0.1:#{BlackStack::StealthBrowserAutomation::Multilogin::mla_local_port}/api/v1/profile/stop?profileId=#{@@profile_name}" #print "Api Call ##{i.to_s} to: #{url}... " uri = URI.parse(url) body = Net::HTTP.get(uri) res = JSON.parse(body) success += 1 if res['status'].to_s == 'OK' #puts "response: #{res['status'].to_s}" end # while i<max && && success<max_success_required raise "Error requesting Mimic to stop (tried #{i.to_s} times)" if success<max_success_required else raise 'Cannot destroy browser due unknown browser type' end rescue => e self.release raise e end #end # if self.isItMe? # reseteo las variables @@browser = nil @@pid = nil @@profile_name = nil @@driver = nil @@type = nil end
Returns the PID of the browser. Returns always nill if the browser is Mimic.
# File lib/browserfactory.rb, line 521 def self.getPid return @@pid end
Returns true if @@browser is not nil, or @@profile_name is not nil. Otherwise, returns false If the browser type is Chrome, this method will return true if the PROFILE_PID_LIST_FILENAME
has one or more PIDs registered. For more information, read documentations about PROFILE_PID_LIST_FILENAME
.
# File lib/browserfactory.rb, line 412 def self.isCreated if @@type == BlackStack::BrowserFactory::TYPE_CHROME if (self.readPidToProfileList(@@profile_name).size>0) return true end end !@@browser.nil? || !@@profile_name.nil? end
Call isCreated() method. For more information, read the documentation of isCreated() method.
# File lib/browserfactory.rb, line 423 def self.isCreated? self.isCreated() end
Retorna true si el perfil en @@profile_name esta siendo trabajado por este proceso
# File lib/browserfactory.rb, line 428 def self.itIsMe? # el archivo tiene una unica primera linea de ancho fijo line_chars = 100 # obtengo el id de este proceso que esta corriendo my_pid = PROCESS.pid # bloqueo el archivo de este perfil @@fd_profile_lock.flock(File::LOCK_EX) if !@@fd_profile_lock.nil? # valido que my_pid no tenga mas caracteres que al ancho fijo raise 'Cannot work browser profile lockfile because the lenght if the id of this process is larger than the max number of chars allowed (#{line_chars.to_s})' if line_chars < my_pid.size # obtengo el id del proceso que se reservo este perfil @@fd_profile_lock.seek(-line_chars, IO::SEEK_CUR) if !@@fd_profile_lock.nil? lock_pid = @@fd_profile_lock.read.to_s # libero el archivo de este perfil @@fd_profile_lock.flock(File::LOCK_UN) if !@@fd_profile_lock.nil? # return my_pid.strip.upcase == lock_pid.strip.upcase end
Launch a chrome browser.
# File lib/browserfactory.rb, line 770 def self.launch_chrome(proxy=nil, load_timeout=DEFAULT_LOAD_TIMEOUT, user_agent=nil, profile_name=nil, chromedriver_path=nil) # @@browser = nil @@driver = nil @@type = BlackStack::BrowserFactory::TYPE_CHROME @@profile_name = profile_name # #fname = PROFILE_LOCK_FILENAME.gsub('%PROFILE_NAME%', @@profile_name) #File.open(fname,'w') if !File.file?(fname) #@@fd_profile_lock = File.open(fname, 'r+') # agent = @@arAgents[0] if user_agent.nil? agent = user_agent if !user_agent.nil? # #959 - valido que el parametro agents sea un array - porque si se pasa un string se lo procesa como array y se toma un UA name de un char. #raise "Array expected in parameter agents in BrowserFactory.create." if !(agent.is_a? String) # while (@@browser == nil) begin # Levantar el flag de reserva a mi favor self.lock() # #self.lockProfile() # self.lockProfileList() # client = Selenium::WebDriver::Remote::Http::Default.new begin client.read_timeout = load_timeout # for newest selenium webdriver version rescue => e client.timeout = load_timeout # deprecated in newest selenium webdriver version end # se agregan extensiones por el issue #1171 # # UPDATE: Las extensiones en JavaScript para eliminar rastros, dejan rastros en si mismas. # switches = ["lang=en", "--log-level=3", "--user-agent=#{agent}", "disable-infobars", "load-extension=#{File.absolute_path('./chrome_extension')}"] # if (profile_name!=nil) # issue #964 filename = "chrome_profiles/#{profile_name}/#{profile_name}/Default/Preferences" if ( File.file?(filename) == true ) File.delete(filename) end # issue #964 filename = "chrome_profiles/#{profile_name}/#{profile_name}/Local State" if ( File.file?(filename) == true ) File.delete(filename) end # configuro el browser para que use este perfil switches += ["--user-data-dir=chrome_profiles/#{profile_name}/#{profile_name}"] end if (proxy!=nil) switches += proxy.chrome_switches end Selenium::WebDriver::Chrome.driver_path = chromedriver_path if !chromedriver_path.nil? Selenium::WebDriver::Chrome.driver_path = DEFAULT_CHROMEDRIVER_PATH if chromedriver_path.nil? @@driver = Selenium::WebDriver.for :chrome, :switches => switches bridge = @@driver.instance_variable_get(:@bridge) service = bridge.instance_variable_get(:@service) process = service.instance_variable_get(:@process) @@pid = process.pid.to_s # es el PID del chromedriver @@driver.manage.window().maximize() @@browser = PampaBrowser.new(@@driver) @@browser.proxy = proxy.clone if !proxy.nil? @@browser.agent_name = agent @@browser.profile_name = profile_name self.updateProfileList # rescue => e # TODO: Se atrapa una expecion porque sigue ocurreiendo el error reportado en el issue #134 y #129 # p1 = PROCESS.list().select { |p| p[:ppid] == PROCESS.pid && p[:executablepath] =~ /chromedriver\.exe/ }.first if (p1 != nil) @@pid = p1[:pid] self.updateProfileList end self.releaseProfileList() # #self.releaseProfile() # Liberar el flag de reserva de creacion del browser self.release() # disparar la exepcion raise e end # #self.releaseProfile() # Liberar el flag de reserva de creacion del browser self.release() end # while # @@browser end
This method will lock the lockfile regarding this host, in order to get all processes in the same host launching one browser at time.
Since chromedriver can't take one starting request at time, this method is to force the BrowserFactory
classs, running in many processes of the same host at the same time, to try
to launch browsers at the same time.
# File lib/browserfactory.rb, line 533 def self.lock @@fd_host_lock.flock(File::LOCK_EX) if @@type == BlackStack::BrowserFactory::TYPE_CHROME end
This method is to open no more than one browser of the same profile at time. It will lock the file at the begining. If the content of the file is the PID of this process, so this process is good to start the browser. If the content of the file is different than the PID of this process, but such content is not equal to the PID of any other process alive, so this process is good to start the browser. If the content of the file is different than the PID of this process, and such content is equal to the PID of another process alive, this method unlock the file and will raise an exception.
# File lib/browserfactory.rb, line 451 def self.lockProfile # el archivo tiene una unica primera linea de ancho fijo line_chars = 100 # bloqueo el archivo de este perfil @@fd_profile_lock.flock(File::LOCK_EX) # obtengo el id de este proceso que esta corriendo my_pid = PROCESS.pid # valido que my_pid no tenga mas caracteres que al ancho fijo raise 'Cannot work browser profile lockfile because the lenght if the id of this process is larger than the max number of chars allowed (#{line_chars.to_s})' if line_chars < my_pid.size # obtengo el id del proceso que se reservo este perfil lock_pid = @@fd_profile_lock.read.to_s # if lock_pid.size == 0 @@fd_profile_lock.write my_pid @@fd_profile_lock.write ' ' * (line_chars - my_pid.size) @@fd_profile_lock.flush elsif lock_pid.strip.upcase != my_pid.strip.upcase # verifico si el proceso que reservo este perfil esta activo p = PROCESS.list().select { |h| h[:pid].strip.upcase == lock_pid.strip.upcase }.first lock_process_is_alive = !p.nil? # levanto una excepcion de bloqueo si este perfil fue reservado por otro proceso y el proceso sigue activo if lock_process_is_alive # libero el archivo de este perfil @@fd_profile_lock.flock(File::LOCK_UN) # raise 'Profile is locked' # me reservo este perfil si fue reservado por un proceso que ya no esta activo else # if lock_process_is_alive @@fd_profile_lock.seek(-line_chars, IO::SEEK_CUR) @@fd_profile_lock.write my_pid @@fd_profile_lock.write ' ' * (line_chars - my_pid.size) @@fd_profile_lock.flush end # if lock_process_is_alive else # lock_pid.strip.upcase == my_pid.strip.upcase # go ahead end # if lock_pid.size == 0 end
Lock PROFILE_PID_LIST_FILENAME
file. For more information, read documentations about PROFILE_PID_LIST_FILENAME
.
# File lib/browserfactory.rb, line 353 def self.lockProfileList() #@@fd_profile.flock(File::LOCK_EX) end
Launch a Mimic browser.
# File lib/browserfactory.rb, line 663 def self.mimic(mimic_profile_id, load_timeout=DEFAULT_LOAD_TIMEOUT) # @@type = BlackStack::BrowserFactory::TYPE_MIMIC @@profile_name = mimic_profile_id # #fname = PROFILE_LOCK_FILENAME.gsub('%PROFILE_NAME%', @@profile_name) #File.open(fname,'w') if !File.file?(fname) #@@fd_profile_lock = File.open(fname,'r+') begin # Levantar el flag de reserva a mi favor self.lock() # #self.lockProfile() url_2 = nil i = 0 max = 3 while (i<max && url_2.nil?) i += 1 # cierro el perfil por si estaba corriendo url = "http://127.0.0.1:#{BlackStack::StealthBrowserAutomation::Multilogin::mla_local_port}/api/v1/profile/stop?profileId=#{@@profile_name}" uri = URI.parse(url) body = Net::HTTP.get(uri) res = JSON.parse(body) url = "http://127.0.0.1:#{BlackStack::StealthBrowserAutomation::Multilogin::mla_local_port}/api/v1/profile/start?automation=true&profileId=#{@@profile_name}" uri = URI.parse(url) body = Net::HTTP.get(uri) res = JSON.parse(body) if !res.has_key?('status') if i>=max self.release() # libero el flag de reserva de turno para crar un browser raise "Error requesting Mimic to start (try #{i.to_s}): #{res.to_s}" end elsif res['status'] != 'OK' if i>=max self.release() # libero el flag de reserva de turno para crar un browser raise "Error requesting Mimic to start (try #{i.to_s}): #{res.to_s}" end else begin url_2 = res['value'] if !url_2.nil? #uri_2 = URI.parse(url_2) #req_2 = Net::HTTP::Get.new(url_2) #res_2 = Net::HTTP.start(uri_2.host, uri_2.port, :use_ssl => true, :verify_mode => OpenSSL::SSL::VERIFY_NONE) {|http| # http.request(req_2) #} @@driver = Selenium::WebDriver.for(:remote, :url => url_2) @@driver.manage.timeouts.page_load = load_timeout #bridge = @@driver.instance_variable_get(:@bridge) #service = bridge.instance_variable_get(:@service) #process = service.instance_variable_get(:@process) #@@pid = process.pid.to_s # es el PID del chromedriver @@driver.manage.window().maximize() @@browser = PampaBrowser.new(@@driver) # Watir::Browser.new(@@driver) end rescue => e raise e if i>=max end end # if !res.has_key?('status') end # while rescue => e # #self.releaseProfile() # libero el flag de reserva de turno para crar un browser self.release() # raise e end # #self.releaseProfile() # libero el flag de reserva de turno para crar un browser self.release() # @@browser end
Returns an array of strings with the PIDs registered for this browser. For more information, read documentations about PROFILE_PID_LIST_FILENAME
. This method is used for chrome browsers only.
# File lib/browserfactory.rb, line 388 def self.readPidToProfileList(profile_name) if profile_name != nil fname = PROFILE_PID_LIST_FILENAME.gsub('%PROFILE_NAME%', profile_name) s = File.read(fname) return s.split(",") else return [] end end
Releases the lockfile regarding this host.
# File lib/browserfactory.rb, line 538 def self.release @@fd_host_lock.flock(File::LOCK_UN) if @@type == BlackStack::BrowserFactory::TYPE_CHROME end
This method is to open no more than one browser of the same profile at time. It will unlock the file at the begining.
# File lib/browserfactory.rb, line 491 def self.releaseProfile # libero el archivo de este perfil @@fd_profile_lock.flock(File::LOCK_UN) end
Unlock PROFILE_PID_LIST_FILENAME
file. For more information, read documentations about PROFILE_PID_LIST_FILENAME
.
# File lib/browserfactory.rb, line 359 def self.releaseProfileList() #@@fd_profile.flock(File::LOCK_UN) end
Remove a PID from the PROFILE_PID_LIST_FILENAME
file. For more information, read documentations about PROFILE_PID_LIST_FILENAME
. This method is used for chrome browsers only.
# File lib/browserfactory.rb, line 376 def self.removePidToProfileList(profile_name, pid) fname = PROFILE_PID_LIST_FILENAME.gsub('%PROFILE_NAME%', profile_name) s = File.read(fname) s = s.gsub("#{pid},", "") f = File.open(fname, 'w') f.write(s) f.close end
Toma una captura del area visible de la pagina. Mas informacion: stackoverflow.com/questions/25543651/screenshot-of-entire-webpage-using-selenium-webdriver-rc-rails
# File lib/browserfactory.rb, line 510 def self.screenshot(filename) @@driver.save_screenshot(filename) end
Returns the description string from a browser type code
# File lib/browserfactory.rb, line 345 def self.typeDesc(type) return 'Chrome' if @@type == BlackStack::BrowserFactory::TYPE_CHROME return 'Mimic' if @@type == BlackStack::BrowserFactory::TYPE_MIMIC raise 'Unknown browser type' end
Returns the list of supported browsers
# File lib/browserfactory.rb, line 340 def self.types [BlackStack::BrowserFactory::TYPE_CHROME, BlackStack::BrowserFactory::TYPE_MIMIC] end
Add the PID of this browser; and the PID of all the child processes of this, with name like /chrome.exe/. For more information, read documentations about PROFILE_PID_LIST_FILENAME
. This method is used for chrome browsers only.
# File lib/browserfactory.rb, line 401 def self.updateProfileList addPidToProfileList(@@profile_name, @@pid) PROCESS.list().select { |p| p[:ppid] == @@pid && p[:executablepath] =~ /chrome\.exe/ }.each { |p2| addPidToProfileList(@@profile_name, p2[:pid]) } end