module TestRailRSpecIntegration
Public Class Methods
Adds a documentation formatter to the rspec if one is not there already.
# File lib/files/testrail_rspec_integration.rb, line 234 def self.add_formatter_for(config) # For some reason, adding a custom formatter will remove any other formatters. # Thus during execution nothing gets printed to the screen. Need to add another # formatter to indicate some sort of progress found_doc_formatter = false config.formatters.each do |fm| if (fm.class == RSpec::Core::Formatters::DocumentationFormatter) found_doc_formatter = true break end end unless found_doc_formatter config.add_formatter "doc" end end
Registers a callback custom formatter to an rspec. The new test run is created from the results of the tests. This is in effect the opposite of the method above (register_rspec_integration
).
# File lib/files/testrail_rspec_integration.rb, line 425 def self.add_rspec_callback(config, product, add_formatter: true) config.add_formatter TestRailRSpecIntegration::TestRailPlanFormatter TestRailRSpecIntegration::TestRailPlanFormatter.set_product(product) if add_formatter TestRailRSpecIntegration.add_formatter_for(config) end end
Filters an rspec run by testrail_id's for bridge Filters an rspec run by testcases found in a particular testrun on testrail.
# File lib/files/testrail_rspec_integration.rb, line 287 def self.filter_rspecs_by_test_run(config, test_run_cases) # This lambda gets called once for each example # Here value is an array of string test case ID's. config.filter_run_including testrail_id: lambda { |value| @@total_count += 1 unless value.is_a? Array @@skip_count += 1 puts "ERROR! testcase has invalid testrail ID: #{value}. Value should be an array, got: #{value.class}".red return false end # The test id's are strings. Convert them to integers to make comparison easier test_ids = value.collect { |str| str.to_i } # Compute the intersection using the handy &() method intersect = test_run_cases.keys & test_ids # Do not include if the test cases have already been run and have ALL passed. # (That would be a waste of time to rerun test's that have already passed) pass_count = 0 skip_count = 0 # Do include if the intersection contains a test id if intersect.size > 0 test_ids.each do |id| test_case = test_run_cases[id] if test_case.nil? next end # puts " #{id} temp id: #{test_case.temp_id} Status: #{test_case.status}, " pass_count += 1 if test_case.status == :passed skip_count += 1 if test_case.status == :pending end all_passed = pass_count == test_ids.count all_skipped = skip_count == test_ids.count if all_passed @@skip_count += 1 puts "Skipping test case #{value}, because all tests already passed" end if all_skipped @@skip_count += 1 puts "Skipping test case #{value}, because all tests marked pending" end do_execute = (pass_count + skip_count) != test_ids.count @@run_count += 1 if do_execute do_execute else @@skip_count += 1 false end } end
Takes care of filtering out tests that are NOT assigned to the user. So essentially runs only tests specified in a testrun in testrail, and that are assigned to a particular user. config - The Rspec configuration user_id - An integer ID corresponding to the testrail user test_run_cases - A hash of TestCase instances
# File lib/files/testrail_rspec_integration.rb, line 255 def self.filter_rspecs_by_test_run_and_user(config, user_id, test_run_cases) config.filter_run_including testrail_id: lambda { |value| # The test id's are strings. Convert them to integers to make comparison easier test_ids = value.collect { |str| str.to_i } # Compute the intersection using the handy &() method intersect = test_run_cases.keys & test_ids assigned_to_ids = [] # Do include if the intersection contains a test id if intersect.size > 0 test_ids.each do |id| test_case = test_run_cases[id] if test_case.nil? next end assigned_to_ids << test_case.assigned_to end # return true to execute the test if any one of the testcase ID's is assigned to the user do_execute = assigned_to_ids.include? user_id if do_execute puts "Assigned to user. Including testcase ID's: #{value}" else puts "Not assigned to user: Skipping #{value}" end do_execute else false end } end
Filters an rspec run by test_id's for canvas. This is used for filtering out test cases that have already been run previously, say on a previous test run that was aborted early and restarted. In this case we skip tests that already passed or were marked as pending (rspec for skipped)
# File lib/files/testrail_rspec_integration.rb, line 340 def self.filter_rspecs_by_testid(config, test_run_cases) # This lambda gets called once for each example # Here value is an array of string test case ID's. config.filter_run_including test_id: lambda { |id| @@total_count += 1 id = id.to_i # The test id's are integers, and in canvas there is only one ID per test case, NOT an array like Bridge in_run = test_run_cases.keys.include?( id ) # Do include if the intersection contains a test id if in_run test_case = test_run_cases[id] if (test_case.status == :passed) @@skip_count += 1 puts "Skipping test case #{id}, because it has already passed" return false end if (test_case.status == :pending) @@skip_count += 1 puts "Skipping test case #{id}, because it is marked pending" return false end @@run_count += 1 return true # do execute this test else @@skip_count += 1 puts "Skipping test case #{id}, because it was not in test_run_cases" return false end } end
# File lib/files/testrail_rspec_integration.rb, line 229 def self.get_run_count @@run_count end
# File lib/files/testrail_rspec_integration.rb, line 225 def self.get_skip_count @@skip_count end
# File lib/files/testrail_rspec_integration.rb, line 221 def self.get_total_count @@total_count end
The param is an RSPEC config The second param is a symbol for which product to hook into
# File lib/files/testrail_rspec_integration.rb, line 377 def self.register_rspec_integration(config, product, add_formatter: true) # Runs test cases as found in a test run on testrail # This will select test examples to run based off of what test rail defines, not what # the file pattern on the command line defines. # That is, this will take a test run (int test rail), and run all the cases defined in it. # First clear any filters passed in from the command line config.inclusion_filter = nil test_run_cases = TestRailOperations.get_test_run_cases(ENV["TESTRAIL_RUN_ID"].to_i) user_id = nil unless ENV["TESTRAIL_ASSIGNED_TO"].nil? user_json = TestRailOperations.get_test_rail_user_by_email(ENV["TESTRAIL_ASSIGNED_TO"]) user_id = user_json["id"] puts "Testrail assigned to: #{user_json}" end if user_id TestRailRSpecIntegration.filter_rspecs_by_test_run_and_user(config, user_id, test_run_cases) else case(product) when :bridge TestRailRSpecIntegration.filter_rspecs_by_test_run(config, test_run_cases) when :canvas TestRailRSpecIntegration.filter_rspecs_by_testid(config, test_run_cases) end end config.add_formatter TestRailRSpecIntegration::TestRailPlanFormatter TestRailRSpecIntegration::TestRailPlanFormatter.set_product(product) if add_formatter TestRailRSpecIntegration.add_formatter_for(config) end # Captures and posts results for any remaining test case results in @@cases that don't fill a full batch config.after(:suite) do |suite| total_cases = TestRailPlanFormatter.class_variable_get(:@@cases) if total_cases.size > 0 TestRailRSpecIntegration::TestRailPlanFormatter.post_results total_cases end end end