class Scrapinghub::Jobs

Public Class Methods

new(api_key:) click to toggle source

Initialize a new Jobs API client

@param api_key [String] Scrapinghub API key

@return Object

# File lib/scrapinghub/jobs.rb, line 18
def initialize(api_key:)
  @api_key = api_key
end

Public Instance Methods

delete(args) click to toggle source

Delete one or more jobs.

@param project [Fixnum] the project’s numeric ID @param job [String, Array<String>] the ID of a specific job to delete

@return [Kleisli::Left] if validation fails (e.g. bad authentication) or

if there were any low-level exceptions (e.g. the host is down), with a
message detailing the failure.

@return [Kleisli::Right] if the operation was successful.

# File lib/scrapinghub/jobs.rb, line 141
def delete(args)
  options = { body: args, basic_auth: { username: @api_key } }
  Try { self.class.post("/api/jobs/delete.json", options) }.to_either >-> response {
    if response.code == 200
      Right(response)
    else
      Left(response)
    end
  }
end
list(args) click to toggle source

Retrieve information about jobs.

@param project [Fixnum] the project’s numeric ID @param job [String, Array<String>] (optional) ID(s) of specific jobs to

retrieve

@param spider [String] (optional) a spider name (only jobs belonging to

this spider will be returned)

@param state [String] (optional) return only jobs with this state. Valid

values: "pending", "running", "finished"

@param has_tag [String, Array<String>] (optional) return only jobs

containing the given tag(s)

@param lacks_tag [String, Array<String>] (optional) return only jobs not

containing the given tag(s)

@param count [Fixnum] (optional) maximum number of jobs to return

@return [Kleisli::Left] if validation fails (e.g. bad authentication) or

if there were any low-level exceptions (e.g. the host is down), with a
message detailing the failure.

@return [Kleisli::Right] if the operation was successful.

# File lib/scrapinghub/jobs.rb, line 48
def list(args)
  options = { query: args, basic_auth: { username: @api_key } }
  Try { self.class.get("/api/jobs/list.json", options) }.to_either >-> response {
    if response.code == 200
      Right(response)
    else
      Left(response)
    end
  }
end
schedule(args) click to toggle source

Schedule a job.

@param project [Fixnum] the project’s numeric ID @param spider [String] the spider name @param add_tag [String, Array<String>] (optional) add tag(s) to the job @param priority [Fixnum] (optional) set the job priority: possible values

range from 0 (lowest priority) to 4 (highest priority), default is 2

@param extra [Hash] (optional) extra parameters passed as spider

arguments

@return [Kleisli::Left] if validation fails (e.g. bad authentication) or

if there were any low-level exceptions (e.g. the host is down), with a
message detailing the failure.

@return [Kleisli::Right] if the operation was successful.

# File lib/scrapinghub/jobs.rb, line 78
def schedule(args)
  extra = args.delete(:extra) || {}
  options = { body: args.merge(extra), basic_auth: { username: @api_key } }
  Try { self.class.post("/api/schedule.json", options) }.to_either >-> response {
    if response.code == 200
      Right(response)
    else
      Left(response)
    end
  }
end
stop(args) click to toggle source

Stop one or more running jobs.

@param project [Fixnum] the project’s numeric ID @param job [String] the ID of a job to stop

@return [Kleisli::Left] if validation fails (e.g. bad authentication) or

if there were any low-level exceptions (e.g. the host is down), with a
message detailing the failure.

@return [Kleisli::Right] if the operation was successful.

# File lib/scrapinghub/jobs.rb, line 163
def stop(args)
  options = { body: args, basic_auth: { username: @api_key } }
  Try { self.class.post("/api/jobs/stop.json", options) }.to_either >-> response {
    if response.code == 200
      Right(response)
    else
      Left(response)
    end
  }
end
update(args) click to toggle source

Update information about jobs.

@param project [Fixnum] the project’s numeric ID @param job [String, Array<String>] (optional) ID(s) of specific jobs to

update

@param spider [String] (optional) query on spider name to update @param state [String] (optional) query on jobs with this state to update.

Valid values: "pending", "running", "finished"

@param has_tag [String, Array<String>] (optional) query on jobs

containing the given tag(s) to update

@param lacks_tag [String, Array<String>] (optional) query on jobs not

containing the given tag(s) to update

@param add_tag [String, Array<String>] (optional) tag(s) to add to the

queried jobs

@param remove_tag [String, Array<String>] (optional) tag(s) to remove

from the queried jobs

@return [Kleisli::Left] if validation fails (e.g. bad authentication) or

if there were any low-level exceptions (e.g. the host is down), with a
message detailing the failure.

@return [Kleisli::Right] if the operation was successful.

# File lib/scrapinghub/jobs.rb, line 119
def update(args)
  options = { body: args, basic_auth: { username: @api_key } }
  Try { self.class.post("/api/jobs/update.json", options) }.to_either >-> response {
    if response.code == 200
      Right(response)
    else
      Left(response)
    end
  }
end