class RestScrapyd

Don’t define a ‘RestScrapyd::Client` and include it in the class to avoid yardoc choking (plus two newlines to avoid this getting into the documentation)

Attributes

project[RW]

@!attribute project Default project name (can also be passed as hash argument to constructor).

Public Instance Methods

addversion(version, egg, project=self.project) click to toggle source

Add a version to a project, creating the project if it doesn’t exist. @param version [String] Project version @param egg [String, IO] Filename or file to upload, must be a Scrapy egg @return [Hash<String, Object>] Response, right now just +{“spiders”: <n>}+ @see scrapyd.readthedocs.org/en/latest/api.html#addversion-json

# File lib/rest-scrapyd.rb, line 38
def addversion(version, egg, project=self.project)
  egg = File.open(egg, 'rb') if egg.is_a? String
  post('addversion.json', project: project, version: version, egg: egg)
end
cancel(job, project=self.project) click to toggle source

Cancel a scheduled or running job @param job [String] Jobid to cancel @param project [String] Project name @return [String] Previous state of job @see scrapyd.readthedocs.org/en/latest/api.html#cancel-json

# File lib/rest-scrapyd.rb, line 68
def cancel(job, project=self.project)
  post('cancel.json', project: project, job: job)['prevstate']
end
delproject(project) click to toggle source

Delete a project and all its uploaded versions. @param project [String] Project name @see scrapyd.readthedocs.org/en/latest/api.html#delproject-json

# File lib/rest-scrapyd.rb, line 114
def delproject(project)
  post('delproject.json', project: project)
end
delversion(version, project=self.project) click to toggle source

Delete a project version. @param version [String] Project version @param project [String] Project name @see scrapyd.readthedocs.org/en/latest/api.html#delversion-json

# File lib/rest-scrapyd.rb, line 107
def delversion(version, project=self.project)
  post('delversion.json', project: project, version: version)
end
listjobs(project=self.project) click to toggle source

Get the list of pending, running and finished jobs of some project. @param project [String] Project name @return [Hash<String, Array<Hash>>] Jobs for project @see scrapyd.readthedocs.org/en/latest/api.html#listjobs-json

# File lib/rest-scrapyd.rb, line 99
def listjobs(project=self.project)
  get('listjobs.json', project: project).reject{|k,v| k=='status'}
end
listprojects() click to toggle source

Get the list of projects uploaded to this Scrapy server. @return [Array<String>] Available projects @see scrapyd.readthedocs.org/en/latest/api.html#listprojects-json

# File lib/rest-scrapyd.rb, line 75
def listprojects
  get('listprojects.json')['projects']
end
listspiders(project=self.project) click to toggle source

Get the list of spiders available in the last version of some project. @param project [String] Project name @return [Array<String>] Available spiders for project @see scrapyd.readthedocs.org/en/latest/api.html#listspiders-json

# File lib/rest-scrapyd.rb, line 91
def listspiders(project=self.project)
  get('listspiders.json', project: project)['spiders']
end
listversions(project=self.project) click to toggle source

Get the list of versions available for some project. @param project [String] Project name @return [Array<String>] Available versions for project @see scrapyd.readthedocs.org/en/latest/api.html#listversions-json

# File lib/rest-scrapyd.rb, line 83
def listversions(project=self.project)
  get('listversions.json', project: project)['versions']
end
schedule(spider, version, settings={}, arguments={}) click to toggle source

Schedule a spider run (also known as a job). @param spider [String] Spider name @param version [String] Project version @param settings [Hash<String, String>] Additional Scrapy settings @param arguments [Hash<String, String>] Additional spider arguments @option arguments [String] :project ({self.project}) Project name @return [String] Job id @see scrapyd.readthedocs.org/en/latest/api.html#schedule-json

# File lib/rest-scrapyd.rb, line 51
def schedule(spider, version, settings={}, arguments={})
  # +setting+ may appear multiple times, so can't use hash
  params = {project: self.project, spider: spider}.to_a
  params += settings.to_a.map{|k,v| ["setting", "#{k}=#{v}"]}
  params += arguments.to_a
  # rest-core doesn't (yet) support array payload, encode ourselves
  params.map{|p| p[0] = p[0].to_s} # like stringify_keys
  params = RestCore::Middleware.percent_encode(params)
  s_headers = headers.merge({'Content-Type' => 'application/x-www-form-urlencoded'})
  post('schedule.json', params, {}, headers: s_headers)['jobid']
end