Table of Contents - webrobots-0.1.2 Documentation
Pages
Classes and Modules
- Nokogiri
- Nokogiri::HTML
- Nokogiri::HTML::Document
- WebRobots
- WebRobots::Error
- WebRobots::ParseError
- WebRobots::RobotsTxt
- WebRobots::RobotsTxt::AccessControlLine
- WebRobots::RobotsTxt::AgentLine
- WebRobots::RobotsTxt::AllowLine
- WebRobots::RobotsTxt::CrawlDelayLine
- WebRobots::RobotsTxt::DisallowLine
- WebRobots::RobotsTxt::ExtentionLine
- WebRobots::RobotsTxt::Line
- WebRobots::RobotsTxt::Parser
- WebRobots::RobotsTxt::Record
- Webrobots
Methods
- ::new — WebRobots
- ::new — WebRobots::ParseError
- ::new — WebRobots::RobotsTxt
- ::new — WebRobots::RobotsTxt::Record
- ::new — WebRobots::RobotsTxt::Line
- ::unfetchable — WebRobots::RobotsTxt
- #_reduce_none — WebRobots::RobotsTxt::Parser
- #allow? — WebRobots::RobotsTxt
- #allow? — WebRobots::RobotsTxt::Record
- #allow? — WebRobots::RobotsTxt::AllowLine
- #allow? — WebRobots::RobotsTxt::DisallowLine
- #compile — WebRobots::RobotsTxt::Line
- #compile — WebRobots::RobotsTxt::AgentLine
- #compile — WebRobots::RobotsTxt::AccessControlLine
- #compile — WebRobots::RobotsTxt::CrawlDelayLine
- #crawl_delay — WebRobots::RobotsTxt
- #default? — WebRobots::RobotsTxt::Record
- #error! — WebRobots::RobotsTxt
- #find_record — WebRobots::RobotsTxt
- #match? — WebRobots::RobotsTxt::Record
- #match? — WebRobots::RobotsTxt::AccessControlLine
- #meta_robots — Nokogiri::HTML::Document
- #nofollow? — Nokogiri::HTML::Document
- #noindex? — Nokogiri::HTML::Document
- #options — WebRobots::RobotsTxt
- #parse_meta_robots — Nokogiri::HTML::Document
- #target — WebRobots::RobotsTxt
- #to_s — WebRobots::ParseError