class RDF::CLI

Individual formats can modify options by updating {Reader.options} or {Writer.options}. Format-specific commands are taken from {Format.cli_commands} for each loaded format, which returns an array of lambdas taking arguments and options.

Status updates should be logged to ‘opts.info`. More complicated information can be added to `:messages` key within `opts`, if present.

Other than ‘help`, all commands parse an input file.

Multiple commands may be added in sequence to execute a pipeline.

@example Creating Reader-specific options:

class Reader
  def self.options
    [
      RDF::CLI::Option.new(
        symbol: :canonicalize,
        on: ["--canonicalize"],
        description: "Canonicalize URI/literal forms.") {true},
      RDF::CLI::Option.new(
        symbol: :uri,
        on: ["--uri STRING"],
        description: "URI.") {|v| RDF::URI(v)},
    ]
  end

@example Creating Format-specific commands:

class Format
  def self.cli_commands
    {
      count: {
        description: "",
        parse: true,
        lambda: ->(argv, opts) {}
      },
    }
  end

@example Adding a command manually

class MyCommand
  RDF::CLI.add_command(:count, description: "Count statements") do |argv, opts|
    count = 0
    RDF::CLI.parse(argv, opts) do |reader|
      reader.each_statement do |statement|
        count += 1
      end
    end
    options[:logger].info "Parsed #{count} statements"
  end
end

Format-specific commands should verify that the reader and/or output format are appropriate for the command.

Constants

COMMANDS

Built-in commands. Other commands are imported from the Format class of different readers/writers using {RDF::Format#cli_commands}. ‘COMMANDS` is a Hash who’s keys are commands that may be executed by {RDF::CLI.exec}. The value is a hash containing the following keys:

  • ‘description` used for providing information about the command.

  • ‘parse` Boolean value to determine if input files should automatically be parsed into `repository`.

  • ‘help` used for the CLI help output.

  • ‘lambda` code run to execute command.

  • ‘filter` value is a Hash whose keys are matched against selected command options. All specified `key/value` pairs are compared against the equivalent key in the current invocation.

    If an Array, option value (as a string) must match any value of the array (as a string)
    If a Proc, it is passed the option value and must return `true`.
    Otherwise, the option value (as a string) must equal the  `value` (as a string).
  • ‘control` Used to indicate how (if) command is displayed

  • ‘repository` Use this repository, if set

  • ‘options` an optional array of `RDF::CLI::Option` describing command-specific options.

  • ‘option_use`: A hash of option symbol to option usage, used for overriding the default status of an option for this command.

@return [Hash{Symbol => Hash{Symbol => Object}}]

OPTIONS

Options to setup, may be modified by selected command. Options are also read from {RDF::Reader#options} and {RDF::Writer#options}. When a specific input- or ouput-format is selected, options are also discovered from the associated subclass reader or writer. @return [Array<RDF::CLI::Option>]

Attributes

repository[RW]

Repository containing parsed statements @return [RDF::Repository]

Public Class Methods

abort(msg) click to toggle source

@param [String] msg @return [void]

# File lib/rdf/cli.rb, line 719
def self.abort(msg)
  Kernel.abort "#{basename}: #{msg}"
end
add_command(command, **options, &block) click to toggle source

Add a command.

@param [#to_sym] command @param [Hash{Symbol => String}] options @option options [String] description @option options [String] help string to display for help @option options [Boolean] parse parse input files in to Repository, or not. @option options [Array<RDF::CLI::Option>] options specific to this command @yield argv, opts @yieldparam [Array<String>] argv @yieldparam [Hash] opts @yieldreturn [void]

# File lib/rdf/cli.rb, line 661
def self.add_command(command, **options, &block)
  options[:lambda] = block if block_given?
  COMMANDS[command.to_sym] ||= options
end
basename() click to toggle source

@return [String]

# File lib/rdf/cli.rb, line 380
def self.basename() File.basename($0) end
commands(format: nil, **options) click to toggle source

@overload commands(**options)

@param [Hash{Symbol => Object}] options already set
@return [Array<String>] list of executable commands

@overload commands(format: :json, **options)

Returns commands as JSON, for API usage.
@param [:json] format
@param [Hash{Symbol => Object}] options already set
@return [Array{Object}]
  Returns an array of commands including the command symbol
# File lib/rdf/cli.rb, line 597
def self.commands(format: nil, **options)
  # First, load commands from other formats
  load_commands

  case format
  when :json
    COMMANDS.map do |k, v|
      v = v.merge(symbol: k, options: v.fetch(:options, []).map(&:to_hash))
      v.delete(:lambda)
      v.delete(:help)
      v.delete(:options) if v[:options].empty?
      v[:control] == :none ? nil : v
    end.compact
  else
    # Subset commands based on filter options
    cmds = COMMANDS.reject do |k, c|
      c.fetch(:filter, {}).any? do |opt, val|
        case val
        when Array
          !val.map(&:to_s).include?(options[opt].to_s)
        when Proc
          !val.call(options[opt])
        else
          val.to_s != options[opt].to_s
        end
      end
    end

    sym_len = cmds.keys.map {|k| k.to_s.length}.max
    cmds.keys.sort.map do |k|
      "%*s: %s" % [sym_len, k, cmds[k][:description]]
    end
  end
end
exec(args, output: $stdout, option_parser: nil, messages: {}, **options) click to toggle source

Execute one or more commands, parsing input as necessary

@param [Array<String>] args @param [IO] output @param [OptionParser] option_parser @param [Hash{Symbol => Hash{Symbol => Array}}] messages used for conveying non primary-output which is structured. @param [Hash{Symbol => Object}] options @return [Boolean]

# File lib/rdf/cli.rb, line 483
def self.exec(args, output: $stdout, option_parser: nil, messages: {}, **options)
  option_parser ||= self.options(args)
  options[:logger] ||= option_parser.options[:logger]
  output.set_encoding(Encoding::UTF_8) if output.respond_to?(:set_encoding) && RUBY_PLATFORM == "java"

  # Separate commands from file options; arguments already extracted
  cmds, args = args.partition {|e| COMMANDS.include?(e.to_sym)}

  if cmds.empty?
    usage(option_parser)
    raise ArgumentError, "No command given"
  end

  if cmds.first == 'help'
    on_cmd = cmds[1]
    cmd_opts = COMMANDS.fetch(on_cmd.to_s.to_sym, {})
    if on_cmd && cmd_opts[:help]
      usage(option_parser, cmd_opts: cmd_opts, banner: "Usage: #{self.basename.split('/').last} #{COMMANDS[on_cmd.to_sym][:help]}")
    elsif on_cmd
      usage(option_parser, cmd_opts: cmd_opts)
    else
      usage(option_parser)
    end
    return
  end

  # Make sure any selected command isn't filtered out
  cmds.each do |c|
    COMMANDS[c.to_sym].fetch(:filter, {}).each do |opt, val|
      case val
      when Array
        unless val.map(&:to_s).include?(options[opt].to_s)
          usage(option_parser, banner: "Command #{c.inspect} requires #{opt} in #{val.map(&:to_s).inspect}, not #{options.fetch(opt, 'null')}")
          raise ArgumentError, "Incompatible command #{c} used with option #{opt}=#{options[opt]}"
        end
      when Proc
        unless val.call(options[opt])
          usage(option_parser, banner: "Command #{c.inspect} #{opt} inconsistent with #{options.fetch(opt, 'null')}")
          raise ArgumentError, "Incompatible command #{c} used with option #{opt}=#{options[opt]}"
        end
      else
        unless val.to_s == options[opt].to_s
          usage(option_parser, banner: "Command #{c.inspect} requires compatible value for #{opt}, not #{options.fetch(opt, 'null')}")
          raise ArgumentError, "Incompatible command #{c} used with option #{opt}=#{options[opt]}"
        end
      end
    end

    # The command may specify a repository instance to use
    options[:repository] ||= COMMANDS[c.to_sym][:repository]
  end

  # Hacks for specific options
  options[:logger].level = Logger::INFO if options[:verbose]
  options[:logger].level = Logger::DEBUG if options[:debug]
  options[:format] = options[:format].to_sym if options[:format]
  options[:output_format] = options[:output_format].to_sym if options[:output_format]

  # Allow repository to be set via option.
  # If RDF::OrderedRepo is present, use it if the `ordered` option is specified, otherwise extend an Array.
  @repository = options[:repository] || case
    when RDF.const_defined?(:OrderedRepo) then RDF::OrderedRepo.new
    when options[:ordered] then [].extend(RDF::Enumerable, RDF::Queryable)
    else RDF::Repository.new
  end

  # Parse input files if any command requires it
  if cmds.any? {|c| COMMANDS[c.to_sym][:parse]}
    start = Time.new
    count = 0
    self.parse(args, **options) do |reader|
      reader.each_statement {|st| @repository << st}
      # Remember prefixes from reading
      options[:prefixes] ||= reader.prefixes
    end
    secs = Time.new - start
    options[:logger].info "Parsed #{repository.count} statements with #{@readers.join(', ')} in #{secs} seconds @ #{count/secs} statements/second."
  end

  # Run each command in sequence
  cmds.each do |command|
    COMMANDS[command.to_sym][:lambda].call(args,
      output: output,
      messages: messages,
      **options.merge(repository: repository))
  end

  # Normalize messages
  messages.each do |kind, term_messages|
    case term_messages
    when Hash
    when Array
      messages[kind] = {result: term_messages}
    else
      messages[kind] = {result: [term_messages]}
    end
  end

  if options[:statistics]
    options[:statistics][:reader] = @readers.first unless (@readers || []).empty?
    options[:statistics][:count] = @repository.count
  end
end
formats(reader: false, writer: false) click to toggle source

@return [Array<String>] list of available formats

# File lib/rdf/cli.rb, line 668
def self.formats(reader: false, writer: false)
  f = RDF::Format.sort_by(&:to_sym).
    select {|ft| (reader ? ft.reader : (writer ? ft.writer : (ft.reader || ft.writer)))}.
    inject({}) do |memo, r|
      memo.merge(r.to_sym => r.name)
  end
  sym_len = f.keys.map {|k| k.to_s.length}.max
  f.map {|s, t| "%*s: %s" % [sym_len, s, t]}
end
load_commands() click to toggle source

Load commands from formats @return [Hash{Symbol => Hash{Symbol => Object}}]

# File lib/rdf/cli.rb, line 635
def self.load_commands
  unless @commands_loaded
    RDF::Format.each do |format|
      format.cli_commands.each do |command, options|
        options = {lambda: options} unless options.is_a?(Hash)
        add_command(command, **options)
      end
    end
    @commands_loaded = true
  end
  COMMANDS
end
options(argv, format: nil) click to toggle source

Return OptionParser set with appropriate options

The yield return should provide one or more commands from which additional options will be extracted. @overload options(argv)

@param [Array<String>] argv
@return [OptionParser]

@overload options(argv, format: :json)

@param [Array<String>] argv
@param [:json] format (:json)
@return [Array<RDF::CLI::Option>]
  Returns discovered options
# File lib/rdf/cli.rb, line 394
def self.options(argv, format: nil)
  options = OptionParser.new
  cli_opts = OPTIONS.map(&:dup)
  logger = Logger.new($stderr)
  logger.level = Logger::WARN
  logger.formatter = lambda {|severity, datetime, progname, msg| "#{severity} #{msg}\n"}
  opts = options.options = {logger: logger}

  # Pre-load commands
  load_commands

  # Add options for the specified command(s)
  cmds, args = argv.partition {|e| COMMANDS.include?(e.to_sym)}
  cmds.each do |cmd|
    Array(RDF::CLI::COMMANDS[cmd.to_sym][:options]).each do |option|
      # Replace any existing option with the same symbol
      cli_opts.delete_if {|cli_opt| cli_opt.symbol == option.symbol}

      # Add the option, unless disabled or removed
      cli_opts.unshift(option)
    end

    # Update usage of options for this command
    RDF::CLI::COMMANDS[cmd.to_sym].fetch(:option_use, {}).each do |sym, use|
      if opt = cli_opts.find {|cli_opt| cli_opt.symbol == sym}
        opt.use = use
      end
    end
  end

  cli_opts.each do |cli_opt|
    next if opts.key?(cli_opt.symbol)
    on_args = cli_opt.on || []
    on_args << cli_opt.description if cli_opt.description
    options.on(*on_args) do |arg|
      opts[cli_opt.symbol] = cli_opt.call(arg, options)
    end
  end

  if format == :json
    # Return options
    cli_opts.map(&:to_hash)
  else
    options.banner = "Usage: #{self.basename} command+ [options] [args...]"

    options.on_tail('-V', '--version', 'Display the RDF.rb version and exit.') do
      puts RDF::VERSION; exit(0)
    end

    show_help = false
    options.on_tail("-h", "--help", "Show this message") do
      show_help = true
    end

    begin
      args = options.parse!(args)
    rescue OptionParser::InvalidOption, OptionParser::InvalidArgument, ArgumentError => e
      abort e
    end

    # Make sure options are processed first
    if show_help
      self.usage(options); exit(0)
    end

    options.args = cmds + args
    options
  end
end
parse(files, evaluate: nil, format: nil, encoding: Encoding::UTF_8, **options) { |reader| ... } click to toggle source

Parse each file, $stdin or specified string in ‘options` yielding a reader

@param [Array<String>] files @param [String] evaluate from command-line, rather than referenced file @param [Symbol] format (:ntriples) Reader symbol for finding reader @param [Encoding] encoding set on the input @param [Hash{Symbol => Object}] options sent to reader @yield [reader] @yieldparam [RDF::Reader] @return [nil]

# File lib/rdf/cli.rb, line 690
def self.parse(files, evaluate: nil, format: nil, encoding: Encoding::UTF_8, **options, &block)
  if files.empty?
    # If files are empty, either use options[:execute]
    input = evaluate ? StringIO.new(evaluate) : $stdin
    input.set_encoding(encoding )
    if !format
      sample = input.read
      input.rewind
    end
    r = RDF::Reader.for(format|| {sample: sample})
    raise ArgumentError, "Unknown format for evaluated input" unless r
    (@readers ||= []) << r
    r.new(input, **options) do |reader|
      yield(reader)
    end
  else
    options[:format] = format if format
    files.each do |file|
      RDF::Reader.open(file, **options) do |reader|
        (@readers ||= []) << reader.class.to_s
        yield(reader)
      end
    end
  end
end
usage(options, cmd_opts: {}, banner: nil) click to toggle source

Output usage message

# File lib/rdf/cli.rb, line 466
def self.usage(options, cmd_opts: {}, banner: nil)
  options.banner = banner if banner
  $stdout.puts options
  $stdout.puts "Note: available commands and options may be different depending on selected --input-format and/or --output-format."
  $stdout.puts "Available commands:\n\t#{self.commands(**options.options).join("\n\t")}"
  $stdout.puts "Available formats:\n\t#{(self.formats).join("\n\t")}"
end