class OcflTools::OcflValidator

Class to perform validation actions on POSIX directories that potentially contain OCFL objects.

Attributes

inventory[R]

@return {OcflTools::OcflInventory} an OcflInventory instance that represents an inventory.json file, if the directory contains a valid OCFL object.

ocfl_object_root[R]

@return [Pathname] the full local filesystem path to the OCFL object root directory.

ocfl_version[RW]

@return [String] the version of OCFL that this validator object is targeting.

verify[R]

@return {OcflTools::OcflVerify} an OcflVerify instance that represents the results of requesting verification of an OcflInventory.

version_format[R]

@return [String] the discovered version format of the object, found by inspecting version directory names.

Public Class Methods

new(ocfl_object_root) click to toggle source

@param [Pathname] ocfl_object_root is a the full local filesystem path to the object directory.

# File lib/ocfl_tools/ocfl_validator.rb, line 22
def initialize(ocfl_object_root)
  unless File.directory? ocfl_object_root
    raise "#{ocfl_object_root} is not a directory!"
  end

  @digest           = nil
  @version_format   = nil
  @ocfl_version     = nil
  @ocfl_object_root = ocfl_object_root
  @my_results       = OcflTools::OcflResults.new
  @inventory        = nil # some checks create an inventory object; have a way to get at that.
  @verify           = nil # some checks create a verify object; have a way to get at that.
end

Public Instance Methods

results() click to toggle source

Get the current summation of results events for this instance, including a roll-up of any verify actions. @return [OcflTools::OcflResults] current validation results.

# File lib/ocfl_tools/ocfl_validator.rb, line 38
def results
  @my_results.add_results(@verify.results) unless @verify.nil?
  @my_results
end
validate_ocfl_object_root(digest: nil) click to toggle source

Perform an OCFL-spec validation of the given object directory. If given the optional digest value, verify file content using checksums in inventory file will fail if digest is not found in manifest or a fixity block. This validates all versions and all files in the object_root. If you want to just check a specific version, call {verify_directory}. @param [String] digest optional digest to use, if one wishes to use values in the fixity block instead of the official OCFL digest values. @return {OcflTools::OcflResults} event results

# File lib/ocfl_tools/ocfl_validator.rb, line 47
def validate_ocfl_object_root(digest: nil)
  # calls verify_structure, verify_inventory and verify_checksums.
  verify_structure
  verify_inventory # returns a diff. results object; merge it?
  verify_manifest
  if !digest.nil?
    verify_fixity(digest: digest)
  else
    verify_checksums
  end
  results # this copies verify.results into our main results object, if it exists.
end
verify_checksums(inventory_file = " click to toggle source

The default checksum test assumes you want to test all likely files on disk against whatever version of the inventory.json (hopefully the latest!) is in the root directory. Otherwise, if you give it a version 3 inventory, it'll check v1…v3 directories on disk against the inventory's manifest, but won't check >v4. {#verify_structure} will, however, let you know if your most recent inventory goes to v3, but there's a v4 directory in your object root. @param [Pathname] inventory_file fully-qualified path to a valid OCFL inventory.json. @return {OcflTools::OcflResults} of event results

# File lib/ocfl_tools/ocfl_validator.rb, line 258
    def verify_checksums(inventory_file = "#{@ocfl_object_root}/inventory.json")
      # validate inventory.json checksum against inventory.json.<sha256|sha512>
      # validate files in manifest against physical copies on disk.
      # cross_check digestss.
      # Report out via @my_results.
      # Inventory file does not exist; create a results object, record this epic fail, and return.
      @my_results ||= OcflTools::OcflResults.new
      unless File.exist?(inventory_file)
        @my_results.error('E215', 'verify_checksums', "Expected inventory file #{inventory_file} not found.")
        return @my_results
      end

      begin
        @inventory = load_inventory(inventory_file)
      rescue OcflTools::Errors::ValidationError
        @my_results.error('E210', 'verify_checksums', "Unable to process inventory file #{inventory_file}.")
        return @my_results
      end

      # if @digest is set, use that as the digest for checksumming.
      # ( but check inventory.fixity to make sure it's there first )
      # Otherwise, use the value of inventory.digestAlgorithm
      # TODO: files_on_disk here LIES if contentDirectory value is wrong!
#      files_on_disk = OcflTools::Utils::Files.get_versions_dir_files(@ocfl_object_root, @inventory.version_id_list.min, @inventory.version_id_list.max)
      # WORK OUT what contentDirectory is by inspecting all paths in inventory.manifest.
      files_on_disk = []
      working_files = OcflTools::Utils::Files.invert_and_expand(@inventory.manifest).keys
      # gives me version/contentDir/<foo>
      # ....and I know that '/' is the delimiter, and element 0 is version. I want element 1, the content_dir value.
      manifest_content_dirs = []
      working_files.each do | file |
        manifest_content_dirs << file.split('/')[1]
      end
      manifest_content_dirs.uniq!
      manifest_content_dirs.each do | mcd |
        # We should check that path on disk to see if there's content there.
        @inventory.versions.keys.each do | version |
          my_files = OcflTools::Utils::Files.get_dir_files("#{@ocfl_object_root}/#{version}/#{mcd}")
          # And expand it to a full file path (returns an array, so flatten it out and append to files_on_disk.
          OcflTools::Utils::Files.expand_filepaths(my_files, "#{@ocfl_object_root}/#{version}/#{mcd}").each do | file |
            files_on_disk << file
          end
        end
        files_on_disk.uniq! # Just to make sure there's no funky business.
      end

      # Now generate checksums for the files we found on disk, and Hash them.
      disk_checksums = OcflTools::Utils::Files.create_digests(files_on_disk, @inventory.digestAlgorithm)
      # Get an equivalent hash by manipulating the inventory.manifest hash.
      manifest_checksums = OcflTools::Utils::Files.invert_and_expand_and_prepend(@inventory.manifest, @ocfl_object_root)
      # Returns OcflTools::OcflResults object; either new or the one passed in with new content.
      OcflTools::Utils.compare_hash_checksums(disk_checksums: disk_checksums, inventory_checksums: manifest_checksums, results: @my_results)
    end
verify_directory(version) click to toggle source

We may also want to only verify a specific directory, not the entire object. For example, if we've just added a new version, we might want to just check those files and not the rest of the object (esp. if it has some very large version directories). @param [Integer] version directory to verify @return {OcflTools::OcflResults} of verify events

# File lib/ocfl_tools/ocfl_validator.rb, line 604
def verify_directory(version)
  # start by getting version format and directories.
  get_version_format # sets @version_format, one way or another.

  version_name = @version_format % version.to_i
  # Make sure this directory actually exists.
  unless File.directory?("#{@ocfl_object_root}/#{version_name}")
    raise "Requested version directory doesn't exist!"
  end

  # OK, now we need an inventory.json to tell use what the contentDirectory should be.
  if File.exist?("#{ocfl_object_root}/#{version_name}/inventory.json")
    my_content_dir = OcflTools::Utils::Inventory.get_contentDirectory("#{ocfl_object_root}/#{version_name}/inventory.json")
    @inventory     = OcflTools::OcflInventory.new.from_file("#{ocfl_object_root}/#{version_name}/inventory.json")
  else
    my_content_dir = OcflTools::Utils::Inventory.get_contentDirectory("#{ocfl_object_root}/inventory.json")
    @inventory     = OcflTools::OcflInventory.new.from_file("#{ocfl_object_root}/inventory.json")
  end

  # Get a list of fully-resolvable files for this version directory from disk.
  my_files_on_disk = OcflTools::Utils::Files.get_version_dir_files(@ocfl_object_root, version)

  # Now process my_inventory.manifest
  # Flip and invert  it.
  manifest_checksums = OcflTools::Utils::Files.invert_and_expand_and_prepend(@inventory.manifest, @ocfl_object_root)

  # Now we need to trim manifest_checksums to the stuff that only matches
  # ocfl_object_root/version_string/content_dir
  filtered_checksums = {}
  manifest_checksums.each do |file, digest|
    if file =~ %r{^#{ocfl_object_root}/#{version_name}/#{my_content_dir}}
      filtered_checksums[file] = digest
    end
  end

  # Now generate checksums for the files we found on disk, and Hash them.
  disk_checksums = OcflTools::Utils::Files.create_digests(my_files_on_disk, @inventory.digestAlgorithm)

  # Finally! Pass them to checksum checker.
  OcflTools::Utils.compare_hash_checksums(disk_checksums: disk_checksums, inventory_checksums: filtered_checksums, results: @my_results, context: "verify_directory #{version_name}")
end
verify_fixity(inventory_file: " click to toggle source

Performs checksum validation of files listed in the inventory's fixity block. @param [Pathname] inventory_file fully-qualified path to a valid OCFL inventory.json. @param [String] digest string value of the algorithm to use for this fixity check. This value must exist as a key in the object's fixity block. @return {OcflTools::OcflResults} of event results

# File lib/ocfl_tools/ocfl_validator.rb, line 64
def verify_fixity(inventory_file: "#{@ocfl_object_root}/inventory.json", digest: 'md5')
  # Gets the appropriate fixity block, calls compare_hash_checksums

  begin
    @inventory = load_inventory(inventory_file)
  rescue OcflTools::Errors::ValidationError
    @my_results.error('E210', 'verify_fixity', "Unable to process inventory file #{inventory_file}.")
    return @my_results
  end

  # Since fixity blocks are not required to be complete, we just validate what's there.
  # So get the fixity block, flip it, expand it, checksum it against the same files on disk.

  if @inventory.fixity.empty?
    @my_results.error('E111', "verify_fixity #{digest}", "No fixity block in #{inventory_file}!")
    return @my_results
  end

  unless @inventory.fixity.key?(digest)
    @my_results.error('E111', "verify_fixity #{digest}", "Requested algorithm #{digest} not found in fixity block.")
    return @my_results
  end

  fixity_checksums = OcflTools::Utils::Files.invert_and_expand_and_prepend(@inventory.fixity[digest], @ocfl_object_root)

  my_files_on_disk = fixity_checksums.keys

  # Warn if there are less files in requested fixity block than in manifest.
  if @inventory.manifest.keys.size > fixity_checksums.keys.size
    missing_files = @inventory.manifest.keys.size - fixity_checksums.keys.size
    @my_results.warn(
      'W111',
      "verify_fixity #{digest}",
      "#{missing_files} files in manifest are missing from fixity block."
    )
  end

  # check these files exist on disk before trying to make checksums!
  my_files_on_disk.each do |file|
    unless File.file? file
      @my_results.error('E111', "verify_fixity #{digest}", "File #{file} in fixity block not found on disk.")
      my_files_on_disk.delete(file)
    end
  end

  disk_checksums = OcflTools::Utils::Files.create_digests(my_files_on_disk, digest)

  # And now we can compare values!
  OcflTools::Utils.compare_hash_checksums(
    disk_checksums: disk_checksums,
    inventory_checksums: fixity_checksums,
    results: @my_results,
    context: "verify_fixity #{digest}"
  )
end
verify_inventory(inventory_file = " click to toggle source

Creates an {OcflInventory} for the given inventory.json, then creates an {OcflVerify} instance of it and verifies it. @param [Pathname] inventory_file fully-qualified path to a valid OCFL inventory.json. @return {OcflTools::OcflResults} event results

# File lib/ocfl_tools/ocfl_validator.rb, line 666
def verify_inventory(inventory_file = "#{@ocfl_object_root}/inventory.json")
  # Load up the object with ocfl_inventory, push it through ocfl_verify.
  @my_results ||= OcflTools::OcflResults.new
  # If inventory_file does not exist, load_inventory will throw and log an E215.
  begin
    @inventory = load_inventory(inventory_file)
    @inventory = OcflTools::OcflInventory.new.from_file(inventory_file)
    @verify    = OcflTools::OcflVerify.new(@inventory)
    @verify.check_all # creates & returns @results object from OcflVerify
  # This could be OcflTools::Errors::ValidationError now.
  rescue OcflTools::Errors::ValidationError
    # I don't think we need to throw this E210 any more.
    @my_results.error('E210', 'verify_inventory', "Unable to process inventory file #{inventory_file}.")
    return @my_results
  end
end
verify_manifest(inventory_file = " click to toggle source

Given an inventory file, do the files mentioned in the manifest exist on disk? This is a basic file existence cross-check. @param [Pathname] inventory_file fully-qualified path to a valid OCFL inventory.json. @return {OcflTools::OcflResults} of event results

# File lib/ocfl_tools/ocfl_validator.rb, line 124
def verify_manifest(inventory_file = "#{@ocfl_object_root}/inventory.json")
  @my_results ||= OcflTools::OcflResults.new
  unless File.exist?(inventory_file)
    @my_results.error('E215', 'verify_files', "Expected inventory file #{inventory_file} not found.")
    return @my_results
  end

  begin
    @inventory = load_inventory(inventory_file)
  rescue OcflTools::Errors::ValidationError
    @my_results.error('E210', 'verify_fixity', "Unable to process inventory file #{inventory_file}.")
    return @my_results
  end

  # These are arrays, not hashes, so they must be sorted for the comparison below to work.
  files_in_manifest  = OcflTools::Utils::Files.invert_and_expand_and_prepend(@inventory.manifest, @ocfl_object_root).keys.sort
  files_on_disk      = OcflTools::Utils::Files.get_versions_dir_files(@ocfl_object_root, @inventory.version_id_list.min, @inventory.version_id_list.max).sort

  # E012: Check that the contentDirectory value in inventory is present on disk.
  # 3.3.1 Content directory
  # Version directories MUST contain a designated content sub-directory if the version contains files to be preserved,
  # and SHOULD NOT contain this sub-directory otherwise.
  @inventory.versions.keys.each do | version |
    # Are there files mentioned in the manifest for this version?
    files_in_this_version = files_in_manifest.grep(/^#{@ocfl_object_root}\/#{version}/)
    if files_in_this_version.size > 0
      if Dir.exist?("#{@ocfl_object_root}/#{version}/#{@inventory.contentDirectory}")
        @my_results.info('I200', 'verify_manifest', "OCFL 3.3.1 Expected content directory #{version}/#{@inventory.contentDirectory} found.")
      else
        @my_results.error('E012', 'verify_manifest', "OCFL 3.3.1 Expected content directory #{version}/#{@inventory.contentDirectory} not found.")
        # IF WE GET THIS RESULT, then files_on_disk is likely not going to work (it expects contentDirectory to NOT LIE).
      end
    else
      if Dir.exist?("#{@ocfl_object_root}/#{version}/#{@inventory.contentDirectory}")
        @my_results.warn('W102', 'verify_manifest', "OCFL 3.3.1 version #{version} contentDirectory should not be empty.")
      else
        @my_results.info('I200', 'verify_manifest', "OCFL 3.3.1 version #{version} does not have (and should not have) a contentDirectory directory.")
      end
    end
  end

  # get_versions_dir_files relies on contentDirectory value being correct. Check!
  # If the prefix of files_in_manifest DOES NOT MATCH object_root/<ver>/contentDirectory, we have a problem.
  case
    when files_in_manifest.size > 0
      working_files = OcflTools::Utils::Files.invert_and_expand(@inventory.manifest).keys
      # gives me version/contentDir/<foo>
      # ....and I know that '/' is the delimiter, and element 0 is version. I want element 1, the content_dir value.
      manifest_content_dirs = []
      working_files.each do | file |
        manifest_content_dirs << file.split('/')[1]
        # Need this from manifest: /object_root/version/[contentDirectory]/file
      end
      manifest_content_dirs.uniq!
      manifest_content_dirs.each do | mcd |
      if "#{mcd}" != "#{@inventory.contentDirectory}"
        # Don't need this; version_structure test will pick up on this directory's presence.
        # @my_results.error('E111', 'verify_manifest', "content directory '#{mcd}' discovered in manifest DOES NOT match inventory contentDirectory value!")
        # We should check that path on disk to see if there's content there.
        @inventory.versions.keys.each do | version |
          my_files = OcflTools::Utils::Files.get_dir_files("#{@ocfl_object_root}/#{version}/#{mcd}")
          # And expand it to a full file path (returns an array, so flatten it out and append to files_on_disk.
          OcflTools::Utils::Files.expand_filepaths(my_files, "#{@ocfl_object_root}/#{version}/#{mcd}").each do | file |
            files_on_disk << file
          end
        end
        files_on_disk.uniq! # Just to make sure there's no funky business.
      end
    end
  end

  # we only need the files (keys), not the digests here.
  if files_on_disk == files_in_manifest
    @my_results.ok('O200', 'verify_manifest', "All files in inventory were found on disk.")
    @my_results.ok('O200', 'verify_manifest', "All discovered files on disk are referenced in inventory file.")
    # Now call verify_versions_across_inventories;  this will check to make sure all prior inventories match this one.
    verify_versions_across_inventories
    return @my_results
  end

  missing_from_disk = []
  files_in_manifest.each do | file |
    # Is there a match to a file in files_on_disk ?
    unless files_on_disk.include? file  # Unless this returns true, the file's not there.
      missing_from_disk << file
    end
  end

  missing_from_manifest = []
  files_on_disk.each do | file |
    unless files_in_manifest.include? file
      missing_from_manifest << file
    end
  end

  unless missing_from_manifest.empty?
    missing_from_manifest.each do |missing|
      @my_results.error('E111', 'verify_manifest', "Unexpected file #{missing} discovered on disk.")
    end
    if missing_from_disk.empty?
      if files_in_manifest.size < 1
        @my_results.warn('W111', 'verify_manifest', 'No files were found in manifest.')
      else
        @my_results.ok('O200', 'verify_manifest', "All files in inventory were found on disk.")
      end
    end
  end

  unless missing_from_disk.empty?
    missing_from_disk.each do |missing|
      @my_results.error('E111', 'verify_manifest', "Expected file #{missing} not found on disk.")
    end
    if missing_from_manifest.empty?
      # This is NOT TRUE if there were no files_on_disk in the first place!
      if files_on_disk.size < 1
        @my_results.warn('W111', 'verify_manifest', "No files discovered on disk.")
      else
        @my_results.ok('O200', 'verify_manifest', "All discovered files on disk are referenced in inventory file.")
      end
    end
  end
  # Now call verify_versions_across_inventories;  this will check to make sure all prior inventories match this one.
  verify_versions_across_inventories
  @my_results
end
verify_structure() click to toggle source

Do all the files and directories in the object_dir conform to spec? Are there inventory.json files in each version directory? (warn if not in version dirs) Deduce version dir naming convention by finding the v1 directory; apply that format to other dirs. @return {OcflTools::OcflResults} of event results

# File lib/ocfl_tools/ocfl_validator.rb, line 316
def verify_structure
  error = nil
  @my_results ||= OcflTools::OcflResults.new
  # 1. use get_version_format to determine the format used for version directories.
  #    If we can't deduce it by inspection of the object_root, ERROR and try and process using site-wide defaults.
  if get_version_format == false
    @my_results.error('E111', 'verify_structure', 'OCFL unable to determine version format by inspection of directories.')
    @error = true
  end

  object_root_dirs  = []
  object_root_files = []

  Dir.chdir(@ocfl_object_root)
  Dir.glob('*').select do |file|
    object_root_dirs << file if File.directory? file
    object_root_files << file if File.file? file
  end

  # 1b. What happens if some this directory is just completely empty?
  if object_root_dirs.size == 0 && object_root_files.size == 0
    @my_results.error('E100', 'verify_sructure', "Object root directory #{@ocfl_object_root} is empty.")
    return @my_results
  end

  # 2. Check object root directory for required files.
  # We have to check the top of inventory.json to get the appropriate digest algo.
  # This is so we don't cause get_digestAlgorithm to throw up if inventory.json doesn't exist.
  file_checks = ['inventory.json']

  # 2a. What digest should the inventory.json sidecar be using? Ask inventory.json.
  # 2b. What's the highest version we should find here?
  # 2c. What should our contentDirectory value be?
  if File.exist? "#{@ocfl_object_root}/inventory.json"
    begin
      @inventory = load_inventory("#{@ocfl_object_root}/inventory.json")
      json_digest      = OcflTools::Utils::Inventory.get_digestAlgorithm("#{@ocfl_object_root}/inventory.json")
      contentDirectory = OcflTools::Utils::Inventory.get_contentDirectory("#{@ocfl_object_root}/inventory.json")
      expect_head      = OcflTools::Utils::Inventory.get_value("#{@ocfl_object_root}/inventory.json", 'head')
      file_checks << "inventory.json.#{json_digest}"
    rescue OcflTools::Errors::ValidationError
      # We couldn't load up the inventory; use site defaults.
      contentDirectory = OcflTools.config.content_directory
      json_digest      = OcflTools.config.digest_algorithm
      file_checks << "inventory.json.#{json_digest}"
    end
  else
    # If we can't get these values from a handy inventory.json, use the site defaults.
    contentDirectory = OcflTools.config.content_directory
    json_digest      = OcflTools.config.digest_algorithm
    file_checks << "inventory.json.#{json_digest}"
  end

  # Error if a required file is not found in the object root.
  # This is now just the check for inventory.json and sidecar file.
  file_checks.each do |file|
    unless object_root_files.include? file
      @my_results.error('E102', 'verify_structure', "Object root does not include required file #{file}")
      error = true
    end
    object_root_files.delete(file)
  end

  # NamAsTe file checks:
  # C1: There should be only 1 file in the root dir beginning with '0=ocfl_object_'
  # C2: That file should match the expected value of OCFL_version (e.g. '0=ocfl_object_1.0')
  # C3: The content of that file should match the filename, less the leading '0='
  root_namaste_files = []
  Dir.glob('0=ocfl_object_*').select do |file|
    root_namaste_files << file if File.file? file
  end

  # C1: We need EXACTLY ONE of these files.
  if root_namaste_files.size == 0
    @my_results.error('E103', 'verify_structure', 'Object root does not include required NamAsTe file.')
    error = true
  end

  if root_namaste_files.size > 1
    @my_results.error('E104', 'verify_structure', "Object root contains multiple NamAsTe files: #{root_namaste_files}")
    error = true
  end

  # C2 and C3 here.
  # If we're dealing with 1 or more ocfl_object_files, process them for correctness.
  unless root_namaste_files.size == 0 || root_namaste_files.size == nil

    # What OCFL version are we looking for? Pull the default value if not otherwise set.
    @ocfl_version ||= OcflTools.config.ocfl_version

    root_namaste_files.each do | file |

      # C2: Is this file the expected version?
      if file != "0=ocfl_object_#{@ocfl_version}"
        @my_results.error('E107', 'verify_structure', "Required NamAsTe file in object root is for unexpected OCFL version: #{file}")
        error = true
      end

      # C3: does the file content match the file name?
      # Cut the first 2 characters from the filename; what remains is the expected content.
      expected_content = file.slice(2..file.size)

      # We use &:gets here instead of &:readline so we don't throw an exception if the file doesn't have content.
      first_line = File.open("#{@ocfl_object_root}/#{file}", &:gets)

      # Handle 'the Namaste file is empty' case.
      if first_line == nil
        @my_results.error('E105', 'verify_structure', 'Required NamAsTe file in object root directory has no content!')
        error = true
        object_root_files.delete(file)
        next
      end

      # it'll have a \n on the end. Remove it, then verify for correct content.
      if first_line.chomp! != expected_content
        @my_results.error('E106', 'verify_structure', 'Required NamAsTe file in object root directory does not contain expected string.')
        error = true
      end
      object_root_files.delete(file)
    end
  end

  # 3. Error if there are extraneous files in object root.
  unless object_root_files.empty?
    @my_results.error('E101', 'verify_structure', "Object root contains noncompliant files: #{object_root_files}")
    error = true
  end

  # 4. Warn if the optional 'logs' directory is found in the object root.
  if object_root_dirs.include? 'logs'
    @my_results.warn('W111', 'verify_structure', 'OCFL 3.1 optional logs directory found in object root.')
    object_root_dirs.delete('logs')
  end

  # 5. Warn if the optional 'extensions' directory is found in object root.
  if object_root_dirs.include? 'extensions'
    @my_results.warn('W111', 'verify_structure', 'OCFL 3.1 optional extensions directory found in object root.')
    object_root_dirs.delete('extensions')
  end

  begin
    version_directories = OcflTools::Utils::Files.get_version_directories(@ocfl_object_root)
  rescue OcflTools::Errors::ValidationError => e
    e.details.each do | code, messages |
      messages.each do | msg |
        @my_results.error(code, 'verify_structure', msg)
      end
    end
    # If we actually throw a validation error, we can't proceed: no version directories found!
    return @my_results
  end

  remaining_dirs = object_root_dirs - version_directories

  # 6. Error if there are extraneous/unexpected directories in the object root.
  unless remaining_dirs.empty?
    @my_results.error('E100', 'verify_structure', "Object root contains noncompliant directories: #{remaining_dirs}")
    error = true
  end

  # 7. Version directories must be a continuous sequence, starting at v1.
  version_dir_count = version_directories.size
  count = 0

  until count == version_dir_count
    count += 1
    expected_directory = @version_format % count
    # just check to see if it's in the array version_directories.
    # We're not *SURE* that what we have is a continous sequence starting at 1;
    # just that they're valid version dir names, sorted in ascending order, and they exist.
    if version_directories.include? expected_directory
      # Could verbose log this here.
      # @my_results.info('I200', 'verify_sructure', "Expected version directory #{expected_directory} found.")
    else
      @my_results.error('E013', 'verify_structure', "Expected version directory #{expected_directory} missing from directory list #{version_directories} ")
      error = true
    end
  end

  # 8. Error if the head version in the inventory does not match the highest version directory discovered in the object root.
  unless expect_head.nil? # No point checking this is we've already failed the root inventory.json check.
    if version_directories[-1] != expect_head
      @my_results.error('E111', 'verify_structure', "Inventory file expects a highest version of #{expect_head} but directory list contains #{version_directories} ")
      error = true
    else
      # could log an 'expected head version found' here.
    end
  end

  # CHECK VERSION DIRECTORY CONTENTS
  # This is setup for the next round of checks.
  # For the version_directories we *do* have, are they cool?
  version_directories.each do |ver|
    version_dirs  = []
    version_files = []

    Dir.chdir("#{@ocfl_object_root}/#{ver}")
    Dir.glob('*').select do |file|
      version_dirs << file if File.directory? file
      version_files << file if File.file? file
    end

    # 9. Warn if inventory.json and sidecar are not present in version directory.
    file_checks = []
    if File.exist? "#{@ocfl_object_root}/#{ver}/inventory.json"
      begin
        @inventory = load_inventory("#{@ocfl_object_root}/#{ver}/inventory.json")
        json_digest      = OcflTools::Utils::Inventory.get_digestAlgorithm("#{@ocfl_object_root}/#{ver}/inventory.json")
        file_checks << 'inventory.json'
        file_checks << "inventory.json.#{json_digest}"
        versionContentDirectory = OcflTools::Utils::Inventory.get_contentDirectory("#{@ocfl_object_root}/#{ver}/inventory.json")
        if versionContentDirectory != contentDirectory
          @my_results.error('E111', 'verify_structure', "contentDirectory value #{versionContentDirectory} in version #{ver} does not match expected contentDirectory value #{contentDirectory}.")
          error = true
        end
      rescue OcflTools::Errors::ValidationError
        # We couldn't load up the inventory; use site defaults.
        # We should also record the error in @my_results?
        json_digest = OcflTools.config.digest_algorithm
        file_checks << 'inventory.json'
        file_checks << "inventory.json.#{json_digest}"
        error = true
      end
    else
      file_checks << 'inventory.json'         # We look for it, even though we know we won't find it, so we can log the omission.
      file_checks << 'inventory.json.sha512'  # We look for it, even though we know we won't find it, so we can log the omission.
    end

    file_checks.each do |file|
      if version_files.include? file
        # The inventory file in the highest version directory MUST match the inventory file in the object root.
        case file
          when 'inventory.json'
            case ver
              # expect_head is nil if there's no inventory.json in the object root, so this test won't run against nothing.
              when expect_head
                # expand_filepaths(@ocfl_object_root)
                my_files = ["#{@ocfl_object_root}/inventory.json", "#{@ocfl_object_root}/#{ver}/inventory.json"]
                # create_digests
                my_digests = OcflTools::Utils::Files.create_digests(my_files, 'sha512')
                # We know there must be only 2 values here, and they should be equal.
                if my_digests.values[0] != my_digests.values[1]
                  @my_results.error('E111', 'verify_structure', "Inventory.json in root and highest version #{ver} MUST match.")
                end
                # Only 1 case block, no 'else'
              end
            # Only 1 case block, no 'else'
          end
        # Expected file exists; delete it from our check list and move on.
        version_files.delete(file)
      else
        @my_results.warn('W111', 'verify_structure', "OCFL 3.1 optional #{file} missing from #{ver} directory")
        version_files.delete(file)
      end
    end

    # 10. Error if files other than inventory & sidecar found in version directory.
    unless version_files.empty?
      @my_results.error('E011', 'verify_structure', "non-compliant files #{version_files} in #{ver} directory")
      error = true
    end

    ###### CHECK THIS; I think this is superseded by the verify_manifest content_dir check.
    # 11. WARN if a contentDirectory exists, but is empty.
    if version_dirs.include? contentDirectory
      version_dirs.delete(contentDirectory)
    end

    # 12. Warn if any directories other than the expected 'content' directory are found in the version directory.
    # This is the "Moab Excepion" to allow for legacy Moab object migration - a 'manifests' directory would be here.
    unless version_dirs.empty?
      @my_results.warn('W101', 'version_structure', "OCFL 3.3 version directory should not contain any directories other than the designated content sub-directory. Additional directories found: #{version_dirs}")
      error = true
    end
  end

  # If we get here without errors (warnings are OK), we passed!
  if error.nil?
    @my_results.ok('O111', 'verify_structure', 'OCFL 3.1 Object root passed file structure test.')
  end
  @my_results
end
verify_version(version) click to toggle source

Different from verify_directory. Verify_version is all versions of the object, up to and including this one. Verify_directory is just check the files and checksums inside that particular version directory. Verify_version(@head) is the canonical way to check an entire object? @param [Integer] version of object to verify @return {OcflTools::OcflResults}

# File lib/ocfl_tools/ocfl_validator.rb, line 652
def verify_version(version)
  # calls verify_directory for 1...n versions.
  count = 1 # start at the bottom
  until count > version # count to the top
    verify_directory(count)
    count += 1
  end
  @my_results
end

Private Instance Methods

compare_inventories(version) click to toggle source

PRIVATE METHOD called by verify_versions_across_inventories Tries to load up an inventory file from the given version directory, and the inventory in the previous version directory.

# File lib/ocfl_tools/ocfl_validator.rb, line 732
def compare_inventories(version)
  # @version_format is important here.
  if get_version_format == false
    @my_results.error('E111', 'compare_inventories', 'OCFL unable to determine version format by inspection of directories.')
    return
  end

  current_version = @version_format % version.to_i
  current_inventory = OcflTools::OcflInventory.new.from_file("#{@ocfl_object_root}/#{current_version}/inventory.json")

  previous_version_int = version - 1
  previous_version = @version_format % previous_version_int.to_i


  if !File.exist? "#{@ocfl_object_root}/#{previous_version}/inventory.json"
    @my_results.error('E111', 'compare_inventories', "OCFL unable to locate previous inventory file at #{@ocfl_object_root}/#{previous_version}/inventory.json.")
    return
  end

  previous_inventory = OcflTools::OcflInventory.new.from_file("#{@ocfl_object_root}/#{previous_version}/inventory.json")

  # Now we have two inventories, we can get their versions blocks.
  #puts current_inventory.versions
  compare_inventories_to_version(current_inventory, previous_inventory, previous_version_int)
end
compare_inventories_to_version(current_inventory, previous_inventory, version) click to toggle source

Given 2 inventories and a version, step down thru versions until you reach 1 and compare their states.

# File lib/ocfl_tools/ocfl_validator.rb, line 759
def compare_inventories_to_version(current_inventory, previous_inventory, version)
  # increment thru versions, calling compare_inventories_version for each.
  until version == 0
    compare_inventories_version(current_inventory, previous_inventory, version)
    version -= 1
  end
end
compare_inventories_version(current_inventory, previous_inventory, version) click to toggle source

Get the version state from each inventory and compare.

# File lib/ocfl_tools/ocfl_validator.rb, line 768
def compare_inventories_version(current_inventory, previous_inventory, version)
  current_version_string = @version_format % version.to_i
  current_version_block = current_inventory.versions[current_version_string]
  previous_version_block = previous_inventory.versions[current_version_string]

  # message, user, created are WARN if different. state is ERROR if different.
  if current_version_block['message'] != previous_version_block['message']
    @my_results.warn('W270', 'compare_inventories_version', "OCFL 3.7 version message mismatch between inventory files: version #{version} message block in #{current_inventory.head}/inventory.json differs from previous inventory.json.")
  end

  if current_version_block['user'] != previous_version_block['user']
    @my_results.warn('W272', 'compare_inventories_version', "OCFL 3.7 version user mismatch between inventory files: version #{version} user block in #{current_inventory.head}/inventory.json differs from previous inventory.json.")
  end

  if current_version_block['created'] != previous_version_block['created']
    @my_results.warn('W271', 'compare_inventories_version', "OCFL 3.7 version created mismatch between inventory files: version #{version} created block in #{current_inventory.head}/inventory.json differs from previous inventory.json.")
  end

  if current_version_block['state'] != previous_version_block['state']
    @my_results.error('E270', 'compare_inventories_version', "OCFL 3.7 version state mismatch between inventory files: version #{version} state block in #{current_inventory.head}/inventory.json differs from previous inventory.json.")
  end

end
get_version_format() click to toggle source
# File lib/ocfl_tools/ocfl_validator.rb, line 792
def get_version_format
  begin
    @my_results ||= OcflTools::OcflResults.new
    @version_format ||= OcflTools::Utils::Files.get_version_format(@ocfl_object_root)
    @my_results.ok('O111', 'version_format', 'OCFL conforming first version directory found.')
    return @version_format
  rescue OcflTools::Errors::ValidationError => e
    # OcflTools::Utils::Files.get_version_format doesn't set errors, so capture them here.
    e.details.each do | code, messages |
      # code is a string, messages is an array.
      messages.each do | msg |
        @my_results.error(code, 'load_inventory', msg)
      end
    end
    # Add on another error explaining how we got here.
    @my_results.error('E111', 'version_format', 'OCFL unable to determine version format by inspection of directories.')
    @my_results.warn('W111', 'version_format', "Attempting to process using default value: #{OcflTools.config.version_format}")
    @version_format = OcflTools.config.version_format
    return false
  end
end
load_inventory(inventory_file) click to toggle source

load up an inventory file and handle any errors. Returns an inventory file is syntatically correct; false if otherwise.

# File lib/ocfl_tools/ocfl_validator.rb, line 686
def load_inventory(inventory_file)
  @my_results ||= OcflTools::OcflResults.new
  OcflTools::OcflInventory.new.from_file(inventory_file)
  # The generic 'something went wrong but I don't know what'; not sure if we should keep this.
  rescue RuntimeError => e
    @my_results.error('E210', 'load_inventory', "#{e}")
    raise
  rescue OcflTools::Errors::ValidationError => e
    e.details.each do | code, messages |
      # code is a string, messages is an array.
      messages.each do | msg |
        @my_results.error(code, 'load_inventory', msg)
      end
    end
    raise # re-raise the error.
end
verify_versions_across_inventories() click to toggle source

Compares the state blocks for all versions across all inventories in the object, and errors if the state block for a given version differs between inventory files. NOTE: this is a private method that should only be called by verify_manifest. That way, we know that @inventory is valid and set by the time we get here.

# File lib/ocfl_tools/ocfl_validator.rb, line 707
def verify_versions_across_inventories
  # OCFL 3.7: In the case that prior version directories include an inventory file
  # there will be multiple inventory files describing prior versions within the OCFL
  # Object. Each version block in each prior inventory file must represent the same
  # object state as the corresponding version block in the current inventory file.
  # Additionally, the values of the created, message and user keys in each version
  # block in each prior inventory file should have the same values as the
  # corresponding keys in the corresponding version block in the current inventory file.

  current_version = @inventory.version_id_list.max
  # Nothing to do if there's only 1 version. Other checks will catch if the inventory
  # in the v1 directory doesn't match the inventory in the object_root.
  return if current_version == 1

  prior_versions = @inventory.version_id_list.max - 1

  until current_version == 1
    compare_inventories(current_version)
    current_version -= 1
  end
end