class GitKeyvalue::KeyValueRepo

Provides a GET/PUT-style interface for a git repo. In effect, it presents the repo as a key/value store, where the keys are file paths (relative to the repo’s root) and the values are the contents of those files.

Requirements

Known good with ruby 1.9.3 and git 1.7.9.6.

Performance & resource usage

Not performant. Must clone the repo before performing any operations. Needs whatever disk space is required for a repo clone. Clears this space when the object is destroyed.

Object lifetime

Stores the local repo in the OS’s temporary directory. Therefore, you should not expect this object to remain valid across automated housekeeping events that might destroy this directory.

Footnote on shallow cloning

Okay, technically, this does not clone the entire repo. For better performance it does a “shallow clone” of the repo, which grabs only the files necessary to represent the HEAD commit. Such a shallow clone is officially enough to enable GET operations, which read only those files anyway. However, according to the git-clone docs, the shallow clone is not officially enough to enable git-push to update those files on the remote repo. However, this seems like a bug in the git-clone docs since, in reality, a shallow clone is enough and should be enough for pushing new commits, since a new commit only needs to reference its parent commit(s).

The bottom line: by using shallow cloning for better perf, this class is relying on undocumented behavior in git-push. This works fine as of git version 1.7.9.6. I see no reason to expect this to break in the future, since this undocumented behavior follows directly from git’s data model, which is stable. However, if it does break, and you want to switch to using the documented git behavior, then set USE_SHALLOW_CLONING to false.

Constants

USE_SHALLOW_CLONING

whether to git-clone only the HEAD commit of the remote repo

Attributes

path_to_repo[R]

@return [String] absolute filesystem path of the local repo

repo_url[R]

@return [String] URL of the remote git repo

Public Class Methods

new(repo_url) click to toggle source

Clones the remote repo, failing if it is invalid or inaccessible.

@param repo_url [String] URL of a valid, network-accessible, permissions-accessible git repo

As it clones the entire repo, this may take a long time if you are manipulating a large remote repo. Keeps the repo in the OS’s temporary directory, so you should not expect this object to remain valid across automated cleanups of that temporary directory (which happen, for instance, typically at restart).

@raise [KeyValueGitError] if unable to clone the repo

# File lib/git_keyvalue.rb, line 185
def initialize(repo_url)
  @repo_url = repo_url
  @path_to_repo = Dir.mktmpdir('KeyValueGitTempDir')
  if USE_SHALLOW_CLONING
    # experimental variant. uses undocumented behaviour of
    # git-clone. This is because setting --depth 1 produces a
    # shallow clone, which according to the docs does not let you
    # git-push aftewards, but in reality should and does let you
    # git-push. This is a bug in the git documentation.
    success = system('git','clone','--depth','1',@repo_url,@path_to_repo)  
  else
    # stable variant. uses documented behavior of git-clone
    success = system('git','clone',@repo_url,@path_to_repo) 
  end
  if not success
    raise KeyValueGitError, 'Failed to initialize, because could not clone the remote repo: ' + repo_url + '. Please verify this is a valid git URL, and that any required network connection or login credentials are available.'
  end
  ObjectSpace.define_finalizer(self, self.class.make_finalizer(@path_to_repo))
end

Private Class Methods

make_finalizer(tmp_dir) click to toggle source

@return [Proc] proc which removes the temporary local clone of the repo

# File lib/git_keyvalue.rb, line 61
def self.make_finalizer(tmp_dir)
  proc do
    puts 'KeyValueRepo: Remove local repo clone in ' + tmp_dir
    FileUtils.remove_entry_secure(tmp_dir)
  end
end

Public Instance Methods

get(path_in_repo) click to toggle source

Get contents of a file, or nil if it does not exist

@param path_in_repo [String] relative path of repo file to get @return [String,nil] string contents of file, or nil if non-existent

# File lib/git_keyvalue.rb, line 211
def get(path_in_repo)
  outer_get(path_in_repo) { |abspath| File.read(abspath) }
end
getfile(path_in_repo, dest_path) click to toggle source

Copies the repo file at path_in_repo to dest_path

@param path_in_repo [String] relative path of repo file to get @param dest_path [String] path to which to copy the gotten file

Does no validation regarding dest_path. If dest_path points to a file, it will overwrite that file. If it points to a directory, it will copy into that directory.

# File lib/git_keyvalue.rb, line 224
def getfile(path_in_repo, dest_path)
  outer_get(path_in_repo) { |abspath| FileUtils.cp(abspath, dest_path) }
end
put(path_in_repo, string_value) click to toggle source

Sets the contents of the file at path_in_repo, creating it if necessary

@param path_in_repo [String] relative path of repo file to add or update @param string_value [String] the new contents for the file at this path

# File lib/git_keyvalue.rb, line 234
def put(path_in_repo, string_value)
  path_in_repo = blindly_relativize_path(path_in_repo)
  outer_put(path_in_repo) {
    # create parent directories if needed
    FileUtils.mkdir_p(File.dirname(path_in_repo))
    # write new file contents
    File.open(path_in_repo,'w') { |f| f.write(string_value) }
  }    
end
putfile(path_in_repo, src_file_path) click to toggle source

Sets the contents of the file at path, creating it if necessary

@param path_in_repo [String] relative path of repo file to add or update @param src_file_path [String] file to use for replacing path_in_repo

# File lib/git_keyvalue.rb, line 250
def putfile(path_in_repo, src_file_path)
  path_in_repo = blindly_relativize_path(path_in_repo)
  outer_put(path_in_repo) {
    # create parent directories if needed
    FileUtils.mkdir_p(File.dirname(path_in_repo))
    # copy file at src_file_path into the path_in_repo
    abspath = Pathname.new(File.join(@path_to_repo,path_in_repo)).to_s
    FileUtils.cp(src_file_path, abspath)
  }
end

Private Instance Methods

blindly_relativize_path(maybe_abspath) click to toggle source

Strips any initial / chars from maybe_abspath

@param [String] maybe_abspath @return [String]

# File lib/git_keyvalue.rb, line 105
def blindly_relativize_path(maybe_abspath)
  (maybe_abspath.split('').drop_while {|ch| ch=='/'}).join
end
isFileExistingWithinRepo(path_in_repo) click to toggle source

Checks if path_in_repo points to a file existing in the repo.

@param [String] path_in_repo @return [Boolean] whether

Even if path_in_repo starts with /, it will be interpreted as relative to the repo’s root.

# File lib/git_keyvalue.rb, line 88
def isFileExistingWithinRepo(path_in_repo)
  abspath = Pathname.new(File.join(@path_to_repo,path_in_repo))
  # see if the file exists and is a file
  if abspath.file?
    # and if it's within the repo
    abspath.realpath.to_s.start_with?(Pathname.new(@path_to_repo).realpath.to_s)
  else
    false
  end
end
outer_get(path_in_repo) { |abspath| ... } click to toggle source

Ensure a file exists and execute a GET-like operation, passed as a block.

@param path_in_repo [String] relative path of a repo file @yieldparam abspath [String] absolute filesystem path for the block to GET @yieldreturn [Object,nil] result of GETting the file, or nil if the block returned its value through side-effects @return [Object,nil] the result returned by the block, or nil if the file does not exist @raise [KeyValueGitError] if cannot pull from the repo @raise [Exception] if the block raises an Exception

Updates the local repo. Verifies the file exists at path_in_repo. If it does not exist or is outside of the repo, returns nil. Otherwise, returns the result of calling the block.

This method will raise whatever the block raises

# File lib/git_keyvalue.rb, line 124
def outer_get(path_in_repo)
  update_local_repo
  if not isFileExistingWithinRepo(path_in_repo)
    nil
  else
    abspath = Pathname.new(File.join(@path_to_repo,path_in_repo)).realpath.to_s
    yield abspath
  end
end
outer_put(path_in_repo) { || ... } click to toggle source

Prepares and executes a PUT-like operation, passed as a block

@param path_in_repo [String] relative path of repo file @yield @return the result returned by the block

@raise [KeyValueGitError] if can’t pull or push the repo @raise [Exception] if the block raises an Exception

Update the local repo, changes to its root directory, then calls the block to execute the PUT operation on the repo’s working tree. Then commits and push that change to the remote repo.

# File lib/git_keyvalue.rb, line 147
def outer_put(path_in_repo)
  update_local_repo
  Dir.chdir(@path_to_repo) do

    yield

    # add and commit to repo
    system('git','add',path_in_repo)
    system('git','commit','-m','\'git-keyvalue: updating ' + path_in_repo + '\'')
    success = system('git','push')
    if not success
      # restore local repo to a good state
      system('git','clean','--force','-d')
      # report the failure
      raise KeyValueGitError, 'Failed to push commit with updated file. This could be because someone else pushed to the repository in the middle of this operation. If this is the problem, you should be able simply to re-try this operation. If the problem is deeper, you might create a fresh object before re-trying.'
    end
  end
end
update_local_repo() click to toggle source

Updates the local clone of the repo @raise [KeyValueGitError] if cannot pull the repo.

# File lib/git_keyvalue.rb, line 71
def update_local_repo
  Dir.chdir(@path_to_repo) do
    success = system('git','pull')
    if not success
      raise KeyValueGitError, 'Failed to pull updated version of the repo, even though it was cloned successfully. Aborting.'
    end
  end
end