Knife EC Backup

Umbrella Project: Knife

Project State: Active

Issues {Response Time Maximum}[https://github.com/chef/chef-oss-practices/blob/master/repo-management/repo-states.md]: 14 days

Pull Request {Response Time Maximum}[https://github.com/chef/chef-oss-practices/blob/master/repo-management/repo-states.md]: 14 days

Description

knife-ec-backup can backup and restore the data in an Enterprise Chef Server installation, preserving the data in an intermediate, editable text format. It is similar to the knife download and knife upload commands and uses the same underlying libraries, but also includes workarounds for objects not yet supported by those tools and various Server API deficiencies. The long-run goal is to improve knife download, knife upload and the Chef Infra Server API and deprecate this tool.

Requirements

This knife plugin requires Chef Infra Client 11.8+.

Server Support

This plugin currently supports Enterprise Chef 11 and Chef Infra Server 12+. Support for the beta key rotation features is provided via the --with-keys-sql flag, but users of this feature should note that this may change once the Chef Infra Server supports an API-based export of the key data.

Installation

Chef Infra Server Install (Recommended)

This gem is installed with chef-server-core 12.0.0 and newer.

For Private Chef 11 (or Enterprise Chef 11) you'll need to download and build locally to get the correct dependencies, either with git clone or by downloading the .zip file. Once unpacked, run:

/opt/opscode/embedded/bin/gem build knife-ec-backup.gemspec
/opt/opscode/embedded/bin/gem install knife-ec-backup*gem --no-ri --no-rdoc -V

Note on installing with existing development tools:

The latest versions of knife-ec-backup require gems with native extensions, thus you must install a standard build toolchain. To install knife-ec-backup without installing libpq development headers on your system, try the following:

/opt/opscode/embedded/bin/gem install knife-ec-backup – –with-pg-config=/opt/opscode/embedded/postgresql/9.2/bin/pg_config

This uses the libpq headers that are included in the Chef Infra Server package installed in /opt/opscode.

Chef Workstation Install (Unsupported)

On systems other than the Chef Infra Server, installation of this gem is not tested or supported. However, if you attempt to do so you will need the postgresql libraries installed.

For example, on macOS:

brew install libpq
gem install knife-ec-backup -- --with-pg-config=/usr/local/Cellar/libpq/9.2/bin/pg_config

The current location of pg_config can be determined with brew info libpq.

Running tests

$ bundle install
$ bundle exec rspec spec/

If bundle install fails on the pg gem and the note above does not work for you, try:

$ brew install postgres (if not present)
$ ARCHFLAGS="-arch x86_64" gem install pg
$ bundle exec rspec spec/

Build from source

Clone the git repository and run the following from inside:

gem build knife-ec-backup.gemspec
gem install knife-ec-backup*gem

Configuration

Permissions

Note that most users in an EC installation lack the permissions to pull all of the data from all organizations and other users. This plugin REQUIRES THE PIVOTAL KEY AND WEBUI KEY from the Chef Infra Server. It is recommended that you run this from a frontend Enterprise Chef Infra Server, you can use –user and –key to pass the pivotal information along.

Subcommands

Common Options

The following options are supported across all subcommands:

knife ec backup DEST_DIR (options)

Path: If you have chef-client installed as well, you may need to invoke this as /opt/opscode/embedded/bin/knife ec backup backup

Options

Creates a repository of an entire Enterprise Chef / Private Chef server.

The format of the repository is based on the knife-essentials (knife download) format and looks like this:

users
  <name>.json
user_acls
  <name>.json
organizations
  <orgname>
    acls
      <type>
        <name>.json
    clients
      <name>.json
    containers
      <name>.json
    cookbooks
      <name>-<version>
    data_bags
      <bag name>
        <item name>
    environments
      <name>.json
    groups
      <name>.json
    nodes
      <name>.json
    roles
      <name>.json
    org.json
    members.json
    invitations.json

This compares very closely with the “knife download /” from an OSC server:

clients
  <name>.json
cookbooks
  <name>-<version>
data_bags
  <bag name>
    <item name>
environments
  <name>.json
nodes
  <name>.json
roles
  <name>.json
users
  <name>.json>

knife ec restore DEST_DIR (options)

Restores all data from the specified DEST_DIR to an Enterprise Chef / Private Chef server. DEST_DIR should be a backup directory created by knife ec backup

Options

knife ec key export [FILENAME]

Create a json representation of the users table from the Chef Infra Server database. If no argument is given, the name of the backup is key_dump.json.

Please note, most users should use knife ec backup with the --with-user-sql option rather than this command.

knife ec key import [FILENAME]

Import a json representation of the users table from FILENAME to the the Chef Infra Server database. If no argument is given, the filename is assumed to be key_dump.json.

Please note, most user should use knife ec restore with the --with-user-sql option rather than this command.

Known Bugs

to the number of organizations in your backup and waiting for the pool to fill before running knife ec restore