Composable Validations

Gem for validating complex JSON payloads.

Features

Requirements

Install

gem install composable_validations

Quick guide

This gem allows you to build a validator - how/when you call this validation is up to you.

Basic example

Say we want to validate a payload that specifies a person with name and age. E.g. {"person" => {"name" => "Bob", "age" => 28}}

require 'composable_validations'

include ComposableValidations

# building validator function
validator = a_hash(
  allowed_keys("person"),
  key("person", a_hash(
    allowed_keys("name", "age"),
    key("name", non_empty_string),
    key("age", non_negative_integer))))

# invalid payload with non-integer age
payload = {
  "person" => {
    "name" => 123,
    "age" => "mistake!"
  }
}

# container for error messages
errors = {}

# application of the validator to the payload with default error messages
valid = default_errors(validator).call(payload, errors)

if valid
  puts "payload is valid"
else
  # examine error messages collected by validator
  puts errors.inspect
end

In the example above the payload is invalid and as a result valid has value false and errors contains:

{"person/name"=>["must be a string"], "person/age"=>["must be an integer"]}

Note that invalid elements of the payload are identified by exact path within the payload.

Sinatra app

When using this gem in your application code you would only include ComposableValidations module in classes responsible for validation.

Extending previous example into Sinatra app:

require 'sinatra'
require 'json'
require 'composable_validations'

post '/' do
  payload = JSON.parse(request.body.read)
  validator = PersonValidator.new(payload)

  if validator.valid?
    status(204)
  else
    status(422)
    validator.errors.to_json
  end
end

class PersonValidator
  include ComposableValidations
  attr_reader :errors

  def initialize(payload)
    @payload = payload
    @errors = {}

    @validator = a_hash(
      allowed_keys("person"),
      key("person", a_hash(
        allowed_keys("name", "age"),
        key("name", non_empty_string),
        key("age", non_negative_integer))))
  end

  def valid?
    default_errors(@validator).call(@payload, @errors)
  end
end

Arrays

The previous examples showed validation of a JSON object. We can also validate JSON arrays. Let’s add list of hobbies to our person object from the previous examples:

{
  "person" => {
    "name" => "Bob",
    "age" => 28,
    "hobbies" => ["knitting", "horse riding"]
  }
}

We will also not accept people with fewer than two hobbies. Validator for this payload:

a_hash(
  allowed_keys("person"),
  key("person", a_hash(
    allowed_keys("name", "age", "hobbies"),
    key("name", non_empty_string),
    key("age", non_negative_integer),
    key("hobbies", array(
      min_size(2),
      each(non_empty_string))))))

Try to apply this validator to the payload containing invalid list of hobbies

...
"hobbies" => ["knitting", {"not" => "allowed"}, "horse riding"]
...

and you’ll get errors specifying exactly where the invalid element is:

{"person/hobbies/1"=>["must be a string"]}

Dependent validations

Sometimes we need to ensure that elements of the payload are in certain relation.

We can ensure simple relations between keys using validators key_greater_than_key, key_less_than_key etc. Check out Composability for example of simple relation between keys.

Uniqueness

For uniqueness validation follow the example in Custom validators.

Key concepts

Validators

Validator is a function returning boolean value and having following signature:

lambda { |validated_object, errors_hash, path| ... }

This gem comes with basic validators like a_hash, array, string, integer, float, date_string, etc. You can find complete list of validators below. Adding new validators is explained in (Custom validators).

Combinators

Validators can be composed using two combinators:

Return values of above combinators are themselves validators. This way they can be further composed into more powerful validation rules.

Composability

We want to validate object representing opening hours of a store. E.g. store opened from 9am to 5pm would be represented by

{"from" => 9, "to" => 17}

Let’s start by building validator ensuring that payload is a hash where both from and to are integers:

a_hash(
  key("from", integer),
  key("to", integer))

We also want to make sure that extra keys like

{"from" => 9, "to" => 17, "something" => "wrong"}

are not allowed. Let’s fix it by using allowed_keys validator:

a_hash(
  allowed_keys("from", "to"),
  key("from", integer),
  key("to", integer))

Better, but we don’t want to allow negative hours like this:

{"from" => -1, "to" => 17}

We can fix it by using more specific integer validator:

a_hash(
  allowed_keys("from", "to"),
  key("from", non_negative_integer),
  key("to", non_negative_integer))

Let’s assume here that we represent store opened all day as

{"from" => 0, "to" => 24}

so hours greater than 24 should also be invalid. We can validate hour by composing non_negative_integer validator with less_or_equal using fail_fast combinator:

hour = fail_fast(non_negative_integer, less_or_equal(24))

a_hash(
  allowed_keys("from", "to"),
  key("from", hour),
  key("to", hour))

This validator still has a little problem. Opening hours like this are not rejected:

{"from" => 21, "to" => 1}

We have to make sure that closing is not before opening. We can do it by using key_greater_than_key validator:

key_greater_than_key("to", "from")

and our validator will look like this:

a_hash(
  allowed_keys("from", "to"),
  key("from", hour),
  key("to", hour),
  key_greater_than_key("to", "from"))

That looks good, but it’s not complete yet. a_hash validator applies all validators to the provided payload by using run_all combinator. This behaviour is problematic if our from or to keys are missing or are not valid integers. Payload

{"from" => "abc", "to" => 17}

will cause an exception as key_greater_than_key can not compare string to integer. Let’s fix it by using fail_fast and run_all combinators:

a_hash(
  allowed_keys("from", "to"),
  fail_fast(
    run_all(
      key("from", hour),
      key("to", hour)),
    key_greater_than_key("to", "from")))

This way if from and to are not both valid hours we will not be comparing them.

You can see this validator reused in a bigger example below.

Path to an invalid element

Validation errors on deeply nested JSON structure will always contain exact path to the invalid element.

Example

Let’s say we validate stores. Example of store object:

store = {
  "store" => {
    "name"        => "Scrutton Street",
    "description" => "large store",
    "opening_hours" => {
      "monday"   => {"from" =>  9, "to" => 17},
      "tuesday"  => {"from" =>  9, "to" => 17},
      "wednesday"=> {"from" =>  9, "to" => 17},
      "thursday" => {"from" =>  9, "to" => 17},
      "friday"   => {"from" =>  9, "to" => 17},
      "saturday" => {"from" => 10, "to" => 16}
    },
    "employees"=> ["bob", "alice"]
  }
}

Definition of the store validator (using from_to built in the previous section):

hour = fail_fast(non_negative_integer, less_or_equal(24))

from_to = a_hash(
  allowed_keys("from", "to"),
  fail_fast(
    run_all(
      key("from", hour),
      key("to", hour)),
    key_greater_than_key("to", "from")))

store_validator = a_hash(
  allowed_keys("store"),
  key("store",
    a_hash(
      allowed_keys("name", "description", "opening_hours", "employees"),
      key("name", non_empty_string),
      optional_key("description"),
      key("opening_hours",
        a_hash(
          allowed_keys("monday", "tuesday", "wednesday", "thursday", "friday", "saturday", "sunday"),
          optional_key("monday",    from_to),
          optional_key("tuesday",   from_to),
          optional_key("wednesday", from_to),
          optional_key("thursday",  from_to),
          optional_key("friday",    from_to),
          optional_key("saturday",  from_to),
          optional_key("sunday",    from_to))),
      key("employees", array(each(non_empty_string))))))

Let’s say we try to validate store that has Wednesday opening hours invalid (closing time before opening time) like this:

...
"wednesday"=> {"from" => 9, "to" => 7},
...

Now we use store validator to fill in the collection of errors using default error messages:

errors = {}
  result = default_errors(store_validator).call(store, errors)

Result is false and we get validation error in the errors hash:

{"store/opening_hours/wednesday/to" => ["must be greater than from"]}

You can find this example in functional spec ready for experiments.

Overriding error messages

This gem comes with set of default error messages. There are few ways to provide your own error messages.

Local override

You can override error message when building your validator:

a_hash(
  key("from", integer("custom error message")),
  key("to", integer("another custom error message")))

This approach is good if you need just few specialized error messages for different parts of your payload.

Global override

If you need to change some of the error messages across all your validators you can provide map of error messages. Keys in the map are symbols matching names of basic validators:

error_overrides = {
    string:  "not a string",
    integer: "not an integer"
  }

  errors = {}
  errors_container = ComposableValidations::Errors.new(errors, error_overrides)
  result = validator.call(valid_data, errors_container, nil)

Note that your error messages don’t need to be strings. You could for example use rendering function that returns combination of error code, error context and human readable message:

error_overrides = {
    key_greater_than_key: lambda do |validated_object, path, key1, key2|
      {
        code: 123,
        context: [key1, key2],
        message: "#{key1}=#{object[key1]} is not less than or equal to #{key2}=#{object[key2]}"
      }
    end
  }

  errors = {}
  errors_container = ComposableValidations::Errors.new(errors, error_overrides)
  result = validator.call(valid_data, errors_container, nil)

And when applied to invalid payload your validator will return an error:

{
    "store/opening_hours/wednesday/to"=>
      [
        {
          :code=>123,
          :context=>["to", "from"],
          :message=>"to=17 is not less than or equal to from=24"
        }
      ]
  }

You can experiment with this example in the specs.

Override error container

You can override error container class and provide any error collecting behaviour you need. The only method error container must provide is:

def add(msg, path, object)

where * msg is a symbol of an error or an array where first element is a symbol of error and remaining elements are context needed to render the error message. * path represents a path to the invalid element within the JSON object. It is an array of strings (keys in hash map) and integers (indexes in array). * object is a validated object.

Example of error container that just collects error paths:

class CollectPaths
  attr_reader :paths

  def initialize
    @paths = []
  end

  def add(msg, path, object)
    @paths << path
  end
end

validator = ...
errors_container = CollectPaths.new
result = validator.call(valid_data, errors_container, nil)

and example of the value of errors_container.paths after getting an error:

[["store", "opening_hours", "wednesday", "to"]]

You can experiment with this example in the spec.

Custom validators

You can create your own validators as functions returning lambdas with signature

lambda { |validated_object, errors_hash, path| ... }

Use error helper function to add errors to the error container and functions validate, precheck and nil_or to avoid boilerplate.

Example

Let’s say we have an ActiveRecord model Store and API allowing update of the store name. We will be receiving payload:

{ name: 'new store name' }

We can build validator ensuring uniqueness of the store name:

a_hash(
  allowed_keys('name'),
  key('name',
    non_empty_string,
    unique_store_name))

where unique_store_name is defined as:

def unique_store_name
  lambda do |store_name, errors, path|
    if !Store.exists?(name: store_name)
      true
    else
      error(errors, "has already been taken", store_name, path)
    end
  end
end

Note that we could simplify this code by using validate helper method:

def unique_store_name
  validate("has already been taken") do |store_name|
    !Store.exists?(name: store_name)
  end
end

We could also generalize this function and end up with generic ActiveModel attribute uniqueness validator ready to be reused:

def unique(klass, attr_name)
  validate("has already been taken") do |attr_value|
    !klass.exists?(attr_name => attr_value)
  end
end

a_hash(
  allowed_keys('name'),
  key('name',
    non_empty_string,
    unique(Store, :name)))

API

ruby errors = {} default_errors(validator).call(validated_object, errors)

ruby array( each_in_slice(0..-2, normal_element_validator), each_in_slice(-1..-1, special_last_element_validator))

ruby precheck(float) { |v| v == 'infinity' }

ruby validate('must be "hello"') { |v| v == 'hello' }