class Chainer::FunctionNode

Attributes

inputs[RW]
outputs[RW]
rank[RW]

Public Class Methods

new() click to toggle source
# File lib/chainer/function_node.rb, line 17
def initialize
  @rank = 0
  @inputs = nil
  @outputs = nil

  @retained_output_data = nil
  @input_indexes_to_retain = nil
  @output_indexes_to_retain = nil
end

Public Instance Methods

apply(inputs) click to toggle source

Computes output variables and grows the computational graph.

Basic behavior is expressed in the documentation of `FunctionNode`. @param [Chainer::Variable, Numo::NArray] inputs If the element is an Numo::NArray,

it is automatically wrapped with `Chainer::Variable`.

@return [Array<Chainer::Variable>] A tuple of output `Chainer::Variable` objectts.

# File lib/chainer/function_node.rb, line 53
def apply(inputs)
  input_vars = inputs.map { |x| Chainer::Variable.as_variable(x) }
  in_data = input_vars.map(&:data)
  requires_grad = input_vars.map(&:requires_grad).any?

  # Forward propagation
  @input_indexes_to_retain = nil
  @output_indexes_to_retain = nil
  outputs = forward(in_data)
  raise TypeError, "#{outputs.class} not Array" unless outputs.is_a?(Array)

  ret = outputs.map { |y| Chainer::Variable.new(y, requires_grad: requires_grad) }

  if Chainer.configuration.enable_backprop
    # Topological ordering
    @rank = input_vars.size > 0 ? input_vars.map(&:rank).max : 0

    # Add backward edges
    ret.each { |y| y.creator_node = self }
    @inputs = input_vars.map(&:node)
    # Add forward edges (must be weak references)
    @outputs = ret.map { |y| WeakRef.new(y.node) }

    unless @input_indexes_to_retain.nil?
      @input_indexes_to_retain.each do |index|
        input_vars[index].retain_data
      end
    end

    unless @output_indexes_to_retain.nil?
      retained_data = []
      @output_indexes_to_retain.each do |index|
        ret[index].retain_data
        retained_data << outputs[index]
      end
      @retained_output_data = Array(retained_data)
    end
  end

  ret
end
backward(target_indexes, grad_outputs) click to toggle source

Computes gradients w.r.t. specified inputs given output gradients.

This method is used to compute one step of the backpropagation corresponding to the forward computation of this function node. Given the gradients w.r.t. output variables, this method computes the gradients w.r.t. specified input variables. Note that this method does not need to compute any input gradients not specified by `target_input_indexes` It enables the function node to return the input gradients with the full computational history,

in which case it supports *differentiable backpropagation* or *higher-order differentiation*.

@param [Array<Integer>] target_indexes Indices of the input variables w.r.t. which the gradients are required.

It is guaranteed that this tuple contains at least one element.

@param [Array<Chainer::Variable>] grad_outputs Gradients w.r.t. the output variables.

If the gradient w.r.t. an output variable is not given, the corresponding element is `None`.

@return [Array<Chainer::Variable>] Array of Chainer::Variable that represent the gradients.

# File lib/chainer/function_node.rb, line 148
def backward(target_indexes, grad_outputs)
  [nil] * target_indexes.size
end
backward_accumulate(target_input_indexes, grad_outputs, grad_inputs) click to toggle source

Computes gradients w.r.t. specified inputs and accumulates them.

This method provides a way to fuse the backward computation and the gradient accumulations

in the case that the multiple functions are applied to the same variable.

Users have to override either of this method or `backward`. It is often simpler to implement `backward` and is recommended if you do not need to provide efficient gradient accumulation.

@param [Array<Integer>] target_input_indexes Indices of the input variables w.r.t. which the gradients are required.

It is guaranteed that this tuple contains at least one element.

@param [Array<Chainer::Variable>] grad_outputs Gradients w.r.t. the output variables.

If the gradient w.r.t. an output variable is not given, the corresponding element is `None`.

@param [Array<Chainer::Variable>] grad_inputs Gradients w.r.t. the input variables specified by `target_input_indexes`.

These values are computed by other computation paths.
If there is no gradient value existing for the variable, the corresponding element is ``None``.

@return [Array<Chainer::Variable>] Array of variables that represent the gradients w.r.t. specified input variables.

# File lib/chainer/function_node.rb, line 167
def backward_accumulate(target_input_indexes, grad_outputs, grad_inputs)
  gxs = backward(target_input_indexes, grad_outputs)

  len_gxs = gxs.size
  if len_gxs == @inputs.size
    gxs = target_input_indexes.map { |i| gxs[i] }
  elsif len_gxs != target_input_indexes.size
    raise ArgumentError, "number of gradients returned by #{impl_name} (#{label}) is incorrect."
  end

  gxs.zip(grad_inputs).map do |gx, g_input|
    if g_input.nil?
      gx
    elsif gx.nil?
      g_input
    else
      gx + g_input
    end
  end
end
forward(inputs) click to toggle source

Computes the output arrays from the input arrays.

@param [Array] inputs input array(s) @return [Array] output array(s)

# File lib/chainer/function_node.rb, line 99
def forward(inputs)
  raise TypeError, "mustt inputs > 0, inputs size is #{inputs.size}" if inputs.size.zero?
  # TODO GPU
  forward_cpu(inputs)
end
forward_cpu(inputs) click to toggle source

Computes the output arrays from the input Numo::NArray.

@param [Array<Numo::NArray>] inputs Numo::NArray objects. @return [Array<Numo::NArray>] Array of output arrays.

# File lib/chainer/function_node.rb, line 109
def forward_cpu(inputs)
  raise NotImplementedError
end
get_retained_inputs() click to toggle source

Returns a Array of retained input variables.

This method is used to retrieve the input variables retained in `forward`.

@return [Array] a Array of retained input variables.

# File lib/chainer/function_node.rb, line 193
def get_retained_inputs
  @input_indexes_to_retain.map { |index| @inputs[index].get_variable }
end
get_retained_outputs() click to toggle source

Returns a Array of retained output variables.

This method is used to retrieve the input variables retained in `forward`.

@return [Array] a Array of retained input variables.

# File lib/chainer/function_node.rb, line 202
def get_retained_outputs
  ret = []
  outputs = @outputs

  new_outputs = outputs.dup
  outputs_modified = false

  @output_indexes_to_retain.zip(@retained_output_data) do |index, data|
    output =  outputs[index].__getobj__
    if output.nil?
      output_var = Chainer::Variable.new(data)
      output_var.creator_node = self
      new_outputs[index] = WeakRef.new(output_var)
      outputs_modified = true
    else
      output_var = output.get_variable
    end

    ret << output_var
  end

  if outputs_modified
    @outputs = Array(new_outputs)
  end

  ret
end
label() click to toggle source

Short text that represents the function.

The default implementation returns its type name. Each function should override it to give more information.

# File lib/chainer/function_node.rb, line 31
def label
  self.class.name
end
output_data() click to toggle source

A tuple of the retained output arrays. This property is mainly used by $Function$. Users basically do not have to use this property; use $get_retained_outputs$ instead.

# File lib/chainer/function_node.rb, line 38
def output_data
  raise RuntimeError, 'retained output data is gone' if @retained_output_data.nil?
  out_data = [nil] * @outputs.size
  @output_indexes_to_retain.zip(@retained_output_data).each do |index, data|
    out_data[index] = data
  end
  out_data
end
retain_inputs(indexes) click to toggle source

Lets specified input variable nodes keep data arrays.

By calling this method from `forward`, the function node can specify which inputs are required for backprop. The input variables with retained arrays can be obtained by `get_retained_inputs` from `backward`.

Note that **this method must not be called from the outside of forward method.** @param [Integer, Array] indexes Indexes of input variables that the function does not require for backprop.

# File lib/chainer/function_node.rb, line 120
def retain_inputs(indexes)
  @input_indexes_to_retain = indexes
end
retain_outputs(indexes) click to toggle source

Lets specified output variable nodes keep data arrays.

By calling this method from `forward`, the function node can specify which outputs are required for backprop. If this method is not called, any output variables are not marked to keep the data array at the point of returning from `apply`. The output variables with retained arrays can be obtained by `get_retained_outputs` from `backward`. Note that **this method must not be called from the outside of forward method.** @param [Integer, Array] indexes Indexes of input variables that the function does not require for backprop.

# File lib/chainer/function_node.rb, line 131
def retain_outputs(indexes)
  @output_indexes_to_retain = indexes
end
unchain() click to toggle source

Purges in/out nodes and this function node itself from the graph.

# File lib/chainer/function_node.rb, line 231
def unchain
  @outputs.each do |y|
    y_ref = y.()
    unless y_ref.nil?
      y_ref.unchain
    end
  end
  @inputs = nil
end

Private Instance Methods

impl_name() click to toggle source
# File lib/chainer/function_node.rb, line 243
def impl_name
  self.class.name
end