Modulation
- Explicit Dependency Management for Ruby¶ ↑
Modulation
| mɒdjʊˈleɪʃ(ə)n | Music - a change from one key to another in a piece of music.
INSTALL | GUIDE | API | EXAMPLES | RDOC
Modulation
provides an alternative way of organizing your Ruby code. Modulation
lets you explicitly import and export declarations in order to better control dependencies in your codebase. Modulation
helps you refrain from littering the global namespace with a myriad modules, or complex multi-level nested module hierarchies.
Using Modulation
, you will always be able to tell where a class or module comes from, and you'll have full control over which parts of a module's code you wish to expose to the outside world. Modulation
can also help you write Ruby code in a functional style, minimizing boilerplate code.
Note:
Modulation
is not a replacement for RubyGems. Rather,Modulation
is intended for managing dependencies between source files inside your Ruby applications. Though it does support loading gems that were written usingModulation
, it is not intended as a comprehensive solution for using third-party libraries.
Features¶ ↑
-
{Complete isolation of each module}[#organizing-your-code-with-modulation] prevents literring of the global namespace.
-
Explicit {exporting}[#exporting-declarations] and {importing}[#importing-declarations] of methods and constants lets you control the public interface for each module, as well as keep track of all dependencies in your code.
-
{Tagged paths}[#using-tags-to-designate-common-subdirectories] simplify the management of dependencies in large applications.
-
{Lazy Loading}[#lazy-loading] improves start up time and memory consumption.
-
{Hot module reloading}[#reloading-modules] streamlines your development process.
-
{Module mocking}[#mocking-modules] facilitates testing.
-
{Dependency introspection}[#dependency-introspection] lets you introspect your dependencies at runtime.
-
{Application packing}[#packing-applications-with-modulation] lets you bundle your code in a single, optionally obfuscated file (WIP).
Rationale¶ ↑
You're probably asking yourself “what the ****?” , but when your Ruby app grows and is split into multiple files loaded using #require
, you'll soon hit some issues:
-
Once a file is
#require
d, any class, module or constant in it is available to any other file in your codebase. All “globals” (classes, modules, constants) are loaded, well, globally, in a single namespace. Name conflicts are easy in Ruby. -
To avoid class name conflicts, classes need to be nested under a single hierarchical tree, sometime reaching 4 levels or more. Just look at Rails.
-
Since a
#require
d class or module can be loaded in any file and then made available to all files, it's easy to lose track of where it was loaded, and where it is used. -
There's no easy way to hide implementation-specific classes or methods. Yes, there's
#private
,#private_constant
etc, but by default everything is#public
! -
Extracting functionality is harder when modules are namespaced and dependencies are implicit.
-
Writing reusable functional code requires wrapping it in modules using
class << self
,def self.foo ...
,extend self
orinclude Singleton
(the pain of implementing singletons in Ruby has been discussed before.)
There's a recent discussion on the Ruby bug tracker regarding possible solutions to the problem of top-level name collision. Hopefully, the present gem could contribute to an eventual “official” API.
Personally, I have found that managing dependencies with #require
in large codebases is… not as elegant or painfree as I would expect from a first-class development environment. I also wanted to have a better solution for writing in a functional style.
So I came up with Modulation
, a small gem that takes a different approach to organizing Ruby code: any so-called global declarations are hidden unless explicitly exported, and the global namespace remains clutter-free. All dependencies between source files are explicit, visible, and easy to understand.
Installing Modulation
¶ ↑
You can install the Modulation
using gem install
, or add it to your Gemfile
:
gem 'modulation'
Organizing your code with Modulation
¶ ↑
Modulation
builds on the idea of a Ruby Module
as a “collection of methods and constants”. Using modulation, each Ruby source file becomes a module. Modules usually export method and constant declarations (usually an API for a specific, well-defined functionality) to be shared with other modules. Modules can also import declarations from other modules. Anything not exported remains hidden inside the module and normally cannot be accessed from the outside.
Each source file is evaluated in the context of a newly-created Module
instance, with some additional methods for introspection and miscellaneous operations such as hot reloading.
Modulation
provides alternative APIs for loading modules. Instead of using require
and require_relative
, we use import
, import_map
etc, discussed in detail in the API reference.
Basic Usage¶ ↑
Exporting declarations¶ ↑
Any class, module or constant be exported using #export
:
export :User, :Session class User ... end class Session ... end
A module may also expose a set of methods without using class << self
, for example when writing in a functional style:
seq.rb
export :fib, :luc def fib(n) (0..1).include?(n) ? n : (fib(n - 1) + fib(n - 2)) end def luc(n) (0..1).include?(n) ? (2 - n) : (luc(n - 1) + luc(n - 2)) end
app.rb
require 'modulation' Seq = import('./seq') puts Seq.fib(10)
Another way to export methods and constants is by passing a hash to #export
:
module.rb
export( foo: :bar, baz: -> { 'hello' }, MY_CONST: 42 ) def bar :baz end
app.rb
m = import('./module') m.foo #=> :baz m.baz #=> 'hello' m::MY_CONST #=> 42
Any capitalized key will be interpreted as a const, otherwise it will be defined as a method. If the value is a symbol, Modulation
will look for the corresponding method or const definition and will treat the key as an alias.
The export
method can be called multiple times. Its behavior is additive:
# this: export :foo, :bar # is the same as this: export :foo export :bar
Importing declarations¶ ↑
Declarations from another module can be imported using #import
:
require 'modulation' Models = import('./models') ... user = Models::User.new(...) ...
Alternatively, a module interested in a single declaration from another module can use the following technique:
require 'modulation' User = import('./models')::User ... user = User.new(...)
A word about paths¶ ↑
Paths given to import
are always considered relative to the importing file, unless they are absolute (e.g. /home/dave/repo/my_app
), specify a tag or reference a gem. This is true for all Modulation
APIs that accept path arguments.
Advanced Usage¶ ↑
Using tags to designate common subdirectories¶ ↑
Normally, module paths are always relative to the file calling the #import
method, just like #require_relative
. This can become a problem once you start moving your source files around. In addition, in applications where your source files are arranged in multiple directories, it can quickly become tedious to do stuff like Post = import('../models/post')
.
Modulation
provides an alternative to relative paths in the form of tagged sources. A tagged source is simply a path associated with a label. For example, an application may tag lib/models
simply as @models
. Once tags are defined, they can be used when importing files, e.g. import('@models/post')
.
To define tags, use Modulation.add_tags
:
Modulation.add_tags( models: '../lib/models', views: '../lib/views' ) ... User = import '@models/user'
Importing all source files in a directory¶ ↑
To load all source files in a directory you can use #import_all
:
import_all('./ext')
Groups of modules providing a uniform interface can also be loaded using #import_map
:
API = import_map('./math_api') #=> hash mapping filenames to modules API.keys #=> ['add', 'mul', 'sub', 'div'] API['add'].(2, 2) #=> 4
The #import_map
takes an optional block to transform hash keys:
API = import_map('./math_api') { |name, mod| name.to_sym } API.keys #=> [:add, :mul, :sub, :div] API[:add].(2, 2) #=> 4
Importing methods into classes and objects¶ ↑
Modulation
provides the #extend_from
and #include_from
methods to include imported methods in classes and objects:
module Sequences extend_from('./seq.rb') end Sequences.fib(5) # extend integers require 'modulation' class Integer include_from('./seq.rb') def seq(kind) send(kind, self) end end 5.seq(:fib)
The #include_from
method accepts an optional list of symbols to import:
class Integer include_from './seq.rb', :fib end 5.fib
Default exports¶ ↑
A module may wish to expose just a single class or constant, in which case it can use #export_default
:
user.rb
export_default :User class User ... end
app.rb
require 'modulation' User = import('./user') User.new(...)
The default exported value can also be defined directly thus:
config.rb
export_default( host: 'localhost', port: 1234, ... )
app.rb
require 'modulation' config = import('./config') db.connect(config[:host], config[:port])
Circular dependencies¶ ↑
Circular dependencies, while not the best practice for organizing a code base, are sometimes useful. Modulation
supports circular dependencies, with the exception of modules with default exports.
Accessing a module's root namespace from nested modules within itself¶ ↑
The special constant MODULE
allows you to access the containing module from nested modules or classes. This lets you call methods defined in the module's root namespace, or otherwise introspect the module:
export :AsyncServer # Await a promise-like callable def await calling_fiber = Fiber.current p = ->(v = nil) {calling_fiber.resume v} yield p Fiber.yield end class AsyncServer < SomeTCPServer def async_read MODULE.await {|p| on_read {|data| p.(data)}} end end
Accessing the global namespace¶ ↑
If you need to access the global namespace inside a module just prefix the class name with double colons:
class ::GlobalClass ... end ::ENV = { ... } what_is = ::THE_MEANING_OF_LIFE
Programmatic module creation¶ ↑
In addition to loading modules from files, modules can be created dynamically at runtime using Modulation.create
. You can create modules by supplying a hash prototype, a string or a block:
# Using a hash prototype m = Modulation.create( add: -> x, y { x + y }, mul: -> x, y { x * y } ) m.add(2, 3) m.mul(2, 3) # Using a string m = Modulation.create <<~RUBY export :foo def foo :bar end RUBY m.foo # Using a block m = Modulation.create do { |mod| export :foo def foo :bar end class mod::BAZ ... end } m.foo
The creation of a objects using a hash prototype is also available as a separate gem called eg.
Unit testing modules¶ ↑
Methods and constants that are not exported can be tested using the #__expose!
method. Thus you can keep implementation details hidden, while being able to easily test them:
parser.rb
export :parse def parse(inp) split(inp).map(&:to_sym) end # private method def split(inp) inp.split(',').map(&:strip) end
test_seq.rb
require 'modulation' require 'minitest/autorun' Parser = import('../lib/parser').__expose! class FibTest < Minitest::Test def test_that_split_trims_split_parts assert_equal(%w[abc def ghi], Parser.split(' abc ,def , ghi ')) end end
Mocking modules¶ ↑
Modules loaded by Modulation
can be easily mocked when running tests or specs, using Modulation.mock
:
require 'minitest/autorun' require 'modulation' module MockStorage extend self def get_user(user_id) { user_id: user_id, name: 'John Doe', email: 'johndoe@gmail.com' } end end class UserControllerTest < Minitest::Test def test_user_storage Modulation.mock('../lib/storage', MockStorage) do controller = UserController.new ... end end end
Modulation.mock
accepts a module path and a receiver, and the module stays mocked within the given block.
Lazy Loading¶ ↑
Modulation
allows the use of lazy-loaded modules - loading of modules only once they're needed by the application, in similar fashion to Module#auto_load
. To lazy load modules use the #auto_import
method, which takes a constant name and a path:
export :foo auto_import :BAR, './bar' def foo # the bar module will only be loaded once this method is called MODULE::BAR end
Lazy-loaded constants must always be qualified. When referring to a lazy-loaded constant from the module's top namespace, use the
MODULE
namespace, as shown above.
The #auto_import
method can also take a hash mapping constant names to paths. This is especially useful when multiple concerns are grouped under a single namespace:
export_default :SuperNet module SuperNet auto_import( HTTP1: './http1', HTTP2: './http2', WebSockets: './websockets' ) end SuperNet::HTTP1 #=> loads the http1 module
Reloading modules¶ ↑
Modules can be reloaded at run-time for easy hot code reloading:
require 'modulation' SQL = import('./sql') ... SQL.__reload!
Another way to reload modules is using Modulation.reload
, which accepts a module or a filename:
require 'filewatcher' FileWatcher.new(['lib']).watch do |fn, event| if(event == :changed) Modulation.reload(fn) end end
When a module is reloaded, its entire content - constants and methods - will be replaced. That means that any code using that module could continue to use it without even being aware it was reloaded, providing its API has not changed.
Reloading of modules with default exports is also possible. Modulation
will
extend the exported value with a #__reload!
method. The value will need to be reassigned:
require 'modulation' settings = import('settings') ... settings = settings.__reload!
Please note that Modulation
does not include a directory watcher that automatically reloads changed modules. This is due to multiple considerations that include the chosen threading model, or the reactor engine in use, or even the chosen solution for watching files (whether it's an external gem or an internal tool).
It is, however, quite trivial to watch files using {directory_watcher
}:
require 'directory_watcher' dw = DirectoryWatcher.new 'lib', glob: '**/*.rb', interval: 2, pre_load: true dw.add_observer do |*events| events.each do |e| next unless e.type == :modified Modulation.reload e.path end end dw.start
Retaining state between reloads¶ ↑
Before a module is reloaded, all of its methods and constants are removed. In some cases, a module might need to retain state across reloads. You can do this by simply using instance variables:
export :value, :inc @counter ||= 0 def value @counter end def incr @counter += 1 end
Care must be taken not to reassign values outside of methods, as this will overwrite any value retained in the instance variable. To assign initial values, use the ||=
operator as in the example above. See also the reload example.
Dependency introspection¶ ↑
Modulation
allows runtime introspection of dependencies between modules. You can interrogate a module's dependencies (i.e. the modules it imports) by calling #__depedencies
:
m1.rb
import ('./m2')
app.rb
m1 = import('./m1') m1.__depedencies #=> [<Module m2>]
You can also iterate over a module's entire dependency tree by using #__traverse_dependencies
:
m1 = import('./m1') m1.__traverse_dependencies { |mod| ... }
To introspect reverse dependencies (modules using a particular module), use #__dependent_modules
:
m1 = import('./m1') m1.__depedencies #=> [<Module m2>] m1.__dependencies.first.__dependent_modules #=> [<Module m1>]
Running Modulation-based applications¶ ↑
Modulation
provides a binary script for running Modulation-based applications. mdl
is a wrapper around Ruby that loads your application's main file as a module, and then runs your application's entry point method. Let's look at a sample application:
app.rb
def greet(name) puts "Hello, #{name}!" end def main print "Enter your name: " name = gets greet(name) end
To run this application, execute mdl app.rb
, or mdl run app.rb
. mdl
will automatically require the modulation
gem and call the application's entry point, #main
.
Packing applications with Modulation
¶ ↑
Note: application packing is at the present time an experimental feature. There might be security concerns for packaging your app, such as leaking filenames from the developer's machine.
Modulation
can also be used to package your entire application into a single portable file that can be copied to another machine and run as is. To package your app, use mdl pack
. This command will perform a dynamic analysis of all the app's dependencies and will put them together into a single Ruby file.
For more information have a look at the app example.
Writing gems using Modulation
¶ ↑
Modulation
can be used to write gems, providing fine-grained control over your gem's public APIs and letting you hide any implementation details. In order to allow loading a gem using either #require
or #import
, code your gem's main file normally, but add require 'modulation/gem'
at the top, and export your gem's main namespace as a default export, e.g.:
require 'modulation/gem' export_default :MyGem module MyGem ... MyClass = import('my_gem/my_class') ... end
Importing gems using Modulation
¶ ↑
Gems written using modulation can also be loaded using #import
. If modulation does not find the module specified by the given relative path, it will attempt to load a gem by the same name. It is also possible to load specific files inside modules by specifying a sub-path:
require 'modulation' MyFeature = import 'my_gem/my_feature'
Note: Since there's not much of a point in
#import
ing gems that do not useModulation
to export symbols,Modulation
will refuse to import any gem that does not depend onModulation
.
Writing modules that patch external classes or modules¶ ↑
It is generally recommended you refrain from causing side effects or patching external code in your modules. When you do have to patch external classes or modules (i.e. core, stdlib, or some third-party code) in your module, it's useful to remember that any module may be eventually reloaded by the application code. This means that any patching done during the loading of your module must be idempotent, i.e. have the same effect when performed multiple times. Take for example the following module code:
module ::Kernel # aliasing #sleep more than once will break your code alias_method :orig_sleep, :sleep def sleep(duration) STDERR.puts "Going to sleep..." orig_sleep(duration) STDERR.puts "Woke up!" end end
Running the above code more than once would cause an infinite loop when calling Kernel#sleep
. In order to prevent this situation, modulation provides the Module#alias_method_once
method, which prevents aliasing the original method more than once:
module ::Kernel # alias_method_once is idempotent alias_method_once :orig_sleep, :sleep def sleep(duration) STDERR.puts "Going to sleep..." orig_sleep(duration) STDERR.puts "Woke up!" end end
Coding style recommendations¶ ↑
-
Import modules into constants, not variables:
ruby Settings = import('./settings')
-
Place your exports at the top of your module, followed by
#require
s, followed by#import
s:
“`ruby export :foo, :bar, :baz
require 'json'
Core = import('./core')
… “`
API Reference¶ ↑
Kernel
¶ ↑
Kernel#auto_import_map(path, options = {})
¶ ↑
Returns a hash mapping keys to corresponding module files inside the given directory path. Modules are loaded automatically upon accessing hash keys.
Kernel#import(path)
¶ ↑
Returns a loaded module identified by the given path. The path can contain tags
Kernel#import_all(path)
¶ ↑
Kernel#import_map(path, options = {})
¶ ↑
Module
¶ ↑
Module#__module_info
¶ ↑
Returns a hash containing information about the module. This currently includes the following entries:
location|Absolute module file path exported_symbols|Array containing all symbols exported by the module
Module#__reload!
¶ ↑
Module#alias_method_once(new_name, old_name)
¶ ↑
Module#auto_import(sym, path)
¶ ↑
Module#export(*symbols)
¶ ↑
Module#export_default(value)
¶ ↑
Module#export_from_receiver(receiver)
¶ ↑
Module#extend_from(path)
¶ ↑
Module#include_from(path, *symbols)
¶ ↑
Module::MODULE
¶ ↑
Modulation
¶ ↑
Modulation.full_backtrace!
¶ ↑
Modulation.reload
¶ ↑
Why you should not use Modulation
¶ ↑
-
Modulation
is not production-ready. -
Modulation
is not thread-safe. -
Modulation
doesn't play well with rdoc/yard. -
Modulation
(probably) doesn't play well withMarshal
. -
Modulation
(probably) doesn't play well with code-analysis tools.