module SemanticLogger
@formatter:off
Send log messages to Bugsnag
Example:
SemanticLogger.add_appender(appender: :bugsnag)
Forward all log messages to Elasticsearch.
Example:
SemanticLogger.add_appender( appender: :elasticsearch, url: 'http://localhost:9200' )
Forward all log messages to Elasticsearch one at a time via a HTTP post.
Note:
-
Other than in very low volume environments it is recommended to rather use the Elasticsearch appender, since it supports bulk logging.
Example:
SemanticLogger.add_appender( appender: :elasticsearch_http, url: 'http://localhost:9200' )
File appender
Writes log messages to a file or open iostream
Forward log entries to a Graylog server.
Example:
SemanticLogger.add_appender( appender: :graylog, url: 'udp://localhost:12201' )
Notes:
-
trace is not supported by Graylog, so trace level logging will appear as debug in Graylog.
In the Graylog Web UI search screen, it is recommended to include the following fields:
`duration`, `level`, `message`, `metric`, `name`, `tags
Send log messages to honeybadger
Example:
SemanticLogger.add_appender(appender: :honeybadger)
Log
to any HTTP(S) server that accepts log messages in JSON form
Features:
-
JSON Formatted messages.
-
Uses a persistent http connection, if the server supports it.
-
SSL encryption (https).
Example:
SemanticLogger.add_appender( appender: :http, url: 'http://localhost:8088/path' )
Forward all log messages to Apache Kafka.
Example:
SemanticLogger.add_appender( appender: :kafka, # At least one of these nodes must be available: seed_brokers: ["kafka1:9092", "kafka2:9092"], # Set an optional client id in order to identify the client to Kafka: client_id: "my-application", )
Send log messages to NewRelic
The :error and :fatal log entries will show up under “Applications” > “Application Name” > “Events” > “Errors” in New Relic.
Example:
SemanticLogger.add_appender(appender: :new_relic)
Forward all log messages to RabbitMQ.
Example:
SemanticLogger.add_appender( appender: :rabbitmq, # Name of the queue in RabbitMQ where to publish the logs. This queue will be bound to "amqp.direct" exchange. queue: 'semantic_logger', # This host will be used for RabbitMQ connection. # NOTE this is different than :host option which is used by the logger directly. rabbitmq_host: '127.0.0.1', # RabbitMQ credentials username: 'my-username', password: 'my-secrect-pass', # All other options accepted by Bunny.new call vhost: 'production', )
Send log messages to sentry
Example:
SemanticLogger.add_appender(appender: :sentry)
Splunk log appender.
Use the official splunk gem to log messages to Splunk.
Example
SemanticLogger.add_appender( appender: :splunk, username: 'username', password: 'password', host: 'localhost', port: 8089, scheme: :https, index: 'main' )
Splunk log appender using the Splunk HTTP(S) listener.
Use the newer, faster and more complete JSON over HTTP interface for Splunk.
To configure Splunk to receive log messages via this appender:
http://dev.splunk.com/view/event-collector/SP-CAAAE7F
Example
SemanticLogger.add_appender( appender: :splunk_http, url: 'http://localhost:8080', token: '70CA900C-3D7E-42A4-9C79-7975D1C422A8' )
Send log messages to local syslog, or remote syslog servers over TCP or UDP.
Example:
# Log to a local Syslog daemon SemanticLogger.add_appender(appender: :syslog)
Example:
# Log to a remote Syslog server over TCP: SemanticLogger.add_appender( appender: :syslog, url: 'tcp://myloghost:514' )
Example:
# Log to a remote Syslog server over UDP: SemanticLogger.add_appender( appender: :syslog, url: 'udp://myloghost:514' )
Example:
# Log to a remote Syslog server using the CEE format over TCP: SemanticLogger.add_appender( appender: :syslog, url: 'tcp://myloghost:514' )
Send log messages to any standard Ruby logging class.
Forwards logging call to loggers such as Logger, log4r, etc.
Base
logger
Abstract base class for loggers Implements common behavior such as log level, default text formatter etc
Logger
class variable mix-in
Lazy initialize a logger class variable with instance accessor By including this mix-in into any class it will define a class level logger and also make it accessible via instance methods
Example:
require 'semantic_logger' SemanticLogger.default_level = :debug SemanticLogger.add_appender(io: $stdout, formatter: :color) class ExternalSupplier # Create class and instance logger methods include SemanticLogger::Loggable def call_supplier(amount, name) logger.debug "Calculating with amount", { amount: amount, name: name } # Measure and log on completion how long the call took to the external supplier logger.measure_info "Calling external interface" do # Code to call the external supplier ... end end end
Notes:
-
To forcibly replace Rails or any other existing logging methods use `prepend` instead of `include`. For example:
ExternalSupplier.prepend SemanticLogger::Loggable
Send Metrics to NewRelic
The :error and :fatal log entries will show up under “Applications” > “Application Name” > “Events” > “Errors” in New Relic.
Example:
SemanticLogger.add_appender(metric: :new_relic)
Forward application metrics to SignalFx.
Example:
SemanticLogger.add_appender( metric: :signalfx, token: 'SIGNALFX_ORG_ACCESS_TOKEN' )
Abstract Subscriber
Abstract base class for all appenders.
Constants
- LEVELS
Logging levels in order of most detailed to most severe
- VERSION
Public Class Methods
Return a logger for the supplied class or class_name
# File lib/semantic_logger/semantic_logger.rb, line 9 def self.[](klass) Logger.new(klass) end
Add a new logging appender as a new destination for all log messages emitted from Semantic Logger
Appenders
will be written to in the order that they are added
If a block is supplied then it will be used to customize the format of the messages sent to that appender. See SemanticLogger::Logger.new
for more information on custom formatters
Parameters
file_name: [String] File name to write log messages to. Or, io: [IO] An IO Stream to log to. For example $stdout, $stderr, etc. Or, appender: [Symbol|SemanticLogger::Subscriber] A symbol identifying the appender to create. For example: :bugsnag, :elasticsearch, :graylog, :http, :mongodb, :new_relic, :splunk_http, :syslog, :wrapper Or, An instance of an appender derived from SemanticLogger::Subscriber For example: SemanticLogger::Appender::Http.new(url: 'http://localhost:8088/path') Or, logger: [Logger|Log4r] An instance of a Logger or a Log4r logger. level: [:trace | :debug | :info | :warn | :error | :fatal] Override the log level for this appender. Default: SemanticLogger.default_level formatter: [Symbol|Object|Proc] Any of the following symbol values: :default, :color, :json Or, An instance of a class that implements #call Or, A Proc to be used to format the output from this appender Default: :default filter: [Regexp|Proc] RegExp: Only include log messages where the class name matches the supplied. regular expression. All other messages will be ignored. Proc: Only include log messages where the supplied Proc returns true The Proc must return true or false.
Examples:
# Send all logging output to Standard Out (Screen) SemanticLogger.add_appender(io: $stdout) # Send all logging output to a file SemanticLogger.add_appender(file_name: 'logfile.log') # Send all logging output to a file and only :info and above to standard output SemanticLogger.add_appender(file_name: 'logfile.log') SemanticLogger.add_appender(io: $stdout, level: :info)
# Send Semantic logging output to an existing logger require 'logger' require 'semantic_logger' # Built-in Ruby logger log = Logger.new($stdout) log.level = Logger::DEBUG SemanticLogger.default_level = :debug SemanticLogger.add_appender(logger: log) logger = SemanticLogger['Example'] logger.info "Hello World" logger.debug("Login time", user: 'Joe', duration: 100, ip_address: '127.0.0.1')
# File lib/semantic_logger/semantic_logger.rb, line 166 def self.add_appender(**args, &block) appender = Logger.processor.appenders.add(**args, &block) # Start appender thread if it is not already running Logger.processor.start appender end
Add signal handlers for Semantic Logger
Two signal handlers will be registered by default:
-
Changing the log_level:
The log level can be changed without restarting the process by sending the log_level_signal, which by default is 'USR2' When the log_level_signal is raised on this process, the global default log level rotates through the following log levels in the following order, starting from the current global default level: :warn, :info, :debug, :trace If the current level is :trace it wraps around back to :warn
-
Logging a Ruby thread dump
When the signal is raised on this process, Semantic Logger will write the list of threads to the log file, along with their back-traces when available For JRuby users this thread dump differs form the standard QUIT triggered Java thread dump which includes system threads and Java stack traces. It is recommended to name any threads you create in the application, by calling the following from within the thread itself: Thread.current.name = 'My Worker'
Also adds JRuby
Garbage collection logging so that any garbage collections that exceed the time threshold will be logged. Default: 100 ms Currently only supported when running JRuby
Note:
To only register one of the signal handlers, set the other to nil Set gc_log_microseconds to nil to not enable JRuby Garbage collections
# File lib/semantic_logger/semantic_logger.rb, line 279 def self.add_signal_handler(log_level_signal = "USR2", thread_dump_signal = "TTIN", gc_log_microseconds = 100_000) if log_level_signal Signal.trap(log_level_signal) do index = default_level == :trace ? LEVELS.find_index(:error) : LEVELS.find_index(default_level) new_level = LEVELS[index - 1] self["SemanticLogger"].warn "Changed global default log level to #{new_level.inspect}" self.default_level = new_level end end if thread_dump_signal Signal.trap(thread_dump_signal) do logger = SemanticLogger["Thread Dump"] Thread.list.each do |thread| # MRI re-uses the main thread for signals, JRuby uses `SIGTTIN handler` thread. next if defined?(JRuby) && (thread == Thread.current) logger.backtrace(thread: thread) end end end if gc_log_microseconds && defined?(JRuby) listener = SemanticLogger::JRuby::GarbageCollectionLogger.new(gc_log_microseconds) Java::JavaLangManagement::ManagementFactory.getGarbageCollectorMXBeans.each do |gcbean| gcbean.add_notification_listener(listener, nil, nil) end end true end
Returns [SemanticLogger::Subscriber] a copy of the list of active appenders for debugging etc. Use SemanticLogger.add_appender
and SemanticLogger.remove_appender
to manipulate the active appenders list
# File lib/semantic_logger/semantic_logger.rb, line 191 def self.appenders Logger.processor.appenders.to_a end
Returns [String] name of this application for logging purposes Note: Not all appenders use `application`
# File lib/semantic_logger/semantic_logger.rb, line 65 def self.application @application end
Override the default application
# File lib/semantic_logger/semantic_logger.rb, line 70 def self.application=(application) @application = application end
Returns the current backtrace level
# File lib/semantic_logger/semantic_logger.rb, line 42 def self.backtrace_level @backtrace_level end
Sets the level at which backtraces should be captured for every log message.
By enabling backtrace capture the filename and line number of where message was logged can be written to the log file. Additionally, the backtrace can be forwarded to error management services such as Bugsnag.
Warning:
Capturing backtraces is very expensive and should not be done all the time. It is recommended to run it at :error level in production.
# File lib/semantic_logger/semantic_logger.rb, line 35 def self.backtrace_level=(level) @backtrace_level = level # For performance reasons pre-calculate the level index @backtrace_level_index = level.nil? ? 65_535 : Levels.index(level) end
Returns the current backtrace level index For internal use only
# File lib/semantic_logger/semantic_logger.rb, line 48 def self.backtrace_level_index @backtrace_level_index end
Clear out all previously registered appenders
# File lib/semantic_logger/semantic_logger.rb, line 183 def self.clear_appenders! Logger.processor.close end
Close all appenders and flush any outstanding messages.
# File lib/semantic_logger/semantic_logger.rb, line 202 def self.close Logger.processor.close end
Returns the global default log level
# File lib/semantic_logger/semantic_logger.rb, line 21 def self.default_level @default_level end
Sets the global default log level
# File lib/semantic_logger/semantic_logger.rb, line 14 def self.default_level=(level) @default_level = level # For performance reasons pre-calculate the level index @default_level_index = Levels.index(level) end
# File lib/semantic_logger/semantic_logger.rb, line 496 def self.default_level_index Thread.current[:semantic_logger_silence] || @default_level_index end
Returns [String] name of this environment for logging purposes Note: Not all appenders use `environment`
# File lib/semantic_logger/semantic_logger.rb, line 76 def self.environment @environment end
Override the default environment
# File lib/semantic_logger/semantic_logger.rb, line 81 def self.environment=(environment) @environment = environment end
If the tag being supplied is definitely a string then this fast tag api can be used for short lived tags
# File lib/semantic_logger/semantic_logger.rb, line 313 def self.fast_tag(tag) return yield if tag.nil? || tag == "" t = Thread.current[:semantic_logger_tags] ||= [] begin t << tag yield ensure t.pop end end
Flush all queued log entries disk, database, etc.
All queued log messages are written and then each appender is flushed in turn.
# File lib/semantic_logger/semantic_logger.rb, line 197 def self.flush Logger.processor.flush end
Returns [String] name of this host for logging purposes Note: Not all appenders use `host`
# File lib/semantic_logger/semantic_logger.rb, line 54 def self.host @host ||= Socket.gethostname.force_encoding("UTF-8") end
Override the default host name
# File lib/semantic_logger/semantic_logger.rb, line 59 def self.host=(host) @host = host end
Returns the check_interval which is the number of messages between checks to determine if the appender thread is falling behind.
# File lib/semantic_logger/semantic_logger.rb, line 480 def self.lag_check_interval Logger.processor.lag_check_interval end
Set the check_interval which is the number of messages between checks to determine if the appender thread is falling behind.
# File lib/semantic_logger/semantic_logger.rb, line 486 def self.lag_check_interval=(lag_check_interval) Logger.processor.lag_check_interval = lag_check_interval end
Returns the amount of time in seconds to determine if the appender thread is falling behind.
# File lib/semantic_logger/semantic_logger.rb, line 492 def self.lag_threshold_s Logger.processor.lag_threshold_s end
:nodoc
# File lib/semantic_logger/semantic_logger.rb, line 389 def self.named_tagged(hash) return yield if hash.nil? || hash.empty? raise(ArgumentError, "#named_tagged only accepts named parameters (Hash)") unless hash.is_a?(Hash) begin push_named_tags(hash) yield ensure pop_named_tags end end
Supply a callback to be called whenever a log entry is created. Useful for capturing appender specific context information.
Parameters object: [Object | Proc] [Proc] the block to call. [Object] any object on which to call #call.
Example:
SemanticLogger.on_log do |log| log.set_context(:honeybadger, Honeybadger.get_context) end
Example:
module CaptureContext def call(log) log.set_context(:honeybadger, Honeybadger.get_context) end end SemanticLogger.on_log(CaptureContext)
Note:
-
This callback is called within the thread of the application making the logging call.
-
If these callbacks are slow they will slow down the application.
# File lib/semantic_logger/semantic_logger.rb, line 240 def self.on_log(object = nil, &block) Logger.subscribe(object, &block) end
Returns [Integer] the number of log entries waiting to be written to the appenders.
When this number grows it is because the logging appender thread is not able to write to the appenders fast enough. Either reduce the amount of logging, increase the log level, reduce the number of appenders, or look into speeding up the appenders themselves
# File lib/semantic_logger/semantic_logger.rb, line 474 def self.queue_size Logger.processor.queue.size end
Remove an existing appender Currently only supports appender instances
# File lib/semantic_logger/semantic_logger.rb, line 175 def self.remove_appender(appender) return unless appender Logger.processor.appenders.delete(appender) appender.close end
After forking an active process call SemanticLogger.reopen
to re-open any open file handles etc to resources.
Note:
Not all appender's implement reopen. Check the code for each appender you are using before relying on this behavior.
# File lib/semantic_logger/semantic_logger.rb, line 212 def self.reopen Logger.processor.reopen end
Silence noisy log levels by changing the default_level
within the block
This setting is thread-safe and only applies to the current thread
Any threads spawned within the block will not be affected by this setting
silence can be used to both raise and lower the log level within the supplied block.
Example:
# Perform trace level logging within the block when the default is higher SemanticLogger.default_level = :info logger.debug 'this will _not_ be logged' SemanticLogger.silence(:trace) do logger.debug "this will be logged" end
Parameters
new_level The new log level to apply within the block Default: :error
Example:
# Silence all logging for this thread below :error level SemanticLogger.silence do logger.info "this will _not_ be logged" logger.warn "this neither" logger.error "but errors will be logged" end
Note:
#silence does not affect any loggers which have had their log level set explicitly. I.e. That do not rely on the global default level
# File lib/semantic_logger/semantic_logger.rb, line 460 def self.silence(new_level = :error) current_index = Thread.current[:semantic_logger_silence] Thread.current[:semantic_logger_silence] = Levels.index(new_level) yield ensure Thread.current[:semantic_logger_silence] = current_index end
Run Semantic Logger
in Synchronous mode.
I.e. Instead of logging messages in a separate thread for better performance, log them using the current thread.
# File lib/semantic_logger/semantic_logger.rb, line 504 def self.sync! Logger.sync! end
Running in synchronous mode?
# File lib/semantic_logger/semantic_logger.rb, line 509 def self.sync? Logger.sync? end
Add the tags or named tags to the list of tags to log for this thread whilst the supplied block is active.
Returns result of block.
Tagged example:
SemanticLogger.tagged(12345, 'jack') do logger.debug('Hello World') end
Named Tags (Hash) example:
SemanticLogger.tagged(tracking_number: 12345) do logger.debug('Hello World') end
Notes:
-
Tags should be a list without any empty values, or contain any array.
-
`logger.tagged` is a slower api that will flatten the example below: `logger.tagged([['first', nil], nil, ['more'], 'other'])`
to the equivalent of:
`logger.tagged('first', 'more', 'other')`
-
# File lib/semantic_logger/semantic_logger.rb, line 345 def self.tagged(*tags, &block) return yield if tags.empty? # Allow named tags to be passed into the logger if tags.size == 1 tag = tags[0] return tag.is_a?(Hash) ? named_tagged(tag, &block) : fast_tag(tag, &block) end begin push_tags(*tags) yield ensure pop_tags(tags.size) end end