class Sidekiq::Metrics::Histogram
Implements space-efficient but statistically useful histogram storage. A precise time histogram stores every time. Instead we break times into a set of known buckets and increment counts of the associated time bucket. Even if we call the histogram a million times, we’ll still only store 26 buckets. NB: needs to be thread-safe or resiliant to races.
To store this data, we use Redis’ BITFIELD command to store unsigned 16-bit counters per bucket per klass per minute. It’s unlikely that most people will be executing more than 1000 job/sec for a full minute of a specific type.
Constants
- BUCKET_INTERVALS
This number represents the maximum milliseconds for this bucket. 20 means all job executions up to 20ms, e.g. if a job takes 280ms, it’ll increment bucket. Note we can track job executions up to about 5.5 minutes. After that, it’s assumed you’re probably not too concerned with its performance.
- FETCH
- HISTOGRAM_TTL
- LABELS
Attributes
Public Class Methods
# File lib/sidekiq/metrics/shared.rb, line 71 def initialize(klass) @klass = klass @buckets = Array.new(BUCKET_INTERVALS.size) { Counter.new } end
Public Instance Methods
# File lib/sidekiq/metrics/shared.rb, line 62 def each buckets.each { |counter| yield counter.value } end
# File lib/sidekiq/metrics/shared.rb, line 84 def fetch(conn, now = Time.now) window = now.utc.strftime("%d-%H:%-M") key = "#{@klass}-#{window}" conn.bitfield_ro(key, *FETCH) end
# File lib/sidekiq/metrics/shared.rb, line 66 def label(idx) LABELS[idx] end
# File lib/sidekiq/metrics/shared.rb, line 90 def persist(conn, now = Time.now) buckets, @buckets = @buckets, [] window = now.utc.strftime("%d-%H:%-M") key = "#{@klass}-#{window}" cmd = [key, "OVERFLOW", "SAT"] buckets.each_with_index do |counter, idx| val = counter.value cmd << "INCRBY" << "u16" << "##{idx}" << val.to_s if val > 0 end conn.bitfield(*cmd) if cmd.size > 3 conn.expire(key, HISTOGRAM_TTL) key end
# File lib/sidekiq/metrics/shared.rb, line 76 def record_time(ms) index_to_use = BUCKET_INTERVALS.each_index do |idx| break idx if ms < BUCKET_INTERVALS[idx] end @buckets[index_to_use].increment end