class Racecar::MessageDeliveryError

MessageDeliveryError wraps an Rdkafka error and tries to give specific hints on how to debug or resolve the error within the Racecar context.

Attributes

rdkafka_error[R]

Public Class Methods

new(rdkafka_error, delivery_handle) click to toggle source
# File lib/racecar/message_delivery_error.rb, line 18
def initialize(rdkafka_error, delivery_handle)
  raise rdkafka_error unless rdkafka_error.is_a?(Rdkafka::RdkafkaError)

  @rdkafka_error = rdkafka_error
  @delivery_handle = delivery_handle
end
partition_from_delivery_handle(delivery_handle) click to toggle source

partition_from_delivery_handle takes an rdkafka delivery handle and returns a human readable version of the partition. It handles the case where the partition is unknown.

# File lib/racecar/message_delivery_error.rb, line 11
def self.partition_from_delivery_handle(delivery_handle)
  partition = delivery_handle&.create_result&.partition
  # -1 is rdkafka-ruby's default value, which gets eventually set by librdkafka
  return "no yet known" if partition.nil? || partition == -1
  partition.to_s
end

Public Instance Methods

code() click to toggle source
# File lib/racecar/message_delivery_error.rb, line 27
def code
  @rdkafka_error.code
end
to_s() click to toggle source
# File lib/racecar/message_delivery_error.rb, line 31
    def to_s
      msg = <<~EOM
        Message delivery finally failed:
        #{@rdkafka_error.to_s}

        #{explain}
      EOM
    end

Private Instance Methods

explain() click to toggle source
# File lib/racecar/message_delivery_error.rb, line 42
    def explain
      case @rdkafka_error.code
      when :msg_timed_out # -192
        <<~EOM
          Could not deliver message within Racecar.config.message_timeout.

          This can happen for various reasons, but most commonly because the connection to the broker is interrupted or there is no leader available. Check the broker's logs or the network for more insight.

          Upstream documentation:
          https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md#error-local-time-out
        EOM

      when :msg_size_too_large # 10
        <<~EOM
          Could not deliver message, since it is bigger than either the broker's or Racecar's maximum message size.

          The broker's config option on the topic is called "max.message.bytes" and the broker wide default is "message.max.bytes". The client's is "message.max.bytes". Take extra care to distinguish this from similarly named properties for receiving/consuming messages (i.e. Racecar.config.max_bytes is NOT related).

          Racecar's limit is currently not configurable and uses librdkafka's default of 1 MB (10³ bytes). As of writing, librdkafka will send at least one message regardless of this limit. It is therefore very likely you're hitting the broker's limit and not Racecar's/librdkafka's.

          Upstream documentation:
          broker per topic: https://docs.confluent.io/platform/current/installation/configuration/topic-configs.html#topicconfigs_max.message.bytes
          broker default:   https://docs.confluent.io/platform/current/installation/configuration/broker-configs.html#brokerconfigs_message.max.bytes
          client:           https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md
        EOM

      when :unknown_topic_or_part # 3
        partition = self.class.partition_from_delivery_handle(@delivery_handle)

        <<~EOM
          Could not deliver message, since the targeted topic or partition (#{partition}) does not exist.

          Check that there are no typos, or that the broker's "auto.create.topics.enable" is enabled. For freshly created topics with auto create enabled, this may appear in the beginning (race condition on creation and publishing).

          Upstream documentation:
          broker setting: https://docs.confluent.io/platform/current/installation/configuration/broker-configs.html#brokerconfigs_auto.create.topics.enable
          client:         https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md#topic-metadata-propagation-for-newly-created-topics
                          https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md#topic-auto-creation
        EOM

      when :record_list_too_large # 18
        <<~EOM
          Tried to deliver more messages in a batch than the broker's segment size.

          Either increase the broker's "log.segment.bytes", or decrease any of the client's related settings "batch.num.messages", "batch.size" or "message.max.bytes". None of these are configurable through Racecar yet, as the defaults should be sufficient and sane.

          Upstream documentation:
          broker: https://docs.confluent.io/platform/current/installation/configuration/broker-configs.html#brokerconfigs_log.segment.bytes
          client: https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md
        EOM

      when :topic_authorization_failed # 29
        <<~EOM
          Failed to deliver message because of insufficient authorization to write into the topic.

          Double check that it is not a race condition on topic creation. If it isn't, verify the ACLs are correct.

          Upstream documentation:
          https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md#unknown-or-unauthorized-topics
        EOM

      else
        <<~EOM
          No specific information is available for this error. Consider adding it to Racecar. You can find generally helpful information in the upstream documentation:
          https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md
          https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md
        EOM
      end
    end