Cassandra plugin for Fluentd¶ ↑
Cassandra output plugin for Fluentd.
Implemented using the Datastax Ruby Driver for Apache Cassandra gem and targets CQL3 and Cassandra 1.2 - 3.x
Warning¶ ↑
This project is in an alpha state, so configuration params could be changed without changing of major version.
Be careful before updating.
Installation¶ ↑
via RubyGems
fluent-gem install fluent-plugin-cassandra-driver td-agent-gem install fluent-plugin-cassandra-driver
Quick Start¶ ↑
Cassandra Configuration¶ ↑
# Create keyspace (via CQL) CREATE KEYSPACE metrics WITH strategy_class='org.apache.cassandra.locator.SimpleStrategy' AND strategy_options:replication_factor=1; # Create table (column family) CREATE TABLE logs (id varchar, timestamp timestamp, json text, PRIMARY KEY (id, timestamp)) WITH CLUSTERING ORDER BY (timestamp DESC);
Fluentd.conf Configuration¶ ↑
<match cassandra.**> type cassandra_driver # fluent output plugin file name (sans fluent_plugin_ prefix) hosts 127.0.0.1 # comma delimited string of hosts keyspace metrics # cassandra keyspace column_family logs # cassandra column family ttl 60 # cassandra ttl (optional, default is 0) schema # cassandra column family schema (see example below) pop_data_keys # pop values from the fluentd hash when storing it as json (optional, default is true) json_column json # column where store all remaining data from fluentd (optional) </match>
Schema example¶ ↑
# hash of hashes :column_damily_key => {:fluentd_record_key => :type_from_list} # or :column_damily_key => :type_from_list # then :fluentd_record_key will be the same as :column_damily_key '{:id => {:ident => nil}, :timestamp => {:timestamp => :time}}'
Available mappings: * :integer * :string * :timeuuid * :time
All nil types will be recognized as string.
Tests¶ ↑
TODO