Cassandra plugin for Fluentd¶ ↑
Cassandra output plugin for Fluentd.
Implemented using the Datastax Ruby Driver for Apache Cassandra gem and targets CQL3 and Cassandra 1.2 - 3.x
Installation¶ ↑
via RubyGems : rubygems.org/gems/fluentd-plugin-cassandra-cqlfunction
fluent-gem install fluentd-plugin-cassandra-cqlfunction
Fluentd.conf Configuration¶ ↑
Filter Plugin:¶ ↑
<filter **> @type cassandra_selector # fluent filter plugin host 127.0.0.1,127.0.0.2 # default => 127.0.0.1 port 9042 # default => 9092 username # default => null password # default => null connect_timeout # default => 10 (sec) keyspace ex # cassandra keyspace tablename tb_ex # cassandra table field fieldA,fieldB # select field normal field_json fieldC # select field json string on base(ex fieldC='{"a":"1"}') where_condition fieldA='xxx' and fieldB=':keyfrominput;' # keyfrominput (fieldB=':a;' --> fieldB='1') </filter>
ex Filter ::¶ ↑
input -> {'a':'1'} output 1 rec -> {'a':'1', 'fieldA':'xxx', 'fieldB':'yyy'} output 2+ rec -> {'a':'1', 'data_cassandra': [{fieldA':'xxx', 'fieldB':'yyy'},{fieldA':'aaa', 'fieldB':'bbb'}]}
Output Plugin:¶ ↑
<match **> @type cassandra_upsert host 127.0.0.1,127.0.0.2 port 9042 username # default => null password # default => null connect_timeout # default => 10 (sec) keyspace ex tablename tb_ex case_insert_value fieldPk='xxx', fieldB=':keyfrominput;' #For insert case case_update_value fieldA='xxx', fieldB=':keyfrominput;' #For update case where_condition_upd fieldPk='xxx' or fieldPk=':keyfrominput;' </match> <match **> @type cassandra_insert host 127.0.0.1,127.0.0.2 port 9042 username # default => null password # default => null connect_timeout # default => 10 (sec) keyspace ex tablename tb_ex insert_value fieldPk='xxx', fieldB=':keyfrominput;' </match> <match **> @type cassandra_update host 127.0.0.1,127.0.0.2 port 9092 username # default => null password # default => null connect_timeout # default => 10 (sec) keyspace ex tablename tb_ex update_value fieldPk='xxx', fieldB=':keyfrominput;' where_condition_upd fieldPk='xxx' or fieldPk=':keyfrominput;' </match>