class LogStash::Outputs::Swift::TemporaryFileFactory
Since the file can contains dynamic part, we have to handle a more local structure to allow a nice recovery from a crash.
The local structure will look like this.
<TEMPORARY_PATH>/<UUID>/<prefix>/ls.s3.localhost.%Y-%m-%dT%H.%m.tag_es_fb.part1.txt.gz
Since the UUID should be fairly unique I can destroy the whole path when an upload is complete. I do not have to mess around to check if the other directory have file in it before destroying them.
Constants
- FILE_MODE
- GZIP_ENCODING
- GZIP_EXTENSION
- STRFTIME
- TXT_EXTENSION
Attributes
counter[RW]
current[RW]
encoding[RW]
prefix[RW]
temporary_directory[RW]
Public Class Methods
new(prefix, tags, encoding, temporary_directory)
click to toggle source
# File lib/logstash/outputs/swift/temporary_file_factory.rb, line 29 def initialize(prefix, tags, encoding, temporary_directory) @counter = 0 @prefix = prefix @tags = tags @encoding = encoding @temporary_directory = temporary_directory @lock = Mutex.new rotate! end
Public Instance Methods
rotate!()
click to toggle source
# File lib/logstash/outputs/swift/temporary_file_factory.rb, line 41 def rotate! @lock.synchronize { @current = new_file increment_counter @current } end
Private Instance Methods
current_time()
click to toggle source
# File lib/logstash/outputs/swift/temporary_file_factory.rb, line 62 def current_time Time.now.strftime(STRFTIME) end
extension()
click to toggle source
# File lib/logstash/outputs/swift/temporary_file_factory.rb, line 50 def extension gzip? ? GZIP_EXTENSION : TXT_EXTENSION end
generate_name()
click to toggle source
# File lib/logstash/outputs/swift/temporary_file_factory.rb, line 66 def generate_name filename = "ls.s3.#{SecureRandom.uuid}.#{current_time}" if tags.size > 0 "#{filename}.tag_#{tags.join('.')}.part#{counter}.#{extension}" else "#{filename}.part#{counter}.#{extension}" end end
gzip?()
click to toggle source
# File lib/logstash/outputs/swift/temporary_file_factory.rb, line 54 def gzip? encoding == GZIP_ENCODING end
increment_counter()
click to toggle source
# File lib/logstash/outputs/swift/temporary_file_factory.rb, line 58 def increment_counter @counter += 1 end
new_file()
click to toggle source
# File lib/logstash/outputs/swift/temporary_file_factory.rb, line 76 def new_file uuid = SecureRandom.uuid name = generate_name path = ::File.join(temporary_directory, uuid) key = ::File.join(prefix, name) FileUtils.mkdir_p(::File.join(path, prefix)) io = if gzip? # We have to use this wrapper because we cannot access the size of the # file directly on the gzip writer. IOWrappedGzip.new(::File.open(::File.join(path, key), FILE_MODE)) else ::File.open(::File.join(path, key), FILE_MODE) end TemporaryFile.new(key, io, path) end