class TSPScraper::Converter
Public Class Methods
raw_csv_to_hash(raw_csv)
click to toggle source
TSP CSV has a janky format:
- Each field in a row, except for the first field, is prepended by a space
For this reason, we manually clean up the CSV instead of using the Ruby CSV library, especially since we know TSP does not use quotation marks in their CSV fields… at least for now.
# File lib/tsp_scraper/converter.rb, line 7 def self.raw_csv_to_hash(raw_csv) raw_data = raw_csv.lines.map { |row| row.strip.split(',').map { |field| field.strip } } headers = raw_data.shift data = [] raw_data.each do |d| hash = { date: Date.parse(d.first), funds: {} } headers.each_with_index do |header, index| next if index == 0 hash[:funds][header] = BigDecimal(d[index]) unless d[index].empty? end data.push(hash) end data end