module Puppet::Pops::Parser::InterpolationSupport
This module is an integral part of the Lexer. It defines interpolation support PERFORMANCE NOTE: There are 4 very similar methods in this module that are designed to be as performant as possible. While it is possible to parameterize them into one common method, the overhead of passing parameters and evaluating conditional logic has a negative impact on performance.
Constants
- PATTERN_VARIABLE
Public Instance Methods
Enqueues lexed tokens until either end of input, or the given brace_count is reached
# File lib/puppet/pops/parser/interpolation_support.rb 188 def enqueue_until brace_count 189 scn = @scanner 190 ctx = @lexing_context 191 queue = @token_queue 192 queue_base = @token_queue[0] 193 194 scn.skip(self.class::PATTERN_WS) 195 queue_size = queue.size 196 until scn.eos? do 197 token = lex_token 198 if token 199 if token.equal?(queue_base) 200 # A nested #interpolate_dq call shifted the queue_base token from the @token_queue. It must 201 # be put back since it is intended for the top level #interpolate_dq call only. 202 queue.insert(0, token) 203 next # all relevant tokens are already on the queue 204 end 205 token_name = token[0] 206 ctx[:after] = token_name 207 if token_name == :RBRACE && ctx[:brace_count] == brace_count 208 qlength = queue.size - queue_size 209 if qlength == 1 210 # Single token is subject to replacement 211 queue[-1] = transform_to_variable(queue[-1]) 212 elsif qlength > 1 && [:DOT, :LBRACK].include?(queue[queue_size + 1][0]) 213 # A first word, number of name token followed by '[' or '.' is subject to replacement 214 # But not for other operators such as ?, +, - etc. where user must use a $ before the name 215 # to get a variable 216 queue[queue_size] = transform_to_variable(queue[queue_size]) 217 end 218 return 219 end 220 queue << token 221 else 222 scn.skip(self.class::PATTERN_WS) 223 end 224 end 225 end
This is the starting point for a double quoted string with possible interpolation The structure mimics that of the grammar. The logic is explicit (where the former implementation used parameters/structures) given to a generic handler. (This is both easier to understand and faster).
# File lib/puppet/pops/parser/interpolation_support.rb 17 def interpolate_dq 18 scn = @scanner 19 ctx = @lexing_context 20 before = scn.pos 21 # skip the leading " by doing a scan since the slurp_dqstring uses last matched when there is an error 22 scn.scan(/"/) 23 value,terminator = slurp_dqstring() 24 text = value 25 after = scn.pos 26 loop do 27 case terminator 28 when '"' 29 # simple case, there was no interpolation, return directly 30 return emit_completed([:STRING, text, scn.pos-before], before) 31 when '${' 32 count = ctx[:brace_count] 33 ctx[:brace_count] += 1 34 # The ${ terminator is counted towards the string part 35 enqueue_completed([:DQPRE, text, scn.pos-before], before) 36 # Lex expression tokens until a closing (balanced) brace count is reached 37 enqueue_until count 38 break 39 when '$' 40 varname = scn.scan(PATTERN_VARIABLE) 41 if varname 42 # The $ is counted towards the variable 43 enqueue_completed([:DQPRE, text, after-before-1], before) 44 enqueue_completed([:VARIABLE, varname, scn.pos - after + 1], after - 1) 45 break 46 else 47 # false $ variable start 48 text += terminator 49 value,terminator = slurp_dqstring() 50 text += value 51 after = scn.pos 52 end 53 end 54 end 55 interpolate_tail_dq 56 # return the first enqueued token and shift the queue 57 @token_queue.shift 58 end
# File lib/puppet/pops/parser/interpolation_support.rb 60 def interpolate_tail_dq 61 scn = @scanner 62 ctx = @lexing_context 63 before = scn.pos 64 value,terminator = slurp_dqstring 65 text = value 66 after = scn.pos 67 loop do 68 case terminator 69 when '"' 70 # simple case, there was no further interpolation, return directly 71 enqueue_completed([:DQPOST, text, scn.pos-before], before) 72 return 73 when '${' 74 count = ctx[:brace_count] 75 ctx[:brace_count] += 1 76 # The ${ terminator is counted towards the string part 77 enqueue_completed([:DQMID, text, scn.pos-before], before) 78 # Lex expression tokens until a closing (balanced) brace count is reached 79 enqueue_until count 80 break 81 when '$' 82 varname = scn.scan(PATTERN_VARIABLE) 83 if varname 84 # The $ is counted towards the variable 85 enqueue_completed([:DQMID, text, after-before-1], before) 86 enqueue_completed([:VARIABLE, varname, scn.pos - after + 1], after - 1) 87 break 88 else 89 # false $ variable start 90 text += terminator 91 value,terminator = slurp_dqstring 92 text += value 93 after = scn.pos 94 end 95 end 96 end 97 interpolate_tail_dq 98 end
# File lib/puppet/pops/parser/interpolation_support.rb 146 def interpolate_tail_uq 147 scn = @scanner 148 ctx = @lexing_context 149 before = scn.pos 150 value,terminator = slurp_uqstring 151 text = value 152 after = scn.pos 153 loop do 154 case terminator 155 when '' 156 # simple case, there was no further interpolation, return directly 157 enqueue_completed([:DQPOST, text, scn.pos-before], before) 158 return 159 when '${' 160 count = ctx[:brace_count] 161 ctx[:brace_count] += 1 162 # The ${ terminator is counted towards the string part 163 enqueue_completed([:DQMID, text, scn.pos-before], before) 164 # Lex expression tokens until a closing (balanced) brace count is reached 165 enqueue_until count 166 break 167 when '$' 168 varname = scn.scan(PATTERN_VARIABLE) 169 if varname 170 # The $ is counted towards the variable 171 enqueue_completed([:DQMID, text, after-before-1], before) 172 enqueue_completed([:VARIABLE, varname, scn.pos - after + 1], after - 1) 173 break 174 else 175 # false $ variable start 176 text += terminator 177 value,terminator = slurp_uqstring 178 text += value 179 after = scn.pos 180 end 181 end 182 end 183 interpolate_tail_uq 184 end
This is the starting point for a un-quoted string with possible interpolation The logic is explicit (where the former implementation used parameters/strucures) given to a generic handler. (This is both easier to understand and faster).
# File lib/puppet/pops/parser/interpolation_support.rb 105 def interpolate_uq 106 scn = @scanner 107 ctx = @lexing_context 108 before = scn.pos 109 value,terminator = slurp_uqstring() 110 text = value 111 after = scn.pos 112 loop do 113 case terminator 114 when '' 115 # simple case, there was no interpolation, return directly 116 enqueue_completed([:STRING, text, scn.pos-before], before) 117 return 118 when '${' 119 count = ctx[:brace_count] 120 ctx[:brace_count] += 1 121 # The ${ terminator is counted towards the string part 122 enqueue_completed([:DQPRE, text, scn.pos-before], before) 123 # Lex expression tokens until a closing (balanced) brace count is reached 124 enqueue_until count 125 break 126 when '$' 127 varname = scn.scan(PATTERN_VARIABLE) 128 if varname 129 # The $ is counted towards the variable 130 enqueue_completed([:DQPRE, text, after-before-1], before) 131 enqueue_completed([:VARIABLE, varname, scn.pos - after + 1], after - 1) 132 break 133 else 134 # false $ variable start 135 text += terminator 136 value,terminator = slurp_uqstring() 137 text += value 138 after = scn.pos 139 end 140 end 141 end 142 interpolate_tail_uq 143 nil 144 end
Interpolates unquoted string and transfers the result to the given lexer (This is used when a second lexer instance is used to lex a substring)
# File lib/puppet/pops/parser/interpolation_support.rb 241 def interpolate_uq_to(lexer) 242 interpolate_uq 243 queue = @token_queue 244 until queue.empty? do 245 lexer.enqueue(queue.shift) 246 end 247 end
# File lib/puppet/pops/parser/interpolation_support.rb 227 def transform_to_variable(token) 228 token_name = token[0] 229 if [:NUMBER, :NAME, :WORD].include?(token_name) || self.class::KEYWORD_NAMES[token_name] || @taskm_keywords[token_name] 230 t = token[1] 231 ta = t.token_array 232 [:VARIABLE, self.class::TokenValue.new([:VARIABLE, ta[1], ta[2]], t.offset, t.locator)] 233 else 234 token 235 end 236 end