class Catalog
This class models a node catalog. It is the thing meant to be passed from server to client, and it contains all of the information in the catalog, including the resources and the relationships between them.
@api public
Attributes
@return [Integer] catalog format version number. This value is constant
for a given version of Puppet; it is incremented when a new release of Puppet changes the API for the various objects that make up the catalog.
The UUID of the catalog
Some metadata to help us compile and generally respond to the current state.
The id of the code input to the compiler.
A String representing the environment for this catalog
The actual environment instance that was used during compilation
Whether this catalog was retrieved from the cache, which affects whether it is written back out again.
Whether this is a host catalog, which behaves very differently. In particular, reports are sent, graphs are made, and state is stored in the state database. If this is set incorrectly, then you often end up in infinite loops, because catalogs are used to make things that the host catalog needs.
Inlined file metadata for non-recursive find A hash of title => metadata
The host name this is a catalog for.
Inlined file metadata for recursive search A hash of title => { source => [metadata, …] }
How long this catalog took to retrieve. Used for reporting stats.
Some metadata to help us compile and generally respond to the current state.
The catalog version. Used for testing whether a catalog is up to date.
Public Class Methods
# File lib/puppet/resource/catalog.rb 407 def self.from_data_hash(data) 408 result = new(data['name'], Puppet::Node::Environment::NONE) 409 410 result.tag(*data['tags']) if data['tags'] 411 result.version = data['version'] if data['version'] 412 result.code_id = data['code_id'] if data['code_id'] 413 result.catalog_uuid = data['catalog_uuid'] if data['catalog_uuid'] 414 result.catalog_format = data['catalog_format'] || 0 415 416 environment = data['environment'] 417 if environment 418 result.environment = environment 419 result.environment_instance = Puppet::Node::Environment.remote(environment.to_sym) 420 end 421 422 result.add_resource( 423 *data['resources'].collect do |res| 424 Puppet::Resource.from_data_hash(res) 425 end 426 ) if data['resources'] 427 428 if data['edges'] 429 data['edges'].each do |edge_hash| 430 edge = Puppet::Relationship.from_data_hash(edge_hash) 431 source = result.resource(edge.source) 432 unless source 433 raise ArgumentError, _("Could not intern from data: Could not find relationship source %{source} for %{target}") % 434 { source: edge.source.inspect, target: edge.target.to_s } 435 end 436 edge.source = source 437 438 target = result.resource(edge.target) 439 unless target 440 raise ArgumentError, _("Could not intern from data: Could not find relationship target %{target} for %{source}") % 441 { target: edge.target.inspect, source: edge.source.to_s } 442 end 443 edge.target = target 444 445 result.add_edge(edge) 446 end 447 end 448 449 result.add_class(*data['classes']) if data['classes'] 450 451 result.metadata = data['metadata'].inject({}) { |h, (k, v)| h[k] = Puppet::FileServing::Metadata.from_data_hash(v); h } if data['metadata'] 452 453 recursive_metadata = data['recursive_metadata'] 454 if recursive_metadata 455 result.recursive_metadata = recursive_metadata.inject({}) do |h, (title, source_to_meta_hash)| 456 h[title] = source_to_meta_hash.inject({}) do |inner_h, (source, metas)| 457 inner_h[source] = metas.map {|meta| Puppet::FileServing::Metadata.from_data_hash(meta)} 458 inner_h 459 end 460 h 461 end 462 end 463 464 result 465 end
Puppet::Graph::SimpleGraph::new
# File lib/puppet/resource/catalog.rb 312 def initialize(name = nil, environment = Puppet::Node::Environment::NONE, code_id = nil) 313 super() 314 @name = name 315 @catalog_uuid = SecureRandom.uuid 316 @catalog_format = 1 317 @metadata = {} 318 @recursive_metadata = {} 319 @classes = [] 320 @resource_table = {} 321 @resources = [] 322 @relationship_graph = nil 323 324 @host_config = true 325 @environment_instance = environment 326 @environment = environment.to_s 327 @code_id = code_id 328 329 @aliases = {} 330 331 if block_given? 332 yield(self) 333 finalize 334 end 335 end
Public Instance Methods
Add classes to our class list.
# File lib/puppet/resource/catalog.rb 74 def add_class(*classes) 75 classes.each do |klass| 76 @classes << klass 77 end 78 79 # Add the class names as tags, too. 80 tag(*classes) 81 end
# File lib/puppet/resource/catalog.rb 124 def add_resource(*resources) 125 resources.each do |resource| 126 add_one_resource(resource) 127 end 128 end
Add `resources` to the catalog after `other`. WARNING: adding multiple resources will produce the reverse ordering, e.g. calling `add_resource_after(A, [B,C])` will result in `[A,C,B]`.
# File lib/puppet/resource/catalog.rb 112 def add_resource_after(other, *resources) 113 resources.each do |resource| 114 other_title_key = title_key_for_ref(other.ref) 115 idx = @resources.index(other_title_key) 116 if idx.nil? 117 raise ArgumentError, _("Cannot add resource %{resource_1} after %{resource_2} because %{resource_2} is not yet in the catalog") % 118 { resource_1: resource.ref, resource_2: other.ref } 119 end 120 add_one_resource(resource, idx+1) 121 end 122 end
# File lib/puppet/resource/catalog.rb 97 def add_resource_before(other, *resources) 98 resources.each do |resource| 99 other_title_key = title_key_for_ref(other.ref) 100 idx = @resources.index(other_title_key) 101 if idx.nil? 102 raise ArgumentError, _("Cannot add resource %{resource_1} before %{resource_2} because %{resource_2} is not yet in the catalog") % 103 { resource_1: resource.ref, resource_2: other.ref } 104 end 105 add_one_resource(resource, idx) 106 end 107 end
Create an alias for a resource.
# File lib/puppet/resource/catalog.rb 182 def alias(resource, key) 183 ref = resource.ref 184 ref =~ /^(.+)\[/ 185 class_name = $1 || resource.class.name 186 187 newref = [class_name, key].flatten 188 189 if key.is_a? String 190 ref_string = "#{class_name}[#{key}]" 191 return if ref_string == ref 192 end 193 194 # LAK:NOTE It's important that we directly compare the references, 195 # because sometimes an alias is created before the resource is 196 # added to the catalog, so comparing inside the below if block 197 # isn't sufficient. 198 existing = @resource_table[newref] 199 if existing 200 return if existing == resource 201 resource_declaration = Puppet::Util::Errors.error_location(resource.file, resource.line) 202 msg = if resource_declaration.empty? 203 #TRANSLATORS 'alias' should not be translated 204 _("Cannot alias %{resource} to %{key}; resource %{newref} already declared") % 205 { resource: ref, key: key.inspect, newref: newref.inspect } 206 else 207 #TRANSLATORS 'alias' should not be translated 208 _("Cannot alias %{resource} to %{key} at %{resource_declaration}; resource %{newref} already declared") % 209 { resource: ref, key: key.inspect, resource_declaration: resource_declaration, newref: newref.inspect } 210 end 211 msg += Puppet::Util::Errors.error_location_with_space(existing.file, existing.line) 212 raise ArgumentError, msg 213 end 214 @resource_table[newref] = resource 215 @aliases[ref] ||= [] 216 @aliases[ref] << newref 217 end
Apply our catalog to the local host. @param options [Hash{Symbol => Object}] a hash of options @option options [Puppet::Transaction::Report] :report
The report object to log this transaction to. This is optional, and the resulting transaction will create a report if not supplied.
@return [Puppet::Transaction] the transaction created for this
application
@api public
# File lib/puppet/resource/catalog.rb 230 def apply(options = {}) 231 Puppet::Util::Storage.load if host_config? 232 233 transaction = create_transaction(options) 234 235 begin 236 transaction.report.as_logging_destination do 237 transaction_evaluate_time = Puppet::Util.thinmark do 238 transaction.evaluate 239 end 240 transaction.report.add_times(:transaction_evaluation, transaction_evaluate_time) 241 end 242 ensure 243 # Don't try to store state unless we're a host config 244 # too recursive. 245 Puppet::Util::Storage.store if host_config? 246 end 247 248 yield transaction if block_given? 249 250 transaction 251 end
# File lib/puppet/resource/catalog.rb 282 def classes 283 @classes.dup 284 end
Puppet::Graph::SimpleGraph#clear
# File lib/puppet/resource/catalog.rb 269 def clear(remove_resources = true) 270 super() 271 # We have to do this so that the resources clean themselves up. 272 @resource_table.values.each { |resource| resource.remove } if remove_resources 273 @resource_table.clear 274 @resources = [] 275 276 if @relationship_graph 277 @relationship_graph.clear 278 @relationship_graph = nil 279 end 280 end
@param resource [A Resource] a resource in the catalog @return [A Resource, nil] the resource that contains the given resource @api public
# File lib/puppet/resource/catalog.rb 133 def container_of(resource) 134 adjacent(resource, :direction => :in)[0] 135 end
Create a new resource and register it in the catalog.
# File lib/puppet/resource/catalog.rb 287 def create_resource(type, options) 288 klass = Puppet::Type.type(type) 289 unless klass 290 raise ArgumentError, _("Unknown resource type %{type}") % { type: type } 291 end 292 resource = klass.new(options) 293 return unless resource 294 295 add_resource(resource) 296 resource 297 end
filter out the catalog, applying block
to each resource. If the block result is false, the resource will be kept otherwise it will be skipped
# File lib/puppet/resource/catalog.rb 506 def filter(&block) 507 # to_catalog must take place in a context where current_environment is set to the same env as the 508 # environment set in the catalog (if it is set) 509 # See PUP-3755 510 if environment_instance 511 Puppet.override({:current_environment => environment_instance}) do 512 to_catalog :to_resource, &block 513 end 514 else 515 # If catalog has no environment_instance, hope that the caller has made sure the context has the 516 # correct current_environment 517 to_catalog :to_resource, &block 518 end 519 end
Make sure all of our resources are “finished”.
# File lib/puppet/resource/catalog.rb 300 def finalize 301 make_default_resources 302 303 @resource_table.values.each { |resource| resource.finish } 304 305 write_graph(:resources) 306 end
# File lib/puppet/resource/catalog.rb 308 def host_config? 309 host_config 310 end
Make the default objects necessary for function.
# File lib/puppet/resource/catalog.rb 338 def make_default_resources 339 # We have to add the resources to the catalog, or else they won't get cleaned up after 340 # the transaction. 341 342 # First create the default scheduling objects 343 Puppet::Type.type(:schedule).mkdefaultschedules.each { |res| add_resource(res) unless resource(res.ref) } 344 345 # And filebuckets 346 bucket = Puppet::Type.type(:filebucket).mkdefaultbucket 347 if bucket 348 add_resource(bucket) unless resource(bucket.ref) 349 end 350 end
The relationship_graph
form of the catalog. This contains all of the dependency edges that are used for determining order.
@param given_prioritizer [Puppet::Graph::Prioritizer] The prioritization
strategy to use when constructing the relationship graph. Defaults the being determined by the `ordering` setting.
@return [Puppet::Graph::RelationshipGraph] @api public
# File lib/puppet/resource/catalog.rb 261 def relationship_graph(given_prioritizer = nil) 262 if @relationship_graph.nil? 263 @relationship_graph = Puppet::Graph::RelationshipGraph.new(given_prioritizer || prioritizer) 264 @relationship_graph.populate_from(self) 265 end 266 @relationship_graph 267 end
Remove the resource from our catalog. Notice that we also call 'remove' on the resource, at least until resource classes no longer maintain references to the resource instances.
# File lib/puppet/resource/catalog.rb 355 def remove_resource(*resources) 356 resources.each do |resource| 357 ref = resource.ref 358 title_key = title_key_for_ref(ref) 359 @resource_table.delete(title_key) 360 aliases = @aliases[ref] 361 if aliases 362 aliases.each { |res_alias| @resource_table.delete(res_alias) } 363 @aliases.delete(ref) 364 end 365 remove_vertex!(resource) if vertex?(resource) 366 @relationship_graph.remove_vertex!(resource) if @relationship_graph and @relationship_graph.vertex?(resource) 367 @resources.delete(title_key) 368 # Only Puppet::Type kind of resources respond to :remove, not Puppet::Resource 369 resource.remove if resource.respond_to?(:remove) 370 end 371 end
Look a resource up by its reference (e.g., File).
# File lib/puppet/resource/catalog.rb 374 def resource(type, title = nil) 375 # Retain type if it's a type 376 type_name = type.is_a?(Puppet::CompilableResourceType) || type.is_a?(Puppet::Resource::Type) ? type.name : type 377 type_name, title = Puppet::Resource.type_and_title(type_name, title) 378 type = type_name if type.is_a?(String) 379 title_key = [type_name, title.to_s] 380 result = @resource_table[title_key] 381 if result.nil? 382 # an instance has to be created in order to construct the unique key used when 383 # searching for aliases 384 res = Puppet::Resource.new(type, title, { :environment => @environment_instance }) 385 386 # Must check with uniqueness key because of aliases or if resource transforms title in title 387 # to attribute mappings. 388 result = @resource_table[[type_name, res.uniqueness_key].flatten] 389 end 390 result 391 end
# File lib/puppet/resource/catalog.rb 397 def resource_keys 398 @resource_table.keys 399 end
# File lib/puppet/resource/catalog.rb 393 def resource_refs 394 resource_keys.collect{ |type, name| name.is_a?( String ) ? "#{type}[#{name}]" : nil}.compact 395 end
# File lib/puppet/resource/catalog.rb 401 def resources 402 @resources.collect do |key| 403 @resource_table[key] 404 end 405 end
Returns [typename, title] when given a String with “Type”. Returns [nil, nil] if '[' ']' not detected.
# File lib/puppet/resource/catalog.rb 86 def title_key_for_ref( ref ) 87 s = ref.index('[') 88 e = ref.rindex(']') 89 if s && e && e > s 90 a = [ref[0, s], ref[s+1, e-s-1]] 91 else 92 a = [nil, nil] 93 end 94 return a 95 end
# File lib/puppet/resource/catalog.rb 467 def to_data_hash 468 metadata_hash = metadata.inject({}) { |h, (k, v)| h[k] = v.to_data_hash; h } 469 recursive_metadata_hash = recursive_metadata.inject({}) do |h, (title, source_to_meta_hash)| 470 h[title] = source_to_meta_hash.inject({}) do |inner_h, (source, metas)| 471 inner_h[source] = metas.map {|meta| meta.to_data_hash} 472 inner_h 473 end 474 h 475 end 476 477 { 478 'tags' => tags.to_a, 479 'name' => name, 480 'version' => version, 481 'code_id' => code_id, 482 'catalog_uuid' => catalog_uuid, 483 'catalog_format' => catalog_format, 484 'environment' => environment.to_s, 485 'resources' => @resources.map { |v| @resource_table[v].to_data_hash }, 486 'edges' => edges.map { |e| e.to_data_hash }, 487 'classes' => classes, 488 }.merge(metadata_hash.empty? ? 489 {} : {'metadata' => metadata_hash}).merge(recursive_metadata_hash.empty? ? 490 {} : {'recursive_metadata' => recursive_metadata_hash}) 491 end
Convert our catalog into a RAL catalog.
# File lib/puppet/resource/catalog.rb 494 def to_ral 495 to_catalog :to_ral 496 end
Convert our catalog into a catalog of Puppet::Resource
instances.
# File lib/puppet/resource/catalog.rb 499 def to_resource 500 to_catalog :to_resource 501 end
Store the classes in the classfile.
# File lib/puppet/resource/catalog.rb 522 def write_class_file 523 # classfile paths may contain UTF-8 524 # https://puppet.com/docs/puppet/latest/configuration.html#classfile 525 classfile = Puppet.settings.setting(:classfile) 526 Puppet::FileSystem.open(classfile.value, classfile.mode.to_i(8), "w:UTF-8") do |f| 527 f.puts classes.join("\n") 528 end 529 rescue => detail 530 Puppet.err _("Could not create class file %{file}: %{detail}") % { file: Puppet[:classfile], detail: detail } 531 end
Produce the graph files if requested.
Puppet::Graph::SimpleGraph#write_graph
# File lib/puppet/resource/catalog.rb 550 def write_graph(name) 551 # We only want to graph the main host catalog. 552 return unless host_config? 553 554 super 555 end
Store the list of resources we manage
# File lib/puppet/resource/catalog.rb 534 def write_resource_file 535 # resourcefile contains resources that may be UTF-8 names 536 # https://puppet.com/docs/puppet/latest/configuration.html#resourcefile 537 resourcefile = Puppet.settings.setting(:resourcefile) 538 Puppet::FileSystem.open(resourcefile.value, resourcefile.mode.to_i(8), "w:UTF-8") do |f| 539 to_print = resources.map do |resource| 540 next unless resource.managed? 541 "#{resource.ref.downcase}" 542 end.compact 543 f.puts to_print.join("\n") 544 end 545 rescue => detail 546 Puppet.err _("Could not create resource file %{file}: %{detail}") % { file: Puppet[:resourcefile], detail: detail } 547 end
Private Instance Methods
# File lib/puppet/resource/catalog.rb 137 def add_one_resource(resource, idx=-1) 138 title_key = title_key_for_ref(resource.ref) 139 if @resource_table[title_key] 140 fail_on_duplicate_type_and_title(resource, title_key) 141 end 142 143 add_resource_to_table(resource, title_key, idx) 144 create_resource_aliases(resource) 145 146 resource.catalog = self if resource.respond_to?(:catalog=) 147 add_resource_to_graph(resource) 148 end
# File lib/puppet/resource/catalog.rb 157 def add_resource_to_graph(resource) 158 add_vertex(resource) 159 @relationship_graph.add_vertex(resource) if @relationship_graph 160 end
# File lib/puppet/resource/catalog.rb 151 def add_resource_to_table(resource, title_key, idx) 152 @resource_table[title_key] = resource 153 @resources.insert(idx, title_key) 154 end
# File lib/puppet/resource/catalog.rb 163 def create_resource_aliases(resource) 164 # Explicit aliases must always be processed 165 # The alias setting logic checks, and does not error if the alias is set to an already set alias 166 # for the same resource (i.e. it is ok if alias == title 167 explicit_aliases = [resource[:alias]].flatten.compact 168 explicit_aliases.each {| given_alias | self.alias(resource, given_alias) } 169 170 # Skip creating uniqueness key alias and checking collisions for non-isomorphic resources. 171 return unless resource.respond_to?(:isomorphic?) and resource.isomorphic? 172 173 # Add an alias if the uniqueness key is valid and not the title, which has already been checked. 174 ukey = resource.uniqueness_key 175 if ukey.any? and ukey != [resource.title] 176 self.alias(resource, ukey) 177 end 178 end
# File lib/puppet/resource/catalog.rb 563 def create_transaction(options) 564 transaction = Puppet::Transaction.new(self, options[:report], prioritizer) 565 transaction.tags = options[:tags] if options[:tags] 566 transaction.ignoreschedules = true if options[:ignoreschedules] 567 transaction.for_network_device = Puppet.lookup(:network_device) { nil } || options[:network_device] 568 569 transaction 570 end
Verify that the given resource isn't declared elsewhere.
# File lib/puppet/resource/catalog.rb 573 def fail_on_duplicate_type_and_title(resource, title_key) 574 # Short-circuit the common case, 575 existing_resource = @resource_table[title_key] 576 return unless existing_resource 577 578 # If we've gotten this far, it's a real conflict 579 error_location_str = Puppet::Util::Errors.error_location(existing_resource.file, existing_resource.line) 580 msg = if error_location_str.empty? 581 _("Duplicate declaration: %{resource} is already declared; cannot redeclare") % { resource: resource.ref } 582 else 583 _("Duplicate declaration: %{resource} is already declared at %{error_location}; cannot redeclare") % { resource: resource.ref, error_location: error_location_str } 584 end 585 raise DuplicateResourceError.new(msg, resource.file, resource.line) 586 end
# File lib/puppet/resource/catalog.rb 559 def prioritizer 560 @prioritizer = Puppet::Graph::SequentialPrioritizer.new 561 end
An abstracted method for converting one catalog into another type of catalog. This pretty much just converts all of the resources from one class to another, using a conversion method.
# File lib/puppet/resource/catalog.rb 591 def to_catalog(convert) 592 result = self.class.new(self.name, self.environment_instance) 593 594 result.version = self.version 595 result.code_id = self.code_id 596 result.catalog_uuid = self.catalog_uuid 597 result.catalog_format = self.catalog_format 598 result.metadata = self.metadata 599 result.recursive_metadata = self.recursive_metadata 600 601 map = {} 602 resources.each do |resource| 603 next if virtual_not_exported?(resource) 604 next if block_given? and yield resource 605 606 newres = resource.copy_as_resource 607 newres.catalog = result 608 609 if convert != :to_resource 610 newres = newres.to_ral 611 end 612 613 # We can't guarantee that resources don't munge their names 614 # (like files do with trailing slashes), so we have to keep track 615 # of what a resource got converted to. 616 map[resource.ref] = newres 617 618 result.add_resource newres 619 end 620 621 message = convert.to_s.tr "_", " " 622 edges.each do |edge| 623 # Skip edges between virtual resources. 624 next if virtual_not_exported?(edge.source) 625 next if block_given? and yield edge.source 626 627 next if virtual_not_exported?(edge.target) 628 next if block_given? and yield edge.target 629 630 source = map[edge.source.ref] 631 unless source 632 raise Puppet::DevError, _("Could not find resource %{resource} when converting %{message} resources") % { resource: edge.source.ref, message: message } 633 end 634 635 target = map[edge.target.ref] 636 unless target 637 raise Puppet::DevError, _("Could not find resource %{resource} when converting %{message} resources") % { resource: edge.target.ref, message: message } 638 end 639 640 result.add_edge(source, target, edge.label) 641 end 642 643 map.clear 644 645 result.add_class(*self.classes) 646 result.merge_tags_from(self) 647 648 result 649 end
# File lib/puppet/resource/catalog.rb 651 def virtual_not_exported?(resource) 652 resource.virtual && !resource.exported 653 end