[2025-07-23 16:45:46,009][ INFO][PID:459459] Marking build as starting [2025-07-23 16:45:46,051][ INFO][PID:459459] Checking for cancel request [2025-07-23 16:45:46,054][ INFO][PID:459459] VM allocation process starts [2025-07-23 16:45:46,088][ INFO][PID:459459] Trying to allocate VM: ResallocHost, ticket_id=4071480, requested_tags=['copr_builder', 'arch_aarch64'] [2025-07-23 16:45:51,093][ INFO][PID:459459] Checking for cancel request [2025-07-23 16:45:56,097][ INFO][PID:459459] Checking for cancel request [2025-07-23 16:46:01,098][ INFO][PID:459459] Checking for cancel request [2025-07-23 16:46:04,178][ INFO][PID:459459] Allocated host ResallocHost, ticket_id=4071480, hostname=98.81.166.128, name=aws_aarch64_normalreserved_prod_05764994_20250723_164416, requested_tags=['copr_builder', 'arch_aarch64'] [2025-07-23 16:46:04,179][ INFO][PID:459459] Allocating ssh connection to builder [2025-07-23 16:46:04,180][ INFO][PID:459459] Checking that builder machine is OK [2025-07-23 16:46:04,551][ INFO][PID:459459] Running remote command: copr-builder-ready epel-9-aarch64 [2025-07-23 16:56:04,778][ INFO][PID:459459] Checking Red Hat subscription (attempt #1) Checking Red Hat subscription (attempt #2) Checking Red Hat subscription (attempt #3) Checking Red Hat subscription (attempt #4) Checking Red Hat subscription (attempt #5) Checking Red Hat subscription (attempt #6) Checking Red Hat subscription (attempt #7) Checking Red Hat subscription (attempt #8) Checking Red Hat subscription (attempt #9) Checking Red Hat subscription (attempt #10) Checking Red Hat subscription (attempt #11) Checking Red Hat subscription (attempt #12) Checking Red Hat subscription (attempt #13) Checking Red Hat subscription (attempt #14) Checking Red Hat subscription (attempt #15) Checking Red Hat subscription (attempt #16) Checking Red Hat subscription (attempt #17) Checking Red Hat subscription (attempt #18) Checking Red Hat subscription (attempt #19) Checking Red Hat subscription (attempt #20) Checking Red Hat subscription (attempt #21) Waiting for Red Hat subscription timeouted! [2025-07-23 16:56:04,779][ INFO][PID:459459] Warning: Permanently added '98.81.166.128' (ED25519) to the list of known hosts. [2025-07-23 16:56:04,779][ ERROR][PID:459459] Re-try request for task on 'ResallocHost, ticket_id=4071480, hostname=98.81.166.128, name=aws_aarch64_normalreserved_prod_05764994_20250723_164416, requested_tags=['copr_builder', 'arch_aarch64']': Builder wasn't ready, trying a new one [2025-07-23 16:56:04,780][ INFO][PID:459459] Releasing VM back to pool [2025-07-23 16:56:04,791][ INFO][PID:459459] Retry #1 (on other host) [2025-07-23 16:56:04,792][ INFO][PID:459459] Checking for cancel request [2025-07-23 16:56:04,792][ INFO][PID:459459] VM allocation process starts [2025-07-23 16:56:04,805][ INFO][PID:459459] Trying to allocate VM: ResallocHost, ticket_id=4071727, requested_tags=['copr_builder', 'arch_aarch64'] [2025-07-23 16:56:07,826][ INFO][PID:459459] Allocated host ResallocHost, ticket_id=4071727, hostname=13.217.45.251, name=aws_aarch64_normalreserved_prod_05765330_20250723_165225, requested_tags=['copr_builder', 'arch_aarch64'] [2025-07-23 16:56:07,826][ INFO][PID:459459] Allocating ssh connection to builder [2025-07-23 16:56:07,827][ INFO][PID:459459] Checking that builder machine is OK [2025-07-23 16:56:08,217][ INFO][PID:459459] Running remote command: copr-builder-ready epel-9-aarch64 [2025-07-23 17:06:08,454][ INFO][PID:459459] Checking Red Hat subscription (attempt #1) Checking Red Hat subscription (attempt #2) Checking Red Hat subscription (attempt #3) Checking Red Hat subscription (attempt #4) Checking Red Hat subscription (attempt #5) Checking Red Hat subscription (attempt #6) Checking Red Hat subscription (attempt #7) Checking Red Hat subscription (attempt #8) Checking Red Hat subscription (attempt #9) Checking Red Hat subscription (attempt #10) Checking Red Hat subscription (attempt #11) Checking Red Hat subscription (attempt #12) Checking Red Hat subscription (attempt #13) Checking Red Hat subscription (attempt #14) Checking Red Hat subscription (attempt #15) Checking Red Hat subscription (attempt #16) Checking Red Hat subscription (attempt #17) Checking Red Hat subscription (attempt #18) Checking Red Hat subscription (attempt #19) Checking Red Hat subscription (attempt #20) Checking Red Hat subscription (attempt #21) Waiting for Red Hat subscription timeouted! [2025-07-23 17:06:08,455][ INFO][PID:459459] Warning: Permanently added '13.217.45.251' (ED25519) to the list of known hosts. [2025-07-23 17:06:08,456][ ERROR][PID:459459] Re-try request for task on 'ResallocHost, ticket_id=4071727, hostname=13.217.45.251, name=aws_aarch64_normalreserved_prod_05765330_20250723_165225, requested_tags=['copr_builder', 'arch_aarch64']': Builder wasn't ready, trying a new one [2025-07-23 17:06:08,457][ INFO][PID:459459] Releasing VM back to pool [2025-07-23 17:06:08,473][ INFO][PID:459459] Retry #2 (on other host) [2025-07-23 17:06:08,473][ INFO][PID:459459] Checking for cancel request [2025-07-23 17:06:08,474][ INFO][PID:459459] VM allocation process starts [2025-07-23 17:06:08,496][ INFO][PID:459459] Trying to allocate VM: ResallocHost, ticket_id=4072008, requested_tags=['copr_builder', 'arch_aarch64'] [2025-07-23 17:06:11,518][ INFO][PID:459459] Allocated host ResallocHost, ticket_id=4072008, hostname=44.211.175.194, name=aws_aarch64_spot_prod_05765647_20250723_170425, requested_tags=['copr_builder', 'arch_aarch64'] [2025-07-23 17:06:11,519][ INFO][PID:459459] Allocating ssh connection to builder [2025-07-23 17:06:11,520][ INFO][PID:459459] Checking that builder machine is OK [2025-07-23 17:06:11,903][ INFO][PID:459459] Running remote command: copr-builder-ready epel-9-aarch64 [2025-07-23 17:16:12,134][ INFO][PID:459459] Checking Red Hat subscription (attempt #1) Checking Red Hat subscription (attempt #2) Checking Red Hat subscription (attempt #3) Checking Red Hat subscription (attempt #4) Checking Red Hat subscription (attempt #5) Checking Red Hat subscription (attempt #6) Checking Red Hat subscription (attempt #7) Checking Red Hat subscription (attempt #8) Checking Red Hat subscription (attempt #9) Checking Red Hat subscription (attempt #10) Checking Red Hat subscription (attempt #11) Checking Red Hat subscription (attempt #12) Checking Red Hat subscription (attempt #13) Checking Red Hat subscription (attempt #14) Checking Red Hat subscription (attempt #15) Checking Red Hat subscription (attempt #16) Checking Red Hat subscription (attempt #17) Checking Red Hat subscription (attempt #18) Checking Red Hat subscription (attempt #19) Checking Red Hat subscription (attempt #20) Checking Red Hat subscription (attempt #21) Waiting for Red Hat subscription timeouted! [2025-07-23 17:16:12,135][ INFO][PID:459459] Warning: Permanently added '44.211.175.194' (ED25519) to the list of known hosts. [2025-07-23 17:16:12,136][ ERROR][PID:459459] Re-try request for task on 'ResallocHost, ticket_id=4072008, hostname=44.211.175.194, name=aws_aarch64_spot_prod_05765647_20250723_170425, requested_tags=['copr_builder', 'arch_aarch64']': Builder wasn't ready, trying a new one [2025-07-23 17:16:12,137][ INFO][PID:459459] Releasing VM back to pool [2025-07-23 17:16:12,151][ ERROR][PID:459459] Backend process error: Three host tried without success: {'98.81.166.128', '44.211.175.194', '13.217.45.251'} [2025-07-23 17:16:12,151][WARNING][PID:459459] Switching not-finished job state to 'failed' [2025-07-23 17:16:12,152][ INFO][PID:459459] Worker failed build, took 1826.1584610939026 [2025-07-23 17:16:12,152][ INFO][PID:459459] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "@redhat-et", "project_name": "flightctl-dev", "project_dirname": "flightctl-dev", "submitter": "packit", "ended_on": 1753290972.1517944, "started_on": 1753289145.9933333, "submitted_on": null, "status": 0, "chroot": "epel-9-aarch64", "arch": "aarch64", "buildroot_pkgs": [], "task_id": "9314114-epel-9-aarch64", "build_id": 9314114, "package_name": "flightctl", "package_version": "latest~217~g8f3f19a4-1.20250723164500586083.main.217.g8f3f19a4", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@redhat-et/flightctl-dev/flightctl", "git_hash": "997f13938cb2c86ef73a6d72e2a250591e052dbf", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@redhat-et/flightctl-dev--packit", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@redhat-et/flightctl-dev/epel-9-aarch64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_aarch64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@redhat-et/flightctl-dev", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@redhat-et/flightctl-dev", "result_dir": "09314114-flightctl", "built_packages": "", "id": 9314114, "mockchain_macros": { "copr_username": "@redhat-et", "copr_projectname": "flightctl-dev", "vendor": "Fedora Project COPR (@redhat-et/flightctl-dev)" } } ] } [2025-07-23 17:16:12,189][ INFO][PID:459459] Sending fedora-messaging bus message in build.end [2025-07-23 17:16:12,944][ INFO][PID:459459] Compressing /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/epel-9-aarch64/09314114-flightctl/builder-live.log by gzip [2025-07-23 17:16:12,946][ INFO][PID:459459] Running command 'gzip /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/epel-9-aarch64/09314114-flightctl/builder-live.log' as PID 738556 [2025-07-23 17:16:12,947][ INFO][PID:459459] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/epel-9-aarch64/09314114-flightctl/builder-live.log) stdout: stderr: gzip: /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/epel-9-aarch64/09314114-flightctl/builder-live.log: No such file or directory [2025-07-23 17:16:12,948][ ERROR][PID:459459] Unable to compress file /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/epel-9-aarch64/09314114-flightctl/builder-live.log [2025-07-23 17:16:12,948][ INFO][PID:459459] Compressing /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/epel-9-aarch64/09314114-flightctl/backend.log by gzip [2025-07-23 17:16:12,949][ INFO][PID:459459] Running command 'gzip /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/epel-9-aarch64/09314114-flightctl/backend.log' as PID 738558