[2026-04-20 13:21:34,072][ INFO][PID:2002621] Marking build as starting [2026-04-20 13:21:34,200][ INFO][PID:2002621] Checking that /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/repodata/repomd.xml exists [2026-04-20 13:21:34,311][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:21:34,312][ INFO][PID:2002621] VM allocation process starts [2026-04-20 13:21:34,408][ INFO][PID:2002621] Trying to allocate VM: ResallocHost, ticket_id=2422028, requested_tags=['arch_aarch64', 'copr_builder'] [2026-04-20 13:21:39,416][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:21:44,421][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:21:49,424][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:21:54,435][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:21:59,444][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:22:04,447][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:22:09,451][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:22:14,453][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:22:19,456][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:22:24,460][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:22:29,465][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:22:34,468][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:22:39,471][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:22:44,472][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:22:49,474][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:22:54,478][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:22:59,480][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:23:04,513][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:23:09,515][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:23:14,517][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:23:19,518][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:23:24,519][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:23:29,521][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:23:34,523][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:23:39,524][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:23:44,530][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:23:49,532][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:23:54,533][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:23:59,548][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:24:04,560][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:24:09,563][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:24:14,567][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:24:19,578][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:24:24,593][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:24:29,593][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:24:34,596][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:24:39,598][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:24:44,601][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:24:49,604][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:24:54,606][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:24:59,609][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:25:04,612][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:25:09,618][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:25:14,620][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:25:19,621][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:25:24,623][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:25:29,624][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:25:34,626][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:25:39,628][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:25:44,632][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:25:49,637][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:25:54,638][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:25:59,639][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:26:04,640][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:26:09,642][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:26:14,646][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:26:19,662][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:26:24,663][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:26:29,668][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:26:34,669][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:26:39,671][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:26:44,676][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:26:49,677][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:26:52,791][ INFO][PID:2002621] Allocated host ResallocHost, ticket_id=2422028, hostname=3.92.47.51, name=aws_aarch64_normal_prod_07132071_20260420_132508, requested_tags=['arch_aarch64', 'copr_builder'] [2026-04-20 13:26:52,792][ INFO][PID:2002621] Allocating ssh connection to builder [2026-04-20 13:26:52,792][ INFO][PID:2002621] Checking that builder machine is OK [2026-04-20 13:26:53,214][ INFO][PID:2002621] Installed copr-rpmbuild version: 1.8 [2026-04-20 13:26:53,216][ INFO][PID:2002621] Running remote command: copr-builder-ready fedora-43-aarch64 [2026-04-20 13:26:53,501][ INFO][PID:2002621] Red Hat subscription not needed for fedora-43-aarch64 Builder is ready to be used [2026-04-20 13:26:53,505][ INFO][PID:2002621] Filling build.info file with builder info [2026-04-20 13:26:53,543][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:26:53,545][ INFO][PID:2002621] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "@redhat-et", "project_name": "flightctl-dev", "project_dirname": "flightctl-dev", "submitter": "packit", "ended_on": null, "started_on": 1776691613.5452766, "submitted_on": null, "status": 3, "chroot": "fedora-43-aarch64", "arch": "aarch64", "buildroot_pkgs": [], "task_id": "10348578-fedora-43-aarch64", "build_id": 10348578, "package_name": "flightctl", "package_version": "1.2.0~main~357~g17573020-1.20260420132041108055.main.444.g17573020", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@redhat-et/flightctl-dev/flightctl", "git_hash": "8497d404e49f12ee30469da8093c611b845475e9", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@redhat-et/flightctl-dev--packit", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@redhat-et/flightctl-dev/fedora-43-aarch64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_aarch64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@redhat-et/flightctl-dev", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@redhat-et/flightctl-dev", "result_dir": "10348578-flightctl", "built_packages": "", "id": 10348578, "mockchain_macros": { "copr_username": "@redhat-et", "copr_projectname": "flightctl-dev", "vendor": "Fedora Project COPR (@redhat-et/flightctl-dev)" } } ] } [2026-04-20 13:26:53,620][ INFO][PID:2002621] Sending fedora-messaging bus message in build.start [2026-04-20 13:26:54,374][ INFO][PID:2002621] Sending fedora-messaging bus message in chroot.start [2026-04-20 13:26:54,411][ INFO][PID:2002621] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10348578-fedora-43-aarch64 --chroot fedora-43-aarch64 --detached [2026-04-20 13:26:54,960][ INFO][PID:2002621] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '3.92.47.51' (ED25519) to the list of known hosts. [2026-04-20 13:26:54,962][ INFO][PID:2002621] Downloading the builder-live.log file, attempt 1 [2026-04-20 13:26:54,965][ INFO][PID:2002621] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.92.47.51 copr-rpmbuild-log [2026-04-20 13:26:59,988][ INFO][PID:2002621] Periodic builder liveness probe: alive [2026-04-20 13:26:59,991][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:27:05,002][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:27:10,003][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:27:15,012][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:27:20,015][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:27:25,022][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:27:30,026][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:27:35,038][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:27:40,044][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:27:45,046][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:27:50,047][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:27:55,051][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:28:00,222][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:28:05,239][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:28:10,241][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:28:15,267][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:28:20,274][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:28:25,283][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:28:30,289][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:28:35,291][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:28:40,295][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:28:45,297][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:28:50,310][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:28:55,327][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:29:00,330][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:29:05,338][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:29:10,339][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:29:15,345][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:29:20,347][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:29:25,350][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:29:30,353][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:29:35,356][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:29:40,360][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:29:44,409][ INFO][PID:2002621] Downloading results from builder [2026-04-20 13:29:44,410][ INFO][PID:2002621] rsyncing of mockbuilder@3.92.47.51:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl started [2026-04-20 13:29:44,412][ INFO][PID:2002621] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@3.92.47.51:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl/ &> /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl/build-10348578.rsync.log [2026-04-20 13:29:44,785][ INFO][PID:2002621] rsyncing finished. [2026-04-20 13:29:44,786][ ERROR][PID:2002621] Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-04-20 13:29:44,790][ ERROR][PID:2002621] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-04-20 13:29:54,793][ INFO][PID:2002621] rsyncing of mockbuilder@3.92.47.51:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl started [2026-04-20 13:29:54,797][ INFO][PID:2002621] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@3.92.47.51:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl/ &> /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl/build-10348578.rsync.log [2026-04-20 13:29:55,058][ INFO][PID:2002621] rsyncing finished. [2026-04-20 13:29:55,063][ ERROR][PID:2002621] Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-04-20 13:29:55,066][ ERROR][PID:2002621] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-04-20 13:30:05,067][ INFO][PID:2002621] rsyncing of mockbuilder@3.92.47.51:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl started [2026-04-20 13:30:05,068][ INFO][PID:2002621] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@3.92.47.51:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl/ &> /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl/build-10348578.rsync.log [2026-04-20 13:30:05,226][ INFO][PID:2002621] rsyncing finished. [2026-04-20 13:30:05,228][ ERROR][PID:2002621] Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-04-20 13:30:05,229][ ERROR][PID:2002621] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-04-20 13:30:15,231][ ERROR][PID:2002621] Re-try request for task on 'ResallocHost, ticket_id=2422028, hostname=3.92.47.51, name=aws_aarch64_normal_prod_07132071_20260420_132508, requested_tags=['arch_aarch64', 'copr_builder']': Unable to finish after 3 SSH attempts [2026-04-20 13:30:15,234][ INFO][PID:2002621] VM Release request [2026-04-20 13:30:15,254][ INFO][PID:2002621] Retry #1 (on other host) [2026-04-20 13:30:15,255][ INFO][PID:2002621] Checking that /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/repodata/repomd.xml exists [2026-04-20 13:30:15,256][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:30:15,257][ INFO][PID:2002621] VM allocation process starts [2026-04-20 13:30:15,283][ INFO][PID:2002621] Trying to allocate VM: ResallocHost, ticket_id=2422401, requested_tags=['arch_aarch64', 'copr_builder'] [2026-04-20 13:30:20,291][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:30:25,293][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:30:30,297][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:30:35,301][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:30:40,303][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:30:45,315][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:30:50,329][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:30:55,331][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:31:00,333][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:31:05,336][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:31:10,338][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:31:15,351][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:31:20,353][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:31:25,354][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:31:30,358][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:31:35,361][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:31:40,370][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:31:45,372][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:31:50,380][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:31:55,380][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:32:00,382][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:32:05,389][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:32:10,391][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:32:15,392][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:32:20,397][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:32:25,399][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:32:30,401][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:32:35,402][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:32:40,404][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:32:45,406][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:32:50,408][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:32:55,416][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:33:00,419][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:33:05,429][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:33:10,431][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:33:15,435][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:33:20,438][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:33:25,442][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:33:30,448][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:33:35,450][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:33:40,471][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:33:45,473][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:33:50,475][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:33:55,484][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:34:00,487][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:34:05,490][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:34:10,492][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:34:15,497][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:34:20,499][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:34:25,505][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:34:30,510][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:34:35,513][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:34:40,520][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:34:45,524][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:34:50,528][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:34:55,532][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:35:00,535][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:35:05,537][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:35:10,541][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:35:15,546][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:35:20,554][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:35:25,558][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:35:30,563][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:35:35,566][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:35:40,568][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:35:45,573][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:35:50,574][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:35:55,579][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:36:00,583][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:36:05,590][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:36:10,609][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:36:15,627][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:36:20,631][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:36:25,634][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:36:30,638][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:36:33,716][ INFO][PID:2002621] Allocated host ResallocHost, ticket_id=2422401, hostname=34.226.214.156, name=aws_aarch64_normalreserved_prod_07132552_20260420_133339, requested_tags=['arch_aarch64', 'copr_builder'] [2026-04-20 13:36:33,719][ INFO][PID:2002621] Allocating ssh connection to builder [2026-04-20 13:36:33,720][ INFO][PID:2002621] Checking that builder machine is OK [2026-04-20 13:36:33,988][ ERROR][PID:2002621] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: Warning: Permanently added '34.226.214.156' (ED25519) to the list of known hosts. Load key "/home/copr/.ssh/id_rsa": Permission denied Permission denied, please try again. Permission denied, please try again. mockbuilder@34.226.214.156: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password). [2026-04-20 13:36:43,992][ ERROR][PID:2002621] Re-try request for task on 'ResallocHost, ticket_id=2422401, hostname=34.226.214.156, name=aws_aarch64_normalreserved_prod_07132552_20260420_133339, requested_tags=['arch_aarch64', 'copr_builder']': Unable to finish after 1 SSH attempts [2026-04-20 13:36:44,001][ INFO][PID:2002621] VM Release request [2026-04-20 13:36:44,054][ INFO][PID:2002621] Retry #2 (on other host) [2026-04-20 13:36:44,058][ INFO][PID:2002621] Checking that /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/repodata/repomd.xml exists [2026-04-20 13:36:44,065][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:36:44,076][ INFO][PID:2002621] VM allocation process starts [2026-04-20 13:36:44,150][ INFO][PID:2002621] Trying to allocate VM: ResallocHost, ticket_id=2422599, requested_tags=['arch_aarch64', 'copr_builder'] [2026-04-20 13:36:49,167][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:36:54,169][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:36:59,179][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:37:04,181][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:37:09,196][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:37:14,210][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:37:19,215][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:37:24,227][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:37:29,228][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:37:34,245][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:37:39,253][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:37:44,259][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:37:49,273][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:37:54,280][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:37:59,306][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:38:04,311][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:38:09,312][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:38:14,335][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:38:19,343][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:38:24,352][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:38:29,363][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:38:34,389][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:38:39,404][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:38:44,408][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:38:49,414][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:38:54,422][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:38:59,428][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:39:04,429][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:39:09,444][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:39:14,446][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:39:19,454][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:39:24,457][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:39:29,460][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:39:34,463][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:39:39,468][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:39:44,509][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:39:49,511][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:39:54,513][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:39:59,518][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:40:04,532][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:40:09,533][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:40:14,535][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:40:19,541][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:40:24,542][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:40:29,543][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:40:34,553][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:40:39,554][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:40:44,555][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:40:49,558][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:40:54,560][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:40:59,563][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:41:04,564][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:41:09,568][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:41:14,569][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:41:19,571][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:41:24,572][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:41:29,575][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:41:34,579][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:41:39,580][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:41:44,583][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:41:49,584][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:41:54,587][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:41:59,590][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:42:04,709][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:42:09,712][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:42:14,715][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:42:19,716][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:42:24,720][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:42:29,721][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:42:34,729][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:42:39,734][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:42:44,736][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:42:49,740][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:42:54,749][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:42:59,750][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:43:04,753][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:43:09,758][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:43:14,774][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:43:19,785][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:43:24,789][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:43:29,794][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:43:34,800][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:43:39,804][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:43:44,813][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:43:49,815][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:43:54,818][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:43:59,830][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:44:04,836][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:44:09,838][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:44:14,843][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:44:19,855][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:44:24,864][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:44:29,869][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:44:34,876][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:44:39,877][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:44:44,879][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:44:49,885][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:44:54,888][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:44:59,889][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:45:04,894][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:45:09,901][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:45:14,904][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:45:19,912][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:45:24,917][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:45:29,931][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:45:34,933][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:45:39,944][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:45:44,952][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:45:49,957][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:45:54,959][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:45:59,969][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:46:04,972][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:46:09,974][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:46:14,980][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:46:19,982][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:46:24,984][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:46:29,988][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:46:34,989][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:46:39,991][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:46:44,995][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:46:49,997][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:46:55,001][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:47:00,002][ INFO][PID:2002621] Checking for cancel request [2026-04-20 13:47:02,895][ INFO][PID:2002621] Allocated host ResallocHost, ticket_id=2422599, hostname=98.80.12.123, name=aws_aarch64_eserved_prod_07133233_20260420_134358, requested_tags=['arch_aarch64', 'copr_builder'] [2026-04-20 13:47:02,896][ INFO][PID:2002621] Allocating ssh connection to builder [2026-04-20 13:47:02,899][ INFO][PID:2002621] Checking that builder machine is OK [2026-04-20 13:47:03,037][ ERROR][PID:2002621] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: Warning: Permanently added '98.80.12.123' (ED25519) to the list of known hosts. Load key "/home/copr/.ssh/id_rsa": Permission denied Permission denied, please try again. Permission denied, please try again. mockbuilder@98.80.12.123: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password). [2026-04-20 13:47:13,039][ ERROR][PID:2002621] Re-try request for task on 'ResallocHost, ticket_id=2422599, hostname=98.80.12.123, name=aws_aarch64_eserved_prod_07133233_20260420_134358, requested_tags=['arch_aarch64', 'copr_builder']': Unable to finish after 1 SSH attempts [2026-04-20 13:47:13,040][ INFO][PID:2002621] VM Release request [2026-04-20 13:47:13,072][ ERROR][PID:2002621] Backend process error: Three host tried without success: {'34.226.214.156', '3.92.47.51', '98.80.12.123'} [2026-04-20 13:47:13,072][WARNING][PID:2002621] Switching not-finished job state to 'failed' [2026-04-20 13:47:13,076][ INFO][PID:2002621] Worker failed build, took 1219.5275058746338 [2026-04-20 13:47:13,090][ INFO][PID:2002621] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "@redhat-et", "project_name": "flightctl-dev", "project_dirname": "flightctl-dev", "submitter": "packit", "ended_on": 1776692833.0727825, "started_on": 1776691613.5452766, "submitted_on": null, "status": 0, "chroot": "fedora-43-aarch64", "arch": "aarch64", "buildroot_pkgs": [], "task_id": "10348578-fedora-43-aarch64", "build_id": 10348578, "package_name": "flightctl", "package_version": "1.2.0~main~357~g17573020-1.20260420132041108055.main.444.g17573020", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@redhat-et/flightctl-dev/flightctl", "git_hash": "8497d404e49f12ee30469da8093c611b845475e9", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@redhat-et/flightctl-dev--packit", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@redhat-et/flightctl-dev/fedora-43-aarch64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_aarch64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@redhat-et/flightctl-dev", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@redhat-et/flightctl-dev", "result_dir": "10348578-flightctl", "built_packages": "", "id": 10348578, "mockchain_macros": { "copr_username": "@redhat-et", "copr_projectname": "flightctl-dev", "vendor": "Fedora Project COPR (@redhat-et/flightctl-dev)" } } ] } [2026-04-20 13:47:13,282][ INFO][PID:2002621] Sending fedora-messaging bus message in build.end [2026-04-20 13:47:13,395][ INFO][PID:2002621] Compressing /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl/builder-live.log by gzip [2026-04-20 13:47:13,400][ INFO][PID:2002621] Running command 'gzip /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl/builder-live.log' as PID 2360681 [2026-04-20 13:47:13,549][ INFO][PID:2002621] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl/builder-live.log) [2026-04-20 13:47:13,550][ INFO][PID:2002621] Compressing /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl/backend.log by gzip [2026-04-20 13:47:13,552][ INFO][PID:2002621] Running command 'gzip /var/lib/copr/public_html/results/@redhat-et/flightctl-dev/fedora-43-aarch64/10348578-flightctl/backend.log' as PID 2360716