[2025-07-17 14:50:13,561][ INFO][PID:3685698] Marking build as starting [2025-07-17 14:50:13,634][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:50:13,637][ INFO][PID:3685698] VM allocation process starts [2025-07-17 14:50:13,656][ INFO][PID:3685698] Trying to allocate VM: ResallocHost, ticket_id=3971126, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-07-17 14:50:16,708][ INFO][PID:3685698] Allocated host ResallocHost, ticket_id=3971126, hostname=140.211.168.240, name=copr_osuosl_p09_prod_05599670_20250717_142708, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-07-17 14:50:16,708][ INFO][PID:3685698] Allocating ssh connection to builder [2025-07-17 14:50:16,709][ INFO][PID:3685698] Checking that builder machine is OK [2025-07-17 14:50:18,158][ INFO][PID:3685698] Running remote command: copr-builder-ready centos-stream-9-ppc64le [2025-07-17 14:50:19,194][ INFO][PID:3685698] Red Hat subscription not needed for centos-stream-9-ppc64le Builder is ready to be used [2025-07-17 14:50:19,195][ INFO][PID:3685698] Filling build.info file with builder info [2025-07-17 14:50:19,196][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:50:19,197][ INFO][PID:3685698] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "rhcontainerbot", "project_name": "podman-next", "project_dirname": "podman-next", "submitter": "packit", "ended_on": null, "started_on": 1752763819.196862, "submitted_on": null, "status": 3, "chroot": "centos-stream-9-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9282265-centos-stream-9-ppc64le", "build_id": 9282265, "package_name": "buildah", "package_version": "102:1.40.0-1.20250717144909619735.main.202.g1c0fb347c", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rhcontainerbot/podman-next/buildah", "git_hash": "bc83a35f6122b944c80f20188c3a34d84e2310a3", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "rhcontainerbot/podman-next--packit", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://dl.fedoraproject.org/pub/epel/9/Everything/$basearch/", "id": "https_dl_fedoraproject_org_pub_epel_9_Everything_basearch", "name": "Additional repo https_dl_fedoraproject_org_pub_epel_9_Everything_basearch" } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le", "arch_power9" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/podman-next", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next", "result_dir": "09282265-buildah", "built_packages": "", "id": 9282265, "mockchain_macros": { "copr_username": "rhcontainerbot", "copr_projectname": "podman-next", "vendor": "Fedora Project COPR (rhcontainerbot/podman-next)" } } ] } [2025-07-17 14:50:19,259][ INFO][PID:3685698] Sending fedora-messaging bus message in build.start [2025-07-17 14:50:20,150][ INFO][PID:3685698] Sending fedora-messaging bus message in chroot.start [2025-07-17 14:50:20,192][ INFO][PID:3685698] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9282265-centos-stream-9-ppc64le --chroot centos-stream-9-ppc64le --detached [2025-07-17 14:50:21,561][ INFO][PID:3685698] Downloading the builder-live.log file, attempt 1 [2025-07-17 14:50:21,563][ INFO][PID:3685698] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.240 copr-rpmbuild-log [2025-07-17 14:50:26,567][ INFO][PID:3685698] Periodic builder liveness probe: alive [2025-07-17 14:50:26,568][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:50:31,568][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:50:36,570][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:50:41,570][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:50:46,572][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:50:51,573][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:50:56,574][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:51:01,575][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:51:06,576][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:51:11,577][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:51:16,578][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:51:21,579][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:51:26,581][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:51:31,582][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:51:36,583][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:51:41,584][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:51:46,586][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:51:51,587][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:51:56,588][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:52:01,589][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:52:06,590][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:52:11,591][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:52:16,592][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:52:21,594][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:52:26,595][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:52:31,596][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:52:36,597][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:52:41,599][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:52:46,600][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:52:51,601][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:52:56,602][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:53:01,603][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:53:06,604][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:53:11,605][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:53:16,606][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:53:21,607][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:53:26,608][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:53:31,609][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:53:36,611][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:53:41,612][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:53:46,613][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:53:51,614][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:53:56,615][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:54:01,616][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:54:06,617][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:54:11,618][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:54:16,619][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:54:21,139][ ERROR][PID:3685698] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-17 14:54:21,620][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:54:26,621][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:54:31,139][WARNING][PID:3685698] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-17 14:54:31,622][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:54:36,623][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:54:41,140][ INFO][PID:3685698] Downloading the builder-live.log file, attempt 2 [2025-07-17 14:54:41,142][ INFO][PID:3685698] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.240 copr-rpmbuild-log [2025-07-17 14:54:41,624][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:54:46,625][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:54:51,626][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:54:56,627][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:55:01,629][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:55:06,630][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:55:11,631][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:55:16,632][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:55:21,633][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:55:26,634][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:55:31,635][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:55:36,636][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:55:41,277][ ERROR][PID:3685698] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-17 14:55:41,637][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:55:46,639][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:55:51,279][WARNING][PID:3685698] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-17 14:55:51,640][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:55:56,641][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:56:01,280][ INFO][PID:3685698] Downloading the builder-live.log file, attempt 3 [2025-07-17 14:56:01,282][ INFO][PID:3685698] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.240 copr-rpmbuild-log [2025-07-17 14:56:01,643][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:56:06,647][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:56:11,648][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:56:16,650][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:56:21,651][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:56:26,652][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:56:31,653][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:56:36,654][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:56:41,655][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:56:46,656][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:56:51,657][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:56:56,659][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:57:01,421][ ERROR][PID:3685698] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-17 14:57:01,660][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:57:06,661][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:57:11,426][WARNING][PID:3685698] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-17 14:57:11,662][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:57:16,669][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:57:21,426][ INFO][PID:3685698] Downloading the builder-live.log file, attempt 4 [2025-07-17 14:57:21,428][ INFO][PID:3685698] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.240 copr-rpmbuild-log [2025-07-17 14:57:21,674][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:57:26,676][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:57:31,677][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:57:36,678][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:57:41,680][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:57:46,684][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:57:51,686][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:57:56,687][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:58:01,689][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:58:06,690][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:58:11,692][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:58:16,693][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:58:21,566][ ERROR][PID:3685698] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-17 14:58:21,694][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:58:26,695][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:58:31,571][ ERROR][PID:3685698] Giving up for unstable SSH, failures: 1752764071.1398265, 1752764151.2798102, 1752764231.426025, 1752764311.570945 [2025-07-17 14:58:31,572][ ERROR][PID:3685698] Re-try request for task on 'ResallocHost, ticket_id=3971126, hostname=140.211.168.240, name=copr_osuosl_p09_prod_05599670_20250717_142708, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-07-17 14:58:31,572][ INFO][PID:3685698] Releasing VM back to pool [2025-07-17 14:58:31,585][ INFO][PID:3685698] Retry #1 (on other host) [2025-07-17 14:58:31,586][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:58:31,586][ INFO][PID:3685698] VM allocation process starts [2025-07-17 14:58:31,600][ INFO][PID:3685698] Trying to allocate VM: ResallocHost, ticket_id=3971209, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-07-17 14:58:36,601][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:58:41,602][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:58:46,603][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:58:51,605][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:58:56,606][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:59:01,607][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:59:06,608][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:59:11,609][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:59:16,611][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:59:21,612][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:59:26,613][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:59:31,615][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:59:36,616][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:59:41,617][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:59:46,618][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:59:51,619][ INFO][PID:3685698] Checking for cancel request [2025-07-17 14:59:56,620][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:00:01,621][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:00:06,622][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:00:11,623][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:00:16,624][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:00:19,684][ INFO][PID:3685698] Allocated host ResallocHost, ticket_id=3971209, hostname=140.211.168.9, name=copr_osuosl_p09_prod_05600366_20250717_145639, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-07-17 15:00:19,684][ INFO][PID:3685698] Allocating ssh connection to builder [2025-07-17 15:00:19,685][ INFO][PID:3685698] Checking that builder machine is OK [2025-07-17 15:00:21,161][ INFO][PID:3685698] Running remote command: copr-builder-ready centos-stream-9-ppc64le [2025-07-17 15:00:22,179][ INFO][PID:3685698] Red Hat subscription not needed for centos-stream-9-ppc64le Builder is ready to be used [2025-07-17 15:00:22,180][ INFO][PID:3685698] Filling build.info file with builder info [2025-07-17 15:00:22,183][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:00:22,183][ INFO][PID:3685698] Not re-notifying FE and msg buses for the new host. [2025-07-17 15:00:22,184][ INFO][PID:3685698] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9282265-centos-stream-9-ppc64le --chroot centos-stream-9-ppc64le --detached [2025-07-17 15:00:23,566][ INFO][PID:3685698] Downloading the builder-live.log file, attempt 1 [2025-07-17 15:00:23,567][ INFO][PID:3685698] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.9 copr-rpmbuild-log [2025-07-17 15:00:28,605][ INFO][PID:3685698] Periodic builder liveness probe: alive [2025-07-17 15:00:28,606][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:00:33,606][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:00:38,607][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:00:43,608][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:00:48,609][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:00:53,610][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:00:58,611][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:01:03,612][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:01:08,613][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:01:13,614][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:01:18,615][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:01:23,617][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:01:28,618][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:01:33,619][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:01:38,620][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:01:43,621][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:01:48,622][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:01:53,623][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:01:58,624][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:02:03,625][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:02:08,626][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:02:13,627][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:02:18,628][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:02:23,629][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:02:28,630][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:02:33,631][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:02:38,632][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:02:43,633][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:02:48,634][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:02:53,635][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:02:58,636][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:03:03,637][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:03:08,638][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:03:13,639][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:03:18,640][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:03:23,641][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:03:28,642][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:03:33,643][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:03:38,644][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:03:43,645][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:03:48,646][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:03:53,647][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:03:58,648][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:04:03,649][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:04:08,650][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:04:13,651][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:04:18,652][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:04:23,653][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:04:28,654][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:04:33,655][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:04:38,656][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:04:43,657][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:04:48,658][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:04:53,660][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:04:58,661][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:05:03,662][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:05:08,663][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:05:13,664][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:05:18,665][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:05:23,666][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:05:28,667][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:05:33,668][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:05:38,669][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:05:43,670][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:05:48,671][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:05:53,673][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:05:58,674][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:06:03,675][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:06:08,684][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:06:13,689][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:06:18,691][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:06:23,693][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:06:28,693][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:06:33,695][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:06:38,696][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:06:43,698][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:06:48,699][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:06:53,701][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:06:58,707][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:07:03,712][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:07:08,713][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:07:13,722][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:07:18,723][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:07:23,726][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:07:28,728][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:07:33,729][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:07:38,731][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:07:43,733][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:07:48,734][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:07:53,735][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:07:58,741][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:08:03,750][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:08:08,756][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:08:13,757][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:08:18,758][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:08:23,761][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:08:28,762][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:08:33,764][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:08:38,765][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:08:43,767][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:08:48,769][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:08:53,772][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:08:58,773][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:09:03,775][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:09:08,779][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:09:13,781][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:09:18,786][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:09:23,788][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:09:28,795][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:09:33,798][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:09:38,805][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:09:43,808][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:09:48,816][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:09:53,818][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:09:58,825][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:10:03,828][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:10:08,832][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:10:13,842][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:10:18,843][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:10:23,847][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:10:28,865][ INFO][PID:3685698] Periodic builder liveness probe: alive [2025-07-17 15:10:28,867][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:10:33,875][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:10:38,878][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:10:43,883][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:10:48,884][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:10:53,892][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:10:58,894][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:11:03,896][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:11:08,904][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:11:13,917][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:11:18,918][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:11:23,924][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:11:28,925][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:11:33,927][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:11:38,930][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:11:43,935][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:11:48,938][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:11:53,945][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:11:58,958][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:12:03,960][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:12:08,961][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:12:13,963][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:12:18,969][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:12:23,970][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:12:28,975][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:12:33,977][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:12:38,981][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:12:43,982][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:12:48,985][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:12:53,986][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:12:58,989][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:13:03,992][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:13:08,995][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:13:13,997][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:13:18,998][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:13:23,999][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:13:29,000][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:13:34,002][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:13:39,005][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:13:44,007][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:13:49,009][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:13:54,010][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:13:59,011][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:14:04,014][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:14:09,016][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:14:14,017][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:14:19,018][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:14:24,019][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:14:29,020][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:14:34,022][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:14:39,025][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:14:44,027][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:14:49,028][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:14:54,030][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:14:59,033][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:15:04,034][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:15:09,036][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:15:14,038][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:15:19,039][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:15:24,040][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:15:29,043][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:15:34,046][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:15:39,049][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:15:44,051][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:15:49,052][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:15:54,054][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:15:59,055][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:16:04,057][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:16:09,061][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:16:14,063][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:16:19,064][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:16:24,066][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:16:29,067][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:16:34,068][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:16:39,072][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:16:44,074][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:16:49,076][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:16:54,077][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:16:59,079][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:17:04,080][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:17:09,083][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:17:14,084][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:17:19,085][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:17:24,088][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:17:29,091][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:17:34,101][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:17:39,102][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:17:44,104][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:17:49,106][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:17:54,108][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:17:59,111][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:18:04,112][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:18:09,113][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:18:14,115][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:18:19,116][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:18:24,117][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:18:29,118][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:18:34,119][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:18:39,120][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:18:44,121][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:18:49,123][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:18:54,124][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:18:59,125][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:19:04,126][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:19:09,127][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:19:14,128][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:19:19,129][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:19:24,131][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:19:29,132][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:19:34,133][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:19:39,134][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:19:44,136][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:19:49,137][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:19:54,138][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:19:59,139][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:20:04,140][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:20:09,142][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:20:14,143][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:20:19,144][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:20:24,145][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:20:29,153][ INFO][PID:3685698] Periodic builder liveness probe: alive [2025-07-17 15:20:29,153][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:20:34,154][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:20:39,155][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:20:44,157][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:20:49,158][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:20:54,159][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:20:59,160][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:21:04,161][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:21:09,162][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:21:14,163][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:21:19,164][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:21:24,165][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:21:29,167][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:21:34,167][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:21:39,169][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:21:44,170][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:21:49,171][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:21:54,172][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:21:59,173][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:22:04,174][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:22:09,175][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:22:14,176][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:22:19,178][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:22:24,178][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:22:29,180][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:22:34,181][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:22:39,182][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:22:44,183][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:22:49,184][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:22:54,185][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:22:59,187][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:23:04,188][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:23:09,189][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:23:14,190][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:23:19,191][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:23:24,192][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:23:29,193][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:23:34,194][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:23:39,196][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:23:44,197][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:23:49,198][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:23:54,199][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:23:59,200][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:24:04,201][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:24:09,202][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:24:14,203][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:24:19,204][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:24:24,205][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:24:29,206][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:24:34,207][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:24:39,208][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:24:44,209][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:24:49,210][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:24:54,211][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:24:59,212][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:25:04,214][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:25:09,215][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:25:14,216][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:25:19,217][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:25:24,218][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:25:29,219][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:25:34,220][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:25:39,221][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:25:44,222][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:25:49,223][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:25:54,224][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:25:59,225][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:26:04,227][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:26:09,228][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:26:14,229][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:26:19,230][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:26:24,231][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:26:29,232][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:26:34,233][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:26:39,234][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:26:44,236][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:26:49,237][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:26:54,238][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:26:59,240][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:27:04,241][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:27:09,243][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:27:14,244][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:27:19,245][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:27:24,246][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:27:29,247][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:27:34,248][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:27:39,250][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:27:44,251][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:27:49,251][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:27:54,252][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:27:59,254][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:28:03,139][ ERROR][PID:3685698] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-17 15:28:04,255][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:28:09,256][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:28:13,139][WARNING][PID:3685698] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-17 15:28:14,257][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:28:19,258][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:28:23,140][ INFO][PID:3685698] Downloading the builder-live.log file, attempt 2 [2025-07-17 15:28:23,142][ INFO][PID:3685698] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.9 copr-rpmbuild-log [2025-07-17 15:28:24,259][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:28:29,260][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:28:34,261][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:28:39,262][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:28:44,264][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:28:49,265][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:28:54,268][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:28:59,270][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:29:04,273][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:29:09,275][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:29:14,276][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:29:19,277][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:29:23,209][ ERROR][PID:3685698] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-17 15:29:24,278][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:29:29,279][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:29:33,211][WARNING][PID:3685698] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-17 15:29:34,280][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:29:39,282][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:29:43,217][ INFO][PID:3685698] Downloading the builder-live.log file, attempt 3 [2025-07-17 15:29:43,223][ INFO][PID:3685698] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.9 copr-rpmbuild-log [2025-07-17 15:29:44,288][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:29:49,290][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:29:54,291][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:29:59,298][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:30:04,300][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:30:09,302][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:30:14,303][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:30:19,304][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:30:24,305][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:30:29,312][ INFO][PID:3685698] Periodic builder liveness probe: dead [2025-07-17 15:30:43,295][ ERROR][PID:3685698] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-17 15:30:53,297][WARNING][PID:3685698] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-17 15:31:03,298][ INFO][PID:3685698] Downloading the builder-live.log file, attempt 4 [2025-07-17 15:31:03,299][ INFO][PID:3685698] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.9 copr-rpmbuild-log [2025-07-17 15:31:29,383][ ERROR][PID:3685698] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 140.211.168.9 port 22: Connection timed out [2025-07-17 15:32:03,367][ ERROR][PID:3685698] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-17 15:32:13,372][ ERROR][PID:3685698] Giving up for unstable SSH, failures: 1752766093.1399193, 1752766173.2111418, 1752766253.2975218, 1752766333.3725932 [2025-07-17 15:32:39,453][ ERROR][PID:3685698] SSH connection lost on #2 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 140.211.168.9 port 22: Connection timed out [2025-07-17 15:33:49,524][ ERROR][PID:3685698] SSH connection lost on #3 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 140.211.168.9 port 22: Connection timed out [2025-07-17 15:34:59,594][ ERROR][PID:3685698] SSH connection lost on #4 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 140.211.168.9 port 22: Connection timed out [2025-07-17 15:35:09,595][ ERROR][PID:3685698] Can't ssh to cancel build. [2025-07-17 15:35:09,595][ ERROR][PID:3685698] Re-try request for task on 'ResallocHost, ticket_id=3971209, hostname=140.211.168.9, name=copr_osuosl_p09_prod_05600366_20250717_145639, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': SSH problems when downloading live log: Resalloc ticket FAILED [2025-07-17 15:35:09,596][ INFO][PID:3685698] Releasing VM back to pool [2025-07-17 15:35:09,605][ INFO][PID:3685698] Retry #2 (on other host) [2025-07-17 15:35:09,605][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:35:09,606][ INFO][PID:3685698] VM allocation process starts [2025-07-17 15:35:09,656][ INFO][PID:3685698] Trying to allocate VM: ResallocHost, ticket_id=3971885, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-07-17 15:35:12,696][ INFO][PID:3685698] Allocated host ResallocHost, ticket_id=3971885, hostname=140.211.168.158, name=copr_osuosl_p09_prod_05600983_20250717_152459, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-07-17 15:35:12,697][ INFO][PID:3685698] Allocating ssh connection to builder [2025-07-17 15:35:12,697][ INFO][PID:3685698] Checking that builder machine is OK [2025-07-17 15:35:14,085][ INFO][PID:3685698] Running remote command: copr-builder-ready centos-stream-9-ppc64le [2025-07-17 15:35:15,156][ INFO][PID:3685698] Red Hat subscription not needed for centos-stream-9-ppc64le Builder is ready to be used [2025-07-17 15:35:15,157][ INFO][PID:3685698] Filling build.info file with builder info [2025-07-17 15:35:15,161][ INFO][PID:3685698] Checking for cancel request [2025-07-17 15:35:15,161][ INFO][PID:3685698] Not re-notifying FE and msg buses for the new host. [2025-07-17 15:35:15,162][ INFO][PID:3685698] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9282265-centos-stream-9-ppc64le --chroot centos-stream-9-ppc64le --detached [2025-07-17 15:35:16,617][ INFO][PID:3685698] Downloading the builder-live.log file, attempt 1 [2025-07-17 15:35:16,619][ INFO][PID:3685698] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.158 copr-rpmbuild-log [2025-07-17 15:35:25,813][ INFO][PID:3685698] Cancel request succeeded out: success err: Warning: Permanently added '140.211.168.158' (ED25519) to the list of known hosts. [2025-07-17 15:35:25,813][ INFO][PID:3685698] Downloading results from builder [2025-07-17 15:35:25,814][ INFO][PID:3685698] rsyncing of mockbuilder@140.211.168.158:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09282265-buildah started [2025-07-17 15:35:25,815][ INFO][PID:3685698] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@140.211.168.158:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09282265-buildah/ &> /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09282265-buildah/build-09282265.rsync.log [2025-07-17 15:35:27,528][ INFO][PID:3685698] rsyncing finished. [2025-07-17 15:35:27,529][ INFO][PID:3685698] Releasing VM back to pool [2025-07-17 15:35:27,541][ INFO][PID:3685698] Searching for 'success' file in resultdir [2025-07-17 15:35:27,542][ ERROR][PID:3685698] Build failed: Backend process error: No success file => build failure [2025-07-17 15:35:27,543][ INFO][PID:3685698] Finished build: id=9282265 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/rhcontainerbot/podman-next chroot=centos-stream-9-ppc64le [2025-07-17 15:35:27,545][ ERROR][PID:3685698] Backend process error: No success file => build failure [2025-07-17 15:35:27,545][ INFO][PID:3685698] Worker failed build, took 12.384104251861572 [2025-07-17 15:35:27,546][ INFO][PID:3685698] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "rhcontainerbot", "project_name": "podman-next", "project_dirname": "podman-next", "submitter": "packit", "ended_on": 1752766527.5457132, "started_on": 1752766515.161609, "submitted_on": null, "status": 0, "chroot": "centos-stream-9-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9282265-centos-stream-9-ppc64le", "build_id": 9282265, "package_name": "buildah", "package_version": "102:1.40.0-1.20250717144909619735.main.202.g1c0fb347c", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rhcontainerbot/podman-next/buildah", "git_hash": "bc83a35f6122b944c80f20188c3a34d84e2310a3", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "rhcontainerbot/podman-next--packit", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://dl.fedoraproject.org/pub/epel/9/Everything/$basearch/", "id": "https_dl_fedoraproject_org_pub_epel_9_Everything_basearch", "name": "Additional repo https_dl_fedoraproject_org_pub_epel_9_Everything_basearch" } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le", "arch_power9" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/podman-next", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next", "result_dir": "09282265-buildah", "built_packages": "", "id": 9282265, "mockchain_macros": { "copr_username": "rhcontainerbot", "copr_projectname": "podman-next", "vendor": "Fedora Project COPR (rhcontainerbot/podman-next)" } } ] } [2025-07-17 15:35:27,596][ INFO][PID:3685698] Sending fedora-messaging bus message in build.end [2025-07-17 15:35:27,622][ INFO][PID:3685698] Compressing /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09282265-buildah/builder-live.log by gzip [2025-07-17 15:35:27,623][ INFO][PID:3685698] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09282265-buildah/builder-live.log' as PID 3953761 [2025-07-17 15:35:27,625][ INFO][PID:3685698] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09282265-buildah/builder-live.log) [2025-07-17 15:35:27,627][ INFO][PID:3685698] Compressing /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09282265-buildah/backend.log by gzip [2025-07-17 15:35:27,628][ INFO][PID:3685698] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09282265-buildah/backend.log' as PID 3953762