[2025-07-17 16:18:19,581][  INFO][PID:4146615] Marking build as starting
[2025-07-17 16:18:19,663][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:18:19,666][  INFO][PID:4146615] VM allocation process starts
[2025-07-17 16:18:19,696][  INFO][PID:4146615] Trying to allocate VM: ResallocHost, ticket_id=3972279, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-07-17 16:18:22,746][  INFO][PID:4146615] Allocated host ResallocHost, ticket_id=3972279, hostname=140.211.168.190, name=copr_osuosl_p09_prod_05601169_20250717_153510, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-07-17 16:18:22,746][  INFO][PID:4146615] Allocating ssh connection to builder
[2025-07-17 16:18:22,747][  INFO][PID:4146615] Checking that builder machine is OK
[2025-07-17 16:18:24,131][  INFO][PID:4146615] Running remote command: copr-builder-ready centos-stream-9-ppc64le
[2025-07-17 16:18:25,261][  INFO][PID:4146615] Red Hat subscription not needed for centos-stream-9-ppc64le
Builder is ready to be used

[2025-07-17 16:18:25,261][  INFO][PID:4146615] Filling build.info file with builder info
[2025-07-17 16:18:25,262][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:18:25,263][  INFO][PID:4146615] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "rhcontainerbot",
            "project_name": "podman-next",
            "project_dirname": "podman-next",
            "submitter": "packit",
            "ended_on": null,
            "started_on": 1752769105.2632728,
            "submitted_on": null,
            "status": 3,
            "chroot": "centos-stream-9-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9282660-centos-stream-9-ppc64le",
            "build_id": 9282660,
            "package_name": "ramalama",
            "package_version": "0.11.0-1000.1.20250717161755192265.main.32.g64ca9cf",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rhcontainerbot/podman-next/ramalama",
            "git_hash": "5a4890189af5b1dad517c4894fdf0f4b807c2fc8",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "rhcontainerbot/podman-next--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://dl.fedoraproject.org/pub/epel/9/Everything/$basearch/",
                    "id": "https_dl_fedoraproject_org_pub_epel_9_Everything_basearch",
                    "name": "Additional repo https_dl_fedoraproject_org_pub_epel_9_Everything_basearch"
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/podman-next",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next",
            "result_dir": "09282660-ramalama",
            "built_packages": "",
            "id": 9282660,
            "mockchain_macros": {
                "copr_username": "rhcontainerbot",
                "copr_projectname": "podman-next",
                "vendor": "Fedora Project COPR (rhcontainerbot/podman-next)"
            }
        }
    ]
}
[2025-07-17 16:18:25,343][  INFO][PID:4146615] Sending fedora-messaging bus message in build.start
[2025-07-17 16:18:26,115][  INFO][PID:4146615] Sending fedora-messaging bus message in chroot.start
[2025-07-17 16:18:26,147][  INFO][PID:4146615] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9282660-centos-stream-9-ppc64le --chroot centos-stream-9-ppc64le --detached
[2025-07-17 16:18:27,606][  INFO][PID:4146615] Downloading the builder-live.log file, attempt 1
[2025-07-17 16:18:27,608][  INFO][PID:4146615] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.190 copr-rpmbuild-log
[2025-07-17 16:18:32,665][  INFO][PID:4146615] Periodic builder liveness probe: alive
[2025-07-17 16:18:32,666][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:18:37,667][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:18:42,668][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:18:47,669][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:18:52,670][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:18:57,672][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:19:02,674][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:19:07,675][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:19:12,676][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:19:17,677][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:19:22,678][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:19:27,679][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:19:32,681][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:19:37,682][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:19:42,683][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:19:47,684][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:19:52,685][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:19:57,686][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:20:02,688][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:20:07,689][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:20:12,690][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:20:17,691][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:20:22,692][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:20:27,695][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:20:32,695][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:20:37,696][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:20:42,698][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:20:47,700][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:20:52,703][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:20:57,704][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:21:02,706][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:21:07,707][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:21:12,708][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:21:17,709][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:21:22,710][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:21:27,711][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:21:32,712][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:21:37,714][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:21:42,715][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:21:47,716][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:21:52,717][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:21:57,718][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:22:02,719][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:22:07,720][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:22:12,722][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:22:17,723][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:22:21,139][ ERROR][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-17 16:22:22,724][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:22:27,725][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:22:31,140][WARNING][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-17 16:22:32,726][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:22:37,728][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:22:41,141][  INFO][PID:4146615] Downloading the builder-live.log file, attempt 2
[2025-07-17 16:22:41,142][  INFO][PID:4146615] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.190 copr-rpmbuild-log
[2025-07-17 16:22:42,729][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:22:47,730][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:22:52,732][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:22:57,733][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:23:02,734][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:23:07,735][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:23:12,736][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:23:17,737][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:23:22,738][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:23:27,739][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:23:32,740][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:23:37,741][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:23:41,282][ ERROR][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-17 16:23:42,742][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:23:47,744][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:23:51,282][WARNING][PID:4146615] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-17 16:23:52,745][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:23:57,746][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:24:01,283][  INFO][PID:4146615] Downloading the builder-live.log file, attempt 3
[2025-07-17 16:24:01,284][  INFO][PID:4146615] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.190 copr-rpmbuild-log
[2025-07-17 16:24:02,747][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:24:07,748][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:24:12,749][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:24:17,750][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:24:22,751][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:24:27,752][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:24:32,753][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:24:37,754][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:24:42,755][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:24:47,756][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:24:52,757][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:24:57,758][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:25:01,423][ ERROR][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-17 16:25:02,759][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:25:07,760][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:25:11,423][WARNING][PID:4146615] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-17 16:25:12,762][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:25:17,763][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:25:21,424][  INFO][PID:4146615] Downloading the builder-live.log file, attempt 4
[2025-07-17 16:25:21,425][  INFO][PID:4146615] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.190 copr-rpmbuild-log
[2025-07-17 16:25:22,764][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:25:27,765][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:25:32,766][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:25:37,767][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:25:42,768][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:25:47,769][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:25:52,770][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:25:57,771][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:26:02,772][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:26:07,774][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:26:12,776][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:26:17,777][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:26:21,562][ ERROR][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-17 16:26:22,778][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:26:27,779][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:26:31,563][ ERROR][PID:4146615] Giving up for unstable SSH, failures: 1752769351.1402154, 1752769431.2828581, 1752769511.4238071, 1752769591.5631747
[2025-07-17 16:26:31,564][ ERROR][PID:4146615] Re-try request for task on 'ResallocHost, ticket_id=3972279, hostname=140.211.168.190, name=copr_osuosl_p09_prod_05601169_20250717_153510, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-07-17 16:26:31,564][  INFO][PID:4146615] Releasing VM back to pool
[2025-07-17 16:26:31,573][  INFO][PID:4146615] Retry #1 (on other host)
[2025-07-17 16:26:31,573][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:26:31,574][  INFO][PID:4146615] VM allocation process starts
[2025-07-17 16:26:31,624][  INFO][PID:4146615] Trying to allocate VM: ResallocHost, ticket_id=3972342, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-07-17 16:26:34,665][  INFO][PID:4146615] Allocated host ResallocHost, ticket_id=3972342, hostname=140.211.168.94, name=copr_osuosl_p09_prod_05601440_20250717_155130, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-07-17 16:26:34,665][  INFO][PID:4146615] Allocating ssh connection to builder
[2025-07-17 16:26:34,666][  INFO][PID:4146615] Checking that builder machine is OK
[2025-07-17 16:26:36,121][  INFO][PID:4146615] Running remote command: copr-builder-ready centos-stream-9-ppc64le
[2025-07-17 16:26:37,179][  INFO][PID:4146615] Red Hat subscription not needed for centos-stream-9-ppc64le
Builder is ready to be used

[2025-07-17 16:26:37,180][  INFO][PID:4146615] Filling build.info file with builder info
[2025-07-17 16:26:37,182][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:26:37,183][  INFO][PID:4146615] Not re-notifying FE and msg buses for the new host.
[2025-07-17 16:26:37,184][  INFO][PID:4146615] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9282660-centos-stream-9-ppc64le --chroot centos-stream-9-ppc64le --detached
[2025-07-17 16:26:38,427][  INFO][PID:4146615] Downloading the builder-live.log file, attempt 1
[2025-07-17 16:26:38,428][  INFO][PID:4146615] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.94 copr-rpmbuild-log
[2025-07-17 16:26:43,427][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:26:48,428][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:26:53,429][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:26:58,430][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:27:03,431][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:27:08,433][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:27:13,434][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:27:18,435][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:27:23,436][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:27:28,437][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:27:33,438][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:27:38,439][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:27:43,440][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:27:48,441][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:27:53,442][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:27:58,443][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:28:03,444][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:28:08,446][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:28:13,447][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:28:18,448][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:28:23,449][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:28:28,450][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:28:33,457][  INFO][PID:4146615] Periodic builder liveness probe: alive
[2025-07-17 16:28:33,457][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:28:38,458][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:28:43,459][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:28:48,460][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:28:53,461][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:28:58,462][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:29:03,463][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:29:08,464][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:29:13,465][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:29:18,466][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:29:23,467][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:29:28,468][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:29:33,469][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:29:38,470][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:29:43,471][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:29:48,472][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:29:53,473][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:29:58,474][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:30:03,475][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:30:08,477][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:30:13,477][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:30:18,479][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:30:23,480][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:30:25,120][ ERROR][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-17 16:30:28,481][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:30:33,482][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:30:35,121][WARNING][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-17 16:30:38,483][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:30:43,484][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:30:45,122][  INFO][PID:4146615] Downloading the builder-live.log file, attempt 2
[2025-07-17 16:30:45,123][  INFO][PID:4146615] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.94 copr-rpmbuild-log
[2025-07-17 16:30:48,485][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:30:53,486][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:30:58,487][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:31:03,488][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:31:08,489][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:31:13,490][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:31:18,491][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:31:23,492][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:31:28,493][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:31:33,494][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:31:38,495][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:31:43,496][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:31:45,258][ ERROR][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-17 16:31:48,497][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:31:53,498][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:31:55,263][WARNING][PID:4146615] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-17 16:31:58,499][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:32:03,500][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:32:05,264][  INFO][PID:4146615] Downloading the builder-live.log file, attempt 3
[2025-07-17 16:32:05,265][  INFO][PID:4146615] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.94 copr-rpmbuild-log
[2025-07-17 16:32:08,501][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:32:13,502][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:32:18,503][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:32:23,504][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:32:28,505][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:32:33,506][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:32:38,507][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:32:43,508][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:32:48,509][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:32:53,511][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:32:58,512][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:33:03,513][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:33:05,403][ ERROR][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-17 16:33:08,514][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:33:13,515][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:33:15,404][WARNING][PID:4146615] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-17 16:33:18,516][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:33:23,517][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:33:25,405][  INFO][PID:4146615] Downloading the builder-live.log file, attempt 4
[2025-07-17 16:33:25,406][  INFO][PID:4146615] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.94 copr-rpmbuild-log
[2025-07-17 16:33:28,518][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:33:33,519][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:33:38,521][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:33:43,522][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:33:48,523][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:33:53,524][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:33:58,525][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:34:03,526][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:34:08,527][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:34:13,529][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:34:18,530][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:34:23,531][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:34:25,526][ ERROR][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-17 16:34:28,532][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:34:33,533][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:34:35,526][ ERROR][PID:4146615] Giving up for unstable SSH, failures: 1752769835.121592, 1752769915.263818, 1752769995.4046018, 1752770075.526809
[2025-07-17 16:34:35,527][ ERROR][PID:4146615] Re-try request for task on 'ResallocHost, ticket_id=3972342, hostname=140.211.168.94, name=copr_osuosl_p09_prod_05601440_20250717_155130, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-07-17 16:34:35,528][  INFO][PID:4146615] Releasing VM back to pool
[2025-07-17 16:34:35,536][  INFO][PID:4146615] Retry #2 (on other host)
[2025-07-17 16:34:35,537][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:34:35,537][  INFO][PID:4146615] VM allocation process starts
[2025-07-17 16:34:35,586][  INFO][PID:4146615] Trying to allocate VM: ResallocHost, ticket_id=3972387, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-07-17 16:34:38,617][  INFO][PID:4146615] Allocated host ResallocHost, ticket_id=3972387, hostname=140.211.168.61, name=copr_osuosl_p09_prod_05601812_20250717_161825, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-07-17 16:34:38,617][  INFO][PID:4146615] Allocating ssh connection to builder
[2025-07-17 16:34:38,618][  INFO][PID:4146615] Checking that builder machine is OK
[2025-07-17 16:34:40,071][  INFO][PID:4146615] Running remote command: copr-builder-ready centos-stream-9-ppc64le
[2025-07-17 16:34:41,116][  INFO][PID:4146615] Red Hat subscription not needed for centos-stream-9-ppc64le
Builder is ready to be used

[2025-07-17 16:34:41,117][  INFO][PID:4146615] Filling build.info file with builder info
[2025-07-17 16:34:41,122][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:34:41,122][  INFO][PID:4146615] Not re-notifying FE and msg buses for the new host.
[2025-07-17 16:34:41,123][  INFO][PID:4146615] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9282660-centos-stream-9-ppc64le --chroot centos-stream-9-ppc64le --detached
[2025-07-17 16:34:42,377][  INFO][PID:4146615] Downloading the builder-live.log file, attempt 1
[2025-07-17 16:34:42,379][  INFO][PID:4146615] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-07-17 16:34:47,378][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:34:52,379][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:34:57,380][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:35:02,381][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:35:07,382][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:35:12,383][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:35:17,384][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:35:22,385][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:35:27,386][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:35:32,387][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:35:37,389][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:35:42,390][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:35:47,391][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:35:52,392][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:35:57,393][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:36:02,394][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:36:07,395][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:36:12,396][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:36:17,397][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:36:22,398][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:36:27,399][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:36:32,400][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:36:37,402][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:36:42,403][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:36:47,404][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:36:52,405][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:36:57,407][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:37:02,408][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:37:07,409][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:37:12,410][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:37:17,413][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:37:22,414][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:37:27,415][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:37:32,416][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:37:37,417][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:37:42,418][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:37:47,419][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:37:52,420][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:37:57,421][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:38:02,422][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:38:07,423][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:38:12,425][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:38:17,426][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:38:22,427][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:38:27,428][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:38:32,429][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:38:37,435][  INFO][PID:4146615] Periodic builder liveness probe: alive
[2025-07-17 16:38:37,436][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:38:42,436][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:38:47,437][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:38:52,438][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:38:57,439][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:39:02,440][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:39:07,441][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:39:12,443][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:39:17,445][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:39:22,446][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:39:27,447][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:39:32,448][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:39:37,449][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:39:42,451][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:39:47,451][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:39:52,452][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:39:57,453][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:40:02,455][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:40:07,456][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:40:12,457][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:40:17,458][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:40:22,459][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:40:27,460][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:40:32,461][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:40:37,462][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:40:42,463][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:40:47,464][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:40:52,465][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:40:57,466][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:41:02,468][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:41:07,469][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:41:12,469][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:41:17,471][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:41:22,471][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:41:27,473][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:41:32,474][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:41:37,475][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:41:42,476][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:41:47,477][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:41:52,478][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:41:57,479][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:42:02,480][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:42:07,481][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:42:12,482][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:42:17,483][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:42:22,484][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:42:27,485][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:42:32,486][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:42:37,487][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:42:42,489][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:42:47,490][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:42:52,491][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:42:57,492][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:43:02,493][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:43:07,494][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:43:12,495][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:43:17,496][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:43:22,497][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:43:27,498][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:43:32,499][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:43:37,500][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:43:42,501][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:43:47,502][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:43:52,503][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:43:57,504][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:44:02,505][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:44:07,506][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:44:12,507][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:44:17,508][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:44:22,509][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:44:27,510][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:44:32,511][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:44:37,512][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:44:42,513][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:44:47,514][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:44:52,515][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:44:57,517][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:45:02,518][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:45:07,519][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:45:12,520][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:45:17,521][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:45:22,522][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:45:27,523][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:45:32,524][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:45:37,525][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:45:42,526][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:45:47,527][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:45:52,529][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:45:57,530][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:46:02,531][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:46:07,532][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:46:12,533][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:46:17,534][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:46:22,535][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:46:27,536][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:46:32,538][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:46:37,539][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:46:42,540][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:46:47,541][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:46:52,542][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:46:57,543][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:47:02,544][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:47:07,545][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:47:12,546][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:47:17,547][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:47:22,548][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:47:27,549][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:47:32,551][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:47:37,552][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:47:42,553][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:47:47,556][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:47:52,557][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:47:57,558][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:48:02,559][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:48:07,560][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:48:12,561][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:48:17,562][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:48:22,563][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:48:27,564][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:48:32,566][  INFO][PID:4146615] Checking for cancel request
[2025-07-17 16:48:37,572][  INFO][PID:4146615] Periodic builder liveness probe: dead
[2025-07-17 16:49:37,714][ ERROR][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Connection timed out during banner exchange
Connection to 140.211.168.61 port 22 timed out

[2025-07-17 16:50:47,826][ ERROR][PID:4146615] SSH connection lost on #2 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Connection timed out during banner exchange
Connection to 140.211.168.61 port 22 timed out

[2025-07-17 16:51:57,894][ ERROR][PID:4146615] SSH connection lost on #3 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.61 port 22: Connection timed out

[2025-07-17 16:52:30,140][ ERROR][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-17 16:52:40,140][WARNING][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-17 16:52:50,141][  INFO][PID:4146615] Downloading the builder-live.log file, attempt 2
[2025-07-17 16:52:50,142][  INFO][PID:4146615] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-07-17 16:53:07,921][ ERROR][PID:4146615] SSH connection lost on #4 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.61 port 22: Connection timed out

[2025-07-17 16:53:17,922][ ERROR][PID:4146615] Can't ssh to cancel build.
[2025-07-17 16:53:50,210][ ERROR][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-17 16:54:00,211][WARNING][PID:4146615] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-17 16:54:10,212][  INFO][PID:4146615] Downloading the builder-live.log file, attempt 3
[2025-07-17 16:54:10,213][  INFO][PID:4146615] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-07-17 16:55:10,281][ ERROR][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-17 16:55:20,282][WARNING][PID:4146615] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-17 16:55:30,283][  INFO][PID:4146615] Downloading the builder-live.log file, attempt 4
[2025-07-17 16:55:30,284][  INFO][PID:4146615] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-07-17 16:56:30,351][ ERROR][PID:4146615] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-17 16:56:40,352][ ERROR][PID:4146615] Giving up for unstable SSH, failures: 1752771160.1409254, 1752771240.2114072, 1752771320.282448, 1752771400.352095
[2025-07-17 16:56:40,353][ ERROR][PID:4146615] Re-try request for task on 'ResallocHost, ticket_id=3972387, hostname=140.211.168.61, name=copr_osuosl_p09_prod_05601812_20250717_161825, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': SSH problems when downloading live log: Resalloc ticket FAILED
[2025-07-17 16:56:40,353][  INFO][PID:4146615] Releasing VM back to pool
[2025-07-17 16:56:40,362][ ERROR][PID:4146615] Backend process error: Three host tried without success: {'140.211.168.190', '140.211.168.61', '140.211.168.94'}
[2025-07-17 16:56:40,363][WARNING][PID:4146615] Switching not-finished job state to 'failed'
[2025-07-17 16:56:40,363][  INFO][PID:4146615] Worker failed build, took 1319.2402217388153
[2025-07-17 16:56:40,364][  INFO][PID:4146615] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "rhcontainerbot",
            "project_name": "podman-next",
            "project_dirname": "podman-next",
            "submitter": "packit",
            "ended_on": 1752771400.363099,
            "started_on": 1752770081.1228774,
            "submitted_on": null,
            "status": 0,
            "chroot": "centos-stream-9-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9282660-centos-stream-9-ppc64le",
            "build_id": 9282660,
            "package_name": "ramalama",
            "package_version": "0.11.0-1000.1.20250717161755192265.main.32.g64ca9cf",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rhcontainerbot/podman-next/ramalama",
            "git_hash": "5a4890189af5b1dad517c4894fdf0f4b807c2fc8",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "rhcontainerbot/podman-next--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://dl.fedoraproject.org/pub/epel/9/Everything/$basearch/",
                    "id": "https_dl_fedoraproject_org_pub_epel_9_Everything_basearch",
                    "name": "Additional repo https_dl_fedoraproject_org_pub_epel_9_Everything_basearch"
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/podman-next",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next",
            "result_dir": "09282660-ramalama",
            "built_packages": "",
            "id": 9282660,
            "mockchain_macros": {
                "copr_username": "rhcontainerbot",
                "copr_projectname": "podman-next",
                "vendor": "Fedora Project COPR (rhcontainerbot/podman-next)"
            }
        }
    ]
}
[2025-07-17 16:56:40,416][  INFO][PID:4146615] Sending fedora-messaging bus message in build.end
[2025-07-17 16:56:40,441][  INFO][PID:4146615] Compressing /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09282660-ramalama/builder-live.log by gzip
[2025-07-17 16:56:40,442][  INFO][PID:4146615] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09282660-ramalama/builder-live.log' as PID 94103
[2025-07-17 16:56:40,444][  INFO][PID:4146615] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09282660-ramalama/builder-live.log)
[2025-07-17 16:56:40,444][  INFO][PID:4146615] Compressing /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09282660-ramalama/backend.log by gzip
[2025-07-17 16:56:40,445][  INFO][PID:4146615] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09282660-ramalama/backend.log' as PID 94104