[2025-09-11 16:12:34,723][  INFO][PID:3926108] Marking build as starting
[2025-09-11 16:12:34,752][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:12:34,753][  INFO][PID:3926108] VM allocation process starts
[2025-09-11 16:12:34,763][  INFO][PID:3926108] Trying to allocate VM: ResallocHost, ticket_id=5010123, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-11 16:12:39,765][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:12:44,767][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:12:49,776][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:12:54,777][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:12:59,779][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:13:04,784][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:13:09,786][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:13:14,787][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:13:19,789][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:13:24,790][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:13:29,793][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:13:34,795][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:13:39,800][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:13:44,801][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:13:49,802][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:13:54,807][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:13:59,808][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:14:04,812][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:14:09,815][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:14:14,826][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:14:19,829][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:14:24,833][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:14:29,835][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:14:34,842][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:14:39,846][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:14:44,847][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:14:49,849][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:14:54,854][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:14:59,856][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:15:04,858][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:15:09,860][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:15:14,865][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:15:19,866][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:15:24,870][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:15:29,874][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:15:34,875][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:15:39,876][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:15:44,883][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:15:49,888][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:15:54,892][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:15:59,894][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:16:04,901][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:16:09,904][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:16:14,909][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:16:19,917][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:16:24,925][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:16:29,928][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:16:34,935][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:16:39,943][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:16:44,951][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:16:49,952][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:16:54,955][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:16:59,956][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:17:04,958][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:17:09,964][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:17:14,965][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:17:19,967][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:17:24,973][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:17:29,974][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:17:34,976][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:17:39,979][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:17:44,984][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:17:49,993][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:17:55,008][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:18:00,009][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:18:05,012][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:18:10,015][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:18:15,018][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:18:20,025][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:18:25,026][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:18:30,027][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:18:35,028][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:18:40,030][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:18:45,032][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:18:50,042][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:18:53,060][  INFO][PID:3926108] Allocated host ResallocHost, ticket_id=5010123, hostname=50.16.21.23, name=aws_x86_64_spot_prod_06718056_20250911_161613, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-11 16:18:53,066][  INFO][PID:3926108] Allocating ssh connection to builder
[2025-09-11 16:18:53,068][  INFO][PID:3926108] Checking that builder machine is OK
[2025-09-11 16:18:53,552][  INFO][PID:3926108] Running remote command: copr-builder-ready centos-stream-9-x86_64
[2025-09-11 16:18:53,840][  INFO][PID:3926108] Red Hat subscription not needed for centos-stream-9-x86_64
Builder is ready to be used

[2025-09-11 16:18:53,841][  INFO][PID:3926108] Filling build.info file with builder info
[2025-09-11 16:18:53,842][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:18:53,843][  INFO][PID:3926108] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "networkmanager",
            "project_name": "NetworkManager-main",
            "project_dirname": "NetworkManager-main",
            "submitter": null,
            "ended_on": null,
            "started_on": 1757607533.8428638,
            "submitted_on": null,
            "status": 3,
            "chroot": "centos-stream-9-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "9546506-centos-stream-9-x86_64",
            "build_id": 9546506,
            "package_name": "NetworkManager",
            "package_version": "1:1.55.3-34173.copr.40aa27690c",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/networkmanager/NetworkManager-main/NetworkManager",
            "git_hash": "1e47a11ce10cbc5932c22875ed9874ab6b32a8eb",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "networkmanager/NetworkManager-main--270211e8-4064-43c3-b2b7-d110ee8d0df9",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/networkmanager/NetworkManager-main/centos-stream-9-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://copr-be.cloud.fedoraproject.org/results/nmstate/nm-build-deps/centos-stream-9-$basearch/",
                    "id": "https_copr_be_cloud_fedoraproject_org_results_nmstate_nm_build_deps_centos_stream_9_basearch",
                    "name": "Additional repo https_copr_be_cloud_fedoraproject_org_results_nmstate_nm_build_deps_centos_stream_9_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/networkmanager/NetworkManager-main",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/networkmanager/NetworkManager-main",
            "result_dir": "09546506-NetworkManager",
            "built_packages": "",
            "id": 9546506,
            "mockchain_macros": {
                "copr_username": "networkmanager",
                "copr_projectname": "NetworkManager-main",
                "vendor": "Fedora Project COPR (networkmanager/NetworkManager-main)"
            }
        }
    ]
}
[2025-09-11 16:18:53,890][  INFO][PID:3926108] Sending fedora-messaging bus message in build.start
[2025-09-11 16:18:56,271][  INFO][PID:3926108] Sending fedora-messaging bus message in chroot.start
[2025-09-11 16:18:56,330][  INFO][PID:3926108] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9546506-centos-stream-9-x86_64 --chroot centos-stream-9-x86_64 --detached
[2025-09-11 16:18:56,717][  INFO][PID:3926108] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '50.16.21.23' (ED25519) to the list of known hosts.


[2025-09-11 16:18:56,721][  INFO][PID:3926108] Downloading the builder-live.log file, attempt 1
[2025-09-11 16:18:56,727][  INFO][PID:3926108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@50.16.21.23 copr-rpmbuild-log
[2025-09-11 16:19:01,778][  INFO][PID:3926108] Periodic builder liveness probe: alive
[2025-09-11 16:19:01,778][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:19:06,795][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:19:11,797][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:19:16,799][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:19:21,803][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:19:26,805][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:19:31,807][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:19:36,815][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:19:41,817][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:19:46,822][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:19:51,823][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:19:56,827][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:20:01,829][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:20:06,830][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:20:11,832][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:20:16,841][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:20:21,842][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:20:26,847][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:20:31,850][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:20:36,855][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:20:41,858][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:20:46,861][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:20:51,865][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:20:56,874][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:21:01,875][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:21:06,877][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:21:11,878][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:21:16,880][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:21:21,884][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:21:26,888][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:21:31,893][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:21:36,899][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:21:41,901][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:21:46,911][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:21:51,916][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:21:56,927][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:22:01,933][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:22:06,934][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:22:11,941][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:22:16,944][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:22:21,948][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:22:26,962][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:22:31,968][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:22:36,969][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:22:41,984][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:22:46,992][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:22:52,000][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:22:57,004][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:23:02,010][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:23:07,015][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:23:12,024][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:23:17,025][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:23:22,026][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:23:27,028][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:23:32,031][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:23:37,035][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:23:42,036][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:23:47,039][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:23:52,041][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:23:57,050][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:24:02,052][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:24:07,053][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:24:12,054][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:24:17,055][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:24:22,057][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:24:27,059][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:24:32,060][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:24:37,065][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:24:42,069][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:24:47,072][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:24:52,075][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:24:57,077][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:25:02,078][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:25:07,080][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:25:12,083][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:25:17,085][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:25:22,097][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:25:27,103][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:25:32,105][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:25:37,110][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:25:42,113][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:25:47,116][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:25:52,119][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:25:57,120][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:26:02,121][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:26:07,126][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:26:12,128][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:26:17,129][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:26:22,131][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:26:27,134][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:26:32,135][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:26:37,136][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:26:42,142][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:26:47,147][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:26:52,155][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:26:57,164][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:27:02,165][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:27:07,169][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:27:12,171][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:27:17,173][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:27:22,174][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:27:27,175][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:27:32,177][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:27:37,179][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:27:42,180][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:27:47,184][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:27:52,185][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:27:57,189][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:28:02,190][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:28:07,194][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:28:12,195][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:28:17,197][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:28:22,199][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:28:27,201][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:28:32,202][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:28:37,205][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:28:42,206][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:28:47,208][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:28:52,209][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:28:57,211][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:29:02,233][  INFO][PID:3926108] Periodic builder liveness probe: alive
[2025-09-11 16:29:02,234][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:29:07,236][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:29:12,237][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:29:17,239][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:29:22,240][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:29:27,241][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:29:32,243][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:29:37,244][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:29:42,245][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:29:47,252][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:29:52,254][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:29:57,256][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:30:02,257][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:30:07,258][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:30:12,261][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:30:17,262][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:30:22,264][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:30:27,268][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:30:32,270][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:30:37,276][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:30:42,278][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:30:47,280][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:30:52,281][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:30:57,282][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:31:02,284][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:31:07,286][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:31:12,292][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:31:17,295][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:31:22,296][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:31:27,298][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:31:32,299][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:31:37,300][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:31:42,302][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:31:47,304][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:31:52,306][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:31:57,307][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:32:02,308][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:32:07,309][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:32:12,310][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:32:17,311][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:32:22,313][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:32:27,314][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:32:32,317][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:32:37,320][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:32:42,322][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:32:47,323][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:32:52,324][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:32:57,327][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:33:02,330][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:33:07,331][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:33:12,333][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:33:17,336][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:33:22,338][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:33:27,339][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:33:32,340][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:33:37,341][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:33:42,343][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:33:47,344][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:33:52,345][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:33:57,346][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:34:02,349][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:34:07,353][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:34:11,749][ ERROR][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-11 16:34:12,354][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:34:17,355][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:34:21,750][WARNING][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-11 16:34:22,357][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:34:27,358][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:34:31,751][  INFO][PID:3926108] Downloading the builder-live.log file, attempt 2
[2025-09-11 16:34:31,754][  INFO][PID:3926108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@50.16.21.23 copr-rpmbuild-log
[2025-09-11 16:34:32,359][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:34:37,360][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:34:42,362][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:34:47,363][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:34:52,364][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:34:57,366][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:35:02,367][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:35:07,369][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:35:12,370][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:35:17,371][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:35:22,373][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:35:27,374][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:35:31,826][ ERROR][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-11 16:35:32,375][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:35:37,376][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:35:41,832][WARNING][PID:3926108] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-11 16:35:42,378][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:35:47,379][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:35:51,833][  INFO][PID:3926108] Downloading the builder-live.log file, attempt 3
[2025-09-11 16:35:51,835][  INFO][PID:3926108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@50.16.21.23 copr-rpmbuild-log
[2025-09-11 16:35:52,380][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:35:57,381][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:36:02,382][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:36:07,383][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:36:12,384][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:36:17,386][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:36:22,387][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:36:27,388][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:36:32,389][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:36:37,390][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:36:42,392][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:36:47,393][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:36:51,903][ ERROR][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-11 16:36:52,395][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:36:57,396][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:37:01,904][WARNING][PID:3926108] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-11 16:37:02,397][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:37:07,398][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:37:11,905][  INFO][PID:3926108] Downloading the builder-live.log file, attempt 4
[2025-09-11 16:37:11,907][  INFO][PID:3926108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@50.16.21.23 copr-rpmbuild-log
[2025-09-11 16:37:12,399][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:37:17,401][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:37:22,402][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:37:27,403][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:37:32,404][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:37:37,405][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:37:42,406][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:37:47,407][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:37:52,408][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:37:57,409][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:38:02,411][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:38:07,412][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:38:11,975][ ERROR][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-11 16:38:12,413][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:38:17,414][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:38:21,977][ ERROR][PID:3926108] Giving up for unstable SSH, failures: 1757608461.750376, 1757608541.8329043, 1757608621.9046812, 1757608701.9770284
[2025-09-11 16:38:21,978][ ERROR][PID:3926108] Re-try request for task on 'ResallocHost, ticket_id=5010123, hostname=50.16.21.23, name=aws_x86_64_spot_prod_06718056_20250911_161613, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-09-11 16:38:21,978][  INFO][PID:3926108] Releasing VM back to pool
[2025-09-11 16:38:21,994][  INFO][PID:3926108] Retry #1 (on other host)
[2025-09-11 16:38:21,995][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:38:21,996][  INFO][PID:3926108] VM allocation process starts
[2025-09-11 16:38:22,014][  INFO][PID:3926108] Trying to allocate VM: ResallocHost, ticket_id=5010776, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-11 16:38:27,015][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:38:32,016][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:38:37,017][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:38:42,018][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:38:47,019][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:38:52,020][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:38:57,021][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:39:02,023][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:39:07,024][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:39:12,027][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:39:17,031][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:39:22,038][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:39:27,050][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:39:32,051][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:39:37,052][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:39:42,054][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:39:47,056][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:39:52,057][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:39:57,059][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:40:02,060][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:40:07,063][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:40:12,064][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:40:17,066][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:40:22,068][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:40:27,070][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:40:32,073][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:40:37,074][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:40:42,078][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:40:47,081][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:40:52,085][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:40:57,086][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:41:02,087][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:41:07,088][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:41:12,090][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:41:17,092][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:41:22,093][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:41:27,094][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:41:32,097][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:41:37,098][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:41:40,160][  INFO][PID:3926108] Allocated host ResallocHost, ticket_id=5010776, hostname=98.82.180.181, name=aws_x86_64_spot_prod_06718661_20250911_163954, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-11 16:41:40,161][  INFO][PID:3926108] Allocating ssh connection to builder
[2025-09-11 16:41:40,162][  INFO][PID:3926108] Checking that builder machine is OK
[2025-09-11 16:41:40,610][  INFO][PID:3926108] Running remote command: copr-builder-ready centos-stream-9-x86_64
[2025-09-11 16:41:40,819][  INFO][PID:3926108] Red Hat subscription not needed for centos-stream-9-x86_64
Builder is ready to be used

[2025-09-11 16:41:40,820][  INFO][PID:3926108] Filling build.info file with builder info
[2025-09-11 16:41:40,821][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:41:40,821][  INFO][PID:3926108] Not re-notifying FE and msg buses for the new host.
[2025-09-11 16:41:40,822][  INFO][PID:3926108] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9546506-centos-stream-9-x86_64 --chroot centos-stream-9-x86_64 --detached
[2025-09-11 16:41:41,139][  INFO][PID:3926108] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '98.82.180.181' (ED25519) to the list of known hosts.


[2025-09-11 16:41:41,141][  INFO][PID:3926108] Downloading the builder-live.log file, attempt 1
[2025-09-11 16:41:41,142][  INFO][PID:3926108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.82.180.181 copr-rpmbuild-log
[2025-09-11 16:41:46,150][  INFO][PID:3926108] Periodic builder liveness probe: alive
[2025-09-11 16:41:46,151][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:41:51,151][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:41:56,152][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:42:01,154][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:42:06,155][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:42:11,156][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:42:16,157][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:42:21,159][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:42:26,160][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:42:31,161][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:42:36,163][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:42:41,164][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:42:46,166][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:42:51,167][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:42:56,169][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:43:01,170][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:43:06,171][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:43:11,172][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:43:16,174][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:43:21,176][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:43:26,178][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:43:31,181][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:43:36,183][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:43:41,184][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:43:46,185][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:43:51,187][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:43:56,188][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:44:01,189][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:44:06,190][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:44:11,191][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:44:16,193][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:44:21,194][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:44:26,195][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:44:31,196][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:44:36,197][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:44:41,198][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:44:46,200][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:44:51,201][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:44:56,202][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:45:01,204][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:45:06,205][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:45:11,207][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:45:16,208][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:45:21,209][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:45:26,210][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:45:31,211][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:45:36,212][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:45:41,213][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:45:46,215][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:45:51,217][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:45:56,218][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:46:01,220][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:46:06,224][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:46:11,225][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:46:16,226][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:46:21,228][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:46:26,230][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:46:31,236][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:46:36,237][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:46:41,238][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:46:46,239][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:46:51,241][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:46:56,242][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:47:01,243][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:47:06,245][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:47:11,247][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:47:16,248][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:47:21,249][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:47:26,251][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:47:31,252][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:47:36,253][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:47:41,254][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:47:46,255][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:47:51,257][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:47:56,258][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:48:01,260][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:48:06,263][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:48:11,266][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:48:16,267][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:48:21,268][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:48:26,269][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:48:31,270][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:48:36,272][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:48:41,275][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:48:46,277][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:48:51,282][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:48:56,284][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:49:01,285][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:49:06,286][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:49:11,287][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:49:16,288][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:49:21,290][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:49:26,291][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:49:31,292][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:49:36,293][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:49:41,294][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:49:46,296][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:49:51,297][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:49:56,299][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:50:01,302][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:50:06,304][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:50:11,305][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:50:16,307][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:50:21,310][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:50:26,312][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:50:31,314][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:50:36,315][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:50:41,371][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:50:46,390][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:50:51,499][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:50:56,505][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:51:01,555][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:51:06,629][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:51:11,632][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:51:16,658][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:51:21,663][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:51:26,667][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:51:31,824][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:51:36,877][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:51:41,896][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:51:46,912][  INFO][PID:3926108] Periodic builder liveness probe: alive
[2025-09-11 16:51:46,993][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:51:51,997][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:51:57,001][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:52:02,060][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:52:07,123][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:52:12,159][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:52:17,194][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:52:22,230][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:52:27,265][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:52:32,268][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:52:37,271][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:52:42,284][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:52:47,286][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:52:52,329][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:52:57,443][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:53:02,450][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:53:07,474][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:53:12,480][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:53:17,488][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:53:22,494][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:53:27,496][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:53:32,501][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:53:37,506][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:53:42,524][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:53:46,051][ ERROR][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-11 16:53:47,537][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:53:52,543][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:53:56,055][WARNING][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-11 16:53:57,547][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:54:02,548][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:54:06,056][  INFO][PID:3926108] Downloading the builder-live.log file, attempt 2
[2025-09-11 16:54:06,058][  INFO][PID:3926108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.82.180.181 copr-rpmbuild-log
[2025-09-11 16:54:06,072][ ERROR][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-11 16:54:07,565][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:54:12,574][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:54:16,075][WARNING][PID:3926108] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-11 16:54:17,582][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:54:22,584][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:54:26,076][  INFO][PID:3926108] Downloading the builder-live.log file, attempt 3
[2025-09-11 16:54:26,077][  INFO][PID:3926108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.82.180.181 copr-rpmbuild-log
[2025-09-11 16:54:27,588][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:54:32,597][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:54:37,603][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:54:42,605][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:54:47,608][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:54:52,609][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:54:57,610][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:55:02,615][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:55:07,621][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:55:12,625][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:55:17,635][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:55:22,638][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:55:26,145][ ERROR][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-11 16:55:27,640][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:55:32,643][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:55:36,150][WARNING][PID:3926108] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-11 16:55:37,645][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:55:42,647][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:55:46,153][  INFO][PID:3926108] Downloading the builder-live.log file, attempt 4
[2025-09-11 16:55:46,155][  INFO][PID:3926108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.82.180.181 copr-rpmbuild-log
[2025-09-11 16:55:47,648][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:55:52,649][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:55:57,651][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:56:02,653][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:56:07,657][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:56:12,659][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:56:17,660][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:56:22,665][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:56:27,678][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:56:32,680][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:56:37,687][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:56:42,692][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:56:46,228][ ERROR][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-11 16:56:47,694][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:56:52,696][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:56:56,232][ ERROR][PID:3926108] Giving up for unstable SSH, failures: 1757609636.055244, 1757609656.0754852, 1757609736.1507483, 1757609816.2324708
[2025-09-11 16:56:56,233][ ERROR][PID:3926108] Re-try request for task on 'ResallocHost, ticket_id=5010776, hostname=98.82.180.181, name=aws_x86_64_spot_prod_06718661_20250911_163954, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-09-11 16:56:56,234][  INFO][PID:3926108] Releasing VM back to pool
[2025-09-11 16:56:56,246][  INFO][PID:3926108] Retry #2 (on other host)
[2025-09-11 16:56:56,247][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:56:56,247][  INFO][PID:3926108] VM allocation process starts
[2025-09-11 16:56:56,259][  INFO][PID:3926108] Trying to allocate VM: ResallocHost, ticket_id=5011247, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-11 16:57:01,261][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:57:06,264][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:57:11,265][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:57:16,267][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:57:21,268][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:57:26,269][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:57:31,270][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:57:34,323][  INFO][PID:3926108] Allocated host ResallocHost, ticket_id=5011247, hostname=34.228.255.220, name=aws_x86_64_spot_prod_06719449_20250911_165553, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-11 16:57:34,323][  INFO][PID:3926108] Allocating ssh connection to builder
[2025-09-11 16:57:34,324][  INFO][PID:3926108] Checking that builder machine is OK
[2025-09-11 16:57:34,737][  INFO][PID:3926108] Running remote command: copr-builder-ready centos-stream-9-x86_64
[2025-09-11 16:57:34,944][  INFO][PID:3926108] Red Hat subscription not needed for centos-stream-9-x86_64
Builder is ready to be used

[2025-09-11 16:57:34,946][  INFO][PID:3926108] Filling build.info file with builder info
[2025-09-11 16:57:34,950][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:57:34,952][  INFO][PID:3926108] Not re-notifying FE and msg buses for the new host.
[2025-09-11 16:57:34,953][  INFO][PID:3926108] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9546506-centos-stream-9-x86_64 --chroot centos-stream-9-x86_64 --detached
[2025-09-11 16:57:35,278][  INFO][PID:3926108] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '34.228.255.220' (ED25519) to the list of known hosts.


[2025-09-11 16:57:35,279][  INFO][PID:3926108] Downloading the builder-live.log file, attempt 1
[2025-09-11 16:57:35,280][  INFO][PID:3926108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.228.255.220 copr-rpmbuild-log
[2025-09-11 16:57:40,279][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:57:45,281][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:57:50,282][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:57:55,286][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:58:00,287][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:58:05,290][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:58:10,292][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:58:15,298][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:58:20,300][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:58:25,304][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:58:30,307][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:58:35,311][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:58:40,313][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:58:45,316][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:58:50,318][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:58:55,320][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:59:00,328][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:59:05,329][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:59:10,330][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:59:15,332][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:59:20,335][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:59:25,336][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:59:30,341][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:59:35,343][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:59:40,344][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:59:45,347][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:59:50,352][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 16:59:55,354][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:00:00,360][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:00:05,361][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:00:10,363][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:00:15,367][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:00:20,368][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:00:25,369][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:00:30,373][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:00:35,375][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:00:40,377][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:00:45,378][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:00:50,380][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:00:55,381][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:01:00,382][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:01:05,384][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:01:10,386][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:01:15,392][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:01:20,408][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:01:25,409][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:01:30,411][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:01:35,413][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:01:40,415][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:01:45,416][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:01:50,436][  INFO][PID:3926108] Periodic builder liveness probe: alive
[2025-09-11 17:01:50,437][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:01:55,443][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:02:00,444][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:02:05,446][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:02:10,449][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:02:15,450][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:02:20,452][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:02:25,453][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:02:30,456][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:02:35,457][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:02:40,458][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:02:45,460][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:02:50,461][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:02:55,462][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:03:00,465][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:03:05,466][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:03:10,467][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:03:15,470][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:03:20,471][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:03:25,472][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:03:30,473][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:03:35,474][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:03:40,475][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:03:45,476][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:03:50,477][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:03:55,479][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:04:00,480][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:04:05,482][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:04:10,484][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:04:15,485][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:04:20,486][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:04:25,487][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:04:30,488][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:04:35,489][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:04:40,490][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:04:45,500][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:04:50,502][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:04:55,506][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:05:00,507][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:05:05,510][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:05:10,511][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:05:15,512][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:05:20,513][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:05:25,514][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:05:30,516][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:05:35,517][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:05:40,519][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:05:45,520][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:05:50,521][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:05:55,522][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:06:00,523][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:06:05,524][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:06:10,525][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:06:15,526][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:06:20,527][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:06:25,528][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:06:30,529][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:06:35,530][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:06:40,532][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:06:45,533][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:06:50,535][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:06:55,536][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:07:00,541][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:07:05,543][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:07:10,544][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:07:15,546][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:07:20,547][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:07:25,548][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:07:30,550][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:07:35,551][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:07:40,553][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:07:45,554][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:07:50,556][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:07:55,557][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:08:00,558][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:08:05,559][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:08:10,560][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:08:15,562][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:08:20,563][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:08:25,564][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:08:30,565][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:08:35,566][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:08:40,567][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:08:45,568][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:08:50,570][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:08:55,572][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:09:00,573][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:09:05,574][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:09:10,575][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:09:12,171][ ERROR][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-11 17:09:15,582][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:09:20,584][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:09:22,178][WARNING][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-11 17:09:25,585][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:09:30,586][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:09:32,179][  INFO][PID:3926108] Downloading the builder-live.log file, attempt 2
[2025-09-11 17:09:32,180][  INFO][PID:3926108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.228.255.220 copr-rpmbuild-log
[2025-09-11 17:09:35,587][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:09:40,589][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:09:45,590][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:09:50,591][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:09:55,594][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:10:00,602][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:10:05,604][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:10:10,605][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:10:15,606][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:10:20,607][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:10:25,608][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:10:30,609][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:10:32,249][ ERROR][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-11 17:10:35,610][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:10:40,613][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:10:42,250][WARNING][PID:3926108] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-11 17:10:45,615][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:10:50,618][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:10:52,251][  INFO][PID:3926108] Downloading the builder-live.log file, attempt 3
[2025-09-11 17:10:52,252][  INFO][PID:3926108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.228.255.220 copr-rpmbuild-log
[2025-09-11 17:10:55,619][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:11:00,620][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:11:05,621][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:11:10,622][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:11:15,623][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:11:20,624][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:11:25,625][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:11:30,628][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:11:35,630][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:11:40,631][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:11:45,632][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:11:50,638][  INFO][PID:3926108] Periodic builder liveness probe: alive
[2025-09-11 17:11:50,639][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:11:52,322][ ERROR][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-11 17:11:55,640][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:12:00,641][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:12:02,325][WARNING][PID:3926108] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-11 17:12:05,643][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:12:10,645][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:12:12,326][  INFO][PID:3926108] Downloading the builder-live.log file, attempt 4
[2025-09-11 17:12:12,328][  INFO][PID:3926108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.228.255.220 copr-rpmbuild-log
[2025-09-11 17:12:15,646][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:12:20,647][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:12:25,649][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:12:30,651][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:12:35,653][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:12:40,655][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:12:45,658][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:12:50,659][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:12:55,662][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:13:00,663][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:13:05,666][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:13:10,667][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:13:12,397][ ERROR][PID:3926108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-11 17:13:15,668][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:13:20,670][  INFO][PID:3926108] Checking for cancel request
[2025-09-11 17:13:22,398][ ERROR][PID:3926108] Giving up for unstable SSH, failures: 1757610562.1786222, 1757610642.2503173, 1757610722.3257647, 1757610802.3985045
[2025-09-11 17:13:22,399][ ERROR][PID:3926108] Re-try request for task on 'ResallocHost, ticket_id=5011247, hostname=34.228.255.220, name=aws_x86_64_spot_prod_06719449_20250911_165553, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-09-11 17:13:22,399][  INFO][PID:3926108] Releasing VM back to pool
[2025-09-11 17:13:22,414][ ERROR][PID:3926108] Backend process error: Three host tried without success: {'50.16.21.23', '34.228.255.220', '98.82.180.181'}
[2025-09-11 17:13:22,415][WARNING][PID:3926108] Switching not-finished job state to 'failed'
[2025-09-11 17:13:22,415][  INFO][PID:3926108] Worker failed build, took 947.4630680084229
[2025-09-11 17:13:22,416][  INFO][PID:3926108] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "networkmanager",
            "project_name": "NetworkManager-main",
            "project_dirname": "NetworkManager-main",
            "submitter": null,
            "ended_on": 1757610802.4151056,
            "started_on": 1757609854.9520376,
            "submitted_on": null,
            "status": 0,
            "chroot": "centos-stream-9-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "9546506-centos-stream-9-x86_64",
            "build_id": 9546506,
            "package_name": "NetworkManager",
            "package_version": "1:1.55.3-34173.copr.40aa27690c",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/networkmanager/NetworkManager-main/NetworkManager",
            "git_hash": "1e47a11ce10cbc5932c22875ed9874ab6b32a8eb",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "networkmanager/NetworkManager-main--270211e8-4064-43c3-b2b7-d110ee8d0df9",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/networkmanager/NetworkManager-main/centos-stream-9-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://copr-be.cloud.fedoraproject.org/results/nmstate/nm-build-deps/centos-stream-9-$basearch/",
                    "id": "https_copr_be_cloud_fedoraproject_org_results_nmstate_nm_build_deps_centos_stream_9_basearch",
                    "name": "Additional repo https_copr_be_cloud_fedoraproject_org_results_nmstate_nm_build_deps_centos_stream_9_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/networkmanager/NetworkManager-main",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/networkmanager/NetworkManager-main",
            "result_dir": "09546506-NetworkManager",
            "built_packages": "",
            "id": 9546506,
            "mockchain_macros": {
                "copr_username": "networkmanager",
                "copr_projectname": "NetworkManager-main",
                "vendor": "Fedora Project COPR (networkmanager/NetworkManager-main)"
            }
        }
    ]
}
[2025-09-11 17:13:22,445][  INFO][PID:3926108] Sending fedora-messaging bus message in build.end
[2025-09-11 17:13:22,471][  INFO][PID:3926108] Compressing /var/lib/copr/public_html/results/networkmanager/NetworkManager-main/centos-stream-9-x86_64/09546506-NetworkManager/builder-live.log by gzip
[2025-09-11 17:13:22,472][  INFO][PID:3926108] Running command 'gzip /var/lib/copr/public_html/results/networkmanager/NetworkManager-main/centos-stream-9-x86_64/09546506-NetworkManager/builder-live.log' as PID 216074
[2025-09-11 17:13:22,474][  INFO][PID:3926108] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/networkmanager/NetworkManager-main/centos-stream-9-x86_64/09546506-NetworkManager/builder-live.log)
[2025-09-11 17:13:22,474][  INFO][PID:3926108] Compressing /var/lib/copr/public_html/results/networkmanager/NetworkManager-main/centos-stream-9-x86_64/09546506-NetworkManager/backend.log by gzip
[2025-09-11 17:13:22,475][  INFO][PID:3926108] Running command 'gzip /var/lib/copr/public_html/results/networkmanager/NetworkManager-main/centos-stream-9-x86_64/09546506-NetworkManager/backend.log' as PID 216075