[2025-08-25 13:57:29,523][ INFO][PID:1565584] Marking build as starting [2025-08-25 13:57:29,575][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:57:29,576][ INFO][PID:1565584] VM allocation process starts [2025-08-25 13:57:29,601][ INFO][PID:1565584] Trying to allocate VM: ResallocHost, ticket_id=4707838, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-25 13:57:34,603][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:57:39,604][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:57:44,605][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:57:49,606][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:57:54,607][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:57:59,608][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:58:04,609][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:58:09,610][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:58:14,611][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:58:19,612][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:58:24,613][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:58:29,614][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:58:34,615][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:58:39,617][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:58:44,618][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:58:49,619][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:58:54,620][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:58:59,621][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:59:04,622][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:59:09,623][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:59:14,624][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:59:19,625][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:59:24,626][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:59:29,627][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:59:34,628][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:59:39,629][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:59:44,630][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:59:49,631][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:59:54,633][ INFO][PID:1565584] Checking for cancel request [2025-08-25 13:59:59,634][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:00:04,635][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:00:09,637][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:00:14,638][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:00:19,639][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:00:24,640][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:00:29,641][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:00:34,642][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:00:39,643][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:00:44,644][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:00:49,646][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:00:54,647][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:00:59,648][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:01:04,649][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:01:09,650][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:01:14,651][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:01:19,652][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:01:24,653][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:01:29,655][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:01:34,656][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:01:39,657][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:01:44,658][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:01:49,659][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:01:54,660][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:01:59,661][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:02:04,666][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:02:09,667][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:02:14,668][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:02:19,669][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:02:24,670][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:02:29,671][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:02:34,672][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:02:39,673][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:02:44,675][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:02:49,676][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:02:54,677][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:02:59,678][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:03:04,679][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:03:09,680][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:03:14,682][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:03:19,683][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:03:24,684][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:03:29,685][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:03:34,686][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:03:39,687][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:03:44,688][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:03:47,709][ INFO][PID:1565584] Allocated host ResallocHost, ticket_id=4707838, hostname=44.201.137.88, name=aws_x86_64_spot_prod_06411620_20250825_140145, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-25 14:03:47,710][ INFO][PID:1565584] Allocating ssh connection to builder [2025-08-25 14:03:47,711][ INFO][PID:1565584] Checking that builder machine is OK [2025-08-25 14:03:48,132][ INFO][PID:1565584] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-08-25 14:03:48,320][ INFO][PID:1565584] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-08-25 14:03:48,321][ INFO][PID:1565584] Filling build.info file with builder info [2025-08-25 14:03:48,322][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:03:48,322][ INFO][PID:1565584] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "alternateved", "project_name": "bleeding-emacs", "project_dirname": "bleeding-emacs", "submitter": "alternateved", "ended_on": null, "started_on": 1756130628.3226402, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9473314-fedora-rawhide-x86_64", "build_id": 9473314, "package_name": "emacs", "package_version": "1:30.2.50-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/alternateved/bleeding-emacs/emacs", "git_hash": "be4b44e7b4753436aeeeb7ac730956f2d064b2ae", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "alternateved/bleeding-emacs--alternateved", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/alternateved/bleeding-emacs/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/alternateved/bleeding-emacs", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/alternateved/bleeding-emacs", "result_dir": "09473314-emacs", "built_packages": "", "id": 9473314, "mockchain_macros": { "copr_username": "alternateved", "copr_projectname": "bleeding-emacs", "vendor": "Fedora Project COPR (alternateved/bleeding-emacs)" } } ] } [2025-08-25 14:03:48,359][ INFO][PID:1565584] Sending fedora-messaging bus message in build.start [2025-08-25 14:03:49,024][ INFO][PID:1565584] Sending fedora-messaging bus message in chroot.start [2025-08-25 14:03:49,049][ INFO][PID:1565584] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9473314-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-08-25 14:03:49,365][ INFO][PID:1565584] Downloading the builder-live.log file, attempt 1 [2025-08-25 14:03:49,366][ INFO][PID:1565584] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.201.137.88 copr-rpmbuild-log [2025-08-25 14:03:54,373][ INFO][PID:1565584] Periodic builder liveness probe: alive [2025-08-25 14:03:54,374][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:03:59,375][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:04:04,376][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:04:09,377][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:04:14,378][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:04:19,380][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:04:24,381][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:04:29,382][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:04:34,383][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:04:39,384][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:04:44,385][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:04:49,386][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:04:54,388][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:04:59,389][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:05:04,391][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:05:09,392][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:05:14,393][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:05:19,394][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:05:24,395][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:05:29,397][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:05:34,398][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:05:39,402][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:05:44,408][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:05:49,409][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:05:54,410][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:05:59,411][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:06:04,412][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:06:09,414][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:06:14,415][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:06:19,416][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:06:24,418][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:06:29,426][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:06:34,427][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:06:39,428][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:06:44,429][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:06:49,431][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:06:54,432][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:06:59,433][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:07:04,434][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:07:09,436][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:07:14,437][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:07:19,439][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:07:24,440][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:07:29,441][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:07:34,442][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:07:39,443][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:07:44,444][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:07:49,446][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:07:54,449][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:07:59,455][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:08:04,457][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:08:09,458][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:08:14,459][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:08:19,461][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:08:24,462][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:08:29,464][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:08:34,469][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:08:39,472][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:08:44,478][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:08:49,487][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:08:54,489][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:08:59,492][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:09:04,493][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:09:09,495][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:09:14,501][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:09:19,503][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:09:24,511][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:09:29,514][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:09:34,522][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:09:39,523][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:09:44,525][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:09:49,528][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:09:54,531][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:09:59,532][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:10:04,537][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:10:09,548][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:10:14,555][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:10:19,559][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:10:24,561][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:10:29,563][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:10:34,569][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:10:39,570][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:10:44,572][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:10:49,575][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:10:54,582][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:10:59,585][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:11:04,590][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:11:09,595][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:11:14,599][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:11:19,601][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:11:24,604][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:11:29,606][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:11:34,612][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:11:39,619][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:11:44,624][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:11:49,626][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:11:54,630][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:11:59,635][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:12:04,636][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:12:09,640][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:12:14,647][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:12:19,650][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:12:24,653][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:12:29,655][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:12:34,657][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:12:39,658][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:12:44,660][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:12:49,664][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:12:54,665][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:12:59,666][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:13:04,668][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:13:09,670][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:13:14,673][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:13:19,674][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:13:24,676][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:13:29,680][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:13:34,681][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:13:39,682][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:13:44,683][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:13:49,685][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:13:50,490][ ERROR][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 14:13:54,693][ INFO][PID:1565584] Periodic builder liveness probe: alive [2025-08-25 14:13:54,694][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:13:59,695][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:14:00,491][WARNING][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-25 14:14:04,696][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:14:09,697][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:14:10,492][ INFO][PID:1565584] Downloading the builder-live.log file, attempt 2 [2025-08-25 14:14:10,494][ INFO][PID:1565584] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.201.137.88 copr-rpmbuild-log [2025-08-25 14:14:14,698][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:14:19,699][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:14:24,700][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:14:29,702][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:14:34,703][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:14:39,704][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:14:44,705][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:14:49,708][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:14:54,710][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:14:59,711][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:15:04,712][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:15:09,714][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:15:10,566][ ERROR][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 14:15:14,717][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:15:19,720][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:15:20,569][WARNING][PID:1565584] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-25 14:15:24,725][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:15:29,727][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:15:30,572][ INFO][PID:1565584] Downloading the builder-live.log file, attempt 3 [2025-08-25 14:15:30,574][ INFO][PID:1565584] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.201.137.88 copr-rpmbuild-log [2025-08-25 14:15:34,729][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:15:39,732][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:15:44,735][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:15:49,739][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:15:54,740][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:15:59,750][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:16:04,752][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:16:09,753][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:16:14,754][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:16:19,756][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:16:24,757][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:16:29,758][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:16:30,646][ ERROR][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 14:16:34,759][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:16:39,760][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:16:40,650][WARNING][PID:1565584] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-25 14:16:44,761][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:16:49,763][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:16:50,651][ INFO][PID:1565584] Downloading the builder-live.log file, attempt 4 [2025-08-25 14:16:50,653][ INFO][PID:1565584] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.201.137.88 copr-rpmbuild-log [2025-08-25 14:16:54,764][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:16:59,765][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:17:04,771][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:17:09,772][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:17:14,773][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:17:19,774][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:17:24,775][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:17:29,777][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:17:34,778][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:17:39,779][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:17:44,780][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:17:49,783][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:17:50,724][ ERROR][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 14:17:54,788][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:17:59,789][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:18:00,726][ ERROR][PID:1565584] Giving up for unstable SSH, failures: 1756131240.4916604, 1756131320.5697935, 1756131400.6503615, 1756131480.7261028 [2025-08-25 14:18:00,727][ ERROR][PID:1565584] Re-try request for task on 'ResallocHost, ticket_id=4707838, hostname=44.201.137.88, name=aws_x86_64_spot_prod_06411620_20250825_140145, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-08-25 14:18:00,727][ INFO][PID:1565584] Releasing VM back to pool [2025-08-25 14:18:00,741][ INFO][PID:1565584] Retry #1 (on other host) [2025-08-25 14:18:00,742][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:18:00,744][ INFO][PID:1565584] VM allocation process starts [2025-08-25 14:18:00,762][ INFO][PID:1565584] Trying to allocate VM: ResallocHost, ticket_id=4708306, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-25 14:18:05,764][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:18:10,765][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:18:15,766][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:18:20,768][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:18:25,774][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:18:30,775][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:18:35,776][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:18:40,779][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:18:45,781][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:18:50,783][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:18:55,786][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:19:00,788][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:19:05,792][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:19:10,795][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:19:15,798][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:19:20,802][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:19:25,804][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:19:28,934][ INFO][PID:1565584] Allocated host ResallocHost, ticket_id=4708306, hostname=18.233.168.104, name=aws_x86_64_spot_prod_06411970_20250825_141739, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-25 14:19:28,939][ INFO][PID:1565584] Allocating ssh connection to builder [2025-08-25 14:19:28,940][ INFO][PID:1565584] Checking that builder machine is OK [2025-08-25 14:19:29,361][ INFO][PID:1565584] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-08-25 14:19:29,569][ INFO][PID:1565584] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-08-25 14:19:29,570][ INFO][PID:1565584] Filling build.info file with builder info [2025-08-25 14:19:29,570][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:19:29,571][ INFO][PID:1565584] Not re-notifying FE and msg buses for the new host. [2025-08-25 14:19:29,572][ INFO][PID:1565584] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9473314-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-08-25 14:19:29,890][ INFO][PID:1565584] Downloading the builder-live.log file, attempt 1 [2025-08-25 14:19:29,892][ INFO][PID:1565584] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@18.233.168.104 copr-rpmbuild-log [2025-08-25 14:19:34,891][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:19:39,892][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:19:44,893][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:19:49,899][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:19:54,900][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:19:59,902][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:20:04,903][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:20:09,906][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:20:14,908][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:20:19,910][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:20:24,919][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:20:29,921][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:20:34,922][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:20:39,924][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:20:44,926][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:20:49,927][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:20:54,928][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:20:59,929][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:21:04,931][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:21:09,932][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:21:14,934][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:21:19,936][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:21:24,937][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:21:29,938][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:21:34,940][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:21:39,941][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:21:44,942][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:21:49,944][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:21:54,945][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:21:59,947][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:22:04,950][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:22:09,952][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:22:14,962][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:22:19,963][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:22:24,965][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:22:29,968][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:22:34,970][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:22:39,971][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:22:44,972][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:22:49,973][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:22:54,975][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:22:59,976][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:23:04,978][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:23:09,981][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:23:14,982][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:23:19,985][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:23:24,990][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:23:29,993][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:23:35,002][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:23:40,003][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:23:45,004][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:23:50,005][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:23:55,020][ INFO][PID:1565584] Periodic builder liveness probe: alive [2025-08-25 14:23:55,022][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:24:00,027][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:24:05,029][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:24:10,030][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:24:15,032][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:24:20,033][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:24:25,034][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:24:30,036][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:24:35,038][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:24:40,040][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:24:45,042][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:24:50,043][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:24:55,045][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:25:00,046][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:25:05,048][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:25:10,051][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:25:15,052][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:25:20,053][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:25:25,054][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:25:30,056][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:25:35,057][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:25:40,058][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:25:45,059][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:25:50,061][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:25:55,064][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:26:00,065][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:26:05,066][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:26:10,067][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:26:15,069][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:26:20,070][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:26:25,072][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:26:30,075][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:26:35,080][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:26:40,082][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:26:45,084][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:26:50,088][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:26:55,094][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:27:00,095][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:27:05,099][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:27:10,100][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:27:15,102][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:27:20,108][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:27:25,114][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:27:30,115][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:27:35,117][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:27:40,123][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:27:45,126][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:27:50,128][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:27:55,132][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:28:00,134][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:28:05,140][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:28:10,145][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:28:15,154][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:28:20,156][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:28:25,157][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:28:30,161][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:28:35,166][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:28:40,170][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:28:45,174][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:28:50,175][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:28:55,176][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:29:00,178][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:29:05,179][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:29:10,181][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:29:15,183][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:29:20,186][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:29:25,187][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:29:30,189][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:29:35,192][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:29:40,195][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:29:45,197][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:29:50,199][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:29:55,201][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:30:00,202][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:30:05,204][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:30:10,209][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:30:15,210][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:30:20,212][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:30:25,214][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:30:30,216][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:30:35,219][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:30:40,220][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:30:45,222][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:30:50,224][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:30:55,225][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:31:00,226][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:31:05,228][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:31:10,229][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:31:15,232][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:31:20,235][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:31:25,236][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:31:30,237][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:31:35,239][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:31:40,243][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:31:45,244][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:31:50,245][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:31:55,246][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:32:00,248][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:32:05,251][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:32:10,252][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:32:14,421][ ERROR][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 14:32:15,253][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:32:20,256][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:32:24,422][WARNING][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-25 14:32:25,258][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:32:30,259][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:32:34,423][ INFO][PID:1565584] Downloading the builder-live.log file, attempt 2 [2025-08-25 14:32:34,426][ INFO][PID:1565584] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@18.233.168.104 copr-rpmbuild-log [2025-08-25 14:32:35,261][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:32:40,262][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:32:45,264][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:32:50,266][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:32:55,268][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:33:00,270][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:33:05,272][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:33:10,274][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:33:15,277][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:33:20,280][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:33:25,283][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:33:30,286][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:33:34,495][ ERROR][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 14:33:35,288][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:33:40,292][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:33:44,497][WARNING][PID:1565584] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-25 14:33:45,293][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:33:50,295][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:33:54,498][ INFO][PID:1565584] Downloading the builder-live.log file, attempt 3 [2025-08-25 14:33:54,500][ INFO][PID:1565584] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@18.233.168.104 copr-rpmbuild-log [2025-08-25 14:33:55,305][ INFO][PID:1565584] Periodic builder liveness probe: alive [2025-08-25 14:33:55,306][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:34:00,308][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:34:05,313][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:34:10,315][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:34:15,317][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:34:20,318][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:34:25,319][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:34:30,321][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:34:35,323][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:34:40,325][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:34:45,326][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:34:50,331][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:34:54,574][ ERROR][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 14:34:55,334][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:35:00,336][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:35:04,578][WARNING][PID:1565584] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-25 14:35:05,338][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:35:10,339][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:35:14,581][ INFO][PID:1565584] Downloading the builder-live.log file, attempt 4 [2025-08-25 14:35:14,587][ INFO][PID:1565584] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@18.233.168.104 copr-rpmbuild-log [2025-08-25 14:35:15,342][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:35:20,344][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:35:25,349][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:35:30,351][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:35:35,352][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:35:40,354][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:35:45,356][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:35:50,357][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:35:55,358][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:36:00,362][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:36:05,364][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:36:10,366][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:36:14,667][ ERROR][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 14:36:15,369][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:36:20,371][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:36:24,670][ ERROR][PID:1565584] Giving up for unstable SSH, failures: 1756132344.4227693, 1756132424.497434, 1756132504.5782523, 1756132584.670608 [2025-08-25 14:36:24,671][ ERROR][PID:1565584] Re-try request for task on 'ResallocHost, ticket_id=4708306, hostname=18.233.168.104, name=aws_x86_64_spot_prod_06411970_20250825_141739, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-08-25 14:36:24,672][ INFO][PID:1565584] Releasing VM back to pool [2025-08-25 14:36:24,695][ INFO][PID:1565584] Retry #2 (on other host) [2025-08-25 14:36:24,696][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:36:24,701][ INFO][PID:1565584] VM allocation process starts [2025-08-25 14:36:24,723][ INFO][PID:1565584] Trying to allocate VM: ResallocHost, ticket_id=4708787, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-25 14:36:29,727][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:36:34,728][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:36:39,731][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:36:44,732][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:36:49,733][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:36:54,734][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:36:59,735][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:37:04,736][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:37:09,741][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:37:14,745][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:37:19,747][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:37:24,748][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:37:29,749][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:37:34,753][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:37:39,754][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:37:44,755][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:37:49,757][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:37:54,760][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:37:59,763][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:38:04,767][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:38:09,769][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:38:14,771][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:38:19,773][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:38:24,774][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:38:29,775][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:38:34,777][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:38:39,778][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:38:44,779][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:38:49,780][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:38:54,781][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:38:59,783][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:39:04,788][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:39:09,789][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:39:14,791][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:39:19,793][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:39:24,794][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:39:29,795][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:39:34,796][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:39:39,798][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:39:44,800][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:39:49,801][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:39:54,807][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:39:59,808][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:40:04,809][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:40:09,812][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:40:14,815][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:40:19,817][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:40:24,818][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:40:29,820][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:40:34,821][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:40:39,822][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:40:44,824][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:40:49,825][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:40:54,826][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:40:59,827][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:41:04,829][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:41:09,830][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:41:14,831][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:41:19,832][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:41:24,833][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:41:29,834][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:41:34,835][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:41:39,837][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:41:42,954][ INFO][PID:1565584] Allocated host ResallocHost, ticket_id=4708787, hostname=54.80.72.37, name=aws_x86_64_spot_prod_06412959_20250825_143925, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-25 14:41:42,955][ INFO][PID:1565584] Allocating ssh connection to builder [2025-08-25 14:41:42,955][ INFO][PID:1565584] Checking that builder machine is OK [2025-08-25 14:41:43,366][ INFO][PID:1565584] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-08-25 14:41:43,566][ INFO][PID:1565584] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-08-25 14:41:43,567][ INFO][PID:1565584] Filling build.info file with builder info [2025-08-25 14:41:43,568][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:41:43,569][ INFO][PID:1565584] Not re-notifying FE and msg buses for the new host. [2025-08-25 14:41:43,571][ INFO][PID:1565584] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9473314-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-08-25 14:41:43,905][ INFO][PID:1565584] Downloading the builder-live.log file, attempt 1 [2025-08-25 14:41:43,906][ INFO][PID:1565584] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.80.72.37 copr-rpmbuild-log [2025-08-25 14:41:48,915][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:41:53,916][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:41:58,917][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:42:03,919][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:42:08,921][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:42:13,922][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:42:18,923][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:42:23,925][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:42:28,926][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:42:33,927][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:42:38,928][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:42:43,929][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:42:48,931][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:42:53,932][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:42:58,938][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:43:03,940][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:43:08,941][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:43:13,945][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:43:18,946][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:43:23,948][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:43:28,950][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:43:33,953][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:43:38,955][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:43:43,958][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:43:48,959][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:43:53,961][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:43:58,968][ INFO][PID:1565584] Periodic builder liveness probe: alive [2025-08-25 14:43:58,969][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:44:03,970][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:44:08,971][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:44:13,973][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:44:18,974][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:44:23,975][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:44:28,976][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:44:33,979][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:44:38,980][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:44:43,981][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:44:48,982][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:44:53,984][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:44:58,985][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:45:03,986][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:45:08,988][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:45:13,989][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:45:18,990][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:45:23,991][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:45:28,993][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:45:33,994][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:45:38,995][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:45:43,997][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:45:48,999][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:45:54,000][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:45:59,001][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:46:04,002][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:46:09,003][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:46:14,005][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:46:19,006][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:46:24,008][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:46:29,009][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:46:34,010][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:46:39,012][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:46:44,013][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:46:49,014][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:46:54,015][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:46:59,016][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:47:04,017][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:47:09,019][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:47:14,020][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:47:19,021][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:47:24,022][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:47:29,024][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:47:34,026][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:47:39,031][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:47:44,036][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:47:49,038][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:47:54,039][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:47:59,040][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:48:04,041][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:48:09,042][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:48:14,044][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:48:19,045][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:48:24,046][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:48:29,048][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:48:34,049][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:48:39,051][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:48:44,052][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:48:49,053][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:48:54,055][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:48:59,056][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:49:04,057][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:49:09,058][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:49:14,059][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:49:19,060][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:49:24,062][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:49:29,063][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:49:34,064][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:49:39,065][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:49:44,066][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:49:49,067][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:49:54,069][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:49:59,071][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:50:04,073][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:50:09,078][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:50:14,079][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:50:19,080][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:50:24,081][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:50:29,083][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:50:34,084][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:50:39,086][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:50:44,089][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:50:49,095][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:50:54,096][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:50:59,097][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:51:04,098][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:51:09,101][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:51:14,102][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:51:19,104][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:51:24,112][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:51:29,113][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:51:34,114][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:51:39,116][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:51:44,117][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:51:49,118][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:51:54,119][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:51:59,120][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:52:04,121][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:52:09,122][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:52:13,409][ ERROR][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 14:52:14,124][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:52:19,126][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:52:23,413][WARNING][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-25 14:52:24,127][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:52:29,128][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:52:33,414][ INFO][PID:1565584] Downloading the builder-live.log file, attempt 2 [2025-08-25 14:52:33,416][ INFO][PID:1565584] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.80.72.37 copr-rpmbuild-log [2025-08-25 14:52:34,133][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:52:39,138][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:52:44,139][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:52:49,140][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:52:54,141][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:52:59,142][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:53:04,145][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:53:09,147][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:53:14,149][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:53:19,150][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:53:24,151][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:53:29,152][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:53:33,487][ ERROR][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 14:53:34,153][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:53:39,154][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:53:43,488][WARNING][PID:1565584] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-25 14:53:44,155][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:53:49,157][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:53:53,489][ INFO][PID:1565584] Downloading the builder-live.log file, attempt 3 [2025-08-25 14:53:53,490][ INFO][PID:1565584] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.80.72.37 copr-rpmbuild-log [2025-08-25 14:53:54,158][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:53:59,169][ INFO][PID:1565584] Periodic builder liveness probe: alive [2025-08-25 14:53:59,169][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:54:04,170][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:54:09,172][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:54:14,173][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:54:19,175][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:54:24,176][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:54:29,177][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:54:34,178][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:54:39,179][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:54:44,181][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:54:49,182][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:54:53,560][ ERROR][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 14:54:54,183][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:54:59,186][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:55:03,563][WARNING][PID:1565584] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-25 14:55:04,189][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:55:09,190][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:55:13,564][ INFO][PID:1565584] Downloading the builder-live.log file, attempt 4 [2025-08-25 14:55:13,567][ INFO][PID:1565584] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.80.72.37 copr-rpmbuild-log [2025-08-25 14:55:14,191][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:55:19,192][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:55:24,193][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:55:29,195][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:55:34,196][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:55:39,197][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:55:44,198][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:55:49,200][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:55:54,201][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:55:59,206][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:56:04,209][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:56:09,214][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:56:13,642][ ERROR][PID:1565584] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 14:56:14,215][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:56:19,216][ INFO][PID:1565584] Checking for cancel request [2025-08-25 14:56:23,645][ ERROR][PID:1565584] Giving up for unstable SSH, failures: 1756133543.4134517, 1756133623.488537, 1756133703.5631967, 1756133783.64518 [2025-08-25 14:56:23,646][ ERROR][PID:1565584] Re-try request for task on 'ResallocHost, ticket_id=4708787, hostname=54.80.72.37, name=aws_x86_64_spot_prod_06412959_20250825_143925, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-08-25 14:56:23,647][ INFO][PID:1565584] Releasing VM back to pool [2025-08-25 14:56:23,660][ ERROR][PID:1565584] Backend process error: Three host tried without success: {'44.201.137.88', '18.233.168.104', '54.80.72.37'} [2025-08-25 14:56:23,660][WARNING][PID:1565584] Switching not-finished job state to 'failed' [2025-08-25 14:56:23,661][ INFO][PID:1565584] Worker failed build, took 880.0909581184387 [2025-08-25 14:56:23,662][ INFO][PID:1565584] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "alternateved", "project_name": "bleeding-emacs", "project_dirname": "bleeding-emacs", "submitter": "alternateved", "ended_on": 1756133783.6607726, "started_on": 1756132903.5698144, "submitted_on": null, "status": 0, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9473314-fedora-rawhide-x86_64", "build_id": 9473314, "package_name": "emacs", "package_version": "1:30.2.50-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/alternateved/bleeding-emacs/emacs", "git_hash": "be4b44e7b4753436aeeeb7ac730956f2d064b2ae", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "alternateved/bleeding-emacs--alternateved", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/alternateved/bleeding-emacs/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/alternateved/bleeding-emacs", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/alternateved/bleeding-emacs", "result_dir": "09473314-emacs", "built_packages": "", "id": 9473314, "mockchain_macros": { "copr_username": "alternateved", "copr_projectname": "bleeding-emacs", "vendor": "Fedora Project COPR (alternateved/bleeding-emacs)" } } ] } [2025-08-25 14:56:23,690][ INFO][PID:1565584] Sending fedora-messaging bus message in build.end [2025-08-25 14:56:23,714][ INFO][PID:1565584] Compressing /var/lib/copr/public_html/results/alternateved/bleeding-emacs/fedora-rawhide-x86_64/09473314-emacs/builder-live.log by gzip [2025-08-25 14:56:23,716][ INFO][PID:1565584] Running command 'gzip /var/lib/copr/public_html/results/alternateved/bleeding-emacs/fedora-rawhide-x86_64/09473314-emacs/builder-live.log' as PID 2026632 [2025-08-25 14:56:23,717][ INFO][PID:1565584] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/alternateved/bleeding-emacs/fedora-rawhide-x86_64/09473314-emacs/builder-live.log) [2025-08-25 14:56:23,718][ INFO][PID:1565584] Compressing /var/lib/copr/public_html/results/alternateved/bleeding-emacs/fedora-rawhide-x86_64/09473314-emacs/backend.log by gzip [2025-08-25 14:56:23,719][ INFO][PID:1565584] Running command 'gzip /var/lib/copr/public_html/results/alternateved/bleeding-emacs/fedora-rawhide-x86_64/09473314-emacs/backend.log' as PID 2026634