[2026-01-30 14:28:37,705][ INFO][PID:1108900] Marking build as starting [2026-01-30 14:28:37,767][ INFO][PID:1108900] Checking that /var/lib/copr/public_html/results/ksurma/sphinx-9-control/fedora-rawhide-x86_64/repodata/repomd.xml exists [2026-01-30 14:28:37,769][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:28:37,770][ INFO][PID:1108900] VM allocation process starts [2026-01-30 14:28:37,789][ INFO][PID:1108900] Trying to allocate VM: ResallocHost, ticket_id=1104215, requested_tags=['copr_builder', 'arch_x86_64'] [2026-01-30 14:28:40,805][ INFO][PID:1108900] Allocated host ResallocHost, ticket_id=1104215, hostname=52.90.9.204, name=aws_x86_64_spot_prod_01774927_20260130_141948, requested_tags=['copr_builder', 'arch_x86_64'] [2026-01-30 14:28:40,805][ INFO][PID:1108900] Allocating ssh connection to builder [2026-01-30 14:28:40,806][ INFO][PID:1108900] Checking that builder machine is OK [2026-01-30 14:28:41,130][ INFO][PID:1108900] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2026-01-30 14:28:41,328][ INFO][PID:1108900] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2026-01-30 14:28:41,328][ INFO][PID:1108900] Filling build.info file with builder info [2026-01-30 14:28:41,329][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:28:41,330][ INFO][PID:1108900] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "ksurma", "project_name": "sphinx-9-control", "project_dirname": "sphinx-9-control", "submitter": "ksurma", "ended_on": null, "started_on": 1769783321.3299515, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "10078997-fedora-rawhide-x86_64", "build_id": 10078997, "package_name": "ghc9.8", "package_version": "9.8.4-17", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/ksurma/sphinx-9-control/ghc9.8", "git_hash": "90395f50ef9599f178bd173d0c3c32a9166748fb", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "ksurma/sphinx-9-control--ksurma", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/ksurma/sphinx-9-control/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/", "id": "https_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch", "name": "Additional repo https_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/ksurma/sphinx-9-control", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/ksurma/sphinx-9-control", "result_dir": "10078997-ghc9.8", "built_packages": "", "id": 10078997, "mockchain_macros": { "copr_username": "ksurma", "copr_projectname": "sphinx-9-control", "vendor": "Fedora Project COPR (ksurma/sphinx-9-control)" } } ] } [2026-01-30 14:28:41,388][ INFO][PID:1108900] Sending fedora-messaging bus message in build.start [2026-01-30 14:28:41,956][ INFO][PID:1108900] Sending fedora-messaging bus message in chroot.start [2026-01-30 14:28:41,985][ INFO][PID:1108900] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10078997-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2026-01-30 14:28:42,291][ INFO][PID:1108900] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '52.90.9.204' (ED25519) to the list of known hosts. [2026-01-30 14:28:42,292][ INFO][PID:1108900] Downloading the builder-live.log file, attempt 1 [2026-01-30 14:28:42,294][ INFO][PID:1108900] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.90.9.204 copr-rpmbuild-log [2026-01-30 14:28:47,301][ INFO][PID:1108900] Periodic builder liveness probe: alive [2026-01-30 14:28:47,302][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:28:52,302][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:28:57,303][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:29:02,304][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:29:07,305][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:29:12,306][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:29:17,307][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:29:22,308][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:29:27,337][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:29:32,338][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:29:37,340][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:29:42,342][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:29:47,343][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:29:52,344][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:29:57,346][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:30:02,347][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:30:07,349][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:30:12,350][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:30:17,351][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:30:22,352][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:30:27,353][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:30:32,355][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:30:37,377][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:30:42,378][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:30:47,379][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:30:52,380][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:30:57,381][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:31:02,382][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:31:07,383][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:31:12,384][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:31:17,386][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:31:22,469][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:31:27,502][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:31:32,503][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:31:37,504][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:31:42,506][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:31:47,507][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:31:52,508][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:31:57,509][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:32:02,510][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:32:07,511][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:32:12,512][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:32:17,514][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:32:22,515][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:32:27,516][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:32:32,518][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:32:37,520][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:32:42,521][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:32:47,769][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:32:53,197][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:32:58,198][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:33:03,200][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:33:08,201][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:33:13,620][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:33:18,621][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:33:23,622][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:33:28,624][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:33:33,625][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:33:38,708][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:33:43,709][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:33:48,710][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:33:53,760][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:33:58,762][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:34:03,764][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:34:08,765][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:34:13,766][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:34:15,557][ ERROR][PID:1108900] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-30 14:34:18,768][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:34:24,112][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:34:25,558][WARNING][PID:1108900] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-30 14:34:29,114][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:34:34,115][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:34:35,559][ INFO][PID:1108900] Downloading the builder-live.log file, attempt 2 [2026-01-30 14:34:35,560][ INFO][PID:1108900] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.90.9.204 copr-rpmbuild-log [2026-01-30 14:34:39,115][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:34:44,116][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:34:49,117][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:34:54,118][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:34:59,119][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:35:04,120][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:35:09,122][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:35:14,123][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:35:19,277][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:35:24,278][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:35:29,279][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:35:34,280][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:35:35,613][ ERROR][PID:1108900] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-30 14:35:39,281][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:35:44,388][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:35:45,622][WARNING][PID:1108900] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-30 14:35:49,389][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:35:54,390][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:35:55,623][ INFO][PID:1108900] Downloading the builder-live.log file, attempt 3 [2026-01-30 14:35:55,624][ INFO][PID:1108900] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.90.9.204 copr-rpmbuild-log [2026-01-30 14:35:59,392][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:36:04,393][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:36:09,394][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:36:14,395][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:36:19,397][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:36:24,398][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:36:29,399][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:36:34,400][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:36:39,401][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:36:44,402][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:36:49,403][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:36:54,404][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:36:55,671][ ERROR][PID:1108900] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-30 14:36:59,405][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:37:04,406][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:37:05,683][WARNING][PID:1108900] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-30 14:37:09,509][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:37:14,513][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:37:15,684][ INFO][PID:1108900] Downloading the builder-live.log file, attempt 4 [2026-01-30 14:37:15,685][ INFO][PID:1108900] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.90.9.204 copr-rpmbuild-log [2026-01-30 14:37:19,514][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:37:24,521][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:37:29,522][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:37:34,523][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:37:39,524][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:37:44,525][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:37:49,526][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:37:54,527][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:37:59,528][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:38:04,637][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:38:09,639][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:38:14,640][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:38:15,748][ ERROR][PID:1108900] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-30 14:38:19,641][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:38:24,643][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:38:25,938][ ERROR][PID:1108900] Giving up for unstable SSH, failures: 1769783665.5583098, 1769783745.6229625, 1769783825.6834471, 1769783905.9383907 [2026-01-30 14:38:25,939][ ERROR][PID:1108900] Re-try request for task on 'ResallocHost, ticket_id=1104215, hostname=52.90.9.204, name=aws_x86_64_spot_prod_01774927_20260130_141948, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-01-30 14:38:25,940][ INFO][PID:1108900] Releasing VM back to pool [2026-01-30 14:38:25,965][ INFO][PID:1108900] Retry #1 (on other host) [2026-01-30 14:38:25,966][ INFO][PID:1108900] Checking that /var/lib/copr/public_html/results/ksurma/sphinx-9-control/fedora-rawhide-x86_64/repodata/repomd.xml exists [2026-01-30 14:38:25,966][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:38:25,967][ INFO][PID:1108900] VM allocation process starts [2026-01-30 14:38:25,978][ INFO][PID:1108900] Trying to allocate VM: ResallocHost, ticket_id=1104327, requested_tags=['copr_builder', 'arch_x86_64'] [2026-01-30 14:38:28,994][ INFO][PID:1108900] Allocated host ResallocHost, ticket_id=1104327, hostname=3.80.238.22, name=aws_x86_64_spot_prod_01775120_20260130_142525, requested_tags=['copr_builder', 'arch_x86_64'] [2026-01-30 14:38:29,338][ INFO][PID:1108900] Allocating ssh connection to builder [2026-01-30 14:38:29,339][ INFO][PID:1108900] Checking that builder machine is OK [2026-01-30 14:38:29,657][ INFO][PID:1108900] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2026-01-30 14:38:29,900][ INFO][PID:1108900] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2026-01-30 14:38:29,901][ INFO][PID:1108900] Filling build.info file with builder info [2026-01-30 14:38:30,026][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:38:30,027][ INFO][PID:1108900] Not re-notifying FE and msg buses for the new host. [2026-01-30 14:38:30,028][ INFO][PID:1108900] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10078997-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2026-01-30 14:38:30,353][ INFO][PID:1108900] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '3.80.238.22' (ED25519) to the list of known hosts. [2026-01-30 14:38:30,354][ INFO][PID:1108900] Downloading the builder-live.log file, attempt 1 [2026-01-30 14:38:30,355][ INFO][PID:1108900] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.80.238.22 copr-rpmbuild-log [2026-01-30 14:38:35,355][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:38:40,356][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:38:45,357][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:38:50,363][ INFO][PID:1108900] Periodic builder liveness probe: alive [2026-01-30 14:38:50,364][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:38:55,365][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:39:00,367][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:39:05,368][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:39:10,372][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:39:15,374][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:39:20,376][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:39:25,377][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:39:30,378][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:39:35,379][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:39:40,380][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:39:45,381][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:39:50,382][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:39:55,383][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:40:00,384][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:40:05,385][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:40:10,387][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:40:15,388][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:40:20,389][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:40:25,390][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:40:30,391][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:40:35,392][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:40:40,394][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:40:45,395][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:40:50,396][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:40:55,397][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:41:00,398][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:41:05,399][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:41:10,400][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:41:15,401][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:41:20,402][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:41:25,403][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:41:30,404][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:41:35,406][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:41:40,407][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:41:45,408][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:41:50,408][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:41:55,410][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:42:00,410][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:42:05,411][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:42:10,412][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:42:15,413][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:42:20,414][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:42:25,415][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:42:30,416][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:42:35,417][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:42:40,418][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:42:45,419][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:42:50,420][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:42:55,421][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:43:00,422][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:43:05,423][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:43:10,424][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:43:15,425][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:43:20,426][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:43:25,427][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:43:30,428][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:43:35,429][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:43:40,430][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:43:45,431][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:43:50,431][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:43:55,432][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:44:00,433][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:44:05,434][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:44:10,435][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:44:15,436][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:44:20,437][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:44:25,438][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:44:30,439][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:44:35,440][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:44:40,442][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:44:45,443][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:44:50,444][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:44:55,445][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:45:00,446][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:45:05,447][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:45:10,447][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:45:15,448][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:45:20,449][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:45:25,450][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:45:30,451][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:45:35,452][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:45:40,453][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:45:45,454][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:45:50,455][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:45:55,456][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:46:00,457][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:46:05,458][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:46:10,459][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:46:15,783][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:46:20,784][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:46:25,786][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:46:30,787][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:46:35,788][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:46:40,790][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:46:45,791][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:46:50,792][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:46:55,794][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:47:00,795][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:47:05,796][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:47:10,797][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:47:15,802][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:47:20,803][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:47:25,804][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:47:30,807][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:47:35,808][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:47:40,810][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:47:45,811][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:47:50,812][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:47:55,813][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:48:00,814][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:48:05,815][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:48:10,816][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:48:15,818][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:48:20,819][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:48:25,821][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:48:30,822][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:48:35,823][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:48:40,824][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:48:45,825][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:48:50,838][ INFO][PID:1108900] Periodic builder liveness probe: alive [2026-01-30 14:48:50,839][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:48:55,840][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:49:00,841][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:49:05,842][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:49:10,843][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:49:15,651][ ERROR][PID:1108900] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-30 14:49:15,844][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:49:20,845][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:49:25,671][WARNING][PID:1108900] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-30 14:49:25,846][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:49:30,847][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:49:35,672][ INFO][PID:1108900] Downloading the builder-live.log file, attempt 2 [2026-01-30 14:49:35,678][ INFO][PID:1108900] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.80.238.22 copr-rpmbuild-log [2026-01-30 14:49:35,848][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:49:40,851][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:49:45,854][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:49:50,860][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:49:55,861][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:50:00,862][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:50:05,863][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:50:10,864][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:50:15,866][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:50:20,867][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:50:25,868][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:50:30,869][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:50:35,767][ ERROR][PID:1108900] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-30 14:50:35,878][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:50:40,879][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:50:45,773][WARNING][PID:1108900] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-30 14:50:45,881][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:50:50,883][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:50:55,774][ INFO][PID:1108900] Downloading the builder-live.log file, attempt 3 [2026-01-30 14:50:55,775][ INFO][PID:1108900] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.80.238.22 copr-rpmbuild-log [2026-01-30 14:50:55,884][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:51:00,885][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:51:05,886][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:51:10,892][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:51:15,893][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:51:20,895][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:51:25,896][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:51:30,897][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:51:35,898][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:51:40,903][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:51:45,904][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:51:50,905][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:51:55,842][ ERROR][PID:1108900] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-30 14:51:55,906][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:52:00,907][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:52:05,843][WARNING][PID:1108900] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-30 14:52:05,932][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:52:10,934][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:52:15,844][ INFO][PID:1108900] Downloading the builder-live.log file, attempt 4 [2026-01-30 14:52:15,852][ INFO][PID:1108900] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.80.238.22 copr-rpmbuild-log [2026-01-30 14:52:15,935][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:52:20,936][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:52:25,938][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:52:30,940][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:52:35,941][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:52:40,986][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:52:45,989][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:52:50,990][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:52:55,991][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:53:00,992][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:53:05,993][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:53:10,994][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:53:15,922][ ERROR][PID:1108900] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-30 14:53:15,996][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:53:20,997][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:53:25,933][ ERROR][PID:1108900] Giving up for unstable SSH, failures: 1769784565.6712797, 1769784645.773469, 1769784725.843387, 1769784805.933696 [2026-01-30 14:53:25,935][ ERROR][PID:1108900] Re-try request for task on 'ResallocHost, ticket_id=1104327, hostname=3.80.238.22, name=aws_x86_64_spot_prod_01775120_20260130_142525, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-01-30 14:53:25,936][ INFO][PID:1108900] Releasing VM back to pool [2026-01-30 14:53:25,947][ INFO][PID:1108900] Retry #2 (on other host) [2026-01-30 14:53:25,948][ INFO][PID:1108900] Checking that /var/lib/copr/public_html/results/ksurma/sphinx-9-control/fedora-rawhide-x86_64/repodata/repomd.xml exists [2026-01-30 14:53:25,948][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:53:25,949][ INFO][PID:1108900] VM allocation process starts [2026-01-30 14:53:25,961][ INFO][PID:1108900] Trying to allocate VM: ResallocHost, ticket_id=1104502, requested_tags=['copr_builder', 'arch_x86_64'] [2026-01-30 14:53:29,014][ INFO][PID:1108900] Allocated host ResallocHost, ticket_id=1104502, hostname=98.82.165.65, name=aws_x86_64_normal_prod_01775085_20260130_142426, requested_tags=['copr_builder', 'arch_x86_64'] [2026-01-30 14:53:29,014][ INFO][PID:1108900] Allocating ssh connection to builder [2026-01-30 14:53:29,015][ INFO][PID:1108900] Checking that builder machine is OK [2026-01-30 14:53:29,397][ INFO][PID:1108900] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2026-01-30 14:53:29,621][ INFO][PID:1108900] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2026-01-30 14:53:29,622][ INFO][PID:1108900] Filling build.info file with builder info [2026-01-30 14:53:29,839][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:53:29,840][ INFO][PID:1108900] Not re-notifying FE and msg buses for the new host. [2026-01-30 14:53:29,840][ INFO][PID:1108900] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10078997-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2026-01-30 14:53:30,222][ INFO][PID:1108900] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '98.82.165.65' (ED25519) to the list of known hosts. [2026-01-30 14:53:30,223][ INFO][PID:1108900] Downloading the builder-live.log file, attempt 1 [2026-01-30 14:53:30,225][ INFO][PID:1108900] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.82.165.65 copr-rpmbuild-log [2026-01-30 14:53:35,224][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:53:40,230][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:53:45,238][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:53:50,239][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:53:55,241][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:54:00,242][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:54:05,243][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:54:10,245][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:54:15,248][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:54:20,249][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:54:25,250][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:54:30,251][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:54:35,252][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:54:40,256][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:54:45,257][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:54:50,258][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:54:55,260][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:55:00,261][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:55:05,262][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:55:10,262][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:55:15,263][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:55:20,264][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:55:25,265][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:55:30,266][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:55:35,267][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:55:40,268][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:55:45,269][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:55:50,270][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:55:55,271][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:56:00,272][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:56:05,273][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:56:10,274][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:56:15,275][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:56:20,276][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:56:25,277][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:56:30,278][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:56:35,279][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:56:40,280][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:56:45,280][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:56:50,282][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:56:55,282][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:57:00,283][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:57:05,284][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:57:10,285][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:57:15,286][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:57:20,287][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:57:25,288][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:57:30,289][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:57:35,291][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:57:40,291][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:57:45,292][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:57:50,293][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:57:55,294][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:58:00,295][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:58:05,296][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:58:10,297][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:58:15,298][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:58:20,299][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:58:25,300][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:58:30,301][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:58:35,302][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:58:40,303][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:58:45,304][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:58:50,304][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:58:55,310][ INFO][PID:1108900] Periodic builder liveness probe: alive [2026-01-30 14:58:55,311][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:59:00,311][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:59:05,312][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:59:10,313][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:59:15,314][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:59:20,315][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:59:25,316][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:59:30,317][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:59:35,318][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:59:40,318][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:59:45,319][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:59:50,320][ INFO][PID:1108900] Checking for cancel request [2026-01-30 14:59:55,321][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:00:00,322][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:00:05,323][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:00:10,324][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:00:15,325][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:00:20,326][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:00:25,327][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:00:30,328][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:00:35,329][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:00:40,329][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:00:45,331][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:00:50,332][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:00:55,333][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:01:00,334][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:01:05,335][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:01:10,356][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:01:15,434][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:01:20,435][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:01:25,436][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:01:30,437][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:01:35,438][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:01:40,439][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:01:45,440][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:01:50,440][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:01:55,441][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:02:00,442][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:02:05,443][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:02:10,444][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:02:15,445][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:02:20,446][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:02:25,447][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:02:30,450][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:02:35,451][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:02:40,452][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:02:45,453][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:02:50,454][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:02:55,455][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:03:00,456][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:03:05,457][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:03:10,458][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:03:15,459][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:03:20,463][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:03:25,464][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:03:30,465][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:03:35,466][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:03:40,467][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:03:45,468][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:03:50,469][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:03:55,471][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:04:00,472][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:04:05,473][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:04:10,474][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:04:15,474][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:04:20,475][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:04:25,476][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:04:30,477][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:04:35,478][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:04:40,479][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:04:45,480][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:04:50,481][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:04:55,483][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:05:00,484][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:05:05,485][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:05:10,486][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:05:15,487][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:05:20,488][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:05:25,489][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:05:30,490][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:05:35,491][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:05:40,492][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:05:45,493][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:05:50,494][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:05:55,495][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:06:00,496][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:06:05,500][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:06:10,505][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:06:15,506][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:06:20,519][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:06:25,520][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:06:30,521][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:06:35,522][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:06:40,523][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:06:45,524][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:06:50,525][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:06:55,526][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:07:00,527][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:07:05,528][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:07:10,530][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:07:15,531][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:07:20,532][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:07:25,535][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:07:30,537][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:07:35,545][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:07:40,546][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:07:45,547][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:07:50,548][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:07:55,549][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:08:00,550][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:08:05,551][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:08:10,552][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:08:15,553][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:08:20,554][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:08:25,555][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:08:30,556][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:08:35,557][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:08:40,558][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:08:45,560][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:08:50,563][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:08:55,571][ INFO][PID:1108900] Periodic builder liveness probe: alive [2026-01-30 15:08:55,572][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:09:00,572][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:09:05,573][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:09:10,574][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:09:15,576][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:09:20,578][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:09:25,579][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:09:30,580][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:09:35,586][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:09:40,589][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:09:45,590][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:09:50,591][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:09:55,593][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:10:00,617][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:10:05,619][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:10:10,621][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:10:15,622][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:10:20,624][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:10:25,628][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:10:30,629][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:10:35,631][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:10:40,635][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:10:45,636][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:10:50,637][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:10:55,639][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:11:00,642][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:11:05,647][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:11:10,649][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:11:15,651][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:11:20,653][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:11:25,657][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:11:30,660][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:11:35,661][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:11:40,666][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:11:45,667][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:11:50,670][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:11:55,672][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:12:00,676][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:12:05,682][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:12:10,684][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:12:15,685][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:12:20,693][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:12:25,696][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:12:30,699][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:12:35,702][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:12:40,703][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:12:45,708][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:12:50,709][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:12:55,711][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:13:00,712][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:13:05,714][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:13:10,715][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:13:15,716][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:13:20,717][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:13:25,719][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:13:30,725][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:13:35,726][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:13:40,728][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:13:45,729][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:13:50,730][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:13:55,731][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:14:00,732][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:14:05,733][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:14:10,734][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:14:15,735][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:14:20,736][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:14:25,737][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:14:30,738][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:14:35,740][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:14:40,742][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:14:45,743][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:14:50,744][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:14:55,745][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:15:00,748][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:15:05,749][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:15:10,750][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:15:15,751][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:15:20,752][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:15:25,977][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:15:30,978][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:15:35,979][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:15:40,980][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:15:45,982][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:15:50,983][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:15:55,984][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:16:00,987][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:16:05,994][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:16:10,995][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:16:15,997][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:16:20,998][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:16:25,999][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:16:30,999][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:16:36,000][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:16:41,001][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:16:46,002][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:16:51,003][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:16:56,004][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:17:01,004][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:17:06,005][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:17:11,006][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:17:16,007][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:17:21,008][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:17:26,009][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:17:31,011][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:17:36,012][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:17:41,013][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:17:46,014][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:17:51,014][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:17:56,015][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:18:01,017][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:18:06,017][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:18:11,018][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:18:16,019][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:18:21,020][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:18:26,021][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:18:31,022][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:18:36,023][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:18:41,025][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:18:46,027][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:18:51,029][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:18:56,040][ INFO][PID:1108900] Periodic builder liveness probe: alive [2026-01-30 15:18:56,043][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:19:01,045][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:19:06,048][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:19:11,050][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:19:16,051][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:19:21,052][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:19:26,053][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:19:31,056][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:19:36,057][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:19:41,059][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:19:46,061][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:19:51,063][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:19:56,064][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:20:01,065][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:20:06,066][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:20:11,067][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:20:16,068][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:20:21,070][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:20:26,071][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:20:31,072][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:20:36,073][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:20:41,074][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:20:46,373][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:20:51,374][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:20:56,375][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:21:01,376][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:21:06,377][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:21:11,461][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:21:16,714][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:21:21,715][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:21:26,716][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:21:31,717][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:21:36,718][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:21:41,719][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:21:46,721][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:21:51,723][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:21:56,724][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:22:01,725][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:22:06,726][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:22:11,729][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:22:16,730][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:22:21,731][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:22:26,732][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:22:31,733][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:22:36,734][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:22:41,734][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:22:46,735][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:22:51,743][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:22:56,744][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:23:01,746][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:23:06,747][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:23:11,759][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:23:16,760][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:23:21,761][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:23:26,762][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:23:31,763][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:23:36,766][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:23:41,767][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:23:46,768][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:23:51,769][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:23:56,770][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:24:01,773][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:24:06,774][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:24:11,775][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:24:16,776][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:24:21,777][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:24:26,829][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:24:31,830][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:24:36,833][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:24:41,834][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:24:46,835][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:24:51,836][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:24:56,837][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:25:01,838][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:25:06,839][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:25:11,842][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:25:16,843][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:25:21,844][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:25:26,845][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:25:31,894][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:25:36,895][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:25:41,896][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:25:46,897][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:25:51,898][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:25:56,899][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:26:01,900][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:26:06,901][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:26:11,902][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:26:16,903][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:26:21,904][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:26:26,905][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:26:31,905][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:26:36,906][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:26:41,907][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:26:46,908][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:26:51,909][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:26:56,910][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:27:01,914][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:27:06,915][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:27:11,916][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:27:16,917][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:27:21,918][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:27:26,919][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:27:31,919][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:27:36,920][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:27:41,921][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:27:46,922][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:27:51,948][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:27:56,950][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:28:01,951][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:28:06,952][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:28:11,953][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:28:16,955][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:28:21,956][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:28:26,957][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:28:31,958][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:28:36,959][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:28:41,960][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:28:46,961][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:28:51,962][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:28:56,970][ INFO][PID:1108900] Periodic builder liveness probe: alive [2026-01-30 15:28:56,970][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:29:01,972][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:29:06,973][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:29:11,974][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:29:16,975][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:29:21,976][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:29:26,976][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:29:31,977][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:29:36,979][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:29:41,980][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:29:46,981][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:29:51,982][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:29:56,983][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:30:01,984][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:30:06,986][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:30:11,986][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:30:16,988][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:30:21,989][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:30:26,992][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:30:31,992][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:30:36,994][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:30:41,995][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:30:46,996][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:30:51,997][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:30:57,160][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:31:02,162][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:31:07,162][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:31:12,164][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:31:17,164][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:31:22,165][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:31:27,166][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:31:32,167][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:31:37,168][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:31:42,170][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:31:47,171][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:31:52,172][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:31:57,173][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:32:02,174][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:32:07,175][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:32:12,176][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:32:17,177][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:32:22,178][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:32:27,179][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:32:32,180][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:32:37,181][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:32:42,267][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:32:47,268][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:32:52,269][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:32:57,270][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:33:02,271][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:33:07,280][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:33:12,282][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:33:17,283][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:33:22,285][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:33:27,286][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:33:32,294][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:33:37,295][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:33:42,296][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:33:47,297][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:33:52,298][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:33:57,299][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:34:02,300][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:34:07,301][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:34:12,302][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:34:17,304][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:34:22,307][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:34:27,309][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:34:32,310][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:34:37,311][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:34:42,313][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:34:47,314][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:34:52,459][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:34:57,469][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:35:02,482][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:35:07,493][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:35:12,495][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:35:17,496][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:35:22,497][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:35:27,498][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:35:32,505][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:35:37,512][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:35:42,513][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:35:47,516][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:35:52,520][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:35:57,529][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:36:02,534][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:36:07,535][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:36:12,538][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:36:17,539][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:36:22,540][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:36:27,546][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:36:32,548][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:36:37,551][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:36:42,555][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:36:47,560][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:36:52,561][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:36:57,562][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:37:02,563][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:37:07,602][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:37:12,694][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:37:17,695][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:37:22,696][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:37:27,697][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:37:32,699][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:37:37,700][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:37:42,703][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:37:47,710][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:37:52,711][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:37:57,713][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:38:02,716][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:38:07,717][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:38:12,718][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:38:17,719][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:38:22,720][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:38:27,723][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:38:32,724][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:38:37,725][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:38:42,727][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:38:47,729][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:38:52,731][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:38:57,742][ INFO][PID:1108900] Periodic builder liveness probe: alive [2026-01-30 15:38:57,743][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:39:02,744][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:39:07,745][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:39:12,747][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:39:17,748][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:39:22,828][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:39:27,830][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:39:32,832][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:39:37,833][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:39:42,834][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:39:47,835][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:39:52,836][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:39:57,837][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:40:02,839][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:40:07,840][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:40:12,841][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:40:17,842][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:40:22,843][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:40:27,845][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:40:32,848][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:40:37,850][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:40:42,851][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:40:47,852][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:40:53,173][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:40:58,176][ INFO][PID:1108900] Checking for cancel request [2026-01-30 15:40:59,535][ INFO][PID:1108900] Downloading results from builder [2026-01-30 15:40:59,536][ INFO][PID:1108900] rsyncing of mockbuilder@98.82.165.65:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/ksurma/sphinx-9-control/fedora-rawhide-x86_64/10078997-ghc9.8 started [2026-01-30 15:40:59,537][ INFO][PID:1108900] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@98.82.165.65:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/ksurma/sphinx-9-control/fedora-rawhide-x86_64/10078997-ghc9.8/ &> /var/lib/copr/public_html/results/ksurma/sphinx-9-control/fedora-rawhide-x86_64/10078997-ghc9.8/build-10078997.rsync.log [2026-01-30 15:41:00,317][ INFO][PID:1108900] rsyncing finished. [2026-01-30 15:41:00,318][ INFO][PID:1108900] Releasing VM back to pool [2026-01-30 15:41:00,334][ INFO][PID:1108900] Searching for 'success' file in resultdir [2026-01-30 15:41:00,335][ ERROR][PID:1108900] Build failed: Backend process error: No success file => build failure [2026-01-30 15:41:00,337][ INFO][PID:1108900] Finished build: id=10078997 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/ksurma/sphinx-9-control chroot=fedora-rawhide-x86_64 [2026-01-30 15:41:00,340][ ERROR][PID:1108900] Backend process error: No success file => build failure [2026-01-30 15:41:00,341][ INFO][PID:1108900] Worker failed build, took 2850.500951051712 [2026-01-30 15:41:00,342][ INFO][PID:1108900] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "ksurma", "project_name": "sphinx-9-control", "project_dirname": "sphinx-9-control", "submitter": "ksurma", "ended_on": 1769787660.3410912, "started_on": 1769784809.84014, "submitted_on": null, "status": 0, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "10078997-fedora-rawhide-x86_64", "build_id": 10078997, "package_name": "ghc9.8", "package_version": "9.8.4-17", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/ksurma/sphinx-9-control/ghc9.8", "git_hash": "90395f50ef9599f178bd173d0c3c32a9166748fb", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "ksurma/sphinx-9-control--ksurma", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/ksurma/sphinx-9-control/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/", "id": "https_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch", "name": "Additional repo https_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/ksurma/sphinx-9-control", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/ksurma/sphinx-9-control", "result_dir": "10078997-ghc9.8", "built_packages": "", "id": 10078997, "mockchain_macros": { "copr_username": "ksurma", "copr_projectname": "sphinx-9-control", "vendor": "Fedora Project COPR (ksurma/sphinx-9-control)" } } ] } [2026-01-30 15:41:00,396][ INFO][PID:1108900] Sending fedora-messaging bus message in build.end [2026-01-30 15:41:00,429][ INFO][PID:1108900] Compressing /var/lib/copr/public_html/results/ksurma/sphinx-9-control/fedora-rawhide-x86_64/10078997-ghc9.8/builder-live.log by gzip [2026-01-30 15:41:00,430][ INFO][PID:1108900] Running command 'gzip /var/lib/copr/public_html/results/ksurma/sphinx-9-control/fedora-rawhide-x86_64/10078997-ghc9.8/builder-live.log' as PID 1502381 [2026-01-30 15:41:00,646][ INFO][PID:1108900] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/ksurma/sphinx-9-control/fedora-rawhide-x86_64/10078997-ghc9.8/builder-live.log) [2026-01-30 15:41:00,647][ INFO][PID:1108900] Compressing /var/lib/copr/public_html/results/ksurma/sphinx-9-control/fedora-rawhide-x86_64/10078997-ghc9.8/backend.log by gzip [2026-01-30 15:41:00,648][ INFO][PID:1108900] Running command 'gzip /var/lib/copr/public_html/results/ksurma/sphinx-9-control/fedora-rawhide-x86_64/10078997-ghc9.8/backend.log' as PID 1502427