[2025-08-25 15:28:08,087][ INFO][PID:2266549] Marking build as starting [2025-08-25 15:28:08,138][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:28:08,139][ INFO][PID:2266549] VM allocation process starts [2025-08-25 15:28:08,193][ INFO][PID:2266549] Trying to allocate VM: ResallocHost, ticket_id=4709969, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-25 15:28:11,211][ INFO][PID:2266549] Allocated host ResallocHost, ticket_id=4709969, hostname=54.89.230.7, name=aws_x86_64_spot_prod_06414057_20250825_150213, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-25 15:28:11,211][ INFO][PID:2266549] Allocating ssh connection to builder [2025-08-25 15:28:11,212][ INFO][PID:2266549] Checking that builder machine is OK [2025-08-25 15:28:11,697][ INFO][PID:2266549] Running remote command: copr-builder-ready fedora-eln-x86_64 [2025-08-25 15:28:11,901][ INFO][PID:2266549] Red Hat subscription not needed for fedora-eln-x86_64 Builder is ready to be used [2025-08-25 15:28:11,902][ INFO][PID:2266549] Filling build.info file with builder info [2025-08-25 15:28:11,902][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:28:11,903][ INFO][PID:2266549] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@CoreOS", "project_name": "continuous", "project_dirname": "continuous", "submitter": "https://api.github.com/users/cgwalters", "ended_on": null, "started_on": 1756135691.9036984, "submitted_on": null, "status": 3, "chroot": "fedora-eln-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9478301-fedora-eln-x86_64", "build_id": 9478301, "package_name": "rpm-ostree", "package_version": "2025.10.5.g2646a78b-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@CoreOS/continuous/rpm-ostree", "git_hash": "7ecc4ee542e311e0994caebea02c8e2aa8ac3e18", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@CoreOS/continuous--https://api.github.com/users/cgwalters", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@CoreOS/continuous/fedora-eln-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc/fedora-eln-x86_64/", "id": "copr_rhcontainerbot_bootc", "name": "Additional repo copr_rhcontainerbot_bootc" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@CoreOS/continuous", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@CoreOS/continuous", "result_dir": "09478301-rpm-ostree", "built_packages": "", "id": 9478301, "mockchain_macros": { "copr_username": "@CoreOS", "copr_projectname": "continuous", "vendor": "Fedora Project COPR (@CoreOS/continuous)" } } ] } [2025-08-25 15:28:11,958][ INFO][PID:2266549] Sending fedora-messaging bus message in build.start [2025-08-25 15:28:12,677][ INFO][PID:2266549] Sending fedora-messaging bus message in chroot.start [2025-08-25 15:28:12,704][ INFO][PID:2266549] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9478301-fedora-eln-x86_64 --chroot fedora-eln-x86_64 --detached [2025-08-25 15:28:13,047][ INFO][PID:2266549] Downloading the builder-live.log file, attempt 1 [2025-08-25 15:28:13,048][ INFO][PID:2266549] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.89.230.7 copr-rpmbuild-log [2025-08-25 15:28:18,061][ INFO][PID:2266549] Periodic builder liveness probe: alive [2025-08-25 15:28:18,062][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:28:23,063][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:28:28,064][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:28:33,066][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:28:38,067][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:28:43,068][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:28:48,069][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:28:53,071][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:28:58,072][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:29:03,073][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:29:08,074][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:29:13,075][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:29:18,077][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:29:23,078][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:29:28,081][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:29:33,082][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:29:38,083][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:29:43,084][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:29:48,085][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:29:53,087][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:29:58,089][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:30:03,093][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:30:08,095][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:30:13,096][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:30:18,097][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:30:23,098][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:30:28,099][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:30:33,103][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:30:38,108][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:30:43,109][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:30:48,110][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:30:53,113][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:30:58,115][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:31:03,117][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:31:08,119][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:31:13,120][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:31:18,121][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:31:23,123][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:31:28,124][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:31:33,126][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:31:38,128][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:31:43,129][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:31:48,134][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:31:53,136][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:31:58,139][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:32:03,140][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:32:08,145][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:32:13,146][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:32:18,150][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:32:23,151][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:32:28,153][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:32:33,158][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:32:38,159][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:32:43,160][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:32:48,162][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:32:53,163][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:32:58,164][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:33:03,165][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:33:08,167][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:33:13,168][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:33:18,170][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:33:23,171][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:33:28,172][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:33:33,175][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:33:38,176][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:33:43,178][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:33:48,179][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:33:53,181][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:33:58,182][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:34:03,183][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:34:08,184][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:34:13,185][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:34:15,709][ ERROR][PID:2266549] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 15:34:18,186][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:34:23,187][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:34:25,710][WARNING][PID:2266549] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-25 15:34:28,189][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:34:33,190][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:34:35,711][ INFO][PID:2266549] Downloading the builder-live.log file, attempt 2 [2025-08-25 15:34:35,712][ INFO][PID:2266549] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.89.230.7 copr-rpmbuild-log [2025-08-25 15:34:35,723][ ERROR][PID:2266549] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 15:34:38,192][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:34:43,193][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:34:45,727][WARNING][PID:2266549] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-25 15:34:48,195][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:34:53,196][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:34:55,727][ INFO][PID:2266549] Downloading the builder-live.log file, attempt 3 [2025-08-25 15:34:55,729][ INFO][PID:2266549] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.89.230.7 copr-rpmbuild-log [2025-08-25 15:34:55,743][ ERROR][PID:2266549] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 15:34:58,201][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:35:03,202][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:35:05,746][WARNING][PID:2266549] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-25 15:35:08,203][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:35:13,204][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:35:15,747][ INFO][PID:2266549] Downloading the builder-live.log file, attempt 4 [2025-08-25 15:35:15,749][ INFO][PID:2266549] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.89.230.7 copr-rpmbuild-log [2025-08-25 15:35:18,206][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:35:23,208][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:35:28,212][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:35:33,220][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:35:38,222][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:35:43,223][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:35:48,225][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:35:53,226][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:35:58,227][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:36:03,229][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:36:08,232][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:36:13,233][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:36:15,819][ ERROR][PID:2266549] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-25 15:36:18,234][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:36:23,235][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:36:25,821][ ERROR][PID:2266549] Giving up for unstable SSH, failures: 1756136065.7101328, 1756136085.7270477, 1756136105.74681, 1756136185.820866 [2025-08-25 15:36:25,822][ ERROR][PID:2266549] Re-try request for task on 'ResallocHost, ticket_id=4709969, hostname=54.89.230.7, name=aws_x86_64_spot_prod_06414057_20250825_150213, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-08-25 15:36:25,823][ INFO][PID:2266549] Releasing VM back to pool [2025-08-25 15:36:25,835][ INFO][PID:2266549] Retry #1 (on other host) [2025-08-25 15:36:25,836][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:36:25,837][ INFO][PID:2266549] VM allocation process starts [2025-08-25 15:36:25,850][ INFO][PID:2266549] Trying to allocate VM: ResallocHost, ticket_id=4710157, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-25 15:36:28,863][ INFO][PID:2266549] Allocated host ResallocHost, ticket_id=4710157, hostname=34.227.190.188, name=aws_x86_64_normal_prod_06414523_20250825_152200, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-25 15:36:28,863][ INFO][PID:2266549] Allocating ssh connection to builder [2025-08-25 15:36:28,864][ INFO][PID:2266549] Checking that builder machine is OK [2025-08-25 15:36:29,335][ INFO][PID:2266549] Running remote command: copr-builder-ready fedora-eln-x86_64 [2025-08-25 15:36:29,554][ INFO][PID:2266549] Red Hat subscription not needed for fedora-eln-x86_64 Builder is ready to be used [2025-08-25 15:36:29,554][ INFO][PID:2266549] Filling build.info file with builder info [2025-08-25 15:36:29,558][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:36:29,559][ INFO][PID:2266549] Not re-notifying FE and msg buses for the new host. [2025-08-25 15:36:29,559][ INFO][PID:2266549] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9478301-fedora-eln-x86_64 --chroot fedora-eln-x86_64 --detached [2025-08-25 15:36:29,904][ INFO][PID:2266549] Downloading the builder-live.log file, attempt 1 [2025-08-25 15:36:29,906][ INFO][PID:2266549] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.227.190.188 copr-rpmbuild-log [2025-08-25 15:36:34,907][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:36:39,916][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:36:44,918][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:36:49,919][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:36:54,927][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:36:59,929][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:37:04,933][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:37:09,935][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:37:14,936][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:37:19,938][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:37:24,939][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:37:29,941][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:37:34,942][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:37:39,944][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:37:44,949][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:37:49,950][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:37:54,951][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:37:59,952][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:38:04,954][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:38:09,958][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:38:14,959][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:38:19,969][ INFO][PID:2266549] Periodic builder liveness probe: alive [2025-08-25 15:38:19,969][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:38:24,970][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:38:29,972][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:38:34,974][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:38:39,975][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:38:44,976][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:38:49,977][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:38:54,978][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:38:59,980][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:39:04,981][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:39:09,983][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:39:14,984][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:39:19,985][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:39:24,989][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:39:29,991][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:39:34,992][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:39:39,993][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:39:44,994][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:39:49,996][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:39:54,997][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:39:59,999][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:40:05,001][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:40:10,003][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:40:15,004][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:40:20,005][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:40:25,006][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:40:30,007][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:40:35,011][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:40:40,013][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:40:45,015][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:40:50,017][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:40:55,018][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:41:00,019][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:41:05,020][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:41:10,021][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:41:15,022][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:41:20,023][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:41:25,025][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:41:30,027][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:41:35,028][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:41:40,029][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:41:45,030][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:41:50,031][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:41:55,033][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:42:00,035][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:42:05,036][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:42:10,037][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:42:15,039][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:42:20,040][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:42:25,042][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:42:30,043][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:42:35,046][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:42:40,048][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:42:45,051][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:42:50,053][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:42:55,055][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:43:00,056][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:43:05,058][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:43:10,060][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:43:15,061][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:43:20,064][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:43:25,066][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:43:30,069][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:43:35,070][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:43:40,072][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:43:45,076][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:43:50,077][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:43:55,078][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:44:00,080][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:44:05,085][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:44:10,086][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:44:15,089][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:44:20,092][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:44:25,094][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:44:30,096][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:44:35,100][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:44:40,101][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:44:45,102][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:44:50,104][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:44:55,105][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:45:00,107][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:45:05,109][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:45:10,111][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:45:15,112][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:45:20,113][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:45:25,115][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:45:30,116][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:45:35,118][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:45:40,119][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:45:45,120][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:45:50,121][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:45:55,123][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:46:00,124][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:46:05,125][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:46:10,126][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:46:15,127][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:46:20,128][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:46:25,129][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:46:30,130][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:46:35,132][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:46:40,133][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:46:45,134][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:46:50,135][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:46:55,136][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:47:00,137][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:47:05,138][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:47:10,139][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:47:15,140][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:47:20,142][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:47:25,144][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:47:30,146][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:47:35,147][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:47:40,149][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:47:45,150][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:47:50,151][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:47:55,152][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:48:00,155][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:48:05,160][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:48:10,162][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:48:15,163][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:48:20,173][ INFO][PID:2266549] Periodic builder liveness probe: alive [2025-08-25 15:48:20,173][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:48:25,174][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:48:30,176][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:48:35,178][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:48:40,179][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:48:45,180][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:48:50,183][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:48:55,185][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:49:00,187][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:49:05,189][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:49:10,192][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:49:15,193][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:49:20,196][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:49:25,199][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:49:30,201][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:49:35,203][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:49:40,206][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:49:45,210][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:49:50,212][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:49:55,214][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:50:00,215][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:50:05,218][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:50:10,223][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:50:15,225][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:50:20,228][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:50:25,229][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:50:30,234][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:50:35,236][ INFO][PID:2266549] Checking for cancel request [2025-08-25 15:50:37,509][ INFO][PID:2266549] Downloading results from builder [2025-08-25 15:50:37,511][ INFO][PID:2266549] rsyncing of mockbuilder@34.227.190.188:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree started [2025-08-25 15:50:37,515][ INFO][PID:2266549] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@34.227.190.188:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/ &> /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/build-09478301.rsync.log [2025-08-25 15:50:38,559][ INFO][PID:2266549] rsyncing finished. [2025-08-25 15:50:38,560][ INFO][PID:2266549] Releasing VM back to pool [2025-08-25 15:50:38,583][ INFO][PID:2266549] Searching for 'success' file in resultdir [2025-08-25 15:50:38,585][ INFO][PID:2266549] Going to sign pkgs from source: 9478301-fedora-eln-x86_64 in chroot: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64 [2025-08-25 15:50:38,586][ INFO][PID:2266549] Calling '/bin/sign -u @CoreOS#continuous@copr.fedorahosted.org -p' (attempt #1) [2025-08-25 15:50:38,926][ INFO][PID:2266549] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-debugsource-2025.10.5.g2646a78b-1.eln151.x86_64.rpm' (attempt #1) [2025-08-25 15:50:40,012][ INFO][PID:2266549] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-debugsource-2025.10.5.g2646a78b-1.eln151.x86_64.rpm [2025-08-25 15:50:40,012][ INFO][PID:2266549] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-debuginfo-2025.10.5.g2646a78b-1.eln151.x86_64.rpm' (attempt #1) [2025-08-25 15:50:41,894][ INFO][PID:2266549] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-debuginfo-2025.10.5.g2646a78b-1.eln151.x86_64.rpm [2025-08-25 15:50:41,896][ INFO][PID:2266549] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-libs-debuginfo-2025.10.5.g2646a78b-1.eln151.x86_64.rpm' (attempt #1) [2025-08-25 15:50:42,801][ INFO][PID:2266549] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-libs-debuginfo-2025.10.5.g2646a78b-1.eln151.x86_64.rpm [2025-08-25 15:50:42,802][ INFO][PID:2266549] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-2025.10.5.g2646a78b-1.eln151.x86_64.rpm' (attempt #1) [2025-08-25 15:50:43,828][ INFO][PID:2266549] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-2025.10.5.g2646a78b-1.eln151.x86_64.rpm [2025-08-25 15:50:43,829][ INFO][PID:2266549] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-libs-2025.10.5.g2646a78b-1.eln151.x86_64.rpm' (attempt #1) [2025-08-25 15:50:44,583][ INFO][PID:2266549] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-libs-2025.10.5.g2646a78b-1.eln151.x86_64.rpm [2025-08-25 15:50:44,584][ INFO][PID:2266549] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-devel-2025.10.5.g2646a78b-1.eln151.x86_64.rpm' (attempt #1) [2025-08-25 15:50:45,337][ INFO][PID:2266549] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-devel-2025.10.5.g2646a78b-1.eln151.x86_64.rpm [2025-08-25 15:50:45,338][ INFO][PID:2266549] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-2025.10.5.g2646a78b-1.eln151.src.rpm' (attempt #1) [2025-08-25 15:50:46,628][ INFO][PID:2266549] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/rpm-ostree-2025.10.5.g2646a78b-1.eln151.src.rpm [2025-08-25 15:50:46,634][ INFO][PID:2266549] Sign done [2025-08-25 15:50:46,635][ INFO][PID:2266549] Incremental createrepo run, adding 09478301-rpm-ostree into https://download.copr.fedorainfracloud.org/results/@CoreOS/continuous/fedora-eln-x86_64, (auto-create-repo=True) [2025-08-25 15:50:46,641][ INFO][PID:2266549] Running command 'copr-repo --batched /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64 --add 09478301-rpm-ostree --no-appstream-metadata' as PID 2456202 [2025-08-25 15:50:48,726][ INFO][PID:2266549] Finished after 2 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64 --add 09478301-rpm-ostree --no-appstream-metadata) [2025-08-25 15:50:48,727][ INFO][PID:2266549] Getting build details [2025-08-25 15:50:48,727][ INFO][PID:2266549] Listing built binary packages in /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree [2025-08-25 15:50:48,728][ INFO][PID:2266549] Built packages: ['rpm-ostree 2025.10.5.g2646a78b', 'rpm-ostree-debugsource 2025.10.5.g2646a78b', 'rpm-ostree-debuginfo 2025.10.5.g2646a78b', 'rpm-ostree-devel 2025.10.5.g2646a78b', 'rpm-ostree-libs-debuginfo 2025.10.5.g2646a78b', 'rpm-ostree-libs 2025.10.5.g2646a78b'] [2025-08-25 15:50:48,729][ INFO][PID:2266549] build details: {'built_packages': 'rpm-ostree 2025.10.5.g2646a78b\nrpm-ostree-debugsource 2025.10.5.g2646a78b\nrpm-ostree-debuginfo 2025.10.5.g2646a78b\nrpm-ostree-devel 2025.10.5.g2646a78b\nrpm-ostree-libs-debuginfo 2025.10.5.g2646a78b\nrpm-ostree-libs 2025.10.5.g2646a78b'} [2025-08-25 15:50:48,730][ INFO][PID:2266549] Retrieving pubkey [2025-08-25 15:50:48,730][ INFO][PID:2266549] Calling '/bin/sign -u @CoreOS#continuous@copr.fedorahosted.org -p' (attempt #1) [2025-08-25 15:50:49,066][ INFO][PID:2266549] Added pubkey for user @CoreOS project continuous into: /var/lib/copr/public_html/results/@CoreOS/continuous/pubkey.gpg [2025-08-25 15:50:49,067][ INFO][PID:2266549] Finished build: id=9478301 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@CoreOS/continuous chroot=fedora-eln-x86_64 [2025-08-25 15:50:49,069][ INFO][PID:2266549] Worker succeeded build, took 859.5101408958435 [2025-08-25 15:50:49,070][ INFO][PID:2266549] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@CoreOS", "project_name": "continuous", "project_dirname": "continuous", "submitter": "https://api.github.com/users/cgwalters", "ended_on": 1756137049.069525, "started_on": 1756136189.559384, "submitted_on": null, "status": 1, "chroot": "fedora-eln-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9478301-fedora-eln-x86_64", "build_id": 9478301, "package_name": "rpm-ostree", "package_version": "2025.10.5.g2646a78b-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@CoreOS/continuous/rpm-ostree", "git_hash": "7ecc4ee542e311e0994caebea02c8e2aa8ac3e18", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@CoreOS/continuous--https://api.github.com/users/cgwalters", "results": { "packages": [ { "name": "rpm-ostree", "epoch": null, "version": "2025.10.5.g2646a78b", "release": "1.eln151", "arch": "x86_64" }, { "name": "rpm-ostree-debugsource", "epoch": null, "version": "2025.10.5.g2646a78b", "release": "1.eln151", "arch": "x86_64" }, { "name": "rpm-ostree-debuginfo", "epoch": null, "version": "2025.10.5.g2646a78b", "release": "1.eln151", "arch": "x86_64" }, { "name": "rpm-ostree-devel", "epoch": null, "version": "2025.10.5.g2646a78b", "release": "1.eln151", "arch": "x86_64" }, { "name": "rpm-ostree", "epoch": null, "version": "2025.10.5.g2646a78b", "release": "1.eln151", "arch": "src" }, { "name": "rpm-ostree-libs-debuginfo", "epoch": null, "version": "2025.10.5.g2646a78b", "release": "1.eln151", "arch": "x86_64" }, { "name": "rpm-ostree-libs", "epoch": null, "version": "2025.10.5.g2646a78b", "release": "1.eln151", "arch": "x86_64" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@CoreOS/continuous/fedora-eln-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc/fedora-eln-x86_64/", "id": "copr_rhcontainerbot_bootc", "name": "Additional repo copr_rhcontainerbot_bootc" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@CoreOS/continuous", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@CoreOS/continuous", "result_dir": "09478301-rpm-ostree", "built_packages": "rpm-ostree 2025.10.5.g2646a78b\nrpm-ostree-debugsource 2025.10.5.g2646a78b\nrpm-ostree-debuginfo 2025.10.5.g2646a78b\nrpm-ostree-devel 2025.10.5.g2646a78b\nrpm-ostree-libs-debuginfo 2025.10.5.g2646a78b\nrpm-ostree-libs 2025.10.5.g2646a78b", "id": 9478301, "mockchain_macros": { "copr_username": "@CoreOS", "copr_projectname": "continuous", "vendor": "Fedora Project COPR (@CoreOS/continuous)" } } ] } [2025-08-25 15:50:49,130][ INFO][PID:2266549] Sending fedora-messaging bus message in build.end [2025-08-25 15:50:49,157][ INFO][PID:2266549] Compressing /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/builder-live.log by gzip [2025-08-25 15:50:49,158][ INFO][PID:2266549] Running command 'gzip /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/builder-live.log' as PID 2456666 [2025-08-25 15:50:49,196][ INFO][PID:2266549] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/builder-live.log) [2025-08-25 15:50:49,196][ INFO][PID:2266549] Compressing /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/backend.log by gzip [2025-08-25 15:50:49,198][ INFO][PID:2266549] Running command 'gzip /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-eln-x86_64/09478301-rpm-ostree/backend.log' as PID 2456677