[2025-10-07 19:42:39,927][ INFO][PID:3750569] Marking build as starting [2025-10-07 19:42:40,227][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:42:40,228][ INFO][PID:3750569] VM allocation process starts [2025-10-07 19:42:40,297][ INFO][PID:3750569] Trying to allocate VM: ResallocHost, ticket_id=5532518, requested_tags=['copr_builder', 'arch_x86_64'] [2025-10-07 19:42:45,299][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:42:46,373][ INFO][PID:3750569] Allocated host ResallocHost, ticket_id=5532518, hostname=44.192.68.139, name=aws_x86_64_spot_prod_07267514_20251007_194118, requested_tags=['copr_builder', 'arch_x86_64'] [2025-10-07 19:42:46,374][ INFO][PID:3750569] Allocating ssh connection to builder [2025-10-07 19:42:46,375][ INFO][PID:3750569] Checking that builder machine is OK [2025-10-07 19:42:46,727][ INFO][PID:3750569] Running remote command: copr-builder-ready fedora-42-x86_64 [2025-10-07 19:42:46,955][ INFO][PID:3750569] Red Hat subscription not needed for fedora-42-x86_64 Builder is ready to be used [2025-10-07 19:42:46,956][ INFO][PID:3750569] Filling build.info file with builder info [2025-10-07 19:42:46,956][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:42:46,961][ INFO][PID:3750569] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@CoreOS", "project_name": "continuous", "project_dirname": "continuous", "submitter": "https://api.github.com/users/cgwalters", "ended_on": null, "started_on": 1759866166.9608035, "submitted_on": null, "status": 3, "chroot": "fedora-42-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9666242-fedora-42-x86_64", "build_id": 9666242, "package_name": "rpm-ostree", "package_version": "2025.11.12.gca4a8b07-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@CoreOS/continuous/rpm-ostree", "git_hash": "7b89c28d2751d885a3ffd66c5fab426e20bead0f", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@CoreOS/continuous--https://api.github.com/users/cgwalters", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@CoreOS/continuous/fedora-42-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc/fedora-42-x86_64/", "id": "copr_rhcontainerbot_bootc", "name": "Additional repo copr_rhcontainerbot_bootc" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@CoreOS/continuous", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@CoreOS/continuous", "result_dir": "09666242-rpm-ostree", "built_packages": "", "id": 9666242, "mockchain_macros": { "copr_username": "@CoreOS", "copr_projectname": "continuous", "vendor": "Fedora Project COPR (@CoreOS/continuous)" } } ] } [2025-10-07 19:42:47,050][ INFO][PID:3750569] Sending fedora-messaging bus message in build.start [2025-10-07 19:42:48,147][ INFO][PID:3750569] Sending fedora-messaging bus message in chroot.start [2025-10-07 19:42:48,198][ INFO][PID:3750569] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9666242-fedora-42-x86_64 --chroot fedora-42-x86_64 --detached [2025-10-07 19:42:48,562][ INFO][PID:3750569] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '44.192.68.139' (ED25519) to the list of known hosts. [2025-10-07 19:42:48,563][ INFO][PID:3750569] Downloading the builder-live.log file, attempt 1 [2025-10-07 19:42:48,565][ INFO][PID:3750569] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.192.68.139 copr-rpmbuild-log [2025-10-07 19:42:53,572][ INFO][PID:3750569] Periodic builder liveness probe: alive [2025-10-07 19:42:53,572][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:42:58,573][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:43:03,574][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:43:08,576][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:43:13,578][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:43:18,579][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:43:23,582][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:43:28,583][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:43:33,584][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:43:38,585][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:43:43,586][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:43:48,587][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:43:53,589][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:43:58,590][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:44:03,591][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:44:08,593][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:44:13,595][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:44:18,596][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:44:23,597][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:44:28,600][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:44:33,601][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:44:38,603][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:44:43,605][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:44:48,615][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:44:53,618][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:44:58,619][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:45:03,620][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:45:08,621][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:45:13,622][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:45:18,623][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:45:23,624][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:45:28,625][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:45:33,626][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:45:38,627][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:45:43,628][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:45:48,630][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:45:53,631][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:45:58,632][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:46:03,633][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:46:08,634][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:46:13,635][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:46:18,637][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:46:23,638][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:46:28,639][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:46:33,640][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:46:38,642][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:46:43,643][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:46:48,644][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:46:53,645][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:46:58,646][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:47:03,647][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:47:08,648][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:47:13,649][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:47:18,651][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:47:23,652][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:47:28,654][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:47:33,655][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:47:38,657][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:47:43,658][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:47:48,659][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:47:53,660][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:47:58,661][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:48:03,662][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:48:08,665][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:48:13,665][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:48:18,666][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:48:23,669][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:48:28,450][ ERROR][PID:3750569] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-10-07 19:48:28,670][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:48:33,671][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:48:38,451][WARNING][PID:3750569] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-10-07 19:48:38,672][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:48:43,673][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:48:48,452][ INFO][PID:3750569] Downloading the builder-live.log file, attempt 2 [2025-10-07 19:48:48,453][ INFO][PID:3750569] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.192.68.139 copr-rpmbuild-log [2025-10-07 19:48:48,674][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:48:53,675][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:48:58,676][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:49:03,677][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:49:08,678][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:49:13,679][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:49:18,680][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:49:23,681][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:49:28,682][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:49:33,683][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:49:38,684][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:49:43,685][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:49:48,522][ ERROR][PID:3750569] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-10-07 19:49:48,686][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:49:53,689][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:49:58,525][WARNING][PID:3750569] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-10-07 19:49:58,690][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:50:03,691][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:50:08,525][ INFO][PID:3750569] Downloading the builder-live.log file, attempt 3 [2025-10-07 19:50:08,527][ INFO][PID:3750569] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.192.68.139 copr-rpmbuild-log [2025-10-07 19:50:08,692][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:50:13,693][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:50:18,694][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:50:23,695][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:50:28,696][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:50:33,698][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:50:38,699][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:50:43,702][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:50:48,704][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:50:53,705][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:50:58,706][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:51:03,707][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:51:08,595][ ERROR][PID:3750569] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-10-07 19:51:08,708][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:51:13,709][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:51:18,599][WARNING][PID:3750569] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-10-07 19:51:18,710][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:51:23,711][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:51:28,599][ INFO][PID:3750569] Downloading the builder-live.log file, attempt 4 [2025-10-07 19:51:28,601][ INFO][PID:3750569] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.192.68.139 copr-rpmbuild-log [2025-10-07 19:51:28,712][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:51:33,713][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:51:38,714][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:51:43,718][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:51:48,719][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:51:53,720][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:51:58,721][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:52:03,722][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:52:08,723][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:52:13,725][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:52:18,726][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:52:23,728][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:52:28,668][ ERROR][PID:3750569] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-10-07 19:52:28,729][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:52:33,730][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:52:38,671][ ERROR][PID:3750569] Giving up for unstable SSH, failures: 1759866518.451344, 1759866598.525223, 1759866678.5992265, 1759866758.6715293 [2025-10-07 19:52:38,672][ ERROR][PID:3750569] Re-try request for task on 'ResallocHost, ticket_id=5532518, hostname=44.192.68.139, name=aws_x86_64_spot_prod_07267514_20251007_194118, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-10-07 19:52:38,672][ INFO][PID:3750569] Releasing VM back to pool [2025-10-07 19:52:38,684][ INFO][PID:3750569] Retry #1 (on other host) [2025-10-07 19:52:38,684][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:52:38,685][ INFO][PID:3750569] VM allocation process starts [2025-10-07 19:52:38,697][ INFO][PID:3750569] Trying to allocate VM: ResallocHost, ticket_id=5532705, requested_tags=['copr_builder', 'arch_x86_64'] [2025-10-07 19:52:43,699][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:52:44,761][ INFO][PID:3750569] Allocated host ResallocHost, ticket_id=5532705, hostname=54.196.117.109, name=aws_x86_64_normalreserved_prod_07267641_20251007_195112, requested_tags=['copr_builder', 'arch_x86_64'] [2025-10-07 19:52:44,762][ INFO][PID:3750569] Allocating ssh connection to builder [2025-10-07 19:52:44,762][ INFO][PID:3750569] Checking that builder machine is OK [2025-10-07 19:52:45,097][ INFO][PID:3750569] Running remote command: copr-builder-ready fedora-42-x86_64 [2025-10-07 19:52:45,294][ INFO][PID:3750569] Red Hat subscription not needed for fedora-42-x86_64 Builder is ready to be used [2025-10-07 19:52:45,295][ INFO][PID:3750569] Filling build.info file with builder info [2025-10-07 19:52:45,295][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:52:45,296][ INFO][PID:3750569] Not re-notifying FE and msg buses for the new host. [2025-10-07 19:52:45,297][ INFO][PID:3750569] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9666242-fedora-42-x86_64 --chroot fedora-42-x86_64 --detached [2025-10-07 19:52:45,628][ INFO][PID:3750569] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '54.196.117.109' (ED25519) to the list of known hosts. [2025-10-07 19:52:45,629][ INFO][PID:3750569] Downloading the builder-live.log file, attempt 1 [2025-10-07 19:52:45,630][ INFO][PID:3750569] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.196.117.109 copr-rpmbuild-log [2025-10-07 19:52:50,629][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:52:55,640][ INFO][PID:3750569] Periodic builder liveness probe: alive [2025-10-07 19:52:55,641][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:53:00,642][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:53:05,643][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:53:10,644][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:53:15,645][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:53:20,646][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:53:25,647][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:53:30,652][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:53:35,653][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:53:40,654][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:53:45,655][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:53:50,656][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:53:55,657][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:54:00,658][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:54:05,659][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:54:10,660][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:54:15,661][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:54:20,662][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:54:25,663][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:54:30,664][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:54:35,665][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:54:40,666][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:54:45,667][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:54:50,669][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:54:55,670][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:55:00,671][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:55:05,672][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:55:10,673][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:55:15,674][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:55:20,675][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:55:25,676][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:55:30,677][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:55:35,678][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:55:40,679][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:55:45,681][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:55:50,682][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:55:55,683][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:56:00,687][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:56:05,688][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:56:10,689][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:56:15,690][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:56:20,692][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:56:25,694][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:56:30,695][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:56:35,696][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:56:40,697][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:56:45,698][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:56:50,699][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:56:55,700][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:57:00,702][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:57:05,706][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:57:10,707][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:57:15,708][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:57:20,709][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:57:25,711][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:57:30,712][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:57:35,713][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:57:40,714][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:57:45,715][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:57:50,716][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:57:55,717][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:58:00,718][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:58:05,719][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:58:10,720][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:58:15,721][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:58:20,722][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:58:25,723][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:58:30,724][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:58:35,725][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:58:40,726][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:58:45,728][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:58:50,729][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:58:55,730][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:59:00,733][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:59:05,738][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:59:10,739][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:59:15,740][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:59:20,741][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:59:25,742][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:59:30,744][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:59:35,747][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:59:40,749][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:59:45,750][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:59:50,751][ INFO][PID:3750569] Checking for cancel request [2025-10-07 19:59:55,752][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:00:00,754][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:00:05,758][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:00:10,759][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:00:15,760][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:00:20,761][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:00:25,763][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:00:30,764][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:00:35,765][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:00:40,768][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:00:45,769][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:00:50,770][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:00:55,771][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:01:00,772][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:01:05,775][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:01:10,777][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:01:15,778][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:01:20,779][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:01:25,780][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:01:30,781][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:01:35,783][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:01:40,785][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:01:45,787][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:01:50,788][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:01:55,789][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:02:00,790][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:02:05,791][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:02:10,792][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:02:15,793][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:02:20,794][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:02:25,795][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:02:30,797][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:02:35,798][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:02:40,799][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:02:45,800][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:02:50,800][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:02:55,809][ INFO][PID:3750569] Periodic builder liveness probe: alive [2025-10-07 20:02:55,810][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:03:00,811][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:03:05,812][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:03:10,813][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:03:15,814][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:03:20,815][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:03:25,816][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:03:30,817][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:03:35,818][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:03:40,819][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:03:45,821][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:03:50,822][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:03:55,823][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:04:00,824][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:04:05,825][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:04:10,826][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:04:15,827][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:04:20,828][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:04:25,829][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:04:30,830][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:04:35,831][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:04:40,833][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:04:45,834][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:04:50,836][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:04:55,837][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:05:00,838][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:05:05,839][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:05:10,841][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:05:15,842][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:05:20,843][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:05:25,844][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:05:30,845][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:05:35,846][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:05:40,847][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:05:45,848][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:05:50,849][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:05:55,850][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:06:00,851][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:06:05,852][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:06:10,854][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:06:15,855][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:06:20,856][ INFO][PID:3750569] Checking for cancel request [2025-10-07 20:06:23,089][ INFO][PID:3750569] Downloading results from builder [2025-10-07 20:06:23,090][ INFO][PID:3750569] rsyncing of mockbuilder@54.196.117.109:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree started [2025-10-07 20:06:23,091][ INFO][PID:3750569] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@54.196.117.109:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/ &> /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/build-09666242.rsync.log [2025-10-07 20:06:23,979][ INFO][PID:3750569] rsyncing finished. [2025-10-07 20:06:23,980][ INFO][PID:3750569] Releasing VM back to pool [2025-10-07 20:06:24,040][ INFO][PID:3750569] Searching for 'success' file in resultdir [2025-10-07 20:06:24,042][ INFO][PID:3750569] Going to sign pkgs from source: 9666242-fedora-42-x86_64 in chroot: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64 [2025-10-07 20:06:24,043][ INFO][PID:3750569] Calling '/bin/sign -u @CoreOS#continuous@copr.fedorahosted.org -p' (attempt #1) [2025-10-07 20:06:24,231][ INFO][PID:3750569] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-libs-2025.11.12.gca4a8b07-1.fc42.x86_64.rpm' (attempt #1) [2025-10-07 20:06:24,855][ INFO][PID:3750569] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-libs-2025.11.12.gca4a8b07-1.fc42.x86_64.rpm [2025-10-07 20:06:24,856][ INFO][PID:3750569] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-2025.11.12.gca4a8b07-1.fc42.x86_64.rpm' (attempt #1) [2025-10-07 20:06:25,394][ INFO][PID:3750569] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-2025.11.12.gca4a8b07-1.fc42.x86_64.rpm [2025-10-07 20:06:25,395][ INFO][PID:3750569] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-debuginfo-2025.11.12.gca4a8b07-1.fc42.x86_64.rpm' (attempt #1) [2025-10-07 20:06:26,555][ INFO][PID:3750569] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-debuginfo-2025.11.12.gca4a8b07-1.fc42.x86_64.rpm [2025-10-07 20:06:26,555][ INFO][PID:3750569] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-debugsource-2025.11.12.gca4a8b07-1.fc42.x86_64.rpm' (attempt #1) [2025-10-07 20:06:27,429][ INFO][PID:3750569] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-debugsource-2025.11.12.gca4a8b07-1.fc42.x86_64.rpm [2025-10-07 20:06:27,430][ INFO][PID:3750569] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-2025.11.12.gca4a8b07-1.fc42.src.rpm' (attempt #1) [2025-10-07 20:06:28,439][ INFO][PID:3750569] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-2025.11.12.gca4a8b07-1.fc42.src.rpm [2025-10-07 20:06:28,440][ INFO][PID:3750569] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-devel-2025.11.12.gca4a8b07-1.fc42.x86_64.rpm' (attempt #1) [2025-10-07 20:06:28,884][ INFO][PID:3750569] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-devel-2025.11.12.gca4a8b07-1.fc42.x86_64.rpm [2025-10-07 20:06:28,885][ INFO][PID:3750569] Calling '/bin/sign -4 -h sha256 -u @CoreOS#continuous@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-libs-debuginfo-2025.11.12.gca4a8b07-1.fc42.x86_64.rpm' (attempt #1) [2025-10-07 20:06:29,492][ INFO][PID:3750569] signed rpm: /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/rpm-ostree-libs-debuginfo-2025.11.12.gca4a8b07-1.fc42.x86_64.rpm [2025-10-07 20:06:29,493][ INFO][PID:3750569] Sign done [2025-10-07 20:06:29,494][ INFO][PID:3750569] Incremental createrepo run, adding 09666242-rpm-ostree into https://download.copr.fedorainfracloud.org/results/@CoreOS/continuous/fedora-42-x86_64, (auto-create-repo=True) [2025-10-07 20:06:29,495][ INFO][PID:3750569] Running command 'copr-repo --batched /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64 --add 09666242-rpm-ostree --no-appstream-metadata' as PID 3870039 [2025-10-07 20:06:30,511][ INFO][PID:3750569] Finished after 1 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64 --add 09666242-rpm-ostree --no-appstream-metadata) [2025-10-07 20:06:30,512][ INFO][PID:3750569] Getting build details [2025-10-07 20:06:30,512][ INFO][PID:3750569] Listing built binary packages in /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree [2025-10-07 20:06:30,513][ INFO][PID:3750569] Built packages: ['rpm-ostree-devel 2025.11.12.gca4a8b07', 'rpm-ostree 2025.11.12.gca4a8b07', 'rpm-ostree-libs 2025.11.12.gca4a8b07', 'rpm-ostree-libs-debuginfo 2025.11.12.gca4a8b07', 'rpm-ostree-debuginfo 2025.11.12.gca4a8b07', 'rpm-ostree-debugsource 2025.11.12.gca4a8b07'] [2025-10-07 20:06:30,514][ INFO][PID:3750569] build details: {'built_packages': 'rpm-ostree-devel 2025.11.12.gca4a8b07\nrpm-ostree 2025.11.12.gca4a8b07\nrpm-ostree-libs 2025.11.12.gca4a8b07\nrpm-ostree-libs-debuginfo 2025.11.12.gca4a8b07\nrpm-ostree-debuginfo 2025.11.12.gca4a8b07\nrpm-ostree-debugsource 2025.11.12.gca4a8b07'} [2025-10-07 20:06:30,514][ INFO][PID:3750569] Retrieving pubkey [2025-10-07 20:06:30,514][ INFO][PID:3750569] Calling '/bin/sign -u @CoreOS#continuous@copr.fedorahosted.org -p' (attempt #1) [2025-10-07 20:06:30,862][ INFO][PID:3750569] Added pubkey for user @CoreOS project continuous into: /var/lib/copr/public_html/results/@CoreOS/continuous/pubkey.gpg [2025-10-07 20:06:30,862][ INFO][PID:3750569] Finished build: id=9666242 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@CoreOS/continuous chroot=fedora-42-x86_64 [2025-10-07 20:06:30,864][ INFO][PID:3750569] Worker succeeded build, took 825.5681846141815 [2025-10-07 20:06:30,865][ INFO][PID:3750569] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@CoreOS", "project_name": "continuous", "project_dirname": "continuous", "submitter": "https://api.github.com/users/cgwalters", "ended_on": 1759867590.8647728, "started_on": 1759866765.2965882, "submitted_on": null, "status": 1, "chroot": "fedora-42-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9666242-fedora-42-x86_64", "build_id": 9666242, "package_name": "rpm-ostree", "package_version": "2025.11.12.gca4a8b07-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@CoreOS/continuous/rpm-ostree", "git_hash": "7b89c28d2751d885a3ffd66c5fab426e20bead0f", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@CoreOS/continuous--https://api.github.com/users/cgwalters", "results": { "packages": [ { "name": "rpm-ostree-devel", "epoch": null, "version": "2025.11.12.gca4a8b07", "release": "1.fc42", "arch": "x86_64" }, { "name": "rpm-ostree", "epoch": null, "version": "2025.11.12.gca4a8b07", "release": "1.fc42", "arch": "x86_64" }, { "name": "rpm-ostree-libs", "epoch": null, "version": "2025.11.12.gca4a8b07", "release": "1.fc42", "arch": "x86_64" }, { "name": "rpm-ostree-libs-debuginfo", "epoch": null, "version": "2025.11.12.gca4a8b07", "release": "1.fc42", "arch": "x86_64" }, { "name": "rpm-ostree-debuginfo", "epoch": null, "version": "2025.11.12.gca4a8b07", "release": "1.fc42", "arch": "x86_64" }, { "name": "rpm-ostree-debugsource", "epoch": null, "version": "2025.11.12.gca4a8b07", "release": "1.fc42", "arch": "x86_64" }, { "name": "rpm-ostree", "epoch": null, "version": "2025.11.12.gca4a8b07", "release": "1.fc42", "arch": "src" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@CoreOS/continuous/fedora-42-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc/fedora-42-x86_64/", "id": "copr_rhcontainerbot_bootc", "name": "Additional repo copr_rhcontainerbot_bootc" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@CoreOS/continuous", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@CoreOS/continuous", "result_dir": "09666242-rpm-ostree", "built_packages": "rpm-ostree-devel 2025.11.12.gca4a8b07\nrpm-ostree 2025.11.12.gca4a8b07\nrpm-ostree-libs 2025.11.12.gca4a8b07\nrpm-ostree-libs-debuginfo 2025.11.12.gca4a8b07\nrpm-ostree-debuginfo 2025.11.12.gca4a8b07\nrpm-ostree-debugsource 2025.11.12.gca4a8b07", "id": 9666242, "mockchain_macros": { "copr_username": "@CoreOS", "copr_projectname": "continuous", "vendor": "Fedora Project COPR (@CoreOS/continuous)" } } ] } [2025-10-07 20:06:30,923][ INFO][PID:3750569] Sending fedora-messaging bus message in build.end [2025-10-07 20:06:30,950][ INFO][PID:3750569] Compressing /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/builder-live.log by gzip [2025-10-07 20:06:30,951][ INFO][PID:3750569] Running command 'gzip /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/builder-live.log' as PID 3870250 [2025-10-07 20:06:30,979][ INFO][PID:3750569] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/builder-live.log) [2025-10-07 20:06:30,979][ INFO][PID:3750569] Compressing /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/backend.log by gzip [2025-10-07 20:06:30,980][ INFO][PID:3750569] Running command 'gzip /var/lib/copr/public_html/results/@CoreOS/continuous/fedora-42-x86_64/09666242-rpm-ostree/backend.log' as PID 3870251