[2025-07-22 18:54:53,867][ INFO][PID:3321565] Marking build as starting [2025-07-22 18:54:53,889][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:54:53,891][ INFO][PID:3321565] VM allocation process starts [2025-07-22 18:54:53,905][ INFO][PID:3321565] Trying to allocate VM: ResallocHost, ticket_id=4035936, requested_tags=['copr_builder', 'arch_x86_64'] [2025-07-22 18:54:56,921][ INFO][PID:3321565] Allocated host ResallocHost, ticket_id=4035936, hostname=44.200.66.197, name=aws_x86_64_spot_prod_05735765_20250722_185128, requested_tags=['copr_builder', 'arch_x86_64'] [2025-07-22 18:54:56,922][ INFO][PID:3321565] Allocating ssh connection to builder [2025-07-22 18:54:56,923][ INFO][PID:3321565] Checking that builder machine is OK [2025-07-22 18:54:57,360][ INFO][PID:3321565] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-07-22 18:54:57,565][ INFO][PID:3321565] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-07-22 18:54:57,566][ INFO][PID:3321565] Filling build.info file with builder info [2025-07-22 18:54:57,573][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:54:57,574][ INFO][PID:3321565] Sending build state back to frontend: { "builds": [ { "timeout": 115200, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "fberat", "project_name": "autoconf.2.72-6", "project_dirname": "autoconf.2.72-6", "submitter": "fberat", "ended_on": null, "started_on": 1753210497.5741985, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9301041-fedora-rawhide-x86_64", "build_id": 9301041, "package_name": "rpm-ostree", "package_version": "2025.8-2", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/fberat/autoconf.2.72-6/rpm-ostree", "git_hash": "ae68e63672d54ce2e47da5eb6d4fd7813408ad26", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "fberat/autoconf.2.72-6--fberat", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/fberat/autoconf.2.72-6", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/fberat/autoconf.2.72-6", "result_dir": "09301041-rpm-ostree", "built_packages": "", "id": 9301041, "mockchain_macros": { "copr_username": "fberat", "copr_projectname": "autoconf.2.72-6", "vendor": "Fedora Project COPR (fberat/autoconf.2.72-6)" } } ] } [2025-07-22 18:54:57,595][ INFO][PID:3321565] Sending fedora-messaging bus message in build.start [2025-07-22 18:54:58,679][ INFO][PID:3321565] Sending fedora-messaging bus message in chroot.start [2025-07-22 18:54:58,746][ INFO][PID:3321565] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9301041-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-07-22 18:54:59,072][ INFO][PID:3321565] Downloading the builder-live.log file, attempt 1 [2025-07-22 18:54:59,073][ INFO][PID:3321565] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.200.66.197 copr-rpmbuild-log [2025-07-22 18:55:04,082][ INFO][PID:3321565] Periodic builder liveness probe: alive [2025-07-22 18:55:04,083][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:55:09,084][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:55:14,086][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:55:19,088][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:55:24,089][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:55:29,091][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:55:34,092][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:55:39,093][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:55:44,096][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:55:49,097][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:55:54,098][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:55:59,099][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:56:04,100][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:56:09,101][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:56:14,102][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:56:19,103][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:56:24,105][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:56:29,105][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:56:34,106][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:56:39,107][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:56:44,120][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:56:49,121][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:56:54,123][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:56:59,124][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:57:04,125][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:57:09,126][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:57:14,127][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:57:19,129][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:57:24,131][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:57:29,134][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:57:34,138][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:57:39,142][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:57:44,143][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:57:49,145][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:57:54,146][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:57:59,147][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:58:04,148][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:58:09,149][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:58:14,151][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:58:19,152][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:58:24,154][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:58:29,155][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:58:34,157][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:58:39,159][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:58:44,165][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:58:45,326][ ERROR][PID:3321565] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-22 18:58:49,167][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:58:54,169][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:58:55,326][WARNING][PID:3321565] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-22 18:58:59,170][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:59:04,172][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:59:05,328][ INFO][PID:3321565] Downloading the builder-live.log file, attempt 2 [2025-07-22 18:59:05,330][ INFO][PID:3321565] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.200.66.197 copr-rpmbuild-log [2025-07-22 18:59:09,173][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:59:14,176][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:59:19,177][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:59:24,179][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:59:29,183][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:59:34,185][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:59:39,186][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:59:44,190][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:59:49,192][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:59:54,194][ INFO][PID:3321565] Checking for cancel request [2025-07-22 18:59:59,200][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:00:04,201][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:00:05,400][ ERROR][PID:3321565] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-22 19:00:09,203][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:00:14,205][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:00:15,403][WARNING][PID:3321565] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-22 19:00:19,207][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:00:24,209][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:00:25,405][ INFO][PID:3321565] Downloading the builder-live.log file, attempt 3 [2025-07-22 19:00:25,407][ INFO][PID:3321565] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.200.66.197 copr-rpmbuild-log [2025-07-22 19:00:29,217][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:00:34,218][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:00:39,221][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:00:44,222][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:00:49,224][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:00:54,225][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:00:59,227][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:01:04,229][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:01:09,233][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:01:14,235][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:01:19,236][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:01:24,237][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:01:25,478][ ERROR][PID:3321565] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-22 19:01:29,241][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:01:34,242][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:01:35,480][WARNING][PID:3321565] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-22 19:01:39,243][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:01:44,246][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:01:45,482][ INFO][PID:3321565] Downloading the builder-live.log file, attempt 4 [2025-07-22 19:01:45,483][ INFO][PID:3321565] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.200.66.197 copr-rpmbuild-log [2025-07-22 19:01:49,247][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:01:54,248][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:01:59,253][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:04,255][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:09,259][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:14,262][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:19,263][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:24,265][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:29,267][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:34,269][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:39,275][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:44,281][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:45,553][ ERROR][PID:3321565] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-22 19:02:49,283][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:54,285][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:55,569][ ERROR][PID:3321565] Giving up for unstable SSH, failures: 1753210735.326929, 1753210815.4031992, 1753210895.480191, 1753210975.5688844 [2025-07-22 19:02:55,570][ ERROR][PID:3321565] Re-try request for task on 'ResallocHost, ticket_id=4035936, hostname=44.200.66.197, name=aws_x86_64_spot_prod_05735765_20250722_185128, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-07-22 19:02:55,574][ INFO][PID:3321565] Releasing VM back to pool [2025-07-22 19:02:55,594][ INFO][PID:3321565] Retry #1 (on other host) [2025-07-22 19:02:55,596][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:55,597][ INFO][PID:3321565] VM allocation process starts [2025-07-22 19:02:55,624][ INFO][PID:3321565] Trying to allocate VM: ResallocHost, ticket_id=4036119, requested_tags=['copr_builder', 'arch_x86_64'] [2025-07-22 19:02:58,667][ INFO][PID:3321565] Allocated host ResallocHost, ticket_id=4036119, hostname=54.164.220.164, name=aws_x86_64_spot_prod_05736073_20250722_190008, requested_tags=['copr_builder', 'arch_x86_64'] [2025-07-22 19:02:58,668][ INFO][PID:3321565] Allocating ssh connection to builder [2025-07-22 19:02:58,672][ INFO][PID:3321565] Checking that builder machine is OK [2025-07-22 19:02:58,971][ INFO][PID:3321565] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-07-22 19:02:59,199][ INFO][PID:3321565] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-07-22 19:02:59,203][ INFO][PID:3321565] Filling build.info file with builder info [2025-07-22 19:02:59,210][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:02:59,213][ INFO][PID:3321565] Not re-notifying FE and msg buses for the new host. [2025-07-22 19:02:59,219][ INFO][PID:3321565] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9301041-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-07-22 19:02:59,584][ INFO][PID:3321565] Downloading the builder-live.log file, attempt 1 [2025-07-22 19:02:59,589][ INFO][PID:3321565] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.164.220.164 copr-rpmbuild-log [2025-07-22 19:03:04,587][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:03:09,591][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:03:14,599][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:03:19,608][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:03:24,610][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:03:29,611][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:03:34,613][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:03:39,615][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:03:44,616][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:03:49,620][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:03:54,621][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:03:59,622][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:04:04,623][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:04:09,625][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:04:14,626][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:04:19,628][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:04:24,630][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:04:29,631][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:04:34,633][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:04:39,637][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:04:44,638][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:04:49,639][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:04:54,641][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:04:59,643][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:05:04,651][ INFO][PID:3321565] Periodic builder liveness probe: alive [2025-07-22 19:05:04,652][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:05:09,653][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:05:14,654][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:05:19,655][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:05:24,656][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:05:29,658][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:05:34,659][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:05:39,660][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:05:44,663][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:05:49,668][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:05:54,669][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:05:59,671][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:06:04,672][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:06:09,674][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:06:14,676][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:06:19,677][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:06:24,678][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:06:29,680][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:06:34,682][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:06:39,683][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:06:44,684][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:06:49,685][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:06:54,688][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:06:59,690][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:07:04,691][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:07:09,692][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:07:14,693][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:07:19,694][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:07:24,696][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:07:29,697][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:07:34,700][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:07:39,701][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:07:44,703][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:07:49,704][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:07:54,706][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:07:59,709][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:08:04,720][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:08:09,732][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:08:14,734][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:08:19,739][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:08:24,741][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:08:29,742][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:08:34,744][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:08:39,747][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:08:44,750][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:08:49,761][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:08:54,765][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:08:59,768][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:09:04,773][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:09:09,779][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:09:14,780][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:09:19,786][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:09:24,789][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:09:29,792][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:09:34,797][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:09:39,801][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:09:44,805][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:09:49,809][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:09:54,811][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:09:59,813][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:10:04,814][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:10:09,816][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:10:14,817][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:10:19,818][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:10:24,821][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:10:29,822][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:10:34,829][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:10:39,830][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:10:44,832][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:10:49,834][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:10:54,835][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:10:59,836][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:11:04,837][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:11:09,839][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:11:14,841][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:11:15,786][ ERROR][PID:3321565] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-22 19:11:19,843][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:11:24,845][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:11:25,787][WARNING][PID:3321565] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-22 19:11:29,846][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:11:34,848][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:11:35,788][ INFO][PID:3321565] Downloading the builder-live.log file, attempt 2 [2025-07-22 19:11:35,791][ INFO][PID:3321565] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.164.220.164 copr-rpmbuild-log [2025-07-22 19:11:39,849][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:11:44,850][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:11:49,852][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:11:54,854][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:11:59,855][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:12:04,857][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:12:09,860][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:12:14,861][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:12:19,862][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:12:24,863][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:12:29,865][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:12:34,873][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:12:35,864][ ERROR][PID:3321565] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-22 19:12:39,876][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:12:44,880][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:12:45,871][WARNING][PID:3321565] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-22 19:12:49,885][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:12:54,886][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:12:55,872][ INFO][PID:3321565] Downloading the builder-live.log file, attempt 3 [2025-07-22 19:12:55,874][ INFO][PID:3321565] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.164.220.164 copr-rpmbuild-log [2025-07-22 19:12:59,888][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:13:04,889][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:13:09,892][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:13:14,895][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:13:19,902][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:13:24,903][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:13:29,904][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:13:34,905][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:13:39,907][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:13:44,908][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:13:49,909][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:13:54,910][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:13:55,947][ ERROR][PID:3321565] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-22 19:13:59,912][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:14:04,916][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:14:05,948][WARNING][PID:3321565] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-22 19:14:09,919][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:14:14,922][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:14:15,951][ INFO][PID:3321565] Downloading the builder-live.log file, attempt 4 [2025-07-22 19:14:15,957][ INFO][PID:3321565] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.164.220.164 copr-rpmbuild-log [2025-07-22 19:14:19,924][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:14:24,926][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:14:29,928][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:14:34,929][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:14:39,932][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:14:44,936][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:14:49,938][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:14:54,939][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:14:59,940][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:15:04,952][ INFO][PID:3321565] Periodic builder liveness probe: alive [2025-07-22 19:15:04,952][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:15:09,954][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:15:14,955][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:15:16,035][ ERROR][PID:3321565] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-22 19:15:19,958][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:15:24,961][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:15:26,037][ ERROR][PID:3321565] Giving up for unstable SSH, failures: 1753211485.7875896, 1753211565.8714566, 1753211645.9486055, 1753211726.0373328 [2025-07-22 19:15:26,040][ ERROR][PID:3321565] Re-try request for task on 'ResallocHost, ticket_id=4036119, hostname=54.164.220.164, name=aws_x86_64_spot_prod_05736073_20250722_190008, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-07-22 19:15:26,041][ INFO][PID:3321565] Releasing VM back to pool [2025-07-22 19:15:26,068][ INFO][PID:3321565] Retry #2 (on other host) [2025-07-22 19:15:26,069][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:15:26,072][ INFO][PID:3321565] VM allocation process starts [2025-07-22 19:15:26,101][ INFO][PID:3321565] Trying to allocate VM: ResallocHost, ticket_id=4036415, requested_tags=['copr_builder', 'arch_x86_64'] [2025-07-22 19:15:31,104][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:15:32,134][ INFO][PID:3321565] Allocated host ResallocHost, ticket_id=4036415, hostname=2620:52:3:1:dead:beef:cafe:c18b, name=copr_hv_x86_64_03_prod_05736272_20250722_190654, requested_tags=['copr_builder', 'arch_x86_64'] [2025-07-22 19:15:32,134][ INFO][PID:3321565] Allocating ssh connection to builder [2025-07-22 19:15:32,135][ INFO][PID:3321565] Checking that builder machine is OK [2025-07-22 19:15:32,776][ INFO][PID:3321565] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-07-22 19:15:33,201][ INFO][PID:3321565] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-07-22 19:15:33,201][ INFO][PID:3321565] Filling build.info file with builder info [2025-07-22 19:15:33,207][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:15:33,209][ INFO][PID:3321565] Not re-notifying FE and msg buses for the new host. [2025-07-22 19:15:33,210][ INFO][PID:3321565] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9301041-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-07-22 19:15:33,792][ INFO][PID:3321565] Downloading the builder-live.log file, attempt 1 [2025-07-22 19:15:33,795][ INFO][PID:3321565] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c18b copr-rpmbuild-log [2025-07-22 19:15:38,794][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:15:43,797][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:15:48,799][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:15:53,802][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:15:58,804][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:16:03,806][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:16:08,807][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:16:13,809][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:16:18,810][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:16:23,811][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:16:28,813][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:16:33,814][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:16:38,815][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:16:43,817][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:16:48,818][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:16:53,819][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:16:58,820][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:17:03,821][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:17:08,822][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:17:13,824][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:17:18,825][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:17:23,826][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:17:28,827][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:17:33,828][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:17:38,830][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:17:43,831][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:17:48,833][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:17:53,834][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:17:58,835][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:18:03,836][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:18:08,838][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:18:13,839][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:18:18,840][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:18:23,841][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:18:28,842][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:18:33,844][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:18:38,845][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:18:43,847][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:18:48,848][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:18:53,849][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:18:58,850][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:19:03,851][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:19:08,852][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:19:13,853][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:19:18,854][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:19:23,855][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:19:28,856][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:19:33,858][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:19:38,859][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:19:43,860][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:19:48,863][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:19:53,864][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:19:58,865][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:20:03,867][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:20:08,868][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:20:13,869][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:20:18,870][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:20:23,871][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:20:28,873][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:20:33,874][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:20:38,875][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:20:43,877][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:20:48,878][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:20:53,879][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:20:58,880][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:21:03,881][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:21:08,882][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:21:13,883][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:21:18,884][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:21:23,885][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:21:28,887][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:21:33,888][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:21:38,889][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:21:43,890][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:21:48,891][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:21:53,892][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:21:58,893][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:22:03,896][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:22:08,897][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:22:13,898][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:22:18,899][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:22:23,901][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:22:28,902][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:22:33,903][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:22:38,904][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:22:43,905][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:22:48,906][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:22:53,907][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:22:58,908][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:23:03,910][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:23:08,911][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:23:13,912][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:23:18,913][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:23:23,914][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:23:28,916][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:23:33,917][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:23:38,918][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:23:43,919][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:23:48,920][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:23:53,921][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:23:58,922][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:24:03,924][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:24:08,925][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:24:13,926][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:24:18,927][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:24:23,929][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:24:28,930][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:24:33,931][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:24:38,932][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:24:43,933][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:24:48,936][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:24:53,937][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:24:58,938][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:25:03,939][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:25:08,947][ INFO][PID:3321565] Periodic builder liveness probe: alive [2025-07-22 19:25:08,947][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:25:13,948][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:25:18,949][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:25:23,950][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:25:28,951][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:25:33,953][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:25:38,954][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:25:43,955][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:25:48,957][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:25:53,958][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:25:58,959][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:26:03,960][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:26:08,961][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:26:13,965][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:26:18,969][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:26:23,970][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:26:28,971][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:26:33,972][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:26:38,973][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:26:43,974][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:26:48,975][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:26:53,976][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:26:58,977][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:27:03,979][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:27:08,980][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:27:13,981][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:27:18,982][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:27:23,983][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:27:28,984][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:27:33,985][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:27:38,987][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:27:43,988][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:27:48,989][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:27:53,990][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:27:58,991][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:28:03,993][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:28:08,994][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:28:13,995][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:28:18,996][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:28:23,998][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:28:28,999][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:28:34,000][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:28:39,001][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:28:44,002][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:28:49,003][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:28:54,005][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:28:59,006][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:29:04,007][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:29:09,008][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:29:14,009][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:29:19,010][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:29:24,011][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:29:29,013][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:29:34,013][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:29:39,015][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:29:44,016][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:29:49,017][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:29:54,018][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:29:59,019][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:30:04,021][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:30:09,022][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:30:14,023][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:30:19,024][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:30:24,026][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:30:29,027][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:30:34,028][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:30:39,029][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:30:44,030][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:30:49,031][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:30:54,032][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:30:59,034][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:31:04,035][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:31:09,036][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:31:14,037][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:31:19,038][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:31:24,039][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:31:29,041][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:31:34,042][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:31:39,043][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:31:44,044][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:31:49,046][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:31:54,049][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:31:59,050][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:32:04,051][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:32:09,053][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:32:14,060][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:32:19,061][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:32:24,062][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:32:29,064][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:32:34,066][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:32:39,067][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:32:44,068][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:32:49,070][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:32:54,073][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:32:59,075][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:33:04,076][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:33:09,077][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:33:14,079][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:33:19,080][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:33:24,081][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:33:29,084][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:33:34,085][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:33:39,089][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:33:44,093][ INFO][PID:3321565] Checking for cancel request [2025-07-22 19:33:46,561][ INFO][PID:3321565] Downloading results from builder [2025-07-22 19:33:46,561][ INFO][PID:3321565] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c18b]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree started [2025-07-22 19:33:46,569][ INFO][PID:3321565] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c18b]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/ &> /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/build-09301041.rsync.log [2025-07-22 19:33:48,096][ INFO][PID:3321565] rsyncing finished. [2025-07-22 19:33:48,097][ INFO][PID:3321565] Releasing VM back to pool [2025-07-22 19:33:48,110][ INFO][PID:3321565] Searching for 'success' file in resultdir [2025-07-22 19:33:48,111][ INFO][PID:3321565] Going to sign pkgs from source: 9301041-fedora-rawhide-x86_64 in chroot: /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64 [2025-07-22 19:33:48,112][ INFO][PID:3321565] Calling '/bin/sign -u fberat#autoconf.2.72-6@copr.fedorahosted.org -p' (attempt #1) [2025-07-22 19:33:48,505][ INFO][PID:3321565] Calling '/bin/sign -4 -h sha256 -u fberat#autoconf.2.72-6@copr.fedorahosted.org -r /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-debuginfo-2025.8-2.fc43.x86_64.rpm' (attempt #1) [2025-07-22 19:33:50,259][ INFO][PID:3321565] signed rpm: /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-debuginfo-2025.8-2.fc43.x86_64.rpm [2025-07-22 19:33:50,260][ INFO][PID:3321565] Calling '/bin/sign -4 -h sha256 -u fberat#autoconf.2.72-6@copr.fedorahosted.org -r /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-2025.8-2.fc43.x86_64.rpm' (attempt #1) [2025-07-22 19:33:51,104][ INFO][PID:3321565] signed rpm: /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-2025.8-2.fc43.x86_64.rpm [2025-07-22 19:33:51,106][ INFO][PID:3321565] Calling '/bin/sign -4 -h sha256 -u fberat#autoconf.2.72-6@copr.fedorahosted.org -r /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-libs-2025.8-2.fc43.x86_64.rpm' (attempt #1) [2025-07-22 19:33:52,145][ INFO][PID:3321565] signed rpm: /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-libs-2025.8-2.fc43.x86_64.rpm [2025-07-22 19:33:52,145][ INFO][PID:3321565] Calling '/bin/sign -4 -h sha256 -u fberat#autoconf.2.72-6@copr.fedorahosted.org -r /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-devel-2025.8-2.fc43.x86_64.rpm' (attempt #1) [2025-07-22 19:33:52,851][ INFO][PID:3321565] signed rpm: /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-devel-2025.8-2.fc43.x86_64.rpm [2025-07-22 19:33:52,852][ INFO][PID:3321565] Calling '/bin/sign -4 -h sha256 -u fberat#autoconf.2.72-6@copr.fedorahosted.org -r /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-2025.8-2.fc43.src.rpm' (attempt #1) [2025-07-22 19:33:54,154][ INFO][PID:3321565] signed rpm: /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-2025.8-2.fc43.src.rpm [2025-07-22 19:33:54,155][ INFO][PID:3321565] Calling '/bin/sign -4 -h sha256 -u fberat#autoconf.2.72-6@copr.fedorahosted.org -r /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-libs-debuginfo-2025.8-2.fc43.x86_64.rpm' (attempt #1) [2025-07-22 19:33:55,173][ INFO][PID:3321565] signed rpm: /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-libs-debuginfo-2025.8-2.fc43.x86_64.rpm [2025-07-22 19:33:55,174][ INFO][PID:3321565] Calling '/bin/sign -4 -h sha256 -u fberat#autoconf.2.72-6@copr.fedorahosted.org -r /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-debugsource-2025.8-2.fc43.x86_64.rpm' (attempt #1) [2025-07-22 19:33:56,042][ INFO][PID:3321565] signed rpm: /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/rpm-ostree-debugsource-2025.8-2.fc43.x86_64.rpm [2025-07-22 19:33:56,044][ INFO][PID:3321565] Sign done [2025-07-22 19:33:56,048][ INFO][PID:3321565] Incremental createrepo run, adding 09301041-rpm-ostree into https://download.copr.fedorainfracloud.org/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64, (auto-create-repo=True) [2025-07-22 19:33:56,049][ INFO][PID:3321565] Running command 'copr-repo --batched /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64 --add 09301041-rpm-ostree --no-appstream-metadata' as PID 3584319 [2025-07-22 19:34:06,886][ INFO][PID:3321565] Finished after 10 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64 --add 09301041-rpm-ostree --no-appstream-metadata) [2025-07-22 19:34:06,887][ INFO][PID:3321565] Getting build details [2025-07-22 19:34:06,888][ INFO][PID:3321565] Listing built binary packages in /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree [2025-07-22 19:34:06,888][ INFO][PID:3321565] Built packages: ['rpm-ostree-devel 2025.8', 'rpm-ostree-debugsource 2025.8', 'rpm-ostree-libs-debuginfo 2025.8', 'rpm-ostree 2025.8', 'rpm-ostree-libs 2025.8', 'rpm-ostree-debuginfo 2025.8'] [2025-07-22 19:34:06,889][ INFO][PID:3321565] build details: {'built_packages': 'rpm-ostree-devel 2025.8\nrpm-ostree-debugsource 2025.8\nrpm-ostree-libs-debuginfo 2025.8\nrpm-ostree 2025.8\nrpm-ostree-libs 2025.8\nrpm-ostree-debuginfo 2025.8'} [2025-07-22 19:34:06,890][ INFO][PID:3321565] Retrieving pubkey [2025-07-22 19:34:06,890][ INFO][PID:3321565] Calling '/bin/sign -u fberat#autoconf.2.72-6@copr.fedorahosted.org -p' (attempt #1) [2025-07-22 19:34:07,068][ INFO][PID:3321565] Added pubkey for user fberat project autoconf.2.72-6 into: /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/pubkey.gpg [2025-07-22 19:34:07,069][ INFO][PID:3321565] Finished build: id=9301041 failed=False timeout=115200 destdir=/var/lib/copr/public_html/results/fberat/autoconf.2.72-6 chroot=fedora-rawhide-x86_64 [2025-07-22 19:34:07,071][ INFO][PID:3321565] Worker succeeded build, took 1113.8613512516022 [2025-07-22 19:34:07,072][ INFO][PID:3321565] Sending build state back to frontend: { "builds": [ { "timeout": 115200, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "fberat", "project_name": "autoconf.2.72-6", "project_dirname": "autoconf.2.72-6", "submitter": "fberat", "ended_on": 1753212847.0712714, "started_on": 1753211733.2099202, "submitted_on": null, "status": 1, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9301041-fedora-rawhide-x86_64", "build_id": 9301041, "package_name": "rpm-ostree", "package_version": "2025.8-2", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/fberat/autoconf.2.72-6/rpm-ostree", "git_hash": "ae68e63672d54ce2e47da5eb6d4fd7813408ad26", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "fberat/autoconf.2.72-6--fberat", "results": { "packages": [ { "name": "rpm-ostree-devel", "epoch": null, "version": "2025.8", "release": "2.fc43", "arch": "x86_64" }, { "name": "rpm-ostree-debugsource", "epoch": null, "version": "2025.8", "release": "2.fc43", "arch": "x86_64" }, { "name": "rpm-ostree-libs-debuginfo", "epoch": null, "version": "2025.8", "release": "2.fc43", "arch": "x86_64" }, { "name": "rpm-ostree", "epoch": null, "version": "2025.8", "release": "2.fc43", "arch": "x86_64" }, { "name": "rpm-ostree", "epoch": null, "version": "2025.8", "release": "2.fc43", "arch": "src" }, { "name": "rpm-ostree-libs", "epoch": null, "version": "2025.8", "release": "2.fc43", "arch": "x86_64" }, { "name": "rpm-ostree-debuginfo", "epoch": null, "version": "2025.8", "release": "2.fc43", "arch": "x86_64" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/fberat/autoconf.2.72-6", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/fberat/autoconf.2.72-6", "result_dir": "09301041-rpm-ostree", "built_packages": "rpm-ostree-devel 2025.8\nrpm-ostree-debugsource 2025.8\nrpm-ostree-libs-debuginfo 2025.8\nrpm-ostree 2025.8\nrpm-ostree-libs 2025.8\nrpm-ostree-debuginfo 2025.8", "id": 9301041, "mockchain_macros": { "copr_username": "fberat", "copr_projectname": "autoconf.2.72-6", "vendor": "Fedora Project COPR (fberat/autoconf.2.72-6)" } } ] } [2025-07-22 19:34:07,103][ INFO][PID:3321565] Sending fedora-messaging bus message in build.end [2025-07-22 19:34:07,156][ INFO][PID:3321565] Compressing /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/builder-live.log by gzip [2025-07-22 19:34:07,157][ INFO][PID:3321565] Running command 'gzip /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/builder-live.log' as PID 3585820 [2025-07-22 19:34:07,186][ INFO][PID:3321565] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/builder-live.log) [2025-07-22 19:34:07,187][ INFO][PID:3321565] Compressing /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/backend.log by gzip [2025-07-22 19:34:07,188][ INFO][PID:3321565] Running command 'gzip /var/lib/copr/public_html/results/fberat/autoconf.2.72-6/fedora-rawhide-x86_64/09301041-rpm-ostree/backend.log' as PID 3585827