[2026-01-05 18:41:50,745][ INFO][PID:3054415] Marking build as starting [2026-01-05 18:41:50,887][ INFO][PID:3054415] Checking that /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/repodata/repomd.xml exists [2026-01-05 18:41:50,890][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:41:50,896][ INFO][PID:3054415] VM allocation process starts [2026-01-05 18:41:50,933][ INFO][PID:3054415] Trying to allocate VM: ResallocHost, ticket_id=664550, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-05 18:41:55,939][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:41:57,014][ INFO][PID:3054415] Allocated host ResallocHost, ticket_id=664550, hostname=2620:52:6:1161:dead:beef:cafe:c18e, name=vmhost_p09_04_prod_00937850_20260105_183447, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-05 18:41:57,015][ INFO][PID:3054415] Allocating ssh connection to builder [2026-01-05 18:41:57,015][ INFO][PID:3054415] Checking that builder machine is OK [2026-01-05 18:41:57,928][ INFO][PID:3054415] Running remote command: copr-builder-ready fedora-41-ppc64le [2026-01-05 18:41:58,472][ INFO][PID:3054415] Red Hat subscription not needed for fedora-41-ppc64le Builder is ready to be used [2026-01-05 18:41:58,473][ INFO][PID:3054415] Filling build.info file with builder info [2026-01-05 18:41:58,477][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:41:58,480][ INFO][PID:3054415] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "ligenix", "project_name": "enterprise-librenms", "project_dirname": "enterprise-librenms", "submitter": "ligenix", "ended_on": null, "started_on": 1767638518.4797864, "submitted_on": null, "status": 3, "chroot": "fedora-41-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9976387-fedora-41-ppc64le", "build_id": 9976387, "package_name": "python-command-runner", "package_version": "1.7.5-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/ligenix/enterprise-librenms/python-command-runner", "git_hash": "dddba86c7a090ba5849963dac24faadccc9e0bee", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "ligenix/enterprise-librenms--ligenix", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-librenms/fedora-41-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/ligenix/enterprise-librenms", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-librenms", "result_dir": "09976387-python-command-runner", "built_packages": "", "id": 9976387, "mockchain_macros": { "copr_username": "ligenix", "copr_projectname": "enterprise-librenms", "vendor": "Fedora Project COPR (ligenix/enterprise-librenms)" } } ] } [2026-01-05 18:41:58,608][ INFO][PID:3054415] Sending fedora-messaging bus message in build.start [2026-01-05 18:41:59,624][ INFO][PID:3054415] Sending fedora-messaging bus message in chroot.start [2026-01-05 18:41:59,679][ INFO][PID:3054415] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9976387-fedora-41-ppc64le --chroot fedora-41-ppc64le --detached [2026-01-05 18:42:00,654][ INFO][PID:3054415] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18e' (ED25519) to the list of known hosts. [2026-01-05 18:42:00,655][ INFO][PID:3054415] Downloading the builder-live.log file, attempt 1 [2026-01-05 18:42:00,659][ INFO][PID:3054415] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18e copr-rpmbuild-log [2026-01-05 18:42:05,690][ INFO][PID:3054415] Periodic builder liveness probe: alive [2026-01-05 18:42:05,691][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:42:10,694][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:42:15,697][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:42:20,698][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:42:25,700][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:42:30,703][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:42:35,711][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:42:40,718][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:42:45,719][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:42:50,725][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:42:55,727][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:43:00,731][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:43:05,736][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:43:10,739][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:43:15,740][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:43:20,742][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:43:25,749][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:43:30,758][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:43:35,759][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:43:40,770][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:43:45,774][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:43:50,776][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:43:55,778][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:44:00,782][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:44:05,784][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:44:10,787][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:44:15,791][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:44:20,799][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:44:25,801][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:44:30,803][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:44:35,804][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:44:40,806][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:44:45,807][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:44:50,808][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:44:55,809][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:45:00,810][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:45:05,811][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:45:10,813][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:45:15,819][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:45:20,821][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:45:25,822][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:45:30,823][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:45:35,825][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:45:40,826][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:45:45,827][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:45:50,828][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:45:55,829][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:46:00,830][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:46:05,831][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:46:10,833][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:46:15,834][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:46:21,093][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:46:26,095][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:46:31,096][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:46:36,098][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:46:41,100][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:46:46,103][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:46:51,104][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:46:56,224][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:47:01,225][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:47:06,227][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:47:11,228][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:47:16,230][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:47:21,240][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:47:26,241][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:47:31,258][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:47:36,365][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:47:41,367][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:47:46,367][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:47:51,368][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:47:56,370][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:48:01,371][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:48:06,372][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:48:11,373][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:48:16,374][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:48:21,375][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:48:26,376][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:48:31,377][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:48:36,546][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:48:41,547][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:48:46,548][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:48:51,549][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:48:56,550][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:49:01,551][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:49:06,552][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:49:11,553][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:49:16,554][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:49:21,555][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:49:26,556][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:49:31,557][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:49:36,558][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:49:41,559][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:49:46,841][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:49:51,843][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:49:56,846][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:50:01,848][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:50:06,849][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:50:11,850][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:50:16,852][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:50:21,853][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:50:26,858][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:50:31,862][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:50:36,863][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:50:41,866][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:50:46,867][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:50:51,868][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:50:56,872][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:51:01,874][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:51:06,875][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:51:11,876][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:51:16,877][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:51:21,878][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:51:26,879][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:51:31,880][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:51:36,881][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:51:41,882][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:51:46,883][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:51:51,884][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:51:56,886][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:52:01,886][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:52:06,895][ INFO][PID:3054415] Periodic builder liveness probe: alive [2026-01-05 18:52:06,908][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:52:11,909][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:52:16,910][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:52:22,200][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:52:27,201][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:52:32,202][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:52:37,203][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:52:42,204][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:52:47,205][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:52:52,206][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:52:57,208][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:53:02,209][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:53:07,210][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:53:12,210][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:53:17,211][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:53:22,212][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:53:27,215][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:53:32,217][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:53:37,219][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:53:42,220][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:53:47,221][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:53:52,223][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:53:57,224][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:54:02,225][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:54:07,226][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:54:12,229][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:54:17,231][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:54:22,232][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:54:27,234][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:54:32,236][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:54:37,238][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:54:42,249][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:54:47,255][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:54:52,260][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:54:57,262][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:55:02,267][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:55:04,697][ INFO][PID:3054415] Downloading results from builder [2026-01-05 18:55:04,705][ INFO][PID:3054415] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18e]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner started [2026-01-05 18:55:04,720][ INFO][PID:3054415] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18e]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/ &> /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/build-09976387.rsync.log [2026-01-05 18:56:04,841][ INFO][PID:3054415] rsyncing finished. [2026-01-05 18:56:04,847][ ERROR][PID:3054415] Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-01-05 18:56:04,847][ ERROR][PID:3054415] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-01-05 18:56:14,848][ INFO][PID:3054415] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18e]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner started [2026-01-05 18:56:14,850][ INFO][PID:3054415] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18e]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/ &> /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/build-09976387.rsync.log [2026-01-05 18:57:14,917][ INFO][PID:3054415] rsyncing finished. [2026-01-05 18:57:14,918][ ERROR][PID:3054415] Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-01-05 18:57:14,918][ ERROR][PID:3054415] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-01-05 18:57:24,918][ INFO][PID:3054415] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18e]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner started [2026-01-05 18:57:24,920][ INFO][PID:3054415] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18e]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/ &> /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/build-09976387.rsync.log [2026-01-05 18:58:25,000][ INFO][PID:3054415] rsyncing finished. [2026-01-05 18:58:25,000][ ERROR][PID:3054415] Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-01-05 18:58:25,001][ ERROR][PID:3054415] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-01-05 18:58:35,001][ ERROR][PID:3054415] Re-try request for task on 'ResallocHost, ticket_id=664550, hostname=2620:52:6:1161:dead:beef:cafe:c18e, name=vmhost_p09_04_prod_00937850_20260105_183447, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 3 SSH attempts [2026-01-05 18:58:35,002][ INFO][PID:3054415] Releasing VM back to pool [2026-01-05 18:58:35,011][ INFO][PID:3054415] Retry #1 (on other host) [2026-01-05 18:58:35,012][ INFO][PID:3054415] Checking that /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/repodata/repomd.xml exists [2026-01-05 18:58:35,013][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:58:35,013][ INFO][PID:3054415] VM allocation process starts [2026-01-05 18:58:35,078][ INFO][PID:3054415] Trying to allocate VM: ResallocHost, ticket_id=664739, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-05 18:58:38,119][ INFO][PID:3054415] Allocated host ResallocHost, ticket_id=664739, hostname=2620:52:6:1161:dead:beef:cafe:c190, name=vmhost_p09_04_prod_00938613_20260105_184620, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-05 18:58:38,567][ INFO][PID:3054415] Allocating ssh connection to builder [2026-01-05 18:58:38,568][ INFO][PID:3054415] Checking that builder machine is OK [2026-01-05 18:58:39,536][ INFO][PID:3054415] Running remote command: copr-builder-ready fedora-41-ppc64le [2026-01-05 18:59:39,604][ ERROR][PID:3054415] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c190 port 22: Connection timed out [2026-01-05 18:59:49,605][ ERROR][PID:3054415] Re-try request for task on 'ResallocHost, ticket_id=664739, hostname=2620:52:6:1161:dead:beef:cafe:c190, name=vmhost_p09_04_prod_00938613_20260105_184620, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts [2026-01-05 18:59:49,606][ INFO][PID:3054415] Releasing VM back to pool [2026-01-05 18:59:49,614][ INFO][PID:3054415] Retry #2 (on other host) [2026-01-05 18:59:49,615][ INFO][PID:3054415] Checking that /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/repodata/repomd.xml exists [2026-01-05 18:59:49,615][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:59:49,616][ INFO][PID:3054415] VM allocation process starts [2026-01-05 18:59:49,650][ INFO][PID:3054415] Trying to allocate VM: ResallocHost, ticket_id=664747, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-05 18:59:52,693][ INFO][PID:3054415] Allocated host ResallocHost, ticket_id=664747, hostname=2620:52:6:1161:dead:beef:cafe:c191, name=vmhost_p09_04_prod_00938622_20260105_184645, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-05 18:59:52,693][ INFO][PID:3054415] Allocating ssh connection to builder [2026-01-05 18:59:52,694][ INFO][PID:3054415] Checking that builder machine is OK [2026-01-05 18:59:53,676][ INFO][PID:3054415] Running remote command: copr-builder-ready fedora-41-ppc64le [2026-01-05 18:59:54,164][ INFO][PID:3054415] Red Hat subscription not needed for fedora-41-ppc64le Builder is ready to be used [2026-01-05 18:59:54,165][ INFO][PID:3054415] Filling build.info file with builder info [2026-01-05 18:59:54,187][ INFO][PID:3054415] Checking for cancel request [2026-01-05 18:59:54,189][ INFO][PID:3054415] Not re-notifying FE and msg buses for the new host. [2026-01-05 18:59:54,189][ INFO][PID:3054415] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9976387-fedora-41-ppc64le --chroot fedora-41-ppc64le --detached [2026-01-05 18:59:55,184][ INFO][PID:3054415] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c191' (ED25519) to the list of known hosts. [2026-01-05 18:59:55,185][ INFO][PID:3054415] Downloading the builder-live.log file, attempt 1 [2026-01-05 18:59:55,186][ INFO][PID:3054415] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c191 copr-rpmbuild-log [2026-01-05 19:00:00,185][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:00:05,186][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:00:10,187][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:00:15,188][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:00:20,189][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:00:25,190][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:00:30,191][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:00:35,192][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:00:40,193][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:00:45,194][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:00:50,195][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:00:55,196][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:01:00,197][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:01:05,198][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:01:10,199][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:01:15,200][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:01:20,201][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:01:25,201][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:01:30,202][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:01:35,203][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:01:40,204][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:01:45,205][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:01:50,206][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:01:55,207][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:02:00,208][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:02:05,209][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:02:10,221][ INFO][PID:3054415] Periodic builder liveness probe: alive [2026-01-05 19:02:10,221][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:02:15,223][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:02:20,224][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:02:25,225][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:02:30,226][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:02:35,227][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:02:40,228][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:02:45,229][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:02:50,230][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:02:55,231][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:03:00,232][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:03:05,232][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:03:10,233][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:03:15,234][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:03:20,235][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:03:25,236][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:03:30,237][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:03:35,238][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:03:40,239][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:03:45,240][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:03:50,241][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:03:55,242][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:04:00,243][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:04:05,243][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:04:10,244][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:04:15,245][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:04:20,246][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:04:25,247][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:04:30,248][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:04:35,249][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:04:40,260][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:04:45,459][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:04:50,461][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:04:55,461][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:05:00,462][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:05:05,464][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:05:10,465][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:05:15,466][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:05:20,467][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:05:25,468][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:05:30,469][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:05:35,470][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:05:40,472][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:05:45,473][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:05:50,474][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:05:55,475][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:06:00,476][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:06:05,477][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:06:10,478][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:06:15,479][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:06:20,480][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:06:25,481][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:06:30,483][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:06:35,484][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:06:40,485][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:06:45,488][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:06:50,493][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:06:55,496][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:07:00,498][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:07:05,500][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:07:10,501][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:07:15,502][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:07:20,503][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:07:25,504][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:07:30,506][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:07:35,508][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:07:40,511][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:07:45,512][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:07:50,524][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:07:55,526][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:08:00,529][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:08:05,531][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:08:10,535][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:08:15,536][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:08:20,537][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:08:25,539][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:08:30,540][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:08:35,546][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:08:40,548][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:08:45,551][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:08:50,552][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:08:55,553][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:09:00,559][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:09:05,564][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:09:10,569][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:09:15,571][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:09:20,574][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:09:25,580][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:09:30,582][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:09:35,583][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:09:40,586][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:09:45,589][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:09:50,591][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:09:55,593][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:10:00,594][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:10:05,598][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:10:10,602][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:10:15,610][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:10:20,614][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:10:25,618][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:10:30,619][ INFO][PID:3054415] Checking for cancel request [2026-01-05 19:10:32,554][ INFO][PID:3054415] Downloading results from builder [2026-01-05 19:10:32,555][ INFO][PID:3054415] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c191]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner started [2026-01-05 19:10:32,560][ INFO][PID:3054415] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c191]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/ &> /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/build-09976387.rsync.log [2026-01-05 19:10:33,321][ INFO][PID:3054415] rsyncing finished. [2026-01-05 19:10:33,322][ INFO][PID:3054415] Releasing VM back to pool [2026-01-05 19:10:33,340][ INFO][PID:3054415] Searching for 'success' file in resultdir [2026-01-05 19:10:33,341][ INFO][PID:3054415] Going to sign pkgs from source: 9976387-fedora-41-ppc64le in chroot: /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le [2026-01-05 19:10:33,345][ INFO][PID:3054415] Calling '/bin/sign -u ligenix#enterprise-librenms@copr.fedorahosted.org -p' (attempt #1) [2026-01-05 19:10:33,613][ INFO][PID:3054415] Calling '/bin/sign -4 -h sha256 -u ligenix#enterprise-librenms@copr.fedorahosted.org -r /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/python3-command-runner-1.7.5-1.fc41.noarch.rpm' (attempt #1) [2026-01-05 19:10:34,368][ INFO][PID:3054415] signed rpm: /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/python3-command-runner-1.7.5-1.fc41.noarch.rpm [2026-01-05 19:10:34,370][ INFO][PID:3054415] Calling '/bin/sign -4 -h sha256 -u ligenix#enterprise-librenms@copr.fedorahosted.org -r /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/python-command-runner-1.7.5-1.fc41.src.rpm' (attempt #1) [2026-01-05 19:10:35,108][ INFO][PID:3054415] signed rpm: /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/python-command-runner-1.7.5-1.fc41.src.rpm [2026-01-05 19:10:35,111][ INFO][PID:3054415] Sign done [2026-01-05 19:10:35,115][ INFO][PID:3054415] Incremental createrepo run, adding 09976387-python-command-runner into https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-librenms/fedora-41-ppc64le, (auto-create-repo=True) [2026-01-05 19:10:35,119][ INFO][PID:3054415] Running command 'copr-repo --batched /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le --add 09976387-python-command-runner --no-appstream-metadata' as PID 3246341 [2026-01-05 19:10:36,497][ INFO][PID:3054415] Finished after 1 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le --add 09976387-python-command-runner --no-appstream-metadata) [2026-01-05 19:10:36,498][ INFO][PID:3054415] Getting build details [2026-01-05 19:10:36,499][ INFO][PID:3054415] Listing built binary packages in /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner [2026-01-05 19:10:36,500][ INFO][PID:3054415] Built packages: ['python3-command-runner 1.7.5'] [2026-01-05 19:10:36,501][ INFO][PID:3054415] build details: {'built_packages': 'python3-command-runner 1.7.5'} [2026-01-05 19:10:36,502][ INFO][PID:3054415] Retrieving pubkey [2026-01-05 19:10:36,503][ INFO][PID:3054415] Calling '/bin/sign -u ligenix#enterprise-librenms@copr.fedorahosted.org -p' (attempt #1) [2026-01-05 19:10:36,776][ INFO][PID:3054415] Added pubkey for user ligenix project enterprise-librenms into: /var/lib/copr/public_html/results/ligenix/enterprise-librenms/pubkey.gpg [2026-01-05 19:10:36,779][ INFO][PID:3054415] Finished build: id=9976387 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/ligenix/enterprise-librenms chroot=fedora-41-ppc64le [2026-01-05 19:10:36,785][ INFO][PID:3054415] Worker succeeded build, took 642.5966176986694 [2026-01-05 19:10:36,788][ INFO][PID:3054415] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "ligenix", "project_name": "enterprise-librenms", "project_dirname": "enterprise-librenms", "submitter": "ligenix", "ended_on": 1767640236.785701, "started_on": 1767639594.1890833, "submitted_on": null, "status": 1, "chroot": "fedora-41-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9976387-fedora-41-ppc64le", "build_id": 9976387, "package_name": "python-command-runner", "package_version": "1.7.5-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/ligenix/enterprise-librenms/python-command-runner", "git_hash": "dddba86c7a090ba5849963dac24faadccc9e0bee", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "ligenix/enterprise-librenms--ligenix", "results": { "packages": [ { "name": "python3-command-runner", "epoch": null, "version": "1.7.5", "release": "1.fc41", "arch": "noarch" }, { "name": "python-command-runner", "epoch": null, "version": "1.7.5", "release": "1.fc41", "arch": "src" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-librenms/fedora-41-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/ligenix/enterprise-librenms", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-librenms", "result_dir": "09976387-python-command-runner", "built_packages": "python3-command-runner 1.7.5", "id": 9976387, "mockchain_macros": { "copr_username": "ligenix", "copr_projectname": "enterprise-librenms", "vendor": "Fedora Project COPR (ligenix/enterprise-librenms)" } } ] } [2026-01-05 19:10:36,896][ INFO][PID:3054415] Sending fedora-messaging bus message in build.end [2026-01-05 19:10:36,958][ INFO][PID:3054415] Compressing /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/builder-live.log by gzip [2026-01-05 19:10:36,961][ INFO][PID:3054415] Running command 'gzip /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/builder-live.log' as PID 3246639 [2026-01-05 19:10:37,008][ INFO][PID:3054415] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/builder-live.log) [2026-01-05 19:10:37,010][ INFO][PID:3054415] Compressing /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/backend.log by gzip [2026-01-05 19:10:37,014][ INFO][PID:3054415] Running command 'gzip /var/lib/copr/public_html/results/ligenix/enterprise-librenms/fedora-41-ppc64le/09976387-python-command-runner/backend.log' as PID 3246644