[2026-01-22 16:35:53,726][ INFO][PID:3209902] Marking build as starting [2026-01-22 16:35:53,838][ INFO][PID:3209902] Checking that /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/repodata/repomd.xml exists [2026-01-22 16:35:53,873][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:35:53,873][ INFO][PID:3209902] VM allocation process starts [2026-01-22 16:35:53,887][ INFO][PID:3209902] Trying to allocate VM: ResallocHost, ticket_id=972826, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-22 16:35:56,931][ INFO][PID:3209902] Allocated host ResallocHost, ticket_id=972826, hostname=2620:52:6:1161:dead:beef:cafe:c194, name=vmhost_p09_04_prod_01477299_20260122_160946, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-22 16:35:56,932][ INFO][PID:3209902] Allocating ssh connection to builder [2026-01-22 16:35:56,932][ INFO][PID:3209902] Checking that builder machine is OK [2026-01-22 16:36:57,004][ ERROR][PID:3209902] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c194 port 22: Connection timed out [2026-01-22 16:37:07,007][ ERROR][PID:3209902] Re-try request for task on 'ResallocHost, ticket_id=972826, hostname=2620:52:6:1161:dead:beef:cafe:c194, name=vmhost_p09_04_prod_01477299_20260122_160946, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts [2026-01-22 16:37:07,008][ INFO][PID:3209902] Releasing VM back to pool [2026-01-22 16:37:07,018][ INFO][PID:3209902] Retry #1 (on other host) [2026-01-22 16:37:07,019][ INFO][PID:3209902] Checking that /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/repodata/repomd.xml exists [2026-01-22 16:37:07,020][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:37:07,020][ INFO][PID:3209902] VM allocation process starts [2026-01-22 16:37:07,030][ INFO][PID:3209902] Trying to allocate VM: ResallocHost, ticket_id=972843, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-22 16:37:10,041][ INFO][PID:3209902] Allocated host ResallocHost, ticket_id=972843, hostname=2620:52:6:1161:dead:beef:cafe:c14e, name=vmhost_p09_03_prod_01477138_20260122_160816, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-22 16:37:10,042][ INFO][PID:3209902] Allocating ssh connection to builder [2026-01-22 16:37:10,042][ INFO][PID:3209902] Checking that builder machine is OK [2026-01-22 16:37:16,870][ INFO][PID:3209902] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2026-01-22 16:37:20,166][ INFO][PID:3209902] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2026-01-22 16:37:20,167][ INFO][PID:3209902] Filling build.info file with builder info [2026-01-22 16:37:20,168][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:37:20,168][ INFO][PID:3209902] Not re-notifying FE and msg buses for the new host. [2026-01-22 16:37:20,169][ INFO][PID:3209902] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10048140-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2026-01-22 16:37:23,578][ INFO][PID:3209902] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c14e' (ED25519) to the list of known hosts. [2026-01-22 16:37:23,579][ INFO][PID:3209902] Downloading the builder-live.log file, attempt 1 [2026-01-22 16:37:23,580][ INFO][PID:3209902] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14e copr-rpmbuild-log [2026-01-22 16:37:28,586][ INFO][PID:3209902] Periodic builder liveness probe: alive [2026-01-22 16:37:28,587][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:37:33,588][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:37:38,589][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:37:43,590][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:37:48,592][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:37:53,594][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:37:58,595][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:38:03,596][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:38:08,597][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:38:13,598][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:38:18,599][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:38:23,601][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:38:28,603][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:38:33,605][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:38:38,606][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:38:43,607][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:38:48,608][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:38:53,609][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:38:58,611][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:39:03,613][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:39:08,615][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:39:13,617][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:39:18,619][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:39:23,621][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:39:28,624][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:39:33,631][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:39:38,634][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:39:43,637][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:39:48,639][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:39:53,640][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:39:58,641][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:40:03,643][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:40:08,644][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:40:13,648][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:40:18,650][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:40:23,651][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:40:28,653][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:40:33,656][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:40:38,658][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:40:43,659][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:40:48,660][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:40:53,661][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:40:58,662][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:41:03,664][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:41:08,665][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:41:13,666][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:41:18,667][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:41:23,673][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:41:28,674][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:41:33,675][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:41:38,676][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:41:43,678][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:41:48,679][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:41:53,680][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:41:58,681][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:42:03,682][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:42:08,685][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:42:13,690][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:42:18,695][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:42:23,696][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:42:28,697][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:42:33,698][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:42:38,700][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:42:43,701][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:42:48,711][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:42:53,718][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:42:58,719][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:43:03,721][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:43:08,727][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:43:13,729][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:43:18,730][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:43:23,733][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:43:28,735][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:43:33,736][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:43:38,737][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:43:43,740][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:43:48,741][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:43:53,743][ INFO][PID:3209902] Checking for cancel request [2026-01-22 16:43:54,845][ INFO][PID:3209902] Downloading results from builder [2026-01-22 16:43:54,846][ INFO][PID:3209902] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c14e]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock started [2026-01-22 16:43:54,847][ INFO][PID:3209902] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c14e]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/ &> /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/build-10048140.rsync.log [2026-01-22 16:44:54,929][ INFO][PID:3209902] rsyncing finished. [2026-01-22 16:44:54,929][ ERROR][PID:3209902] Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-01-22 16:44:54,930][ ERROR][PID:3209902] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details. [2026-01-22 16:45:04,930][ INFO][PID:3209902] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c14e]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock started [2026-01-22 16:45:04,932][ INFO][PID:3209902] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c14e]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/ &> /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/build-10048140.rsync.log [2026-01-22 16:45:08,160][ INFO][PID:3209902] rsyncing finished. [2026-01-22 16:45:08,160][ INFO][PID:3209902] Releasing VM back to pool [2026-01-22 16:45:08,169][ INFO][PID:3209902] Searching for 'success' file in resultdir [2026-01-22 16:45:08,170][ INFO][PID:3209902] Going to sign pkgs from source: 10048140-fedora-rawhide-ppc64le in chroot: /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le [2026-01-22 16:45:08,170][ INFO][PID:3209902] Calling '/bin/sign -u @mock#mock-stable@copr.fedorahosted.org -p' (attempt #1) [2026-01-22 16:45:08,360][ INFO][PID:3209902] Calling '/bin/sign -4 -h sha256 -u @mock#mock-stable@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/mock-lvm-6.6-1.fc44.noarch.rpm' (attempt #1) [2026-01-22 16:45:09,049][ INFO][PID:3209902] signed rpm: /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/mock-lvm-6.6-1.fc44.noarch.rpm [2026-01-22 16:45:09,051][ INFO][PID:3209902] Calling '/bin/sign -4 -h sha256 -u @mock#mock-stable@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/mock-6.6-1.fc44.src.rpm' (attempt #1) [2026-01-22 16:45:09,672][ INFO][PID:3209902] signed rpm: /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/mock-6.6-1.fc44.src.rpm [2026-01-22 16:45:09,673][ INFO][PID:3209902] Calling '/bin/sign -4 -h sha256 -u @mock#mock-stable@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/mock-filesystem-6.6-1.fc44.noarch.rpm' (attempt #1) [2026-01-22 16:45:10,493][ INFO][PID:3209902] signed rpm: /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/mock-filesystem-6.6-1.fc44.noarch.rpm [2026-01-22 16:45:10,495][ INFO][PID:3209902] Calling '/bin/sign -4 -h sha256 -u @mock#mock-stable@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/mock-scm-6.6-1.fc44.noarch.rpm' (attempt #1) [2026-01-22 16:45:11,099][ INFO][PID:3209902] signed rpm: /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/mock-scm-6.6-1.fc44.noarch.rpm [2026-01-22 16:45:11,100][ INFO][PID:3209902] Calling '/bin/sign -4 -h sha256 -u @mock#mock-stable@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/mock-6.6-1.fc44.noarch.rpm' (attempt #1) [2026-01-22 16:45:11,763][ INFO][PID:3209902] signed rpm: /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/mock-6.6-1.fc44.noarch.rpm [2026-01-22 16:45:11,764][ INFO][PID:3209902] Calling '/bin/sign -4 -h sha256 -u @mock#mock-stable@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/mock-rpmautospec-6.6-1.fc44.noarch.rpm' (attempt #1) [2026-01-22 16:45:12,380][ INFO][PID:3209902] signed rpm: /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/mock-rpmautospec-6.6-1.fc44.noarch.rpm [2026-01-22 16:45:12,381][ INFO][PID:3209902] Sign done [2026-01-22 16:45:12,382][ INFO][PID:3209902] Incremental createrepo run, adding 10048140-mock into https://download.copr.fedorainfracloud.org/results/@mock/mock-stable/fedora-rawhide-ppc64le, (auto-create-repo=True) [2026-01-22 16:45:12,383][ INFO][PID:3209902] Running command 'copr-repo --batched /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le --add 10048140-mock --no-appstream-metadata' as PID 3276664 [2026-01-22 16:45:14,348][ INFO][PID:3209902] Finished after 1 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le --add 10048140-mock --no-appstream-metadata) [2026-01-22 16:45:14,349][ INFO][PID:3209902] Getting build details [2026-01-22 16:45:14,350][ INFO][PID:3209902] Listing built binary packages in /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock [2026-01-22 16:45:14,350][ INFO][PID:3209902] Built packages: ['mock-rpmautospec 6.6', 'mock-filesystem 6.6', 'mock 6.6', 'mock-scm 6.6', 'mock-lvm 6.6'] [2026-01-22 16:45:14,350][ INFO][PID:3209902] build details: {'built_packages': 'mock-rpmautospec 6.6\nmock-filesystem 6.6\nmock 6.6\nmock-scm 6.6\nmock-lvm 6.6'} [2026-01-22 16:45:14,351][ INFO][PID:3209902] Retrieving pubkey [2026-01-22 16:45:14,352][ INFO][PID:3209902] Calling '/bin/sign -u @mock#mock-stable@copr.fedorahosted.org -p' (attempt #1) [2026-01-22 16:45:14,660][ INFO][PID:3209902] Added pubkey for user @mock project mock-stable into: /var/lib/copr/public_html/results/@mock/mock-stable/pubkey.gpg [2026-01-22 16:45:14,661][ INFO][PID:3209902] Finished build: id=10048140 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@mock/mock-stable chroot=fedora-rawhide-ppc64le [2026-01-22 16:45:14,664][ INFO][PID:3209902] Worker succeeded build, took 474.49533581733704 [2026-01-22 16:45:14,665][ INFO][PID:3209902] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@mock", "project_name": "mock-stable", "project_dirname": "mock-stable", "submitter": "https://api.github.com/users/Conan-Kudo", "ended_on": 1769100314.6641707, "started_on": 1769099840.168835, "submitted_on": null, "status": 1, "chroot": "fedora-rawhide-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "10048140-fedora-rawhide-ppc64le", "build_id": 10048140, "package_name": "mock", "package_version": "6.6-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@mock/mock-stable/mock", "git_hash": "6f6f9508977c4a3d53beb98306c7c0a416a9e11f", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@mock/mock-stable--https://api.github.com/users/Conan-Kudo", "results": { "packages": [ { "name": "mock", "epoch": null, "version": "6.6", "release": "1.fc44", "arch": "src" }, { "name": "mock-rpmautospec", "epoch": null, "version": "6.6", "release": "1.fc44", "arch": "noarch" }, { "name": "mock-filesystem", "epoch": null, "version": "6.6", "release": "1.fc44", "arch": "noarch" }, { "name": "mock", "epoch": null, "version": "6.6", "release": "1.fc44", "arch": "noarch" }, { "name": "mock-scm", "epoch": null, "version": "6.6", "release": "1.fc44", "arch": "noarch" }, { "name": "mock-lvm", "epoch": null, "version": "6.6", "release": "1.fc44", "arch": "noarch" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@mock/mock-stable/fedora-rawhide-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "bootstrap": "on", "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@mock/mock-stable", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@mock/mock-stable", "result_dir": "10048140-mock", "built_packages": "mock-rpmautospec 6.6\nmock-filesystem 6.6\nmock 6.6\nmock-scm 6.6\nmock-lvm 6.6", "id": 10048140, "mockchain_macros": { "copr_username": "@mock", "copr_projectname": "mock-stable", "vendor": "Fedora Project COPR (@mock/mock-stable)" } } ] } [2026-01-22 16:45:14,727][ INFO][PID:3209902] Sending fedora-messaging bus message in build.end [2026-01-22 16:45:16,125][ INFO][PID:3209902] Compressing /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/builder-live.log by gzip [2026-01-22 16:45:16,126][ INFO][PID:3209902] Running command 'gzip /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/builder-live.log' as PID 3277011 [2026-01-22 16:45:16,136][ INFO][PID:3209902] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/builder-live.log) [2026-01-22 16:45:16,137][ INFO][PID:3209902] Compressing /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/backend.log by gzip [2026-01-22 16:45:16,138][ INFO][PID:3209902] Running command 'gzip /var/lib/copr/public_html/results/@mock/mock-stable/fedora-rawhide-ppc64le/10048140-mock/backend.log' as PID 3277017