[2026-01-13 12:51:26,674][ INFO][PID:2653616] Marking build as starting [2026-01-13 12:51:26,820][ INFO][PID:2653616] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le//repodata/repomd.xml exists [2026-01-13 12:51:27,164][ INFO][PID:2653616] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le//repodata/repomd.xml exists [2026-01-13 12:51:27,517][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:51:27,519][ INFO][PID:2653616] VM allocation process starts [2026-01-13 12:51:27,534][ INFO][PID:2653616] Trying to allocate VM: ResallocHost, ticket_id=792619, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-13 12:51:32,535][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:51:37,536][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:51:42,537][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:51:47,538][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:51:52,539][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:51:57,540][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:52:02,541][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:52:07,542][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:52:12,543][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:52:17,544][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:52:22,545][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:52:27,547][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:52:32,548][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:52:37,549][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:52:42,550][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:52:47,553][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:52:52,555][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:52:57,556][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:53:02,557][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:53:07,561][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:53:12,563][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:53:17,564][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:53:22,565][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:53:27,566][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:53:32,567][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:53:37,568][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:53:42,570][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:53:45,720][ INFO][PID:2653616] Allocated host ResallocHost, ticket_id=792619, hostname=2620:52:6:1161:dead:beef:cafe:c154, name=vmhost_p09_03_prod_01178013_20260113_124753, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-13 12:53:45,721][ INFO][PID:2653616] Allocating ssh connection to builder [2026-01-13 12:53:45,721][ INFO][PID:2653616] Checking that builder machine is OK [2026-01-13 12:53:46,663][ INFO][PID:2653616] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2026-01-13 12:53:47,111][ INFO][PID:2653616] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2026-01-13 12:53:47,112][ INFO][PID:2653616] Filling build.info file with builder info [2026-01-13 12:53:47,113][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:53:47,114][ INFO][PID:2653616] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@copr", "project_name": "copr-pull-requests", "project_dirname": "copr-pull-requests:pr:4113", "submitter": null, "ended_on": null, "started_on": 1768308827.1141958, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "10003794-fedora-rawhide-ppc64le", "build_id": 10003794, "package_name": "copr-cli", "package_version": "2.4.post1-1.git.29.c362ab2", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@copr/copr-pull-requests:pr:4113/copr-cli", "git_hash": "2d6226129b561e4a34c88a1623545964f1d69bca", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@copr/copr-pull-requests--b0e1be89-edf7-40a7-91e7-5afae4ecf398", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 1, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/", "id": "copr_coprdir", "name": "Coprdir repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-dev/fedora-rawhide-ppc64le/", "id": "copr_copr_copr_dev", "name": "Additional repo copr_copr_copr_dev" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4113", "result_dir": "10003794-copr-cli", "built_packages": "", "id": 10003794, "mockchain_macros": { "copr_username": "@copr", "copr_projectname": "copr-pull-requests", "vendor": "Fedora Project COPR (@copr/copr-pull-requests)" } } ] } [2026-01-13 12:53:47,175][ INFO][PID:2653616] Sending fedora-messaging bus message in build.start [2026-01-13 12:53:47,803][ INFO][PID:2653616] Sending fedora-messaging bus message in chroot.start [2026-01-13 12:53:47,833][ INFO][PID:2653616] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10003794-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2026-01-13 12:53:48,794][ INFO][PID:2653616] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c154' (ED25519) to the list of known hosts. [2026-01-13 12:53:48,794][ INFO][PID:2653616] Downloading the builder-live.log file, attempt 1 [2026-01-13 12:53:48,796][ INFO][PID:2653616] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c154 copr-rpmbuild-log [2026-01-13 12:53:53,815][ INFO][PID:2653616] Periodic builder liveness probe: alive [2026-01-13 12:53:53,816][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:53:58,819][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:54:03,824][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:54:08,826][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:54:13,828][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:54:18,829][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:54:23,830][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:54:28,832][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:54:33,835][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:54:38,836][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:54:43,838][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:54:48,847][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:54:53,848][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:54:58,850][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:55:03,851][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:55:08,852][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:55:13,853][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:55:18,854][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:55:23,855][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:55:28,856][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:55:33,857][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:55:38,858][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:55:43,859][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:55:48,859][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:55:53,860][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:55:58,861][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:56:03,862][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:56:08,863][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:56:13,864][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:56:18,865][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:56:23,866][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:56:28,867][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:56:33,868][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:56:38,869][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:56:43,870][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:56:48,899][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:56:53,900][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:56:58,901][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:57:03,902][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:57:08,903][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:57:10,379][ ERROR][PID:2653616] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-13 12:57:13,904][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:57:18,905][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:57:20,380][WARNING][PID:2653616] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-13 12:57:23,906][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:57:28,907][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:57:30,381][ INFO][PID:2653616] Downloading the builder-live.log file, attempt 2 [2026-01-13 12:57:30,385][ INFO][PID:2653616] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c154 copr-rpmbuild-log [2026-01-13 12:57:33,908][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:57:38,909][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:57:43,909][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:57:48,910][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:57:53,911][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:57:58,913][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:58:03,913][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:58:08,915][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:58:13,915][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:58:18,917][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:58:23,918][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:58:28,919][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:58:30,453][ ERROR][PID:2653616] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-13 12:58:33,920][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:58:38,923][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:58:40,474][WARNING][PID:2653616] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-13 12:58:43,924][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:58:48,925][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:58:50,475][ INFO][PID:2653616] Downloading the builder-live.log file, attempt 3 [2026-01-13 12:58:50,595][ INFO][PID:2653616] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c154 copr-rpmbuild-log [2026-01-13 12:58:53,927][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:58:58,930][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:59:03,931][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:59:08,932][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:59:13,933][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:59:18,934][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:59:23,936][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:59:28,937][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:59:33,938][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:59:38,939][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:59:43,940][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:59:48,941][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:59:50,664][ ERROR][PID:2653616] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-13 12:59:53,943][ INFO][PID:2653616] Checking for cancel request [2026-01-13 12:59:58,944][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:00:00,682][WARNING][PID:2653616] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-13 13:00:03,947][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:00:08,948][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:00:10,683][ INFO][PID:2653616] Downloading the builder-live.log file, attempt 4 [2026-01-13 13:00:10,830][ INFO][PID:2653616] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c154 copr-rpmbuild-log [2026-01-13 13:00:13,949][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:00:18,950][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:00:23,951][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:00:28,953][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:00:33,954][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:00:38,955][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:00:43,956][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:00:48,957][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:00:53,958][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:00:58,960][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:01:03,960][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:01:08,961][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:01:10,898][ ERROR][PID:2653616] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-13 13:01:13,963][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:01:18,965][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:01:20,918][ ERROR][PID:2653616] Giving up for unstable SSH, failures: 1768309040.380365, 1768309120.474686, 1768309200.6825657, 1768309280.9185452 [2026-01-13 13:01:20,920][ ERROR][PID:2653616] Re-try request for task on 'ResallocHost, ticket_id=792619, hostname=2620:52:6:1161:dead:beef:cafe:c154, name=vmhost_p09_03_prod_01178013_20260113_124753, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-01-13 13:01:20,921][ INFO][PID:2653616] Releasing VM back to pool [2026-01-13 13:01:20,941][ INFO][PID:2653616] Retry #1 (on other host) [2026-01-13 13:01:20,944][ INFO][PID:2653616] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le//repodata/repomd.xml exists [2026-01-13 13:01:21,254][ INFO][PID:2653616] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le//repodata/repomd.xml exists [2026-01-13 13:01:21,477][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:01:21,478][ INFO][PID:2653616] VM allocation process starts [2026-01-13 13:01:21,492][ INFO][PID:2653616] Trying to allocate VM: ResallocHost, ticket_id=792734, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-13 13:01:26,493][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:01:27,520][ INFO][PID:2653616] Allocated host ResallocHost, ticket_id=792734, hostname=2620:52:6:1161:dead:beef:cafe:c18b, name=vmhost_p09_04_prod_01178116_20260113_125228, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-13 13:01:27,522][ INFO][PID:2653616] Allocating ssh connection to builder [2026-01-13 13:01:27,522][ INFO][PID:2653616] Checking that builder machine is OK [2026-01-13 13:01:28,571][ INFO][PID:2653616] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2026-01-13 13:01:29,069][ INFO][PID:2653616] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2026-01-13 13:01:29,070][ INFO][PID:2653616] Filling build.info file with builder info [2026-01-13 13:01:29,090][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:01:29,091][ INFO][PID:2653616] Not re-notifying FE and msg buses for the new host. [2026-01-13 13:01:29,091][ INFO][PID:2653616] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10003794-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2026-01-13 13:01:30,064][ INFO][PID:2653616] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18b' (ED25519) to the list of known hosts. [2026-01-13 13:01:30,065][ INFO][PID:2653616] Downloading the builder-live.log file, attempt 1 [2026-01-13 13:01:30,066][ INFO][PID:2653616] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18b copr-rpmbuild-log [2026-01-13 13:01:35,065][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:01:40,066][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:01:45,067][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:01:50,068][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:01:55,069][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:02:00,071][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:02:05,073][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:02:10,075][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:02:15,077][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:02:20,078][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:02:25,083][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:02:30,085][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:02:35,087][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:02:40,088][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:02:45,091][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:02:50,095][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:02:55,096][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:03:00,097][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:03:05,098][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:03:10,099][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:03:15,101][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:03:20,103][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:03:25,105][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:03:30,107][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:03:35,110][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:03:40,113][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:03:45,114][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:03:50,115][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:03:55,135][ INFO][PID:2653616] Periodic builder liveness probe: alive [2026-01-13 13:03:55,138][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:04:00,140][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:04:05,141][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:04:10,143][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:04:15,144][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:04:20,144][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:04:25,145][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:04:30,146][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:04:35,148][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:04:40,149][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:04:45,150][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:04:50,151][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:04:55,152][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:05:00,153][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:05:05,154][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:05:10,156][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:05:15,157][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:05:20,158][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:05:25,159][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:05:30,165][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:05:35,167][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:05:40,171][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:05:45,172][ INFO][PID:2653616] Checking for cancel request [2026-01-13 13:05:45,198][ INFO][PID:2653616] Downloading results from builder [2026-01-13 13:05:45,199][ INFO][PID:2653616] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18b]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli started [2026-01-13 13:05:45,201][ INFO][PID:2653616] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18b]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/ &> /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/build-10003794.rsync.log [2026-01-13 13:05:45,870][ INFO][PID:2653616] rsyncing finished. [2026-01-13 13:05:45,871][ INFO][PID:2653616] Releasing VM back to pool [2026-01-13 13:05:45,923][ INFO][PID:2653616] Searching for 'success' file in resultdir [2026-01-13 13:05:45,924][ INFO][PID:2653616] Going to sign pkgs from source: 10003794-fedora-rawhide-ppc64le in chroot: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le [2026-01-13 13:05:45,925][ INFO][PID:2653616] Calling '/bin/sign -u @copr#copr-pull-requests@copr.fedorahosted.org -p' (attempt #1) [2026-01-13 13:05:46,192][ INFO][PID:2653616] Calling '/bin/sign -4 -h sha256 -u @copr#copr-pull-requests@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/copr-cli-2.4.post1-1.git.29.c362ab2.fc44.noarch.rpm' (attempt #1) [2026-01-13 13:05:46,927][ INFO][PID:2653616] signed rpm: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/copr-cli-2.4.post1-1.git.29.c362ab2.fc44.noarch.rpm [2026-01-13 13:05:46,927][ INFO][PID:2653616] Calling '/bin/sign -4 -h sha256 -u @copr#copr-pull-requests@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/copr-cli-2.4.post1-1.git.29.c362ab2.fc44.src.rpm' (attempt #1) [2026-01-13 13:05:47,655][ INFO][PID:2653616] signed rpm: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/copr-cli-2.4.post1-1.git.29.c362ab2.fc44.src.rpm [2026-01-13 13:05:47,656][ INFO][PID:2653616] Sign done [2026-01-13 13:05:47,657][ INFO][PID:2653616] Pulp: create_content: /api/v3/content/rpm/packages/upload/ /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/copr-cli-2.4.post1-1.git.29.c362ab2.fc44.noarch.rpm [2026-01-13 13:05:48,040][ INFO][PID:2653616] [1/2] Uploaded to Pulp: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/copr-cli-2.4.post1-1.git.29.c362ab2.fc44.noarch.rpm [2026-01-13 13:05:48,040][ INFO][PID:2653616] Pulp: create_content: /api/v3/content/rpm/packages/upload/ /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/copr-cli-2.4.post1-1.git.29.c362ab2.fc44.src.rpm [2026-01-13 13:05:48,346][ INFO][PID:2653616] [2/2] Uploaded to Pulp: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/copr-cli-2.4.post1-1.git.29.c362ab2.fc44.src.rpm [2026-01-13 13:05:48,349][ INFO][PID:2653616] Pulp: get_repository: /api/v3/repositories/rpm/rpm/?name=%40copr%2Fcopr-pull-requests%3Apr%3A4113%2Ffedora-rawhide-ppc64le&offset=0&limit=1 [2026-01-13 13:05:48,630][ INFO][PID:2653616] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019bb776-6631-7baf-b66b-60be5efc0f49/ [2026-01-13 13:05:48,631][ INFO][PID:2653616] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019bb776-6631-7baf-b66b-60be5efc0f49/ [2026-01-13 13:05:53,760][ INFO][PID:2653616] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019bb776-6631-7baf-b66b-60be5efc0f49/ [2026-01-13 13:05:53,761][ INFO][PID:2653616] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019bb776-6631-7baf-b66b-60be5efc0f49/ [2026-01-13 13:05:53,921][ INFO][PID:2653616] Successfully modified Pulp repository content /api/pulp/public-copr/api/v3/repositories/rpm/rpm/019ba31e-95e5-7339-a334-274dda4ba11d/ [2026-01-13 13:05:53,921][ INFO][PID:2653616] Pulp: publishing /api/v3/publications/rpm/rpm/ /api/pulp/public-copr/api/v3/repositories/rpm/rpm/019ba31e-95e5-7339-a334-274dda4ba11d/ [2026-01-13 13:05:54,067][ INFO][PID:2653616] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019bb776-7b68-7b7d-a7ae-61748857d655/ [2026-01-13 13:05:54,070][ INFO][PID:2653616] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019bb776-7b68-7b7d-a7ae-61748857d655/ [2026-01-13 13:05:59,231][ INFO][PID:2653616] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019bb776-7b68-7b7d-a7ae-61748857d655/ [2026-01-13 13:05:59,232][ INFO][PID:2653616] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019bb776-7b68-7b7d-a7ae-61748857d655/ [2026-01-13 13:05:59,380][ INFO][PID:2653616] Successfully created Pulp publication of repository /api/pulp/public-copr/api/v3/repositories/rpm/rpm/019ba31e-95e5-7339-a334-274dda4ba11d/ [2026-01-13 13:05:59,384][ INFO][PID:2653616] Pulp: add_content: /api/pulp/public-copr/api/v3/repositories/rpm/rpm/019ba31e-95e5-7339-a334-274dda4ba11d/ (['/api/pulp/public-copr/api/v3/content/rpm/packages/019bb776-63a9-708d-a170-5da8f2386b55/', '/api/pulp/public-copr/api/v3/content/rpm/packages/019bb776-64f2-7ca3-bebb-c90e46583c41/']) [2026-01-13 13:05:59,387][ INFO][PID:2653616] Getting build details [2026-01-13 13:05:59,387][ INFO][PID:2653616] Listing built binary packages in /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli [2026-01-13 13:05:59,388][ INFO][PID:2653616] Built packages: ['copr-cli 2.4.post1'] [2026-01-13 13:05:59,388][ INFO][PID:2653616] build details: {'built_packages': 'copr-cli 2.4.post1'} [2026-01-13 13:05:59,389][ INFO][PID:2653616] Retrieving pubkey [2026-01-13 13:05:59,389][ INFO][PID:2653616] Calling '/bin/sign -u @copr#copr-pull-requests@copr.fedorahosted.org -p' (attempt #1) [2026-01-13 13:05:59,724][ INFO][PID:2653616] Added pubkey for user @copr project copr-pull-requests into: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/pubkey.gpg [2026-01-13 13:05:59,724][ INFO][PID:2653616] Removing /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/copr-cli-2.4.post1-1.git.29.c362ab2.fc44.noarch.rpm, it is stored in Pulp [2026-01-13 13:05:59,725][ INFO][PID:2653616] Removing /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/copr-cli-2.4.post1-1.git.29.c362ab2.fc44.src.rpm, it is stored in Pulp [2026-01-13 13:05:59,726][ INFO][PID:2653616] Finished build: id=10003794 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113 chroot=fedora-rawhide-ppc64le [2026-01-13 13:05:59,727][ INFO][PID:2653616] Worker succeeded build, took 270.63669180870056 [2026-01-13 13:05:59,728][ INFO][PID:2653616] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@copr", "project_name": "copr-pull-requests", "project_dirname": "copr-pull-requests:pr:4113", "submitter": null, "ended_on": 1768309559.7277844, "started_on": 1768309289.0910926, "submitted_on": null, "status": 1, "chroot": "fedora-rawhide-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "10003794-fedora-rawhide-ppc64le", "build_id": 10003794, "package_name": "copr-cli", "package_version": "2.4.post1-1.git.29.c362ab2", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@copr/copr-pull-requests:pr:4113/copr-cli", "git_hash": "2d6226129b561e4a34c88a1623545964f1d69bca", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@copr/copr-pull-requests--b0e1be89-edf7-40a7-91e7-5afae4ecf398", "results": { "packages": [ { "name": "copr-cli", "epoch": null, "version": "2.4.post1", "release": "1.git.29.c362ab2.fc44", "arch": "src" }, { "name": "copr-cli", "epoch": null, "version": "2.4.post1", "release": "1.git.29.c362ab2.fc44", "arch": "noarch" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 1, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/", "id": "copr_coprdir", "name": "Coprdir repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-dev/fedora-rawhide-ppc64le/", "id": "copr_copr_copr_dev", "name": "Additional repo copr_copr_copr_dev" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4113", "result_dir": "10003794-copr-cli", "built_packages": "copr-cli 2.4.post1", "id": 10003794, "mockchain_macros": { "copr_username": "@copr", "copr_projectname": "copr-pull-requests", "vendor": "Fedora Project COPR (@copr/copr-pull-requests)" } } ] } [2026-01-13 13:05:59,786][ INFO][PID:2653616] Sending fedora-messaging bus message in build.end [2026-01-13 13:05:59,815][ INFO][PID:2653616] Compressing /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/builder-live.log by gzip [2026-01-13 13:05:59,817][ INFO][PID:2653616] Running command 'gzip /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/builder-live.log' as PID 2759417 [2026-01-13 13:05:59,866][ INFO][PID:2653616] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/builder-live.log) [2026-01-13 13:05:59,867][ INFO][PID:2653616] Compressing /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/backend.log by gzip [2026-01-13 13:05:59,869][ INFO][PID:2653616] Running command 'gzip /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4113/fedora-rawhide-ppc64le/10003794-copr-cli/backend.log' as PID 2759426