[2025-12-28 03:02:33,987][ INFO][PID:171682] Marking build as starting [2025-12-28 03:02:34,064][ INFO][PID:171682] Checking that /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/repodata/repomd.xml exists [2025-12-28 03:02:34,074][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:02:34,075][ INFO][PID:171682] VM allocation process starts [2025-12-28 03:02:34,099][ INFO][PID:171682] Trying to allocate VM: ResallocHost, ticket_id=540795, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-28 03:02:39,100][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:02:44,101][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:02:49,245][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:02:54,246][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:02:59,247][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:03:04,247][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:03:09,248][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:03:14,249][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:03:19,250][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:03:24,251][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:03:29,252][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:03:34,309][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:03:39,310][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:03:44,311][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:03:49,344][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:03:54,346][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:03:59,347][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:04:04,613][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:04:09,614][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:04:14,615][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:04:19,616][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:04:24,617][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:04:29,618][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:04:34,619][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:04:39,620][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:04:44,621][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:04:49,622][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:04:52,194][ INFO][PID:171682] Allocated host ResallocHost, ticket_id=540795, hostname=2620:52:6:1161:dead:beef:cafe:c18a, name=vmhost_p09_04_prod_00795569_20251228_030024, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-28 03:04:52,195][ INFO][PID:171682] Allocating ssh connection to builder [2025-12-28 03:04:52,195][ INFO][PID:171682] Checking that builder machine is OK [2025-12-28 03:04:53,084][ INFO][PID:171682] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2025-12-28 03:04:53,549][ INFO][PID:171682] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2025-12-28 03:04:53,550][ INFO][PID:171682] Filling build.info file with builder info [2025-12-28 03:04:53,550][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:04:53,551][ INFO][PID:171682] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "amatej", "project_name": "dnf-nightly-sanitizers", "project_dirname": "dnf-nightly-sanitizers", "submitter": "amatej", "ended_on": null, "started_on": 1766891093.5510612, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9954071-fedora-rawhide-ppc64le", "build_id": 9954071, "package_name": "dnf-plugins-core", "package_version": "4.10.1-1.20251228025150161848.master.9.gfa966d8", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/amatej/dnf-nightly-sanitizers/dnf-plugins-core", "git_hash": "95986da92758e0580a338749d4dab64411f374f6", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "amatej/dnf-nightly-sanitizers--amatej", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/amatej/dnf-nightly-sanitizers", "result_dir": "09954071-dnf-plugins-core", "built_packages": "", "id": 9954071, "mockchain_macros": { "copr_username": "amatej", "copr_projectname": "dnf-nightly-sanitizers", "vendor": "Fedora Project COPR (amatej/dnf-nightly-sanitizers)" } } ] } [2025-12-28 03:04:53,621][ INFO][PID:171682] Sending fedora-messaging bus message in build.start [2025-12-28 03:04:54,125][ INFO][PID:171682] Sending fedora-messaging bus message in chroot.start [2025-12-28 03:04:54,152][ INFO][PID:171682] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9954071-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2025-12-28 03:04:55,128][ INFO][PID:171682] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18a' (ED25519) to the list of known hosts. [2025-12-28 03:04:55,129][ INFO][PID:171682] Downloading the builder-live.log file, attempt 1 [2025-12-28 03:04:55,212][ INFO][PID:171682] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18a copr-rpmbuild-log [2025-12-28 03:05:00,134][ INFO][PID:171682] Periodic builder liveness probe: alive [2025-12-28 03:05:00,135][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:05:05,136][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:05:10,137][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:05:15,138][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:05:20,139][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:05:25,140][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:05:30,141][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:05:35,142][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:05:40,142][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:05:45,143][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:05:50,144][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:05:55,145][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:06:00,146][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:06:05,147][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:06:10,148][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:06:15,647][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:06:20,648][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:06:25,649][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:06:30,650][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:06:35,651][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:06:40,652][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:06:45,653][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:06:50,654][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:06:55,655][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:07:00,658][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:07:05,662][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:07:10,664][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:07:15,665][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:07:20,666][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:07:25,667][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:07:30,668][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:07:35,669][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:07:40,670][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:07:45,677][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:07:50,680][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:07:55,681][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:08:00,682][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:08:05,684][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:08:10,687][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:08:15,689][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:08:20,691][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:08:25,692][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:08:30,693][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:08:35,695][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:08:40,698][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:08:45,700][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:08:50,703][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:08:55,705][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:09:00,706][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:09:05,712][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:09:10,720][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:09:15,721][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:09:20,724][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:09:25,726][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:09:30,728][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:09:35,731][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:09:36,370][ ERROR][PID:171682] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-28 03:09:40,734][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:09:45,737][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:09:46,372][WARNING][PID:171682] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-28 03:09:50,739][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:09:55,743][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:09:56,376][ INFO][PID:171682] Downloading the builder-live.log file, attempt 2 [2025-12-28 03:09:56,380][ INFO][PID:171682] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18a copr-rpmbuild-log [2025-12-28 03:10:00,745][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:10:05,746][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:10:10,748][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:10:15,750][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:10:20,755][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:10:25,761][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:10:30,763][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:10:35,766][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:10:40,771][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:10:45,774][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:10:50,776][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:10:55,778][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:10:56,475][ ERROR][PID:171682] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-28 03:11:00,779][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:11:05,780][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:11:06,486][WARNING][PID:171682] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-28 03:11:10,782][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:11:15,784][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:11:16,487][ INFO][PID:171682] Downloading the builder-live.log file, attempt 3 [2025-12-28 03:11:16,489][ INFO][PID:171682] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18a copr-rpmbuild-log [2025-12-28 03:11:20,790][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:11:25,791][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:11:30,792][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:11:35,794][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:11:40,796][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:11:45,797][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:11:50,798][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:11:55,799][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:12:00,800][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:12:05,801][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:12:10,802][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:12:15,803][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:12:16,557][ ERROR][PID:171682] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-28 03:12:20,804][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:12:25,805][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:12:26,567][WARNING][PID:171682] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-28 03:12:30,807][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:12:35,808][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:12:36,568][ INFO][PID:171682] Downloading the builder-live.log file, attempt 4 [2025-12-28 03:12:36,569][ INFO][PID:171682] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18a copr-rpmbuild-log [2025-12-28 03:12:40,809][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:12:45,810][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:12:50,811][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:12:55,813][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:13:00,815][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:13:05,817][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:13:10,818][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:13:15,823][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:13:20,825][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:13:25,826][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:13:30,827][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:13:35,830][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:13:36,638][ ERROR][PID:171682] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-28 03:13:40,833][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:13:45,837][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:13:46,659][ ERROR][PID:171682] Giving up for unstable SSH, failures: 1766891386.3727267, 1766891466.486248, 1766891546.5677679, 1766891626.6588335 [2025-12-28 03:13:46,660][ ERROR][PID:171682] Re-try request for task on 'ResallocHost, ticket_id=540795, hostname=2620:52:6:1161:dead:beef:cafe:c18a, name=vmhost_p09_04_prod_00795569_20251228_030024, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-12-28 03:13:46,662][ INFO][PID:171682] Releasing VM back to pool [2025-12-28 03:13:46,687][ INFO][PID:171682] Retry #1 (on other host) [2025-12-28 03:13:46,689][ INFO][PID:171682] Checking that /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/repodata/repomd.xml exists [2025-12-28 03:13:46,694][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:13:46,696][ INFO][PID:171682] VM allocation process starts [2025-12-28 03:13:46,713][ INFO][PID:171682] Trying to allocate VM: ResallocHost, ticket_id=540941, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-28 03:13:51,717][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:13:56,723][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:14:01,724][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:14:06,725][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:14:11,727][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:14:16,728][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:14:21,729][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:14:26,730][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:14:31,731][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:14:34,814][ INFO][PID:171682] Allocated host ResallocHost, ticket_id=540941, hostname=140.211.168.229, name=copr_osuosl_p09_prod_00795554_20251228_025741, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-28 03:14:34,814][ INFO][PID:171682] Allocating ssh connection to builder [2025-12-28 03:14:34,815][ INFO][PID:171682] Checking that builder machine is OK [2025-12-28 03:14:36,489][ INFO][PID:171682] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2025-12-28 03:14:37,556][ INFO][PID:171682] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2025-12-28 03:14:37,557][ INFO][PID:171682] Filling build.info file with builder info [2025-12-28 03:14:37,577][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:14:37,578][ INFO][PID:171682] Not re-notifying FE and msg buses for the new host. [2025-12-28 03:14:37,579][ INFO][PID:171682] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9954071-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2025-12-28 03:14:38,994][ INFO][PID:171682] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '140.211.168.229' (ED25519) to the list of known hosts. [2025-12-28 03:14:38,995][ INFO][PID:171682] Downloading the builder-live.log file, attempt 1 [2025-12-28 03:14:38,997][ INFO][PID:171682] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.229 copr-rpmbuild-log [2025-12-28 03:14:43,996][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:14:48,997][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:14:53,998][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:14:58,999][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:15:04,008][ INFO][PID:171682] Periodic builder liveness probe: alive [2025-12-28 03:15:04,009][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:15:09,009][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:15:14,011][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:15:19,011][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:15:24,014][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:15:29,015][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:15:34,016][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:15:39,017][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:15:44,018][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:15:49,019][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:15:54,020][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:15:59,021][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:16:04,022][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:16:09,023][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:16:14,024][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:16:19,026][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:16:24,027][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:16:29,028][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:16:34,029][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:16:39,030][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:16:44,031][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:16:49,032][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:16:54,033][ INFO][PID:171682] Checking for cancel request [2025-12-28 03:16:57,140][ INFO][PID:171682] Downloading results from builder [2025-12-28 03:16:57,140][ INFO][PID:171682] rsyncing of mockbuilder@140.211.168.229:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core started [2025-12-28 03:16:57,141][ INFO][PID:171682] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@140.211.168.229:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/ &> /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/build-09954071.rsync.log [2025-12-28 03:16:59,735][ INFO][PID:171682] rsyncing finished. [2025-12-28 03:16:59,736][ INFO][PID:171682] Releasing VM back to pool [2025-12-28 03:16:59,744][ INFO][PID:171682] Searching for 'success' file in resultdir [2025-12-28 03:16:59,744][ INFO][PID:171682] Going to sign pkgs from source: 9954071-fedora-rawhide-ppc64le in chroot: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le [2025-12-28 03:16:59,745][ INFO][PID:171682] Calling '/bin/sign -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -p' (attempt #1) [2025-12-28 03:17:00,018][ INFO][PID:171682] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugins-core-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm' (attempt #1) [2025-12-28 03:17:01,121][ INFO][PID:171682] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugins-core-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm [2025-12-28 03:17:01,121][ INFO][PID:171682] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-local-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm' (attempt #1) [2025-12-28 03:17:01,958][ INFO][PID:171682] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-local-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm [2025-12-28 03:17:01,959][ INFO][PID:171682] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-pre-transaction-actions-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm' (attempt #1) [2025-12-28 03:17:02,785][ INFO][PID:171682] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-pre-transaction-actions-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm [2025-12-28 03:17:02,785][ INFO][PID:171682] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/dnf-utils-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm' (attempt #1) [2025-12-28 03:17:03,622][ INFO][PID:171682] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/dnf-utils-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm [2025-12-28 03:17:03,623][ INFO][PID:171682] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-show-leaves-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm' (attempt #1) [2025-12-28 03:17:04,453][ INFO][PID:171682] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-show-leaves-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm [2025-12-28 03:17:04,454][ INFO][PID:171682] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/dnf-plugins-core-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.src.rpm' (attempt #1) [2025-12-28 03:17:05,299][ INFO][PID:171682] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/dnf-plugins-core-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.src.rpm [2025-12-28 03:17:05,299][ INFO][PID:171682] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-post-transaction-actions-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm' (attempt #1) [2025-12-28 03:17:06,988][ INFO][PID:171682] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-post-transaction-actions-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm [2025-12-28 03:17:06,988][ INFO][PID:171682] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/dnf-plugins-core-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm' (attempt #1) [2025-12-28 03:17:07,852][ INFO][PID:171682] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/dnf-plugins-core-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm [2025-12-28 03:17:07,852][ INFO][PID:171682] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-versionlock-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm' (attempt #1) [2025-12-28 03:17:08,679][ INFO][PID:171682] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-versionlock-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm [2025-12-28 03:17:08,680][ INFO][PID:171682] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-leaves-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm' (attempt #1) [2025-12-28 03:17:09,510][ INFO][PID:171682] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-leaves-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm [2025-12-28 03:17:09,511][ INFO][PID:171682] Calling '/bin/sign -4 -h sha256 -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-modulesync-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm' (attempt #1) [2025-12-28 03:17:10,353][ INFO][PID:171682] signed rpm: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/python3-dnf-plugin-modulesync-4.10.1-1.20251228025150161848.master.9.gfa966d8.fc44.noarch.rpm [2025-12-28 03:17:10,354][ INFO][PID:171682] Sign done [2025-12-28 03:17:10,356][ INFO][PID:171682] Incremental createrepo run, adding 09954071-dnf-plugins-core into https://download.copr.fedorainfracloud.org/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le, (auto-create-repo=True) [2025-12-28 03:17:10,360][ INFO][PID:171682] Running command 'copr-repo --batched /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le --add 09954071-dnf-plugins-core --no-appstream-metadata' as PID 249202 [2025-12-28 03:17:12,453][ INFO][PID:171682] Finished after 2 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le --add 09954071-dnf-plugins-core --no-appstream-metadata) [2025-12-28 03:17:12,455][ INFO][PID:171682] Getting build details [2025-12-28 03:17:12,456][ INFO][PID:171682] Listing built binary packages in /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core [2025-12-28 03:17:12,456][ INFO][PID:171682] Built packages: ['python3-dnf-plugin-show-leaves 4.10.1', 'python3-dnf-plugin-versionlock 4.10.1', 'python3-dnf-plugin-leaves 4.10.1', 'python3-dnf-plugin-pre-transaction-actions 4.10.1', 'python3-dnf-plugin-modulesync 4.10.1', 'dnf-plugins-core 4.10.1', 'python3-dnf-plugin-post-transaction-actions 4.10.1', 'dnf-utils 4.10.1', 'python3-dnf-plugin-local 4.10.1', 'python3-dnf-plugins-core 4.10.1'] [2025-12-28 03:17:12,459][ INFO][PID:171682] build details: {'built_packages': 'python3-dnf-plugin-show-leaves 4.10.1\npython3-dnf-plugin-versionlock 4.10.1\npython3-dnf-plugin-leaves 4.10.1\npython3-dnf-plugin-pre-transaction-actions 4.10.1\npython3-dnf-plugin-modulesync 4.10.1\ndnf-plugins-core 4.10.1\npython3-dnf-plugin-post-transaction-actions 4.10.1\ndnf-utils 4.10.1\npython3-dnf-plugin-local 4.10.1\npython3-dnf-plugins-core 4.10.1'} [2025-12-28 03:17:12,461][ INFO][PID:171682] Retrieving pubkey [2025-12-28 03:17:12,465][ INFO][PID:171682] Calling '/bin/sign -u amatej#dnf-nightly-sanitizers@copr.fedorahosted.org -p' (attempt #1) [2025-12-28 03:17:12,751][ INFO][PID:171682] Added pubkey for user amatej project dnf-nightly-sanitizers into: /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/pubkey.gpg [2025-12-28 03:17:12,752][ INFO][PID:171682] Finished build: id=9954071 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers chroot=fedora-rawhide-ppc64le [2025-12-28 03:17:12,754][ INFO][PID:171682] Worker succeeded build, took 155.17592215538025 [2025-12-28 03:17:12,755][ INFO][PID:171682] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "amatej", "project_name": "dnf-nightly-sanitizers", "project_dirname": "dnf-nightly-sanitizers", "submitter": "amatej", "ended_on": 1766891832.754454, "started_on": 1766891677.5785317, "submitted_on": null, "status": 1, "chroot": "fedora-rawhide-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9954071-fedora-rawhide-ppc64le", "build_id": 9954071, "package_name": "dnf-plugins-core", "package_version": "4.10.1-1.20251228025150161848.master.9.gfa966d8", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/amatej/dnf-nightly-sanitizers/dnf-plugins-core", "git_hash": "95986da92758e0580a338749d4dab64411f374f6", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "amatej/dnf-nightly-sanitizers--amatej", "results": { "packages": [ { "name": "python3-dnf-plugin-show-leaves", "epoch": null, "version": "4.10.1", "release": "1.20251228025150161848.master.9.gfa966d8.fc44", "arch": "noarch" }, { "name": "python3-dnf-plugin-versionlock", "epoch": null, "version": "4.10.1", "release": "1.20251228025150161848.master.9.gfa966d8.fc44", "arch": "noarch" }, { "name": "python3-dnf-plugin-leaves", "epoch": null, "version": "4.10.1", "release": "1.20251228025150161848.master.9.gfa966d8.fc44", "arch": "noarch" }, { "name": "python3-dnf-plugin-pre-transaction-actions", "epoch": null, "version": "4.10.1", "release": "1.20251228025150161848.master.9.gfa966d8.fc44", "arch": "noarch" }, { "name": "python3-dnf-plugin-modulesync", "epoch": null, "version": "4.10.1", "release": "1.20251228025150161848.master.9.gfa966d8.fc44", "arch": "noarch" }, { "name": "dnf-plugins-core", "epoch": null, "version": "4.10.1", "release": "1.20251228025150161848.master.9.gfa966d8.fc44", "arch": "noarch" }, { "name": "python3-dnf-plugin-post-transaction-actions", "epoch": null, "version": "4.10.1", "release": "1.20251228025150161848.master.9.gfa966d8.fc44", "arch": "noarch" }, { "name": "dnf-utils", "epoch": null, "version": "4.10.1", "release": "1.20251228025150161848.master.9.gfa966d8.fc44", "arch": "noarch" }, { "name": "python3-dnf-plugin-local", "epoch": null, "version": "4.10.1", "release": "1.20251228025150161848.master.9.gfa966d8.fc44", "arch": "noarch" }, { "name": "dnf-plugins-core", "epoch": null, "version": "4.10.1", "release": "1.20251228025150161848.master.9.gfa966d8.fc44", "arch": "src" }, { "name": "python3-dnf-plugins-core", "epoch": null, "version": "4.10.1", "release": "1.20251228025150161848.master.9.gfa966d8.fc44", "arch": "noarch" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/amatej/dnf-nightly-sanitizers", "result_dir": "09954071-dnf-plugins-core", "built_packages": "python3-dnf-plugin-show-leaves 4.10.1\npython3-dnf-plugin-versionlock 4.10.1\npython3-dnf-plugin-leaves 4.10.1\npython3-dnf-plugin-pre-transaction-actions 4.10.1\npython3-dnf-plugin-modulesync 4.10.1\ndnf-plugins-core 4.10.1\npython3-dnf-plugin-post-transaction-actions 4.10.1\ndnf-utils 4.10.1\npython3-dnf-plugin-local 4.10.1\npython3-dnf-plugins-core 4.10.1", "id": 9954071, "mockchain_macros": { "copr_username": "amatej", "copr_projectname": "dnf-nightly-sanitizers", "vendor": "Fedora Project COPR (amatej/dnf-nightly-sanitizers)" } } ] } [2025-12-28 03:17:12,856][ INFO][PID:171682] Sending fedora-messaging bus message in build.end [2025-12-28 03:17:12,891][ INFO][PID:171682] Compressing /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/builder-live.log by gzip [2025-12-28 03:17:12,892][ INFO][PID:171682] Running command 'gzip /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/builder-live.log' as PID 249457 [2025-12-28 03:17:12,901][ INFO][PID:171682] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/builder-live.log) [2025-12-28 03:17:12,901][ INFO][PID:171682] Compressing /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/backend.log by gzip [2025-12-28 03:17:12,902][ INFO][PID:171682] Running command 'gzip /var/lib/copr/public_html/results/amatej/dnf-nightly-sanitizers/fedora-rawhide-ppc64le/09954071-dnf-plugins-core/backend.log' as PID 249459