[2026-01-11 14:26:47,443][  INFO][PID:1301366] Marking build as starting
[2026-01-11 14:26:47,485][  INFO][PID:1301366] Checking that /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/repodata/repomd.xml exists
[2026-01-11 14:26:47,540][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:26:47,540][  INFO][PID:1301366] VM allocation process starts
[2026-01-11 14:26:47,578][  INFO][PID:1301366] Trying to allocate VM: ResallocHost, ticket_id=758158, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-11 14:26:50,631][  INFO][PID:1301366] Allocated host ResallocHost, ticket_id=758158, hostname=2620:52:6:1161:dead:beef:cafe:c193, name=vmhost_p09_04_prod_01099313_20260111_140043, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-11 14:26:51,149][  INFO][PID:1301366] Allocating ssh connection to builder
[2026-01-11 14:26:51,149][  INFO][PID:1301366] Checking that builder machine is OK
[2026-01-11 14:26:52,133][  INFO][PID:1301366] Running remote command: copr-builder-ready fedora-42-ppc64le
[2026-01-11 14:27:52,185][ ERROR][PID:1301366] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c193 port 22: Connection timed out

[2026-01-11 14:28:02,186][ ERROR][PID:1301366] Re-try request for task on 'ResallocHost, ticket_id=758158, hostname=2620:52:6:1161:dead:beef:cafe:c193, name=vmhost_p09_04_prod_01099313_20260111_140043, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-01-11 14:28:02,187][  INFO][PID:1301366] Releasing VM back to pool
[2026-01-11 14:28:02,194][  INFO][PID:1301366] Retry #1 (on other host)
[2026-01-11 14:28:02,195][  INFO][PID:1301366] Checking that /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/repodata/repomd.xml exists
[2026-01-11 14:28:02,196][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:28:02,196][  INFO][PID:1301366] VM allocation process starts
[2026-01-11 14:28:02,260][  INFO][PID:1301366] Trying to allocate VM: ResallocHost, ticket_id=758160, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-11 14:28:05,293][  INFO][PID:1301366] Allocated host ResallocHost, ticket_id=758160, hostname=2620:52:6:1161:dead:beef:cafe:c18b, name=vmhost_p09_04_prod_01099392_20260111_140200, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-11 14:28:05,293][  INFO][PID:1301366] Allocating ssh connection to builder
[2026-01-11 14:28:05,294][  INFO][PID:1301366] Checking that builder machine is OK
[2026-01-11 14:28:06,237][  INFO][PID:1301366] Running remote command: copr-builder-ready fedora-42-ppc64le
[2026-01-11 14:28:06,743][  INFO][PID:1301366] Red Hat subscription not needed for fedora-42-ppc64le
Builder is ready to be used

[2026-01-11 14:28:06,744][  INFO][PID:1301366] Filling build.info file with builder info
[2026-01-11 14:28:06,744][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:28:06,745][  INFO][PID:1301366] Not re-notifying FE and msg buses for the new host.
[2026-01-11 14:28:06,745][  INFO][PID:1301366] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9996046-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached
[2026-01-11 14:28:07,715][  INFO][PID:1301366] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18b' (ED25519) to the list of known hosts.


[2026-01-11 14:28:07,716][  INFO][PID:1301366] Downloading the builder-live.log file, attempt 1
[2026-01-11 14:28:07,717][  INFO][PID:1301366] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18b copr-rpmbuild-log
[2026-01-11 14:28:12,725][  INFO][PID:1301366] Periodic builder liveness probe: alive
[2026-01-11 14:28:12,726][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:28:17,726][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:28:22,727][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:28:27,728][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:28:32,729][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:28:37,730][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:28:42,731][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:28:47,732][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:28:52,733][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:28:57,734][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:29:02,735][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:29:07,736][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:29:12,737][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:29:17,738][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:29:22,741][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:29:27,742][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:29:32,743][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:29:37,744][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:29:42,745][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:29:47,746][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:29:52,747][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:29:57,748][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:30:02,749][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:30:07,750][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:30:12,750][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:30:17,752][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:30:22,753][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:30:27,753][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:30:32,754][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:30:37,756][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:30:42,757][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:30:47,757][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:30:52,759][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:30:57,759][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:31:02,761][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:31:07,762][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:31:12,763][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:31:17,763][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:31:22,765][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:31:27,768][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:31:32,768][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:31:37,769][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:31:42,770][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:31:47,771][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:31:52,773][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:31:57,777][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:32:02,778][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:32:07,779][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:32:12,780][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:32:17,781][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:32:22,782][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:32:27,783][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:32:32,784][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:32:37,785][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:32:42,786][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:32:47,787][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:32:52,788][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:32:57,789][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:33:02,790][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:33:07,790][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:33:12,791][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:33:17,793][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:33:22,793][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:33:27,794][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:33:32,795][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:33:37,796][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:33:42,797][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:33:47,798][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:33:52,799][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:33:57,800][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:34:02,801][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:34:07,802][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:34:12,803][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:34:17,827][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:34:22,829][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:34:27,831][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:34:32,832][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:34:37,832][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:34:42,833][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:34:47,834][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:34:52,835][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:34:57,836][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:35:02,837][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:35:07,838][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:35:12,840][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:35:17,841][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:35:22,842][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:35:27,843][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:35:32,844][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:35:37,845][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:35:42,847][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:35:47,848][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:35:52,849][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:35:57,850][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:36:02,851][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:36:07,852][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:36:12,853][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:36:17,855][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:36:22,856][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:36:27,857][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:36:32,857][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:36:37,859][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:36:42,861][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:36:47,896][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:36:52,897][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:36:57,898][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:37:02,899][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:37:07,900][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:37:12,901][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:37:17,909][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:37:22,910][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:37:28,079][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:37:33,080][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:37:38,081][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:37:43,082][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:37:48,083][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:37:53,084][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:37:58,085][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:38:03,087][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:38:08,282][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:38:13,288][  INFO][PID:1301366] Periodic builder liveness probe: alive
[2026-01-11 14:38:13,288][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:38:18,289][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:38:23,290][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:38:28,291][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:38:33,292][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:38:38,369][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:38:43,370][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:38:48,371][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:38:53,372][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:38:58,373][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:39:03,624][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:39:08,625][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:39:13,626][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:39:18,669][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:39:23,670][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:39:28,707][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:39:33,708][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:39:38,713][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:39:43,720][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:39:48,796][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:39:53,797][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:39:58,798][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:40:03,848][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:40:08,854][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:40:13,936][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:40:19,043][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:40:24,045][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:40:29,046][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:40:34,047][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:40:39,048][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:40:44,049][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:40:49,050][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:40:54,062][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:40:59,063][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:41:04,064][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:41:09,183][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:41:14,207][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:41:19,208][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:41:24,209][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:41:29,228][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:41:34,238][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:41:39,462][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:41:44,592][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:41:49,594][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:41:54,595][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:41:59,601][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:42:04,707][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:42:09,708][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:42:14,748][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:42:19,749][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:42:24,750][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:42:29,752][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:42:34,753][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:42:39,966][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:42:44,967][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:42:49,968][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:42:54,969][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:43:00,056][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:43:05,057][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:43:10,129][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:43:15,129][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:43:20,225][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:43:25,226][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:43:30,228][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:43:35,270][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:43:40,270][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:43:45,271][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:43:50,272][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:43:55,295][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:44:00,410][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:44:05,411][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:44:10,417][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:44:15,418][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:44:20,490][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:44:25,491][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:44:30,500][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:44:35,757][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:44:40,914][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:44:45,915][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:44:50,916][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:44:55,917][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:45:00,921][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:45:05,981][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:45:11,177][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:45:16,178][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:45:21,179][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:45:26,180][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:45:31,181][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:45:36,291][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:45:41,316][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:45:46,397][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:45:51,397][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:45:56,398][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:46:01,405][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:46:06,406][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:46:11,695][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:46:16,696][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:46:21,697][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:46:26,697][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:46:31,698][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:46:36,699][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:46:41,700][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:46:46,701][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:46:51,702][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:46:56,703][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:47:01,704][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:47:06,705][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:47:11,706][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:47:16,707][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:47:21,708][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:47:26,709][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:47:31,710][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:47:36,711][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:47:41,711][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:47:46,712][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:47:51,713][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:47:56,714][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:48:01,715][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:48:06,717][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:48:11,718][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:48:16,724][  INFO][PID:1301366] Periodic builder liveness probe: dead
[2026-01-11 14:48:16,724][  INFO][PID:1301366] calling cancel callback, and waiting
[2026-01-11 14:48:37,928][WARNING][PID:1301366] Can't cancel build
out:

err:
Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18b' (ED25519) to the list of known hosts.
Traceback (most recent call last):
  File "/usr/bin/copr-rpmbuild-cancel", line 85, in <module>
    _main()
    ~~~~~^^
  File "/usr/bin/copr-rpmbuild-cancel", line 69, in _main
    with open(pidfile, "r") as pidfd:
         ~~~~^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/copr-rpmbuild/pid'

[2026-01-11 14:48:48,910][ ERROR][PID:1301366] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-11 14:48:58,910][WARNING][PID:1301366] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-11 14:49:08,911][  INFO][PID:1301366] Downloading the builder-live.log file, attempt 2
[2026-01-11 14:49:08,913][  INFO][PID:1301366] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18b copr-rpmbuild-log
[2026-01-11 14:51:10,163][ ERROR][PID:1301366] Re-try request for task on 'ResallocHost, ticket_id=758160, hostname=2620:52:6:1161:dead:beef:cafe:c18b, name=vmhost_p09_04_prod_01099392_20260111_140200, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Resalloc ticket FAILED
[2026-01-11 14:51:10,163][  INFO][PID:1301366] Releasing VM back to pool
[2026-01-11 14:51:10,172][  INFO][PID:1301366] Retry #2 (on other host)
[2026-01-11 14:51:10,172][  INFO][PID:1301366] Checking that /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/repodata/repomd.xml exists
[2026-01-11 14:51:10,173][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:51:10,173][  INFO][PID:1301366] VM allocation process starts
[2026-01-11 14:51:10,205][  INFO][PID:1301366] Trying to allocate VM: ResallocHost, ticket_id=758237, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-11 14:51:13,243][  INFO][PID:1301366] Allocated host ResallocHost, ticket_id=758237, hostname=2620:52:6:1161:dead:beef:cafe:c14e, name=vmhost_p09_03_prod_01100079_20260111_141403, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-11 14:51:13,357][  INFO][PID:1301366] Allocating ssh connection to builder
[2026-01-11 14:51:13,358][  INFO][PID:1301366] Checking that builder machine is OK
[2026-01-11 14:51:14,413][  INFO][PID:1301366] Running remote command: copr-builder-ready fedora-42-ppc64le
[2026-01-11 14:51:14,893][  INFO][PID:1301366] Red Hat subscription not needed for fedora-42-ppc64le
Builder is ready to be used

[2026-01-11 14:51:14,893][  INFO][PID:1301366] Filling build.info file with builder info
[2026-01-11 14:51:14,910][  INFO][PID:1301366] Checking for cancel request
[2026-01-11 14:51:14,911][  INFO][PID:1301366] Not re-notifying FE and msg buses for the new host.
[2026-01-11 14:51:14,911][  INFO][PID:1301366] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9996046-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached
[2026-01-11 14:51:15,876][  INFO][PID:1301366] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c14e' (ED25519) to the list of known hosts.


[2026-01-11 14:51:15,877][  INFO][PID:1301366] Downloading the builder-live.log file, attempt 1
[2026-01-11 14:51:15,878][  INFO][PID:1301366] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14e copr-rpmbuild-log
[2026-01-11 14:51:20,877][  INFO][PID:1301366] calling cancel callback, and waiting
[2026-01-11 14:51:24,525][  INFO][PID:1301366] Cancel request succeeded
out:
success
err:
Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c14e' (ED25519) to the list of known hosts.

[2026-01-11 14:51:24,526][  INFO][PID:1301366] Downloading results from builder
[2026-01-11 14:51:24,526][  INFO][PID:1301366] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c14e]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers started
[2026-01-11 14:51:24,527][  INFO][PID:1301366] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c14e]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers/ &> /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers/build-09996046.rsync.log
[2026-01-11 14:52:24,585][  INFO][PID:1301366] rsyncing finished.
[2026-01-11 14:52:24,585][ ERROR][PID:1301366] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-11 14:52:24,586][ ERROR][PID:1301366] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-11 14:52:34,586][  INFO][PID:1301366] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c14e]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers started
[2026-01-11 14:52:34,587][  INFO][PID:1301366] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c14e]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers/ &> /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers/build-09996046.rsync.log
[2026-01-11 14:53:34,659][  INFO][PID:1301366] rsyncing finished.
[2026-01-11 14:53:34,660][ ERROR][PID:1301366] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-11 14:53:34,660][ ERROR][PID:1301366] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-11 14:53:44,661][  INFO][PID:1301366] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c14e]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers started
[2026-01-11 14:53:44,662][  INFO][PID:1301366] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c14e]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers/ &> /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers/build-09996046.rsync.log
[2026-01-11 14:54:44,760][  INFO][PID:1301366] rsyncing finished.
[2026-01-11 14:54:44,761][ ERROR][PID:1301366] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-11 14:54:44,761][ ERROR][PID:1301366] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-11 14:54:54,762][ ERROR][PID:1301366] Re-try request for task on 'ResallocHost, ticket_id=758237, hostname=2620:52:6:1161:dead:beef:cafe:c14e, name=vmhost_p09_03_prod_01100079_20260111_141403, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 3 SSH attempts
[2026-01-11 14:54:54,763][  INFO][PID:1301366] Releasing VM back to pool
[2026-01-11 14:54:54,777][ ERROR][PID:1301366] Backend process error: Three host tried without success: {'2620:52:6:1161:dead:beef:cafe:c18b', '2620:52:6:1161:dead:beef:cafe:c14e', '2620:52:6:1161:dead:beef:cafe:c193'}
[2026-01-11 14:54:54,778][WARNING][PID:1301366] Switching not-finished job state to 'failed'
[2026-01-11 14:54:54,778][  INFO][PID:1301366] Worker failed build, took 219.8666079044342
[2026-01-11 14:54:54,778][  INFO][PID:1301366] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "music",
            "project_name": "rust-image",
            "project_dirname": "rust-image",
            "submitter": "music",
            "ended_on": 1768143294.7781086,
            "started_on": 1768143074.9115007,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-42-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9996046-fedora-42-ppc64le",
            "build_id": 9996046,
            "package_name": "papers",
            "package_version": "48.5-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/music/rust-image/papers",
            "git_hash": "0f8d629d44829c36a79cf5e6148fea795f3a846b",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "music/rust-image--music",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/music/rust-image/fedora-42-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": true,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/music/rust-image",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/music/rust-image",
            "result_dir": "09996046-papers",
            "built_packages": "",
            "id": 9996046,
            "mockchain_macros": {
                "copr_username": "music",
                "copr_projectname": "rust-image",
                "vendor": "Fedora Project COPR (music/rust-image)"
            }
        }
    ]
}
[2026-01-11 14:54:54,820][  INFO][PID:1301366] Sending fedora-messaging bus message in build.end
[2026-01-11 14:54:55,385][  INFO][PID:1301366] Compressing /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers/builder-live.log by gzip
[2026-01-11 14:54:55,386][  INFO][PID:1301366] Running command 'gzip /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers/builder-live.log' as PID 1377616
[2026-01-11 14:54:55,388][  INFO][PID:1301366] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers/builder-live.log)
[2026-01-11 14:54:55,388][  INFO][PID:1301366] Compressing /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers/backend.log by gzip
[2026-01-11 14:54:55,389][  INFO][PID:1301366] Running command 'gzip /var/lib/copr/public_html/results/music/rust-image/fedora-42-ppc64le/09996046-papers/backend.log' as PID 1377617