[2026-01-27 18:27:27,408][  INFO][PID:1642629] Marking build as starting
[2026-01-27 18:27:27,471][  INFO][PID:1642629] Checking that /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/repodata/repomd.xml exists
[2026-01-27 18:27:27,498][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:27:27,499][  INFO][PID:1642629] VM allocation process starts
[2026-01-27 18:27:27,517][  INFO][PID:1642629] Trying to allocate VM: ResallocHost, ticket_id=1056320, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-27 18:27:30,574][  INFO][PID:1642629] Allocated host ResallocHost, ticket_id=1056320, hostname=2620:52:6:1161:dead:beef:cafe:c14d, name=vmhost_p09_03_prod_01652041_20260127_181959, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-27 18:27:30,575][  INFO][PID:1642629] Allocating ssh connection to builder
[2026-01-27 18:27:30,576][  INFO][PID:1642629] Checking that builder machine is OK
[2026-01-27 18:27:33,044][  INFO][PID:1642629] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2026-01-27 18:28:33,114][ ERROR][PID:1642629] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c14d port 22: Connection timed out

[2026-01-27 18:28:43,123][ ERROR][PID:1642629] Re-try request for task on 'ResallocHost, ticket_id=1056320, hostname=2620:52:6:1161:dead:beef:cafe:c14d, name=vmhost_p09_03_prod_01652041_20260127_181959, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-01-27 18:28:43,123][  INFO][PID:1642629] Releasing VM back to pool
[2026-01-27 18:28:43,176][  INFO][PID:1642629] Retry #1 (on other host)
[2026-01-27 18:28:43,177][  INFO][PID:1642629] Checking that /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/repodata/repomd.xml exists
[2026-01-27 18:28:43,178][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:28:43,178][  INFO][PID:1642629] VM allocation process starts
[2026-01-27 18:28:43,190][  INFO][PID:1642629] Trying to allocate VM: ResallocHost, ticket_id=1056345, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-27 18:28:46,203][  INFO][PID:1642629] Allocated host ResallocHost, ticket_id=1056345, hostname=2620:52:6:1161:dead:beef:cafe:c14e, name=vmhost_p09_03_prod_01652586_20260127_182506, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-27 18:28:46,203][  INFO][PID:1642629] Allocating ssh connection to builder
[2026-01-27 18:28:46,204][  INFO][PID:1642629] Checking that builder machine is OK
[2026-01-27 18:28:54,294][  INFO][PID:1642629] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2026-01-27 18:28:57,572][  INFO][PID:1642629] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2026-01-27 18:28:57,574][  INFO][PID:1642629] Filling build.info file with builder info
[2026-01-27 18:28:57,579][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:28:57,582][  INFO][PID:1642629] Not re-notifying FE and msg buses for the new host.
[2026-01-27 18:28:57,586][  INFO][PID:1642629] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10069758-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2026-01-27 18:28:59,254][  INFO][PID:1642629] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c14e' (ED25519) to the list of known hosts.


[2026-01-27 18:28:59,255][  INFO][PID:1642629] Downloading the builder-live.log file, attempt 1
[2026-01-27 18:28:59,256][  INFO][PID:1642629] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14e copr-rpmbuild-log
[2026-01-27 18:29:04,261][  INFO][PID:1642629] Periodic builder liveness probe: alive
[2026-01-27 18:29:04,261][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:29:09,262][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:29:14,263][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:29:19,264][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:29:24,266][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:29:29,267][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:29:34,268][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:29:39,269][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:29:44,270][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:29:49,287][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:29:54,288][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:29:59,321][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:30:04,350][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:30:09,434][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:30:14,490][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:30:19,491][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:30:24,492][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:30:29,493][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:30:34,495][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:30:39,509][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:30:44,510][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:30:49,511][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:30:54,513][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:30:59,514][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:31:04,515][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:31:09,518][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:31:14,524][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:31:19,527][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:31:24,545][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:31:29,549][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:31:34,553][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:31:39,555][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:31:44,557][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:31:49,559][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:31:54,560][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:31:59,565][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:32:04,568][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:32:09,575][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:32:14,576][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:32:19,579][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:32:24,583][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:32:29,592][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:32:34,593][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:32:39,596][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:32:44,599][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:32:49,603][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:32:54,604][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:32:59,606][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:33:04,610][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:33:09,612][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:33:14,625][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:33:19,725][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:33:24,774][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:33:29,775][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:33:34,776][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:33:39,779][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:33:44,781][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:33:48,441][ ERROR][PID:1642629] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-27 18:33:49,783][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:33:54,784][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:33:58,442][WARNING][PID:1642629] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-27 18:33:59,785][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:34:04,786][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:34:08,443][  INFO][PID:1642629] Downloading the builder-live.log file, attempt 2
[2026-01-27 18:34:08,444][  INFO][PID:1642629] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14e copr-rpmbuild-log
[2026-01-27 18:34:09,787][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:34:14,788][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:34:19,789][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:34:24,790][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:34:29,791][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:34:34,792][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:34:39,793][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:34:44,795][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:34:49,796][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:34:54,798][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:34:59,800][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:35:04,801][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:35:08,512][ ERROR][PID:1642629] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-27 18:35:09,802][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:35:14,803][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:35:18,527][WARNING][PID:1642629] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-27 18:35:19,804][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:35:24,805][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:35:28,528][  INFO][PID:1642629] Downloading the builder-live.log file, attempt 3
[2026-01-27 18:35:28,529][  INFO][PID:1642629] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14e copr-rpmbuild-log
[2026-01-27 18:35:29,806][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:35:34,807][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:35:39,808][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:35:44,812][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:35:49,813][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:35:54,814][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:35:59,815][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:36:04,816][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:36:09,818][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:36:14,819][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:36:19,820][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:36:24,821][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:36:28,590][ ERROR][PID:1642629] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-27 18:36:29,821][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:36:34,823][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:36:38,595][WARNING][PID:1642629] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-27 18:36:39,824][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:36:44,825][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:36:48,596][  INFO][PID:1642629] Downloading the builder-live.log file, attempt 4
[2026-01-27 18:36:48,597][  INFO][PID:1642629] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14e copr-rpmbuild-log
[2026-01-27 18:36:49,826][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:36:54,827][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:36:59,828][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:37:04,829][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:37:09,830][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:37:14,832][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:37:19,833][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:37:24,834][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:37:29,835][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:37:34,837][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:37:39,845][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:37:44,846][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:37:48,664][ ERROR][PID:1642629] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-27 18:37:49,847][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:37:54,848][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:37:58,673][ ERROR][PID:1642629] Giving up for unstable SSH, failures: 1769538838.44215, 1769538918.5277758, 1769538998.5953748, 1769539078.6731436
[2026-01-27 18:37:58,674][ ERROR][PID:1642629] Re-try request for task on 'ResallocHost, ticket_id=1056345, hostname=2620:52:6:1161:dead:beef:cafe:c14e, name=vmhost_p09_03_prod_01652586_20260127_182506, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2026-01-27 18:37:58,674][  INFO][PID:1642629] Releasing VM back to pool
[2026-01-27 18:37:58,682][  INFO][PID:1642629] Retry #2 (on other host)
[2026-01-27 18:37:58,683][  INFO][PID:1642629] Checking that /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/repodata/repomd.xml exists
[2026-01-27 18:37:58,683][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:37:58,684][  INFO][PID:1642629] VM allocation process starts
[2026-01-27 18:37:58,727][  INFO][PID:1642629] Trying to allocate VM: ResallocHost, ticket_id=1056479, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-27 18:38:01,761][  INFO][PID:1642629] Allocated host ResallocHost, ticket_id=1056479, hostname=2620:52:6:1161:dead:beef:cafe:c10e, name=vmhost_p09_02_prod_01650775_20260127_180815, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-27 18:38:01,762][  INFO][PID:1642629] Allocating ssh connection to builder
[2026-01-27 18:38:01,762][  INFO][PID:1642629] Checking that builder machine is OK
[2026-01-27 18:38:03,147][  INFO][PID:1642629] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2026-01-27 18:38:03,811][  INFO][PID:1642629] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2026-01-27 18:38:03,812][  INFO][PID:1642629] Filling build.info file with builder info
[2026-01-27 18:38:03,856][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:38:03,857][  INFO][PID:1642629] Not re-notifying FE and msg buses for the new host.
[2026-01-27 18:38:03,858][  INFO][PID:1642629] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10069758-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2026-01-27 18:38:05,246][  INFO][PID:1642629] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c10e' (ED25519) to the list of known hosts.


[2026-01-27 18:38:05,247][  INFO][PID:1642629] Downloading the builder-live.log file, attempt 1
[2026-01-27 18:38:05,248][  INFO][PID:1642629] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c10e copr-rpmbuild-log
[2026-01-27 18:38:10,247][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:38:15,248][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:38:20,270][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:38:25,307][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:38:30,372][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:38:35,399][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:38:40,405][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:38:45,407][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:38:50,408][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:38:55,434][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:39:00,467][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:39:05,522][  INFO][PID:1642629] Periodic builder liveness probe: alive
[2026-01-27 18:39:05,524][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:39:10,535][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:39:15,542][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:39:20,544][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:39:25,546][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:39:30,549][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:39:35,552][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:39:40,556][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:39:45,563][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:39:50,568][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:39:55,572][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:40:00,575][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:40:05,600][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:40:10,625][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:40:15,634][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:40:20,661][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:40:25,662][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:40:30,676][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:40:35,690][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:40:40,722][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:40:45,744][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:40:50,747][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:40:55,793][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:41:00,796][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:41:05,799][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:41:10,801][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:41:15,805][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:41:20,814][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:41:25,819][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:41:30,824][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:41:35,835][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:41:40,836][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:41:45,839][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:41:50,841][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:41:55,848][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:42:00,849][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:42:05,853][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:42:10,857][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:42:15,867][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:42:20,869][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:42:25,872][  INFO][PID:1642629] Checking for cancel request
[2026-01-27 18:42:27,060][  INFO][PID:1642629] Downloading results from builder
[2026-01-27 18:42:27,140][  INFO][PID:1642629] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10e]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd started
[2026-01-27 18:42:27,148][  INFO][PID:1642629] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10e]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/ &> /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/build-10069758.rsync.log
[2026-01-27 18:43:27,317][  INFO][PID:1642629] rsyncing finished.
[2026-01-27 18:43:27,322][ ERROR][PID:1642629] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-27 18:43:27,323][ ERROR][PID:1642629] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-27 18:43:37,324][  INFO][PID:1642629] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10e]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd started
[2026-01-27 18:43:37,325][  INFO][PID:1642629] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10e]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/ &> /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/build-10069758.rsync.log
[2026-01-27 18:43:38,557][  INFO][PID:1642629] rsyncing finished.
[2026-01-27 18:43:38,558][  INFO][PID:1642629] Releasing VM back to pool
[2026-01-27 18:43:38,567][  INFO][PID:1642629] Searching for 'success' file in resultdir
[2026-01-27 18:43:38,568][  INFO][PID:1642629] Going to sign pkgs from source: 10069758-fedora-rawhide-ppc64le in chroot: /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le
[2026-01-27 18:43:38,568][  INFO][PID:1642629] Calling '/bin/sign -u @meta#netconsd@copr.fedorahosted.org -p' (attempt #1)
[2026-01-27 18:43:38,839][  INFO][PID:1642629] Calling '/bin/sign -4 -h sha256 -u @meta#netconsd@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/netconsd-debugsource-v0.4.1-1.20260127181754468069.pr21.30.g4d87cb2.fc44.ppc64le.rpm' (attempt #1)
[2026-01-27 18:43:39,313][  INFO][PID:1642629] signed rpm: /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/netconsd-debugsource-v0.4.1-1.20260127181754468069.pr21.30.g4d87cb2.fc44.ppc64le.rpm
[2026-01-27 18:43:39,314][  INFO][PID:1642629] Calling '/bin/sign -4 -h sha256 -u @meta#netconsd@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/netconsd-debuginfo-v0.4.1-1.20260127181754468069.pr21.30.g4d87cb2.fc44.ppc64le.rpm' (attempt #1)
[2026-01-27 18:43:39,994][  INFO][PID:1642629] signed rpm: /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/netconsd-debuginfo-v0.4.1-1.20260127181754468069.pr21.30.g4d87cb2.fc44.ppc64le.rpm
[2026-01-27 18:43:39,994][  INFO][PID:1642629] Calling '/bin/sign -4 -h sha256 -u @meta#netconsd@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/netconsd-v0.4.1-1.20260127181754468069.pr21.30.g4d87cb2.fc44.src.rpm' (attempt #1)
[2026-01-27 18:43:40,491][  INFO][PID:1642629] signed rpm: /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/netconsd-v0.4.1-1.20260127181754468069.pr21.30.g4d87cb2.fc44.src.rpm
[2026-01-27 18:43:40,495][  INFO][PID:1642629] Calling '/bin/sign -4 -h sha256 -u @meta#netconsd@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/netconsd-v0.4.1-1.20260127181754468069.pr21.30.g4d87cb2.fc44.ppc64le.rpm' (attempt #1)
[2026-01-27 18:43:41,134][  INFO][PID:1642629] signed rpm: /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/netconsd-v0.4.1-1.20260127181754468069.pr21.30.g4d87cb2.fc44.ppc64le.rpm
[2026-01-27 18:43:41,136][  INFO][PID:1642629] Sign done
[2026-01-27 18:43:41,140][  INFO][PID:1642629] Incremental createrepo run, adding 10069758-netconsd into https://download.copr.fedorainfracloud.org/results/@meta/netconsd/fedora-rawhide-ppc64le, (auto-create-repo=True)
[2026-01-27 18:43:41,146][  INFO][PID:1642629] Running command 'copr-repo --batched /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le --add 10069758-netconsd --no-appstream-metadata' as PID 1775033
[2026-01-27 18:43:42,115][  INFO][PID:1642629] Finished after 0 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le --add 10069758-netconsd --no-appstream-metadata)
[2026-01-27 18:43:42,116][  INFO][PID:1642629] Getting build details
[2026-01-27 18:43:42,117][  INFO][PID:1642629] Listing built binary packages in /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd
[2026-01-27 18:43:42,117][  INFO][PID:1642629] Built packages:
['netconsd v0.4.1', 'netconsd-debuginfo v0.4.1', 'netconsd-debugsource v0.4.1']
[2026-01-27 18:43:42,118][  INFO][PID:1642629] build details: {'built_packages': 'netconsd v0.4.1\nnetconsd-debuginfo v0.4.1\nnetconsd-debugsource v0.4.1'}
[2026-01-27 18:43:42,118][  INFO][PID:1642629] Retrieving pubkey
[2026-01-27 18:43:42,119][  INFO][PID:1642629] Calling '/bin/sign -u @meta#netconsd@copr.fedorahosted.org -p' (attempt #1)
[2026-01-27 18:43:42,313][  INFO][PID:1642629] Added pubkey for user @meta project netconsd into: /var/lib/copr/public_html/results/@meta/netconsd/pubkey.gpg
[2026-01-27 18:43:42,314][  INFO][PID:1642629] Finished build: id=10069758 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@meta/netconsd chroot=fedora-rawhide-ppc64le 
[2026-01-27 18:43:42,317][  INFO][PID:1642629] Worker succeeded build, took 338.45960330963135
[2026-01-27 18:43:42,317][  INFO][PID:1642629] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@meta",
            "project_name": "netconsd",
            "project_dirname": "netconsd",
            "submitter": "packit",
            "ended_on": 1769539422.316974,
            "started_on": 1769539083.8573706,
            "submitted_on": null,
            "status": 1,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "10069758-fedora-rawhide-ppc64le",
            "build_id": 10069758,
            "package_name": "netconsd",
            "package_version": "v0.4.1-1.20260127181754468069.pr21.30.g4d87cb2",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@meta/netconsd/netconsd",
            "git_hash": "57ccfde1c847218913d3076ff449e99accd854a9",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@meta/netconsd--packit",
            "results": {
                "packages": [
                    {
                        "name": "netconsd",
                        "epoch": null,
                        "version": "v0.4.1",
                        "release": "1.20260127181754468069.pr21.30.g4d87cb2.fc44",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "netconsd-debuginfo",
                        "epoch": null,
                        "version": "v0.4.1",
                        "release": "1.20260127181754468069.pr21.30.g4d87cb2.fc44",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "netconsd",
                        "epoch": null,
                        "version": "v0.4.1",
                        "release": "1.20260127181754468069.pr21.30.g4d87cb2.fc44",
                        "arch": "src"
                    },
                    {
                        "name": "netconsd-debugsource",
                        "epoch": null,
                        "version": "v0.4.1",
                        "release": "1.20260127181754468069.pr21.30.g4d87cb2.fc44",
                        "arch": "ppc64le"
                    }
                ]
            },
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@meta/netconsd/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@meta/netconsd",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@meta/netconsd",
            "result_dir": "10069758-netconsd",
            "built_packages": "netconsd v0.4.1\nnetconsd-debuginfo v0.4.1\nnetconsd-debugsource v0.4.1",
            "id": 10069758,
            "mockchain_macros": {
                "copr_username": "@meta",
                "copr_projectname": "netconsd",
                "vendor": "Fedora Project COPR (@meta/netconsd)"
            }
        }
    ]
}
[2026-01-27 18:43:42,406][  INFO][PID:1642629] Sending fedora-messaging bus message in build.end
[2026-01-27 18:43:43,012][  INFO][PID:1642629] Compressing /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/builder-live.log by gzip
[2026-01-27 18:43:43,013][  INFO][PID:1642629] Running command 'gzip /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/builder-live.log' as PID 1775278
[2026-01-27 18:43:43,167][  INFO][PID:1642629] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/builder-live.log)
[2026-01-27 18:43:43,168][  INFO][PID:1642629] Compressing /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/backend.log by gzip
[2026-01-27 18:43:43,169][  INFO][PID:1642629] Running command 'gzip /var/lib/copr/public_html/results/@meta/netconsd/fedora-rawhide-ppc64le/10069758-netconsd/backend.log' as PID 1775293