[2025-12-30 19:06:19,552][ INFO][PID:353277] Marking build as starting [2025-12-30 19:06:19,911][ INFO][PID:353277] Checking that /var/lib/copr/public_html/results/keithbowes/deponejo/fedora-42-ppc64le/repodata/repomd.xml exists [2025-12-30 19:06:19,924][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:06:19,925][ INFO][PID:353277] VM allocation process starts [2025-12-30 19:06:19,942][ INFO][PID:353277] Trying to allocate VM: ResallocHost, ticket_id=582168, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-30 19:06:24,944][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:06:25,980][ INFO][PID:353277] Allocated host ResallocHost, ticket_id=582168, hostname=2620:52:6:1161:dead:beef:cafe:c18a, name=vmhost_p09_04_prod_00858182_20251230_185552, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-30 19:06:25,981][ INFO][PID:353277] Allocating ssh connection to builder [2025-12-30 19:06:25,982][ INFO][PID:353277] Checking that builder machine is OK [2025-12-30 19:07:26,056][ ERROR][PID:353277] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c18a port 22: Connection timed out [2025-12-30 19:07:36,058][ ERROR][PID:353277] Re-try request for task on 'ResallocHost, ticket_id=582168, hostname=2620:52:6:1161:dead:beef:cafe:c18a, name=vmhost_p09_04_prod_00858182_20251230_185552, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts [2025-12-30 19:07:36,059][ INFO][PID:353277] Releasing VM back to pool [2025-12-30 19:07:36,104][ INFO][PID:353277] Retry #1 (on other host) [2025-12-30 19:07:36,108][ INFO][PID:353277] Checking that /var/lib/copr/public_html/results/keithbowes/deponejo/fedora-42-ppc64le/repodata/repomd.xml exists [2025-12-30 19:07:36,109][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:07:36,111][ INFO][PID:353277] VM allocation process starts [2025-12-30 19:07:36,146][ INFO][PID:353277] Trying to allocate VM: ResallocHost, ticket_id=582225, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-30 19:07:41,150][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:07:42,215][ INFO][PID:353277] Allocated host ResallocHost, ticket_id=582225, hostname=2620:52:6:1161:dead:beef:cafe:c14d, name=vmhost_p09_03_prod_00858549_20251230_190008, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-30 19:07:42,216][ INFO][PID:353277] Allocating ssh connection to builder [2025-12-30 19:07:42,216][ INFO][PID:353277] Checking that builder machine is OK [2025-12-30 19:07:43,362][ INFO][PID:353277] Running remote command: copr-builder-ready fedora-42-ppc64le [2025-12-30 19:07:43,943][ INFO][PID:353277] Red Hat subscription not needed for fedora-42-ppc64le Builder is ready to be used [2025-12-30 19:07:43,944][ INFO][PID:353277] Filling build.info file with builder info [2025-12-30 19:07:43,968][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:07:43,974][ INFO][PID:353277] Not re-notifying FE and msg buses for the new host. [2025-12-30 19:07:43,978][ INFO][PID:353277] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9962046-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached [2025-12-30 19:07:45,048][ INFO][PID:353277] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c14d' (ED25519) to the list of known hosts. [2025-12-30 19:07:45,050][ INFO][PID:353277] Downloading the builder-live.log file, attempt 1 [2025-12-30 19:07:45,105][ INFO][PID:353277] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14d copr-rpmbuild-log [2025-12-30 19:07:50,111][ INFO][PID:353277] Periodic builder liveness probe: alive [2025-12-30 19:07:50,125][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:07:55,129][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:08:00,132][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:08:05,145][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:08:10,146][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:08:15,149][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:08:20,153][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:08:25,167][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:08:30,171][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:08:35,177][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:08:40,186][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:08:45,195][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:08:50,200][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:08:55,208][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:09:00,219][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:09:05,226][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:09:10,232][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:09:15,236][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:09:20,238][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:09:25,241][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:09:30,244][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:09:35,249][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:09:40,250][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:09:45,259][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:09:50,261][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:09:55,280][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:10:00,284][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:10:05,289][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:10:10,297][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:10:15,302][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:10:20,308][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:10:25,320][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:10:30,329][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:10:35,331][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:10:40,334][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:10:45,337][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:10:50,342][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:10:55,346][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:10:56,376][ ERROR][PID:353277] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-30 19:11:00,363][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:11:05,370][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:11:06,379][WARNING][PID:353277] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-30 19:11:10,375][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:11:15,377][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:11:16,384][ INFO][PID:353277] Downloading the builder-live.log file, attempt 2 [2025-12-30 19:11:16,392][ INFO][PID:353277] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14d copr-rpmbuild-log [2025-12-30 19:11:20,380][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:11:25,385][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:11:30,401][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:11:35,411][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:11:40,413][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:11:45,420][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:11:50,427][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:11:55,428][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:12:00,431][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:12:05,446][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:12:10,451][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:12:15,457][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:12:16,493][ ERROR][PID:353277] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-30 19:12:20,465][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:12:25,468][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:12:26,509][WARNING][PID:353277] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-30 19:12:30,474][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:12:35,475][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:12:36,513][ INFO][PID:353277] Downloading the builder-live.log file, attempt 3 [2025-12-30 19:12:36,519][ INFO][PID:353277] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14d copr-rpmbuild-log [2025-12-30 19:12:40,485][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:12:45,487][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:12:50,490][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:12:55,492][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:13:00,497][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:13:05,503][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:13:10,509][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:13:15,510][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:13:20,513][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:13:25,520][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:13:30,528][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:13:35,536][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:13:36,598][ ERROR][PID:353277] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-30 19:13:40,538][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:13:45,546][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:13:46,609][WARNING][PID:353277] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-30 19:13:50,551][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:13:55,560][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:13:56,612][ INFO][PID:353277] Downloading the builder-live.log file, attempt 4 [2025-12-30 19:13:56,614][ INFO][PID:353277] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14d copr-rpmbuild-log [2025-12-30 19:14:00,568][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:14:05,573][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:14:10,584][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:14:15,585][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:14:20,587][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:14:25,592][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:14:30,595][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:14:35,598][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:14:40,600][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:14:45,601][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:14:50,602][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:14:55,604][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:14:56,684][ ERROR][PID:353277] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-30 19:15:00,605][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:15:05,606][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:15:06,691][ ERROR][PID:353277] Giving up for unstable SSH, failures: 1767121866.37911, 1767121946.5092077, 1767122026.609371, 1767122106.6910827 [2025-12-30 19:15:06,692][ ERROR][PID:353277] Re-try request for task on 'ResallocHost, ticket_id=582225, hostname=2620:52:6:1161:dead:beef:cafe:c14d, name=vmhost_p09_03_prod_00858549_20251230_190008, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-12-30 19:15:06,693][ INFO][PID:353277] Releasing VM back to pool [2025-12-30 19:15:06,705][ INFO][PID:353277] Retry #2 (on other host) [2025-12-30 19:15:06,706][ INFO][PID:353277] Checking that /var/lib/copr/public_html/results/keithbowes/deponejo/fedora-42-ppc64le/repodata/repomd.xml exists [2025-12-30 19:15:06,707][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:15:06,708][ INFO][PID:353277] VM allocation process starts [2025-12-30 19:15:06,719][ INFO][PID:353277] Trying to allocate VM: ResallocHost, ticket_id=582393, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-30 19:15:09,756][ INFO][PID:353277] Allocated host ResallocHost, ticket_id=582393, hostname=140.211.168.48, name=copr_osuosl_p09_prod_00858824_20251230_191008, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-30 19:15:09,757][ INFO][PID:353277] Allocating ssh connection to builder [2025-12-30 19:15:09,761][ INFO][PID:353277] Checking that builder machine is OK [2025-12-30 19:15:11,644][ INFO][PID:353277] Running remote command: copr-builder-ready fedora-42-ppc64le [2025-12-30 19:15:12,719][ INFO][PID:353277] Red Hat subscription not needed for fedora-42-ppc64le Builder is ready to be used [2025-12-30 19:15:12,726][ INFO][PID:353277] Filling build.info file with builder info [2025-12-30 19:15:12,740][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:15:12,743][ INFO][PID:353277] Not re-notifying FE and msg buses for the new host. [2025-12-30 19:15:12,743][ INFO][PID:353277] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9962046-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached [2025-12-30 19:15:14,863][ INFO][PID:353277] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '140.211.168.48' (ED25519) to the list of known hosts. [2025-12-30 19:15:14,866][ INFO][PID:353277] Downloading the builder-live.log file, attempt 1 [2025-12-30 19:15:14,868][ INFO][PID:353277] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.48 copr-rpmbuild-log [2025-12-30 19:15:19,867][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:15:24,868][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:15:29,875][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:15:34,877][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:15:39,878][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:15:44,879][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:15:49,880][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:15:54,882][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:15:59,884][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:16:04,886][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:16:09,888][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:16:14,889][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:16:19,892][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:16:24,895][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:16:29,901][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:16:34,902][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:16:39,904][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:16:44,906][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:16:49,908][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:16:54,909][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:16:59,911][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:17:04,911][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:17:09,913][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:17:14,914][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:17:19,915][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:17:24,917][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:17:29,919][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:17:34,920][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:17:39,921][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:17:44,922][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:17:49,923][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:17:54,929][ INFO][PID:353277] Periodic builder liveness probe: alive [2025-12-30 19:17:54,930][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:17:59,932][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:18:04,933][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:18:09,935][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:18:14,937][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:18:19,941][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:18:24,942][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:18:29,943][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:18:34,945][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:18:39,946][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:18:44,947][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:18:49,948][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:18:54,949][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:18:59,950][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:19:04,951][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:19:09,952][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:19:14,953][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:19:19,954][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:19:24,955][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:19:29,956][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:19:34,957][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:19:39,958][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:19:44,960][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:19:49,961][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:19:54,962][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:19:59,963][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:20:04,964][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:20:09,965][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:20:14,966][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:20:19,967][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:20:24,969][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:20:29,970][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:20:34,971][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:20:39,972][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:20:44,973][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:20:49,974][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:20:54,974][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:20:59,976][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:21:04,977][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:21:09,978][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:21:14,979][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:21:19,979][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:21:25,002][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:21:30,003][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:21:35,004][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:21:40,005][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:21:45,005][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:21:50,006][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:21:55,007][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:22:00,024][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:22:05,025][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:22:10,026][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:22:15,027][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:22:20,028][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:22:25,029][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:22:30,030][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:22:35,031][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:22:40,032][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:22:45,033][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:22:50,034][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:22:55,035][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:23:00,036][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:23:05,037][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:23:10,038][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:23:15,039][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:23:20,040][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:23:25,041][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:23:30,042][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:23:35,043][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:23:40,043][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:23:45,044][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:23:50,045][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:23:55,046][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:24:00,047][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:24:05,048][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:24:10,049][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:24:15,050][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:24:20,051][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:24:25,052][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:24:30,053][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:24:35,054][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:24:40,055][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:24:45,056][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:24:50,057][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:24:55,058][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:25:00,059][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:25:05,059][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:25:10,060][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:25:15,061][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:25:20,062][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:25:25,063][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:25:30,064][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:25:35,065][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:25:40,068][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:25:45,070][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:25:50,071][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:25:55,072][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:26:00,074][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:26:05,075][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:26:10,076][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:26:15,077][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:26:20,078][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:26:25,079][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:26:30,080][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:26:35,081][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:26:40,082][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:26:45,083][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:26:50,084][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:26:55,085][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:27:00,086][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:27:05,087][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:27:10,088][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:27:15,089][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:27:20,090][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:27:25,090][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:27:30,092][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:27:35,093][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:27:40,094][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:27:45,095][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:27:50,096][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:27:55,102][ INFO][PID:353277] Periodic builder liveness probe: alive [2025-12-30 19:27:55,103][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:28:00,104][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:28:05,105][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:28:10,106][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:28:15,107][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:28:20,108][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:28:25,109][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:28:30,110][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:28:35,111][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:28:40,112][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:28:45,113][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:28:50,114][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:28:55,115][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:29:00,116][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:29:05,117][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:29:10,119][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:29:15,120][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:29:20,121][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:29:25,122][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:29:30,123][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:29:35,124][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:29:40,125][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:29:45,126][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:29:50,127][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:29:55,128][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:30:00,129][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:30:05,130][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:30:10,131][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:30:15,132][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:30:20,133][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:30:25,134][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:30:30,135][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:30:35,136][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:30:40,137][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:30:45,137][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:30:50,138][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:30:55,139][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:31:00,140][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:31:05,141][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:31:10,143][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:31:15,146][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:31:20,147][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:31:25,148][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:31:30,149][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:31:35,150][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:31:40,151][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:31:45,152][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:31:50,153][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:31:55,154][ INFO][PID:353277] Checking for cancel request [2025-12-30 19:31:55,279][ INFO][PID:353277] Downloading results from builder [2025-12-30 19:31:55,279][ INFO][PID:353277] rsyncing of mockbuilder@140.211.168.48:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/keithbowes/deponejo/fedora-42-ppc64le/09962046-deno started [2025-12-30 19:31:55,280][ INFO][PID:353277] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@140.211.168.48:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/keithbowes/deponejo/fedora-42-ppc64le/09962046-deno/ &> /var/lib/copr/public_html/results/keithbowes/deponejo/fedora-42-ppc64le/09962046-deno/build-09962046.rsync.log [2025-12-30 19:32:06,182][ INFO][PID:353277] rsyncing finished. [2025-12-30 19:32:06,183][ INFO][PID:353277] Releasing VM back to pool [2025-12-30 19:32:06,192][ INFO][PID:353277] Searching for 'success' file in resultdir [2025-12-30 19:32:06,193][ ERROR][PID:353277] Build failed: Backend process error: No success file => build failure [2025-12-30 19:32:06,193][ INFO][PID:353277] Finished build: id=9962046 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/keithbowes/deponejo chroot=fedora-42-ppc64le [2025-12-30 19:32:06,195][ ERROR][PID:353277] Backend process error: No success file => build failure [2025-12-30 19:32:06,196][ INFO][PID:353277] Worker failed build, took 1013.4528300762177 [2025-12-30 19:32:06,196][ INFO][PID:353277] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "keithbowes", "project_name": "deponejo", "project_dirname": "deponejo", "submitter": "keithbowes", "ended_on": 1767123126.1960392, "started_on": 1767122112.7432091, "submitted_on": null, "status": 0, "chroot": "fedora-42-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9962046-fedora-42-ppc64le", "build_id": 9962046, "package_name": "deno", "package_version": "2.6.3-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/keithbowes/deponejo/deno", "git_hash": "dc86085bb2ca2b975b489083e20424d07f26fc6d", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "keithbowes/deponejo--keithbowes", "results": null, "appstream": true, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/keithbowes/deponejo/fedora-42-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/keithbowes/deponejo", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/keithbowes/deponejo", "result_dir": "09962046-deno", "built_packages": "", "id": 9962046, "mockchain_macros": { "copr_username": "keithbowes", "copr_projectname": "deponejo", "vendor": "Fedora Project COPR (keithbowes/deponejo)" } } ] } [2025-12-30 19:32:06,266][ INFO][PID:353277] Sending fedora-messaging bus message in build.end [2025-12-30 19:32:06,817][ INFO][PID:353277] Compressing /var/lib/copr/public_html/results/keithbowes/deponejo/fedora-42-ppc64le/09962046-deno/builder-live.log by gzip [2025-12-30 19:32:06,818][ INFO][PID:353277] Running command 'gzip /var/lib/copr/public_html/results/keithbowes/deponejo/fedora-42-ppc64le/09962046-deno/builder-live.log' as PID 516793 [2025-12-30 19:32:06,853][ INFO][PID:353277] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/keithbowes/deponejo/fedora-42-ppc64le/09962046-deno/builder-live.log) [2025-12-30 19:32:06,854][ INFO][PID:353277] Compressing /var/lib/copr/public_html/results/keithbowes/deponejo/fedora-42-ppc64le/09962046-deno/backend.log by gzip [2025-12-30 19:32:06,855][ INFO][PID:353277] Running command 'gzip /var/lib/copr/public_html/results/keithbowes/deponejo/fedora-42-ppc64le/09962046-deno/backend.log' as PID 516794