[2025-11-12 20:02:22,044][ INFO][PID:3508759] Marking build as starting [2025-11-12 20:02:23,035][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:02:23,036][ INFO][PID:3508759] VM allocation process starts [2025-11-12 20:02:23,059][ INFO][PID:3508759] Trying to allocate VM: ResallocHost, ticket_id=6150901, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-11-12 20:02:28,061][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:02:29,089][ INFO][PID:3508759] Allocated host ResallocHost, ticket_id=6150901, hostname=2620:52:6:1161:dead:beef:cafe:c156, name=vmhost_p09_03_prod_07962947_20251112_190541, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-11-12 20:02:29,091][ INFO][PID:3508759] Allocating ssh connection to builder [2025-11-12 20:02:29,091][ INFO][PID:3508759] Checking that builder machine is OK [2025-11-12 20:02:30,392][ INFO][PID:3508759] Running remote command: copr-builder-ready centos-stream-9-ppc64le [2025-11-12 20:03:30,469][ ERROR][PID:3508759] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c156 port 22: Connection timed out [2025-11-12 20:03:40,470][ ERROR][PID:3508759] Re-try request for task on 'ResallocHost, ticket_id=6150901, hostname=2620:52:6:1161:dead:beef:cafe:c156, name=vmhost_p09_03_prod_07962947_20251112_190541, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': Unable to finish after 1 SSH attempts [2025-11-12 20:03:40,471][ INFO][PID:3508759] Releasing VM back to pool [2025-11-12 20:03:40,482][ INFO][PID:3508759] Retry #1 (on other host) [2025-11-12 20:03:40,483][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:03:40,483][ INFO][PID:3508759] VM allocation process starts [2025-11-12 20:03:40,495][ INFO][PID:3508759] Trying to allocate VM: ResallocHost, ticket_id=6150946, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-11-12 20:03:43,510][ INFO][PID:3508759] Allocated host ResallocHost, ticket_id=6150946, hostname=2620:52:6:1161:dead:beef:cafe:c18b, name=vmhost_p09_04_prod_07963077_20251112_190931, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-11-12 20:03:43,511][ INFO][PID:3508759] Allocating ssh connection to builder [2025-11-12 20:03:43,511][ INFO][PID:3508759] Checking that builder machine is OK [2025-11-12 20:03:44,402][ INFO][PID:3508759] Running remote command: copr-builder-ready centos-stream-9-ppc64le [2025-11-12 20:03:44,869][ INFO][PID:3508759] Red Hat subscription not needed for centos-stream-9-ppc64le Builder is ready to be used [2025-11-12 20:03:44,869][ INFO][PID:3508759] Filling build.info file with builder info [2025-11-12 20:03:44,870][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:03:44,871][ INFO][PID:3508759] Not re-notifying FE and msg buses for the new host. [2025-11-12 20:03:44,871][ INFO][PID:3508759] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9792304-centos-stream-9-ppc64le --chroot centos-stream-9-ppc64le --detached [2025-11-12 20:03:45,799][ INFO][PID:3508759] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18b' (ED25519) to the list of known hosts. [2025-11-12 20:03:45,800][ INFO][PID:3508759] Downloading the builder-live.log file, attempt 1 [2025-11-12 20:03:45,801][ INFO][PID:3508759] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18b copr-rpmbuild-log [2025-11-12 20:03:50,816][ INFO][PID:3508759] Periodic builder liveness probe: alive [2025-11-12 20:03:50,817][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:03:55,817][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:04:00,818][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:04:05,819][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:04:10,820][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:04:15,821][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:04:20,822][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:04:25,823][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:04:30,824][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:04:35,825][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:04:40,826][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:04:45,827][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:04:45,869][ ERROR][PID:3508759] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-11-12 20:04:50,828][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:04:55,830][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:04:55,870][WARNING][PID:3508759] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-11-12 20:05:00,831][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:05:05,832][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:05:05,871][ INFO][PID:3508759] Downloading the builder-live.log file, attempt 2 [2025-11-12 20:05:05,872][ INFO][PID:3508759] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18b copr-rpmbuild-log [2025-11-12 20:05:10,833][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:05:15,834][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:05:20,835][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:05:25,836][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:05:30,837][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:05:35,839][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:05:40,840][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:05:45,841][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:05:50,842][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:05:55,846][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:06:00,850][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:06:05,852][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:06:05,942][ ERROR][PID:3508759] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-11-12 20:06:10,855][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:06:15,856][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:06:15,944][WARNING][PID:3508759] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-11-12 20:06:20,858][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:06:25,859][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:06:25,945][ INFO][PID:3508759] Downloading the builder-live.log file, attempt 3 [2025-11-12 20:06:25,947][ INFO][PID:3508759] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18b copr-rpmbuild-log [2025-11-12 20:06:30,860][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:06:35,861][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:06:40,863][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:06:45,864][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:06:50,866][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:06:55,867][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:07:00,868][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:07:05,869][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:07:10,870][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:07:15,871][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:07:20,872][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:07:25,875][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:07:30,876][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:07:35,877][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:07:40,878][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:07:45,879][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:07:50,880][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:07:55,881][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:08:00,883][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:08:05,885][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:08:10,890][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:08:15,893][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:08:20,894][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:08:25,897][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:08:30,900][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:08:35,903][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:08:40,904][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:08:45,905][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:08:50,908][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:08:55,913][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:09:00,919][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:09:05,920][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:09:10,927][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:09:15,928][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:09:20,930][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:09:25,932][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:09:30,935][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:09:35,939][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:09:40,942][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:09:45,947][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:09:50,949][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:09:55,952][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:10:00,958][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:10:05,959][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:10:10,966][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:10:15,969][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:10:20,970][ INFO][PID:3508759] Checking for cancel request [2025-11-12 20:10:22,795][ INFO][PID:3508759] Downloading results from builder [2025-11-12 20:10:22,798][ INFO][PID:3508759] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18b]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09792304-containers-common started [2025-11-12 20:10:22,801][ INFO][PID:3508759] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18b]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09792304-containers-common/ &> /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09792304-containers-common/build-09792304.rsync.log [2025-11-12 20:10:23,436][ INFO][PID:3508759] rsyncing finished. [2025-11-12 20:10:23,439][ INFO][PID:3508759] Releasing VM back to pool [2025-11-12 20:10:23,473][ INFO][PID:3508759] Searching for 'success' file in resultdir [2025-11-12 20:10:23,477][ ERROR][PID:3508759] Build failed: Backend process error: No success file => build failure [2025-11-12 20:10:23,480][ INFO][PID:3508759] Finished build: id=9792304 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/rhcontainerbot/podman-next chroot=centos-stream-9-ppc64le [2025-11-12 20:10:23,485][ ERROR][PID:3508759] Backend process error: No success file => build failure [2025-11-12 20:10:23,487][ INFO][PID:3508759] Worker failed build, took 398.6158866882324 [2025-11-12 20:10:23,490][ INFO][PID:3508759] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "rhcontainerbot", "project_name": "podman-next", "project_dirname": "podman-next", "submitter": "packit", "ended_on": 1762978223.4869225, "started_on": 1762977824.8710358, "submitted_on": null, "status": 0, "chroot": "centos-stream-9-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9792304-centos-stream-9-ppc64le", "build_id": 9792304, "package_name": "containers-common", "package_version": "102:0.67.0~dev-1.20251112200150369033.main.121.g4afce3558e", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rhcontainerbot/podman-next/containers-common", "git_hash": "222dc3235dace21e17435b4f3f8b1d9bafc5ffd9", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "rhcontainerbot/podman-next--packit", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://dl.fedoraproject.org/pub/epel/9/Everything/$basearch/", "id": "https_dl_fedoraproject_org_pub_epel_9_Everything_basearch", "name": "Additional repo https_dl_fedoraproject_org_pub_epel_9_Everything_basearch" } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le", "arch_power9" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/podman-next", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next", "result_dir": "09792304-containers-common", "built_packages": "", "id": 9792304, "mockchain_macros": { "copr_username": "rhcontainerbot", "copr_projectname": "podman-next", "vendor": "Fedora Project COPR (rhcontainerbot/podman-next)" } } ] } [2025-11-12 20:10:23,568][ INFO][PID:3508759] Sending fedora-messaging bus message in build.end [2025-11-12 20:10:24,667][ INFO][PID:3508759] Compressing /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09792304-containers-common/builder-live.log by gzip [2025-11-12 20:10:24,674][ INFO][PID:3508759] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09792304-containers-common/builder-live.log' as PID 3567019 [2025-11-12 20:10:24,684][ INFO][PID:3508759] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09792304-containers-common/builder-live.log) [2025-11-12 20:10:24,687][ INFO][PID:3508759] Compressing /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09792304-containers-common/backend.log by gzip [2025-11-12 20:10:24,694][ INFO][PID:3508759] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/podman-next/centos-stream-9-ppc64le/09792304-containers-common/backend.log' as PID 3567021