[2025-09-08 15:38:04,682][ INFO][PID:4029066] Marking build as starting [2025-09-08 15:38:04,742][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:38:04,743][ INFO][PID:4029066] VM allocation process starts [2025-09-08 15:38:04,756][ INFO][PID:4029066] Trying to allocate VM: ResallocHost, ticket_id=4964313, requested_tags=['copr_builder', 'arch_x86_64'] [2025-09-08 15:38:07,803][ INFO][PID:4029066] Allocated host ResallocHost, ticket_id=4964313, hostname=52.203.203.143, name=aws_x86_64_spot_prod_06669435_20250908_153443, requested_tags=['copr_builder', 'arch_x86_64'] [2025-09-08 15:38:07,803][ INFO][PID:4029066] Allocating ssh connection to builder [2025-09-08 15:38:07,804][ INFO][PID:4029066] Checking that builder machine is OK [2025-09-08 15:38:08,094][ INFO][PID:4029066] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-09-08 15:38:08,293][ INFO][PID:4029066] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-09-08 15:38:08,294][ INFO][PID:4029066] Filling build.info file with builder info [2025-09-08 15:38:08,300][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:38:08,301][ INFO][PID:4029066] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "rhcontainerbot", "project_name": "podman-next", "project_dirname": "podman-next", "submitter": "packit", "ended_on": null, "started_on": 1757345888.301413, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9534383-fedora-rawhide-x86_64", "build_id": 9534383, "package_name": "netavark", "package_version": "102:1.17.0~dev-1.20250908134501975313.main.26.g07dd9e4", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rhcontainerbot/podman-next/netavark", "git_hash": "1c0f039df21e05a8ff23017c66b1bd8efe9f0de9", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "rhcontainerbot/podman-next--packit", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/podman-next", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next", "result_dir": "09534383-netavark", "built_packages": "", "id": 9534383, "mockchain_macros": { "copr_username": "rhcontainerbot", "copr_projectname": "podman-next", "vendor": "Fedora Project COPR (rhcontainerbot/podman-next)" } } ] } [2025-09-08 15:38:08,361][ INFO][PID:4029066] Sending fedora-messaging bus message in build.start [2025-09-08 15:38:08,993][ INFO][PID:4029066] Sending fedora-messaging bus message in chroot.start [2025-09-08 15:38:09,016][ INFO][PID:4029066] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9534383-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-09-08 15:38:09,336][ INFO][PID:4029066] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '52.203.203.143' (ED25519) to the list of known hosts. [2025-09-08 15:38:09,337][ INFO][PID:4029066] Downloading the builder-live.log file, attempt 1 [2025-09-08 15:38:09,338][ INFO][PID:4029066] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.203.203.143 copr-rpmbuild-log [2025-09-08 15:38:14,342][ INFO][PID:4029066] Periodic builder liveness probe: alive [2025-09-08 15:38:14,342][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:38:19,343][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:38:24,344][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:38:29,345][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:38:34,346][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:38:39,347][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:38:44,348][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:38:49,349][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:38:54,350][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:38:59,351][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:39:04,352][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:39:09,353][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:39:15,449][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:39:20,450][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:39:25,451][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:39:30,452][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:39:35,453][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:39:40,454][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:39:45,455][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:39:50,456][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:39:55,457][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:40:00,458][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:40:05,459][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:40:10,460][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:40:15,461][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:40:20,462][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:40:25,464][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:40:30,464][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:40:35,465][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:40:40,467][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:40:45,468][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:40:50,469][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:40:55,470][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:41:00,471][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:41:05,473][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:41:10,474][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:41:15,475][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:41:20,476][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:41:25,477][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:41:30,478][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:41:35,479][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:41:40,480][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:41:45,480][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:41:50,481][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:41:55,482][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:42:00,483][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:42:05,484][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:42:10,485][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:42:15,486][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:42:20,487][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:42:25,488][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:42:30,489][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:42:35,490][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:42:40,491][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:42:45,492][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:42:50,493][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:42:55,494][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:43:00,495][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:43:05,496][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:43:10,497][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:43:15,498][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:43:20,499][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:43:25,500][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:43:30,501][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:43:35,502][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:43:40,503][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:43:45,503][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:43:50,504][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:43:55,505][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:44:00,506][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:44:05,507][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:44:10,508][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:44:15,509][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:44:20,510][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:44:25,511][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:44:30,512][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:44:35,513][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:44:40,514][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:44:45,515][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:44:50,516][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:44:55,517][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:45:00,518][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:45:05,519][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:45:10,521][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:45:15,522][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:45:20,522][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:45:25,523][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:45:30,524][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:45:35,525][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:45:40,526][ INFO][PID:4029066] Checking for cancel request [2025-09-08 15:45:44,776][ INFO][PID:4029066] Downloading results from builder [2025-09-08 15:45:44,777][ INFO][PID:4029066] rsyncing of mockbuilder@52.203.203.143:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark started [2025-09-08 15:45:44,778][ INFO][PID:4029066] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@52.203.203.143:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/ &> /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/build-09534383.rsync.log [2025-09-08 16:25:10,673][ INFO][PID:4029066] rsyncing finished. [2025-09-08 16:25:10,674][ ERROR][PID:4029066] Failed to download data from builder due to rsync error, see the rsync log file for details. [2025-09-08 16:25:10,674][ ERROR][PID:4029066] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details. [2025-09-08 16:25:20,675][ INFO][PID:4029066] rsyncing of mockbuilder@52.203.203.143:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark started [2025-09-08 16:25:20,676][ INFO][PID:4029066] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@52.203.203.143:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/ &> /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/build-09534383.rsync.log [2025-09-08 16:26:20,754][ INFO][PID:4029066] rsyncing finished. [2025-09-08 16:26:20,755][ ERROR][PID:4029066] Failed to download data from builder due to rsync error, see the rsync log file for details. [2025-09-08 16:26:20,755][ ERROR][PID:4029066] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details. [2025-09-08 16:26:30,756][ INFO][PID:4029066] rsyncing of mockbuilder@52.203.203.143:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark started [2025-09-08 16:26:30,757][ INFO][PID:4029066] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@52.203.203.143:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/ &> /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/build-09534383.rsync.log [2025-09-08 16:27:30,837][ INFO][PID:4029066] rsyncing finished. [2025-09-08 16:27:30,838][ ERROR][PID:4029066] Failed to download data from builder due to rsync error, see the rsync log file for details. [2025-09-08 16:27:30,839][ ERROR][PID:4029066] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details. [2025-09-08 16:27:40,839][ ERROR][PID:4029066] Re-try request for task on 'ResallocHost, ticket_id=4964313, hostname=52.203.203.143, name=aws_x86_64_spot_prod_06669435_20250908_153443, requested_tags=['copr_builder', 'arch_x86_64']': Unable to finish after 3 SSH attempts [2025-09-08 16:27:40,840][ INFO][PID:4029066] Releasing VM back to pool [2025-09-08 16:27:40,849][ INFO][PID:4029066] Retry #1 (on other host) [2025-09-08 16:27:40,850][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:27:40,850][ INFO][PID:4029066] VM allocation process starts [2025-09-08 16:27:40,876][ INFO][PID:4029066] Trying to allocate VM: ResallocHost, ticket_id=4964537, requested_tags=['copr_builder', 'arch_x86_64'] [2025-09-08 16:27:43,929][ INFO][PID:4029066] Allocated host ResallocHost, ticket_id=4964537, hostname=44.206.247.253, name=aws_x86_64_spot_prod_06669707_20250908_162608, requested_tags=['copr_builder', 'arch_x86_64'] [2025-09-08 16:27:43,930][ INFO][PID:4029066] Allocating ssh connection to builder [2025-09-08 16:27:43,930][ INFO][PID:4029066] Checking that builder machine is OK [2025-09-08 16:27:44,357][ INFO][PID:4029066] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-09-08 16:27:44,555][ INFO][PID:4029066] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-09-08 16:27:44,555][ INFO][PID:4029066] Filling build.info file with builder info [2025-09-08 16:27:44,557][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:27:44,558][ INFO][PID:4029066] Not re-notifying FE and msg buses for the new host. [2025-09-08 16:27:44,558][ INFO][PID:4029066] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9534383-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-09-08 16:27:44,872][ INFO][PID:4029066] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '44.206.247.253' (ED25519) to the list of known hosts. [2025-09-08 16:27:44,873][ INFO][PID:4029066] Downloading the builder-live.log file, attempt 1 [2025-09-08 16:27:44,874][ INFO][PID:4029066] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.206.247.253 copr-rpmbuild-log [2025-09-08 16:27:49,878][ INFO][PID:4029066] Periodic builder liveness probe: alive [2025-09-08 16:27:49,878][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:27:54,879][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:27:59,880][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:28:04,881][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:28:09,882][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:28:14,883][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:28:19,884][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:28:24,885][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:28:29,886][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:28:34,887][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:28:39,888][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:28:44,889][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:28:49,890][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:28:54,891][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:28:59,892][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:29:04,893][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:29:09,894][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:29:14,895][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:29:19,896][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:29:24,897][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:29:29,898][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:29:34,899][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:29:39,900][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:29:44,901][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:29:49,902][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:29:54,903][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:29:59,904][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:31:40,166][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:31:45,167][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:32:12,422][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:32:32,814][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:32:37,815][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:32:42,816][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:33:55,830][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:34:00,831][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:34:05,832][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:35:27,599][ INFO][PID:4029066] Downloading results from builder [2025-09-08 16:35:27,600][ INFO][PID:4029066] rsyncing of mockbuilder@44.206.247.253:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark started [2025-09-08 16:35:27,601][ INFO][PID:4029066] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@44.206.247.253:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/ &> /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/build-09534383.rsync.log [2025-09-08 16:36:27,677][ INFO][PID:4029066] rsyncing finished. [2025-09-08 16:37:17,182][ ERROR][PID:4029066] Failed to download data from builder due to rsync error, see the rsync log file for details. [2025-09-08 16:37:17,182][ ERROR][PID:4029066] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details. [2025-09-08 16:37:27,183][ INFO][PID:4029066] rsyncing of mockbuilder@44.206.247.253:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark started [2025-09-08 16:37:39,247][ INFO][PID:4029066] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@44.206.247.253:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/ &> /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/build-09534383.rsync.log [2025-09-08 16:38:39,324][ INFO][PID:4029066] rsyncing finished. [2025-09-08 16:39:20,294][ ERROR][PID:4029066] Failed to download data from builder due to rsync error, see the rsync log file for details. [2025-09-08 16:39:20,294][ ERROR][PID:4029066] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details. [2025-09-08 16:39:30,295][ INFO][PID:4029066] rsyncing of mockbuilder@44.206.247.253:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark started [2025-09-08 16:39:30,296][ INFO][PID:4029066] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@44.206.247.253:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/ &> /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/build-09534383.rsync.log [2025-09-08 16:40:30,374][ INFO][PID:4029066] rsyncing finished. [2025-09-08 16:40:42,230][ ERROR][PID:4029066] Failed to download data from builder due to rsync error, see the rsync log file for details. [2025-09-08 16:40:42,230][ ERROR][PID:4029066] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details. [2025-09-08 16:40:52,231][ ERROR][PID:4029066] Re-try request for task on 'ResallocHost, ticket_id=4964537, hostname=44.206.247.253, name=aws_x86_64_spot_prod_06669707_20250908_162608, requested_tags=['copr_builder', 'arch_x86_64']': Unable to finish after 3 SSH attempts [2025-09-08 16:40:52,231][ INFO][PID:4029066] Releasing VM back to pool [2025-09-08 16:40:52,241][ INFO][PID:4029066] Retry #2 (on other host) [2025-09-08 16:40:52,242][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:40:52,242][ INFO][PID:4029066] VM allocation process starts [2025-09-08 16:40:52,322][ INFO][PID:4029066] Trying to allocate VM: ResallocHost, ticket_id=4964587, requested_tags=['copr_builder', 'arch_x86_64'] [2025-09-08 16:40:57,323][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:41:02,324][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:41:07,325][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:41:58,294][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:42:03,295][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:42:08,296][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:42:13,297][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:42:40,414][ INFO][PID:4029066] Allocated host ResallocHost, ticket_id=4964587, hostname=2620:52:3:1:dead:beef:cafe:c190, name=copr_hv_x86_64_03_prod_06669850_20250908_164103, requested_tags=['copr_builder', 'arch_x86_64'] [2025-09-08 16:42:40,414][ INFO][PID:4029066] Allocating ssh connection to builder [2025-09-08 16:42:40,622][ INFO][PID:4029066] Checking that builder machine is OK [2025-09-08 16:42:41,197][ INFO][PID:4029066] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-09-08 16:42:41,506][ INFO][PID:4029066] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-09-08 16:42:41,507][ INFO][PID:4029066] Filling build.info file with builder info [2025-09-08 16:42:41,509][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:42:41,718][ INFO][PID:4029066] Not re-notifying FE and msg buses for the new host. [2025-09-08 16:42:41,719][ INFO][PID:4029066] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9534383-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-09-08 16:42:42,281][ INFO][PID:4029066] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c190' (ED25519) to the list of known hosts. [2025-09-08 16:42:42,282][ INFO][PID:4029066] Downloading the builder-live.log file, attempt 1 [2025-09-08 16:42:42,283][ INFO][PID:4029066] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c190 copr-rpmbuild-log [2025-09-08 16:42:47,288][ INFO][PID:4029066] Periodic builder liveness probe: alive [2025-09-08 16:42:47,288][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:42:52,289][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:43:58,646][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:44:03,647][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:44:08,648][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:45:11,470][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:45:16,471][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:45:21,472][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:45:26,473][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:46:19,934][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:46:24,935][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:46:29,936][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:46:34,937][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:46:39,938][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:46:44,939][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:47:50,670][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:47:55,671][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:48:00,672][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:48:05,673][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:48:10,674][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:50:54,166][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:51:01,454][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:51:07,910][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:51:12,911][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:51:17,912][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:51:22,913][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:51:53,911][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:51:58,912][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:52:04,326][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:52:09,327][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:52:14,328][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:52:25,358][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:52:31,814][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:52:37,646][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:52:46,391][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:52:51,398][ INFO][PID:4029066] Periodic builder liveness probe: alive [2025-09-08 16:52:51,398][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:52:56,399][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:53:01,400][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:53:06,401][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:53:11,402][ INFO][PID:4029066] Checking for cancel request [2025-09-08 16:53:58,713][ INFO][PID:4029066] Downloading results from builder [2025-09-08 16:53:58,713][ INFO][PID:4029066] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c190]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark started [2025-09-08 16:53:58,714][ INFO][PID:4029066] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c190]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/ &> /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/build-09534383.rsync.log [2025-09-08 18:55:16,327][ INFO][PID:4029066] rsyncing finished. [2025-09-08 18:55:16,333][ INFO][PID:4029066] Releasing VM back to pool [2025-09-08 18:55:16,551][ INFO][PID:4029066] Searching for 'success' file in resultdir [2025-09-08 18:55:20,502][ INFO][PID:4029066] Going to sign pkgs from source: 9534383-fedora-rawhide-x86_64 in chroot: /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64 [2025-09-08 18:55:20,503][ INFO][PID:4029066] Calling '/bin/sign -u rhcontainerbot#podman-next@copr.fedorahosted.org -p' (attempt #1) [2025-09-08 18:55:20,837][ INFO][PID:4029066] Calling '/bin/sign -4 -h sha256 -u rhcontainerbot#podman-next@copr.fedorahosted.org -r /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/netavark-1.17.0~dev-1.20250908134501975313.main.26.g07dd9e4.fc44.x86_64.rpm' (attempt #1) [2025-09-08 18:58:57,431][ INFO][PID:4029066] signed rpm: /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/netavark-1.17.0~dev-1.20250908134501975313.main.26.g07dd9e4.fc44.x86_64.rpm [2025-09-08 18:58:57,432][ INFO][PID:4029066] Calling '/bin/sign -4 -h sha256 -u rhcontainerbot#podman-next@copr.fedorahosted.org -r /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/netavark-debuginfo-1.17.0~dev-1.20250908134501975313.main.26.g07dd9e4.fc44.x86_64.rpm' (attempt #1) [2025-09-08 20:58:58,549][ INFO][PID:4029066] signed rpm: /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/netavark-debuginfo-1.17.0~dev-1.20250908134501975313.main.26.g07dd9e4.fc44.x86_64.rpm [2025-09-08 20:58:58,557][ INFO][PID:4029066] Calling '/bin/sign -4 -h sha256 -u rhcontainerbot#podman-next@copr.fedorahosted.org -r /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/netavark-1.17.0~dev-1.20250908134501975313.main.26.g07dd9e4.fc44.src.rpm' (attempt #1) [2025-09-08 20:59:01,864][ INFO][PID:4029066] signed rpm: /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/netavark-1.17.0~dev-1.20250908134501975313.main.26.g07dd9e4.fc44.src.rpm [2025-09-08 20:59:01,865][ INFO][PID:4029066] Calling '/bin/sign -4 -h sha256 -u rhcontainerbot#podman-next@copr.fedorahosted.org -r /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/netavark-debugsource-1.17.0~dev-1.20250908134501975313.main.26.g07dd9e4.fc44.x86_64.rpm' (attempt #1) [2025-09-08 21:18:25,021][ INFO][PID:4029066] signed rpm: /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/netavark-debugsource-1.17.0~dev-1.20250908134501975313.main.26.g07dd9e4.fc44.x86_64.rpm [2025-09-08 21:18:25,021][ INFO][PID:4029066] Sign done [2025-09-08 21:18:25,243][ INFO][PID:4029066] Incremental createrepo run, adding 09534383-netavark into https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64, (auto-create-repo=True) [2025-09-08 21:18:25,285][ INFO][PID:4029066] Running command 'copr-repo --batched /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64 --add 09534383-netavark --no-appstream-metadata' as PID 729280 [2025-09-08 21:24:26,640][ INFO][PID:4029066] Finished after 361 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64 --add 09534383-netavark --no-appstream-metadata) [2025-09-08 21:24:26,669][ INFO][PID:4029066] Getting build details [2025-09-08 21:24:26,672][ INFO][PID:4029066] Listing built binary packages in /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark [2025-09-08 21:24:26,674][ INFO][PID:4029066] Built packages: ['netavark-debuginfo 1.17.0~dev', 'netavark-debugsource 1.17.0~dev', 'netavark 1.17.0~dev'] [2025-09-08 21:24:26,778][ INFO][PID:4029066] build details: {'built_packages': 'netavark-debuginfo 1.17.0~dev\nnetavark-debugsource 1.17.0~dev\nnetavark 1.17.0~dev'} [2025-09-08 21:24:26,781][ INFO][PID:4029066] Retrieving pubkey [2025-09-08 21:24:26,782][ INFO][PID:4029066] Calling '/bin/sign -u rhcontainerbot#podman-next@copr.fedorahosted.org -p' (attempt #1) [2025-09-08 21:24:27,126][ INFO][PID:4029066] Added pubkey for user rhcontainerbot project podman-next into: /var/lib/copr/public_html/results/rhcontainerbot/podman-next/pubkey.gpg [2025-09-08 21:24:27,341][ INFO][PID:4029066] Finished build: id=9534383 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/rhcontainerbot/podman-next chroot=fedora-rawhide-x86_64 [2025-09-08 21:24:27,441][ INFO][PID:4029066] Worker succeeded build, took 16905.7224919796 [2025-09-08 21:24:27,454][ INFO][PID:4029066] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "rhcontainerbot", "project_name": "podman-next", "project_dirname": "podman-next", "submitter": "packit", "ended_on": 1757366667.4409635, "started_on": 1757349761.7184715, "submitted_on": null, "status": 1, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9534383-fedora-rawhide-x86_64", "build_id": 9534383, "package_name": "netavark", "package_version": "102:1.17.0~dev-1.20250908134501975313.main.26.g07dd9e4", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rhcontainerbot/podman-next/netavark", "git_hash": "1c0f039df21e05a8ff23017c66b1bd8efe9f0de9", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "rhcontainerbot/podman-next--packit", "results": { "packages": [ { "name": "netavark-debuginfo", "epoch": 102, "version": "1.17.0~dev", "release": "1.20250908134501975313.main.26.g07dd9e4.fc44", "arch": "x86_64" }, { "name": "netavark-debugsource", "epoch": 102, "version": "1.17.0~dev", "release": "1.20250908134501975313.main.26.g07dd9e4.fc44", "arch": "x86_64" }, { "name": "netavark", "epoch": 102, "version": "1.17.0~dev", "release": "1.20250908134501975313.main.26.g07dd9e4.fc44", "arch": "src" }, { "name": "netavark", "epoch": 102, "version": "1.17.0~dev", "release": "1.20250908134501975313.main.26.g07dd9e4.fc44", "arch": "x86_64" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/podman-next", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next", "result_dir": "09534383-netavark", "built_packages": "netavark-debuginfo 1.17.0~dev\nnetavark-debugsource 1.17.0~dev\nnetavark 1.17.0~dev", "id": 9534383, "mockchain_macros": { "copr_username": "rhcontainerbot", "copr_projectname": "podman-next", "vendor": "Fedora Project COPR (rhcontainerbot/podman-next)" } } ] } [2025-09-08 21:24:28,107][ INFO][PID:4029066] Sending fedora-messaging bus message in build.end [2025-09-08 21:24:28,693][ INFO][PID:4029066] Compressing /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/builder-live.log by gzip [2025-09-08 21:24:28,695][ INFO][PID:4029066] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/builder-live.log' as PID 742804 [2025-09-08 21:24:28,729][ INFO][PID:4029066] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/builder-live.log) [2025-09-08 21:24:28,731][ INFO][PID:4029066] Compressing /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/backend.log by gzip [2025-09-08 21:24:28,732][ INFO][PID:4029066] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/podman-next/fedora-rawhide-x86_64/09534383-netavark/backend.log' as PID 742811