[2026-04-22 18:05:11,649][ INFO][PID:4121800] Marking build as starting [2026-04-22 18:05:11,729][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:05:11,734][ INFO][PID:4121800] VM allocation process starts [2026-04-22 18:05:11,765][ INFO][PID:4121800] Trying to allocate VM: ResallocHost, ticket_id=2464494, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-22 18:05:14,806][ INFO][PID:4121800] Allocated host ResallocHost, ticket_id=2464494, hostname=13.221.207.221, name=aws_x86_64_spot_prod_07920863_20260422_174753, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-22 18:05:14,807][ INFO][PID:4121800] Allocating ssh connection to builder [2026-04-22 18:05:14,808][ INFO][PID:4121800] Checking that builder machine is OK [2026-04-22 18:05:15,159][ INFO][PID:4121800] Installed copr-rpmbuild version: 1.8 [2026-04-22 18:05:15,162][ INFO][PID:4121800] Running remote command: copr-builder-ready srpm-builds [2026-04-22 18:05:15,388][ INFO][PID:4121800] Red Hat subscription not needed for srpm-builds Builder is ready to be used [2026-04-22 18:05:15,391][ INFO][PID:4121800] Filling build.info file with builder info [2026-04-22 18:05:15,392][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:05:15,395][ INFO][PID:4121800] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": null, "enable_net": true, "project_owner": "@copr", "project_name": "PyPI", "project_dirname": "PyPI", "submitter": "ksurma", "ended_on": null, "started_on": 1776881115.395586, "submitted_on": null, "status": 3, "chroot": "srpm-builds", "arch": null, "buildroot_pkgs": null, "task_id": "10356564", "build_id": 10356564, "package_name": "python-gsbparse", "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 5, "source_json": "{\"pypi_package_name\": \"gsbparse\", \"pypi_package_version\": \"0.8.1\", \"spec_generator\": \"pyp2spec\", \"spec_template\": \"\", \"python_versions\": []}", "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": null, "sandbox": "@copr/PyPI--ksurma", "results": null, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "repos": [], "background": true, "distributions_in_build": [ "epel-9", "fedora-42", "fedora-43", "fedora-44", "fedora-rawhide" ], "distributions_in_project": [ "epel-9", "fedora-42", "fedora-43", "fedora-44", "fedora-rawhide" ], "destdir": "/var/lib/copr/public_html/results/@copr/PyPI", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI", "result_dir": "10356564", "built_packages": "", "tags": [ "arch_noarch" ], "id": 10356564, "mockchain_macros": { "copr_username": "@copr", "copr_projectname": "PyPI", "vendor": "Fedora Project COPR (@copr/PyPI)" } } ] } [2026-04-22 18:05:15,444][ INFO][PID:4121800] Sending fedora-messaging bus message in build.start [2026-04-22 18:05:16,074][ INFO][PID:4121800] Sending fedora-messaging bus message in chroot.start [2026-04-22 18:05:16,105][ INFO][PID:4121800] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/10356564 --detached [2026-04-22 18:05:16,539][ INFO][PID:4121800] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '13.221.207.221' (ED25519) to the list of known hosts. [2026-04-22 18:05:16,540][ INFO][PID:4121800] Downloading the builder-live.log file, attempt 1 [2026-04-22 18:05:16,542][ INFO][PID:4121800] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@13.221.207.221 copr-rpmbuild-log [2026-04-22 18:05:21,548][ INFO][PID:4121800] Periodic builder liveness probe: alive [2026-04-22 18:05:21,549][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:05:26,551][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:05:31,553][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:05:32,666][ ERROR][PID:4121800] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-04-22 18:05:36,555][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:05:41,556][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:05:42,677][WARNING][PID:4121800] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-04-22 18:05:46,560][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:05:51,567][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:05:52,678][ INFO][PID:4121800] Downloading the builder-live.log file, attempt 2 [2026-04-22 18:05:52,680][ INFO][PID:4121800] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@13.221.207.221 copr-rpmbuild-log [2026-04-22 18:05:56,573][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:06:01,574][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:06:06,576][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:06:11,581][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:06:16,599][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:06:21,601][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:06:26,610][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:06:31,612][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:06:36,613][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:06:41,615][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:06:46,617][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:06:51,618][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:06:52,749][ ERROR][PID:4121800] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-04-22 18:06:56,620][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:07:01,621][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:07:02,778][WARNING][PID:4121800] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-04-22 18:07:06,623][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:07:11,625][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:07:12,780][ INFO][PID:4121800] Downloading the builder-live.log file, attempt 3 [2026-04-22 18:07:12,787][ INFO][PID:4121800] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@13.221.207.221 copr-rpmbuild-log [2026-04-22 18:07:16,629][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:07:21,631][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:07:26,633][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:07:31,637][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:07:36,641][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:07:41,643][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:07:46,646][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:07:51,648][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:07:56,651][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:08:01,656][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:08:06,659][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:08:11,664][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:08:12,883][ ERROR][PID:4121800] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-04-22 18:08:16,667][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:08:21,671][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:08:22,934][WARNING][PID:4121800] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-04-22 18:08:26,675][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:08:31,680][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:08:32,937][ INFO][PID:4121800] Downloading the builder-live.log file, attempt 4 [2026-04-22 18:08:32,946][ INFO][PID:4121800] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@13.221.207.221 copr-rpmbuild-log [2026-04-22 18:08:36,685][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:08:41,687][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:08:46,690][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:08:51,692][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:08:56,700][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:09:01,708][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:09:06,718][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:09:11,720][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:09:16,724][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:09:21,736][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:09:26,743][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:09:31,750][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:09:33,046][ ERROR][PID:4121800] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-04-22 18:09:36,752][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:09:41,758][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:09:43,143][ ERROR][PID:4121800] Giving up for unstable SSH, failures: 1776881142.6773245, 1776881222.7787735, 1776881302.934879, 1776881383.1428869 [2026-04-22 18:09:43,173][ ERROR][PID:4121800] Re-try request for task on 'ResallocHost, ticket_id=2464494, hostname=13.221.207.221, name=aws_x86_64_spot_prod_07920863_20260422_174753, requested_tags=['arch_noarch', 'copr_builder']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-04-22 18:09:43,222][ INFO][PID:4121800] VM Release request [2026-04-22 18:09:43,244][ INFO][PID:4121800] Retry #1 (on other host) [2026-04-22 18:09:43,245][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:09:43,247][ INFO][PID:4121800] VM allocation process starts [2026-04-22 18:09:43,267][ INFO][PID:4121800] Trying to allocate VM: ResallocHost, ticket_id=2464626, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-22 18:09:48,271][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:09:49,310][ INFO][PID:4121800] Allocated host ResallocHost, ticket_id=2464626, hostname=52.70.193.105, name=aws_x86_64_normal_prod_07920643_20260422_174702, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-22 18:09:49,311][ INFO][PID:4121800] Allocating ssh connection to builder [2026-04-22 18:09:49,312][ INFO][PID:4121800] Checking that builder machine is OK [2026-04-22 18:09:49,584][ INFO][PID:4121800] Installed copr-rpmbuild version: 1.8 [2026-04-22 18:09:49,586][ INFO][PID:4121800] Running remote command: copr-builder-ready srpm-builds [2026-04-22 18:09:49,868][ INFO][PID:4121800] Red Hat subscription not needed for srpm-builds Builder is ready to be used [2026-04-22 18:09:49,869][ INFO][PID:4121800] Filling build.info file with builder info [2026-04-22 18:09:49,885][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:09:49,891][ INFO][PID:4121800] Not re-notifying FE and msg buses for the new host. [2026-04-22 18:09:49,892][ INFO][PID:4121800] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/10356564 --detached [2026-04-22 18:09:50,358][ INFO][PID:4121800] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '52.70.193.105' (ED25519) to the list of known hosts. [2026-04-22 18:09:50,359][ INFO][PID:4121800] Downloading the builder-live.log file, attempt 1 [2026-04-22 18:09:50,365][ INFO][PID:4121800] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.70.193.105 copr-rpmbuild-log [2026-04-22 18:09:55,360][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:10:00,367][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:10:05,370][ INFO][PID:4121800] Checking for cancel request [2026-04-22 18:10:07,667][ INFO][PID:4121800] Downloading results from builder [2026-04-22 18:10:07,679][ INFO][PID:4121800] rsyncing of mockbuilder@52.70.193.105:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/10356564 started [2026-04-22 18:10:07,688][ INFO][PID:4121800] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@52.70.193.105:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/10356564/ &> /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/10356564/build-10356564.rsync.log [2026-04-22 18:10:08,116][ INFO][PID:4121800] rsyncing finished. [2026-04-22 18:10:08,121][ INFO][PID:4121800] VM Release request [2026-04-22 18:10:08,175][ INFO][PID:4121800] Searching for 'success' file in resultdir [2026-04-22 18:10:08,193][ INFO][PID:4121800] Getting build details [2026-04-22 18:10:08,200][ INFO][PID:4121800] Retrieving SRPM info from /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/10356564 [2026-04-22 18:10:08,217][ INFO][PID:4121800] SRPM URL: https://download.copr.fedorainfracloud.org/results/@copr/PyPI/srpm-builds/10356564/python-gsbparse-0.8.1-1.src.rpm [2026-04-22 18:10:08,222][ INFO][PID:4121800] build details: {'srpm_url': 'https://download.copr.fedorainfracloud.org/results/@copr/PyPI/srpm-builds/10356564/python-gsbparse-0.8.1-1.src.rpm', 'pkg_name': 'python-gsbparse', 'pkg_version': '0.8.1-1'} [2026-04-22 18:10:08,232][ INFO][PID:4121800] Finished build: id=10356564 failed=False timeout=108000 destdir=/var/lib/copr/public_html/results/@copr/PyPI chroot=srpm-builds [2026-04-22 18:10:08,246][ INFO][PID:4121800] Worker succeeded build, took 18.354235649108887 [2026-04-22 18:10:08,254][ INFO][PID:4121800] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": null, "enable_net": true, "project_owner": "@copr", "project_name": "PyPI", "project_dirname": "PyPI", "submitter": "ksurma", "ended_on": 1776881408.2461545, "started_on": 1776881389.891919, "submitted_on": null, "status": 1, "chroot": "srpm-builds", "arch": null, "buildroot_pkgs": null, "task_id": "10356564", "build_id": 10356564, "package_name": "python-gsbparse", "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 5, "source_json": "{\"pypi_package_name\": \"gsbparse\", \"pypi_package_version\": \"0.8.1\", \"spec_generator\": \"pyp2spec\", \"spec_template\": \"\", \"python_versions\": []}", "pkg_name": "python-gsbparse", "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI/srpm-builds/10356564/python-gsbparse-0.8.1-1.src.rpm", "uses_devel_repo": null, "sandbox": "@copr/PyPI--ksurma", "results": { "architecture_specific_tags": { "epel-9": { "buildarch": [ "noarch" ] }, "fedora-42": { "buildarch": [ "noarch" ] }, "fedora-43": { "buildarch": [ "noarch" ] }, "fedora-44": { "buildarch": [ "noarch" ] }, "fedora-rawhide": { "buildarch": [ "noarch" ] } }, "name": "python-gsbparse", "epoch": null, "version": "0.8.1", "release": "1" }, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "repos": [], "background": true, "distributions_in_build": [ "epel-9", "fedora-42", "fedora-43", "fedora-44", "fedora-rawhide" ], "distributions_in_project": [ "epel-9", "fedora-42", "fedora-43", "fedora-44", "fedora-rawhide" ], "destdir": "/var/lib/copr/public_html/results/@copr/PyPI", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI", "result_dir": "10356564", "built_packages": "", "tags": [ "arch_noarch" ], "pkg_version": "0.8.1-1", "id": 10356564, "mockchain_macros": { "copr_username": "@copr", "copr_projectname": "PyPI", "vendor": "Fedora Project COPR (@copr/PyPI)" } } ] } [2026-04-22 18:10:08,417][ INFO][PID:4121800] Sending fedora-messaging bus message in build.end [2026-04-22 18:10:08,454][ INFO][PID:4121800] Compressing /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/10356564/builder-live.log by gzip [2026-04-22 18:10:08,458][ INFO][PID:4121800] Running command 'gzip /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/10356564/builder-live.log' as PID 4158520 [2026-04-22 18:10:08,463][ INFO][PID:4121800] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/10356564/builder-live.log) [2026-04-22 18:10:08,465][ INFO][PID:4121800] Compressing /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/10356564/backend.log by gzip [2026-04-22 18:10:08,472][ INFO][PID:4121800] Running command 'gzip /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/10356564/backend.log' as PID 4158521