[2026-04-20 13:33:24,857][ INFO][PID:2171193] Marking build as starting [2026-04-20 13:33:24,948][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:33:24,953][ INFO][PID:2171193] VM allocation process starts [2026-04-20 13:33:24,990][ INFO][PID:2171193] Trying to allocate VM: ResallocHost, ticket_id=2422508, requested_tags=['arch_x86_64', 'copr_builder'] [2026-04-20 13:33:29,999][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:33:35,003][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:33:40,012][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:33:45,014][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:33:50,021][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:33:55,023][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:34:00,030][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:34:05,034][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:34:10,037][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:34:15,048][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:34:20,057][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:34:25,059][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:34:30,061][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:34:35,069][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:34:40,079][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:34:45,094][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:34:50,104][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:34:53,238][ INFO][PID:2171193] Allocated host ResallocHost, ticket_id=2422508, hostname=2620:52:6:1161:dead:beef:cafe:c141, name=vmhost_x86_01_prod_07132470_20260420_133208, requested_tags=['arch_x86_64', 'copr_builder'] [2026-04-20 13:34:53,240][ INFO][PID:2171193] Allocating ssh connection to builder [2026-04-20 13:34:53,241][ INFO][PID:2171193] Checking that builder machine is OK [2026-04-20 13:34:53,735][ ERROR][PID:2171193] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c141' (ED25519) to the list of known hosts. Load key "/home/copr/.ssh/id_rsa": Permission denied Permission denied, please try again. Permission denied, please try again. mockbuilder@2620:52:6:1161:dead:beef:cafe:c141: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password). [2026-04-20 13:35:03,863][ ERROR][PID:2171193] Re-try request for task on 'ResallocHost, ticket_id=2422508, hostname=2620:52:6:1161:dead:beef:cafe:c141, name=vmhost_x86_01_prod_07132470_20260420_133208, requested_tags=['arch_x86_64', 'copr_builder']': Unable to finish after 1 SSH attempts [2026-04-20 13:35:04,000][ INFO][PID:2171193] VM Release request [2026-04-20 13:35:04,076][ INFO][PID:2171193] Retry #1 (on other host) [2026-04-20 13:35:04,080][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:35:04,092][ INFO][PID:2171193] VM allocation process starts [2026-04-20 13:35:04,190][ INFO][PID:2171193] Trying to allocate VM: ResallocHost, ticket_id=2422587, requested_tags=['arch_x86_64', 'copr_builder'] [2026-04-20 13:35:09,197][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:35:14,213][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:35:19,255][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:35:24,304][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:35:29,325][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:35:34,354][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:35:39,359][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:35:44,371][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:35:49,374][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:35:54,427][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:35:59,429][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:36:04,436][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:36:09,456][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:36:14,470][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:36:19,486][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:36:24,488][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:36:29,494][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:36:34,514][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:36:39,517][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:36:44,523][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:36:49,539][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:36:54,547][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:36:59,553][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:37:04,562][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:37:09,572][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:37:14,588][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:37:19,599][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:37:24,658][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:37:29,665][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:37:34,698][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:37:39,799][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:37:44,825][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:37:49,842][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:37:54,848][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:37:59,857][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:38:04,876][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:38:09,905][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:38:14,925][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:38:19,934][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:38:22,936][ INFO][PID:2171193] Allocated host ResallocHost, ticket_id=2422587, hostname=2620:52:6:1161:dead:beef:cafe:c109, name=vmhost_x86_02_prod_07132621_20260420_133450, requested_tags=['arch_x86_64', 'copr_builder'] [2026-04-20 13:38:22,937][ INFO][PID:2171193] Allocating ssh connection to builder [2026-04-20 13:38:22,941][ INFO][PID:2171193] Checking that builder machine is OK [2026-04-20 13:38:23,469][ ERROR][PID:2171193] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c109' (ED25519) to the list of known hosts. Load key "/home/copr/.ssh/id_rsa": Permission denied Permission denied, please try again. Permission denied, please try again. mockbuilder@2620:52:6:1161:dead:beef:cafe:c109: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password). [2026-04-20 13:38:33,553][ ERROR][PID:2171193] Re-try request for task on 'ResallocHost, ticket_id=2422587, hostname=2620:52:6:1161:dead:beef:cafe:c109, name=vmhost_x86_02_prod_07132621_20260420_133450, requested_tags=['arch_x86_64', 'copr_builder']': Unable to finish after 1 SSH attempts [2026-04-20 13:38:33,590][ INFO][PID:2171193] VM Release request [2026-04-20 13:38:33,665][ INFO][PID:2171193] Retry #2 (on other host) [2026-04-20 13:38:33,667][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:38:33,671][ INFO][PID:2171193] VM allocation process starts [2026-04-20 13:38:33,753][ INFO][PID:2171193] Trying to allocate VM: ResallocHost, ticket_id=2422642, requested_tags=['arch_x86_64', 'copr_builder'] [2026-04-20 13:38:38,759][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:38:43,767][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:38:48,777][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:38:53,798][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:38:58,807][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:39:03,817][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:39:08,827][ INFO][PID:2171193] Checking for cancel request [2026-04-20 13:39:12,027][ INFO][PID:2171193] Allocated host ResallocHost, ticket_id=2422642, hostname=98.92.190.76, name=aws_x86_64_normal_prod_07132729_20260420_133639, requested_tags=['arch_x86_64', 'copr_builder'] [2026-04-20 13:39:12,028][ INFO][PID:2171193] Allocating ssh connection to builder [2026-04-20 13:39:12,030][ INFO][PID:2171193] Checking that builder machine is OK [2026-04-20 13:39:12,304][ ERROR][PID:2171193] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: Warning: Permanently added '98.92.190.76' (ED25519) to the list of known hosts. Load key "/home/copr/.ssh/id_rsa": Permission denied Permission denied, please try again. Permission denied, please try again. mockbuilder@98.92.190.76: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password). [2026-04-20 13:39:22,362][ ERROR][PID:2171193] Re-try request for task on 'ResallocHost, ticket_id=2422642, hostname=98.92.190.76, name=aws_x86_64_normal_prod_07132729_20260420_133639, requested_tags=['arch_x86_64', 'copr_builder']': Unable to finish after 1 SSH attempts [2026-04-20 13:39:22,430][ INFO][PID:2171193] VM Release request [2026-04-20 13:39:22,480][ ERROR][PID:2171193] Backend process error: Three host tried without success: {'98.92.190.76', '2620:52:6:1161:dead:beef:cafe:c141', '2620:52:6:1161:dead:beef:cafe:c109'} [2026-04-20 13:39:22,557][WARNING][PID:2171193] Switching not-finished job state to 'failed' [2026-04-20 13:39:22,562][ INFO][PID:2171193] Worker failed build, took 357.7927596569061 [2026-04-20 13:39:22,563][ INFO][PID:2171193] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": null, "enable_net": true, "project_owner": "rhcontainerbot", "project_name": "bootc", "project_dirname": "bootc", "submitter": "packit", "ended_on": 1776692362.5578403, "started_on": 1776692004.7650807, "submitted_on": null, "status": 0, "chroot": "srpm-builds", "arch": "x86_64", "buildroot_pkgs": null, "task_id": "10348620", "build_id": 10348620, "package_name": null, "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 9, "source_json": "{\"script\": \"\\n#!/bin/sh\\n\\ngit config --global user.email \\\"hello@packit.dev\\\"\\ngit config --global user.name \\\"Packit\\\"\\nresultdir=$PWD\\npackit -d prepare-sources --result-dir \\\"$resultdir\\\" --ref main --job-config-index 1 --merged-ref '8dc77c9aed9abab12bb141415c030cbaa1f1279a' --no-create-symlinks https://github.com/bootc-dev/bootc\\n\\n\", \"chroot\": \"fedora-43-x86_64\", \"builddeps\": \"packit cargo git zstd libzstd-devel ostree-devel openssl-devel\", \"resultdir\": null, \"repos\": \"copr://packit/packit-stable\"}", "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": null, "sandbox": "rhcontainerbot/bootc--packit", "results": null, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/packit/packit-stable/fedora-43-x86_64/", "id": "copr_packit_packit_stable", "name": "Additional repo copr_packit_packit_stable" } ], "background": false, "distributions_in_build": [ "centos-stream-10", "centos-stream-9", "fedora-42", "fedora-43", "fedora-44", "fedora-eln", "fedora-rawhide", "opensuse-tumbleweed", "rhel-10", "rhel-9" ], "distributions_in_project": [ "centos-stream-10", "centos-stream-9", "fedora-42", "fedora-43", "fedora-44", "fedora-eln", "fedora-rawhide", "opensuse-tumbleweed", "rhel-10", "rhel-9" ], "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/bootc", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc", "result_dir": "10348620", "built_packages": "", "tags": [ "arch_x86_64" ], "id": 10348620, "mockchain_macros": { "copr_username": "rhcontainerbot", "copr_projectname": "bootc", "vendor": "Fedora Project COPR (rhcontainerbot/bootc)" } } ] } [2026-04-20 13:39:22,689][ INFO][PID:2171193] Sending fedora-messaging bus message in build.end [2026-04-20 13:39:23,862][ INFO][PID:2171193] Compressing /var/lib/copr/public_html/results/rhcontainerbot/bootc/srpm-builds/10348620/builder-live.log by gzip [2026-04-20 13:39:23,870][ INFO][PID:2171193] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/srpm-builds/10348620/builder-live.log' as PID 2252265 [2026-04-20 13:39:23,881][ INFO][PID:2171193] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/srpm-builds/10348620/builder-live.log) stdout: stderr: gzip: /var/lib/copr/public_html/results/rhcontainerbot/bootc/srpm-builds/10348620/builder-live.log: No such file or directory [2026-04-20 13:39:23,886][ ERROR][PID:2171193] Unable to compress file /var/lib/copr/public_html/results/rhcontainerbot/bootc/srpm-builds/10348620/builder-live.log [2026-04-20 13:39:23,976][ INFO][PID:2171193] Compressing /var/lib/copr/public_html/results/rhcontainerbot/bootc/srpm-builds/10348620/backend.log by gzip [2026-04-20 13:39:23,984][ INFO][PID:2171193] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/srpm-builds/10348620/backend.log' as PID 2252291