[2026-03-31 10:59:40,308][  INFO][PID:2663464] Marking build as starting
[2026-03-31 10:59:40,389][  INFO][PID:2663464] Checking that /var/lib/copr/public_html/results/rhcontainerbot/bootc/centos-stream-9-s390x/repodata/repomd.xml exists
[2026-03-31 10:59:40,598][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 10:59:40,599][  INFO][PID:2663464] VM allocation process starts
[2026-03-31 10:59:40,609][  INFO][PID:2663464] Trying to allocate VM: ResallocHost, ticket_id=2110921, requested_tags=['copr_builder', 'arch_s390x']
[2026-03-31 10:59:43,620][  INFO][PID:2663464] Allocated host ResallocHost, ticket_id=2110921, hostname=3.236.142.161, name=aws_x86_64_spot_prod_04408642_20260331_105740, requested_tags=['copr_builder', 'arch_s390x']
[2026-03-31 10:59:43,621][  INFO][PID:2663464] Allocating ssh connection to builder
[2026-03-31 10:59:43,621][  INFO][PID:2663464] Checking that builder machine is OK
[2026-03-31 10:59:43,985][  INFO][PID:2663464] Running remote command: copr-builder-ready centos-stream-9-s390x
[2026-03-31 10:59:44,200][  INFO][PID:2663464] Red Hat subscription not needed for centos-stream-9-s390x
Builder is ready to be used

[2026-03-31 10:59:44,200][  INFO][PID:2663464] Filling build.info file with builder info
[2026-03-31 10:59:44,201][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 10:59:44,202][  INFO][PID:2663464] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "rhcontainerbot",
            "project_name": "bootc",
            "project_dirname": "bootc",
            "submitter": "packit",
            "ended_on": null,
            "started_on": 1774954784.2024193,
            "submitted_on": null,
            "status": 3,
            "chroot": "centos-stream-9-s390x",
            "arch": "s390x",
            "buildroot_pkgs": [],
            "task_id": "10274922-centos-stream-9-s390x",
            "build_id": 10274922,
            "package_name": "bootc",
            "package_version": "202603301748.g7d0cfe64fb-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rhcontainerbot/bootc/bootc",
            "git_hash": "cb29d0a39d2bf7f59c45628515c950d1ce360431",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "rhcontainerbot/bootc--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc/centos-stream-9-s390x/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_s390x"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/bootc",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc",
            "result_dir": "10274922-bootc",
            "built_packages": "",
            "id": 10274922,
            "mockchain_macros": {
                "copr_username": "rhcontainerbot",
                "copr_projectname": "bootc",
                "vendor": "Fedora Project COPR (rhcontainerbot/bootc)"
            }
        }
    ]
}
[2026-03-31 10:59:44,280][  INFO][PID:2663464] Sending fedora-messaging bus message in build.start
[2026-03-31 10:59:45,109][  INFO][PID:2663464] Sending fedora-messaging bus message in chroot.start
[2026-03-31 10:59:45,171][  INFO][PID:2663464] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10274922-centos-stream-9-s390x --chroot centos-stream-9-s390x --detached
[2026-03-31 10:59:45,524][  INFO][PID:2663464] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '3.236.142.161' (ED25519) to the list of known hosts.


[2026-03-31 10:59:45,525][  INFO][PID:2663464] Downloading the builder-live.log file, attempt 1
[2026-03-31 10:59:45,527][  INFO][PID:2663464] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.236.142.161 copr-rpmbuild-log
[2026-03-31 10:59:50,530][  INFO][PID:2663464] Periodic builder liveness probe: alive
[2026-03-31 10:59:50,531][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 10:59:55,531][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:00:00,533][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:00:05,535][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:00:10,536][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:00:15,537][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:00:20,560][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:00:25,561][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:00:30,562][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:00:35,563][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:00:40,564][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:00:45,565][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:00:50,566][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:00:55,567][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:01:00,568][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:01:05,569][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:01:10,570][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:01:15,571][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:01:20,572][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:01:25,573][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:01:30,603][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:01:35,604][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:01:40,605][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:01:45,606][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:01:50,607][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:01:55,608][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:02:00,609][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:02:05,610][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:02:10,611][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:02:15,612][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:02:20,613][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:02:25,614][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:02:30,615][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:02:35,616][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:02:40,617][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:02:45,618][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:02:50,619][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:02:55,620][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:03:00,621][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:03:05,622][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:03:10,623][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:03:15,624][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:03:20,625][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:03:25,626][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:03:30,627][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:03:35,628][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:03:40,629][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:03:45,630][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:03:50,631][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:03:55,632][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:04:00,633][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:04:05,634][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:04:10,635][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:04:15,636][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:04:20,637][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:04:25,638][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:04:30,639][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:04:35,640][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:04:40,641][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:04:45,642][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:04:50,643][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:04:55,644][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:05:00,645][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:05:05,646][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:05:10,647][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:05:15,648][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:05:20,649][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:05:25,650][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:05:30,651][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:05:35,659][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:05:40,660][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:05:45,661][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:05:50,664][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:05:55,665][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:06:00,668][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:06:05,669][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:06:10,674][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:06:15,675][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:06:20,676][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:06:25,677][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:06:30,679][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:06:35,680][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:06:40,681][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:06:45,682][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:06:50,683][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:06:55,689][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:07:00,691][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:07:05,695][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:07:10,696][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:07:15,697][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:07:20,700][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:07:25,704][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:07:30,705][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:07:35,707][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:07:40,709][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:07:45,713][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:07:50,714][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:07:55,715][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:08:00,718][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:08:05,719][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:08:10,721][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:08:15,724][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:08:20,728][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:08:25,732][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:08:30,734][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:08:35,735][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:08:40,738][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:08:45,739][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:08:50,741][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:08:55,743][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:09:00,744][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:09:05,745][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:09:10,749][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:09:15,757][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:09:20,759][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:09:25,763][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:09:30,765][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:09:35,774][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:09:40,787][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:09:45,789][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:09:50,824][  INFO][PID:2663464] Periodic builder liveness probe: alive
[2026-03-31 11:09:50,825][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:09:55,831][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:10:00,836][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:10:05,840][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:10:10,844][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:10:15,847][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:10:20,850][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:10:25,860][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:10:30,866][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:10:35,867][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:10:40,870][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:10:45,880][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:10:50,881][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:10:55,886][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:11:00,892][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:11:05,899][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:11:10,901][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:11:15,903][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:11:20,912][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:11:25,914][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:11:30,915][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:11:35,918][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:11:40,922][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:11:45,924][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:11:50,925][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:11:55,927][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:12:00,929][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:12:05,931][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:12:10,932][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:12:15,933][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:12:20,934][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:12:25,935][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:12:30,936][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:12:35,937][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:12:40,938][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:12:45,939][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:12:50,940][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:12:55,943][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:13:00,945][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:13:05,951][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:13:10,953][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:13:15,955][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:13:20,957][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:13:25,958][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:13:30,959][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:13:35,960][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:13:40,962][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:13:45,965][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:13:50,966][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:13:55,971][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:14:00,975][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:14:05,978][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:14:10,979][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:14:15,989][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:14:20,991][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:14:25,992][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:14:30,995][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:14:36,003][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:14:41,011][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:14:46,012][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:14:51,014][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:14:56,016][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:15:01,018][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:15:06,020][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:15:11,021][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:15:16,023][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:15:21,027][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:15:26,028][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:15:31,031][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:15:36,034][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:15:41,036][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:15:46,037][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:15:51,040][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:15:56,041][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:16:01,042][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:16:06,048][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:16:11,049][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:16:16,050][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:16:21,051][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:16:26,052][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:16:31,053][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:16:36,054][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:16:41,056][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:16:46,060][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:16:51,061][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:16:56,062][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:17:01,064][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:17:06,065][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:17:11,067][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:17:16,068][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:17:21,069][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:17:26,070][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:17:31,071][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:17:36,072][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:17:41,073][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:17:46,074][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:17:51,080][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:17:56,081][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:18:01,082][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:18:06,084][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:18:11,085][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:18:16,086][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:18:21,087][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:18:26,088][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:18:31,089][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:18:36,090][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:18:41,092][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:18:46,093][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:18:51,094][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:18:56,095][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:19:01,096][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:19:06,097][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:19:11,099][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:19:16,100][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:19:21,101][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:19:26,102][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:19:31,103][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:19:36,104][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:19:41,105][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:19:46,107][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:19:51,117][  INFO][PID:2663464] Periodic builder liveness probe: alive
[2026-03-31 11:19:51,118][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:19:56,121][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:20:01,122][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:20:06,123][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:20:11,125][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:20:16,126][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:20:21,128][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:20:26,130][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:20:31,130][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:20:33,621][ ERROR][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-03-31 11:20:36,131][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:20:41,134][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:20:43,623][WARNING][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-03-31 11:20:46,137][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:20:51,139][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:20:53,624][  INFO][PID:2663464] Downloading the builder-live.log file, attempt 2
[2026-03-31 11:20:53,626][  INFO][PID:2663464] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.236.142.161 copr-rpmbuild-log
[2026-03-31 11:20:56,140][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:21:01,141][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:21:06,142][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:21:11,144][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:21:16,145][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:21:21,146][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:21:26,147][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:21:31,148][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:21:36,154][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:21:41,155][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:21:46,156][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:21:51,157][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:21:53,693][ ERROR][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-03-31 11:21:56,158][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:22:01,159][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:22:03,694][WARNING][PID:2663464] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-03-31 11:22:06,160][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:22:11,161][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:22:13,695][  INFO][PID:2663464] Downloading the builder-live.log file, attempt 3
[2026-03-31 11:22:13,696][  INFO][PID:2663464] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.236.142.161 copr-rpmbuild-log
[2026-03-31 11:22:16,162][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:22:21,163][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:22:26,164][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:22:31,165][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:22:36,166][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:22:41,167][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:22:46,168][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:22:51,169][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:22:56,170][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:23:01,171][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:23:06,172][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:23:11,173][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:23:13,765][ ERROR][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-03-31 11:23:16,174][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:23:21,175][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:23:23,767][WARNING][PID:2663464] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-03-31 11:23:26,176][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:23:31,177][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:23:33,768][  INFO][PID:2663464] Downloading the builder-live.log file, attempt 4
[2026-03-31 11:23:33,769][  INFO][PID:2663464] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.236.142.161 copr-rpmbuild-log
[2026-03-31 11:23:36,178][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:23:41,179][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:23:46,180][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:23:51,181][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:23:56,183][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:24:01,184][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:24:06,185][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:24:11,186][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:24:16,187][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:24:21,188][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:24:26,189][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:24:31,190][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:24:33,837][ ERROR][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-03-31 11:24:36,191][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:24:41,192][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:24:43,870][ ERROR][PID:2663464] Giving up for unstable SSH, failures: 1774956043.6233487, 1774956123.6945536, 1774956203.7679198, 1774956283.870397
[2026-03-31 11:24:43,871][ ERROR][PID:2663464] Re-try request for task on 'ResallocHost, ticket_id=2110921, hostname=3.236.142.161, name=aws_x86_64_spot_prod_04408642_20260331_105740, requested_tags=['copr_builder', 'arch_s390x']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2026-03-31 11:24:43,872][  INFO][PID:2663464] Releasing VM back to pool
[2026-03-31 11:24:43,884][  INFO][PID:2663464] Retry #1 (on other host)
[2026-03-31 11:24:43,884][  INFO][PID:2663464] Checking that /var/lib/copr/public_html/results/rhcontainerbot/bootc/centos-stream-9-s390x/repodata/repomd.xml exists
[2026-03-31 11:24:43,885][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:24:43,886][  INFO][PID:2663464] VM allocation process starts
[2026-03-31 11:24:43,900][  INFO][PID:2663464] Trying to allocate VM: ResallocHost, ticket_id=2111441, requested_tags=['copr_builder', 'arch_s390x']
[2026-03-31 11:24:46,916][  INFO][PID:2663464] Allocated host ResallocHost, ticket_id=2111441, hostname=44.223.86.202, name=aws_x86_64_spot_prod_04409248_20260331_112258, requested_tags=['copr_builder', 'arch_s390x']
[2026-03-31 11:24:46,917][  INFO][PID:2663464] Allocating ssh connection to builder
[2026-03-31 11:24:46,917][  INFO][PID:2663464] Checking that builder machine is OK
[2026-03-31 11:24:47,253][  INFO][PID:2663464] Running remote command: copr-builder-ready centos-stream-9-s390x
[2026-03-31 11:24:47,489][  INFO][PID:2663464] Red Hat subscription not needed for centos-stream-9-s390x
Builder is ready to be used

[2026-03-31 11:24:47,489][  INFO][PID:2663464] Filling build.info file with builder info
[2026-03-31 11:24:47,548][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:24:47,549][  INFO][PID:2663464] Not re-notifying FE and msg buses for the new host.
[2026-03-31 11:24:47,549][  INFO][PID:2663464] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10274922-centos-stream-9-s390x --chroot centos-stream-9-s390x --detached
[2026-03-31 11:24:47,884][  INFO][PID:2663464] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '44.223.86.202' (ED25519) to the list of known hosts.


[2026-03-31 11:24:47,885][  INFO][PID:2663464] Downloading the builder-live.log file, attempt 1
[2026-03-31 11:24:47,886][  INFO][PID:2663464] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.223.86.202 copr-rpmbuild-log
[2026-03-31 11:24:52,886][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:24:57,886][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:25:02,887][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:25:07,888][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:25:12,895][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:25:17,896][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:25:22,897][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:25:27,898][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:25:32,899][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:25:37,903][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:25:42,908][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:25:47,909][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:25:52,910][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:25:57,911][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:26:02,912][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:26:07,913][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:26:12,914][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:26:17,915][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:26:22,916][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:26:27,917][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:26:32,920][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:26:37,921][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:26:42,922][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:26:47,923][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:26:52,924][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:26:57,925][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:27:02,926][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:27:07,927][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:27:12,928][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:27:17,929][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:27:22,930][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:27:27,931][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:27:32,932][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:27:37,933][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:27:42,934][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:27:47,937][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:27:52,938][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:27:57,939][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:28:02,940][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:28:07,941][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:28:12,942][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:28:17,943][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:28:22,944][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:28:27,945][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:28:32,946][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:28:37,947][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:28:42,948][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:28:47,949][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:28:52,950][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:28:57,951][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:29:02,952][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:29:07,953][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:29:12,954][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:29:17,955][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:29:22,956][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:29:27,957][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:29:32,998][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:29:37,999][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:29:43,071][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:29:48,072][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:29:53,078][  INFO][PID:2663464] Periodic builder liveness probe: alive
[2026-03-31 11:29:53,079][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:29:58,080][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:30:03,081][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:30:08,082][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:30:13,085][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:30:18,086][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:30:23,087][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:30:28,088][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:30:33,089][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:30:38,091][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:30:43,093][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:30:48,094][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:30:53,095][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:30:58,096][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:31:03,097][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:31:08,098][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:31:13,099][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:31:18,100][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:31:23,101][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:31:28,102][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:31:33,103][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:31:38,104][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:31:43,105][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:31:48,106][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:31:53,106][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:31:58,108][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:32:03,110][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:32:08,112][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:32:13,113][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:32:18,114][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:32:23,115][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:32:28,116][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:32:33,117][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:32:38,118][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:32:43,120][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:32:48,121][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:32:53,122][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:32:58,123][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:33:03,123][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:33:08,125][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:33:13,126][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:33:18,127][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:33:23,128][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:33:28,129][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:33:33,129][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:33:38,130][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:33:43,131][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:33:48,132][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:33:53,139][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:33:58,140][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:34:03,142][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:34:08,143][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:34:13,144][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:34:18,146][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:34:23,147][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:34:28,148][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:34:33,149][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:34:37,126][ ERROR][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-03-31 11:34:38,150][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:34:43,151][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:34:47,126][WARNING][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-03-31 11:34:48,152][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:34:53,153][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:34:57,127][  INFO][PID:2663464] Downloading the builder-live.log file, attempt 2
[2026-03-31 11:34:57,235][  INFO][PID:2663464] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.223.86.202 copr-rpmbuild-log
[2026-03-31 11:34:58,154][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:35:03,156][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:35:08,157][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:35:13,158][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:35:18,159][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:35:23,160][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:35:28,161][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:35:33,162][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:35:38,163][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:35:43,164][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:35:48,165][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:35:53,166][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:35:57,304][ ERROR][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-03-31 11:35:58,167][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:36:03,168][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:36:07,305][WARNING][PID:2663464] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-03-31 11:36:08,169][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:36:13,170][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:36:17,306][  INFO][PID:2663464] Downloading the builder-live.log file, attempt 3
[2026-03-31 11:36:17,307][  INFO][PID:2663464] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.223.86.202 copr-rpmbuild-log
[2026-03-31 11:36:18,172][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:36:23,173][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:36:28,174][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:36:33,175][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:36:38,176][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:36:43,177][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:36:48,184][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:36:53,185][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:36:58,191][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:37:03,192][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:37:08,199][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:37:13,200][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:37:17,446][ ERROR][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-03-31 11:37:18,201][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:37:23,202][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:37:27,454][WARNING][PID:2663464] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-03-31 11:37:28,203][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:37:33,204][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:37:37,455][  INFO][PID:2663464] Downloading the builder-live.log file, attempt 4
[2026-03-31 11:37:37,566][  INFO][PID:2663464] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.223.86.202 copr-rpmbuild-log
[2026-03-31 11:37:38,205][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:37:43,224][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:37:48,225][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:37:53,226][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:37:58,233][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:38:03,234][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:38:08,234][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:38:13,236][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:38:18,242][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:38:23,243][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:38:28,244][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:38:33,245][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:38:37,632][ ERROR][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-03-31 11:38:38,247][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:38:43,248][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:38:47,642][ ERROR][PID:2663464] Giving up for unstable SSH, failures: 1774956887.126914, 1774956967.3056552, 1774957047.4543283, 1774957127.642054
[2026-03-31 11:38:47,643][ ERROR][PID:2663464] Re-try request for task on 'ResallocHost, ticket_id=2111441, hostname=44.223.86.202, name=aws_x86_64_spot_prod_04409248_20260331_112258, requested_tags=['copr_builder', 'arch_s390x']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2026-03-31 11:38:47,645][  INFO][PID:2663464] Releasing VM back to pool
[2026-03-31 11:38:47,688][  INFO][PID:2663464] Retry #2 (on other host)
[2026-03-31 11:38:47,692][  INFO][PID:2663464] Checking that /var/lib/copr/public_html/results/rhcontainerbot/bootc/centos-stream-9-s390x/repodata/repomd.xml exists
[2026-03-31 11:38:47,735][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:38:47,738][  INFO][PID:2663464] VM allocation process starts
[2026-03-31 11:38:47,808][  INFO][PID:2663464] Trying to allocate VM: ResallocHost, ticket_id=2111664, requested_tags=['copr_builder', 'arch_s390x']
[2026-03-31 11:38:50,870][  INFO][PID:2663464] Allocated host ResallocHost, ticket_id=2111664, hostname=100.54.105.75, name=aws_x86_64_spot_prod_04409538_20260331_113428, requested_tags=['copr_builder', 'arch_s390x']
[2026-03-31 11:38:50,870][  INFO][PID:2663464] Allocating ssh connection to builder
[2026-03-31 11:38:50,871][  INFO][PID:2663464] Checking that builder machine is OK
[2026-03-31 11:38:51,214][  INFO][PID:2663464] Running remote command: copr-builder-ready centos-stream-9-s390x
[2026-03-31 11:38:51,422][  INFO][PID:2663464] Red Hat subscription not needed for centos-stream-9-s390x
Builder is ready to be used

[2026-03-31 11:38:51,423][  INFO][PID:2663464] Filling build.info file with builder info
[2026-03-31 11:38:51,442][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:38:51,443][  INFO][PID:2663464] Not re-notifying FE and msg buses for the new host.
[2026-03-31 11:38:51,444][  INFO][PID:2663464] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10274922-centos-stream-9-s390x --chroot centos-stream-9-s390x --detached
[2026-03-31 11:38:51,781][  INFO][PID:2663464] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '100.54.105.75' (ED25519) to the list of known hosts.


[2026-03-31 11:38:51,783][  INFO][PID:2663464] Downloading the builder-live.log file, attempt 1
[2026-03-31 11:38:51,784][  INFO][PID:2663464] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@100.54.105.75 copr-rpmbuild-log
[2026-03-31 11:38:56,784][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:39:01,786][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:39:06,789][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:39:11,790][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:39:16,792][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:39:21,794][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:39:26,795][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:39:31,797][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:39:36,799][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:39:41,801][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:39:46,802][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:39:51,803][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:39:56,811][  INFO][PID:2663464] Periodic builder liveness probe: alive
[2026-03-31 11:39:56,812][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:40:01,813][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:40:06,814][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:40:11,815][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:40:16,816][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:40:21,817][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:40:26,818][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:40:31,819][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:40:36,820][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:40:41,821][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:40:46,822][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:40:51,840][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:40:56,841][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:41:01,842][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:41:06,843][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:41:11,844][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:41:16,845][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:41:21,846][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:41:26,898][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:41:31,899][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:41:36,900][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:41:41,901][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:41:46,902][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:41:51,903][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:41:56,905][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:42:01,906][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:42:06,907][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:42:11,908][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:42:16,909][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:42:21,910][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:42:26,911][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:42:31,914][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:42:36,923][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:42:41,924][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:42:46,925][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:42:51,926][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:42:56,927][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:43:01,928][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:43:06,929][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:43:11,930][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:43:16,931][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:43:21,932][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:43:26,933][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:43:31,934][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:43:36,935][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:43:41,936][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:43:46,944][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:43:51,945][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:43:56,946][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:44:01,947][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:44:06,948][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:44:11,949][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:44:16,950][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:44:21,952][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:44:26,953][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:44:31,954][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:44:36,955][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:44:41,956][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:44:46,956][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:44:51,957][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:44:56,958][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:45:01,959][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:45:06,960][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:45:11,961][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:45:16,962][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:45:21,963][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:45:26,965][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:45:31,965][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:45:36,966][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:45:41,967][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:45:46,968][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:45:51,970][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:45:56,971][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:46:01,971][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:46:06,972][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:46:11,973][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:46:16,974][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:46:21,975][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:46:26,976][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:46:31,977][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:46:36,978][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:46:41,979][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:46:46,980][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:46:51,981][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:46:56,982][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:47:01,984][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:47:06,985][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:47:11,986][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:47:16,987][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:47:21,988][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:47:26,989][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:47:31,990][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:47:36,991][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:47:41,992][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:47:46,993][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:47:51,994][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:47:56,995][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:48:01,996][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:48:06,997][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:48:11,998][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:48:16,999][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:48:22,000][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:48:27,001][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:48:32,002][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:48:37,003][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:48:42,004][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:48:47,005][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:48:52,006][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:48:57,006][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:49:02,007][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:49:07,008][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:49:12,009][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:49:17,010][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:49:22,011][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:49:27,012][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:49:32,013][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:49:37,014][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:49:42,015][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:49:47,016][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:49:52,017][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:49:57,023][  INFO][PID:2663464] Periodic builder liveness probe: alive
[2026-03-31 11:49:57,023][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:50:02,024][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:50:07,025][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:50:12,026][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:50:17,027][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:50:22,028][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:50:27,029][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:50:32,035][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:50:37,036][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:50:42,037][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:50:47,038][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:50:52,039][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:50:57,040][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:51:02,041][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:51:07,042][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:51:12,043][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:51:17,044][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:51:22,045][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:51:27,046][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:51:32,047][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:51:37,048][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:51:42,070][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:51:47,071][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:51:52,072][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:51:57,073][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:52:02,074][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:52:07,075][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:52:12,076][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:52:17,077][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:52:22,078][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:52:27,079][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:52:32,080][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:52:37,081][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:52:42,082][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:52:47,083][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:52:52,084][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:52:57,085][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:53:02,086][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:53:07,087][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:53:12,087][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:53:17,088][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:53:22,089][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:53:27,090][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:53:32,091][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:53:37,092][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:53:42,093][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:53:47,094][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:53:52,095][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:53:57,096][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:54:02,097][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:54:07,098][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:54:12,099][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:54:17,100][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:54:22,101][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:54:27,102][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:54:32,103][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:54:37,104][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:54:42,105][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:54:47,106][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:54:52,107][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:54:57,108][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:55:02,109][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:55:07,110][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:55:12,111][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:55:17,112][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:55:22,112][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:55:27,113][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:55:32,114][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:55:37,124][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:55:42,125][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:55:47,181][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:55:52,182][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:55:57,207][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:56:02,208][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:56:07,209][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:56:12,211][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:56:17,212][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:56:22,213][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:56:27,214][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:56:32,215][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:56:37,216][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:56:42,217][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:56:47,218][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:56:52,219][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:56:57,220][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:57:02,221][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:57:07,221][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:57:12,223][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:57:17,224][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:57:22,252][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:57:27,253][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:57:32,254][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:57:37,256][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:57:42,257][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:57:47,260][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:57:52,262][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:57:57,274][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:58:02,280][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:58:07,281][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:58:12,283][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:58:17,284][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:58:22,285][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:58:27,289][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:58:32,290][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:58:37,291][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:58:42,292][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:58:47,294][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:58:52,295][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:58:57,296][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:59:02,297][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:59:07,298][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:59:12,299][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:59:17,300][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:59:22,301][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:59:27,307][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:59:32,307][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:59:37,308][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:59:42,309][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:59:47,310][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:59:52,311][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 11:59:57,326][  INFO][PID:2663464] Periodic builder liveness probe: alive
[2026-03-31 11:59:57,326][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:00:02,327][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:00:07,328][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:00:12,329][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:00:17,330][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:00:22,331][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:00:27,333][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:00:32,334][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:00:37,339][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:00:42,340][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:00:47,341][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:00:52,343][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:00:57,344][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:01:02,345][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:01:07,346][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:01:12,347][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:01:17,351][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:01:22,354][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:01:27,355][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:01:32,357][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:01:37,358][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:01:42,360][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:01:47,367][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:01:52,369][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:01:57,370][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:02:02,372][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:02:07,376][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:02:07,617][ ERROR][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-03-31 12:02:12,377][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:02:17,379][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:02:17,619][WARNING][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-03-31 12:02:22,386][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:02:27,388][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:02:27,620][  INFO][PID:2663464] Downloading the builder-live.log file, attempt 2
[2026-03-31 12:02:27,707][  INFO][PID:2663464] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@100.54.105.75 copr-rpmbuild-log
[2026-03-31 12:02:32,391][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:02:37,392][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:02:42,393][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:02:47,395][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:02:52,398][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:02:57,400][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:03:02,402][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:03:07,403][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:03:12,405][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:03:17,406][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:03:22,407][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:03:27,411][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:03:27,778][ ERROR][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-03-31 12:03:32,412][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:03:37,413][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:03:37,779][WARNING][PID:2663464] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-03-31 12:03:42,414][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:03:47,415][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:03:47,779][  INFO][PID:2663464] Downloading the builder-live.log file, attempt 3
[2026-03-31 12:03:47,781][  INFO][PID:2663464] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@100.54.105.75 copr-rpmbuild-log
[2026-03-31 12:03:52,416][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:03:57,417][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:04:02,418][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:04:07,420][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:04:12,421][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:04:17,424][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:04:22,504][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:04:27,575][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:04:32,577][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:04:37,578][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:04:42,579][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:04:47,580][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:04:47,847][ ERROR][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-03-31 12:04:52,582][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:04:57,630][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:04:57,850][WARNING][PID:2663464] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-03-31 12:05:02,632][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:05:07,633][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:05:07,851][  INFO][PID:2663464] Downloading the builder-live.log file, attempt 4
[2026-03-31 12:05:07,853][  INFO][PID:2663464] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@100.54.105.75 copr-rpmbuild-log
[2026-03-31 12:05:12,635][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:05:17,637][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:05:22,638][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:05:27,639][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:05:32,640][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:05:37,642][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:05:42,644][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:05:47,645][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:05:52,646][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:05:57,648][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:06:02,649][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:06:07,650][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:06:07,921][ ERROR][PID:2663464] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-03-31 12:06:12,651][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:06:17,652][  INFO][PID:2663464] Checking for cancel request
[2026-03-31 12:06:17,922][ ERROR][PID:2663464] Giving up for unstable SSH, failures: 1774958537.6196783, 1774958617.7791874, 1774958697.8507006, 1774958777.9227052
[2026-03-31 12:06:17,923][ ERROR][PID:2663464] Re-try request for task on 'ResallocHost, ticket_id=2111664, hostname=100.54.105.75, name=aws_x86_64_spot_prod_04409538_20260331_113428, requested_tags=['copr_builder', 'arch_s390x']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2026-03-31 12:06:17,924][  INFO][PID:2663464] Releasing VM back to pool
[2026-03-31 12:06:17,935][ ERROR][PID:2663464] Backend process error: Three host tried without success: {'100.54.105.75', '44.223.86.202', '3.236.142.161'}
[2026-03-31 12:06:17,936][WARNING][PID:2663464] Switching not-finished job state to 'failed'
[2026-03-31 12:06:17,936][  INFO][PID:2663464] Worker failed build, took 1646.4924376010895
[2026-03-31 12:06:17,937][  INFO][PID:2663464] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "rhcontainerbot",
            "project_name": "bootc",
            "project_dirname": "bootc",
            "submitter": "packit",
            "ended_on": 1774958777.9361818,
            "started_on": 1774957131.4437442,
            "submitted_on": null,
            "status": 0,
            "chroot": "centos-stream-9-s390x",
            "arch": "s390x",
            "buildroot_pkgs": [],
            "task_id": "10274922-centos-stream-9-s390x",
            "build_id": 10274922,
            "package_name": "bootc",
            "package_version": "202603301748.g7d0cfe64fb-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rhcontainerbot/bootc/bootc",
            "git_hash": "cb29d0a39d2bf7f59c45628515c950d1ce360431",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "rhcontainerbot/bootc--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc/centos-stream-9-s390x/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_s390x"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/bootc",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc",
            "result_dir": "10274922-bootc",
            "built_packages": "",
            "id": 10274922,
            "mockchain_macros": {
                "copr_username": "rhcontainerbot",
                "copr_projectname": "bootc",
                "vendor": "Fedora Project COPR (rhcontainerbot/bootc)"
            }
        }
    ]
}
[2026-03-31 12:06:18,010][  INFO][PID:2663464] Sending fedora-messaging bus message in build.end
[2026-03-31 12:06:18,055][  INFO][PID:2663464] Compressing /var/lib/copr/public_html/results/rhcontainerbot/bootc/centos-stream-9-s390x/10274922-bootc/builder-live.log by gzip
[2026-03-31 12:06:18,057][  INFO][PID:2663464] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/centos-stream-9-s390x/10274922-bootc/builder-live.log' as PID 3069117
[2026-03-31 12:06:18,129][  INFO][PID:2663464] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/centos-stream-9-s390x/10274922-bootc/builder-live.log)
[2026-03-31 12:06:18,131][  INFO][PID:2663464] Compressing /var/lib/copr/public_html/results/rhcontainerbot/bootc/centos-stream-9-s390x/10274922-bootc/backend.log by gzip
[2026-03-31 12:06:18,132][  INFO][PID:2663464] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/centos-stream-9-s390x/10274922-bootc/backend.log' as PID 3069129