[2025-08-28 18:17:52,929][  INFO][PID:893487] Marking build as starting
[2025-08-28 18:17:52,978][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:17:52,979][  INFO][PID:893487] VM allocation process starts
[2025-08-28 18:17:52,989][  INFO][PID:893487] Trying to allocate VM: ResallocHost, ticket_id=4804727, requested_tags=['copr_builder', 'arch_x86_64']
[2025-08-28 18:17:56,026][  INFO][PID:893487] Allocated host ResallocHost, ticket_id=4804727, hostname=54.164.245.210, name=aws_x86_64_spot_prod_06483248_20250828_180519, requested_tags=['copr_builder', 'arch_x86_64']
[2025-08-28 18:17:56,026][  INFO][PID:893487] Allocating ssh connection to builder
[2025-08-28 18:17:56,027][  INFO][PID:893487] Checking that builder machine is OK
[2025-08-28 18:17:56,440][  INFO][PID:893487] Running remote command: copr-builder-ready fedora-42-x86_64
[2025-08-28 18:17:56,624][  INFO][PID:893487] Red Hat subscription not needed for fedora-42-x86_64
Builder is ready to be used

[2025-08-28 18:17:56,625][  INFO][PID:893487] Filling build.info file with builder info
[2025-08-28 18:17:56,626][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:17:56,626][  INFO][PID:893487] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "che",
            "project_name": "mesa",
            "project_dirname": "mesa",
            "submitter": "che",
            "ended_on": null,
            "started_on": 1756405076.6267884,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-42-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "9500431-fedora-42-x86_64",
            "build_id": 9500431,
            "package_name": "mesa",
            "package_version": "25.3.0-0.20.gitb1d07ae",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/che/mesa/mesa",
            "git_hash": "4046385edd92096717c615943fa77c8f6d3f548a",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "che/mesa--che",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/che/mesa/fedora-42-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/che/llvm/fedora-42-x86_64/",
                    "id": "copr_che_llvm",
                    "name": "Additional repo copr_che_llvm"
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/che/mesa",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/che/mesa",
            "result_dir": "09500431-mesa",
            "built_packages": "",
            "id": 9500431,
            "mockchain_macros": {
                "copr_username": "che",
                "copr_projectname": "mesa",
                "vendor": "Fedora Project COPR (che/mesa)"
            }
        }
    ]
}
[2025-08-28 18:17:56,659][  INFO][PID:893487] Sending fedora-messaging bus message in build.start
[2025-08-28 18:17:57,323][  INFO][PID:893487] Sending fedora-messaging bus message in chroot.start
[2025-08-28 18:17:57,350][  INFO][PID:893487] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9500431-fedora-42-x86_64 --chroot fedora-42-x86_64 --detached
[2025-08-28 18:17:57,653][  INFO][PID:893487] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '54.164.245.210' (ED25519) to the list of known hosts.


[2025-08-28 18:17:57,654][  INFO][PID:893487] Downloading the builder-live.log file, attempt 1
[2025-08-28 18:17:57,655][  INFO][PID:893487] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.164.245.210 copr-rpmbuild-log
[2025-08-28 18:18:02,659][  INFO][PID:893487] Periodic builder liveness probe: alive
[2025-08-28 18:18:02,660][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:18:07,660][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:18:12,661][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:18:17,671][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:18:22,673][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:18:27,673][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:18:32,674][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:18:37,675][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:18:42,676][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:18:47,677][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:18:52,678][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:18:57,679][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:19:02,681][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:19:07,682][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:19:12,684][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:19:17,689][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:19:22,691][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:19:27,694][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:19:32,701][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:19:37,706][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:19:42,707][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:19:47,708][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:19:52,709][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:19:57,710][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:20:02,712][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:20:07,713][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:20:12,714][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:20:17,715][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:20:22,716][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:20:27,718][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:20:32,719][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:20:37,720][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:20:42,721][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:20:47,723][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:20:52,724][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:20:57,725][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:21:02,727][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:21:07,728][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:21:12,731][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:21:17,732][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:21:22,733][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:21:27,734][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:21:32,735][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:21:37,737][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:21:42,738][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:21:47,739][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:21:52,740][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:21:57,741][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:22:02,742][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:22:07,743][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:22:12,744][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:22:17,747][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:22:22,749][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:22:27,750][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:22:32,751][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:22:37,752][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:22:42,753][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:22:47,754][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:22:52,755][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:22:57,756][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:23:02,757][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:23:07,758][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:23:12,759][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:23:17,760][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:23:22,761][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:23:27,762][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:23:32,763][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:23:37,764][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:23:42,765][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:23:47,766][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:23:52,767][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:23:57,768][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:24:02,769][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:24:07,770][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:24:12,771][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:24:17,772][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:24:22,789][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:24:27,815][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:24:32,835][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:24:37,882][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:24:42,894][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:24:47,957][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:24:53,007][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:24:58,013][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:25:03,029][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:25:08,047][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:25:13,108][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:25:18,175][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:25:23,180][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:25:28,265][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:25:33,323][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:25:38,338][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:25:43,406][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:25:48,443][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:25:53,454][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:25:58,468][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:26:03,630][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:26:08,663][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:26:13,764][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:26:18,811][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:26:23,897][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:26:29,025][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:26:34,106][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:26:39,162][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:26:44,237][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:26:49,310][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:26:54,389][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:26:59,467][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:27:04,515][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:27:09,580][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:27:14,613][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:27:19,674][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:27:24,728][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:27:29,740][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:27:34,743][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:27:39,828][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:27:44,839][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:27:49,867][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:27:54,930][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:27:59,998][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:28:05,068][  INFO][PID:893487] Periodic builder liveness probe: alive
[2025-08-28 18:28:05,068][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:28:10,069][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:28:15,133][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:28:20,193][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:28:25,225][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:28:30,239][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:28:35,325][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:28:40,418][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:28:45,469][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:28:50,514][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:28:55,586][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:29:00,650][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:29:05,699][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:29:10,817][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:29:16,216][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:29:21,244][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:29:26,433][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:29:31,548][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:29:36,628][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:29:41,727][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:29:46,798][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:29:51,837][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:29:56,932][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:30:01,961][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:30:07,016][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:30:12,029][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:30:17,035][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:30:22,088][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:30:27,102][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:30:32,142][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:30:37,179][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:30:42,242][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:30:47,245][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:30:52,322][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:30:57,471][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:31:02,542][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:31:07,598][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:31:12,708][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:31:17,726][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:31:22,751][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:31:27,764][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:31:32,837][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:31:37,896][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:31:42,969][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:31:48,008][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:31:53,011][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:31:58,022][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:32:03,075][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:32:08,129][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:32:13,192][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:32:18,197][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:32:23,289][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:32:28,322][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:32:33,398][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:32:38,496][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:32:43,558][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:32:48,565][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:32:53,601][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:32:58,650][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:33:03,661][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:33:08,748][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:33:13,805][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:33:18,808][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:33:23,913][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:33:29,021][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:33:34,063][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:33:39,172][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:33:44,200][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:33:49,284][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:33:54,348][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:33:59,527][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:34:04,710][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:34:09,875][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:34:14,996][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:34:20,049][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:34:25,164][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:34:30,230][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:34:35,235][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:34:40,242][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:34:45,312][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:34:50,365][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:34:55,421][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:35:00,441][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:35:05,449][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:35:10,496][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:35:15,560][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:35:20,578][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:35:25,650][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:35:30,655][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:35:35,730][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:35:40,778][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:35:45,790][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:35:50,840][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:35:55,860][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:36:00,871][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:36:05,878][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:36:10,911][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:36:15,942][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:36:20,947][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:36:25,949][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:36:30,950][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:36:35,967][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:36:41,014][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:36:46,029][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:36:51,034][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:36:56,066][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:37:01,131][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:37:06,216][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:37:11,233][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:37:16,282][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:37:21,339][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:37:26,363][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:37:31,393][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:37:36,412][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:37:41,448][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:37:46,466][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:37:51,485][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:37:56,536][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:38:01,628][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:38:06,672][  INFO][PID:893487] Periodic builder liveness probe: alive
[2025-08-28 18:38:06,733][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:38:11,737][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:38:16,741][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:38:21,753][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:38:26,760][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:38:31,780][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:38:36,783][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:38:41,795][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:38:46,821][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:38:51,822][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:38:56,850][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:38:58,657][  INFO][PID:893487] Downloading results from builder
[2025-08-28 18:38:58,657][  INFO][PID:893487] rsyncing of mockbuilder@54.164.245.210:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa started
[2025-08-28 18:38:58,658][  INFO][PID:893487] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@54.164.245.210:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa/ &> /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa/build-09500431.rsync.log
[2025-08-28 18:49:07,675][  INFO][PID:893487] rsyncing finished.
[2025-08-28 18:49:07,675][ ERROR][PID:893487] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-08-28 18:49:07,676][ ERROR][PID:893487] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-08-28 18:49:17,676][  INFO][PID:893487] rsyncing of mockbuilder@54.164.245.210:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa started
[2025-08-28 18:49:17,677][  INFO][PID:893487] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@54.164.245.210:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa/ &> /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa/build-09500431.rsync.log
[2025-08-28 18:50:17,753][  INFO][PID:893487] rsyncing finished.
[2025-08-28 18:50:17,754][ ERROR][PID:893487] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-08-28 18:50:17,754][ ERROR][PID:893487] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-08-28 18:50:27,758][  INFO][PID:893487] rsyncing of mockbuilder@54.164.245.210:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa started
[2025-08-28 18:50:27,759][  INFO][PID:893487] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@54.164.245.210:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa/ &> /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa/build-09500431.rsync.log
[2025-08-28 18:51:27,856][  INFO][PID:893487] rsyncing finished.
[2025-08-28 18:51:27,857][ ERROR][PID:893487] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-08-28 18:51:27,857][ ERROR][PID:893487] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-08-28 18:51:37,858][ ERROR][PID:893487] Re-try request for task on 'ResallocHost, ticket_id=4804727, hostname=54.164.245.210, name=aws_x86_64_spot_prod_06483248_20250828_180519, requested_tags=['copr_builder', 'arch_x86_64']': Unable to finish after 3 SSH attempts
[2025-08-28 18:51:37,859][  INFO][PID:893487] Releasing VM back to pool
[2025-08-28 18:51:37,868][  INFO][PID:893487] Retry #1 (on other host)
[2025-08-28 18:51:37,869][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:51:37,869][  INFO][PID:893487] VM allocation process starts
[2025-08-28 18:51:37,914][  INFO][PID:893487] Trying to allocate VM: ResallocHost, ticket_id=4804997, requested_tags=['copr_builder', 'arch_x86_64']
[2025-08-28 18:51:40,992][  INFO][PID:893487] Allocated host ResallocHost, ticket_id=4804997, hostname=54.242.106.163, name=aws_x86_64_spot_prod_06483958_20250828_184821, requested_tags=['copr_builder', 'arch_x86_64']
[2025-08-28 18:51:40,993][  INFO][PID:893487] Allocating ssh connection to builder
[2025-08-28 18:51:40,994][  INFO][PID:893487] Checking that builder machine is OK
[2025-08-28 18:51:41,505][  INFO][PID:893487] Running remote command: copr-builder-ready fedora-42-x86_64
[2025-08-28 18:51:41,703][  INFO][PID:893487] Red Hat subscription not needed for fedora-42-x86_64
Builder is ready to be used

[2025-08-28 18:51:41,704][  INFO][PID:893487] Filling build.info file with builder info
[2025-08-28 18:51:41,706][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:51:41,707][  INFO][PID:893487] Not re-notifying FE and msg buses for the new host.
[2025-08-28 18:51:41,708][  INFO][PID:893487] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9500431-fedora-42-x86_64 --chroot fedora-42-x86_64 --detached
[2025-08-28 18:51:42,016][  INFO][PID:893487] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '54.242.106.163' (ED25519) to the list of known hosts.


[2025-08-28 18:51:42,017][  INFO][PID:893487] Downloading the builder-live.log file, attempt 1
[2025-08-28 18:51:42,083][  INFO][PID:893487] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.242.106.163 copr-rpmbuild-log
[2025-08-28 18:51:47,023][  INFO][PID:893487] Periodic builder liveness probe: alive
[2025-08-28 18:51:47,023][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:51:52,024][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:51:57,025][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:52:02,026][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:52:07,027][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:52:12,028][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:52:17,030][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:52:22,031][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:52:27,033][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:52:32,034][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:52:37,035][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:52:42,038][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:52:47,039][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:52:52,040][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:52:57,041][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:53:02,042][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:53:07,043][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:53:12,044][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:53:17,230][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:53:22,231][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:53:27,232][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:53:32,234][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:53:36,560][ ERROR][PID:893487] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-08-28 18:53:37,234][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:53:42,235][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:53:46,568][WARNING][PID:893487] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-08-28 18:53:47,237][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:53:52,238][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:53:56,569][  INFO][PID:893487] Downloading the builder-live.log file, attempt 2
[2025-08-28 18:53:56,571][  INFO][PID:893487] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.242.106.163 copr-rpmbuild-log
[2025-08-28 18:53:57,239][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:54:02,240][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:54:07,241][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:54:12,242][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:54:17,243][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:54:22,244][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:54:27,245][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:54:32,246][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:54:37,248][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:54:42,249][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:54:47,250][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:54:52,251][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:54:56,639][ ERROR][PID:893487] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-08-28 18:54:57,252][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:55:02,253][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:55:06,680][WARNING][PID:893487] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-08-28 18:55:07,254][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:55:12,255][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:55:16,685][  INFO][PID:893487] Downloading the builder-live.log file, attempt 3
[2025-08-28 18:55:16,687][  INFO][PID:893487] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.242.106.163 copr-rpmbuild-log
[2025-08-28 18:55:17,256][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:55:22,257][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:55:27,258][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:55:32,258][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:55:37,260][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:55:42,261][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:55:47,262][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:55:52,263][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:55:57,264][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:56:02,265][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:56:07,266][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:56:12,267][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:56:16,755][ ERROR][PID:893487] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-08-28 18:56:17,268][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:56:22,303][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:56:26,758][WARNING][PID:893487] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-08-28 18:56:27,369][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:56:32,483][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:56:36,759][  INFO][PID:893487] Downloading the builder-live.log file, attempt 4
[2025-08-28 18:56:36,760][  INFO][PID:893487] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.242.106.163 copr-rpmbuild-log
[2025-08-28 18:56:37,485][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:56:42,486][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:56:47,487][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:56:52,488][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:56:57,489][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:57:02,490][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:57:07,491][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:57:12,493][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:57:17,494][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:57:22,495][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:57:27,496][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:57:32,497][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:57:36,828][ ERROR][PID:893487] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-08-28 18:57:37,497][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:57:42,498][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:57:46,877][ ERROR][PID:893487] Giving up for unstable SSH, failures: 1756407226.5685744, 1756407306.6808798, 1756407386.7586393, 1756407466.8775504
[2025-08-28 18:57:46,878][ ERROR][PID:893487] Re-try request for task on 'ResallocHost, ticket_id=4804997, hostname=54.242.106.163, name=aws_x86_64_spot_prod_06483958_20250828_184821, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-08-28 18:57:46,879][  INFO][PID:893487] Releasing VM back to pool
[2025-08-28 18:57:46,887][  INFO][PID:893487] Retry #2 (on other host)
[2025-08-28 18:57:46,888][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:57:46,889][  INFO][PID:893487] VM allocation process starts
[2025-08-28 18:57:46,899][  INFO][PID:893487] Trying to allocate VM: ResallocHost, ticket_id=4805019, requested_tags=['copr_builder', 'arch_x86_64']
[2025-08-28 18:57:49,910][  INFO][PID:893487] Allocated host ResallocHost, ticket_id=4805019, hostname=54.145.219.136, name=aws_x86_64_spot_prod_06483995_20250828_185057, requested_tags=['copr_builder', 'arch_x86_64']
[2025-08-28 18:57:49,911][  INFO][PID:893487] Allocating ssh connection to builder
[2025-08-28 18:57:49,911][  INFO][PID:893487] Checking that builder machine is OK
[2025-08-28 18:57:50,419][  INFO][PID:893487] Running remote command: copr-builder-ready fedora-42-x86_64
[2025-08-28 18:57:50,616][  INFO][PID:893487] Red Hat subscription not needed for fedora-42-x86_64
Builder is ready to be used

[2025-08-28 18:57:50,616][  INFO][PID:893487] Filling build.info file with builder info
[2025-08-28 18:57:50,617][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:57:50,618][  INFO][PID:893487] Not re-notifying FE and msg buses for the new host.
[2025-08-28 18:57:50,618][  INFO][PID:893487] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9500431-fedora-42-x86_64 --chroot fedora-42-x86_64 --detached
[2025-08-28 18:57:50,935][  INFO][PID:893487] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '54.145.219.136' (ED25519) to the list of known hosts.


[2025-08-28 18:57:50,936][  INFO][PID:893487] Downloading the builder-live.log file, attempt 1
[2025-08-28 18:57:50,937][  INFO][PID:893487] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.145.219.136 copr-rpmbuild-log
[2025-08-28 18:57:55,936][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:58:00,937][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:58:05,938][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:58:10,939][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:58:15,940][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:58:20,941][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:58:25,942][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:58:30,943][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:58:35,944][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:58:40,945][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:58:45,946][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:58:50,947][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:58:55,948][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:59:00,949][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:59:05,950][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:59:10,951][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:59:15,952][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:59:20,953][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:59:25,954][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:59:30,955][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:59:35,956][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:59:40,956][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:59:45,957][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:59:50,958][  INFO][PID:893487] Checking for cancel request
[2025-08-28 18:59:55,959][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:00:00,960][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:00:05,961][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:00:10,962][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:00:15,963][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:00:20,964][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:00:25,965][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:00:30,966][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:00:35,967][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:00:40,968][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:00:45,969][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:00:50,970][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:00:55,971][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:01:00,972][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:01:05,973][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:01:10,974][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:01:15,975][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:01:20,976][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:01:25,978][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:01:30,979][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:01:35,980][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:01:40,981][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:01:45,982][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:01:50,988][  INFO][PID:893487] Periodic builder liveness probe: alive
[2025-08-28 19:01:50,988][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:01:55,989][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:02:00,990][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:02:05,991][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:02:10,992][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:02:15,993][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:02:20,994][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:02:25,995][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:02:31,019][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:02:36,099][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:02:36,506][ ERROR][PID:893487] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-08-28 19:02:41,190][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:02:46,202][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:02:46,575][WARNING][PID:893487] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-08-28 19:02:51,245][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:02:56,248][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:02:56,576][  INFO][PID:893487] Downloading the builder-live.log file, attempt 2
[2025-08-28 19:02:56,579][  INFO][PID:893487] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.145.219.136 copr-rpmbuild-log
[2025-08-28 19:03:01,291][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:03:06,293][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:03:11,294][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:03:16,295][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:03:21,296][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:03:26,297][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:03:31,298][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:03:36,299][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:03:41,300][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:03:46,301][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:03:51,302][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:03:56,303][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:03:56,648][ ERROR][PID:893487] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-08-28 19:04:01,304][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:04:06,305][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:04:06,650][WARNING][PID:893487] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-08-28 19:04:11,307][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:04:16,308][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:04:16,651][  INFO][PID:893487] Downloading the builder-live.log file, attempt 3
[2025-08-28 19:04:16,652][  INFO][PID:893487] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.145.219.136 copr-rpmbuild-log
[2025-08-28 19:04:21,309][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:04:26,310][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:04:31,311][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:04:36,312][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:04:41,313][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:04:46,314][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:04:51,315][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:04:56,316][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:05:01,317][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:05:06,318][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:05:11,319][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:05:16,320][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:05:16,720][ ERROR][PID:893487] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-08-28 19:05:21,320][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:05:26,321][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:05:26,728][WARNING][PID:893487] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-08-28 19:05:31,322][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:05:36,323][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:05:36,729][  INFO][PID:893487] Downloading the builder-live.log file, attempt 4
[2025-08-28 19:05:36,730][  INFO][PID:893487] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.145.219.136 copr-rpmbuild-log
[2025-08-28 19:05:41,324][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:05:46,325][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:05:51,326][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:05:56,328][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:06:01,330][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:06:06,331][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:06:11,332][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:06:16,333][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:06:21,334][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:06:26,335][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:06:31,337][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:06:36,338][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:06:36,798][ ERROR][PID:893487] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-08-28 19:06:41,339][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:06:46,340][  INFO][PID:893487] Checking for cancel request
[2025-08-28 19:06:46,802][ ERROR][PID:893487] Giving up for unstable SSH, failures: 1756407766.574968, 1756407846.6506293, 1756407926.7283735, 1756408006.8022416
[2025-08-28 19:06:46,803][ ERROR][PID:893487] Re-try request for task on 'ResallocHost, ticket_id=4805019, hostname=54.145.219.136, name=aws_x86_64_spot_prod_06483995_20250828_185057, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-08-28 19:06:46,803][  INFO][PID:893487] Releasing VM back to pool
[2025-08-28 19:06:46,812][ ERROR][PID:893487] Backend process error: Three host tried without success: {'54.145.219.136', '54.242.106.163', '54.164.245.210'}
[2025-08-28 19:06:46,813][WARNING][PID:893487] Switching not-finished job state to 'failed'
[2025-08-28 19:06:46,813][  INFO][PID:893487] Worker failed build, took 536.1949970722198
[2025-08-28 19:06:46,813][  INFO][PID:893487] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "che",
            "project_name": "mesa",
            "project_dirname": "mesa",
            "submitter": "che",
            "ended_on": 1756408006.8130622,
            "started_on": 1756407470.618065,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-42-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "9500431-fedora-42-x86_64",
            "build_id": 9500431,
            "package_name": "mesa",
            "package_version": "25.3.0-0.20.gitb1d07ae",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/che/mesa/mesa",
            "git_hash": "4046385edd92096717c615943fa77c8f6d3f548a",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "che/mesa--che",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/che/mesa/fedora-42-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/che/llvm/fedora-42-x86_64/",
                    "id": "copr_che_llvm",
                    "name": "Additional repo copr_che_llvm"
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/che/mesa",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/che/mesa",
            "result_dir": "09500431-mesa",
            "built_packages": "",
            "id": 9500431,
            "mockchain_macros": {
                "copr_username": "che",
                "copr_projectname": "mesa",
                "vendor": "Fedora Project COPR (che/mesa)"
            }
        }
    ]
}
[2025-08-28 19:06:46,842][  INFO][PID:893487] Sending fedora-messaging bus message in build.end
[2025-08-28 19:06:46,867][  INFO][PID:893487] Compressing /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa/builder-live.log by gzip
[2025-08-28 19:06:46,869][  INFO][PID:893487] Running command 'gzip /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa/builder-live.log' as PID 1094470
[2025-08-28 19:06:46,870][  INFO][PID:893487] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa/builder-live.log)
[2025-08-28 19:06:46,870][  INFO][PID:893487] Compressing /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa/backend.log by gzip
[2025-08-28 19:06:46,871][  INFO][PID:893487] Running command 'gzip /var/lib/copr/public_html/results/che/mesa/fedora-42-x86_64/09500431-mesa/backend.log' as PID 1094471