[2026-09-15 05:28:24,286][ INFO][PID:2938035] Marking build as starting [2026-09-15 05:28:24,318][ INFO][PID:2938035] Checking that https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915/rhel-8-ppc64le//repodata/repomd.xml exists [2026-09-15 05:28:24,673][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:28:24,693][ INFO][PID:2938035] rpmeta: predicted 54 min for llvm on rhel-8-ppc64le -> keeping powerful builder tag [2026-09-15 05:28:24,694][ INFO][PID:2938035] VM allocation process starts [2026-09-15 05:28:24,699][ INFO][PID:2938035] Trying to allocate VM: ResallocHost, ticket_id=5029416, requested_tags=['arch_ppc64le', 'copr_builder', 'on_demand_powerful'] [2026-09-15 05:28:29,700][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:28:34,701][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:28:39,702][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:28:44,703][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:28:49,704][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:28:54,705][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:28:59,706][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:29:04,706][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:29:09,707][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:29:14,708][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:29:19,709][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:29:24,710][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:29:29,711][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:29:34,711][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:29:39,712][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:29:44,713][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:29:50,869][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:29:55,870][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:30:00,996][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:30:05,997][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:30:10,998][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:30:15,998][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:30:20,999][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:30:26,000][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:30:31,001][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:30:36,002][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:30:41,003][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:30:46,004][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:30:51,005][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:30:56,006][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:31:01,007][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:31:06,007][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:31:11,008][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:31:16,009][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:31:21,010][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:31:26,010][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:31:31,011][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:31:36,012][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:31:41,013][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:31:46,014][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:31:51,015][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:31:56,016][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:32:01,017][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:32:06,017][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:32:11,018][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:32:16,019][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:32:21,020][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:32:26,021][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:32:31,022][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:32:36,023][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:32:41,023][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:32:46,103][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:32:51,104][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:32:56,105][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:33:01,107][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:33:06,107][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:33:11,108][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:33:16,109][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:33:21,111][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:33:26,111][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:33:31,112][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:33:36,113][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:33:41,114][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:33:42,820][ INFO][PID:2938035] Allocated host ResallocHost, ticket_id=5029416, hostname=2620:52:6:1161:dead:beef:cafe:c141, name=vmhost_p_p09_01_prod_37596473_20260915_052703, requested_tags=['arch_ppc64le', 'copr_builder', 'on_demand_powerful'] [2026-09-15 05:33:42,820][ INFO][PID:2938035] Allocating ssh connection to builder [2026-09-15 05:33:42,820][ INFO][PID:2938035] Checking that builder machine is OK [2026-09-15 05:33:43,229][ INFO][PID:2938035] Installed copr-rpmbuild version: 1.9 [2026-09-15 05:33:43,230][ INFO][PID:2938035] Running remote command: copr-builder-ready rhel-8-ppc64le [2026-09-15 05:33:43,725][ INFO][PID:2938035] Checking Red Hat subscription (attempt #1) Red Hat subscription active Builder is ready to be used [2026-09-15 05:33:43,726][ INFO][PID:2938035] Filling build.info file with builder info [2026-09-15 05:33:43,726][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:33:43,726][ INFO][PID:2938035] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "@fedora-llvm-team", "project_name": "llvm-snapshots-big-merge-20260915", "project_dirname": "llvm-snapshots-big-merge-20260915", "submitter": "kkleine", "ended_on": null, "started_on": 1789450423.726675, "submitted_on": null, "status": 3, "chroot": "rhel-8-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "10986683-rhel-8-ppc64le", "build_id": 10986683, "package_name": "llvm", "package_version": "23.1.0-11", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@fedora-llvm-team/llvm-snapshots-big-merge-20260915/llvm", "git_hash": "3197154e9016b07302292cd6087e0e992ea63295", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@fedora-llvm-team/llvm-snapshots-big-merge-20260915--kkleine", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 1, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915/rhel-8-ppc64le/", "id": "copr_base", "module_hotfixes": true, "name": "Copr repository", "priority": null, "repo_gpgcheck": false }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-test-suite/rhel-8-ppc64le/", "id": "copr_fedora_llvm_team_llvm_test_suite", "name": "Additional repo copr_fedora_llvm_team_llvm_test_suite" } ], "background": false, "fedora_review": false, "isolation": "default", "modules": { "toggle": [ { "enable": "swig:4.0" } ] }, "repo_priority": null, "tags": [ "arch_ppc64le", "on_demand_powerful", "on_demand_powerful" ], "with_opts": [ "snapshot_build" ], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915", "result_dir": "10986683-llvm", "built_packages": "", "rpmeta_prediction": 54, "id": 10986683, "mockchain_macros": { "copr_username": "@fedora-llvm-team", "copr_projectname": "llvm-snapshots-big-merge-20260915", "vendor": "Fedora Project COPR (@fedora-llvm-team/llvm-snapshots-big-merge-20260915)" } } ] } [2026-09-15 05:33:43,756][ INFO][PID:2938035] Sending fedora-messaging bus message in build.start [2026-09-15 05:33:44,106][ INFO][PID:2938035] Sending fedora-messaging bus message in chroot.start [2026-09-15 05:33:44,127][ INFO][PID:2938035] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10986683-rhel-8-ppc64le --chroot rhel-8-ppc64le --detached [2026-09-15 05:33:45,039][ INFO][PID:2938035] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c141' (ED25519) to the list of known hosts. [2026-09-15 05:33:45,040][ INFO][PID:2938035] Downloading the builder-live.log file, attempt 1 [2026-09-15 05:33:45,041][ INFO][PID:2938035] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c141 copr-rpmbuild-log [2026-09-15 05:33:50,044][ INFO][PID:2938035] Periodic builder liveness probe: alive [2026-09-15 05:33:50,044][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:33:55,044][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:34:00,045][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:34:05,046][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:34:10,047][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:34:15,048][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:34:20,049][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:34:25,050][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:34:30,051][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:34:35,051][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:34:40,052][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:34:45,053][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:34:50,054][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:34:55,055][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:35:00,056][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:35:05,057][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:35:10,057][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:35:15,058][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:35:20,059][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:35:25,060][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:35:30,061][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:35:35,957][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:35:41,628][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:35:47,239][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:35:52,528][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:35:57,722][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:36:03,223][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:36:08,515][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:36:13,740][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:36:18,741][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:36:23,742][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:36:28,743][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:36:33,744][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:36:38,745][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:36:43,747][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:36:48,748][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:36:53,748][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:36:58,749][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:37:03,750][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:37:08,751][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:37:13,751][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:37:18,752][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:37:23,896][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:37:29,189][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:37:34,192][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:37:39,192][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:37:44,193][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:37:49,194][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:37:54,315][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:37:59,316][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:38:04,316][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:38:09,317][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:38:14,318][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:38:19,319][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:38:24,320][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:38:29,368][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:38:34,369][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:38:39,370][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:38:44,371][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:38:49,372][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:38:54,372][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:38:59,373][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:39:04,374][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:39:09,375][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:39:14,376][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:39:19,377][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:39:24,378][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:39:29,379][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:39:34,380][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:39:39,387][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:39:44,388][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:39:49,389][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:39:54,390][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:39:59,391][ INFO][PID:2938035] Checking for cancel request [2026-09-15 05:40:01,200][ INFO][PID:2938035] Downloading results from builder [2026-09-15 05:40:01,200][ INFO][PID:2938035] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c141]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915/rhel-8-ppc64le/10986683-llvm started [2026-09-15 05:40:01,201][ INFO][PID:2938035] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c141]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915/rhel-8-ppc64le/10986683-llvm/ &> /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915/rhel-8-ppc64le/10986683-llvm/build-10986683.rsync.log [2026-09-15 05:40:03,996][ INFO][PID:2938035] rsyncing finished. [2026-09-15 05:40:03,997][ INFO][PID:2938035] VM Release request [2026-09-15 05:40:04,022][ INFO][PID:2938035] Searching for 'success' file in resultdir [2026-09-15 05:40:04,023][ ERROR][PID:2938035] Build failed: Backend process error: No success file => build failure [2026-09-15 05:40:04,028][ INFO][PID:2938035] Finished build: id=10986683 failed=True timeout=108000 destdir=/var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915 chroot=rhel-8-ppc64le [2026-09-15 05:40:04,029][ ERROR][PID:2938035] Backend process error: No success file => build failure [2026-09-15 05:40:04,034][ INFO][PID:2938035] Worker failed build, took 380.3078701496124 [2026-09-15 05:40:04,034][ INFO][PID:2938035] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "@fedora-llvm-team", "project_name": "llvm-snapshots-big-merge-20260915", "project_dirname": "llvm-snapshots-big-merge-20260915", "submitter": "kkleine", "ended_on": 1789450804.0345452, "started_on": 1789450423.726675, "submitted_on": null, "status": 0, "chroot": "rhel-8-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "10986683-rhel-8-ppc64le", "build_id": 10986683, "package_name": "llvm", "package_version": "23.1.0-11", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@fedora-llvm-team/llvm-snapshots-big-merge-20260915/llvm", "git_hash": "3197154e9016b07302292cd6087e0e992ea63295", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@fedora-llvm-team/llvm-snapshots-big-merge-20260915--kkleine", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 1, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915/rhel-8-ppc64le/", "id": "copr_base", "module_hotfixes": true, "name": "Copr repository", "priority": null, "repo_gpgcheck": false }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-test-suite/rhel-8-ppc64le/", "id": "copr_fedora_llvm_team_llvm_test_suite", "name": "Additional repo copr_fedora_llvm_team_llvm_test_suite" } ], "background": false, "fedora_review": false, "isolation": "default", "modules": { "toggle": [ { "enable": "swig:4.0" } ] }, "repo_priority": null, "tags": [ "arch_ppc64le", "on_demand_powerful", "on_demand_powerful" ], "with_opts": [ "snapshot_build" ], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915", "result_dir": "10986683-llvm", "built_packages": "", "rpmeta_prediction": 54, "id": 10986683, "mockchain_macros": { "copr_username": "@fedora-llvm-team", "copr_projectname": "llvm-snapshots-big-merge-20260915", "vendor": "Fedora Project COPR (@fedora-llvm-team/llvm-snapshots-big-merge-20260915)" } } ] } [2026-09-15 05:40:04,063][ INFO][PID:2938035] Sending fedora-messaging bus message in build.end [2026-09-15 05:40:04,083][ INFO][PID:2938035] Compressing /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915/rhel-8-ppc64le/10986683-llvm/builder-live.log by gzip [2026-09-15 05:40:04,084][ INFO][PID:2938035] Running command 'gzip /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915/rhel-8-ppc64le/10986683-llvm/builder-live.log' as PID 3064361 [2026-09-15 05:40:04,151][ INFO][PID:2938035] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915/rhel-8-ppc64le/10986683-llvm/builder-live.log) [2026-09-15 05:40:04,151][ INFO][PID:2938035] Compressing /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915/rhel-8-ppc64le/10986683-llvm/backend.log by gzip [2026-09-15 05:40:04,152][ INFO][PID:2938035] Running command 'gzip /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-big-merge-20260915/rhel-8-ppc64le/10986683-llvm/backend.log' as PID 3064401