[2026-09-22 15:30:50,785][  INFO][PID:1761863] Marking build as starting
[2026-09-22 15:30:50,860][  INFO][PID:1761863] Checking that /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/epel-9-ppc64le/repodata/repomd.xml exists
[2026-09-22 15:30:50,861][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:30:50,880][  INFO][PID:1761863] rpmeta: predicted 239 min for thunderbird on epel-9-ppc64le -> keeping powerful builder tag
[2026-09-22 15:30:50,882][  INFO][PID:1761863] VM allocation process starts
[2026-09-22 15:30:50,898][  INFO][PID:1761863] Trying to allocate VM: ResallocHost, ticket_id=5186353, requested_tags=['arch_power9', 'arch_ppc64le', 'copr_builder', 'on_demand_powerful']
[2026-09-22 15:30:55,900][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:31:00,901][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:31:02,942][  INFO][PID:1761863] Allocated host ResallocHost, ticket_id=5186353, hostname=2620:52:6:1161:dead:beef:cafe:c185, name=vmhost_p_p09_04_prod_41060349_20260922_150926, requested_tags=['arch_power9', 'arch_ppc64le', 'copr_builder', 'on_demand_powerful']
[2026-09-22 15:31:02,943][  INFO][PID:1761863] Allocating ssh connection to builder
[2026-09-22 15:31:02,943][  INFO][PID:1761863] Checking that builder machine is OK
[2026-09-22 15:31:03,438][  INFO][PID:1761863] Installed copr-rpmbuild version: 1.9
[2026-09-22 15:31:03,439][  INFO][PID:1761863] Running remote command: copr-builder-ready epel-9-ppc64le
[2026-09-22 15:31:03,946][  INFO][PID:1761863] Checking Red Hat subscription (attempt #1)
Red Hat subscription active
Builder is ready to be used

[2026-09-22 15:31:03,947][  INFO][PID:1761863] Filling build.info file with builder info
[2026-09-22 15:31:03,947][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:31:03,947][  INFO][PID:1761863] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "ligenix",
            "project_name": "enterprise-cosmic-testing",
            "project_dirname": "enterprise-cosmic-testing",
            "submitter": "ligenix",
            "ended_on": null,
            "started_on": 1790091063.947823,
            "submitted_on": null,
            "status": 3,
            "chroot": "epel-9-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "11021337-epel-9-ppc64le",
            "build_id": 11021337,
            "package_name": "thunderbird",
            "package_version": "156.0-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/ligenix/enterprise-cosmic-testing/thunderbird",
            "git_hash": "1ca2542e3d0dd764cc742e610526e37976ca71db",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "ligenix/enterprise-cosmic-testing--ligenix",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-cosmic-testing/epel-9-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null,
                    "repo_gpgcheck": false
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-cosmic/epel-9-ppc64le/",
                    "id": "copr_ligenix_enterprise_cosmic_rhel_epel_releasever_basearch",
                    "name": "Additional repo copr_ligenix_enterprise_cosmic_rhel_epel_releasever_basearch"
                },
                {
                    "baseurl": "https://dl.rockylinux.org/pub/rocky/$releasever/devel/$basearch/os/",
                    "id": "https_dl_rockylinux_org_pub_rocky_releasever_devel_basearch_os",
                    "name": "Additional repo https_dl_rockylinux_org_pub_rocky_releasever_devel_basearch_os"
                },
                {
                    "baseurl": "https://repo.almalinux.org/almalinux/$releasever/BaseOS/$basearch/os/",
                    "id": "https_repo_almalinux_org_almalinux_releasever_BaseOS_basearch_os",
                    "name": "Additional repo https_repo_almalinux_org_almalinux_releasever_BaseOS_basearch_os"
                },
                {
                    "baseurl": "https://repo.almalinux.org/almalinux/$releasever/AppStream/$basearch/os/",
                    "id": "https_repo_almalinux_org_almalinux_releasever_AppStream_basearch_os",
                    "name": "Additional repo https_repo_almalinux_org_almalinux_releasever_AppStream_basearch_os"
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9",
                "on_demand_powerful"
            ],
            "with_opts": [
                "jackserver_plugin",
                "libcamera_plugin"
            ],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-cosmic-testing",
            "result_dir": "11021337-thunderbird",
            "built_packages": "",
            "rpmeta_prediction": 239,
            "id": 11021337,
            "mockchain_macros": {
                "copr_username": "ligenix",
                "copr_projectname": "enterprise-cosmic-testing",
                "vendor": "Fedora Project COPR (ligenix/enterprise-cosmic-testing)"
            }
        }
    ]
}
[2026-09-22 15:31:03,989][  INFO][PID:1761863] Sending fedora-messaging bus message in build.start
[2026-09-22 15:31:04,337][  INFO][PID:1761863] Sending fedora-messaging bus message in chroot.start
[2026-09-22 15:31:04,356][  INFO][PID:1761863] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/11021337-epel-9-ppc64le --chroot epel-9-ppc64le --detached
[2026-09-22 15:31:05,446][  INFO][PID:1761863] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c185' (ED25519) to the list of known hosts.


[2026-09-22 15:31:05,446][  INFO][PID:1761863] Downloading the builder-live.log file, attempt 1
[2026-09-22 15:31:05,447][  INFO][PID:1761863] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c185 copr-rpmbuild-log
[2026-09-22 15:31:10,450][  INFO][PID:1761863] Periodic builder liveness probe: alive
[2026-09-22 15:31:10,451][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:31:15,451][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:31:20,452][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:31:25,453][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:31:30,454][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:31:35,455][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:31:40,456][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:31:45,458][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:31:50,459][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:31:55,460][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:32:00,461][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:32:05,462][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:32:10,463][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:32:15,464][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:32:20,465][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:32:25,466][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:32:30,466][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:32:35,467][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:32:40,469][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:32:45,470][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:32:50,471][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:32:55,471][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:33:00,473][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:33:05,474][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:33:10,475][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:33:15,475][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:33:20,476][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:33:25,477][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:33:30,478][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:33:35,479][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:33:40,481][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:33:45,482][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:33:50,483][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:33:55,486][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:34:00,489][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:34:05,490][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:34:10,490][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:34:15,491][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:34:20,492][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:34:25,493][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:34:30,494][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:34:35,496][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:34:40,497][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:34:45,498][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:34:50,499][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:34:55,501][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:35:00,502][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:35:05,503][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:35:10,504][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:35:15,504][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:35:20,505][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:35:25,506][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:35:30,507][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:35:35,508][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:35:40,509][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:35:45,510][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:35:50,511][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:35:55,511][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:36:00,512][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:36:05,513][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:36:10,514][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:36:15,515][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:36:20,516][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:36:25,517][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:36:30,517][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:36:35,520][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:36:40,521][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:36:45,522][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:36:50,522][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:36:55,523][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:37:00,525][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:37:05,531][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:37:10,532][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:37:15,534][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:37:20,534][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:37:25,536][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:37:30,538][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:37:35,540][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:37:40,541][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:37:45,541][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:37:50,542][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:37:55,543][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:38:00,544][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:38:05,616][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:38:10,732][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:38:15,736][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:38:20,750][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:38:25,845][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:38:30,852][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:38:36,510][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:38:41,511][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:38:46,512][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:38:51,514][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:38:56,515][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:39:01,516][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:39:06,517][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:39:11,518][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:39:16,518][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:39:21,672][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:39:27,018][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:39:32,019][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:39:37,020][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:39:42,021][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:39:47,022][  INFO][PID:1761863] Checking for cancel request
[2026-09-22 15:39:49,581][  INFO][PID:1761863] Downloading results from builder
[2026-09-22 15:39:49,582][  INFO][PID:1761863] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c185]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/epel-9-ppc64le/11021337-thunderbird started
[2026-09-22 15:39:49,582][  INFO][PID:1761863] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c185]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/epel-9-ppc64le/11021337-thunderbird/ &> /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/epel-9-ppc64le/11021337-thunderbird/build-11021337.rsync.log
[2026-09-22 15:39:56,983][  INFO][PID:1761863] rsyncing finished.
[2026-09-22 15:39:56,983][  INFO][PID:1761863] VM Release request
[2026-09-22 15:39:56,988][  INFO][PID:1761863] Searching for 'success' file in resultdir
[2026-09-22 15:39:56,989][ ERROR][PID:1761863] Build failed: Backend process error: No success file => build failure
[2026-09-22 15:39:56,994][  INFO][PID:1761863] Finished build: id=11021337 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing chroot=epel-9-ppc64le 
[2026-09-22 15:39:56,995][ ERROR][PID:1761863] Backend process error: No success file => build failure
[2026-09-22 15:39:57,000][  INFO][PID:1761863] Worker failed build, took 533.0523493289948
[2026-09-22 15:39:57,000][  INFO][PID:1761863] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "ligenix",
            "project_name": "enterprise-cosmic-testing",
            "project_dirname": "enterprise-cosmic-testing",
            "submitter": "ligenix",
            "ended_on": 1790091597.0001724,
            "started_on": 1790091063.947823,
            "submitted_on": null,
            "status": 0,
            "chroot": "epel-9-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "11021337-epel-9-ppc64le",
            "build_id": 11021337,
            "package_name": "thunderbird",
            "package_version": "156.0-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/ligenix/enterprise-cosmic-testing/thunderbird",
            "git_hash": "1ca2542e3d0dd764cc742e610526e37976ca71db",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "ligenix/enterprise-cosmic-testing--ligenix",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-cosmic-testing/epel-9-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null,
                    "repo_gpgcheck": false
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-cosmic/epel-9-ppc64le/",
                    "id": "copr_ligenix_enterprise_cosmic_rhel_epel_releasever_basearch",
                    "name": "Additional repo copr_ligenix_enterprise_cosmic_rhel_epel_releasever_basearch"
                },
                {
                    "baseurl": "https://dl.rockylinux.org/pub/rocky/$releasever/devel/$basearch/os/",
                    "id": "https_dl_rockylinux_org_pub_rocky_releasever_devel_basearch_os",
                    "name": "Additional repo https_dl_rockylinux_org_pub_rocky_releasever_devel_basearch_os"
                },
                {
                    "baseurl": "https://repo.almalinux.org/almalinux/$releasever/BaseOS/$basearch/os/",
                    "id": "https_repo_almalinux_org_almalinux_releasever_BaseOS_basearch_os",
                    "name": "Additional repo https_repo_almalinux_org_almalinux_releasever_BaseOS_basearch_os"
                },
                {
                    "baseurl": "https://repo.almalinux.org/almalinux/$releasever/AppStream/$basearch/os/",
                    "id": "https_repo_almalinux_org_almalinux_releasever_AppStream_basearch_os",
                    "name": "Additional repo https_repo_almalinux_org_almalinux_releasever_AppStream_basearch_os"
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9",
                "on_demand_powerful"
            ],
            "with_opts": [
                "jackserver_plugin",
                "libcamera_plugin"
            ],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-cosmic-testing",
            "result_dir": "11021337-thunderbird",
            "built_packages": "",
            "rpmeta_prediction": 239,
            "id": 11021337,
            "mockchain_macros": {
                "copr_username": "ligenix",
                "copr_projectname": "enterprise-cosmic-testing",
                "vendor": "Fedora Project COPR (ligenix/enterprise-cosmic-testing)"
            }
        }
    ]
}
[2026-09-22 15:39:57,046][  INFO][PID:1761863] Sending fedora-messaging bus message in build.end
[2026-09-22 15:39:57,064][  INFO][PID:1761863] Compressing /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/epel-9-ppc64le/11021337-thunderbird/builder-live.log by gzip
[2026-09-22 15:39:57,065][  INFO][PID:1761863] Running command 'gzip /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/epel-9-ppc64le/11021337-thunderbird/builder-live.log' as PID 1904058
[2026-09-22 15:39:57,070][  INFO][PID:1761863] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/epel-9-ppc64le/11021337-thunderbird/builder-live.log)
[2026-09-22 15:39:57,070][  INFO][PID:1761863] Compressing /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/epel-9-ppc64le/11021337-thunderbird/backend.log by gzip
[2026-09-22 15:39:57,071][  INFO][PID:1761863] Running command 'gzip /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/epel-9-ppc64le/11021337-thunderbird/backend.log' as PID 1904061