[2026-01-11 02:12:58,572][  INFO][PID:2778916] Marking build as starting
[2026-01-11 02:12:58,731][  INFO][PID:2778916] Checking that /var/lib/copr/public_html/results/@meta/pystemd/fedora-42-ppc64le/repodata/repomd.xml exists
[2026-01-11 02:12:59,064][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:12:59,064][  INFO][PID:2778916] VM allocation process starts
[2026-01-11 02:12:59,079][  INFO][PID:2778916] Trying to allocate VM: ResallocHost, ticket_id=752434, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-11 02:13:02,101][  INFO][PID:2778916] Allocated host ResallocHost, ticket_id=752434, hostname=2620:52:6:1161:dead:beef:cafe:c189, name=vmhost_p09_04_prod_01084852_20260111_015534, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-11 02:13:02,101][  INFO][PID:2778916] Allocating ssh connection to builder
[2026-01-11 02:13:02,102][  INFO][PID:2778916] Checking that builder machine is OK
[2026-01-11 02:14:02,172][ ERROR][PID:2778916] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c189 port 22: Connection timed out

[2026-01-11 02:14:12,173][ ERROR][PID:2778916] Re-try request for task on 'ResallocHost, ticket_id=752434, hostname=2620:52:6:1161:dead:beef:cafe:c189, name=vmhost_p09_04_prod_01084852_20260111_015534, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-01-11 02:14:12,174][  INFO][PID:2778916] Releasing VM back to pool
[2026-01-11 02:14:12,185][  INFO][PID:2778916] Retry #1 (on other host)
[2026-01-11 02:14:12,185][  INFO][PID:2778916] Checking that /var/lib/copr/public_html/results/@meta/pystemd/fedora-42-ppc64le/repodata/repomd.xml exists
[2026-01-11 02:14:12,186][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:14:12,187][  INFO][PID:2778916] VM allocation process starts
[2026-01-11 02:14:12,197][  INFO][PID:2778916] Trying to allocate VM: ResallocHost, ticket_id=752446, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-11 02:14:15,210][  INFO][PID:2778916] Allocated host ResallocHost, ticket_id=752446, hostname=2620:52:6:1161:dead:beef:cafe:c18d, name=vmhost_p09_04_prod_01084880_20260111_015914, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-11 02:14:15,210][  INFO][PID:2778916] Allocating ssh connection to builder
[2026-01-11 02:14:15,211][  INFO][PID:2778916] Checking that builder machine is OK
[2026-01-11 02:14:16,219][  INFO][PID:2778916] Running remote command: copr-builder-ready fedora-42-ppc64le
[2026-01-11 02:14:16,796][  INFO][PID:2778916] Red Hat subscription not needed for fedora-42-ppc64le
Builder is ready to be used

[2026-01-11 02:14:16,797][  INFO][PID:2778916] Filling build.info file with builder info
[2026-01-11 02:14:16,797][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:14:16,798][  INFO][PID:2778916] Not re-notifying FE and msg buses for the new host.
[2026-01-11 02:14:16,799][  INFO][PID:2778916] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9995034-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached
[2026-01-11 02:15:16,830][ ERROR][PID:2778916] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c18d port 22: Connection timed out

[2026-01-11 02:15:26,840][ ERROR][PID:2778916] Re-try request for task on 'ResallocHost, ticket_id=752446, hostname=2620:52:6:1161:dead:beef:cafe:c18d, name=vmhost_p09_04_prod_01084880_20260111_015914, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-01-11 02:15:26,841][  INFO][PID:2778916] Releasing VM back to pool
[2026-01-11 02:15:26,849][  INFO][PID:2778916] Retry #2 (on other host)
[2026-01-11 02:15:26,849][  INFO][PID:2778916] Checking that /var/lib/copr/public_html/results/@meta/pystemd/fedora-42-ppc64le/repodata/repomd.xml exists
[2026-01-11 02:15:26,850][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:15:26,850][  INFO][PID:2778916] VM allocation process starts
[2026-01-11 02:15:26,901][  INFO][PID:2778916] Trying to allocate VM: ResallocHost, ticket_id=752456, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-11 02:15:29,949][  INFO][PID:2778916] Allocated host ResallocHost, ticket_id=752456, hostname=2620:52:6:1161:dead:beef:cafe:c18a, name=vmhost_p09_04_prod_01084901_20260111_020201, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-11 02:15:29,949][  INFO][PID:2778916] Allocating ssh connection to builder
[2026-01-11 02:15:29,950][  INFO][PID:2778916] Checking that builder machine is OK
[2026-01-11 02:15:30,807][  INFO][PID:2778916] Running remote command: copr-builder-ready fedora-42-ppc64le
[2026-01-11 02:15:31,301][  INFO][PID:2778916] Red Hat subscription not needed for fedora-42-ppc64le
Builder is ready to be used

[2026-01-11 02:15:31,302][  INFO][PID:2778916] Filling build.info file with builder info
[2026-01-11 02:15:31,307][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:15:31,308][  INFO][PID:2778916] Not re-notifying FE and msg buses for the new host.
[2026-01-11 02:15:31,308][  INFO][PID:2778916] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9995034-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached
[2026-01-11 02:15:32,250][  INFO][PID:2778916] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18a' (ED25519) to the list of known hosts.


[2026-01-11 02:15:32,251][  INFO][PID:2778916] Downloading the builder-live.log file, attempt 1
[2026-01-11 02:15:32,252][  INFO][PID:2778916] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18a copr-rpmbuild-log
[2026-01-11 02:15:37,257][  INFO][PID:2778916] Periodic builder liveness probe: alive
[2026-01-11 02:15:37,258][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:15:42,260][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:15:47,261][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:15:52,263][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:15:57,467][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:16:02,468][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:16:07,469][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:16:12,470][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:16:17,472][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:16:22,475][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:16:27,476][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:16:32,477][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:16:37,478][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:16:42,479][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:16:47,480][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:16:52,481][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:16:57,482][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:17:02,484][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:17:07,488][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:17:12,491][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:17:17,528][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:17:22,530][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:17:27,539][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:17:32,540][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:17:38,006][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:17:43,007][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:17:48,009][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:17:53,011][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:17:58,012][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:18:03,014][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:18:08,015][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:18:13,016][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:18:18,019][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:18:23,026][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:18:28,029][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:18:33,034][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:18:38,037][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:18:43,038][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:18:48,167][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:18:53,168][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:18:58,170][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:19:03,171][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:19:08,172][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:19:13,173][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:19:18,175][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:19:23,179][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:19:28,183][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:19:33,184][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:19:38,185][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:19:43,186][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:19:48,187][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:19:53,188][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:19:58,189][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:20:03,190][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:20:08,193][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:20:13,194][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:20:18,195][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:20:23,196][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:20:28,197][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:20:33,199][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:20:38,200][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:20:43,201][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:20:48,204][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:20:53,207][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:20:58,208][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:21:03,209][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:21:08,212][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:21:13,236][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:21:18,238][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:21:23,239][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:21:28,240][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:21:33,241][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:21:38,242][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:21:43,243][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:21:48,244][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:21:53,245][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:21:58,246][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:22:03,247][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:22:08,248][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:22:13,299][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:22:18,332][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:22:23,333][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:22:28,334][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:22:33,335][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:22:38,336][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:22:43,338][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:22:48,339][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:22:53,340][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:22:58,341][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:23:03,342][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:23:08,343][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:23:13,344][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:23:18,346][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:23:23,355][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:23:28,357][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:23:33,358][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:23:38,359][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:23:43,360][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:23:48,375][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:23:53,400][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:23:58,404][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:24:03,405][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:24:08,406][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:24:13,410][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:24:18,411][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:24:23,412][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:24:28,413][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:24:33,418][  INFO][PID:2778916] Checking for cancel request
[2026-01-11 02:24:37,002][  INFO][PID:2778916] Downloading results from builder
[2026-01-11 02:24:37,003][  INFO][PID:2778916] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18a]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@meta/pystemd/fedora-42-ppc64le/09995034-python-pystemd started
[2026-01-11 02:24:37,005][  INFO][PID:2778916] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18a]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@meta/pystemd/fedora-42-ppc64le/09995034-python-pystemd/ &> /var/lib/copr/public_html/results/@meta/pystemd/fedora-42-ppc64le/09995034-python-pystemd/build-09995034.rsync.log
[2026-01-11 02:24:37,680][  INFO][PID:2778916] rsyncing finished.
[2026-01-11 02:24:37,681][  INFO][PID:2778916] Releasing VM back to pool
[2026-01-11 02:24:37,693][  INFO][PID:2778916] Searching for 'success' file in resultdir
[2026-01-11 02:24:37,694][ ERROR][PID:2778916] Build failed: Backend process error: No success file => build failure
[2026-01-11 02:24:37,695][  INFO][PID:2778916] Finished build: id=9995034 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/@meta/pystemd chroot=fedora-42-ppc64le 
[2026-01-11 02:24:37,697][ ERROR][PID:2778916] Backend process error: No success file => build failure
[2026-01-11 02:24:37,698][  INFO][PID:2778916] Worker failed build, took 546.3902614116669
[2026-01-11 02:24:37,698][  INFO][PID:2778916] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@meta",
            "project_name": "pystemd",
            "project_dirname": "pystemd",
            "submitter": "packit",
            "ended_on": 1768098277.698264,
            "started_on": 1768097731.3080025,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-42-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9995034-fedora-42-ppc64le",
            "build_id": 9995034,
            "package_name": "python-pystemd",
            "package_version": "0.15.1-1.20260111021240060733.pr112.10.gf860387",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@meta/pystemd/python-pystemd",
            "git_hash": "08b6880e10e96d4d0980d4f366b6c741b4aa3a47",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@meta/pystemd--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@meta/pystemd/fedora-42-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@meta/pystemd",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@meta/pystemd",
            "result_dir": "09995034-python-pystemd",
            "built_packages": "",
            "id": 9995034,
            "mockchain_macros": {
                "copr_username": "@meta",
                "copr_projectname": "pystemd",
                "vendor": "Fedora Project COPR (@meta/pystemd)"
            }
        }
    ]
}
[2026-01-11 02:24:37,791][  INFO][PID:2778916] Sending fedora-messaging bus message in build.end
[2026-01-11 02:24:38,492][  INFO][PID:2778916] Compressing /var/lib/copr/public_html/results/@meta/pystemd/fedora-42-ppc64le/09995034-python-pystemd/builder-live.log by gzip
[2026-01-11 02:24:38,495][  INFO][PID:2778916] Running command 'gzip /var/lib/copr/public_html/results/@meta/pystemd/fedora-42-ppc64le/09995034-python-pystemd/builder-live.log' as PID 2845210
[2026-01-11 02:24:38,573][  INFO][PID:2778916] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@meta/pystemd/fedora-42-ppc64le/09995034-python-pystemd/builder-live.log)
[2026-01-11 02:24:38,574][  INFO][PID:2778916] Compressing /var/lib/copr/public_html/results/@meta/pystemd/fedora-42-ppc64le/09995034-python-pystemd/backend.log by gzip
[2026-01-11 02:24:38,575][  INFO][PID:2778916] Running command 'gzip /var/lib/copr/public_html/results/@meta/pystemd/fedora-42-ppc64le/09995034-python-pystemd/backend.log' as PID 2845224