[2026-01-08 01:56:57,929][  INFO][PID:3046899] Marking build as starting
[2026-01-08 01:56:58,096][  INFO][PID:3046899] Checking that /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/repodata/repomd.xml exists
[2026-01-08 01:56:58,214][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:56:58,215][  INFO][PID:3046899] VM allocation process starts
[2026-01-08 01:56:58,231][  INFO][PID:3046899] Trying to allocate VM: ResallocHost, ticket_id=702092, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-08 01:57:03,232][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:57:08,234][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:57:13,234][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:57:18,235][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:57:23,237][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:57:28,238][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:57:33,240][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:57:38,241][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:57:43,241][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:57:48,242][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:57:53,243][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:57:58,245][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:58:03,246][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:58:08,247][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:58:13,280][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:58:18,281][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:58:23,282][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:58:28,283][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:58:33,284][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:58:38,285][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:58:43,287][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:58:48,287][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:58:53,288][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:58:58,289][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:59:03,290][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:59:08,292][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:59:13,293][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:59:18,294][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:59:23,295][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:59:28,296][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:59:33,297][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:59:38,298][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:59:43,299][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:59:48,300][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:59:53,301][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 01:59:58,302][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:00:03,303][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:00:08,304][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:00:13,305][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:00:18,307][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:00:23,308][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:00:28,309][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:00:33,310][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:00:38,311][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:00:43,311][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:00:48,312][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:00:53,318][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:00:58,319][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:01:03,319][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:01:08,321][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:01:13,322][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:01:16,352][  INFO][PID:3046899] Allocated host ResallocHost, ticket_id=702092, hostname=2620:52:6:1161:dead:beef:cafe:c18d, name=vmhost_p09_04_prod_01007374_20260108_015608, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-08 02:01:16,353][  INFO][PID:3046899] Allocating ssh connection to builder
[2026-01-08 02:01:16,353][  INFO][PID:3046899] Checking that builder machine is OK
[2026-01-08 02:01:17,325][  INFO][PID:3046899] Running remote command: copr-builder-ready fedora-42-ppc64le
[2026-01-08 02:01:17,822][  INFO][PID:3046899] Red Hat subscription not needed for fedora-42-ppc64le
Builder is ready to be used

[2026-01-08 02:01:17,823][  INFO][PID:3046899] Filling build.info file with builder info
[2026-01-08 02:01:17,823][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:01:17,824][  INFO][PID:3046899] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "mochaa",
            "project_name": "protobuf",
            "project_dirname": "protobuf",
            "submitter": "https://src.fedoraproject.org/user/farchord",
            "ended_on": null,
            "started_on": 1767837677.8247101,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-42-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9983577-fedora-42-ppc64le",
            "build_id": 9983577,
            "package_name": "marble",
            "package_version": "1:25.12.1-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/mochaa/protobuf/marble",
            "git_hash": "5d68522d948b74b93e5c518c9c135131e589d80b",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "mochaa/protobuf--https://src.fedoraproject.org/user/farchord",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/mochaa/protobuf/fedora-42-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/mochaa/protobuf",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/mochaa/protobuf",
            "result_dir": "09983577-marble",
            "built_packages": "",
            "id": 9983577,
            "mockchain_macros": {
                "copr_username": "mochaa",
                "copr_projectname": "protobuf",
                "vendor": "Fedora Project COPR (mochaa/protobuf)"
            }
        }
    ]
}
[2026-01-08 02:01:17,902][  INFO][PID:3046899] Sending fedora-messaging bus message in build.start
[2026-01-08 02:01:18,453][  INFO][PID:3046899] Sending fedora-messaging bus message in chroot.start
[2026-01-08 02:01:18,480][  INFO][PID:3046899] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9983577-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached
[2026-01-08 02:01:19,451][  INFO][PID:3046899] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18d' (ED25519) to the list of known hosts.


[2026-01-08 02:01:19,452][  INFO][PID:3046899] Downloading the builder-live.log file, attempt 1
[2026-01-08 02:01:19,453][  INFO][PID:3046899] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18d copr-rpmbuild-log
[2026-01-08 02:01:24,460][  INFO][PID:3046899] Periodic builder liveness probe: alive
[2026-01-08 02:01:24,460][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:01:29,461][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:01:34,462][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:01:39,463][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:01:44,464][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:01:49,465][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:01:54,466][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:01:59,467][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:02:04,468][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:02:09,469][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:02:14,470][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:02:19,471][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:02:24,472][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:02:29,473][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:02:34,474][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:02:39,475][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:02:44,476][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:02:49,477][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:02:54,478][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:02:59,479][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:03:04,480][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:03:09,481][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:03:14,482][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:03:19,483][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:03:24,484][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:03:29,485][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:03:34,487][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:03:39,488][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:03:44,489][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:03:49,490][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:03:54,491][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:03:59,492][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:04:04,495][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:04:09,496][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:04:14,497][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:04:19,498][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:04:24,499][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:04:29,500][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:04:34,501][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:04:39,502][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:04:44,503][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:04:49,504][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:04:54,505][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:04:59,506][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:05:04,507][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:05:09,508][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:05:14,509][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:05:19,510][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:05:24,511][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:05:29,512][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:05:34,513][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:05:39,514][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:05:44,515][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:05:49,516][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:05:54,517][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:05:59,518][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:06:04,519][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:06:09,520][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:06:14,521][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:06:19,522][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:06:24,522][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:06:29,524][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:06:34,525][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:06:39,526][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:06:44,526][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:06:49,527][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:06:54,528][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:06:59,529][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:07:04,530][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:07:09,531][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:07:14,533][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:07:19,534][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:07:24,535][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:07:29,536][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:07:34,546][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:07:39,547][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:07:44,551][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:07:49,552][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:07:54,553][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:07:59,555][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:08:04,266][  INFO][PID:3046899] Downloading results from builder
[2026-01-08 02:08:04,266][  INFO][PID:3046899] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18d]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble started
[2026-01-08 02:08:04,267][  INFO][PID:3046899] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18d]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble/ &> /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble/build-09983577.rsync.log
[2026-01-08 02:09:04,342][  INFO][PID:3046899] rsyncing finished.
[2026-01-08 02:09:04,395][ ERROR][PID:3046899] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-08 02:09:04,396][ ERROR][PID:3046899] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-08 02:09:14,397][  INFO][PID:3046899] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18d]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble started
[2026-01-08 02:09:14,398][  INFO][PID:3046899] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18d]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble/ &> /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble/build-09983577.rsync.log
[2026-01-08 02:10:14,513][  INFO][PID:3046899] rsyncing finished.
[2026-01-08 02:10:14,517][ ERROR][PID:3046899] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-08 02:10:14,518][ ERROR][PID:3046899] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-08 02:10:24,528][  INFO][PID:3046899] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18d]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble started
[2026-01-08 02:10:24,540][  INFO][PID:3046899] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18d]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble/ &> /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble/build-09983577.rsync.log
[2026-01-08 02:11:24,897][  INFO][PID:3046899] rsyncing finished.
[2026-01-08 02:11:24,898][ ERROR][PID:3046899] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-08 02:11:24,899][ ERROR][PID:3046899] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-08 02:11:34,903][ ERROR][PID:3046899] Re-try request for task on 'ResallocHost, ticket_id=702092, hostname=2620:52:6:1161:dead:beef:cafe:c18d, name=vmhost_p09_04_prod_01007374_20260108_015608, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 3 SSH attempts
[2026-01-08 02:11:34,905][  INFO][PID:3046899] Releasing VM back to pool
[2026-01-08 02:11:34,945][  INFO][PID:3046899] Retry #1 (on other host)
[2026-01-08 02:11:34,945][  INFO][PID:3046899] Checking that /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/repodata/repomd.xml exists
[2026-01-08 02:11:34,946][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:11:34,947][  INFO][PID:3046899] VM allocation process starts
[2026-01-08 02:11:34,984][  INFO][PID:3046899] Trying to allocate VM: ResallocHost, ticket_id=702449, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-08 02:11:39,986][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:11:45,004][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:11:50,030][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:11:55,038][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:12:00,039][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:12:03,235][  INFO][PID:3046899] Allocated host ResallocHost, ticket_id=702449, hostname=2620:52:6:1161:dead:beef:cafe:c18d, name=vmhost_p09_04_prod_01007374_20260108_015608, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-08 02:12:03,236][  INFO][PID:3046899] Allocating ssh connection to builder
[2026-01-08 02:12:03,237][  INFO][PID:3046899] Checking that builder machine is OK
[2026-01-08 02:13:03,338][ ERROR][PID:3046899] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c18d port 22: Connection timed out

[2026-01-08 02:13:13,346][ ERROR][PID:3046899] Re-try request for task on 'ResallocHost, ticket_id=702449, hostname=2620:52:6:1161:dead:beef:cafe:c18d, name=vmhost_p09_04_prod_01007374_20260108_015608, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-01-08 02:13:13,346][  INFO][PID:3046899] Releasing VM back to pool
[2026-01-08 02:13:13,417][  INFO][PID:3046899] Retry #2 (on other host)
[2026-01-08 02:13:13,421][  INFO][PID:3046899] Checking that /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/repodata/repomd.xml exists
[2026-01-08 02:13:13,426][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:13:13,433][  INFO][PID:3046899] VM allocation process starts
[2026-01-08 02:13:13,511][  INFO][PID:3046899] Trying to allocate VM: ResallocHost, ticket_id=702509, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-08 02:13:18,515][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:13:23,517][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:13:28,523][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:13:33,529][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:13:38,532][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:13:43,536][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:13:48,540][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:13:53,542][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:13:58,550][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:14:03,554][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:14:08,558][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:14:13,563][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:14:18,569][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:14:23,573][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:14:28,577][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:14:33,579][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:14:38,581][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:14:43,588][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:14:48,592][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:14:53,640][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:14:58,644][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:15:03,651][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:15:08,652][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:15:13,661][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:15:18,732][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:15:23,744][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:15:28,833][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:15:33,834][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:15:38,835][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:15:43,842][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:15:48,904][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:15:53,906][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:15:58,911][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:16:03,918][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:16:08,919][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:16:13,920][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:16:19,109][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:16:24,111][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:16:29,114][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:16:34,120][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:16:39,122][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:16:44,123][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:16:49,129][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:16:54,133][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:16:59,137][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:17:04,143][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:17:09,148][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:17:14,152][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:17:19,156][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:17:24,162][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:17:29,225][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:17:34,227][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:17:39,243][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:17:44,244][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:17:49,248][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:17:54,250][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:17:59,252][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:18:04,255][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:18:09,258][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:18:14,263][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:18:19,265][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:18:24,266][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:18:29,269][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:18:31,872][  INFO][PID:3046899] Allocated host ResallocHost, ticket_id=702509, hostname=2620:52:6:1161:dead:beef:cafe:c192, name=vmhost_p09_04_prod_01007599_20260108_021223, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-08 02:18:31,873][  INFO][PID:3046899] Allocating ssh connection to builder
[2026-01-08 02:18:31,873][  INFO][PID:3046899] Checking that builder machine is OK
[2026-01-08 02:18:32,831][  INFO][PID:3046899] Running remote command: copr-builder-ready fedora-42-ppc64le
[2026-01-08 02:18:33,429][  INFO][PID:3046899] Red Hat subscription not needed for fedora-42-ppc64le
Builder is ready to be used

[2026-01-08 02:18:33,430][  INFO][PID:3046899] Filling build.info file with builder info
[2026-01-08 02:18:33,549][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:18:33,556][  INFO][PID:3046899] Not re-notifying FE and msg buses for the new host.
[2026-01-08 02:18:33,557][  INFO][PID:3046899] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9983577-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached
[2026-01-08 02:18:34,565][  INFO][PID:3046899] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c192' (ED25519) to the list of known hosts.


[2026-01-08 02:18:34,566][  INFO][PID:3046899] Downloading the builder-live.log file, attempt 1
[2026-01-08 02:18:34,658][  INFO][PID:3046899] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c192 copr-rpmbuild-log
[2026-01-08 02:18:39,595][  INFO][PID:3046899] Periodic builder liveness probe: alive
[2026-01-08 02:18:39,596][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:18:44,599][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:18:49,604][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:18:54,615][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:18:59,620][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:19:04,630][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:19:09,631][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:19:14,634][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:19:19,635][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:19:24,636][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:19:29,639][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:19:34,649][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:19:39,652][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:19:44,660][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:19:49,662][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:19:54,671][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:19:59,676][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:20:04,679][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:20:09,686][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:20:14,698][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:20:19,702][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:20:24,708][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:20:29,709][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:20:34,711][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:20:39,714][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:20:44,727][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:20:49,729][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:20:54,739][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:20:59,747][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:21:04,749][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:21:09,762][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:21:14,764][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:21:19,771][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:21:24,774][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:21:29,790][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:21:34,794][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:21:39,807][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:21:44,814][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:21:49,822][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:21:54,825][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:21:59,827][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:22:04,831][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:22:09,833][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:22:14,836][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:22:19,840][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:22:24,852][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:22:29,908][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:22:34,910][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:22:39,911][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:22:44,912][  INFO][PID:3046899] Checking for cancel request
[2026-01-08 02:22:49,432][  INFO][PID:3046899] Downloading results from builder
[2026-01-08 02:22:49,436][  INFO][PID:3046899] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c192]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble started
[2026-01-08 02:22:49,440][  INFO][PID:3046899] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c192]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble/ &> /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble/build-09983577.rsync.log
[2026-01-08 02:22:51,608][  INFO][PID:3046899] rsyncing finished.
[2026-01-08 02:22:51,608][  INFO][PID:3046899] Releasing VM back to pool
[2026-01-08 02:22:51,632][  INFO][PID:3046899] Searching for 'success' file in resultdir
[2026-01-08 02:22:51,635][ ERROR][PID:3046899] Build failed: Backend process error: No success file => build failure
[2026-01-08 02:22:51,640][  INFO][PID:3046899] Finished build: id=9983577 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/mochaa/protobuf chroot=fedora-42-ppc64le 
[2026-01-08 02:22:51,648][ ERROR][PID:3046899] Backend process error: No success file => build failure
[2026-01-08 02:22:51,649][  INFO][PID:3046899] Worker failed build, took 258.09343695640564
[2026-01-08 02:22:51,650][  INFO][PID:3046899] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "mochaa",
            "project_name": "protobuf",
            "project_dirname": "protobuf",
            "submitter": "https://src.fedoraproject.org/user/farchord",
            "ended_on": 1767838971.6495044,
            "started_on": 1767838713.5560675,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-42-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9983577-fedora-42-ppc64le",
            "build_id": 9983577,
            "package_name": "marble",
            "package_version": "1:25.12.1-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/mochaa/protobuf/marble",
            "git_hash": "5d68522d948b74b93e5c518c9c135131e589d80b",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "mochaa/protobuf--https://src.fedoraproject.org/user/farchord",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/mochaa/protobuf/fedora-42-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/mochaa/protobuf",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/mochaa/protobuf",
            "result_dir": "09983577-marble",
            "built_packages": "",
            "id": 9983577,
            "mockchain_macros": {
                "copr_username": "mochaa",
                "copr_projectname": "protobuf",
                "vendor": "Fedora Project COPR (mochaa/protobuf)"
            }
        }
    ]
}
[2026-01-08 02:22:51,768][  INFO][PID:3046899] Sending fedora-messaging bus message in build.end
[2026-01-08 02:22:51,810][  INFO][PID:3046899] Compressing /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble/builder-live.log by gzip
[2026-01-08 02:22:51,812][  INFO][PID:3046899] Running command 'gzip /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble/builder-live.log' as PID 3245001
[2026-01-08 02:22:51,892][  INFO][PID:3046899] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble/builder-live.log)
[2026-01-08 02:22:51,893][  INFO][PID:3046899] Compressing /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble/backend.log by gzip
[2026-01-08 02:22:51,897][  INFO][PID:3046899] Running command 'gzip /var/lib/copr/public_html/results/mochaa/protobuf/fedora-42-ppc64le/09983577-marble/backend.log' as PID 3245017