[2025-06-12 14:16:32,232][  INFO][PID:2406235] Marking build as starting
[2025-06-12 14:16:32,343][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:16:32,368][  INFO][PID:2406235] VM allocation process starts
[2025-06-12 14:16:32,442][  INFO][PID:2406235] Trying to allocate VM: ResallocHost, ticket_id=3463220, requested_tags=['copr_builder', 'arch_x86_64']
[2025-06-12 14:16:37,466][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:16:42,482][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:16:44,763][  INFO][PID:2406235] Allocated host ResallocHost, ticket_id=3463220, hostname=54.162.219.95, name=aws_x86_64_spot_prod_04770246_20250612_140659, requested_tags=['copr_builder', 'arch_x86_64']
[2025-06-12 14:16:44,772][  INFO][PID:2406235] Allocating ssh connection to builder
[2025-06-12 14:16:44,781][  INFO][PID:2406235] Checking that builder machine is OK
[2025-06-12 14:16:45,372][  INFO][PID:2406235] Running remote command: copr-builder-ready epel-9-x86_64
[2025-06-12 14:16:45,742][  INFO][PID:2406235] Checking Red Hat subscription (attempt #1)
Red Hat subscription active
Builder is ready to be used

[2025-06-12 14:16:45,750][  INFO][PID:2406235] Filling build.info file with builder info
[2025-06-12 14:16:45,754][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:16:45,791][  INFO][PID:2406235] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@copr",
            "project_name": "PyPI",
            "project_dirname": "PyPI",
            "submitter": "ksurma",
            "ended_on": null,
            "started_on": 1749737805.7907245,
            "submitted_on": null,
            "status": 3,
            "chroot": "epel-9-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "9156323-epel-9-x86_64",
            "build_id": 9156323,
            "package_name": "python-cybrid-api-id-python",
            "package_version": "0.123.578-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@copr/PyPI/python-cybrid-api-id-python",
            "git_hash": "e3239e6c2dbd16b484d21bbb209af288168b8b0c",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@copr/PyPI--ksurma",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI/epel-9-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@copr/PyPI",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI",
            "result_dir": "09156323-python-cybrid-api-id-python",
            "built_packages": "",
            "id": 9156323,
            "mockchain_macros": {
                "copr_username": "@copr",
                "copr_projectname": "PyPI",
                "vendor": "Fedora Project COPR (@copr/PyPI)"
            }
        }
    ]
}
[2025-06-12 14:16:45,869][  INFO][PID:2406235] Sending fedora-messaging bus message in build.start
[2025-06-12 14:16:53,425][  INFO][PID:2406235] Sending fedora-messaging bus message in chroot.start
[2025-06-12 14:16:53,544][  INFO][PID:2406235] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9156323-epel-9-x86_64 --chroot epel-9-x86_64 --detached
[2025-06-12 14:16:54,084][  INFO][PID:2406235] Downloading the builder-live.log file, attempt 1
[2025-06-12 14:16:54,096][  INFO][PID:2406235] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.162.219.95 copr-rpmbuild-log
[2025-06-12 14:16:58,943][ ERROR][PID:2406235] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-06-12 14:16:59,133][  INFO][PID:2406235] Periodic builder liveness probe: alive
[2025-06-12 14:16:59,145][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:17:04,158][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:17:08,953][WARNING][PID:2406235] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-06-12 14:17:09,170][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:17:14,186][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:17:18,977][  INFO][PID:2406235] Downloading the builder-live.log file, attempt 2
[2025-06-12 14:17:18,992][  INFO][PID:2406235] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.162.219.95 copr-rpmbuild-log
[2025-06-12 14:17:19,211][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:17:24,229][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:17:29,243][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:17:34,248][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:17:39,271][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:17:44,305][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:17:49,311][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:17:54,317][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:17:59,333][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:18:04,350][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:18:09,357][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:18:14,376][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:18:19,170][ ERROR][PID:2406235] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-06-12 14:18:19,390][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:18:24,401][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:18:29,183][WARNING][PID:2406235] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-06-12 14:18:29,408][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:18:34,418][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:18:39,197][  INFO][PID:2406235] Downloading the builder-live.log file, attempt 3
[2025-06-12 14:18:39,208][  INFO][PID:2406235] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.162.219.95 copr-rpmbuild-log
[2025-06-12 14:18:39,429][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:18:44,448][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:18:49,461][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:18:54,466][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:18:59,502][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:19:04,514][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:19:09,524][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:19:14,531][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:19:19,561][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:19:24,570][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:19:29,577][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:19:34,587][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:19:39,301][ ERROR][PID:2406235] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-06-12 14:19:39,600][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:19:44,612][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:19:49,315][WARNING][PID:2406235] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-06-12 14:19:49,624][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:19:54,639][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:19:59,320][  INFO][PID:2406235] Downloading the builder-live.log file, attempt 4
[2025-06-12 14:19:59,325][  INFO][PID:2406235] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.162.219.95 copr-rpmbuild-log
[2025-06-12 14:19:59,665][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:20:04,672][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:20:09,677][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:20:14,684][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:20:19,701][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:20:24,716][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:20:29,724][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:20:34,736][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:20:39,746][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:20:44,757][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:20:49,764][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:20:54,775][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:20:59,440][ ERROR][PID:2406235] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-06-12 14:20:59,788][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:21:04,797][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:21:09,451][ ERROR][PID:2406235] Giving up for unstable SSH, failures: 1749737828.9531932, 1749737909.1834085, 1749737989.3154845, 1749738069.4514475
[2025-06-12 14:21:09,457][ ERROR][PID:2406235] Re-try request for task on 'ResallocHost, ticket_id=3463220, hostname=54.162.219.95, name=aws_x86_64_spot_prod_04770246_20250612_140659, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-06-12 14:21:09,459][  INFO][PID:2406235] Releasing VM back to pool
[2025-06-12 14:21:09,507][  INFO][PID:2406235] Retry #1 (on other host)
[2025-06-12 14:21:09,520][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:21:09,525][  INFO][PID:2406235] VM allocation process starts
[2025-06-12 14:21:09,565][  INFO][PID:2406235] Trying to allocate VM: ResallocHost, ticket_id=3463318, requested_tags=['copr_builder', 'arch_x86_64']
[2025-06-12 14:21:12,615][  INFO][PID:2406235] Allocated host ResallocHost, ticket_id=3463318, hostname=34.228.30.212, name=aws_x86_64_normalreserved_prod_04770287_20250612_140850, requested_tags=['copr_builder', 'arch_x86_64']
[2025-06-12 14:21:12,623][  INFO][PID:2406235] Allocating ssh connection to builder
[2025-06-12 14:21:12,625][  INFO][PID:2406235] Checking that builder machine is OK
[2025-06-12 14:21:12,983][  INFO][PID:2406235] Running remote command: copr-builder-ready epel-9-x86_64
[2025-06-12 14:21:13,245][  INFO][PID:2406235] Checking Red Hat subscription (attempt #1)
Red Hat subscription active
Builder is ready to be used

[2025-06-12 14:21:13,250][  INFO][PID:2406235] Filling build.info file with builder info
[2025-06-12 14:21:13,255][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:21:13,263][  INFO][PID:2406235] Not re-notifying FE and msg buses for the new host.
[2025-06-12 14:21:13,272][  INFO][PID:2406235] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9156323-epel-9-x86_64 --chroot epel-9-x86_64 --detached
[2025-06-12 14:21:13,687][  INFO][PID:2406235] Downloading the builder-live.log file, attempt 1
[2025-06-12 14:21:13,693][  INFO][PID:2406235] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.228.30.212 copr-rpmbuild-log
[2025-06-12 14:21:18,689][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:21:23,702][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:21:28,710][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:21:33,718][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:21:38,732][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:21:43,737][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:21:48,743][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:21:53,754][  INFO][PID:2406235] Checking for cancel request
[2025-06-12 14:21:56,355][  INFO][PID:2406235] Downloading results from builder
[2025-06-12 14:21:56,361][  INFO][PID:2406235] rsyncing of mockbuilder@34.228.30.212:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64/09156323-python-cybrid-api-id-python started
[2025-06-12 14:21:56,368][  INFO][PID:2406235] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@34.228.30.212:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64/09156323-python-cybrid-api-id-python/ &> /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64/09156323-python-cybrid-api-id-python/build-09156323.rsync.log
[2025-06-12 14:21:56,716][  INFO][PID:2406235] rsyncing finished.
[2025-06-12 14:21:56,719][  INFO][PID:2406235] Releasing VM back to pool
[2025-06-12 14:21:56,758][  INFO][PID:2406235] Searching for 'success' file in resultdir
[2025-06-12 14:21:56,766][  INFO][PID:2406235] Going to sign pkgs from source: 9156323-epel-9-x86_64 in chroot: /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64
[2025-06-12 14:21:56,770][  INFO][PID:2406235] Calling '/bin/sign -u @copr#PyPI@copr.fedorahosted.org -p' (attempt #1)
[2025-06-12 14:21:57,109][  INFO][PID:2406235] Calling '/bin/sign -4 -h sha256 -u @copr#PyPI@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64/09156323-python-cybrid-api-id-python/python3-cybrid-api-id-python-0.123.578-1.el9.noarch.rpm' (attempt #1)
[2025-06-12 14:21:58,001][  INFO][PID:2406235] signed rpm: /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64/09156323-python-cybrid-api-id-python/python3-cybrid-api-id-python-0.123.578-1.el9.noarch.rpm
[2025-06-12 14:21:58,008][  INFO][PID:2406235] Calling '/bin/sign -4 -h sha256 -u @copr#PyPI@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64/09156323-python-cybrid-api-id-python/python-cybrid-api-id-python-0.123.578-1.el9.src.rpm' (attempt #1)
[2025-06-12 14:21:58,894][  INFO][PID:2406235] signed rpm: /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64/09156323-python-cybrid-api-id-python/python-cybrid-api-id-python-0.123.578-1.el9.src.rpm
[2025-06-12 14:21:58,896][  INFO][PID:2406235] Sign done
[2025-06-12 14:21:58,899][  INFO][PID:2406235] Incremental createrepo run, adding 09156323-python-cybrid-api-id-python into https://download.copr.fedorainfracloud.org/results/@copr/PyPI/epel-9-x86_64, (auto-create-repo=True)
[2025-06-12 14:21:58,913][  INFO][PID:2406235] Running command 'copr-repo --batched /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64 --add 09156323-python-cybrid-api-id-python --no-appstream-metadata' as PID 2461878
[2025-06-12 14:24:57,875][  INFO][PID:2406235] Finished after 178 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64 --add 09156323-python-cybrid-api-id-python --no-appstream-metadata)
[2025-06-12 14:24:57,878][  INFO][PID:2406235] Getting build details
[2025-06-12 14:24:57,879][  INFO][PID:2406235] Listing built binary packages in /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64/09156323-python-cybrid-api-id-python
[2025-06-12 14:24:57,883][  INFO][PID:2406235] Built packages:
['python3-cybrid-api-id-python 0.123.578']
[2025-06-12 14:24:57,886][  INFO][PID:2406235] build details: {'built_packages': 'python3-cybrid-api-id-python 0.123.578'}
[2025-06-12 14:24:57,888][  INFO][PID:2406235] Retrieving pubkey
[2025-06-12 14:24:57,895][  INFO][PID:2406235] Calling '/bin/sign -u @copr#PyPI@copr.fedorahosted.org -p' (attempt #1)
[2025-06-12 14:24:58,256][  INFO][PID:2406235] Added pubkey for user @copr project PyPI into: /var/lib/copr/public_html/results/@copr/PyPI/pubkey.gpg
[2025-06-12 14:24:58,259][  INFO][PID:2406235] Finished build: id=9156323 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@copr/PyPI chroot=epel-9-x86_64 
[2025-06-12 14:24:58,287][  INFO][PID:2406235] Worker succeeded build, took 225.02315759658813
[2025-06-12 14:24:58,298][  INFO][PID:2406235] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@copr",
            "project_name": "PyPI",
            "project_dirname": "PyPI",
            "submitter": "ksurma",
            "ended_on": 1749738298.2871075,
            "started_on": 1749738073.2639499,
            "submitted_on": null,
            "status": 1,
            "chroot": "epel-9-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "9156323-epel-9-x86_64",
            "build_id": 9156323,
            "package_name": "python-cybrid-api-id-python",
            "package_version": "0.123.578-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@copr/PyPI/python-cybrid-api-id-python",
            "git_hash": "e3239e6c2dbd16b484d21bbb209af288168b8b0c",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@copr/PyPI--ksurma",
            "results": {
                "packages": [
                    {
                        "name": "python-cybrid-api-id-python",
                        "epoch": null,
                        "version": "0.123.578",
                        "release": "1.el9",
                        "arch": "src"
                    },
                    {
                        "name": "python3-cybrid-api-id-python",
                        "epoch": null,
                        "version": "0.123.578",
                        "release": "1.el9",
                        "arch": "noarch"
                    }
                ]
            },
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI/epel-9-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@copr/PyPI",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI",
            "result_dir": "09156323-python-cybrid-api-id-python",
            "built_packages": "python3-cybrid-api-id-python 0.123.578",
            "id": 9156323,
            "mockchain_macros": {
                "copr_username": "@copr",
                "copr_projectname": "PyPI",
                "vendor": "Fedora Project COPR (@copr/PyPI)"
            }
        }
    ]
}
[2025-06-12 14:24:58,355][  INFO][PID:2406235] Sending fedora-messaging bus message in build.end
[2025-06-12 14:24:58,433][  INFO][PID:2406235] Compressing /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64/09156323-python-cybrid-api-id-python/builder-live.log by gzip
[2025-06-12 14:24:58,442][  INFO][PID:2406235] Running command 'gzip /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64/09156323-python-cybrid-api-id-python/builder-live.log' as PID 2492575
[2025-06-12 14:24:58,470][  INFO][PID:2406235] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64/09156323-python-cybrid-api-id-python/builder-live.log)
[2025-06-12 14:24:58,475][  INFO][PID:2406235] Compressing /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64/09156323-python-cybrid-api-id-python/backend.log by gzip
[2025-06-12 14:24:58,482][  INFO][PID:2406235] Running command 'gzip /var/lib/copr/public_html/results/@copr/PyPI/epel-9-x86_64/09156323-python-cybrid-api-id-python/backend.log' as PID 2492592