[2025-09-19 11:24:07,601][  INFO][PID:714957] Marking build as starting
[2025-09-19 11:24:07,639][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:24:07,640][  INFO][PID:714957] VM allocation process starts
[2025-09-19 11:24:07,678][  INFO][PID:714957] Trying to allocate VM: ResallocHost, ticket_id=5142516, requested_tags=['copr_builder', 'arch_noarch']
[2025-09-19 11:24:12,681][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:24:17,683][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:24:22,684][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:24:27,686][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:24:32,687][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:24:37,693][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:24:42,698][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:24:47,699][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:24:52,700][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:24:57,702][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:25:02,704][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:25:07,706][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:25:12,707][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:25:15,817][  INFO][PID:714957] Allocated host ResallocHost, ticket_id=5142516, hostname=2620:52:3:1:dead:beef:cafe:c292, name=copr_p09_01_prod_06830626_20250919_111522, requested_tags=['copr_builder', 'arch_noarch']
[2025-09-19 11:25:15,818][  INFO][PID:714957] Allocating ssh connection to builder
[2025-09-19 11:25:15,821][  INFO][PID:714957] Checking that builder machine is OK
[2025-09-19 11:25:16,703][  INFO][PID:714957] Running remote command: copr-builder-ready srpm-builds
[2025-09-19 11:25:17,146][  INFO][PID:714957] Red Hat subscription not needed for srpm-builds
Builder is ready to be used

[2025-09-19 11:25:17,147][  INFO][PID:714957] Filling build.info file with builder info
[2025-09-19 11:25:17,148][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:25:17,149][  INFO][PID:714957] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "@python",
            "project_name": "python-in-pulp",
            "project_dirname": "python-in-pulp",
            "submitter": "https://src.fedoraproject.org/user/ksurma",
            "ended_on": null,
            "started_on": 1758281117.1492646,
            "submitted_on": null,
            "status": 3,
            "chroot": "srpm-builds",
            "arch": null,
            "buildroot_pkgs": null,
            "task_id": "9576951",
            "build_id": 9576951,
            "package_name": "python-cvss",
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 10,
            "source_json": "{\"clone_url\": \"https://src.fedoraproject.org/rpms/python-cvss\", \"distgit\": \"fedora\", \"committish\": \"827e67d0bcbec2c0ec6eeda252d2efc288a541f8\"}",
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": null,
            "sandbox": "@python/python-in-pulp--https://src.fedoraproject.org/user/ksurma",
            "results": null,
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": true,
            "destdir": "/var/lib/copr/public_html/results/@python/python-in-pulp",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@python/python-in-pulp",
            "result_dir": "09576951",
            "built_packages": "",
            "tags": [
                "arch_noarch"
            ],
            "id": 9576951,
            "mockchain_macros": {
                "copr_username": "@python",
                "copr_projectname": "python-in-pulp",
                "vendor": "Fedora Project COPR (@python/python-in-pulp)"
            }
        }
    ]
}
[2025-09-19 11:25:17,170][  INFO][PID:714957] Sending fedora-messaging bus message in build.start
[2025-09-19 11:25:18,039][  INFO][PID:714957] Sending fedora-messaging bus message in chroot.start
[2025-09-19 11:25:18,064][  INFO][PID:714957] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/9576951 --detached
[2025-09-19 11:25:18,849][  INFO][PID:714957] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c292' (ED25519) to the list of known hosts.


[2025-09-19 11:25:18,850][  INFO][PID:714957] Downloading the builder-live.log file, attempt 1
[2025-09-19 11:25:18,852][  INFO][PID:714957] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c292 copr-rpmbuild-log
[2025-09-19 11:25:23,860][  INFO][PID:714957] Periodic builder liveness probe: alive
[2025-09-19 11:25:23,861][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:25:28,863][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:25:33,864][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:25:38,865][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:25:43,866][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:25:48,871][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:25:53,873][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:25:58,874][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:26:03,878][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:26:08,880][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:26:13,886][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:26:18,888][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:26:23,890][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:26:28,894][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:26:33,898][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:26:38,906][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:26:43,907][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:26:48,908][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:26:53,910][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:26:58,913][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:27:03,918][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:27:08,920][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:27:13,921][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:27:18,923][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:27:19,140][ ERROR][PID:714957] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-19 11:27:23,927][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:27:28,929][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:27:29,144][WARNING][PID:714957] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-19 11:27:33,931][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:27:38,936][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:27:39,145][  INFO][PID:714957] Downloading the builder-live.log file, attempt 2
[2025-09-19 11:27:39,150][  INFO][PID:714957] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c292 copr-rpmbuild-log
[2025-09-19 11:27:43,943][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:27:48,945][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:27:53,955][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:27:58,956][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:28:03,958][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:28:08,960][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:28:13,969][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:28:18,971][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:28:23,973][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:28:28,992][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:28:33,994][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:28:38,998][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:28:39,234][ ERROR][PID:714957] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-19 11:28:44,000][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:28:49,002][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:28:49,236][WARNING][PID:714957] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-19 11:28:54,006][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:28:59,007][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:28:59,240][  INFO][PID:714957] Downloading the builder-live.log file, attempt 3
[2025-09-19 11:28:59,241][  INFO][PID:714957] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c292 copr-rpmbuild-log
[2025-09-19 11:29:04,009][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:29:09,011][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:29:14,016][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:29:19,019][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:29:24,020][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:29:29,021][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:29:34,022][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:29:39,024][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:29:44,026][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:29:49,027][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:29:54,029][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:29:59,030][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:29:59,311][ ERROR][PID:714957] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-19 11:30:04,032][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:30:09,035][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:30:09,314][WARNING][PID:714957] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-19 11:30:14,037][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:30:19,038][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:30:19,320][  INFO][PID:714957] Downloading the builder-live.log file, attempt 4
[2025-09-19 11:30:19,321][  INFO][PID:714957] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c292 copr-rpmbuild-log
[2025-09-19 11:30:24,040][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:30:29,043][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:30:34,047][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:30:39,048][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:30:44,049][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:30:49,053][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:30:54,054][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:30:59,056][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:31:04,058][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:31:09,059][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:31:14,062][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:31:19,064][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:31:19,406][ ERROR][PID:714957] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-19 11:31:24,077][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:31:29,079][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:31:29,407][ ERROR][PID:714957] Giving up for unstable SSH, failures: 1758281249.144305, 1758281329.2368417, 1758281409.314262, 1758281489.4073915
[2025-09-19 11:31:29,412][ ERROR][PID:714957] Re-try request for task on 'ResallocHost, ticket_id=5142516, hostname=2620:52:3:1:dead:beef:cafe:c292, name=copr_p09_01_prod_06830626_20250919_111522, requested_tags=['copr_builder', 'arch_noarch']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-09-19 11:31:29,413][  INFO][PID:714957] Releasing VM back to pool
[2025-09-19 11:31:29,441][  INFO][PID:714957] Retry #1 (on other host)
[2025-09-19 11:31:29,443][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:31:29,445][  INFO][PID:714957] VM allocation process starts
[2025-09-19 11:31:29,486][  INFO][PID:714957] Trying to allocate VM: ResallocHost, ticket_id=5143027, requested_tags=['copr_builder', 'arch_noarch']
[2025-09-19 11:31:34,492][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:31:39,493][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:31:44,497][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:31:49,498][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:31:54,501][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:31:59,503][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:32:04,506][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:32:09,510][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:32:14,511][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:32:19,516][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:32:24,518][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:32:29,519][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:32:34,522][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:32:37,611][  INFO][PID:714957] Allocated host ResallocHost, ticket_id=5143027, hostname=2620:52:3:1:dead:beef:cafe:c1d7, name=copr_hv_x86_64_04_prod_06830422_20250919_110519, requested_tags=['copr_builder', 'arch_noarch']
[2025-09-19 11:32:37,619][  INFO][PID:714957] Allocating ssh connection to builder
[2025-09-19 11:32:37,632][  INFO][PID:714957] Checking that builder machine is OK
[2025-09-19 11:32:38,238][  INFO][PID:714957] Running remote command: copr-builder-ready srpm-builds
[2025-09-19 11:32:38,563][  INFO][PID:714957] Red Hat subscription not needed for srpm-builds
Builder is ready to be used

[2025-09-19 11:32:38,563][  INFO][PID:714957] Filling build.info file with builder info
[2025-09-19 11:32:38,567][  INFO][PID:714957] Checking for cancel request
[2025-09-19 11:32:38,567][  INFO][PID:714957] Not re-notifying FE and msg buses for the new host.
[2025-09-19 11:32:38,568][  INFO][PID:714957] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/9576951 --detached
[2025-09-19 11:32:39,122][  INFO][PID:714957] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c1d7' (ED25519) to the list of known hosts.


[2025-09-19 11:32:39,123][  INFO][PID:714957] Downloading the builder-live.log file, attempt 1
[2025-09-19 11:32:39,125][  INFO][PID:714957] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c1d7 copr-rpmbuild-log
[2025-09-19 11:32:41,913][  INFO][PID:714957] Downloading results from builder
[2025-09-19 11:32:41,914][  INFO][PID:714957] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c1d7]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@python/python-in-pulp/srpm-builds/09576951 started
[2025-09-19 11:32:41,915][  INFO][PID:714957] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c1d7]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@python/python-in-pulp/srpm-builds/09576951/ &> /var/lib/copr/public_html/results/@python/python-in-pulp/srpm-builds/09576951/build-09576951.rsync.log
[2025-09-19 11:32:42,506][  INFO][PID:714957] rsyncing finished.
[2025-09-19 11:32:42,509][  INFO][PID:714957] Releasing VM back to pool
[2025-09-19 11:32:42,529][  INFO][PID:714957] Searching for 'success' file in resultdir
[2025-09-19 11:32:42,530][  INFO][PID:714957] Getting build details
[2025-09-19 11:32:42,530][  INFO][PID:714957] Retrieving SRPM info from /var/lib/copr/public_html/results/@python/python-in-pulp/srpm-builds/09576951
[2025-09-19 11:32:42,533][  INFO][PID:714957] SRPM URL: https://download.copr.fedorainfracloud.org/results/@python/python-in-pulp/srpm-builds/09576951/python-cvss-3.3-5.src.rpm
[2025-09-19 11:32:42,535][  INFO][PID:714957] build details: {'srpm_url': 'https://download.copr.fedorainfracloud.org/results/@python/python-in-pulp/srpm-builds/09576951/python-cvss-3.3-5.src.rpm', 'pkg_name': 'python-cvss', 'pkg_version': '3.3-5'}
[2025-09-19 11:32:42,537][  INFO][PID:714957] Finished build: id=9576951 failed=False timeout=108000 destdir=/var/lib/copr/public_html/results/@python/python-in-pulp chroot=srpm-builds 
[2025-09-19 11:32:42,541][  INFO][PID:714957] Worker succeeded build, took 3.973109483718872
[2025-09-19 11:32:42,544][  INFO][PID:714957] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "@python",
            "project_name": "python-in-pulp",
            "project_dirname": "python-in-pulp",
            "submitter": "https://src.fedoraproject.org/user/ksurma",
            "ended_on": 1758281562.540923,
            "started_on": 1758281558.5678136,
            "submitted_on": null,
            "status": 1,
            "chroot": "srpm-builds",
            "arch": null,
            "buildroot_pkgs": null,
            "task_id": "9576951",
            "build_id": 9576951,
            "package_name": "python-cvss",
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 10,
            "source_json": "{\"clone_url\": \"https://src.fedoraproject.org/rpms/python-cvss\", \"distgit\": \"fedora\", \"committish\": \"827e67d0bcbec2c0ec6eeda252d2efc288a541f8\"}",
            "pkg_name": "python-cvss",
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": "https://download.copr.fedorainfracloud.org/results/@python/python-in-pulp/srpm-builds/09576951/python-cvss-3.3-5.src.rpm",
            "uses_devel_repo": null,
            "sandbox": "@python/python-in-pulp--https://src.fedoraproject.org/user/ksurma",
            "results": {
                "name": "python-cvss",
                "epoch": null,
                "version": "3.3",
                "release": "5",
                "exclusivearch": [],
                "excludearch": []
            },
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": true,
            "destdir": "/var/lib/copr/public_html/results/@python/python-in-pulp",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@python/python-in-pulp",
            "result_dir": "09576951",
            "built_packages": "",
            "tags": [
                "arch_noarch"
            ],
            "pkg_version": "3.3-5",
            "id": 9576951,
            "mockchain_macros": {
                "copr_username": "@python",
                "copr_projectname": "python-in-pulp",
                "vendor": "Fedora Project COPR (@python/python-in-pulp)"
            }
        }
    ]
}
[2025-09-19 11:32:42,583][  INFO][PID:714957] Sending fedora-messaging bus message in build.end
[2025-09-19 11:32:42,636][  INFO][PID:714957] Compressing /var/lib/copr/public_html/results/@python/python-in-pulp/srpm-builds/09576951/builder-live.log by gzip
[2025-09-19 11:32:42,641][  INFO][PID:714957] Running command 'gzip /var/lib/copr/public_html/results/@python/python-in-pulp/srpm-builds/09576951/builder-live.log' as PID 826017
[2025-09-19 11:32:42,650][  INFO][PID:714957] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@python/python-in-pulp/srpm-builds/09576951/builder-live.log)
[2025-09-19 11:32:42,651][  INFO][PID:714957] Compressing /var/lib/copr/public_html/results/@python/python-in-pulp/srpm-builds/09576951/backend.log by gzip
[2025-09-19 11:32:42,655][  INFO][PID:714957] Running command 'gzip /var/lib/copr/public_html/results/@python/python-in-pulp/srpm-builds/09576951/backend.log' as PID 826018