[2025-07-11 12:06:35,486][  INFO][PID:1647702] Marking build as starting
[2025-07-11 12:06:35,518][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:06:35,522][  INFO][PID:1647702] VM allocation process starts
[2025-07-11 12:06:35,558][  INFO][PID:1647702] Trying to allocate VM: ResallocHost, ticket_id=3893741, requested_tags=['copr_builder', 'arch_noarch']
[2025-07-11 12:06:40,565][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:06:41,735][  INFO][PID:1647702] Allocated host ResallocHost, ticket_id=3893741, hostname=2620:52:3:1:dead:beef:cafe:c29a, name=copr_p09_01_prod_05287322_20250711_114600, requested_tags=['copr_builder', 'arch_noarch']
[2025-07-11 12:06:41,739][  INFO][PID:1647702] Allocating ssh connection to builder
[2025-07-11 12:06:41,740][  INFO][PID:1647702] Checking that builder machine is OK
[2025-07-11 12:06:42,808][  INFO][PID:1647702] Running remote command: copr-builder-ready srpm-builds
[2025-07-11 12:06:43,368][  INFO][PID:1647702] Red Hat subscription not needed for srpm-builds
Builder is ready to be used

[2025-07-11 12:06:43,375][  INFO][PID:1647702] Filling build.info file with builder info
[2025-07-11 12:06:43,376][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:06:43,386][  INFO][PID:1647702] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "@copr",
            "project_name": "PyPI",
            "project_dirname": "PyPI",
            "submitter": "ksurma",
            "ended_on": null,
            "started_on": 1752235603.3860645,
            "submitted_on": null,
            "status": 3,
            "chroot": "srpm-builds",
            "arch": null,
            "buildroot_pkgs": null,
            "task_id": "9266464",
            "build_id": 9266464,
            "package_name": "python-deeploy",
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 5,
            "source_json": "{\"pypi_package_name\": \"deeploy\", \"pypi_package_version\": \"1.50.0\", \"spec_generator\": \"pyp2spec\", \"spec_template\": \"\", \"python_versions\": []}",
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": null,
            "sandbox": "@copr/PyPI--ksurma",
            "results": null,
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": true,
            "destdir": "/var/lib/copr/public_html/results/@copr/PyPI",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI",
            "result_dir": "09266464",
            "built_packages": "",
            "tags": [
                "arch_noarch"
            ],
            "id": 9266464,
            "mockchain_macros": {
                "copr_username": "@copr",
                "copr_projectname": "PyPI",
                "vendor": "Fedora Project COPR (@copr/PyPI)"
            }
        }
    ]
}
[2025-07-11 12:06:43,433][  INFO][PID:1647702] Sending fedora-messaging bus message in build.start
[2025-07-11 12:06:46,123][  INFO][PID:1647702] Sending fedora-messaging bus message in chroot.start
[2025-07-11 12:06:46,180][  INFO][PID:1647702] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/9266464 --detached
[2025-07-11 12:06:47,140][  INFO][PID:1647702] Downloading the builder-live.log file, attempt 1
[2025-07-11 12:06:47,145][  INFO][PID:1647702] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c29a copr-rpmbuild-log
[2025-07-11 12:06:52,157][  INFO][PID:1647702] Periodic builder liveness probe: alive
[2025-07-11 12:06:52,158][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:06:57,163][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:07:02,164][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:07:07,166][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:07:12,167][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:07:17,171][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:07:22,172][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:07:27,177][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:07:32,179][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:07:37,184][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:07:42,192][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:07:47,198][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:07:52,200][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:07:57,202][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:08:02,205][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:08:07,209][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:08:12,210][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:08:17,212][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:08:22,217][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:08:27,218][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:08:32,220][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:08:37,221][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:08:42,222][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:08:47,224][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:08:48,139][ ERROR][PID:1647702] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-11 12:08:52,225][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:08:57,226][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:08:58,140][WARNING][PID:1647702] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-11 12:09:02,227][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:09:07,229][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:09:08,141][  INFO][PID:1647702] Downloading the builder-live.log file, attempt 2
[2025-07-11 12:09:08,142][  INFO][PID:1647702] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c29a copr-rpmbuild-log
[2025-07-11 12:09:12,230][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:09:17,231][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:09:22,232][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:09:27,235][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:09:32,237][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:09:37,238][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:09:42,242][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:09:47,245][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:09:52,246][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:09:57,247][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:10:02,248][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:10:07,249][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:10:08,210][ ERROR][PID:1647702] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-11 12:10:12,250][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:10:17,251][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:10:18,214][WARNING][PID:1647702] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-11 12:10:22,253][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:10:27,255][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:10:28,215][  INFO][PID:1647702] Downloading the builder-live.log file, attempt 3
[2025-07-11 12:10:28,217][  INFO][PID:1647702] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c29a copr-rpmbuild-log
[2025-07-11 12:10:32,257][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:10:37,258][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:10:42,259][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:10:47,261][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:10:52,264][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:10:57,269][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:11:02,270][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:11:07,271][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:11:12,272][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:11:17,273][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:11:22,276][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:11:27,277][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:11:28,285][ ERROR][PID:1647702] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-11 12:11:32,278][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:11:37,281][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:11:38,289][WARNING][PID:1647702] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-11 12:11:42,283][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:11:47,284][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:11:48,290][  INFO][PID:1647702] Downloading the builder-live.log file, attempt 4
[2025-07-11 12:11:48,291][  INFO][PID:1647702] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c29a copr-rpmbuild-log
[2025-07-11 12:11:52,287][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:11:57,292][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:02,293][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:07,295][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:12,296][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:17,297][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:22,299][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:27,300][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:32,301][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:37,302][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:42,304][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:47,305][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:48,361][ ERROR][PID:1647702] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-11 12:12:52,307][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:57,311][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:58,367][ ERROR][PID:1647702] Giving up for unstable SSH, failures: 1752235738.1403146, 1752235818.2140756, 1752235898.2894473, 1752235978.3673732
[2025-07-11 12:12:58,370][ ERROR][PID:1647702] Re-try request for task on 'ResallocHost, ticket_id=3893741, hostname=2620:52:3:1:dead:beef:cafe:c29a, name=copr_p09_01_prod_05287322_20250711_114600, requested_tags=['copr_builder', 'arch_noarch']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-07-11 12:12:58,371][  INFO][PID:1647702] Releasing VM back to pool
[2025-07-11 12:12:58,398][  INFO][PID:1647702] Retry #1 (on other host)
[2025-07-11 12:12:58,400][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:12:58,402][  INFO][PID:1647702] VM allocation process starts
[2025-07-11 12:12:58,421][  INFO][PID:1647702] Trying to allocate VM: ResallocHost, ticket_id=3893879, requested_tags=['copr_builder', 'arch_noarch']
[2025-07-11 12:13:01,450][  INFO][PID:1647702] Allocated host ResallocHost, ticket_id=3893879, hostname=44.200.201.189, name=aws_x86_64_normalreserved_prod_05287661_20250711_120407, requested_tags=['copr_builder', 'arch_noarch']
[2025-07-11 12:13:01,452][  INFO][PID:1647702] Allocating ssh connection to builder
[2025-07-11 12:13:01,452][  INFO][PID:1647702] Checking that builder machine is OK
[2025-07-11 12:13:01,900][  INFO][PID:1647702] Running remote command: copr-builder-ready srpm-builds
[2025-07-11 12:13:02,119][  INFO][PID:1647702] Red Hat subscription not needed for srpm-builds
Builder is ready to be used

[2025-07-11 12:13:02,119][  INFO][PID:1647702] Filling build.info file with builder info
[2025-07-11 12:13:02,127][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:13:02,129][  INFO][PID:1647702] Not re-notifying FE and msg buses for the new host.
[2025-07-11 12:13:02,131][  INFO][PID:1647702] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/9266464 --detached
[2025-07-11 12:13:02,478][  INFO][PID:1647702] Downloading the builder-live.log file, attempt 1
[2025-07-11 12:13:02,483][  INFO][PID:1647702] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.200.201.189 copr-rpmbuild-log
[2025-07-11 12:13:07,478][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:13:12,481][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:13:17,484][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:13:22,492][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:13:27,494][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:13:32,501][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:13:37,504][  INFO][PID:1647702] Checking for cancel request
[2025-07-11 12:13:37,636][  INFO][PID:1647702] Downloading results from builder
[2025-07-11 12:13:37,639][  INFO][PID:1647702] rsyncing of mockbuilder@44.200.201.189:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/09266464 started
[2025-07-11 12:13:37,646][  INFO][PID:1647702] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@44.200.201.189:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/09266464/ &> /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/09266464/build-09266464.rsync.log
[2025-07-11 12:13:37,908][  INFO][PID:1647702] rsyncing finished.
[2025-07-11 12:13:37,908][  INFO][PID:1647702] Releasing VM back to pool
[2025-07-11 12:13:37,935][  INFO][PID:1647702] Searching for 'success' file in resultdir
[2025-07-11 12:13:37,936][  INFO][PID:1647702] Getting build details
[2025-07-11 12:13:37,940][  INFO][PID:1647702] Retrieving SRPM info from /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/09266464
[2025-07-11 12:13:37,941][  INFO][PID:1647702] SRPM URL: https://download.copr.fedorainfracloud.org/results/@copr/PyPI/srpm-builds/09266464/python-deeploy-1.50.0-1.src.rpm
[2025-07-11 12:13:37,944][  INFO][PID:1647702] build details: {'srpm_url': 'https://download.copr.fedorainfracloud.org/results/@copr/PyPI/srpm-builds/09266464/python-deeploy-1.50.0-1.src.rpm', 'pkg_name': 'python-deeploy', 'pkg_version': '1.50.0-1'}
[2025-07-11 12:13:37,946][  INFO][PID:1647702] Finished build: id=9266464 failed=False timeout=108000 destdir=/var/lib/copr/public_html/results/@copr/PyPI chroot=srpm-builds 
[2025-07-11 12:13:37,952][  INFO][PID:1647702] Worker succeeded build, took 35.82245659828186
[2025-07-11 12:13:37,954][  INFO][PID:1647702] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "@copr",
            "project_name": "PyPI",
            "project_dirname": "PyPI",
            "submitter": "ksurma",
            "ended_on": 1752236017.9523878,
            "started_on": 1752235982.1299312,
            "submitted_on": null,
            "status": 1,
            "chroot": "srpm-builds",
            "arch": null,
            "buildroot_pkgs": null,
            "task_id": "9266464",
            "build_id": 9266464,
            "package_name": "python-deeploy",
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 5,
            "source_json": "{\"pypi_package_name\": \"deeploy\", \"pypi_package_version\": \"1.50.0\", \"spec_generator\": \"pyp2spec\", \"spec_template\": \"\", \"python_versions\": []}",
            "pkg_name": "python-deeploy",
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI/srpm-builds/09266464/python-deeploy-1.50.0-1.src.rpm",
            "uses_devel_repo": null,
            "sandbox": "@copr/PyPI--ksurma",
            "results": {
                "name": "python-deeploy",
                "epoch": null,
                "version": "1.50.0",
                "release": "1",
                "exclusivearch": [],
                "excludearch": []
            },
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": true,
            "destdir": "/var/lib/copr/public_html/results/@copr/PyPI",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI",
            "result_dir": "09266464",
            "built_packages": "",
            "tags": [
                "arch_noarch"
            ],
            "pkg_version": "1.50.0-1",
            "id": 9266464,
            "mockchain_macros": {
                "copr_username": "@copr",
                "copr_projectname": "PyPI",
                "vendor": "Fedora Project COPR (@copr/PyPI)"
            }
        }
    ]
}
[2025-07-11 12:13:37,993][  INFO][PID:1647702] Sending fedora-messaging bus message in build.end
[2025-07-11 12:13:38,032][  INFO][PID:1647702] Compressing /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/09266464/builder-live.log by gzip
[2025-07-11 12:13:38,035][  INFO][PID:1647702] Running command 'gzip /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/09266464/builder-live.log' as PID 1708946
[2025-07-11 12:13:38,041][  INFO][PID:1647702] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/09266464/builder-live.log)
[2025-07-11 12:13:38,045][  INFO][PID:1647702] Compressing /var/lib/copr/public_html/results/@copr/PyPI/srpm-builds/09266464/backend.log by gzip