[2025-05-22 14:32:38,856][ INFO][PID:1301349] Marking build as starting [2025-05-22 14:32:39,047][ INFO][PID:1301349] Trying to allocate VM: ResallocHost, ticket_id=3126906, requested_tags=['copr_builder', 'arch_noarch'] [2025-05-22 14:32:42,155][ INFO][PID:1301349] Allocated host ResallocHost, ticket_id=3126906, hostname=54.234.246.31, name=aws_x86_64_spot_prod_04233717_20250522_142411, requested_tags=['copr_builder', 'arch_noarch'] [2025-05-22 14:32:42,158][ INFO][PID:1301349] Allocating ssh connection to builder [2025-05-22 14:32:42,159][ INFO][PID:1301349] Checking that builder machine is OK [2025-05-22 14:32:42,630][ INFO][PID:1301349] Running remote command: copr-builder-ready srpm-builds [2025-05-22 14:32:42,933][ INFO][PID:1301349] Red Hat subscription not needed for srpm-builds Builder is ready to be used [2025-05-22 14:32:42,936][ INFO][PID:1301349] Filling build.info file with builder info [2025-05-22 14:32:42,967][ INFO][PID:1301349] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": null, "enable_net": true, "project_owner": "xsnrg", "project_name": "indi-3rdparty-bleeding", "project_dirname": "indi-3rdparty-bleeding", "submitter": "https://api.github.com/users/knro", "ended_on": null, "started_on": 1747924362.9665725, "submitted_on": null, "status": 3, "chroot": "srpm-builds", "arch": null, "buildroot_pkgs": null, "task_id": "9073260", "build_id": 9073260, "package_name": "indi-maxdomeii", "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 8, "source_json": "{\"type\": \"git\", \"clone_url\": \"https://github.com/indilib/indi-3rdparty\", \"subdirectory\": \"\", \"committish\": \"f3797fc7d469524693e926313fb66c10a488089a\", \"spec\": \"indi-maxdomeii/indi-maxdomeii.spec\", \"srpm_build_method\": \"rpkg\"}", "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": null, "sandbox": "xsnrg/indi-3rdparty-bleeding--https://api.github.com/users/knro", "results": null, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "background": true, "repos": [], "destdir": "/var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding", "result_dir": "09073260", "built_packages": "", "tags": [ "arch_noarch" ], "id": 9073260, "mockchain_macros": { "copr_username": "xsnrg", "copr_projectname": "indi-3rdparty-bleeding", "vendor": "Fedora Project COPR (xsnrg/indi-3rdparty-bleeding)" } } ] } [2025-05-22 14:32:43,015][ INFO][PID:1301349] Sending fedora-messaging bus message in build.start [2025-05-22 14:32:44,269][ INFO][PID:1301349] Sending fedora-messaging bus message in chroot.start [2025-05-22 14:32:44,307][ INFO][PID:1301349] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/9073260 --detached [2025-05-22 14:32:44,713][ INFO][PID:1301349] Downloading the builder-live.log file, attempt 1 [2025-05-22 14:32:44,716][ INFO][PID:1301349] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.234.246.31 copr-rpmbuild-log [2025-05-22 14:32:49,732][ INFO][PID:1301349] Periodic builder liveness probe: alive [2025-05-22 14:34:00,299][ ERROR][PID:1301349] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-05-22 14:34:10,301][WARNING][PID:1301349] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-05-22 14:34:20,303][ INFO][PID:1301349] Downloading the builder-live.log file, attempt 2 [2025-05-22 14:34:20,306][ INFO][PID:1301349] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.234.246.31 copr-rpmbuild-log [2025-05-22 14:35:20,373][ ERROR][PID:1301349] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-05-22 14:35:30,385][WARNING][PID:1301349] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-05-22 14:35:40,391][ INFO][PID:1301349] Downloading the builder-live.log file, attempt 3 [2025-05-22 14:35:40,394][ INFO][PID:1301349] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.234.246.31 copr-rpmbuild-log [2025-05-22 14:36:40,463][ ERROR][PID:1301349] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-05-22 14:36:50,473][WARNING][PID:1301349] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-05-22 14:37:00,482][ INFO][PID:1301349] Downloading the builder-live.log file, attempt 4 [2025-05-22 14:37:00,486][ INFO][PID:1301349] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.234.246.31 copr-rpmbuild-log [2025-05-22 14:38:00,557][ ERROR][PID:1301349] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-05-22 14:38:10,563][ ERROR][PID:1301349] Giving up for unstable SSH, failures: 1747924450.3011005, 1747924530.3850715, 1747924610.4739015, 1747924690.5636995 [2025-05-22 14:38:10,566][ ERROR][PID:1301349] Re-try request for task on 'ResallocHost, ticket_id=3126906, hostname=54.234.246.31, name=aws_x86_64_spot_prod_04233717_20250522_142411, requested_tags=['copr_builder', 'arch_noarch']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-05-22 14:38:10,568][ INFO][PID:1301349] Releasing VM back to pool [2025-05-22 14:38:10,693][ INFO][PID:1301349] Retry #1 (on other host) [2025-05-22 14:38:10,719][ INFO][PID:1301349] Trying to allocate VM: ResallocHost, ticket_id=3127013, requested_tags=['copr_builder', 'arch_noarch'] [2025-05-22 14:38:13,761][ INFO][PID:1301349] Allocated host ResallocHost, ticket_id=3127013, hostname=54.90.90.137, name=aws_aarch64_spot_prod_04233088_20250522_141745, requested_tags=['copr_builder', 'arch_noarch'] [2025-05-22 14:38:13,766][ INFO][PID:1301349] Allocating ssh connection to builder [2025-05-22 14:38:13,769][ INFO][PID:1301349] Checking that builder machine is OK [2025-05-22 14:38:14,216][ INFO][PID:1301349] Running remote command: copr-builder-ready srpm-builds [2025-05-22 14:38:14,544][ INFO][PID:1301349] Red Hat subscription not needed for srpm-builds Builder is ready to be used [2025-05-22 14:38:14,551][ INFO][PID:1301349] Filling build.info file with builder info [2025-05-22 14:38:14,558][ INFO][PID:1301349] Not re-notifying FE and msg buses for the new host. [2025-05-22 14:38:14,565][ INFO][PID:1301349] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/9073260 --detached [2025-05-22 14:38:15,592][ INFO][PID:1301349] Downloading the builder-live.log file, attempt 1 [2025-05-22 14:38:15,597][ INFO][PID:1301349] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.90.90.137 copr-rpmbuild-log [2025-05-22 14:39:46,590][ INFO][PID:1301349] Downloading results from builder [2025-05-22 14:39:46,601][ INFO][PID:1301349] rsyncing of mockbuilder@54.90.90.137:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/srpm-builds/09073260 started [2025-05-22 14:39:46,607][ INFO][PID:1301349] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@54.90.90.137:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/srpm-builds/09073260/ &> /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/srpm-builds/09073260/build-09073260.rsync.log [2025-05-22 14:40:17,903][ INFO][PID:1301349] rsyncing finished. [2025-05-22 14:40:17,905][ INFO][PID:1301349] Releasing VM back to pool [2025-05-22 14:40:17,928][ INFO][PID:1301349] Searching for 'success' file in resultdir [2025-05-22 14:40:17,930][ INFO][PID:1301349] Getting build details [2025-05-22 14:40:17,931][ INFO][PID:1301349] Retrieving SRPM info from /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/srpm-builds/09073260 [2025-05-22 14:40:17,933][ INFO][PID:1301349] SRPM URL: https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/srpm-builds/09073260/indi-maxdomeii-2.1.3.git-20250522143933.src.rpm [2025-05-22 14:40:17,935][ INFO][PID:1301349] build details: {'srpm_url': 'https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/srpm-builds/09073260/indi-maxdomeii-2.1.3.git-20250522143933.src.rpm', 'pkg_name': 'indi-maxdomeii', 'pkg_version': '2.1.3.git-20250522143945'} [2025-05-22 14:40:17,937][ INFO][PID:1301349] Finished build: id=9073260 failed=False timeout=108000 destdir=/var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding chroot=srpm-builds [2025-05-22 14:40:17,944][ INFO][PID:1301349] Worker succeeded build, took 123.38622570037842 [2025-05-22 14:40:17,946][ INFO][PID:1301349] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": null, "enable_net": true, "project_owner": "xsnrg", "project_name": "indi-3rdparty-bleeding", "project_dirname": "indi-3rdparty-bleeding", "submitter": "https://api.github.com/users/knro", "ended_on": 1747924817.9444335, "started_on": 1747924694.5582078, "submitted_on": null, "status": 1, "chroot": "srpm-builds", "arch": null, "buildroot_pkgs": null, "task_id": "9073260", "build_id": 9073260, "package_name": "indi-maxdomeii", "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 8, "source_json": "{\"type\": \"git\", \"clone_url\": \"https://github.com/indilib/indi-3rdparty\", \"subdirectory\": \"\", \"committish\": \"f3797fc7d469524693e926313fb66c10a488089a\", \"spec\": \"indi-maxdomeii/indi-maxdomeii.spec\", \"srpm_build_method\": \"rpkg\"}", "pkg_name": "indi-maxdomeii", "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": "https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/srpm-builds/09073260/indi-maxdomeii-2.1.3.git-20250522143933.src.rpm", "uses_devel_repo": null, "sandbox": "xsnrg/indi-3rdparty-bleeding--https://api.github.com/users/knro", "results": { "name": "indi-maxdomeii", "epoch": null, "version": "2.1.3.git", "release": "20250522143945", "exclusivearch": [], "excludearch": [] }, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "background": true, "repos": [], "destdir": "/var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding", "result_dir": "09073260", "built_packages": "", "tags": [ "arch_noarch" ], "pkg_version": "2.1.3.git-20250522143945", "id": 9073260, "mockchain_macros": { "copr_username": "xsnrg", "copr_projectname": "indi-3rdparty-bleeding", "vendor": "Fedora Project COPR (xsnrg/indi-3rdparty-bleeding)" } } ] } [2025-05-22 14:40:18,483][ INFO][PID:1301349] Sending fedora-messaging bus message in build.end [2025-05-22 14:40:18,502][ INFO][PID:1301349] Compressing /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/srpm-builds/09073260/builder-live.log by gzip [2025-05-22 14:40:18,504][ INFO][PID:1301349] Running command 'gzip /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/srpm-builds/09073260/builder-live.log' as PID 1345291 [2025-05-22 14:40:18,506][ INFO][PID:1301349] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/srpm-builds/09073260/builder-live.log) [2025-05-22 14:40:18,508][ INFO][PID:1301349] Compressing /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/srpm-builds/09073260/backend.log by gzip [2025-05-22 14:40:18,510][ INFO][PID:1301349] Running command 'gzip /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/srpm-builds/09073260/backend.log' as PID 1345292