[2025-02-25 19:31:58,316][ INFO][PID:2057419] Marking build as starting [2025-02-25 19:31:58,424][ INFO][PID:2057419] Trying to allocate VM [2025-02-25 19:33:06,577][ INFO][PID:2057419] Allocated host ResallocHost, ticket_id=1506711, hostname=34.205.246.59, name=aws_x86_64_spot_prod_01454296_20250225_193126 [2025-02-25 19:33:06,579][ INFO][PID:2057419] Allocating ssh connection to builder [2025-02-25 19:33:06,580][ INFO][PID:2057419] Checking that builder machine is OK [2025-02-25 19:33:06,964][ INFO][PID:2057419] Running remote command: copr-builder-ready fedora-40-x86_64 [2025-02-25 19:33:07,199][ INFO][PID:2057419] Red Hat subscription not needed for fedora-40-x86_64 Builder is ready to be used [2025-02-25 19:33:07,200][ INFO][PID:2057419] Filling build.info file with builder info [2025-02-25 19:33:07,201][ INFO][PID:2057419] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "farchord", "project_name": "kde-test", "project_dirname": "kde-test", "submitter": "farchord", "ended_on": null, "started_on": 1740511987.2016366, "submitted_on": null, "status": 3, "chroot": "fedora-40-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "8699719-fedora-40-x86_64", "build_id": 8699719, "package_name": "breeze-gtk", "package_version": "6.3.2-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/farchord/kde-test/breeze-gtk", "git_hash": "1cbbb2f1f4776e2b8e44c2a7c03ed5a93527835f", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "farchord/kde-test--farchord", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/farchord/kde-test/fedora-40-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/farchord/kde-test", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/farchord/kde-test", "result_dir": "08699719-breeze-gtk", "built_packages": "", "id": 8699719, "mockchain_macros": { "copr_username": "farchord", "copr_projectname": "kde-test", "vendor": "Fedora Project COPR (farchord/kde-test)" } } ] } [2025-02-25 19:33:07,231][ INFO][PID:2057419] Sending fedora-messaging bus message in build.start [2025-02-25 19:33:08,102][ INFO][PID:2057419] Sending fedora-messaging bus message in chroot.start [2025-02-25 19:33:08,131][ INFO][PID:2057419] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8699719-fedora-40-x86_64 --chroot fedora-40-x86_64 --detached [2025-02-25 19:33:08,509][ INFO][PID:2057419] Downloading the builder-live.log file, attempt 1 [2025-02-25 19:33:08,512][ INFO][PID:2057419] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.205.246.59 copr-rpmbuild-log [2025-02-25 19:33:13,518][ INFO][PID:2057419] Periodic builder liveness probe: alive [2025-02-25 19:33:36,638][ ERROR][PID:2057419] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-02-25 19:33:46,639][WARNING][PID:2057419] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-02-25 19:33:56,640][ INFO][PID:2057419] Downloading the builder-live.log file, attempt 2 [2025-02-25 19:33:56,642][ INFO][PID:2057419] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.205.246.59 copr-rpmbuild-log [2025-02-25 19:34:56,711][ ERROR][PID:2057419] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-02-25 19:35:06,713][WARNING][PID:2057419] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-02-25 19:35:16,714][ INFO][PID:2057419] Downloading the builder-live.log file, attempt 3 [2025-02-25 19:35:16,716][ INFO][PID:2057419] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.205.246.59 copr-rpmbuild-log [2025-02-25 19:36:16,784][ ERROR][PID:2057419] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-02-25 19:36:26,785][WARNING][PID:2057419] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-02-25 19:36:36,786][ INFO][PID:2057419] Downloading the builder-live.log file, attempt 4 [2025-02-25 19:36:36,787][ INFO][PID:2057419] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.205.246.59 copr-rpmbuild-log [2025-02-25 19:37:36,855][ ERROR][PID:2057419] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-02-25 19:37:46,859][ ERROR][PID:2057419] Giving up for unstable SSH, failures: 1740512026.6397624, 1740512106.7130218, 1740512186.7852783, 1740512266.8592153 [2025-02-25 19:37:46,860][ ERROR][PID:2057419] Re-try request for task on 'ResallocHost, ticket_id=1506711, hostname=34.205.246.59, name=aws_x86_64_spot_prod_01454296_20250225_193126': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-02-25 19:37:46,861][ INFO][PID:2057419] Releasing VM back to pool [2025-02-25 19:37:46,986][ INFO][PID:2057419] Retry #1 (on other host) [2025-02-25 19:37:46,987][ INFO][PID:2057419] Trying to allocate VM [2025-02-25 19:37:50,022][ INFO][PID:2057419] Allocated host ResallocHost, ticket_id=1506769, hostname=2620:52:3:1:dead:beef:cafe:c1d8, name=copr_hv_x86_64_04_prod_01454446_20250225_193440 [2025-02-25 19:37:50,023][ INFO][PID:2057419] Allocating ssh connection to builder [2025-02-25 19:37:50,023][ INFO][PID:2057419] Checking that builder machine is OK [2025-02-25 19:37:50,595][ INFO][PID:2057419] Running remote command: copr-builder-ready fedora-40-x86_64 [2025-02-25 19:37:50,943][ INFO][PID:2057419] Red Hat subscription not needed for fedora-40-x86_64 Builder is ready to be used [2025-02-25 19:37:50,944][ INFO][PID:2057419] Filling build.info file with builder info [2025-02-25 19:37:50,948][ INFO][PID:2057419] Not re-notifying FE and msg buses for the new host. [2025-02-25 19:37:50,949][ INFO][PID:2057419] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8699719-fedora-40-x86_64 --chroot fedora-40-x86_64 --detached [2025-02-25 19:37:51,495][ INFO][PID:2057419] Downloading the builder-live.log file, attempt 1 [2025-02-25 19:37:51,496][ INFO][PID:2057419] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c1d8 copr-rpmbuild-log [2025-02-25 19:39:48,471][ INFO][PID:2057419] Downloading results from builder [2025-02-25 19:39:48,472][ INFO][PID:2057419] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c1d8]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk started [2025-02-25 19:39:48,473][ INFO][PID:2057419] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c1d8]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/ &> /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/build-08699719.rsync.log [2025-02-25 19:39:48,893][ INFO][PID:2057419] rsyncing finished. [2025-02-25 19:39:48,894][ INFO][PID:2057419] Releasing VM back to pool [2025-02-25 19:39:48,903][ INFO][PID:2057419] Searching for 'success' file in resultdir [2025-02-25 19:39:48,904][ INFO][PID:2057419] Going to sign pkgs from source: 8699719-fedora-40-x86_64 in chroot: /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64 [2025-02-25 19:39:48,904][ INFO][PID:2057419] Calling '/bin/sign -u farchord#kde-test@copr.fedorahosted.org -p' (attempt #1) [2025-02-25 19:39:49,077][ INFO][PID:2057419] Calling '/bin/sign -4 -h sha256 -u farchord#kde-test@copr.fedorahosted.org -r /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/breeze-gtk-gtk3-6.3.2-1.fc40.noarch.rpm' (attempt #1) [2025-02-25 19:39:49,829][ INFO][PID:2057419] signed rpm: /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/breeze-gtk-gtk3-6.3.2-1.fc40.noarch.rpm [2025-02-25 19:39:49,830][ INFO][PID:2057419] Calling '/bin/sign -4 -h sha256 -u farchord#kde-test@copr.fedorahosted.org -r /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/breeze-gtk-gtk2-6.3.2-1.fc40.noarch.rpm' (attempt #1) [2025-02-25 19:39:50,718][ INFO][PID:2057419] signed rpm: /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/breeze-gtk-gtk2-6.3.2-1.fc40.noarch.rpm [2025-02-25 19:39:50,719][ INFO][PID:2057419] Calling '/bin/sign -4 -h sha256 -u farchord#kde-test@copr.fedorahosted.org -r /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/breeze-gtk-common-6.3.2-1.fc40.noarch.rpm' (attempt #1) [2025-02-25 19:39:51,624][ INFO][PID:2057419] signed rpm: /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/breeze-gtk-common-6.3.2-1.fc40.noarch.rpm [2025-02-25 19:39:51,625][ INFO][PID:2057419] Calling '/bin/sign -4 -h sha256 -u farchord#kde-test@copr.fedorahosted.org -r /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/breeze-gtk-6.3.2-1.fc40.noarch.rpm' (attempt #1) [2025-02-25 19:39:52,508][ INFO][PID:2057419] signed rpm: /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/breeze-gtk-6.3.2-1.fc40.noarch.rpm [2025-02-25 19:39:52,509][ INFO][PID:2057419] Calling '/bin/sign -4 -h sha256 -u farchord#kde-test@copr.fedorahosted.org -r /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/breeze-gtk-gtk4-6.3.2-1.fc40.noarch.rpm' (attempt #1) [2025-02-25 19:39:53,401][ INFO][PID:2057419] signed rpm: /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/breeze-gtk-gtk4-6.3.2-1.fc40.noarch.rpm [2025-02-25 19:39:53,402][ INFO][PID:2057419] Calling '/bin/sign -4 -h sha256 -u farchord#kde-test@copr.fedorahosted.org -r /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/breeze-gtk-6.3.2-1.fc40.src.rpm' (attempt #1) [2025-02-25 19:39:54,290][ INFO][PID:2057419] signed rpm: /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/breeze-gtk-6.3.2-1.fc40.src.rpm [2025-02-25 19:39:54,291][ INFO][PID:2057419] Sign done [2025-02-25 19:39:54,291][ INFO][PID:2057419] Incremental createrepo run, adding 08699719-breeze-gtk into https://download.copr.fedorainfracloud.org/results/farchord/kde-test/fedora-40-x86_64, (auto-create-repo=True) [2025-02-25 19:39:54,292][ INFO][PID:2057419] Running command 'copr-repo --batched /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64 --add 08699719-breeze-gtk --no-appstream-metadata' as PID 2112218 [2025-02-25 19:39:56,775][ INFO][PID:2057419] Finished after 2 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64 --add 08699719-breeze-gtk --no-appstream-metadata) [2025-02-25 19:39:56,776][ INFO][PID:2057419] Getting build details [2025-02-25 19:39:56,777][ INFO][PID:2057419] Listing built binary packages in /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk [2025-02-25 19:39:56,777][ INFO][PID:2057419] Built packages: ['breeze-gtk-gtk4 6.3.2', 'breeze-gtk-gtk3 6.3.2', 'breeze-gtk-gtk2 6.3.2', 'breeze-gtk 6.3.2', 'breeze-gtk-common 6.3.2'] [2025-02-25 19:39:56,778][ INFO][PID:2057419] build details: {'built_packages': 'breeze-gtk-gtk4 6.3.2\nbreeze-gtk-gtk3 6.3.2\nbreeze-gtk-gtk2 6.3.2\nbreeze-gtk 6.3.2\nbreeze-gtk-common 6.3.2'} [2025-02-25 19:39:56,778][ INFO][PID:2057419] Retrieving pubkey [2025-02-25 19:39:56,778][ INFO][PID:2057419] Calling '/bin/sign -u farchord#kde-test@copr.fedorahosted.org -p' (attempt #1) [2025-02-25 19:39:57,093][ INFO][PID:2057419] Added pubkey for user farchord project kde-test into: /var/lib/copr/public_html/results/farchord/kde-test/pubkey.gpg [2025-02-25 19:39:57,094][ INFO][PID:2057419] Finished build: id=8699719 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/farchord/kde-test chroot=fedora-40-x86_64 [2025-02-25 19:39:57,096][ INFO][PID:2057419] Worker succeeded build, took 126.14742350578308 [2025-02-25 19:39:57,096][ INFO][PID:2057419] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "farchord", "project_name": "kde-test", "project_dirname": "kde-test", "submitter": "farchord", "ended_on": 1740512397.096189, "started_on": 1740512270.9487655, "submitted_on": null, "status": 1, "chroot": "fedora-40-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "8699719-fedora-40-x86_64", "build_id": 8699719, "package_name": "breeze-gtk", "package_version": "6.3.2-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/farchord/kde-test/breeze-gtk", "git_hash": "1cbbb2f1f4776e2b8e44c2a7c03ed5a93527835f", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "farchord/kde-test--farchord", "results": { "packages": [ { "name": "breeze-gtk-gtk4", "epoch": null, "version": "6.3.2", "release": "1.fc40", "arch": "noarch" }, { "name": "breeze-gtk-gtk3", "epoch": null, "version": "6.3.2", "release": "1.fc40", "arch": "noarch" }, { "name": "breeze-gtk-gtk2", "epoch": null, "version": "6.3.2", "release": "1.fc40", "arch": "noarch" }, { "name": "breeze-gtk", "epoch": null, "version": "6.3.2", "release": "1.fc40", "arch": "noarch" }, { "name": "breeze-gtk-common", "epoch": null, "version": "6.3.2", "release": "1.fc40", "arch": "noarch" }, { "name": "breeze-gtk", "epoch": null, "version": "6.3.2", "release": "1.fc40", "arch": "src" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/farchord/kde-test/fedora-40-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/farchord/kde-test", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/farchord/kde-test", "result_dir": "08699719-breeze-gtk", "built_packages": "breeze-gtk-gtk4 6.3.2\nbreeze-gtk-gtk3 6.3.2\nbreeze-gtk-gtk2 6.3.2\nbreeze-gtk 6.3.2\nbreeze-gtk-common 6.3.2", "id": 8699719, "mockchain_macros": { "copr_username": "farchord", "copr_projectname": "kde-test", "vendor": "Fedora Project COPR (farchord/kde-test)" } } ] } [2025-02-25 19:39:57,127][ INFO][PID:2057419] Sending fedora-messaging bus message in build.end [2025-02-25 19:39:57,204][ INFO][PID:2057419] Compressing /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/builder-live.log by gzip [2025-02-25 19:39:57,205][ INFO][PID:2057419] Running command 'gzip /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/builder-live.log' as PID 2112396 [2025-02-25 19:39:57,216][ INFO][PID:2057419] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/builder-live.log) [2025-02-25 19:39:57,217][ INFO][PID:2057419] Compressing /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/backend.log by gzip [2025-02-25 19:39:57,218][ INFO][PID:2057419] Running command 'gzip /var/lib/copr/public_html/results/farchord/kde-test/fedora-40-x86_64/08699719-breeze-gtk/backend.log' as PID 2112397