[2024-11-11 17:33:33,746][ INFO][PID:82989] Marking build as starting [2024-11-11 17:33:33,807][ INFO][PID:82989] Trying to allocate VM [2024-11-11 17:33:36,839][ INFO][PID:82989] Allocated host ResallocHost, ticket_id=5815699, hostname=34.227.173.137 [2024-11-11 17:33:36,840][ INFO][PID:82989] Allocating ssh connection to builder [2024-11-11 17:33:36,840][ INFO][PID:82989] Checking that builder machine is OK [2024-11-11 17:33:37,206][ INFO][PID:82989] Running remote command: copr-builder-ready fedora-41-x86_64 [2024-11-11 17:33:37,437][ INFO][PID:82989] Red Hat subscription not needed for fedora-41-x86_64 Builder is ready to be used [2024-11-11 17:33:37,438][ INFO][PID:82989] Filling build.info file with builder info [2024-11-11 17:33:37,440][ INFO][PID:82989] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "tofik", "project_name": "password-store", "project_dirname": "password-store", "submitter": "tofik", "ended_on": null, "started_on": 1731346417.4398043, "submitted_on": null, "status": 3, "chroot": "fedora-41-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "8243497-fedora-41-x86_64", "build_id": 8243497, "package_name": "tomb", "package_version": "2.11-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/tofik/password-store/tomb", "git_hash": "c387e2f38c49ed9b43e9b33b89211b8e6ae727ea", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "tofik/password-store--tofik", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/tofik/password-store/fedora-41-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/tofik/password-store", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/tofik/password-store", "result_dir": "08243497-tomb", "built_packages": "", "id": 8243497, "mockchain_macros": { "copr_username": "tofik", "copr_projectname": "password-store", "vendor": "Fedora Project COPR (tofik/password-store)" } } ] } [2024-11-11 17:33:37,504][ INFO][PID:82989] Sending fedora-messaging bus message in build.start [2024-11-11 17:33:38,360][ INFO][PID:82989] Sending fedora-messaging bus message in chroot.start [2024-11-11 17:33:38,382][ INFO][PID:82989] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8243497-fedora-41-x86_64 --chroot fedora-41-x86_64 --detached [2024-11-11 17:33:38,767][ INFO][PID:82989] Downloading the builder-live.log file, attempt 1 [2024-11-11 17:33:38,769][ INFO][PID:82989] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.227.173.137 copr-rpmbuild-log [2024-11-11 17:33:43,775][ INFO][PID:82989] Periodic builder liveness probe: alive [2024-11-11 17:33:58,057][ ERROR][PID:82989] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2024-11-11 17:34:08,058][WARNING][PID:82989] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2024-11-11 17:34:18,059][ INFO][PID:82989] Downloading the builder-live.log file, attempt 2 [2024-11-11 17:34:18,060][ INFO][PID:82989] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.227.173.137 copr-rpmbuild-log [2024-11-11 17:35:18,130][ ERROR][PID:82989] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2024-11-11 17:35:28,153][WARNING][PID:82989] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2024-11-11 17:35:38,154][ INFO][PID:82989] Downloading the builder-live.log file, attempt 3 [2024-11-11 17:35:38,169][ INFO][PID:82989] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.227.173.137 copr-rpmbuild-log [2024-11-11 17:36:38,242][ ERROR][PID:82989] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2024-11-11 17:36:48,248][WARNING][PID:82989] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2024-11-11 17:36:58,249][ INFO][PID:82989] Downloading the builder-live.log file, attempt 4 [2024-11-11 17:36:58,251][ INFO][PID:82989] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.227.173.137 copr-rpmbuild-log [2024-11-11 17:37:58,318][ ERROR][PID:82989] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2024-11-11 17:38:08,322][ ERROR][PID:82989] Giving up for unstable SSH, failures: 1731346448.0582707, 1731346528.1538517, 1731346608.248681, 1731346688.3224533 [2024-11-11 17:38:08,323][ ERROR][PID:82989] Re-try request for task on 'ResallocHost, ticket_id=5815699, hostname=34.227.173.137': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2024-11-11 17:38:08,324][ INFO][PID:82989] Releasing VM back to pool [2024-11-11 17:38:08,334][ INFO][PID:82989] Retry #1 (on other host) [2024-11-11 17:38:08,335][ INFO][PID:82989] Trying to allocate VM [2024-11-11 17:38:11,418][ INFO][PID:82989] Allocated host ResallocHost, ticket_id=5815764, hostname=2620:52:3:1:dead:beef:cafe:c150 [2024-11-11 17:38:11,418][ INFO][PID:82989] Allocating ssh connection to builder [2024-11-11 17:38:11,419][ INFO][PID:82989] Checking that builder machine is OK [2024-11-11 17:38:11,978][ INFO][PID:82989] Running remote command: copr-builder-ready fedora-41-x86_64 [2024-11-11 17:38:12,311][ INFO][PID:82989] Red Hat subscription not needed for fedora-41-x86_64 Builder is ready to be used [2024-11-11 17:38:12,312][ INFO][PID:82989] Filling build.info file with builder info [2024-11-11 17:38:12,323][ INFO][PID:82989] Not re-notifying FE and msg buses for the new host. [2024-11-11 17:38:12,324][ INFO][PID:82989] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8243497-fedora-41-x86_64 --chroot fedora-41-x86_64 --detached [2024-11-11 17:38:13,001][ INFO][PID:82989] Downloading the builder-live.log file, attempt 1 [2024-11-11 17:38:13,003][ INFO][PID:82989] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c150 copr-rpmbuild-log [2024-11-11 17:39:00,593][ INFO][PID:82989] Downloading results from builder [2024-11-11 17:39:00,595][ INFO][PID:82989] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c150]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64/08243497-tomb started [2024-11-11 17:39:00,596][ INFO][PID:82989] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c150]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64/08243497-tomb/ &> /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64/08243497-tomb/build-08243497.rsync.log [2024-11-11 17:39:01,300][ INFO][PID:82989] rsyncing finished. [2024-11-11 17:39:01,302][ INFO][PID:82989] Releasing VM back to pool [2024-11-11 17:39:01,311][ INFO][PID:82989] Searching for 'success' file in resultdir [2024-11-11 17:39:01,311][ INFO][PID:82989] Going to sign pkgs from source: 8243497-fedora-41-x86_64 in chroot: /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64 [2024-11-11 17:39:01,312][ INFO][PID:82989] Calling '/bin/sign -u tofik#password-store@copr.fedorahosted.org -p' (attempt #1) [2024-11-11 17:39:01,619][ INFO][PID:82989] Calling '/bin/sign -4 -h sha256 -u tofik#password-store@copr.fedorahosted.org -r /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64/08243497-tomb/tomb-2.11-1.fc41.noarch.rpm' (attempt #1) [2024-11-11 17:39:02,488][ INFO][PID:82989] signed rpm: /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64/08243497-tomb/tomb-2.11-1.fc41.noarch.rpm [2024-11-11 17:39:02,489][ INFO][PID:82989] Calling '/bin/sign -4 -h sha256 -u tofik#password-store@copr.fedorahosted.org -r /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64/08243497-tomb/tomb-2.11-1.fc41.src.rpm' (attempt #1) [2024-11-11 17:39:03,433][ INFO][PID:82989] signed rpm: /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64/08243497-tomb/tomb-2.11-1.fc41.src.rpm [2024-11-11 17:39:03,434][ INFO][PID:82989] Sign done [2024-11-11 17:39:03,434][ INFO][PID:82989] Incremental createrepo run, adding 08243497-tomb into https://download.copr.fedorainfracloud.org/results/tofik/password-store/fedora-41-x86_64, (auto-create-repo=True) [2024-11-11 17:39:03,435][ INFO][PID:82989] Running command 'copr-repo --batched /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64 --add 08243497-tomb --no-appstream-metadata' as PID 120301 [2024-11-11 17:39:04,321][ INFO][PID:82989] Finished after 0 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64 --add 08243497-tomb --no-appstream-metadata) [2024-11-11 17:39:04,322][ INFO][PID:82989] Getting build details [2024-11-11 17:39:04,323][ INFO][PID:82989] Listing built binary packages in /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64/08243497-tomb [2024-11-11 17:39:04,323][ INFO][PID:82989] Built packages: ['tomb 2.11'] [2024-11-11 17:39:04,324][ INFO][PID:82989] build details: {'built_packages': 'tomb 2.11'} [2024-11-11 17:39:04,324][ INFO][PID:82989] Retrieving pubkey [2024-11-11 17:39:04,325][ INFO][PID:82989] Calling '/bin/sign -u tofik#password-store@copr.fedorahosted.org -p' (attempt #1) [2024-11-11 17:39:04,641][ INFO][PID:82989] Added pubkey for user tofik project password-store into: /var/lib/copr/public_html/results/tofik/password-store/pubkey.gpg [2024-11-11 17:39:04,642][ INFO][PID:82989] Finished build: id=8243497 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/tofik/password-store chroot=fedora-41-x86_64 [2024-11-11 17:39:04,644][ INFO][PID:82989] Worker succeeded build, took 52.32074522972107 [2024-11-11 17:39:04,645][ INFO][PID:82989] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "tofik", "project_name": "password-store", "project_dirname": "password-store", "submitter": "tofik", "ended_on": 1731346744.6445491, "started_on": 1731346692.323804, "submitted_on": null, "status": 1, "chroot": "fedora-41-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "8243497-fedora-41-x86_64", "build_id": 8243497, "package_name": "tomb", "package_version": "2.11-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/tofik/password-store/tomb", "git_hash": "c387e2f38c49ed9b43e9b33b89211b8e6ae727ea", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "tofik/password-store--tofik", "results": { "packages": [ { "name": "tomb", "epoch": null, "version": "2.11", "release": "1.fc41", "arch": "src" }, { "name": "tomb", "epoch": null, "version": "2.11", "release": "1.fc41", "arch": "noarch" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/tofik/password-store/fedora-41-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/tofik/password-store", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/tofik/password-store", "result_dir": "08243497-tomb", "built_packages": "tomb 2.11", "id": 8243497, "mockchain_macros": { "copr_username": "tofik", "copr_projectname": "password-store", "vendor": "Fedora Project COPR (tofik/password-store)" } } ] } [2024-11-11 17:39:04,714][ INFO][PID:82989] Sending fedora-messaging bus message in build.end [2024-11-11 17:39:04,736][ INFO][PID:82989] Compressing /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64/08243497-tomb/builder-live.log by gzip [2024-11-11 17:39:04,738][ INFO][PID:82989] Running command 'gzip /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64/08243497-tomb/builder-live.log' as PID 120448 [2024-11-11 17:39:04,742][ INFO][PID:82989] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64/08243497-tomb/builder-live.log) [2024-11-11 17:39:04,743][ INFO][PID:82989] Compressing /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64/08243497-tomb/backend.log by gzip [2024-11-11 17:39:04,744][ INFO][PID:82989] Running command 'gzip /var/lib/copr/public_html/results/tofik/password-store/fedora-41-x86_64/08243497-tomb/backend.log' as PID 120449