[2025-05-13 16:20:45,535][ INFO][PID:2090028] Marking build as starting [2025-05-13 16:20:45,690][ INFO][PID:2090028] Trying to allocate VM: ResallocHost, ticket_id=2963978, requested_tags=['copr_builder', 'arch_x86_64'] [2025-05-13 16:30:04,740][ INFO][PID:2090028] Allocated host ResallocHost, ticket_id=2963978, hostname=35.173.126.15, name=aws_x86_64_spot_prod_03893447_20250513_162619, requested_tags=['copr_builder', 'arch_x86_64'] [2025-05-13 16:30:04,742][ INFO][PID:2090028] Allocating ssh connection to builder [2025-05-13 16:30:04,750][ INFO][PID:2090028] Checking that builder machine is OK [2025-05-13 16:30:05,337][ INFO][PID:2090028] Running remote command: copr-builder-ready epel-9-x86_64 [2025-05-13 16:30:05,738][ INFO][PID:2090028] Checking Red Hat subscription (attempt #1) Red Hat subscription active Builder is ready to be used [2025-05-13 16:30:05,746][ INFO][PID:2090028] Filling build.info file with builder info [2025-05-13 16:30:05,779][ INFO][PID:2090028] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@oisf", "project_name": "suricata-7.0-testing", "project_dirname": "suricata-7.0-testing", "submitter": "jasonish", "ended_on": null, "started_on": 1747153805.7792492, "submitted_on": null, "status": 3, "chroot": "epel-9-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9033529-epel-9-x86_64", "build_id": 9033529, "package_name": "suricata", "package_version": "1:7.0.10-2", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@oisf/suricata-7.0-testing/suricata", "git_hash": "044fc2356b4dc640e550549215be780661bb37c5", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@oisf/suricata-7.0-testing--jasonish", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@oisf/suricata-7.0-testing/epel-9-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@oisf/suricata-7.0-testing", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@oisf/suricata-7.0-testing", "result_dir": "09033529-suricata", "built_packages": "", "id": 9033529, "mockchain_macros": { "copr_username": "@oisf", "copr_projectname": "suricata-7.0-testing", "vendor": "Fedora Project COPR (@oisf/suricata-7.0-testing)" } } ] } [2025-05-13 16:30:05,845][ INFO][PID:2090028] Sending fedora-messaging bus message in build.start [2025-05-13 16:30:10,624][ INFO][PID:2090028] Sending fedora-messaging bus message in chroot.start [2025-05-13 16:30:10,696][ INFO][PID:2090028] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9033529-epel-9-x86_64 --chroot epel-9-x86_64 --detached [2025-05-13 16:30:11,197][ INFO][PID:2090028] Downloading the builder-live.log file, attempt 1 [2025-05-13 16:30:11,206][ INFO][PID:2090028] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@35.173.126.15 copr-rpmbuild-log [2025-05-13 16:30:16,264][ INFO][PID:2090028] Periodic builder liveness probe: alive [2025-05-13 16:33:35,170][ ERROR][PID:2090028] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-05-13 16:33:45,174][WARNING][PID:2090028] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-05-13 16:33:55,194][ INFO][PID:2090028] Downloading the builder-live.log file, attempt 2 [2025-05-13 16:33:55,200][ INFO][PID:2090028] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@35.173.126.15 copr-rpmbuild-log [2025-05-13 16:34:55,284][ ERROR][PID:2090028] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-05-13 16:35:05,290][WARNING][PID:2090028] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-05-13 16:35:15,292][ INFO][PID:2090028] Downloading the builder-live.log file, attempt 3 [2025-05-13 16:35:15,305][ INFO][PID:2090028] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@35.173.126.15 copr-rpmbuild-log [2025-05-13 16:36:15,410][ ERROR][PID:2090028] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-05-13 16:36:25,418][WARNING][PID:2090028] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-05-13 16:36:35,425][ INFO][PID:2090028] Downloading the builder-live.log file, attempt 4 [2025-05-13 16:36:35,435][ INFO][PID:2090028] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@35.173.126.15 copr-rpmbuild-log [2025-05-13 16:37:35,511][ ERROR][PID:2090028] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-05-13 16:37:45,578][ ERROR][PID:2090028] Giving up for unstable SSH, failures: 1747154025.1744194, 1747154105.290405, 1747154185.4186666, 1747154265.5784583 [2025-05-13 16:37:45,582][ ERROR][PID:2090028] Re-try request for task on 'ResallocHost, ticket_id=2963978, hostname=35.173.126.15, name=aws_x86_64_spot_prod_03893447_20250513_162619, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-05-13 16:37:45,586][ INFO][PID:2090028] Releasing VM back to pool [2025-05-13 16:37:45,637][ INFO][PID:2090028] Retry #1 (on other host) [2025-05-13 16:37:45,664][ INFO][PID:2090028] Trying to allocate VM: ResallocHost, ticket_id=2964499, requested_tags=['copr_builder', 'arch_x86_64'] [2025-05-13 16:48:04,385][ INFO][PID:2090028] Allocated host ResallocHost, ticket_id=2964499, hostname=35.175.183.54, name=aws_x86_64_normalreserved_prod_03894544_20250513_163806, requested_tags=['copr_builder', 'arch_x86_64'] [2025-05-13 16:48:04,399][ INFO][PID:2090028] Allocating ssh connection to builder [2025-05-13 16:48:04,407][ INFO][PID:2090028] Checking that builder machine is OK [2025-05-13 16:48:04,848][ INFO][PID:2090028] Running remote command: copr-builder-ready epel-9-x86_64 [2025-05-13 16:48:05,145][ INFO][PID:2090028] Checking Red Hat subscription (attempt #1) Red Hat subscription active Builder is ready to be used [2025-05-13 16:48:05,149][ INFO][PID:2090028] Filling build.info file with builder info [2025-05-13 16:48:05,156][ INFO][PID:2090028] Not re-notifying FE and msg buses for the new host. [2025-05-13 16:48:05,158][ INFO][PID:2090028] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9033529-epel-9-x86_64 --chroot epel-9-x86_64 --detached [2025-05-13 16:48:05,604][ INFO][PID:2090028] Downloading the builder-live.log file, attempt 1 [2025-05-13 16:48:05,608][ INFO][PID:2090028] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@35.175.183.54 copr-rpmbuild-log [2025-05-13 16:48:10,612][ INFO][PID:2090028] Periodic builder liveness probe: alive [2025-05-13 16:54:42,102][ INFO][PID:2090028] Downloading results from builder [2025-05-13 16:54:42,105][ INFO][PID:2090028] rsyncing of mockbuilder@35.175.183.54:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata started [2025-05-13 16:54:42,111][ INFO][PID:2090028] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@35.175.183.54:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/ &> /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/build-09033529.rsync.log [2025-05-13 16:54:43,264][ INFO][PID:2090028] rsyncing finished. [2025-05-13 16:54:43,269][ INFO][PID:2090028] Releasing VM back to pool [2025-05-13 16:54:43,313][ INFO][PID:2090028] Searching for 'success' file in resultdir [2025-05-13 16:54:43,318][ INFO][PID:2090028] Going to sign pkgs from source: 9033529-epel-9-x86_64 in chroot: /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64 [2025-05-13 16:54:43,325][ INFO][PID:2090028] Calling '/bin/sign -u @oisf#suricata-7.0-testing@copr.fedorahosted.org -p' (attempt #1) [2025-05-13 16:54:43,603][ INFO][PID:2090028] Calling '/bin/sign -4 -h sha256 -u @oisf#suricata-7.0-testing@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/suricata-7.0.10-2.el9.src.rpm' (attempt #1) [2025-05-13 16:54:44,848][ INFO][PID:2090028] signed rpm: /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/suricata-7.0.10-2.el9.src.rpm [2025-05-13 16:54:44,850][ INFO][PID:2090028] Calling '/bin/sign -4 -h sha256 -u @oisf#suricata-7.0-testing@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/suricata-7.0.10-2.el9.x86_64.rpm' (attempt #1) [2025-05-13 16:54:45,604][ INFO][PID:2090028] signed rpm: /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/suricata-7.0.10-2.el9.x86_64.rpm [2025-05-13 16:54:45,606][ INFO][PID:2090028] Calling '/bin/sign -4 -h sha256 -u @oisf#suricata-7.0-testing@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/suricata-debuginfo-7.0.10-2.el9.x86_64.rpm' (attempt #1) [2025-05-13 16:54:46,617][ INFO][PID:2090028] signed rpm: /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/suricata-debuginfo-7.0.10-2.el9.x86_64.rpm [2025-05-13 16:54:46,620][ INFO][PID:2090028] Calling '/bin/sign -4 -h sha256 -u @oisf#suricata-7.0-testing@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/suricata-debugsource-7.0.10-2.el9.x86_64.rpm' (attempt #1) [2025-05-13 16:54:47,532][ INFO][PID:2090028] signed rpm: /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/suricata-debugsource-7.0.10-2.el9.x86_64.rpm [2025-05-13 16:54:47,540][ INFO][PID:2090028] Sign done [2025-05-13 16:54:47,550][ INFO][PID:2090028] Incremental createrepo run, adding 09033529-suricata into https://download.copr.fedorainfracloud.org/results/@oisf/suricata-7.0-testing/epel-9-x86_64, (auto-create-repo=True) [2025-05-13 16:54:47,559][ INFO][PID:2090028] Running command 'copr-repo --batched /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64 --add 09033529-suricata --no-appstream-metadata' as PID 2350628 [2025-05-13 16:54:52,963][ INFO][PID:2090028] Finished after 5 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64 --add 09033529-suricata --no-appstream-metadata) [2025-05-13 16:54:52,966][ INFO][PID:2090028] Getting build details [2025-05-13 16:54:52,968][ INFO][PID:2090028] Listing built binary packages in /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata [2025-05-13 16:54:52,971][ INFO][PID:2090028] Built packages: ['suricata-debugsource 7.0.10', 'suricata-debuginfo 7.0.10', 'suricata 7.0.10'] [2025-05-13 16:54:52,975][ INFO][PID:2090028] build details: {'built_packages': 'suricata-debugsource 7.0.10\nsuricata-debuginfo 7.0.10\nsuricata 7.0.10'} [2025-05-13 16:54:52,980][ INFO][PID:2090028] Retrieving pubkey [2025-05-13 16:54:52,982][ INFO][PID:2090028] Calling '/bin/sign -u @oisf#suricata-7.0-testing@copr.fedorahosted.org -p' (attempt #1) [2025-05-13 16:54:53,311][ INFO][PID:2090028] Added pubkey for user @oisf project suricata-7.0-testing into: /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/pubkey.gpg [2025-05-13 16:54:53,315][ INFO][PID:2090028] Finished build: id=9033529 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@oisf/suricata-7.0-testing chroot=epel-9-x86_64 [2025-05-13 16:54:53,330][ INFO][PID:2090028] Worker succeeded build, took 408.1739535331726 [2025-05-13 16:54:53,335][ INFO][PID:2090028] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@oisf", "project_name": "suricata-7.0-testing", "project_dirname": "suricata-7.0-testing", "submitter": "jasonish", "ended_on": 1747155293.3302324, "started_on": 1747154885.1562788, "submitted_on": null, "status": 1, "chroot": "epel-9-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9033529-epel-9-x86_64", "build_id": 9033529, "package_name": "suricata", "package_version": "1:7.0.10-2", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@oisf/suricata-7.0-testing/suricata", "git_hash": "044fc2356b4dc640e550549215be780661bb37c5", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@oisf/suricata-7.0-testing--jasonish", "results": { "packages": [ { "name": "suricata-debugsource", "epoch": 1, "version": "7.0.10", "release": "2.el9", "arch": "x86_64" }, { "name": "suricata-debuginfo", "epoch": 1, "version": "7.0.10", "release": "2.el9", "arch": "x86_64" }, { "name": "suricata", "epoch": 1, "version": "7.0.10", "release": "2.el9", "arch": "src" }, { "name": "suricata", "epoch": 1, "version": "7.0.10", "release": "2.el9", "arch": "x86_64" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@oisf/suricata-7.0-testing/epel-9-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@oisf/suricata-7.0-testing", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@oisf/suricata-7.0-testing", "result_dir": "09033529-suricata", "built_packages": "suricata-debugsource 7.0.10\nsuricata-debuginfo 7.0.10\nsuricata 7.0.10", "id": 9033529, "mockchain_macros": { "copr_username": "@oisf", "copr_projectname": "suricata-7.0-testing", "vendor": "Fedora Project COPR (@oisf/suricata-7.0-testing)" } } ] } [2025-05-13 16:54:53,419][ INFO][PID:2090028] Sending fedora-messaging bus message in build.end [2025-05-13 16:54:53,470][ INFO][PID:2090028] Compressing /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/builder-live.log by gzip [2025-05-13 16:54:53,477][ INFO][PID:2090028] Running command 'gzip /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/builder-live.log' as PID 2351191 [2025-05-13 16:54:53,576][ INFO][PID:2090028] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/builder-live.log) [2025-05-13 16:54:53,579][ INFO][PID:2090028] Compressing /var/lib/copr/public_html/results/@oisf/suricata-7.0-testing/epel-9-x86_64/09033529-suricata/backend.log by gzip