[2024-11-07 16:54:07,415][ INFO][PID:2215961] Marking build as starting [2024-11-07 16:54:07,472][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 16:58:25,585][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5756811, hostname=169.63.97.9 [2024-11-07 16:58:25,586][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 16:58:25,587][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 16:58:26,148][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 16:58:26,429][ INFO][PID:2215961] [2024-11-07 16:58:26,430][ INFO][PID:2215961] Warning: Permanently added '169.63.97.9' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 16:58:26,430][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5756811, hostname=169.63.97.9': Builder wasn't ready, trying a new one [2024-11-07 16:58:26,431][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 16:58:26,441][ INFO][PID:2215961] Retry #1 (on other host) [2024-11-07 16:58:26,442][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 16:58:29,529][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5756838, hostname=169.63.97.9 [2024-11-07 16:58:29,530][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 16:58:29,532][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 16:58:29,817][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 16:58:30,085][ INFO][PID:2215961] [2024-11-07 16:58:30,086][ INFO][PID:2215961] Warning: Permanently added '169.63.97.9' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 16:58:30,086][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5756838, hostname=169.63.97.9': Builder wasn't ready, trying a new one [2024-11-07 16:58:30,087][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 16:58:30,097][ INFO][PID:2215961] Retry #2 (on other host) [2024-11-07 16:58:30,098][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 16:58:33,187][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5756839, hostname=169.63.97.9 [2024-11-07 16:58:33,188][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 16:58:33,189][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 16:58:33,492][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 16:58:33,776][ INFO][PID:2215961] [2024-11-07 16:58:33,776][ INFO][PID:2215961] Warning: Permanently added '169.63.97.9' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 16:58:33,777][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5756839, hostname=169.63.97.9': Builder wasn't ready, trying a new one [2024-11-07 16:58:33,777][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 16:58:33,787][ INFO][PID:2215961] Retry #3 (on other host) [2024-11-07 16:58:33,788][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 16:58:36,870][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5756840, hostname=169.63.97.9 [2024-11-07 16:58:36,871][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 16:58:36,871][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 16:58:37,184][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 16:58:37,480][ INFO][PID:2215961] [2024-11-07 16:58:37,480][ INFO][PID:2215961] Warning: Permanently added '169.63.97.9' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 16:58:37,481][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5756840, hostname=169.63.97.9': Builder wasn't ready, trying a new one [2024-11-07 16:58:37,481][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 16:58:37,492][ INFO][PID:2215961] Retry #4 (on other host) [2024-11-07 16:58:37,492][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 16:58:40,521][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5756841, hostname=169.63.97.9 [2024-11-07 16:58:40,521][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 16:58:40,522][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 16:58:40,826][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 16:58:41,086][ INFO][PID:2215961] [2024-11-07 16:58:41,086][ INFO][PID:2215961] Warning: Permanently added '169.63.97.9' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 16:58:41,087][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5756841, hostname=169.63.97.9': Builder wasn't ready, trying a new one [2024-11-07 16:58:41,087][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 16:58:41,112][ INFO][PID:2215961] Retry #5 (on other host) [2024-11-07 16:58:41,113][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 16:58:44,138][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5756843, hostname=169.63.97.9 [2024-11-07 16:58:44,139][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 16:58:44,139][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 16:58:44,414][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 16:58:44,695][ INFO][PID:2215961] [2024-11-07 16:58:44,696][ INFO][PID:2215961] Warning: Permanently added '169.63.97.9' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 16:58:44,696][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5756843, hostname=169.63.97.9': Builder wasn't ready, trying a new one [2024-11-07 16:58:44,696][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 16:58:44,705][ INFO][PID:2215961] Retry #6 (on other host) [2024-11-07 16:58:44,706][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 16:58:47,795][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5756844, hostname=169.63.97.9 [2024-11-07 16:58:47,796][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 16:58:47,796][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 16:58:48,081][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 16:58:48,353][ INFO][PID:2215961] [2024-11-07 16:58:48,354][ INFO][PID:2215961] Warning: Permanently added '169.63.97.9' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 16:58:48,354][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5756844, hostname=169.63.97.9': Builder wasn't ready, trying a new one [2024-11-07 16:58:48,355][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 16:58:48,366][ INFO][PID:2215961] Retry #7 (on other host) [2024-11-07 16:58:48,367][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 16:58:51,453][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5756845, hostname=169.63.97.9 [2024-11-07 16:58:51,454][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 16:58:51,454][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 16:58:51,780][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 16:58:52,046][ INFO][PID:2215961] [2024-11-07 16:58:52,046][ INFO][PID:2215961] Warning: Permanently added '169.63.97.9' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 16:58:52,047][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5756845, hostname=169.63.97.9': Builder wasn't ready, trying a new one [2024-11-07 16:58:52,047][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 16:58:52,056][ INFO][PID:2215961] Retry #8 (on other host) [2024-11-07 16:58:52,056][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 16:58:55,154][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5756846, hostname=169.63.97.9 [2024-11-07 16:58:55,155][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 16:58:55,155][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 16:58:55,453][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 16:58:55,736][ INFO][PID:2215961] [2024-11-07 16:58:55,737][ INFO][PID:2215961] Warning: Permanently added '169.63.97.9' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 16:58:55,737][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5756846, hostname=169.63.97.9': Builder wasn't ready, trying a new one [2024-11-07 16:58:55,738][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 16:58:55,747][ INFO][PID:2215961] Retry #9 (on other host) [2024-11-07 16:58:55,747][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 17:03:13,887][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5756847, hostname=169.62.17.158 [2024-11-07 17:03:13,888][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 17:03:13,888][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 17:03:14,429][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 17:03:14,691][ INFO][PID:2215961] [2024-11-07 17:03:14,691][ INFO][PID:2215961] Warning: Permanently added '169.62.17.158' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 17:03:14,692][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5756847, hostname=169.62.17.158': Builder wasn't ready, trying a new one [2024-11-07 17:03:14,692][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 17:03:14,708][ INFO][PID:2215961] Retry #10 (on other host) [2024-11-07 17:03:14,709][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 17:04:02,827][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5756913, hostname=169.62.17.158 [2024-11-07 17:04:02,828][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 17:04:02,828][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 17:04:03,394][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 17:04:03,680][ INFO][PID:2215961] [2024-11-07 17:04:03,682][ INFO][PID:2215961] Warning: Permanently added '169.62.17.158' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 17:04:03,682][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5756913, hostname=169.62.17.158': Builder wasn't ready, trying a new one [2024-11-07 17:04:03,683][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 17:04:03,707][ INFO][PID:2215961] Retry #11 (on other host) [2024-11-07 17:04:03,708][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 17:04:21,833][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5756995, hostname=169.62.17.158 [2024-11-07 17:04:21,833][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 17:04:21,834][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 17:04:22,346][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 17:04:22,611][ INFO][PID:2215961] [2024-11-07 17:04:22,612][ INFO][PID:2215961] Warning: Permanently added '169.62.17.158' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 17:04:22,613][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5756995, hostname=169.62.17.158': Builder wasn't ready, trying a new one [2024-11-07 17:04:22,613][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 17:04:22,629][ INFO][PID:2215961] Retry #12 (on other host) [2024-11-07 17:04:22,631][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 17:04:50,764][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5757037, hostname=169.62.17.158 [2024-11-07 17:04:50,765][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 17:04:50,765][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 17:04:51,339][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 17:04:51,632][ INFO][PID:2215961] [2024-11-07 17:04:51,633][ INFO][PID:2215961] Warning: Permanently added '169.62.17.158' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 17:04:51,634][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5757037, hostname=169.62.17.158': Builder wasn't ready, trying a new one [2024-11-07 17:04:51,634][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 17:04:51,675][ INFO][PID:2215961] Retry #13 (on other host) [2024-11-07 17:04:51,679][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 17:05:19,805][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5757093, hostname=169.62.17.158 [2024-11-07 17:05:19,806][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 17:05:19,806][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 17:05:20,392][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 17:05:20,675][ INFO][PID:2215961] [2024-11-07 17:05:20,676][ INFO][PID:2215961] Warning: Permanently added '169.62.17.158' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 17:05:20,677][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5757093, hostname=169.62.17.158': Builder wasn't ready, trying a new one [2024-11-07 17:05:20,677][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 17:05:20,696][ INFO][PID:2215961] Retry #14 (on other host) [2024-11-07 17:05:20,697][ INFO][PID:2215961] Trying to allocate VM [2024-11-07 17:07:38,880][ INFO][PID:2215961] Allocated host ResallocHost, ticket_id=5757147, hostname=150.239.227.147 [2024-11-07 17:07:38,881][ INFO][PID:2215961] Allocating ssh connection to builder [2024-11-07 17:07:38,881][ INFO][PID:2215961] Checking that builder machine is OK [2024-11-07 17:07:39,371][ INFO][PID:2215961] Running remote command: copr-builder-ready fedora-rawhide-s390x [2024-11-07 17:07:39,636][ INFO][PID:2215961] [2024-11-07 17:07:39,637][ INFO][PID:2215961] Warning: Permanently added '150.239.227.147' (ED25519) to the list of known hosts. bash: line 1: copr-builder-ready: command not found [2024-11-07 17:07:39,637][ ERROR][PID:2215961] Re-try request for task on 'ResallocHost, ticket_id=5757147, hostname=150.239.227.147': Builder wasn't ready, trying a new one [2024-11-07 17:07:39,638][ INFO][PID:2215961] Releasing VM back to pool [2024-11-07 17:07:39,652][ ERROR][PID:2215961] Backend process error: Three host tried without success: {'150.239.227.147', '169.63.97.9', '169.62.17.158'} [2024-11-07 17:07:39,653][WARNING][PID:2215961] Switching not-finished job state to 'failed' [2024-11-07 17:07:39,654][ INFO][PID:2215961] Worker failed build, took 812.2411410808563 [2024-11-07 17:07:39,655][ INFO][PID:2215961] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "packit", "project_name": "systemd-systemd-35076", "project_dirname": "systemd-systemd-35076", "submitter": "packit", "ended_on": 1730999259.6537323, "started_on": 1730998447.4125912, "submitted_on": null, "status": 0, "chroot": "fedora-rawhide-s390x", "arch": "s390x", "buildroot_pkgs": [], "task_id": "8227029-fedora-rawhide-s390x", "build_id": 8227029, "package_name": "systemd", "package_version": "257.rc1-1.20241107164754363238.pr35076.24.gaa4723fe9b", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/packit/systemd-systemd-35076/systemd", "git_hash": "3472bb1151f89dd9f6e5a857092cd1232983b21c", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "packit/systemd-systemd-35076--packit", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/packit/systemd-systemd-35076/fedora-rawhide-s390x/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "tags": [ "arch_s390x" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/packit/systemd-systemd-35076", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/packit/systemd-systemd-35076", "result_dir": "08227029-systemd", "built_packages": "", "id": 8227029, "mockchain_macros": { "copr_username": "packit", "copr_projectname": "systemd-systemd-35076", "vendor": "Fedora Project COPR (packit/systemd-systemd-35076)" } } ] } [2024-11-07 17:07:39,743][ INFO][PID:2215961] Sending fedora-messaging bus message in build.end [2024-11-07 17:07:40,506][ INFO][PID:2215961] Compressing /var/lib/copr/public_html/results/packit/systemd-systemd-35076/fedora-rawhide-s390x/08227029-systemd/builder-live.log by gzip [2024-11-07 17:07:40,508][ INFO][PID:2215961] Running command 'gzip /var/lib/copr/public_html/results/packit/systemd-systemd-35076/fedora-rawhide-s390x/08227029-systemd/builder-live.log' as PID 2307084 [2024-11-07 17:07:40,509][ INFO][PID:2215961] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/packit/systemd-systemd-35076/fedora-rawhide-s390x/08227029-systemd/builder-live.log) stdout: stderr: gzip: /var/lib/copr/public_html/results/packit/systemd-systemd-35076/fedora-rawhide-s390x/08227029-systemd/builder-live.log: No such file or directory [2024-11-07 17:07:40,510][ ERROR][PID:2215961] Unable to compress file /var/lib/copr/public_html/results/packit/systemd-systemd-35076/fedora-rawhide-s390x/08227029-systemd/builder-live.log [2024-11-07 17:07:40,511][ INFO][PID:2215961] Compressing /var/lib/copr/public_html/results/packit/systemd-systemd-35076/fedora-rawhide-s390x/08227029-systemd/backend.log by gzip [2024-11-07 17:07:40,512][ INFO][PID:2215961] Running command 'gzip /var/lib/copr/public_html/results/packit/systemd-systemd-35076/fedora-rawhide-s390x/08227029-systemd/backend.log' as PID 2307087