macOS tests are frequently failing during the system package installation phase these days with errors like the following (I had to scroll up a lot to see it):
fatal: unable to access 'https://github.com/Homebrew/homebrew-core/': transfer closed with outstanding read data remaining Error: Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core failed!
It apparently is a GitHub issue that is actively being discussed (and worked on?) at https://github.com/Homebrew/homebrew-core/issues/61772.
The issue still exists and makes many of our pull request checks fail. The Homebrew team has no intention to propose a solution despite the issue affecting lots of users and CI pipelines. They blame GitHub and have closed and locked the issue.
Given their lack of care for this problem but also the various issues that we have had in the past (e.g
pinning), I wonder whether we should not try to remove our dependency on Homebrew, e.g. by switching to MacPorts (@Michael Soegtrop has some experience with it).
Another option (bigger change) would be to try to create the bundle from a Nix build. I'm not even sure it would work, but it would probably be the best possible option with respect to reproducibility.
this issue is probably genuinely due to GitHub, but their support to users generates the same sentiment in most users.
As a partial workaround, they are a bit less aggressive against threads outside of their bug tracker (they have a discourse), which they consider as their TODO list with an “inbox 0” policy.
regarding Nix, do you have a local expert, _and_ can you install Nix on your macOS CI machine? Creating a
/nix “partition” is somewhat invasive (tho they have a wizard for it)
and can you build binaries that are _not_ installed in
/nix, given that binaries really like absolute paths?
Regarding Nix, no, we have knowledgeable people on Nix but not on Nix + macOS. My hope for this solution resided in the understanding that macOS bundles are sorts of isolated filesystems that you just mount when you wish to use them. Maybe I'm mistaken though.
Isn't Vincent knowledgeable in Nix + macOS? We could ask him for advice if needed.
Partly yes. I remember him saying that he was no macOS expert and he used a macOS machine just because that's what IMDEA had given him.
for most Mac users, a .dmg is a filesystem that you mount (at a random place) to copy the contained folder (e.g. CoqIDE.app) into /Applications/CoqIDE-8.12.app. But it’s true that many apps can be run from there, and the app can even be renamed (?), so you’re probably right?
so the question might indeed reduce to “can you create a correct .dmg from Nix”
Yes, and I haven't found answers to this question with a quick Google search so I don't think that it's a common practice.
But let's not forget that I was also suggesting a simpler solution that could be to switch to Mac Ports.
Théo Zimmermann said:
pinning), I wonder whether we should not try to remove our dependency on Homebrew, e.g. by switching to MacPorts (Michael Soegtrop has some experience with it).
The coq platform supports both MacPorts and homebrew. I did quite a few pushes to opam conf packages to add depext lines for MacPorts. For what is needed for core Coq all should be merged upstream meanwhile.
I personally use MacPorts and I would say it is fairly stable. Afaik they have their own infrastructure but it is not said that this is more stable than GitHub longterm.
Probably for our CI we should have our own servers for whatever our CI needs - not only for stability reasons but also to control updates.
This is still an issue, after more than 3 weeks. Homebrew installs fail almost 25% of the time.
We should really look into migrating to MacPorts.
How about Nix? It seems to me that given the status of the OSX package system is the most convenient for users.
Any user can already install Coq on macOS through Nix. If you are talking about using Nix to generate the macOS bundle in our CI, then yes, this is also a solution that I proposed earlier in this thread, although I am not sure how practical it is.
Now it's failing more than half of the time!
It looks like the outstanding problem might have been fixed but the macOS CI is still failing (now all the time) because of some other issue that happened in the meantime:
fatal: 'origin' does not appear to be a git repository fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. fatal: 'origin' does not appear to be a git repository fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. fatal: 'origin' does not appear to be a git repository fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. fatal: 'origin' does not appear to be a git repository fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. Error: Fetching /usr/local/Homebrew/Library/Taps/local/homebrew-openssl failed! Fetching /usr/local/Homebrew/Library/Taps/local/homebrew-python2 failed! fatal: invalid upstream 'origin/master' fatal: invalid upstream 'origin/master'
I am getting a failure here that I don't understand https://github.com/coq/coq/runs/4942920775?check_suite_focus=true
Ah found it
curl: (92) HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1) Error: gtksourceview3: Failed to download resource "gobject-introspection" Download failed: https://ghcr.io/v2/homebrew/core/gobject-introspection/blobs/sha256:6b9375ec9a2e908f441d98379b5cb1b9094ebfbe0f19d1b7a8e35369dfc99052
hopefully this will resolve itself
These HTTP/2 errors seem indeed transient IME, @Michael Soegtrop got this yesterday...
Yes, there seems to be a transient error, see e.g. the discussion at (https://github.com/ocaml/opam/issues/5021)-
It would be interesting to know what curl is used
Last updated: Feb 01 2023 at 16:03 UTC