Stream: Coq devs & plugin devs

Topic: ✔ cannot create regular file '.csdp.cache': Permission de...


view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:04):

Running make -f Makefile.dune test-suite, I get:

dune runtest --no-buffer
ocamlc test-suite/.ocaml_pwd.eobjs/byte/dune__exe__Ocaml_pwd.{cmi,cmo,cmt}
ocamlopt test-suite/.ocaml_pwd.eobjs/native/dune__exe__Ocaml_pwd.{cmx,o}
ocamlopt test-suite/ocaml_pwd.exe
ocaml_pwd test-suite/bin.inc
ocaml_pwd test-suite/libpath.inc
make[1]: Entering directory '/home/mdenes/dev/coq/master/_build/default/test-suite'
cp .csdp.cache.test-suite .csdp.cache
cp: cannot create regular file '.csdp.cache': Permission denied

Does it ring a bell? Stdlib builds fine, so it is a bit strange. The files produced in _build/default have indeed read-only permissions.

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:04):

Deleting _build and rebuilding does not fix the problem.

view this post on Zulip Ali Caglayan (Mar 21 2022 at 13:11):

It's probably best not to use the top makefile for the test-suite for the moment

view this post on Zulip Ali Caglayan (Mar 21 2022 at 13:11):

Just do make -C test-suite

view this post on Zulip Gaëtan Gilbert (Mar 21 2022 at 13:12):

the Makefile.dune test suite target is terrible, but trying to remove it seems like it leads to a political fight so users can keep the pain

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:12):

Pffff

view this post on Zulip Théo Zimmermann (Mar 21 2022 at 13:15):

I agree that it doesn't make sense to keep this target until the test-suite is properly dunerized (which I have no idea when it will happen).

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:16):

Should I open a PR?

view this post on Zulip Gaëtan Gilbert (Mar 21 2022 at 13:17):

also if you do make -C test-suite next time you run dune it will complain about summary.log because of this target

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:18):

is dune build && make -C test-suite expected to work?

view this post on Zulip Gaëtan Gilbert (Mar 21 2022 at 13:18):

y

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:19):

Thanks, will try. But we need an ETA for fixing the mess. Removing non-working targets looks like a very clear win.

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:20):

We can add them back when they are ready.

view this post on Zulip Ali Caglayan (Mar 21 2022 at 13:22):

I am in favor of removing the test-suite targets from the main makefile.

view this post on Zulip Ali Caglayan (Mar 21 2022 at 13:23):

The plan is to eventually remove the makefile for the test-suite, but the ETA for that is unknown. When that happens, correct targets in the main makefile can be added again.

view this post on Zulip Pierre-Marie Pédrot (Mar 21 2022 at 13:24):

please don't change the toplevel commands again, I am starting to get feed up to have to relearn muscle memory every two weeks

view this post on Zulip Pierre-Marie Pédrot (Mar 21 2022 at 13:25):

make test-suite needs to do the job, whatever it does

view this post on Zulip Pierre-Marie Pédrot (Mar 21 2022 at 13:25):

(in dune mode, that is)

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:26):

So how do I fix my problem?

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:26):

I guess I'd need a bit more than "don't use it", then :)

view this post on Zulip Ali Caglayan (Mar 21 2022 at 13:26):

@Pierre-Marie Pédrot are you able to use make test-suite currently?

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:27):

Ali Caglayan said:

Pierre-Marie Pédrot are you able to use make test-suite currently?

Btw, don't we have a CI to answer this kind of questions?

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:29):

The test-suite jobs don't use these targets anyway

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:32):

Actually, test-suite:edge:dune does

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:33):

That seems a problem with dune cache + test suite, what dune version are you using?

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:33):

maybe we should pass --cache=disabled to that target

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:34):

2.9.3

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:34):

Is the cache enabled by default?

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:34):

no

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:34):

but that's a global setting

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:35):

3.0 has a cache rewrite, anyways it is much faster for dev work, but I think that problem may not be fixed by the 3.0 cache rewrite

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:35):

but just the speedup for zero builds is very large

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:35):

λ cat ~/.config/dune/config
(lang dune 2.8)
(display short)

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:35):

how do I check if it is enabled?

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:36):

it seems it is not,

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:36):

I don't know how to check other than the log

view this post on Zulip Pierre-Marie Pédrot (Mar 21 2022 at 13:36):

FWIW make test-suite works for me but I'm not a cache user

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:37):

I guess a bug report with full info may help to see what the problem is

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:37):

full info I mean "reproduction instructions"

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:38):

tho as I meintoned the other day, more info on why the flags field doens't work for you, would be nice

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:38):

maybe there is a bug in how we handle the prelude, but that'd be an obvious one line fix for 2.9.4

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:41):

The pb with broken build systems is a bit like with UIs, building a repro case may be hard when reaching the inconsistent state takes a few specific steps.

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:43):

But I'll try

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:46):

Maxime Dénès said:

The pb with broken build systems is a bit like with UIs, building a repro case may be hard when reaching the inconsistent state takes a few specific steps.

Indeed that's a problem.

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:48):

Do we have an ETA for conversion of the test suite build to dune?

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:48):

Not asking this to put pressure, it could be more effective for me to just wait.

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:49):

For now the experience is I spend 50% on my dev time on build issues, but I'm confident it is going to be much better once the few big issues are fixed.

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:49):

(and the make vs dune redundancy resolved)

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:49):

However, without an estimate on when it lands, I can't take an informed decision on how to schedule my work.

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 13:52):

I don't know how to give an ETA

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:54):

I think we should discuss it, because I think we'll need it, maybe under some hypotheses. All core dev activity is potentially impacted by work on the build system, by definition.

view this post on Zulip Maxime Dénès (Mar 21 2022 at 13:57):

Is there a precise description on the next steps written somewhere, in a very detailed way that could serve as a basis for estimating the resources needed?

view this post on Zulip Maxime Dénès (Mar 21 2022 at 14:00):

(maybe a collection of low level issues)

view this post on Zulip Maxime Dénès (Mar 21 2022 at 14:10):

I think I found the root of the pb

view this post on Zulip Maxime Dénès (Mar 21 2022 at 14:10):

make clean does (among other things) :

find _build_vo/default//lib/coq/ test-suite \( -name '*.vo' -o -name '*.vio' -o -name '*.vos' -o -name '*.vok' -o -name '*.glob' \) -exec rm -f {} +

view this post on Zulip Maxime Dénès (Mar 21 2022 at 14:10):

it does not delete .log files

view this post on Zulip Maxime Dénès (Mar 21 2022 at 14:11):

and then, when running make -f Makefile.dune test-suite, unlike other targets, it does not complain about generated artifacts in the source tree

view this post on Zulip Maxime Dénès (Mar 21 2022 at 14:11):

but copies them as read-only, which then makes the build fail

view this post on Zulip Maxime Dénès (Mar 21 2022 at 14:13):

So switching back to dune after a call to make, even after make clean does not work if test suite was run. Otherwise it works fine.

view this post on Zulip Ali Caglayan (Mar 21 2022 at 17:10):

make clean should ideally just call the clean target of the test-suite makefile

view this post on Zulip Ali Caglayan (Mar 21 2022 at 17:11):

or specifically the copy in _build

view this post on Zulip Ali Caglayan (Mar 21 2022 at 17:11):

Since that is known to work

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 17:32):

Maxime Dénès said:

Is there a precise description on the next steps written somewhere, in a very detailed way that could serve as a basis for estimating the resources needed?

There is not, anyways it is common knowledge by build experts that such an estimation is far from easy c.f. (again) https://signalsandthreads.com/build-systems/

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 17:34):

I think what would help much more than documenting what are likely low-level steps, is to specify and document the use cases developers want from the build system

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 17:35):

already Makefile.dune tries to do this, however we are having some trouble with the ongoing migration for the reasons already explained

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 17:36):

Anyways many things people could help have been already detailed and so far we are not seeing a lot of improvement; similarly with other non-build domains. Such lists are written, I spend hours writing long mails, I never hear back.

view this post on Zulip Emilio Jesús Gallego Arias (Mar 21 2022 at 17:36):

So not something I look forward a lot to keep doing


Last updated: Feb 05 2023 at 20:03 UTC