Stream: coqbot devs & users

Topic: How to troubleshoot coqbot?


view this post on Zulip Imran Iqbal (Dec 12 2020 at 16:51):

Hi, we've tried to use the instructions at https://github.com/coq/bot/#how-to-use-the-coqbot-instance to deploy coqbot across our GitHub org (https://github.com/saltstack-formulas). So far, coqbot only shows up for PRs that are made using branches from the parent repository (non-forked), which GitLab can already handle. E.g. https://github.com/saltstack-formulas/test-formula/pull/2/checks (whereas PR #1 sent from a fork didn't work). What steps are available to troubleshoot this further?

view this post on Zulip Enrico Tassi (Dec 12 2020 at 16:59):

@Théo Zimmermann please move this post to the coqbot stream

view this post on Zulip Notification Bot (Dec 12 2020 at 16:59):

This topic was moved here from #Miscellaneous > How to troubleshoot coqbot? by Théo Zimmermann

view this post on Zulip Imran Iqbal (Dec 12 2020 at 17:53):

Enrico Tassi said:

Théo Zimmermann please move this post to the coqbot stream

Ah, sorry. This stream didn't show up on the left-hand side by default. Thanks for moving it.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 20:47):

I've tried various combinations (of GitHub/GitLab repos and repo paths) with the bot; is there any advantage of using the legacy method (as a regular user account)? I've added it to one repo but currently at Awaiting coqbot’s response -- any idea how long this will take before the invitation is accepted?

view this post on Zulip Imran Iqbal (Dec 12 2020 at 20:49):

OK, the invitation has just been accepted; now to continue testing...

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 20:49):

Dear @Imran Iqbal, with the legacy method, I have to manually validate the invitation. I've just done that.

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 20:49):

I'm available to help you debug your setup now.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 20:49):

Théo Zimmermann said:

Dear Imran Iqbal, with the legacy method, I have to manually validate the invitation. I've just done that.

Thanks for that. Is it a more stable method to use?

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 20:51):

In principle, both should work well. Both are currently in use across various organizations.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 20:51):

OK, I just force-pushed a PR and I can see the outgoing webhook was successful (200). No pipeline in GitLab yet.

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 20:52):

If I understand correctly, you've first tried the GitHub App method?

view this post on Zulip Imran Iqbal (Dec 12 2020 at 20:53):

Yes, I tried the app method first.

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 20:54):

Can you give me a link to a PR from a fork for which it didn't work?

view this post on Zulip Imran Iqbal (Dec 12 2020 at 20:54):

coqbot would be displayed if the PR branch was from the parent repo itself (which GitLab handles anyway) -- I couldn't get it involved with a forked repo... OK, https://github.com/saltstack-formulas/test-formula/pull/4.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 20:55):

^ The force-push I just did sent the outgoing webhook fine from GitHub. Nothing received in GitLab, though (checked webhook there).

view this post on Zulip Imran Iqbal (Dec 12 2020 at 20:56):

I've simplified the GitLab end by matching it up with the GitHub side (https://gitlab.com/saltstack-formulas/test-formula).

view this post on Zulip Imran Iqbal (Dec 12 2020 at 20:58):

Originally tried the repo in the place we wanted it and set the coqbot.toml as mentioned in the README. Thought I'd remove that configuration and just get it working in the basic way first.

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:03):

I've got something weird in the logs. It might be a bug. I'll need to investigate more.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:04):

@Théo Zimmermann OK, shall I check back in an hour or so?

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:04):

Can you confirm first that you didn't forget to give coqbot access to your GitLab repo?

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:05):

Absolutely gave it, with Developer permissions:

 coqbot @coqbot
Given access 1 hour ago

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:07):

OK. I'll look into this more. I'll ping you when I get more info.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:07):

Thanks, I appreciate it.

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:28):

I've identified a first problem linked to the mapping:
https://github.com/saltstack-formulas/test-formula/blob/398a4165f4bd5544d0e2a2b53939b085b60743de/coqbot.toml
The current implementation is unfortunately expecting a repository in the form owner/name.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:30):

If you look, I've removed that file and gone to a basic setup without the coqbot.toml file.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:30):

I've even moved the repository in the GitLab side to match the GitHub side.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:33):

@Théo Zimmermann Is this enough or would I have to set up another test repo with owner/name right from the beginning?

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:33):

It should be fine.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:34):

So what's the next step I can take?

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:34):

I guess you also disabled the GitHub App.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:35):

Yes, so that I could try the account method instead. I tried the GitHub App with the coqbot.toml removed as well.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:35):

(And the repo paths matching on both sides).

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:35):

OK

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:41):

OK, I've found what the issue was.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:42):

Excellent! So there's hope...?

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:43):

Yes, you should see that there are new status checks on https://github.com/saltstack-formulas/test-formula/pull/4

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:43):

Checking...

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:44):

So basically, the main issue was the one I've already explained.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:44):

The owner/repo issue?

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:44):

Yes. And the second issue was that the data was not updated when you removed the file.

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:45):

So basically, I've restarted the bot to remove the data from memory.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:45):

OK, there's one other repo where I configured coqbot.toml -- will that work OK if I change it now?

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:45):

Yes, it should.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:46):

So would it be better if I go back to using the GitHub App instead? We're planning to use this across ~100 repos in our org.

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:46):

Yes, definitely.

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:46):

Both easier to setup and you get more features.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:47):

Excellent, I'm going to try some stuff out now and I'll get back to you if there are any issues. Really appreciate the help!

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:48):

Would you mind opening two issues on the bot's repository to remind me of fixing the two we found out?

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:48):

That way, you'll also be informed when I fix them.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:48):

What was the second issue?

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:49):

That the modification to the coqbot.toml file was not taken into account.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:50):

Ah, OK -- I'll do that shortly.

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:50):

Thanks!

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:50):

BTW, there was another difference I noticed when following the instructions:

The bot will only take care of mirroring the PRs and reporting status checks back so you may still want to activate the mirroring feature for the main branches. To do so, the easiest way is to choose the "CI/CD for external repo" option when creating the GitLab repository. However, you should opt to give the repo by URL rather than with the GitHub button, because we won't need GitLab's own status check reporting feature. (If it is already activated, you can disable this integration in the "Settings" / "Integration" menu).

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:51):

Oh! That's interesting!

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:51):

The sync itself still works but I had to manually trigger it each time.

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:53):

I've heard from another user (@Erik Martin-Dorel) that had the same problem.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:54):

And specifically this part:

(If it is already activated, you can disable this integration in the "Settings" / "Integration" menu)

This is a GitLab bug but it can't be disabled if it's already set up!

Would this even be a problem if we did use the GitHub button instead of giving the repo by URL?

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:56):

Would this even be a problem if we did use the GitHub button instead of giving the repo by URL?

You would get duplicated status reported to your GitHub repo.

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 21:56):

(one by GitLab and one by coqbot)

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:57):

Oh dear, the merge commit performed by coqbot is affecting our commitlint check: https://gitlab.com/saltstack-formulas/test-formula/-/jobs/907388641#L34.

[CI merge] PR #5: docs(readme): make minor adjustment to test PR GitLab pipeline (from fork)

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:58):

The merge commit is a great idea, since it behaves like Travis, etc.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 21:58):

Now I'm going to have to figure out how to ignore that particular commit and check the commits from the PR instead...

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 22:01):

Théo Zimmermann said:

Would this even be a problem if we did use the GitHub button instead of giving the repo by URL?

You would get duplicated status reported to your GitHub repo.

Now that I think about it, you wouldn't get duplicated status reported to your PRs, only to your branches. (Because of these merge commits.)

view this post on Zulip Imran Iqbal (Dec 12 2020 at 22:02):

OK, I may just try setting up the CICD links in the usual way and see how that affects things.

view this post on Zulip Imran Iqbal (Dec 12 2020 at 22:02):

BTW, I've gone back to using the GitHub App and it appears to be working fine!

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 22:02):

Cool!

view this post on Zulip Imran Iqbal (Dec 12 2020 at 22:03):

OK, let me add those two issues that you mentioned...

view this post on Zulip Théo Zimmermann (Dec 12 2020 at 22:03):

Thanks!

view this post on Zulip Imran Iqbal (Dec 17 2020 at 15:20):

@Théo Zimmermann Thanks for helping out earlier, we've got coqbot configured for ~95 repos and it seems to be working really well -- we got caught out badly by https://gitlab.com/gitlab-org/gitlab/-/issues/5667 (which is where we heard about coqbot as a potential workaround).

A couple of questions:

  1. Are there any commands that we can send to @coqbot, such as restarting the pipeline or individual jobs?
  2. Is there any chance of a solution for https://github.com/coq/bot/issues/120 soon? We've got different types of repo under our main org, which is why we wanted to separate repos by namespace (e.g. saltstack-formulas/formulas/github/saltstack-formulas/test-formula instead of saltstack-formulas/test-formula). Would be glad to test any developmental efforts.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 20:17):

@Imran Iqbal Regarding 1., since you've installed with the GitHub App in the end, you should see "Re-run" buttons on failed jobs and pipelines. I've just opened a PR updating the doc. See https://github.com/coq/bot/pull/122. More specifically, see https://github.com/Zimmi48/bot/blob/document-re-run/README.md#github-checks-features.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 20:18):

Thanks, I'll check that out.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 20:27):

Théo Zimmermann said:

Imran Iqbal Regarding 1., since you've installed with the GitHub App in the end, you should see "Re-run" buttons on failed jobs and pipelines. I've just opened a PR updating the doc. See https://github.com/coq/bot/pull/122. More specifically, see https://github.com/Zimmi48/bot/blob/document-re-run/README.md#github-checks-features.

This would be good if it could be set for other repos as well; we have this problem quite often: https://github.com/Zimmi48/bot/blob/document-re-run/README.md#post-comment-when-a-pull-request-does-not-respect-certain-standards.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 20:32):

Do you wish for the same exact comment or for a configurable message?

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 20:32):

Regarding 2., I've just pushed a fix. I'm pretty confident it will work, but you need to wait for deployment to be complete before you can test it.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 20:33):

Even the same comment would be useful for the time being. Ultimately, end users would prefer to be able to customise that to their needs. Probably in the coqbot.toml, right?

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 20:33):

Right.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 20:33):

Théo Zimmermann said:

Regarding 2., I've just pushed a fix. I'm pretty confident it will work, but you need to wait for deployment to be complete before you can test it.

Great! Please ping me when it's ready to test.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 20:33):

You can check this: https://github.com/coq/bot/runs/1572857089. It should be a few minutes.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 20:34):

Sure; I've got a test repo set up already.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 20:34):

Great!

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 20:35):

Can you open an issue about the comment feature so that I don't forget?

view this post on Zulip Imran Iqbal (Dec 17 2020 at 20:35):

Sure, I'll do that soon, just juggling a couple of things at the minute...

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:04):

Théo Zimmermann said:

You can check this: https://github.com/coq/bot/runs/1572857089. It should be a few minutes.

Interesting! It's definitely working, the pipeline has been triggered from the PR that is introducing coqbot.toml (without it even being in the master branch just yet). The integration isn't showing up in the PR but that will probably work once coqbot.toml is available in the master branch.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:08):

Yes! A second PR after coqbot.toml is in the master branch has resulted in the coqbot integration working. I've got other issues now with pre-commit but that doesn't appear to have anything to do with this change. Looks like all is working well!

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:09):

Can you give me a link to the first PR? I'm surprised by what you describe.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:09):

I was expecting only the configuration file in the default branch to have an effect.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:10):

Sure: https://github.com/saltstack-formulas/test-formula/pull/9

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:10):

This was the pipeline that was triggered by that PR: https://gitlab.com/saltstack-formulas/formulas/github/saltstack-formulas/test-formula/-/pipelines/231565413.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:12):

OK the explanation for the first PR is simply that GitLab has created a redirection from https://gitlab.com/saltstack-formulas/test-formula to https://gitlab.com/saltstack-formulas/formulas/github/saltstack-formulas/test-formula.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:13):

Right, so if I hadn't moved the repo, it probably wouldn't have worked.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:13):

So new repos under that namespace wouldn't work until the coqbot.toml is available.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:13):

Yes, that's what I would expect.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:14):

Ah, my pre-commit problem was due to caching, so that's fixed as well. I'll probably take the chance and move our repos over. Or perhaps I'll just create them fresh, to avoid the redirects. Thanks again!

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:18):

FYI after checking the log, there might still be remaining issues. Please do not move all your repositories to the new location just yet and be ready to report any issue you notice in practice to me.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:19):

Any specific checks you want me to make on the test repo? There won't be much more activity other than there, since I won't be moving any of the other repos over after what you've just mentioned.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:21):

OK so the issue is actually related to something I've already discussed with my intern this summer and had ask him to open an issue about but I don't think he did.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:21):

Basically, we store a correspondence table between GitHub and GitLab repository.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:22):

When we are called on a GitHub repo, we can check the coqbot.toml file in the default branch if we don't know the correspondence yet.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:22):

Right.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:22):

But when we are called on a GitLab repo, we don't have a fallback other than trying the same full name on the GitHub side if we don't have the correspondence in store.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:23):

There are two consequences of this:

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:23):

  1. when you move your repository to the new location, some status checks report will be lost (I think this is what I saw in the log).

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:24):

when you move your repository to the new location

Would the same happen if I recreated the repo in the new location, instead of moving it?

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:25):

No because there would be no remaining status checks.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:25):

The second consequence is possibly more problematic.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:26):

  1. If the bot is killed while a pipeline is running (can happen on redeploy or if the pipelines run for more than 30 minutes and there is no activity on any repository in the meantime---then Heroku can kill the bot), then some status checks could also be lost for the same reason.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:27):

And that wouldn't happen with the default repo location?

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:27):

No, because that would be the fallback location if there is nothing in the correspondence table.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:27):

I don't know how much it would be an actual problem in practice though.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:27):

Gotcha.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:28):

But anyway, there is a clear fix which is to load all the config files for all the repositories where the bot is installed when it restarts.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:28):

Yes, that sounds appropriate. I'll be available for further testing if you need me!

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:28):

I'll try to find some time to implement it during the Christmas holidays.

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:29):

Thanks again for your feedback (and don't forget about opening the issue we talked about earlier today).

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:29):

Of course, whenever that's available. We're good for the time being. Thanks for your help. I'll get around to adding that other issue soon.

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:29):

Still juggling!

view this post on Zulip Théo Zimmermann (Dec 17 2020 at 21:29):

Perfect, thanks!

view this post on Zulip Imran Iqbal (Dec 17 2020 at 21:30):

No problem. Bye for now.


Last updated: Jan 31 2023 at 09:01 UTC