Imran Iqbal (Dec 12 2020 at 16:51): Imran Iqbal (Dec 12 2020 at 16:51):Hi, we've tried to use the instructions at https://github.com/coq/bot/#how-to-use-the-coqbot-instance to deploy coqbot across our GitHub org (https://github.com/saltstack-formulas). So far, coqbot only shows up for PRs that are made using branches from the parent repository (non-forked), which GitLab can already handle. E.g. https://github.com/saltstack-formulas/test-formula/pull/2/checks (whereas PR #1 sent from a fork didn't work). What steps are available to troubleshoot this further?
Enrico Tassi (Dec 12 2020 at 16:59):@Théo Zimmermann please move this post to the coqbot stream
Notification Bot (Dec 12 2020 at 16:59):This topic was moved here from #Miscellaneous > How to troubleshoot coqbot? by Théo Zimmermann
Imran Iqbal (Dec 12 2020 at 17:53):Enrico Tassi said:
Théo Zimmermann please move this post to the coqbot stream
Ah, sorry. This stream didn't show up on the left-hand side by default. Thanks for moving it.
Imran Iqbal (Dec 12 2020 at 20:47):I've tried various combinations (of GitHub/GitLab repos and repo paths) with the bot; is there any advantage of using the legacy method (as a regular user account)? I've added it to one repo but currently at Awaiting coqbot’s response -- any idea how long this will take before the invitation is accepted?
Imran Iqbal (Dec 12 2020 at 20:49):OK, the invitation has just been accepted; now to continue testing...
Théo Zimmermann (Dec 12 2020 at 20:49):Dear @Imran Iqbal, with the legacy method, I have to manually validate the invitation. I've just done that.
Théo Zimmermann (Dec 12 2020 at 20:49):I'm available to help you debug your setup now.
Imran Iqbal (Dec 12 2020 at 20:49):Théo Zimmermann said:
Dear Imran Iqbal, with the legacy method, I have to manually validate the invitation. I've just done that.
Thanks for that. Is it a more stable method to use?
Théo Zimmermann (Dec 12 2020 at 20:51):In principle, both should work well. Both are currently in use across various organizations.
Imran Iqbal (Dec 12 2020 at 20:51):OK, I just force-pushed a PR and I can see the outgoing webhook was successful (200). No pipeline in GitLab yet.
Théo Zimmermann (Dec 12 2020 at 20:52):If I understand correctly, you've first tried the GitHub App method?
Imran Iqbal (Dec 12 2020 at 20:53):Yes, I tried the app method first.
Théo Zimmermann (Dec 12 2020 at 20:54):Can you give me a link to a PR from a fork for which it didn't work?
Imran Iqbal (Dec 12 2020 at 20:54):coqbot would be displayed if the PR branch was from the parent repo itself (which GitLab handles anyway) -- I couldn't get it involved with a forked repo... OK, https://github.com/saltstack-formulas/test-formula/pull/4.
Imran Iqbal (Dec 12 2020 at 20:55):^ The force-push I just did sent the outgoing webhook fine from GitHub. Nothing received in GitLab, though (checked webhook there).
Imran Iqbal (Dec 12 2020 at 20:56):I've simplified the GitLab end by matching it up with the GitHub side (https://gitlab.com/saltstack-formulas/test-formula).
Imran Iqbal (Dec 12 2020 at 20:58):Originally tried the repo in the place we wanted it and set the coqbot.toml as mentioned in the README. Thought I'd remove that configuration and just get it working in the basic way first.
Théo Zimmermann (Dec 12 2020 at 21:03):I've got something weird in the logs. It might be a bug. I'll need to investigate more.
Imran Iqbal (Dec 12 2020 at 21:04):@Théo Zimmermann OK, shall I check back in an hour or so?
Théo Zimmermann (Dec 12 2020 at 21:04):Can you confirm first that you didn't forget to give coqbot access to your GitLab repo?
Imran Iqbal (Dec 12 2020 at 21:05):Absolutely gave it, with Developer permissions:
coqbot @coqbot
Given access 1 hour ago
OK. I'll look into this more. I'll ping you when I get more info.
Imran Iqbal (Dec 12 2020 at 21:07):Thanks, I appreciate it.
Théo Zimmermann (Dec 12 2020 at 21:28):I've identified a first problem linked to the mapping:
https://github.com/saltstack-formulas/test-formula/blob/398a4165f4bd5544d0e2a2b53939b085b60743de/coqbot.toml
The current implementation is unfortunately expecting a repository in the form owner/name.
Imran Iqbal (Dec 12 2020 at 21:30):If you look, I've removed that file and gone to a basic setup without the coqbot.toml file.
Imran Iqbal (Dec 12 2020 at 21:30):I've even moved the repository in the GitLab side to match the GitHub side.
Imran Iqbal (Dec 12 2020 at 21:33):@Théo Zimmermann Is this enough or would I have to set up another test repo with owner/name right from the beginning?
Théo Zimmermann (Dec 12 2020 at 21:33):It should be fine.
Imran Iqbal (Dec 12 2020 at 21:34):So what's the next step I can take?
Théo Zimmermann (Dec 12 2020 at 21:34):I guess you also disabled the GitHub App.
Imran Iqbal (Dec 12 2020 at 21:35):Yes, so that I could try the account method instead. I tried the GitHub App with the coqbot.toml removed as well.
Imran Iqbal (Dec 12 2020 at 21:35):(And the repo paths matching on both sides).
Théo Zimmermann (Dec 12 2020 at 21:35):OK
Théo Zimmermann (Dec 12 2020 at 21:41):OK, I've found what the issue was.
Imran Iqbal (Dec 12 2020 at 21:42):Excellent! So there's hope...?
Théo Zimmermann (Dec 12 2020 at 21:43):Yes, you should see that there are new status checks on https://github.com/saltstack-formulas/test-formula/pull/4
Imran Iqbal (Dec 12 2020 at 21:43):Checking...
Théo Zimmermann (Dec 12 2020 at 21:44):So basically, the main issue was the one I've already explained.
Imran Iqbal (Dec 12 2020 at 21:44):The owner/repo issue?
Théo Zimmermann (Dec 12 2020 at 21:44):Yes. And the second issue was that the data was not updated when you removed the file.
Théo Zimmermann (Dec 12 2020 at 21:45):So basically, I've restarted the bot to remove the data from memory.
Imran Iqbal (Dec 12 2020 at 21:45):OK, there's one other repo where I configured coqbot.toml -- will that work OK if I change it now?
Théo Zimmermann (Dec 12 2020 at 21:45):Yes, it should.
Imran Iqbal (Dec 12 2020 at 21:46):So would it be better if I go back to using the GitHub App instead? We're planning to use this across ~100 repos in our org.
Théo Zimmermann (Dec 12 2020 at 21:46):Yes, definitely.
Théo Zimmermann (Dec 12 2020 at 21:46):Both easier to setup and you get more features.
Imran Iqbal (Dec 12 2020 at 21:47):Excellent, I'm going to try some stuff out now and I'll get back to you if there are any issues. Really appreciate the help!
Théo Zimmermann (Dec 12 2020 at 21:48):Would you mind opening two issues on the bot's repository to remind me of fixing the two we found out?
Théo Zimmermann (Dec 12 2020 at 21:48):That way, you'll also be informed when I fix them.
Imran Iqbal (Dec 12 2020 at 21:48):What was the second issue?
Théo Zimmermann (Dec 12 2020 at 21:49):That the modification to the coqbot.toml file was not taken into account.
Imran Iqbal (Dec 12 2020 at 21:50):Ah, OK -- I'll do that shortly.
Théo Zimmermann (Dec 12 2020 at 21:50):Thanks!
Imran Iqbal (Dec 12 2020 at 21:50):BTW, there was another difference I noticed when following the instructions:
The bot will only take care of mirroring the PRs and reporting status checks back so you may still want to activate the mirroring feature for the main branches. To do so, the easiest way is to choose the "CI/CD for external repo" option when creating the GitLab repository. However, you should opt to give the repo by URL rather than with the GitHub button, because we won't need GitLab's own status check reporting feature. (If it is already activated, you can disable this integration in the "Settings" / "Integration" menu).
Théo Zimmermann (Dec 12 2020 at 21:51):Oh! That's interesting!
Imran Iqbal (Dec 12 2020 at 21:51):The sync itself still works but I had to manually trigger it each time.
Théo Zimmermann (Dec 12 2020 at 21:53):I've heard from another user (@Erik Martin-Dorel) that had the same problem.
Imran Iqbal (Dec 12 2020 at 21:54):And specifically this part:
(If it is already activated, you can disable this integration in the "Settings" / "Integration" menu)
This is a GitLab bug but it can't be disabled if it's already set up!
Would this even be a problem if we did use the GitHub button instead of giving the repo by URL?
Théo Zimmermann (Dec 12 2020 at 21:56):Would this even be a problem if we did use the GitHub button instead of giving the repo by URL?
You would get duplicated status reported to your GitHub repo.
Théo Zimmermann (Dec 12 2020 at 21:56):(one by GitLab and one by coqbot)
Imran Iqbal (Dec 12 2020 at 21:57):Oh dear, the merge commit performed by coqbot is affecting our commitlint check: https://gitlab.com/saltstack-formulas/test-formula/-/jobs/907388641#L34.
[CI merge] PR #5: docs(readme): make minor adjustment to test PR GitLab pipeline (from fork)
Imran Iqbal (Dec 12 2020 at 21:58):The merge commit is a great idea, since it behaves like Travis, etc.
Imran Iqbal (Dec 12 2020 at 21:58):Now I'm going to have to figure out how to ignore that particular commit and check the commits from the PR instead...
Théo Zimmermann (Dec 12 2020 at 22:01):Théo Zimmermann said:
Would this even be a problem if we did use the GitHub button instead of giving the repo by URL?
You would get duplicated status reported to your GitHub repo.
Now that I think about it, you wouldn't get duplicated status reported to your PRs, only to your branches. (Because of these merge commits.)
Imran Iqbal (Dec 12 2020 at 22:02):OK, I may just try setting up the CICD links in the usual way and see how that affects things.
Imran Iqbal (Dec 12 2020 at 22:02):BTW, I've gone back to using the GitHub App and it appears to be working fine!
Théo Zimmermann (Dec 12 2020 at 22:02):Cool!
Imran Iqbal (Dec 12 2020 at 22:03):OK, let me add those two issues that you mentioned...
Théo Zimmermann (Dec 12 2020 at 22:03):Thanks!
Imran Iqbal (Dec 17 2020 at 15:20):@Théo Zimmermann Thanks for helping out earlier, we've got coqbot configured for ~95 repos and it seems to be working really well -- we got caught out badly by https://gitlab.com/gitlab-org/gitlab/-/issues/5667 (which is where we heard about coqbot as a potential workaround).
A couple of questions:
@coqbot, such as restarting the pipeline or individual jobs?saltstack-formulas/formulas/github/saltstack-formulas/test-formula instead of saltstack-formulas/test-formula). Would be glad to test any developmental efforts. Théo Zimmermann (Dec 17 2020 at 20:17):@Imran Iqbal Regarding 1., since you've installed with the GitHub App in the end, you should see "Re-run" buttons on failed jobs and pipelines. I've just opened a PR updating the doc. See https://github.com/coq/bot/pull/122. More specifically, see https://github.com/Zimmi48/bot/blob/document-re-run/README.md#github-checks-features.
Imran Iqbal (Dec 17 2020 at 20:18):Thanks, I'll check that out.
Imran Iqbal (Dec 17 2020 at 20:27):Théo Zimmermann said:
Imran Iqbal Regarding 1., since you've installed with the GitHub App in the end, you should see "Re-run" buttons on failed jobs and pipelines. I've just opened a PR updating the doc. See https://github.com/coq/bot/pull/122. More specifically, see https://github.com/Zimmi48/bot/blob/document-re-run/README.md#github-checks-features.
This would be good if it could be set for other repos as well; we have this problem quite often: https://github.com/Zimmi48/bot/blob/document-re-run/README.md#post-comment-when-a-pull-request-does-not-respect-certain-standards.
Théo Zimmermann (Dec 17 2020 at 20:32):Do you wish for the same exact comment or for a configurable message?
Théo Zimmermann (Dec 17 2020 at 20:32):Regarding 2., I've just pushed a fix. I'm pretty confident it will work, but you need to wait for deployment to be complete before you can test it.
Imran Iqbal (Dec 17 2020 at 20:33):Even the same comment would be useful for the time being. Ultimately, end users would prefer to be able to customise that to their needs. Probably in the coqbot.toml, right?
Théo Zimmermann (Dec 17 2020 at 20:33):Right.
Imran Iqbal (Dec 17 2020 at 20:33):Théo Zimmermann said:
Regarding 2., I've just pushed a fix. I'm pretty confident it will work, but you need to wait for deployment to be complete before you can test it.
Great! Please ping me when it's ready to test.
Théo Zimmermann (Dec 17 2020 at 20:33):You can check this: https://github.com/coq/bot/runs/1572857089. It should be a few minutes.
Imran Iqbal (Dec 17 2020 at 20:34):Sure; I've got a test repo set up already.
Théo Zimmermann (Dec 17 2020 at 20:34):Great!
Théo Zimmermann (Dec 17 2020 at 20:35):Can you open an issue about the comment feature so that I don't forget?
Imran Iqbal (Dec 17 2020 at 20:35):Sure, I'll do that soon, just juggling a couple of things at the minute...
Imran Iqbal (Dec 17 2020 at 21:04):Théo Zimmermann said:
You can check this: https://github.com/coq/bot/runs/1572857089. It should be a few minutes.
Interesting! It's definitely working, the pipeline has been triggered from the PR that is introducing coqbot.toml (without it even being in the master branch just yet). The integration isn't showing up in the PR but that will probably work once coqbot.toml is available in the master branch.
Imran Iqbal (Dec 17 2020 at 21:08):Yes! A second PR after coqbot.toml is in the master branch has resulted in the coqbot integration working. I've got other issues now with pre-commit but that doesn't appear to have anything to do with this change. Looks like all is working well!
Théo Zimmermann (Dec 17 2020 at 21:09):Can you give me a link to the first PR? I'm surprised by what you describe.
Théo Zimmermann (Dec 17 2020 at 21:09):I was expecting only the configuration file in the default branch to have an effect.
Imran Iqbal (Dec 17 2020 at 21:10):Sure: https://github.com/saltstack-formulas/test-formula/pull/9
Imran Iqbal (Dec 17 2020 at 21:10):This was the pipeline that was triggered by that PR: https://gitlab.com/saltstack-formulas/formulas/github/saltstack-formulas/test-formula/-/pipelines/231565413.
Théo Zimmermann (Dec 17 2020 at 21:12):OK the explanation for the first PR is simply that GitLab has created a redirection from https://gitlab.com/saltstack-formulas/test-formula to https://gitlab.com/saltstack-formulas/formulas/github/saltstack-formulas/test-formula.
Imran Iqbal (Dec 17 2020 at 21:13):Right, so if I hadn't moved the repo, it probably wouldn't have worked.
Imran Iqbal (Dec 17 2020 at 21:13):So new repos under that namespace wouldn't work until the coqbot.toml is available.
Théo Zimmermann (Dec 17 2020 at 21:13):Yes, that's what I would expect.
Imran Iqbal (Dec 17 2020 at 21:14):Ah, my pre-commit problem was due to caching, so that's fixed as well. I'll probably take the chance and move our repos over. Or perhaps I'll just create them fresh, to avoid the redirects. Thanks again!
Théo Zimmermann (Dec 17 2020 at 21:18):FYI after checking the log, there might still be remaining issues. Please do not move all your repositories to the new location just yet and be ready to report any issue you notice in practice to me.
Imran Iqbal (Dec 17 2020 at 21:19):Any specific checks you want me to make on the test repo? There won't be much more activity other than there, since I won't be moving any of the other repos over after what you've just mentioned.
Théo Zimmermann (Dec 17 2020 at 21:21):OK so the issue is actually related to something I've already discussed with my intern this summer and had ask him to open an issue about but I don't think he did.
Théo Zimmermann (Dec 17 2020 at 21:21):Basically, we store a correspondence table between GitHub and GitLab repository.
Théo Zimmermann (Dec 17 2020 at 21:22):When we are called on a GitHub repo, we can check the coqbot.toml file in the default branch if we don't know the correspondence yet.
Imran Iqbal (Dec 17 2020 at 21:22):Right.
Théo Zimmermann (Dec 17 2020 at 21:22):But when we are called on a GitLab repo, we don't have a fallback other than trying the same full name on the GitHub side if we don't have the correspondence in store.
Théo Zimmermann (Dec 17 2020 at 21:23):There are two consequences of this:
Théo Zimmermann (Dec 17 2020 at 21:23): Imran Iqbal (Dec 17 2020 at 21:24):when you move your repository to the new location
Would the same happen if I recreated the repo in the new location, instead of moving it?
Théo Zimmermann (Dec 17 2020 at 21:25):No because there would be no remaining status checks.
Théo Zimmermann (Dec 17 2020 at 21:25):The second consequence is possibly more problematic.
Théo Zimmermann (Dec 17 2020 at 21:26): Imran Iqbal (Dec 17 2020 at 21:27):And that wouldn't happen with the default repo location?
Théo Zimmermann (Dec 17 2020 at 21:27):No, because that would be the fallback location if there is nothing in the correspondence table.
Théo Zimmermann (Dec 17 2020 at 21:27):I don't know how much it would be an actual problem in practice though.
Imran Iqbal (Dec 17 2020 at 21:27):Gotcha.
Théo Zimmermann (Dec 17 2020 at 21:28):But anyway, there is a clear fix which is to load all the config files for all the repositories where the bot is installed when it restarts.
Imran Iqbal (Dec 17 2020 at 21:28):Yes, that sounds appropriate. I'll be available for further testing if you need me!
Théo Zimmermann (Dec 17 2020 at 21:28):I'll try to find some time to implement it during the Christmas holidays.
Théo Zimmermann (Dec 17 2020 at 21:29):Thanks again for your feedback (and don't forget about opening the issue we talked about earlier today).
Imran Iqbal (Dec 17 2020 at 21:29):Of course, whenever that's available. We're good for the time being. Thanks for your help. I'll get around to adding that other issue soon.
Imran Iqbal (Dec 17 2020 at 21:29):Still juggling!
Théo Zimmermann (Dec 17 2020 at 21:29):Perfect, thanks!
Imran Iqbal (Dec 17 2020 at 21:30):No problem. Bye for now.
Last updated: May 28 2023 at 18:29 UTC