disclaimer: (this is my first time properly using Yocto)
Today evening I was smashing out many stupid fixes left and right for Yocto recepies that weren’t building. (I was trying to build my own yocto-image for my TinkerBoard 2… see: getting my tinkerboard up and running) One of those “stupid fixes” I just find so hilarious in terms of:
- the sheer struggle to hunt endlessly for bugs with the most simple/stupid origin
- the “how the most minor changes can cause endless trouble in places soooo far away” with programming (especially in pkging)
- also a lot the stupidity/ridiculousness of the state of Open Source Software Rene Rebe often talks about in his Live Packaging sessions (where he very often encounters similar bugs you chase endlessly because of some in-your-eyes stupid and tiny change)
- a Error-Message of a program being FAR from what the actual problem is
This bug was the recepie src/yocto/poky/meta/recipes-connectivity/mobile-broadband-provider-info/mobile-broadband-provider-info_git.bb do_fetch failing. and it took me about 1:30h to figure out the cause of it…
so I looked in the log file for this recipie, where I found at the bottom (as you expect errors to be at the bottom because the program would exit when encountering an error… right????? WRONG more later)
DEBUG: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; export PATH="/source/yocto/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/source/yocto/poky/scripts:/source/yocto/build/tmp/work/armv8a-poky-linux/mobile-broadband-provider-info/1_20210805-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux:/source/yocto/build/tmp/work/armv8a-poky-linux/mobile-broadband-provider-info/1_20210805-r0/recipe-sysroot/usr/bin/crossscripts:/source/yocto/build/tmp/work/armv8a-poky-linux/mobile-broadband-provider-info/1_20210805-r0/recipe-sysroot-native/usr/sbin:/source/yocto/build/tmp/work/armv8a-poky-linux/mobile-broadband-provider-info/1_20210805-r0/recipe-sysroot-native/usr/bin:/source/yocto/build/tmp/work/armv8a-poky-linux/mobile-broadband-provider-info/1_20210805-r0/recipe-sysroot-native/sbin:/source/yocto/build/tmp/work/armv8a-poky-linux/mobile-broadband-provider-info/1_20210805-r0/recipe-sysroot-native/bin:/source/yocto/poky/bitbake/bin:/source/yocto/build/tmp/hosttools"; export HOME="/home/root"; LANG=C git -c core.fsyncobjectfiles=0 -c gc.autoDetach=false clone --bare --mirror https://gitlab.gnome.org/git/GNOME/mobile-broadband-provider-info.git /source/yocto/build/../packages/git2/gitlab.gnome.org.git.GNOME.mobile-broadband-provider-info.git --progress failed with exit code 128, no output
ERROR: Fetcher failure: Unable to find revision 11f2247eccd3c161b8fd9b41143862e9fb81193c in branch master even from upstream
ERROR: Fetcher failure for URL: 'git://gitlab.gnome.org/GNOME/mobile-broadband-provider-info.git;protocol=https;branch=master'. Unable to fetch URL from any source.
DEBUG: Python function base_do_fetch finished
DEBUG: Python function do_fetch finished
right above the ‘ERROR Fetcher failure’ there was a DEBUG which showed a command, which I would expect to be the command causing the ‘ERROR Fetcher failure’ in the next line (AGAIN WRONG more later), so I Inspected this command that is run, which is basically git clone https://gitlab.gnome.org/git/GNOME/mobile-broadband-provider-info.git… I looked at the repo on gitlab.gnome.org, It exists, copied the URL from there and was able to clone it. WTF
It already took me a few minutes to realize the extra “/git” after the domain. Ah I’ve solved it, the SRC_URI simply needs updating to not include this “/git” and done. WRONG again the SRC_URI in mobile-broadband-provider-info_git.bb did not have this wired “/git”. Off I went looking for other files, who could modify the SRC_URI of this recipie or the whole gitlab.gnome.org location… no results. and also bitbake -e mobile-broadband-provider-info showed the correct SRC_URI meaning the “/git” had to come from somewhere outside of the recepie code and inside the yocto fetching logic…
After much talking to ChatGPT about Yocto Fetching Intricasies, googeling if someone had a similar issue and looking through the bitbake src (bitbake/lib/bb/fetch2/git.py … added some print calls 😉) without any results… I eventually realized, that the line from the log ERROR: Fetcher failure for URL: 'git://gitlab.gnome.org/GNOME/mobile-broadband-provider-info.git;protocol=https;branch=master'. Unable to fetch URL from any source. does not have the “/git” … WHERE TF does this /git come from???!!!! (not from any recepie-files, not from the bitbake fetch logic, … I feel like I have checked all places where the URL could be changed)
WELL… I eventually read the more upper parts of the log-file…
WARNING: Failed to fetch URL git://gitlab.gnome.org/GNOME/mobile-broadband-provider-info.git;protocol=https;branch=master, attempting MIRRORS if available
The fetch from the origin URL failed and the command I was looking at was downloading one of the many Mirrors yocto tries out. So the wired URL with the “/git” inserted in the middle of the original URL is simply one mirror Yocto tries after the original URL fails. (I don’t like this behaviour as it caused loads of confusion for me in the log… because this “MIRROR” was the last on the list 🤦♂️) (I do now understand how Yocto MIRROR and PREMIRROR work…)
OK, so then WHY does the original URL fail to download? I can checkout the repo manually!! (also I checked the commit of SRCREV does exist in the repo (https://gitlab.gnome.org/GNOME/mobile-broadband-provider-info/-/commit/11f2247eccd3c161b8fd9b41143862e9fb81193c))
WELL the commands run before the “Failed to fetch URL …” line are some git commands: (all run in ‘/source/yocto/build/../packages/git2/gitlab.gnome.org.GNOME.mobile-broadband-provider-info.git’)
git branch --contains 11f2247eccd3c161b8fd9b41143862e9fb81193c --list master 2> /dev/null | wc -l
git remote
git remote rm origin
git remote add --mirror=fetch origin https://gitlab.gnome.org/GNOME/mobile-broadband-provider-info.git
git fetch -f --progress https://gitlab.gnome.org/GNOME/mobile-broadband-provider-info.git refs/*:refs/*
git fetch -f --progress https://gitlab.gnome.org/GNOME/mobile-broadband-provider-info.git refs/*:refs/*
git prune-packed
git pack-refs --all
git pack-redundant --all | xargs -r rm
git branch --contains 11f2247eccd3c161b8fd9b41143862e9fb81193c --list master 2> /dev/null | wc -l
but none of them failed with any error when I ran them myself…
The ones with “wc -l” resulted in 0 however… BUT I CHECKED THE COMMIT EXISTS… (asked ChatGPT about how “git branch —contains hash —list master” works)
I ran the command without the “wc -l” (no output) and then also without the “—list master” and then I spotted it:
master was renamed to main.
some quick googeling brought up some relevant discussions of Gnome renaming master to main:
- https://discourse.gnome.org/t/renaming-master-branch-to-mainline/841
- the official proposal: https://mail.gnome.org/archives/desktop-devel-list/2019-April/msg00049.html
- on the last sentence: I AM NOT IN as this change cost me 1:30h of my life
- and on “making GNOME a project that everyone wants to use and develop for”… actually this change definetely pushes me away from Gnome (even more)
I mean it is stupid, that there is a branch as part of the SRC_URI (where Yocto then correctly checks if the SRCREV is actually a commit in that branch…), the commit is what identifies a version of a source unambiguously, so why care about the branch as well?
The fact that Yocto tries this wired “/git” mirror as the last one, making the log confusing… also not so great.
But the real thing is… changing names of branches because some people don’t like the words…