**** BEGIN LOGGING AT Wed Nov 30 02:59:56 2022 Nov 30 07:46:58 good morning **** BEGIN LOGGING AT Wed Nov 30 13:02:59 2022 Nov 30 14:06:20 Does bitbake offer a way to list all available build targets? So a list that would include all PROVIDES for all not skipped recipes including the `virtual/*` targets and possible `world` as well? Nov 30 14:30:09 bitbake-layers show-recipes Nov 30 14:32:16 that only shows recipes, but not all build targets. It wouldn't show `virtual/kernel` for instance, but you can use it to build the currently preferred kernel recipe. Nov 30 14:37:07 It also doesn't list `perl-native` so even `pn-buildlist` after `bitbake -g world` is more complete... Nov 30 14:47:49 `bitbake -s` also shows `perl-native` but still no `virtual/*` packages or `world`... Nov 30 22:39:10 is it somehow possible to conditionally modify the DEPENDS of a package, making the decision dependend on changes in the git repo which the SRC_URI is pointing to? Nov 30 22:39:26 I mean at build time, not as a fixed distro config Nov 30 22:40:55 basically I have a terrible setup which I can not get rid of, where auth tokens are being commited to a submodule twice per day and those are needed to access some npm dependencies which the main recipe needs; of course this is causing unnecessary builds... so I am figuring if I could somehow first evaluate if sources for the main recipe have changed, and if they did - only then trigger the build of Nov 30 22:40:57 the dependency package which would fetch the access token needed for building Nov 30 23:00:43 Jin^eLD: something like DEPENDS = "${@'A' if 'B' in d.getVar('SRC_URI') else 'C'}" Nov 30 23:01:21 Jin^eLD: ah, you said in the remote repo. That is harder :( Nov 30 23:01:52 Jin^eLD: you could probably put a function call in there which checked but it wouldn't be nice or perform well Nov 30 23:02:05 yes.. its terrible, two git repos, one is meant to be a submodule but I split them up into two independent packages Nov 30 23:02:20 into two recipes I mean Nov 30 23:02:55 so I am looking for if SRC_URI of A says stuff changed, then DEPENDS = "add B", otherwise don't Nov 30 23:06:26 RP: I wonder if there is an event which could help me? looing into bb.event now, but not sure if that makes sense Nov 30 23:07:17 or anything I could hook onto that decides that "my code has changed due to new commits, I need to rebuild" - thats where I would try to hack DEPENDS Nov 30 23:07:42 Jin^eLD: you basically need to make bitbake parse the recipe every time to trigger the check Nov 30 23:08:29 the "A" recipe is part of my image, so it would get looked at as a dependency to the image Nov 30 23:08:59 the idea is that inside "A" I want to DEPEND on "B" only if "A" sees it has new git commits in its SRC_URI repo Nov 30 23:09:42 Jin^eLD: I get the idea but remember that bitbake caches DEPENDS, it doesn't reparse the recipe on every call. It also has no idea when it needs to reparse the recipe Nov 30 23:10:10 so in order to make DEPENDS change, it would have to reparse that recipe every time Nov 30 23:10:10 ooh, thats what you meant Nov 30 23:10:20 right, now I get it Nov 30 23:10:42 so one more "how to do it" question Nov 30 23:11:19 BB_DONT_CACHE = "1" in the recipe Nov 30 23:11:42 oh that's simple, cool :) Nov 30 23:12:18 so then the more difficult part I guess - any hint where to hook into the decision if a rebuild of A is needed due to its own SRC_URI change or not? Nov 30 23:12:26 or rather contents of the SRC_URI Nov 30 23:13:06 Jin^eLD: in most cases people make the recipe AUTOREV since if the upstream revision changes, they want the rebuild Nov 30 23:13:27 autorev recipes automatically set don't cache Nov 30 23:14:12 that would work on the devel setup, but not on a release setup where I pin down what to build using tags, however that damn auth token repo has to remain AUTOREV of course Nov 30 23:14:37 I do not care if the token repo gets rebuilt too often, that is fast and its not part of the resulting image Nov 30 23:14:54 i.e. thats my "B" package in the scenario Nov 30 23:14:58 in meta-openamp we just moved to a discrete and -dev that disable the -dev version unless set in preferred version.. that seems to work well with the AUTOREV Nov 30 23:15:12 sorry not preferred version, I mean preferred provider Nov 30 23:15:30 * RP -> afk Nov 30 23:15:38 fray: could you elaborate a bit, I am not yet getting it Nov 30 23:15:38 (modeled after linux-yocto and linux-yocto-dev) Nov 30 23:15:51 let me look at linux-yocto-dev Nov 30 23:16:00 https://github.com/OpenAMP/meta-openamp/tree/master/recipes-openamp/libmetal Nov 30 23:16:13 You'll see there is a libmetal_version.bb and a libmetal-dev_git.bb Nov 30 23:16:36 By default you build "libmetal" and get libmetal_version.bb. If you try to run libmetal-dev you will get an error the recipe is not available.. Nov 30 23:16:52 if you set PREFERRED_PROVIDER_libmetal = "libmetal-dev" and then build either libmetal or libmetal-dev you will get libmetal-dev Nov 30 23:17:14 the common libmetal.inc is where the 'magic' is.... Nov 30 23:17:35 right, but I do not yet see the connection to the problem I am trying to solve...? or am I missing it Nov 30 23:17:41 note though in libmetal-dev_git.bb, the SRCREV setting.. that is important as well, it prevents even disabled items from triggering AUTOREV Nov 30 23:17:48 basically here you decide if you are preferring the pinned or the AUTOREV version Nov 30 23:17:51 Maybe I misunderstood your issue.. Nov 30 23:18:20 We wanted to EITHER use a pinned version or AUTOREV... but if we pinned, we never wanted AUTOREV to show up or it would cause off-line build issues Nov 30 23:18:37 this allows the PREFERRED_VERSION to select the pinned or autorev explicitly.. and solves the second issue as well Nov 30 23:18:53 OK, I get it, but that is not the problem I am facing Nov 30 23:19:03 ok, sorry I misunderstood then Nov 30 23:19:40 my issue is that I can only build A (no matter if its pinned or autoreved) if I checked out "B" which has to be autorev Nov 30 23:19:56 because "B" provides some login crap which A needs to fetch npm dependencies from Nov 30 23:20:31 What is your DEPENDS set to? Nov 30 23:20:46 B: DEPENDS = "A" ? Nov 30 23:20:53 other way around Nov 30 23:20:58 A: DEPENDS = "B" Nov 30 23:21:08 because you can't build A without B, but you only need B if A has changed Nov 30 23:21:53 A depends on B, but you only want B if you are going to rebuild A? Nov 30 23:22:02 yes, exactly! Nov 30 23:22:49 AFAIK there is no way to do that. Each recipe is independent of the others. If B is autorev, then the only way for A to know (depends) that B has or has not changed is to query upstream for the rev and hope the resulting hash is the same Nov 30 23:23:12 no, no, wait Nov 30 23:23:16 A does not need to know if B has changed Nov 30 23:23:30 A needs to know if A has changed and add B as its DEPENDS only if A did indeed change Nov 30 23:23:38 If A depends on B, then it does have to know if B has changed... Nov 30 23:24:19 The dependency system doesn't know if a change to B will change A or not. So depends initiates a "cehck if B changed, if B changed then A changes" behavior Nov 30 23:24:42 but it does so only if A says DEPENDS = "B", right? Nov 30 23:24:44 It might be possible to use hash-exclusions on a to disable that behavior... MAYBE something like: Nov 30 23:25:57 basically, A uses some output from B and B changes often, A changes seldom and builds for a long time Nov 30 23:26:06 (in A) DEPEND_ON_B = "B" DEPENDS += "${DEPEND_ON_B}" DEPENDS[vardepsexclude] += "DEPEND_ON_B" Nov 30 23:26:18 but A only needs to be rebuilt if A changes, it does not care about the interim changes in b Nov 30 23:26:31 that is going to say.. A depends on B.. but the hash of A is not dependency upon B.. (maybe) Nov 30 23:26:56 not sure that will work, as the task hash brings in depdnencies automatically and I don't know if that will actually exclude the dependencies Nov 30 23:26:57 what effect would that have? Nov 30 23:27:03 h Nov 30 23:27:05 h Nov 30 23:27:10 "hm" is what I wanted to write :) Nov 30 23:27:14 simple case, ignoring dependencies Nov 30 23:27:18 recipe has: Nov 30 23:27:28 A = 1 B = 2 C = 3 Nov 30 23:27:38 but if I decide to rebuild "A" because of changes in "A", will it _then_ also rebuild "B"? Nov 30 23:27:38 that results in a recipe dependency of 0xABCD Nov 30 23:27:57 if you change A to be = 2.. then the recipe hash (not dependency) could become 0xBCDE Nov 30 23:28:08 system says BCDE isn't the same as ABCD so it needs to build Nov 30 23:28:18 with vardepsexclude you can do something like: Nov 30 23:28:44 A_PRIME = '1' A = '${A_PRIME}' A[vardepsexclude] = 'A_PRIME' B = 2 C = 3 Nov 30 23:28:53 the hash will change if A, B or C change.. but NOT if A_PRIME changes Nov 30 23:29:33 let me try that Nov 30 23:29:33 extending this into dependencies is where I don't know what happens. I know know if vardepexcluding a DEPENDS component will change the dependeny hashes Nov 30 23:30:30 but, just thinking back of my original idea... is there a way/event in "A" to know that "A" now needs to be rebuilt, because the fetcher detected a commit in A's SRC_URI? Nov 30 23:30:32 but in the normal case, if you declare B is needed for A to build, then if B changes the hash of A changes Nov 30 23:31:30 I don't know. You can assume though if a do_build runs it has to be built (there may be other tasks). but tasks of A can NOT affect behavior of B.. only the _OUTPUT_ of the build of one recipe can affect another (through the sysroot) Nov 30 23:32:06 but for do_build it would be too late to inject "B" ass a dependency... Nov 30 23:32:31 I would want to intercept that earlier and then say "OK, depdency of A:do_build is B Nov 30 23:32:37 or something like that Nov 30 23:32:47 correct Nov 30 23:32:57 Anytime AFTER parse is too late Nov 30 23:33:12 and any event saying "I'm going to build this" will be after parse Nov 30 23:33:47 with two distinct recipes, I don't think there is anyway to do what you want without someone convincing the system to ignore hash values you would normally inherit on B.. Nov 30 23:34:26 trying your vardepsexclude suggestion now... Nov 30 23:34:27 if you ever use bitbake-diffsigs, often it will tell you package you depend on changed which is why YOUR recipe changed.. that is the only way to break this.. remove that dependency and I'm not sure there is a way to do it Nov 30 23:34:43 I knwo vardepsexclude works WITHIN a recipe, just not sure beyond that scope Nov 30 23:34:58 well, if it was one recipe - I can not have two git SRC_URIs I think and I also can not ignore changes in a submodule if B was a submodule? or can I? Nov 30 23:35:45 yes you can Nov 30 23:35:56 I do? how? Nov 30 23:35:57 easy to have multiple git.... Nov 30 23:36:21 just two space separated git lines? Nov 30 23:36:35 where does it end up then, I mean in ${S} terms? Nov 30 23:36:49 finding an example Nov 30 23:37:18 question is if that would help me at all though? Nov 30 23:37:56 because then I would somehow need a way to say "fetch git uri 2 only if git uri 1 changes" Nov 30 23:38:46 SRC_URI = "git://github.com/Xilinx/linux-xlnx.git;protocol=https;name=machine \\n git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-5.15;destsuffix=yocto-kmeta" Nov 30 23:39:04 that downloads two different ones. The destsuffix is the name of the directory it's placed into Nov 30 23:39:09 the 'name' is how the component is referred to.. Nov 30 23:39:32 so instead of SRCREV it becomes SRCREV_machine = "..." and SRCREV_kmeta = "..." (ignore 'type', that isn't relevant here) Nov 30 23:39:51 ok, let me think Nov 30 23:40:05 so the 'machine' goes into the 'git' directory (default), while kmeta goes into 'yocto-kmeta' (bothin the WORKDIR) Nov 30 23:40:31 right Nov 30 23:40:40 but I am still thinking of how that would help me now Nov 30 23:41:56 I have a "leading" tree and a "subtree" (the same A and B analogy) and I want to rebuild the whole recipe only if the leading tree changes, but not if the subtree changes Nov 30 23:42:11 at least in a single recipe.. then you can probably use the vardepsexclude or something else to exclude checking the second (B) item Nov 30 23:42:46 but if I exclude checking the B item, will it rebuild B if it builds A? or will it always ignore B? Nov 30 23:43:44 because once I see "OK, I need to build A", then I want the HEAD (AUTOREV) of B Nov 30 23:43:48 there is probably a way to say only use SRCREV_A not SRCREV_B for hashing.. but I'm not sure what it is Nov 30 23:44:30 AUTOREV is a do it or don't prospect.. thats why I always tell people not to use AUTOREV... I'm really not sure if this can help Nov 30 23:45:03 I am testing your original vadepsexclude via DEPENDS solution now, lets see if that actually solves it Nov 30 23:51:50 ok so first part looks good, when I build "A" it rebuilds B if there are changes in B, Nov 30 23:52:37 now lets see how it copes with changes in A and then with changes in A and in B, but I think that first part was actually what I wanted Dec 01 00:03:13 almost Dec 01 00:03:40 so when I build A its fine and behaves as I would epect it Dec 01 00:03:43 *expect Dec 01 00:03:55 but now C (the rootfs) depends on A Dec 01 00:04:17 and it seems C gets rebuild when B gets rebuild, eventhough B is not part of C and A did nto get rebuild due to the vardepsexcudle Dec 01 00:04:38 but this can probaby be tuned by adding anothe exclude on B on the rootfs Dec 01 00:37:45 hmm, can't figure out the chain how B triggers a C rebuild Dec 01 00:38:57 something for tomorrow I guess Dec 01 00:39:34 fray, RP: thank you for your help! I did get a bit further **** ENDING LOGGING AT Thu Dec 01 02:59:57 2022