**** BEGIN LOGGING AT Fri Sep 25 02:59:57 2020 Sep 25 06:22:33 what's the status for 3.1.3 ? on track for release today ? Sep 25 06:44:59 * mcfrisk wonders if finding and fixing bitbake and build script race conditions will be my job until retirement in 25 years? It's been the last 20 or so already. Even bazel had race conditions in bazel installation itself which had decided to download java code into $HOME and failed to auto-update them on build servers. all hope lost? Sep 25 07:00:00 * mcfrisk also wonders when SW crash root causes will finally be detected before seeing them in real product testing. static analyzers and compilers are able to find roughly 95% of them at compile time but developers seem to be really ignorant at those tools. Things like checking return values or pointers in C/C++ are really simple things which everyone seems to get wrong. Not much has improved in last 20 Sep 25 07:00:06 * mcfrisk years, which means next 25 years until my retirement will bethe same story. Or can go and rust really break through and get rid of these? java didn't seem to help much, seen so many null pointer crashes there... I think I need a beer already Sep 25 07:17:37 kergoth: we should definitely start using that for BBHandledException Sep 25 07:18:47 mcfrisk: The time I've been spending with intermittent failures does start to get to you after a while! Sep 25 07:31:50 RP: ignorance is bliss. if one stops ignoring issues and treats all of them as bugs to fix, it easily becomes a full time job and a career. with the broad deployment of continuous integration and test setups, jeez, so many problems become very visible in projects and organizations. Trick is to get order of magnitude improvements and fix the root causes before they cause issues. static analysis for C and Sep 25 07:31:56 C++, validation for build scripts? Sep 25 07:32:58 mcfrisk: how many issues do you see from bitbake itself? Just wondering how we're doing... Sep 25 07:36:00 RP: a bit too many, actually. doing zeus to dunfell updates, it's too easy to get bitbake to 'crash' or hang when meta layers are all messed/mixed up. Once those are sorted, things on yocto and open source side are better. It's usually the company specific SW and build scripts which have races and issues. Though lately also meta-java... Sep 25 07:38:18 oh and the BSP layers.. they can screw everything up. I think some of mine are even corrupting sstate cache somehow. have to wipe tmp for all builds just be sure.. Sep 25 07:56:20 mcfrisk: bad metadata messing up sstate is a tough one. Do you have examples of crashing or hanging bitbake? That sounds like something we should try and avoid Sep 25 07:59:23 RP: bitbake crashes only when meta layers have odd things, wrong branch or some parsing errors. I'm often able to find out manually what the problems are. examples are lost already. Hangs, well also happen for same reasons sometimes. And then there is the 'bitbake -c devshell foo' open for more than 24 hours and exiting console just hangs bitbake. Have to kill processes or build container to recover. Sep 25 08:01:53 mcfrisk: well, this is open source and we are happy to try and improve things if we can Sep 25 08:10:05 Any ideas on how to use that `aws` tool to download stuff from an s3 bucket from a yocto recipe? Sep 25 08:10:45 manuel1985 it a jva tool right Sep 25 08:10:52 java Sep 25 08:11:12 I think it's python but let me look it up Sep 25 08:11:30 regadless the way to go is a native recipe Sep 25 08:12:54 ptsneves: Sep 25 08:12:54 manuel@debian:~$ head -1 /usr/local/bin/aws Sep 25 08:12:54 #!/usr/bin/python3 Sep 25 08:13:14 What do you mean with `native recipe`? Sep 25 08:13:26 a recipe for the build system to use. Sep 25 08:13:32 so to run on the host Sep 25 08:14:54 Yeah that's what I'm intending to do. The question is: Do I have `aws` available from a docker recipe? I would like to run `aws s3 cp s3://myfile ${WORKDIR}`. Didn't try yet, but I guess I don't have `aws` available in a recipe. Sep 25 08:17:39 manuel1985 can you clarify what you mean by running from a docker recipe? Sep 25 08:19:30 I would like to put a `do_fetch()` function in my recipe which downloads this file from the s3 bucket. And in this function I'd call `aws` Sep 25 08:20:33 I can't try it out right now, but I assume it won't work, because in an issue unrelated to this I once tried to run `docker pull` in the same way in a recipe, and I think it told me it can't find `docker` altough it's installed on the machine the build was running on Sep 25 08:20:54 right, inside builds you have minimal commands from the host Sep 25 08:21:06 so that you don't end up relying on stuff on the host which isn't tracked by dependencies Sep 25 08:21:25 the hack is to add aws to HOSTTOOLS, but then you need to document that everyone using the recipe needs to install those things Sep 25 08:21:35 the proper fix is a recipe that installs aws, and DEPEND on that Sep 25 08:22:29 fwiw, there is already a s3 fetcher Sep 25 08:23:25 i suspect it has aged badly as it assumes aws is present on $PATH so will hit the same issue, but that's easily fixed Sep 25 08:28:57 Alright, gonna look for it :) That was valuable information, thanks for the insight on the inner workings of yocto! aws is available in my $PATH so hopefully I won't run into problems :) Sep 25 08:41:50 "inside builds you have minimal commands from the host" is saying that aws *won't* be on $PATH Sep 25 08:41:57 thus the hosttools hack Sep 25 08:42:27 there should be an aws-native recipe that can get magically depended on Sep 25 08:52:00 Hi! I use dunfell. How do I add the kernel .bin file to the boot partition of my SD card? I added INITRAMFS_IMAGE = "core-image-tiny-initramfs" and INITRAMFS_IMAGE_BUNDLE = "1" which bitbakes a file Image-initramfs-imx8mmevk.bin Sep 25 08:53:06 But the core-image-minimal-imx8mmevk.wic.gz doesn't seem to contain that Image-initramfs-imx8mmevk.bin Sep 25 08:54:04 With yocto "sumo" I found out I have to add this line to my conf file: IMAGE_BOOTFILES += "${KERNEL_IMAGETYPE}-initramfs-${MACHINE}.bin" Sep 25 08:54:19 But it doesn't have an effect with dunfell Sep 25 08:59:02 Anyway, when I'm in u-boot and do "fatls mmc 1:1" I just see 2 files: Image and my device tree .dtb file Sep 25 09:03:12 Image is the kernel image? Sep 25 09:04:13 yes Sep 25 09:04:19 RP: I'm going to ping this again :-) https://www.openembedded.org/pipermail/bitbake-devel/2019-May/019998.html Sep 25 09:05:43 ernstp: didn't we try testing that and it caused problems? :/ Sep 25 09:06:27 RP: I'm not aware of that. I can search the ML Sep 25 09:07:02 ernstp: I think my other concern would be special casing specific fetchers in the core code. That is usually a bad sign and we've usually been able to avoid doing it Sep 25 09:07:30 I wish I had the time to sit down and properly look at these things :( Sep 25 09:09:26 RP: 1) git protocol is blocked. 2) we download an git archive tar from a PREMIRROR, everyone is happy. 3) a recipe is updated with a new sha1 4) yocto tries to do "git fetch" in the archive, but it's configured for the git protocol and that's still blocked Sep 25 09:10:03 RP: so this is very much related to the PREMIRROR system, which is a bit more "core" than just the git fetcher itself Sep 25 09:11:03 ernstp: are you doing 2) manually? Sep 25 09:11:33 RP: no that works out of the box Sep 25 09:12:05 RP: there are only a couple of recipes with git protocol gits Sep 25 09:12:07 @ern Sep 25 09:12:16 ernstp what are you trying to solve? Sep 25 09:12:50 ptsneves: I submitted a patch some time ago, pinged it again: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13278 Sep 25 09:13:04 ernstp from your description and commit message it sounds the behavior is already taken care by a fresh clone Sep 25 09:13:21 which would be triggered if the hash was not found in the local downloads Sep 25 09:13:44 ernstp: Can you at least resend the patch with more of the info above. That does make the situation clearer and helps. I'm not promising to merge it but it would help Sep 25 09:13:54 ptsneves: you get ERROR: prelink-native-1.0+gitAUTOINC+05aeafd053-r0 do_unpack: Fetcher failure: Sep 25 09:13:55 fatal: reference is not a tree: 05aeafd053e56356ec8c62f4bb8f7b95bae192f3 Sep 25 09:14:21 ernstp: I note I'm the owner of that bug :( Sep 25 09:14:26 sorry Sep 25 09:14:42 RP: i get an email every time you push it to the next milestone :-) Sep 25 09:14:56 ernstp it is interesting, because we were victims of a similar issue but found that the behavior in yocto was correct. Sep 25 09:15:32 ernstp: I have way more bugs than its feasible for me to handle :( Sep 25 09:15:32 ptsneves: it's annoying to have to go into Jenkins servers and clear their download cache etc... Sep 25 09:16:02 RP: yeah, I'll just ping you once every year :-) Sep 25 09:16:20 but yeah I can resend it to the list, with a longer message Sep 25 09:17:13 fbre: then what's the issue? Sep 25 09:17:17 ernstp: please do that rather than just ping :) Sep 25 09:17:48 there was no testing I think: https://lists.openembedded.org/g/bitbake-devel/search?ev=0&q=%22Replace+old+git+with+mirror%22 Sep 25 09:19:07 ernstp: or I forgot to report back :( Sep 25 09:20:47 RP: might need to rebase it also, master has of course moved... Sep 25 09:23:57 the issue i have with that patch is that it couples the __init__ with git fetcher Sep 25 09:25:09 trying to see an alternative Sep 25 09:26:55 aaaarghhh (facepalm) - the viariable is not IMAGE_BOOTFILES but IMAGE_BOOT_FILES X) :') Sep 25 09:27:11 That has changed from sumo to dunfell Sep 25 09:27:22 now it works Sep 25 09:27:22 ptsneves: i wondered if it could be made more generic, but I wanted to be conservative and limit it as much as possible to reduce risk Sep 25 09:29:36 > 1) git protocol is blocked. 2) we download an git archive tar from a PREMIRROR, everyone is happy. 3) a recipe is updated with a new sha1 4) yocto tries to do "git fetch" in the archive, but it's configured for the git protocol and that's still blocked Sep 25 09:30:07 from this description i do not understand why the git fetch does not try to fetch from the upstream mirror as in the first time Sep 25 09:31:00 it is the issue right? Sep 25 09:44:06 ptsneves: yeah you run into a "Fetcher failure". The exception says "General fetcher exception when something happens incorrectly" Sep 25 09:45:50 ptsneves: yeah ok there should be an additional item in the list Sep 25 09:49:26 it does fetch it from the upstream mirror but it does that to a different path, git.yoctoproject.org.git.prelink-cross.git vs git.yoctoproject.org.prelink-cross.git Sep 25 09:49:38 so that's what my patch does, it fixes that link Sep 25 10:18:30 anyone konws the status for 3.1.3 ? on track for release today ? Sep 25 10:50:21 yann: QA report due next week. Its a little behind Sep 25 10:50:32 yann: it is built and in QA Sep 25 10:54:46 is the tag available somewhere already ? I'm preparing our first dunfell-based release and i'd prefer to use that one from the start Sep 25 10:55:47 for now I've taken 44ab5a84770bf as closest guess I could make :) Sep 25 11:08:25 So I built a generic core-image-minimal qemuarmv5 build in dunfell but I don't get a virtual console working although doing "runqemu ... nographic" is fine. Is that something that should be expected to work as per qemuarm core-image-minimal? Sep 25 11:11:23 if you pass nographic then you should get a login prompt, yes Sep 25 11:45:59 yann: the QA email has the revisions in (sent to the yocto list) Sep 25 11:46:09 yann: basically current tip of dunfell Sep 25 12:44:02 RP: I recieved the 3.2M3 notifications but not the 3.1.3 one Sep 25 12:45:42 in my .conf, I specify a WKS_FILE. Where do I actually put the file? Sep 25 12:47:51 I've copied the directdisk-bootloader-config.wks, common.wks.inc and directdisk-bootloader-config.cfg from ~/poky/scripts/lib, but have no clue where to put them or whether this is the right way of doing things Sep 25 12:49:05 my main objective is to have a custom directdisk-bootloader-config.cfg without changing the one in the poky repo (since touching the poky repo is considered bad practice) Sep 25 12:50:14 RP: in the yocto-3.2_M3.rc2 thread you wrote "The 3.1.3 build should also be arriving shortly" but https://lists.yoctoproject.org/g/yocto/search?q=3.1.3 does not show anything more precise than "3.1.3 will be built soon" afterwards - and I still can't wrap my mind around the ML archive UI to be able to see the latest mails in the list Sep 25 14:34:50 yann: the email didn't make it to the list :( Sep 25 14:36:21 yann: I've resent Sep 25 14:38:42 RP: got it, thx! Sep 25 14:39:37 I was wondering, I remember sometimes seeing a meta-oe sha1 in such reports - what's the rule here ? Sep 25 14:40:23 yann: not sure meta-oe has ever been listed there Sep 25 14:40:43 yann: meta-oe isn't used in the autobuilders release build target Sep 25 14:42:27 ok, I'm probably confusing with a different report Sep 25 14:42:52 we probably should test more but it is what it is Sep 25 14:42:52 meta-oe should not have been in yocto list Sep 25 14:43:34 yann Khen sends out world build status from time to time Sep 25 14:43:40 for meta-oe Sep 25 14:44:22 armpit: Yann Khen? I've not heard of them before :) Sep 25 14:44:42 hehe, forgot the comma Sep 25 14:44:42 :) Sep 25 14:44:56 and misspelled Sep 25 14:50:21 yann, oecore is in the QA listing Sep 25 14:53:30 hi, building dunfell from poky/crops, at the end, bitbake stays a lot of time (>30min) stop on "Summary: There were 39 WARNING messages shown." without completing. Is this normal ? Sep 25 14:54:18 ad__: is it doing something with buildhistory? Its probably doing something but 30mins is a long time Sep 25 14:54:55 ad__: its also possible there are processes which aren't exiting. What does ps ax show as running? Sep 25 14:55:43 Morning folks Sep 25 14:55:55 morning sgw Sep 25 14:55:58 RP: I just send a couple of RFC patches for the target dumping Sep 25 14:56:00 RP, it's stop in this way https://pasteboard.co/JsJje2c.png Sep 25 14:56:14 ad__: doesn't crops need to publish/copy-out sstate and such ? Sep 25 14:56:30 ad__: hmm, so probably not buildhistory Sep 25 14:56:31 (for persistence between builds) Sep 25 14:57:02 biab Sep 25 14:58:01 RP, ps aux | grep python does not show any process Sep 25 14:58:25 i only see containerd-shim and docker run Sep 25 14:58:34 sgw: you're implying we already have a set of commands and they're just not running? Sep 25 14:59:05 ad__: I'm no crops expert but that sounds it should have exited then :/ Sep 25 14:59:27 i tried CTRL+C nothing happen, so, closed the terminal Sep 25 14:59:29 let's see Sep 25 14:59:59 now re-running Sep 25 15:02:35 now on rebuild it completed fine Sep 25 15:02:48 well, thanks anyway for the support Sep 25 15:03:40 zeddii: I found it, my defconfig file was not found because I had accidentally copy pasted a similar but not identical filename Sep 25 15:04:08 zeddii: it was obvious once I found the actuall directory that bitbake was looking in, but I still wish it had told me Sep 25 15:04:37 ad__: that's really what a sstate rsync would look like :) Sep 25 15:05:09 yann, ah ok, thanks. Will study on it. Sep 25 15:05:48 30min looks like a lot, but I've see some docker instances (gitlab.com's) being really slow when it comes to copying data, even inside a single docker container Sep 25 15:06:39 copying an unpacked yocto SDK (not even an extended one) does takes more than that time Sep 25 15:15:22 ecdhe: there is a patch for that in master now Sep 25 15:34:26 JPEW, fray: core-image-sato builds and passes testimage with path filtering :) Sep 25 15:34:37 RP: Cool Sep 25 15:35:07 * RP should probably do a binary image compare next, see if anything weird happened. Sep 25 15:35:28 the more I look at the pseudo logs in WORKDIR, the more I think there are things which are very very wrong today when reusing builddirs Sep 25 15:39:05 Hmm, I bet the values returned by the config in python-native probably don't apply to a mingw prebuilt binary windows python installation. Guess I should test it, though Sep 25 15:44:45 jonmason: do you have thoughts on the cortex-m0 patch in master-next? Sep 25 15:53:15 RP: there seems to be two paths, one for testimage class that uses OEQemuTarget and the other for HW , which seems to have a QemuTarget class also Sep 25 15:53:57 The HW/QemuTarget path had a cmd dumper, but the OEQemuTarget did not. Sep 25 15:55:21 sgw: and that works over serial? Sep 25 15:57:22 (the dump commands are run over the serial interface) Sep 25 15:58:17 RP: yes the code I provided runs over the qemu serial, since I still had your Took x seconds to login code, I saw that multiple times. Sep 25 15:58:57 sgw: ok, cool. I'm just surprised its this simple! Sep 25 15:59:32 as ever its all about knowing which small number of magic lines to put where :) Sep 25 15:59:34 Took me a while to figure out is was not going to be complex as I had to backtrace where things where coming from. Sep 25 16:00:00 sgw: we really need to simplify some of it Sep 25 16:00:04 And then figuring out I did not have 'd' available to get the args and ... Sep 25 16:00:48 sgw: indeed, the lack of d can be a shock :) Sep 25 16:01:04 Yesh I thought having two qemu targets was wrong, they seem to serve very different purposes. That's a longer term clean-up I think Sep 25 16:01:42 RP: rburton mentioned that to me. I was going to take a look after lunch. Sep 25 16:01:58 It initially confused me because the QemuTarget did have access to the Dumpers, but I figured out the OEQemuTarget is not of the same class, OEQemuTarget is of the OESSHTarget Class Sep 25 16:02:27 jonmason: ok, np. I just need a hint on whether it is right or not :) Sep 25 16:02:52 sgw: those names look quite poor :/ Sep 25 16:03:01 I will wait until next week and then and if anyone has comments then move it out of RFC, unless your comfortable with it now. Sep 25 16:03:18 sgw: I'll probably put it in testing for builds over the weekend Sep 25 16:03:37 Sounds fair, how often has the network failure been happening? Sep 25 16:03:37 sgw: it sounds good at least :) Sep 25 16:03:50 sgw: not very often, just often enough to be annoying Sep 25 16:03:58 once every week or two Sep 25 16:04:21 RP: the previous m0plus looked okay. So I'm going to assume without looking that it should be fine. But I'll have an answer in an hour or so Sep 25 16:04:42 jonmason: np, I just didn't know it ross had talked to you Sep 25 16:05:14 Similarly, I have cortex m series queued but was waiting on the cortex a stuff to get sorted first Sep 25 16:05:45 That's also why I was messing with zephyr, as it's a good test bed Sep 25 16:05:55 jonmason: I've left that series as I don't think its right unfortunately :( Sep 25 16:06:42 if we make changes like that this late in the release I need to be convinced they're right and I'm not (I think I said as much) :/ Sep 25 16:06:51 I'm doing v3 now, I just keep getting pulled into other stuff. It should be out today Sep 25 16:07:06 jonmason: ok Sep 25 16:08:00 And if this needs to wait until after gatesgarth is released, that seems reasonable Sep 25 16:09:03 jonmason: If its just adding files we have a bit more flexibility, will depend on the changes Sep 25 16:15:09 Then I'll take that into account and maybe do some of the organization in a follow-up series Sep 25 16:17:44 Hmm, I wonder if it'd be useful to create a recipetool or devtool command to take a specified .bb file and shove it into a temporary layer and build it there after adjusting priorities and verifying it'll be the selected provider, as an alternative to bitbake -b Sep 25 16:18:01 course that assumes it's not in one already, so perhaps not Sep 25 16:18:15 is yocto sumo so old that I should be using something else? Sep 25 16:18:18 just thinking about quick and dirty work on a new recipetool-create'd recipe Sep 25 16:18:36 https://wiki.yoctoproject.org/wiki/Releases shows it was released on april 2018 Sep 25 16:18:55 and has a current upstream support level of EOL Sep 25 16:19:00 A vendor shipped a bsp based on sumo in mid 2019 Sep 25 16:19:01 but it depends on your perspective, of course Sep 25 16:19:31 RP: I am looking at something else regarding package.manifest creation, ideally a way to create a custom manifest. I am thinking of 2 possibly ways, I am sure there are more. 1) bbclass that gets inherited or 2) extend the format_pkg_list() in lib/oe/utils.py to take a custom ret_format and handle in there with getVar("TBD_CUSTOM_MANIFEST"). Thoughts? Sep 25 16:19:47 How hard is it to upgrade? I've never done it, but it looks like I might need to based on the fact that it can no longer pull source for stuff like htop and sudo Sep 25 16:20:21 sgw: I remember nothing about package manifests :/ Sep 25 16:20:52 ecdhe: have a read of the migration guides in the manual for thud and later ? Sep 25 16:21:19 If you're just using standard yocto project its easier than a ton of BSPs and custom recipes Sep 25 16:23:25 RP, a board vendor shipped us a bsp based on sumo. But I've got another board for which the vendor shipped buildroot. Sep 25 16:23:43 RP: fair enough, I will dig around some and come up with a proposal, more about checking on creating interface that a bbclass could use vs have kind of script called from a variable to create the manifest Sep 25 16:24:00 ecdhe: we can't know how hard that BSP will be to work with something more recent Sep 25 16:25:22 I wanted the new board to use yocto instead. Since the new board's buildroot just downloads a U-Boot and kernel source tree and builds them, I cloned the first vendor's Sumo project and changed the git repos, defconfigs, device tree references etc, and have, for the most part, gotten yocto to execute the same basic build steps that buildroot was taking. Sep 25 16:25:42 RP: My build is failing at the moment because I can't acquire all the package source. Sep 25 16:26:10 ecdhe: I'd probably have done something similar to that FWIW Sep 25 16:26:25 But I'd really prefer to get my ported BSP fulling building on sumo before I migrate to a newer version Sep 25 16:26:47 ecdhe: that would seem sensible Sep 25 16:27:01 Since I have the original sumo build environment (all 60 gigs) I might be interested in transplanting cached copies of the no-longer attainable source Sep 25 16:27:31 Then once I get a build I can migrate to Dunfell or so Sep 25 16:28:15 or even the pending October release Sep 25 16:29:45 ecdhe: seems reasonable to me. dunfell is our LTS so I'd try and get to that at least Sep 25 16:33:15 RP, my sumo build is failing saying it can't download http://ftp.sudo.ws/sudo/dist/sudo-1.8.22.tar.gz Sep 25 16:34:55 If I find sudo-1.8.22.tar.gz in my old yocto workspace, do I just copy it to the same relative path in the new workspace? Sep 25 16:35:46 It's in ./build/downloads/sudo-1.8.22.tar.gz Sep 25 16:36:01 or is there some other step that I'll need to take Sep 25 16:37:00 ecdhe: that will work. I'm surprised it doesn't find it from the yocto project mirrors Sep 25 16:37:31 RP: don't underestimate the power of badly written BSP layers and wrappers :) Sep 25 16:37:53 RP: I wouldn't be surprised if they replaced the mirrors with theirs or entirely removed them :) Sep 25 16:38:18 qschulz: I'm doing my best not to think too hard about it Sep 25 16:38:41 RP: hehehe, if that makes you sleep better :D Sep 25 16:39:14 ecdhe: are you building on the same machine different Yocto projects? In that case, you can share the DL_DIR and SSTATE_DIR, it should be just fine (and if it isn't, scream at vendor) Sep 25 16:39:42 and not have to duplicate your download directory (and benefit from shared sstate cache dir :) ) Sep 25 16:40:06 rburton: good news is that buildhistory on an image without my pseudo changes built in one builddir and one with the changes don't show any permissions issues. They do show pam differences :/ Sep 25 16:50:12 JPEW: nice build corruption issue. PACKAGECONFIG_append_pn-shadow = " pam", bitbake shadow, remove that line, bitbake shadow, /etc/pam.d/* still present Sep 25 16:56:05 Hmm, because ${D} isn't cleared out? Sep 25 16:59:25 nice Sep 25 17:01:14 jonmason: Reviewed-off-by? nice typo :D Sep 25 17:01:31 crap Sep 25 17:01:33 JPEW: WORKDIR Sep 25 17:02:22 RP: it does globbing in WORKDIR? Sep 25 17:02:47 rburton: yes, its clear when you read the recipe why it breaks Sep 25 17:03:01 its the second time recently I've seen this :( Sep 25 17:03:35 that's a horrible thing to do Sep 25 17:03:55 I'm starting to think the fetcher shouldn't just dump files in WORKDIR Sep 25 17:04:56 RP: Oh, ya. That's a really gross way to do the recipe Sep 25 17:05:46 Using PACKAGECONFIG in SRC_URI seems like a bit of a red flag Sep 25 17:05:50 I noticed a problem with the ssh pregen keys recipe where if you remove a file from the ssh sources directory, the file still ends up in the image :( Sep 25 17:06:19 its because the fetcher evidently doesn't/can't clean up WORKDIR Sep 25 17:07:12 there SRC_URI = "file://sshdir/" so its a directory of files Sep 25 17:08:32 hm fun Sep 25 17:09:10 two issues are different but related :( Sep 25 17:11:03 i wonder how much will break if WORKDIR isn't actually the parent of S and friends Sep 25 17:11:34 ie WORKDIR was /work/ alongside /temp/ and so on Sep 25 17:11:50 then you could blast it away on re-fetch Sep 25 17:12:26 rburton: I think fetch just had to stop poking into WORKDIR. Maybe we mark recipes as "transitioned" with a flag that changes the fetcher behaviour? Sep 25 17:12:38 and then warn on the recipes which don't set it? Sep 25 17:12:47 where would the fetcher put files otherwise? Sep 25 17:13:10 rburton: named directory configured by a variable? or just a common name? Sep 25 17:13:31 call it WORKDIR and put it in /work/ ;) Sep 25 17:13:31 ${WORKDIR}/fetched-files/ ? Sep 25 17:14:15 for bug 13803, I noticed: Sep 25 17:14:18 ERROR: setUpModule (devtool) [ (subunit.RemotedTestCase) Sep 25 17:14:46 What does the RemotedTestCase mean? Sep 25 17:14:59 moto-timo: oe-selftest -j for parallelism Sep 25 17:15:27 moto-timo: means subunit and testtools are proxing the test results over a pipe from multiple processes Sep 25 17:15:28 RP: thanks. That might explain why I can't get the copytree (directory) target to work Sep 25 17:15:46 RP: ${WORKDIR}/oe-local-files ? that'd match what devtool is using :) Sep 25 17:16:16 qschulz: match an existing standard?! ;-) Sep 25 17:16:30 qschulz: seriously, thanks, I hadn't remembered it was doing that Sep 25 17:16:53 I instrumented the heck out of the code, but it's only been passing in files shutil.copy2 mode and so even when I force copytree it's invalid Sep 25 17:16:54 lol Sep 25 17:17:09 ok, I'll run with -j Sep 25 17:17:26 or just send the patch Sep 25 17:17:28 RP: been bitten by it :D though... I hate how. We use SRC_URI = "file://oe-local-files.patch;patchdir=.." for patching files coming from upstream recipes.... Sep 25 17:17:33 but I'd rather see it working Sep 25 17:18:08 obviously this does not work with devtool putting the local files in a different "directory layout" Sep 25 17:19:24 RP: but honestly, +1 for putting files in a different dir than WORKDIR, it's super confusing Sep 25 17:21:51 RP: the issue is with recipes only having "file://"s where you'd actually have S = ${WORKDIR} and that wouldn't work. But that's pretty easy to error on since that wouldn't make any sense anymore Sep 25 17:21:56 (thinking out loud :) ) Sep 25 18:05:11 at least the shutils.copytree is being called now, but it hasn't gotten to a test where __pycache__ would be copied yet. Getting close! Sep 25 18:34:13 if devtool reset times out while trying to clean the sysroot how can I manually reset it? I removed the workspace folder but devtool status still lists it as present Sep 25 18:48:31 RP: it built! It built! Sep 25 18:48:38 now, will it run? Sep 25 19:05:01 RP: eeeetts aliiiieeeeeve!!!! Sep 25 19:30:00 RP: just sent v2 out. Hopefully it is more towards your liking :) Sep 25 19:30:43 ecdhe: great! Sep 25 19:30:51 jonmason: thanks, will take a look Sep 25 20:28:09 Within a recipe that builds some C code, you might have serveral tasks, like do_fetch, do_configure, do_build, do_install, do_deploy. Steps like fetch, build, install, deploy cause data to be copied/transformed from one domain/directory to another. Sep 25 20:29:50 in this situation, do_build ${S} => ${B}, do_install ${B} => ${D}, do_deploy ${B} => ${DEPLOY_DIR} Sep 25 20:30:31 What about when you're not building C code? What if you're downloading a static file from a http server and putting it right into the deploy directory Sep 25 20:31:28 The do_fetch task would still bring the file from the remote webserver to ${S} Sep 25 20:31:54 But in what task would you perform the ${S} => ${DEPLOY_DIR} copy? Sep 25 20:32:25 Does it break convention too badly to have do_deploy() copy from ${S} instead of ${B}? Sep 25 20:43:50 ecdhe: nothing wrong with that, plenty of recipes do it. a lot of upstream project buildsystems don't support a source vs build/object separation. Sep 25 21:31:59 Thanks kergoth. I just want to avoid making unmaintainable messes, and I haven't worked with yocto long enough to have an intuitive sense of smell for worst practices Sep 25 21:59:18 The fun thing about testing do_unpack rm improvements is you break build directories quickly :( Sep 25 22:10:50 kergoth: think http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=7adaca6ff63e0d3495a1cd0cc5e5221b8c82a4a6 will work to fix do_unpack? Sep 25 22:45:49 dumb question : im trying to modify QB_OPT_APPEND in local.conf but can't see any changes taking effect when using "runqemu". Am I missing something obvious? Sep 25 22:46:50 bitbake -e can determine its an extra operation when added but no change in value so I guess someone overwrites the value after local.conf is parsed. If so is there a "right" way to do this? Sep 25 22:47:47 RP: seems reasonable. i'm assuming utils.remove() uses rm, not python? i'd have used pathlib personally, but that's personal preference, the concept is sound. the other thing i'd wonder is whether anyone is using prefuncs on do_unpack, as the order would change if that's the case. i doubt anyone is, though :) Sep 25 23:17:36 kergoth: I have found meson.bbclass is making some assumptions but that is easily fixed Sep 25 23:17:53 kergoth: utils.remove uses a mixture of things depending on what we've found is fastest Sep 25 23:29:00 kergoth: we have a different fun class of bug. Modify do_unpack and it will rerun but prepare_recipe_sysroot does not. It will however uninstall "bad" dependencies from the sysroot **** ENDING LOGGING AT Sat Sep 26 02:59:57 2020