**** BEGIN LOGGING AT Fri Oct 02 02:59:57 2020 Oct 02 07:22:13 Hi, I've created a patch with git for the linux kernel. The patch changes defconfig, bitbake succeeds but it looks like the kernel is not picking up the changes, how can I debug this? Oct 02 07:26:53 KBUILD_DEFCONFIG ?= "socfpga_defconfig"KCONFIG_MODE ?= "--alldefconfig" is used by the kernel recipe, maybe this overwrites the defconfig? Oct 02 07:56:17 Quick question, if I want to do post-processing commands on regular image I can simply use ${sysroot} Oct 02 07:56:52 What is the corresponding SDK host? SDK_HOST_ROOT? Oct 02 07:59:45 It can't be that difficult :P Oct 02 08:04:43 wertigon do not know by heart but have a look at poky/meta/classes/populate_sdk.bbclass and its siblings Oct 02 08:08:58 Hmm, STAGING_DIR_HOST? Gonna try it atleast Oct 02 08:53:54 Ok, so my problem; trying to make the SDK for dunfell branch Oct 02 08:54:13 SDK generates qemu for all architectures (except arm64, which is the one I use) Oct 02 08:54:47 This means I get around 1.5 GB of wasted qemu images Oct 02 08:55:19 Best solution would be to never generate these at all - only takes a lot of time to generate for all these architectures Oct 02 08:56:08 But what I'm implementing now is to simply remove these artifacts with a post-process command Oct 02 08:57:47 Probably a dumb question, but when are patches applied, after or before do_compile? Oct 02 08:58:10 aka; why the fuck am I generating qemu-system-ppc.exe and qemu-system-aarch.exe? Oct 02 08:58:34 guest021: Should be before Oct 02 08:59:44 wertigon: thank you Oct 02 09:01:03 guest021: right after unpack. You can check for yourself by using the -g argument to bitbake and then read the dot files, the task dependencies are explained, you'd have seen that indeed, do_compile depends on do_patch of the same recipe Oct 02 09:01:05 :) Oct 02 09:01:26 guest021: or bitbake my-recipe -g -u taskexp for a GUI Oct 02 09:13:28 wertigon: better solution would be to change nativesdk-qemu Oct 02 09:13:38 there's a variable for 'what machines to build' Oct 02 09:14:37 rburton: Sounds good, is that as a qemu recipe extension then? :) Oct 02 09:14:56 well you can do it in your layer.conf Oct 02 09:15:22 or propose a patch to make it just build the right target qemu Oct 02 09:16:07 Ah, just set QEMU_TARGETS Oct 02 09:16:21 And that I should be able to do from image file yes I see now Oct 02 09:16:23 Thanks Oct 02 09:17:05 no, not the image recipe Oct 02 09:17:36 i didn't mean layer.conf, but definitely not an image recipe Oct 02 09:17:54 ok... Oh, read local.conf Oct 02 09:17:55 mb Oct 02 09:20:12 Otherwise, yeah, a recipes-devtool/qemu/qemu_%.bbappend should take care of it Oct 02 09:57:26 That bbappend did the trick Oct 02 09:58:08 However... the SDK package is still 2GB large Oct 02 10:06:26 is it debug symbols? static builds? most likely worth comparing with a linux host SDK to see where the space goes Oct 02 11:47:32 rpm 4.16: Deprecate the Berkeley DB backend (it is still the default however) Oct 02 11:47:42 rburton: facepalm of the month? :) Oct 02 11:48:14 awww i saved the release notes to pocket to read Oct 02 11:48:26 the question is does f33 use bdb still, or sqlite Oct 02 11:49:00 they also switched to libgcrypt from openssl, i sent a few patches upstream to fix bugs in it before they switched Oct 02 11:53:14 rburton: https://src.fedoraproject.org/rpms/rpm/blob/master/f/rpm.spec Oct 02 11:53:27 still bdb,not ndb or sqlite Oct 02 11:53:38 I guess we have to follow Oct 02 11:53:41 maybe next release Oct 02 12:18:15 One long recompile later, the Linux SDK zipped size is also 2G Oct 02 12:18:27 That seems excessive to me Oct 02 12:24:19 Hmm, the thing that takes the most space is the board support Oct 02 12:25:58 So, debug symbols only take up 200M, not that bad Oct 02 12:33:22 What the... Oct 02 12:36:43 Grepping around in the board support SDK folder, it says the entire folder is 2.7G but using du -hs on every individual folder it adds up to 1G Oct 02 12:52:26 wertigon: huge number of tiny files? --apparently-size will tell you the real sizes, not disk usage. Oct 02 12:52:55 rburton: probably that yeah Oct 02 12:53:12 dare I do a tree | wc -l? Oct 02 12:54:06 Hey, I'm using an out-of-tree defconfig file (pulled directly from my linux) to set the options for the kernel in yocto, but the resulting kernel differs, why? Oct 02 12:54:46 guest021: Most probably you turned off a setting that automatically turned on/off other settings. Oct 02 12:54:58 I know that I'm doing something wrong, but I can't figure out what Oct 02 12:55:15 For instance, if you turn off ipv6 support you will also turn off a lot of options below that Oct 02 12:56:02 So, yocto might ignore parts of the defconfig file, if other settings are existent? Oct 02 12:56:04 guest021: check that it's using your defconfig first, is it in SRC_URI? is your defconfig taken and not another one (check in ${WORKDIR}/temp/log.do_fetch for the list of paths checked). Oct 02 12:56:30 It is in SCR_URI, if I remove the corresponding line I get an error message that no defconfig file could be found Oct 02 12:56:33 guest021: yocto not, the kernel yes. But if the defconfig are identical and the sources too, if you build manually or with yocto you shouldn't have much of a difference Oct 02 12:57:30 guest021: how did you create your defconfig? Oct 02 12:57:39 guest021: how is your defconfig named in Yocto? Oct 02 12:58:34 defconfig Oct 02 12:59:06 by hand Oct 02 12:59:08 guest021: For the record, you will have a really awful experience trying to manage the full kernel definition, better to just write fragment files. Oct 02 12:59:42 guest021: by hand => no. Use menuconfig, save with defconfig Oct 02 12:59:52 *save with savedefconfig Oct 02 13:00:52 I did that initially, but I had the same problem Oct 02 13:01:35 guest021: keep using menuconfig/savedefconfig :) Oct 02 13:01:46 guest021: at least we know it's not that ;) Oct 02 13:02:26 guest021: can you check with bitbake virtual/kernel -c menuconfig that your options are here? Oct 02 13:03:22 rburton: apparent size is even smaller in all aspects Oct 02 13:03:31 well, yes Oct 02 13:03:32 Than disk usage Oct 02 13:03:54 unless you have a file system that compresses, that's always going to be true, it just makes it easier to see where the space is actually going Oct 02 13:04:14 I get undefined references in In function `show_help': Oct 02 13:04:28 rburton: Yeah, true... Ooooooh Oct 02 13:04:29 there are tools to analyse where space goes. if you run gnome, baobob is one. Oct 02 13:04:40 I'm just looking at directories now Oct 02 13:06:14 ... Wait are you frickin kidding me Oct 02 13:06:30 i think you found it Oct 02 13:06:33 guest021: uh? Oct 02 13:06:36 Yeah Oct 02 13:06:45 a built kernel tree? Oct 02 13:06:47 a swapfile! Oct 02 13:07:11 $ du -hs .git Oct 02 13:07:11 1,7G .git Oct 02 13:07:21 why is there a git tree inside the sdk Oct 02 13:07:31 My question exactly! Oct 02 13:07:37 a kernel tree, i bet Oct 02 13:07:57 Yup Oct 02 13:10:27 Now I just gotta know Oct 02 13:10:41 Do I file a bug report with TI or openembedded :D Oct 02 13:11:04 well does it replicate with no TI layers and machine=qemuarm Oct 02 13:11:08 if so then OE Oct 02 13:11:11 otherwise, TI Oct 02 13:12:52 sudo apt-get install libncurses5-dev fixed the error in menuconfig Oct 02 13:13:13 So what's the next step qschulz Oct 02 13:16:37 guest021: well, you enabled/disabled some KConfig options in your defconfig right? check that it's actually there (won't be, but at least, we're sure and you've got a way faster way to check if your defconfig is used instead of booting the kernel :) Oct 02 13:18:00 I'm currently looking for CONFIG_JFFS2_FS, but I haven't found the fitting submenu yet Oct 02 13:18:45 guest021: use '/' to search :) Oct 02 13:18:59 it'll tell you if it's enabled or not Oct 02 13:19:04 thank you qschulz Oct 02 13:19:39 It's disabled but it should be enabled Oct 02 13:19:56 is it ok to modify a recipe's variable from the image recipe? if so, what will be the right way to do? Oct 02 13:23:04 Ok so the "board-support" is simply the linux kernel tree that was used to compile the SDK target. Oct 02 13:23:24 Is there any reason whatsoever to have this source code here? Oct 02 13:24:34 Since what menuconfig shows me is consistent with the behavior of the kernel image (missing options), I would conclude that the defconfig file is not properly used Oct 02 13:25:43 I will bring it up with TI when possible, but it seems to me the only things we need are kernel headers and the bzImage Oct 02 13:27:33 xtron: conf data is global. recipe data is local. Oct 02 13:27:43 wertigon: the whole source tree is a bit mad yeah Oct 02 13:28:14 xtron: you actually cnanot modify a variable from a recipe in another recipe Oct 02 13:28:41 xtron: https://twitter.com/TheYoctoJester/status/1217166071519744000?s=20 Oct 02 13:29:10 guest021: yep, then you need to dig a bit deeper into the classes/tasks used by your kernel recipe Oct 02 13:29:52 I would try to debug kernel_do_kernel_configme task, do_configure as well and try to see what's happening (have a look at WORKDIR/temp/log.do_ for that Oct 02 13:31:57 qschulz, yes, I did this in the recipe -> DEFAULT_ENFORCING_pn-image-recipe-name = "permissive" Oct 02 13:34:12 xtron: this does either nothing or it can just be DEFAULT_ENFORCING? Oct 02 13:34:47 qschulz, need to put it some *.conf, right? Oct 02 13:39:40 xtron: if it's for the image recipe and not package recipes, put it into the image recipe. If it's for package recipes and not image recipes, put it in each package recipe that needs it. If both image and package recipes, put in all of them. Oct 02 13:40:17 xtron: or/and use your machine conf files, or depending on what exactly you want to do, probably a new distro Oct 02 13:41:22 qschulz That was interesting: it uses olddefconfig as target, instead of defconfig which was the expected behavior from from my side Oct 02 13:42:59 guest021: olddefconfig is run after a defconfig is loaded IIRC, it's just to make sure there are defaults for newly itnroduced variables without asking for users' input Oct 02 13:43:08 guest021: so that's **probably** okay Oct 02 13:44:29 HOSTCC scripts/basic/fixdep GEN ./Makefile HOSTCC scripts/kconfig/conf.o HOSTCC scripts/kconfig/zconf.tab.o HOSTLD scripts/kconfig/confscripts/kconfig/conf --olddefconfig Kconfig.config:6168:warning: override: reassigning to symbol LOCALVERSION Oct 02 13:45:19 It looks to me a bit like Kconfig overwrites other settings, is this correct? Oct 02 13:47:26 other settings? KConfig is the only way to setup options in the kernel so I'm not sure to follow? Oct 02 13:50:09 Sorry, I meant to say that Kconfig fills .config with the default values independent of the defconfig file. Can the machine settings influence the .config? Oct 02 14:01:14 guest021: defconfig is basically a .config with all the "guess-able by Kconfig" options (dependency chains ("selects") and defaults) Oct 02 14:02:17 except when you use a defconfig created with a slightly different kernel (different version for example), in which case it's encouraged to use olddefconfig to try to find some defaults for those non guess-able Kconfig options Oct 02 14:02:25 (from recollection, I've rarely used it) Oct 02 14:02:59 guest021: machine settings could influence if your kernel recipe supports config fragments (.cfg or .scc IIRC) Oct 02 14:05:45 I've added KBUILD_DEFCONFIG ?= "defconfig" and removed the file://defconfig and the kernel remains unchanged Oct 02 14:06:23 This tells us that the out-of-tree defconfig file is ignored Oct 02 14:06:52 But why? Oct 02 14:15:11 rburton: Simply by removing the kernel tree I managed to go from 2.0G in SDK size to 324M Oct 02 14:15:18 That's 83.5% size reduction Oct 02 14:15:37 thats more like the size I expect a SDK to be Oct 02 14:15:43 Yeah, me too Oct 02 14:15:48 After compression ofc Oct 02 14:15:54 file a bug with TI :) Oct 02 14:16:30 Will do, I'll send it to the guy who has contact with TI and they need to solve it :) Oct 02 14:17:42 Could also be that we've done something wrong on our end Oct 02 14:17:53 Like removed a recipe that should be there Oct 02 14:41:03 I don't get this error message: A KBUILD_DEFCONFIG 'socfpga_defconfig' was specified, but not present in the source tree ; All I do is use the official meta-altera layer Oct 02 14:43:47 Is it possible that bitbake may end up in a corrupted state? Oct 02 14:45:15 guest021: you need socfpga_defconfig file in arch//boot/dts in your kernel, is that the case? Oct 02 14:45:37 yes, it's the official layer by a silicon vendor Oct 02 14:46:48 guest021: well, yocto disagrees with you :D Oct 02 14:47:03 Hi, is there a canonical way to add an html page to an image with the license information? Oct 02 14:51:42 Okay, the repo changed... It's always the dumbest thing Oct 02 14:53:08 guest021: :) Oct 02 14:53:26 guest021: still your defconfig should have been used Oct 02 14:54:01 Ok, time to go off the clock, have a nice weekend y'all! Oct 02 14:54:50 And thanks for the help :) Oct 02 14:54:56 Take care Oct 02 15:08:55 fray: am i correct in thinking archiver's do_ar_original needs to use process_submodules() to ensure we get those included? Oct 02 15:11:33 RP kergoth regarding the issues I chatted about the other day regarding _sysconfigdata Oct 02 15:12:09 I found that removing export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata" in python3native.bbclass seemd to resolve the issue Oct 02 15:12:22 I found that removing export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata" in python3native.bbclass seems to resolve the issue Oct 02 15:15:24 I can't remember if one of you all sent me this link: https://github.com/openembedded/openembedded-core/compare/master...kergoth:python-sysconfig Oct 02 15:15:29 dusry: I suspect that will break something else :/ Oct 02 15:15:58 It was enough for me to get devtool to put my recipe in workspace. I will try a clean build. Oct 02 15:18:36 heh Oct 02 15:18:37 Exception: ModuleNotFoundError: No module named '_sysconfigdata' Oct 02 15:18:49 dusry: well I'm glad i was reading irc today Oct 02 15:19:08 i hate that bit of python Oct 02 15:19:11 The elusive error strikes again. Oct 02 15:19:53 So just for my own edification, OE doesn't use the python included in the host system? Oct 02 15:20:29 I noticed it builds a lot of tools for the host that the host already has. Oct 02 15:21:47 well, both Oct 02 15:22:02 it uses the host python3 where it can, for example bitbake is python so obviously that runs in the host py3 Oct 02 15:22:18 but some recipes need to use modules that we need to build, so you need a python3-native Oct 02 15:22:35 this looks a bit like its using the host py3 but variables are set up to use the py3-native Oct 02 15:25:25 i wonder if kanavin_home can remember the quirks around this bit of python Oct 02 15:25:28 Thanks for the info. I'm really hoping to learn as much as I can about this so I can start contributing. Oct 02 15:26:58 so the trigger is the recipe inheriting py3native Oct 02 15:27:53 kergoth's patch doesn't look obviously wrong at least Oct 02 15:28:47 iirc it fixed the common case but there were particular recipes that broke as a result Oct 02 15:28:53 bah Oct 02 15:29:05 iirc they unset the variable, so the wrapper passing it becomes problematic, or so? Oct 02 15:29:11 I'm trying a build for my evm board right now. Oct 02 15:29:18 Curious to see what happens. Oct 02 15:29:23 kergoth: ah yes glib unsets it Oct 02 15:29:25 we'd have to use our own variable instead, most likely, so the wrapper can act conditionally? Oct 02 15:29:39 but why does glib unset it Oct 02 15:29:47 tahts a good question Oct 02 15:29:50 for the same reason maybe Oct 02 15:30:04 dusry: is there a bug for this? Oct 02 15:30:31 rburton: I did suggest adding a second name for this so we can control it properly Oct 02 15:30:47 I haven't checked. I assume so because there was another thread on the mailing list about it where RP and kergoth chimed in. Oct 02 15:31:02 i can't even remember what this variable does Oct 02 15:31:04 it's kind of annoying that we need python-native at all, is it only to allow access to the patched cross sysconfig/site that aligns with the target python? Oct 02 15:31:20 rburton: sstate reuse. avoids the sysconfig being bound tot he host system Oct 02 15:31:23 iirc anyway Oct 02 15:31:30 since its normally renamed by that Oct 02 15:31:33 _sysconifg_blahblah Oct 02 15:32:02 kergoth: we need native py as some deps we need are C python modules Oct 02 15:32:31 I did have some patches to let pure native modules be added to the search path for the host python but the problem of C py modules remains Oct 02 15:33:39 kergoth: we need config from a version that matches the target version exactly iirc Oct 02 16:13:39 No issues with build after removing that bit about _sysconfigdata. Guess I got lucky. :) Oct 02 16:23:31 * armpit that is weird, a recipe can cause gnome not to start with devshell Oct 02 16:42:11 If anyone wants nightmares, redir.c in bash, here_document_to_fd() is the thing tripping up the pseudo abort() Oct 02 17:38:18 * mcfrisk grabs a beer, second zeus to dunfell update just got integrated with minor regressions. cheers! Oct 02 18:42:17 woo: ERROR: An uncaught exception occurred in runqueue | ETA: 0:00:14 Oct 02 19:02:18 mcfrisk, congrats Oct 02 20:18:37 how can I get the qemu command line parameters that results in invoking runqemu? I would like to start qemu without runqemu script. Oct 02 20:34:05 mcfrisk: nice! :) **** ENDING LOGGING AT Sat Oct 03 02:59:57 2020