**** BEGIN LOGGING AT Mon Mar 07 02:59:58 2016 Mar 07 08:37:06 JaMa: gm. Finally no failures for meta-initramfs Mar 07 08:38:53 now, these failures in do_make_scripts for the modules... Mar 07 08:39:12 good morning Mar 07 08:48:03 hi mckoan Mar 07 09:00:52 ant_work: but all qemux86 external modules failes for whatever reason Mar 07 09:04:50 ant_work: ciao, how's it going ? Mar 07 10:16:49 ant_work, there is a problem with the source mirror I'm trying to track down. Mar 07 10:18:01 ok, thanks again Mar 07 13:51:01 Hello all. I'm looking for some explanation/help on how to employ volatile configuration files (like /etc/hosts) with a read-only filesystem. My attempts so far have failed (extending base-files, trying to deploy "broken" symlinks that point across to a read-write boot partition, etc.) Mar 07 13:51:22 Does anyone have a moment to help me understand a route forward? Mar 07 13:52:24 heh Mar 07 13:52:36 Maybe try emailing the list also Mar 07 13:52:44 hmm, which list :) Mar 07 13:52:57 hmm, good question for the architecture list Mar 07 13:53:29 http://lists.openembedded.org/mailman/listinfo/openembedded-architecture Mar 07 13:53:32 True :-) Mar 07 13:53:34 well it depends a lot on the use case, i'd say Mar 07 13:53:57 btw, do you have a read only rootfs going? Mar 07 13:54:49 I expect kergoth would have suggestions, but he is likely asleep Mar 07 13:54:50 Yes, I have basically everything working that I need (BSP seems to work with the FPGA, my other services were compiled and seem to run properly, etc.). The issue is that some of those services have configuration files that would be better left read-write. Mar 07 13:55:14 can you define where those magic files are? Mar 07 13:55:41 in that case, point them to a rw mounted FS at a known location. Mar 07 13:55:41 Yes, they're all in /etc at this point. /etc/hosts, /etc/network/interfaces and /etc/omniORB.cfg Mar 07 13:56:14 our companies way is, we have something like /etc/oursupercoolconfigstuff, which is a distinct fs that can be runtime remounted to RW Mar 07 13:56:35 Right, I tried that. What I was attempting to do is have my recipe for netutils, for example, install a soft link pointed at /media/FAT/etc/hosts for /etc/hosts. However no matter what I did with the packaging stages, that "broken" link would never be collected or installed. Mar 07 13:56:44 what about unionfs: http://unionfs.filesystems.org/ Mar 07 13:57:25 eengie: well so is your question rather "how do i install a symlink that appears to be broken at build time" or "where do i place my FS anyways" Mar 07 13:57:26 That's a new one for me...I'll look into it. Mar 07 13:57:32 not sure i'd call /etc/hosts or /etc/interfaces rw in a mainly-ro environment Mar 07 13:57:46 thats why there exists nss modules to dynamically add local hostname to host lookup Mar 07 13:58:20 rburton: yeah, or other network managing infrastructure Mar 07 13:58:31 LetoThe2nd: I'm only detailing what I've attempted to this point. So one route I went was yes, trying to install broken symlinks that would be valid once the filesystem mounted on startup. Mar 07 13:59:49 the general problem sounds like something that people should have solved Mar 07 14:00:08 LetoThe2nd: I also tried using base-files "volatiles" variable to create the symlinks to the /var/volatile location on startup followed by an init script that would copy the files from the boot partition over to the linked files. That also didn't work (seemed to cause populate-volatiles.sh to spew errors, and the files in question were again never links...always replaced by actual files). Mar 07 14:00:38 CroftonIwork: right! That's exactly what's been going through my head...I can't be the first one to do this. Google-foo is letting me down though. Mar 07 14:04:25 (E-mailing the architecture list now by the way -- thanks for the suggestion.) Mar 07 14:05:08 I suspect this is a piece of the puzzle that people do not share as much as other pieces :) Mar 07 14:05:35 and sounds like an area we should look for best practices and document add to meta-data Mar 07 14:05:43 TO MAKE EMBEDDED GREAT AGAIN Mar 07 14:05:46 arrggh Mar 07 14:05:50 LOL Mar 07 14:05:54 * Crofton|work uses the eye bleach Mar 07 14:06:24 * Crofton|work trundles off to troll a twitter bot Mar 07 14:06:56 That's basically what I was hoping to find -- a best practices for this type of thing. The symlink idea felt like the most "clean" way to execute this behavior... And as I failed and tried new things, I'm increasingly going for totally batty solutions. Mar 07 14:07:45 eengie: a "neat" solution is to change any recipes that want mutable state to use a mutable directory, and keep /etc read only Mar 07 14:08:31 It gives us somethiing to talk about on the architecture list from the usual boring changes :) Mar 07 14:10:43 "stateless linux" as endorsed by those nice poeple at redhat is a good way forward for readonly rootfs, https://fedoraproject.org/wiki/StatelessLinux Mar 07 14:10:47 rburton: fair point. The omniORB requirement would be simple enough but it would be nice to keep some configuration files where users are accustomed to finding them (like hosts) Mar 07 14:11:06 eengie: sure, but why is hosts considered mutable? Mar 07 14:11:40 Just thinking of certain end-users I know of that might need to hard-code endpoint names that aren't discovered by other conventional means. Mar 07 14:13:15 people are weird Mar 07 14:13:50 Well, yes. Mar 07 14:14:07 Some like the "hard way" Mar 07 14:16:13 sounds like not having a read only rootfs would be the simple solution :) Mar 07 14:16:52 True, however I can see some of these guys wanting to do the tarballed root filesystem approach too. Just too many conflicting interests on this one. Mar 07 14:17:07 users eh Mar 07 14:51:48 As I continue to dig, maybe I'm going about this all wrong. Should I be defining a pkg_postinst_* function rather than trying to directly deploy the links within the package? Mar 07 14:52:22 For example, add one for the netutils that moves /etc/hosts over to /var/volatile/tmp/ and then creates the link at /etc/hosts pointing to that new file. Mar 07 14:52:58 What I've been doing thus far is jacking around with do_install and do_deploy tasks. Mar 07 14:54:21 the rule for most packaging engines is: Mar 07 14:54:32 If the link is static and will never change, do_install.. Mar 07 14:54:55 if the link is dynamic, something could change it (-including-) putting stuff into a volatile location, it should be done in a postinst function Mar 07 14:55:24 Package managers like RPM, can be used to validate the integrity of the system. If the link changes, then it can't be in the package -- or the validation can and will fail Mar 07 14:55:46 that's why you'd use a postinst script... also for things like alternatives or files that can only be determined at runtime Mar 07 14:57:30 fray: thanks for clarifying. So then I could for example check in that method if someone has set a variable (of mine) using @bb.utils and then from there make the decision for each affected recipe to swap the file with a link? Mar 07 14:57:50 That sounds a lot more clever. Mar 07 14:58:03 probably not.. Mar 07 14:58:26 the postinst scripts MUST be shell.. and they will be fully expanded before they are stored into the package.. Mar 07 14:58:33 you can do things like: Mar 07 14:58:54 if [ -e /var/volatile/.stamp ]; then ln -s /bin/foo /var/volatile/bar ; fi Mar 07 14:59:03 (actuallY did that wrong.. it'd be: Mar 07 14:59:17 if [ -e $D/var/volatile/.stamp ]; then ln -s /bin/foo $D/var/volatile/bar ; fi Mar 07 14:59:25 Hmm... I'm looking at the oe-core/meta/recipes-core/initscripts recipe which seems to be using @bb.utils.contains in an if-statement followed by what do appear to be bash commands. Mar 07 14:59:48 any actions you take need to be prefixed with a $D --or-- you need to detect $D is set and exit 1.. (this is the trigger of the location during a cross install) Mar 07 15:00:02 yes, that bash command is expanded prior to package generation time.. Mar 07 15:00:13 so what they're saying is "run this command if this condition during the build process is set" Mar 07 15:00:22 that is allowed, but after the package is generated, the condition is locked down Mar 07 15:01:34 hmm... Mar 07 15:04:48 you have to remember whatever you put in pre or post install/remove are evaluated (pythonwise) at build time.. the output must be shell code.. Mar 07 15:05:01 if you are producing RPM packages you can query the result using rpm -qp --scripts Mar 07 15:05:26 if you are doing opkg or deb, use ar to extract the archive -- then tar to pull apart the 'control' and view the scripts Mar 07 15:08:19 Right, thanks. I'm trying this out now a few different ways (one for each recipe) to see how far along I get. Mar 07 15:09:08 Is the /var/volatile/.stamp expected to exist though based on some other recipe? I'm looking at my deployed rootfs and it doesn't exist. Mar 07 15:09:32 that was a made up example Mar 07 15:09:47 nothing I said should be considered real, other then as an example of what could be done Mar 07 15:11:11 Fair enough :-) just checking Mar 07 15:22:36 fray: is there a means to export a template of the postinst function so that I can parameterize it similar to how one can "export" other functions between recipes? Mar 07 15:23:24 the useradd and alternatives functionality are provided in bbclasses. In these bbclasses we more or less create templatized scripts. Then the user can set a simple variable in the recipe to control the eventual actions Mar 07 15:23:40 I would suggest those two as examples.. note the useradd one is very complicated.. the alternatives one less so Mar 07 15:23:49 Alright, thanks Mar 07 15:24:16 forgot there is a third 'update-rc.d' as well.. Mar 07 15:24:43 update-alternatives is fairly simple.. (still has a lot of complex actions.. but it's easier to read then the useradd) Mar 07 15:25:11 looks like the update-rc.d is the most simple one. has the preinst script, plus the anon python to enable it Mar 07 16:21:38 fray: something unrelated that has been bugging me is that I have two init.d scripts that are starting out of order. The one that should start after the other is specified with a Required-Start with the provides name of the other script. So for example, it depends on foo, so Required-Start: $foo. Mar 07 16:22:34 Other than ensuring the files are installed into init.d is there some way to convince update-rc.d to retain that dependency order? It seems to be creating the rcS* scripts however it wishes. Mar 07 16:23:23 using sysvinit I assume? (not systemd)? If so, you need to pass the rightt parametes to the init setup.. basically you define the numbers to use.. the requires-start/stop stuff isn't used by the system.. Mar 07 16:23:58 (having to parse that stuff is a lot slower then using static numbers.... and boot speed is an issue on these systems) Mar 07 16:24:03 Ah... Mar 07 16:24:11 got it. Mar 07 16:24:20 systemd does have deps, but you pay for it in complexity Mar 07 16:27:28 Alright that ...helps a bit. But the bummer is it looks like mountall gets started after networking, so trying to ensure the configuration is based off the user's changes (in a mounted filesystem) won't work. Mar 07 16:28:10 Hmmm.. Mar 07 16:30:09 thats why your configuration stuff usually needs to be in /etc since it's part of root.. Mar 07 16:30:39 if you are mounting an external resource for that, you'll need to introduce your own initscript that verifies that drivers are loaded and then mount the point properly Mar 07 16:30:52 not an unusual addition by any means.. Mar 07 16:31:07 (networking is started before mountall to allow for NFS..) Mar 07 16:31:20 what you want is a 'mount-some' for local or config... Mar 07 16:34:17 Right. I specify my own fstab file as it is so I'm basically ensuring I'm not trying to hook up any network storage at the time that the table is used. Mar 07 18:31:18 fray: any chance you're still around? I'm still fighting this thing. I tried this: pkg_postinst_${PN} () { #!/bin/sh ORIG="/etc/network/interfaces" DEST="${SD_CARD_LOCATION}${ORIG}" ln -sf ${ORIG} ${DEST} } -- Where SD_CARD_LOCATION is a variable in my recipe. Mar 07 18:32:14 so SD_CARD_LOCATION should have been expanded in the postinst Mar 07 18:32:16 The output file system has an actual file at /etc/network/interfaces rather than my link. Looking up in my build temp directory, I see a broken link, interfaces, pointed off to somewhere in my overall build structure rather than ${DEST} Mar 07 18:32:41 the problem though is that code would have run during image generation, and the wrong dest was used.. Mar 07 18:33:17 you need to have '$D' in there.. to specify the right "root location".. during a cross install that will be the path to your filesystem Mar 07 18:33:22 in an on-target install it's blank Mar 07 18:34:18 ...and do you have your ln 'right'.. don't you want to link /etc/network/interfaces to /sdcard/etc/network/interfaces? Mar 07 18:34:28 if so your args are backwards.. Mar 07 18:34:40 ln -s Mar 07 18:37:53 been staring at this too long I think. Alright, I was under the wrong impression based on your example. I know you said not to treat it as actual, but based on it I got the impression that when postinst was running, it would be in a fakeroot-like filesystem (so /var would actually be the final embedded system's /var) Mar 07 18:38:21 So really this should flip around to be ln -sf ${DEST} ${D}${ORIG} since ${D} is going to be the actual target filesystem location Mar 07 18:39:39 the script is run in a 'fakeroot'.. but it's not in a 'chroot'. So you need the $D.. (use $D and not ${D}.. they have different meanings..) Mar 07 18:39:58 $D is shell expanded.. ${D} is expanded at bitbake variable eval time) Mar 07 18:41:00 okay, that's a little confusing since ${D} would be valid syntax in the shell. So am I able to use my other variables as well? Like the SD_CARD_LOCATION (but as $SD_CARD_LOCATION rather than with braces)? Mar 07 22:28:36 ohh god.. it's that fake Intel news channel videos.. Mar 07 22:29:30 I swear these are the same actors as that corruption video from a couple yearsa go Mar 07 22:29:32 ugh Mar 07 22:29:40 this might make me stabby Mar 07 22:30:37 at least I can mute.. Mar 07 22:46:40 fray: ? Mar 07 23:23:02 bluelightning, gm Mar 07 23:27:39 hi Crofton|work Mar 07 23:32:29 so you have your own place now? Mar 07 23:37:21 Crofton|work: we do yes Mar 07 23:37:30 even has a garage for all my junk :) Mar 07 23:37:47 you carried junk around the world? Mar 07 23:38:07 yeah, well, you get a container, you might as well fill it :) Mar 07 23:38:20 besides, some of it never left NZ Mar 07 23:38:29 heh **** ENDING LOGGING AT Tue Mar 08 02:59:59 2016