**** BEGIN LOGGING AT Wed Mar 06 02:59:57 2019 Mar 06 07:13:16 Tofe: Build Rocko for Hammerhead overnight, seems to have the same issue as Sumo. I'll try to pull logs in a bit to see if it's the same issue Mar 06 07:13:32 Then will try to downgrade systemd a bit to see if that solves it Mar 06 07:13:41 Just to try to get to the cause of it Mar 06 07:16:43 Herrie|Laptop: I just don't see how systemd is involved here (apart if udev is involved) Mar 06 07:16:53 (Morning!) Mar 06 07:17:04 Tofe: I'm also not sure Mar 06 07:17:18 Just trying to pin down where the issue might come from Mar 06 07:19:07 And to exclude some other issues Mar 06 07:19:24 I could also try to just put only new systemd on pyro and see what that does Mar 06 07:20:11 Right now I only see udev and gcc as possible causes Mar 06 07:20:48 But udev is part of systemd no? Mar 06 07:20:58 yes Mar 06 07:21:16 quite independant still, but yes Mar 06 07:21:32 I'll just try some combinations to see if I can pin it down somehow Mar 06 07:21:38 Then at least we know where to look Mar 06 07:22:37 maybe we could try sumo's kernel inside a pyro initrd Mar 06 07:23:25 that way we could rule out gcc for the kernel, and it would only be udev left Mar 06 07:26:10 Tofe: Yeah also option Mar 06 07:26:18 I can try that later today as well Mar 06 07:38:44 Herrie|Laptop: the dyndns url for exiv2 is official ? Mar 06 07:39:22 https://github.com/Exiv2/exiv2/releases <-- yep Mar 06 07:39:46 merged Mar 06 07:43:00 Tofe: Yup. JaMa should cherry pick for other branches Mar 06 08:10:41 Tofe: Rocko seems the same as Sumo on Hammerhead: https://bpaste.net/show/59360ec1f6dd Mar 06 08:10:47 [ 2.565420] initrd: PANIC for reason: Couldn't find data partition. Mar 06 08:11:31 ok; I'm not very surprised Mar 06 08:12:11 Tofe: Me neither Mar 06 08:12:20 Let me flash the stable release and toy with boot.img from Sumo Mar 06 08:12:24 hm Mar 06 08:12:31 newer systemd? Mar 06 08:12:37 which version? Mar 06 08:12:39 bshah: We were on 232 with pyro Mar 06 08:12:43 Which worked OK Mar 06 08:12:50 new version is 237+? Mar 06 08:13:08 Rocko is on 234 Mar 06 08:13:25 Sumo on 237 Mar 06 08:13:42 Thud is 239 Mar 06 08:13:59 got last_kmsg? Mar 06 08:14:27 bshah: See above bpaste Mar 06 08:14:29 bshah: also important: it switches from gcc 6 to gcc 7.3 Mar 06 08:14:33 https://bpaste.net/show/59360ec1f6dd Mar 06 08:15:33 bshah: The 3.18 kernel devices seem to work. It's the 3.4 ones that are problematic Mar 06 08:15:42 Or at least Hammerhead Mar 06 08:15:51 I haven't tested the other 3.4 ones yet Mar 06 08:15:54 3.18 on aarch64, versus 3.4 on arm32 Mar 06 08:16:00 I think it is same issue which made Plasma Mobile drop 3.4 devices with switch to bionic Mar 06 08:16:28 bshah: oh I wasn't aware of that, do you have details? Mar 06 08:17:01 search halium group :P Mar 06 08:17:04 Give me moment Mar 06 08:17:23 ah ok I thought there was a blog post or something :) Mar 06 08:19:28 if you look at logs Nov 11-13 is to look for Mar 06 08:19:42 ok thanks Mar 06 08:19:57 Tofe: Seems that for Hammerhead we might be able to do mainline kernel: https://www.plasma-mobile.org/2018/03/26/plasma_mobile_on_open_devices.html Mar 06 08:20:55 Herrie|Laptop: well, we could give it a try, but look also at yesterday's discussion between me and saidinesh5 Mar 06 08:21:31 Tofe: Yeah seems we need to wait for 5.1 Mar 06 08:22:59 bshah: you sure about the date ? http://logs.nslu2-linux.org/livelogs/halium/halium.20181112.txt Mar 06 08:23:21 boo Mar 06 08:23:25 check 11? Mar 06 08:23:30 almost empty Mar 06 08:23:39 rip log broke Mar 06 08:23:46 ah, oops Mar 06 08:23:52 https://github.com/Halium/projectmanagement/issues/96 some discussion here Mar 06 08:23:57 ok thanks Mar 06 08:24:15 you can ping minlexx / alexy.min in the channel Mar 06 08:24:36 because for us it would mean quite a big change: no touchpad, no hammerhead, no mako... it would be sad Mar 06 08:25:50 Tofe: for me it is Nov 13 in telegram logs Mar 06 08:26:00 ah, "Problematic code (chase_symlinks() in src/basic/fs-util.c, src/basic/mount-util.c), that prevents systemd from booting on our 3.4 kernel was added in systmd-233" Mar 06 08:26:04 so maybe it is offset by +530 hur Mar 06 08:27:47 ah yes, here are some bits: http://logs.nslu2-linux.org/livelogs/halium/halium.20181114.txt -- and I was even involved a bit :D Mar 06 08:29:12 Tofe: When I flash boot.img from sumo on working pyro image I get the same issue Mar 06 08:30:07 Herrie|Laptop: yup, because userland isn't involved here, it's just kernel+initrd Mar 06 08:31:02 um no, kernel + systemd Mar 06 08:31:07 initrd is not involved :P Mar 06 08:32:25 Log just for reference: https://bpaste.net/show/f61d77686d5e Mar 06 08:33:25 bshah: initrd is involved, as it's where it stops :p Mar 06 08:33:56 and it contains udev, which might have the same issue as systemd core Mar 06 08:34:40 so, well, we're both correct, in a way Mar 06 08:36:35 Looking at the offending commits, I don't really see why this should start to give the issues Mar 06 08:40:15 Herrie|Laptop: there could be some syscall involved, eventually bugged on 3.4 Mar 06 08:41:07 Tofe: I also noticed that 237 in Sumo is using https://github.com/systemd/systemd-stable/tree/46659f7deb962f55c728e70597e37c2a3ab6326d Mar 06 08:41:16 While there are quite some commits afterwards Mar 06 08:41:28 I'll try latest commit to see if I can get that to build, maybe it'll solve stuff Mar 06 09:04:04 The status for mainline on N5 is better than what I first thought (there's some modem support for instance); so at some point it might be worth a try. Still, if we could find a solution that also works for touchpad and mako.... Mar 06 09:06:33 huh ? https://github.com/flto/linux/commit/e820bd2dc91097c399cf019cb09dfdfbf4afc22c Mar 06 09:10:35 Tofe: :) Mar 06 09:10:47 I knew pmOS guys were doing something with it Mar 06 09:12:30 bshah: how "unstable" was the lastest mainline hammerhead you had, for plasma mobile ? Mar 06 09:12:39 super Mar 06 09:12:45 ah, that much Mar 06 09:14:43 Well that was also 1 year ago;) Mar 06 09:32:46 Mako mainline doesn't seem to have much work done, however it seems to be not too far off with the N7 mainline as per: https://wiki.postmarketos.org/wiki/Google_Nexus_4_(lg-mako)#Mainline Mar 06 09:34:21 Herrie|Laptop: I would be too sorry to lose mako. But touchpad and hammerhead, I'd like to keep them a little bit longer Mar 06 09:34:26 wouldn't* Mar 06 09:34:34 Tofe: Same here Mar 06 09:34:38 My N4 is dead ;) Mar 06 09:35:11 mine isn't, but I bet there aren't that many people with a spare working N4, wanting to put LuneOS on it today Mar 06 09:36:46 * JaMa bought new battery to hammerhead yesterday Mar 06 09:42:32 Having a mainline kernel on a device would be a nice exercice, plus a nice-to-have Mar 06 09:43:23 It would let us see if we did any architecture mistake regarding hardware abstraction Mar 06 09:43:58 plus, it's the same toolchain all the way Mar 06 09:44:09 ok, now I'm tempted Mar 06 09:47:25 It would solve a lot of kernel issues with systemd for a very long time to come :P Mar 06 09:47:37 yup :D Mar 06 10:23:23 JaMa: For when you're back: seeing this warning @ Sumo, might be good to address sometime. https://bpaste.net/show/77efcae4fbd9 Mar 06 10:25:34 Tofe: OK got newer systemd 237 from systemd-stable to behave Mar 06 10:25:50 Needed to adjust 1 patch and drop a bunch of others because they were merged upstream Mar 06 10:26:05 Now building image Mar 06 11:37:58 Tofe: Same issue, but was expected really Mar 06 11:46:28 ok Mar 06 11:50:01 I'll try Sumo with systemd 232 from Pyro Mar 06 11:50:04 See what that does Mar 06 11:50:08 Build anyway go quickly Mar 06 11:50:42 you just need to build the bootimg Mar 06 11:51:27 How do I do that again? Mar 06 11:53:09 bb android-initramfs-bootimg or something like that, let me find it Mar 06 11:54:51 android-kernel-bootimg Mar 06 11:55:05 then you just boot it through fastboot Mar 06 11:56:17 Ah that saves the whole qtbase & webengine rebuild :D Mar 06 11:59:12 yup; that's why I only built that for hammerhead yesterday, it was just a matter of minutes, from scratch Mar 06 12:00:06 of course the userland won't be proper, but if we reach userland then we've already succeeded :) Mar 06 12:19:37 updated sumo builds finished on jenkins, now rebuilding rocko with latest fixes (and hammerhead build added for both) Mar 06 15:12:09 Tofe: https://bpaste.net/show/7131d352e76f Mar 06 15:12:20 Thats with systemd232 on sumo Mar 06 15:12:30 Looks slightly better I'd say Mar 06 15:15:49 I'll do a whole image now to see what that does Mar 06 15:18:41 Yes, it's what it's supposed to do... So to summarize, it's systemd-232 with gcc-7.3, right? Mar 06 15:20:49 Tofe: Yeah and a few patches to get systemd232 to behave with glibc 2.27 Mar 06 15:21:09 ok Mar 06 15:21:18 Most of it which is also in here: https://www.mail-archive.com/pld-cvs-commit@lists.pld-linux.org/msg417435.html Mar 06 15:21:43 I'll push my branch shortly Mar 06 15:21:50 That gives us a better understanding of the issue then: it's within systemd-udev, and not related to gcc 7.3 Mar 06 15:22:23 and this issue is only on hammerhead, right? Mar 06 15:23:00 JaMa: I haven't tested Tenderloin & Mako Mar 06 15:23:05 I suspect all 3.4 kernel devices tbh Mar 06 15:23:23 elvispre: Onyx uses 3.4 too right? Mar 06 15:24:14 Seems Onyx uses 3.4 as per https://github.com/corpuscle/android_kernel_oneplus_onyx/blob/luneos/cm-14.1-wip/Makefile#L1 Mar 06 15:30:49 JaMa: What I did: Added this patch: https://bpaste.net/show/fe4e14a892af to fix the glibc 2.27 and GCC7 issues Mar 06 15:32:04 Then simply copied the whole systemd (232) folders from pyro to sumo (oe-core and meta-webos-ports) and added https://github.com/openembedded/openembedded-core/blob/sumo/meta/recipes-core/systemd/systemd-machine-units_1.0.bb as well Mar 06 15:32:08 Then it seems to build Mar 06 15:32:23 yes, sorry, I meant that it wasn't shown on tissot nor qemux86 Mar 06 15:32:57 JaMa: Likely Mar 06 15:33:08 Seems it's something in the fs bits of 3.4 kernel that's the issue Mar 06 15:33:31 systemd guys long time ago already stated that "they don't care about old kernels, just update kernel" Mar 06 15:37:14 yeah, the docs say that 3.13 is needed since 2 years ago https://github.com/systemd/systemd/pull/5519/commits/32d9c84250f86bd8c17f5ab4e9485820861f01f7 Mar 06 15:38:00 3.11 since 3,5 years ago https://github.com/systemd/systemd/commit/a0c3e16b7bce78c7ce42e18050106f61cfe7a94a Mar 06 15:39:29 Yes Mar 06 15:39:40 We managed to keep it working with some patches to kernel & systemd so far Mar 06 15:39:41 since 226, but unfortunately the commit doesn't say why it was needed, only that it's 2y old kernel version :/ Mar 06 15:40:00 However in 233 they broke it somehow Mar 06 15:45:18 And we didn't really get to the source of it yet. Discussion in https://github.com/Halium/projectmanagement/issues/96 seems to suggest that https://github.com/systemd/systemd-stable/commit/c4f4fce79e157800212f3cba1b21870097030e81 might be the culprit, but I don't see much wrong with that code. Mar 06 15:49:19 I don't see why either, or how he came to this conclusion Mar 06 15:49:43 it might be the place where the behavior is faulty, but not the right commit Mar 06 15:50:21 I wouldn't even know how to debug this stuff :/ Mar 06 15:55:45 Tofe: Couldn't you gdb it and put some breakpoints at suspected calls? Mar 06 16:06:58 Came across this one, thought it would be good to have as well: https://github.com/webOS-ports/meta-webos-ports/pull/316/commits/f18fdba95db22369e7a2ade0f9de6129aaf18c99 Mar 06 16:07:31 JaMa: I guess that should be cherry picked to rocko/sumo/thud and slightly updated as well Mar 06 16:19:37 ok, will do, IIRC I had to update this one when rebasing the branches, so it should trigger conflicts now as well Mar 06 16:20:16 Herrie|Laptop: this actually looks pretty good Mar 06 16:20:56 Tofe: What? Mar 06 16:21:10 rocko images almost done on jenking Mar 06 16:21:11 s Mar 06 16:21:12 ah ok it was already there Mar 06 16:21:42 Herrie|Laptop: the mmc partname patch Mar 06 16:21:54 Tofe: Yeah just updated with the > 9 partitions Mar 06 16:21:59 We might see that on some targets Mar 06 17:00:41 should I delete all unstable images except latest or is anyone interested in some other ones (like different MACHINEs than hammerhead,tissot,qemux86 built before)? Mar 06 17:01:01 we're almost out of space again /dev/xvdc1 985G 928G 57G 95% /home/jenkins Mar 06 17:13:17 JaMa: for unstable, you can proceed Mar 06 17:19:14 Tofe: Log from Hammerhead Sumo with systemd 232: https://bpaste.net/show/67cc6f86531d Mar 06 17:22:56 too bad you don't have the beginning, as it seems the halium image isn't mounted correctly Mar 06 17:24:18 but if you boot a second time, there shouldn't be the cp -arv output Mar 06 17:24:44 (anyway we should just remove the verbose option, it's useless here Mar 06 17:24:46 ) Mar 06 17:27:26 Tofe: https://bpaste.net/show/ddc7e7096012 Mar 06 17:27:30 That's 2nd boot Mar 06 17:28:30 https://github.com/webOS-ports/meta-webos-ports/pull/317 remove useless verbosity Mar 06 17:28:42 (let's concentrate on sumo now) Mar 06 17:30:29 [ 2.886557] cpio: skipping unsafe symlink to '/sbin/healthd' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract Mar 06 17:30:45 my latest commit is missing, it seems Mar 06 17:31:45 https://github.com/Tofee/meta-webos-ports/commit/70b9092d579c9456f6769b33e31e3bea7ea8891a Mar 06 17:31:53 let me include that in my latest PR... Mar 06 17:32:09 should be fixed in busybox used in sumo now Mar 06 17:32:21 well maybe not, I've only added it to oe-core Mar 06 17:32:23 well... it seems not... Mar 06 17:32:26 not included in LuneOS Mar 06 17:32:28 ok Mar 06 17:32:56 Well I took: https://github.com/webOS-ports/webos-ports-setup/blob/sumo/conf/layers.txt Mar 06 17:33:02 Which seems it might not have your commit Mar 06 17:33:05 I manually added it now Mar 06 17:33:24 Actually it should have it :S Mar 06 17:33:29 Since it's pointing to head Mar 06 17:33:37 Herrie|Laptop: it was only in thud Mar 06 17:33:43 (I think ?) Mar 06 17:33:43 Ah Mar 06 17:33:52 Well I added it locally Mar 06 17:34:06 I'll PR it right away for sumo Mar 06 17:34:33 Just flashing the zImage-dtb-hammerhead.fastboot should work right? Mar 06 17:35:27 mmmh it *was* there... or am I allucinating :) Mar 06 17:35:27 I think it was only in thud, then I've asked if it was meant for sumo and moved it myself, will check from my end as well Mar 06 17:35:59 Yeah UI works now, rotation works Mar 06 17:36:34 Audio not, but could be I miss the PA bits Mar 06 17:36:53 https://github.com/Tofee/meta-webos-ports/blob/sumo/meta-luneos/recipes-core/initrdscripts/initramfs-boot-android/init.sh#L126 meh Mar 06 17:37:22 Herrie|Laptop: you most likely are missing the latest commits, somehow Mar 06 17:37:35 Tofe: Seems so Mar 06 17:37:55 but anyway, it's good news: we have a fallback plan Mar 06 17:38:47 Yeah Mar 06 17:40:10 I'm trying to devise a smart plan to debug a bit this udev issue Mar 06 17:40:12 Not sure how we want to go about this for now? Do we downgrade to systemd232 or we stay at newer and try to find solution? Mar 06 17:41:17 give me some days first so that I look into it; if I don't find a fix within a week, we'll try to go with systemd-232 Mar 06 17:42:55 Tofe: this one should go to pyro as well, right? initramfs-boot-android: init.sh: be less verbose for initial user data copy Mar 06 17:43:11 55G releases Mar 06 17:43:11 73G luneos-unstable-rocko Mar 06 17:43:11 73G luneos-unstable-sumo Mar 06 17:43:11 179G luneos-unstable-thud Mar 06 17:43:11 292G luneos-testing Mar 06 17:43:21 we might need to prune testing builds as well Mar 06 17:43:34 JaMa: yes please, pyro, and thud as well Mar 06 17:47:10 JaMa: I'd say go ahead and we'll kick off new ones? Mar 06 17:47:20 Should we switch testing to Sumo? Mar 06 17:47:53 * JaMa accidentally dropped tissot unstable builds from fileserver :/ Mar 06 17:48:18 I'm fine with that, do we need another release or stable build from testing before we do that? Mar 06 17:48:42 switching unstable between rocko, sumo, thud is quite annoying and time consuming on slow builder Mar 06 17:48:43 JaMa: I think we should do Sumo and then prepare for a release? Mar 06 17:48:47 Tofe: ^ ? Mar 06 17:49:33 Seems we're in similar state for most targets on Sumo v.s. Pyro (except for the systemd issue), so I see no reasons to stick to Pyro for now Mar 06 17:51:39 Would also save you needing to maintain 4 layers ;) Mar 06 17:51:52 Would bring it back to Sumo & Thud... Any whatever comes after Thud Mar 06 17:55:54 udev doesn't have a ground-breaking output... Mar 06 17:55:57 https://bpaste.net/show/152bbb30c06d Mar 06 17:56:23 Herrie|Laptop: yes, having a release based on sumo looks like a good step Mar 06 17:57:38 Wow! tl;dr You guys are definitely all back in action :) Mar 06 18:19:23 Here's the pyro udevd output... notice the subtle change ? :) https://bpaste.net/show/c92ca92ad49c Mar 06 18:20:01 this reminds me that I saw some commit in systemd about changing the place where hwdb rules are stored, or something like that Mar 06 18:57:07 before switching testing to sumo, I should cleanup the images and sstate on fileserver, ready to drop everything there is and rebuild from sumo? Mar 06 18:57:31 so last pyro build will be the last stable release without any sstate? Mar 06 18:58:43 or I can remove the images/ipk and do one more pyro build from sstate to have lastest pyro branch before switching to sumo Mar 06 19:02:44 there are ~100 commits in meta-webos-ports/pyro mostly OSE migration since last release it seems Mar 06 19:04:26 JaMa: they're all also in sumo, right? Mar 06 19:07:34 anyway, image on sumo works, so that's a go for me Mar 06 19:08:54 yes they are Mar 06 19:09:16 just that if we find something wrong with sumo later, we won't have these in separate pyro build to compare Mar 06 19:10:09 I can create luneos-testing-pyro with the current pyro build and leave it on fileserver for a while (or we can compare against local build from latest pyro) Mar 06 19:10:49 I think we'll just need the latest testing pyro build, the older ones we won't have any use Mar 06 19:11:11 these are new since doppio release: https://github.com/webOS-ports/webos-ports-setup/commit/74be875eaf37fa68494d8fcc5a2b63a97d55509c Mar 06 19:11:33 ok, so I'll create luneos-testing-pyro with latest Mar 06 19:11:53 thanks Mar 06 19:12:12 oh, shit the systemd .patch change causes qtwebengine and a lot of stuff to rebuild.. it will take a while again Mar 06 19:12:54 I'm not in a hurry :) I'm trying to get udevd to output some verbose debug... Mar 06 19:19:15 always when I'm finally close to catching-up on the builder some small change invalidates most of the sstate :) Mar 06 19:19:33 but with testing switching to sumo it will be just 2 builds instead of 4 Mar 06 19:20:33 22G luneos-stable Mar 06 19:20:33 20G luneos-stable-staging Mar 06 19:20:33 56G luneos-testing Mar 06 19:20:33 0 luneos-unstable Mar 06 19:20:33 25G luneos-unstable-rocko Mar 06 19:20:35 21G luneos-unstable-sumo Mar 06 19:20:38 84G luneos-unstable-thud Mar 06 19:20:55 looks better on the fileserver :) Mar 06 19:20:58 /dev/xvdc1 985G 502G 483G 51% /home/jenkins Mar 06 20:01:34 ok I've reached the same point as the other fellow, i.e. chase_symlinks Mar 06 20:03:03 to summarize, we give a list of directories to parse (for the rules), it tries to resolve the canonical absolute paths, and it aborts, leaving the list of directories to search empty. Mar 06 20:03:46 I'm digging a little bit further; this kind of systemd issue rings a bell Mar 06 20:25:39 failed fstat(/etc): Bad file descriptor <--- this means that fstat on a file in a tmpfs mounted directory is not working Mar 06 20:37:48 but that's strange, because this part was already there in 232... Mar 06 20:44:47 ok, https://github.com/systemd/systemd-stable/commit/e187369587b1c6a5f65a12e7ec0bf7844905d014#diff-bcad68c477b6651521e880c40b7a9b40 is the real culprit. Mar 06 20:46:08 When going into chase_symlinks, we do some "fstat" that are really not well supported in the older kernels Mar 06 20:49:12 If we don't use nspawn, I could write a brutal patch that redirects chase_symlinks to canonicalize_file_name Mar 06 21:14:06 Herrie: care to test https://github.com/webOS-ports/meta-webos-ports/compare/sumo...Tofee:sumo?expand=1 when you've got some time ?... Mar 06 21:14:19 JaMa: ^ looks ok to PR, if it works well ? Mar 06 21:14:27 (from the recipe point of view) Mar 06 21:16:35 Tofe: yes, it's fine, small nitpick that the closing quite shouldn't be indented :) Mar 06 21:17:36 is it needed only for sumo or rocko as well? Mar 06 21:20:13 JaMa: only for sumo, as we won't use rocko Mar 06 21:20:21 ... and thud, for that matter Mar 06 21:20:50 it's really a kernel on life-support... Mar 06 21:22:25 JaMa: fixed the closing quote ! :p Mar 06 21:48:45 perfect :) Mar 06 21:56:40 I noticed that the porting guide (https://www.webos-ports.org/wiki/Porting_Guide) hasn't been updated in quite a while. Are the instructions there still good or have recommendations changed? In particular, I was wondering if I should be following the Halium porting guide (http://docs.halium.org/en/latest/porting/first-steps.html) in place of the "Building the Android pieces" section. Mar 06 21:58:05 novaldex|away: any idea what could be causing these? I haven't seen them before builder upgrade: Mar 06 21:58:08 jenkins@bonaire:~$ dmesg | grep INFO Mar 06 21:58:10 [1093513.028033] INFO: task git:18417 blocked for more than 120 seconds. Mar 06 21:58:13 [1093513.028234] INFO: task git:18418 blocked for more than 120 seconds. Mar 06 21:58:15 [1093633.867935] INFO: task git:18417 blocked for more than 120 seconds. Mar 06 21:58:18 [1093633.868135] INFO: task git:18418 blocked for more than 120 seconds. Mar 06 21:58:20 [1102454.454029] INFO: task kworker/u16:1:21288 blocked for more than 120 seconds. Mar 06 21:58:23 [1102454.454358] INFO: task ninja:26662 blocked for more than 120 seconds. Mar 06 21:58:26 [1102454.454601] INFO: task clang:14560 blocked for more than 120 seconds. Mar 06 21:58:28 [1102454.454842] INFO: task aarch64-linux-a:15029 blocked for more than 120 seconds. Mar 06 21:58:31 [1102454.455062] INFO: task clang:15033 blocked for more than 120 seconds. Mar 06 21:58:34 [1102454.455305] INFO: task clang:15042 blocked for more than 120 seconds. Mar 06 21:59:44 they are only in 2 groups 3 days ago, groups 3 hours between, so maybe it was just some one-time issue Mar 06 22:29:11 reticivis: Halium guide is good to follow for Android bits. Once you completed https://docs.halium.org/en/latest/porting/build-sources.html#initialize we've got some custom steps to follow Mar 06 22:29:29 Since we don't use Google's GCC toolchain but newer GCC and own build system for integration into LuneOS. Mar 06 22:31:44 Tofe: Good work on the patch. Shouldn't something like http://cgit.openembedded.org/openembedded-core/tree/meta/recipes-core/systemd/systemd/0013-Make-root-s-home-directory-configurable.patch?h=rocko help to get this working? Seems related, but might not be full fix Mar 06 22:35:21 Tofe: I meant more something like: http://openlinux.windriver.com/overc/sources/core2_64/systemd-219+stable+git0+85a6fabdd3-r0.3/0010-Make-root-s-home-directory-configurable.patch Mar 06 23:00:35 Tofe: This might be a "more proper" fix? https://github.com/devurandom/gentoo-patches/blob/2cca58e47c155282cf514ef85989dd6e1b3f701f/sys-apps/systemd-234-r4/c7b2630c9154722e2295f3dbfbb19f3305676ecf.patch Mar 06 23:01:28 Or this one seems related too: https://github.com/balabit-deps/balabit-os-7-systemd/blob/d15deca2cfbc3ad72fdb9ae4d872fbe745388269/debian/patches/service-relax-PID-file-symlink-chain-checks-a-bit-8133.patch **** ENDING LOGGING AT Thu Mar 07 02:59:57 2019