**** BEGIN LOGGING AT Tue Apr 23 02:59:57 2019 Apr 23 04:59:06 whats the difference of mka mkbootimg from mka halium-boot? Apr 23 06:33:57 @TuxThePenguin [], it is due to missing config_devtmpfs Apr 23 06:36:27 @TuxThePenguin [], [Edit] it is due to missing CONFIG_DEVTMPFS Apr 23 06:54:31 Ok I have changed some stuff around, it took a while but it makes testing things a bit more streamlined and easy Apr 23 06:54:53 So now I'm going to change a few kernel settings at a time and see if I can find what was causing panics **** BEGIN LOGGING AT Tue Apr 23 07:43:30 2019 Apr 23 07:44:21 I was originally going to try and port Halium to the Ace 3 LTE (loganreltexx) but it runs 3.4, has 8GB of storage, has garbage specs and had a similar kernel panic issue that I couldn't figure out because the old kernel version doesn't have the ram logging in an easy way Apr 23 07:45:03 Surprisingly it did actually compile, it just wasn't worth making it actually boot Apr 23 07:49:52 Flohack: I have some old 3.4 kernels working with Halium 5.1 and pretty recent systemd Apr 23 07:51:18 But it's a quite fragile situation Apr 23 08:01:51 @TuxThePenguin [3.18, I always think it's 3.10 lol], I am working on 3.10.49 Apr 23 08:03:00 FINALLY, I just narrowed down the kernel panics to these config lines https://paste.ubuntu.com/p/hY74byMwkh/ Apr 23 08:21:08 you can skip AppArmor for now Apr 23 08:21:19 I don't know if it's apparmor yet Apr 23 08:21:33 I'm still looking for exactly which line is causing it Apr 23 08:21:50 Although if this next boot fails I can be pretty sure it's AppArmor Apr 23 08:24:10 it is probably disabling SELinux and replacing it with AppArmor Apr 23 08:25:13 Ok yeah I think it's AppArmor, I'm just doing the last test now Apr 23 08:33:31 And as it turns out that was not the last test, multiple lines were causing the panic, I'm trying to disable all lines related to apparmor now Apr 23 08:39:31 Hmm thats still not working Apr 23 08:44:23 @DEEP73 [I am working on 3.10.49], Cool do yu managed to get Halium rootfs and container running? Apr 23 08:44:31 @DEEP73 [I am working on 3.10.49], [Edit] Cool did you managed to get Halium rootfs and container running? Apr 23 08:55:18 @TuxThePenguin [Ok yeah I think it's AppArmor, I'm just doing the last test now], There are ut specific port for apparmor that has to be patched Apr 23 08:55:33 @TuxThePenguin [Ok yeah I think it's AppArmor, I'm just doing the last test now], [Edit] There is ut specific port for apparmor that has to be patched Apr 23 08:56:08 But thats for later Apr 23 08:57:06 I have also problems with apparmor on 3.18 but at least it doesnt prevents booting Apr 23 08:58:02 @Flohack [Cool did you managed to get Halium rootfs and container running?], I built halium-boot.. I will test it a week later .. I have only one phone right now, so can't mess with it until I get another Apr 23 08:58:53 But last time I checked , my device didn't boot and I got no ssh accesd Apr 23 09:00:44 Ok I see so you need more time. I need someone who has already ssh to check lxc container startup for me (which istn working still) Apr 23 09:15:30 anyone know this error? … https://paste.ubuntu.com/p/S2W4Ymsxp2/ Apr 23 09:17:59 @nanu_c [There is ut specific port for apparmor that has to be patched], I'm actually not 100% sure what it is, I'm just going through enabling small amounts of kernel options at a time to see how many I can get enabled without it panicking Apr 23 09:19:06 @TuxThePenguin [I'm actually not 100% sure what it is, I'm just going through enabling small amo …], Do you get ssh access if it is not panicking? Apr 23 09:20:11 So far only telnet Apr 23 09:22:54 And an inaccessible MTP device called "Failed to boot" when I use halium-boot Apr 23 09:23:16 @TuxThePenguin [So far only telnet], Ok then smth might be wrong with systemd for you Apr 23 09:23:38 What does dmesg say in telnet? Apr 23 09:25:10 Hold on and Ill put up a paste Apr 23 09:26:41 Thankfully I can boot it without it panicking now, before when it was panicking the dmesg ramoops logs were always badly corrupted https://paste.ubuntu.com/p/6KCkKSr6T6/ Apr 23 09:28:33 It seems it cannot mount /dev and /run directories: … ```[01-01 22:11:28.200] [3][529: exe]Begin: Running /scripts/init-bottom ... … [01-01 22:11:28.200] [3][529: exe]mount: mounting /dev on /root/dev failed: No such file or directory … [01-01 22:11:28.200] [3][530: exe]mount: mounting /dev on /root/dev failed: No such file or directory Apr 23 09:28:33 … [01-01 22:11:28.200] [0][1: init]done. … [01-01 22:11:28.200] [2][531: exe]mount: mounting /run on /root/run failed: No such file or directory … [01-01 22:11:28.200] [2][532: run-init]run-init: current directory on the same filesystem as the root: Not a typewriter … [01-01 22:11:28.200] [0][1: init]Target filesystem doesn't have requested /sbin/i Apr 23 09:28:33 nit. … [01-01 22:11:28.203] [1][533: run-init]run-init: current directory on the same filesystem as the root: Not a typewriter … [01-01 22:11:28.203] [2][534: run-init]run-init: current directory on the same filesystem as the root: Not a typewriter … [01-01 22:11:28.203] [2][535: run-init]run-init: current directory on the same filesystem as the ro Apr 23 09:28:33 ot: Not a typewriter … [01-01 22:11:28.203] [2][536: run-init]run-init: current directory on the same filesystem as the root: Not a typewriter … [01-01 22:11:28.203] [2][537: run-init]run-init: current directory on the same filesystem as the root: Not a typewriter … [01-01 22:11:28.206] [0][1: init]No init found. Try passing init= bootarg.``` Apr 23 09:28:43 uh got too long Apr 23 09:28:57 It seems it cannot mount /run and /dev dirs Apr 23 09:29:07 so that support might be missing somehow in kernel but how? Apr 23 09:29:15 Well thats probably not the worst thing that can happen Apr 23 09:29:19 ```[01-01 22:11:28.200] [3][529: exe]Begin: Running /scripts/init-bottom ... … [01-01 22:11:28.200] [3][529: exe]mount: mounting /dev on /root/dev failed: No such file or directory … [01-01 22:11:28.200] [3][530: exe]mount: mounting /dev on /root/dev failed: No such file or directory … [01-01 22:11:28.200] [0][1: init]done.``` Apr 23 09:29:31 Well it is because then your init script stops there Apr 23 09:29:49 `[01-01 22:11:28.206] [0][1: init]No init found. Try passing init= bootarg.` Apr 23 09:30:45 So the good news is it fails late on init-bottom but that wont help you much now ^^ Apr 23 09:30:52 What did you disable in kernel config Apr 23 09:32:28 Right now I am down to 4 things that deviate from what the halium checker tool for UBports wants, CONFIG_SECURITY_APPARMOR_BOOTPARAM_VALUE=1, CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=0 Apr 23 09:33:04 [Edit] Right now I am down to 4 things that deviate from what the halium checker tool for UBports wants, CONFIG_SECURITY_APPARMOR_BOOTPARAM_VALUE=1, CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=0, CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE=1 and CONFIG_DEFAULT_SECURITY_APPARMOR=y Apr 23 09:33:49 This is the current config https://paste.ubuntu.com/p/kFWtDMVpfX/ Apr 23 09:34:17 And with those 4 the kernel panicks? strange Apr 23 09:34:30 I'm not sure yet, still testing Apr 23 09:35:11 But at least one of those is causing it Apr 23 09:36:46 Ok I just compiled with CONFIG_SECURITY_APPARMOR_BOOTPARAM_VALUE=1 and it's fine so it's down to 3 now Apr 23 09:36:47 Can you look in telnet if you got /dev and /run folders Apr 23 09:37:11 I do have /dev and /run folders Apr 23 09:37:16 @TuxThePenguin [Ok I just compiled with CONFIG_SECURITY_APPARMOR_BOOTPARAM_VALUE=1 and it's fine …], stange Apr 23 09:37:22 Ah wait Apr 23 09:37:34 There is no /root/dev Apr 23 09:37:49 Only /root/android, /root/data and /root/userdata Apr 23 09:37:53 hmm ok so mount failed maybe Apr 23 09:40:11 Any idea what might cause that? Apr 23 09:40:51 I imagine it's something to do with the initrd since I don't think it's getting to the Android stage? Apr 23 09:44:14 @TuxThePenguin where did you put the rootfs? Apr 23 09:44:19 it should be /data/rootfs.img Apr 23 09:44:56 Oh god Im a total idiot Apr 23 09:44:58 the process is like halium-boot/hybris-boot initramfs -> Linux rootfs with systemd or upstart -> Android start inside LXCcontainer Apr 23 09:45:11 [Edit] the process is like halium-boot/hybris-boot initramfs -> Linux rootfs with systemd or upstart -> Android start inside LXC container Apr 23 09:45:14 I didn't realise it was supposed to be /data/rootfs.img Apr 23 09:45:29 I thought you flashed the image to the data partition Apr 23 09:45:40 Oh wow I feel stupid lol Apr 23 09:46:22 also rootfs.img is Linux rootfs, not system.img you compiled in Halium Apr 23 09:46:30 Yeah I know Apr 23 09:46:41 Also I just found one of the kernel options that causes it to panic Apr 23 09:47:19 I was basically extracting the hybris reference rootfs directly onto the data partition Apr 23 09:47:31 @TuxThePenguin [I was basically extracting the hybris reference rootfs directly onto the data pa …], oh lol fail ^^ Apr 23 09:47:39 Please use the installer script, its there for a reason Apr 23 09:47:45 Its also doing post-install steps Apr 23 09:48:04 For example setting an SHH password Apr 23 09:48:09 I wasn't using the installer script since at first my device didn't have fastboot working (that was using the oreo bootstack) Apr 23 09:48:10 [Edit] For example setting an SSH password Apr 23 09:48:25 Should the device be in recovery or fastboot? Apr 23 09:48:32 Nothing to do with fastboot, the halium-install uses only adb Apr 23 09:48:35 recovery Apr 23 09:48:56 Ok, oh yeah I remember, I was also using a newer TWRP version which disables ADB at the start Apr 23 09:50:28 where should i place the pid_namespace patch? sorry I just started yesterday Apr 23 09:50:29 hehe fix all that, and then do http://docs.halium.org/en/latest/porting/install-build/reference-rootfs.html#install-rootfs-and-system-img-alternative-method Apr 23 09:50:43 Alright I'll just finish finding which options cause the panics and then I will compile and install a working kernel and system with the script Apr 23 09:51:29 And yeah I'm using TWRP 3.1.1 now so I have ADB access (I really wish they didn't disable it, ADB is a very useful thing for a recovery to have) Apr 23 10:04:14 I found the two kernel configurations that were causing the panics! CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=0 … CONFIG_DEFAULT_SECURITY_APPARMOR=y Apr 23 10:04:49 well, basically disabling SELinux and enabling AppArmor Apr 23 10:05:13 this was suggested from the beginning, but now you've found it the hard way Apr 23 10:06:03 Yeah pretty much, hopefully until later that will prevent as many problems as possible though (there were other things in there that I thought may have been causing it) Apr 23 10:07:18 Also I didn't think SELinux was the problem since in previous logs where it hadn't panicked there were lines saying that selinux was disabled Apr 23 10:37:51 Umm I ran the installer script and it seems to not want to create one of the filesystems https://paste.ubuntu.com/p/xk57XFwMC2/ Apr 23 10:49:04 Hmm maybe you are really out of space? Happens frequently to me since Android tree is so huge 😆 Apr 23 10:49:18 Oh ON the device Apr 23 10:49:31 is your data partition original size again Apr 23 10:49:37 after you have fiddled with it 😆 Apr 23 10:50:15 Well no idk. either of that Apr 23 10:50:39 Ah and try this halium-install from JBB: https://github.com/JBBgameich/halium-install/ Apr 23 10:50:58 and dont forget to specify -p reference Apr 23 10:57:42 Oh yeah I forgot about that script lol Apr 23 10:58:09 And nah my device isn't out of space, the system partition alone is 6GB and I have checked to make sure all filesystems are the full size of their respective partiitons Apr 23 10:58:14 [Edit] And nah my device isn't out of space, the system partition alone is 6GB and I have checked to make sure all filesystems are the full size of their respective partitions Apr 23 10:59:12 try the other one. If no help, put a bash -x or so somewhere sop you can see what it is trying to do. If it is on host or device Apr 23 11:15:17 Heyyyy it booted Apr 23 11:15:27 SSH is working Apr 23 11:15:37 cool! Gimme 5 Apr 23 11:15:49 Congrats Apr 23 11:18:14 @TuxThePenguin [SSH is working], Can you immediately do me a favour and type lxc-info -n android Apr 23 11:19:17 Hmm, the connection to the device is very unstable, I can log in but then after some time or immediately, it's random, the whole terminal gets locked up and I have to reboot the device to get it to connect again Apr 23 11:19:34 @TuxThePenguin [Hmm, the connection to the device is very unstable, I can log in but then after …], The issue is most likely with your host pc Apr 23 11:19:52 NetworkManager loves to take control of the interface Apr 23 11:19:52 Ill see if I can connect from a different device Apr 23 11:19:56 @vanyasem [NetworkManager loves to take control of the interface], And then freaking nuke it Apr 23 11:20:07 So you need to type killall ssh Apr 23 11:20:11 Killing all ssh processes Apr 23 11:20:16 And then connect again Apr 23 11:20:37 Also a tip - you can configure nm to ignore the interface of your phone Apr 23 11:20:47 Or well disable it for a while for the sake of sanity Apr 23 11:21:12 @TuxThePenguin [Ill see if I can connect from a different device], You need to add your RNDIS device to the network manager ignore list Apr 23 11:21:12 On kde you can even see that it tries to connect to your phones interface from the tray applet Apr 23 11:21:21 Thats a nuisance Apr 23 11:21:38 We should have dhcpd also running on ssh mode, not only on telnet Apr 23 11:21:51 It seem to work much better with NM then Apr 23 11:22:13 Yeah with DHCPD it worked very reliably Apr 23 11:23:01 Add this to the network manager conf somehow: `unmanaged-devices=interface-name:` Apr 23 11:23:30 Note that tzhe interface name always changes when you change the USB port, so I have a habit to use always the same with my "porting" USB cable ;) Apr 23 11:24:01 Yeah I have the device always plugged into the same USB port anyway Apr 23 11:25:46 Ok yeah it seems to be working much better now Apr 23 11:26:11 Oh wait no it happened again Apr 23 11:26:36 hmmm dang ^^ Apr 23 11:27:44 If I knew where my PCMCIA USB card was I would have the perfect machine for this Apr 23 11:32:07 I found it! Apr 23 11:32:09 Alright Apr 23 11:33:47 I have an old machine running barebones Debian Jessui so I'm hoping that will work Apr 23 11:35:46 Alright it's SSHed in Apr 23 11:35:50 Lets see if this works Apr 23 11:36:46 @Flohack [Can you immediately do me a favour and type lxc-info -n android], It returns "Name: android" and "State: STOPPED" Apr 23 11:39:03 @TuxThePenguin [It returns "Name: android" and "State: STOPPED"], Ok so you got maybe the same issue as myself Apr 23 11:39:09 Can you type lxc-checkconfig Apr 23 11:39:16 and paatebin me the output? Apr 23 11:39:58 I don't really think I can pastebin you the output since this laptop is so old that no web browsers really function on it lol, Ill see what I can do though Apr 23 11:42:58 Screenshot? Picture? 😆 Apr 23 11:43:23 Im trying, you will have to bear with me for a few minutes while I get my beyond jank solution sorted :P Apr 23 11:43:55 haha ye ssure no worries Apr 23 11:49:28 In the jbbgamrich repo is also a connect script. It takes care of everything even on changing ssh keys Apr 23 11:49:54 I just got it working anyway lol Apr 23 11:50:04 So from my main computer I SSHed into my old laptop Apr 23 11:50:10 Then from that I SSHed into the phone lol Apr 23 11:50:40 https://github.com/JBBgameich/halium-install/blob/master/connect.py Apr 23 11:50:47 https://paste.ubuntu.com/p/fbc4r8Th2g/ Apr 23 11:50:57 @Flohack Apr 23 11:51:47 `connect.py -p ssh -u phablet` Apr 23 11:51:59 Or same with telnet Apr 23 11:52:43 On a separate note the first command the halium porting guide tells me to do isn't working https://paste.ubuntu.com/p/5HcGW8Skyr/ Apr 23 11:54:09 You have to mount it as rw and then do it again Apr 23 11:55:27 Ok that fixed one thing but it's still saying "cat: '/var/lib/lxc/android/rootfs/ueventd*.rc': No such file or directory" Apr 23 11:56:00 There is no rootfs folder inside /var/lib/lxc/android Apr 23 11:56:03 @TuxThePenguin [Ok that fixed one thing but it's still saying "cat: '/var/lib/lxc/android/rootfs …], Thats becasue your container is stopped Apr 23 11:56:09 probably Apr 23 11:56:12 Yeah that makes sense Apr 23 11:56:18 @TuxThePenguin [Ok that fixed one thing but it's still saying "cat: '/var/lib/lxc/android/rootfs …], Grats on getting to a ssh connection Apr 23 11:56:57 @TuxThePenguin [@Flohack], Ok can you try lxc-start -F -n android to start it in foreground? Apr 23 11:57:05 DOnt be shocked about 2000 lines of errors 😆 Apr 23 11:57:18 the config looks OKish Apr 23 11:57:42 This doesn't look like too many errors lol https://paste.ubuntu.com/p/QTt8cDMHNc/ Apr 23 11:57:43 `Cgroup cpuset: enabled` @nanu_c ^^ Apr 23 11:58:10 LOL Apr 23 11:58:20 Then smth went wrong with your android container Apr 23 11:59:16 There is no /system folder although I'm not sure if thats supposed to be in a container somewhere Apr 23 12:01:13 @TuxThePenguin [There is no /system folder although I'm not sure if thats supposed to be in a co …], Did you already run lxc-check-config (or whatever the command is) Apr 23 12:01:40 https://paste.ubuntu.com/p/fbc4r8Th2g/ Apr 23 12:01:46 Time to keep changing kernel configs lol Apr 23 12:02:39 That looks like everything you need to get the container going, I gotta get going though, but good luck Apr 23 12:03:05 @Flohack [We should have dhcpd also running on ssh mode, not only on telnet], it should just work now Apr 23 12:03:26 @NotKit [it should just work now], It does not on my Ubuntu 16.04, still have to disable NM for that interface Apr 23 12:03:55 are you using latest rootfs? I commited fix some months ago Apr 23 12:05:20 For things like these what configs would I change to fix them? … "Veth pair device: missing" … "Macvlan: missing" … "Vlan: missing" Apr 23 12:16:24 @NotKit [are you using latest rootfs? I commited fix some months ago], Maybe not... I am not at my compute rnow Apr 23 12:22:06 So I have been able to fix most of the missing things with easy configuration changes but there are a few that I am stuck on https://paste.ubuntu.com/p/5DRMS44tmX/ … Cgroup memory controller should be enabled and so should CONFIG_IP6_NF_TARGET_MASQUERADE although apparently they aren't and I'm not sure what to change for the others Apr 23 12:28:59 oh, and I meant UBPorts rootfs, not Halium one Apr 23 12:35:16 @NotKit [oh, and I meant UBPorts rootfs, not Halium one], Oh OK will try that today. BTW is it fine to use 32bit rootfs with the 64bit kernel and device tree stuff? Apr 23 12:40:27 generally yes, but there can be corner cases like broken 32-bit hwcomposer ioctls on MediaTek devices or missing libs Apr 23 12:44:27 @TuxThePenguin [So I have been able to fix most of the missing things with easy configuration ch …], As long as the stuff in the first 2 sections doesnt have anything in red that says missing, the container should be able to start and run Apr 23 12:45:27 @NotKit [generally yes, but there can be corner cases like broken 32-bit hwcomposer ioctl …], Ok I hope this wont happen on Nexus 6P 😆 Apr 23 12:45:30 Since its not starting for you I have a feeling that you might have a corrupted system.img, can you manually mount that in TWRP? Apr 23 12:45:46 Ill have a look Apr 23 12:45:57 @Tygerpro [As long as the stuff in the first 2 sections doesnt have anything in red that sa …], I got all green and still no start because of weird cpuset errors (no permissions in cgroup folders) Apr 23 12:46:38 @Flohack [I got all green and still no start because of weird cpuset errors (no permission …], Interesting, which device was that on? Apr 23 12:46:39 @Flohack [Im posting my lxc startup errors here too: https://pastebin.com/amu9jc5g], See here Apr 23 12:46:48 Nexus 6P Apr 23 12:48:14 Ok the installer script left a system.img file in the data partition which mounts fine in TWRP, should it be an image file on the data partition or should it be flashed to the system partition which is currently blank? Apr 23 12:48:46 @Flohack [See here], I see alot of "read-only file system" did you do "touch .writable-folder" (or whatever that command is) to boot it as a r/w filesystem? Apr 23 12:49:31 I have noticed that my device is also starting up with a read-only filesystem Apr 23 12:50:30 are you booting UBPorts? Apr 23 12:50:45 I'm booting the Halium reference filesystem Apr 23 12:50:53