**** BEGIN LOGGING AT Mon Nov 25 02:59:58 2013 Nov 25 03:01:18 jake42: I have no choice, gta02 is the only device I use :( Nov 25 03:10:38 nightshift? Nov 25 03:41:11 DocScrutinizer05: hey, no, have woken up already. Nov 25 03:42:38 DocScrutinizer05: the irq line from pcf50633 seem to be asserted (and btw I can easily observe GPIO changes, e.g. AUX button, live over JTAG with a fully stopped CPU, JTAG does rock), and the irq is pending (in s3c interrupt controller) but masked and for some reason my patch doesn't lead to pending bit being cleared. Nov 25 03:48:24 Hopefully will be able to continue this evening, dayjob's calling now, I'm probably already late. Nov 25 04:01:39 hmm Nov 25 04:02:27 yeah, jtag boundaryscan is a great tool that most people have no idea of Nov 25 04:02:59 recently somebody tried to explain to me jtag was a debug interface for the cpu Nov 25 04:03:05 ;-P Nov 25 04:03:58 of the class of remote-gdb, a better serial interface Nov 25 04:04:28 actually originally this wasn't even a planned usecase for jtag Nov 25 04:09:09 btw isn't your TZ just 2h ahead of mine? Nov 25 04:10:35 duh, 3h. Ahh Mr Putin. Prolly the only reasonable thing he ever did in his life Nov 25 04:12:58 * DocScrutinizer05 wonders if the pending IRQ can only get reset by resetting the PCF50633 IRQ register Nov 25 04:14:22 when the CPU GPIO IRQ is level triggered, then resetting the IRQ-pending bit in CPU doesn't help anything. But I'm sure you already considered that Nov 25 04:16:33 usual IRQ scheme for intelligent peripherals: the peripheral has a pulldown open collector that asserts the IRQ on a GPIO (which often is even shared between multiple peripherals). The IRQ handler is servicing all peripherals and "harvesting" all the IRQ pending bits set *in the peripheral IRQ controllers* Nov 25 04:18:43 when the IRQ triggering condition (like low bat voltage or overtemp) in PCF50633 persists, then you even need to disable the IRQ, otherwise it gets re-triggered immediately after you harvesting it Nov 25 04:20:54 of cpurse this doesn't apply when the peripheral IRQ trigger isn't defined as a stage (overtemp) but rather as a statechange (normal -> overtemp) Nov 25 04:21:11 s/stage/state/ Nov 25 04:21:13 DocScrutinizer05 meant: of cpurse this doesn't apply when the peripheral IRQ trigger isn't defined as a state (overtemp) but rather as a statechange (normal -> overtemp) Nov 25 04:22:19 too long ago since I last time read pcf50633 datasheet, sorry Nov 25 04:23:12 but what I still recall is: this chip had quite a number of flaws and unexpected behaviour Nov 25 04:24:14 and been generally nasty and not exactly smart design Nov 25 04:26:43 hooray we go gaga Nov 25 05:45:21 DocScrutinizer05: I've actually tried using JTAG for boundary scan some time ago, the facility is surprisingly absent from free software tools (or obscure and underdocumented as in UrJTAG): https://webcache.googleusercontent.com/search?hl=en&q=cache:Nvj639Ip-zgJ:http://comments.gmane.org/gmane.comp.debugging.openocd.devel/23336%2Bmanual+boundary+scan+openocd&ct=clnk Nov 25 05:46:01 yeah Nov 25 05:46:26 DocScrutinizer05: btw, there's an ARM "semihosting" protocol that allows the target (debuggee) to make calls on host, e.g. to do input/output, so it sort of provides "serial console" over JTAG. Nov 25 05:46:27 free software isn't really about hardware, and JTAG is a hw thing in the end Nov 25 05:46:46 hehe Nov 25 05:47:04 yeah, but that's not what I've been told JTAG was for Nov 25 05:47:27 it was more like "it controlls the CPU so you can debug it" Nov 25 05:47:46 "boundary scan? what's that?" Nov 25 05:48:14 I do not like being at UTC+4 all the time. I get up and it's still dark, I get back home from work and it's already dark. We're now 2 hours off the real astronomical TZ (the first our was added by Soviet government early in the 1900s). Nov 25 05:48:40 hah Nov 25 05:49:04 I'd like to have summer time al year long Nov 25 05:49:29 though to be honest, right now it wouldn't make a difference Nov 25 05:49:50 time for a nap, hope I'll wake up before shops close Nov 25 05:50:05 DocScrutinizer05: good night, and see you soon. Always a pleasure to talk to you. Nov 25 05:50:11 o/ Nov 25 05:50:14 :-) Nov 25 05:50:15 same Nov 25 18:48:42 Hah, managed to make FR continue resuming by manipulating specific interrupt controller registers via jtag, hmm, that might lead to a solution eventually. Nov 25 18:52:12 :) Nov 25 18:54:36 It's still unclear why it manages to enter an interrupt for external controller and the handler doesn't ack it so it reenters again and again -> halt on resume. Nov 25 18:54:54 But I guess a workaround good enough can be implemented. Nov 25 19:00:22 hacking your only phone with jtag, needs a lot of courage and skill, kudos to you :) Nov 25 19:00:52 JaMa: btw, I'm rather glad to see you still hanging out here Nov 25 19:01:52 JTAG can't harm my phone, only help fixing :) Nov 25 19:02:39 unfurtunately hanging here + doing small changes in SHR before running new builds, is all I can do lately Nov 25 19:03:04 too much work in other parts of OE, not really specific or beneficial for SHR Nov 25 19:03:33 JaMa: $ job? Nov 25 19:08:14 JaMa: btw, I hope your offer to guide me on a small bicycle tour still holds, I'd be very glad to come cycle with you and my wife says that's a great idea too :) Nov 25 19:13:34 PaulFertser: yes for job and yes for bicycle tour Nov 25 19:14:22 PaulFertser: maybe I'll have to loose few pounds to keep up with you two :) working from home is really bad for my stamina Nov 25 19:32:18 JaMa: hehe, I'm afraid I'm not in the best condition either, there's nowhere to go by bicycle from the place I live now (still Moscow but suburbs). Nov 25 20:01:51 PaulFertser: pcf50533 not restting a IRW triggered by a buttonpress? Nov 25 20:02:02 s/IRW/IRQ/ Nov 25 20:02:03 DocScrutinizer05 meant: PaulFertser: pcf50533 not restting a IRQ triggered by a buttonpress? Nov 25 20:02:48 which IRQ are you referring to, in particular? Nov 25 20:06:09 * DocScrutinizer05 wonders what PaulFertser actually is running to do those suspend/resume tests. AIUI the suspend and particularly resume is triggered by powerbutton press? powerbutton is a *very* special critter that sends a first warning IRQ and CPU needs to trigger a watchdog or reset IRQ to not get reset from PCF50633 8s later. This is a thing implemented in kernel Nov 25 20:07:15 so when kernel gets stuck and can't do an ordered shutdown (or whatever else the action supposed to get triggered by powerbutton) then pcf50633 resets the CPU hard after a few seconds Nov 25 20:07:27 all disclaimer: IIRC Nov 25 20:17:01 DocScrutinizer05: no, something other is going on there: the kernel is not acking an external interrupt once (for whatever reason) and then it keeps coming again and again but because all ext interrupts are masked it doesn't call any handler, the irq is left unacked. Nov 25 20:17:26 DocScrutinizer05: I'm running a shell while cycle with rtcwake. Nov 25 20:17:41 aaah, nifty Nov 25 20:19:26 DocScrutinizer05: iirc that pcf50633 irq was supposed to just wake the cpu up when the battery is low to allow it to signal the user and shutdown cleanly. Nov 25 20:19:50 for ack of external IRQ, you are aware that you need I2C powered to talk to pcf50633 to do the ack? Nov 25 20:20:44 lindi-: radekp: If anyone wants to try, here's the possible solution, seems to work here (at least it lasts considerably longer than the previous time without the patch): http://paste.debian.net/67730/ (on top of SHR's 2.6.39.4). Will tell more tomorrow morning, have to go to sleep now. Nov 25 20:21:04 DocScrutinizer05: it's masked anyway, I just need to tell the SoC's interrupt controller to calm down :) Nov 25 20:21:16 DocScrutinizer05: JaMa: good night, see you soon folks :) Nov 25 20:21:23 cya Nov 25 20:21:37 a masked interupt however is a problem Nov 25 20:21:52 Once the kernel resumes fully, it'll unmask it and the driver will handle it. Nov 25 20:22:10 your goal is NOT to make the IRQ controller shut up, but to reset the external IRQ Nov 25 20:22:39 The issue here is that I see an external interrupt pending but no actual EINT unmasked. So the SoC-level IRQ fires again and again, and is never acknoledged as no handler for EINT is unmasked. Nov 25 20:23:09 Once the kernel fully resumes it'll unmask the corresponding EINT and will handle it accordingly. Nov 25 20:23:17 how can IRQ fire when it's masked? Nov 25 20:24:00 If it's not acknoledged, it'll fire again. See SRCPND + INTPND in the SoC's manual. Nov 25 20:24:16 when? Nov 25 20:24:31 I'd say as soon as it gets unmasked Nov 25 20:24:36 The question is why the hell it got masked but the parent irq bit not acknoledged. Who knows, probably some race condition somewhere, too hard to track down. Nov 25 20:25:13 Even if all EINTs are masked the exception will get raised again and again unless something writes 1 to SRCPND and INTPND. Nov 25 20:25:33 that can't be correct Nov 25 20:26:20 since when CPOU gets an exception on every single clock cycle, it will not do anything reasonable, but simply jitter between to opcodes Nov 25 20:26:28 2442B page 14-7 SRCPND description. Nov 25 20:26:28 CPU* Nov 25 20:26:56 Sorry, really have to sleep now... Nov 25 20:27:05 n8 o/ :) Nov 25 20:27:38 My workaround seems to be fine so far: 100 cycles and still going while I had about 4 hangs for the same period with the same config before. Nov 25 20:27:55 the problem is in distinguishing between wake-event and IRQ which are both triggered by same level on same GPIO Nov 25 20:28:53 DocScrutinizer05: I'll try to explain what I've understood so far about this issue tomorrow. Nov 25 20:29:04 SoC needs to resume on wake-event with IRQs masked, crank up all power domains and whatnot, THEN unmask IRQ and same moment jump into IRQ-handler that does the service Nov 25 20:30:03 the trick is to deliberately mask the "normal" IRQ before going suspend Nov 25 20:30:09 gnite Nov 25 20:30:56 yay usable kernel for GTA02 - that sounds good Nov 25 20:31:19 the SoC needs to not get disturbed during resume-from-suspend, until it got ready to service IRQs again Nov 25 20:31:49 few months ago i have given up and made QtMoko branch based on 2.6.28 - it would be nice to delete it :) Nov 25 20:34:29 radekp: I've upgraded to 3.2 in SHR/master lately, because I needed 3.0+ for new systemd Nov 25 20:34:44 JaMa: hi Nov 25 20:34:55 background story, maybe loosely related: on N900 you need to do "sleep5&echo ram>$suspendsysnode" since otherwise you can't release CR key on keyboard, device goes to suspend, wake up from suspend with CR key released and no message/event created for that ->result: kbd locked Nov 25 20:35:20 JaMa: yup those old kernels are pain - i had to backport and badly hack devtmpfs to 2.6.28 to make it working again... Nov 25 20:36:16 ~lart jama for using systemd on embedded Nov 25 20:36:16 * apt executes killall -HUP jama for using systemd on embedded Nov 25 20:37:44 systemd is explicitly ignoring and abandoning needs of embedded devices. See /usr Nov 25 20:39:05 "we don't need /usr anymore since nowadays every system has a 10GB root partition at least" Nov 25 20:39:59 "thus we can abandon udev and move everything into systemd" Nov 25 20:40:45 dbus? suuuure, gets started by process#1 Nov 25 20:41:34 * DocScrutinizer05 waits for Poettering's next move: PA mandatory part of system init aka systemd Nov 25 20:41:46 step 3: wayland Nov 25 20:42:04 DocScrutinizer05: maybe there is small chance that Lennart will find another project and leaves systemd maintainance to some sane people? Nov 25 20:42:27 "why support framebuffer console when nowadays every system is using wayland or the obsolete ancestor X11?" Nov 25 20:43:14 well I don't see big benefit of separate /usr anymore Nov 25 20:43:38 honestly and seriously, I don't think embedded devices are suited for systemd Nov 25 20:43:59 JaMa: MHM Nov 25 20:44:28 JaMa: Nov 25 20:44:29 we're using upstart in webOS and I don't think it's better than systemd :) Nov 25 20:44:30 IroN900:~# df -h Nov 25 20:44:31 Dateisystem Size Used Avail Use% EingehÃĪngt auf Nov 25 20:44:32 rootfs 228M 168M 56M 76% / Nov 25 20:44:34 ubi0:rootfs 228M 168M 56M 76% / Nov 25 20:45:07 I'm absolutely sure upstart is better than systemd, in that regard Nov 25 20:46:09 /dev/mmcblk0p2 2,0G 857M 1,1G 45% /home Nov 25 20:46:38 IroN900:~# ll -d /opt Nov 25 20:46:39 drwxr-xr-x 38 root root 4096 2013-06-23 11:04 /opt Nov 25 20:47:14 /opt/pymaemo/usr/lib/python2.5 2,0G 857M 1,1G 45% /usr/lib/python2.5 Nov 25 20:47:28 ~optification Nov 25 20:47:28 optification is a inventive duct tape workaround to reclaim space in fs root, done due to the fact the systeminit *and* partitioning is FUBAR, http://wiki.maemo.org/Documentation/Maemo_5_Developer_Guide/Packaging,_Deploying_and_Distributing/Installing_under_opt_and_MyDocs, or ""OMG - I wish they looked into FHS and moved /usr to eMMC"", http://www.pathname.com/fhs/pub/fhs-2.3.html#PURPOSE2 bullet1,2 and fhs-2.3.html#PURPOSE16 dot3" Nov 25 20:48:21 now tell me what's Poettering's rationale why we can ignore FHS Nov 25 20:48:57 well, you can still have a separate /usr, you just need to mount it in initrd Nov 25 20:49:24 allegedly this doesn't work with systemd Nov 25 20:49:34 (but i guess I should not interrupt your so productive rant) Nov 25 20:50:11 and btw the system I quoted above doesn't even *have* any initrd Nov 25 20:51:10 since initrd is a botch to cope with unknown platform config, so "one kernel ( / system) fits all" Nov 25 20:53:02 why would I implement initrd on a embedded system that runs a highly customized kernel and core system anyway? Nov 25 20:54:11 i am using it on qtmoko/GTA04 for some obscure booting options Nov 25 20:54:26 initrd got invented to overcome installation issues with former linux systems where you had to build the kernel to include all the fs (and other) drivers you need to crank up the system until #1 process Nov 25 20:54:27 e.g. root on extended partition Nov 25 20:55:14 and for rebooting from one partition to another - IMO quite nice... Nov 25 20:55:40 initrd not needed for any of those purposes Nov 25 20:56:08 you got uBoot for that Nov 25 20:56:08 hmm right that could have been done with uboot Nov 25 20:56:13 and kexec Nov 25 20:57:04 embedded is not X86-ISA Nov 25 20:57:15 but it was nice opportunity for playing with klibc - or how is it called... Nov 25 21:01:07 DocScrutinizer05: btw uboot cant boot from extended partitions and it does not have ext4 support (but patch exists) Nov 25 21:01:54 DocScrutinizer05: and running os from ext3 is really big problem - unless you dont mind that it's slow as hell Nov 25 21:02:09 well, if you actually need >4 systems in your multiboot then you probably got other problems on embedded than just extended partitions Nov 25 21:02:19 ;-) Nov 25 21:02:38 DocScrutinizer05: i do need it for testing regressions Nov 25 21:03:10 consider rewriting partitoon table in MBR Nov 25 21:03:42 or patch uBoot Nov 25 21:05:33 oki, but still i will like my 22kb initramfs which works perfectly for me Nov 25 21:08:22 partitioning A: 1GB systemA; 1GBsystemB; 1GB systemC; 1GB systemD; 8GB unused (dummy) Nov 25 21:08:23 partitioning B: 4GB dummy; 1GB systemE; 1GB systemF; 1GB systemG... Nov 25 21:09:55 also you don't really need a primary partition for each system, only for each different kernel you want to run, AIUI Nov 25 21:11:12 on N900: Nov 25 21:11:15 IroN900:~# cat /proc/mtd Nov 25 21:11:17 dev: size erasesize name Nov 25 21:11:18 mtd0: 00020000 00020000 "bootloader" Nov 25 21:11:20 mtd1: 00060000 00020000 "config" Nov 25 21:11:21 mtd2: 00040000 00020000 "log" Nov 25 21:11:23 mtd3: 00200000 00020000 "kernel" Nov 25 21:11:24 mtd4: 00200000 00020000 "initfs" Nov 25 21:11:26 mtd5: 0fb40000 00020000 "rootfs" Nov 25 21:11:32 initfs unused since they decided it works better without Nov 25 21:11:56 alas it's too small to format it with ubi Nov 25 21:12:02 :-) Nov 25 21:12:06 or :-S Nov 26 02:12:48 1360 cycles and counting, yay, the workaround works. Nov 26 02:14:46 already awake or just interrupted sleep to check cycles? :) Nov 26 02:18:21 JaMa: good morning :) Nov 26 02:18:34 Have to wake up to go to work soon. Nov 26 02:45:59 PaulFertser: don't tell me you already are awake again! Nov 26 02:46:43 you have the wrong job ;-) Nov 26 02:47:09 can't even be 7o' at moscow TZ Nov 26 02:49:19 PaulFertser: take care your patch is not only introducing a duct tape fix based on changes in timing of a race condition ;-) Nov 26 02:51:38 AIUI the critical coditions to be met are: mask IRQs and allow wake-events before suspending. After resume first initialize system, particularly I2C and other stuff needed to talk to pcf50633, only then unmask IRQs and allow pending IRQ from PCF50633 to invoke irq-handler Nov 26 02:52:03 DocScrutinizer05: I could have woken up an hour later but it was nicer to wake and go together with my wife. Nov 26 02:53:04 DocScrutinizer05: I have 3 jobs actually, all different: part-time engineer, 1-day-a-week "consultant", university teacher :) Nov 26 02:53:11 irq handler in turn masks IRQs (on SoC) again - inplicitly usually, and talks to PCF50633 to service/flush the IRQ cause there and thus de-assert the IRQ line to GPIO, then irq-handler unmasks IRQs again Nov 26 02:53:24 eeeek, teacher Nov 26 02:53:31 :-D Nov 26 02:53:56 for me teacher only when I'm also chanop ;-) Nov 26 02:55:46 DocScrutinizer05: do you mean you're teaching too now? Nov 26 02:55:59 nah, prolly only babbling Nov 26 02:56:49 I don't feel like 1st level helpdesk in *this* channel. Which makes the channel kinda special Nov 26 02:57:55 you know I'm in other channels where matters are really like in primary school sometimes **** ENDING LOGGING AT Tue Nov 26 02:59:58 2013