**** BEGIN LOGGING AT Wed May 02 02:59:58 2012 May 02 10:19:24 I can see my device crash when it runs out of memory, all lights go off except power, and uptime goes back to zero whenI can log back in again, but I don't see anything in /sys/kernel/debug/crashlog. am I really meant to be seeing anything there? May 02 10:20:11 also, with the default setup, oomkiller should be running, and I've set my own proces (deliberately leaking memory) to have a high oom_adj score, and I can see that it's oom_score keeps rising as it keeps consuming memory, so I _expect_ to see that process killed, not thebox crash. May 02 10:20:22 anyone have any ideas on where I could look to see what's going wrong? May 02 10:35:21 karlp: try to reduce /proc/sys/vm/overcommit_ratio May 02 10:35:54 should I be getting an oom_dump_tasks log somewhere too? May 02 10:36:06 maybe the oom killer fires, but only after the kernel has already dropped various things from ram that it can reload from filesystem May 02 10:36:17 that might delay the watchdog process or something May 02 10:36:26 at least that's one way that could lead to reboot without a stack trace May 02 10:36:47 I guess I should set up remote syslog too, that might get me a few more seconds. May 02 10:37:14 not sure if remote syslog is fast enough May 02 10:37:43 http://www.kernel.org/doc/Documentation/sysctl/vm.txt says that overcommit_ratio only matters when overcommit_memory is 2? May 02 10:38:07 ah May 02 10:38:45 I'm not sure I want to completely disable overcommitting for everything, May 02 10:38:53 though it'ðs probably worth trying out, May 02 10:39:18 I just had hoped that this oom_adj knob would just mean I could rely on my task getting killed _always_ first. May 02 10:39:47 I can see it have a vsize of ~1000% of memory before it crshes May 02 10:49:45 CommitLimit: 30996 kB May 02 10:49:45 Committed_AS: 10564 kB May 02 10:49:57 total used free shared buffers May 02 10:49:57 Mem: 61992 34340 27652 0 2136 May 02 10:50:13 my limit seems to be lower May 02 10:50:47 karlp: do you have panic_on_oom set? May 02 10:52:40 no, I was experimenting with that too, but it's off right now, May 02 10:55:23 I've got commitLimit 14900, committed_AS at 238300 right now. May 02 10:56:12 > 238300 May 02 10:56:15 fffffffffffffffffffffffffffffffffffffffff May 02 10:57:26 yeah, that's my process that just keeps mallocing every 100ms to try and test out this May 02 10:57:46 yeah, turning overcommit_memory to 0 just means "-ash: can't fork" May 02 10:58:36 ok, maybe I turned the overcommit_ratio down too low May 02 11:08:50 well, ok. turning overcommit_memory to 2 (no overcommitting beyond overcommit_ratio) does indeed make my bad process die and the system keeps running. May 02 11:09:45 though there's nothing in the logs abotu OOM, so I guess that's actually my app crashing from the malloc failure/segfault rather than being killed for using too much memory May 02 11:53:25 ok, has anyone got any other opinions on setting overcommit_memory to 2? I'd rather not change entire system behaviour just to make sure that any badly written apps of mine crash first May 02 11:53:54 I've completely failed to get the OOM killer to actually kill anything before the system restarts, presumably due to watchdog timeouts May 02 13:40:18 build #10 of s3c24xx is complete: Failure [failed compile_4] Build details are at http://buildbot.openwrt.org:8010/builders/s3c24xx/builds/10 May 02 14:14:26 hey, what was the fix for this kernel_menuconfig issue? http://pastebin.com/gbVZhyWZ May 02 14:18:03 meh, solved it, did make defconfig - nothing, make kernel_menuconfig still breaks, then make defconfig again and now i can make kernel_menuconfig .. May 02 14:18:29 is there a compile defintion to know whether I'm on trunk or backfire? I have some routines from trunk libuci in my own code, that not surprisingly break when I try and build from trunk May 02 14:25:30 karlp: you can include $(INCLUDE_DIR)/version.mk in your package makefile May 02 14:25:42 karlp: and add your own definition to the cflags based on version checks May 02 14:25:45 in trunk May 02 14:26:04 ah, backfire has the file as well May 02 14:26:12 so you should be able to check the version based on that May 02 14:26:33 hmm, got an example of that or a package that might do this? May 02 14:28:15 those two version.mk files look totally different May 02 14:28:28 no, but you could do ifneq ($(findstring 10.03,$(VERSION_CODE)),) TARGET_CFLAGS += -DIS_BACKFIRE=1 endif May 02 14:28:44 it may only work in a recent version of the backfire branch May 02 14:28:51 if that's enough, then this should work May 02 14:28:59 when using backfire, you should use something recent anyway May 02 14:29:09 we regularly update our backfire branch, May 02 14:29:32 I just want to give trunk a try for this memory issue, and catch up on how things generally work on trunk. May 02 14:31:24 this does not look like a tar archive? May 02 14:32:17 define 'this' May 02 14:32:53 http://pastebin.com/wynjk8C6 May 02 14:33:17 I guess something in my Build/Prepare doesn't like it when trying to build from copying source instead of checking out of svn May 02 14:33:48 no idea where that tarball came from or how it was generated May 02 14:52:23 is there a known problem with libjson-c library in today's trunk? http://pastebin.com/y6YpEK1p May 02 14:52:47 simplethings like, "let's try the behaviour with trunk" always seem to take longer than thangs that are expected to take a long time :| May 02 15:32:23 07:53:25 < karlp> ok, has anyone got any other opinions on setting overcommit_memory to 2? I'd rather not change entire system behaviour just to make sure that any badly written apps of mine crash first May 02 15:32:27 monit May 02 15:40:33 florian * r31550 /trunk/target/linux/brcm63xx/patches-3.3/ (35 files): [brcm63xx] use a define for the poll and debounce interval May 02 15:40:34 florian * r31551 /trunk/package/openssl/ (Makefile patches/210-no-OPENSSL_ia32cap_P.patch): [package] openssl: update to 1.0.1b May 02 15:53:25 Weedy_lappy: I'm already using monit. May 02 15:53:38 monit hasn't been helping. May 02 15:53:51 if my app just mallocs out of control, eventually the system crashes. May 02 15:53:57 karlp: y u no if totalmem gt 60% for 6 cycles then restart May 02 15:54:43 if monit isn;t killing apps before they kill the host you have it setup wrong May 02 15:56:48 hmm, I hadn't tried that, I'd been using monit just to make sure the process was running at all. May 02 15:57:02 I'll look at the rest of monit's options, that might be even simpler. May 02 17:20:06 Anyone else having problems accessing https://dev.openwrt.org/timeline ??? I'm getting Gateway timeout for quite some time... May 02 17:42:23 [florian]: Hey. Do I get it right that we don't have adsl support fot the bcm63xx chips? May 02 17:42:41 <[florian]> xl0: you are right, we don't May 02 17:44:43 :/ May 02 17:45:06 Thx. May 02 17:45:51 <[florian]> yw May 02 18:27:23 juhosg * r31552 /trunk/target/linux/generic/config-3.3: linux/3.3: add more missing symbols May 02 18:27:24 juhosg * r31553 /trunk/target/linux/mpc83xx/ (12 files in 2 dirs): mpc83xx: add support for 3.3 May 02 18:27:27 juhosg * r31554 /trunk/target/linux/mpc83xx/Makefile: mpc83xx: switch to 3.3 May 02 18:27:28 juhosg * r31555 /trunk/target/linux/mpc83xx/ (config-2.6.36 patches-2.6.36/): mpc83xx: remove 2.6.36 support May 02 19:17:59 juhosg * r31556 /trunk/target/linux/ramips/image/Makefile: (log message trimmed) May 02 19:17:59 ramips: Make serial console for DIR-300-B1 work again May 02 19:17:59 Make serial console for DIR-300-B1 work again. May 02 19:17:59 With some recent change, "console=ttyS1,57600" was removed from the May 02 19:17:59 default kernel command line, this adds the necessary mkcmdline May 02 19:18:00 incantation to image/Makefile to add it there for this target. May 02 19:18:00 This is quite likely broken for other targets as well. May 02 19:18:01 juhosg * r31557 /trunk/target/linux/ramips/ (4 files in 4 dirs): (log message trimmed) May 02 19:18:02 ramips: W306R V2.0 - GPIO and Network fixes May 02 19:18:02 Fixed: GPIO typos and confirmed GPIO_BUTTON_RESET May 02 19:18:03 Fixed: Lan & Wan reversed: swaped "eth0.2" with "eth0.1" by May 02 19:18:03 removing a line(default is correct), and reversed the May 02 19:18:04 Lan/wan layout LLLLW to WLLLL. May 02 19:18:04 Added: image/Makefile now builds -factory.bin files. I am May 02 19:28:08 juhosg * r31558 /packages/net/ (8 files in 6 dirs): May 02 19:28:08 Add vpnc-scripts package with up-to-date script May 02 19:28:08 This removes the hard-coded masquerading configuration from vpnc-script. May 02 19:28:08 It's possible to do it in a hook now, but it also shouldn't be necessary, May 02 19:28:08 because we should be invoking the hotplug scripts to plumb the interface May 02 19:28:08 properly anyway. May 02 19:28:09 Signed-off-by: David Woodhouse May 02 19:28:10 juhosg * r31559 /packages/net/openconnect/ (Makefile files/ files/openconnect.sh files/run-openconnect): May 02 19:28:10 Add openconnect protocol support to network scripts May 02 19:28:10 This is very basic — username/password only, and doesn't even check May 02 19:28:11 certificates which it *definitely* should. But it's a good start. May 02 19:28:11 Signed-off-by: David Woodhouse May 02 19:28:12 juhosg * r31560 /packages/net/openl2tp/Makefile: May 02 19:28:12 Remove local copy of pppol2tp PPP module from openl2tp May 02 23:40:08 build #5 of adm5120 is complete: Success [build successful] Build details are at http://buildbot.openwrt.org:8010/builders/adm5120/builds/5 May 03 02:07:18 hi there **** ENDING LOGGING AT Thu May 03 03:00:01 2012