**** BEGIN LOGGING AT Sun Sep 30 03:00:01 2018 Sep 30 05:33:49 folks. a coredump can be done with the process keeping running, right? Sep 30 05:41:45 yes, with gdb Sep 30 05:42:36 (and probably other tools) Sep 30 05:50:26 yeah, but obviously not with standard signal means Sep 30 05:50:42 see man 2 signal and man 5 core Sep 30 05:51:10 both say coredump implies process termination Sep 30 05:53:08 well, to avoid falling for XY problem: infobot stops for a >2h each night at 00:20 UTC, and neither Tim (owner) nor I have any idea what's causing this. How would you try to tackle this shit? Sep 30 05:53:46 it's not sigcore, but just a type of signals Sep 30 05:53:52 ie. sigquit is core too Sep 30 05:54:05 I know Sep 30 05:54:31 as for the bot, check cron? Sep 30 05:54:44 or scripts the bot is using, might be some log managing Sep 30 05:54:53 or db rehash Sep 30 05:55:11 I suggested this to Tim a dozen times already. Even suspected logrotate the culprit. No reply by Tim Sep 30 05:55:24 do you have access to the box? Sep 30 05:55:30 I need a commandline I demand him doing Sep 30 05:55:38 no Sep 30 05:55:51 some monitoring then? Sep 30 05:56:20 now we're talking. What exactly= Sep 30 05:56:23 ? Sep 30 05:56:29 disk/network usage Sep 30 05:56:36 meh Sep 30 05:56:43 ie. bot uploading something big Sep 30 05:56:45 nothing too unusual Sep 30 05:57:15 if it's a bot stoppping responding, it might be some script running and blocking everything else Sep 30 05:57:28 which would mean turning that script to run in parallel might help Sep 30 05:57:32 to start with, I have no clear indication *why* the bot goes offline Sep 30 05:59:45 I can remote-reboot the bot and already talked Tim into this reboot providing some diagnostic output. Alas I didn't find the one smart command to show what I need to know, about system status Sep 30 06:01:21 it might be as simple as some script blocking the bot Sep 30 06:01:42 which obviously wouldnt show in most diagnostics Sep 30 06:01:46 ps doesn't show anything suspicious Sep 30 06:02:02 easiest would be adding some debug log showing what funcs are starting and when Sep 30 06:02:36 and popping that single print in most funcs Sep 30 06:11:03 would seem like the bot goes OOM Sep 30 06:12:55 which would sound as some script trying to run big data Sep 30 06:14:17 well yeah. I could speculate all day long. I have no clue what to talk Tim into, to *fix* the issue Sep 30 06:15:07 ~seen rimriker Sep 30 06:15:09 DocScrutinizer05: i haven't seen 'rimriker' Sep 30 06:15:13 ~seen timriker Sep 30 06:15:14 timriker <~TimRiker@bzflag/projectlead/TimRiker> was last seen on IRC in channel #infobot, 25d 10h 35m 30s ago, saying: 'added a netstat -ntp|grep perl'. Sep 30 06:15:34 clearly he has no time to fix it, and no one else has the access, which basically boils down to nothing is going to be done Sep 30 06:16:06 that's a pretty useless revelation Sep 30 06:16:22 but clearly directs to the deeper issue Sep 30 06:16:32 so??? Sep 30 06:16:38 you can speculate all day long, without results Sep 30 06:16:44 so??? Sep 30 06:17:42 do you want to tell me you're short of ideas how to tackle this? Noted Sep 30 06:18:06 I'm too, right now Sep 30 06:18:09 i've already shared few ideas, but without access they are unusable Sep 30 06:19:30 I have 'access', just not the type of access you might think of when you hear the term Sep 30 06:20:47 >>saying: 'added a netstat -ntp|grep perl'<< is what results from my type of "access", I demanded for a netstat to rule out uplink clogging Sep 30 06:21:21 nah, it more sounds as some homecleaning script running in the bot Sep 30 06:21:35 and I can trigger this and all the other diagnostic rools by doing a bot reboot Sep 30 06:21:47 which takes a while, doesnt hit big cpu/network, but still blocks further execution Sep 30 06:21:51 tools* Sep 30 06:27:14 please check this https://pastebin.com/k2eACYn6 it has some errors that might well be worth a closer look and inverstigation. alas the bot is perl and thus I'm totally lost. /join #infobot for sourcecode etc Sep 30 06:38:31 I gonna provide a reboot log based on operational state now Sep 30 06:38:40 ~+status Sep 30 06:38:41 Since Sun Sep 30 02:25:36 2018, there have been 0 modifications, 0 questions, 0 dunnos, 0 morons and 0 commands. I have been awake for 4h 13m 4s this session, and currently reference 119769 factoids. I'm using about 80728 kB of memory. With 0 active forks. Process time user/system 12.89/0.57 child 0/0 Sep 30 06:41:27 ~wb Sep 30 06:41:33 oops Sep 30 06:41:36 https://pastebin.com/cssbAu0W Sep 30 06:43:09 gosh, does that PoS critter hang on reboot now? Sep 30 06:46:37 chat.freenode.net should be irc.freenode.com ? Sep 30 06:48:00 check shell ulimits Sep 30 06:48:19 ulimit -a Sep 30 06:50:52 well, when I want to tell Tim "please add a diagnostics line to check ulinit -a", I need a rationale WHY I think this is a relevant info, what we expect it to be, and what we hope to fix by messing with it Sep 30 06:51:08 Out of memory! Sep 30 06:51:12 this maybe Sep 30 06:51:48 would be interesting to find that string exact in scripts Sep 30 06:51:49 you seen the timestamps of those loglines in ibot's log? Sep 30 06:52:14 could it could pinpoint the place where it does that big job Sep 30 06:52:42 (although it might be generic perl err msg) Sep 30 06:53:02 timestamps or not, it probably has more than one problem Sep 30 06:53:10 it is Sep 30 06:55:28 fsckng bot hangs on reboot!!! >:-( Sep 30 06:55:31 perlmonks link to debug things: https://www.perlmonks.org/?node_id=598685 Sep 30 06:55:46 also suggests checking ulimit Sep 30 06:56:28 which actually migh be a GOOD thing, catching it while it does whatever it does when it takes 2h+ to re-login Sep 30 06:56:59 ooh, no, that been me now, should stop messing with bot when tired Sep 30 06:57:02 maybe it has some task to do which it tries to do for months but dies by killing Sep 30 06:57:20 [2018-09-30 08:44:58] --> infobot (ibot@208.53.50.136) has joined this channel. Sep 30 06:57:21 [2018-09-30 08:44:58] *** ChanServ sets mode: +v infobot Sep 30 06:58:13 ~wb Sep 30 06:58:14 It's great to be back! Sep 30 06:58:21 ~+status Sep 30 06:58:22 Since Sun Sep 30 06:56:10 2018, there have been 0 modifications, 1 question, 0 dunnos, 0 morons and 0 commands. I have been awake for 2m 11s this session, and currently reference 119769 factoids. I'm using about 80264 kB of memory. With 0 active forks. Process time user/system 1.29/0.12 child 0/0 Sep 30 06:58:34 ~+uptime Sep 30 06:58:36 - Uptime for apt - Sep 30 06:58:36 Now: 2m 24s running infobot 1.5.4 (SVN) -- linux Sep 30 06:58:37 1: 59d 8h 41m 19s running infobot 1.5.4 (SVN) -- linux, ended Sun Nov 14 18:39:57 2010 Sep 30 06:58:37 2: 57d 3h 9m 23s running infobot 1.5.4 (SVN) -- linux, ended Fri Jun 26 20:39:27 2009 Sep 30 06:58:38 3: 53d 8h 24m 4s running infobot 1.5.4 (SVN) -- linux, ended Tue Nov 7 05:34:20 2017 Sep 30 07:00:23 random crap to debug memory in perl: http://perldoc.perl.org/perldebguts.html#Debugging-Perl-memory-usage Sep 30 07:01:03 might actually be useful if you get a coredump and want to step through it Sep 30 07:02:36 http://perldoc.perl.org/perldebguts.html#Debugging-Perl-Memory-Usage (proper link) Sep 30 07:02:49 http://laforge.gnumonks.org/blog/20180929-fernvale-discount/ Sep 30 07:03:49 2g is being killed of around the world Sep 30 07:03:56 might be the reason of lack of interest Sep 30 07:04:02 *off Sep 30 07:23:08 yep, sure Sep 30 07:25:44 yet there are amazing things going on, like Michaela Spacefalconia trying to re-release a (Openmoko Freerunner) TI Calypso chipset based 2G modem module Sep 30 07:27:00 I for one think ^^^ this stuff is more rewarding than Calypso Sep 30 19:31:58 stupid question - but i must ask nonethelese - the telco in my country uses overhead lines, and our area is also prone to lightning strikes in summer. what is the best way (besides disconnecting, of course) to protect the adsl modem/router from this? Sep 30 19:33:29 nothing protects you from direct (or even in neighbourhood) lightning hit Sep 30 19:33:45 my friend had such incident few years ago Sep 30 19:33:58 lightning hit the house 2-3 buildings away Sep 30 19:34:19 his modem way flying across the room leaving sparks and burns trail Sep 30 19:35:03 s/way/was Sep 30 19:37:46 gosh Sep 30 19:38:02 at least no one's reported such here Sep 30 19:38:59 i have heard few stories about ethernet networks hung around the neighbourhoods too, those tend to burn routers/switches first Sep 30 19:39:18 so just dont worry and keep your data safe Sep 30 19:39:44 ie. store backups in different locations Sep 30 19:44:36 well our modems dead now. no idea when isp will provide replacement. Sep 30 19:45:21 there are some telephone line isolators, but as i've said, nothing protects from close hit Sep 30 19:45:47 some power strips provide filtering for modem lines Sep 30 19:46:12 isp doesn't like those. :) Sep 30 19:46:36 they just have to pass high freqs, while still protecting from voltage spikes Sep 30 19:46:42 they complained that it causes the signal to be 'sent back' to their more expensive equipment :D Sep 30 19:46:48 rrrright Sep 30 19:47:07 i never got time to study how they really work to see if their claim was correct (i doubt) Sep 30 19:50:06 that might be true for some cheapo ones Sep 30 20:04:07 guess we must be sure to disconnect then, assuming storm happens when we're around **** ENDING LOGGING AT Mon Oct 01 03:00:02 2018