**** BEGIN LOGGING AT Fri Sep 11 03:00:00 2009
Sep 11 06:51:29 <Danicasti> Buongiorno!
Sep 11 08:27:40 <Decodecoding> There's a RC2 for 2.3.0! Where can I see a changelog?
Sep 11 09:00:36 <iurgi> ~.
Sep 11 09:03:36 <dasjoe> Your session didn't crash
Sep 11 09:11:31 <iurgi> hhahah
Sep 11 09:11:33 <iurgi> thanks ;)
Sep 11 09:11:36 <iurgi> it actually did
Sep 11 09:11:44 <iurgi> but it got restored before the link was cut
Sep 11 09:11:45 <iurgi> ;)
Sep 11 11:57:57 <epienbro> blogic, you there?
Sep 11 11:58:15 <blogic> yes
Sep 11 11:58:20 <epienbro> ah great
Sep 11 11:58:29 <blogic> just arrived
Sep 11 11:58:37 <blogic> will dissappear for 5 minutes in a sec
Sep 11 11:58:42 <blogic> need to get juice :)
Sep 11 11:58:43 <blogic> brb
Sep 11 11:59:06 <epienbro> did you already have some time to check the uCLibc/sscanf bug I reported?
Sep 11 11:59:21 <blogic> no, i can do it now
Sep 11 11:59:26 <blogic> gimme a moment to setup
Sep 11 11:59:37 <blogic> i worked till 4am and only just arrived at my work desk :)
Sep 11 12:01:29 <epienbro> take your time
Sep 11 12:02:58 <blogic> what does %127c do ?
Sep 11 12:03:03 <blogic> never seen that syntax
Sep 11 12:03:44 <epienbro> read at most 127 characters and put it in a char*
Sep 11 12:03:53 <blogic> ah ok
Sep 11 12:03:59 <blogic> never seen that
Sep 11 12:04:09 <epienbro> I can't use %s there as that might introduce buffer overflows
Sep 11 12:04:19 <blogic> yeah
Sep 11 12:04:39 <blogic> ok
Sep 11 12:04:45 <blogic> let me make a test package to work with
Sep 11 12:08:53 <epienbro> or do you know of another format string which I can use to archieve the same effect (without introducing possible buffer overflows) ?
Sep 11 12:09:25 <blogic> let me think about it for a sec
Sep 11 12:09:34 <blogic> hmm
Sep 11 12:09:41 <blogic> you can strstr for a " "
Sep 11 12:09:55 <blogic> if it is below 127 use %s else use strncpy
Sep 11 12:10:07 <blogic> there is a define to find out if you run uclibc
Sep 11 12:10:15 <blogic> so you can make the workaround libc depended
Sep 11 12:10:53 <blogic> let me check the uclibc docs if %127c is supported
Sep 11 12:11:44 <epienbro> I couldn't find anything about it in the 'differences between uClibc and Glibc' document
Sep 11 12:11:51 <blogic> #ifdef __UCLIBC__
Sep 11 12:12:02 <blogic> you can use that to make the workaround
Sep 11 12:12:09 <blogic> i can reproduce the bug
Sep 11 12:12:10 <blogic> ok
Sep 11 12:12:35 <blogic> i will dig into the uclibc source code later to see if it is a missing feature or a bug, so we can report it upstream
Sep 11 12:12:49 <epienbro> great :)
Sep 11 12:12:58 <blogic> soooo
Sep 11 12:13:07 <blogic> we need to think about the gui
Sep 11 12:13:15 <blogic> you said you are makeing a plugin thing
Sep 11 12:13:29 <blogic> ideally we have a lua c binding to get status data into luci
Sep 11 12:13:30 <epienbro> with 'gui' you mean the webinterface right ?
Sep 11 12:13:34 <blogic> yes
Sep 11 12:13:39 <blogic> webui
Sep 11 12:14:02 <blogic> and, how do you store the config atm ?!
Sep 11 12:14:09 <blogic> i mean, do you have config files ?
Sep 11 12:14:14 <blogic> let me download src
Sep 11 12:14:18 <blogic> is there a svn trunk ?!
Sep 11 12:14:22 <blogic> or git ?
Sep 11 12:14:23 <epienbro> the config is stored in a config file in ~/.config/NNTPGrab
Sep 11 12:14:34 <epienbro> http://svn.openftd.org/svn/nntpgrab
Sep 11 12:14:35 <blogic> ok
Sep 11 12:14:40 <epienbro> the plugin stuff is in the trunk
Sep 11 12:14:54 <blogic> ok
Sep 11 12:15:13 <epienbro> but the entire configuration stuff is bundled as a seperate plugin, so we can rewrite that if necessary
Sep 11 12:15:22 <blogic> ok
Sep 11 12:15:29 <blogic> well
Sep 11 12:15:34 <blogic> you should look at uci
Sep 11 12:15:38 <blogic> there is a libuci.so
Sep 11 12:15:44 <blogic> very easy and clean to do
Sep 11 12:15:51 <epienbro> do you have a link to the uci documentation ?
Sep 11 12:15:56 <blogic> yeah
Sep 11 12:16:02 <blogic> 1 sec
Sep 11 12:16:11 <blogic> ah
Sep 11 12:16:19 <blogic> there is only docs for the lua binding
Sep 11 12:16:21 <blogic> grml
Sep 11 12:16:50 <blogic> ok, so you have a nntpgrab daemon running, it does the work right ? and then a gui to control it over ip i asusme ?!
Sep 11 12:17:02 <epienbro> yes that's right
Sep 11 12:17:15 <epienbro> except for that uclibc bug, but we can workaround that for now if necessary
Sep 11 12:17:48 <blogic> ok
Sep 11 12:17:59 <blogic> how does the client daemon communication work ?
Sep 11 12:18:20 <epienbro> http://nntpgrab.nl/wiki/index.php/Protocol
Sep 11 12:19:04 <blogic> so you can use it using telnet nice
Sep 11 12:19:08 <blogic> easily done then
Sep 11 12:19:29 <blogic> there are 2 ways to do this
Sep 11 12:19:40 <blogic> 1) use the link you pasted
Sep 11 12:20:03 <blogic> 2) use the link you pasted only for commands and teach nntpgrab to dump its state into uci-state
Sep 11 12:20:14 <blogic> luci can handle uci-state files nativley
Sep 11 12:20:33 <epienbro> how does the webui communicate with uci ?
Sep 11 12:20:40 <blogic> and the c code for duming the state in nntpgrab is 100 lines if we use ucix and libuci
Sep 11 12:21:06 <blogic> local uci = require("luci.model.uci").cursor_state()
Sep 11 12:21:30 <blogic> ip = uci:get("network", "wan", "ipaddr")
Sep 11 12:21:33 <blogic> like that :)
Sep 11 12:21:50 <blogic> et me find an example
Sep 11 12:21:57 <blogic> do you have a fon20 ?
Sep 11 12:22:01 <epienbro> yes
Sep 11 12:22:08 <blogic> do you have ssh ?
Sep 11 12:22:09 <blogic> type
Sep 11 12:22:12 <epienbro> of course
Sep 11 12:22:12 <blogic> uci show
Sep 11 12:22:26 <blogic> and ls /etc/config/
Sep 11 12:22:32 <epienbro> one sec, I need to switch the fon back on first
Sep 11 12:22:44 <blogic> the conecpt is package.section.option
Sep 11 12:22:46 <blogic> so
Sep 11 12:22:58 <blogic> uci get network.wan.ipaddr
Sep 11 12:23:19 <blogic> will tell you option ipaddr in section wan in the file /etc/config/network
Sep 11 12:23:40 <blogic> uci set network.wan.ipaddr=1.1.1.1 works aswell
Sep 11 12:23:56 <blogic> until you do uci commit, the changes are stored in /tmp/.uci/
Sep 11 12:24:03 <blogic> also, there is a uci-state
Sep 11 12:24:08 <blogic> we store runtime data there
Sep 11 12:24:23 <blogic> it is stored inside /var/state/
Sep 11 12:24:58 <blogic> options can be lists
Sep 11 12:25:05 <blogic> sections are typed or named
Sep 11 12:25:13 <blogic> uci is wonderfull :)
Sep 11 12:25:17 <blogic> its like a mini DB
Sep 11 12:25:31 <blogic> i think we can easily make it work
Sep 11 12:25:38 <blogic> what items will the webui have ?
Sep 11 12:25:50 <blogic> a list of download, a way to add a new dl
Sep 11 12:25:54 <blogic> what else is tehr e?
Sep 11 12:26:17 <epienbro> but if I understand correctly, uci only reads and writes to files? how do the applications receive a notification whenever a file changes?
Sep 11 12:26:58 <blogic> we use fonstated for this
Sep 11 12:27:04 <blogic> it is a event queuing daemon
Sep 11 12:27:10 <blogic> any app can send an event
Sep 11 12:27:18 <blogic> and we have lua/bash/c hanlders
Sep 11 12:27:36 <blogic> in a webui form level, we have validation (also uci based) and event levels
Sep 11 12:27:56 <blogic> so if you have a form in the webui you can define what events are fired depending on which fileds are edited
Sep 11 12:28:06 <blogic> it is a full feature model view control
Sep 11 12:28:11 <blogic> framework
Sep 11 12:28:28 <blogic> anyhow, makeing a webi will be trivial
Sep 11 12:28:46 <blogic> we have a lua binding that talks to transmission using the http rpc socket
Sep 11 12:28:55 <blogic> its a similar setup
Sep 11 12:29:15 <epienbro> sounds very interesting :)
Sep 11 12:29:49 <blogic> as for the size and dependencies
Sep 11 12:29:58 <blogic> transmission has many deps and eats ram
Sep 11 12:30:02 <epienbro> do you have documentation or example code for the fonstated c handler ?
Sep 11 12:30:21 <blogic> it is not a direct bindign you can make
Sep 11 12:30:34 <blogic> its a script that fires that must trigger your code
Sep 11 12:30:44 <blogic> i mean, what changes do you want to catch ?
Sep 11 12:30:53 <blogic> like a new file being queued ?
Sep 11 12:31:52 <blogic> i need to test samba in a sec and get 2.3.0.0 ready for release, i will prepare a small example over the next few days, that you can use as a reference
Sep 11 12:31:55 <blogic> ok ?
Sep 11 12:32:14 <epienbro> let's say the user wants to add a file to download queue using the webui..what is the communication flow to archieve that ?
Sep 11 12:32:49 <blogic> ok
Sep 11 12:33:13 <blogic> webui uses a lua binding to talk to nntpgrab using the network socket
Sep 11 12:33:38 <blogic> then, wehnever the page is opened, we load the current state over the network socket and display it
Sep 11 12:33:46 <blogic> that is how we do it for transmission
Sep 11 12:34:08 <epienbro> ah okay
Sep 11 12:34:23 <epienbro> I'll experiment with it and see what I can come up with :)
Sep 11 12:36:40 <epienbro> thanx for the information so far :)
Sep 11 12:37:33 <blogic> sure
Sep 11 12:37:42 <blogic> i will compile nntgrab and try to make it run
Sep 11 12:38:10 <epienbro> I'll publish the Makefile's later today
Sep 11 12:38:50 <epienbro> on second thought, I might also create a subversion repository feed for it for the time being
Sep 11 14:43:01 <epienbro> blogic, I documented the current progress of the port at http://nntpgrab.nl/wiki/index.php/Fonosfera
Sep 11 15:00:32 <blogic> epienbro: oki
Sep 11 15:00:35 <blogic> i will try it later on
Sep 11 15:12:44 <blogic> i think i have dlna running
Sep 11 15:19:19 <blogic> indeed
Sep 11 15:19:21 <blogic> i can see the server
Sep 11 15:19:25 <blogic> and mount it :)
Sep 11 15:19:30 <blogic> but i cannot see the content
Sep 11 15:19:31 <blogic> ?!
Sep 11 15:32:39 <epienbro> hmm..when compiling the luci package I get a strange segmentation fault while executing the LUA compiler (luac) which I didn't have 2 days ago
Sep 11 15:32:55 <blogic> ?!
Sep 11 15:32:59 <blogic> what dostro ?1
Sep 11 15:33:02 <epienbro> for i in $(find dist -name *.lua -not -name debug.lua); do luac -s -o $i $i; done
Sep 11 15:33:02 <epienbro> /bin/sh: line 1: 16559 Segmentation fault      (core dumped) luac -s -o $i $i
Sep 11 15:33:02 <epienbro> make[5]: *** [luacompile] Error 139
Sep 11 15:33:02 <epienbro> make[5]: Leaving directory `/home/erik/fon-ng/openwrt/build_dir/mips/luci-0.9+svn/libs/core'
Sep 11 15:33:04 <blogic> and what lua version ?
Sep 11 15:33:07 <epienbro> fedora rawhide, x86_64
Sep 11 15:33:11 <blogic> hmm
Sep 11 15:33:32 <epienbro> lua-5.1.4-4.fc12.x86_64
Sep 11 15:33:39 <blogic> make package/luci/clean
Sep 11 15:33:44 <blogic> and try again
Sep 11 15:35:41 <epienbro> nope, doesn't help
Sep 11 15:36:05 <epienbro> the last update of the fedora lua package was back in july so that probably isn't the cause..
Sep 11 15:37:34 <epienbro> probably some rawhide b0rkedness..
Sep 11 15:44:41 <blogic> grml
Sep 11 18:06:06 <epienbro> blogic, I just tried to build everything on a Fedora 11 i586 environment (with a complete clean checkout) and the segmentation fault also occurs there...
Sep 11 18:07:14 <xMff> maybe you managed to trigger a lua bug
Sep 11 18:07:15 <epienbro> were there any special changes published in luci in the last few days?
Sep 11 18:07:35 <xMff> unlikely
Sep 11 18:08:02 <xMff> luac is just the lua bytecode compiler, it's highly unlikely that any lua code can trigger it
Sep 11 18:08:16 <xMff> *can trigger a segfault
Sep 11 18:08:40 <xMff> a random guess, what filesystem you're on?
Sep 11 18:08:44 <epienbro> ext4
Sep 11 18:09:00 <epienbro> but the strange thing is that it worked just fine two days ago on the same computer...
Sep 11 18:09:08 <xMff> hmm
Sep 11 18:09:24 <xMff> can you run gdb on the core dump?
Sep 11 18:09:54 <xMff> and check the backtrace
Sep 11 18:09:55 <epienbro> I need to find the core file first...
Sep 11 18:10:14 <epienbro> it isn't in the same folder as indicated by make
Sep 11 18:11:05 <xMff> what's in /proc/sys/kernel/core_pattern ?
Sep 11 18:11:18 <epienbro> 'core'
Sep 11 18:11:24 <xMff> hm
Sep 11 18:11:35 <epienbro> but in Fedora a new system was introduced recently named 'abrt' which can catch segfaults
Sep 11 18:12:09 <xMff> find /home/erik/fon-ng/openwrt/build_dir/mips/luci-0.9+svn -name 'core*'  :)
Sep 11 18:12:10 <epienbro> though I haven't found any lua chrashes yet..
Sep 11 18:13:13 <xMff> you could try to disable the bytecode compiling to see if it works then
Sep 11 18:13:26 <xMff> it's an option in menuconfig
Sep 11 18:14:27 <xMff> luac puts the final binaries in the same path as the source lua files
Sep 11 18:14:36 <xMff> maybe this does not work well on ext4 for some reason
Sep 11 18:14:47 <xMff> under certain workloads or so
Sep 11 18:15:38 <epienbro> could be..
Sep 11 18:16:35 <xMff> you could do this:   echo /tmp/core > /proc/sys/kernel/core_pattern
Sep 11 18:16:43 <xMff> to ensure that the core dumps end up in /tmp
Sep 11 18:17:06 <epienbro> ah that's a nice one :)
Sep 11 18:20:29 <epienbro> hmz..strangely enough no core file gets created in /tmp...
Sep 11 18:21:41 <xMff> this is strange indeed
Sep 11 18:21:45 <xMff> ulimit stuff is right?
Sep 11 18:21:47 <epienbro> never mind, I forgot to set 'ulimit -c unlimited'
Sep 11 18:21:52 <epienbro> now retrying
Sep 11 18:23:10 <epienbro> ah, I've got the core file now
Sep 11 18:23:19 <epienbro> now installing the lua debug symbols..
Sep 11 18:28:19 <epienbro> Core was generated by `luac -s -o dist/usr/lib/lua/luci/util.lua dist/usr/lib/lua/luci/util.lua'.
Sep 11 18:28:20 <epienbro> Program terminated with signal 11, Segmentation fault.
Sep 11 18:28:20 <epienbro> #0  0x08054042 in newkey (L=<value optimized out>, t=<value optimized out>, key=0xbf8e0584) at ltable.c:420
Sep 11 18:28:20 <epienbro> 420	      gnext(n) = gnext(mp);  /* chain new position */
Sep 11 18:28:20 <epienbro> (gdb) bt full
Sep 11 18:28:21 <epienbro> #0  0x08054042 in newkey (L=<value optimized out>, t=<value optimized out>, key=0xbf8e0584) at ltable.c:420
Sep 11 18:28:26 <epienbro>         othern = 0x0
Sep 11 18:28:28 <epienbro>         n = 0x97b83a8
Sep 11 18:28:30 <epienbro>         mp = 0x97b84f8
Sep 11 18:28:32 <epienbro> #1  0x0805433d in resize (L=<value optimized out>, t=<value optimized out>, nasize=<value optimized out>, nhsize=159014920) at ltable.c:314
Sep 11 18:28:35 <epienbro>         i = <value optimized out>
Sep 11 18:28:41 <epienbro>         oldasize = 134562621
Sep 11 18:28:43 <epienbro>         oldhsize = 159014920
Sep 11 18:28:45 <xMff> use pastebin.com please
Sep 11 18:28:45 <epienbro>         nold = 0xbf8e0f04
Sep 11 18:28:47 <epienbro> #2  0x0805b9b3 in llex (ls=0x4, seminfo=<value optimized out>) at llex.c:347
Sep 11 18:28:49 <epienbro>         sep = <value optimized out>
Sep 11 18:28:51 <epienbro> #3  0x0805ca6c in ?? ()
Sep 11 18:28:55 <epienbro> No symbol table info available.
Sep 11 18:28:57 <epienbro> <<cut>>
Sep 11 18:28:59 <epienbro> #23 0x08050180 in open_func (ls=0xbf8e0f04, fs=0x63) at lparser.c:350
Sep 11 18:29:01 <epienbro>         L = 0xb806f68c
Sep 11 18:29:03 <epienbro>         f = <value optimized out>
Sep 11 18:29:05 <epienbro> Backtrace stopped: previous frame inner to this frame (corrupt stack?)
Sep 11 18:29:55 <epienbro> sorry, will do that next time
Sep 11 18:30:00 <epienbro> but this is all the interesting stuff
Sep 11 18:32:45 <epienbro> anyway, it looks like some memory corruption..
Sep 11 18:32:54 <xMff> indeed
Sep 11 18:33:04 <xMff> is /home/erik/fon-ng/openwrt/build_dir/mips/luci-0.9+svn/libs/core/dist/usr/lib/lua/luci/util.lua still plaintext?
Sep 11 18:33:10 <epienbro> but what's also strange is that the input file is the same as the output file
Sep 11 18:33:19 <xMff> this is intended
Sep 11 18:33:32 <xMff> it's written this way in the makefile
Sep 11 18:34:00 <epienbro> that specific file is still plain text, but other files in that folder are binary
Sep 11 18:34:14 <xMff> can you repeat the command on this file with a different output file?
Sep 11 18:35:56 <epienbro> that works just fine
Sep 11 18:36:20 <xMff> I see
Sep 11 18:36:47 <epienbro> but when input == output it works as well..
Sep 11 18:37:01 <xMff> hmm
Sep 11 18:37:05 <epienbro> it only segfaults when called from the fon build environment
Sep 11 18:37:42 <xMff> this is strange, I mean luac is executed on each file, it's not that it is some continuously running process or so...
Sep 11 18:38:04 <epienbro> perhaps some environment variable?
Sep 11 18:38:30 <xMff> but maybe it's indeed some ext4 behavioural thing, some buffering stuff or so when processing many small files in a row
Sep 11 18:39:11 <xMff> what's the fonosfera svn url?
Sep 11 18:39:35 <epienbro> http://svn.fonosfera.org/fon-ng/trunk
Sep 11 18:40:11 <xMff> ty
Sep 11 18:42:26 <xMff> try applying this patch on the luci folder: http://luci.pastebin.com/m79143964
Sep 11 18:45:08 <epienbro> now rebuilding..
Sep 11 18:48:17 <epienbro> for i in $(find dist -name *.lua -not -name debug.lua); do luac -s -o $i.b $i; mv $i.b $i; done
Sep 11 18:48:17 <epienbro> /bin/sh: line 1: 30099 Segmentation fault      (core dumped) luac -s -o $i.b $i
Sep 11 18:48:17 <epienbro> mv: cannot stat `dist/usr/lib/lua/luci/util.lua.b': No such file or directory
Sep 11 18:48:17 <epienbro> make[5]: *** [luacompile] Error 1
Sep 11 18:48:27 <epienbro> still a segfault..
Sep 11 18:48:58 <xMff> vary strange
Sep 11 18:49:08 <epienbro> it sure is..
Sep 11 18:49:22 <xMff> my only recommendation for now is to disable bytecode compiling then
Sep 11 18:53:14 <xMff> epienbro: can you please post the backtrace in pastebin again? I want to write a bug report to the Lua mailing list
Sep 11 18:53:28 <epienbro> sure, one sec
Sep 11 18:54:20 <epienbro> I also just added a 'echo $$i' to the change you just proposed..and the other files succeed just fine..it's just the util.lua file on which the segfault occurs
Sep 11 18:54:32 <xMff> interesting
Sep 11 18:58:10 <epienbro> http://luci.pastebin.com/m1b17dd14
Sep 11 19:02:17 <xMff> thanks
Sep 11 19:10:45 <epienbro> xMff, I now compiled everything with the Luci 'Build Target' set to 'Stripped' and it looks like everything builds just fine now :)
Sep 11 19:11:03 <xMff> yeah
Sep 11 19:11:08 <xMff> use "full source"
Sep 11 19:11:19 <xMff> this will ease debugging if you intend to do development
Sep 11 19:11:31 <epienbro> oh okay
Sep 11 19:11:59 <xMff> with stripped all comments are removed from the source files, this means that line numbers etc. will be off in bachtraces and error messages
Sep 11 19:12:16 <epienbro> makes sense
Sep 11 19:12:21 <epienbro> will rebuild right away :)
Sep 11 21:09:16 <xMff> epienbro: I'll post a detailled bug report now, got in touch with some ppl in #lua. Will you be around here later on? In case of additional questions
Sep 11 21:11:30 <epienbro> I'll remain here :)
Sep 11 21:11:49 <xMff> good
Sep 11 22:20:55 <epienbro> blogic, I just updated the wiki at http://nntpgrab.nl/wiki/index.php/Fonosfera with more information about building NNTPGrab for the Fonera, the current issues with it and the binaries of the current work-in-progress
**** ENDING LOGGING AT Sat Sep 12 02:59:56 2009