**** BEGIN LOGGING AT Thu Nov 20 02:59:56 2008 Nov 20 05:36:52 does anyone know in which thread ContentProvider methods are called? Nov 20 05:36:52 is it in a binder thread or all in the same thread? Nov 20 05:38:15 in binder threads. Nov 20 05:38:24 (could happen concurrently with thw main thread, and concurrently with one another) Nov 20 05:38:41 ahh, the docs mention this in app model Nov 20 05:39:00 if it happened in the main thread, you'd have risks of inter-app deadlocks. Nov 20 05:49:43 i'm trying a trick in my app to have the content provider close down the open database, replace the file with something downloaded from the net, and then open everything back up again Nov 20 05:50:01 abusing the fact that sqlite3 databases are portable Nov 20 05:50:44 my database is about 6 or 7M in size, and constructing it piece by piece is excruciatingly slow for the phone. it seems better to just download it prebuilt from the server. Nov 20 05:51:49 then subsequent synchronization is based on replaying a changelog, not syncing all the data all over again Nov 20 05:52:26 sounds reasonable Nov 20 05:52:35 after all if it's a database you can just ask for the records since the last sync Nov 20 05:53:09 thats exactly what it does. and that same protocol can be used for first time "slow refreshes", but with this much data on a mobile phone is *awful* :) Nov 20 05:54:35 the protocol introduces quite a bit of its own expressive overhead, and the database construction is very heavy. Nov 20 05:55:30 I've been pondering similar for ondevice storage of maps Nov 20 05:57:04 maemo mapper 1.x uses the filesystem Nov 20 05:57:37 maps/long/lat basically Nov 20 05:57:45 2.x uses sqllite, but seems WAY slower Nov 20 05:57:46 although that might be for unrelated reasons Nov 20 06:26:10 anyone can help me with audio recording? Nov 20 07:13:00 what the heck is this android_metadata table that android puts into my databases? Nov 20 07:13:00 just tells the locale of the app? why would that need to go in the database? Nov 20 08:59:51 well this is interesting. if you close a database with active cursors dangling out there some really odd things happen Nov 20 09:00:28 the cursors think they are open (isClosed is false), but anytime you requery them they say "Invalid statement in fillWindow()" Nov 20 09:15:51 hmm, this has to be some type of bug in android? Nov 20 09:16:36 i wonder if i could work around it by tracking all the cursors i hand out in my provider and then going through and closing all of them when i need to close the database Nov 20 09:19:40 you don't close your cursors? :) Nov 20 09:21:06 romainguy: what i'm trying to do is implement an "optimization" in Five whereby first-time sync happens by simply downloading a pre-built sqlite3 database from the server Nov 20 09:21:40 which is working great by just closing the open database in the content provider, moving the new one in place, then loading it all back up Nov 20 09:21:58 however, any activities which had a cursor into the old database never get closed, but also don't work anymore. calling requery on them says "Invalid statement in fillWindow", though they report that they are still opened and working (isClosed() is false) Nov 20 09:22:01 i expect them not to work, of course, but i also expect them to report that they are closed or invalid. Nov 20 09:22:24 so since they dont show that they are invalid, and can't requery, none of the activities can ever refresh themselves until destroyed and remade Nov 20 09:22:53 can't you just broadcast an intent to ask your activities to refresh their cursors? Nov 20 09:24:23 yes, i thought about that. but then i'd have to handle this quirky behaviour in every place that hangs onto a cursor... Nov 20 09:24:23 which is quite a few Nov 20 09:26:04 although i guess that won't help me, since requery will never work right Nov 20 09:26:29 so there would still need to be special code to remake the cursors Nov 20 09:26:29 hmm Nov 20 09:27:02 i wish there were some way to get inside the cursor and just refresh whatever needs refreshing to make requery work again. Nov 20 09:28:20 you should ask jham on #android Nov 20 09:28:40 i might be able to wrap Cursor and add some extra info to make it possible to build a brand new cursor when i close the old database Nov 20 09:29:46 he's not in there :) Nov 20 09:30:42 i think the problem is that cursor holds a compiled statement which is valid only in the context of an open database handle. so you can't go pulling that handle out from under it and expecting requery to work Nov 20 09:31:06 and there's some other logic elsewhere which prevents the cursor from being considered closed Nov 20 09:32:48 anyway, this problem can wait until tomorrow. night folks :) Nov 20 09:35:15 jasta: why don't you simply open the new db, copy the data with sql statements, close the new db and send a data change notification Nov 20 09:37:14 the whole reason i'm doing it this way in the first place is because there's a LOT of data. invoking rename() on the filesystem is going to be a hell of a lot faster than copying and reintroducing the data this way. Nov 20 09:37:36 the size of the database im testing with is 6MB. Nov 20 09:38:16 yeah I know it would be faster Nov 20 09:38:19 but it doesn't work :) Nov 20 09:38:28 at least with open cursors Nov 20 09:38:44 yes but i think at least tracking the cursors i hand out would be a better solution than copying all that data Nov 20 09:39:10 i believe it would work to just add the uncompiled statement to a wrapped cursor and implementing requery in a special way when i close the underlying database Nov 20 09:41:52 ill try it tomorrow :) Nov 20 09:42:33 although, hmm, in practice since this should only happen when the database is either corrupt or new, i guess i should be considering even a small overhead introduced for the common case worse. Nov 20 12:25:17 http://digitalspaghetti.me.uk/way-to-rip-off-early-adopters-t-mobile-uk :( Nov 20 13:50:24 digitalspaghetti, yeah.. that's the price you pay when you buy stuff on the first/2nd month after launch :( Nov 20 14:32:43 tauno_: yea, but you would expect that after 3, maybe 6 months Nov 20 14:32:43 not less than a month after the phone Nov 20 14:34:10 i've never seen anymore post a link to their own blog somewhere :) that's a new one for me atleast Nov 20 14:34:46 not to mention this is the dev channel :) Nov 20 14:37:43 I'll post one in the general channel then ;) Nov 20 14:45:44 morning Nov 20 18:40:25 good morning Nov 20 18:47:10 good evening Nov 20 18:47:14 ;) Nov 20 18:49:04 :) Nov 20 20:56:46 romainguy: i'm considering your approach from yesterday of just merging in the data. but do you know of the proper way to disable indexing and such to do this bulk insert? Nov 20 20:57:02 will a transaction in sqlite3 accomplish the same effect? Nov 20 20:57:22 i dont know that much about efficient sqlite3 utilization Nov 20 20:57:24 no idea Nov 20 20:57:32 databases are not my thing :) Nov 20 20:59:33 also i believe this will block access to the database while this is going on, so my app is going to need to dance around that somehow... Nov 20 21:00:56 cursors are snapshot cursors Nov 20 21:01:02 so at least you can still use them Nov 20 21:02:08 i wonder if i can just layer onto asyncquery some logic that retries failed queries for a short while Nov 20 21:02:16 that would be the most general solution i think Nov 20 21:03:04 it sure would be nice to have remote exceptions tho :) Nov 20 21:04:36 im gonna revise my sync process quite a bit as well to store everything locally first then bulk update it in the provider Nov 20 21:04:38 sqlite's locking is fairly primitive IIRC. Lots of stuff freezes out other clients. Nov 20 21:04:48 yes i know that it is Nov 20 21:04:52 What's the underlying problem? Nov 20 21:05:44 just trying to come up with a strategy for handling a long running operation which will prevent access to the db Nov 20 21:06:18 Long running database operation? How much data are we talking about here? Nov 20 21:07:02 Write your own pseudo-journal and feed from that a record or two at a time? Or do you need atomicity for the whole thing? Nov 20 21:11:09 i need atomicity, and we are talking a lot of data. Nov 20 21:11:42 the database for my server is 6mb, which is certainly not what i expect the upper bound to be Nov 20 21:11:54 Then honestly, you're kinda SOL. Sqlite wasn't designed for checkpointing, which is what you really want. Nov 20 21:12:08 and all the data would need to be copied in Nov 20 21:12:20 Sorry, s/checkpointing/snapshotting/. Nov 20 21:12:50 what i will do is just allow for failure and wait for the process to complete in the ui Nov 20 21:12:55 Can you block writes only? If so, you could make a copy of the file for readres. Nov 20 21:13:17 so there will be a window while this updates that the user is locked out of the app Nov 20 21:13:40 i think my strategy will work fine honestly Nov 20 21:14:01 its an uncommon case for the app Nov 20 21:27:34 An uncommon app for the uncommon man Nov 21 02:54:10 oh great. somehow i introduced a bug in Five which causes HttpClient#execute to hang in ThreadSafeClientConnManager#getConnection **** ENDING LOGGING AT Fri Nov 21 02:59:57 2008