**** BEGIN LOGGING AT Mon Apr 26 02:59:56 2010 Apr 26 15:27:06 hi, does someone know where ofono stores the pin? Apr 26 15:27:20 I deleted all entries in /var/lib/ofono Apr 26 15:27:37 but it still remembers the pins I have entered previously Apr 26 15:31:36 wagi: I don't know but run ofono with strace and look to the open calls to find out that, or read the code. ;) Apr 26 15:32:01 stupid me Apr 26 15:32:17 of course, I should remove the device Apr 26 15:32:31 as long it is powered it doesn't forget the pin Apr 26 15:32:32 :) Apr 26 15:33:10 hehe ;) Apr 26 15:40:16 cu Apr 26 16:45:42 holtmann: Are you actually fine with IntegratedCircuitCardIdentifier as a property name? Apr 26 16:46:10 It fits with what we do for MobileNetworkCode etc. Apr 26 16:46:21 However ICCID might be better actually. Apr 26 16:46:29 Since even the iPhone just shows that value. Apr 26 16:47:03 I think we have to cut down a little bit with the plain text here and just use acronyms. Apr 26 16:48:02 The MNC/MCC are used more commonly Apr 26 16:48:15 for ICCID I agree it can be a simple acronym Apr 26 16:48:53 However, do you want to name it something like SimCardId? Apr 26 16:49:10 No. Either ICCID or the long one. Apr 26 16:49:35 We should not be making something up. Unless it is something simple like "Name". Apr 26 16:49:47 In the end its just the SIM Card identifier Apr 26 16:50:02 And we don't do shit like InternationalMobileSubscriberIdentity Apr 26 16:50:15 Perhaps that one needs to change as well :) Apr 26 16:50:35 Maybe just calling that IMSI would be better. Apr 26 16:50:52 Also MobileCountryCode could just be CountryCode or MCC. Apr 26 16:51:15 Nah Apr 26 16:51:39 CountryCode is too confusing and I personally hate acronyms Apr 26 16:51:53 my vote is for SimIdentity and SubscriberIdentity Apr 26 16:51:56 Also ICCID = Identity actually. The Sim is part in the interface name already. Apr 26 16:52:12 or maybe CardIdentity Apr 26 16:52:27 ModuleIdentity. Apr 26 16:52:35 Can we find more names ;) Apr 26 16:53:05 Btw, ETSI specs refer to UICC/SIM as 'Card' or 'Smart Card' Apr 26 16:53:14 So even Sim Toolkit is referred to as 'Card toolkit' Apr 26 16:53:47 So CardIdentity is actually a good name Apr 26 16:53:55 Okay. I am fine with that. Apr 26 16:54:35 Just make sure the docs mention ICCID since that is what most technical people remember. Apr 26 16:54:49 So is this suppose to be available even on locked SIM cars. Apr 26 16:54:55 s/cars/cards/ Apr 26 17:03:13 Yes Apr 26 17:03:21 the ICCID is the card ID, not the IMSI Apr 26 17:03:35 So you could have two SIM cards with same IMSI but different ICCID Apr 26 17:10:47 balrog-k1n: Btw, we need to call the sim_inserted watches after we initialize the sim & determine the phase Apr 26 17:11:12 balrog-k1n: The current code can cause EFecc to be read before the phase Apr 26 17:30:28 denkenz: I like CardIdentity, too Apr 26 17:37:04 akiniemi: Great, always good to have consensus :) Apr 26 17:45:42 holtmann: How close is the HDLC part to being able to replace PPP HDLC parser? Apr 26 18:06:25 denkenz: can that cause any problems? iirc the specs don't say exactly when EFphase needs to be read Apr 26 18:09:58 balrog-k1n: I remember we cached EFecc incorrectly when the phase wasn't read properly Apr 26 18:10:51 balrog-k1n: And even if it works out in the end, it is cleaner, at least move the sim_initialize before the inserted watch list notification Apr 26 18:11:06 balrog-k1n: So the phase EF is queued first Apr 26 18:11:41 denkenz: then we need to fix caching (too) :) Apr 26 18:12:46 ok i'll try to change the initialization order Apr 26 18:13:08 balrog-k1n: Well, the best solution is to call inserted watches in the ef_phase callback Apr 26 18:13:36 balrog-k1n: That way we keep the old behavior exactly Apr 26 18:15:43 balrog-k1n: In the end none of this probably matters, but you'd have to convince yourself that it works Apr 26 18:16:03 balrog-k1n: If the code flow is made explicit, then you don't have to convince yourself ;) Apr 26 18:16:51 denkenz: well on the other hand it's easier to spot the bugs earlier :) probably things that subscribe to sim inserted but instead should subscribe to sim ready Apr 26 18:18:56 balrog-k1n: Only the voice call is actually going to watch for the inserted state Apr 26 18:19:07 balrog-k1n: For the purpose of EF access anyway Apr 26 18:19:22 yeah Apr 26 18:22:25 balrog-k1n: But in the end we should make it explicit, that code path is way too tricky already Apr 26 18:23:34 denkenz: i have no problem changing it, i just don't see why it's safer or better to read efphase first -- no caching should happen before imsi is read Apr 26 18:23:46 balrog-k1n: Actually it can Apr 26 18:24:15 balrog-k1n: We already had this case on phonesim Apr 26 18:24:24 let's fix the cache'ing then Apr 26 18:24:57 Ok, I mis-stated Apr 26 18:25:04 no caching happens before imsi is read Apr 26 18:25:16 However, pre-PIN EFs can be read after the IMSI is known Apr 26 18:25:44 yeah, but then it's totally safe to cache them, we just gain time Apr 26 18:26:46 and hence we know the phase... ok I buy that Apr 26 18:30:56 balrog-k1n: talk to me about the BER/CTLV compositor, do you think my idea is faster or slower? Apr 26 18:33:07 denkenz: the idea of hinting when the length is not going to cross 0x80? it would save us some checks for sure, but seems complicated Apr 26 18:33:37 that memmove can only happen once, when the length crosses the 0x80 border because we have added something to the TLV, Apr 26 18:33:57 because length starts to occupy 2 bytes instead of 1 Apr 26 18:34:00 So what about the BER-TLV memmove? Apr 26 18:34:37 when we have ctlvs inside the BER-TLV? Apr 26 18:34:57 yep, or are you counting on the ber-tlv offset optimization I proposed? Apr 26 18:35:10 it'll happen once for the entire BER-TLV and once for every CTLV that crosses 128B Apr 26 18:35:42 but we usually set the length of the CTLVs ahead, i.e. when no data is in it yet Apr 26 18:35:46 so no memmove is needed Apr 26 18:36:16 My biggest problem is that it makes the encoder really complicated Apr 26 18:36:59 yeah, it's a little complicated, but sounds like anything else will also be complicated Apr 26 18:37:27 The biggest culprit is display_text Apr 26 18:37:41 unless we allocated a new buffer for the contents of every TLV and then at the end assembled everything by copying the buffers into one big buffer Apr 26 18:37:49 The encoder would have to encode the text, check its length, allocate a buffer of length + foo for the DCS Apr 26 18:38:24 I'd rather encoders look like foo_append_byte, foo_append_short() Apr 26 18:39:04 yeah, that would be easier Apr 26 18:39:37 but i did that encoding of text in the code and it doesn't seem so terrible (?) Apr 26 18:39:56 The text encoding is a necessary evil Apr 26 18:40:12 though not in display_text, only get_inkey/get_input send text in response Apr 26 18:40:14 However, the mms parameters unit test was just a little ugly for my tastes Apr 26 18:40:37 Btw, we might actually need encoders for all proactive commands Apr 26 18:40:44 the reponse needs to be squeezed into a AT+CSIM commands or some other command, and i think these are limited to 255 bytes Apr 26 18:41:36 The reason being that some modems decode the STK commands for us Apr 26 18:41:49 ah Apr 26 18:41:50 And then send different unsolicited notifications depending on the command Apr 26 18:42:06 i like the mms parameters test because decoding and encoding looks pretty much the same Apr 26 18:42:09 So we either make stk_proactive_command official API, or we re-encode them in the plugin Apr 26 18:42:42 If we need to re-encode arbitrary commands, then having a growable-buffer is a good idea Apr 26 18:43:01 yes, maybe let's have the decoding inside src/stkutil.c but not use it directly in src/stk.c Apr 26 18:43:13 instead drivers can use it if they need Apr 26 18:43:45 Problem becomes that we expose entirety of stkutil.h as ofono API Apr 26 18:43:52 I really don't want that Apr 26 18:45:03 hmm Apr 26 18:45:09 Of course encoding-reencoding is also bad Apr 26 18:45:22 But some modems are sane and simply send us the PDU as is Apr 26 18:45:44 if we want to support the modems that decode STK commands for us then we will have to have either the decoding or encoding of TLVs exposed as ofono api, i'm afraid Apr 26 18:46:04 Actually we can play a trick Apr 26 18:46:43 We keep the PDU format, but for modems that send decoded commands, the driver has to be in-tree and accesses the stkutil functionality to re-encode it Apr 26 18:47:38 Those drivers will have to compose the data structures manually anyway Apr 26 18:48:05 but that keeps the sane modems' workload minimal and our interface really clean Apr 26 18:48:40 Of course maybe I'm just being an idiot ;) Apr 26 18:49:55 sounds ok, just the reencoding may be a non-trivial amount of work Apr 26 18:50:09 compared to parsing something and putting into C structs Apr 26 18:50:41 balrog-k1n: We do it only for the commands we support Apr 26 18:50:57 do you know if nokia modems or any other popular modem does it? Apr 26 18:51:17 And quite a bit of dataobj are covered by terminal responses anyway Apr 26 18:51:31 I know of two major manufacturers that do it Apr 26 18:51:33 yeah Apr 26 18:51:35 40 iirc Apr 26 18:51:39 And two others send PDUs Apr 26 18:51:45 So 50/50 split :) Apr 26 18:52:21 I don't know how the Nokia interface looks though Apr 26 18:52:52 okay Apr 26 18:53:20 From looking at stkutil.c so far, it seems that display_text is about the trickiest one Apr 26 18:53:21 i assume someone would have screamed on the list already if they didn't send the pdu :) Apr 26 18:53:35 Nah, nobody is that far along Apr 26 18:54:31 Also, I think of it as an investment for ModemEmulator or rewriting phonesim Apr 26 18:54:38 We'd need to encode commands anyway Apr 26 18:54:43 oh yes Apr 26 18:54:55 re display_text, we already did some of the logic to use the best text encoding, in sms Apr 26 18:55:15 yep, your code seemed very similar Apr 26 18:55:38 the problem with get_inkey/get_input responses though is that there are some bits for the uicc to indicate what encoding it wants the response to use, Apr 26 18:56:17 but i couldn't find the exact information saying which encoding and what bit needs to be set Apr 26 18:56:37 heh, probably nobody uses that :P Apr 26 18:57:09 Look at the qualifier, that usually contains that type of info Apr 26 18:58:34 And in theory we can tell this to the UI and reject anything that it sends not fitting to the particular DCS Apr 26 18:59:40 yeah, it seems like writing an UI for this is going to be quite complicated Apr 26 19:00:00 Not our bug ;) Apr 26 19:00:26 i'm thinking it would be good to have a reference implementation Apr 26 19:00:40 otherwise nobody is going to make a full blown UI :) Apr 26 19:01:11 Let our / Nokia UI guys worry about that Apr 26 19:01:33 Right now we need something to make sure it works as expected, hence I want heavy unit tests Apr 26 19:03:01 good idea Apr 26 19:03:30 although this doesn't help to test with real modems to see if the response we send is what is expected, and if the apps actually work Apr 26 19:04:09 agreed, however we have phonesim which is a reasonable first test Apr 26 19:05:01 i read one of the montly reports that said ofono v. 0.20 delivered SIM application toolkit :) Apr 26 19:05:19 yeah, silliness Apr 26 19:06:17 so do you agree with having a proper encoder and leaving the interface as is? At least as a direction to start out with? Apr 26 19:07:28 the interface meaning the bunch of *_constr_* functions i added? Apr 26 19:07:57 no, the proactive command one Apr 26 19:08:11 ah yeah Apr 26 19:08:16 so it accepts PDUs only Apr 26 19:08:32 exactly Apr 26 19:09:10 yep, sounds okay, we can always change it if needed Apr 26 19:09:54 Ok cool, so given that we have a proper encoder requirement now Apr 26 19:10:26 also some potential mms code could use it Apr 26 19:10:37 What can we do to lessen the pain? I really don't like set_data() Apr 26 19:10:45 Oh? MMS has TLVs? :) Apr 26 19:11:14 oh i don't know, i assumed the TLV in the unit test is some mms thing :) Apr 26 19:11:27 Nah, I just picked it out of the air Apr 26 19:12:11 on top of the set_data() we can easily do the _append_byte/short/text() functions Apr 26 19:13:08 Yeah but we still do data-copies Apr 26 19:14:56 i suspect gbytearray does a lot more copying Apr 26 19:15:59 Heh Apr 26 19:17:26 so what about: tlv_builder_init(iter, tag), tlv_builder_open_container(iter, relocatable, tag), tlv_builder_close_container() <- does the memmove Apr 26 19:17:43 And leave append_byte/short/data parts Apr 26 19:18:04 Would that be simpler to implement? Apr 26 19:18:55 hmm yeah, this may be simpler to implement Apr 26 19:19:33 so initially you would reserve one byte for the length and mmove if it turns out longer? Apr 26 19:20:23 So lets take the case of CTLV, if you set the relocatable to FALSE, we set the length to 1 byte and assume that never changes Apr 26 19:20:35 So there is no memmove at the end regardless Apr 26 19:20:50 If relocatable is FALSE, we start at an offset for a 2-byte length Apr 26 19:21:07 If at the end we find that we have less, we memmove Apr 26 19:21:29 For BER-TLV there will never be memmoves if we use the proposed optimization Apr 26 19:21:53 we simply always reserve the maximum length and then reformat the header on each close_container Apr 26 19:21:54 because we use the pad bytes instead of memmoving? Apr 26 19:21:59 exactly Apr 26 19:22:36 Then pre-allocate the GByteArray to 256 or some magic value Apr 26 19:22:47 Hopefully 99% of the time we do a single malloc and no memmoves Apr 26 19:22:58 the thing is i don't think the relocatable flag helps here, Apr 26 19:23:12 with either complexity or the amount of copying Apr 26 19:23:38 well it may help if the users always set it right Apr 26 19:23:43 why not? You just check the first length byte, if its valid then don't memmove Apr 26 19:23:45 but then it adds complexity Apr 26 19:24:01 Otherwise memmove + fixup Apr 26 19:24:18 currently you don't memmove anyway if the length is < 0x80 Apr 26 19:24:38 and if it's longer then you only do it once, and copy only this number of bytes, not the entire ctlv Apr 26 19:25:14 See, that is the issue, you'd rather memmove 127 bytes than 16k Apr 26 19:25:56 you never have to memmove 16k, Apr 26 19:26:24 once the length has crossed 0x80, the length occupies two bytes and no further copying happens Apr 26 19:26:55 This is what I'm proposing... Apr 26 19:27:06 ok Apr 26 19:27:26 [14:23] why not? You just check the first length byte, if its valid then don't memmove Apr 26 19:27:47 So we only memmove if relocatable == TRUE & length so far is less than 128 Apr 26 19:27:55 ah ok Apr 26 19:28:13 the implementation i sent never moves more than 127 bytes either Apr 26 19:28:58 Nod, but it always does it, whereas I'm proposing we flag it Apr 26 19:29:15 normally the length will be known ahead and there won't be any moves, except in the container BER-TLV, but we can use the stuff bytes to avoid this Apr 26 19:29:41 it doesn't do it always, only when the length crosses 127 bytes, Apr 26 19:29:49 and we didn't know ahead it would cross 127 bytes Apr 26 19:29:54 it works like this: Apr 26 19:30:10 * at the start reserves one byte for length, Apr 26 19:30:34 * the user can call set_length successively as it adds data, or call it once if it knows the length already Apr 26 19:31:04 only if it adds data successively and crosses 127 bytes, there'll be a memmove Apr 26 19:32:12 well yeah, you're right that there'll always be the memmove if we implement the _append_byte api Apr 26 19:32:47 heh :) Apr 26 19:33:25 my main concern was it puts too much work on the encoder, where 95% of them are like 3 append_byte calls Apr 26 19:33:49 They always know whether they're relocatable or not Apr 26 19:34:02 And we can trust the user in this case Apr 26 19:34:11 So no need to be ultra-paranoid Apr 26 19:34:51 the only work on the encoder is comparing if the length exceeds the buffer size or 128 bytes on every append Apr 26 19:35:34 I'm fine with doing it that way too Apr 26 19:37:22 However, you _can_ trust the hint in this case to save yourself some work Apr 26 19:38:59 so we would set relocatable flag whenever there's a text field in the CTLV? Apr 26 19:40:15 Basically, along with one or two others that can be > 127 bytes Apr 26 19:41:46 If we know the length of the entire structure, then maybe open_container_with_length is even better Apr 26 19:43:03 the relocatable=TRUE case is same as setting initial length to 128 and then calling set_length again at the end Apr 26 19:44:06 Anyway, you already have the code, just make it look prettier for the end-user :) Apr 26 19:44:17 My main objection is set_length / byte by byte copy Apr 26 19:45:02 Someone will save off a pointer to the wrong offset and do nasty stuff Apr 26 19:45:13 Don't expose those details to the user Apr 26 19:47:37 oops Apr 26 19:48:34 did you see my last comments or ? Apr 26 19:48:47 21:43 < denkenz> If we know the length of the entire structure, then maybe Apr 26 19:48:47 open_container_with_length is even better Apr 26 19:48:47 21:44 < balrog-k1n> the relocatable=TRUE case is same as setting initial length Apr 26 19:48:47 to 128 and then calling set_length again at the end Apr 26 19:48:55 is the last thing i said Apr 26 19:49:21 [14:44] Anyway, you already have the code, just make it look prettier for the end-user :) Apr 26 19:49:23 [14:44] My main objection is set_length / byte by byte copy Apr 26 19:49:24 [14:45] Someone will save off a pointer to the wrong offset and do nasty stuff Apr 26 19:49:26 [14:45] Don't expose those details to the user Apr 26 19:50:03 e.g. get rid of set_length and get_data, replace by append_data and I'm actually good with that Apr 26 19:50:38 ah, sounds good Apr 26 19:51:22 do we want to supply the tag including the cr flag, or split in two parameters like i did? Apr 26 19:53:16 So the CR can be a separate param Apr 26 19:53:38 I think you went a little overboard on the BER-TLV tags, for the STK case they're not necessary Apr 26 19:56:38 also should it all sit in stkutil, or only the append_byte, append_short, append_text stuff? this way it remains compatible with our decoding functions Apr 26 19:57:28 Right now I have a feeling we simply don't need to solve the general composition case Apr 26 19:57:38 So making it explicitly for STK is fine Apr 26 20:01:45 Oh, and you might also want to add some comments explicitly stating who has to free the pdu in the end Apr 26 20:02:49 e.g. the caller does it or do it at builder_close() or something Apr 26 20:05:16 Nobody should really see anything besides stk_command_encode() function :) Apr 26 20:08:46 i guess for the terminal responses it's easiest to just have the caller supply a static buffer and its size Apr 26 20:14:51 balrog-k1n: If you're sure about the 256 byte limit, then sure Apr 26 21:19:13 balrog-k1n: One other thing I just thought of, do we want to cache our SIM files by ICCID? Apr 26 21:20:19 That would allow us to cache EFli/EFpl & EFecc and save a few more round trips on startup Apr 26 22:17:49 denkenz: i don't know, i have little idea of how unique iccid is Apr 26 22:18:08 iccid + phase i guess Apr 26 22:23:23 balrog-k1n: ICCID is unique for each smart card I believe Apr 26 22:24:45 And yes, ICCID + Phase Apr 26 22:32:36 denkenz: ah, neat Apr 26 22:33:10 i can propose a patch tomorrow Apr 26 22:33:40 we may want to ditch cacheing files / SMS fragments by imsi even Apr 26 22:36:22 SMS fragments by IMSI is actually correct Apr 26 22:36:39 Remember, you can have one Smart Card + multiple IMSIs Apr 26 22:38:10 balrog-k1n: However, there's still a question of whether a REFRESH IMSI can trigger a refresh of other files as well Apr 26 22:39:03 In which case storing by ICCID maybe not be correct Apr 26 22:46:27 Not urgent, lets do more research once we start messing with REFRESH of IMSI Apr 26 22:48:34 i guess the switching between imsis is persistent so someone might do the REFRESH in another phone and the iccid would not change but EFecc or EFli might Apr 26 22:50:13 Yeah, the whole REFRESH IMSI is bizarre Apr 26 22:50:25 We really need to figure out what it actually touches Apr 27 01:56:54 denkenz: my patch to gdbus is still waiting review on the mailing list. ;) Apr 27 02:05:32 denkenz: ping Apr 27 02:18:48 padovan: All gdbus patches are domain of holtmann Apr 27 02:19:03 yang_ubuntu: pong Apr 27 02:20:39 holtmann: can you review my gdbus patch, please? Apr 27 02:20:56 It's called "Add send_method_call to g_dbus" Apr 27 02:23:48 denkenz: This is Yang Gu. I changed my name from ygu5 to yang*. I have two questions here. Apr 27 02:25:09 1. For 8.20, parse_dataobj_imei(), I think there is no need to pass user as char **, char * is enough. Do you think so? Apr 27 02:27:00 I found we have some inconsistency when parsing comprehension tlv. Sometimes we use char *, and other times we use char **. Apr 27 02:30:29 2. Why struct stk_command_display_text is used in parse_get_inkey()? Is this on purpose? They are similar, but not identical. The difference is the ctlv of immediate_response. Apr 27 02:32:57 yang_ubuntu: We use char** when we malloc, char* when we don't Apr 27 02:33:34 so imei should probably be char * Apr 27 02:34:08 parse_get_inkey is probably just a bug Apr 27 02:35:28 agree, I will make the patch for them. Apr 27 02:43:56 denkenz: can we have some mechanism to 'write through' the AT command from DUN client side? Apr 27 02:44:35 write through? Apr 27 02:44:48 denkenz: yeah. the client side send us AT command Apr 27 02:44:57 'us'? Apr 27 02:45:02 I'm confused Apr 27 02:45:07 and directly feed to modem that talk with AT command Apr 27 02:45:28 you know we are acted as DUN server side Apr 27 02:45:35 err, nope Apr 27 02:45:42 We have the driver concept Apr 27 02:45:52 if we __know__ the modem talking AT Apr 27 02:45:58 We don't ;) Apr 27 02:46:18 Besides, for DUN server you don't actually interact with the modem driver Apr 27 02:46:27 sure, i know Apr 27 02:46:39 so what are you asking? Apr 27 02:46:45 zhenhua: Even if we would knew, the answer is still no. If you give a DUN client full access to all AT commands, it can screw with the whole telephony stack. Apr 27 02:46:52 And that would be a serious issue. Apr 27 02:47:35 Even simple stuff like CGDCONT is a major pain. Apr 27 02:47:37 holtmann: i know, but we can only register callbacks for few commands Apr 27 02:48:07 like ATD*99 Apr 27 02:48:38 If we are having the AT emulator in between anyway, then we do this properly and emulate all AT commands. So that is also works on ISI and other binary modems. Apr 27 02:48:50 denkenz: but for HFP AG, you will have ATD and CLCC etc Apr 27 02:49:16 So make a proxy ATD that detects whether this is GPRS, Voice or Data call Apr 27 02:49:17 holtmann: yes, that's the major reason to have this emulator in core, i think. Apr 27 02:49:22 Then calls the right one appropriately Apr 27 02:49:49 denkenz: i am working on that now. :) Apr 27 02:50:33 zhenhua: There is no shortcut here. We have to do the hard work to get this right once. After that everything will be easier in the future. Apr 27 02:50:34 hope to send out in this week. i am stuck with distro bugs last week. Apr 27 02:51:11 holtmann: that's fine. **** ENDING LOGGING AT Tue Apr 27 02:59:57 2010