**** BEGIN LOGGING AT Sun Jun 03 03:00:02 2018
Jun 03 03:06:54 <timeless> DocScrutinizer: yes.
Jun 03 03:10:10 <timeless> Unfortunately, this is a basic flaw in the way Debian packages work. The binaries don't include enough information to capture how they were built
Jun 03 03:12:52 <timeless> Romoxa (Oleg Romashin) most of the build optimizations and continued working on fennec for a while
Jun 03 08:34:10 <enyc> timeless: hrrm... isn't that what '-dbg' and source packages are for?
Jun 03 08:34:34 <timeless> no
Jun 03 08:34:44 <enyc> timeless: reproducible-builds project being helpful, you can check you can rebuild he same checksum binary etc.?
Jun 03 08:35:00 <timeless> they don't provide a way to guarantee a complete build env
Jun 03 08:35:22 <timeless> enyc: the problem is that a binary package doesn't tell you which precise versions of dependent packages
Jun 03 08:35:39 <timeless> it doesn't tell you which version of the compiler was used
Jun 03 08:35:41 <timeless> or the linker
Jun 03 08:35:42 <timeless> or...
Jun 03 08:35:50 <enyc> timeless: aaat the build-log i see
Jun 03 08:35:59 <timeless> yes, if you manage to get matching checksums, you know that you reproduced things
Jun 03 08:36:14 <timeless> but, when you don't (which will happen 99.99% of the time, due to things that embed timecodes)
Jun 03 08:36:19 <timeless> you don't know anything
Jun 03 08:36:24 <enyc> timeless: surely its' expected to use tha same common pbuilder / debian release
Jun 03 08:36:40 <enyc> timeless: thats part of why they use pbuilder,  create new chroot with minimum deps for building  each time
Jun 03 08:37:14 <enyc> but then i understand it depends what  'updates'  to a release were applied before/after a package was built
Jun 03 08:37:36 <enyc> e.g.  gcc with retpoline fixes -- has package been rebuilt-or-not, etc etc
Jun 03 08:38:27 <timeless> enyc: anyway, the point is that debian packages do not encode enough info
Jun 03 08:38:51 <timeless> if the package is from along time ago, (5-10 years), you're SOL if the build environment is long gone
Jun 03 08:39:12 <enyc> have you made a propased patch to dpkg/tools etc.?, oloked into options for build logs/info to be stored eithre separately or embedded in the debs in a way that won't affect existing systems installing them...?
Jun 03 08:39:37 <timeless> no
Jun 03 08:39:44 <timeless> for one, i'm not sure it's entirely solvable
Jun 03 08:39:55 <timeless> one could at least encode the version of installed packages
Jun 03 08:40:18 <enyc> the pbuilder creates buildlogs
Jun 03 08:40:29 <DocScrutinizer05> embeeded timestamps, HAHA great. I guess a VM with patched time_t time(time_t *t)  that simply returns a configurable constant might be useful
Jun 03 08:41:09 <timeless> there's a faketime module, i use it to talk to openssl
Jun 03 08:41:11 <enyc> well acutalyl reproducible-builds work to get rid of all build times etc. of course
Jun 03 08:42:29 <timeless> enyc: anyway, build logs that are on a server that don't persist aren't helpful
Jun 03 08:42:33 <Maxdamantus> I think I've noticed that particular packages in nix have workarounds for removing build times from things that store them.
Jun 03 08:42:38 <timeless> remember, DocScrutinizer05 is asking about packages from 5-10 years ago
Jun 03 08:42:53 <DocScrutinizer05> actually jonwil is
Jun 03 08:42:55 <timeless> the source servers in a number of cases are gone
Jun 03 08:43:03 <timeless> heck, even the build env distros are gone
Jun 03 08:43:04 <Maxdamantus> (and as a standard feature, file times are all set to `0` after building a derivation in the store)
Jun 03 08:43:09 <timeless> (which scratchbox,...)
Jun 03 08:43:27 <enyc> Maxdamantus: thats' to make the build reproducible ?
Jun 03 08:44:07 <Maxdamantus> enyc: not exactly.
Jun 03 08:44:20 <Maxdamantus> enyc: the build should be reproducible enough without that.
Jun 03 08:44:22 <timeless> really, to some extent, anything that isn't part of the shipping .deb is probably a bad thing to rely on
Jun 03 08:44:50 <timeless> it's pretty easy to not end up w/ the associated "dbg" packages in case no one used them and the repo got lost
Jun 03 08:45:12 <enyc> well you say the problem isn't solvab/e, mut didn't rcatch what parts ooyou want solved and why,  what the compatibilite problems would be, and how you do intend to solve those that should be solved.
Jun 03 08:45:44 <timeless> anyway,... solvable? maybe, it'd probably require logging all installed package versions and checksums for all files
Jun 03 08:46:16 <timeless> the question is ~ can one rebuild a build env in order to rebuild a package?
Jun 03 08:46:34 <timeless> that's what's desired and in a number of cases desirable
Jun 03 08:46:52 <Maxdamantus> That should be relatively doable in nix.
Jun 03 08:46:54 <timeless> a fun version of this problem:
Jun 03 08:46:57 <enyc> timeless: hrrm like i mentiont to leste people they shouldn't be expecting to reply on current githumb for contributors .. if you want to keep and reprouce/upstream fixes you need things like copyright file and clear status... hey-ho.
Jun 03 08:47:04 <Maxdamantus> (though that's not relevant in this case)
Jun 03 08:47:12 <timeless> a package X depends on a build-dep Y w/ some vague version requirement (not absolute)
Jun 03 08:47:27 <timeless> in one version of Y, it defines a macro w/ one value, in another, it defines it with a different value
Jun 03 08:47:28 <DocScrutinizer05> could at least from now on make, cmake et al take care to include all info to the .deb per default?
Jun 03 08:47:55 <Maxdamantus> What is "all info"? Every package on the build system?
Jun 03 08:48:07 <Maxdamantus> or every package transitively reachable through the dependencies?
Jun 03 08:48:15 <timeless> the resulting X package will have totally different shapes depending on the define, which is not something that you can tell by looking at the package
Jun 03 08:48:21 <Maxdamantus> or just the dependencies?
Jun 03 08:48:52 <DocScrutinizer05> ouch
Jun 03 08:49:20 <timeless> Maxdamantus: it would be safer if it did all packages, but for privacy reasons everyone would object to that
Jun 03 08:49:53 <timeless> i think from a reproducible perspective, if your build env was able to record all packages that contributed to files that were opened during the build process
Jun 03 08:49:58 <timeless> you'd get something vaguely good
Jun 03 08:50:12 <DocScrutinizer05> :nod:
Jun 03 08:50:19 <DocScrutinizer05> augias task
Jun 03 08:51:11 <timeless> and again, what do you do if something is modifying how you see files
Jun 03 08:51:13 <Maxdamantus> I'm not sure how it normally works on debian-based systems, though I think it hasn't worked this way with maemo, but: if everyone agrees that all official packages are built only by an automatic build server, it should be fairly reproducible.
Jun 03 08:51:28 <timeless> Maxdamantus: i worked on maemo
Jun 03 08:51:37 <timeless> the build servers more or less did things in parallel
Jun 03 08:51:40 <Maxdamantus> since the build maintainers can just keep a git repository with all deb sources.
Jun 03 08:51:52 <timeless> you could have races where different packages were built/used in parallel
Jun 03 08:51:59 <timeless> such things were real problems
Jun 03 08:52:14 <timeless> specifically, i tried to build a source code index of a given binary release
Jun 03 08:52:14 <Maxdamantus> Heh, fun.
Jun 03 08:52:17 <timeless> it wasn't really possible
Jun 03 08:52:31 <timeless> because different binaries used different versions of dependent packages
Jun 03 08:52:32 * DocScrutinizer05 idly muses about keeping complete VMs in source 'repo'
Jun 03 08:52:49 <DocScrutinizer05> one for each package - OUUUFF
Jun 03 08:52:50 <timeless> DocScrutinizer05: real professional projects do that, more or less
Jun 03 08:53:06 <timeless> not archiving the build env leads to problems like this
Jun 03 08:53:20 <Maxdamantus> tbh, the complete VMs thing doesn't seem particularly unattainable.
Jun 03 08:53:31 <Maxdamantus> If your storage mechanism involves deduplication, it should be fine.
Jun 03 08:54:05 <Maxdamantus> eg, one obvious way of achieving that would be to fire off every build in an isolated snapshot of the filesystem.
Jun 03 08:54:34 <Maxdamantus> then you just save the filesystem tree as a git tree in a bit repo.
Jun 03 08:54:34 <timeless> Maxdamantus: i used dedup on the source system to avoid blowing up space w/ my weekly index
Jun 03 08:54:43 <timeless> (i used zfs)
Jun 03 08:55:04 <sparre> At one of my customers, we launch a VM on Amazon for every build-and-test run.
Jun 03 08:55:19 <Maxdamantus> Yeah, I was thinking of the "snapshot" part in terms of btrfs, but zfs works too.
Jun 03 08:55:30 <timeless> this was over a decade ago
Jun 03 08:55:33 * DocScrutinizer05 looked into dedup solutions a lot, for local workstation (btrfs, ext4) - it's a PITA and nothing _really_ worked
Jun 03 08:55:56 <timeless> in my case, since the files were generally predictably named, it was pretty easy
Jun 03 08:56:07 <timeless> i built to a stage, got rid of the package versions from directory names
Jun 03 08:56:15 <timeless> and then rsync'd over the previous week's files
Jun 03 08:56:23 <timeless> and snapshotted
Jun 03 08:56:41 <DocScrutinizer05> yeah, that's somewhat sustainable and feasible
Jun 03 08:57:06 <timeless> but, that didn't solve the stupid "x uses a@1; y uses a@2; z uses b@2 which uses a@?"
Jun 03 08:57:23 <DocScrutinizer05> :nod:
Jun 03 08:59:05 <DocScrutinizer05> the problem however feels like homegrown, the Godfathers of unix had a pretty clear notion about how to tackle this at very least on .SharedObject library level
Jun 03 08:59:35 <timeless> sharedobjects are one things, but it doesn't cover macros
Jun 03 08:59:42 <DocScrutinizer05> "new" stuff should strictly follow same principles
Jun 03 08:59:59 <DocScrutinizer05> yeah, sure, that's why >>at least for...<<
Jun 03 09:01:09 <timeless> anyway.... it sucked... and was very frustrating not being able to answer questions about what a binary was thinking
Jun 03 09:01:10 <DocScrutinizer05> coders would need to learn to *always* think fore/back-ward compatibility
Jun 03 09:02:02 <timeless> grr. cat wants to go out
Jun 03 09:02:30 <DocScrutinizer05> but yeah, when you make a new (version of) package, you usually don't give a shit about backward and even less forward compatibility
Jun 03 09:18:17 * DocScrutinizer05 a few days ago looked into snap packages first time. Also sort of PITA, NFC where the apache running in snap env is keeping its vhost config
Jun 03 09:20:38 <DocScrutinizer05> the damn nextcloud snap package even has a LetsEncrypt running, but I can't find how to enable apache HTTPS
Jun 03 09:30:19 <timeless> is mod_md in hat thing?
Jun 03 09:45:33 <jonwil> I think as far as build prerequisites go, the only option is to assume that if "configure" doesn't complain about stuff being out of date, its suitable to use.
Jun 03 20:24:08 <Vajb> hehe interesting read of maemo builds back there
Jun 03 20:24:54 <Vajb> last nite i wanted to prepare build enviroment for my upcoming android course...
Jun 03 20:25:10 <Vajb> can admit that i don't like android studio
**** ENDING LOGGING AT Mon Jun 04 03:00:00 2018