A piece of Planet Debian
March 2016
S M T W T F S
    1 2 3 4
5
6 7 8 9
10
 11 12
13
14
 15 16 17 18 19
20
21
 22 23 24 25 26
27 28 29 30 31    

Archives

Tags:

RSS Atom
Reproducible builds: week 47 in Stretch cycle

What happened in the reproducible builds effort between March 13th and March 19th 2016:

Toolchain fixes

  • Petter Reinholdtsen uploaded naturaldocs/1.51-1.1 which makes the output reproducible. Original patch by Chris Lamb.
  • Damyan Ivanov uploaded libpdf-api2-perl/2.025-2 which will make internal font ID reproducible.
  • Christian Hofstaedtler uploaded ruby2.3/2.3.0-5 which sets gzip embedded mtime field to fixed value for rdoc-generated compressed javascript data.

Packages fixed

The following packages have become reproducible due to changes in their build dependencies: diction, doublecmd, ruby-hiredis, vdr-plugin-epgsearch.

The following packages became reproducible after getting fixed:

Some uploads fixed some reproducibility issues, but not all of them:

Patches submitted which have not made their way to the archive yet:

  • #818128 on nethack by Reiner Herrmann: implement support for SOURCE_DATE_EPOCH, set LC_ALL to C, and ensure deterministic build order when running parallel builds.
  • #818111 on debian-keyring by Satyam Zode: fix the order of files in md5sums.
  • #818067 on ncurses by Niels Thykier: strip trailing whitespaces introduced when using dash as system shell.
  • #818230 on aircrack-ng by Reiner Herrmann: build assembly code as a separate .o file.
  • #818419 on mutt by Daniel Shahaf: use C locale when listing files to be put in README.Patches.
  • #818430 on ruby-coveralls by Dhole: ensure UTC is used as the timezone when generating the documentation.
  • #818686 on littlewizard by Reiner Herrmann: use the C locale in the script for iterating over the files.
  • #818704 on strigi by Reiner Herrmann: sort keys when traversing hashes in makecode.pl.

Package reviews

44 reviews have been removed, 40 added and 5 updated in the previous week.

Chris Lamb has reported 16 FTBFS.

Posted Mon 21 Mar 2016 04:44:50 PM CET Tags:
Reproducible builds: week 46 in Stretch cycle

What happened in the reproducible builds effort between March 6th and March 12th:

Packages fixed

The following packages have become reproducible due to changes in their build dependencies: dfc, gap-openmath, gnubik, gplanarity, iirish, iitalian, monajat, openimageio, plexus-digest, ruby-fssm, vdr-plugin-dvd, vdr-plugin-spider.

The following packages became reproducible after getting fixed:

  • adduser/3.114 by Niels Thykier.
  • bsdmainutils/9.0.7 by Michael Meskes.
  • criu/2.0-1 by Salvatore Bonaccorso.
  • genometools/1.5.8+ds-2 by Sascha Steinbiss.
  • gfs2-utils/3.1.8-1 uploaded by Bastian Blank, fix by Christoph Berg.
  • gmerlin/1.2.0~dfsg+1-5 by IOhannes m zmölnig.
  • heroes/0.21-14 by Stephen Kitt.
  • kmc/2.3+dfsg-3 by Sascha Steinbiss.
  • polyml/5.6-3 by James Clarke.
  • sed/4.2.2-7.1 by Niels Thykier.
  • snpomatic/1.0-3 by Sascha Steinbiss.
  • tantan/13-4 by Sascha Steinbiss.

Some uploads fixed some reproducibility issues, but not all of them:

Patches submitted which have not made their way to the archive yet:

  • #817979 on modernizr by Sascha Steinbiss: sort list of files included in feature-detects.js.
  • #818027 on snapper by Sascha Steinbiss: always use /bin/sh as shell.

tests.reproducible-builds.org

Always use all cores on armhf builders. (h01ger)

Improve the look of Debian dashboard. (h01ger)

Package reviews

118 reviews have been removed, 114 added and 15 updated in the previous week.

15 FTBFS have been filled by Chris Lamb.

New issues: xmlto_txt_output_locale_specific.

Misc.

Lunar seeks new maintainers for diffoscope, several mailing lists, and these very weekly reports.

Posted Mon 14 Mar 2016 12:41:34 PM CET Tags:
Reproducible builds: week 45 in Stretch cycle

What happened in the reproducible builds effort between February 28th and March 5th:

Toolchain fixes

  • Antonio Terceiro uploaded gem2deb/0.27 that forces generated gemspecs to use the date from debian/changelog.
  • Antonio Terceiro uploaded gem2deb/0.28 that forces generated gemspecs to have their contains file lists sorted.
  • Robert Luberda uploaded ispell/3.4.00-5 which make builds of hashes reproducible.
  • Cédric Boutillier uploaded ruby-ronn/0.7.3-4 which will make the output locale agnostic. Original patch by Chris Lamb.
  • Markus Koschany uploaded spring/101.0+dfsg-1. Fixed by Alexandre Detiste.

Ximin Luo resubmitted the patch adding the --clamp-mtime option to Tar on Savannah's bug tracker.

Lunar rebased our experimental dpkg on top of the current master branch. Changes in the test infrastructure are required before uploading a new version to our experimental repository.

Reiner Herrmann rebased our custom texlive-bin against the latest uploaded version.

Packages fixed

The following 77 packages have become reproducible due to changes in their build dependencies: asciidoctor, atig, fuel-astute, jekyll, libphone-ui-shr, linkchecker, maven-plugin-testing, node-iscroll, origami-pdf, plexus-digest, pry, python-avro, python-odf, rails, ruby-actionpack-xml-parser, ruby-active-model-serializers, ruby-activerecord-session-store, ruby-api-pagination, ruby-babosa, ruby-carrierwave, ruby-classifier-reborn, ruby-compass, ruby-concurrent, ruby-configurate, ruby-crack, ruby-css-parser, ruby-cucumber-rails, ruby-delorean, ruby-encryptor, ruby-fakeweb, ruby-flexmock, ruby-fog-vsphere, ruby-gemojione, ruby-git, ruby-grack, ruby-htmlentities, ruby-jekyll-feed, ruby-json-schema, ruby-listen, ruby-markerb, ruby-mathml, ruby-mini-magick, ruby-net-telnet, ruby-omniauth-azure-oauth2, ruby-omniauth-saml, ruby-org, ruby-origin, ruby-prawn, ruby-pygments.rb, ruby-raemon, ruby-rails-deprecated-sanitizer, ruby-raindrops, ruby-rbpdf, ruby-rbvmomi, ruby-recaptcha, ruby-ref, ruby-responders, ruby-rjb, ruby-rspec-rails, ruby-rspec, ruby-rufus-scheduler, ruby-sass-rails, ruby-sass, ruby-sentry-raven, ruby-sequel-pg, ruby-sequel, ruby-settingslogic, ruby-shoulda-matchers, ruby-slack-notifier, ruby-symboltable, ruby-timers, ruby-zip, ticgit, tmuxinator, vagrant, wagon, yard.

The following packages became reproducible after getting fixed:

Some uploads fixed some reproducibility issues, but not all of them:

Patches submitted which have not made their way to the archive yet:

  • #816209 on elog by Reiner Herrmann: use printf instead of echo which is shell-independent.
  • #816214 on python-pip by Reiner Herrmann: removes timestamp from generated Python scripts.
  • #816230 on rows by Reiner Herrmann: tell grep to always treat the input as text.
  • #816232 on eficas by Reiner Herrmann: use printf instead of echo which is shell-independent.

Florent Daigniere and bancfc reported that linux-grsec was currently built with GRKERNSEC_RANDSTRUCT which will prevent reproducible builds with the current packaging.

tests.reproducible-builds.org

pbuilder has been updated to the last version to be able to support Build-Depends-Arch and Build-Conflicts-Arch. (Mattia Rizzolo, h01ger)

New package sets have been added for Subgraph OS, which is based on Debian Stretch: packages and build dependencies. (h01ger)

Two new armhf build nodes have been added (thanks Vagrant Cascadian) and integrated in our Jenkins setup with 8 new armhf builder jobs. (h01ger)

strip-nondeterminism development

strip-nondeterminism version 0.016-1 was released on Sunday 28th. It will now normalize the POT-Creation-Date field in GNU Gettext .mo files. (Reiner Herrmann) Several improvements to the packages metadata have also been made. (h01ger, Ben Finney)

Package reviews

185 reviews have been removed, 91 added and 33 updated in the previous week.

New issue: fileorder_in_gemspec_files_list.

43 FTBFS bugs were reported by Chris Lamb, Martin Michlmayr, and gregor herrmann.

Misc.

After merging the patch from Dhiru Kholia adding support for SOURCE_DATE_EPOCH in rpm, Florian Festi opened a discussion on the rpm-ecosystem mailing list about reproducible builds.

On March 4th, Lunar gave an overview of the general reproducible builds effort at the Internet Freedom Festival in Valencia.

Posted Thu 10 Mar 2016 09:48:05 PM CET Tags:
Reproducible builds: week 44 in Stretch cycle

What happened in the reproducible builds effort between February 21th and February 27th:

Toolchain fixes

Didier Raboud uploaded pyppd/1.0.2-4 which makes PPD generation deterministic.

Emmanuel Bourg uploaded plexus-maven-plugin/1.3.8-10 which sorts the components in the components.xml files generated by the plugin.

Guillem Jover has implemented stable ordering for members of the control archives in .debs.

Chris Lamb submitted another patch to improve reproducibility of files generated by cython.

Packages fixed

The following packages have become reproducible due to changes in their build dependencies: dctrl-tools, debian-edu, dvdwizard, dymo-cups-drivers, ekg2, epson-inkjet-printer-escpr, expeyes, fades, foomatic-db, galternatives, gnuradio, gpodder, gutenprint icewm, invesalius, jodconverter-cli latex-mk, libiio, libimobiledevice, libmcrypt, libopendbx, lives, lttnganalyses, m2300w, microdc2, navit, po4a, ptouch-driver, pxljr, tasksel, tilda, vdr-plugin-infosatepg, xaos.

The following packages became reproducible after getting fixed:

Some uploads fixed some reproducibility issues, but not all of them:

tests.reproducible-builds.org

The reproducibly tests for Debian now vary the provider of /bin/sh between bash and dash. (Reiner Herrmann)

diffoscope development

diffoscope version 50 was released on February 27th. It adds a new comparator for PostScript files, makes the directory tests pass on slower hardware, and line ordering variations in .deb md5sums files will not be hidden anymore.

Version 51 uploaded the next day re-added test data missing from the previous tarball.

diffoscope is looking for a new primary maintainer.

Package reviews

87 reviews have been removed, 61 added and 43 updated in the previous week.

New issues: captures_shell_variable_in_autofoo_script, varying_ordering_in_data_tar_gz_or_control_tar_gz.

30 new FTBFS have been reported by Chris Lamb, Antonio Terceiro, Aaron M. Ucko, Michael Tautschnig, and Tobias Frost.

Misc.

The release team reported on their discussion about the topic of rebuilding all of Stretch to make it self-contained (in respect to reproducibility).

Christian Boltz is hoping someone could talk about reproducible builds at the openSUSE conference happening June 22nd-26th in Nürnberg, Germany.

Posted Sat 05 Mar 2016 12:54:41 PM CET Tags:
Reproducible builds: week 43 in Stretch cycle

What happened in the reproducible builds effort between February 14th and February 20th 2016:

Toolchain fixes

Yaroslav Halchenko uploaded cython/0.23.4+git4-g7eed8d8-1 which makes its output deterministic. Original patch by Chris Lamb.

Didier Raboud uploaded pyppd/1.0.2-3 to experimental which now serialize PPD deterministically.

Lunar submitted two patches for lcms to add a way for clients to set the creation date/time in profile headers and initialize all bytes when writing named colors.

Packages fixed

The following packages have become reproducible due to changes in their build dependencies: dbconfig-common, dctrl-tools, dvdwizard, ekg2, expeyes, galternatives, gpodder, icewm, latex-mk, libiio, lives, navit, po4a, tasksel, tilda, vdr-plugin-infosatepg, xaos.

The following packages became reproducible after getting fixed:

Some uploads fixed some reproducibility issues, but not all of them:

Unknown status:

  • tomcat7/7.0.68-1 by Emmanuel Bourg (test suite fails in test environment).

Patches submitted which have not made their way to the archive yet:

  • #814840 on tor by Petter Reinholdtsen: use the UTC timezone when calling asciidoc.
  • #815082 on arachne-pnr by Dhole: use the C locale to format the changelog date.
  • #815192 on manpages-de by Reiner Herrmann: tell grep to always treat the input as text so that it works with non-UTF-8 locales.
  • #815193 on razorqt by Reiner Herrmann: tell grep to always treat the input as text so that it works with non-UTF-8 locales.
  • #815250 on jacal by Reiner Herrmann: use the C locale to format the build date.
  • #815252 on colord by Lunar: remove extra timestamps when generating CMF and spectra and implement support for SOURCE_DATE_EPOCH.

reproducible.debian.net

Two new package sets have been added: freedombox and freedombox_build-depends. (h01ger)

diffoscope development

diffoscope version 49 was released on February 17th. It continues to improve handling of debug symbols for ELF files. Their content will now be compared separately to make them more readable. The search for matching debug packages is more efficient by looking only for .deb files in the same parent directory. Alongside more bug fixes, support for ICC profiles has been added, and libarchive is now also used to read metadata for ar archives.

strip-nondeterminism development

Reiner Herrmann added support to normalize Gettext .mo files.

Package reviews

170 reviews have been removed, 172 added and 54 updated in the previous week.

34 new FTBFS bugs have been opened by Chris Lamb, h01ger and Reiner Herrmann.

New issues added this week: lxqt_translate_desktop_binary_file_matched_under_certain_locales, timestamps_in_manpages_generated_by_autogen.

Improvements to the prebuilder script: avoid ccache, skip disorderfs hook if device nodes cannot be created, compatibility with grsec trusted path execution (Reiner Herrmann), code cleanup (Esa Peuha).

Misc.

Steven Chamberlain highlighted reproducibility problems due to differences in how Linux and FreeBSD handle permissions for symlinks. Some possible ways forward have been discussed on the reproducible-builds mailing list.

Bernhard M. Wiedemann reported on some reproducibility tests made on OpenSuse mentioning the growing support for SOURCE_DATE_EPOCH.

If you are eligible for Outreachy or Google Summer of Code, consider spending the summer working on reproducible builds!

Posted Sun 21 Feb 2016 11:07:07 PM CET Tags:
Reproducible builds: week 42 in Stretch cycle

What happened in the reproducible builds effort between February 7th and February 13th 2016:

Toolchain fixes

  • James McCoy uploaded devscripts/2.16.1 which makes dcmd supports .buildinfo files. Original patch by josch.
  • Lisandro Damián Nicanor Pérez Meyer uploaded qt4-x11/4:4.8.7+dfsg-6 which make files created by qch reproducible by using a fixed date instead of the current time. Original patch by Dhole.

Norbert Preining rejected the patch submitted by Reiner Herrmann to make the CreationDate not appear in comments of DVI / PS files produced by TeX. He also mentioned that some timestamps can be replaced by using the -output-comment option and that the next version of pdftex “will have patches inspired by reproducible build to mitigate the effects (see SOURCE_DATE_EPOCH patches)”.

Packages fixed

The following packages have become reproducible due to changes in their build dependencies: abntex, apt-dpkg-ref, arduino, c++-annotations, cfi, chaksem, clif, cppreference-doc, dejagnu, derivations, ecasound, fdutils, gnash, gnu-standards, gnuift, gsequencer, gss, gstreamer0.10, gstreamer1.0, harden-doc, haskell98-report, iproute2, java-policy, libbluray, libmodbus, lizardfs, mclibs, moon-buggy, nurpawiki, php-sasl, shishi, stealth, xmltex, xsom.

The following packages became reproducible after getting fixed:

Some uploads fixed some reproducibility issues, but not all of them:

Patches submitted which have not made their way to the archive yet:

  • #813944 on cvm by Reiner Herrmann: remove gzip headers, fix permissions of some directories and the order of the md5sums.
  • #814019 on latexdiff by Reiner Herrmann: remove the current build date from documentation.
  • #814214 on rocksdb by Chris Lamb: add support for SOURCE_DATE_EPOCH.

reproducible.debian.net

A new armhf build node has been added (thanks to Vagrant Cascadian) and integrated into the Jenkins setup for 4 new armhf builder jobs. (h01ger)

All packages for Debian testing (Stretch) have been tested on armhf in just 42 days. It took 114 days to get the same point for unstable back when the armhf test infrastructure was much smaller.

Package sets have been enabled for testing on armhf. (h01ger)

Packages producing architecture-independent (“Arch:all”) binary packages together with architecture dependent packages targeted for specific architectures will now only be tested on matching architectures. (Steven Chamberlain, h01ger)

As the Jenkins setup is now made of 252 different jobs, the overview has been split into 11 different smalller views. (h01ger)

Package reviews

222 reviews have been removed, 110 added and 50 updated in the previous week.

35 FTBFS reports were made by Chris Lamb, Danny Edel, and Niko Tyni.

Misc.

The recordings of Ludovic Courtès' talk at FOSDEM’16 about reproducible builds and GNU Guix is now available. One can also have a look at slides from Fabian Keil's talk about ElecrtroBSD and Baptiste Daroussin's talk about FreeBSD packages.

Posted Sun 14 Feb 2016 10:47:30 PM CET Tags:
Reproducible builds: week 41 in Stretch cycle

What happened in the reproducible builds effort this week:

Toolchain fixes

After remarks from Guillem Jover, Lunar updated his patch adding generation of .buildinfo files in dpkg.

Packages fixed

The following packages have become reproducible due to changes in their build dependencies: dracut, ent, gdcm, guilt, lazarus, magit, matita, resource-agents, rurple-ng, shadow, shorewall-doc, udiskie.

The following packages became reproducible after getting fixed:

  • disque/1.0~rc1-5 by Chris Lamb, noticed by Reiner Herrmann.
  • dlm/4.0.4-2 by Ferenc Wágner.
  • drbd-utils/8.9.6-1 by Apollon Oikonomopoulos.
  • java-common/0.54 by by Emmanuel Bourg.
  • libjibx1.2-java/1.2.6-1 by Emmanuel Bourg.
  • libzstd/0.4.7-1 by Kevin Murray.
  • python-releases/1.0.0-1 by Jan Dittberner.
  • redis/2:3.0.7-2 by Chris Lamb, noticed by Reiner Herrmann.
  • tetex-brev/4.22.github.20140417-3 by Petter Reinholdtsen.

Some uploads fixed some reproducibility issues, but not all of them:

  • anarchism/14.0-4 by Holger Levsen.
  • hhvm/3.11.1+dfsg-1 by Faidon Liambotis.
  • netty/1:4.0.34-1 by Emmanuel Bourg.

Patches submitted which have not made their way to the archive yet:

  • #813309 on lapack by Reiner Herrmann: removes the test log and sorts the files packed into the static library locale-independently.
  • #813345 on elastix by akira: suggest to use the $datetime placeholder in Doxygen footer.
  • #813892 on dietlibc by Reiner Herrmann: remove gzip headers, sort md5sums file, and sort object files linked in static libraries.
  • #813912 on git by Reiner Herrmann: remove timestamps from documentation generated with asciidoc, remove gzip headers, and sort `md5sums and tclIndex files.

reproducible.debian.net

For the first time, we've reached more than 20,000 packages with reproducible builds for sid on amd64 with our current test framework.

Vagrant Cascadian has set up another test system for armhf. Enabling four more builder jobs to be added to Jenkins. (h01ger)

Package reviews

233 reviews have been removed, 111 added and 86 updated in the previous week.

36 new FTBFS bugs were reported by Chris Lamb and Alastair McKinstry.

New issue: timestamps_in_manpages_generated_by_yat2m. The description for the blacklisted_on_jenkins issue has been improved. Some packages are also now tagged with blacklisted_on_jenkins_armhf_only.

Misc.

Steven Chamberlain gave an update on the status of FreeBSD and variants after the BSD devroom at FOSDEM’16. He also discussed how jails can be used for easier and faster reproducibility tests.

The video for h01ger's talk in the main track of FOSDEM’16 about the reproducible ecosystem is now available.

Posted Mon 08 Feb 2016 04:42:05 PM CET Tags:
Reproducible builds: week 40 in Stretch cycle

What happened in the reproducible builds effort between January 24th and January 30th:

Media coverage

Holger Levsen was interviewed by the FOSDEM team to introduce his talk on Sunday 31st.

Toolchain fixes

Jonas Smedegaard uploaded d-shlibs/0.63 which makes the order of dependencies generated by d-devlibdeps stable accross locales. Original patch by Reiner Herrmann.

Packages fixed

The following 53 packages have become reproducible due to changes in their build dependencies: appstream-glib, aptitude, arbtt, btrfs-tools, cinnamon-settings-daemon, cppcheck, debian-security-support, easytag, gitit, gnash, gnome-control-center, gnome-keyring, gnome-shell, gnome-software, graphite2, gtk+2.0, gupnp, gvfs, gyp, hgview, htmlcxx, i3status, imms, irker, jmapviewer, katarakt, kmod, lastpass-cli, libaccounts-glib, libam7xxx, libldm, libopenobex, libsecret, linthesia, mate-session-manager, mpris-remote, network-manager, paprefs, php-opencloud, pisa, pyacidobasic, python-pymzml, python-pyscss, qtquick1-opensource-src, rdkit, ruby-rails-html-sanitizer, shellex, slony1-2, spacezero, spamprobe, sugar-toolkit-gtk3, tachyon, tgt.

The following packages became reproducible after getting fixed:

Some uploads fixed some reproducibility issues, but not all of them:

  • gnubg/1.05.000-4 by Russ Allbery.
  • grcompiler/4.2-6 by Hideki Yamane.
  • sdlgfx/2.0.25-5 fix by Felix Geyer, uploaded by Gianfranco Costamagna.

Patches submitted which have not made their way to the archive yet:

  • #812876 on glib2.0 by Lunar: ensure that functions are sorted using the C locale when giotypefuncs.c is generated.

diffoscope development

diffoscope 48 was released on January 26th. It fixes several issues introduced by the retrieval of extra symbols from Debian debug packages. It also restores compatibility with older versions of binutils which does not support readelf --decompress.

strip-nondeterminism development

strip-nondeterminism 0.015-1 was uploaded on January 27th. It fixes handling of signed JAR files which are now going to be ignored to keep the signatures intact.

Package reviews

54 reviews have been removed, 36 added and 17 updated in the previous week.

30 new FTBFS bugs have been submitted by Chris Lamb, Michael Tautschnig, Mattia Rizzolo, Tobias Frost.

Misc.

Alexander Couzens and Bryan Newbold have been busy fixing more issues in OpenWrt.

Version 1.6.3 of FreeBSD's package manager pkg(8) now supports SOURCE_DATE_EPOCH.

Ross Karchner did a lightning talk about reproducible builds at his work place and shared the slides.

Posted Tue 02 Feb 2016 12:39:09 AM CET Tags:
ASLR now!

Address space layout randomization helps to protect against buffer overflow attacks as it becomes harder for an attacker to turn a programming error into an exploitable security hole. The first implementation for Linux is 15 years old. Support in mainline kernel and toolchains have been available for a good while now. But to work, ASLR also needs executables to be built as position independent. And as Hanno Böck from the fuzzing project gently reminded me at 32C3, almost no executables in Debian are built as such, while it is now the default in Windows, Mac OS X, OpenBSD, and Fedora to name just a few other systems.

PIE has the reputation of having a performance hit. While true, especially for register constrained architectures like i386, it makes a difference only for the executable itself, not any shared library it uses as they are already built as position independent. Also, several optimizations have been made since the early days. GCC 5 will reuse the PIC hard register (which is also good for libraries). On amd64, GCC 5 and binutils 2.26 will do copy relocations. More improvements for i386 are being worked on.

I sincerly believe that users are way more likely to notice a compromised system than a slightly slower one.

Tracking progress

Since version 2.5.40, lintian will now issue a warning1 when it detects that a binary has not been compiled as a position independent executable. Kudos to Niels Thykier. Now that we can track our progress, I'm calling every Debian Developers: let's try to get as many ASLR-compatible executables in Stretch!

How to enable PIE

Thanks to all contributors over the past years who have improved our toolchain, we now have a fairly easy way to enable hardening flags with dpkg-buildflags. For packages using dh, it now boils down with adding on top of debian/rules:

export DEB_BUILD_MAINT_OPTIONS = hardening=+all

You can even test if a package supports all hardening flags without any changes running DEB_BUILD_OPTIONS=hardening=+all dpkg-buildpackage. Running lintian or hardening-check can then tell you if the protections have been successfully enabled.

Hardening by default?

But do we really need to change so many packages individually? The difference between the current default features and all hardening features are pie and bindnow. Could we turn them by default and do binNMUs instead of requiring actions from maintainers?

I guess the question boils down to: how many packages would require a (one-liner) change to turn off the pie or bindnow features if they were on by default?

To get the beginning of an answer, I took the top 502 (according to popcon installations) source packages shipping non-position independent executables. I've try to rebuild them enabling all hardening flags through DEB_BUILD_OPTIONS.

Out of 49 packages3:

  • 32 (65%) built fine and produced PIE binaries: acpi, bc, bind9, bsd-mailx, bsdmainutils, bzip2, cpio, cron, debianutils, diffutils, dpkg, file, fontconfig, gettext, glib2.0, glibc, gnupg, gzip, hostname, ifupdown, iputils, logrotate, m4, mutt, nano, net-tools, netcat, netkit-ftp, netkit-telnet, os-prober, pam, util-linux.
  • 4 (8%) built fine but did not compiled hardened binaries: discover, mawk, mlocate, patch.
  • 13 (27%) failed to build, with some of these expected failures, e.g. for *-static or GRUB: bash, busybox, coreutils, e2fsprogs, grub2, insserv, iptables, kbd, ncurses, newt, openssl, pciutils, perl.

The results are really encouraging. Especially taking in account that some of these packages are part of the “tricky and weird” set. To know for sure, we would need a mass-rebuild of the archive with DEB_BUILD_OPTIONS=hardening=+all in the environment. Any takers?

  1. Verification of the whole archive by the latest version of lintian is still in progress by the time I'm writing these lines. According to Niels it should take 3-4 more days to look at all available packages. ↩

  2. As always, UDD does wonders:

    SELECT packages.source, MAX(popcon.insts) AS insts
  FROM lintian, popcon, packages
 WHERE lintian.tag = 'hardening-no-pie'
   AND lintian.package_arch = 'amd64'
   AND popcon.package = lintian.package
   AND packages.package = popcon.package
   AND packages.distribution = 'debian'
   AND packages.release = 'sid'
 GROUP BY packages.source
 ORDER BY MAX(popcon.insts) DESC
 LIMIT 50;
 ↩

  3. acpid currently fail to build from source in sid. ↩

Posted Wed 27 Jan 2016 12:54:52 AM CET Tags:
Reproducible builds: week 39 in Stretch cycle

What happened in the reproducible builds effort between January 17th and January 23rd:

Toolchain fixes

James McCoy uploaded subversion/1.9.3-2 which removes -Wdate-time from CPPFLAGS passed to swig enabling several packages to build again.

The switch made in binutils/2.25-6 to use deterministic archives by default had the unfortunate effect of breaking a seldom used feature of make. Manoj Srivastava asked on debian-devel the best way to communicate the changes to Debian users. Lunar quickly came up with a patch that displays a warning when Make encounters “deterministic” archives. Manoj made it available in make/4.1-2 together with a NEWS file advertising the change.

Following Guillem Jover's comment on the latest patch to make mtimes of packaged files deterministic, Daniel Kahn Gillmor updated and extended the patch adding the --clamp-mtime option to GNU Tar.

Mattia Rizzolo updated texlive-bin in the “reproducible” experimental repository.

Packages fixed

The following packages became reproducible after getting fixed:

Some uploads fixed some reproducibility issues, but not all of them:

Patches submitted which have not made their way to the archive yet:

reproducible.debian.net

Transition from reproducible.debian.net to the more general tests.reproducible-builds.org has started. More visual changes are coming. (h01ger)

A plan on how to run tests for F-Droid has been worked out. (hc, mvdan, h01ger) A first step has been made by adding a Jenkins job to setup an F-Droid build environment. (h01ger)

diffoscope development

diffoscope 46 has been released on January 19th, followed-up by version 47 made available on January 23rd. Try it online at try.diffoscope.org!

The biggest visible change is the improvement to ELF file handling. Comparisons are now done section by section, using the most appropriate tool and options to get meaningful results, thanks to Dhole's work and Mike Hommey's suggestions. Also suggested by Mike, symbols for IP-relative ops are now filtered out to remove clutter.

Understanding differences in ELF files belonging to Debian packages should also be much easier as diffoscope will now try to extract debug information from the matching dbgsym package. This means objdump disassembler should output line numbers for packages built with recent debhelper as long as the associated debug package is in the same directory.

As diff tends to consume huge amount of memory on large inputs, diffoscope has a limit in place to prevent crashes. diffoscope used to display a difference every time the limit was hit. Because this was confusing in case there were actually no differences, a hash is now internally computed to only report a difference when one exists.

Files in archives and other container members are now compared in the original order. This should not matter in most case but overall give more predictable results.

Debian .buildinfo files are now supported.

Amongst other minor fixes and improvements, diffoscope will now properly compare symlinks in directories. Thanks Tuomas Tynkkynen for reporting the problem.

Package reviews

70 reviews have been removed, 125 added and 33 updated in the previous week, gcc-5 amongst others.

25 FTBFS issues have been filled by Chris Lamb, Daniel Stender, Martin Michlmayr.

Misc.

The 16th FOSDEM will happen in Brussels, Belgium on January 30-31st. Several talks will be about reproducible builds: h01ger about the general ecosystem, Fabian Keil about the security oriented ElectroBSD, Baptiste Daroussin about FreeBSD packages, Ludovic Courtès about Guix.

Posted Sun 24 Jan 2016 10:11:21 PM CET Tags:

This blog is powered by ikiwiki.