Quality requirements for software in Debian

should there be a minimal quality standard?

After an attempt to package Hadoop for Debian I had to learn that the quality of
the upstream is too low to warrant an inclusion into Debian. Should there be 
minimal quality standards for software to be included in Debian? What could 
these standards be? If such standards would exist, it would be easy to challenge
an ITP for a crapy PHP content management system...?

==Motivation==

* Is there any reason, why we shouldn't just include anything in Debian?

* Social Contract: "We will make the best system we can, so that free works will
  be widely distributed and used." , "... we will provide an integrated system 
  of high-quality materials ..."
  
* Caring for users may also mean to shield them from bad solutions.
  
* Why is quality important:
  * few bugs
  * fast fixes for bugs / security holes
  * usability (no crashes, no data loss, adaptable for different environments)
  * Users should only learn systems that we could expect to be around for a 
    longer time.
  * few migrations, so users don't have to migrate data, update servers to often
  * are developers also users? it should be pleasant to work on the code.

* Make reasoning which software should enter Debian more objective, less personal
* Debian contains that many software, that the problem is now to keep crap
  outside of Debian
* It's a good advertisment for software to be officially packaged for Debian
* Debian Policy says nothing about upstream code quality?

==Types of quality metrics==

* general: code duplication, test coverage, coupling, complexity, 
  established best practices (JSLint, pylint)
* java: Findbugs, PMD
* Project management: Has bugtracker, mailinglist, (D)VCS, long term support
  releases, proper versioning scheme ( http://semver.org ), code review system,
  number of developers, maturity
* Documentation: for Users, Developers
* Being used by other projects; unfortunately there are many cases of poor
  projects that are necessary to satisfy dependencies of others.
* What upstream thinks about code quality
* http://wiki.debian.org/DebianMentorsNet#Metrics
* lintian-extra

==How to interprete the metrics==

* no hard limit - use social pressure :-)
* just like "signals" in google's pagerank algorithm
* "points" are inherited from dependencies, e.g. each PHP CMS inherits partly
  the quality metrics of PHP
* "Uniqueness" as a metric - extra points, if it's the only of its kind
  - use debtags for this: if two packages cannot be discriminated with debtags,
    probably they're quite equivalent
* Position in dependency tree: If many packages depends on it, it gets points
    (unless it is causing issues with those dependencies).
* Popcon numbers
* UDD info - about bugs for un-maintained packages candidates for removal using bapase: 
  http://udd.debian.org/cgi-bin/bapase.cgi
  (particularly useful for "packages maintained by NMU's alone")
* Amount of CVE-IDs per lines of code or installations
* Amount of comments per Code (ohcount may help here)
* Amount and quality of documentation

==Implementation in Debian==

* Just measure packages that are already in Debian and publish metrics. Thus
  people can compare their ITP software against the metrics.
  
* Dream: People could use the metrics setup of Debian even _before_ filling the
  ITP and attach the metrics to their ITP.

* Should Debian provide project hosting for not (yet) packaged stuff with all 
  kinds of metrics, project tools, to help upstream?

* Do we need work distribution systems like SetiAtHome (BOINC)?

* Migration to testing regulated also according to the computer quality
  index (for example: a package with low index gets migrated only if explicitly
  requested)

==Related projects==

* http://qa.debian.org/daca Debian Automated Code Analysis (Raphael Geissert)

===Outside Debian===
* http://www.sqo-oss.org/biblio
* http://www.qualipso.org/ -- EU-funded research project about "Trust and Quality in OSS"
* http://www.qualoss.org/ -- another research project
* list of related projects: http://www.qualoss.org/related-projects
* Coccinelle http://coccinelle.lip6.fr/

==Related Ideas==

* Make it a best practice that ITPs should be supported by 2 DDs.

* Should Debian have peer review? Every upload needs to peer reviewed by another
  DM/DD? (If every package would use GIT that could be done with gerrit... just
  dreaming :-)
  - Something on top of patch-tracker.debian.org?

* Isn't it a wonderful area for master thesis and PhDs?

* Protocols for removing packages

* The idea of "upload certificates" I mentioned at the end is further discussed here: http://madduck.net/blog/2005.08.11:rcs-uploads/. The GPG concept of "notations" might be useful here. Rhonda suggested to check if DAK can handle multi-signature uploads. I would be happy to discuss this further with anyone interested. I do not remember with whom I already worked on this idea. There is no actual code yet.

=== Reviews in Debian ===

* Have reviews (required/optional) between the upstream->testing migration