Surveys in Debian
This is my page trying to list surveys done by members of the
debian project, which also
gives some pointers to advice
and why I care.
Most of the survey discussions are posted to
and I try to contribute constructive criticism when possible.
I'd welcome suggestions for the contents of this page.
Surveys by debian developers
- bits from the DMs/NMs/AMs?,
- The construction
and deconstruction of the legitimacy of people and ideas in the Free
Software Community - A study about Debian,
(2006-07, results, method, pt and partial en)
- Poll of debian-user readers about the GNU FDL,
(2005-04, results, some method)
- Debian usage survey,
- The debian-women supporters survey,
(2004-07, some results, method)
- Strawpoll on proper usage of @debian.org email address,
(2004-01, results, method)
Surveys on debian, rather than by developers
- Academy of Management Journal, Oct 2007, Vol. 50 Issue 5,
p1079-1106, 28p, Siobhán O'Mahony and Fabrizio Ferraro
by André Felipe Machado)
- Managing Open Source Software as an Integrated Part of Business,
Niklas Vainio (2006-01, results to appear?)
- Political Motives of Developers for Collaboration on GNU/Linux,
(2004-07, results, method, non-free)
- FLOSSPOLS Second Survey,
(2005-03, results to appear?)
- YAQ - Yet Another Questionnaire,
(2004-07, results, method (in German-language dissertation), non-free)
- Short Survey,
(2003-05, in dissertation)
- Who is Doing it,
(2001-07, results 2001-08, method, non-free)
Surveys on free software in general
- Open Source Case Studies, IDABC - Interchange of Data Between Administrations (viewed 2008-03)
- DebianInThePublicSector, WMartinBorgert and TorstenWerner (viewed 2008-03)
- Free software in public administration, beta, Ciarán O'Riardon (viewed 2008-03)
I find it disappointing that non-developer surveys take a lot
longer to publish results, only about half seem to publish at all,
and non-free software terms like CC-nd, -nc and -by
are common for ones which do.
I have not listed projects which don't look like they'll ever publish.
Advice to surveyors
Because I criticise bad surveys,
people expect me to give advice about how to write good surveys.
I'm not sure why: there are more than enough elementary statistics
texts which cover surveying far better than I can.
It is not a subject which is hugely different for debian.
I like "Statistics Explained" by
John Parry Lewis and Alasdair Traill, but I'm not sure how easy it
is to get and it's rather big.
Much easier to obtain is
"Statistics Without Tears: A Primer for Non-Mathematicians"
by Derek Rowntree.
If you want something online, start with
Some Aspects of Study Design
from Gerard E. Dallal's "Little Handbook of Statistical Practice" and follow the link to the full thing when you're ready,
but I strongly suggest getting "Tears" from your local library too.
If you know of a free software work on the subject, please let me know
and I'll promote it here.
A survey is evidence, not proof.
You will probably not convince those who vehemently disagree with you
and how many "floating voters" you convince is correlated with how
good your survey is.
Basically, as with any other surveys, the golden rules are:
- Privately, start the write-up before you start collecting data
- what population you are trying to sample,
- what question(s) you are trying to investigate,
- your method of sampling (sometimes you can do better than a
self-selecting listener opinion poll, while other times you
need to compile a census of all members rather than sampling them),
- the desired strength of outcome (if you can),
- and any prior beliefs you hold about the topic;
- Be as simple, clear and neutral as you can:
in question phrasing and procedure details for a questionnaire,
or in measurement methods for automated data collection,
or so on;
- Be considerate:
Remember that respondants are giving time to your research;
- try to be polite when asking people to participate,
- tell people how and when their responses will be published
(if you're doing interviews, do this at the end, to avoid biasing your data:
some people will tell you what they think you want to hear; but include
it in the mailshot if you're asking people to send back forms),
- accommodate community opinions (so for a survey about debian,
please release the results and analysis as free software),
- test the survey yourself before involving any other people
- and try to follow any applicable guidelines, as far as possible.
- Carry out a pilot survey. Use it to debug the survey,
as well as estimating the sample size you
need to get the desired strength of outcome;
- Carry out the main survey;
- The first summary results should be a simple descriptive compilation
Leave any commentary until the second part, then try to use any
applicable methods to support or debunk your commentary;
- Publish as much as you can,
so others can check, reproduce or extend your findings.
To recap: define, clear, considerate, pilot, survey, summary, publish.
For more specific details, consider this advice from:
Who am I and why I care
I am a debian developer who studied degree-level statistics
for about ten years.
I have been trained to do surveys by public and private sector groups.
I trained others on-and-off for about 5 years.
I'll link an example of my work here when it's online.
These days, I earn a living offering web site visitor analysis,
accessibility checking and market research, along with some programming.
I want to help debian contributors carry out better surveys and
so help manage their debian work better.
The project is large and anecdotes don't always help.
Some think that if you state an opinion, you must be ready
to defend it forever, else it has less value:
I reject that and believe good solid research will make
collections of "ordinary" opinions defensible against this view.
I find it really frustrating to see research projects
surveying debian, but wasting our time.
It is especially annoying when these projects are funded with large amounts
of government money which could have gone to groups related to
FSF Europe, but for politics.
Thanks to Andreas Schuldei for motivating me to work on this.
Many thanks to Enrico Zini for commenting on an early draft and
suggesting some "be considerate" advice.
Thanks to all who commented on the announcement
All errors are still probably mine, though.
You can copy this page under the same terms
my personal site.
17:00, 23 Apr 2008