Home

Final report

Slides/Presentations

Workshop

Press coverage

related research

people

 

Free/Libre and Open Source Software:
Survey and Study


FLOSS

OBJECTIVES
In recent years, there has been a significant increase in the productive time spent by citizens of an information and knowledge-based society that does not lend itself to easy measurement in monetary terms. The Internet and related technologies are by far the leading cause of this phenomenon. It enables the formation of distributed, collaborative networks of individuals engaged in the task of cooperatively creating and generating value while governed by a loose and informal structure. By creating an environment of "value-in-participation" this technology encourages, or has encouraged, the participation of individuals in such collaborative networks without the explicit transfer of money among network participants (Ghosh, 1998) (1).

The result of such forms of spending time is that individuals may be using quantifiable resources - their time, especially time on-line, which can be measured in various ways - to generate economically valuable goods and services which are far less quantifiable, as they are frequently provided to others within the network or outside it without explicit and direct (hence measurable) monetary payment. Certainly there are clear implications of such value generation in monetary terms, many often even measurable by the individuals concerned, in terms of indirect income realised through the conversion of reputation capital or goodwill generated for related, or subsidiary, services. Nevertheless accurate or even approximate appreciation of the primary services provided without the use of money through collaborative networks is almost impossible, in a direct sense, through the usual methods of monetary quantification - due to the absence (or relatively low presence) of money in the system.

Unfortunately the consequent of this, given that most models and techniques for economic evaluation and measurement require the use of money, is that the activity of such non-monetary economic networks is left unmeasured, at least in any significant and useful quantifiable sense. Although there are studies and models for quantitative analysis of non-monetary values (e.g. the measurement of knowledge) in an economy they tend to be useful primarily in judging the influence of such values, e.g. knowledge, within organisations, markets or other social structures for which the forms of measurement are clearly dominated by monetary indicators. Measurement is far more complex and ambiguous in a context where the essential and primary economic activity - the generation of value through collaborative networks - is unusual in its avoidance of the use of money as mode of exchange.

The best known example of such a context is the Open Source/Free Software phenomenon. The phenomenon itself - the development of software through collaborative, informal networks of professional or amateur programmers, and the networked distribution making it available to developers and end-users free of charge - has been widely and well documented. Editorials and policy papers have been written on the impact of the OS/F software movement on the computer industry, business in general and the economy at large. However, few models have been developed to explain why or how the system works, to rationally explain the functioning of collaborative networks without primary dependence on money.

Speculation is common regarding the monetary worth of such collaborative development, speculation which translates into widely fluctuating share prices for the companies that have devoted much of their business plans to the OS/F software philosophy. But hard data on the monetary value generated by this phenomenon is almost non-existent. Indeed, hard data on any aspect of the OS/FS phenomenon is rare - even the actual number of developers or software involoved was a matter of guesswork until the Orbiten Free Software Survey carried out this June (Ghosh and Ved Prakash, 2000) (2).

This project aims to remedy this lack of information by starting at the very beginning: by conducting a survey to generate a unique base of primary data on OS/FS usage and development; identifying indicators to measure value creation and dissemination in the OS/FS arena; identifying business models based on these indicators; identifying the impact of and recommending changes in government policy and regulatory environments with regards to OS/FS; finally, development of a base for extending these to the broader economic measurement of non-monetary and trans-monetary activity (i.e. where monetary reasons may be secondary rather than primary, or conditional on other, non-monetary motives) in the information society, beyond the domain of OS/FS.

The specific features of this project are:

  1. The collation of a base of hard data (until now no such data exist) on the importance and role of OS/F software in today's economies and an impact assessment for policy and decision-making.
    This task will be carried out through the execution of a comprehensive survey of likely user organizations and developers of OS/F software, with detailed interviews of a selected representative sample/panel. The expected survey findings related to this task would include: measured dependence of organisations on OS/F software; displacement value of OS/F software; licensing and dissemination advantages/disadvantages of OS/F software; impact of specific policies and regulation on OS/F software.

  2. The development of indicators for the measurement of value creation within the OS/F software communities, especially in order to identify the distribution patterns of contribution within OS/F software communities and projects (3).
    This task will be performed through the survey, and in addition through scanning a large base of OS/F software source code (following the precedent of the Orbiten Survey). This would also involve the development of models and analytical tools (including software) to measure and evaluate contribution of value in OS/F software communities, specifically in order to find techniques to best allocate resources in distributed software development efforts.

  3. The measurement of contributions and identification of dependence on such contributions as are provided by user organisations OS/F software, including government/international institutions, on the developer community and on project development at large.

  4. The evaluation and identification of business models and best practices in the OS/F software community, especially the transition to and from commercial software operations.

  5. Creating and implementing a specific plan for the broad dissemination of the results of this project to four different types of audiences: the government, policy-making and regulator community; the statistical community; the business community; and the OS/FS developer community itself.


B.6 PROJECT WORKPLAN

INTRODUCTION
A key concern throughout this project, which has a significant bearing on the eventual methodology of the survey as well as other implementation techniques such as development and identification of indicators, is the issue of non-monetary vs. monetary measurement. While the economic impact of non-monetary value-generating activities clearly requires the translation or appreciation of such generated value into monetary terms, which are clearly and discretely quantifiable, this project makes the assumption that it is more useful to find a model of quantifiable measures for non-monetary value within its own context before attempting to translate such value, perhaps poorly measured, into monetary equivalence.

Although monetary reasons may indeed be behind decisions taken by members of the OS/FS community, and are most likely behind decisions taken by user organizations (for example, companies that use Linux rather than a commercial operating system are certainly influenced by its lack of a price-tag), they may not be immediately quantifiable. In other cases, such as the dynamics and interactions between groups of developers, monetary indicators may have little or no apparent bearing on decisions taken, e.g. to contribute to a specific software development project. Moreover, as one of the overall objectives of this project is to highlight or identify indicators that may be useful as a basis for examination and modeling of non-monetary/trans-monetary economic activities beyond the domain of OS/FS users and developers, it is attempted to aim towards a survey and analysis methodology that can provide measurement indicators even without quantifiable monetary inputs (which survey respondents may not be able to provide).

Such a system of measurement should be able to determine, or approximate within a model, relative value perceptions for different goods and services across varying collaborative network communities; detect levels of concentration of value production and use; monitor the interdependence - the exchange and trade - between economic network communities; and identify points of dependence, intersection and interaction between the non-monetary value in digital network communities and the domain of monetary value compensation in the organisations and environments hospitable to them.

As quantitative non-monetary measures may vastly differ based on the type and domain of activity being examined, it is useful to start with a subject that has the following attributes:

  • discreteness, so that value generated can be quantified;
  • documentation, in order to identify and locate transactions and participation as clearly as possible;
  • self-interest, in that participants in such activity can be identified and are likely to be willing to provide thedegree of cooperation with any measurement/study that may be required or helpful;
  • monetary equivalence, where activity or value generated finds parallels in the monetary economy allowing for approximations to be made in terms of monetary value of generated products and services.

Furthermore, it is practical to choose a domain where the impact of non-monetary activity on the economy at large is significant, in terms of business interests as well as government policy and decision-making. Open source/free software fits all these criteria perfectly. Software is discrete, clearly broken into applications, packages, lines of code; the participating community is less so, but individual names of participants can be identified, as can project/group membership and to a large extent software (i.e. output) authorship, "ownership" and contribution.

OS/F Software is well documented - at the bare minimum, software is in itself documentation, with dependencies between applications/packages and the implied dependencies between the respective project member communities a crucial part of source code. Software is also documented in terms of author credits, though this is less reliable data. Finally, OS/F software communities work through their collaborative interaction in networks usually implemented as mailing lists or other on-line discussion groups - which is documentation, by definition - and these resources are usually available to the public, or may be accessible for research/study purposes.

The OS/F software community has a huge self-interest in measurement; members of the developer (and even user) community keenly follow their impact on the wider economy as shown in the form of media coverage, take-up by business interests and government, etc. Anecdotal evidence is ample: when the first ever survey of OS/F software contribution and authorship was released (Ghosh & Ved Prakash, 2000) the web site was deluged with over 100,000 visits a day from interested members of the OS/FS community worldwide. That several suggestions were offered on improving the accuracy of the survey demonstrates the willingness of community members to participate in research activities that better document, measure and evaluate their efforts.

Monetary equivalence is also more clearly apparent at first sight in the OS/F software area. Software is commercial; commercial equivalents of most OS/F software packages are widely available with known monetary values (i.e. price tags). Software programmers form a single labour market that is increasingly global in scope, with the same actors floating between the OS/F and commercial software communities, providing a ready measure for their development effort and inputs in terms of time and/or lines of code, which have well-known market values in monetary terms (e.g. euro per person-hour for Perl programming, etc). Monetary equivalence is a complex if useful system of measurement, and its ease of application in the domain of OS/F software relative to other possible non-monetary economic domains is of clear benefit. (Valuing the Linux operating system in the following way: number of copies in use × price of equivalent product e.g. Windows 2000 may be debatable - if Linux was a priced product it might have fewer copies in use, let alone support an unusual distribution and development model - but clearly useful as a start.) Moreover, it may be possible to draw monetary measures of software that although not within the original definition (4) of "open source" or "free software" is provided (by commercial developers) free of charge, such as the e-mail application Eudora: at least for practical reasons, as users/survey respondents are not necessarily aware of the technical and legal distinctions between classes of software when it does not cost them any money, it is possible to include monetary measures of such commercial software by treating them as loss-leaders and/or calculating cost on the basis of advertiser revenue (for advertiser-supported software).

Finally, in addition to the above criteria, there are two major advantages to building a model of non-monetary/trans-monetary economic measures for the OS/F software community. First, experience in measuring the OS/F software community, especially the tools and models developed may be useful or scale to the measurement of broader, less clearly defined but obviously economically valuable activities e.g. citizen participation in subject-specific discussion groups on the Internet, where communities have created valuable dynamic resources ranging in topic from computer programming to neurosciences. Second, OS/F software has a large impact on business, both in the immediate, practical terms of utility of such software and such "non-pricing" models, as well as in terms of alternative creative ideas for business structure models themselves. Similarly, OS/F software has a large impact on the effectiveness of government policy (and in turn, government decisions can have a large impact on the effectiveness of the OS/F software development model) - ranging from the incentives to shift more of business rapidly onto the sphere of e-commerce, to the simple cost advantages of OS/F software in implementing on-line assisted schemes for education in industrial and developing countries.


PROJECT OVERVIEW AND WORKPLAN OUTLINE
The workplan is organised in two main analytical streams co-dependent on the third stream, which takes the form of a multi-part market survey. The first stream (work package 1) involves the creation of a preliminary working model for the OS/FS community, in particular the identification of indicators that could usefully provide measures of OS/FS economic activity, as well as the determination of the basis for broader indicators beyond the OS/FS community. Based on the inputs from this, the survey stream (work package 2 and 3) will gather the primary data through questionnaires and interviews with user and developer organisations and companies throughout Europe, as well as the analysis of a representative sample of software source code. The outputs of the survey feed back into the first stream, to validate the effectiveness of the indicators chosen. The second analytical stream (work packages 4 to 7) will involve the analysis of the survey results from user and developer perspectives, identification of best practices, and assessment of policy and regulatory implications and impact, taking as inputs the specific data resulting from the survey. Finally, the dissemination package (work package 8) will distribute the outputs of the whole project to targeted groups as defined.

1. Development of indicators and preliminary survey model

This work package will result in the identification and development of indicators for non-monetary and trans-monetary economic activity, specifically for the domain of OS/F software but with the perspective of broadbasing the indicators eventually.

The purpose of indicators identified and/or developed during this work package is: to determine what the survey (work package 2) should cover; to assess the utility and reliability of measurement methods and indicators; to determine the applicability of such measures to areas beyond OS/F software communities; to assess the practicalities of such forms of measurement of non-monetary or trans-monetary economic activity as compared with conventional monetary measurement.

The last two points, especially, will depend on feedback from the survey stream (work packages 2 and 3) before final completion of this work package.

The identification and development of indicators in this work package will also determine the precise model and methodology for the survey stream (work packages 2 and 3) taking into account the relative usefulness of different indicators versus the costs and complexity of acquiring data through the survey in a way practical for the scope of this project.

2. Survey

This work package contains the main portion of data-gathering involved in this project. This would survey likely user organisations and developer communities of OS/F software, with detailed interviews of a representative sample of selected users/developers.

The survey findings would be expected to measure, for example: dependence on OS/F software; displacement value of OS/F software - i.e. direct savings as a result of non-use of commercial software; indirect cost benefits of OS/F software; licensing and dissemination advantages of OS/F software - i.e. the increased ease of rapid and widespread of such software as a direct result of flexible licensing and the lack of contract/purchase requirements. Organisations will be asked to identify the cost transfer to OS/FS development that they make in the following ways: directly employing OS/FS developers as programmers or consultants; hiring programmers thereby indirectly paying for their OS/FS development in "free time"; publicising or providing opportunities and contexts for publicity for OS/FS development, etc.

The survey would also include questions on business models and government/ regulatory policies related work packages 6 & 7.

The methodology of the survey would be tailored to the preliminary survey model evolved through the identification of relevant indicators in work package 1. However, the survey is expected to be in two or more parts/phases, as the sample group is diverse. There may also be inputs to the survey questionnaire/interview based on the requirements of workpackages 6 & 7.

3. Software source code analysis

This work package would be draw on existing work undertaken by the Orbiten Free Software Survey (Ghosh & Ved Prakash, 2000), which has scanned OS/F software source code in order to identify authorship and concentration of contribution. Several useful indicators will be determined actual data available on specific OS/F software development projects. Where feasible, software tools will be developed / existing tools (e.g. free software developed by Orbiten for the Free Software Survey) updated in order to analyse source code for new data. Measured indicators will help identify: concentration of development within distributed efforts (concentration in "wealth"); concentration of value flowing to users or specific projects ("dependence") from other projects; "mobility" or the dynamic measure of the exchange or movement of developers/authors between projects and distributed development efforts over time; dependencies between specific distributed development projects, which can highlight the relative importance or "trade exchange rate" between multiple OS/FS projects in linked distributed development efforts.

The methodology of this software source would be tailored to the preliminary survey model evolved through the identification of relevant indicators in work package 1.

4. User-level analysis of survey

This work package involves the analysis of the survey (work package 2) from the point of view of users of free software. It will analyse the results and indicators specific to benefits achieved by users of OS/FS software, following the measurement indicators listed above under work package 2 (e.g. cost savings of OS/FS, support and updates through OS/FS, etc).
As a result of the survey of user organizations, this task involves identifying, for example, measured dependence on OS/F software; displacement value of OS/F software - i.e. direct savings as a result of non-use of commercial software; indirect cost benefits of OS/F software; licensing and dissemination advantages of OS/F software - i.e. the increased ease of rapid and widespread of such software as a direct result of flexible licensing and the lack of contract/purchase requirements. It will result in a working paper which will later be integrated during work package 6.

5. Developer-level analysis of survey

This work package involves the analysis of the survey (work package 2) from the point of view of developers of free software. It will analyse the results and indicators specific to benefits received by developers of OS/FS software, following the measurement indicators listed above under work package 2 (e.g. payments for service and support, indirect payments through employment of developers as consultants etc). As a result of the survey of user organisations, this task involves identifying direct or indirect "subsidies" to OS/F software development efforts. Survey results will be analysed to identify the cost transfer to OS/FS development that organisations make in, for example, the following ways: directly employing OS/FS developers as programmers/consultants; hiring programmers thereby indirectly paying for their OS/FS development in "free time"; publicising or providing opportunities and contexts for publicity for OS/FS development, etc. This workpackage will result in a working paper which will later be integrated during work package 6.

6. Best practices and software business models

This work package involves the evaluation and identification of business models and best practices in the OS/F software community, especially the transition to and from commercial software operations. This will be based on the survey stream (work packages 2 and 3). Further, this workpackage will draw on a review of literature, and interviews/interaction with OS/FS and commercial software developers and a panel of experts in the field.

One focus will be identifying and modelling the process whereby OS/FS community members can translate their efforts into monetary value through the provision of commercial services, as well as the process whereby commercial software companies and other organisations can translate their proprietary products into additional, possibly non-monetary value through their participation in the OS/F software community. Development business models, i.e. models where the prime evaluation criterion is the successful and efficient distributed development of OS/F software rather than specific revenue objectives, are also analysed and best practices identified as part of this task. This work package would also serve the purpose of integrating the outputs of the twin survey-analysis work packages (4 & 5) into a single study.

7. Policy impact and recommendations

This work package will analyse and assess the impact of and dependence on government policy decisions, whether existing or potential on distributed OS/F software development. This will be based on the survey stream (work packages 2 and 3), literature review, and interviews/interaction with OS/FS and commercial software developers, government and other policy-making bodies.

The focus will be two-fold: first, the impact of existing government policies on the growth of OS/FS communities and take-up of OS/F software by business and society as a whole. The impact of new policies to encourage OS/FS will be examined, as well as existing policies that discourage its growth. Based on the survey (work package 2) and further analysis, recommendations will be made with respect to the policy/regulatory environment. A further process in this task would be to identify areas of public policy wherein the benefits of and dependencies on OS/F software as found through the survey can be mapped and new areas of efficiency in public policy implementation discovered as a result.

8. Integration/Dissemination package

  • This work package would essentially integrate the outputs of results from previous work packages (in particular work packages 1, 3, 6 & 7) into a final report. However, as an ongoing process of widespread dissemination, one of the key objectives of this project, will start almost at the beginning of the project (i.e. cosynchronous with work package 1). This is intended to target four groups:
  • the policy and decision-making community (including governments, multilateral agencies, regulatory bodies, legislatures and of course the Commission itself), which will be interested in what the indicators suggest about the sources of performance, the impact of OS/FS on society and the economy, and the impact of specific existing and recommended policy measures on distributed development efforts.
  • the statistical community (including statistical departments of government and multilateral bodies as well as academic and research institutes/universities), which will be interested, primarily in the methodological issues underlying the derivation of the specific indicators of this study.
  • the business community, which will be interested in identifying models whereby they can generate commercial value out of OS/F software and/or use the OS/F software approach to augment their existing business strategies.
  • the OS/FS developer community itself, which would greatly appreciate more and specific understanding about its functionings and methods, and could use the project results to improve efficiency in distributed development efforts.

In terms of readiness of project results, the order of dissemination of results will probably be the reverse of the above list. OS/FS developers and possibly the business community will perhaps be interested in results of the survey itself, even before the final report and integration of data. The latter will be the most useful, however, for the policy-making and statistical communities.

Therefore, a web-site for dissemination of project results will be developed towards the start of the project and maintained and publicised throughout its duration. It will provide timely results especially from the survey and software source code analysis (work packages 2 & 3) as well as making available outputs (e.g. working papers) of the other work packages as and when they are prepared.

A final seminar/workshop will be held for participants from these different targeted communities as well as the wider public to help disseminate the project results, at the end of the period of the project.

A final report/book will be the end-result of the integration of all the project elements, incorporating in addition feedback received from the workshop.

References/Survey of literature-->


(1) Ghosh, R. A, "Cooking pot markets: an economic model for the trade in free goods and services on the Internet" in First Monday, Vol. 3 Issue 3, March 1998, http://www.firstmonday.org/issues/issue3_3/ghosh/

(2) Ghosh, R.A. and Ved Prakash, V., Orbiten Free Software Survey, 1st ed., May 2000, http://orbiten.org

(3) For descriptions of the functioning and management of Open Source/Free Software projects see for example Lerner, Josh and Jean Tirole (2000) "The Simple Economics of Open Source," NBER working paper 7600, www.nber.org/papers/w7600 (march); Kollock, P. (1999), The Economies of Online Cooperation: Gifts and Public Goods in Cyberspace. Chapter 7, Communities in Cyberspace, M.A. Smith and P. Kollock eds. London, Routledge; Lakhani, Karim and von Hippel, Eric (May 2000), "How Open Source software works: Free user-to-user assistance", MIT Sloan School of Management Working Paper #4117.

(4) For original definitions of "Open Source" see http://www.opensource.org/osd.html for the original definition of the term "free software", which forms the basis for and preceded the term "open source", see the the definition by the Free Software Foundation at http://www.gnu.ai.mit.edu/philosophy/free-sw.html