Freely-accessible or freely-licensed?

There are actually two issues here, the first being that members of these communities want access to research about themselves without having to pay the average $20-$30 an article.  While important, this also overshadows a more fundamental concern: communities like Wikipedia, Apache, Creative Commons, and OLPC were founded on the idea of providing free and open software, hardware, or educational content to the world.   The Wikimedia Foundation’s mission statement is “to empower and engage people around the world to collect and develop educational content under a free license or in the public domain.”  That is pretty clear-cut, and those of us with obligations to both our own academic community and the Wikipedia community are having more and more problems with negotiating those competing tensions.

In a sense, this is related to how the major ethical dilemma with 19th and early 20th century anthropologists wasn’t about giving ‘their natives’ a copy of their manuscripts. Rather, it was that most anthropologists were participating in systems of colonialism, which were in direct opposition to the interests of the people they studied.  Now, I am in no way arguing that the same kind of power relation exists between academics who study Wikipedians and the Wikipedian community, or that the issue open educational sources is on the same ethical level as colonialism.   As an aside, contemporary anthropologists have documented this shift from ‘studying down’ to ‘studying up’, although I would say that most academics who research open communities like Wikipedia are now ‘studying across’ — but that interesting subject is for another blog post.   But I bring it up because unlike with the Trobriand Islanders, the communities that we study are now beginning to articulate their concerns with how we perform and publish our research, and it is something that we need to listen to.

So to return to the core issue at hand: why is the Wikipedian community (and the Wikimedia Foundation) supporting research that will be copyrighted and bound up in publications which further support an intellectual property regime they clearly stand against?   And what does it mean for us as academic researchers to give back to the communities we study?   It obviously goes beyond being willing to send a copy of a PDF to an interested Wikipedian over e-mail, or even hosting a freely-accessible copy of our copyrighted PDFs on our websites (which many of us do, even when we’re not supposed to). For those of us studying Wikipedia, Creative Commons, Scratch, or a number of open content creation communities, it means releasing our research under a Creative Commons license, as this has become the standard for releasing everything other than code.

Now, the moment I say this, all the academics breathe a heavy sigh, knowing that such a request is impossible, given the current academic system in which we are entrenched. Even the Journal of Computer Mediated Communication, one of the few top-tier open access journals in the social sciences, is copyrighted by the publisher. Some academic superstars like Lawrence Lessig have been able to get their books published from a university press while still being released under a CC license, but not all of us are Lawrence Lessig. Especially for graduate students and junior faculty, who are desperately trying to get their research published anywhere, when the paper finally gets accepted and that copyright assignment form comes in your inbox, the last thing you want to do is start a losing battle over CC-BY-SAing your paper. However, I do have to give a shoutout to Joseph Reagle, who spent a massive amount of effort getting MIT Press to let him publish his book on Wikipedia under a CC license (although with a number of restrictions), but it is unclear the extent to which this will continue in the future.

A partial solution: freely-licensed figures, ‘used with permission’ in copyrighted research papers

So now I finally get to the solution that this blog post was supposed to be entirely about. We academics who study open content communities have an obligation to release our research under free licenses. This does not mean that we have to release our research papers under CC-BY-SA, which is all but impossible for most of us. What it means is that we must release our findings, results, and conclusions under such licenses, and thanks to how copyright works, we can do this through the existing system. Conclusions and abstracts are easy: we just re-write them. We should actually be in the habit of re-writing our densely-worded abstracts and conclusions under a more succinct and human-readable for the communities we study anyway.

However, there is also a way to do this with figures, charts, and graphs. This idea came to me when I saw a copyrighted article in the ACM library (from the Association for Computing Machinery, where a significant amount of Wikipedia research is published) which used a photo someone else took “with permission.” This kind of thing happens regularly enough for the ACM to have a rather sane policy on it: “The author’s copyright transfer applies only to the work as a whole, and not to any embedded objects owned by third parties. An author who embeds an object, such as an art image that is copyrighted by a third party, must obtain that party’s permission to include the object, with the understanding that the entire work may be distributed as a unit in any medium.” I haven’t checked any other publication houses, but I’ve seen this kind of situation happen in so many different books and papers that it could provide a nice loophole in for most of academia.

For most research on Wikipedia, the figures, charts, and graphs are the most interesting aspects of the research, and these can be released under a CC-BY or CC-BY-SA license, and then used with permission in an ACM article. The ACM’s main concern is that they need authors to assign copyright to them in order to make sure publication goes smoothly, and as long as the ‘original author’ of the image is completely fine with having the image in the work and published by the ACM, everyone is happy. I’m no lawyer, but I think this would work with releasing figures, charts, and graphs, even though the copyright policy only qualifies the legal phrase with an example of art images copyrighted by third parties. This doesn’t work as well with many forms of qualitative research, such as historical or interview-based research in which the goal is to elaborate on specific case studies. Still, figures and conceptual diagrams are also useful in those kinds of papers, and can be added to an alternative documentation of a research project, which is possibly co-extensive with but not identical to the research paper.

I’ve actually been putting my charts and graphs up on Wikimedia Commons for quite some time (you can check them all out on my user gallery), even before I realized that copyright was even an issue.   These figures are present in my published papers, many of which are copyrighted by the ACM.  Thankfully, it turns out that this is actually compatible copyright-wise, but this is only solid because I uploaded them to Commons before assigning copyright to the ACM.  It is less clear if someone can retroactively release such images.

But that issue aside, my graphs and charts can live in both worlds, serving members of both communities.  For my quantitative research, these graphs contain my core findings about the rise of bots and assisted editing tools, for example. I have yet to document my previous research projects in a way that would be helpful to others.  More on that in the section below, but I think that even just uploading figures to Commons is a good start.  And it is incredibly painless, especially given that uploading to Commons is a lot easier now than it has been in the past.

Research documentation on Meta-Wiki

Documentation of research projects could take place quite nicely in a new Research: namespace that some great people at the Wikimedia Foundation have provided to document planed, current, and past research projects on Meta-Wiki, the wiki that is used to coordinate many tasks which are common to all language versions of Wikipedia, as well as projects like Wikisource or Wiktonary. You can see a very rough example of one of these that I am working on with as part of my summer research  fellowship with the Wikimedia Foundation: an incomplete but still interesting study of new users that fellow-Fellow Jonathan Morgan and I are doing.

The documentation page is not written like an academic article, although it does give Wikipedians and researchers alike something that is arguably more important. It gives information necessary to replicate the study, for example, how we sampled for new users and what coding schema we used to track new user participation in community spaces. It also contains a few sentences about the motivation of the study, and a few sentences about each of the results. And critically, it contains the graphs which clearly indicate that since 2004, fewer users are participating in community spaces in their first thirty days of joining the project.  If I wanted to write this up into an academic article (which I do plan to), I can do so in such a way that is both suitable for the ACM or another academic publisher, while keeping all the existing content on the documentation page freely-licensed.

Now, to be on the safe side, it may be wise to release these graphs under a CC-BY license instead of a CC-BY-SA one, because the Share Alike requirement might require some other researcher to release an entire academic paper under a CC-BY-SA license if they use one of my CC-BY-SA figures.   However, I do not think this is the case, because as I am the original copyright holder, I can choose to give permission to using images in my own academic papers.   This is a common misconception with Share Alike and CC licenses in general — while I can never revoke my license once I make it, I am not bound by those terms in my own work, and can release the image under as many free and non-free licenses as I choose.   For example, if it is entirely my own image that I license with CC-BY-SA, I do not have to release every work that builds on it under CC-BY-SA, just as I can license the work for commercial use even if I choose a CC license that prohibits commercial use.

Research isn’t a paper

In all, I think that many of the seemingly-intractable problems stem from the false assumption that research projects are entirely encapsulated in a series of papers, and so the demand to ‘freely license your research’ is heard as ‘freely license your papers’.  However, academics already think of research projects as these long processes which spawn multiple papers, and so there is no reason why a research project could not also spawn a freely-licensed documentation space which does not prohibit the publishing of research papers.  Certainly there are many aspects of research papers which would not be included, and there is a risk that these documentation spaces would be second-class reports which are always incomplete compared to the research paper.  Though it is a bit patronizing to universally assume that community members don’t want that dense theoretical analysis of how distributed cognition flows in the actor-network, I think that a facts, figures, and abstracts version would suffice for most.

Given the current academic systems in which we are currently entrenched, I think that this is a good short-term solution, especially for graduate students and other junior scholars who do not have the political capital to change the way in which existing publication regimes operate.  And who knows, perhaps by creating alternative, freely-licensed spaces for documenting research, these publications will recognize the need to make research, though not necessarily research papers, freely accessible and open to all.

Abstract: In this paper, we examine the social roles of software tools in the English-language Wikipedia, specifically focusing on autonomous editing programs and assisted editing tools. This qualitative research builds on recent research in which we quantitatively demonstrate the growing prevalence of such software in recent years. Using trace ethnography, we show how these often-unofficial technologies have fundamentally transformed the nature of editing and administration in Wikipedia. Specifically, we analyze „vandal fighting‟ as an epistemic process of distributed cognition, highlighting the role of non-human actors in enabling a decentralized activity of collective intelligence. In all, this case shows that software programs are used for more than enforcing policies and standards. These tools enable coordinated yet decentralized action, independent of the specific norms currently in force.

Download the full paper (PDF, open access from ACM)

This week, I’m presenting a poster at WikiSym 2009 on “The Social Roles of Bots and Assisted Editing Tools.”  Most of the work is distilled from my thesis.

Abstract: This project investigates various software programs as non-human social actors in Wikipedia, arguing that their influence must not be overlooked in research of the on-line encyclopedia project. Using statistical and archival methods, the roles of assisted editing programs and bots are examined. First, the proportion of edits made by these non-human actors is significantly more than previously described in earlier research. Second, these actors have moved into new spaces, changing not just the practice of article writing and reviewing, but also administrative work.

Download the Poster (PDF)

Download the Extended Abstract (PDF)

And if you are interested in this topic, check out the full paper, The Work of Sustaining Order in Wikipedia: The Banning of a Vandal.

Jimmy Wales speaking at the conference keynote, by GreenReaper, CC BY-SA 3.0

A few months ago, I had the pleasure of presenting at the first (hopefully annual) WikiConference New York, sponsored by the Wikimedia New York City chapter with assistance from Free Culture @ NYU and the Information Law Institute at NYU’s law school. I know that I am atrociously late in writing this post, but I’m not really writing it for the Wikipedians out there. Rather, the WikiConference was an interesting experiment that seemed to apply Wikipedia’s philosophy towards editing to a conference, resulting in what the organizers called a “modified unconference.”

I had never heard of unconferences before, but they are apparently growing increasingly common in tech/programming circles, especially as precursors or followups to traditional conferences. The idea is that in order to keep administratve costs low, you don’t really organize the conference into pre-determined panels, roundtables, and keynotes. Instead, you have a general theme, a good number of open rooms, and a good number of eager participants, who set the topics of individual sessions for themselves and move from room to room on a fluid, ad-hoc basis.  The only rule is the “rule of two feet” – if you don’t like what is going on in the room you are in, leave and find another one.

The conference organizers apparently decided that this was too anarchistic, and instead opted to have a limited number of traditional sessions.  I was on one of the structured sessions, presenting my research on bots and assisted editing tools on the “Quality and Governance” panel.  It was also decided that the “open space”  time was to be segmented into blocks of concurrent sessions.  There was going to be a specific agenda for each of the open space sessions, but they were to be determined at the conference, not before; in addition, the process was to be open to anyone who wanted to propose a session.  While it seemed like an odd way to run a conference (and a bit scary seeing blank space dominate the schedule), it worked incredibly well.

Open space board at WikiConference NYC, by me, CC BY-SA 3.0

We had use of five rooms of various sizes, and one of them was dedicated for refreshments and mingling.  Outside of the largest room (which was used for each day’s opening keynote), there were sheets of paper taped to the wall, creating a table for rooms and timeslots.  After the first day’s opening keynote, sheets of paper, tape, and markers were passed around, and anybody could write something down, tape it to the wall under a timeslot/room combination, and that would be part of the initial schedule.

Given that most of us had never participated in this before, there was a good amount of milling around in front of the schedule wall – five minutes in, nobody had put up a single topic for any timeslot.  Feeling compelled to ake some initiative, I asked someone who was going to be on my panel that afternoon how he felt about a topic on macro-level decision making.  Specifically, I was interested in the approval of flagged revisions – the controversial software feature that would require some edits be approved before going live.  He suggested that I make it broader, and simply write “How do we make decisions?”

The open space wall, by Cary Bass, CC BY-SA 2.5

That seemed like a better and broader topic, so I grabbed some paper and one of the markers, wrote it down in my chicken-scratch handwriting, and taped it to the wall under the first timeslot for the second-biggest room.  Shortly after, three other sheets came up, on quite diverse topics: notability standards, libraries and librarians in Wikipedia, and translation/foreign languages.  Some had even put up sheets for other time slots, touching on nineteen issues that touched on just about every topic in and around Wikipedia.

According to the conventions of open space, the person who put the topic up was expected to start the session on time, say a few words to frame the issue, and then wrap things up at the end.  As the session began, I did just that, telling the room that I had originally thought of this as a discussion about the decision-making around large scale issues like flagged revisions.  However, it is probably good that I was not the moderator, because the room quickly got off the topic of macro-level decision-making and moved into the micro.  We ended up talking extensively about the wide variety of decisions that are made every day – whether to keep or delete a potentially unnotable article, to make an editor into an administrator, and more.

NewYorkBrad asking a question, by Sage Ross, CC BY 3.0

While this was not what I originally envisioned for the session, I was glad that the format had allowed such a swift change.  Had I been delegated to craft a speech, panel, discussion, or roundtable in a traditional conference, I probably would have taken it into a direction that most people did not want to go – of the twenty-something open sessions in the two days, nobody proposed a session on flagged revisions.  Unconferences are supposed to be directed by and for the benefit of the participants, and this was certainly the case.  In any case, the discussion on decision-making went rather well, although a moderator did end up emerging because our session ended up being one of the most popular open sessions, filling up the 75-person classroom.

Yet like in Wikipedia, the unconference didn’t simply devolve into a mass populist mob, reaching for the lowest common denominator.  The fact that we had multiple rooms, a couple of them small conference rooms, meant that less popular topics got their fair share of space.  One open session that I found interesting was on systemic bias – the fact that Wikipedia tends to implicitly favor certain topics, styles, or stances because of the demographic makeup of its contributors.  This tends to not be that popular of a topic, and only a handful of us showed up to discuss this (in my opinion) quite important issue.  However, this resulted in a very thought-provoking discussion among the five of us – that’s about three percent of the conference – who felt a need to identify, theorize, and fix Wikipedia’s systemic biases.

Open session: editing an article for the Wikipedia Signpost

Taken by GreenReaper, CC BY-SA 3.0

Another strength of the open unconference is its radical flexibility.  On the second day, the question/answer session in opening keynote speech turned into a strong debate among a few of the participants.  Because this stops others from asking questions, the typical move at conferences is to stop the debate and pledge to continue it later.  I’ve seen it happen at many conferences, but due to the rigid structure of most conferences, the continuing discussion rarely happens.  Yet in this case, the keynote speech was to be followed by open space sessions.  Realizing that there was an empty slot avaliable in one of the small rooms, the debate that emerged in the keynote Q/A was instantly given its own session.

We also had sets of lightning talks, which were presented in a keynote style.  For those of you who don’t know, lighting talks are short 3-7 minute presentations that anyone can give on the fly.  While lightning talks are held in many conferences I have been to, they tend to be pushed to the background.  Like poster sessions, lightning talks usually take place during established break periods (like lunch), or during other sessions.  This means that the only people who view them are other lightning talkers.  In our case, the lightning talks were after the lunch hour and when no other sessions were being held.  This way, I feel that the presenters got a much broader audience.

Andrew Gradman giving a lightning talk, by Sage Ross, CC BY-SA 3.0

In all, I think that the open unconference was a great success.  However, I don’t think that the “open space” model is adequate on its own – which is why I was glad that there were a limited number of keynotes and pre-arranged panels.  I was on one of the panels (discussing “Quality and Governance”), and got to give a standard 15 minute structured conference presentation, as did my fellow panelists.  I feel that that format is valuble, because I don’t think my research findings on bots and assisted editing tools (or any research findings, for that matter) could have been presented in an open space session or a lightning talk.  The two kinds of sessions are meant to facilitate two different kinds of activities: structured panels and keynotes frame discussions, while the open spaces let participants take it in any way they desire.  For example, I was very excited when the last open session of the conference turned into a user-driven showcase of assisted editing tools – completely unprovoked by myself, I promise.  Another session (one of my favorite) was a workshop in which all the participants worked collectively on writing a news article about the conference for Wikipedia’s community newspaper, the Wikipedia Signpost.

I’m not sure if these kinds of activities would have happened at a more traditional conference – and if they did, they would have probably required a lot more planning. One thing is certain though: the cost of the conference, which was the main reason for the unconference movement, was practically nil. It was completely run by volunteers, and only expenses were refreshments and food.

We throw around the words “collective intelligence” and “wisdom of the crowds” quite a bit to describe “Web 2.0″ sites like Wikipedia, but we hardly define what we mean when we use any of those terms, which is why they largely remain scare-quoted. Because of this, the door has been left wide open for scientists and journalistic defenders of science to critique Wikipedia and other social media sites as being relativist, collectivist mobs who can do no more than aggregate the baseline opinion of what the masses perceive to be Truth. While Wikipedia does have epistemic standards, the open question is how such an epistemology can be operationalized and enforced. To answer such a question, I examine Wikipedia in light of a distinction between an infrastructure of knowing (everything required to evaluate a statement as true/false) and an infrastructure of knowledge production (everything required to bring forth new statements with claims to truth/falsity). While the Wikipedian epistemology on the encyclopedic level is purely evaluative, refusing to publish original research and instead relying on reliable sources, this process is made possible by a non-encyclopedic form of knowledge production.

In short, in order for there to exist an infrastructure of knowing such that the evaluation of encyclopedia articles becomes possible, there must exist an infrastructure of knowledge production to generate and evaluate claims regarding the acts of editing. These include statements like “this edit is vandalism and needs to be reverted” or “this user is disruptive and needs to be blocked” – which require their own epistemic order for evaluation. Taking a cue from laboratory studies of scientific practice, I detail the way in which epistemic standards are “black boxed” into material technologies. In the same way that a mass spectrometer is the reification of dozens of now-unproblematic theories from physics, chemistry, and mathematics, so do various technological programs used by self-described “vandal fighters” reify Wikipedia’s epistemic standards. Similarly, in the same way that various technologies had to be developed to allow experimental science to trump philosophical reasoning (like laboratory reports, which made experimental findings circulatable), so have various technologies been developed that make Wikipedia’s mechanisms of epistemic verification and enforcement possible.

By detailing all the human and non-human actors at work in the banning of a vandal, I show how a group of seemingly-disconnected editors contributed to a process of knowledge production necessary for the enforcement of epistemic standards. In this way, collective intelligence was made possible in Wikipedia, but not because of a mystical or anarchistic wisdom of crowds. Instead, these encyclopedic epistemic standards were able to be enforced because various human and non-human actors were constantly working to hold together an infrastructure of non-encyclopedic knowledge production.

Link: Working Within Wikipedia: Infrastructures of Knowing and Knowledge Production (PDF, 901 KB)