Abstract: In this paper, we examine the social roles of software tools in the English-language Wikipedia, specifically focusing on autonomous editing programs and assisted editing tools. This qualitative research builds on recent research in which we quantitatively demonstrate the growing prevalence of such software in recent years. Using trace ethnography, we show how these often-unofficial technologies have fundamentally transformed the nature of editing and administration in Wikipedia. Specifically, we analyze „vandal fighting‟ as an epistemic process of distributed cognition, highlighting the role of non-human actors in enabling a decentralized activity of collective intelligence. In all, this case shows that software programs are used for more than enforcing policies and standards. These tools enable coordinated yet decentralized action, independent of the specific norms currently in force.

Download the full paper (PDF, open access from ACM)

This week, I’m presenting a poster at WikiSym 2009 on “The Social Roles of Bots and Assisted Editing Tools.”  Most of the work is distilled from my thesis.

Abstract: This project investigates various software programs as non-human social actors in Wikipedia, arguing that their influence must not be overlooked in research of the on-line encyclopedia project. Using statistical and archival methods, the roles of assisted editing programs and bots are examined. First, the proportion of edits made by these non-human actors is significantly more than previously described in earlier research. Second, these actors have moved into new spaces, changing not just the practice of article writing and reviewing, but also administrative work.

Download the Poster (PDF)

Download the Extended Abstract (PDF)

And if you are interested in this topic, check out the full paper, The Work of Sustaining Order in Wikipedia: The Banning of a Vandal.

Jimmy Wales speaking at the conference keynote, by GreenReaper, CC BY-SA 3.0

A few months ago, I had the pleasure of presenting at the first (hopefully annual) WikiConference New York, sponsored by the Wikimedia New York City chapter with assistance from Free Culture @ NYU and the Information Law Institute at NYU’s law school. I know that I am atrociously late in writing this post, but I’m not really writing it for the Wikipedians out there. Rather, the WikiConference was an interesting experiment that seemed to apply Wikipedia’s philosophy towards editing to a conference, resulting in what the organizers called a “modified unconference.”

I had never heard of unconferences before, but they are apparently growing increasingly common in tech/programming circles, especially as precursors or followups to traditional conferences. The idea is that in order to keep administratve costs low, you don’t really organize the conference into pre-determined panels, roundtables, and keynotes. Instead, you have a general theme, a good number of open rooms, and a good number of eager participants, who set the topics of individual sessions for themselves and move from room to room on a fluid, ad-hoc basis.  The only rule is the “rule of two feet” – if you don’t like what is going on in the room you are in, leave and find another one.

The conference organizers apparently decided that this was too anarchistic, and instead opted to have a limited number of traditional sessions.  I was on one of the structured sessions, presenting my research on bots and assisted editing tools on the “Quality and Governance” panel.  It was also decided that the “open space”  time was to be segmented into blocks of concurrent sessions.  There was going to be a specific agenda for each of the open space sessions, but they were to be determined at the conference, not before; in addition, the process was to be open to anyone who wanted to propose a session.  While it seemed like an odd way to run a conference (and a bit scary seeing blank space dominate the schedule), it worked incredibly well.

Open space board at WikiConference NYC, by me, CC BY-SA 3.0

We had use of five rooms of various sizes, and one of them was dedicated for refreshments and mingling.  Outside of the largest room (which was used for each day’s opening keynote), there were sheets of paper taped to the wall, creating a table for rooms and timeslots.  After the first day’s opening keynote, sheets of paper, tape, and markers were passed around, and anybody could write something down, tape it to the wall under a timeslot/room combination, and that would be part of the initial schedule.

Given that most of us had never participated in this before, there was a good amount of milling around in front of the schedule wall – five minutes in, nobody had put up a single topic for any timeslot.  Feeling compelled to ake some initiative, I asked someone who was going to be on my panel that afternoon how he felt about a topic on macro-level decision making.  Specifically, I was interested in the approval of flagged revisions – the controversial software feature that would require some edits be approved before going live.  He suggested that I make it broader, and simply write “How do we make decisions?”

The open space wall, by Cary Bass, CC BY-SA 2.5

That seemed like a better and broader topic, so I grabbed some paper and one of the markers, wrote it down in my chicken-scratch handwriting, and taped it to the wall under the first timeslot for the second-biggest room.  Shortly after, three other sheets came up, on quite diverse topics: notability standards, libraries and librarians in Wikipedia, and translation/foreign languages.  Some had even put up sheets for other time slots, touching on nineteen issues that touched on just about every topic in and around Wikipedia.

According to the conventions of open space, the person who put the topic up was expected to start the session on time, say a few words to frame the issue, and then wrap things up at the end.  As the session began, I did just that, telling the room that I had originally thought of this as a discussion about the decision-making around large scale issues like flagged revisions.  However, it is probably good that I was not the moderator, because the room quickly got off the topic of macro-level decision-making and moved into the micro.  We ended up talking extensively about the wide variety of decisions that are made every day – whether to keep or delete a potentially unnotable article, to make an editor into an administrator, and more.

NewYorkBrad asking a question, by Sage Ross, CC BY 3.0

While this was not what I originally envisioned for the session, I was glad that the format had allowed such a swift change.  Had I been delegated to craft a speech, panel, discussion, or roundtable in a traditional conference, I probably would have taken it into a direction that most people did not want to go – of the twenty-something open sessions in the two days, nobody proposed a session on flagged revisions.  Unconferences are supposed to be directed by and for the benefit of the participants, and this was certainly the case.  In any case, the discussion on decision-making went rather well, although a moderator did end up emerging because our session ended up being one of the most popular open sessions, filling up the 75-person classroom.

Yet like in Wikipedia, the unconference didn’t simply devolve into a mass populist mob, reaching for the lowest common denominator.  The fact that we had multiple rooms, a couple of them small conference rooms, meant that less popular topics got their fair share of space.  One open session that I found interesting was on systemic bias – the fact that Wikipedia tends to implicitly favor certain topics, styles, or stances because of the demographic makeup of its contributors.  This tends to not be that popular of a topic, and only a handful of us showed up to discuss this (in my opinion) quite important issue.  However, this resulted in a very thought-provoking discussion among the five of us – that’s about three percent of the conference – who felt a need to identify, theorize, and fix Wikipedia’s systemic biases.

Open session: editing an article for the Wikipedia Signpost

Taken by GreenReaper, CC BY-SA 3.0

Another strength of the open unconference is its radical flexibility.  On the second day, the question/answer session in opening keynote speech turned into a strong debate among a few of the participants.  Because this stops others from asking questions, the typical move at conferences is to stop the debate and pledge to continue it later.  I’ve seen it happen at many conferences, but due to the rigid structure of most conferences, the continuing discussion rarely happens.  Yet in this case, the keynote speech was to be followed by open space sessions.  Realizing that there was an empty slot avaliable in one of the small rooms, the debate that emerged in the keynote Q/A was instantly given its own session.

We also had sets of lightning talks, which were presented in a keynote style.  For those of you who don’t know, lighting talks are short 3-7 minute presentations that anyone can give on the fly.  While lightning talks are held in many conferences I have been to, they tend to be pushed to the background.  Like poster sessions, lightning talks usually take place during established break periods (like lunch), or during other sessions.  This means that the only people who view them are other lightning talkers.  In our case, the lightning talks were after the lunch hour and when no other sessions were being held.  This way, I feel that the presenters got a much broader audience.

Andrew Gradman giving a lightning talk, by Sage Ross, CC BY-SA 3.0

In all, I think that the open unconference was a great success.  However, I don’t think that the “open space” model is adequate on its own – which is why I was glad that there were a limited number of keynotes and pre-arranged panels.  I was on one of the panels (discussing “Quality and Governance”), and got to give a standard 15 minute structured conference presentation, as did my fellow panelists.  I feel that that format is valuble, because I don’t think my research findings on bots and assisted editing tools (or any research findings, for that matter) could have been presented in an open space session or a lightning talk.  The two kinds of sessions are meant to facilitate two different kinds of activities: structured panels and keynotes frame discussions, while the open spaces let participants take it in any way they desire.  For example, I was very excited when the last open session of the conference turned into a user-driven showcase of assisted editing tools – completely unprovoked by myself, I promise.  Another session (one of my favorite) was a workshop in which all the participants worked collectively on writing a news article about the conference for Wikipedia’s community newspaper, the Wikipedia Signpost.

I’m not sure if these kinds of activities would have happened at a more traditional conference – and if they did, they would have probably required a lot more planning. One thing is certain though: the cost of the conference, which was the main reason for the unconference movement, was practically nil. It was completely run by volunteers, and only expenses were refreshments and food.

We throw around the words “collective intelligence” and “wisdom of the crowds” quite a bit to describe “Web 2.0″ sites like Wikipedia, but we hardly define what we mean when we use any of those terms, which is why they largely remain scare-quoted. Because of this, the door has been left wide open for scientists and journalistic defenders of science to critique Wikipedia and other social media sites as being relativist, collectivist mobs who can do no more than aggregate the baseline opinion of what the masses perceive to be Truth. While Wikipedia does have epistemic standards, the open question is how such an epistemology can be operationalized and enforced. To answer such a question, I examine Wikipedia in light of a distinction between an infrastructure of knowing (everything required to evaluate a statement as true/false) and an infrastructure of knowledge production (everything required to bring forth new statements with claims to truth/falsity). While the Wikipedian epistemology on the encyclopedic level is purely evaluative, refusing to publish original research and instead relying on reliable sources, this process is made possible by a non-encyclopedic form of knowledge production.

In short, in order for there to exist an infrastructure of knowing such that the evaluation of encyclopedia articles becomes possible, there must exist an infrastructure of knowledge production to generate and evaluate claims regarding the acts of editing. These include statements like “this edit is vandalism and needs to be reverted” or “this user is disruptive and needs to be blocked” – which require their own epistemic order for evaluation. Taking a cue from laboratory studies of scientific practice, I detail the way in which epistemic standards are “black boxed” into material technologies. In the same way that a mass spectrometer is the reification of dozens of now-unproblematic theories from physics, chemistry, and mathematics, so do various technological programs used by self-described “vandal fighters” reify Wikipedia’s epistemic standards. Similarly, in the same way that various technologies had to be developed to allow experimental science to trump philosophical reasoning (like laboratory reports, which made experimental findings circulatable), so have various technologies been developed that make Wikipedia’s mechanisms of epistemic verification and enforcement possible.

By detailing all the human and non-human actors at work in the banning of a vandal, I show how a group of seemingly-disconnected editors contributed to a process of knowledge production necessary for the enforcement of epistemic standards. In this way, collective intelligence was made possible in Wikipedia, but not because of a mystical or anarchistic wisdom of crowds. Instead, these encyclopedic epistemic standards were able to be enforced because various human and non-human actors were constantly working to hold together an infrastructure of non-encyclopedic knowledge production.

Link: Working Within Wikipedia: Infrastructures of Knowing and Knowledge Production (PDF, 901 KB)

These non-encyclopedic wikispaces in and around Wikipedia are used to organize most of the largely invisible work required to maintain and further develop the encyclopedia. In fact, some of the project’s most active pages are not hotly-contested encyclopedia articles, but rather these ‘meta’ pages which are used to make collective consensus decisions about various issues. In maintaining and developing this aspect of the encyclopedia, the Wikipedian community takes advantage of the wiki media to do so in a unique form of digital governance. Social power structures still exist, but the wiki-based nature of the site allows authority to be largely distributed and decentralized, in stark contrast to traditional forms of knowledge production.

However, such a social structure and media use has not always been present in Wikipedia. In the first year of its existence, most of the coordination of invisible maintenance work and resolution of ‘meta’ issues took place almost exclusively on e-mail listservs. I demonstrate that this media use corresponded to a social structure that took founder Jimmy Wales to be the unquestioned leader of the project, in charge of resolving issues when they arose among the small community of editors. Yet as the project grew, this listserv-mediated, “benevolent dictator” governance model did not scale to meet the rapid increase of both individual editors and editorial issues.

In response to various controversies in which the benevolent dictator model led to backlashes from the project’s growing editorial base, I show how pages in the wiki began to be used for a new, distributed form of governance. Instead of a monarchical model tempered by a centralized discussion forum, this model took advantage of features in the wiki media to enable a more direct and participatory system of governance. However, both the wiki media and the governance model proved inadequate and were subsequently refined in response to various issues faced by the project. The result, I show in this historical account, is the current instantiation of authority and media technology in and around Wikipedia, which has evolved significantly since in the project’s seven year history.

Scholars have long theorized how media technologies fundamentally reshapes the way in which we exist both as individuals and as a society. In an age of information overload, the history of Wikipedia’s co-evolving media use and governance model gives us a powerful lesson regarding the way in which the development of social structures and media technologies are fundamentally interrelated in the digital era.

Director of the Library of Alexandria, Dr. Ismail Serageldin gave a keynote speech on the first day of Wikimania 2008 titled, New Paradigms for New Tomorrows.  It was quite thoughtful and inspiring – the man is one of the most amazing individuals I have heard.  He is learned in so many different areas of academic and cultural knowledge, as well as incredibly wise.  I would recommend watching the video of his speech, but if you are pressed for time you can read my notes.

This talk is more somber, as we must mention challenges. The theme for this conference is “changing the shape of wisdom” – yes, that is true, very, very true.

Problems and promises

Globalization – nation-states are becoming less important, people are moving together and faster. It generating great wealth and leaving people behind. Globalization is not seen as beneficial by some, people who feel they are being crushed by interests outside of their control. Population growth is increasingly putting pressure on natural resources. Access to fundamental rights – food, shelter, safety, education – are being denied. Young people are pressing for education, employment. Societal progression – it has happened, among women especially. But there is more to be done. Wars, terrorism, refugees, rebellions, child soldiers – why do we have them now?

What does it cost of equip a soldier vs. a classroom? Books or bombs? That is the question. Napoleon Bonaparte said, “In the long run the sword is always beaten by the spirit” He proved that quote, as his victory was wiped away, his legacy was his system of governance, of education-based government bureaucracy, civil codes, etc.

We need peace – we cannot continue to wage war. We have universal declaration of human rights. In Article 19, everyone has the right to freedom of opinion and expression to seek, receive information from a multiplicity of sources.

Freedom of expression

We must re-affirm the polity of humans that share those ideals. Books will stay. Book banning: censorship contravenes human rights. Information cannot be denied – like standing before a tidal wave. Social pressure is a form of censorship – freedom of expression must be protected.

Freedom of access

Print can sometimes be out of reach – not just out of print but too pricey. “We can provide all information to all people at all times” – our motto at the Library. Free flow of information is important. We’ve had a shortage of books – now with the Internet we have too much information. Like going from a drought to fire hose. We need it just right – not just for children of the rich, but children of the poor.

Sometimes copyright issues can put things out of reach. 70% of works are in copyright and out of print. But copyright is in Universal Declaration of human rights too. My credo: all information to all people at all times. Innovators and authors must benefit, but public must have broad access.

What the BA is doing

Massive amount of archiving about Egypt – ancient documents, modern Egypt documents. Digitizing 5k books / month. Creating a Supercourse for free education of all kinds throughout Egypt. Development gateway – websites of NGOs available in their local language. We want to partner with open source efforts.

The future looks great in terms of technology. The future in terms of society is another story.

In defense of values

We talk about the knowledge-based society, but we must remember that knowledge is more than information. Data -> information -> knowledge -> wisdom. Wise decisions made are different than knowledgeable decisions. No scientific answer to the normative. Knowledge requires we rethink values. Values of science: Truth, honor, creativity and imagination, constructive subversiveness – advance by overthrowing the old, tolerance of engagement, and arbitration of disputes based on rationality, evidence, and discussion.

Requires participation – wiki community has shown this. Power of the civil society forces government to be responsible and capable. Putnam – Making Democracy Work. Civil society thrives on information flows.

Parable of story of the flute and children – adjudicate who gets the flute:

1. I am poor, others are rich – equity
2. I can play the flute, others cannot – utility
3. I made the flute myself – entitlement

No scientific answer to this question.

In defense of youth

Newton, Einstein, Heisenberg, Watson, Turing – they were all young.

To change the shape of wisdom we have to trust our youth, dare to dream, we can do things different. There are forces of Arab society against us, but we can win. Look at Antarctica – a whole continent kept for a bunch of penguins and science, free of econ interests and military bases. Martin Luther King and the civil rights movement, and now Obama is a nominee for President.

It is not impossible to provide all information to all people at all times!

Wikiversity does not limit to a university style education – primary, secondary, and post-secondary education. It also does not offer degrees, no certificates, no titles.

Launched in August 2006. Scope: learning materials, activities, and communities. Major questions still to be addressed: What is learning the wiki way? How is it distinguished from other WMF projects?

Aspects of Wikiversity different from Wikipedia: It includes original research, and that is what I will be talking about. We have flexibility in NPOV. We setup Wikiversity to work out what it means to learn in the Wiki way. We want to say something to the wider world of education.

What is research in the wiki way? Several challenges: editing data – should this be allowed? Should Wikversity host any kind of research, like Nazism? Creationism? We all have our own worldviews, epistemologies, philosophies and Wikiversity challenges them.

Current research: Bloom Clock – Distributed data about what is currently in flower. People add logs to individual plants pages. Also provides a learning community on identifying plants.

Action research based on understanding a context through changing it. Usually collaborative and iterative. This project is about defining and developing Wikiversity. However, it is problematic to define as distinct from what happens anyway on Wikiversity. A Review Board might be in order, but it is possibly in tension with community processes.

The Wikimedia foundation installation is huge:

• 10,000,000,000 views per month
• 50,000 http objects/sec
• Hardware budget: $1.5 million
• Bandwidth costs: 25k/months
• Physical hosting: $10k/month
• X86_64 software, ubuntu with customized packages, RAMdisks

MediaWiki basic goals: allow open, collaborative editing of a large encyclopedia; also be easy to setup for smaller installations. It runs on LAMP, or LAMP on steroids (Squid, Memcached, many other optimizations). Wikipedia used to crash all the time, but it doesn’t now. We can focus on long term.

Recently: we got login unification working. [to thunderous applause], got flagged revisions in open beta [to no applause]. We are working on better mobile support. We’ve been improving localization.

What not to expect next year: flying cars, WYSIWYG (but you may get better template and table tools), peer-to-peer Wikipedia

What to expect: more tech staff, better operations (data center updates and expansions), dumps and backup capacity), and software.

In the software:

• Revision deletion: there is some version of a page we want to delete, but we don’t want to delete other revisions. We have oversight, but that is a hack. Hacks are bad.
• Vandalism and abuse: Globally block an IP address across all WMF projects
• Extension:AbuseFilter – automatic abuse filter for admins
• Threads and comment system
• Wikimedia Commons – especially uploading – will be updated so that “actual humans can use it.”  AfD on commons is “pretty scary” – sometimes I can’t find out why my file has been deleted, and if I can’t find it, well…

Scientific papers that have been written about Wikipedia are interesting, and there are many things to do with quantitative or statistical analysis. I however want to take a Sociological approach and ask: how does Wikipedia organize its members into the project?

Real Utopia is a concept from Eric Wright, a professor of Sociology at Wisconsin-Madison. It puts into place very idealistic places. It is an egalitarianism of many kinds, radical direct participatory democracy, where all are given the conditions necessary to ensure human flourishing. Two real utopias. 1: Participatory budgeting in Porto Alegre, Brazil. They created a system of dual power between residents and municipal assembly. 2: British Citizen’s Assembly: 160 randomly selected individuals charged with creating an electoral system.

Real utopian aspects of Wikipedia: Full and open participation, pragmatic orientation, direct and deliberative, consensus formation, alternative dispute resolution, devolution, non-hierarchical, democratic.

Future challenges: every new, innovative, and exciting project faces challenges when it begins to become bureaucratized and institutionalized. Successful institutions must be highly responsive to their members, that is, democratic. Wales remains an authority of last resort. Should the Wikimedia foundation institute a system of dual power? A volunteer assembly has already been suggested.
Perhaps a randomly selected Wikipedian Assembly, a jury picked from members of the community to resolve disputes? Who owns Wikipedia? We discussed in the Board Panel about selling a project. WMF owns all the technology, but content production – who owns that? The community does.

Question: Are you aware of What Wikipedia is Not:Democracy? Yes, and that is wrong. Consensus is democracy – it is implied.

My only real problem with Edo’s presentation was his answer to the question about WWIN:Democracy, a longstanding policy that is widely characterized as “Wikipedia is not a democracy.”  If that was what the policy said, I would have entirely agreed with Edo – consensus is effectively a democratic form of governance, when all members of a political community are taken into account when determining consensus.  However, this is not what it says.  The policy (taken from an emphatic posting on the Wikipedia mailing list by Jimmy Wales in 2005) says: “Wikipedia is not an experiment in democracy or any other political system.”

This is an entirely different idea, one with a good amount of nuance.  Wikipedia obviously contains many elements of a democracy, but democracy is not what Wikipedia is primarily about.  To rephrase: Wikipedia primarily is a project to create an encyclopedia that will give everyone the sum total of human knowledge in their own language, not primarily a project to adhere to the principles of democracy.  Now, many people will say that democracy is what makes such a project possible, and I will heartily agree.  However, changing how Wikipedia is run must be justified not in terms of democracy, but encyclopedia building.  That is, democracy in Wikipedia is not an end in itself, only a means to building an encyclopedia.

With this in mind, I feel that Edo’s presentation is flawed insofar as it couches policy suggestions in a language foreign to Wikipedia.  However, I must admit I do that too: my senior thesis on Wikipedia’s legal structure concluded that Wikipedian law (whatever that was) contained traces of Continental Law and Common Law in its legal systems, which created conflict and need to be reconsiled.  Talk about a case of wikilawyering.  Every time I feel like busting out Michel Foucault and writing on Wikipedia in that manner, I must remind myself that Wikipedia does not exist to decenter the liberal-democratic humanist subject or problematize existing knowledge/power regimes.  Yes, it might very well do that, but I am wary of anyone who uses any particular theory to claim not what Wikipedia is, but what Wikipedia should be.

When looking over his paper, I feel that he is more interested in “exporting” Wikipedia’s model to other forms than importing the concept of a real utopia into Wikipedia.  I commend him for that.  I also appreciate his call for a “social history of Wikipedia” – a project that I will be undertaking in my thesis next academic year.