Growth

Wikimedia Foundation projects Growth Increasing new editor retention in mid-size wikis. Group: Core Experiences Start: 2018-07-01 Team members: Gergő Tisza (staff software engineer), Benoît Evellin (community relations specialist), Elena Tonkovidova (senior test engineer), Kosta Harlan (staff software engineer), Sergio Gimeno (senior software engineer), Morten Warncke-Wang (staff data scientist), Rita Ho (principal user experience designer), Mariya Shilova (technical program manager) Management: Dennis Mburugu (engineering), Kirsten Stoller (senior product manager) Updates: Current project updates Work with us and involve your community Get updates on your talk page! Cross-departmental Editor Growth Coordination Committee Progress: We post data analysis updates from time to time.

To see updates on these initiatives and the specific projects underneath them, see the Growth team updates page or sign up for our team's newsletter.

• Understanding first day (EditorJourney)
• Personalized first day
  • Welcome survey
  • Engagement emails (potential project)
  • Newcomer homepage
  • Newcomer tasks
  • Structured tasks
    • Add a link
    • Add an image
    • Copyedit
  • User profile (potential project)
  • Mentor dashboard
• Focus on help desk
  • Help panel
• Positive reinforcement
• Community configuration
• Newcomer Experience Projects

Most of these happen via the GrowthExperiments extension. A small part of the analytics work happened within the WikimediaEvents extension.

Deployment table[edit]

The table below is a quick way to see which Growth team initiatives are deployed in which wikis. See the specific project pages for details.

In the table below, wikis in bold (Arabic Wikipedia, Bengali Wikipedia, Czech Wikipedia, Spanish Wikipedia) are our pilot wikis, with dedicated resources.^[1]

All versions of Wikipedia now have the Growth features as the default onboarding process. The list below details the ones where the configuration is not the default one.

Learn more about getting the features.

Wiki	Welcome survey	Help panel	Newcomer homepage	Newcomer tasks	Link recommendation	Image recommendation	Mentor dashboard	New impact module
All Wikipedia not listed below	Done	Done	Done	Done	Varies by wiki. See T304110.	N Not done	Available at all wikis, activated where a list of mentors has been created[2]	N Not done
Arabic Wikipedia	Done	Done	Done	Done		Done		A/B testing (50%)
Bengali Wikipedia	Done	Done	Done	Done		Done		A/B testing (50%)
Czech Wikipedia	Done	Done	Done	Done		Done		A/B testing (50%)
German Wikipedia	A/B testing (80%)	A/B testing (80%)	A/B testing (80%)	Done		N Not done		N Not done
Spanish Wikipedia	Done	Done	Done	Done		Done		A/B testing (50%)
Greek Wikipedia	Done	Done	Done	Done		Done		N Not done
Persian Wikipedia	Done	Done	Done	Done		Done		N Not done
French Wikipedia	Done	Done	Done	Done		Done		N Not done
French Wiktionary	Done	Done	Done	N Not done		N Not done		N Not done
Indonesian Wikipedia	Done	Done	Done	Done		Done		N Not done
Polish Wikipedia	Done	Done	Done	Done		Done		N Not done
Portuguese Wikipedia	Done	Done	Done	Done		Done		N Not done
Romanian Wikipedia	Done	Done	Done	Done		Done		N Not done
Test Wikipedia	Done	Done	Done	Done		Done		N Not done
Turkish Wikipedia	Done	Done	Done	Done		Done		N Not done
Vietnamese Wikipedia	Test (see task T218920 task T325376)	Done	Done	Done		Done		N Not done

The Growth Team formed in July 2018. The team supports the Audiences department's "New Content" program, and, in turn, the Wikimedia Foundation's long-term goal of "Knowledge Equity". Those initiatives are about building strong communities that bring broad and deep content to people all over the world. One way to accomplish this is by engaging new contributors in Wikimedia projects. Unfortunately, very few people who start contributing continue to contribute in the days and weeks after they begin. This is because contributing requires many cultural and technical skills that are difficult to learn. The Growth Team's objective is to address this problem through software changes that help retain new contributors in mid-size Wikimedia projects, starting with Wikipedias.

We are beginning our work with mid-size Wikipedias for a few reasons. Those projects have a lot of potential for growth, but also have challenges that may be helped through software. Much of the motivation for this team's work comes from the New Editor Experiences research project, which was undertaken in 2017. In the research, the Wikimedia Foundation worked with the Czech and Korean Wikipedia communities to understand the challenges that stand in the way of new contributors. Continuing that relationship, we will be testing software changes on a few pilot Wikipedias (see the table) before deploying to other mid-sized Wikipedias. Our software changes will usually fall into one of these categories:

• In-context help: changes to the Wikipedia editing experience to surface helpful guidance around technical and conceptual challenges.
• Human-to-human help: forums and programs that connect editors to each other so that experienced contributors can help newer contributors succeed.
• Task recommendations: suggesting edits that will help newcomers learn wiki skills.

For information about the Growth team that existed from 2013–2015, see Growth/Growth 2014.

For information about the predecessor to this team, see Collaboration.

Past projects by various iterations of the team and volunteer developers:

• AfC improvements
• Map Improvements 2018
• Edit Review Improvements
• Thanks
• Structured Discussions, formerly Flow, discussion and collaboration software. Now in production as opt-in for talk pages, help pages and village pump-style discussion forums, on select wikis, but new requests are currently not applied.
• Notifications (Echo), a redesign of the notifications system in Wikipedia. It was deployed to all Wikimedia wikis in November 2013.
• New Page Triage, redesigning the New Page Patrol experience.
• WikiLove, a tool designed to encourage community health and reduce "bite".
• MoodBar, a tool designed to encourage feedback from new users, with Feedback Dashboard supporting review by experienced members.
• Article Feedback Tool, a tool designed to encourage readers to provide feedback on articles.

As stated in the Audiences department’s 2018-2019 annual plan, the Growth team’s goal is:

Increase retention of new contributors in the target wikis, Czech and Korean Wikipedias.

To determine whether we’re meeting our goal, we have specified metrics and set targets for them:

The first metric, “new contributor retention”, is the main metric we use to measure our impact.  The second metric, “mentor retention”, will be relevant for interventions that involve mentorship. As we focus our work, we will consider whether that second metric is the right fit for the team at this time.

We need to create a more specific definition of “new contributor retention” in order for it to be useful.  See the section below on “What does it mean to ‘retain new contributors’?” for our team’s thinking.

In the future, this section will contain links to the Growth team's reports that track our metrics.

For the purposes of this team, “new contributors” are people who have registered accounts on a wiki project, even if they have not made an edit. To “retain” them means that they make edits and continue to make edits over time. Though we hope that contributors are retained for months and years, we know that they will first have to make it through their first days and weeks. Those first days and weeks are more likely to be the Growth team’s focus.

There are different periods of time over which we can measure retention. One definition might be “second month retention”, meaning a contributor is "retained" if they edit during their first month and also edit during their second month. But another definition might be “second week retention”, meaning a contributor is "retained" if they edit during their first week and also edit during their second week.

As we work, the Growth team will determine which types of retention are most important to track and to increase.  See this page for a formal definition of retention.

Wikimedia projects need to have a constant stream of new contributors in order to become hubs for broad and deep content. New contributors replace experienced contributors who stop editing. They also increase the diversity of the editing community. This stream of new editors allows projects to grow their content while increasing the diversity of their content. But in many Wikimedia projects, the number of active contributors is decreasing or staying the same, and the retention rate of new contributors is also decreasing or staying the same. This issue was studied in the 2013 paper The Rise and Decline of an Open Collaboration System: How Wikipedia's reaction to popularity is causing its decline. The authors showed that active contributors in English Wikipedia began to shrink in 2007, with the major cause being increasing technical and cultural barriers for contributors. Since then, similar patterns have been seen in other Wikimedia projects.

Data showing the state of active contributors and contributor retention in Wikimedia projects is forthcoming.

Beyond the high-level numbers themselves, additional research suggests that many new contributors do not reach their full potential because contributing is both technically and culturally complicated. For instance, it is common for excited new contributors to have their first edits reverted or their first pages deleted without an informative explanation. This causes them to be confused and disappointed, and ultimately not to continue editing.

Together, the numbers, research, and stories, mean that we can do better to engage and retain new contributors. In particular, the Wikimedia Foundation decided to focus on mid-size Wikipedias because those projects need to grow their content, have challenges with contributor retention, and there is potential for software to help.

To better understand what causes new contributors to stay and to leave, the Wikimedia Foundation completed the New Editor Experiences research project in 2017.  The project identified the main challenges experienced by new contributors to Wikipedia, and a set of potential solutions.  See the “Challenges” and “Solutions” sections below for the details.

The Growth team formed to take action on the challenges and solutions identified by the research project. We will continue our relationship with the Czech and Korean communities by testing changes in those wikis first. When we discover changes that increase contributor retention, we will engage other communities to consider deploying those changes in more wikis.

Although much valuable research has already been done to help our team understand new editors, important open questions remain. The team records and prioritizes our open questions on this page. We will be deliberate about using research, data, surveys, and feature instrumentation to answer them.

The following is a list of healthy characteristics that we think wikis should exhibit with respect to new contributors.  They are guiding principles for our team, and we hope to help make them a reality in the wikis we work with.

• Diverse contributors lead to wikis with broad and deep content.
• Contributors who want to make useful improvements to a wiki should all be able to find a place in the community where they fit in.
• Contributors should be able to learn to edit on their own.
• Contributors of any skill level should be able to ask for and receive help from other community members.
• Contributors should be able to learn and edit in a supportive and non-confrontational environment.

There are challenges that stand in the way of the principles above.  The main challenges were identified by the New Editor Experiences research project that was completed in 2017.  With the help of Czech and Korean community members, researchers interviewed 64 contributors to Czech and Korean Wikipedias to learn about why they started editing, what they found easy or difficult, and if they've stopped editing, why they stopped. The project surfaced a set of technical, conceptual, and cultural challenges, summarized below.  Our team’s work will help wikis overcome these challenges so that the principles can become reality.

• Technical: new contributors struggle with specific skills needed to contribute.
  • Editing: though Visual Editor is helpful for new contributors, it is hard for them to learn the process of building, citing, and publishing.
  • Communication: new contributors have trouble finding and using Talk pages. This is because Talk pages do not use Visual Editor, and because they work differently than other internet discussion systems.
  • Finding help: many wikis have scattered and inconsistent help materials that are difficult for new contributors to find and use.
• Conceptual: new contributors have trouble learning core wiki policies and best practices.  The following are the concepts that are most challenging.
  • Community: many new contributors do not realize there is an active community behind each wiki project.
  • Verifiability and citations: all content must be attributed to reliable sources.
  • Notability: all content must have garnered enough broad attention to deserve an article.
  • Encyclopedic style and neutral point of view: articles should present content without bias toward one side of an argument.
  • Copyright: content should not violate the licensing of its sources.
• Cultural: the wiki environment can be discouraging to new contributors who are trying to find where they fit in.
  • Personas: new contributors have different objectives for why they are editing, but it is hard to figure out how to accomplish them.
  • Framing: the way that contributors communicate impacts whether new contributors stay. Negative feedback can discourage further contribution.

The Growth team will attempt multiple solutions to the above challenges, and we will learn and expand as we go along.  These are the findings from the New Editor Experiences project that will guide the solutions we attempt:

• Intermediaries: many new contributors have succeeded because a partner or mentor helped them learn to edit.  These partners are frequently found in off-wiki places like schools, libraries, or meetups.
• Iterative learning: new contributors have more success when they learn editing skills over time in safe environments.
• Outside help: new contributors frequently search for help outside of Wikipedia, either in off-wiki communications, or through other internet resources.

Those trends lead to two main categories of solutions to the challenges described above.  The work of our team will usually fit into one of these categories:

• Human-to-human help: forums and programs that connect contributors to each other so that experienced contributors can help newer contributors succeed.  This can include help desk forums, mentorship opportunities, or even off-wiki connections.
• In-context help: changes to the Wikipedia editing experience to surface helpful guidance around technical and conceptual challenges.  This is guidance surfaced at the time it is needed, in the place where it is needed -- as opposed to on a separate page that may be difficult to find.

This page contains the list of ideas that the team has considered and discussed with communities.

The Growth Team follows several principles as we work toward our objective.

• We keep in close communication with the communities our team affects, so that our work remains grounded in reality.
• We start with small interventions, learn quickly, and adjust our approach.
• We make decisions based on evidence from experiments, data, and user research.
• We take the time to equip ourselves with the tools we need to be flexible and work quickly.
• The software we build gives flexibility to the communities who use it, so they can modify and improve it on their own.
• When we know something doesn’t work, we remove it cleanly and learn for the future.
• We follow the Wikimedia Product Guidance for working with communities.
  • Please report bugs at Phabricator, using the Growth tag
  • How we triage incoming tasks and other chores, is detailed at Growth/Team/Chore list.
  • Our daily chores are documented at Growth/Team/Chores

Please review this guide describing how to engage with the Growth team.

These are links and research that underpin the background, principles, challenges, and solutions above, and that continue to shape our team’s thinking.

• The Rise and Decline of an Open Collaboration System: How Wikipedia's reaction to popularity is causing its decline
• New Editor Experiences
• Post-registration editor survey
• Formal definitions of editor retention
• Mentoring in Wikipedia: A Clash of Cultures
• Tea & Sympathy: Crafting Positive New User Experiences on Wikipedia
• The original Growth team from 2013 - 2014.
• Research on the impact of the Wikipedia Adventure
• Evaluating the impact of the Wikipedia Teahouse on newcomer socialization and retention
• Accept, decline, postpone: How newcomer productivity is reduced in English Wikipedia by pre-publication review
• The Pipeline of Online Participation Inequalities: The Case of Wikipedia Editing
• Research on New editors' first session and retention
• Fostering Public Good Contributions with Symbolic Awards: A Large-Scale Natural Field Experiment at Wikipedia