Life, as we are inclined to picture it, relies on the existence of diversity in the world. The existence of different species makes it possible for living things to exploit different ways of making a living in the world. And that all stems from lineages to split and generate new species.

Major waves of diversification have occurred at various times in the history of the Earth; the Cambrian explosion being the best known. During the Devonian, a geological period between 419 and 359 million years ago, the surface of the Earth was colonized by green plants. The first forests arose as plants evolved the ability to produce wood. Later in the Devonian, the first seed-producing plants evolved. This period of rapid plant evolution and greening of the land surface is known as the Devonian explosion. This greening of the land surface of the Earth represented a huge increase in biomass, which triggered a major decrease in atmospheric carbon dioxide and increase in oxygen; this in turn triggered changes in ocean nutrient concentrations and cycling patterns. Although this topic was briefly covered in the article about the Devonian, Wikipedia had no article about the Devonian explosion until a student in Mitch Cruzan’s Plant Evolution class spun off a stand-alone about the Devonian explosion. By spinning off this article, the student editor was able to add a lot of valuable information to Wikipedia that would have unbalanced the main article about the Devonian, had it been added there. In addition, the creation of a separate article increased the visibility of this information as a plant biology topic, as distinct from the parent article about a geological period.

Life on land posed a variety of new challenges for plants including the need for support (air is much less buoyant than water) and reproduction. In the water, it’s possible for sex cells to swim from male to female, but this doesn’t work so well on land. Beyond this is the fact that although reproduction in plants is superficially similar to reproduction in animals, there are actually two separate generations – a haploid gametophyte generation and a diploid sporophyte generation. As plants colonized the land there was an evolutionary trend towards endospory – the retention of the gametophyte generation within the sporophyte generation. While this important process was mentioned a few times in Wikipedia prior to this term, the creation of the endospory in plants article by a student in this class filled a major gap in Wikipedia’s coverage.

People often think of species as distinct entities that are unable to interbreed or, if they do interbreed, the resulting offspring is sterile (like a mule). In reality, a lot of species are capable of interbreeding; a group of species that frequently engage in natural hybridization is called a syngameon. As hybrids back-cross with members of their parental species, genetic material can be exchanged between species, a process which is called introgression. Introgressive hybridization in plants is especially important because of its use by plant breeders to improve important crop species like wheat and ornamental species like daffodils. Student editors in the class were able to create new articles about these two topics; while the syngameon article is entirely new, Wikipedia already had an article on introgression, but its section on plants was just two sentences long. With a broad topic like introgression, it can sometimes be difficult to decide whether you should expand a section of an existing article or create a new daughter article, but in this case the importance of introgression in plant breeding makes the creation of a new article a very reasonable choice.

Student editors in the class also created articles on related topics like transgenerational epigenetic inheritance in plants, pollinator-mediated selection, photoautotrophism and temporal plasticity, while others expanded the developmental selection and heterospory articles. Advanced undergraduate classes like this one can bring the ability to recognize — and fill — important gaps like this one to Wikipedia. Without their participation, lacunae like these might otherwise go unnoticed.

Interested in improving Wikipedia in your subject area? Visit teach.wikiedu.org for more information on how to use Wikipedia as a teaching tool in higher education classrooms.

Image credit: Eduard Riou (1838-1900) from The World Before the Deluge 1872, United States, Public domain, via Wikimedia Commons

The old principle that knowledge is power has been proven true in the online space in a way precedent only by the innovation of print. Free knowledge is designed to be shareable and shared online. It is evident that as the custodians of one of its flagship projects, Wikipedia, we should always consider if we could afford disengaging from conversation about the power that is created with it. This reflection is especially relevant in any global movement whose collective actions weigh enough to make a difference globally.

Read the introduction to the debate

Read John Weitzmann’s take on the issue on March 16th

Match made in (online) heaven

Emergence of Wikipedia, Wikimedia Commons and other such projects wouldn’t have been possible without reliable, standardised mechanisms of controlling creative outputs by ceding rights to them. Creative Commons licences are a societally recognised tool to do just that. Of course Wikipedia could have gone with any tool of a release of rights. But because of the diffusion of Creative Commons licences and the community behind it willing to translate, improve and finally use the licensing it makes sense that CC licensing is present on Wikipedia to such an extent.

It is a joyous feedback loop – Wikipedia has many contributors so the intake of CC licensing is massive. Then the images and materials licensed in that way start functioning in other contexts and projects. The two ideas: of a tool and of a knowledge-building practice are mutually reinforcing. No wonder that there is a significant personal overlap between two communities of contributors.

How is all this relevant to AI? Free knowledge is an ecosystem with many dependencies and a focus only on our immediate surroundings is too narrow. It is important to see that if we care about free knowledge we cannot dismiss major developments related to CC licences. That includes uses of photos to train AI in facial recognition that harms society.

The neutrality perspective

Many conversations on systemic problems affecting the Wikimedia movement and its endeavours are routinely scrutinised through a lens of project neutrality. There is no doubt that neutrality is – and should be – the cornerstone of creating encyclopaedic content. Even then one must account for implicit bias and ensure plurality of well-founded perspectives to get close to the ideal. But the very act of presenting true information is a political act. Enabling people to share in the sum of human knowledge is an ideological choice of letting everyone share the power. As noble as it is, it is sadly not a “neutral” one as there are many empowered proponents of the view that it is better if others are not able to make well-informed choices.

Artificial Intelligence is a neutral tool in the sense that the meaning of an elegantly designed algorithm can only be revealed through its intentional use by a human being. AI can be an empowering instrument of welfare or a weaponised tool of warfare. And, for that matter, so can be the data used to train it. We cannot, on one hand, claim that sharing in the sum of human knowledge is our end game with all the liberation that it brings and, on the other, pretend that uses of that knowledge that lead to disempowerment, disenfranchising and dehumanising of us humans are completely beyond our area of reflection.

“AI can be an empowering instrument of welfare or a weaponised tool of warfare.”

The thing is, we already care about the topic. We ensure transparency of our projects, we devise ethical frameworks and collective oversight for building and testing our own AI models, we don’t collect any personal user data to provide a safe harbour from ever-present algorithmically powered surveillance. Stretching our imagination to the damaging effects of facial recognition proliferation is a natural consequence not only of what we say we believe in but also of what we do.

What to do?

When discussing whether we should care that facial recognition training is built upon openly licensed photos I often sense that people are worried that showing interest will inevitably lead to a conclusion that the commons are the problem. Obviously they are not – I am convinced that their contribution to the world is of an overwhelmingly net positive value. The resolution of the “to care or not to care” dilemma is not to stop contributing to the commons.

What is more, the fact that we have a noble mission has to be weighed against our capacity to tackle the complexity of systemic problems. Surely, we cannot be responsible for all the bad that is happening online whenever anyone decides to use openly licensed content. The thing is that we have a unique chance to tackle a problem in a climate that is favourable to regulating it. It is neither too early – we already have evidence of bad and good applications of AI in almost any area of life. Nor is it too late, as the European Union has only begun debating a proposal of how to regulate AI.

“Wikimedians have an opportunity to frame the vocabulary of AI use and propagate the idea that it is not the technology that is good or bad but how we choose to use it should be regulated. “

The general objectives are pretty clear. Wikimedians have an opportunity to frame the vocabulary of AI use and propagate the idea that it is not the technology that is good or bad but how we choose to use it should be regulated. Even for face or voice recognition we can imagine an array of uses that assist those of us who are vision-impaired for example. The problem is that the revenue is in developing surveillance techniques and that the data that we provide to the world is employed to produce toxic results.

Regulating problematic AI uses

The European Commission proposed an AI Act that comes in handy as it determines AI uses that are unaccepted and rules for those that are considered high-risk. The first category includes a few cases where facial recognition may play a part, such as evaluation of trustworthiness based on social behaviour or personality, creating a societal score that is then used in social contexts unrelated to the primary data and real-time biometric identification. Public authorities would be forbidden to make use of AI in all these cases, where facial recognition is an indispensable tool. We need to discuss if this is enough or if there are any other uses of facial recognition AI that should make their way to this list.

There is also a dimension of the prohibition that concerns our projects directly. The data on the activity of our editors and contributors can become a part of a social scoring system or an evaluation of trustworthiness as well. A well-vetted list of forbidden AI uses also helps protect our communities.

Is AI a singularity point for open licensing?

On a different level, perhaps it is time to overview the open licensing system from the perspective of fundamental rights and to discuss whether there is a case to be made for an introduction of protection of people’s personal rights unrelated to copyright. Sure, this is not an easy problem to solve as these types of rights can be abused to protect public information for example. But if this is where the world is going with AI we need to follow with brainstorming remedies. When copyright was going from strength to strength, we didn’t just advise people to manage somehow, we gave them tools to do so.

Caring and actively devising ways to protect people from misuse of the imagery that portrays them, their families and their children may become important for the future uptake of open licensing. If people start associating open licensing with a gateway to an abuse of their rights and a tool aiding an oppressive system of control, they won’t use it. They may also object to others using it when it involves them as photo subjects.

Currently it seems that now the only way to somehow protect photos of people against the use for facial recognition training is to publish them under copyright. Then at least one can point to damages from a copyright breach when requesting a pull-out from such a database. It is certain that we wouldn’t want to bring about that result through our refusal to engage with the issue.

Wikimedia.brussels introduces a new format: debate. Our regular contributors as well as guest authors look at one topic from various sides. The arguments may be contrary, or they may point to different priorities. Contributors cast light on the complexity of an issue that doesn’t lend itself to an easy one-way solution. It is up to our Readers to choose the most appealing point of view or appreciate the diversity of perspectives.

***

These days, searchable Creative Commons-licensed resources include over 600 million items. Many of these are photos and out of them, a large number depicts humans – and their faces. While CC licensing does not touch upon the rights of subjects of photographs, the licences enable the author to waive many of their rights making possible, for example, reuse of images portraying people.

At Wikimedia, we are of course fans of open and free licensing – all content in projects such as Wikipedia or Wikimedia Commons is available for further reuse. We love when people do that because practising Free Knowledge is only possible with frictionless sharing, adapting, remixing and building upon what already exists. But as we see the availability of these resources as a force for good, should we care if they are used in a way that harms people?

Openly licensed photos depicting humans are massively fed into large databases used for training AI in facial recognition. Staggering numbers of these images are freely and openly licensed, coming from services such as Flickr and licensed in every possible way, including non-commercial use restrictions. According to researcher and artist Adam Harvey, author of Exposing.ai project, we are talking about millions of images, across multiple databases. Is that in any way harmful?

Literally everyone who develops artificial intelligence meant to deal with humans through recognising, categorising, tracking and profiling us and our behaviours needs training data. AI can be used for perfectly legitimate and hugely beneficial purposes, for example as assistants for people with disabilities. It can also be used by a bank that, wanting to understand our lifestyle to establish our credit scoring, combs through the internet and social media looking for our vacation photos and random snapshots. MegaFace, one of the largest datasets identified by Harvey and containing close to 5 million images leading to almost 700 thousand identities has been used by Tok Tok, Facebook, Google and Clearview.ai. Downloaded, according to the New York Times, thousands of times, these images were also requested by Europol, Danish and Turkish police, a Russian security agency, or the US Intelligence Advanced Research Projects Activity (IARPA), to name a few.

Photos available seemingly for the benefit of all humanity are then used to train obscure social media algorithms used to profile us for ads, to modify our behaviour and stay angry at each other. They are also instrumental in developing state surveillance by a number of states and agencies, and even these seemingly benevolent do it without little public scrutiny as AI uses are new and as yet unregulated. Finally, the subjects of beautiful wedding photos, friends gatherings, and artistic portraits are largely unaware of the fact that their memories and past time become “cannon meat” in the fight for economic and political dominance. Military and security-related uses of AI, from training drone face recognition software to emotion recognition at border control, remains vastly unregulated and unsupervised. 

None of this is the fault of the photographers, their subjects or projects that disseminate open content and free knowledge. It is rather a side consequence of a well working system, and neither that system or its participants can be held liable for all the evil in the world, including abuses of that content or personality rights. 

But is that enough to write off concerns about obscure and harmful uses of AI? Maybe we do have a more elusive ethical responsibility? And if we do, how can we approach the unintended consequences of our good work in a way that doesn’t stifle proliferation of free and open resources that, for the most part, benefit humanity? Or perhaps the “for the most part” argument is enough and we simply need to do our work and leave worries about AI dystopia to others?

Anna Mazgal, Senior EU Policy Advisor at FKAGEU points out that we cannot, on one hand, claim that sharing in the sum of human knowledge is our end game and, on the other, pretend that uses of that knowledge that lead to disempowerment, disenfranchising and dehumanising of us humans are completely beyond our area of reflection.

John Weizmann, Legal Counsel at Wikimedia Deutschland argues that we must not leverage copyright and licence grants to counter problematic developments in AI – unless we’re ready to jeopardise Open Content altogether.

Let us know what you think!

References:

• Harvey, Adam. LaPlace, Jules, Exposing.ai, 2021, https://exposing.ai, retrieved on 2022-02-14;
• Nech, Aaron and Kemelmacher-Shlizerman, Ira, “Level Playing Field for Million Scale Face Recognition”, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, p. 3406-3415;

By Arturo Borrero González, Site Reliability Engineer, Wikimedia Cloud Services Team

One of the most important and successful products provided by the Wikimedia Cloud Services team at the Wikimedia Foundation is Toolforge, a hosting service commonly known in the industry as Platform as a Service (PaaS). In particular, it is a platform that allows users and developers to run and use a variety of applications with the ultimate goal of helping the Wikimedia mission from the technical side.

Toolforge is powered by two different backend engines, Kubernetes and Grid Engine. The two backends have traditionally offered different features for tool developers. But as time moves forward we’ve learnt that Kubernetes is the future. Explaining why is the purpose of this blog post: we want to share more information and reasoning behind this mindset.

There are a number of reasons that make Grid Engine poorly suitable to remain as execution backend in Toolforge:

• There has not been a new Grid Engine release (bug fixes, security patches, or otherwise) since 2016. This doesn’t feel like a project being actively developed or maintained.
• The grid has poor support and controls for important aspects such as high availability, fault tolerance and self-recovery.
• Maintaining a healthy grid requires plenty of manual operations, like manual queue cleanups in case of failures, hand-crafted script for pooling/depooling nodes, etc.
• There is no good or modern monitoring support for the grid, and we need to craft and maintain several monitoring pieces for proper observability and to be able to do proper maintenance.
• The grid is also strongly tied to the underlying operating system release version. Migrating from one Debian version to the next is painful (a dedicated blog post about this will follow shortly).
• The grid imposes a strong dependency on NFS, another old technology. We would like to reduce dependency on NFS overall, and in the future we will explore NFS-free approaches for Toolforge.
• In general, Grid Engine is old software, old technology, which can be replaced by more modern approaches for providing an equivalent or better service.

As mentioned above, our desire is to cover all our grid-like needs with Kubernetes, a technology which has several benefits:

• Good high availability, fault tolerance and self-recovery semantics, constructs and facilities.
• Maintaining a running Kubernetes cluster requires little manual operations.
• There are good monitoring and observability options for Kubernetes deployments, including seamless integration with industry standards like prometheus.
• Our current approach to deploying and upgrading Kubernetes is independent of the underlying operating system.
• While our current Kubernetes deployment uses NFS as a central component, there is support for using other, more modern, approaches for the kind of shared storage needs we have in Toolforge.
• In general, Kubernetes is a modern technology, with a vibrant and healthy community, that enables new use cases and has enough flexibility to adapt legacy ones.

The relationship between Toolforge and Grid Engine has been interesting over the years. The grid has been used for quite a lot of time, we have plenty of documentation and established good practices. On the other hand, the grid is hard to maintain, imposes a heavy burden on the WMCS team and is a technology we must eventually discontinue. How to accommodate the two realities is a refreshing challenge, one that we hope to tackle together in the near future. A tradeoff exists here, but it is clear to us which option is best.

So we will work on deprecating and removing Grid Engine and migrating use cases into Kubernetes. This deprecation, however, will be done with care, as we know our technical community relies on the grid for some of the most relevant Toolforge use cases. And some of those workflows will need some adaptation in order to be fully supported on Kubernetes.

Stay tuned for more information on present and next works surrounding the Wikimedia Toolforge service. The next blog post will share more concrete details.

01/03/2022-07/03/2022

Mapping campaigns

• [1] Stefan Keller illustrated the Swiss project of the month > with sophisticated graphics.
• Raphaelmirc reported > on the results of the Mapathon in City Xexéu, Brazil, and refers > to the website of UMBRAOSM, the organisation that arranges these Mapathons in Brazil.

Mapping

• Enock Seth drew mappers’ attention to the updated Maxar Premium Imagery around Kumasi, Ghana on the Talk-gh mailing list and recommended how to work with the updated imagery.
• On the German OSM forum, mappers discussed > the so-called ‘security vandalism’ in Poland and Ukraine. The term describes re-tagging or deleting military features. They concluded that it seems to be against the will of the community in Poland and such vandalism in Poland can and should be reverted without involving DWG.
• Dakon has drafted a proposal for railway:signal:height=* to map the height of all signals at a node that do not have a dedicated :height=* subtag.

Community

• Amanda McCann issued her regular report covering her OSM activities for January 2022.
• On Saturday 5 March the local chapter of YouthMappers in Guatemala held a basic OpenStreetMap workshop with the support of HOT and several universities in Guatemala.
• OpenStreetMap Belgium’s Mapper of the Month for March is Søren Johannessen from Denmark.
• On Saturday 26 February several people from Latin America participated in a ‘note-a-thon’ (we reported earlier). User Mapeadora gave > her impressions of the event.

Local chapter news

• OpenStreetMap US published their March 2022 Newsletter.

Events

• In commemoration of International Women’s Day, GeoChicas and YouthMappers held a virtual workshop on Thursday 3 March to provide training in technical tools to incorporate a city into the ‘Women’s Streets’ project.
• On Tuesday 22 March the Trufi Association and the Institute for Transportation and Development (ITDP) will present a webinar on ‘Why Open Data Matters for Cycling’.

Education

• In the eleventh part of her Derrynaflan Trail series, Anne-Karoline Distel showed how to 3D map a more complex building than a simple house: the Crohane Church.
• Anne-Karoline Distel has also posted episode 12 of the Derrynaflan Trail on Youtube, showing how 3D models can be created from OSM data.

Humanitarian OSM

• Michael Montani, a GIS consultant for the UN Department of Operational Support, explained, on the HOT discussion list, how the UNSOS peacekeepers in Somalia are using OSM data to support their intervention. He also mentioned their concern about quality problems from some mapping projects located on the HOT Tasking Manager in areas with security problems that UNSOS monitors.Alessandro, a UNMapper, already expressed his frustration about these quality problems (we reported earlier) two months ago. He continued in a new blog post, which triggered some comments suggesting that eventual intervention from the OSMF Data working group may be required.
• Bo Percival introduced the new members of the HOT tech team.

Software

• yuiseki has developed an online tool that visualises the editor of each building object on a map with user icons. The source code is available on GitHub.

Did you know …

• … the various Geotools: ‘Reprojector’, ‘How Big’, ‘Gimme Geodata’ and ‘Radius’ of Hans Hack?
• … that LeafletJS, an open-source JavaScript library for interactive maps, also mobile-friendly, was born in Ukraine 11 years ago?
• … openSenseMap, the map with open sensor data? The senseBox offers students and interested citizens an OSM platform to collect and publish measurement data.

OSM in the media

• The Register reported on the StreetComplete app being rejected from the Google Play Store due to it having a donation link.
• Guillaume Rischard (OSMF Chair) created his own map of Kharkiv in half an hour to compare it with a publication in the NY Times. Result: ‘The New York Times did not use OpenStreetMap, but an inferior, probably commercial source. Otherwise their map would be better.’

Other “geo” things

• In a Twitter thread, Mateusz Fafinski showed how cartographic choices in mapping the Russian invasion of Ukraine can influence our perceptions.
• Dr. Jeffrey Lewis showed, on Twitter, how traffic information can be used to track military movements – hours before it was officially announced. Google reacted and deactivated the traffic layer for Ukraine.
• Webcams on German motorways are currently offline. Some speculate that this is to limit open source intelligence (via Twitter ).
• The UK Health and Safety Executive have created a database of fine-grained population estimates to assist in risk assessment; the National Population Database. A small sample of the database covering part of South Manchester is available online for interactive use. The types of data available (school enrolment, hospital beds, stadium capacity, etc.) may be of interest to OSM mappers.

Upcoming Events

Where	What	Online	When	Country
Marburg	FOSSGIS 2022		2022-03-09 – 2022-03-12
Lyon	EPN des Rancy : Technique de cartographie et d’édition		2022-03-12
臺北市	OpenStreetMap x Wikidata Taipei #38		2022-03-14
San Jose	South Bay Map Night	✓	2022-03-16
	149.Treffen des OSM-Stammtisches Bonn		2022-03-15
Lüneburg	Lüneburger Mappertreffen (online)		2022-03-15
London	Geomob London		2022-03-16
Großarl	4. Virtueller OpenStreetMap Stammtisch Österreich		2022-03-16
City of Subiaco	Social Mapping Sunday: Shenton Park		2022-03-20
Habay-la-Neuve	Réunion des contributeurs OpenStreetMap, Habay-la-Neuve		2022-03-21
	OSMF Engineering Working Group meeting		2022-03-21
	Why Open Data Matters for Cycling: Visualizing a Cycling City		2022-03-22
Barcelona	Geomob Barcelona		2022-03-22
City of Nottingham	OSM East Midlands/Nottingham meetup (online)	✓	2022-03-22
Decatur County	OSM US Mappy Hour		2022-03-24
	[Online] OpenStreetMap Foundation board of Directors – public videomeeting		2022-03-24
IJmuiden	OSM Nederland bijeenkomst (online)	✓	2022-03-26
06060	Reunión Bimestral OSM-LatAm		2022-03-26
Perth	Social Mapping Online	✓	2022-03-27
Bremen	Bremer Mappertreffen (Online)	✓	2022-03-28
San Jose	South Bay Map Night	✓	2022-03-30
Ville de Bruxelles – Stad Brussel	Virtual OpenStreetMap Belgium meeting	✓	2022-03-29
Tucson	State of the Map US		2022-04-01 – 2022-04-03

Note:
If you like to see your event here, please put it into the OSM calendar. Only data which is there, will appear in weeklyOSM.

This weeklyOSM was produced by Lejun, MatthiasMatthias, Nakaner, Nordpfeil, PierZen, SK53, Sammyhawkrad, Strubbl, TheSwavu, YoViajo, derFred, ztzthu.

• Albania report: Traditional Food Photography
• Belgium report: Shakespeare is dead – Contemporary playwriting festival; Public Domain Day Belgium 2022 (10/02) report
• Estonia report: An examples of a visual storytelling – two virtual exhibitions
• France report: Wikimedian in Residence in Clermont Auvergne
• Italy report: New agreement for Wiki Loves Monument Italy 2022
• Kosovo report: Traditional Food Photography
• Netherlands report: New photo collections of Alkmaar, Wiki goes Caribbean meeting, contemporary art Wikidata import and knowledge platform for GLAMS
• New Zealand report: National Digital Forum and Editing in a Time of COVID
• Nigeria report: 1Lib1Ref 2022 Kwara
• Poland report: New GLAM-Wiki partnerships and cooperations
• Serbia report: A month in the sign of 1Lib1Ref
• Sweden report: Additional photos from Swedish missionairies; Historical maps of Ukraine
• UK report: Khalili Collections
• Ukraine report: Stand with Ukraine!
• USA report: Women’s History Month activities
• AvoinGLAM report: February in AvoinGLAM
• Content Partnerships Hub report: Encyclopaedia of Life release their 2 million species descriptions under CC0
• Structured Data on Wikimedia Commons report: Editing SDC with OpenRefine; Monthly OpenRefine and Wikimedia office hours
• Calendar: March’s GLAM events

Women’s History Month in March serves as a designated time to commemorate women and highlight their contributions to society. As we showcase the many women who have shaped our world, however, their struggles to be recognized and seen for their many achievements also become clear.

This is especially true in the information landscape, the universe of resources we turn to when we’re looking for knowledge. For as long as written history, women — especially Black, Indigenous, and women of color — have been left out of the record, out of historical narratives, and traditional sources of knowledge — an issue that many institutions and publications today are trying to address.

“For as long as written history, women — especially Black, Indigenous, and women of color — have been left out of the record.”

Sadly, we see this reflected when it comes to the representation and participation of women — cis, transgender, and non-binary — on Wikipedia and other Wikimedia projects. Wikipedia is powered by humans, so it is vulnerable to human biases. Currently, only 18% of the content in all Wikimedia projects, including biographies on Wikipedia, are of women. Further, only 15% of Wikimedia contributors are women.

But there is good news. In recent years, a growing number of passionate people and ongoing gender equity initiatives have been working to close the gender gap on Wikipedia and beyond.

The Wikimedia Foundation, the nonprofit that operates Wikipedia and its companion free knowledge projects, strives to create an inclusive, equitable living record of history, stories, and contexts. Project Rewrite was launched in 2021 with this exact mission in mind: calling attention to the gender bias across the information landscape, highlighting efforts underway to close these gaps, and inviting everyone to get involved — not just in March, but all year long.

“But there is good news. A growing number of passionate people and ongoing gender equity initiatives have been working to close the gender gap on Wikipedia and beyond.”

Every action counts. There are several efforts underway to join and support — from campaigns to add missing articles about women to Wikipedia, to training events for new volunteers, and more.

Are you ready to join us?

It’s easy to start with the following Project Rewrite Checklist, sharing ten ways to help close the gender gap on Wikipedia and beyond. Let us know your progress with #ProjectRewrite on social media.

The Project Rewrite Checklist

You can help to close the gender gap on Wikipedia and beyond! Looking for a place to start? Explore the Project Rewrite Checklist for ten ways to get involved. Every action counts.

1. Get the facts: Did you know only 18% of the content in all Wikimedia projects, including biographies on Wikipedia, are about women? We believe one of the first steps in solving the gender gap is through understanding the problem. See the latest research and stats.
2. Listen to a podcast: What causes the gender gap on Wikipedia in the first place? Tune into the “dot com” podcast to learn about some of the biggest challenges and hear from volunteers working to fix them.
3. Watch a panel discussion: While the gender gap problem exists on Wikipedia, the solution requires changes outside of it. Leaders in academia, journalism, and the nonprofit sector discuss how we can work together for change.
4. Read an article: Glamour interviews four amazing volunteers who are writing themselves into history. Read how they first started with editing articles on Wikipedia and how they are now making the online encyclopedia more equitable, one edit at a time.
5. Create a Wikipedia account: You did your reading and you are ready to take action. So what’s next? Start by creating your Wikipedia account. To prepare for your first edit, review a guide from Art+Feminism, a non-profit focused on closing information gaps related to gender, feminism, and the arts on Wikipedia.
6. Start small: Small changes can have big impacts. Many Wikipedia articles about women need small improvements, such as more information or sources. Get started by updating articles on this list from Women in Red, a project to reduce systemic bias in the Wikimedia movement.
7. Tag images: Looking for something to do while taking a quick break? Take just a few seconds to go through images of women’s suffrage activists and write descriptions, captions, and tags related to what you see to increase search engine capabilities. The tool was created by Wiki in Africa to improve visual representation on the media repository Wikimedia Commons.
8. Upload a photo: Photos are critical in shaping the story about women and their work on Wikipedia. Do you or your institution have freely-licensed photos of notable women? Upload them to Wikimedia Commons through the #VisibleWikiWomen campaign to add the face to the name on Wikipedia.
9. Find an event: Maybe you are looking to join a community or expand your skills in editing. Whatever your goal, several events happening in March and the rest of the year welcome, guide, and support new editors.
10. Join a campaign: Keep up the momentum all year long! There are dozens of campaigns, contests, and groups you can join to make a difference. Whether it is by adding missing references, uploading photographs, or creating new biographies on overlooked women, you can impact the information people around the world use on a daily basis.

Visit the Project Rewrite page to learn more.

Barbara Kandek is a Communications Associate at the Wikimedia Foundation.

NANöR is an active user of Arabic Wiki Projects and Arabic communities, a member of the Middle East and Africa grant committee, and the Core Organizing Team (COT) for Wikimania 2022.

FACT about My Wikidata Journey:
I am not a Librarian or a programmer!
Yet Wikidata had room for my enthusiasm

Before I head straight through how my journey with Wikidata started, here is a surprising truth you might have in common with me. I don’t have a background in programming or any IT software expertise, I only have the basics. Yet I support Wikidata and I hit the milestone of having more than 5,000 edits in 6 months!

My relation with Wikidata started with my initial journey with the big sister Wikipedia. It all started as the usual, creating a new page or article. Back then, I only used to add links via Wikidata. At a later stage I started to go to Wikidata items and started to edit the labels, and if I had enough courage I started to work on the description part with the basic understanding I had back then.  

I wanted to expand my knowledge in the programming part and I was excited to explore how to organize the knowledge and data on Wikidata. I did not actually know that there was training on how to uplift our skills and use Wikidata the right way. It also enriched my knowledge on the various tools I can make use of in the future. I learned about training from another active Wikipedian who shared the event with me and told me that there was a training course on Wikidata through Wiki Education. I directly applied and signed up for the course in July 2021 and it lasted for 3 weeks. It was super beneficial experience for me as I expanded my knowledge in a methodological way. I did all the learning models they provided in the course (8 training sessions), which provided me with a lot of knowledge in a systematic way. 

Undoubtedly, the training boosted my confidence to get familiar with Wikidata terms Q, P, E, how to add referencing, how to evaluate items relevancy and value even if I do not know much about the topic. The training made me explore these aspects and accelerated my learning through an expert Wikdatan’s guidance, not forgetting to add that the resources given about tools were so helpful. 

I was in parallel working with a colleague in Arabic Wikipedia on archiving and documenting historical resources. We started to sort it on Wikidata then move it to Arabic Wikipedia. Everything I learned through the training was transformed and applied during the documentation workshop focused on historical resources. 

Because I love to transfer knowledge to newcomers so as not to be afraid of Wikidata complexity, I contacted the Wiki World Heritage User Group, since they have good experience in Wikidata. We launched under their supervision a workshop for 6 females on the documentation of monuments in Aleppo. The workshop was for 2 weeks, we learned how to collect data on spreadsheets, learned about the tools (Quick Statements, Wikidata Query Service), and implemented what was previously learned from Wiki Education’s Wikidata Institute course. 

Since I believe in closing the gender gap on Wikidata, my inner motivation to support newcomers and my active launching of workshops, I felt that I can be a guide in this dimension, like a guide other beginners through the basics of Wikidata.

Driven by these beliefs, I launched a workshop regarding filtering and documenting terms, phrases and words in a reliable dictionary that started in Tunisia with the members of Wikimedia Levant User Group.This workshop was for 6 weeks, where it had 10 female participants. Proudly, we worked on a book that included around 850 words that needed documentation, where we documented 740 words and phrases in Arabic. This  progress can be tracked through the workshop dashboard.

This made the participants have an initial idea about Wikidata and break the fear of joining this world. They learned from an expert view point. The group work was really interactive and they were excited, each participant helped in adding or editing a minimum of 50 items during the workshop. I can recall a conversation I had with one of the participants, “Wikidata is really easy we thought it was so hard,” and I guided the conversation to ensure that “Wikidata is not easy, but the expert \ institution community is simplifying it through those trainings and previous workshops to help us support free knowledge.”

I always encourage newcomers and beginners to learn Wikidata the right way because a simple contribution done the right way will make them excited to try more and more.

My long term goal is to support the community in closing the gender gap when launching those kinds of workshops, Every contribution matters!

Want to take the same Wikidata class User:NANöR took? Visit wikiedu.org/wikidata for current offerings.

Image credit: NANöR, CC BY-SA 3.0, via Wikimedia Commons

Last year, I started a small pilot under the OpenSpeaks project for building voice data as a foundational layer for speech synthesis research and application development. To test some of the learning in this field, I started building a wordlist by collecting words from multiple sources including Odia Wikipedia and Odia Wiktionary, and started recording pronunciations using Lingua Libre. Recently, the pilot hit a 55,000 pronunciation milestone. The repository also includes pronunciations of 5,600 words in Baleswaria, the northern dialect of Odia. All the recordings were also released under a Public Domain (Creative Commons CC0 1.0) release on Wikimedia Commons. These recordings make the largest repository of Public-Domain voice data in Odia, and add to another 4,000+ recordings of sentences in Odia on Mozilla Common Voice.

The “Odia pronunciation” category on Wikimedia Commons that houses these recordings also includes further contributions from many other volunteer contributors that are available under CC-BY and CC-BY-SA Licenses. The dialect has been scarcely studied before and a comprehensive corpus of all the spoken words was never made available. As a bonus of this pilot recording project, two word corpus are being built and expanded in both Odia and Baleswaria.

Source: https://commons.wikimedia.org/wiki/File:13_Lingua_Libre_tut_Odia_-_Publish_to_Wikimedia_Commons.png

Recording a word using Lingua Libre. A detailed tutorial in Odia explains how to record words from the Odia Wiktionary. User:Psubhashish CC-BY-SA-4.0.

Collecting words for building wordlists

To record pronunciations of words, a unique list of words are often used. Such lists exist for many languages as Natural Language Processing (NLP) research generally requires such corpuses. In the case of Odia, many of us in the Wikimedia community had created a list of 500K unique words back in 2017 primarily using the content that we ourselves created on Odia Wikipedia at that time. There also exists a few other lists such as PanLex and UniLex, as pointed out by Bengali Wikimedian Mahir Morshed over a conversation, which use data crawled from multiple sources. Nikhil Mohan Patnaik, a scientist and co-founder of nonprofit Srujanika that is involved in digitization and scientific resource-building in Odia, once shared the frustration that most wordlists in Odia did not have a decent diversity of topics, especially words that find contemporary use or use in science and technology.

Through a series of interconnected steps, it was possible to create a wordlist and clean it further. Cleaning up a wordlist is a must as the collection of textual data often includes content that is not always edited for spelling and other mistakes such as the use of alien characters in a particular writing system (say, Cyrillic characters in Korean text). Data dumps from Odia Wikipedia, Odia Wikisource, and Odia Wiktionary (which itself contains over 100K words and lexical definitions that are taken from the 1931-1941 lexicon “Purnnachandra Ordiya Bhashakosha”) were the primary sources. At the beginning of this pilot, I started expanding the existing wordlist on a few fronts. The Odia Wikipedia by this time not only has a few thousand more articles, but the quality of the articles have been consciously improved by community effort. More importantly, individual editors who lead projects, such as articles on politicians and films by Sangram Keshari Senapati, medicine-related articles by Subas Chandra Rout and articles on species and more from both plant and animal kingdom by Sraban Kumar Mishra, have created a rich body of diverse articles. Many annual editing sprints focusing on diciplines such as feminism and social sciences are also helping diversify Odia Wikipedia slowly. It was useful to add words by downloading and scraping the Wikipedia data dump and also look beyond. Despite these efforts, a small community of volunteer editors still translates into some spelling mistakes. Within three days of running a bot, I could fix over 10,000 errors just inside Odia Wikipedia. There are many more and some would even need manual reading, discovery, and editing. Wikisource tends to retain many spelling mistakes or writing styles, which could drift from the standard writing style that the original author of a book uses. While recording words in a dialect helps expand the speech diversity, plain typing and other errors can only increase the total number count of words. It is possible to manually notice many mistyped words during the recording process and it was possible to correct them on the wordlist side.

Odia newspapers that have a text archive and new sites became the primary sources for contemporary topics and the Bigyan Diganta magazine was key for science and technology related words. By web-scraping, crowdsourcing such source, converting publicly available text typed using legacy encoding systems (such as ASCII) into text with Unicode encoding by using encoding converters, and cleaning up such data with a mixed approach (in a semi-automated process of using a script written by Wikimedian T. Shrinivasan and a manual cleanup) helped add more words to the wordlist. A tutorial by Wikimedian Tito Dutta was also useful for the wordlist creation before this script was available.

These steps also helped expand the diversity in terms of topics as Wikipedia content has many levels of gaps—starting from a significantly high number of articles of local relevance in terms of geography, religion, and ethnicity because of the existing contributor diversity to a much higher level of coverage of medicine-related articles, to a staggeringly low-level of coverage of articles in topics such as feminism or human-computer interaction or social justice. While the publicly available text includes many known gaps, collating all available words is still required from a wordlist standpoint.

Lexicographers generally segregate words as headwords or lemmas and forms. From a pronunciation library standpoint, forms are important as they carry phonological features such as intonation. For over 10,000 of the 500K+ list, I also created a spreadsheet with rules to produce more than 100 forms from a single lemma. Later, continued interactions with Wikimedians Bodhisattwa Mondal and Mahir helped me understand that there is a place for lemmas and forms for creation of Lexemes on Wikidata, and work towards Abstract Wikipedia, an ambitious project that would eventually help translate definitions and descriptions used on Wikipedia into multiple languages. Abstract Wikipedia can potentially be the kind of automation that might reduce the workload of many Wikipedians as volunteer labor is an extreme privilege in many parts of the world. The demand for way too much manual work across Wikimedia projects can not only lead to contributor burnout but can also create an entry-level huge barrier for many who have no-limited time and other forms of affordability to volunteer.

The wordlist creation also helped in listing for the first time headwords/lemmas that did not make it to available dictionaries either because of lack of wider coverage or lack of subject experts. Emergence of neologies because of COVID is an example of how unexpected situations add new words to a language. The wordlist has some such words. What was more useful is to create forms from headwords using concatenation forumas (e.g. “COVID-led” and “COVID-related” are two forms of the lemma “COVID”). Recording forms are important as speech incorporates intonation and other variations even though a lemma and its corresponding forms will have repetition of the same lemma. The audio library does contain some such cases. Building the wordlist in a speech-led process also helped create lemmas along with definitions and forms in some cases and that prepared ground for importing them into a Wikidata lexeme structure.

The recording process

Lingua Libre recording setup for batch recording of word pronunciations. Package cushioning pads are used for frugal soundproofing. User:Psubhashish CC-BY-SA 4.0

The recording process includes recording with a professional desktop studio microphone directly into Lingua Libre. The latter being a web application that works well on the Firefox browser, it was possible to change the system settings on the computer and make the external mic (instead of the computer’s built-in mic which has low quality reception) the default mic. The mic also has rotating knobs to adjust the gain during the recording as Lingua Libre has no other effective means of cleaning up audio post recording. Some amount of soundproofing was done by adding four package cushioning pads while the microphone and the computer were placed over a cotton mat that covered the wooden table underneath. The recording process includes recording with a professional desktop studio microphone. Some amount of soundproofing was done by adding four package cushioning pads while the microphone and the computer were placed over a cotton mat that covered the wooden table underneath. All these steps were frugal and used for ensuring acoustic soundproofing. Lastly, Lingua Libre’s two levels of sound monitoring and reviewing were key to ensuring that the audio quality is right before uploading. I have created on the Odia Wiktionary a tutorial with the step-by-step process for using Lingua Libre for Odia entries from Wiktionary.

Platforms and hardware used

This pilot uses platforms that are open source and otherwise adhere to the philosophy of Openness. OpenSpeaks started as a project on Wikiversity for helping archivists that are creating multimedia archival documentations of low-medium-resource languages. It includes a range of Open Educational Resources (OER), open strategies and even templates for specific needs in different linguistic and other demographic environments. Lingua Libre is a web platform for recording pronunciations of a list of words in a language or dialect. The project supports virtually all languages that can be written using a writing system with Unicode-compliance encoding. Common Voice is a web platform by Mozilla that encourages contributors to record pronunciations of sentences and review recordings made by others. It only accepts at the moment sentences available under Public Domain and the recordings made are also made available under a Public Domain release after they are anonymized. The script created by T. Shrinivasan for creating a unique list of words from any text file (with .txt or even .xml or .json extensions) is available on GitHub and can be tailor-made to accommodate the needs of different languages typed using varied writing systems.

Things in the horizon

The recorded words are now uploaded on Wikimedia Commons paving a way for them to be used across Wikimedia projects, starting with lexemes and Wiktionary, and also for other NLP research and application development. As a native speaker of Baleswaria who always switches code between the dialect and the standard Odia, it was also important to document the variations in intonation and accent which are essential attributes that need to be documented for speech synthesis. The repository severely lacks phonological diversity as all the 55,000 recordings are made by a single male speaker. The next step is to document the ethnolinguistic strategies that can be useful for other low-medium-resource languages as a part of the OpenSpeaks project which focuses on helping archivists with strategies and tools for multimedia archival of languages with low and limited resources. In the meantime, the wordlists are made available on GitHub.

I'll start with a bit of general administrivia. First, our migration of Wikimedia code review & CI to GitLab continues, and we're mindful that people could use regular updates on progress. Second, I need to think through some stuff about the project, and doing that in writing is helpful for all involved. I'm going to try writing occasional blog entries here for both purposes.

Now on to the main topic of this post: Access control for groups and projects on the Wikimedia GitLab instance.

The tl;dr: We've been modeling access to things on GitLab by using groups under /people to contain individual users and then granting those groups access to things under /repos. This has been tricky to explain and doesn't work as well at a technical level as we'd hoped, so we're mostly scrapping the distinction, and moving control of project access to individual memberships in groups under /repos. This should be easier to think about, simpler to manage, and seems like it will suit our needs better. Read on for the nitty-gritty detail.

(Thanks to @Dzahn, @Majavah, @bd808, @AntiCompositeNumber, and @thcipriani for helping me think through the issues underlying this post.)

Background

During the GitLab consultation, when we were working on building up a model of how we'd use GitLab for Wikimedia projects, we wrote up a draft policy for managing users and their access to projects.

GitLab supports Groups. GitLab groups are similar to GitHub's concept of organizations, although the specifics differ. Groups can contain:

• Other, nested groups
• Individual projects (repositories & metadata)
• Users as members; members of other groups can be invited to a group
  • A user who is a member of a top-level group is also a member of every group it contains

We've since changed the original draft policy in some small ways - in particular, we decided to move most projects into a top-level /repos group in order to offer shared CI runners (see T292094). You can read the policy we landed on at the latest revision of GitLab/Policy on mediawiki.org.

The basic idea was that we would separate groups out into:

1. Sub-groups of /repos: Namespaces for projects, split up by functional area of code
2. Sub-groups of /people: Namespaces for individual users, split up by organizational units like:
   • Volunteer group
   • Teams at organizations such as the WMF, WMDE, etc.

Groups in /people could then be given access to projects under /repos.

Our hope was that this would let us decouple the management of groups of humans from the individual projects they work on, and ease onboarding for new contributors. A new member of the WMF Release Engineering team, for example, could be added to a single group and then have access to all the things they need to do their job.

We intended for most /people groups to be owned by their members, who would in turn have ownership-level access to their projects under /repos, allowing for contributors to a project to manage access and invite new contributors.

As a concrete example:

• https://gitlab.wikimedia.org/repos/cloud contains various cloud services projects
• The Wikimedia Foundation Cloud Services team and volunteer cloud administrators are modeled by membership in:
  • https://gitlab.wikimedia.org/people/wmf-team-cloud-services
  • https://gitlab.wikimedia.org/people/volunteer-group-cloud-admin

Problems with this scheme

I've been proceeding under this plan as people request the creation of GitLab project groups, but there turn out to be some problems.

First, it doesn't seem like permission inheritance for nested groups with other groups as members works the way you'd expect & hope: See T300939 - "GitLab group permissions are not inherited by sub-groups for groups of users invited to the parent repo".

Second, users have concerns about equity of access and tight coupling of things like employment with a specific organization to project access. We didn't have any intention of modeling any group of users as second-class citizens within this scheme, but it seems to create the impression of one all the same. It's also striking that the set of projects people work on just isn't that cleanly mapped to any particular organizational structure. Once you've been a technical contributor for a while, you've almost certainly collected responsibilities that no org chart reflects accurately.

Finally, and maybe most importantly, this is a complicated way to do things. People have a hard time thinking about it, and it requires a lot of explanation. That seems bad for an abstraction that we'd like to be basically self-serve for most users.

Proposed solution

Mostly, my plan is to use groups closer to how they seem to be designed:

1. Sub-groups of /repos will contain both individual contributor memberships and projects
2. Except in occasional one-off cases, access should be granted at the level of a containing group rather than at the level of individual projects, so as to avoid micromanaging access to many projects.
3. We'll keep /people in mind as a potential solution for some problems (for example, it might be a good tool for synchronizing groups of users from LDAP and granting access to certain projects on that basis), but not rely on it for anything at the moment.

There are some unanswered questions here, but I plan to redraft the policy doc, move existing project layouts to this scheme, and start creating new project groups on this basis in the coming week or so.

My main philosophical takeaway here is that I work with a bunch of anarchists, and it's always best to plan accordingly.

Originally, one of our goals for this migration was avoiding a repeat of the weird, nested morass that is our current set of Gerrit permissions. While it would be a good idea to keep the structure of things on GitLab flatter and easier to think about, I'm no longer that worried about it. Some of the complexity is inherent to any large set of projects and contributors; some of it just reflects a long-lived technical culture that's emergent and largely self-governing, tendencies that nearly always resist well-intentioned efforts to rationalize and map structure to things like official organizational layout.

I’ve taken a break from the blog for too long! Time to update some on current work. We’re doing a final push on the video.js-based frontend media player for MediaWiki’s TimedMediaHandler, with some new user interface bits, better mobile support, and laying the foundation for smoother streaming in the future.

Among other things, I’m doing some cleanup on the AudioFeeder component in the ogv.js codec shim, which is still used in Safari on iOS devices and older Macs.

This abstracts a digital sound output channel with an append-only buffer, which can be stopped/started, the volume changed, and the current playback position queried.

When I was starting on this work in 2014 or so, Internet Explorer 11 was supported so I needed a Flash backend for IE, and a Web Audio backend for Safari… at the time, the only way to create the endless virtual buffer in Web Audio was using a ScriptProcessorNode, which ran its data-manipulation callback on the main thread. This required a fairly large buffer size for each callback to ensure the browser’s audio thread had data available if the main thread was hung up on drawing a video frame or something.

Fast forward to 2022: IE 11 and Flash are EOL and I’ve been able to drop them from our support matrix. Safari and other browsers still support ScriptProcessorNode, but it’s been officially deprecated for a while in favor of AudioWorklets.

I’ve been meaning to look into upgrading with an AudioWorklet backend but hadn’t had need; however I’m seeing some performance problems with the current code on Safari Technology Preview, especially on my slower 2015 MacBook Pro which doesn’t grok VP9 in hardware so needs the shim. :) Figured it’s worth taking a day or two to see if I can avoid a perf regression on old Macs when the next Safari update comes out.

So first — what’s a worklet? This is an interface that’s being adopted by a few web bits (I think some CSS animation bits are using these too) to have a fairly structured way of loading little scripts into a dedicated worker thread (the worklet) to do specific things off-main-thread that are performance critical (audio, layout, animation).

An AudioWorkletNode hooks into the Web Audio graph, giving something similar to a ScriptProcessorNode but where the processing callback runs in the worklet, on an AudioWorkletProcessor subclass. The processor object has audio-specific stuff like the media time at the start of the buffer, and is given input/output channels to read/write.

For an ever-growing output, we use 0 inputs and 1 output; ideally I can support multichannel audio as well, which I never bothered to do in the old code (for simplicity it downmixed everything to stereo). Because the worklet processors run on a dedicated thread, the data comes in small chunks — by default something like 128 samples — whereas I’d been using like 8192-sample buffers on the main thread! This allows you to have low latency, if you prefer it over a comfy buffer.

Communicating with worker threads traditionally sucks in JavaScript — unless you opt into the new secure stuff for shared memory buffers you have to send asynchronous messages; however those messages can include structured data so it’s easy to send Float32Arrays full of samples around.

The AudioWorkletNode on the main thread gets its own MessagePort, which connects to a fellow MessagePort on the AudioWorkletProcessor in the audio thread, and you can post JS objects back and forth, using the standard “structured clone” algorithm for stripping out local state.

I haven’t quite got it running yet but I think I’m close. ;) On node creation, an initial set of queued buffers are sent in with the setup parameters. When audio starts playing, after the first callback copies out its data it posts some state info back to the main thread, with the audio chunk’s timestamp and the number of samples output so far.

The main thread’s AudioFeeder abstraction can then pair those up to report what timestamp within the data feed is being played now, with compensation for any surprises (like the audio thread missing a callback itself, or a buffer underrun from the main thread causing a delay).

When stopping, instead of just removing the node from the audio graph, I’ve got the main thread sending down a message that notifies the worklet code that it can safely stop, and asking for any remaining data back. This is important if we’ve maintained a healthy buffer of decoded audio; in case we continue playback from the same position, we can pass the buffers back into the new worklet node.

I kinda like the interface now that I’m digging in it. Should work… Either tonight or tomorrow hope to sort that out and get ogv.js updated in-tree again in TMH.

This International Women’s Day, we’re celebrating our partnership with the Smithsonian American Women’s History Initiative, an effort to amplify American women’s accomplishments to the public. American museums house rich materials about notable and regional figures, and Wikipedia provides a space to share untold stories. So Wiki Education worked with the Smithsonian to design a project that would bring women’s stories to Wikipedia’s readers for years to come.

In four Wiki Scholars courses, museum professionals who work at one of the Smithsonian’s nearly 200 Affiliates collaborated with each other and Wiki Education’s team to add and expand biographies of notable women on Wikipedia. Over 6 weeks, they learned how to use Wikimedia projects as tools in their work to preserve and share knowledge with the public. All told, we trained 74 museum professionals how to edit Wikipedia, representing 53 different Smithsonian Affiliate museums, and they improved more than 160 articles. By embedding Wikipedia know-how within their institution, the Smithsonian has developed a network of new Wikipedians to continue this important work both through their own editing and through organizing local projects.

This is the story of how we worked together to bring high-quality information about historic women to the public — and how other organizations can make that happen for their faculty, staff, or members.

There are two key components to this project:

1. museum professionals from across the United States learn how to edit Wikipedia; 
2. we expand public knowledge of notable American women from across the U.S. by leveraging museum collections and materials.

1) Wiki Scholars courses teach museum professionals how to edit Wikipedia

During the 6-week courses, Wiki Education’s team of Wikipedia experts facilitated collaborative group sessions among the museum professionals and provided ongoing support as the scholars made their first edits. We worked together as they incorporated published information about notable and underrepresented American women from their collections onto Wikipedia, helping them navigate Wikipedia’s technical, procedural, and cultural practices. One participant reported that “it was great to have timely responses to even the smallest questions. It was also great to just have a safe environment in class to be confused while starting up the project.” When asked if the experience met their expectations, one participant said, “YES! I was eager to learn the back-end and was gobsmacked about how much actually goes into the development and editing of pages. I really had no idea about the back end of how the Wikipedia process comes together.”

Our tried-and-tested Wiki Scholars course curriculum brings “newbies” into the community in a relatively short period, and we’re thrilled with how much participants enjoy the whole experience. One said that the course “exceeded [their] expectations with an exceptionally well thought out curriculum and a thoughtful instructor who made the content digestible.”

On top of learning how Wikipedia works, this course gave employees across Smithsonian Affiliate institutions the unique chance to collaborate with each other and learn about work at other museums.

Now, they’re primed to continue adding archival materials to Wikipedia, and they can find the tools and support they need to organize in their regions. Deborah Krieger, the Exhibit & Program Coordinator for the Museum of Work & Culture in Woonsocket, Rhode Island, has already hosted an edit-a-thon, where she and fellow participants helped improve articles related to five women featured in the museum’s recent exhibit, Rhode Island Women Create. Another participant said, “I learned exactly what I came in to do and can use these tools for additional programming!”

2) The public benefits from untold stories about American women

With its wide availability, Wikipedia presents a unique chance to democratize knowledge about notable figures in U.S. history. But who traditionally determines what’s ‘notable’? Both the volunteer editors and the publications available to them, since Wikipedia biographies require citations from reputable sources. The community of editors in the United States typically cluster geographically in major cities like New York, Washington, DC, and San Francisco. Those active editors have built strong initiatives in their localities, running editing events, responding to local interest in Wikimedia projects, and fostering fun and fulfilling communities that keep editors engaged for years.

Thanks to those organizers, we’ve seen an influx of new editors in these regions over the last two decades. They excel at writing local legends into Wikipedia, making their stories more widely known. So how do we shed light on the hidden figures from other parts of the country?

Wiki Education has been eager to activate editors outside of the major clusters who can build Wikipedia outreach into their professional lives. This project with the Smithsonian presented a great opportunity to bring together Wikipedia novices from across the United States, working with people from Rhode Island, Michigan, Montana, New Mexico, and 21 other states.

By bringing editors from diverse regions to Wikipedia, we are able to tell the story of American history beyond the names we know from our history books, especially when these editors have access to hyper-local archives and materials. Museum professionals identified the untold stories from their institutions’ own collections and brought them to a broader audience than the visitors who walk through their doors. As one participant, Freya Liggett, of the Northwest Museum of Arts and Culture in Spokane, Washington, said, “Projects like the Smithsonian American Women’s History Initiative can make significant contributions to Wikipedia’s content and open new ways for people to connect with resources at your museum.”

Freya developed a new biography for Wikipedia, writing about Estelle Reel. The suffragist and politician served as the national Superintendent of Indian Schools between 1898 and 1910. After the “many news stories [in 2021] about the grim legacy of North American Indian boarding schools,” Liggett thought it was important to add Reel’s role in the history of Indian schools, thus documenting the “individuals behind America’s assimilation-based education policies and the effects on Native children.” Though Reel’s role in developing racist curriculum to assimilate Indian children into white society is not a pretty one, these stories deserve to be accessible to the public, especially to honor Indian culture and history.

Erica Schumann, a member of the Development Team at the Fort Worth Museum of Science and History, created an article about Opal Lee, widely known as the “Grandmother of Juneteenth.” Erica says she was shocked Opal Lee didn’t have an article, despite the considerable amount of national media attention she’d gotten. The Fort Worth Museum of Science and History had been working with Lee’s family to develop an exhibit display recognizing her for her achievements. So Erica dove right in to create her biography in her sandbox, a private drafting space on Wikipedia. While Erica’s draft was still in her sandbox, another editor created the article; Erica ended up moving her drafted text into the article others had started. “When I started sandboxing the article, I had no idea Juneteenth would become a federal holiday just a couple weeks later!” she said. “In the middle of the course, on June 17, 2021, Opal Lee saw her dream become a reality as she joined President Biden as he signed the bill formally establishing Juneteenth as a national holiday. This led to Ms. Lee gaining a significant amount of national and international attention over the course of just a few days, and it was incredible to see all the views the article was immediately getting! It was fantastic to see the article being updated in real time to reflect Ms. Lee’s huge accomplishment, and I am so grateful I got to be a part of that editing experience.” In only six months, Opal Lee’s biography has reached more than 20,000 readers, bringing one Texan’s story to a huge audience thanks to Wikipedia and editors like Schumann.

In 2021, a Wikipedia editor approached the Hagley Museum and Library with a rights and reproductions request: could they have images of the patent models Hagley had in their collection for inventor William E. Sawyer? Hagley Registrar Jennifer Johns immediately saw Wikipedia as another way to generate interest in their collection. As a part of the Wiki Scholars course, Jennifer expanded biographies of four notable women: Frances Gabe, Harriet Tracy, Clarissa Britain, and Anne Marguerite Hyde de Neuville. All are subjects of Hagley’s collection; the first three are inventors and the fourth is an artist in the collection whose work Jennifer likes. Because Jennifer had access to the museum’s collections, she was able to use photographs she’d taken of the patent models. Hagley is opening a new patent model exhibit, Nation of Inventors, in spring 2022, and Jennifer has made it her mission to ensure all the represented have Wikipedia biographies. After that, she says, she’ll tackle a larger project: ensuring all women inventors in Hagley’s collection have Wikipedia articles.

Above, you can see the quantitative impact this group has had over the past year. Their hard work (adding 81,000 words!!) has reached nearly 3 million people in less than a year. Now, anyone with access to the internet can learn about Anne Burlak, a labor organizer from Pennsylvania who helped shape labor standards for textile unions. Perhaps they’ll read about Cornelia Clarke, a nature photographer from Grinnell, Iowa, or Minnie M. Cox, the first Black postmaster in Mississippi.

3) We brought unique perspectives to Wikipedia

This project had another unintended impact on Wikipedia: 86% of the Wiki Scholars use “She/Her” pronouns.

You’ve probably heard of the Wikipedia gender gap — that far more people who identify as men edit Wikipedia than those who identify as women or different gender identities. The Wikimedia Foundation, the nonprofit that hosts Wikipedia and other projects, releases periodic “Community Insights” reports, which include demographic data on Wikipedia’s editing community. The 2021 Community Insights report shows the progress that’s been made on that front recently: Globally, women made up 15% of contributors, but in Northern America, where Wiki Education’s programs operate, that number is 22%. In contrast, self-reported survey data from across Wiki Education’s programs show that 67% of our program participants identify as women. The Smithsonian collaboration had an even higher percentage of participants who use “She/Her” pronouns: 86%!

Wiki Education has shown time and time again that providing a space for structured learning and discussion — like the weekly Zoom classes — helps new Wikipedia editors tackle the work it takes to write high-quality Wikipedia articles, especially for the first time. On this International Women’s Day, we’re celebrating that we’ve created a supportive learning environment that brings more diversity to the projects. As one Smithsonian Wiki Scholar put it, “REPRESENTATION MATTERS. Our course was focused on populating Wikipedia with notable women. The benefit is that now at least 50 or so new articles will be online for women who otherwise would have no online presence. That matters.” That matters for representation in Wikipedia’s content, and it matters for representation among the editors.

How other institutions can celebrate International Women’s Day and Women’s History Month by developing a similar project

Our team works personally with organizations like the Smithsonian to set up Wikipedia training courses that align with their mission and bring untold stories to Wikipedia. We’re eager to continue this work, but we need your help. You can sponsor a course like this one for your team. This unique, fun professional development experience is fulfilling for scholars as they share knowledge with the world, and we can’t wait to bring more subject-matter experts into our community.

If you’re interested in beginning a conversation about buying out a customized course for members or staff of your organization, contact us at partner@wikiedu.org.

Photo credits: Smithsonian Institution Archives, Record Unit 371, Box 02, Folder: December 1975, Image No. 75-14850-05; Bain News Service, publisher, Public domain, via Wikimedia Commons; The White House, Public domain, via Wikimedia Commons; Unknown authorUnknown author, Public domain, via Wikimedia Commons; Baroness Anne-Marguerite-Henriette Hyde de Neuville, Public domain, via Wikimedia Commons.

Other languages: Bahasa Indonesia, Deutsch, English, Kirundi, español, français, norsk bokmål, polski, português, português do Brasil, suomi, svenska, čeština, русский, українська, עברית, العربية, বাংলা, 中文, 日本語, ꯃꯤꯇꯩ ꯂꯣꯟ, 한국어

Latest tech news from the Wikimedia technical community. Please tell other users about these changes. Not all changes will affect you. Translations are available.

Problems

• There was a problem with some interface labels last week. It will be fixed this week. This change was part of ongoing work to simplify the support for skins which do not have active maintainers. [1]

Changes later this week

• The new version of MediaWiki will be on test wikis and MediaWiki.org from 8 March. It will be on non-Wikipedia wikis and some Wikipedias from 9 March. It will be on all wikis from 10 March (calendar).

Tech news prepared by Tech News writers and posted by bot • Contribute • Translate • Get help • Give feedback • Subscribe or unsubscribe.

Getting all the government agencies of the world into Wikidata is obviously a big, hairy and audacious goal, which means it’s just in line with everything else we do on Wikidata!

There is a saying, there is only one way to eat an elephant, a bite at a time. The way we usually organize when eating these bites is in WikiProjects, and so, of course, we have them on Wikidata as well.

An introduction to WikiProject Govdirectory

Inspired by WikiProject EveryPolitician, and quite natural for this project, the WikiProject Govdirectory is divided per country. Getting to know the context and all nuances of a country can be quite tricky, and the quality and availability of sources vary wildly.

But even countries are too big bites to chew when even a small country like Sweden has over 600 government agencies. So within each country, we are usually breaking it down even further, often down to each specific type of agency, like municipalities or courts. On this level, it becomes possible to make progress in a short amount of time and make sure that each part is modeled with high quality without overly complex queries.

To easier get an overview of the state of each bite, we have adopted a rating system. We give from 0 to 5 stars, but already when getting to two stars, the data is starting to become usable.

Link to the project: https://www.wikidata.org/wiki/Wikidata:WikiProject_Govdirectory

Wikidata and beyond!

While we love Wikidata, and find editing and making queries a soothing and relaxing pastime, the interface of Wikidata isn’t really tailored for the general internet visitor. So to make the data about government agencies more accessible, and even more so, actionable, we have created an external website that extracts data from Wikidata and presents it in a very different way.

Instead of focusing on the statements and editing and adding data, this interface is optimized for citizens to get in touch with their government officials to solve whatever need they have.

Find the website at: https://www.govdirectory.org/

There could, obviously, be many more uses beyond Wikidata itself, tailored for other groups or use cases. With Wikidata’s information being licensed CC0, imagination is indeed the limit!

Let’s make government agencies on Wikidata even better – together

First, check if someone already started working on the country you are interested in. You might be in luck and can join ongoing work. If not, we have created several guides to get you started with the project pages. 

But the real work, and an essential one, is to find high-quality sources of what agencies exist and how they are structured. This will help you figure out how to model the items on Wikidata. We also encourage looking at other countries and see how they have modeled items and made queries to be able to produce accurate and up-to-date lists of what is in Wikidata.

If you need any help, we have a centralized talk page that is pretty well watched. Every week on Fridays, we organize a collaboration hour in a video chat. You are welcome to join with questions about anything, or just to see what people are working on right now. It is a very informal and casual meeting and no commitments are needed to join.

Do you want to get involved? Add yourself to the participants and start looking around.

SHA-1 Git hashes can be mapped to code review or code repository URL to offer a web visualization with additional context.

The resolve-hash command allows to get such URL from a Git hash, or another VCS reference. It can search Phabricator, Gerrit, GitHub and GitLab currently.

Ouf of the box, it will detect your ~/.arcrc configuration and use GitHub public API. You can create a small YAML configuration file it to add Gerrit and GitLab in the mix.

Install it. Use it.

The resolve-hash package is available on PyPI.

$ pip install resolve-hash

$ resolve-hash 6411f75775a7aa8db
https::⃫github.com/10up/simple-local-avatars/commit/6411f75775a7aa8db2ef097d70b12926018402c1

Specific use cases

Projects moved from GitHub to GitLab

GitLab requires any query to the search API to be authenticated. You can generate a personal access token in your user settings; the API scope is enough, so check only read_api.

Then you can add create a $HOME/.config/resolve-hash.conf file with the following content:

# GitLab
gitlab_public_token: glpat-sometoken

For Wikimedia contributors

Gerrit exposes a REST API. To use it, create a $HOME/.config/resolve-hash.conf file with the following content:

# Gerrit REST API
gerrit:
  - https://gerrit.wikimedia.org/r/

Gerrit will be then queried before GitHub:

$ resolve-hash 311d17f289470
https::⃫gerrit.wikimedia.org/r/c/mediawiki/core/+/768149

Note if you’ve configured Arcanist to interact with phabricator.wikimedia.org, your configuration in ~/.arcrc is used BEFORE the Gerrit one. Tell me if you’re in that case, we’ll allow to order resolution strategies.

What inspired this project?

Terminator allows plugins to improve the behavior of the terminal. Some plugins allows to expressions like Bug:1234 to offer a link to the relevant bug tracker.

What if we can detect hashes, especially VCS hashes, to offer a link to the code review system, like Phabricator or Gerrit, or at least to a public code hosting facility like GitHub?

What’s next?

We can add support for private instances of GitHub Enterprise and GitLab. Code I wrote in VCS package is already ready to accept any GitHub or GitLab URL, and is so prepared to accept a specific instance, so it’s a matter of declare new configuration options and add the wrapper code in VcsHashSearch class.

A cache would be useful to speed up the process. Hashes are stable enough for that.

Write a Terminator plugin, so we solve the root problem described above.

The code is extensible enough to search other kind of hashes than commits, but I’m not sure we’ve reliable sources of hashes for know files or packages.

References

• Nasqueron DevCentral
  • Kanban workboard
  • Repository
  • Report an issue
• Python package repository
  • Package on PyPI
  • README

22/02/2022-28/02/2022

Mapping campaigns

• Andrés Gómez Casanova reported > on the progress of the ‘Note-a-thons’, an initiative of the MaptimeBogota group, which started this year and aims to resolve open notes in OSM in cooperation with other Latin American communities.

Mapping

• The Polish OSM community presented a mapping project to help Ukrainian refugees by preparing a map (Menu in , , ) that shows refugee reception points and seeks to add Ukrainian names for Polish cities.
• Fizzie41 reported on how the Australian mapper community has made progress on resolving open OpenStreetMap notes.
• Richlv described a procedure to clean up untagged ways that are not members of any relations using JOSM.
• Before-and-after micromapping examples have become popular, particularly on the OSM subreddit.
  User whatismoss has now extended
  the genre with a diary post about a shopping mall involving both 3D and indoor mapping.
• Voting on the following proposals has closed:
  • transformer=*, to refine power transformers’ classification to make it clearer and remove redundancy with substation=*, was successful with 19 votes for, 0 votes against and 1 abstention.
  • boundary=forest, the latest version of a tagging scheme for forest boundaries (managed or unmanaged forests), was also successful with 18 votes for, 0 votes against and 0 abstentions.

Community

• The next online meeting of the Latin American OSM community is on Saturday 26 March. OSM Mexico will be in charge of the coordination and more information is available on the event’s wiki page > .
• OSM Fukushima members have released videos introducing weeklyOSM 601 and 602 . They discover buildings in Tonga with extended roofs and discuss how to map them.
• Ilya Zverev shared > his personal view about mapping in difficult times and the risks of mapping objects under dispute, but also expresses the value of OSM in support of humanitarian activities.

OpenStreetMap Foundation

• Mikel Maron shared his view on the draft resolution on membership prerequisites.
• The OSMF board has rejected the application by MapUganda for local chapter status. A key reason was that there is insufficient separation between MapUganda’s commercial and community activities.
• Heather Leson, in her diary, provided an update on how the HOT Governance Working Group, which she leads, plans to look at HOT governance moving from a governance model with membership control of the Board to a more diverse and inclusive model, and asked for volunteers to assist in this process of organisational change.The second part of the diary entry concerned current activities of the LCCWG (Local Chapters and Community) related to Codes of Conduct (CoC).

Education

• Ever wanted to know what is under the hood of the OSM-driven route planner for cycle touring, cycle.travel? Richard Fairhurst presented an in-depth look at his site as part of the talk programme of the (virtual) 2022 Cycle Touring Festival.
• raphaelmirc recommended > a number of OSM Brazil videos (in Portuguese) created to improve mapping in OSM.
• User unen reported on the aims and objectives of HOT’s ‘Asia Pacific OSM Help Desk’ and invited people to book an appointment for one of the sessions available every Monday (09:00 UTC) and Wednesday (06:00 UTC).

Maps

• Anne-Karoline Distel described in her diary how she created a 3D-printed model, using OSM data, of the churchyard in Magorban for visually impaired people.
• On gnulinux.ch > Ralf Hersel introduced readers to the osMap project, which displays OpenStreetMap data using names in selected Latin character set languages worldwide. For example English, French, Spanish …

Programming

• An OSM wiki page has been created for the 2022 Google Summer of Code. Compared with previous years, participants will be required to have a more in-depth understanding of both OSM and software components of the project. Mentor organisations will be informed by Google on 6 March.
• Pieter Vander Vennet wrote about the new implementation of MapComplete’s language picker.
• RadicallyOpenSecurity (ROS) has audited the security of Nominatim’s codebase and data.
• Trufi’s lead developer gave a video guided tour of the Trufi code base to a group of new volunteer developers.

Releases

• The February 2022 maintenance version (17.0.3) of Vespucci has been released.
• A February 2022 version of OrganicMaps has been released for Android and iOS. The most important changes concern updated map data (4 February), fixes in routing and fixes in styles (e.g., parking, house numbers and railway stations).
• StreetComplete v41 now allows users to measure the widths of streets and cycleways, as well as the height below overhead obstructions, using smartphone cameras with augmented reality. You can read about the results of the first tests regarding expected accuracy of AR measurement on GitHub.

Other “geo” things

• Russian geographers have published > an open letter against military operations in Ukraine, signed by more than 1800 geographers.

Upcoming Events

Where	What	Online	When	Country
Xexéu	Mapathona na Cidade Xexéu – PE -Brasil – Edifícios, Estradas, Pontos de Interesses e Área Verde		2022-03-05 – 2022-03-06
	OSMF Engineering Working Group meeting		2022-03-07
Meatu	Map Tanzania to help protect girls from FGM for International Womens’ Day	✓	2022-03-08
Marburg	FOSSGIS 2022		2022-03-09 – 2022-03-12
Michigan	Michigan Meetup	✓	2022-03-10
München	Münchner OSM-Treffen		2022-03-10
Berlin	165. Berlin-Brandenburg OpenStreetMap Stammtisch	✓	2022-03-10
Lyon	EPN des Rancy : Technique de cartographie et d’édition		2022-03-12
臺北市	OpenStreetMap x Wikidata Taipei #38		2022-03-14
San Jose	South Bay Map Night	✓	2022-03-16
	149.Treffen des OSM-Stammtisches Bonn		2022-03-15
Lüneburg	Lüneburger Mappertreffen (online)		2022-03-15
London	Geomob London		2022-03-16
Großarl	4. Virtueller OpenStreetMap Stammtisch Österreich		2022-03-16
City of Subiaco	Social Mapping Sunday: Shenton Park		2022-03-20
Habay-la-Neuve	Réunion des contributeurs OpenStreetMap, Habay-la-Neuve		2022-03-21
Barcelona	Geomob Barcelona		2022-03-22
City of Nottingham	OSM East Midlands/Nottingham meetup (online)	✓	2022-03-22
	[Online] OpenStreetMap Foundation board of Directors – public videomeeting		2022-03-24
Perth	Social Mapping Online	✓	2022-03-27

Note:
If you like to see your event here, please put it into the OSM calendar. Only data which is there, will appear in weeklyOSM.

This weeklyOSM was produced by Nordpfeil, PierZen, SK53, Sammyhawkrad, SomeoneElse, Strubbl, TheSwavu, YoViajo, derFred.

The Wikimedia movement’s commitment to provide reliable, verifiable information to the world becomes even more critical in times of crisis. The ongoing invasion of Ukraine has already caused unimaginable pain and suffering and impacted millions. Yet in times of upheaval, from pandemics to political turmoil to natural disasters, Wikimedians come together in the service of our collective mission. People are coming to the Wikimedia projects to learn facts, and Wikimedians around the world are collaborating to share their knowledge.

In addition to the work being done on the Wikimedia projects to document this crisis in 100 languages, people across the movement are hard at work to support the affected communities. In the most wiki way, we are starting a page to coordinate efforts on Meta-Wiki – and your help is needed. Anyone in the movement is encouraged to list their Wikimedia activities related to this crisis or ideas they have to help, so that we can collaborate and support each other where needed.

The Wikimedia Foundation stands in solidarity with the communities–those directly affected in Ukraine and all others who work to protect access to free knowledge. We are also reviewing the potential impacts that this crisis and the corresponding threats of censorship being made by the Russian government could have on the entire movement. We remain committed to sharing information as quickly as we are able, and we look forward to hearing from others across the movement.

Wikipedia’s articles about US federal laws tend to have high numbers of pageviews — despite the fact that they’re often incomplete. Like all articles, those about public policy tend to focus in on what was of particular interest to the volunteers who have contributed to it in the past, leaving other sections missing or underdeveloped. That’s where Nicole Mathew’s Capstone Course in American Politics came in. Her Oakland University students each took a federal law from 1947 to 2012 and updated them. Since the students’ work this fall, the articles they edited have been viewed more than 300,000 times.

One popular article was that of the National Security Act of 1947, which restructured the military and intelligence agencies in the United States post-World War II. The student overhauled the article, which used to have only seven paragraphs and six citations. Today, thanks to the student’s work, it’s a fully developed article with twenty-one citations as well as detailed descriptions of its legislative history and provisions.

Another article the class expanded was the Civil Rights Act of 1960, which addressed voting rights and a number of other civil rights issues. The student editor added significant information about the background and legislative history of this act, helping place it in its historical context.

Another civil rights legislation article to get improved was the Civil Rights Act of 1968, the landmark law establishing hate crimes, the Indian Civil Rights Act, the Fair Housing Act, and the Anti-Riot Act. Since 2019, the article had included a warning banner asking readers to expand each section about the ten Titles in the act; the student editor finally did, extensively documenting what each one covers. The student also added sections to the background and legislative history of the article.

Like civil rights, immigration is still today a topic of legislative discussion. A student editor in the course expanded Wikipedia’s coverage of the Immigration and Nationality Act of 1965, significantly documenting its legislative history. The student also added information about its background and more effectively detailed the provisions in the legislation.

Before another student editor worked on it, Wikipedia’s article on the Fair Packaging and Labeling Act was what’s known on Wikipedia as a stub — a short article without much information or many citations. A student editor significantly expanded it, adding sections on the act’s background, legislative history, and provisions.

Other students in the course also added information to the Freedom of Access to Clinic Entrances Act, the McKinney-Vento Homess Assistance Act, the Public Health Cigarette Smoking Act, and the Health Insurance Portability and Accountability Act, or HIPPA, which has been in the news recently.

In all these examples, students are better enabling United States citizens to understand the laws of their country — the context they emerged from, the path they took to become law, and what they state. Having an informed citizenry requires having neutral, fact-based information about laws and other public policy topics — a key role Wikipedia can play.

If you’re a politics or public policy instructor interested in having your students help improve Wikipedia’s coverage, reach out! Visit teach.wikiedu.org for more information.

Image credit: Ralf Roletschek (talk) – Infos über Fahrräder auf fahrradmonteur.de Wikis in der Ausbildung, CC BY-SA 3.0, via Wikimedia Commons

Wikidata tinkers with the concept of maxlag that has existed in MediaWiki for some years in order to slow automated editing at times of lag in various systems.

Here you will find a little introduction to MediaWiki maxlag, and the ways that Wikidata hooks into the value, altering it for its needs.

As you can see above, a high maxlag can cause automated editing to reduce or stop on wikidata.org

MediaWiki maxlag

Maxlag was introduced in MediaWiki 1.10 (2007), moving to api.php only implementation in 1.27 (2016).

If you are running MediaWiki on a replicated database cluster, then maxlag will indicate the number of seconds of replication database lag.

Since MediaWiki 1.29 (2017) you have also been able to increase maxlag reported to users depending on the number of jobs in the job queue using the $wgJobQueueIncludeInMaxLagFactor setting.

The maxlag parameter can be passed to api.php through a URL parameter or POST data. It is an integer number of seconds.

If the specified lag is exceeded at the time of the request, the API returns an error (with 200 status code, see task T33156) like the following

Manual:Maxlag parameter

Code language: JSON / JSON with Comments (json)

Users can retrieve the maxlag at any time by sending a value of -1 (the lag is always 0 or more, so this always presents the error.

Generally speaking the maxlag on most sites with replication should be below a second.

Users of Wikimedia sites are recommended to use a maxlag=5 value, and this is the default in may tools such as Pywikibot. You can read more about this on the API Etiquette page.

Modifying maxlag

Back in 2018 in response to ongoing Wikidata dispatch lag related issues, we implemented a way to modify how and when the maxlag error was shown to users from extensions. Conveniently the maxlag error already included a type value, and we had the plan to add another!

The new hook ApiMaxLagInfo was born (gerrit change, documentation)

Receivers of the hook will get the MediaWiki calculated $lagInfo, and can decide if they have a system that is more lagged than the current lag they have been passed. If they do, they can overwrite this $lagInfo and pass it on.

In the diagram below we can see this in action

• MediaWiki determins the maxlag to be 0.7s from its sources (replication & optional jobqueue)
• Hook implementation 1 determines its own maxlag to be 0.1s, and decides to not change the already existing 0.7s
• Hook implementation 2 determines its own maxlag to be 1.5s, so it replaces the lower 0.7s
• The final maxlg is 1.5s and this is what is used when checking the maxlag parameter provides by the user, or when displaying the lagged value to the user

Factors

As with the optional MediaWiki job queue maxlag integration, all usages of the ApiMaxLagInfo hook generally come with their own configurable factor.

This is due to the expectation that ~5 seconds of lag is when automated clients should back off.

For some systems, we actually want that to happen at say 2 minutes of lag, or when the job queue has 1000 entries. The factor allows this translation.

Code language: PHP (php)

Dispatching

Dispatching or change propagation has been part of Wikibase since the early days. This mechanism keeps track of changes that happen on Wikibase, emitting events in the form of MediaWiki Jobs to any clients (such as Wikipedia) that need to be notified about the change for one reason or another.

Historically dispatching has had some issues with being slow, which in turn leads to updates not reaching sites such as Wikipedia in a reasonable amount of time. This is deemed to be bad as it means that things such as vandalism fixes take longer to appear.

Before adding dispatch lag to max lag the Wikidata team already had monitoring for the lag, and would often run additional copies of the dispatch process to clear backlogs.

You can find numerous issues relating to dispatch lag issues, and before this was added to maxlag Wikidata admins would normally go around talking to editors making large numbers of edits, of blocking bots.

• T253746: high dispatch lag in Wikidata (27 May 2020)
• T178624: enwiki dispatch lag is 8 hours
• T193349: Ongoing dispatch problems (April 2018)
• T194602: Reoccuring dispatch problems (May 2018)
• T171263: Wikidata Dispatcher and Job Queue is overflowed

Dispatch lag was origionally added as a type of maxlag in 2018, and was the first usage of the ApiMaxLagInfo hook.

The value used was the median lag for all clients with a factor applied. This was calculated during each request, as this median lag value was readily available in Wikibase code.

Code language: PHP (php)

Later in 2018 we inadvertently made dispatching much faster. The whole system was rewritten in 2021 as part of T48643: [Story] Dispatching via job queue (instead of cron script), and a decision was made to no longer include dispatch lag in maxlag.

Queryservice updates

The query service update lag was added as a type of maxlag on Wikidata since 2019 due to the ongoing issues that the query service was having staying up to date with Wikidata. You can find an ADR on the decision in the Wikidata.org MediaWiki extension.

The value used was calculated in a maintenance script that is run every minute. This script takes the lastUpdated values of query service backends as recorded in prometheus looking for the backend that is the next most lagged server from the median. This value is then stored in a cache with a TTL of around 70 seconds. (code)

Code language: PHP (php)

During every request, this cached value is then checked and a factor applied. (code)

Code language: PHP (php)

In 2021 the Wikidata Query Service had fully switched over to its’ new streaming updater which should mostly tackle the lag issues.

The post Wikidata maxlag, via the ApiMaxLagInfo hook appeared first on addshore.

Цей пост також доступний українською мовою.

On 1 March 2022 the Wikimedia Foundation received a Russian government demand to remove content related to the unprovoked invasion of Ukraine posted by volunteer contributors to Russian Wikipedia. The Wikimedia Foundation and the movement we are part of have never backed down in the face of government threats to deny people their fundamental human right to access free, open, and verifiable information. 

The information available on Wikipedia is sourced and shared by volunteers who invest time and effort to ensure that the content is fact-based and reliable. Many continue to do so in adverse circumstances: As the invasion continues, Ukrainian volunteers have continued to add content and make edits to Wikipedia, even in face of deep hardship.

Tuesday’s takedown request threatened censorship. Denying people access to reliable information, at a time of crisis, can have life-altering consequences. We join Wikimedia Russia, the independent Russia based Wikimedia affiliate and the broader community of Russian Wikipedia volunteers in defending the right of  volunteers to continue their diligent work of editing Wikipedia with the most up-to-date and reliable information available related to Russia’s invasion of Ukraine. 

Wikipedia is a crucial second draft of history, freely and openly providing important historical and current context of Russia, Ukraine and many other topics. When the invasion of Ukraine began on February 23rd, volunteer editors immediately started to create Wikipedia articles to inform the world about unfolding events. As of March 3rd, the English Wikipedia article about the invasion has been viewed over 11.3 million times, and there are articles about the topic in over 99 languages. 

As ever, Wikipedia is an important source of reliable, factual information in this crisis. In recognition of this important role, we will not back down in the face of efforts to censor and intimidate members of our movement. We stand by our mission to deliver free knowledge to the world. 

The annual Coolest Tool Award celebrated software tools created and used by the Wikimedia community. Here are this year’s winners.

By André Klapper, Staff Developer Advocate, The Wikimedia Foundation

Wiki communities around the globe have diverse use cases and technical needs. Volunteer developers are often the first to experiment with new ideas, build local and global solutions and enhance the experience in our software. The Coolest Tool Award aims to acknowledge that, and make volunteer developers’ work more visible.

The third edition of the Coolest Tool Award took place on 14 January 2022. The event was live streamed on Youtube in the MediaWiki channel. The video is also available on Wikimedia Commons. The award was organized and selected by the Coolest Tool Academy 2021 and friends, based on nominations from our communities.

In total, ten tools were awarded in the categories listed below, and four more tools received honorable mentions. A tool is a piece of software used in the Wikimedia ecosystem. Examples for tools include gagdets, MediaWiki extensions, websites, web services, APIs, applications, bots, or user scripts.

Thank you for all your work!

2021 winners

Experience — Intuitive and easy to use

WikiShootMe is a map of images and geographical locations that helps you find points of interest that need photos. It is both intuitive but also powerful, and it works well on both desktop and mobile.

To quote from a nomination, “I often show this tool to newcomers. It really helps them understand Wikidata and the relationship with Wikimedia Commons.”

Developer — Tools that primarily serve developers

Quarry is a public interface to query replica databases of public Wikimedia wikis, via a web browser. You can also share your queries with others.

Newcomer — New tools or tools by new developers

GlobalWatchlist shows your watched pages across all wikis on a single page.

Diversity — Tools that help include a variety of people, languages, cultures

Humaniki allows to explore the gender gap in content of Wikimedia projects by dimensions such as country, language, or date of birth.

Quality — Tools to improve content quality

Depictor is a game for filling in the gaps of Structured Data on Wikimedia Commons by adding depicts statements to images. It is mobile friendly and a fun way to make image search better.

Tiny — Small tools and tools that do one thing well

diffedit is a user script that enables editing directly from viewing the differences between two versions of the same wiki page. It is very useful for patrolling.

Versatile — Tools that support multiple workflows

Convenient Discussions is a user script that allows the user to post and edit comments without switching to a separate page. You do not need to search code for wiki markup, read talk pages through diffs, or deal with edit conflicts.

Reusable — Serves many wikis and projects

Wudele lets you quickly schedule a poll or an event by setting up potential options and letting others vote on the best time or choice.

Editor — Tools that augment editing

PAWS is a Jupyter notebook deployment. It allows to create and share documents which contain live code. You can run scripts that help with creating visualizations or write technical documentation and tutorials that help others.

Eggbeater — Tools in use for more than 10 years

Earwig’s Copyvio Detector finds possible copyright violations in Wikipedia articles by searching the web, following the article’s links, and checking with external image databases. It is multilingual and an essential tool for maintaining Wikipedia’s quality.

Honorable mentions

• CopyPatrol allows identifying recent copyright violations on several Wikimedia projects.
• MediaWiki CLI makes it easy to set up a MediaWiki instance, services for development purposes, and to run many integration tools.
• VizQuery lets you query Wikidata by property and property value. It offers autocomplete and the search results show images.
• And for WBStack, let’s quote one of its users: “The true promise of Wikidata comes from the ability to build a federation of Wikibases all using the same data primitives, and WBStack is making that promise a reality by letting anyone create a fully functioning Wikibase at the press of a button.”

Big thanks to everyone who nominated their favorite tools, everyone who contributed code, and everyone who spread the word about the Coolest Tool Award!

For more details on the awarded projects, please see the Coolest_Tool_Award/2021 page.

andré — for the 2021 Coolest Tool Academy

The Wikimedia Foundation’s Global Advocacy team works to protect and promote a global regulatory environment in which everyone can freely share in the sum of all knowledge. The month of February 2022 has been a busy one when it comes to legislative and regulatory developments around the world that shape people’s ability to participate in the free knowledge movement. In case you blinked, we’re happy to catch you up. In this article we review the most important developments that have preoccupied us and the actions we’ve taken to advance fundamental rights online such as the freedom of expression, privacy, and access to knowledge. We’ll also highlight the team’s work to protect the public-interest Internet, and our vision of an online ecosystem in which everyone can freely produce, access, share and remix information. As a global movement, Wikimedians can uphold this vision together.  

To learn more about our team and the work we do, follow us on Twitter (@WikimediaPolicy) or sign-up to the Wikimedia public policy mailing list. The team’s Meta page is under construction.

US Legislative Developments

• EARN IT Act: The Global Advocacy team opposed the EARN IT Act, which would significantly weaken protections that online intermediaries, like Wikipedia, have from liability concerning child sexual abuse material (CSAM) and undermine encryption. The Foundation has signed an open letter opposing the bill alongside a coalition of more than 60 other concerned organizations, which was published by the Center for Democracy and Technology. The team also published a blog post outlining the negative impact that the legislation, and others like it, would have on freedom of expression, privacy, and open knowledge projects like Wikipedia. Although the legislation passed out of the US Senate Judiciary Committee, the team’s blog post and open letter were introduced into the official record. 
• Copyright and related rights: The Public Policy team advocated for copyright standards that would support the free and open nature of the internet on two counts. First, we signed an open letter against the Journalism Competition & Preservation Act (JCPA), which proposed granting media organizations the right to restrict third parties from linking to external content. These basic fair uses are essential to millions of websites and online services like Wikipedia, as they use links to connect content to online communities large and small. To further advocate for copyright norms that enable the free exchange of knowledge, we also submitted comments to the US Copyright Office ahead of upcoming proceedings evaluating technical standards for protecting copyrighted works online. 

Latin America and the Caribbean

• Chile: The Chilean Congress has been considering a very concerning bill to regulate digital platforms since September 2021. If approved, the bill has the potential to become a misguided influence on similar regulations throughout the region. Both international and Chilean groups, including Wikimedia Chile and WMF, have expressed concern over the overly broad nature of the bill, lack of consultation with civil society, and last-minute hearings around the legislation. The Global Advocacy team has been supporting Wikimedia Chile to analyze the bill and its legal implications for open knowledge projects like Wikipedia.

Asia

• SIM Card Registration Act in the Philippines. We endorsed a coalition letter to the President of the Philippines urging him to veto the SIM Card Act. The act targets telecommunications and social media entities in such a way that ultimately criminalizes pseudonymity online. The Act forbids the use of pseudonyms when registering accounts with social media, establishing a minimum of 6 years in prison and/or a harsh fine. It sets a worrying precedent of curtailing freedom of expression and privacy in a way that could apply to other online sites, including Wikimedia projects. 
• Myanmar Draft Cybersecurity Law: A draft Cyber Security Law in Myanmar has been introduced one year after a military coup deposed the country’s democratically elected government. The law includes overly broad and poorly defined prohibitions and requirements that are inconsistent with international standards related to freedom of expression. Additionally, the military would have the legal authority to order content deleted, block digital platforms, seize individuals’ devices, and criminalize the use of VPN all without due process. Civil society and human rights organizations have supported a statement by the Global Network Initiative (GNI) calling for the withdrawal of the law. 

European Union

• Our colleagues from the Free Knowledge Advocacy Group EU have been promoting the interests of EU Wikimedia Affiliates. The Digital Services Act (DSA) is in trilogue, i.e. the three main EU bodies have adopted their respective positions and are now negotiating over a  common version. We support language adopted by the Parliament that differentiates between a) terms of use of a platform operator and b) the rules set by the communities that use the platforms. In addition, we endorse the Parliament’s view on intermediary liability that leaves room for community-lead content moderation. 

Additional Developments

• Online Safety Bills: the Global Advocacy team published a blog post on the common pitfalls of online safety regulations. Such bills generally address the symptoms rather than the causes of online harms and, in the process, threaten open knowledge communities and individuals’ fundamental human rights around privacy, freedom of expression, and access to knowledge.  Online safety bills have been proposed in Australia, the UK, and Canada, and we are closely following the debates around them.
• Human Rights Policy: The Global Advocacy team participated in an Office Hour with the Board of Trustees on 17 February to brief the Movement on the recently adopted Human Rights Policy and to answer questions from the community. The briefing emphasized the Foundation’s commitment to protecting and upholding the human rights of Wikimedia users and consulting the community. 

This post is also available in English.

1 березня 2022 року Фонд Вікімедіа отримав від уряду Росії вимогу видалити контент щодо неспровокованого вторгнення в Україну, розміщений волонтерами російської Вікіпедії. Рух та Фонд Вікімедіа ніколи не відступали перед загрозами урядів позбавити людей їх основного права на доступ до безкоштовної, відкритої і перевіреної інформації. 

Інформацію, доступну у Вікіпедії, збирають та поширюють волонтери, які витрачають власний час та зусилля, щоб забезпечити надійний, оснований на фактах контент. В той час як вторгнення триває, українські волонтери продовжують додавати новий контент та редагувати Вікіпедію незважаючи на важкі часи.

Запит на блокування інформації російським урядом у вівторок – це пряма загроза цензури. У часи кризи заборона доступу до надійного джерела інформації може мати критичні наслідки для життя людей. Ми приєднуємося до Вікімедіа Росія, незалежної російської філії Вікімедіа і ширшої спільноти волонтерів російської Вікіпедії у захисті права волонтерів продовжувати свою старанну роботу з редагування Вікіпедії, використовуючи найактуальнішу і достовірну інформацію, пов’язану з вторгненням Росії в Україну.

Вікіпедія є важливим джерелом, що вільно та відкрито надає сучасний та історичний контекст щодо Росії, України та багатьох інших актуальних тем. 23 лютого, коли почалось вторгнення в Україну, наші редактори-волонтери відразу почали створювати статті у Вікіпедії, щоб інформувати світ про актуальний стан речей. Станом на 3 березня стаття в англійській Вікіпедії про вторгнення була переглянута більше 11,3 мільйонів разів. Статті на цю тему написані більш ніж 99-ма мовами. 

Вікіпедія була і залишається важливим джерелом надійної та достовірної інформації в умовах кризи. Усвідомлюючи свою важливу роль, ми не відступимо перед спробами піддати цензурі статті та перед залякуванням членів нашого руху. Ми дотримуємося нашої місії – поширювати вільні знання по всьому світу.

The Wikimedia Movement is one of the most important, collaborative, and community driven initiatives in the world, it is also a meeting place to produce knowledge in a participatory way. But what happens when half of the population is underrepresented or not represented at all in these projects?

Currently, only 18% of the content in all Wikimedia projects, including biographies on Wikipedia, are of women. About 25% of movement organizers, who lead events, build community identity, and direct priorities for the movement, are also women. Historically, more men than women have edited Wikipedia. This has a direct result on the kind of information that is covered and how that information is presented.

Change first begins with knowledge and on International Women’s Day this year, the Foundation aims to shed light on the gender bias across the information landscape by sharing the facts and highlighting the Wikimedia movement’s efforts in closing these gaps.

On  March 8th, the Wikimedia Foundation will be hosting a virtual panel discussion #BreaktheBias – Celebrating women’s achievement and taking action for equity with movement leaders who are working to improve gender diversity in the Wikimedia projects and moderated by our very own CEO, Maryana Iskander. It will take place virtually at 16:00 to 17:00 UTC and you can join the event by clicking here. 

Join us to learn more about the movement’s achievements, challenges, and opportunities in closing the gender gap. Together, we can build a more inclusive, equitable, and diverse community! 

Panelists will include: 

Carmen Alcazar – Wikimedian of the Year 2021; President of Wikimedia Mexico, Founder of Editatona.

Erina Mukuta – Project Lead, Wikimedia Community User Group Uganda

Mónica Bonilla-Parra – Project Coordinator, Wikimedia Colombia

Rosie Stephenson-Goodknight – Trustee, Wikimedia Foundation

We would like to thank all the people who participated in the event from so many places in the world, and with so many diverse experiences. It was an honor to have the presence of so many inspiring leaders, and the moderation of our CEO, Maryana Iskander. In this link you can find the recording of the event which was live streamed on our YouTube channel.
We hope to see you again soon!

On Friday, February 18, 2022, the Trust and Safety Policy Team hosted a Panel Q&A event with community members to discuss the relevance of the Universal Code of Conduct(UCoC) and Enforcement Guidelines, particularly to small and medium sized wikis. As you may be aware, the draft for the Enforcement Guidelines has been completed and is  moving to a community ratification vote from 7 – 21 March 2022. 

This panel discussion was organized as part of efforts to create awareness about the UCoC, its Enforcement Guidelines and the upcoming ratification vote. It had about 25 staff and volunteers attending with 4 panelists; Uzoma Ozurumba from the Igbo community, Ruby Brown from the Open Foundation West Africa, Chabota Kanguya from the Chichewa community  and Nahid Sultan from the Bengali community. The event, which lasted for over an hour, covered various issues ranging from the importance of the existence of a UCoC to the need for small/medium wikis to participate in the ratification vote and get their views/interests represented.

One of the main topics of discussion was whether or not the UCoC and its Enforcement Guidelines are important/necessary. Uzoma remarked that the UCoC is particularly important for smaller wikis because they tend to be fragile and vulnerable so having a UCoC will create a culture of accountability, trust, inclusivity and safety. She also added that having the UCoC without the Enforcement Guidelines is like having a “toothless security/guard dog that can’t bark or bite to protect you”. Isaac mentioned that it will create a safer environment for contributors to collaborate in harmony with people from diverse cultures and backgrounds. Nahid also added that smaller wikis have fewer active users and thus fewer resources and are at risk for higher levels of harassment and toxicity so the UCoC Enforcement Guidelines would be useful at creating equitable standards across all communities. 

On why it is necessary for (small/medium sized) communities to vote on the UCoC Enforcement Guidelines, panelists agreed that it is particularly important for community members from smaller wikis to vote to get their voices/views represented in the vote. Uzoma and Isaac particularly mentioned that since the UCoC and EG are going to be binding on everyone, consensus is very important hence the need for everyone to participate. Ruby also added that smaller wikis may feel that their votes could be insignificant because of their size but their opinion and vote still matters and are equally important.

We rounded up the discussion with questions on the U4C and UCoC related training for administrators and functionaries. Panelists echoed the need for that to exist especially for smaller wikis that may not have the existing structures and systems to such a committee to support them with dealing with these issues. 

                The ratification vote for the UCoC Enforcement Guidelines will be happening from March 7, 2022 to March 21, 2022. You can read more about how to vote, the UCoC and the Enforcement Guidelines to feel empowered to vote! Also, you can find a full recording of the panel discussion here.

I was writing some user documentation for RedirectManager just now, and wrote this sentence-and-a-bit:

If you provide a name of an existing page to create as a redirect, an error message will be shown and no redirect will be created. Similarly, if you

I was going to say something about how it's not possible to create a redirect to a page that doesn't exist. This isn't a limitation of MediaWiki, but when I was writing the RedirectManager API I thought it would be good to prevent these "dangling redirects". It wasn't until I came to write the documentation that I realised the most obvious use-case: creating a redirect to a page that you're in the process of creating! As in, while writing a new page, you want to add a shortcut to it — hardly a rare thing, I think.

This is why I really like "documentation-driven development", where one writes the docs first and pretends that they're describing features that already exist. It really does help focus the mind on what's required of the code, and (as in today's example) highlights things that might otherwise be overlooked.

So I'll now go and change the API error to a warning, and not show it at all in the UI (although it might be worth having some indication that the target page doesn't exist).

https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Extension:RedirectManager

Цей пост також доступний українською мовою.
Этот пост также доступен на русском языке.

Update: As the conflict has continued, the Wikimedia Foundation received a Russian government demand to remove content from Russian Wikipedia. You can read the latest update and our response.

The Wikimedia Foundation supports a global movement of volunteers and communities around the world. We stand with our volunteers in Ukraine, Russia, and around the world who are calling for an immediate and peaceful resolution to the conflict.

The invasion of Ukraine has resulted in the senseless loss of life and has also been accompanied by information warfare online. The spread of disinformation about the ongoing crisis affects the safety of people who depend on facts to make life-and-death decisions and interferes with everyone’s right to access open knowledge. Ukrainian volunteers have heroically continued to add content and make edits to Wikipedia and other Wikimedia projects ensuring that people everywhere continue to have access to neutral, reliable information in times of crisis, and demonstrating our shared belief in knowledge as a human right.

The Wikimedia Foundation is actively working with affected communities to identify potential threats to information on Wikimedia projects, and supporting volunteer editors and administrators who serve as a first line of defense against manipulation of facts and knowledge.

We join those calling for a peaceful end to the conflict, and will continue to support the efforts of those contributing to a strong digital commons that keeps knowledge open, neutral and free.