Diversity Dashboard that provides gender gap statistics of Wikipedia and its sister projects
Wikipedia is one of the top 2 visited websites globally. Wikipedia holds several million biographies, which form an incredible dataset about WHO the project finds important. However, evaluating how diverse they are will give an understanding of the gender gap in the broader knowledge pool.
Humaniki is a merger of two previous Wikimedia diversity dashboard tools WHGI and Denelezh that help address this challenge. These tools provide gender diversity metrics in the content of Wikipedia and its sister projects.
Proactively encouraging people to write about women and other genders by providing diversity metrics and creating awareness of the gender gap. Creating an Inclusive Web.
4 members (2 Data Scientists, 1 Frontend Engineer, 1 UXR/D)
September - October 2020
UX Generalist (Research + Design)
Qualtrics, Miro, Google Suite, Taguette, Zotero, Figma, Balsamiq, Trello
Semi-structured interviews, Focus group study, Open card sorting
The previous tools were built with an engineering first approach and their websites have huge traffic, so they stand as a proof of concept but the technical team wants to understand the motivations and needs of diversity-focused community members to direct their development efforts in an efficient manner and create an effective and measurable impact.
Decisions to make:
Which features to retain and which ones to discontinue?
What new features will create a high impact?
I followed the participatory design approach wherein I involved a defined set of stakeholders in the research process. I identified the research objectives based on the initial discussions with the stakeholders.
Who are the users?
How the users are using existing tools?
What are their behaviors, goals, motivations, and needs?
Elicit feature requirements by understanding the challenges community members face with the current tools
Identify other diversity metrics needed by the community for collaborative decision making
I conducted a literature review to find other similar initiatives and existing tools to inform our research strategy and identify the target participant pool to define qualifying criteria for the research. I identified 4 participant profiles:
Representatives of existing diversity organizations
Occasional diversity editors
Tech experts working on similar tools
Researchers from gender diversity domain (as gender is a highly sensitive topic, we needed to bring in expert guidance)
Understanding different participant groups guided our selection of the research methodology and the user recruitment plan.
I used qualitative research methodology to understand community frustrations and needs with the goal to elicit feature requirements for the new tool. Behavioral interviewing to discover how the interviewee addressed challenges in specifically related environments.
I planned three types of interview sessions for 3 different participant groups:
N = 10
User: Technical experts, Community representatives, and Researchers.
Focus: Discussing potential integrations, community needs, and interface guidelines to better inform our design.
Open Card Sorting
N = 7
Focus: Prioritizing existing features and understanding the context of the use of the product. with classic co-design activities like open card sorting.
N = 3 groups (6 users)
User: Editors and Representatives from the same wikiprojects
Focus: Understanding internal community dynamics by unearthing the unknown unknowns
We used statistically non-representative stratified sampling to build a sample of 23 community stakeholders.
1. Recruiting community representatives, tech experts, researchers through direct communication channels.
2. Recruiting diversity editors by posting screener surveys on social media platforms and community discussion channels.
Participant Qualifying Criteria (editors):
1+ years of contribution experience
English speaker (researcher’s limitation)
Has contributed or led efforts in the diversity space
(+1 optional) Has used previous tools
50% Male and Female Ratio
Qualtrics Survey Software
Maintaining a Database of respondents (with consent)
Participant Recruitment Posts
Facebook groups of gender diversity organizations
We reimbursed community members who signed up through the screener survey to motivate participation from unknown groups.
Wikipedia Internal Discussion Channel
Information Sheet and Consent Form
Designing the interview protocols for different participant groups and sending all the participant relevant details to maintain transparency and expectations
Information Sheet and Consent Form
Interview Study | Data Gathering
Open Card Sorting activity with editors
To find how different users prioritize existing features and understand the context of the use
To learn about ways in which editors work towards improving the coverage of topics concerning women or other underrepresented groups in Wikipedia.
To understand additional data points needed to make actionable diversity edits.
How expert users currently create awareness and what are the challenges?
Making users share screen and learn how they are using existing tools, as well as how they approach creating awareness about other gaps.
“Learning the creative problem solving of expert users that can be transformed to designing large scale solutions.”
- Sejal Khatri
Data Transcription and Analysis
I triangulated the gathered data and synthesized the findings delivering 4 actionable themes mapped with 10 elicited features and 6 user personas.
From interviews, I developed a deeper understanding of our 4 participant user groups, which guided our definition of 6 identified archetypes of Humaniki’s potential users.
Persona Prioritization Matrix
We created a prioritization matrix to understand the ease of use in using the previous tools in comparison with the value those tools provide to them.
This helped us scope down the designs keeping in consideration our primary persona.
* These personas are mapped based on the researcher’s interpretations of their background and challenges.
We identified four key themes from the list of elicited feature requirements from the community.
1. Improving the usability of the data we already collect by making it more shareable, more searchable, and more re-usable.
2. Expanding the analysis dimensions of the existing data including different attributes of humans, the snapshot date of the data was collected, and the interface language.
3. Providing actionable insights by highlighting editing opportunities on Wikidata and Wikimedia projects.
4. Maintain highly used features and statistics already in use by the community
Implications for Design
We converged our ideas into a list of deliverable features by using a tweaked version of the Centre for Development of Creative Thinking’s matrix (COCD box), which allowed us to select the final feature list based on ease of implementation and impact of the feature.
Shortlisted Features for MVP
Deliverables and Impact
Publishing research findings for different audiences, blog posts for the community, detailed research reports for the Wikimedia Foundation, user epics for development team.
Documenting research ideas to spur future research.
Five shortlisted features for the MVP
Delivering a UX Roadmap to strategize future development efforts.
Potential to increase the user base by more than 50% by catering the non-English speaking communities.
Research Report for Wikimedia Foundation and Community Experts
Blog Posts for Community
Grant Report for Wikimedia Foundation
Evaluative Research (Blurb)