Humaniki

Diversity Dashboard that provides gender gap statistics of Wikipedia and its sister projects

Overview

Wikipedia is one of the top 2 visited websites globally. Wikipedia holds several million biographies, which form an incredible dataset about WHO the project finds important. However, evaluating how diverse they are will give an understanding of the gender gap in the broader knowledge pool.

Humaniki is a merger of two previous Wikimedia diversity dashboard tools WHGI and Denelezh that help address this challenge. These tools provide gender diversity metrics in the content of Wikipedia and its sister projects.

Goal:

Proactively encouraging people to write about women and other genders by providing diversity metrics and creating awareness of the gender gap. Creating an Inclusive Web.

Team

4 members (2 Data Scientists, 1 Frontend Engineer, 1 UXR/D)

Duration

September - October 2020

Role

UX Generalist (Research + Design)

Tools

Qualtrics, Miro, Google Suite, Taguette, Zotero, Figma, Balsamiq, Trello

Methodology

Semi-structured interviews, Focus group study, Open card sorting

Problem Space

The previous tools were built with an engineering first approach and their websites have huge traffic, so they stand as a proof of concept but the technical team wants to understand the motivations and needs of diversity-focused community members to direct their development efforts in an efficient manner and create an effective and measurable impact.

Decisions to make:

Which features to retain and which ones to discontinue?
What new features will create a high impact?

Research Background

I followed the participatory design approach wherein I involved a defined set of stakeholders in the research process. I identified the research objectives based on the initial discussions with the stakeholders.

Research Objectives

Identify

Who are the users?
How the users are using existing tools?
What are their behaviors, goals, motivations, and needs?

Key Objectives

Elicit feature requirements by understanding the challenges community members face with the current tools

Identify other diversity metrics needed by the community for collaborative decision making

Secondary Research

I conducted a literature review to find other similar initiatives and existing tools to inform our research strategy and identify the target participant pool to define qualifying criteria for the research. I identified 4 participant profiles:

Representatives of existing diversity organizations
Occasional diversity editors
Tech experts working on similar tools
Researchers from gender diversity domain (as gender is a highly sensitive topic, we needed to bring in expert guidance)

Understanding different participant groups guided our selection of the research methodology and the user recruitment plan.

Research Methods

I used qualitative research methodology to understand community frustrations and needs with the goal to elicit feature requirements for the new tool. Behavioral interviewing to discover how the interviewee addressed challenges in specifically related environments.

I planned three types of interview sessions for 3 different participant groups:

Semi-Structured Interviews

N = 10

User: Technical experts, Community representatives, and Researchers.

Focus: Discussing potential integrations, community needs, and interface guidelines to better inform our design.

Open Card Sorting

N = 7

User: Editors

Focus: Prioritizing existing features and understanding the context of the use of the product. with classic co-design activities like open card sorting.

Group Interviews

N = 3 groups (6 users)

User: Editors and Representatives from the same wikiprojects

Focus: Understanding internal community dynamics by unearthing the unknown unknowns

Participant Recruitment

We used statistically non-representative stratified sampling to build a sample of 23 community stakeholders.

1. Recruiting community representatives, tech experts, researchers through direct communication channels.

2. Recruiting diversity editors by posting screener surveys on social media platforms and community discussion channels.

Participant Qualifying Criteria (editors):

1+ years of contribution experience
English speaker (researcher’s limitation)
Has contributed or led efforts in the diversity space
(+1 optional) Has used previous tools
50% Male and Female Ratio

Qualtrics Survey Software

Maintaining a Database of respondents (with consent)

Participant Recruitment Posts

Facebook groups of gender diversity organizations

We reimbursed community members who signed up through the screener survey to motivate participation from unknown groups.

Wikipedia Internal Discussion Channel

Interview Protocol

Information Sheet and Consent Form

Designing the interview protocols for different participant groups and sending all the participant relevant details to maintain transparency and expectations

Prototol:

For Editors, For Researchers/Organizers.

Information Sheet and Consent Form

Interview Study | Data Gathering

Open Card Sorting activity with editors

To find how different users prioritize existing features and understand the context of the use
To learn about ways in which editors work towards improving the coverage of topics concerning women or other underrepresented groups in Wikipedia.
To understand additional data points needed to make actionable diversity edits.

How expert users currently create awareness and what are the challenges?

Making users share screen and learn how they are using existing tools, as well as how they approach creating awareness about other gaps.

“Learning the creative problem solving of expert users that can be transformed to designing large scale solutions.”

- Sejal Khatri

Data Transcription and Analysis

Synthesizing Findings

I triangulated the gathered data and synthesized the findings delivering 4 actionable themes mapped with 10 elicited features and 6 user personas.

Personas

From interviews, I developed a deeper understanding of our 4 participant user groups, which guided our definition of 6 identified archetypes of Humaniki’s potential users.

Persona Prioritization Matrix

We created a prioritization matrix to understand the ease of use in using the previous tools in comparison with the value those tools provide to them.

This helped us scope down the designs keeping in consideration our primary persona.

* These personas are mapped based on the researcher’s interpretations of their background and challenges.

Key Themes

We identified four key themes from the list of elicited feature requirements from the community.

1. Improving the usability of the data we already collect by making it more shareable, more searchable, and more re-usable.

2. Expanding the analysis dimensions of the existing data including different attributes of humans, the snapshot date of the data was collected, and the interface language.

3. Providing actionable insights by highlighting editing opportunities on Wikidata and Wikimedia projects.

4. Maintain highly used features and statistics already in use by the community

Implications for Design

Feasibility

We converged our ideas into a list of deliverable features by using a tweaked version of the Centre for Development of Creative Thinking’s matrix (COCD box), which allowed us to select the final feature list based on ease of implementation and impact of the feature.

Shortlisted Features for MVP

Deliverables and Impact

Publishing research findings for different audiences, blog posts for the community, detailed research reports for the Wikimedia Foundation, user epics for development team.