Racism and digital design: how online platforms can counteract discrimination


After discovering the extent of discrimination against black guests and hosts on Airbnb, researcher Michael Luca and his colleagues developed a toolkit to help managers recognize and mitigate discrimination on online platforms.

Platforms like Airbnb and Uber can reduce the prevalence of discrimination among their users by making wise design choices, says Luca, Lee J. Styslinger III Associate Professor of Business Administration at Harvard Business School, who studies the platform design for over a decade. .

“The choices you make as a manager can have a profound impact on the inclusiveness of the systems you create,” says Luca, who also co-authored a new case study, Racial Discrimination on Airbnb: The Role of Platform. Design. “Platform companies should consider the implications of design choices, including unintended consequences such as discrimination.”

A 2015 study by Luca and colleagues Benjamin Edelman, now an economist at Microsoft, and Dan Svirsky, an economist at Uber, used booking requests from 20 fake Airbnb profiles without photos to assess discrimination among 6,400 Airbnb hosts. in five US cities. To isolate the issue of race, half of the fake profiles were given names that birth records show are common among whites, while half had names common among African Americans.

“The choices you make as a manager can have a profound impact on the inclusiveness of the systems you create.”

Requests made from profiles with African American-sounding names were about 16% less likely to be accepted. According to the study, discrimination was pervasive across price ranges and a variety of neighborhoods, but most rejections came from hosts who had never hosted a black guest. In a related article, researchers found that African American hosts in New York generally charged less than other hosts on the platform for similar listings.

“Research shows that African American guests were rejected more often than white guests,” Luca says. “More generally, the evidence points to disparities on both sides of the market – African American guests and hosts face discrimination.”

After the research was made public, policymakers took notice. Users started pushing back. The hashtag #AirbnbWhileBlack has become trending. In 2016, Airbnb CEO Brian Chesky responded to the findings and publicly acknowledged that the potential for discrimination on the platform had not presented itself to him or his two co-founders before the site launched, a blind spot he attributed in part to the fact that all three of the company’s founders were white men.

Airbnb set up a task force to determine the scope of the problem and evaluate proposed fixes, some of which had been recommended and described by Luca in a Harvard Business Review article co-authored with Ray Fisman. The company has since made a series of announcements about its efforts to measure and mitigate racial discrimination among users, most recently through Project Lighthouse.

Poor design invites discrimination

Airbnb’s early design choices were intended to facilitate a user’s trust in both the platform and other users, but some of those choices had the unintended consequence of enabling discrimination. For example, when it launched in 2008, the names and profile photos of potential guests featured prominently on Airbnb requests that hosts saw before booking. This design contrasted with existing platforms such as eBay, where product images dominated pages and users remained anonymous.

At the time, Chesky reportedly said, “Access is built on trust, and trust is built on transparency. When you remove anonymity, it brings out the best in people. We think anonymity doesn’t has no place in the future of Airbnb or the sharing economy.”

But putting users’ photos front and center turned out to make discrimination easier, according to Luca’s research. In response to research and growing public pressure, hosts now only see a guest’s profile picture once a reservation has been confirmed. Additionally, the company has updated its policies and terms of service. Luca also points to the increased use of Airbnb’s “Instant Book” feature, which allows guests to book rentals before hosts view their profiles, as a step in the right direction.

Through experimentation, Luca says, platform companies like Airbnb can highlight design elements that align with both company performance goals and inclusivity.

“If you’re running an online marketplace,” he says, “you have to ask yourself if discrimination is likely to be a problem and what design choices might mitigate the risk of discrimination. If you’re only optimizing for growth at short-term metrics and ignoring the potential for discrimination, you could create a blind spot.”

A Manager’s Toolkit for Platform Design

According to Luca, the first step to creating inclusive online platforms is for designers and decision-makers to recognize the potential for discrimination.

Luca and Svirsky describe a framework for making inclusive design choices in a forthcoming journal article Marketing Intelligence Review. The following is a condensed version:

To raise awareness. Digital platform builders need to recognize how their design choices and algorithms can discriminate in a market. Managers can be proactive in investigating and addressing the problem. For example, Uber created a cross-functional Equity Task Force comprised of economists, data scientists, lawyers, and product managers to explore issues of discrimination.

Measure discrimination. Many platforms do not know the racial, ethnic or sexual makeup of their trading participants. Regular reporting on issues and successes of users who may be at risk of discrimination can help companies uncover and address issues.

Retain sensitive data. In many cases, a simple but effective change is to withhold potentially sensitive user information, such as race and gender, in the early stages of engagement with the platform.

Automate taking into account algorithmic biases. Automation and algorithms can help reduce bias, as in the case of Airbnb’s Instant Book feature. However, discrimination can also occur through algorithms. Algorithms can be biased by changing their inputs, but this forces managers to think about their diversity and equity goals. For example, LinkedIn redesigned its recruitment search tool to ensure that the gender breakdown of search results matches the gender breakdown for that profession as a whole. If 30% of data scientists are women, a recruiter looking for data scientists would see 30% female candidates in the search results.

Think like an architect of choice. Principles of choice architecture can help reduce discrimination. For example, people tend to use whatever option is set to default, so resetting options to default with inclusivity in mind can be a useful strategy. Companies can also consider increasing the visibility of their anti-discrimination policies to help raise awareness.

Experiment to measure the effects. Platforms can incorporate efforts to measure discrimination into their experimental tests to understand the impact of different design choices.

Be transparent. Platforms should make their work on discrimination issues transparent and open channels of communication with managers and designers. It is also essential to evaluate selectivity measurement methods and associated design changes over time.

About the Author

Kristen Senz is a writer and social media editor for Harvard Business School Working Knowledge.

[Image: RyanJLane]

Related reading

Share your ideas below.


Comments are closed.