Donation Allocation Decision Aid
412 Food Rescue (412FR) is a nonprofit based in Pittsburgh, Pennsylvania. The mission of 412FR is to prevent perfectly good food from entering the waste stream. The organization does so by running daily operations to save surplus food from going to waste by redistributing it to communities in need.
I worked with Dr. Min Kyung Lee from summer 2018 to spring 2019 to design a decision aid to assist 412 employees with donation allocation. The algorithm was collectively built by multiple stakeholders in 412 networks and offers recommended recipients for each donation.
I joined the team as a software engineer & research assistant to build the algorithm. I started working as the main UX designer to design the dashboard interface in Fall 2018.
Project Time: 1 year
Team: Dr. Min Kyung Lee Research Team
Role: Software Engineer, UX Designer
40% of the food produced gets wasted.
412 Food Rescue prevents perfectly good food from getting into waste stream by redistributing it to communities in need.
We created a decision-aid that helps 412 Food Rescue improve the fairness and efficiency of its donation allocation.
An allocation algorithm using the participatory framework
It balances varying interests from different stakeholders in a moral, legitimate way.
A decision-aid dashboard to show and explain results
It is well-incorporated into 412's current allocation workflow and presents the recommendation outcome with easy-to-understand explanations.
✓ Integrated into the current allocation workflow
✓ Present top-recommended recipients
✓ Show detailed information upon request
✓ Map delivery route
Getting to know the context and the problems
Our team worked closely with the service’s stakeholders to build the algorithm and evaluate the framework through a series of studies.
What’s the process of allocating a donation?
What tools and information are dispatchers using when making allocation decisions?
What pain points do they have in the current process?
Research Activity 1 - Contextual Inquiry about donation allocation at 412 FR site
Research Activity 2 - Interview with all stakeholders in the food rescue service system
How do people define “a fair allocation”?
What factors will they consider if asked to make an allocation?
Based on the research, We identified 4 stakeholders in the food rescue ecosystem.
Connection between the three other stakeholders
"I want to make fair and efficient allocation decisions."
Donate food to 412 Food Rescue
"I want to reallocate good food to places in need."
Receive donations from 412 Food Rescue
"It's important for us to deliver healthy and diverse food to our clients"
Deliver donations from donors to recipients
"I prefer delivery tasks with a shorter distance."
Consolidated Insights - Models
Model 1 - 412FR Operational Model
Model 2 - Allocation Decision Model
Consolidated Insights - Identifying Painpoints
Painpoint 1 Fragmented Information
Dispatchers need to refer to other documents/websites.
Painpoint 2 The decision making process relies a lot of dispatchers' past experience
It's hard to train new employees.
Some recipients may be ignored because the dispatchers don’t remember their information.
Some factors may be underestimated. Distance, as the most straightforward information on the screen, is usually valued much more important than other factors.
Painpoint 3 The decision-making process has little input from the other stakeholders
412 Employees are the only ones who are making the decision.
We brainstormed possible solutions to assist the dispatchers in the allocation process. We explored different levels of automation, from completely manual selection to a full automatic allocation process.
A collective participatory framework that enables people to build algorithmic policy for their communities.
Our framework focuses on direct participation in designing algorithmic governance. The key idea of the framework is to construct a computational model representing each individual stakeholder and to have those models vote on their individuals’ behalf.
Based on the background research, we identified several factors that stakeholders considered important when making allocation decisions.
Building Pairwise Model
Participants were asked to pick between two generated recipients in 50 simulated situations. A regression analysis was performed to create a decision model for each person.
Building Scoring Model
We asked participants to create a second model by making scoring rules. They tested how the scoring model works by applying them in 3-5 pairwise comparisons and then adjusted the models accordingly.
Both models were tested on pairwise comparisons. The results were shown to the participants and they were asked to select the one that best represented their voice.
For every donation, we used each stakeholder's computational model to generate an individual rank of recipients. The individual ranks are then aggregated using the Borda voting rule to produce the final ranking of recipients.
We got all donation allocation records from 3/05/2018 to 8/07/2018 from 412 Food Rescue and created a simulation environment to reallocate donations based on the algorithm. Afterward, we compared the allocation results with the original human decisions based on the following metrics:
Average distance to the donor location.
Min & Max donations per recipient received.
SD of donations per recipient received.
Percentage of organizations that have received at least one donation
Recipient's regularity of receiving donations (interval between two donations)
Drive decision making with AI predictions
Goals and Challenges:
Explain the algorithmic recommendations.
Convince users of the validity of the recommendation.
Give users the flexibility to make final decisions themselves.
Information dispatchers used in the allocation
Donor and Recipient location
Contact information and operational hours
Recipient's information (food access level, poverty rate, etc.)
Information provided by the prediction
List of top recipients
Borda score ranking
Other stakeholders' preferences
Exploration 1 - Explaining the rationale behind the recommendation results.
Showing the users simple explanations of the recommendation outcome helps them make informed choices and increases the credibility of the decision-aid. We explored how to use data visualization to explain and justify the results:
Visualization 1: Feature performance
What makes this recipient rank higher? What are its strengths and weaknesses?
Visualization 2: Stakeholder Ranking
How does the recipient rank in different stakeholder groups?
Final Pick and Revision
To help the users identify each recipient's strengths and weaknesses, we applied color-coding to the feature performances. The icon would be highlighted if the recipient ranks the top 10% in this feature. (min/max 10 % depends on correlation.)
Exploration 2 - Interface Architecture
1. User testing
We tested the interface with five dispatchers and five non-staff (who were given basic allocation rules beforehand) to evaluate the decision-aid's effectiveness in helping users make more informed donation-allocation decisions. This process was to simulate the situation for both experienced and new employees. The users were asked to think aloud during the process and participate in interviews afterward.
Problem 1: Information Overflow
412 Food Rescue has expanded its size since summer 2018. It hired 5 more dispatchers to handle the increasing number of donations. In the design process, our team has been evaluating the system with expert dispatchers who would love to review every detail of the candidates. However, the interface turned out to be too complicated for those newly hired employees who were given large amounts of allocation work every day.
Problem 2: Too many options
Some users think it's unnecessary to show all 12 options since they will just go with the first/second one.
2. Incorporating Feedback
We knew there was a need to simplify the interface. To decide what information to keep, I organized a card-sorting activity to ask the users to rank the information by importance. Then I aggregated the results by averaging each component's location.
Show fewer options by default and load more options upon request.
The users can choose the amount of information that they want to read. Information that is less important is hidden by default and can be accessed upon request.
3. Synchronization Accommodating with 412FR's internal organizational changes
Introduction of weekly scheduled donation
December 2018, 412 Food Rescue introduced the weekly scheduled donation feature. It allowed 412 Food Rescue to create models that generate recurring donations from donors to recipients. Thus, I explored ways to incorporate the weekly scheduled donation information in the decision-aid.
I interviewed with the 412 Food Rescue and found out that they would like to give more opportunities to recipients without weekly scheduled donations. Thus I decided to go with option 2. At the same time, I added a section on the dashboard to display the weekly donation information.
Donor's and Recipients' Locations
List of top-ranked recipients
Top 5 options listed by default
Recipient's Detail Information
Contact and Operational Hrs
Feature Data and Performance
Weekly Donation Information
Recap of iteration changes
Less Is More
We reduced the information shown on the detail card.
1. Feature Performance (Why this recipient ranks high)
Features are highlighted in yellow when an organization is in the top 10% of recipients ranked by that factor.
2. Weekly Donation Information
We incorporated this new feature from 412FR that records weekly scheduled donations (introduced in 12/2018)
3. Total Donations in the last three months
4. Additional Information available after scrolling
Many users brought up that they would just go with the top option in the user testing.
The top 5 options are shown by default. Users can request more recommendations by clicking the 'load more' button.
We updated both the interface and decision model to accommodate the 412FR internal change - the introduction of weekly scheduled donations.
What I have learned from this one-year experience?
Great designs don't just appear overnight
– they take time to get right.
Less is More
Communication within the team and with users plays an important role in the product lifecycle.