top of page

WeBuildAI

Donation Allocation Decision Aid

Background

412 Food Rescue (412FR) is a nonprofit based in Pittsburgh, Pennsylvania. The mission of 412FR is to prevent perfectly good food from entering the waste stream. The organization does so by running daily operations to save surplus food from going to waste by redistributing it to communities in need.

I worked with Dr. Min Kyung Lee from summer 2018 to spring 2019 to design a decision aid to assist 412 employees with donation allocation. The algorithm was collectively built by multiple stakeholders in 412 networks and offers recommended recipients for each donation.  

I joined the team as a software engineer & research assistant to build the algorithm. I started working as the main UX designer to design the dashboard interface in Fall 2018. 

Publication: http://minlee.net/materials/Publication/2019-CSCW-AI_Participation.pdf

​

Project Time:  1 year

Team:  Dr. Min Kyung Lee Research Team

Role:  Software Engineer, UX Designer

40% of the food produced gets wasted.

412 Food Rescue prevents perfectly good food from getting into waste stream by redistributing it to communities in need. 

We created a decision-aid that helps 412 Food Rescue improve the fairness and efficiency of its donation allocation. 
noun_Machine Learning_2300276 (1).png

An allocation algorithm using the participatory framework

It balances varying interests from different stakeholders in a moral, legitimate way. 

A decision-aid dashboard to show and explain results

noun_dashboard_665064.png

It is well-incorporated into 412's current allocation workflow and presents the recommendation outcome with easy-to-understand explanations.

Current Product

Current Solution

interface-components.png

✓ Integrated into the current allocation workflow

✓ Present top-recommended recipients

✓ Show detailed information upon request

✓ Map delivery route

​

​

Interaction Video

Research 

Getting to know the context and the problems

Research

Methods

Our team worked closely with the service’s stakeholders to build the algorithm and evaluate the framework through a series of studies. 

  • What’s the process of allocating a donation?

  • What tools and information are dispatchers using when making allocation decisions?

  • What pain points do they have in the current process?

Research Activity 1 -  Contextual Inquiry about donation allocation at 412 FR site

Research Activity 2 -  Interview with all stakeholders in the food rescue service system​

  • ​How do people define “a fair allocation”?

  • What factors will they consider if asked to make an allocation?

Stakeholders

Based on the research, We identified 4 stakeholders in the food rescue ecosystem. 

dispatcher.png

412 Employee

​Connection between the three other stakeholders

"I want to make fair and efficient allocation decisions."

donor.png

Donor

Donate food to 412 Food Rescue

"I want to reallocate good food to places in need."

client.png

Recipient

Receive donations from 412 Food Rescue

"It's important for us to deliver healthy and diverse food to our clients"

volunteer.png

Volunteer

Deliver donations from donors to recipients

"I prefer delivery tasks with a shorter distance."

Consolidated Insights - Models

Model 1 - 412FR Operational Model

Process.png

Model 2 - Allocation Decision Model

workflow.png

Consolidated Insights - Identifying Painpoints

Painpoint 1  Fragmented Information

  • Dispatchers need to refer to other documents/websites.

tools.png

Painpoint 2  The decision making process relies a lot of dispatchers' past experience

  • It's hard to train new employees.

  • Some recipients may be ignored because the dispatchers don’t remember their information. 

  • Some factors may be underestimated. Distance, as the most straightforward information on the screen, is usually valued much more important than other factors.

​

Painpoint 3  The decision-making process has little input from the other stakeholders 

  • 412 Employees are the only ones who are making the decision. 

Ideation

We brainstormed possible solutions to assist the dispatchers in the allocation process. We explored different levels of automation, from completely manual selection to a full automatic allocation process. 

Ideation.png
Typing on a Computer

Building Algorithms

A collective participatory framework that enables people to build algorithmic policy for their communities.

Building Algorithm

General Method

Our framework focuses on direct participation in designing algorithmic governance. The key idea of the framework is to construct a computational model representing each individual stakeholder and to have those models vote on their individuals’ behalf.

Step 1

Identifying Inputs

Based on the background research, we identified several factors that stakeholders considered important when making allocation decisions. 

Step 2

Building Pairwise Model

Participants were asked to pick between two generated recipients in 50 simulated situations. A regression analysis was performed to create a decision model for each person. 

Step 3

Building Scoring Model

We asked participants to create a second model by making scoring rules. They tested how the scoring model works by applying them in  3-5 pairwise comparisons and then adjusted the models accordingly.

Step 4

Model Comparison

Both models were tested on pairwise comparisons. The results were shown to the participants and they were asked to select the one that best represented their voice.

For every donation, we used each stakeholder's computational model to generate an individual rank of recipients. The individual ranks are then aggregated using the Borda voting rule to produce the final ranking of recipients. 

algorithm-model.png

Model Evaluation

We got all donation allocation records from 3/05/2018 to 8/07/2018 from 412 Food Rescue and created a simulation environment to reallocate donations based on the algorithm. Afterward, we compared the allocation results with the original human decisions based on the following metrics:

​

  • Average distance to the donor location.

  • Min & Max donations per recipient received. 

  • SD of donations per recipient received. 

  • Percentage of organizations that have received at least one donation

  • Recipient's regularity of receiving donations (interval between two donations)

Results

results (2).png

Designing Interface

Drive decision making with AI predictions

Goals and Challenges: 

  • Explain the algorithmic recommendations. 

  • Convince users of the validity of the recommendation.

  • Give users the flexibility to make final decisions themselves. 

Designing Interface

Interface Components

  • Information dispatchers used in the allocation

    • Donor and Recipient location

    • Contact information and operational hours

    • Recipient's information (food access level, poverty rate, etc.)

  • Information provided by the prediction

    • List of top recipients

    • Borda score ranking

    • Other stakeholders' preferences

Design Explorations

Exploration 1 - Explaining the rationale behind the recommendation results.

Showing the users simple explanations of the recommendation outcome helps them make informed choices and increases the credibility of the decision-aid. We explored how to use data visualization to explain and justify the results: 

Visualization 1:  Feature performance 

What makes this recipient rank higher? What are its strengths and weaknesses?

feature.png

Visualization 2: Stakeholder Ranking 

How does the recipient rank in different stakeholder groups?

ranking.png

Final Pick and Revision

To help the users identify each recipient's strengths and weaknesses, we applied color-coding to the feature performances. The icon would be highlighted if the recipient ranks the top 10% in this feature. (min/max 10 % depends on correlation.)

final-pick.png

Exploration 2 -  Interface Architecture

Design Iterations

1. User testing

We tested the interface with five dispatchers and five non-staff (who were given basic allocation rules beforehand) to evaluate the decision-aid's effectiveness in helping users make more informed donation-allocation decisions. This process was to simulate the situation for both experienced and new employees. The users were asked to think aloud during the process and participate in interviews afterward.

Problem 1: Information Overflow

412 Food Rescue has expanded its size since summer 2018. It hired 5 more dispatchers to handle the increasing number of donations. In the design process, our team has been evaluating the system with expert dispatchers who would love to review every detail of the candidates. However, the interface turned out to be too complicated for those newly hired employees who were given large amounts of allocation work every day. 

Problem 2: Too many options

Some users think it's unnecessary to show all 12 options since they will just go with the first/second one. 

2. Incorporating Feedback

We knew there was a need to simplify the interface. To decide what information to keep, I organized a card-sorting activity to ask the users to rank the information by importance. Then I aggregated the results by averaging each component's location. 

card-sorting.png

Solution 1: 

Show fewer options by default and load more options upon request.

fewer-options.png

Solution 2: 

The users can choose the amount of information that they want to read. Information that is less important is hidden by default and can be accessed upon request. 

Untitled_Artwork 14.png

3. Synchronization   Accommodating with 412FR's internal organizational changes

Introduction of weekly scheduled donation 

December 2018, 412 Food Rescue introduced the weekly scheduled donation feature. It allowed 412 Food Rescue to create models that generate recurring donations from donors to recipients. Thus, I explored ways to incorporate the weekly scheduled donation information in the decision-aid. 

Group 6.png

I interviewed with the 412 Food Rescue and found out that they would like to give more opportunities to recipients without weekly scheduled donations. Thus I decided to go with option 2. At the same time, I added a section on the dashboard to display the weekly donation information. 

Untitled_Artwork 17.png

New Wireframes

new-wireframe1.png
new-wireframe2.png
new-wireframe3.png
mockup.png

Final Solution

interface-components.png

Map

  • Donor's and Recipients' Locations

  • Delivery Route

List of top-ranked recipients

  • Top 5 options listed by default

  • Feature Performance

  • Load More

Recipient's Detail Information

  • Contact and Operational Hrs

  • Feature Data and Performance

  • Weekly Donation Information

Recap of iteration changes

card.png

Less Is More

We reduced the information shown on the detail card. 

​

1. Feature Performance (Why this recipient ranks high)

Features are highlighted in yellow when an organization is in the top 10% of recipients ranked by that factor.

​

2. Weekly Donation Information

We incorporated this new feature from 412FR that records weekly scheduled donations (introduced in 12/2018)

​

3. Total Donations in the last three months

4. Additional Information available after scrolling

Stakeholder Ranking

​

Easy Decision

Many users brought up that they would just go with the top option in the user testing. 

​

The top 5 options are shown by default. Users can request more recommendations by clicking the 'load more' button. 

list.png
weekly.gif
weekly-donation.png

Synchronization

We updated both the interface and decision model to accommodate the 412FR internal change -  the introduction of weekly scheduled donations.

Simple Explanation

We use simple texts to describe the feature performance by default. Users can view the detailed data and graphs by clicking 'Show Details'

text.png

Take-aways

What I have learned from this one-year experience?

Great designs don't just appear overnight

– they take time to get right. 

Less is More 

Communication within the team and with users plays an important role in the product lifecycle.

bottom of page