Data Shows Shipt’s “Black-Box” Algorithm Reduces Pay of 40% of Workers
OCTOBER 15, 2020 | COWORKER.ORG
Through collective pay transparency, workers are exposing pay disparities and fighting back against information asymmetry.
When Shipt - the grocery delivery service owned by Target Corporation - launched, it used a transparent pay model to compensate its workers. For every delivery - or “shop,” as workers call them - the worker received a 7.5% commission on the order total, plus $5. Starting in early 2020, Shipt began paying some workers using a new, black-box algorithm that workers call “V2”. On September 30th, Shipt finalized the “V2” roll-out, ditched its transparent payscale, and joined the majority of other gig economy platforms in paying all workers using a proprietary algorithm.
Since Shipt began using “V2” to determine worker pay, workers have been airing grievances about the black-box algorithm lowering payouts and undermining their ability to accurately estimate their wages. Shipt continues to dismiss these claims, insisting that under V2, “average base pay levels remain consistent overall, and slightly higher in some markets.”
After one week on @Shipt's new pay steal (scale) structure, I'm down $80. It's looking like Shipt will skim $400 from me this month.
— shipt_shopper (@shipt_shopper) October 1, 2020
.@shipt completely ignored our calls for transparent pay. Their greed to cut pay against overwhelming objections & moves to censure us by removing critics from company controlled groups is shameful! Shipts goal is clear: to increase profits at our expense! https://t.co/Z2P4jZTzAQ
— ShiptShopper (@ShopperShipt) September 11, 2020
As the “V2” rollout began, shoppers started collecting their payout information - saving screenshots, creating spreadsheets, and compiling them into a collective database. To expand workers’ capacity to crowdsource this information, Coworker.org formed a partnership with Dan Calacci, a Phd student at MIT, who created an SMS chat-bot that transcribes Shipt payout records and analyzes them in real time. The result is the largest existing database of Shipt shopper pay information - 6,503 data points from 213 Shipt Shoppers.
What we found
By comparing what shoppers are earning under V2 to what they would have earned under V1 - Shipt’s previous pay model - Calacci’s analysis of the data offers a rare picture of how Shipt’s transition to a black-box algorithm is affecting worker pay. The report’s main findings include:
- 41% of the shoppers we’ve collected data from are earning less under the new V2 payment scheme.
- Those shoppers that are making less under the new V2 scheme are making 11% less on average per-shop.
- The number of shoppers earning less under V2 is growing, as V2 payouts appear to be declining over time. 60% of shoppers having earned less from V2 in the past week as of this report (Oct. 7th), and the average V2 payout has fallen by about 15% since the algorithm was rolled out to all shoppers.
- However, across all shops and workers, the V2 pay scheme does technically pay slightly more on average - but not all shoppers are reaping this benefit.
The report’s findings stand in contrast to Shipt’s statements about the new algorithm. In aggregate, V2 pays shoppers slightly more than they were paid under V1, but when the experiences of individual shoppers are compared, it becomes clear that while some may be benefiting from V2, others are effectively getting a pay cut. As Calacci explains in the report, “any earnings increases under the V2 algorithm are not being distributed evenly.” “Without more transparency for workers by Shipt or broader, worker-driven research like this project,” he writes, “why this discrepancy exists will remain a mystery.”
To learn more about our findings, our methodology, and the data we used, click here to read the full report prepared by MIT’s Dan Calacci.
Backing up worker grievances
Workers have long been aware of the inconsistencies surrounding the V2 pay model, and have been speaking out for months about its effects. The collected data appears to back up many of their claims. Our analysis shows that while some workers may be making more under the new algorithm, a full 41% have effectively taken a pay cut.
“I feel like it’s a slap in the face to all shoppers who are getting this pay decrease when the shoppers have literally been holding the company up during this global pandemic.” -Sarah
“I'm disabled and barely able to work at all. So when I do work, I need to be paid fair, so I don't have to hope and pray for promos when tips don't come through or orders that I can actually do become scarce.” - Shawn
“Shoppers deserve to be compensated for the work they do, and they deserve to know the pay scale with which they are being paid. V2/V3 takes money from shoppers and allows Shipt to pay shoppers what they feel they deserve. -Lauren
In Figure 1, each square represents a shopper and is color coded based on the average amount they make per shop under V2, compared with the amount they would have made under V1. According to the report, “A majority of workers making less are making between $.50 and $3 less per shop. These differences amount to an average of an 11% pay cut for workers who are making less.”
Lower payouts do not appear to be random. As the report explains:
And there’s another mystery here. Shipt claims that it’s algorithm estimates “effort” for each shop. If it were simply bad at doing this, and had some random error, we’d expect the lower payouts to be evenly distributed across workers. But that’s not what we see. Instead, we see a group of workers who are consistently making less under the new algorithm.
Without more transparency from Shipt, we can only speculate about why this might be. “If this new pay structure is better for shoppers,” explained a shopper named Lauren, “Shipt needs to be transparent as to how they come up with the payout per shop.”
Things might be getting worse
Our data also shows that payouts under V2 appear to be decreasing over time. In the week of October 7th, data shows that the number of shoppers making less increased to more than 60%.
The opaque nature of Shipt’s black-box algorithm means it’s possible that Shipt continues to tweak the algorithm, or adjust the frequency and amount of “promo pay” opportunities that increase payout on a shop-by-shop basis. As The Shipt List - a collective of Shipt shoppers pushing for changes from the company - explains, “Algorithmically generated pay is a very appealing concept for gig companies since pay can be tweaked and lowered at any time without announcement or warning to workers, and opaque ‘tweaks in the formula’ can be called a ‘glitch’ on their side when something goes wrong.”
Without transparency from Shipt, worker-powered pay transparency efforts like this one are one of the few tools shoppers have to track changes in pay, identify inequities, and demand changes from the company.
Shipt workers are taking action
It is worth stating the implications of our report’s findings plainly: in the midst of a global pandemic - as many grocery stores, including Target, agreed to modest pay increases as “hazard pay” - Shipt continued to roll out a new payment model that appears to have reduced the pay of 2 in 5 workers. As these workers put themselves at risk to provide an essential service to Target’s customers, they struggled (and continue to struggle) to adapt to a new pay scale. The contrast between Target’s decision to increase wages for employees, and its pay cut for many Shipt workers points to the company’s increased reliance on a “shadow” workforce that is not guaranteed a minimum wage, granted paid leave or other benefits, and whose compensation model is no longer transparent.
In response to Shipt’s nationwide rollout of the black-box algorithm, Shipt shoppers have announced a nationwide protest and boycott of the platform. Beginning on October 17, shoppers will be refusing to take orders on the platform for three days. They are calling on customers to support the protests by placing no orders during this period. The three day work stoppage will conclude with in-person demonstrations at Target HQ in Minneapolis and Shipt HQ in Birmingham, demanding a return to a transparent pay structure.
To learn more about these actions and how Shipt shoppers are organizing for change, visit the Shipt List website.