On the Effect of Mixed Intelligence on Gig-based Food Delivery

Given the growth in adoption of gig-based crowdsourcing food delivery platforms, couriers often find themselves forced to work longer hours and on multiple platforms to earn a fair living wage. Although various research efforts have been dedicated to make these platforms more efficient and fair, we question the benefit of deploying such a diverse set of models simultaneously, i.e., mixed intelligence, and their effect on the decisions made by couriers. In this work, we run extensive experiments on real data from popular food delivery platforms, modeling various matching algorithms as well as autonomous crowd decision models, and showcase the negative effect of designing models ignoring the assumption of mixed intelligence within this gig-economy.


INTRODUCTION
Food delivery platforms have become more popular with the emergence of gig-based platforms, such as GrubHub and DoorDash1 .As opposed to classical restaurant-based delivery models, in which restaurants independently hire a dedicated set of delivery couriers for their own purposes, in crowdsourced delivery platforms, drivers are not employed or controlled by the platform, but join willingly for their own gain.In crowdsourced delivery, drivers2 complete delivery orders in an opportunistic manner, based on their private constraints and utilities, which implicitly links the quality of service to the availability of drivers in these platforms.
Motivation.Recent research efforts in this domain are focused on developing efficient delivery order assignment models to improve the quality of service and system performance [5,7], with some focused on the fairness of such assignment models when it comes to the driver's experience [3,8].However, all of existing works, as far as we know, assume that their models and algorithms are the only providers of the food delivery service in a region, and that drivers will blindly follow their assignments and/or routing decisions.
We argue that these assumptions are far from reality, due to mixed intelligence, as multiple service providers exist on the market.Not all service providers offer the same model of order assignment, as some make decisions on behalf of drivers while others simply present recommendations, and more often than not, drivers have autonomy over their decisions on which platform(s) to commit to.Furthermore, there has been an increased adoption of the recently coined term "multi-apping", as described in driver help blogs [1,2], in which drivers work for more than platform at a time to increase their income.All of these factors create a complex system in which multiple forms of intelligence, i.e., order assignment models and algorithms, compete against each other, which is what we refer to as mixed intelligence.
Paper Contribution.In our opinion, there is a need to reevaluate existing assignment models in a mixed intelligence scenario, to truly assess the effects of these models on system performance, quality of service, and drivers' benefit.The main objective of this work is to emphasize the importance of evaluating models under the assumption of mixed intelligence, and accordingly, we design and develop an agent-based food delivery simulator to perform such an evaluation.The main contribution of this paper is not the simulator in itself 3 , but rather the presentation of the preliminary results of our investigation on the effect of mixed intelligence on gig-based food delivery.
In this paper, we use real food delivery data from GrubHub [7], to emulate a stream of orders incoming into the system as a whole, and we implement various models of service providers generating various sets of assignments and recommendations to drivers, allowing us to simulate the expected behavior of the couriers given these assignment decisions.Our results indicate that although performance is optimal when there is a monopoly of intelligence, i.e., a single service provider in control of all orders and drivers, it degrades immensely when combined with even just one other source of intelligence.We hope that the results shown in this paper would be a motivating factor for future directions of research in this field; considering mixed intelligence when designing assignment models and algorithms.
Paper outline.In the remainder of this paper, we start with the system model definitions in Section 2 and the order assignment algorithms used for evaluation, followed by the evaluation setup and results in Section 3. Finally, we conclude the paper with a brief review of related work and a discussion of future work.

GIG-BASED FOOD DELIVERY 2.1 System Model
As depicted in Fig. 1, in typical crowdsourced food delivery platforms, the service provider acts as the broker between the clients, represented by their food orders, and the drivers.Drivers are assumed to be individual and rational participants, with personal schedules, constraints, and utilities.They have the flexibility to autonomously pick the orders to complete based on their personal preferences and spatio-temporal constraints, or to accept order assignments from the platform.Moreover, drivers have the opportunity to participate in multiple platforms simultaneously, in an attempt to increase their chances of getting more orders to complete.
Orders are defined by their spatio-temporal endpoints.The spatial endpoints represent the location of the restaurant preparing the food order and the delivery destination of that order.Meanwhile, the temporal constraints represent the time at which the order is ready for pickup, and the time window through which that order can be delivered within reasonable delay.Orders with no flexibility, i.e., a zero-threshold of delay, expect to be picked up right away and routed through the shortest path to their destinations.On the other hand, orders with more flexibility, i.e., with some maximum threshold on delay, tolerate longer routes, possibly allowing drivers to complete other orders along the way.
Service providers vary in their level of involvement in the driver's decision making process.Service providers that have control over a fleet of vehicles are capable of making matching decisions and assigning orders to their drivers, similar to the model in [3,5].On the other end of the spectrum, service providers that act as middleagents offer no guidance or recommendations to the drivers and simply act as a repository of order information, allowing the drivers to make their individual and autonomous decisions 4 .However, the most common type of service provided by these platforms is a recommendation service, in which the provider assigns orders to drivers with the option for drivers to decline these assignments if they don't fit their personal constraints, similar to that offered by Grubhub and DoorDash. 4That is rarely seen in reality, as providers typically get involved in the decision process 2.2 Food Delivery Assignment 2.2.1 Order Matching.This is the most common food delivery assignment model in literature, as shown in Fig. 2, in which the drivers and the customers have no autonomy, and the service provider makes order assignments optimizing some system-level objective.The advantage of that model is that order assignments can be optimized and that service providers can provide better quality of service to their customers and drivers.However, a major drawback of this model is that it is assumes that drivers will accept all assignments made for them and that they are not committed to any other platforms.
Figure 2: In the matching model of food delivery, the service provider makes order assignment decisions on behalf of its drivers.
For the scope of this paper, we implement two algorithms for order matching; (1) Greedy Assignment is an online order assignment algorithm, in which orders arriving in a stream are processed individually upon their arrival, as shown in Algorithm 1.In our definition of the greedy assignment algorithm, a newly arriving order is assigned to the driver that incurs the lowest marginal cost to their original schedule, as a result of adding that new order to it.To be more specific, the cost of assigning a set of orders to a driver's schedule is defined as the sum of delivery delays of each order on that schedule, and delivery delay is defined as the difference between the actual delivery time of an order and its earliest possible delivery time based on the shortest path from the restaurant to the destinations.We note that if the delay to the original schedule exceeds the maximum delay threshold allowed by any order on that schedule, the cost of addition is set to an infinitely large value to indicate the infeasibility of that addition.
In Algorithm 1, when a new order, , arrives to the system, its marginal cost of assignment is computed for each driver, , in the set of available drivers .Then, the driver with the lowest marginal cost is identified, and the order, , is assigned to that driver with the lowest marginal cost,  * .As shown in the example in Figure 3, when the new order, indicated in orange, is added to the first driver's schedule, it adds a delay of 10 minutes to each of the preexisting orders on that driver's schedule, leading to a marginal cost of 20 minutes.Meanwhile, the addition of the new order to the second driver's schedule leads to a marginal cost of 40 minutes.Thus, it's best to assign the new order to the first driver.
(2) FoodMatch is similarly an online assignment algorithm, in which food orders are periodically clustered into batches and then matched to available drivers [5].In the FoodMatch algorithm, the information of available orders and couriers is collected in real time, and periodically an offline assignment decision is made with Adding that new order to the first driver's schedule adds a 10 minute delay to each of their previously committed orders (b) , as opposed to the second driver, which would incur a cost of 20 minute delay per order (c).The greedy assignment algorithm will assign the new order to the first driver with a scheduling cost of 20 minutes.Add  to 's schedule 7: Calculate cost of new schedule ( ′ ) 8: if   is the lowest so far then end if 13: end for 14: Assign  to  * the available information from that period of time.Within a single period, the algorithm is defined over two phases; order batching and order matching.
In the order batching phase, an attempt is made to batch the delivery orders into batches of a maximum size of 3 based on their spatio-temporal characteristics.The batch size of 3 is inferred from the maximum number of orders that a courier can carry on their bikes in India 5 .The small batch size allows for the design of a simple batching algorithm, in which all possible groupings of 3 orders are considered and the batches with minimum schedule cost are created.Finally, in the order matching phase, the generated batches are assigned to the drivers that are closest to the first pickup location of the batch in a greedy fashion.

Autonomous
Delivery.On the other hand, autonomous delivery is the most common delivery assignment model in the reality of gig-based platform, as these platforms market themselves as the alternative option to traditional delivery jobs that offers drivers control over their hours and order decisions.Although seemingly 5 The geographical area on which the original work focuses.
having full autonomy, these platforms still get involved with the driver decisions, albeit indirectly.Typically, the service provider does not provide information about all orders in the vicinity, and only recommend a certain subset based on their internal optimization models.
In autonomous models, it is assumed that we have a stream of orders and a dynamic set of couriers that changes over time.When a new order comes in, as shown in Figure 4, the service provider recommends it to the closest driver first.If the driver accepts it, it is assigned to them, and if not, the order is recommended to the next closest driver, and so on.Drivers make decisions on whether to commit to an order or not based on their own personal utilities and constraints.Some of the most common utility models that we implemented and evaluated are; • Distance, in which the courier decides on whether to commit to a food order or not based on their distance from the restaurant.• Income Rate, in which the courier decides on whether to commit to a food order or not based on the ratio between the revenue incoming and the expected time to be committed to the delivery.• Schedule Cost, in which the courier decides on whether to commit to a food order or not based on the marginal cost of adding the new order to their schedule, similar to the metric used in the Greedy Algorithm defined above.

EXPERIMENTAL EVALUATION
In this section, we use simulations based on real meal delivery traces to evaluate the effect of drivers adopting a diverse set of delivery assignment models on the overall performance and fairness of the system.

Experimental Setup
We developed an agent-based simulator with the purpose of evaluating the effectiveness of various delivery choice and assignment algorithms on the quality of service in food delivery platforms.All simulations were executed on a machine running macOS version 12.1, with 8-Core Intel Core i9 machine and 32 GB of RAM.3.1.1Dataset and road network.Meal delivery information are extracted from real traces obtained from GitHub, which were originally used in [7].The GrubHub dataset contains 240 instances generated from anonymized data from real meal delivery operations at Grubhub.It is a rich dataset, with information on couriers' initial locations and their temporal availability, as well as the spatial information of the restaurants and the actual times at which the meals were ready for delivery.The dataset contains data for 240 days that are derived from a set of 10 seed instances to synthesize food preparation delays and slow traffic, among other things.In this work, we based our simulations on the original 10 seed files, since our objective was to evaluate the assignment mechanisms against each other in a typical setting.As location information in the original dataset is perturbed to ensure anonymity, all locations are converted to pairwise (, ) coordinates, which are imported into our simulator and treated as grid coordinates on a Manhattan-like graph.
3.1.2Simulation process.Depending on the day used, the details of the corresponding locations and times are loaded into the simulator.The simulation time is set to a single day with each time step at the granularity of a single minute; creating a total simulation time of 1440 time steps, following the time granularity within the GrubHub datset.
Couriers / drivers are generated with exact starting locations based on the spatio-temporal properties available in the data traces, and are assumed to only be available to participate in the platform at the times that they are indicated to be online and active.When an order request arrives or when a driver becomes available, we emulate the matching process of their corresponding service provider, based on the type of service offered by that provider.A novel contribution of this work is the implementations of mixed intelligence, to emulate a realistic gig-based economy, in which drivers have access to multiple service providers.

Evaluated Mechanisms.
Due to space limitations, in this section, we present the results obtained from implementing only three different models of assignment; (1) autonomous with drivers optimizing for distance, (2) Greedy, and (3) FoodMatch.We note that due to the definitions and assumptions made in the FoodMatch algorithm [5], drivers can be assigned a maximum of  orders, and they can only subscribe to a single service provider within a day.

Evaluation Metrics.
We measure various sets of metrics, to better evaluate the performance and fairness of the mechanisms adopted on the drivers, customers, as well as the platform as a whole.For the platform, we measure the number of orders completed (order rate), the number of drivers involved in the completion of rides (participation rate).For the orders, we measure the average delay for pickup (measured as the difference between the order's ready time and its actual delivery time).For drivers, we measure the average revenue per driver (measured as the total revenue collected from completed rides), and their average idle time.

Evaluation Results
3.2.1 Autonomous vs. assigned.Our first set of experiments were designed to evaluate the performance of each of the assignment models as a single intelligence in the system, and compare them against each other.No randomization or synthesis of data was performed in these experiments, as we fed the raw GrubHub data traces directly into the simulation.Results shown in Table 1 confirm the correctness of the simulations.Pure autonomous based on distance has the lowest completion rate with the lowest participation rate, since orders are recommended to drivers that are nearby who usually accept them and move directly to them, leading to a mediocre performance.Greedy assignment achieves a 20% higher acceptance rate, since the provider is more involved in the decision process and assigns orders to drivers whose schedules won't be drastically altered from the new assignment.Finally, FoodMatch [5] is superior due to its batching process, which enables drivers to efficiently use their time.The batching period used to obtain these results was 10 units of time, representing 10 minutes of real traces time.

Effect of batch period.
In our second set of experiments, we decided to evaluate the effect of varying the batch period, from 10 to 60 on the performance of FoodMatch, to better understand the algorithm.As the results in Figure 5 indicate that larger batching periods lead to a worse performance, which is expected in a food delivery platform.The interesting pattern we noticed was in the courier participation rate, which initially decreased until it hit a convergence spot at around 40, and then started to gradually increase even with low order delivery rates.We believe that this is due to the assignment algorithm recruiting more drivers in attempts to complete as many orders as possible within a long batch period, Figure 5: Investigating the effect of the length of the batching period on the performance of FoodMatch [5] but even with such an increased recruitment rate, the orders face extreme delays (due to the delay in processing), and end up not being completed.

Effect of mixed intelligence.
In our third set of experiments, we include the notion of mixed intelligence in our simulations, and combine both the autonomous with the assigned models in various ways.In these experiments, we ran an extensive set of simulations with various ratios and types of intelligence models, of which we present some sample results that confirm the motivation behind this work.All results shown below are an average of 20 simulations with different random seeds.
The results in Figure 6 represent the order delivery rate of a combination of three sets of assignment models at different rates.We compare between a setup in which drivers either make autonomous decisions or sign up with FoodMatch, and another in which drivers either sign up with a Greedy provider or FoodMatch.Moreover, we vary these degrees of mixed intelligence from 0 to 1 at increments of 0.1.For example, in the experiment with Autonomous + FoodMatch at a mixed intelligence rate of 0.4, an average 40% of the drivers make autonomous decisions while the rest follow the FoodMatch assignments.As indicated by the results, mixing intelligence within the system reduced the performance greatly.Even with two powerful algorithms such as Greedy and FoodMatch, with individual delivery rates that were 0.7 and 1.0 respectively, combining them leads to a delivery rate that is lower that 0.6 due to the partitioning in decision making.
Next, we repeat the above experiment, focusing on Autonomous + FoodMatch, while varying the batch size of FoodMatch, with results shown in Figure7, we note the extreme degradation in performance, even with only 20% of the drivers choosing to not be part of FoodMatch.We also note the slight increase in completion rate as the batch period increases for the mixed intelligence scenarios, which we attribute to autonomous drivers picking up the slack when batch periods get longer, especially that the rate in increase gets higher with higher mixed rates, such as 0.8.
Finally, we note that this degradation in completion rates has a direct effect on the participation rates as well, as indicated in Figure 7b, which reduced by half in some scenarios.This reduced  participation is a direct negative implication on the drivers, their income, and their livelihood.

RELATED WORK
Matching in food delivery.Originally the food delivery problem was first defined as a special instance of the Vehicle Routing problem with Time Windows [4], and it was focused on optimizing delivery services originating from the restaurants themselves.The notion of scheduling non-dedicated couriers for individual deliveries was first defined in [7], in which both the routing of the couriers as well as the scheduling of orders were considered.Various works have attempted to solve the problem, but with unrealistic and incomplete assumptions [6,9], and until recently only a few pieces of work have attempted to solve the problem under realistic assumptions.In [5], Joshi at al. propose a scalable matching algorithm and evaluated it against real datasets from GrubHub and Swiggy, and in [3], Gupta et al. extended that framework to provide more fair assignments to drivers.More recently, in [8], Singh et al. approached the fair assignment problem from a stochastic point of view, in which they developed a randomized dependent rounding based efficient sampling algorithm, to fairly distributed the load over the drivers, such that their expected travel cost is minimized.

CONCLUSION AND FUTURE WORK
In this work, we highlight the reality of mixed intelligence in gigbased food delivery, and emphasize the importance of evaluating models under the assumption of such mixed intelligence.Our extensive simulation results demonstrate the extreme degradation in performance of even the best assignment algorithms when coupled with just one other form of assignment, due to the partitioning of courier dedication and order availability.
In our future work, we aim to further investigate these models of mixed intelligence, and develop assignment models that are couriercentric, instead of platform-centric, to improve the performance of the system as a whole and accordingly the experience and fairness of the couriers in the system.

Figure 1 :
Figure1: A food delivery platform is composed of three main parties; the drivers, the orders, and the service provider acting as the middle-agent.

Figure 3 :
Figure 3: Example scenario with 2 drivers already committed to some orders, when a new order arrives to the platform (a).Adding that new order to the first driver's schedule adds a 10 minute delay to each of their previously committed orders (b) , as opposed to the second driver, which would incur a cost of 20 minute delay per order (c).The greedy assignment algorithm will assign the new order to the first driver with a scheduling cost of 20 minutes.

Figure 4 :
Figure 4: In the autonomous model of food delivery, drivers have the autonomy to accept or reject the orders assigned to them by the service provider.

Figure 6 :
Figure 6: Mixing intelligence within the system reduces the performance greatly.Even with two powerful algorithms such as Greedy and FoodMatch.

Figure 7 :
Figure 7: Even at the smallest rate of mixed intelligence, the order completion rate is reduced by 10% and participation rate is affected accordingly.

Table 1 :
Performance of different assignment models and algorithms as single sources of intelligence.Results indicate expected performance, in which FoodMatch excels in both order completion rate, delay, and driver idle time.