“Sometimes It’s Like Putting the Track in Front of the Rushing Train”: Having to Be ‘On Call’ for Work Limits the Temporal Flexibility of Crowdworkers

Research suggests that the temporal flexibility advertised to crowdworkers by crowdsourcing platforms is limited by both client-imposed constraints (e.g., strict completion times) and crowdworkers’ tooling practices (e.g., multitasking). In this article, we explore an additional contributor to workers’ limited temporal flexibility: the design of crowdsourcing platforms, namely requiring crowdworkers to be ‘on call’ for work. We conducted two studies to investigate the impact of having to be ‘on call’ on workers’ schedule control and job control. We find that being ‘on call’ impacted (1) participants’ ability to schedule their time and stick to planned work hours, and (2) the pace at which participants worked and took breaks. The results of the two studies suggest that the ‘on-demand’ nature of crowdsourcing platforms can limit workers’ temporal flexibility by reducing schedule control and job control. We conclude the article by discussing the implications of the results for (a) crowdworkers, (b) crowdsourcing platforms, and (c) the wider platform economy.


INTRODUCTION
Research suggests that the temporal flexibility of people working on crowdsourcing platforms is limited by both client-imposed constraints (e.g., strict completion times) and workers' tooling practices (e.g., multitasking).As part of the "just-in-time" platform economy [45], on-demand crowdsourcing platforms are advertised to crowdworkers as offering them temporal flexibility: flexibility in terms of when and for how long they choose to work [86,145].However, research suggests that crowdworkers do not benefit from the temporal flexibility advertised by crowdsourcing platforms [110,114].For example, each client (i.e., private companies or individual consumers) [87] provides their own completion times on jobs, which workers must adhere to [111].In addition, since clients post jobs on-demand, workers have to wait for work and make themselves available to 'catch' new jobs whenever clients make them available.Furthermore, research suggests that the external tools workers use to 'catch' new jobs promote the temporal fragmentation of workers' (a) work-life boundaries (i.e., of workers' schedules) and (b) work practices (i.e., by increasing workers' need to use their time multitasking) [160].Thus, the tools workers use to 'catch' jobs also limit workers' temporal flexibility.
In this article, we explore an additional contributor to crowdworkers' limited temporal flexibility: the design of crowdsourcing platforms, namely requiring crowdworkers to be 'on call'.Being 'on call' is a somewhat known [78,114,149] but undefined and increasingly popular working time arrangement within the platform economy, central to the operation of geographically tethered platform work (e.g., ride-hailing services) [162] and online crowdsourcing platforms alike.In the context of crowdsourcing platforms, we define being 'on call' as a working time arrangement that requires crowdworkers to wait and search for jobs for an undetermined amount of time, often without getting paid, because of the platforms' lack of predictable work availability and lack of work assignment.At least seven platforms are known to have adopted this working time arrangement (i.e., Amazon Mechanical Turk, Clickworker, Hive Micro, Microworkers, Neevo, PicoWorkers, and UHRS) [110], and over time more can be expected given the growing demand for 'flexible' working arrangements [119].However, the lack of predictable work availability and lack of work assignment results in competition between crowdworkers [111,114], who have to accept jobs before other workers on a first-come, first-served basis, in an auction-like system [51].We argue in this article that being 'on call' can further limit workers' temporal flexibility and exacerbate precarious working conditions.Thus, revealing the platform architectures that make the exploitation of workers possible is pivotal to changing the flexibility discourse of the platform economy (e.g., individual freedom and flexibility [8]), and demanding decent work standards [74] for the workers (e.g., realised temporal flexibility [18] and fair pay [157]).
We conducted two studies to investigate the impact of having to be 'on call' on workers' schedule control [103] and job control [156].In the two studies, we asked the following Research Questions (RQs): (1) RQ Study 1: How does having to be 'on call' for work limit crowdworkers' control over scheduling their time?(2) RQ Study 2: How does having to be 'on call' for work limit crowdworkers' control over their work pace?
Thus, Study 1 is a time-use-diary study that investigates how having to be 'on call' can limit U.S.-based workers' control over how they schedule their time.Study 2 is a video analysis study of twelve 90-minute working sessions that investigates how having to be 'on call' can limit workers' control over the pace at which they conduct their work.Workers' time scheduling and work pace are of particular interest in this work as they are predictors of schedule control [103] and job control [63], which are both, in turn, predictors of wellbeing [63,103].
We find that having to be 'on call' impacted: (1) participants' ability to schedule their time and stick to planned hours of work, and (2) the pace at which participants worked and took breaks.The results of Study 1 show that participants' ability to schedule their time and stick to planned hours 18:3 of work was particularly impacted by the lack of predictable work availability on the crowdsourcing platform.Even though participants started and finished work roughly when they intended to, participants worked on average 2 hours less than planned and spent on average 22% of their daily working time on unpaid 'on-call' activities such as waiting and searching for new jobs.In addition, the data suggest that participants' workdays were significantly more fragmented (i.e., broken into more work sessions) than workers planned, with work distributed across twice as many periods of work as desired.This study suggests that a lack of predictable work availability can reduce workers' schedule control by impacting their ability to schedule their time and stick to planned work hours.
The results of Study 2 also show that the pace at which participants worked and took breaks was particularly impacted by the lack of work assignment on the crowdsourcing platform.We observed that during the twelve 90-minute working sessions recorded, participants spent on average 17% of their working time on unpaid 'on-call' activities such as waiting and searching for new jobs.In addition, working on the platform was overall characterised by three distinct periods of work intensity: periods of low, moderate, and high work intensity.For example, during periods of low work intensity, participants worked at a slower pace, filling their unpaid time with activities resembling break-taking due to a lack of available work (i.e., browsing the internet or watching TV shows), whilst monitoring the platform for new jobs.In contrast, participants worked at a higher pace during periods of high work intensity, engaging in task switching to quickly 'catch' new work but not taking any breaks.This study suggests that a lack of work assignment can reduce workers' job control by impacting the pace at which they work and frequency with which they take breaks.
Taken together, the results of the two studies suggest that the 'on-demand' nature of crowdsourcing platforms, which requires workers to be 'on call' for work and accept jobs before other workers, can limit U.S.-based workers' temporal flexibility, reducing their schedule control and job control.Reduced schedule control and job control are known for having negative impacts on health and wellbeing [107].We conclude the article by discussing the implications of the results for (a) the people working on crowdsourcing platforms, (b) the design of crowdsourcing platforms, and (c) the wider platform economy.Overall, this work makes three main contributions that extend and contribute to the existing HCI and CSCW research examining the working conditions of crowdworkers [60,75,82]: (1) A definition of what it means to be 'on call' for work on some existing crowdsourcing platforms.(2) A measure to quantify the amount of unpaid time that crowdworkers have to spend being 'on call' for work.(3) Empirical evidence that being 'on call' for work impacts workers' control over their daily schedule planning (Study 1) and work pace (Study 2).

The Promise of Temporal Flexibility
The platform economy is seen (although not without dispute) as a flexible alternative to traditional job opportunities.For instance, a report published in 2016 by the World Bank describes working within the platform economy as a flexible earning opportunity, where people can work on online platforms from home and set their own schedules [163].However, the narrative of 'flexibility' promoted by institutions such as the World Bank is a source of contention among scholars, who argue that platform work can perpetuate precarious working conditions for the workers on the pretext of flexibility [8].
Nevertheless, more people than ever have to work within the platform economy as a result of the COVID-19 pandemic [88].Working within the platform economy offers a way to earn additional income, which has been important during the COVID-19 pandemic lockdowns and in helping people cope with the cost of living crisis brought on by high inflation in the early 2020s [144].Besides the obvious financial incentives, one of the main selling points of platform economy advocates is workers' flexibility.More precisely, platform work is advertised as offering workers temporal flexibility.For example, Uber drivers reportedly value the ride-hailing service's flexible work schedules and ability to adapt hourly to their time demands [34].Another example of temporally flexible platform work can be found on crowdsourcing platforms.Crowdsourcing platforms are touted as offering the people working on these platforms temporal flexibility because of the short temporal nature of the work [161], known as 'crowdwork', 'microwork', or 'cloudwork'.Examples of work available on these platforms include data entry jobs such as receipt transcriptions, image or video tagging, dataset cleaning, or survey completion [48].
Academic and industry researchers alike widely use crowdsourcing platforms in their work.For example, within CSCW and HCI, researchers have used crowdsourcing platforms to study privacy in online social networks [120] or to understand online news behaviours [16].Within industry, Artificial Intelligence (AI) researchers extensively use crowdsourcing platforms to train Machine Learning (ML) algorithms.Large tech companies such as Amazon, Facebook, and Google, as well as AI start-ups, temporarily hire people from crowdsourcing platforms to build and label large training datasets for ML applications, such as product recommendations, image or speech recognition, or traffic prediction [124].Amid an expansion in the use of AI, the use of crowdsourcing platforms is expected to grow globally at an annual rate of 26% [102,138].However, crowdsourcing platforms are emblematic of precarious work, being characterised by short-term work opportunities that do not have a fixed place of work [155].
Far from being a pastime that people engage in within their spare time, crowdsourcing platforms became a primary employer for many who require, or in some cases desire, the flexibility promoted by on-demand crowdsourcing platforms [45].Crowdsourcing platforms grew in popularity in the mid-2000s for people who needed an income working at home or on the move [93].Years later, crowdsourcing platforms have been criticised for their low wages [81], lack of transparency between workers and clients [94], and enablement of a 'work-anywhere' attitude that has the potential to fragment workers' work-life boundaries [160].The 'work-anywhere' element of crowdsourcing platforms, also observed in the case of digital nomads [38], is considered to offer people a great amount of flexibility [108].
Those who work flexibly on crowdsourcing platforms do so for personal and financial reasons [164].In some cases, workers' circumstances mean that working outside of the traditional workforce becomes one of the only options for employment.For example, some workers lack the physical mobility to search for work outside of their homes (e.g., are not able to travel to a workplace with a physical location) [166]; other workers are not available to work within rigid industrial hours (e.g., are not able to work the hours set by traditional jobs) [58]; and finally, other workers are housed in prisons [80] or refugee camps [67].Therefore, people unable to find work in formal labour markets have to work on crowdsourcing platforms.
Previous work suggests that although people who work on crowdsourcing platforms value the temporal flexibility advertised by the platform, they do not benefit from it.Prior research shows that crowdworkers value having autonomy over scheduling their own time [46].However, there is evidence to suggest that crowdworkers do not benefit from the temporal flexibility advertised by the platform.For example, since workers have no protection from low demands in work [55], they are not able to complete jobs when they prefer.Therefore, crowdworkers can end up having limited temporal flexibility [114].
Having to Be 'On Call' for Work Limits the Temporal Flexibility of Crowdworkers 18:5 In this article, we argue that having to be 'on call' for work on crowdsourcing platforms is a contributor to workers' limited temporal flexibility.Research examining the temporal flexibility of workers has shown that it is limited by both client-imposed constraints [111] and workers' tooling practices [160].On the one hand, even though some workers might be attracted to the temporal flexibility that crowdsourcing platforms can offer [164], there is a lack of worker-controlled temporal flexibility on these platforms [114].Instead, it is clients who have flexibility: they control the 'when' and 'for how long' aspects of jobs.For example, as soon as a client posts a set of surveys to be completed by crowdworkers, workers have to be ready to compete against other workers to 'catch' it [111].Workers can 'catch' new jobs manually, or use tools to 'catch' jobs on their behalf and notify them of new work [101].Therefore, the temporal flexibility advertised by crowdsourcing platforms is limited by client-imposed constraints.
On the other hand, the tools workers use to 'catch' work promote temporal fragmentation of workers' work-life boundaries and work practices [160].First, the tools promote temporal fragmentation of workers' work-life boundaries by increasing workers' on-demand availability during non-work time.For example, job-catching tools notify workers of new work at unpredictable times of the day, sometimes outside their 'working hours' [160].Second, the tools promote temporal fragmentation of workers' work practices by enabling task switching and multitasking behaviour.For example, workers need to frequently monitor the tools for new work even when trying to complete jobs [160].Therefore, the temporal flexibility advertised by crowdsourcing platforms is limited by workers' tooling practices.We next explore what we know about being 'on call' for work.

Having to Be 'On Call' for Work
Within the platform economy, workers have to be 'on call' for work [49].Formally, on-call work is characterised by the ILO (International Labour Organization) as a working time arrangement that involves variable and unpredictable hours of work (from zero hours to full-time work) [130].On-call arrangements emerged in the past decade in industrialised economies as a way of scaling staffing in a short amount of time, in response to changing business needs.Such working time arrangements are commonly found within the platform economy [130].We built on the ILO's description of on-call work to define being 'on call' in the context of crowdsourcing platforms.
In the context of crowdsourcing platforms, we define being 'on call' as a working time arrangement that requires crowdworkers to wait and search for jobs for an undetermined amount of time, often without getting paid, because of the platforms' lack of predictable work availability and lack of work assignment.We next describe the lack of predictable work availability and lack of work assignment found on some existing crowdsourcing platforms.
In this article, we argue that the problem of being 'on call' for work on crowdsourcing platforms is twofold: (1) there is a lack of predictable work availability, and (2) there is a lack of work assignment.First, the lack of predictable work availability is mainly due to an oversupply of labour on crowdsourcing platforms, which makes jobs scarce [73].The oversupply of labour generates a labour force that competes for the better-paid jobs [8].Within the wider platform economy, labour platforms also have an oversupply of workers [73], which makes the workers a 'disposable labour force' that platforms can quickly replace [122].Further, the oversupply of labour means that there are more crowdworkers completing jobs on these platforms than available jobs, which results in competition between the workers.In other words, there are more workers completing jobs on the platform than available jobs.The lack of predictable work availability results in workers not knowing when clients are going to post jobs on the platform.Further, workers do not know for how long work will be available on the platform [114].We describe the impact that not knowing when and for how long clients will post jobs has on the workers in the forthcoming sections (Sections 2.2.1 and 2.2.2).
Second, the lack of work assignment is due to the jobs being made available to most of the workers online, rather than workers being matched by the platform with suitable jobs.Whereas some on-call work within parts of the platform economy is assigned to workers algorithmically (e.g., Uber) [113], workers on crowdsourcing platforms such as Amazon Mechanical Turk, Clickworker, Hive Micro, Microworkers, Neevo, PicoWorkers, or Microsoft's UHRS (Universal Human Relevance System) are not assigned work [110].Instead, crowdworkers have to accept work from the pool of jobs available [105].In the case of crowdsourcing platforms, clients recruit workers on an as-needed basis to work on jobs.Recruiting participants quickly, also known as lowering crowd recruitment latency (i.e., the time until a worker accepts a job that was just posted [79]), has been of interest to researchers wanting to optimise on-demand real-time crowdsourcing [64].Workers can be recruited either using on-demand recruiting (i.e., workers are recruited by clients when needed) or using retainers (i.e., workers are added to a waiting pool and are assigned jobs when needed) [89].For example, on-demand recruiting of crowdworkers has been used to pre-recruit workers with a latency of 2 minutes [20].In contrast, the retainer model for recruitment has been used to pre-recruit workers within 2 seconds [19].Therefore, because crowdworkers are not assigned work, they have to be 'on call' to 'catch' jobs.The lack of work assignment means that workers have to accept available jobs before other workers.We describe the impact that not having work assigned has on workers in the forthcoming sections (Sections 2.2.1 and 2.2.2).
While 'on call', crowdworkers are in a state of hypervigilance [75], having to be ready to work on jobs whenever they become available [149].During this time, workers carry out unpaid work such as searching and waiting for work to become available on the platform [17].Toxtli et al. [149] identified that crowdworkers spend a median of 11 minutes daily on activities related to hypervigilance: (1) watching over clients' profiles, (2) searching for jobs, (3) managing queued jobs, (4) searching for filtered jobs, and (5) checking worker's qualification.Out of these five activities, Toxtli et al. [149] describe two as being 'on call': (1) watching over clients' profiles and (2) managing queued jobs.The remaining three activities are described as "identifying good work" [149, p. 6].On a closer look at the data of Toxtli et al. [149], participants spent on average 28 minutes (SD = 56.8min) on activities related to hypervigilance.Calculating the mean revealed the outliers in their data (i.e., the participants who spent the most time in each activity), which they manually inspected to understand the details of the participants.Out of the five activities belonging to hypervigilance (enumerated previously), participants spent the most time watching over clients' profiles, for an average of 15 minutes.This activity was described as being 'on call'.Given that, on average, participants in the study of Toxtli et al. [149] spent most of their time on an activity described as being 'on call', we believe there is value in further quantifying (and describing) the amount of time crowdworkers spend 'on call' for work.Therefore, in this article, we build on the description by Toxtli et al. [149] of being 'on call' on a crowdsourcing platform by defining, quantifying, and describing being 'on call' for work on a large crowdsourcing platform.We next review what is known about schedule control and job control to begin to understand the potential impact of being 'on call' for crowdworkers.

2.2.1
The Impact of Being 'On Call' on Schedule Control.Schedule control, or temporal flexibility in work schedules, involves the extent to which workers can determine the hours and duration of work [103].Schedule control is believed to minimise the disruptiveness of role blurring and, in turn, enhance work-home integration [68,139].In addition, flexibility enactment theory [106] and work-family border theory [37] state that when workers have control over how to schedule their time, they can better attend to the demands of the work and life domains.Therefore, a high level of control over time use can minimise the negative effects of long hours on work-family relations 18:7 [90].Furthermore, increased schedule control, such as limiting excessive working hours, can result in less fatigue and fewer sleep problems, and workers matching their work hours with their circadian rhythm [11].
Research examining the temporal flexibility of crowdworkers shows that crowdworkers have difficulties scheduling their time because of client-imposed constraints.In particular, not knowing when clients will make work available means that some workers cannot detach from the platform.In an interview study by Lehdonvirta [114] with 10 crowdworkers, participants report finding it hard to mentally detach from work, thinking about the jobs and money they might be missing by not being online to catch 'tasks' when clients make them available.Workers' inability to properly detach from work likely negatively impacts their ability to recover and replenish energy resources properly [142].Consequentially, Lehdonvirta [114] suggests that having to be 'on call' might limit crowdworkers' control over scheduling their time, rather than delivering on the promises of temporal flexibility.
Research also shows that workers have difficulties scheduling their time because of their tooling practices.In particular, notifications from the tools workers use to 'catch' jobs can intrude into workers' non-work time.Williams et al. [160] show that the tools workers use to 'catch' work promote temporal fragmentation of workers' work-life boundaries.In their study, they interviewed 21 full-time crowdworkers and showed that although workers try to follow a 'routine work schedule', the tools they use notify them of new work at unpredictable times, sometimes outside their 'working hours' [160].For example, participants report interrupting their non-work activities (e.g., time with families or spent resting) to work on the platform when tools notified them of new jobs.Furthermore, workers in the study of Williams et al. [160] report feeling motivated to remain 'on call' on the platform partially because they enjoy the sense of serendipity that comes from their tools uncovering unexpected jobs, especially during non-work hours [160].
In summary, prior work shows that crowdworkers have difficulties scheduling their time because of client-imposed constraints and workers' tooling practices.However, the results of prior work do not tell us how being 'on call' impacts the control workers have over when and for how long to work, and their ability to stick to their work schedules.In this article, we aim to explore the extent to which the design of the large crowdsourcing platform, namely requiring crowdworkers to be 'on call' for work, is an additional contributor to workers' limited control over their work scheduling.Evidence suggests that a lack of work time control can result in stressful work environments and decreased health outcomes [66].Furthermore, having some degree of flexibility in working hours is considered to be an important element of overall job satisfaction [11].Thus, if we want to change things to avoid these kinds of negative outcomes, we need to understand the extent and impact of the difficulties participants encounter when scheduling their time.Therefore, in our first study, we ask the following RQ: RQ Study 1: How does having to be 'on call' for work limit crowdworkers' control over scheduling their time?

2.2.2
The Impact of Being 'On Call' on Job Control.Job control is defined as the perceived ability to exert some influence over one's work environment to make it more rewarding and less threatening [63].Perceived job control, including control over pace of work, has positive impacts on job satisfaction and job performance [91,156].However, low levels of perceived job control are linked to negative outcomes, such as job dissatisfaction, work-related stress, and mental and physical ill health [21,22,148].In the case of on-call workers, high job control is positively associated with job satisfaction, having the potential to mitigate on-call stress symptoms [13,115].Research suggests that not all forms of flexibility are beneficial towards work-life balance; however, job control is the most crucial resource towards decreasing work intensity [62].
'Work pace' is a working time dimension that reflects the intensity of work and forms a component of job control.Work pace, as opposed to other working time dimensions that relate to how much control people have over when and for how long to work, describes how quickly work is completed [54].Task switching and multitasking, and not being able to take breaks from work, can intensify the pace of work [59,98].High work pace can lead to fatigue [53], exhaustion [125], and work-life conflict [35].
Research examining the temporal flexibility of crowdworkers shows that crowdworkers have difficulties choosing at what pace they would like to work because of client-imposed constraints.In particular, having to multitask to catch 'task' when posted by clients can increase workers' pace of work.The structural constraints of crowdsourcing platforms have led to the creation of a competitive marketplace in which only the fastest and most alert workers get to work on the better-paid jobs [111,114].In contrast, slower workers see jobs disappear before they can accept them.Therefore, the competitiveness of the platform has led to workers adopting job-finding tools to 'catch' work.Lehdonvirta [114] reports that the job-finding tools used by the participants in his interview study notified them when a new job became available.As a result, some participants reported stopping and putting everything aside to complete the job.However, the use of job-finding tools can lead to task switching and multitasking as workers need to frequently monitor the tools even when trying to complete jobs [160].Despite the common belief that multitasking allows people to use their time more flexibly, evidence suggests that multitasking lowers wellbeing and self-rated performance [104].Furthermore, task switching induced by interruptions, such as the notifications from job-finding tools, can increase feelings of frustration and stress [27,117,118], and intensify the pace of work [59].
Research also shows that workers have difficulties choosing at what pace they would like to work because of their tooling practicesin particular, continuously receiving notifications from tools can make it difficult for workers to take breaks.Williams et al. [160] show that the tools workers use to 'catch' work promote fragmentation of workers' work practices by enabling task switching and multitasking behaviour.Furthermore, as the availability of work on the platform is variable, workers are under time pressure to 'catch' work before other workers.In turn, the variable availability of work can increase workers' task switching and multitasking activities and decrease their ability to take breaks and disconnect from work.In the study by Williams et al. [160], some of the participants reported difficulties detaching from crowdwork and the process of searching for jobs.In particular, a participant reported difficulties detaching from work to take breaks because of fear they might miss out on potential earnings when work suddenly became available.In addition, a prior investigation of break-taking among crowdworkers also revealed that workers had concerns that taking breaks would decrease their earnings [112].Evidence suggests that having control over taking breaks at the right time from working at a high pace can provide workers relief [43,135].However, a small number of breaks, or a lack of breaks, can result in sedentary behaviour and musculoskeletal symptoms [76], and depletion of cognitive resources, which leads to fatigue and burnout [44].
In summary, prior work shows that crowdworkers have difficulties choosing their pace of work because of client-imposed constraints and workers' tooling practices.However, the results of prior work do not tell us how much control workers have over their work pace.In this article, we aim to explore the extent to which the design of the crowdsourcing platform, namely requiring crowdworkers to be 'on call' for work, is an additional contributor to workers' limited control over their work pace.Limited control over the pace of work can increase workers' task switching and multitasking activities, and decrease workers' ability to take breaks and disconnect from work, negatively impacting their health and wellbeing.In this work, we therefore aim to characterise in detail the 'on call' problem of crowdsourcing platforms, with a focus on not only how being 'on call' impacts workers' control over their work schedules (Study 1) but also the impact of being 'on call' on workers' control over their work pace.Therefore, in our second study, we ask the following RQ: RQ Study 2: How does having to be 'on call' for work limit crowdworkers' control over their work pace?

Research Approach
In this article, we investigate how the design of a large crowdsourcing platform, namely requiring crowdworkers to be 'on call' because of the platform's lack of predictable work availability and work assignment, is a contributor to workers' limited temporal flexibility.To answer our RQs, we look at the impact that having to be 'on call' had on workers' temporal flexibility at two levels.In the first part of the article, we focus on the bigger picture of how having to be 'on call' can limit workers' control over scheduling their time at the level of a working day.In the second part of the article, we 'zoom in' to the moment-to-moment experience of workers to understand how having to be 'on call' can limit workers' control over their work pace at the level of a working session.Therefore, in Study 1, we present a time-use-diary study conducted to investigate the impact of having to be 'on call' on workers' ability to plan and stick to their work schedules.Next, in Study 2, we present a video analysis study of 18 hours of screen recordings conducted to investigate the impact of being 'on call' on job control over workers' work pace.We begin by presenting Study 1.

STUDY 1: TIME-USE-DIARY STUDY
In the next sections, we present the time-use-diary study we conducted to investigate how being 'on call' for work can limit crowdworkers' control over scheduling their time.We begin by describing how we recruited participants for the study and the data collection procedure.

Design.
We conducted a time-use-diary study to investigate how having to be 'on call' can limit crowdworkers' control over their work schedules at the level of a working day.Time-use diaries are a multidisciplinary research method used within the social sciences (e.g., [26]), psychology (e.g., [129]), and sociology (e.g., [41]) to capture how people use their time and document the time spent on daily activities.Time-use diaries have been used in the past to explore daily patterns of work and flexibility of work [7].
In this study, we used two time-use diaries.First, we used the '7-day Work Schedule' diary to get an overview of when participants planned to work during the upcoming week.As shown in Figure 1, participants could mark with an 'X' each 15-minute interval when they plan to work on the crowdsourcing platform in the next 7 days.Participants could also tick the 'No work' box if they did not plan to work on the crowdsourcing platform on a particular day.
Second, we used the '24-hour Everyday Activities' diary to get a detailed record of participants' activities over a single 24-hour period.As can be seen in Figure 2, participants were asked to record a detailed description of their activities every 10 minutes throughout the day.In addition, we used the '24-hour Everyday Activities' diary to understand how well crowdworkers could stick to the schedule they had previously made at the start of the week using the '7-day Work Schedule' diary.We based these two time diaries on the United Kingdom 2014-2015 Time Use Survey [65].
3.1.2Procedure.We hosted the two diaries online using Microsoft Excel Online.We created separate files for each Diary Participant (DP), and we assigned each participant a unique link to their files.The links could only be accessed by the participants and the researchers.Participants were able to access the files containing the two diaries in their browsers (e.g., on a computer or phone) using the online spreadsheet software Microsoft Excel Online or on their desktop using the Microsoft Excel desktop application.
Once recruited, we asked participants to first complete the '7-day Work Schedule' diary (see Figure 1).We asked participants to mark with an 'X' each time interval when they had planned to work on the crowdsourcing platform in the next 7 days.We used this diary to gather data on when participants planned to work during the week.On the following day, we asked participants to complete the '24-hour Everyday Activities' diary (see Figure 2).We asked participants to write down a description of their activities throughout the day.Participants were sent three reminders during the day to complete the '24-hour Everyday Activities' diary.We used this diary to gather data on when participants actually worked during the week.In this way, it was possible to see how well participants were able to stick to the plans that they made the previous day. 1 reports the demographics of the participants who took part in this study.We recruited 19 participants.We compensated the participants with $20 USD for their time.Participants were required to earn a significant amount of their income (over 50%) from working on crowdsourcing platforms to participate in the study.We recruited participants who made a 18:11 large portion of their income through crowdsourcing platforms because workers who depend on online work for their living report spending a sizable portion of their workday being 'on call' for work to appear, compared to workers who have other sources of income [114].

Participants. Table
All participants were based in the United States.We recruited only participants based in the United States to mitigate some of the cultural differences that may affect people's relationship with time and, in turn, with how they plan their time.For example, cross-cultural research studies conducted with crowdworkers based in the United States and India, the two largest crowdworker groups [47], have found that workers display intertemporal differences across (a) time of day and (b) across the serial order in which they participated (i.e., earlier or later in data collection) [31]; in other words, crowdworkers from the United States and India vary demographically across the time of day at which data is collected and serial position, suggesting the existence of intertemporal variations among countries.Other studies have illustrated that crowdworkers across different countries work on the platforms at different rates and times [134] and display different patterns of both motivation and social desirability effects [6].Finally, research suggests that data collected on crowdsourcing platforms as part of cross-cultural comparison studies can be systematically different, highlighting cultural differences including social orientation (individualism vs. collectivism), social desirability, and thinking styles (holistic vs. analytic) [154].
In terms of gender, of the 19 participants, 11 (58%) identified as women, 6 (32%) as men, and 1 (5%) as non-binary; our sample had more women than the average population of crowdworkers reported in previous studies, which had a more mixed workforce [47].In terms of age, 16 (84%) participants were in the age range of 24 to 54 years; our sample was therefore consistent with the average population of crowdworkers, which ranges between the ages of 30 and 39 [32].In terms of education level, 10 participants (53%) reported having some college/technical training, and 8 participants (42%) reported holding a university undergraduate degree (e.g., bachelor's); this is in line with previous studies reporting that crowdworkers are likely to have a college degree [32].A further 2 participants (11%) reported holding university post-graduate program degrees (e.g., master's).Finally, in terms of income, participants earned on average 90% (SD = 15) of their income from working on crowdsourcing platforms.

Ethical Considerations.
Before taking part in the study, we briefed participants about the study's purpose and data confidentiality practices.The participation of the workers was voluntary, and informed consent was obtained from all participants.The study had institutional research ethics approval: UCLIC/1718/013/Staff Cox/Lascau/Brumby.
Throughout the '24-hour Everyday Activities' diary, we regularly prompted participants to avoid disclosing sensitive information.For example, we noted in the diaries that if there was something they felt was too private to record, participants should write 'personal' in the text fields.Furthermore, we pseudonymised the entries and separated workers' usernames from the diaries to preserve confidentiality.

Analysis.
The analysis is based on diary data from 18 participants.Out of the 19 participants recruited, we excluded the data from one participant (i.e., DP2) from the analysis.This is because DP2's '24-hour Everyday Activities' diary revealed that they did not engage in any crowdsourcing-related activities (e.g., completing jobs or waiting for work), except for filing in the diary itself.To code the remaining participants' diary entries, we used the coding schema provided in the NatCen report [123] of the United Kingdom 2014-2015 Time Use Survey [65].For example, sleeping activities were marked with '110', and actively working on jobs on the crowdsourcing platform was marked with '1110'.We excluded from the last measure the amount of time during which participants filled in the diary.Furthermore, we argue that actively working on jobs is also part of participants' experiences of being 'on call' because workers have to monitor job-catching tools even when they are actively completing jobs [160].However, for the purpose of our study, we did not include this paid activity in our measurement of being 'on call' for work.Instead, in addition to the codes provided in the NatCen report, we constructed two new code categories for the unpaid time participants had to spend 'on call' for work: (1) '1392' for activities related to waiting for work (example activities include "Reading stuff online and waiting for work" and "Waiting for work to do on [name of platform]") and (2) '1393' for activities related to searching for work on the crowdsourcing platform (example activities include "checked [name of platform] for jobs I've missed overnight" and "scanned [name of platform] for batches").We next present the results of the study.

Results
Table 2 provides an overview of the observed differences between how participants planned to work for a given day on the crowdsourcing platform, as captured by the '7-day Work Schedule' diary, compared to how they actually worked, as captured by the '24-hour Everyday Activities' diary.For all statistical analysis, we used a paired sample t-test since the data were normally distributed as assessed by the Shapiro-Wilk test (α = .05).We judged effects significant if they reached a 0.05 significance level.We explain each of the measures in more detail in the following sections.

Start and End Time of Workday.
We first consider the time of day that participants planned to start and end their workday, and whether our participants then actually kept to these plans on a given day.We found that participants planned to start their workday in the morning at around 8 am (M = 08:06, SD = 1 hr 35 min).In reality, participants actually started their workday a little later than planned (M = 08:21, SD = 2 hr).However, a paired sample t-test found there to be no significant difference between planned and actual start time of the workday, t( 17  19:00, SD = 3 hr).A paired sample t-test again found there to be no significant difference between the planned and actual end time of the workday, t(17) = 1.07, p = .30,95% CI = [-36 min, 1 hr 50 min].These results show that participants planned to work what can be considered traditional working hours, starting work in the morning and finishing work in the early evening.There was therefore no evidence that participants did not keep to their planned start and finish times.

Number of Hours Worked.
We next consider whether participants were able to actually work the number of hours planned within their working day.We found that while participants planned to work on average for 8 hr 17 min (SD = 2 hr 45 min), they were only able to actually work 6 hr 13 min (SD = 2 hr 33 min).A paired sample t-test found that participants actually worked significantly fewer hours (-2 hr 03 min) than planned, t(17) = 2.99, p = .008,95% CI = [-3 hr 30 min, -36 min].Therefore, results of the diary study show that our participants worked on average 2 hours less than planned.Furthermore, out of the total time worked, participants spent on average 1 hr 23 min (SD = 3 hr 30 min) waiting and searching for work.In other words, participants spent on average 22% of their daily working time on unpaid 'on-call' activities such as waiting and searching for new jobs.

Fragmentation of Workday.
To better understand why participants were not able to work as many hours as planned despite starting and ending their workday as planned, we next drill down to consider periods of work.We define a period of work as a block of contiguous work (either planned or actually reported) that is separated from another period of work by a break of at least 10 minutes.For example, DP17 planned to work across two distinct periods of work: (1) 08:30 to 13:00 and (2) 14:00 to 17:30.However, DP17's actual work pattern was far more fragmented than planned, with work being done in many more shorter periods of work than planned: (1) 08:00 to 09:10, (2) 09:50 to 10:50, (3) 11:00 to 11:10, (4) 11:40 to 12:50, and (5) 14:20 to 17:00.From this example, we can see that the reason DP17 worked 1 hr 90 min less than planned is because work was done over five periods of work instead of two, and that each of these work periods was far shorter in duration than planned.In other words, DP17's workday was far more fragmented than they had planned.
To examine whether participants' workdays were more fragmented than planned, we consider both the frequency and the duration of the periods of work throughout the workday.Although participants planned to work across only three distinct periods in the day (M = 3.06, SD = 1.63), they actually worked across six periods (M = 6.17,SD = 3.37).A paired sample t-test found this difference to be significant, t(17) = 3.62, p = .002,95% CI = [1.30, 4.92], suggesting that participants' work schedules were far more fragmented than planned.Therefore, the results of the diary study show that our participants' workdays were more fragmented than planned, with work distributed across twice as many work periods as planned.In terms of the duration of these work periods, we found that participants planned for each period of work to be on average 3 hr 22 min (SD = 1 hr 57 min).In reality, each period of work was actually only 1 hr 24 min (SD = 1 hr 11 min), which was significantly shorter in duration than planned, t(17) = -4.42,p < .001,95% CI = [-2 hr 54 min, -1 hr 01 min].Therefore, results of the diary study show that the work periods our participants worked were far shorter than planned.

Discussion
Taken together, the results of the diary study show that our participants' workdays were far more fragmented than they had planned: there were on average twice as many work periods as planned, and each of these was far shorter than planned.Moreover, this increased fragmentation of the workday meant that participants were on average working for 2 hours less than planned, despite starting and ending their workday roughly when they had planned.These findings are interesting because most of our participants wanted to work a 'shift' on the crowdsourcing platform, setting out clear start and end times when they wanted to work on the platform and generally sticking to them.Workers, then, are responding to the imperatives of the platform within this constraint; they are not just 'jumping to attention' at any point jobs arrive.
Our results also suggest that participants worked on average for 2 hours less than planned.In contrast, participants in the interview study of Lehdonvirta [114] report having to frequently work longer hours than planned.However, it is unclear if participants in the aforementioned study were speaking about the amount of time they had to work (i.e., for how long) or the times of the day at which they were working (i.e., when).In our study, we aimed to differentiate between the two concepts and provide objective measures.It could be that participants in the study of Lehdonvirta [114] were speaking about both scenarios.Nevertheless, the fact that in our study participants worked 2 hours fewer than they had planned illustrates the issue with being 'on call' on crowdsourcing platforms: with participants having set aside time to work a 'shift', the data suggest they could not 'fill' this time and maximise their earnings.This situation indicates a paucity of work available on the platform that meets workers' requirements, meaning that workers end up using the time already assigned inefficiently trying to find work.Lack of suitable work at a given time can also be seen in the case of platform workers on freelancing platforms, such as Upwork, where freelancers report spending a high number of unpaid hours on finding jobs [29].
This study also yields new insights into the relationship between the flexibility and fragmentation of work in crowdsourcing platforms.Our results paint a picture of workers seeking to develop some kind of routine, but where the platform's architecture prevents them from doing so.At a workday level, this leads to high temporal fragmentation of workers' schedules.The temporal fragmentation makes it harder for workers to schedule their time across the day in a way that balances demands on their time.Irregular and unpredictable schedules of work can disrupt daily or weekly routines [14], as well as personal relationships [10], and thus reduce control over work-life boundaries [40].Furthermore, non-standard work schedules are associated with anxiety and irritability [39], as well as decreased sleep quality [153], and can have adverse effects on mental health [131].However, it is not only unpredictability in working time that can worsen the working conditions of crowdworkers but also the economic instabilities, as evidence suggests that unpredictable and unstable work schedules can lead to economic insecurity [15,25].
In summary, the schedule-level data tells us something about the temporal fragmentation of the working day within the 'shifts' that workers plan to fit in around their other commitments.Even when workers set aside time to work (rather than simply being continuously available), scheduling in the face of a 'flexible' platform with variable availability of work is almost impossible, and workers end up fitting life around the times when work is available.The fact that workers miss out on almost 2 hours of work in a given 'shift' but do not attempt to 'catch up' on this time at 18:15 the end of the shift might imply that workers are aware that after a certain hour they will not find work on the platform, or they are simply not keen to make use of the 'flexibility' that the platform offers.
What we did not see from the data was how having to be 'on call' for work influences the pace at which workers complete individual jobs and find and manage work on the platform.Work pace is another important part of job control, so to understand this aspect, in Study 2, we 'zoomed in' to the moment-to-moment experience of crowdworkers.

Limitations.
We recruited participants who made a large portion of their income through crowdsourcing platforms, as they are known to spend a significant proportion of their time waiting for work [114].However, we did not set any other pre-screening criteria when we recruited the participants, such as minimum job completion rates or minimum earnings.Therefore, we cannot know if any of the participants in our study were highly experienced workers.Observing more experienced workers could have created potential biases towards the interpretation of our results, as the workers might have developed specific strategies to schedule their time.Nevertheless, for this work, a general understanding of planning and fragmentation is sufficient.Still, it could be that a more focused or stratified sample based on experience rather than time might yield insights into whether planning effectiveness develops with experience, and whether there are ceiling effects (i.e., a point at which experience cannot overcome the fundamental architecture of the platform).Furthermore, we asked 19 workers to keep the two time-use diaries, whereas time-use diaries are usually administrated to larger samples (e.g., [65]).Therefore, results in the study might not necessarily be generalisable to the wider population of crowdworkers.However, the results provide an initial overview of the limitations workers encounter when scheduling their time.Future work should consider administering the diaries to a larger sample of participants to examine the scale and significance of the issues generated by having to be 'on call' observed in our study.We asked participants in the time-use-diary study to record their work activities throughout the day, in line with prior studies that have asked participants to keep diaries of their work activities (e.g., [3,42,126]).Asking participants to record their activities every 10 minutes throughout the day could have been disruptive to participants' work.The potential disruptiveness is a common disadvantage of diary studies [42].However, the in situ and in-the-moment nature of these kinds of diaries is an important advantage [30,92].We have reason to believe that in this working context, the disruptive effects of the diaries may have been less apparent.Working on crowdsourcing platforms is a fragmented activity; as larger jobs are decomposed into short jobs, the disruptions to participants' work should have been minimal.The disruption should have been minimal because filling in the diaries could be done quickly, which should have made recovering from the interruption easier [121].Moreover, the interruptions were relevant to the participants' main jobs (e.g., working on the crowdsourcing platform), an aspect which should have also made it easier for the participants to recover from the interruptions [1,71].Despite the additional workload, participants in our study engaged with the diaries throughout the diary day and completed all the required fields.Subsequent studies could employ alternative methods (e.g., the Experience Sampling Method [109,152]) and alternative time frames (e.g., hourly reports) to examine the extent to which workers encounter difficulties in planning and sticking to their schedules.It might be that a lower fidelity of data might still serve the purpose.

STUDY 2: VIDEO ANALYSIS STUDY
In the following sections, we present the video analysis study of more than 18 hours of screen recordings conducted to investigate how having to be 'on call' for work can limit workers' control over the pace at which they work and thus reduce job control.We begin by describing how we recruited participants for the study, the data collection procedure, and analytic strategies.

Method
4.1.1Design.We conducted a video analysis study to investigate how having to be 'on call' can limit workers' control over their work pace at the level of a working session.Inspired by previous research that studied the workflows of crowdworkers [77], we gathered 18 hours of screen recordings of participants working on the crowdsourcing platform.The screen recordings provided us with a rich dataset of the participants' behaviours on the crowdsourcing platform.Compared to diary studies, screen recordings require little effort on participants' side [30].Further, compared to direct observation or shadowing, screen recordings do not require researchers to be physically present in the same room with the participants.Therefore, instead of relying on participants' notes during a diary study or on researchers' notes during direct observations or shadowing, the screen recordings enabled us to capture a comprehensive picture of the participants' behaviours.Finally, in our study, we annotated each screen recording to describe the activities we saw participants doing (e.g., working on a job, searching for new jobs to work on, switching between windows, taking breaks).Furthermore, to better understand the events in these videos, we sent follow-up questions to participants to elaborate on their actions recorded in the videos.

Procedure.
A job was advertised on the crowdsourcing platform with a maximum completion time of 24 hours, meaning that the participants had 24 hours to record their screens, upload the screen recordings, and submit the job.Before agreeing to complete the job, we presented participants with an information page that contained details about the study.On the information page, we mentioned the following: (1) We ask participants in the study to record their screens for 90 minutes using a remote usability testing platform.(2) We will hold everything that appears on participants' screens under strict confidentiality, and we will delete the recordings after data analysis.(3) We expect participants to take breaks during the 90-minute recording, and ask them to inform us if they are about to take any breaks by leaving a message on their screens.
After participants accepted the job, they were taken to the usability testing platform to record their computer screen for 90 minutes.Where participants were asked by the platform to enter their full names and email addresses to begin recording their screens, we instructed participants to enter the usernames they used on the crowdsourcing platform and a random e-mail address (e.g., address@e-mail.com).All participants complied with our instructions.We paid participants within 24 hours of submitting the job.
After watching each video, we identified brief video clips about which we wanted to learn more.The selection of these clips focused on moments when we could see participants readjusting their work activities once a new job became available on the platform.We identified 98 clips in total, and after identifying these, we sent follow-up messages to each participant to provide detailed comments on three of their video clips.We contacted participants via an API that supported messaging the workers using only their usernames, as we did not collect participants' e-mail addresses.In the message, we included our questions and links to the video fragments.We hosted the video fragments on Microsoft OneDrive.For security purposes, we scheduled the links to the videos to expire 2 weeks after we had sent them to the participants.Participants received an additional $5 for answering these follow-up questions.We received detailed annotations on 18 video clips from six participants.Each of the three clips that we sent to participants to comment on covered different types of interactions with the crowdsourcing platform: interactions with the platform, productivity tools, or other tools such as forums.The following is an example of the kind of question we sent to a participant: "Looking back at how you went about completing your work, I was wondering how come you chose to check the platform while you were completing a job?Here is the secure link to a fragment of the video recording where you can see the job I'm referring to: [. ..]". 3 reports the demographics of the participants who took part in this study.We recruited 12 participants.We compensated the participants with $10 for their time; participants could then receive an additional $5 for answering our follow-up questions, bringing the pay for the study to $15.All of the participants were based in the United States, and the total number of jobs participants completed ranged from 525 to 125,778 (M = 34,465, SD = 37,917).The study was open to both experienced workers (n = 9) (i.e., workers with more than 10,000 jobs completed [137]) and novice workers (n = 3) (i.e., workers with less than 1,000 jobs completed [137]); this is because we wanted to capture workers' activities regardless of their experience.Of the 12 participants, 7 participants (58%) identified as men and 5 (42%) as women; this is in line with previous studies reporting a mixed workforce [47].Participants' age range was 27 to 51 years (M = 37.3 years, SD = 7.0 years); this is in line with previous studies reporting that the majority of crowdworkers population is between the ages of 30 and 39 [32].In terms of education level, 5 (42%) had some college/technical training, 4 participants (33%) reported holding a university undergraduate program degree (e.g., bachelor's), 2 participants reported holding high school diplomas (17%), and 1 a university post-graduate program degree (e.g., master's); this is in line with previous studies reporting that workers are likely to have a college degree [32].

Ethical Considerations.
There are, naturally, several privacy concerns over collecting the kind of video data necessary for this study.We took two main measures to mitigate as best as possible any potential risks to the participants.First, we held extensive conversations with the departmental and faculty ethics committees of our university.We obtained the appropriate ethical review committee approval prior to conducting the study and complied with all aspects of the approvalthe study had institutional research ethics approval: UCLIC/1718/013/Staff Cox/Lascau/Brumby.
Before taking part in the study, we briefed participants about the study's purpose and data confidentiality practices.The participation of the workers was voluntary, and informed consent was obtained from all participants.Further, we committed to giving participants control over the recordings: participants could start, pause, or restart the screen recordings at any point.The recordings were stored on participants' computers until they had decided to upload and share them with us.
Second, we obtained the appropriate data protection registration approval prior to conducting the study and adhered to the institutional data management requirements.In this sense, we maintained the participants' privacy and that of the clients by keeping the screen recordings secure and removing any confidential information.Further, we were aware of the risks of participants accidentally disclosing confidential information in the screen recordings and therefore prompted participants not to reveal any sensitive information.Where personal information such as clients' usernames appeared in the videos (e.g., where a worker was completing a job posted by a client), we anonymised the information within the job by blurring it out, unlinking it from the videos, and discarding it alongside the original videos.We also removed any other screen recordings not relevant to our study (e.g., screen recordings of non-crowdwork).
4.1.5Analysis.We next give an overview of how we analysed the video data collected.We begin by describing how we measured the activities recorded in the video data, followed by describing how we contextualised these activities.
A. Preparing the Data for Analysis.As this study is only concerned with participants' behaviour whilst working on the crowdsourcing platform, the first author initially reviewed the recordings to discard any sections where the participants were not engaging in crowdwork activities.Furthermore, as this study is only concerned with participants' behaviours, she additionally discarded any sections where the contents of the jobs were showing.
Once the first author discarded these sections, together with the second author, they familiarised themselves with the screen recordings by initially independently viewing the recordings to gain a good understanding of the entire data; this process was comparable to a thematic analysis [23,24].Specifically, since we aimed in this study to investigate how having to be 'on call' can limit workers' control over the pace at which they work, the two authors initially annotated two unpaid work activities that participants had to engage in whilst 'on call' on the crowdsourcing platform: (1) Waiting for new jobs [165] and (2) Searching for jobs [17].
Annotating the two unpaid work activities enabled us to define each activity's start and end times and the frequency of occurrence across the dataset.We describe the process of analysing the two activities in subsection 'E.Analysing the Activities Recorded'.
We describe in the next subsection how we measured the two activities.B. Measuring the Amount of Unpaid Time Participants Spent whilst 'On Call' for Work.Consistent with the measure we used in Study 1, we measured the amount of time participants spent whilst 'on call' for work by aggregating the amount of unpaid time participants spent (1) Waiting for new jobs and (2) Searching for jobs.Both of these activities have been described previously in the context of the unpaid work activities that crowdworkers have to engage in when working on crowdsourcing platforms [17,165].Similarly to Study 1, in this study, we annotated the two activities to measure the amount of unpaid time participants spent being 'on call' for work.We next give an overview of how we measured these two activities: (1) Waiting for new jobs: We measured the amount of time participants were waiting for new jobs to become available on the platform.Waiting for new jobs is part of being 'on call' because of the crowdsourcing platform's lack of predictable work availability (i.e., workers do not know when clients are going to make new jobs available [17]).We observed when participants were waiting for new jobs by annotating the instances in which participants were not interacting directly with the crowdsourcing platform or any crowdwork-related work (e.g., unpaid work like contacting clients, tracking earnings, reading forums or reviews, or checking qualifications).Whilst waiting for work, participants filled their time with activities resembling break-taking (e.g., playing video games, browsing the internet, or watching Netflix).In this study, we differentiated between 'activities resembling break-taking' and actual breaks by asking participants to inform us during the screen recordings if they were about to take any breaks (we report the number of breaks participants took in Section 4.2).( 2) Searching for jobs: We measured the amount of time participants were searching for jobs to complete.Searching for jobs is part of being 'on call' because it requires workers to engage in unpaid work such as filtering through potential work [149].We observed when participants searched for jobs by annotating the instances in which participants visited the crowdsourcing platform's main page (i.e., the page where jobs appear as they are posted), and when participants visited their external tools to adjust observable parameters in the tools [77].
C. Describing How Participants Spent Their Unpaid Time whilst 'On-Call'.In addition to reviewing and annotating the two activities that we used to measure the amount of unpaid time participants spent whilst 'on call' for work, we annotated three other activities.We annotated the following three activities to describe how participants spent their unpaid 'on-call' time: (1) 'Catching' new jobs, (2) Managing the queue of jobs, and (3) Doing other unpaid work.First, 'Catching new jobs' has been described previously as one of the main activities that crowdworkers have to engage in [111,160].Second, 'Managing the queue of jobs' is one of the two activities that were previously used to briefly describe being 'on call' on a crowdsourcing platform [149].Finally, 'Doing other unpaid work' has been described previously in quantifying the invisible labour of crowdworkers [149].We chose to focus on these three activities because we were interested to learn from our data about how having to be 'on call' can influence participants' work activities.Specifically, we wanted to quantify these three activities to describe how participants spent their 'on-call' time.In this sense, we further annotated the video data to define when each of these three work activities occurred; this approach enabled us to define the frequency of occurrence across the dataset.We next give an overview of how we quantified these three work activities, which we used to describe how participants spent their unpaid 'on-call' time: (3) 'Catching' new jobs: We observed whether participants were 'catching' new jobs as they became available on the crowdsourcing platform.We chose this activity to describe being 'on call' for work because crowdworkers have to 'catch' new jobs as they become available [111,160]-we argue that this is because of the crowdsourcing platform's lack of work assignment (i.e., the crowdsourcing platform does not assign jobs for workers to complete).We measured the number of jobs participants caught by recording the frequency with which participants were (a) manually accepting new jobs and (b) the frequency with which participants' external tools were 'catching' jobs on their behalf.We excluded from the analysis jobs that did not meet participants' criteria and that they therefore did not attempt to 'catch'; it was possible to record participants' criteria when they adjusted observable parameters in their external tools.(4) Managing the queue of jobs: We observed whether participants were managing the jobs they had 'queued' up to work on.Managing the queue of jobs is one of the two activities that were previously used to briefly describe being 'on call' on a crowdsourcing platform [149].We chose this activity to describe being 'on call' because workers have to line up jobs to complete in the queue of jobs, then filter out any unsuitable jobs that they will not complete (e.g., fraudulent jobs or low-paying jobs) [149].We observed the number of jobs participants had in their queue by examining the workers' job queues right before they started working on a job.The queue of jobs was displayed either on the crowdsourcing platform's queue of jobs or as part of the external tools that participants were using to catch jobs and maximise their earnings.For 10 of the participants, the queue of jobs was displayed on the crowdsourcing platform's jobs queue, whereas for the remaining 2 participants, the queue was displayed as part of their external tools.For the participants who had the queue displayed on the crowdsourcing platform's queue of jobs and had the "auto-accept next job" feature on, we subtracted the number of jobs they were completing (or returning) one after the other, as participants were working their way through the queue.( 5) Doing other unpaid work: Finally, we observed whether participants were engaging in any other unpaid work.We chose this activity to describe being 'on call' because crowdworkers have to engage in a variety of unpaid work just to secure paid work [149] (in addition to activities like waiting, searching, and catching jobs, or managing the queue of jobs, which we excluded from this measure).We defined unpaid work to include a wide variety of unpaid activities such as (a) contacting clients, (b) tracking earnings, (c) reading forums or reviews, and (d) checking their qualifications.This list of unpaid activities is consistent with the description by Gupta [77] of crowdworkers' workflows and with the invisible labour activities examined by Toxtli et al. [149].We next describe how we observed each unpaid activity.We first observed when participants contacted clients by annotating the instances in which participants sent clients messages using the contact form provided by the platform.We next observed when participants tracked their earnings by annotating the instances in which they visited the crowdsourcing platform's earnings section.Further, we observed when participants read forums or reviews by annotating the instances in which participants visited worker forums or read client or job reviews left by other workers.Finally, we observed when participants checked their job qualifications by annotating the instances in which they visited the crowdsourcing platform's Qualifications page.

D. Measuring and Describing the Amount of Paid Time Participants Spent
Working whilst 'On Call'.Finally, we reviewed and annotated the video data showing the participants actively working on jobs.Similarly to the measure in Study 1, we argue that this paid activity is also part of participants' experiences of being 'on call' for work because workers have to monitor job-catching tools even when they are actively completing jobs [160].However, for the purpose of our study, we did not include this paid activity in our definition of being 'on call' for work, nor did we use this activity to measure the amount of unpaid time participants spent whilst 'on call'.Instead, we used this activity to measure and describe the amount of paid time participants spent working on the platform.Hence, we present this activity separately.We next give an overview of how we measured this activity: (6) Actively working on jobs: We measured the amount of time participants were actively working on jobs.We observed the number of job participants completed by annotating the video data to define (a) when participants started working on a job (i.e., the moment participants clicked on the 'Work' button to begin working on a job) and (b) when participants stopped working on a job (i.e., the moment participants clicked on the 'Submit' or 'Return' buttons to stop working on a job).As part of measuring the amount of time participants were actively working on jobs, we also included (a) any amount of time spent on jobs that were started but then returned, as well as (b) any amount of time spent on jobs that eventually timed out [149]-in comparison, Toxtli et al. [149] categorised these two measures as 'unpaid work'.However, in our study, we included these other two activities to measure the amount of paid time participants spent working, whilst still being 'on call' for work.
In summary, to measure the amount of unpaid time participants spent whilst 'on call' for work, we annotated two key work activities: (1) Waiting for new jobs and (2) Searching for jobs.Next, to describe how participants spent their 'on-call' time, we additionally annotated three other unpaid activities: (3) 'Catching' new jobs, (4) Managing the queue of jobs, and (5) Doing other unpaid work.Finally, to measure and describe the amount of paid time that participants spent working on the platform, whilst still being 'on call' for work, we annotated one other activity: (6) Actively working on jobs.We next describe how we analysed the six activities.
E. Analysing the Activities Recorded.The first and second authors independently coded all instances related to the six activities observed within the 18 hours of video recordings.They coded the video data deductively (i.e., top-down).In this sense, the first and second authors deductively generated initial codes based on the RQ asked, the initial viewing of the recordings, and existing literature (e.g., 'catching' new jobs [160] or managing the queue of jobs [149]).Based on the codes, they developed a preliminary codebook to help guide the analysis of the study; they refined the codebook throughout the analysis of the video data.
This analysis resulted in 14 codes grouped into six main categories: (1) Waiting for new jobs (e.g., code, 'playing video game'), (2) Searching for jobs (e.g., code, 'searching for job on main page'), (3) Doing other unpaid work (e.g., code, 'contacting clients'), (4) 'Catching' new jobs (e.g., code, 'catching job manually'), ( 5) Managing the queue of jobs (e.g., code, 'checking the work queue on the platform'), and (6) Actively working on jobs (e.g., code, 'starting to complete job').The two authors annotated the video data iteratively, until no other notable instances related to the six activities were identified.Through the process of annotating the data, they were able to generate a robust list of all of the instances of activities we were interested in measuring in this study.
Throughout the coding process, the first and second authors collaboratively examined in detail the six activities recorded and shared their own understandings of these activities and discussed any disagreements.Additionally, after coding the video data, the first author contacted the participants with follow-up questions: participants were asked to describe some of the behaviours recorded in the videos.Following up with the participants allowed us to check and refine our codes against their descriptions.Finally, all authors collaboratively discussed the activities recorded and asked probing questions to build a shared understanding of the video data.Throughout these group discussions, the authors explored the meaning of the data, and reflected on how their biases and subjectivity might be affecting the reading of the data [24] F. Contextualising the Activities Recorded.Additionally, the first author inductively translated the codes into observable patterns (i.e., comparable to the themes of thematic analysis [23,24]).During this process, the author looked for patterns of activities across the whole dataset rather than only within each video recording.Furthermore, the first author sought patterns across the activities by combining the annotations made and observing relationships between them, instead of contrasting the instances of the six activities in a direct manner [50].This approach enabled the first author to examine the relationship between the instances and measure the amount of time participants engaged with them.Throughout this process, we identified in the video data three distinct periods of work intensity.
We observed variability in work intensity over the 90-minute periods.The variability suggests that, for the participants in our study, working on the crowdsourcing platform was characterised by three distinct periods of work intensity, which were influenced by the number of jobs participants had in their work queue: (1) Periods of low work intensity, in which participants had zero or only one job lined up to complete in the queue; (2) Periods of moderate work intensity, in which participants had between two and five jobs lined up; and (3) Periods of high work intensity, in which participants had six or more jobs lined up.We used six jobs as the cutoff point for periods of high work intensity, as it is approximately equal to 2 standard deviations from the overall mean.
We observed the number of jobs participants had in their queue by examining the workers' queues right before they started working on a job.The queue of jobs was displayed on either the crowdsourcing platform's queue of jobs or as part of their external tools.We chose to focus on the number of jobs participants had in their queues, as these can be an indicator of work intensity on the crowdsourcing platform.Since each queued job has a time limit, crowdworkers report having to monitor the time limits of the jobs to ensure they get to complete all the queued jobs without the jobs expiring or timing out when working on them [111].Moreover, if workers are not near their work spaces, monitoring the queue of jobs is one of the most common mobile job for the workers [160].Therefore, we argue that the more jobs workers have in their queue, the more their work intensifies (i.e., the workers' pace increases).
On average, we identified four (M = 4.22, SD = 1.96, range: 0-8) distinct periods of work intensity of the three types described previously (i.e., low, moderate, and high).We next present the results of the study, in which we used the three distinct periods of work intensity to describe how having to be 'on call' for work limited participants' control over the pace at which they worked.

Compliance with Instructions.
We asked the 12 participants to share 90 minutes of video capturing their screen while working on the crowdsourcing platform.Participants generally complied with this instruction, although there was some variability in the duration of footage that was submitted (M = 93 min 34 sec, SD = 7 min 28 sec, range: 86 min 37 sec to 113 min 28 sec).4 presents an overview of the activities participants engaged in during the working sessions recorded.We next describe the six work activities across the whole dataset.

Activities Undertaken. Table
First, participants spent on average 13 minutes (M = 12 min 49 sec, SD = 9 min 52 sec) waiting for new jobs.Second, participants spent on average 16 minutes (M = 16 min 14 sec, SD = 2 min 50 sec) searching for jobs.Taken together, participants spent on average 17% of their unpaid working time waiting and searching for jobs whilst working on the crowdsourcing platform; we calculated this value by summing up all of the unpaid time participants in our study spent waiting and searching for jobs during the twelve 90-minute working sessions recorded.In other words, participants spent on average 17% of their working time on unpaid 'on-call' activities such as waiting and searching for new jobs.Finally, participants submitted 64 jobs in total (M = 5.33, SD = 2.46); we use the term submitted to differentiate between jobs that participants worked on and submitted for review, and jobs that participants worked on but had to return or had expired.In total, participants returned 46 jobs (M = 3.83, SD = 3.71); therefore, participants returned on average 42% of the total number of jobs caught.Furthermore, participants spent on average 17 minutes (M = 17 min 15 sec, SD = 12 min 19 sec) working on the 64 jobs.
In the next section, we describe how the aforementioned activities occurred during the three periods of work intensity observed in the data.Table 5 presents an overview of the amount of time participants engaged in the activities during periods of low, moderate, and high work intensity.In addition, Figure 3 presents a stacked bar chart of the activities across the three intensity periods and the whole dataset, for easier comparison.Next, we first report descriptive statistics from the Fig. 4. Participant P8 working with three different external tools to find new jobs.In the first two screenshots, the participant switched between two tabs to check the list of jobs available on the crowdsourcing platform using two different tools.In the last screenshot, the participant configured the settings on another tool to search only for jobs paying more than $1.99.All screenshots are presented as sketches in this article to maintain workers' and clients' privacy.
data.The descriptive statistics help us build a detailed picture of how participants were spending their time being 'on call' for work during the three periods of work intensity.Additionally, to give a more nuanced understanding, we describe examples from specific video clips along with the detailed comments that the participants provided to explain what was happening.

Periods of Low Work Intensity.
We observed that participants spent 39% of their time in periods of low work intensity, for an average of 33 minutes (M = 33 min 22 sec, SD = 16 min 9 sec).First, during periods of low work intensity, participants spent on average 25 minutes (M = 25 min 14 sec, SD = 16 sec) waiting for new jobs.Second, participants on average 15 minutes (M = 15 min 6 sec, SD = 1 min 54 sec) searching for jobs.Third, participants managed to catch new jobs approximately every 17 minutes on average (M = 17 min 5 sec, SD = 10 min 55 sec).Fourth, participants had zero or only one job lined up in the queue of jobs to complete.Fifth, participants spent on average 10 minutes doing unpaid work (M = 10 min 5 sec, SD = 47 sec).Sixth, participants spent on average 20 minutes working on jobs (M = 19 min 57 sec, SD = 1 min 36 sec).Finally, during these periods of low work intensity, we observed that participants were working at a slow pace during these periods, working on one job or on no jobs whatsoever.Participants were left waiting for new jobs that met their selection criteria to become available on the platform.During this time, participants filled their unpaid time with activities resembling break-taking (e.g., browsing the internet or watching TV shows).We next describe these activities.
Since only a low amount of work was available on the platform during these periods, participants had to spend a large amount of time searching for new jobs.To automatically search the platform for new jobs, participants used open source external tools.We observed each participant using on average three different external tools each (M = 3.28, SD = 1.64).We observed participants frequently switching between different external tools as they were waiting for work (N = 15).For example, as presented in Figure 4, we observed participant P9 using a dual-monitor setup and switching between three different tools over 30 seconds in one clip.When asked about this clip, the participant said that they used this setup to manage their work: I use two monitors because it is advantageous for [name of platform] (and productivity in general while doing any number of computer based jobs).[name of platform]'s own interface is limited, and many people, myself included, find it necessary to use various [tools] and extensions that help find, sort, filter, organize, and accept jobs.Thus, at any given time I might have a number of different windows running for these purposes, so having two monitors is extremely helpful to manage my work flow.-P9 Participants programmed their external tools to help them monitor the platform for new jobs that met their selection criteria (e.g., setting a minimum payment amount for a job).We observed participants actively changing the parameters of their external tools based on the availability of jobs on the platform over time.For example, participant P8 initially set their external tool to only alert for new jobs that paid more than $2.99.After not 'catching' any jobs for 10 minutes, the participant changed the parameters in their tool to return jobs paying more than $1.99.After 12 minutes of making this change, the tool had auto-accepted five jobs paying more than $1.99, and the participant worked on these jobs.When we asked P8 to elaborate on this video clip, they said that "While waiting for work [tool] will automatically grab anything that is $1.99 and above . . .I still count this as «working»".

Periods of Moderate Work Intensity.
We observed that participants spent 35% of their time in periods of moderate work intensity, for an average of 32 minutes (M = 32 min 9 sec, SD = 15 min 47 sec).First, during periods of moderate work intensity, participants spent on average 9 minutes (M = 9 min 4 sec, SD = 1 min 36 sec) waiting for new jobs.Second, participants spent on average 16 minutes (M = 16 min 12 sec, SD = 2 min 51 sec) searching for jobs.Third, participants managed to catch new jobs approximately every 5 minutes on average (M = 4 min 50 sec, SD = 3 min 39 sec).Fourth, participants had between two and five jobs (M = 2.96, SD = 1.78) lined up in the queue of jobs to complete.Fifth, participants spent on average 5 minutes doing unpaid work (M = 4 min 39 sec, SD = 49 sec).Sixth, participants spent on average 20 minutes working on jobs (M = 19 min 41 sec, SD = 11 min 12 sec).Finally, during these periods of moderate work intensity, we observed that participants were working at a moderate pace, actively working on jobs, as well as doing activities that are necessary for work (e.g., contacting clients, tracking earnings, reading forums or reviews, or checking qualifications).Unlike during periods of lower work intensity, participants were able to take a few breaks.We next describe these activities.
Since a moderate amount of work was available on the platform during these periods, participants had to increase their work pace to find and evaluate new work.We observed participants frequently (N = 16) switching between different external tools to catch jobs.Once participants caught a new job, they had to quickly decide if they wanted to keep it or return it.We observed participants using community-based reviews to find out more information to help evaluate the jobs in their queue.Furthermore, we frequently observed participants returning jobs (M = 3.82, SD = 3.73) from their queue of jobs.We measured the number of jobs participants returned by counting the instances in which participants clicked a job's 'Return' button on the crowdsourcing platform.There are many reasons the participant might want to return a job.We learnt about the reasons from the responses we received from the participants.For example, participants returned jobs because they did not think they could finish them before the completion time expired, or because they were not sure that the work they did would be accepted, or because of a problem with the job.We also observed cases (N = 6) in which participants stopped working on a job because a new, better-paid job became available to work on instead.To give an example, as presented in Figure 5, P8 returned a job that paid $2.00.P8 had eight jobs in their jobs queue (including ours).In their jobs queue, the most recent job they caught (which also happened to have the shortest time remaining) paid $2.00.The job did not have an estimated completion time, so P8 turned to a tool to check the hourly rate of the job for other workers who completed the same job.Other workers reviewed the job as "underpaid" (at a pay rate of $5.25/hour), and as a result, P8 returned the job.When asked about this clip, the participant said that they usually swap lower-paying jobs for higher-paying jobs: One other factor that I should mention is money.By that, I mean it can matter sometimes what the new job is paying and what the current job you're working on is paying.Usually this is a decider in jobs that you haven't started yet.If I was working on a low paying job like $.50 and the new job that came in was higher paying like $4 you might Fig. 5. Participant P8 returning a job that was underpaid.In the first screenshot, the participant checked their queue of jobs and noticed they caught a job that paid $2.00.To check the hourly rate of the job for other workers who completed the same job, P8 used an external tool, exemplified in the second screenshot.Other workers reviewed the job as "underpaid" (at a pay rate of $5.25/hour), and as a result, P8 returned the job in the last screenshot.
think that I would return the $.50 job and forget about it and just start on the $4 job since it pays more and I won't have to worry about [not] having enough time . . .[I]f my queue starts filling up and the time limits are conflicting then I will return lowerpaying jobs for the higher paying jobs but that is because I haven't started any of the jobs yet.-P8 As well as working on the crowdsourcing platform and doing activities that are necessary for crowdsourcing, participants took a few breaks (M = 1.57,SD = 0.78, range: 1-3), for an average of 5 minutes each (M = 4 min 46 sec, SD = 3 min 52 sec).The most common reason reported by participants for taking a break was to get a drink.Other break activities included checking social media (e.g., Twitter, Facebook, Reddit), spending time with children or pets, or doing other noncrowdwork.

Periods of High Work Intensity.
We observed that participants spent 26% of their time in periods of high work intensity, for an average of 23 minutes (M = 23 min 33 sec, SD = 16 min 19 sec).First, during periods of high work intensity, participants spent on average 3 minutes (M = 2 min 36 sec, SD = 18 sec) waiting for new jobs.Second, participants spent on average 17 minutes (M = 17 min 25 sec, SD = 1 min 49 sec) searching for jobs.Third, participants managed to catch new jobs approximately every 3 minutes on average (M = 2 min 46 sec, SD = 3 min 43 sec).Fourth, participants had at least six jobs (M = 6.23,SD = 1.26) lined up in the queue of jobs to complete.Fifth, participants spent on average 7 minutes doing unpaid work (M = 6 min 39 sec, SD = 47 sec).Sixth, participants spent on average 15 minutes working on jobs (M = 14 min 58 sec, SD = 11 min 46 sec).Finally, during these periods of high work intensity, we observed that participants were working at a faster pace, engaging in task switching to 'catch' new work.During these periods, participants in our study did not take any breaks.We next describe these activities.
Since a high amount of work was available on the platform during these periods, notifications for new jobs often came at points when participants were already working on a job.There were 34 occasions in the video data in which we observed participants switching away from the job they were currently working on to catch a new job that had just become available.Participants configured their external tools to notify them when new jobs became available that met their personal selection criteria to avoid missing out on jobs.For example, some of the external tools use pop-up visual notifications that give information on the name of the job, the name of the client, the payment amount, and the time expected to complete the job.Although these notifications helped participants be fast at grabbing new jobs, they could also distract them from working on an active job.To give an example of how these external tools could distract participants, in one video clip exemplified in Figure 6, participant P6 was halfway through working on a job when they were Fig. 6.Participant P6 switching away from the job they were completing to accept a new job that had just become available on the platform.In the first screenshot, the participant was halfway through a data collection job when suddenly they switched to another tab because a tool notified them via a sound alert that a new job had just become available.In the second screenshot, the participant reviewed the job and accepted it, adding it to the jobs queue.
alerted of a new one job by one of their external tools.When asked about this clip, the participant said they switched away from working on the job because the tool notified them that a new job had become available via a sound alert.After switching, checking, and accepting this new job, the participant returned to the job they had previously been completing: [Y]ou mentioned [the tool] and I am glad you did!The reason I keep checking that so aggressively is there is an Alarm that sounds whenever a job is available that meets my preset criteria.So in the video it may seem like I am beyond obsessively checking it but in reality it was a really busy day and that's why you see me click it sometimes in a millisecond.I just glance at what made it beep and resume my work.-P6 As new jobs became available at a faster pace, we observed participants using community-based reviews to find out more information to help evaluate the jobs in their queue (N = 12).For example, participants used scripts to check the client's history and the estimated hourly pay of the job.When asked to elaborate on one of these clips, participant P11 said that they were looking at the number of reviews a client had, and the time and hourly wage information of the job: [I] usually look at the number of reviews and whether there are any rejections or blocks.I then look at the hourly average . . .I use time and hourly wage information to decide if I should do a job or will I miss something more lucrative.-P11 Unlike during periods of lower work intensity, participants did not take breaks during periods of high work intensity.One reason for this is that participants had at least five jobs lined up in their queue of jobs during these high availability periods, and these jobs often had short completion deadlines.For example, as seen in Figure 7, in one video clip, participant P7 started working on a job that only had 20 minutes left until the completion deadline.Despite the job being advertised as needing 40 minutes to be completed, the participant proceeded to check one of the external tools that they were using and discovered that other crowdworkers had reported completing this job in around 18 minutes.After checking this information, participant P7 decided to work on the job.After 10 minutes of working on the job, a message was displayed: 'Feel free to take a break before the next round'.At this point, the participant switched back to their jobs queue to check how much time they had left on the job, where they learnt that they only had 9 minutes and 43 seconds remaining.At this point, P7 exclaimed: "I don't really have time for a break . . .oh god . . .I'm tired." P7 then immediately returned to the job and resumed working on it.Eventually, the job expired.P7 did not manage to submit it on time and did not get paid for working on it.When Fig. 7. Participant P7 unable to take breaks whilst working because of a job that had a short completion deadline.In the first screenshot, P7 was two-thirds into completing an experimental psychology job, which asked them to take a break before continuing working.In the second screenshot, the participant switched to the jobs queue to check how much time they had left on the job, where they learnt that they only had 9 minutes and 43 seconds remaining; P7 then immediately returned to the job and resumed working on it.Eventually, as seen in the last screenshot, the job expired, and P7 did not manage to submit it on time and did not get paid for their work.
asked about this clip, participant P7 elaborated that because of the time limits on the job, they were unable to take a break while working: Sometimes I have to rush to complete and submit [the jobs] within like 10 minutes, than [sic] oh my god, I have to go do that, and sometimes it's like putting the track in front of the rushing train sort of situation where the moment I am done with one job, the amount of time that I have left to complete the job in the queue is just barely enough to get that done, and then the amount to get the third one in the queue done is barely enough.So it keeps like this, one thing after another, after another, after another, until several hours have passed and I barely have time to pee or get something to drink.It's not the best situation to be in, but this is the situation I am in right now.-P7

Discussion
Taken together, the results of the video analysis study show that the variable availability of work on the crowdsourcing platform observed in Study 1 influenced the pace at which workers were working.It was easy for the work pace of the workers in our study to increase during busy moments.During these times, workers had multiple jobs queued up.To manage these fragments of work, workers had to switch between jobs and external tools to keep on top of everything.However, the 'quiet' periods when job stacks are empty also have knock-on effects, fragmenting the rest of the day.Having planned to set aside time to work on the crowdsourcing platform, workers were keen to make the most of this time.However, this time is not necessarily fungible; if there are few jobs, workers do not necessarily have the option to simply stop work and 'make it up' later.Instead, they remained 'on call', neither actively working on jobs but unable to turn off from searching and waiting for new jobs.Thus, workers were hoping to make as much as possible in the time available.Conversely, if there were plenty of available jobs, workers were reluctant to take breaks as planned and continue working without breaks.This finding fits with prior work that reports that crowdworkers are not able to take breaks because of the time demands they were under [112].Our study suggests that the time demands workers reported in prior work were likely due to workers having to work under the time pressures of 'catching' work before other workers and not taking breaks when work is available.
In terms of unpaid time, the results of the study suggest that workers in our study spent on average 17% of their working time on unpaid 'on-call' activities such as waiting and searching for new jobs during the twelve 90-minute working sessions recorded.Additionally, we observed that workers had to return 42% of the total number of jobs caught and had to continue waiting and searching for jobs to secure paid work.Moreover, the results of Study 1 further suggest that workers spent on average 22% of their daily working time on unpaid 'on-call' activities such as waiting and searching for new jobs.Waiting and searching for work is a form of work itself, unpaid and largely invisible [143].There are no wage guarantees on crowdsourcing platforms [55], and workers are also not paid for the time they are doing essential admin and 'meta-work'.Therefore, as there are generally more low-paying jobs than high-paying jobs on crowdsourcing platforms [81], crowdworkers have to compete against each other for the higher-paying jobs whilst 'on call' for work.Furthermore, for each 90 minutes of video recordings, workers in our study spent an average of 12 minutes on unpaid work activities (i.e., contacting clients, tracking earnings, reading forums or reviews, and checking qualifications) and on waiting and searching for jobs.In a survey study by Berg [17], for every 60 minutes spent on the crowdsourcing platform, workers spent 18 minutes searching for jobs and performing unpaid preparatory work (i.e., unpaid work).In comparison, in our study, for every 60 minutes of 'at work' time, participants spent on average 8 minutes on unpaid work.
Additionally, the study by Toxtli et al. [149] identified that crowdworkers spend 33% (i.e., a median of 33 minutes) of their time daily on unpaid work.We note that the aforementioned study categorised as 'unpaid work' two activities that we considered to be part of actively working on jobs (see Section 4.1.5),rather than part of unpaid work: (1) "Starting jobs but then returning them" and ( 2) "Doing jobs that eventually timeout" [149, p. 9].The latter activity (i.e., "Doing jobs that eventually timeout") was the most time-consuming activity observed in the study by Toxtli et al. [149] (for a median of 4.5 minutes and 37% of workers), whereas the first activity (i.e., "Starting jobs but then returning them") was the second most time-consuming activity observed in their study (for a median of 4.2 minutes and 92% of workers).However, in our study, we did not categorise these two activities as unpaid work, but instead we categorised them as 'Actively working on jobs', since the aim of our study was not to measure the amount of time workers have to take part in unpaid work activities (other than unpaid 'on-call' activities like waiting and searching for new jobs), but to observe the impact of being 'on call' on workers' schedules and work pace.Thus, since we did not categorise these two uttermost time-consuming activities as unpaid work, it is difficult to compare the amount of unpaid time reported in our study with the amount of unpaid time reported in the work of Toxtli et al. [149].Nevertheless, we hope that by reporting the amount of unpaid time workers spent in our study (an average of 12 minutes over 90 minutes of work), we can help further quantify the amount of time crowdworkers spent on unpaid work.Unpaid work ultimately impacts workers' wages-workers' already low hourly wages [81] go down from $3.76 to $2.83 when accounting for unpaid work [149].Thus, paying workers at a rate for jobs that gives them 'slack' for unpaid work (i.e, the non-task aspects of taking on jobs) would provide recognition of this invisible work, but could also provide the platform and clients with an incentive to reduce the time and effort involved in doing unpaid meta-work (e.g., with better tooling, robust and reliable jobs).
Finally, the results of our study suggest that workers had to multitask whenever new jobs became available.Toxtli et al. [149] pose the question "How exactly does multi-tasking and context switching relate to invisible labor?" (p.20).We provide an initial answer to this question by showing that the time pressures of being 'on call' to secure work before other workers shaped the task switching behaviours of our workers.Workers had to switch their attention away from the job at hand when new work became available.Switching between monitoring and performing work has also been observed in the case of knowledge workers in the study of Renaud et al. [132], where 84% of survey respondents kept their e-mail running in the background while working.However, constant task switching is considered to be more taxing than actual interruptions (e.g., being notified by a new e-mail or job) because of the frequent switching of attention from one activity to the other [69], and the added time pressures of responding to interruptions [117].Therefore, we argue that the time-urgent 'on call' nature makes the crowdsourcing platform a multitasking environment, in which workers have to be quick to react and switch away from their job when new work becomes available.Jobs, especially ones using time-based measures of performance, need to be designed with this behaviour in mind [72]-in some cases, multitasking might be a participant-level random effect that comes out in the wash, but for certain kinds of paradigm (e.g., memory-focused research), the constant meta-work being done during participation could have a systematic impact on results.

Limitations.
We conducted a detailed video analysis.A strength of video data is that you can actually see what is happening; therefore, nuances and details can be expected.The nuanced data meant that we could identify interesting clips of activity and share these with the participants to get more information about what was going on.There are, however, also limitations of the video study.Video is intensive to gather and analyse.As a result, this was a relatively small study with 12 participants recording only 90 minutes of activity.Although small in size (N = 12), it allowed us to generate a rich set of data to investigate the work practices of workers.However, the results in the study might not necessarily be generalisable to the wider population of crowdworkers and across different crowdsourcing platforms due to the small sample.To increase the generalisability of the results, the results of the study could be validated with a large-scale survey experiment (e.g., 1,000 participants working on different crowdsourcing platforms).Additionally, the results of the study could be validated with a system-level simulation or model, addressing questions such as "What are the costs of this problem to each type of stakeholder (e.g., workers, clients, platforms)?" or "What is the optimal behaviour for each type of stakeholder?". 1urther, the study was open to both experienced (i.e., more than 10,000 jobs) and novice workers (i.e., with less than 500 jobs completed), who self-selected to participate in the study.As a result, our sample consisted of more experienced participants (n = 9) than novice participants (n = 3).However, observing more experienced workers could have created potential biases towards the interpretation of our results, as the workers might have developed specific strategies to manage the pace of their work and take breaks.Future work should explicitly compare how novice and more experienced workers experience being 'on call'; it could yield valuable insights into developing strategies to cope and reveal ceiling effects, where even very experienced workers cannot deal with having to be 'on call' for call.
In terms of methods, one limitation of the study is that we could not record all of the activities of four participants who were using a second screen in their work.This limitation is because the screen recording tool we asked participants to use for the screen recordings only allowed participants to record one screen at a time.Thus, while we could tell when a worker switched from one screen to the other (i.e., the windows on the first screen became inactive), we could not record the activities displayed on the screens of four participants.In the data analysis, we only accounted for the activities that took place on participants' main screens.Where no activity was shown on the participants' screen (because they were using a second screen), we skipped these moments in the recordings as no activities were in focus and only analysed the moments that activity was on the recorded screen.Alternative screen recording software that allows recording multiple screens would have improved our ability to capture working contexts fully.
Another methodological challenge in this study is that, throughout, participants were working on our job by recording their screen and other jobs.This situation made setting the rate of pay difficult; we wanted to pay fairly, but at the same time, too high a rate of pay might have affected Having to Be 'On Call' for Work Limits the Temporal Flexibility of Crowdworkers 18:31 participants' behaviour in terms of work rate and job-finding behaviour.Thus, calibrating pay to be fair but does not unduly influence participants' behaviour is a difficult question for future work.Furthermore, we observed most activity during sessions to take place in an internet browser.It could be possible for future work to corroborate these findings by automatically recording browser activity.Indeed, such activity log studies have been useful in other areas to learn about large-scale patterns of users over extended periods of time (e.g., [2,147,158]).Logging is not quite this simple, though; we saw that not all work on the job occurred within the job's browser tab.For example, participants sometimes switched from the job tab to search and find information in a different tab.This switch meant that they were doing a Google search or reading a webpage.This activity may appear for a logger unrelated to the job, but in the video, we can see that these searches are clearly being done as part of the job.
Additionally, in the data analysis, we counted the time spent on work that was rejected as part of 'working time' while being on call.However, given the data collection method, it is difficult to identify which work was rejected and which was not, since it usually takes time to know whether a client will pay.Thus, future work could employ mechanisms that detect rejected work to better measure the amount of unpaid time participants spent whilst 'on call'.
A final methodological limitation is that we asked participants to record their screens at random points throughout the day for 90 minutes.Whereas the results of Study 1 suggest that crowdworkers tend to work in blocks of around 90 minutes, survey studies have indicated that workers report longer typical work sessions for full-time crowdworkers of approximately 300 minutes [112].Thus, future work could ask workers to record their screens for more extended periods and, at the same time, observe any potential variations in job intensity while being 'on call' for work.The challenge, of course, is to develop ways to measure aspects of behaviour that retain some of the fidelity of video coding (without the labour involved in coding it) without ending up with telemetry-based measures that might misclassify (e.g., work vs. non-work) activity.A promising method is described in the work of Toxtli et al. [149].They present a computational mechanism for quantifying the invisible labour of crowdworkers.The mechanism is built into a webpage plugin that can detect and quantify paid and invisible labour.In this sense, the plugin detects when a crowdworker is doing paid work or when they are doing invisible work, and then measures how much time the crowdworker puts into each of these two activities.In the future, the computational mechanism by Toxtli et al. [149] could build on our definition of being 'on call' on the crowdsourcing platform to quantify the amount of time crowdworkers spent whilst 'on call'.Furthermore, researchers could use their plugin to investigate to what extent crowdworkers have to be 'on call' for work on other crowdsourcing platforms and digital labour platforms.

GENERAL DISCUSSION
We situated the findings of the two studies with prior work in the previous Discussion sections of the article.Next, we summarise the contributions and the results of the two studies.We complete the article by discussing the implications of the results for (a) the people working on crowdsourcing platforms, (b) the design of crowdsourcing platforms, and (c) the wider platform economy.

Summary of Contributions and Results
In this article, we make three main contributions that extend and contribute to the existing HCI and CSCW research examining the working conditions of crowdworkers [60,75,82].In summary, our three main contributions are (1) defining, (2) quantifying, and (3) describing being 'on call' on a large crowdsourcing platform.We summarise the three contributions next.
The first contribution of this article is a definition of what it means to be 'on call' for work on some existing crowdsourcing platforms.Being 'on call' is a somewhat known [78,114,149] but undefined and increasingly popular working time arrangement of some existing crowdsourcing platforms [110].Therefore, in this work, we defined being 'on call' as a working time arrangement that requires crowdworkers to wait and search for jobs for an undetermined amount of time, often without getting paid, because of the platforms' lack of predictable work availability and lack of work assignment.By defining 'on-call' time, we were able to quantify and describe the ways in which this type of working time arrangement employed by existing crowdsourcing platforms contributes to workers' limited temporal flexibility.
The second contribution of this article is a measure to quantify the amount of unpaid time that crowdworkers have to spend being 'on call' for work.We quantified the amount of unpaid time crowdworkers spend waiting and searching for new jobs whilst being 'on call' for work.Whilst previous work quantified the amount of time crowdworkers spend searching for jobs [17,149], it did not quantify the amount of time crowdworkers spend waiting for jobs to become available.Overall, the results of Study 1 suggest that crowdworkers spent on average 22% of their daily working time on unpaid 'on-call' activities such as waiting and searching for new jobs.Additionally, the results of Study 2 further suggest that workers spent on average 17% of their working time on the two unpaid activities during the twelve 90-minute working sessions recorded.By quantifying unpaid 'on call' time, we extend the existing research examining the invisible labour of crowdworkers [75,149].
Finally, the third contribution of this article is empirical evidence that being 'on call' for work impacts workers' control over their daily schedule planning and work pace.Previous research suggests that the temporal flexibility of people working on crowdsourcing platforms is limited by both client-imposed constraints (e.g., strict job completion times) [111] and crowdworkers' tooling practices (e.g., increased multitasking) [160].In this work, we explored an additional contributor to workers' limited temporal flexibility: the design of crowdsourcing platforms, namely requiring crowdworkers to be 'on call' for work.We find in Study 1 that having to be 'on call' impacted participants' ability to schedule their time and stick to planned hours of work.Furthermore, we find in Study 2 that having to be 'on call' impacted the pace at which participants were able to work.Overall, the two studies we have presented in this article show that having to be 'on call' limits workers' temporal flexibility.Thus, by describing 'on-call' time, we extend the existing research examining the working conditions of crowdworkers by showing that being 'on call' for work impacts workers' control over their daily schedule planning (Study 1) and work pace (Study 2).

Implications for People Working on Crowdsourcing Platforms
Prior work suggests that the temporal flexibility of people working on crowdsourcing platforms is limited by both client-imposed constraints [111] and workers' tooling practices [160].The results of the studies we have presented show that the design of the crowdsourcing platform, namely requiring crowdworkers to be 'on call' for work because of the platform's lack of predictable work availability and lack of work assignment, is a contributor to crowdworkers' limited temporal flexibility.We next discuss the implications of the results.

Lack of Predictable Work Availability
. Study 1 investigated the relationship between crowdworkers' planned and actual working times.The results of the study suggest that the platform's lack of predictable work availability can lead to workers having limited control over the planning of their work schedules.Further, the study showed that participants could roughly keep to planned start and finish times for 'shifts' on the crowdsourcing platform.However, participants worked 2 hours fewer than they had planned.Furthermore, the workday was more fragmented than planned between these times though, with work distributed across twice as many pressures under which workers have to work.We know that when under time pressure, people tend to gather less information and to act quickly when making decisions [36].Time pressure affects human judgement, as well as decision-making [5], calling into question the validity of dataused in both industry and academic publications-provided by crowdworkers.Future work will be required to assess the impact of time pressure on crowdworkers' judgement and decision-making.
Furthermore, a high work pace can increase fatigue [53] and exhaustion [125] for the crowdworkers.Taking regular breaks can alleviate feelings of fatigue and exhaustion [43,135] and replenish their energy resources [142].However, participants in our study were not able to take breaks during periods of high work intensity.Lack of break-taking is of interest to the ongoing conversation about the working conditions of crowdworkers [75].For example, crowdworkers based in the United States do not benefit from state law paid rest breaks at work ("a paid 10-minute rest period for each 4-hour work period" [128]) since crowdsourcing platforms are in large part unregulated.In comparison, drivers on Uber's on-demand ride-hailing service can work up to a maximum of 10 hours before having to take a 6-hour break from completing trips [151].Thus, the results of our study extend prior work examining the invisible work of crowdworkers by showing that participants in our study were not able to take breaks during periods of high work intensity.
Finally, work scheduling and work pace are two components of temporal precarity.Temporal precarity is defined as the unpredictability, uncertainty, and insecurity workers experience with respect to work scheduling and work pace [100].Prior research exploring the sustainability of platform work has criticised the growing 'Uberization' of the workforce [84] and the exacerbation of work precarity that platform workers experience [57,159]; it has also asked for an investigation of the work precarity of platform work [9].Thus, the results of our two studies extend the current understanding of how platform workers experience temporal precarity [110].The results show that the current design of the crowdsourcing platform, namely requiring crowdworkers to be 'on call' because of the platform's lack of predictable work availability and, importantly, lack of work assignment, limits workers' control over their work schedules and work pace.

Implications for the Design of Crowdsourcing Platforms
Can anything be done about having crowdworkers be 'on call' for work?It is challenging to make practicable suggestions for improvements because, as other authors have noted, these issues are effectively 'features' and not 'bugs' in these platforms [110,111,114,160].There may be room to mitigate some of the worst effects of the temporal flexibility issue, and we focus on these here.

Lack of Predictable Work Availability.
Thinking specifically about increasing the temporal flexibility of crowdworkers, Williams et al. [160] suggest that temporal flexibility could be increased by, for example, crowdworkers having the "ability to limit their work hours to an 8-hour window during the day"(p.22).However, the results of our first study suggest that this suggestion would not help with some of the temporal fragmentation issues that our participants experienced.Our participants seemed to favour planning 'shifts' of work-clearly defined periods when they intended to be working.For the most part, our participants were able to stick to the start and end time of these shifts, but within these shifts, work was highly fragmented.The tool-related temporal fragmentation Williams et al. [160] noticed is another side effect of the nature of the platform and workers' desires to maximise the work they can do in the time they have set aside-an hour is not a fungible unit for workers in that they might be able to work right now, but not in 2 hours.To earn what they need when they can work, they need to use tools to help find more work.Having limited 'blocks' of work in this way may help with the 'leakage' of crowdwork into non-work time.However, it would not fix the high work intensity of workers who try to make as much as possible in the time available, which is a corollary of the limited availability of work on the platform.
Instead, crowdworkers might benefit from crowdsourcing platforms enforcing maximum working hours.Since crowdworkers are neither employed directly by clients nor employed by crowdsourcing platforms, they do not benefit from the labour and social rights that come with formal employment, such as maximum working hours regulations and access to a union.Instead, the 'workers' are classified by crowdsourcing platforms as Independent Contractors [75] and are free to work for an unlimited number of hours.The recommendation to enforce maximum working hours on the platform might be beneficial to reduce the increasingly higher competitiveness on crowdsourcing platforms.However, the issue of a small number of high-quality jobs on crowdsourcing platforms remains unaddressed.

Lack of Work Assignment.
Crowdworkers have to invest their time into building jobcatching tools and browsing community forums to find work [101].The results of Study 2 suggest that the lack of work assignment impacts the temporal flexibility of crowdworkers as workers compete against each other to 'catch' the higher-paying jobs.Therefore, one might recommend assigning workers jobs algorithmically (like in the case of Uber) instead of workers having to accept work from the pool of available jobs before other workers.Arguably, assigning workers to jobs could reduce the amount of time workers spend searching for jobs [149].Several methods for assigning jobs to crowdworkers, rather than having workers 'catch' them, have been suggested (e.g., [48,85]).For example, workers could be assigned jobs based on skills, expertise, past experience, job preferences, or personal interests.Hettiachchi et al. [83] argue that if crowdsourcing platforms assigned compatible jobs to crowdworkers, workers would spend less time and effort finding jobs.Regardless of how workers would be assigned jobs, Jones [99] makes the argument that job assignment should privilege the free time and autonomy of the workers.However, assigning workers jobs algorithmically is not a quick fix.We discuss in the next sections how Uber 18:35 drivers, despite being assigned trips algorithmically, still have to spend 40% of their working time waiting for a fare [34] due to the high competitiveness on the platform.
A few readers might be frustrated about the lack of tangible design recommendations to increase crowdworkers' temporal flexibility.Designers of crowdsourcing platforms can be intentional and impactful with their work in not building new features on top of an existing technology or not building a new technology altogether if, ultimately, it may not be possible to 'solve' design 'issues' [61] unless significant changes are made to the platform architecture and the business model of crowdsourcing.In this sense, we acknowledge that designers of crowdsourcing platforms do not operate in a vacuum but have to work with the business model of these platforms, as well as with stakeholders' objectives-further, even when researchers have tried to 'layer' tools on top of these platforms, the platforms have often terminated their access [95,136].Thus, the 'fix' for these issues is a big and ongoing problem.
However, policymakers need to continue to design policy that improves the working conditions of crowdworkers [149].In particular, we argue that policymakers need to be made aware of the potentially problematic working time arrangements perpetuated by some crowdsourcing platforms-that is, requiring crowdworkers to be 'on call' for work because of the platforms' lack of predictable work availability and lack of work assignment.We believe that revealing the platform architectures that make the exploitation of crowdworkers possible is pivotal to policy innovation and, ultimately, changing the flexibility discourse of the platform economy (e.g., individual freedom and flexibility [8]), and demanding decent work standards [74] for the workers (e.g., realised temporal flexibility [18] and fair pay [157]).Thus, this work contributes to the larger conversation about overlooked unpaid labour by defining, quantifying, and describing what it means to be 'on call' on a large crowdsourcing platform.Although the responsibility for improving crowdworkers' working conditions needs to be shifted away from the workers to crowdsourcing platforms and policymakers, crowdworkers themselves require financial and, consequentially, temporal support to cooperate and take collective action [136].Thus, technologies that provide insights into crowdworkers' working conditions could support the collective voice and action of the workers [95].For example, tools that reveal the platform architectures that make the temporal exploitation of crowdworkers possible (e.g., [94,149]) could empower the workers, as well as partners and advocates (e.g., scholars, unions, the public, or designers), to further critique and protest poor working conditions [95].
Going forward, AI practitioners are showing a growing interest in improving sociotechnical AI systems by incorporating more ethical practices in AI and ML models [141].For example, technology companies have been focusing on developing responsible AI frameworks that embody values such as fairness, accountability, and transparency [127].Within the goal of creating responsible AI, there is this opportunity to consider how responsible AI supply chains (or "fairtrade AI") can be created, focusing on human labour.We argue that AI systems generated with human labour that have good working conditions could potentially create more responsible AI systems.The resulting AI systems could be less biased, have better data quality, and have more market value.Furthermore, given that online platform work may become more regulated, technology companies might benefit from leading in the space of responsible AI supply chains rather than adapting their practices to new regulations.Defining a set of standards for responsible AI supply chains could be achieved through interdisciplinary collaborations between crowdworkers, clients, policymakers, civil society, scholars, and industry practitioners with expertise in HCI, AI, ML, psychology, and social science.
Finally, we leave the reader with a few open-ended questions.What if workers were assigned work algorithmically, instead of having to accept work from the pool of available jobs before other workers?Would this change increase workers' temporal flexibility?Would workers spend less time monitoring the platform for work?Would they have more control over how they do their work?Would they multitask less?Would they be able to take more breaks?Would clients miss out on flexibility?These are all questions for future work.

Implications for the Wider Platform Economy
With the rise of the "just-in-time" workforce within the platform economy, there has been a shift from 'scheduled' to 'on-demand' expectations from both services (e.g., Netflix) and people (e.g., Uber drivers) [110].As a result of the shift to 'on-demand' expectations, the speed at which platforms workers must accomplish jobs has increased, with consumers becoming the principal benefactors of the quick services within the platform economy [33,70].In contrast, we see that online platform workers such as crowdworkers do not benefit from the same flexibility as consumers [110].The results of the two studies presented in this article suggest that the temporal flexibility of crowdworkers is limited by having to be 'on call' to respond to the temporal demands of customers (i.e., clients).We next discuss the implications of our results for the wider platform economy.

Lack of Predictable Work Availability.
In this article, we argue that the lack of predictable work availability on crowdsourcing platforms is due to an oversupply of labour.Within the wider platform economy, labour platforms have an oversupply of workers [73], which makes the workers a 'disposable labour force' that can be quickly replaced [122].As a result of the COVID-19 pandemic, platform economy services are facing an increase in labour supply [52].The increase in labour supply has resulted in platform workers spending more time 'on call' for work.For example, in the case of Deliveroo, the largest on-demand food delivery service, riders began spending more time waiting for work due to the increase in labour supply [12,28].Whilst the number of riders working during the evenings increased, the number of orders stagnated.Consequently, riders spent more time waiting for work and earnings dropped.
The results of Study 1 suggest that workers spent on average 22% of their daily working time on unpaid 'on-call' activities such as waiting and searching for new jobs.Additionally, the results of Study 2 further suggest that workers spent on average 17% of their working time on unpaid 'on-call' activities such as waiting and searching for new jobs.In the context of the wider platform economy, Uber drivers report spending 40% of their time waiting for a fare [34].In February 2021, the Supreme Court of the United Kingdom ruled that the time Uber drivers spend working is not restricted only to the time drivers are driving customers to their destination but also covers any time the driver is logged into the Uber app, waiting to accept trips [146].In other words, Uber drivers ought to be paid for the time they spend waiting for a fare.The proposed Directive is a positive move in the right direction for improving the working conditions of people working in the platform economy.However, at the time of submission of the manuscript for this article, Uber is still yet to comply with the Supreme Court's ruling.Further, it remains to be seen how crowdsourcing platforms will transpose the Directive into practice across the regions in which they operate.
Finally, the effects of the oversupply of labour within the platform economy can also be seen on freelancing platforms such as Upwork, where freelancers report spending a high number of unpaid hours on waiting and searching for jobs because of a lack of available work [29].As a result, freelancers have to adapt their tool and software usage to support the temporal rhythms of their work, which, although grant the freelancers' high levels of temporal flexibility to find work at different times throughout the day, also blurs the lines between work and non-work [96].Therefore, we see more and more examples of people working within the platform economy that are impacted by the narrative of flexibility and individual freedom [8]. 18:37

Lack of Work Assignment.
In this article, we argue that the lack of work assignment on crowdsourcing platforms is due to the jobs being made available to most of the workers online, rather than workers being matched by the platform with suitable jobs.For example, in the case of Lyft and Uber's ride-hailing services, drivers are assigned work algorithmically.This form of algorithmic management is the most common type of automation found across the platform economy.Nevertheless, although Lyft and Uber drivers are assigned trips algorithmically, drivers still have to spend 40% of their working time waiting for a fare [34].Once a fare becomes available, riders have a mere 15 seconds to assess the offer based on the information provided, reach the screen, and accept a trip [116,150].If drivers do not want to accept a ride or are too slow to do so, they are penalised: their 'acceptance rate' drops and drivers risk having their accounts deactivated [133].Therefore, drivers have to work under similar time pressures of having to respond quickly to new jobs as the crowdworkers who participated in our second study, but cannot realistically opt out of accepting rides as they risk losing their jobs temporarily or permanently.
Whilst one might recommend assigning crowdworkers jobs algorithmically (like in the case of Uber; however, the decisions should be made transparent to the workers [113]), assigning jobs algorithmically is not a quick fix.We know from the experiences of on-demand ride-hailing drivers that the forms of algorithmic management employed by apps such as Uber can result in drivers spending longer hours than initially planned logged into the app just to be assigned jobs [133], but risk getting their accounts suspended if they decline too many jobs.Furthermore, in the case of Upwork, freelancers and clients are matched through algorithmic assignment based on a set of attributes that enables freelancers and clients to search for one another and get matched [97].
Upwork also notifies freelancers about potential jobs that might be a good match for their skill sets.However, algorithmic matching is not enough to match freelancers and clients on Upwork.Freelancers and clients also use the platform's communication channels and evaluation metrics to supplement the match-making process [97].Therefore, whilst working alongside the algorithm became culturally the most common type of automation within the platform economy, this arrangement seems far from the paradise of individual freedom and flexibility [8] envisioned for the future of work.
"Can we foresee a future crowd workplace in which we would want our children to participate?",Kittur et al. [105, p. 1] called on the HCI community in 2013 to consider a longer-term perspective for the future of crowdsourcing platforms.In this sense, the community has been aiming towards a future of work in which tomorrow's generation would participate in proudly.Alas, 'task assignment' remains one of the greatest roadblocks to achieving this aim [105], not only for crowdsourcing platforms but also for the wider platform economy.Furthermore, decent work standards such as realised temporal flexibility also remain a great roadblock to achieving this aim.

CONCLUSION
We presented two studies that show how having to be 'on call' on a large crowdsourcing platform, namely requiring crowdworkers to be 'on call' for work, contributes to workers' limited temporal flexibility.We argue that crowdworkers have to be 'on call' for work because of the platform's lack of predictable work availability and lack of work assignment.
Study 1 was a time-use-diary study in which 18 participants completed a (planning) 1-week work schedule diary at the start of the week and an (actual) 1-day activity diary.Results suggest that although participants started and finished work roughly when they intended to, participants worked on average 2 hours less than planned and spent on average 22% of their daily working time on unpaid 'on-call' activities such as waiting and searching for new jobs.In addition, the data suggests that participants' workdays were significantly more fragmented than workers planned, with work distributed across twice as many periods of work as desired.Therefore, the results of

18 : 9 Fig. 1 .
Fig. 1.Example of a filled-in '7-day Work Schedule' time-use diary from DP1. Participants were asked to mark with an 'X' each time interval when they had planned to work on the crowdsourcing platform over the next 7 days.Participants could tick the 'No work' box if they did not plan to work on the crowdsourcing platform on a particular day.

Fig. 2 .
Fig. 2. Example of a filled-in '24-hour Everyday Activities' time-use diary from DP1.The diary included columns for participants to enter main and secondary activities, as well as locations in their own words.Participants were asked to write down their activities throughout the day.

Fig.
Fig. Overview of the activities participants engaged in during periods of low, moderate, and high work intensity, and across the whole dataset, shown in a stacked bar chart.

Table 2 .
Difference between Actual and Planned the Crowdsourcing Platform Work from Study 1 DiariesA period of work is defined as a block of contiguous work that is separated from another period of work by a break of at least 10 minutes.Note: df = 17, * p < .01,* * p < .001.

Table 4 .
Overview of the Activities Participants Engaged in during the Working Sessions Recorded, across the Whole Dataset

Table 5 .
Overview of the Amount of Time Participants Engaged in the Activities during Periods of Low, Moderate and High Work Intensity 19 min 57 sec 1 min 36 sec 19 min 41 sec 11 min 12 sec 14 min 58 sec 11 min 46 sec Note that 'catching jobs' describes the frequency with which participants were claiming new jobs.