Abstract
Current cloud deletion mechanisms fall short in meeting users’ various deletion needs. They assume all data is deleted the same way—data is temporally removed (or hidden) from users’ cloud accounts before being completely deleted. This assumption neglects users’ desire to have data completely deleted instantly or their preference to have it recoverable for a more extended period. To date, these preferences have not been explored. To address this gap, we conducted a participatory study with four groups of active cloud users (five subjects per group). We examined their deletion preferences and the information they require to aid deletion. In particular, we explored how users want to delete cloud data and identify what information about cloud deletion they consider essential, the time it should be made available to them, and the communication channel that should be used. We show that cloud deletion preferences are complex and multi-dimensional, varying between subjects and groups. Information about deletion should be within reach when needed, for instance, be part of deletion controls. Based on these findings, we discuss the implications of our study in improving the current deletion mechanism to accommodate these preferences.
1 INTRODUCTION
Empowering users regarding data retention and deletion is very important in environments such as the cloud, where users do not have direct control over the infrastructure. Clouds can provide significant challenges to users, especially verifying the data handling practices of service providers. In fact, the current mechanisms for cloud deletion give users little to no control over how their data is disposed of [25, 44, 53, 54].
This work extends our previous work [54] on cloud deletion. We investigate two issues; users’ cloud deletion preferences and the information that may support users’ deletion needs in the cloud. We seek to answer the following key research questions: (1) how do users want to delete cloud data? and (2) how do they want to be informed about deletion? Despite evidence [25, 44, 53, 54] that users have various deletion needs, current cloud deletion mechanisms fall short in meeting these needs. For instance, they give users little to no control over how their data should be deleted (i.e., whether data should be completely deleted or recoverable). Prior studies informing the design of deletion mechanisms have primarily focused on understanding cloud deletion and users’ understanding of online deletion. While these efforts are relevant for future designs and the development of deletion mechanisms, they lack insights into deletion preferences or contextual reasons why users may want data to be deleted in a certain way. They have investigated only the narrow issues of data deletion, such as why people delete and the challenges they face. With this focus, they have neglected users’ other needs or the underlying reasons they want to delete.
Our previous study [54] has shown that cloud users have different motivations to delete. For instance, users may delete to tidy up their account, manage old data, or preserve their privacy. These different motivations for deletion may require different ways of deleting — for instance, while deleting just to tidy up one’s account may not need to be complete (moving data to the ‘trash’ folder may be enough), deleting to preserve one’s privacy may require deleted data completely removed from one’s cloud account. Moreover, when a user is unsure of the importance of a file, the user may delete it but hope to recover it when needed. Existing literature does not provide insights on these different needs or the ways to address them. Currently, users are limited regarding how they can delete their cloud data; all data stored in the cloud is temporarily removed from users’ localities before being completely deleted. However, this is in contrast to how users may want to delete their data. Cloud providers offer no help or controls to meet these different needs [53].
Regarding information about cloud deletion, our previous work [54] also suggested that users are not satisfied with how information about deletion is shared and distributed. For instance, participants stated that information about deletion — unlike other information, such as storage size — is found in privacy policies only and is not easily accessible. Nonetheless, privacy policy shortcomings are well understood in the literature. They have surfeit information, are difficult to understand, and are usually tailored to demonstrate compliance with legal requirements [23]. Moreover, concerning data deletion, information about deletion is usually compact and short, missing other aspects of data deletion, which may impair transparency. None of the existing studies about the cloud provides insights on what information about deletion users consider important, and when and where it should be shared with them.
To address this gap, we asked the following research questions: (1) how do cloud users classify cloud data, that is, what kinds of data do they treat similarly and differently with regards to sensitivity and importance?; (2) how do users want data to be deleted, what preferences can be identified from this, considering (a) deletion under individual contexts and (b) social contexts/shared folders? Are these preferences consistent?; and (3) is it feasible to design deletion mechanisms that satisfy their deletion preferences? To address the second research question, we examined the following: (1) what information about deletion users consider important, (2) when do users want this information to be presented to them, and (3) where users prefer to find such information.
In summary, our work makes the following contributions.
Cloud data classification. We identify three categories that users commonly use to classify data stored in the cloud. Cloud users generally categorize data under the following groups: (1) essential and sensitive, (2) less important and less sensitive, and (3) important but less sensitive. Essential and sensitive —this is a group of data items that users consider to be necessary or useful and private. Less important and less sensitive —a group of data that users consider less valuable, easy to produce, and less private whereas important but less sensitive data is the data they consider useful and hard to produce but they are happy with other people knowing or seeing it.
Characterization of cloud user deletion preferences. We identify and characterize cloud user deletion preferences with regards to deleting in individual and social contexts. Our analysis uncovers four characteristics of cloud user deletion preferences, including their complexity and dimensions. Most critically, we discuss how the reason for deletion, the perceived importance (file utility) of the file being deleted, the size of the file, file sharing context, sensitivity, and the storage size underpin these preferences.
An understanding of users’ preferences regarding cloud deletion information. We find that users consider information on cloud deletion scarce, not useful, and that it is usually presented to them at the wrong time through a wrong channel. They prefer that technical information about deletion tasks (e.g., how deletion is carried out in the cloud) be made available to them in blogs while information about who has access to data stored or deleted from the cloud and what happens when data is deleted should also be made available through other channels (e.g., frequently asked questions) besides privacy policies.
The rest of the article is organized as follows. We continue with the discussion of previous studies on cloud deletion in Section 2. Section 3 provides a detailed summary of our previous work on cloud deletion [54]. Section 4 gives a general overview of the approach behind our study. We present the results of the first activity (data sorting) in Section 5, the second activity (deletion preferences) in Section 6, and the last activity (deletion information) in Section 7. Section 8 comprises the discussion and implications of our study and several guidelines for developers of cloud storage platforms. We present our conclusions in Section 9.
2 RELATED WORK
2.1 Cloud Deletion
Both formal studies and anecdotal evidence suggest that deletion is essential to cloud users. Users delete for various reasons; however, failure to do so can lead to unintended disclosures, clutters, regrets and emotional trauma [35, 44, 53]. While there has been some work focusing on understanding challenges of deleting from other platforms, such as social media [2, 17, 22, 61], user studies on cloud deletion are novel and sparse [25, 44].
Most usability studies [3, 19, 33, 75] around the cloud focus on other aspects, such as sharing, synchronization, perceptions, and privacy. Earlier studies argued that not all users intended to use the cloud storage; they were mainly accidental users—commonly surprised to find their data in the cloud. The mismatch between users’ expectations and the reality of using the cloud has been cited as the main reason. Users usually misunderstand file synchronization, sharing, and deletion. For example, in the study by Capra et al. [8], some participants manually synced their files to the cloud. Regarding deletion, previous studies have reported mixed perceptions. Ion et al. [19] reported that most users understood that cloud deletion was not instant and permanent; they believed files were still recoverable for some time after deletion. However, younger participants in the cross-generational study [3] believed that deletion was instant and permanent. In our previous study [54], some participants did not understand that deleting from a shared folder would affect all collaborators. Khan et al. explained that the misunderstanding of how shared folders work often leads to users refraining from deleting old files, even when they have the access rights to do so [25].
In a non-cloud context, Murillo et al. found that misunderstandings and unfounded expectations of deletion are limited to user interfaces [44]. Lack of understanding explanation or impact of deletion on the interface can lead to users not correctly assessing the effects of deleting messages [59]. Others [19, 44] also argue that limited understanding of what happens in the backend may lead to unexpected results. Some studies [44, 53] have noted that, in order for users to have a better understanding of deletion, they are expected to understand deletion concepts such as backend, timeliness, backup, derived information, completeness, anonymization, fine-grained, and shared copies. However, these concepts are not known among users; after interviewing 36 cloud users, Ion et al. found that users do not understand and know timeliness or data retention [19]. While these studies highlight the need for understanding deletion, none suggests how users can acquire such knowledge or what are users’ preferences regarding the types of deletion they want concerning particular types of data. Also, it is necessary to study how users delete data in settings when it is shared with others using the cloud service.
Offering complete deletion in the cloud is still a challenge. Hao et al. [14] argue that it is impossible to ensure data deletion in the cloud using software-based approaches without physically tampering with the disk. Our previous work [53] has also identified and discussed the challenges of guaranteeing deletion in the cloud, highlighting that its salient infrastructure and operational features make it difficult to completely delete data from the cloud. Despite this, some efforts have been made to ensure deletion in the cloud. For example, Tang et al. built a policy-based deployable cloud storage system (i.e., FADE) to protect deleted data [69]. Another scheme that offers fine-grained deletion was proposed by Mo et al. [43]. This scheme is based on a key modulation function (KMF) that allows users to delete individual data items without re-encrypting the rest of the data. Nonetheless, these cryptographic-based solutions work by denying access to the deleted data. They do not remove the deleted data from the cloud, which means that the data can still be leaked through other attacks (e.g., brute force attacks or incorrect usage of cryptography libraries during implementation [49]). Moreover, current cloud services do not specify the type of deletion they offer or whether deleted data can still be recovered.
2.2 Classifying Data
Previous works on deletion report that deciding what data to delete or keep is a significant challenge for most users because it involves predicting the future—users are not good at predicting their data preferences over time. They also must keep in mind the type of data they are deleting, the importance of the data, and whether they will need such data in the future [3, 13, 54, 68, 73, 74]. This process is ongoing and usually relies on two distinctive extremes: hoarding (i.e., keeping data even if it is not valuable) and minimalism (i.e., avoiding storing too much data or regularly deleting data) [73]. It is decided on a highly individual level dependent on context, service, and usefulness [44]. In group settings (e.g., shared folders), it is even more challenging because users must consider the future information needs of other collaborators [51].
Deciding on what data to keep or delete from the cloud may require users to understand how the cloud or cloud deletion works [44, 54]. Consequently, several researchers have investigated how users classify data. For example, some users classify data through the lens of similarity [7]. Other times, their approaches are contextual [75]. Voida et al. found that users segmented cloud data into multiple mental places, for example, work or family [75]. In shared repositories, users usually view data as theirs or belonging to others but rarely as co-owned (i.e., common ownership) [51]. This usually leads to users forgetting about the existence of these data. However, when shown old data, most users tend to delete it rather than adopt other remediations [25]. To help users classify cloud data, Khan et al. [26] proposed a tool to identify whether a file was sensitive and useful. Their results showed improvements over the state-of-the-art baselines, from 26% to 159%. This also improved the prediction of whether the user will keep or delete the file by 10%. However, Khan et al. [26] argue that basic metadata and demographic information are not particularly strong predictors of users’ file-management decisions.
Other tools have been proposed to help users manage their cloud data. Khan et al. [25] developed a retrospective cloud data management tool and found that 83% of their participants preferred deleting at least one file they saw from their accounts. Brackenbury et al. [7] developed a tool to help users manage similar files with the intuition that they should be managed similarly. They found that users were more likely to accept recommendations to delete the files than to move them. While these tools are promising, their performance suggests that automatic cloud file management (including deletion) is still at an early stage. Currently, users are still responsible for classifying and managing their cloud files.
2.3 User Preferences
One of the many motivations for cloud deletion is privacy [25, 44]. Prior research has extensively focused on understanding users’ privacy preferences regarding different personal data and technology. Efforts have focused on online social networks [12, 30, 41, 42], smartphone privacy [20, 21], advertising [38, 77], location [1, 34, 39], and data-sharing preferences [36, 46, 62]. Results from these studies have led to various privacy mechanisms and improved user interfaces for users to control who can see their posts and avoid regrets in social media, for instance [12, 41, 42]. Also, in other contexts, such as online advertising, users may like to state their preferences not to be tracked or select the kind of ads they want to see, with Melicher et al. [38] reporting that users found this beneficial and offering them a sense of control. To address users’ privacy concerns in smartphones, users can give various applications different access permissions [28, 29]. Regarding deletion, design and longitudinal data management studies point to a growing need for tools and mechanisms that can support users in decisions to keep and discard their data [13, 74]. They emphasize how individuals’ practices and preferences differ in their daily lives. For example, users often want to delete similar files even if they were not stored in the same folder [7]. Moreover, they frequently delete files they consider not sensitive and useless. However, they always want to recover work and school-related files if they were deleted by mistake [26]. Studies around deletion preferences concerning cloud deletion are minimal though these two studies evidence some preferences for deletion. In other areas, studies around user preferences have enhanced users’ control over their personal data and privacy; insights on deletion preferences are missing, particularly when it comes to deletion from the cloud.
2.4 Privacy Policies and Terms of Service
Today’s online services use notice and choice (i.e., consent) as the paradigm for giving consent online [63]. User agreements, Terms of Service agreements, and Privacy policies are expected to contain the service provider’s data practices (i.e., data collection and use), presumably providing users with sufficient knowledge to help them make informed decisions on whether they should disclose their information or stop using the service [24]. With regards to deletion, they are expected to explain how data is deleted and provide the service provider’s deletion practices, including retention policy and recovery terms. Nonetheless, the vast amount of literature has reported on the usability problems of current privacy policies. The majority of these are long, unreadable, and contain irrelevant information [24, 37, 50]. Some are not aligned with user privacy concerns [11]. Others fail to inform users, leaving them helpless [58]. To actively help users manage their privacy, the authors of [58] suggest that control mechanisms must be relevant, actionable, and understandable. They also identify four main dimensions to consider when designing to provide notices: timing, when should a notice be presented; channel, how should the notice be delivered; modality, how the information should be conveyed; and control, how choice options are integrated into the notice. In the context of cloud deletion, improving these notices will help users understand the provider’s deletion practices and inform their choices with regards to deletion. In this work, we explore what information about deletion in the cloud is essential to users, when it should be presented to users (timing), and the channel that should be used.
3 PREVIOUS STUDY - DELETION PRACTICES
In our previous study on cloud deletion [54], we explored several key questions fundamental to usable privacy and security with regards to cloud deletion:
Motivating factors behind cloud users’ need to delete
The challenges they face or factors that underpin their failure to delete
The strategies that they employ to circumvent the challenges
The deletion experience that users want
Using semi-structured interviews (n = 26) and grounded theory analysis, we identified four key drivers that motivate users to delete, three key themes that explain deletion failures, and various coping strategies that users adopt to deal with failure to delete.
We found that users’ motivations to delete are privacy, expertise, policy, and storage driven.
Privacy driven - This group of participants deleted because they did not trust the cloud provider, wanted to forget certain files, and wanted to avoid future conflicts.
Expertise driven - Participants’ motives to delete were based on users’ understanding of the cloud deletion and ability to delete. When participants were confident that they could delete, they were more likely to delete than those who were not confident.
Policy driven - Some participants deleted due to extrinsic policies of their workplaces or perceived value of the files.
Storage driven - Participants were mostly driven to delete to free up storage and avoid clutter.
With regards to deletion failures, our analysis revealed that some deletion failures were due to limited access to information about deletion, interface issues, and users’ cloud deletion mental models. Participants revealed that very often information about deletion is not enough or not easily accessible as other information, such as how to upgrade storage. Others stated that sometimes their cloud application crashes or they struggle to find some deletion features, for example, deleting items from cloud recycle/trash cans. We also found that some deletion failures stem from the misunderstandings and the concepts that some users have about cloud deletion. For instance, some cloud users refrain from deleting from shared folders because they are not sure whether the file they want to delete will be removed from the visibility of other shared folder members. Others, for example, those who are privacy driven, may end up having files in their ‘deleted items’ folder without their knowledge because they may think deletion is permanent.
The study also revealed that cloud users adopt various strategies to address their challenges to delete. For example, some participants prefer to delete from certain devices (e.g., sync folders in their personal computers) or platforms from which they are confident they will be able to delete, while others choose to filter what files get stored in their cloud accounts so that they may not require the need to delete. We also learnt that the choice of strategy is not consistent It is always changing depending on the nature of the challenge, their expertise level, the device they are using, and the reason they want to delete. For instance, participants who delete to free storage favored the strategies that created space (e.g., deleting other files) while those whose motivation to delete is privacy related will adopt a strategy that removed the file from the storage.
Our work revealed that some cloud users desire to have transparency regarding deletion; they wanted information about how service providers delete or handle deleted data to be made freely available. Others expressed the desire to have a complete deletion (i.e., permanent deletion of data and metadata) as they stated that they were not confident about how data is deleted. Some suggested that they lacked control over the deletion process and would prefer having control over how deletion should be executed, for example, how long deleted files should remain in the deleted items folder. With regards to getting help with deletion, others suggested having dedicated services for answering queries about the deletion. Figure 1 gives a summary of these findings.
Fig. 1. Adapted. Key Findings from our previous study [54].
Overall, our previous work helped us to understand users’ cloud deletion practices and how users cope with deletion failures. However, it did not give insights on users’ deletion preferences or inclinations concerning the information about the deletion. Participants expressed their desire to have complete deletion and control over deletion. Nonetheless, our work did not explore these views in depth, for example, we did not ask our participants to explain the scenarios in which they would want complete deletion or whether they would want complete deletion for all their cloud data. With regards to information about deletion, participants expressed the desire to have transparency about the deletion and more information about cloud deletion. However, we did not examine the kind of information about cloud deletion that they wanted or where they would expect to find it. In this article, we aim to address these limitations and provide insights on users’ cloud deletion preferences, highlighting situations in which users may want complete deletion over other types of deletion. We also provide an understanding regarding information about deletion, what information users consider essential, and where and when they would want to have access to it.
4 METHODOLOGY
To investigate users’ deletion preferences and information requirements, we used three participatory action research (PAR) [5] tasks. PAR involves participation and action from a group of people who are affected by the same problem and act together to tackle it. As a collaborative research methodology, it offers researchers the opportunity to co-develop or investigate with users. It stresses users’ lived experiences, social changes, and their construction knowledge, which can be useful for solving their everyday challenges [4]. This facilitates discovering and developing solutions that are viable and useful to users. Moreover, we chose PAR because it is a well-established method in human-computer interaction (HCI) to explore complex issues with users [5, 48]. Cloud deletion is not an easy topic for all users [44, 54]; gathering a group of users to explore it is likely to yield better results than discussing it with individuals on a one-to-one basis. We chose the sorting (i.e., grouping) method for each PAR because sorting is a natural cognitive process routinely used in everyday life on which many evaluations and decision-making processes rely [31].
The rationale behind each PAR follows.
PAR 1 - Data sorting. We chose data sorting activity because the findings of our previous work [54] suggested that sometimes participants considered the file type when choosing which file to delete, for example, when deleting to free space, participants reported that they usually choose the file they did not consider to be important. In this current study, we wanted to know whether participants’ perception of the file (i.e., importance and sensitivity) also influenced their deletion choice. Letting participants group these data types beforehand seemed reasonable before asking them how they would delete such data.
PAR 2 - Deletion preferences. Participants deleted cloud data for various reasons [54]. We hypothesized that this might be linked to deletion types; thus, asking them to sort data this way seemed an intuitive way to understand how participants want their data to be deleted.
PAR 3 - Information Requirements. We asked participants to categorize information about deletion in terms of how important it is to see it and the communication channel because prior studies, for example, Murillo et al., suggest that users would perceive deletion more accurately if they understood the deletion concepts we used [44]. Thus, we created these concepts and examined whether participants valued some more than others and when and where they would want such information to be made available. Allowing participants to group concepts using our predefined concepts helped us to observe these differences easily.
Our study involved completing a pre-study survey and then the three (3) activities/PARs as shown in Figure 2. These PARs are described in detail in Sections 5, 6, and 7. In this section, we focus on giving details on the overall method, the study procedure, recruitment, data collection, and data analysis.
Fig. 2. The survey was taken prior to attending the study sessions. Participants first classified data types, then completed the task-based exercises.
Study Procedure
Before users could attend a session for task-based exercises, participants were asked to complete an online pre-survey that, in addition to obtaining demographic details, assessed their perception of cloud deletion practices.
Selected participants (criteria explained in the next section) were invited to our labs to complete the rest of the study. The first activity involved sorting data types individually, then as a group. The second activity concerned deletion preferences and the last activity was about deletion information. PAR 3 and 3 activities were group tasks. The lead researcher moderated all sessions. At the beginning of each session, the lead researcher explained the general aim of the study, what was required of the participants in each task, answered participants’ queries, started discussions around each PAR (including probing participants to explain their reasoning behind their sorting preferences or ask the other group members to give their opinions), managed time, and ensured that the objectives of the study were addressed. The facilitator also ensured that all study materials were available and that all categorizations were photographed before, during, and after activities.
Recruitment, Ethics, and Data Collection
After obtaining an ethics clearance, we recruited participants through social media, word of mouth, and ads around the university and the city center. Interested respondents were encouraged to complete a screening form. The purpose of this questionnaire was to identify active cloud users who were 18 or older meeting three or more of the following:
Having deleted from the cloud through more than one device or interface (so that they could provide their experiences based on more than one interface)
Having more than one cloud account (so that they could share their experiences based on more than one provider)
Sharing some folders (to be able to gather their deletion preferences from shared files)
Experienced some challenges when deleting (to learn about how they would solve their challenges)
Interested in cloud deletion
Being able to attend the participatory study
Sixty-five (65) people (40% identified as male) responded to our ads and completed the screening questionnaire. Sixty stated that they have deleted in the cloud, 17 of whom experienced some challenges while deleting, 76.9% sharing folders, 46.2 % had more than one account, and only 3 people stated they could not attend to participate in the study.
In the end, 20 (50% females) participants were invited to take part in the study. We divided them into four equal groups. Each participant was given a day and time when the study would take place and was asked to confirm their availability. While the initial group allocation was random, for diversity purposes, we ensured that each group contained at least three non-student participants, not more than three people of the same gender, and diverse age groups. We considered this sample sufficient for the study and complexity of the topic. Deletion as a topic is not as popular as privacy. Consequently, a smaller number of participants allowed us to probe and ask follow-up questions during the session. Complex topics are easy to explore using the participatory method [5, 48]. Moreover, Oates and Alevizou [45] suggest that conducting studies with groups of five to eight is sufficient to generate discussions and provide valuable insights. Table 1 lists the demographics of all of the participants. We obtained consent to record audio and take pictures (i.e., pictures of the props without participants’ faces) during the sessions. Each participant received compensation worth $7.00 for their time.
| Code | Group | Gender | Age | Employment | Accounts | Cloud Services |
|---|---|---|---|---|---|---|
| P1 | A | Female | 21–25 | Student | 2–3 | Dropbox, iCloud, Google Drive |
| P4 | A | Male | 31–35 | full-time | 4–5 | Dropbox, Box, Google Drive, OneDrive |
| P11 | A | Female | 31–35 | full-time | 2–3 | Google Drive |
| P17 | A | Male | 18–20 | Student | 2–3 | Dropbox, Google Drive, OneDrive |
| P18 | A | Male | 31–35 | full-time | 6 + | Dropbox, iCloud, Google Drive, OneDrive |
| P2 | B | Male | 31–35 | PhD student | 2–3 | Google Drive Box |
| P5 | B | Female | 31–35 | full-time | 2–3 | Dropbox, Google Drive |
| P7 | B | Female | 18–20 | Student | 1 | Google Drive |
| P13 | B | Male | 26–30 | full-time | 2–3 | iCloud, Google Drive |
| P19 | B | Female | 41–45 | full-time | 4–5 | iCloud, Google Drive, Dropbox |
| P14 | C | Male | 26–30 | full-time | 2–3 | iCloud, Google Drive, OneDrive |
| P3 | C | Male | 26–30 | PhD student | 2–3 | iCloud, Google Drive |
| P8 | C | Male | 21–25 | full-time | 2–3 | Google Drive, OneDrive |
| P9 | C | Female | 18–20 | Student | 1 | Google Drive |
| P20 | C | Female | 31–35 | full-time | 2–3 | Google Drive, OneDrive |
| P10 | D | Male | 26–30 | part-time | 2–3 | Dropbox, iCloud, Google Drive |
| P6 | D | Female | 26–30 | Unemployed | 2–3 | Dropbox, Google Drive |
| P12 | D | Female | 26–30 | PhD student | 1 | Google Drive |
| P15 | D | Female | 31–35 | Student | 4–5 | Google Drive, OneDrive |
| P16 | D | Male | 36–40 | full-time | 2–3 | Google Drive, Box, Amazon Cloud Drive |
Table 1. Summary: Study Demographics
In total, we collected 224 min worth of audio from all of the group sessions and took a total of 94 images (188 MB). Sessions took an average of 63 min, excluding breaks.
Pilot Studies
After obtaining ethics clearance, we ran three pilot study sessions with three different groups of participants (i.e., four participants per group). These sessions were used to understand how long each task would take to complete, assess whether the research protocol is realistic and workable, and whether the study props were enough to complete the studies. Pre-tests were run as if they were the final study; participants completed the ethics process, and the researcher conducted the study as if the results were going to be used for the final report. However, during the pilot study, the researcher noted how the study was going, noting challenges and how participants engaged with the props. At the end of each session, the participants were asked how they found the study, what they struggled with, and what they would remove from the study.
As a result of the pilot study, we removed some data types because participants suggested that they would not usually have such data in the cloud and felt similar to other data types we already had. In the end, we removed 12 data types. Some data types were removed to reduce the time to complete PAR1 and PAR2 tasks. Initially, we did not provide participants with deletion types. However, we discovered that without deletion types, participants decided on only two options, to delete and not to delete. Consequently, this did not provide any variation on how they would delete their data. However, after introducing deletion types in the second session, we noticed differences in how they wanted data to be deleted. This also led to rich discussions about deletion types. Moreover, we initially had one break between PAR1 and PAR2. However, after the first pre-test, we allowed participants to take breaks after each session if they wanted to. Data collected during the pilot studies was not used in the final results.
Analysis
After transcribing all of the audio and photos, we performed qualitative analysis—a thematic theory approach. To generate a codebook, a lead researcher independently coded all of the data from the first group. The first stage of analysis involved the identification of various data classifications from photos from PAR1 using the open coding technique [9, 16]. This first focused on the individual sorting and then the group sorting. The lead researcher recorded the characteristics of each group and classified similar groups since they had overlapping features. For example, groups that participants labeled personal, private, important were grouped together because they overlapped. Then, the PAR 1 section of the transcripts was coded, mainly to understand the groups’ reasons, which helped inform the decision behind grouping certain groups together. This process led to the first codebook. After identifying the high-level groups, the second researcher analyzed the same photos and transcripts to determine agreement or disagreement with the first coder. The two researchers then discussed the codebook, especially the high-level groups identified from the data.
The lead researcher then coded the second and third PARs following the same process, with the second coder confirming and discussing the codebook. The only difference in analyzing PAR2 and PAR3 was that we used closed coding [9, 16]. Initial codes for PAR2 and PAR3 were the categories we used in the activities. For example, PAR2 categories were the four deletion types we used in the deletion activity. After compiling a codebook, two researchers independently coded the rest of the scripts from the remaining groups using a single codebook. The Cohen’s kappa coefficient agreement was found to be 0.72, showing a high degree of agreement between the two researchers. After the independent step, the two researchers collaborated to refine the disputed codes that resulted in disagreements. We found that this was due to researchers interpreting some codes differently; thus, the codebook was refined to clarify them. After the initial coding was complete, further analysis revealed themes and categories about users’ deletion preferences.
After coding the first two sessions, we did not see any variation within the groups that participants generated in PAR1. In some cases, the contents of the grouping would have more data types or different data types, but the definition of the groups by the participants remained the same. However, we continued with the rest of the group session to confirm whether we had reached saturation point.
5 PAR1 – DATA SORTING
Activity Design
To explore the deletion preferences, we developed a sorting task that required participants to sort various types of data (e.g., meme videos, passport copies) individually and as a group. We adopted a free-sorting technique and asked participants to categorize given data types according to how they perceive them so that similar data types are gathered together. The categories were not predetermined so that participants could create as many groups as they saw fit. There were 26 data types in total, selected from various research work and online sources [15, 18, 27, 40, 64]. We listed the suggested data types from all of our sources, then generated more data types like those indicated in the listed sources. Based on this list, we then discussed which data types to include or exclude from our study. These were data types that users were familiar with, particularly around the context of cloud usage. We wanted a list of data types that covered most aspects of an individual’s life and may pose privacy reasons for participants if not handled well. Table 2 shows the list of all data used in our study. The purpose of this task was to visualize and identify participants’ intuitive categories and investigate whether these categories would have any influence on their deletion preferences. On average, this activity took each participant 5 min and around 13 min when part of a group.
| Data types | |||
|---|---|---|---|
| Medical report/information | Music videos | Old birthday video | Children photos (Family) |
| Rifle licence | Honeymoon photos | Genetic information | Facebook downloaded data |
| Immigration documents | WhatsApp backup | Family photos | Research data |
| Personal information | Meme videos | Job application letter | 4 MB video clips |
| Biometric data | Meme images | Legal documents | Business contracts |
| Passport copy | 3 GB wildlife video | E-books | Friends photos |
| Old bank statements | Pet care information |
Table 2. List of Data Types Used in the Data Sorting Activity
PAR1 – Findings
Following open coding, we identified three themes that show how participants categorised cloud data.
We found that people classify data differently—they create various groups, sometimes overlapping. Participants frequently categorised data according to sensitivity, utility, content, and use. Common groups included: sensitive, personal, important, less important and less sensitive, miscellaneous, and sensitive and important. Some participants highlighted that important data is the data that they consider useful (utility), hard to get, and do not want to lose whereas sensitive or personal data is the data they consider private and can be used by others to identify them. For consistency purposes, we grouped similar groups and recorded them as a single group. As a result, we ended up with three groups: important and sensitive, less important and less sensitive, and important but less sensitive.
Important and sensitive: This group contained data that participants described as private, personal and that they did not want to share with unknown or unauthorised people.
Less important and less sensitive: This consisted of data that participants considered less useful, easy to reproduce, and less private.
Important but less sensitive: This group contained data that participants considered useful and hard to reproduce but were happy about other people knowing about it.
We observed some differences in the data sorting activity between individuals and groups. The results of our sorting activity are shown in Table 3. We discuss these differences next.
Individual sorting. Individually, participants generally classified data into four to five groups. The most common groups were personal, sensitive and important, not important, miscellaneous, and work. Other groups were named entertainment, family, and less sensitive and less important. Similar groups (according to their description or properties) were joined together, and we found that participants had fewer data types categorised as important but less sensitive. The important and sensitive group included data that was about them (e.g., copy of a passport) or data related to their families (e.g., family photos). Data types perceived to be less important and less sensitive included music videos and Facebook downloaded data.1
Group sorting.
In group settings, participants categorised data into three or four groups. The groups were fewer in number and contained more data types. After grouping these collections into our three broad groups, the less important and less sensitive group had fewer items while the important and sensitive group had the most items. Some data types were split between the two groups. For example, two groups classified WhatsApp data as unimportant and less sensitive, whereas the other two categorised it as important and sensitive. When referring to photos, P5 noted:
“This is about family, private things. Like children’s photos.” P5 Group B
P8 stated that meme videos and images were more suitable under unimportant data:
“I would have these under entertainment, something like that. This is not important....” P8 Group C
Individual versus Group context. Comparing the sorting between the individuals and groups, the number of data types classified as important and sensitive increased in the group sorting more than during individual sorting. Furthermore, most data types classified as important but less sensitive during individual sorting ended up in the important and sensitive category in group settings. We posit that the differences in the results may be due to the fact that, in group settings, participants discussed different risks concerning each type of data, which may have influenced their choice. For instance, participants discussed different threats that could affect disputed data types or how such data can merely be misused (e.g., data being used to impersonate the owner).
“People can impersonate you, something like that. I can use [business contracts], I can claim to be you....” P16 Group D
These discussions may have impacted individual users’ perception of some data because all of the data types discussed in this manner were generally moved to the sensitive and important group. For instance, during the individual data sorting task, some group A participants classified WhatsApp backup data as not important and less sensitive. However, during group sorting, where the risk was discussed, participants agreed that such data should be classified as sensitive and important because WhatsApp data may contain personal and private information.
“Do you know they back up everything... WhatsApp backup would contain a lot of things I assume... messages, pictures, am not sure... maybe contacts. You don’t want people to know that... I would say private.” P18 Group A
We discuss the implications of this finding further in Section 8.
6 PAR2 – DELETION METAPHOR
Activity Design
To help elicit cloud deletion preferences, we developed a garbage collection metaphor. Using metaphors is a well-known HCI technique to help users think about digital objects as they would think about real-world objects to increase their familiarity with them [6, 10]. We also used our metaphor to minimize participants introducing their own metaphors to the study, which may lead to inconclusive results. We learned from our previous study [54] that users can possess incomplete or incorrect mental models of cloud deletion. This PAR was divided into two parts.
The first part of this activity focused on household waste management. Using an A3 sheet of paper, we presented a diagram of a house and five empty boxes depicting different ways of managing household waste. The five boxes had a picture of a fireplace, shredder, green bin, grey bin, and compost bin. Each box had a description of what the box represents, and a list of properties associated with the represented method of waste management. We then created eight labels representing household waste. These included the following: old bank statement, confidential letter, fizzy or soda can, newspaper, milk carton, rotten apples, candy wrappers, and old working computer keyboard. The task was for participants to place each type of waste in the appropriate box (bin). We chose the garbage metaphor because it somehow depicted deletion, that is, disposal of unwanted material with which most participants are familiar. Using this metaphor also helped us further build rapport. Figure 3(a) shows participants completing the metaphor task.
Fig. 3. Tasks: (a) Before completing the deletion preferences task, a metaphor task was used to help users appreciate cloud deletion — users exploring different ways of managing waste. (b) During the deletion preferences task, users classified how they would delete data from a shared folder.
In the second part, we tailored our metaphor and used its main concepts to design a deletion preference activity that required participants to sort out how they would delete cloud data. Rather than boxes depicting different bins, the boxes now depicted four different types of deletion and their properties.
We considered the four main types of deletion or deletion properties according to previous literature [53, 57] and named them for easy understanding.
Complete deletion — Deletion that removes all copies of data permanently from the cloud.
Soft deletion or partial deletion — Nominal deletion, in which some parts of the deleted data can still be recovered.
Camouflage deletion — Deletion that moves data from the users’ locality but not permanently removed and can always be recovered.
Trash can deletion — Deletion that allows recovery for some time before data is completely erased.
Not delete — This in when an individual does not want to delete.
For this task, we had 28 data labels (from the data sorting task) for participants to use for the deletion task. Each group was required to categorise how they would want such data to be deleted under two different conditions: (1) when deleting from a personal account and (2) deleting from shared folders.
PAR 2 – Findings
After open coding (explained in Section 4), we proceeded to identify themes and relationships between our data, especially those that explained users’ deletion preferences. We identified four themes that explained their desires and needs regarding deletion in the cloud. We further compared these preferences with the results from the data sorting task and found that deletion preferences (or needs) are not always aligned with how participants perceive data. Participants’ deletion preferences are multi-dimensional, different, and not consistent within individual and social contexts.
Comparing the deletion preferences with the data sorting activity, we found that individuals’ preferences are often not aligned with how they classify or perceive data. Individuals may desire a deletion type that may not necessarily be a conventional way to delete certain data types. For instance, data classified as personal or private may not necessarily be disposed of by a deletion type that destroys it completely and instantly to prevent it from falling into the wrong hands or being abused. Deletion preferences are not fixed to any particular group of data. Participants explained that data types could not be tied to a specific type of deletion because deletion needs (requirements) are different and constantly changing. For instance, Groups C and D preferred complete and permanent deletion when deleting data perceived as unimportant and less sensitive, arguing that such data is unnecessary and might not be needed in the future. However, in some cases, Group C did not opt for complete and permanent deletion for the same data. Table 4 shows the deletion preferences for deleting various data types from an individual cloud account for all of the groups.
| Data | Complete Deletion | Soft Deletion | Camouflage Deletion | Trashcan Deletion | Not to delete |
|---|---|---|---|---|---|
| Medical report/information | A, B, C | D | |||
| Rifle licence | A, B, C | D | |||
| Immigration documents | A, B | C | D | ||
| Personal information | A, B | C | D | ||
| Biometric data | A, B, D | C | |||
| Passport copy | A, B, C | D | |||
| Old bank statements | A, B, D | C | |||
| Business contracts | A, B | C | D | ||
| Music videos | C, D | A | B | ||
| Honeymoon photos | A | B | C, D | ||
| WhatsApp backup | A | B, C | D | ||
| Meme videos | C, D | A | B | ||
| Meme images | C, D | A | B | ||
| 3 GB wildlife video | B, C, D | A | |||
| Children photos | A | B | D | C | |
| OS installation file | C, D | A, B | |||
| Friends photos | A | B | D | C | |
| Facebook downloaded data | A, C, D | B | |||
| Old birthday video | A, B | C, D | |||
| Genetic information | A, B, C | D | |||
| Family photos | B | A, D | C | ||
| Application letter | B | A | C, D | ||
| Legal documents | A, B | C | D | ||
| E-book (PDFs) | C | A | B | D | |
| Pet care information | C, D | A | B | ||
| 4 MB video clip | C, D | A | B | ||
| Research information | A | B, C | D |
Table 4. Summary of Data Deletion Preferences for Deleting from an Individual Account by Each Group
Within individual contexts, participants’ deletion preferences vary and depend on various factors. During the preference tasks, participants expressed distinct deletion preferences. We found that participants’ deletion needs concerning a particular group of data or individual data types vary. We observed that these preferences were different even when the group had discussed and agreed on the risks or how to classify a particular data type; each group would argue for a different type of deletion. For instance, during the sorting task, all groups classified photos belonging to a friend as important but less sensitive. However, during the deletion preferences task, each group decided to have this type of data deleted differently.
“Friend’s photo can be very important. I say let’s keep it unless we have another copy.” P9 Group C
“Think about this, why would I have my mate’s private photo? I know it’s not private, but I am sure it would be something funny. I would delete it just so I can always get it back....” P10 Group D
During coding, we also found suggestions that deletion preferences within individual contexts change over time. Changes in participants’ lives or how they use the cloud may impact their deletion preferences. We found evidence that when cloud usage, file utility, social life, storage needs, and privacy needs changes, participants’ preferences may also change.
“I used to work as a photographer. I never deleted anything. Now, I just delete.” P4 Group A
“I would never completely delete [research data]. The university can delete it. We are not allowed to delete data before 10 years unless it’s special data.” P3 Group C
“It depends on the service provider; if they treat recycle bin as separate space then I can choose trash can. If not, I am completely deleting it.” P18 Group A
“We need to know why we are deleting first. We will always have different results depending on context. If today I have cancer, I want to keep that to myself, permanently delete. Tomorrow, I am well and I don’t care who knows about [it].” P2 Group B
We also found that deletion preferences are complex and multi-dimensional in nature; they depend on different factors, including the reason for deletion, file sensitivity, file utility, and size.
“How much will I pay to get this info; I will keep it [genetic info] if it is expensive, maybe... again, it never changes. If it’s cheap, I will permanently delete it.” P12 Group D
When the reason for deletion is to tidy the account (e.g., deleting memes and music videos that they do not consider important or sensitive), they prefer a quicker method (i.e., soft deletion), citing that completeness in such a case is not essential. Despite this reason, some participants stated that non-essential data might require complete deletion since that data may not be necessary for the future.
“This is permanent deletion, why would you keep this? Unless they are the videos you made. I will never need this.” P7 Group B
Concerning space, participants preferred soft deletion when the deleted files do not count towards the storage quota but complete deletion when they count.
“I didnt know deleted folder can count. Maybe that’s why I used to buy storage all the time.” P19 Group B
“There is no point to do trash can if it is not freeing your storage.” P18 Group A
“It’s immigration documents, where do I put them? If someone can recover them, you are in trouble. This is the cloud; I prefer to immediately remove it completely.” P13 Group B
“If applying for visa extension, am concerned that this [data] can be stolen but I need this readily available. But after I get my visa, I may want to delete [it] quickly.” P3 Group C
“I remember stories about DropBox. Deleted files came back... it shows they don’t delete. Permanent deletion for me, please.” P16 Group D
“Birthday videos are great. My mom would be mad at us for deleting them. I would only delete them if I have a copy somewhere. ” P7 Group B
“Its big, random. Why keep it? I would completely delete it.” P8 Group C
Participants highlighted that sometimes their choice of deletion might be influenced by the device they use to access the cloud. They argued that some interfaces provide better usability, influencing their deletion choice. For instance, Groups B and C argued that they would mostly want complete deletion if they are using a sync folder on a computer or a computer browser versus when using a mobile application. Some participants reasoned that it is easier and effortless to identify mistakes and recover files using web browsers or sync folders versus mobile applications.
“Have you ever tried to delete from your phone? iPhone used to be so small. I would permanently delete useless things which don’t matter. If it works [related], maybe not.” P20 Group C
Regarding social context or deleting from shared folders, we found that despite data classification groups, participants preferred soft deletion or not to delete files over complete deletion. We also found that, unlike in individual contexts in which preferences vary a lot, in a social context, preferences differ but are highly dependent on the situation. Participants’ choices mostly fluctuated between not deleting, camouflage deletion (i.e., always recoverable) and soft deletion. Table 5 shows the deletion preferences of deleting various data types from a family shared folder for all groups.
| Data | Complete Deletion | Soft Deletion | Camouflage Deletion | Trashcan Deletion | Not to delete |
|---|---|---|---|---|---|
| Medical report/information | A, B | C,D | |||
| Rifle licence | A, B, C | D | |||
| Immigration documents | B | A, C, D | |||
| Personal information | A, B | C, D | |||
| Biometric data | A, B, D | C | |||
| Passport copy | B | A, C, D | |||
| Old bank statements | A, B, C, D | ||||
| Business contracts | A, C | B, D | |||
| Music videos | C, D | A, B | |||
| Honeymoon photos | B | A, C, D | |||
| WhatsApp backup | A, B, C, D | ||||
| Meme videos | C, D | A, B | |||
| Meme images | C, D | A, B | |||
| 3 GB wildlife video | C, D | A, B | |||
| Children photos | A, B, C, D | ||||
| OS installation file | A, B, C, D | ||||
| Friends photos | A, B, C, D | ||||
| Facebook downloaded data | A, B, C, D | ||||
| Old birthday video | A, B, C, D | ||||
| Genetic information | A, B, D | C | |||
| Family photos | A, B, C, D | ||||
| Application letter | A, B | C, D | |||
| Legal documents | A, B | C, D | |||
| E-book (PDFs) | A, C, D | ||||
| Pet care information | A, B, C, D | ||||
| 4 MB video clip | C, D | A, B | |||
| Research information | A, B, C, D |
Table 5. Summary of Data Deletion Preferences for Deleting from a Family Shared Folder
Participants’ deletion preferences over shared folders also change over time. However, we found that their deletion preferences are influenced by the type of relationship between shared folder members, social life, the total number of users involved, authorship status, and perceived trust.
“My brother would permanently delete everything. He doesn’t like his old pictures. With family, I would say I want to recover every time.” P7 Group B
Regarding shared folders among work colleagues, participants preferred not to delete because some members of the group may still be using the data targeted for deletion. However, when all group members have authorized the deletion, they preferred complete deletion with no recovery.
“It happened at work recently. They deleted everything and our manager was not happy. We were lucky we could still recover them.” P14 Group C
“My group went to the lecturer and said I didn’t do anything. I changed groups and went into our shared folder and deleted everything. I would want permanent deletion for that.” P17 Group A
“If you delete something, maybe you should have everyone in the shared folder agree. Imagine twelve people. Two is easy. If there are too many people, you just delete to hide it from everyone. If they realize, you bring it back.” P18 Group A
“Communication is important. If everyone agrees, then why not [delete permanently]. It works well with few people.” P3 Group C
“If I uploaded it, I can easily delete it. If it’s permanent, I can always upload it again.” P17 Group A
“Our holiday folder is no delete folder. I would say no delete or deletion which I can recover.” P6 Group D
7 PAR3 – INFORMATION REQUIREMENTS
Activity Design
This activity focused on information about the deletion. We created 23 labels containing information related to the cloud and deletion. This information covered deletion concepts suggested to be important by prior research [44], such as time, shared folders, copies, backend, and user interface (UI). Table 6 contains the list of the information used in our study. Participants were asked to categorize these in three ways: (1) the order of importance with regards to deletion, (2) the point (time) when they would prefer to see the information, and (3) where they would expect or prefer to find that information (channel). However, unlike the data sorting activity (PAR1), for which categories were not given, in this task, categories were given to the participants beforehand. The categories were given to simplify and reduce task time. Despite being given categories, participants were encouraged to add more categories they considered applicable.
| Deletion Concepts | |
|---|---|
| Accountability | Who has access to deleted data in the cloud |
| Who has access to all data stored in the cloud | |
| Backend, Anonymisation | What happens to deleted data |
| Backend | How data is deleted |
| How data is stored in the cloud | |
| How much storage size is left | |
| The extent of data deletion (e.g., complete deletion, soft deletion) | |
| The location where data is stored | |
| The number of copies of data stored in the cloud | |
| What happens when a user deletes one’s cloud account | |
| How copies of data created by the provider are deleted | |
| Shared Folders | How data is deleted from a shared folder |
| How shared folders work | |
| In whose account data in shared folders resides | |
| How deletion from shared folders works (e.g., who has the right to delete) | |
| Time | The time it takes the provider to completely delete from the recycle bin |
| The time it takes to completely delete data from the cloud | |
| The time it takes for all copies of data to be deleted from the cloud | |
| The time it takes to completely delete a cloud account | |
| User Interface | How data is deleted from a web interface |
| How data is deleted from a “sync folder” or “cloud folder” in My Computer | |
| How data is deleted from the cloud using a smartphone | |
| Data Recovery | Data recovery after data has been deleted from ‘deleted folder’ or ‘trash can’ |
Table 6. Summary of Deletion Concepts Considered and Used in the Study
Regarding importance, we gave participants three categories: the most important information, less important information, and neutral group. Regarding Time, they could choose between Before signing up or deleting, During usage or deletion, and After deletion. Last, with regards to communication channels, participants could choose from Privacy Policies, Blog pages, Interactive Dialogs, Adverts, and Frequently Asked Questions (FAQs). We chose these channels because they are mostly used to distribute information about cloud services.
To ensure common ground for deletion concepts, the lead researcher allowed the participants to familiarise themselves with concepts and explained them to the participants, especially those that confused them. For example, some participants were not familiar with the “sync folder.”
PAR3 – Findings
We first present the results of PAR3 in Table 7. We show how participants categorized data with regards to importance, time, and channel of communication. Table 8 discusses and summarises these categorizations. We conclude this section by presenting the lessons learnt from this activity.
| Deletion Concept | Importance | Time | Channel |
|---|---|---|---|
| Accountability | Participants considered info about who has access to their deleted data critical. | They prefer to know this information before using the cloud. In some cases, they preferred this info after they have deleted their data. | They mostly expect this information to be in privacy policies. |
| Anonymisation Backend | Participants had a divided opinion over the importance of knowing what happens to deleted data. Some participants considered it critical while others said it was not. | They preferred having this knowledge before using the cloud or deleting from the cloud. | Participants expect to be able to have access to this info through privacy policies, blog pages, and interactive dialogs. |
| Backend | All of the groups perceived info about how data is stored not essential. However, they mostly perceive backend info important, if not critical, with regards to deletion. | They generally want to know this info before using the cloud or attempting to delete from it. Some argued that they would want to know much storage is left and the extent of deletion while deleting or after deleting. | They prefer this info to be shared through privacy policies, blog pages, and interactive dialogs. Participants stated they would expect info about the extent of deletion in privacy policies and interactive dialogs. Some participants believed that info about the deletion of copies of data should be made available in blog pages. They also suggested getting info about the amount of storage left and the number of copies of data existing in the cloud to be fully communicated in the account page or dashboard. One group stated that they would expect info about where data is stored to be made available in advertisements. |
| Shared folder | Participants considered info about shared folders to be critical, particularly, how data is deleted from shared folders and who can delete from a shared folder. One group argued that info about whose account shared data resided in was not critical to know compared with how data is deleted from shared folders. | They prefer getting information about deleting from shared folders before using the cloud or deleting from it. However, they highlighted that they would want to know who can delete from shared folders while using the cloud or when attempting to delete. | Participants preferred having information about shared folders in privacy policies, blog pages, and interactive pop-ups. For instance, they would expect to know who can delete through interactive dialogs when attempting to delete. Furthermore, they would also expect info about how data is deleted from shared folders to be distributed through FAQs and blog pages. |
| Time | Info about how long it takes to delete is considered at the very least important if not critical to know. Participants mostly considered knowing the duration of deleting data completely from the cloud critical. | Participants expressed that info concerning duration should mostly be made available before signing up to use the cloud. However, info about how long it will take to delete all copies of data from the cloud should be made available before and after deleting. | Participants mostly expected info about the duration to be found in privacy policies when it is about completely deleting an account and deleting a “sync” folder from their local machine. However, they mostly expect info about the duration to be in blog pages, interactive dialogs, and FAQs. They did not expect any of this info to be shared through their cloud dashboard/account pages or advertisements. |
| Recovery | Participants perceive info about whether they can recover data from the cloud to be critical. | They expect to know this info while using the cloud or when they are deleting data from the cloud. Some participants argued that this info is better known before signing up or attempting to delete. | Info about data recovery was mainly expected to be found in blogs and FAQs. |
Table 8. Summary of Deletion Information Preferences with Regards to Deletion Concepts
Following PAR3 and open coding (described in detail in Section 4), we learnt the following.
Information about deletion is less visible.
While deleting or when considering deletion, users prefer to have relevant information that will inform their decisions.
Presenting users with information at the right time will inform users’ decisions better.
The channel of communication is essential. Important information should be made available in different places.
Essential information should not be limited to privacy policies.
Information About Deletion Is Less Visible
Activity 3 revealed that most participants were unaware of what information about cloud deletion was available or where they could access it. For instance, during PAR3, many participants asked us whether all information used in the study was indeed available. They reported not having seen some of the information and expressed their desire to have access to such information. For example, all of the participants from one group debated whether cloud providers have information about data recovery after deleting from a “trash can” or “deleted item” folder on their website. They mostly agreed that this information is not available in the cloud. We also learned that most participants generally assume that all important information about deletion will be found in privacy policies.
“These are the things I would never think of; this would be important for the company, not me. Anyway, this is probably in the privacy policy of Dropbox.” P6 Group D
Users Prefer Precise and Relevant Information
Participants do not want to have all of the information about deletion through one channel of communication. They explained that having too much information in one place can be overwhelming and may cause people not to be interested. We learned that participants prefer to be exposed only to information critical for decision-making while using or deleting from the cloud. For instance, information about how data is stored in the cloud was deemed not important, but all groups highlighted that how deletion from shared folders works was very critical to know; thus, they would expect to have easy access to that information.
“People like you would love knowing this kind of things. I don’t really think its important to me. I don’t care how it is stored. Can I get it when I want it, yes... deletion maybe... sometimes you want to know what happens when you delete something.” P7 Group B
“For it’s like... can I recover this file. If yes, great. Google should get me to that information easily.” P15 Group D
“People should care about this and how to delete from shared folders. Everything down there is okay in my opinion.” P18 Group A
Presenting Information at the Right Time Informs Users’ Decisions
Participants prefer to have the information they consider critical or important presented to them before signing up for cloud services or before attempting to delete it. They preferred knowing information that concerns ‘how’ and ‘who’ handles data before they sign up for a service (e.g., information about who has access to all data stored in the cloud). Participants also highlighted that they need to be informed about the status of the system after they have deleted it. We also learned that most participants prefer knowing how long it will take for copies of data to be deleted after they have requested deletion.
“Yes, it may influence my decision to sign up. You don’t want to sign up, then later find that I cannot delete my data. But, again, I don’t really do this when I want to register [to a cloud service], it’s too much [information].” P18 Group A
“In Mac[OS] after you install something, they ask you if you want to keep a file, I always delete it. That’s cool because I need that information then.” P2 Group B
“You don’t want to delete and then be asked if you are sure the following day. I would panic. Information should just be there at the right time.” P6 Group D
Importance and Channel of Communication
We learned that participants generally prefer to have most of the information they consider critical in privacy policies,though, this is contextual. However, unlike our previous study [54], in which participants mentioned the desire to find some information in ads, in this study advertisements were least preferred for sharing information. We also found that information considered least important may also be preferred in privacy policies and interactive dialogs.
“This is the kind of information you find in privacy policies. Let’s put it under there. Imagine seeing this on your screen. It’s overwhelming....” P19 Group B
Beyond Privacy Policies
While privacy policies usually have information about deletion and participants expected most of the information to be in privacy policies, we learned that users do not always want to have all the information about deletion in the privacy policies (see Table 7). Our activity revealed that users occasionally prefer to have some information made available through channels such as the provider’s blog posts, account dashboard, ads, FAQs, and interactive dialogs. We learned that participants often expect technical information (i.e., information about how data deletion is achieved or completed) in blogs and FAQs. For example, all of the participants highlighted that information about how to delete using the web interface or mobile application should be made available in FAQs and blogs. We also learned that information about ‘who’ and ‘what’ is expected to be found in privacy policies while information about duration is expected to be instant and made available through interactive dialogs, blogs, and FAQs.
“I know we put this under privacy policies. But I don’t read them. Sometimes, you want this kind of information somewhere nice. A short blog explaining what happens to deleted data would be useful.” P13 Group B
8 DISCUSSION
Our study revealed various insights concerning cloud deletion and information preferences. In this section, we review our activities and discuss the implication of our findings regarding deletion mechanisms with respect to deleting in individual and social settings. We conclude this section by further informing policy around cloud deletion.
8.1 Data Sorting Activity
Our data sorting (or data perception) task showed that data sensitivity varies and depends on individuals’ and groups’ understanding of risk pertaining to the data targeted for deletion. This complements prior works [40, 60]. This finding further reestablishes that users’ perception (including evaluation of risk) of data is subjective, particularly narrow in individual settings versus groups. Somehow, this is expected; groups discuss risk better. However, it does highlight the possibilities of users’ underestimation of risk leading to data being deleted in a manner that leaves it vulnerable or causes damage.
The sorting task also suggests that data utility may be as valuable to users as sensitivity when choosing how data should be treated. This is not to say that there is a trade-off between the two; rather, it is considered important as sensitivity. From our study, data considered important tend to influence group composition or properties. Participants grouped these data types despite some having varied perceived sensitivity. It may be interesting to investigate how much data utility influences perceived sensitivity or whether the two are mutually exclusive.
Our findings also revealed that risk was not the only aspect that influenced sorting. Some groups discussed content and context; other groups touched on the element of trust and safety. For example, all groups classified honeymoon photos as “important and sensitive.” However, in some cases, participants mentioned that it was not that the data was “important” utility-wise but because they aimed to protect the dignity of what the photo might contain. Concerning children’s photos, some participants argued that while the photos themselves might be of value to them, they may not contain sensitive content. However, they are protecting the safety of children. Participants also discussed the element of trust during friends’ photos. All the groups agreed that friends’ photos were likely to contain common content between them and their friends. Therefore, they may be important only in terms of “value,” which may include an emotional attachment. Some participants mentioned that their friends would trust that they would keep such data safe.
During group sorting, we also learned that data types were more likely to be moved from other groups to the “important and sensitive” group than vice versa. While discussing such data, participants who had initially categorised data in a different way were more inclined to agree to have the data in the important and sensitive category than they initially thought. However, when discussing pet care information, participants who had initially classified this as sensitive or important data agreed to have it under unimportant and less sensitive. They mostly argued this information was more about the animal’s health than about them personally, though they know this information is linked to them. Prior studies [55, 71, 72] on pet wearables have found that pet owners do not always understand that pet wearable devices may also collect sensitive information about the pet owners.
8.2 Deletion Preferences - How do Users Want to Delete Cloud Data?
Users’ desire to delete data vary and is dependent on various factors. While existing literature [52, 69, 70] has mostly accepted that perceived high sensitivity drives the choice for the most destructible deletion method, our findings suggest that this is not held in some cases. In fact, deletion preferences do not entirely rely on how data is perceived or categorised. For instance, data perceived as not sensitive may warrant a destructible method of deletion. This suggests that the choice of deletion in the cloud goes beyond sensitivity; it is multi-dimensional and depends on the context of deletion. In individual contexts, deletion preferences are complex, diverse, and change over time as users’ lives change. In social contexts, they seem to be dependent on relationships, trust, creators, authorships, and their offline life situations. These differences suggest that users should be given choices regarding how they may delete data in the cloud.
8.3 Deletion Information - How Do They Want to be Informed About Deletion?
Our results suggest that users want to know more about deletion than how data is stored in the cloud. This reaffirms our earlier study’s findings that users are interested in knowing what happens to their data during deletion [54]. All of the groups in our study only suggested that how data is stored in the cloud was not critical to know, somehow highlighting that most of the information about deletion is essential. This discovery also supports the idea that users need to understand the deletion concepts better to make informed decisions. We also learned that most participants prefer knowing most information before deleting, sometimes even before signing up. This finding supports our argument that the user’s choice of a service provider may depend on how the provider deletes data or communicates about the deletion. Surprisingly, most participants preferred to have most of the information about deletion in privacy policies despite a surplus of literature stating that users do not read privacy policies. We posit that this may stem from users mixing where they expect to find information and where it should be placed. Nonetheless, our findings suggest that participants consider deletion information essential to contributing to how service providers should handle their data, or maybe participants searched for such information in privacy policies and could not find it. This may also suggest that users do not know what information about deletion exists in privacy policies. Therefore, they make assumptions about where it should be.
8.4 Deletion Controls - Individual Settings
Based on the results we obtained, we envision next-generation deletion controls with the following properties:
Intelligent and Personalized.
In comparison with sharing preferences [36, 46, 62], deletion preferences also depend on various factors: file attributes, relationships between owners, and mental models. We found that deletion preferences are not fixed; they change based on the context and time, suggesting that deletion needs or requirements cannot be generalized. There is no one-size-fits-all solution for meeting these preferences.
Our results also suggest that users sometimes prefer more than one type of deletion for particular data depending on the context. For instance, users may choose complete deletion or soft deletion for less essential and less sensitive data. This suggests that users do not only use one type of deletion for certain data but may use a different type depending on the context. Thus, the complete deletion may not only be used for privacy purposes but also for destroying data they deem unnecessary. Moreover, we found that participants do not always choose the destructive method of deletion for data that they consider private or confidential. We learned that they also consider how easy it is to get or reproduce the data. This suggests that deletion preferences are complicated and do not always reflect users’ privacy concerns.
These findings suggest that translating users’ deletion preferences into deletion controls will be challenging because the preferences are numerous, different, and dependent on many factors. There is a need for intelligent deletion controls that can adapt and accommodate different deletion preferences. Designers may have to employ AI techniques (such as machine learning) to implement smart deletion controls that can actively assist users in their deletion tasks—for instance, automatically clustering data types (as we observed people do themselves) with similar deletion preferences. Previous studies [7, 26] show some promising results regarding data classification concerning similarity, sensitivity, and usefulness. Cloud systems could automatically learn the deletion preferences for particular data on a per-user basis and establish suitable deletion defaults.
Interface-aware.
As people use different interfaces (i.e., web interfaces, mobile apps, and sync folders) to access the cloud depending on their need, sharing audience, cost, and accessibility, designers should account for these factors. This suggests that users may be willing to incur the cost of setting their deletion preferences or switching devices to delete easily. Our previous work [54] attests to this; users switch between devices to avoid facing deletion challenges. Therefore, designers should consider whether one interface can contain all of the necessary features required to offer deletion preferences or whether some features may be excluded. For example, to minimise effort when deleting through mobile devices, deletion mechanisms could be simplified to avoid complexity and cost. However, for web interfaces, users may be presented with more detailed controls.
Layered.
Our results suggest that the level of detail of deletion preferences differs across users; fine-grained controls may interest some users while others may find them too demanding. Therefore, we call on designers to develop deletion mechanisms that are layered — not only limited to fine-grained deletion choices but also coarse-grained choices for users who may find them too demanding. Fine-grained controls could focus on file properties and sharing context to account for costs and other use cases, such as associating a deletion type with a particular device. Coarse-grained deletion controls could include default settings that are easier to understand.
Retrospective.
Our results suggest that deletion preferences may change over time. This suggests that users may have specific requirements for deleting old data or data that have not been modified for over a long period. As stated by prior work [25], users have an interest in managing old data; therefore, deletion mechanisms should cater to the deletion of old data. Users could be given a chance to define when data should be considered old and specify how it should be deleted. For instance, users could be notified when data has become old and be requested to take action on how it should be deleted.
8.5 Deletion Controls - Social Contexts (Multi-party Deletion Mechanisms)
Multi-party issues are well understood in other domains, such as social media [56, 65, 66, 67], but are very limited in cloud computing [76]. In this section, we discuss how multi-party deletion in the cloud could be understood or improved.
Multiuser-aware.
We found that participants refrained from deleting files they did not create or author from shared folders. Therefore, designers should consider implementing deletion controls that take into account the multi-user nature of shared folders. For instance, members of a shared folder can agree beforehand which folders or files could be deleted. A file creator can specify whether others can delete a file before and during upload. This may reduce conflicts that may arise from deleting from shared folders or choosing the type of deletion (e.g., recoverable or not) to use.
Conflicts.
Deleting from shared folders may cause conflicts as deletion preferences can be different (or not align). The research could focus on understanding collaborative decision-making regarding deletion and what conflicts could arise from shared folders. Also, investigations could be conducted regarding what mechanisms could be implemented to reduce such conflicts. Current mechanisms allow uploaders to define privacy settings (including deletion) of the content, but there is no support for other members of the shared folder who may disagree or have different deletion preferences. Designers should also consider providing reactive approaches to enable users to deal with conflicts or reestablish deletion rules or preferences. An intriguing endeavor would include investigating how group decisions actually translate to the choice of deletion in practice.
Ownership.
Due to varying deletion requirements that arose from deleting from social contexts activity, we think ownership within shared folders may need to be defined. In a shared folder, one may assume that the uploader is the owner and, therefore, has the right to determine how an item should be deleted. However, there are cases in which an item may be considered co-owned by others, for example, photos. Understanding co-ownership is critical to design tools for managing deletion in the cloud.
Context-Aware Mechanisms.
This work and our previous work [54] did not investigate conflicts and consequences of deleting from social contexts. It is not clear whether these preferences include other people or whether they considered others before deleting. Moreover, if they did, it is not clear what they actually considered. Context-aware mechanisms could be used to help users consider others before executing their deletion preferences. Audience visualisation could be employed to help users understand who will be impacted by their deletion decisions. This can reduce the chances of accidental deletions (and conflicts) that may affect other people.
8.6 Deletion Information
Relevant Information at the Right Time.
A prior study [44] suggests that information about deletion should cover at least six areas: backend, time, backup, derived information, anonysization, and shared copies. While the authors of the prior study do not suggest how and where such information could be presented to users, our study suggests that this information should not only be constrained to privacy policies but should be made available through other channels as well. Our results also suggest that sharing these concepts at the right time may improve users’ understanding of deletion. For instance, explaining the retention period (i.e., time) immediately after users have deleted may help them realize that data is not entirely removed from the cloud. Similarly, users may also understand that the deletion process may sometimes include anonymisation before data is removed entirely from the cloud.
Our previous work [54] suggested that users seek help (i.e., look for information on deletion) in order to delete (i.e., accomplish a task). However, this study further confirms that users seek information to delete only and not to improve their knowledge about the deletion. They search for information about deletion only when they need it. Therefore, designers could categorize information about deletion into two groups: primary information and secondary information. Primary information could contain information that is essential for deletion and is needed by users to make confident decisions about deletion. Secondary information could focus on information that is not contextualised but may be necessary for users. Our results suggest that critical information should be easily accessible, particularly during deletion, while other information could be made available to users upon request or through other channels. Moreover, research should not only focus on improving privacy-related information on policies but should also investigate what deletion information is relevant for privacy policies.
Deletion Status and Summaries.
Our results highlighted the importance of giving users the status of their deletion action; users want to know whether deletion completed successfully and what it means with regards to deleted data. Designers could inform users about the effects of their deletion: file moved to “deleted items folder” or how long it will take to remove the file from the cloud completely. Moreover, deleted data could have a timer showing when it is going to be completely removed from the deleted item folder. This would help users understand retention better — which data is still recoverable and which one is not. For those who delete for privacy reasons, they would also get assurance that their data is removed from the cloud entirely.
Our results also suggest the need for deletion summaries. This feature would allow users to see their deletion records and help them audit their accounts, allowing them to reverse their decisions. For instance, through summaries, users may see which data has been deleted, how it was deleted and whether it is recoverable. Users could set how frequently they want to receive these reports or get them on demand.
Limitations
One disadvantage of participatory action research is domination. It is possible that one or two participants may have dominated the group and overruled others. This may have limited the ability of others to freely express their views. To mitigate this, the researcher conducting the fieldwork stepped in to the discussions and encouraged quiet group members to share their views.
This study was exploratory and mainly qualitative in nature. While the sample was diverse and roughly balanced across different demographic variables, such as gender, education, and employment, other variables such as age were varied but less balanced (18–45 years). We encourage additional studies with a larger and more diverse cloud user population, though getting older participants may be challenging as they tend to engage less with new technologies [47]. Moreover, while four groups with five participants are sufficient to create insightful discussions, it also limits the total range of experiences. Five people may have fewer experiences than a group of ten [45]. It is also possible that, in some cases, the minority opinion can be one that may lead to the opinion being discarded easily. We attempted to minimise this by having an odd number of participants per group to create a chance of having a minority opinion of more than one. The hypothesis that we raised over the preferences on deletion and information about deletion should be quantitatively tested in a subsequent confirmatory study.
Despite explaining all of the concepts to all participants during each session, it is possible that some concepts were misunderstood or where interpreted differently. Our prior study [54] showed that users have different understandings of the cloud and cloud deletion. We attempted in this study to create a shared understanding of all of the concepts we used. However, we still posit that some concepts may have been misunderstood or conflated, for example, camouflage deletion and soft deletion or dashboard/UI and pop-up dialogs. We also noticed that during PAR3, some groups ignored the dashboard/UI category. This category may have been ambiguous and confused the participants. We noticed this when analysing advertisements as well; in our prior study [54], participants expressed the lack of information about deletion on cloud storage advertisements. However, during this study, only one group suggested that some information should be in the advertisement. Future studies could clarify these discrepancies.
While participants expressed different preferences for deleting data from cloud-based storage, it is possible that, in practice, they may not engage in such deletion preferences. In addition, as our study revealed, these preferences may change over time (not stable), suggesting that the deletion behaviour or the choice of deletion may also change. Also, some participants mentioned that they would not delete some of the data we presented to them, suggesting that users may not delete their data despite the availability of deletion mechanisms/operations. This is highlighted by some studies in digital hoarding, for example, Sweeten et al. [68]. Future studies should explore the differences between the deletion preferences and the actual deletion behaviour.
Lastly, during some exercises, few participants showed signs that might have led to task exhaustion. To mitigate this and the negative impact this might have in the results—e.g., participants rushing through the tasks, we introduced breaks in between the tasks.
9 CONCLUSION
As technology continues to evolve, features such as the deletion of data should also change and become flexible to meet customers’ needs. This study lays the foundation for better cloud deletion controls and interfaces. We investigated cloud deletion preferences and the information that supports deletion in the cloud. We have shown that users have different deletion requirements—different data requires different types of deletion. Our results also show that deletion preferences differ significantly across people depending on their needs—there is no one size fits all solution. Controls should be intelligent and personalized, interface-aware, layered, retrospective, and multi-user aware. Our activities have shown how complex deletion preferences are highlighting the challenges of designing controls for deletion. Our results also provide useful insights on how information about deletion can be improved to inform users better when deleting. In particular, we show what information about deletion users consider important, where they want it to be made available, and at which point in time of their use of the cloud.
With our work, we hope to bring attention to and inspire positive change in the space of data deletion in the cloud, pushing towards a distinct approach and truly user-centered deletion mechanisms. A plethora of existing literature has established that users’ perception heavily influences their behaviour. Thus, it is essential to consider how users perceive various types of cloud deletion and their effects. This may help policymakers better understand how users perceive risk and how to ensure that users are protected from harms that may result from deletion. Our findings can also help service providers reflect on the type of information users view as important. Providers showing users what types of deletion methods are available may give users the impression that their providers understand their concerns and care about data deletion. There is also a need to have more deletion awareness campaigns—particularly regarding deletion in the cloud—covering its uses and consequences since increasingly more data is being stored in the cloud. Relevant agencies should use relevant deletion terms to help people understand that deletion methods are different and have different outcomes.
ACKNOWLEDGMENTS
We would like to thank all the participants who took part in the study, and the editor and reviewers who provided us with valuable feedback. This research was supported by the UK EPSRC REPHRAIN: Research centre on Privacy, Harm Reduction and Adversarial Influence online (EP/V011189/1).
Footnotes
1 After the Cambridge Analytica scandal [32], Facebook allowed its users to download all of the data that they have shared on the network. https://www.facebook.com/help/212802592074644.
Footnote
- [1] . 2015. Your location has been shared 5,398 times!: A field study on mobile app privacy nudging. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15), Seoul, Republic of Korea, April 18-23, 2015, , , , and (Eds.). ACM, New York, NY, 787–796.
DOI: Google ScholarDigital Library
- [2] . 2013. Tweets are forever: A large-scale quantitative analysis of deleted tweets. In Computer Supported Cooperative Work (CSCW’13), San Antonio, TX, February 23-27, 2013, , , , and (Eds.). ACM, London, United Kingdom, 897–908.
DOI: Google ScholarDigital Library
- [3] . 2019. Back to real pictures: A cross-generational understanding of users’ mental models of photo cloud storage. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1–24.Google Scholar
Digital Library
- [4] . 2004. A review of the literature on the benefits and drawbacks of participatory action research. First Peoples Child & Family Review 1, 1 (2004), 19–32.Google Scholar
Cross Ref
- [5] . 2012. Participatory research methods: A methodological approach in motion. Historical Social Research/Historische Sozialforschung Vol. 37 (2012), 191–222.Google Scholar
- [6] . 2006. The reification of metaphor as a design tool. ACM Transactions on Computer-Human Interaction 13, 4 (2006), 490–530.Google Scholar
Digital Library
- [7] . 2021. Files of a feather flock together? Measuring and modeling how users perceive file similarity in cloud storage. In SIGIR’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11–15, 2021, , , , , , and (Eds.). ACM, New York, NY, 787–797.
DOI: Google ScholarDigital Library
- [8] . 2014. File synchronization and sharing: User practices and challenges. Proceedings of the American Society for Information Science and Technology 51, 1 (2014), 1–10.Google Scholar
Cross Ref
- [9] . 2006. Business Research Methods. Vol. 9. McGraw-Hill, New York, New York.Google Scholar
- [10] . 2004. Human-computer Interaction. Pearson Education, London.Google Scholar
Digital Library
- [11] . 2005. Examining Internet privacy policies within the context of user privacy values. IEEE Transactions on Engineering Management 52, 2 (
May 2005), 227–237.DOI: Google ScholarCross Ref
- [12] . 2014. KnowMe and ShareMe: Understanding automatically discovered personality traits from social media and user sharing preferences. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 955–964.Google Scholar
Digital Library
- [13] . 2020. “It’s a scavenger hunt”: Usability of websites’ opt-out and data deletion choices. In CHI’20: CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, April 25-30, 2020, , , , , , , , , , , , and (Eds.). ACM, New York, NY,1–12.
DOI: Google ScholarDigital Library
- [14] . 2016. Deleting secret data with public verifiability. IEEE Transactions on Dependable and Secure Computing 13, 6 (2016), 617–629.
DOI: Google ScholarDigital Library
- [15] . Scan and Save Images of Your Passport and Prescriptions When Traveling. Life hacker. Retrieved June 22, 2022 from https://lifehacker.com/scan-and-save-images-of-your-passport-and-prescriptions-927527185Google Scholar
- [16] Antony Bryant and Kathy Charmaz (Eds.). 2007. The Sage handbook of grounded theory (1st ed.). SAGE Publications, 1 Oliver’s Yard, 55 City Road, London, EC1Y 1SP. Google Scholar
Cross Ref
- [17] . 2015. Face/Off: Preventing privacy leakage from photos in social networks. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, Denver, CO, October 12-16, 2015, , , and (Eds.). ACM, New York, NY, 781–792.
DOI: Google ScholarDigital Library
- [18] Backing up to Google Drive. Meta. Retrieved June 11, 2022 from https://faq.whatsapp.com/en/android/28000019/Google Scholar
- [19] Iulia Ion, Niharika Sachdeva, Ponnurangam Kumaraguru, and Srdjan Capkun. 2011. Home is safer than the cloud!: Privacy concerns for consumer cloud storage. In Symposium On Usable Privacy and Security, SOUPS’11, Pittsburgh, PA, USA - July 20-22, 2011, Lorrie Faith Cranor (Ed.). ACM, New York, NY, USA, 1–20. Google Scholar
Digital Library
- [20] . 2017. To permit or not to permit, that is the usability question: Crowdsourcing mobile apps’ privacy permission settings. Proceedings on Privacy Enhancing Technologies 2017, 4 (2017), 119–137.Google Scholar
Cross Ref
- [21] . 2015. Crowdsourced exploration of security configurations. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15), Seoul, Republic of Korea, April 18–23, 2015, , , , and (Eds.). ACM, New York, NY, 467–476.
DOI: Google ScholarDigital Library
- [22] . 2012. Facebook and privacy: It’s complicated. In Symposium on Usable Privacy and Security (SOUPS’12), Washington, DC, July 11–13, 2012, (Ed.). ACM, New York, NY, 9.
DOI: Google ScholarDigital Library
- [23] . 2020. The dilemma of user engagement in privacy notices: Effects of interaction modes and habituation on user attention. ACM Transactions on Privacy and Security 23, 1 (2020), 1–38.Google Scholar
Digital Library
- [24] . 2009. A “nutrition label” for privacy. In Proceedings of the 5th Symposium on Usable Privacy and Security (SOUPS 09), Mountain View, CA, July 15–17, 2009 (ACM International Conference Proceeding Series), (Ed.). ACM, New York, NY, 12.
DOI: Google ScholarDigital Library
- [25] . 2018. Forgotten but not gone: Identifying the need for longitudinal data management in cloud storage. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI’18). ACM, New York, NY, Article
543 , 12 pages.DOI: Google ScholarDigital Library
- [26] . 2021. Helping users automatically find and manage sensitive, expendable files in cloud storage. In 30th USENIX Security Symposium, USENIX Security 2021, August 11–13, 2021, and (Eds.). USENIX Association, Berkeley, CA, 1145–1162. https://www.usenix.org/conference/usenixsecurity21/presentation/khan-mohammad.Google Scholar
- [27] . 2012. Protecting the privacy and security of sensitive customer data in the cloud. Computer Law & Security Review 28, 3 (2012), 308-319.
DOI: Google ScholarCross Ref
- [28] . 2016. Follow my recommendations: A personalized privacy assistant for mobile app permissions. In 12th Symposium on Usable Privacy and Security (SOUPS’16), Denver, CO, June 22–24, 2016. USENIX Association, Berkeley, CA, 27–41. https://www.usenix.org/conference/soups2016/technical-sessions/presentation/liu.Google Scholar
- [29] . 2016. When privacy meets usability: Unobtrusive privacy permission recommendation system for mobile apps based on crowdsourcing. IEEE Transactions on Services Computing 11 (2016), 864–878.Google Scholar
- [30] . 2011. Analyzing Facebook privacy settings: User expectations vs. reality. In Proceedings of the 2011 ACM SIGCOMM Conference on Internet Measurement Conference. ACM, New York, NY, 61–70.Google Scholar
Digital Library
- [31] Luc Pauwels and Dawn Mannay (Eds.). 2020. The SAGE Handbook of Visual Research Methods (1st ed.). SAGE Publications, 1 Oliver’s Yard, 55 City Road, London, EC1Y 1SP. Google Scholar
Cross Ref
- [32] . 2019. Facebook understood how dangerous the Trump-linked data firm Cambridge Analytica could be much earlier than it previously said. Retrieved June 16, 2022 from https://www.businessinsider.com/cambridge-analytica-a-guide-to-the-trump-linked-data-firm-that-harvested-50- million-facebook-profiles-2018-3?r=US&IR=T##what-did-cambridge-analytica-do-1.Google Scholar
- [33] . 2012. That syncing feeling: Early user experiences with the cloud. In Designing Interactive Systems Conference 2012 (DIS’12), Newcastle Upon Tyne, United Kingdom, June 11–15, 2012. ACM, New York, NY, 544–553.
DOI: Google ScholarDigital Library
- [34] . 2015. If you are happy and you know it, say “I’m here”: Investigating parents’ location-sharing preferences. In Proceedings of Human-Computer Interaction—INTERACT 2015—15th IFIP TC 13 International Conference, Bamberg, Germany, September 14–18, 2015, Part III (Lecture Notes in Computer Science), , , , , , and (Eds.), Vol. 9298. Springer, Bamberg, Germany, 315–332.
DOI: Google ScholarDigital Library
- [35] . 2017. Stories from survivors: Privacy & security practices when coping with intimate partner abuse. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, May 06-11, 2017, , , , , , , and (Eds.). ACM, New York, NY, USA, 2189–2201.
DOI: Google ScholarDigital Library
- [36] . 2010. Access control for home data sharing: Attitudes, needs and practices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 645–654.Google Scholar
Digital Library
- [37] . 2008. The cost of reading privacy policies. ISJLP 4 (2008), 543.Google Scholar
- [38] . 2016. (Do Not) track me sometimes: Users’ contextual preferences for web tracking. Proceedings on Privacy Enhancing Technologies 2016, 2 (2016), 135–154.Google Scholar
Cross Ref
- [39] . 2017. User interactions and permission use on Android. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, 362–373.
DOI: Google ScholarDigital Library
- [40] . 2017. Information sensitivity typology: Mapping the degree and type of risk consumers perceive in personal data sharing. Journal of Consumer Affairs 51, 1 (2017), 133–161.Google Scholar
Cross Ref
- [41] . 2016. How socially aware are social media privacy controls? Computer 49, 3 (2016), 96–99.Google Scholar
Digital Library
- [42] . 2017. REACT: REcommending access control decisions to social media users. In Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017, Sydney, Australia, July 31 - August 03, 2017, , , and (Eds.). ACM, New York, NY, 421–426.
DOI: Google ScholarDigital Library
- [43] . 2014. Two-party fine-grained assured deletion of outsourced data in cloud systems. In IEEE 34th International Conference on Distributed Computing Systems (ICDCS’14), Madrid, Spain, June 30–July 3, 2014. IEEE, New York, NY, 308–317.
DOI: Google ScholarDigital Library
- [44] . 2018. “If I press delete, it’s gone” —user understanding of online data deletion and expiration. In 14th Symposium on Usable Privacy and Security (SOUPS’18). USENIX Association, Baltimore, MD, 329–339. https://www.usenix.org/conference/soups2018/presentation/murillo.Google Scholar
- [45] . 2017. Conducting Focus Groups for Business and Management Students. SAGE, London UK.Google Scholar
- [46] . 2005. A study of preferences for sharing and privacy. In Extended Abstracts Proceedings of the 2005 Conference on Human Factors in Computing Systems (CHI 2005), Portland, Oregon, USA, April 2-7, 2005, and (Eds.). ACM, New York, NY, 1985–1988.
DOI: Google ScholarDigital Library
- [47] . 2011. Diffusion of technology: Frequency of use for younger and older adults. Ageing International 36, 1 (
01 Mar 2011), 123–145.DOI: Google ScholarCross Ref
- [48] . 2003. Reflections on participatory research. Area 35, 1 (2003), 46–54.Google Scholar
Cross Ref
- [49] . 2019. Usability smells: An analysis of developers’ struggle with crypto libraries. In 15th Symposium on Usable Privacy and Security (SOUPS’19), Santa Clara, CA, August 11–13, 2019, (Ed.). USENIX Association, 245–257. https://www.usenix.org/conference/soups2019/presentation/patnaik.Google Scholar
- [50] . 2007. What’s wrong with online privacy policies? Communications of the ACM 50, 9 (2007), 103–108.Google Scholar
Digital Library
- [51] . 2009. Yours, mine and (not) ours: Social influences on group information repositories. In Proceedings of the 27th International Conference on Human Factors in Computing Systems (CHI’09), Boston, MA, April 4–9, 2009. ACM, New York, NY, 2095–2098.
DOI: Google ScholarDigital Library
- [52] . 2011. A secure cloud backup system with assured deletion and version control. In 2011 International Conference on Parallel Processing Workshops (ICPPW’11), Taipei, Taiwan, September 13–16, 2011, and (Eds.). IEEE, New York, NY, 160–167.
DOI: Google ScholarDigital Library
- [53] . 2016. Assured deletion in the cloud: Requirements, challenges and future directions. In Proceedings of the 2016 ACM on Cloud Computing Security Workshop CCSW’16), Vienna, Austria, October 28, 2016, , , , , , and (Eds.). ACM, New York, NY, 97–108.
DOI: Google ScholarDigital Library
- [54] . 2017. “I feel stupid I can’t delete...”: A study of users’ cloud deletion practices and coping strategies. In 13th Symposium on Usable Privacy and Security (SOUPS’17), Santa Clara, CA, July 12–14, 2017. USENIX Association, 241–256. https://www.usenix.org/conference/soups2017/technical-sessions/presentation/ramokapane.Google Scholar
- [55] . 2019. Does my dog really need a gadget?: What can we learn from pet owners’ amotivations for using pet wearables?. In ACI’19: 6th International Conference on Animal-Computer Interaction, Haifa, Israel, November 12–14, 2019. ACM, New York, NY, 6:1–6:6.
DOI: Google ScholarDigital Library
- [56] . 2018. “You don’t want to be the next meme”: College students’ workarounds to manage privacy in the era of pervasive photography. In 14th Symposium on Usable Privacy and Security (SOUPS’18), Baltimore, MD, August 12–14, 2018, and (Eds.). USENIX Association, 143–157. https://www.usenix.org/conference/soups2018/presentation/rashidiGoogle Scholar
- [57] . 2013. SoK: Secure data deletion. In 2013 IEEE Symposium on Security and Privacy (SP’13), Berkeley, CA, May 19–22, 2013. IEEE,New York, NY, 301–315.
DOI: Google ScholarDigital Library
- [58] . 2017. Designing effective privacy notices and controls. IEEE Internet Computing 21, 3 (
May 2017), 70–77.DOI: Google ScholarDigital Library
- [59] . 2020. Exploring user perceptions of deletion in mobile instant messaging applications. Journal of Cybersecurity 6, 1 (2020), tyz016.
DOI: Google ScholarCross Ref
- [60] . 2019. Internet users’ perceptions of information sensitivity–insights from Germany. International Journal of Information Management 46 (2019), 142–150.Google Scholar
Digital Library
- [61] . 2013. “I read my Twitter the next morning and was astonished”: A conversational perspective on Twitter regrets. In 2013 ACM SIGCHI Conference on Human Factors in Computing Systems (CHI’13), Paris, France, April 27–May 2, 2013, , , and (Eds.). ACM, New York, NY, 3277–3286.
DOI: Google ScholarDigital Library
- [62] . 2016. Sharing personal content online: Exploring channel choice and multi-channel behaviors. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, CA, May 7-12, 2016, , , , , and (Eds.). ACM, New York, NY, 101–112.
DOI: Google ScholarDigital Library
- [63] . 2014. Beyond notice and choice: Privacy, norms, and consent. Journal of High Technology Law 14 (2014), 370.Google Scholar
- [64] . Is it Safe to Store Personal IDs Like Scanned Copies of Passport, on Google Drive? Quora. Retrieved June 11, 2022 from https://www.quora.com/Is-it-safe-to-store-personal-IDs-like-scanned-copies-of-passport-on-Google-drive.Google Scholar
- [65] . 2016. Resolving multi-party privacy conflicts in social media. IEEE Transactions on Knowledge and Data Engineering 28, 7 (2016), 1851–1863.
DOI: Google ScholarCross Ref
- [66] . 2018. Multiparty privacy in social media. Communications of the ACM 61, 8 (
July 2018), 74–81.DOI: Google ScholarDigital Library
- [67] . 2017. Photo privacy conflicts in social media: A large-scale empirical study. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, 3821–3832.
DOI: Google ScholarDigital Library
- [68] . 2018. Digital hoarding behaviours: Underlying motivations and potential negative consequences. Computers in Human Behavior 85 (2018), 54–60.Google Scholar
Cross Ref
- [69] . 2010. FADE: Secure overlay cloud storage with file assured deletion. In Proceedings of Security and Privacy in Communication Networks — 6th International ICST Conference (SecureComm’10), Singapore, September 7–9, 2010 (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering), and (Eds.), Vol. 50. Springer, Berlin, 380–397.
DOI: Google ScholarCross Ref
- [70] . 2012. Secure overlay cloud storage with access control and assured deletion. IEEE Transactions on Dependable and Secure Computing 9, 6 (2012), 903–916.
DOI: Google ScholarDigital Library
- [71] . 2020. Pets without PETs: On pet owners’ under-estimation of privacy concerns in pet wearables. Proceedings on Privacy Enhancing Technologies 2020, 1 (2020), 143–164.Google Scholar
Cross Ref
- [72] . 2019. Buddy’s wearable is not your buddy: Privacy implications of pet wearables. IEEE Security & Privacy 17, 3 (2019), 28–39.Google Scholar
Cross Ref
- [73] . 2018. Hoarding and minimalism: Tendencies in digital data preservation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI’18), Montreal, QC, Canada, April 21–26, 2018, , , , and (Eds.). ACM, New York, NY, 587.
DOI: Google ScholarDigital Library
- [74] . 2019. Keeping and discarding personal data: Exploring a design space. In Proceedings of the 2019 Designing Interactive Systems Conference (DIS’19), San Diego, CA, June 23–28, 2019, , , , and (Eds.). ACM, New York, NY, 1463–1477.
DOI: Google ScholarDigital Library
- [75] . 2013. Turbulence in the clouds: Challenges of cloud-based information work. In 2013 ACM SIGCHI Conference on Human Factors in Computing Systems (CHI’13), Paris, France, April 27 – May 2, 2013, , , and (Eds.). ACM, New York, NY, 2273–2282.
DOI: Google ScholarDigital Library
- [76] . 2020. Cloudy with a chance of misconceptions: Exploring users’ perceptions and expectations of security and privacy in cloud office suites. In 16th Symposium on Usable Privacy and Security (SOUPS’20). USENIX Association, Berkeley, CA, 359–377. https://www.usenix.org/conference/soups2020/presentation/wermke.Google Scholar
- [77] . 2018. Shopping as a social activity: Understanding people’s categorical item sharing preferences on social networks. In Joint Proceedings of the ACM IUI 2018 Workshops Co-located with the 23rd ACM Conference on Intelligent User Interfaces (ACM IUI’18), Tokyo, Japan, March 11, 2018 (CEUR Workshop Proceedings), and (Eds.), Vol. 2068. CEUR-WS.org, New York, NY, 12. http://ceur-ws.org/Vol-2068/humanize4.pdf.Google Scholar
Index Terms
What Users Want From Cloud Deletion and the Information They Need: A Participatory Action Study
Recommendations
Cloud Storage as the Infrastructure of Cloud Computing
ICICCI '10: Proceedings of the 2010 International Conference on Intelligent Computing and Cognitive InformaticsAs an emerging technology and business paradigm, Cloud Computing has taken commercial computing by storm. Cloud computing platforms provide easy access to a company’s high-performance computing and storage infrastructure through web services. With cloud ...
Design and Implementation of HPC-SA in OpenStack Cloud Platform
CIMSIM '15: Proceedings of the 2015 Seventh International Conference on Computational Intelligence, Modelling and SimulationPrivate cloud application with on-premise model offers benefits for security, reliability and high performance. However, not all SMBs and enterprises have enough resources to host an appliance or service. Private cloud application with off- premise ...
Cloud Computing Security: From Single to Multi-clouds
HICSS '12: Proceedings of the 2012 45th Hawaii International Conference on System SciencesThe use of cloud computing has increased rapidly in many organizations. Cloud computing provides many benefits in terms of low cost and accessibility of data. Ensuring the security of cloud computing is a major factor in the cloud computing environment, ...











Comments