Human-Swarm Interaction in Semi-voluntary Search and Rescue Operations Opportunities and Challenges

In search and rescue (SAR) operations, drones can provide clear and timely situational overview data and object identification. However, the current one-to-one relationship between operators and drones limits scalability. Swarm solutions have been proposed to overcome this limitation, but there are few examples of control concepts for SAR operations. Human-swarm interaction (HSI) presents new challenges in terms of task design, control loops, communication, and managing uncertainty. We present an exploratory study of integrating drone swarms into SAR organizations, with a focus on challenges and opportunities for HSI. Our findings highlight the need for a holistic approach to drone swarm systems design, development, and integration. Careful system and task design is vital to reduce operator workload, maximize situational awareness, and maintaining effective communication among SAR team members. Building trust through technology exposure and training is also important. We identify several key research avenues, including adaptive and intelligent swarm control mechanisms, trust dynamics between operators and swarms, participatory design work, legal and operational frameworks, and the organizational impact of drone swarm integration. Overall, this paper contributes to HSI and SAR research by addressing research gaps concerning the integration effects and constraints of drone swarms in current work settings. The study highlights the potential for implementing drone swarms in semi-voluntary SAR organizations, while emphasizing the importance of considering the tasks and interactions between humans and drones when assessing overall system performance.


INTRODUCTION
Swarming, as a technology, enables multiple autonomous agents to coordinate their actions.As a behavior, swarming emerges from the interactions between agents following a set of rules that shapes the local interactions between agents and objects in their environment [13].Multiple application areas for swarms of Unmanned Aerial Vehicles (UAVs or drones) have been proposed, including forest firefighting [2], generating maps for first responders [3], assisting in reducing emergency response times [7], and search and rescue operations [9].
In search and rescue (SAR) operations, search teams are often comprised of volunteer recruits managed by a more stable command structure.This poses a challenge in remote and sparsely populated mission areas [17].Drones are also deployed as they can explore areas that are otherwise inaccessible or dangerous to humans [16,26].By utilizing various sensors (e.g., optical or infrared) and computer vision systems, drones can provide clear and timely situational overview data and object identification [16,18].Although additional drones could be desirable for these purposes, the current one-to-one relationship between operators and drones limits scalability.Swarm solutions have been proposed to overcome this but there are few examples of control concepts for SAR operations [16,18].Meshcheryakov and colleagues [17] presents a swarm implementation for SAR use but only briefly discusses the work processes of the operator or how the swarm fits into the overall SAR organization.Additional challenges for swarm deployment in time-critical SAR operations includes creating shared mental models between autonomous agents and humans [8] and the increased data quantity from multiple video streams [16].In more general terms, human-swarm interaction (HSI) presents new challenges in terms of task design, control loops, communication, and managing uncertainty [13].The stochastic and non-intuitive behavior of swarms can lead to increased mental workload and severity of errors [10,11].However, increased drone (and swarm) autonomy also has the potential to reduce an operator's workload and increase their situational awareness by shifting cognitive resources from flight control to task supervision and management [10].Generally, a rise in drone numbers improves human-swarm team performance to a point, but eventually is subject to the law of decreasing marginal returns.Therefore it is important to consider the tasks and interactions between humans and drones when assessing overall system performance [11].
Trust is another human factors challenge associated with supervisory control and human-autonomy teaming [5,11].Uncertainty and vulnerability are central aspects of trust.In teams, actors rely on others to perform according to their respective roles and responsibilities to support their common goal [14,21].Having trust in the independent operational capability and competence of one's teammates is important in complex team endeavors.Trust in automation can vary between individuals and within organizations, and be affective or analytical in nature, both of which are important for automation acceptance.Environmental variability, automation transparency, and operator training are also contributing factors.In addition, actors must feel that they trust the automation and be able to reason about the possibilities and limitations of the technology [14].Trust is a mediating factor between drone swarm reliability and operator reliance, where increased operator reliance-through trust-can contribute to increased performance (and complacency) but is not enough for proper system integration and use [11,12].
Introduction of new technology can affect the perceived control of coordinators [25] and experienced teams are generally more able to control new technology [6].In a command and control (C2) setting, a commander can exercise control to influence or coordinate team activities.Decision-making skills, situational awareness, information flow, and available technology can affect how control is applied in-and to-a system [22].Many command systems utilize a remote control room that relies on effective verbal communication with field agents.The probability for success is increased if mission command and field agents establish a shared situational awareness and understanding about tactical plans, operational instructions, and events in the field [22].In another C2 paradigm, agile organizations can be described as edge organizations where tactical and operational decisions are made by people closer to (or in) the field.Information is more readily available on a peer-to-peer basis rather than through a centralized command function, and other constraints associated with centralized control are eliminated [1].
As indicated above, there is a lack of research on how drone swarms can be integrated into the operations and activities of SAR organizations.This paper present the design, results, and implications of an initial and exploratory workshop study into the integration requirements for drone swarms in SAR organizations.Three research questions structured our work: (1) How can SAR organizations structure their field operations?(2) How are drones a part of that process?(3) What are the potential challenges and opportunities in deploying autonomous drone swarms in semi-voluntary SAR organizations?
In the following sections, we present the study design and procedure, data analysis and results, discuss the implications of our results, and suggest future research opportunities.

METHOD
Organizations and work processes are impacted by technological advances.Consequences of change are hard to anticipate but necessary to scrutinize [27].This evokes the envisioned world problem [27]: how can results from studies in the current field inform future stages, since the technology will change the current field?By using artifact-based methods and letting practitioners explore the potential of the new technology, this issue can be alleviated.Therefore, to study future (i.e., non-existing) systems, one research approach is to include real-world challenges, recruit knowledgeable participants, and use real-world incidents as a starting point.In workshops, researchers rely on recordings and note taking [28].
Workshops are suitable for ill-defined and prospective research areas, combining the firsthand information of observations and reflective insights of interviews.In a workshop study with firefighters, Bjurling et al. [2] explored how drone swarms could be used in a forest firefighting work context.The workshops were each conducted in two phases.The first phase was focused on describing current work processes, needs, and motivations.The second phase focused on work as envisioned, where participants were asked how they might deploy a drone swarm to solve the same task.Based on this approach, the current study conducted two workshops with SAR domain experts to discuss the potential implications of drone swarms in their organization.

Participants
Three participants (two male, one female) from a Swedish semivoluntary SAR organization were recruited for the study.The participants were 30, 53, and 55 years of age and had approximately five years of experience in their respective roles.One participant was a drone pilot, one was a dog handler, and the last participant an expert in the organization's mission command function.The participants were recruited through personal network contacts.Informed consent was obtained from the participants prior to data collection.
No monetary compensation was offered to the participants.

Materials
The workshops were conducted remotely via Zoom.The online whiteboard tool Mural was used to enable collaboration between the participants in the Zoom call session.On Mural, different roles were represented by different icons, see Figure 1.There was also a map retrieved from Open Street Map.The participants received additional information about the study and filled out a consent form.The workshop leader had an interview guide with questions based on the analysis method and specific SAR questions covering work tasks, procedures, and tools.

Procedure
The study came in three parts.The first part was an initial interview with one of the participants where the workshop leader got to understand the fundamentals of the organization.The collected information was later used to prepare for the workshop.The second part was a short pilot study with two university students where the procedure and technical aspects of the workshop were tested.
No major changes were made to the procedure following the pilot.The third part was the primary workshops.The first workshop followed the procedure outline described by Bjurling et al. [2] by dividing the workshop into two phases.The first phase focused on the organization's work as currently performed.The discussion addressed the roles within the organization and the different goals they pursue during their missions.This first phase took 90 minutes, followed by a short break.Before the second phase began, the workshop leader introduced the concept of drone swarms by explaining their general structure, behavior, and capabilities.Next, the second phase focused on how the participants envisioned drone swarms, if available, could be integrated in their activities.The second phase of the workshop took 60 minutes, adding up to a total of 150 minutes.The workshop was recorded.Detailed fieldnotes were also taken and were complemented after video reviews.
The second workshop was scheduled to collect additional information about the mission commander's C2 strategies for the search teams, and to discuss initial analysis results from the first workshop.The second workshop included two of the initial three participants from the first workshop, specifically the mission commander and drone pilot.Informed consent was obtained again.The workshop was conducted as a single discussion session focused on specific topics from the initial analysis.The online Mural was again used to allow participants to externalize their thoughts.The workshop lasted for approximately 70 minutes.

Analysis
Cognitive Work Analysis (CWA) was selected to analyze the workshop data.When designing a new system, one must consider the resources available and their capabilities and limitations [19].Using CWA, researchers and designers can identify roles and functions for actors and system capabilities.CWA can also identify constraints imposed by the environment on operators and their actions [24].CWA also supports a formative design approach by enabling the identification of requirements that need to be satisfied for an artefact to be well-integrated in a sociotechnical system.This is typically done in multiple analytical steps.
For this study, the selected CWA analysis steps were: Work Domain Analysis; Activity Analysis in Work Domain Terms; and Activity Analysis in Decision Making terms.This study applied the analytical frameworks suggested by Stanton and Jenkins [23].This included the abstraction hierarchy for work domain analysis, and the contextual activity template and decision tree for the activity analyses.To identify challenges and opportunities regarding the introduction of swarms in the system, inspiration was taken from Read, Salmon, and Lenné [20] where the CWA Design Toolkit is introduced.CWA can be used to generate design ideas for novel systems [15,20].In the first steps of this process, one part is identifying pain points of the users.Fieldnotes from the second phase of the first workshop and the full second workshop were combined and analyzed for this study utilizing the CWA processes previously mentioned.A CWA software tool [4] was used to facilitate the analysis.

RESULTS
Here, we present the results of the CWA-the work domain analysis and the two activity analyses-and our findings pertaining to swarm constraints in semi-voluntary SAR organizations.

Work Domain Analysis
For this analysis we present each level of the abstraction hierarchy in turn.
Functional purpose.One specific and central functional purpose was identified for the analyzed system; to cover the identified area of interest and thus increasing the possibility of finding the missing person.Values and priority measures.Seven values and priority measures were identified.The first was for the searchers to deliver a probability of detection (POD) for the area they most recently searched through.This measure can be summarized as the probability that a missing person would be found from a search of the area.The measure is an estimate presented in percentage points, ranging from zero to 100 (higher is better).The other measures and assessments were: sector completion; group coordination; staying in directed area; search progress; determined circumstances; and regulatory compliance.Purpose-related functions.Nine purpose-related functions were identified.The first was for the patrol leaders to maintain searcher formation.The remaining functions were: deliver report of searched area; communicate between the different groups; coordination of the search operation; go out in field; plan operation; assess searcher report credibility; identify the last position of the lost person; and determine wind information.Object-related processes.17 object-related processes were identified.These were: following the searchers in patrol; walk in a specific formation (called serpentine) with the dogs; collect wind information; fly with the drone; check drone video feed; draw drone search path; searchers requesting permission go off script; confirming searchers' whereabouts; communicate with different searchers; communicate with police; define search areas; define search sectors (within areas); decide upon search teams; send search teams on patrol; identify objects of interest; calling searchers; and return to command center.Physical objects.13 physical objects were identified as the most important: searchers; dogs; drones; SD-cards; search programs; physical maps; tracking apps; telephones; information about the missing person; Google Maps; education for searchers on how to perform a search; location probability statistics; and compasses.

Activity Analyses
In work domain terms, control tasks were the object-related processes (presented in the previous section) mapped onto six situations extracted from the SAR mission description.These situations included the initial team setup, starting an initial search, and arrival and setup of new personnel.Searching out in the field, reporting back to the command center, and finding the person of interest were the other three.The object-related processes were relevant to varying degrees for each of these situations.For brevity, we will not present the mappings in full here, however the contextual activity template is presented in Figure 2 of the appendix.
In decision making terms, the activity analysis was made from the drone pilot's perspective.The process is initiated by the alert and getting to the search area.The pilot's positioning is based on several information factors, including wind, physical obstacles, pattern creating possibilities, drone battery, and missing person information.The system state addresses, for instance, whether the pilot can create an accurate representation of the searched area or fly in this area given the drone's battery levels.The drone pilot's options includes, e.g., considering whether it is possible to physically fly in this area or dividing the search area into smaller sections.The goal is chosen from the alternatives a) create grid-system, b) perform tight search, or c) maintain altitude.The target state thus considers whether the drone pilot should fly in a particular area, fly in a certain direction, or divide the searching into smaller bits.The pilot must also consider whether the selected option allows them to maintain eye contact with the drone.They then ask themselves "How do I fly here?How can I avoid draining battery?How can I maintain altitude?"

Swarm Constraints
Three object-related processes (actions) were deemed most relevant to the introduction of drone swarms: fly with the drone, confirming searchers' whereabouts, and check drone video feed.Additionally, the data highlighted two major constraints: the need for human judgment, and ability to control the swarms.The comments relevant to flying with the drones, for example, concerned the need for frameworks for how to fly, and emphasized the need to maintain a line-of-sight and see what the drones are doing.Currently, in Sweden, there must always be a legally responsible drone pilot in control of each drone, maintaining visual eye contact with their drone.The drones also must "report back" to the pilot and it must be clear where the drones have flown.There are also external constraints that must be considered, like wind and drone battery, that affects the possibility to fly with the drones.The comments related to confirming searchers' whereabouts mostly concerned the need for human control and human judgement.Comment about the distance and maintaining visual line-of-sight are relevant to this action, as well as letting individual drones fly away from the swarm.
In general, participants found it difficult to trust machines when it came to handling human lives.This, in turn, required humans to deliver the POD estimation.Similar observations can be made regarding the comments relevant to checking drone video feeds.For example, there were several mentions about the need to be able to see both individual and multiple video feeds.There were also concerns about the time factor, where, on the one hand, additional drones could collect more data faster but, on the other hand, too much data could lead to information and communication bottlenecks.This can result in the risk of seeing objects of interest hours later.Furthermore, besides the initial area setup, drone swarms would affect all phases of the SAR mission.The three most relevant actions all take place during the search.Swarms also affects the identification of an object of interest because it is difficult to trust the drones autonomously making judgements about the relevance of potential objects or people of interest.
Concerning how a drone swarm could affect the work domain, several observations were made.The swarm changes the drone pilot's role in the system.The swarm, through its behavior, can make it more difficult to follow search structure instructions, follow rules, and deliver a good POD, which in turn can complicate the possibility of finding the missing person.However, the work can also be simplified if the drones were capable of autonomous decision making.This could result in faster POD deliveries and finding missing persons more quickly.However, the tasks of confirming searcher whereabouts can also be increasingly challenging because of this, as it can be hard to discern where the drones have been.The operator's perspective is interesting with respect to control tasks in decision-making terms; constraints could include having to consider battery levels of multiple drones and maintaining altitude.However, re-dispatching drones to searched areas could be easier with swarms.Multiple cameras could also contribute to better area representations and deliver them more quickly.Furthermore, we juxtaposed the discussions about trusting swarming drones, search dogs, and volunteers.The search organization trusts the dogs because the dog handler confirms their findings.They trust the search patrol leader's judgement because they can see the volunteers all the times (who also are human and believed to make better decisions).As drones are not human, the same trust does not apply to them.They can miss important details.If they fly too far away they are not considered to be sufficiently controlled by the operator.
In summary, drone swarms pose a challenge in terms of operator control and trust which, in turn, makes it challenging for the command center to control the operation.

DISCUSSION
We will structure our discussion to address our three research questions in turn.
RQ1: How can SAR organizations structure their field operations?The SAR mission description is similar to a C2 remote control room approach [22] where most of the planning takes place.In this case, the mission commander oversees the mission from a station in the general mission area.However, searchers, dog patrols, pilots and drones, and the volunteers in the field are responsible for delivering PODs, maintaining structure, and following directions.The participating SAR organization therefore also contains elements of edge organizations [1].There is also an emphasis on verbal information flow and communication between groups and with the mission commander, as reflected in C2 systems literature [22], e.g., when searchers ask to go off-script or deliver PODs.The communication serves the higher goal of having a shared situational awareness of the mission.
RQ2: How are drones a part of that process?Drones are used to gather information about the search area.Furthermore, the operator must continuously return to the command center to report what they have seen on the feed, similar to how dog handlers and searchers must return to the command center and deliver PODs.Drones are used throughout the search process, from the initial search to when the person is either found or the search is called off.During the search, the operator continuously makes sure that the area can be searched in a way that adheres to flying rules and the drone capabilities.The goal is to create a search grid system and maintain altitude.This sheds some light on how the introduction of drone swarms will change how the operator structures their job in order to maintain alignment with both the system's overall goals and the operator's previous drone operation goals.
RQ3: What are the potential challenges and opportunities in deploying autonomous drone swarms in semi-voluntary SAR organizations?
The swarm can be detrimental to shared situational awareness-a major constraint-if the operator cannot keep track of the swarm to the same extent as today's search dogs, search volunteers, or singular drones.It could be unclear what the drones have done or, most importantly, what they have seen or not seen.This, too, highlights a perceived need for human decision making to foster a shared situational awareness [17].The required human judgement and operator swarm control, combined with additional drones, could increase task switching (e.g., between monitoring, managerial, or troubleshooting tasks) and mental workload.If not designed properly, the task of supervising the drone swarm could become more cognitively taxing than teleoperating a single drone, and the system performance may suffer accordingly [11].The added task of monitoring multiple video feeds, and its associated attentional demands, could result in a loss of the operator's situational awareness.The operator's ability to maintain altitude and sustain a focused search could also be negatively impacted by the increased information load (depending on technical implementation choices).However, since the SAR organization has rules to follow, there is a constant need for human judgement (not only from a search perspective) adding to the swarm operator's workload.To mitigate this, swarming drones could be designed to adapt formation, maintain altitude, avoid obstructions, and adapt to wind and battery levels while preserving the operator's means of executive and supervisory control.
There are also interesting team aspects.Like in C2, there is a focus on communication and shared understanding.Also, the lack of trust in the swarm reflects the uncertainty towards automation discussed in the literature [14].In addition, the participants were concerned about introducing drone swarms in their SAR organization because of the critical nature of searching for missing persons.Furthermore, external circumstances like wind may affect trust if it would cause the drones to fly over areas that are prohibited.These findings reflects how trust mediates between automation reliability and human reliance on automation [12].In conclusion, the rules for flying drones must change for drone swarms to be applicable to SAR activities, both in terms of reducing legal risk, and reducing operator and command center workload.At the same time, SAR organizations should consider reducing their operational control of search tasks to maximize the utility of additional drones [11].This includes building trust through technology exposure and experience, and demonstrated technology capability.

CONCLUSIONS
Through our exploratory study we have identified potential effects and constraints that arise from integrating drone swarms into current SAR work settings.We found that managing a drone swarms presents challenges in terms of increased workload, attentional demands, and decreased situational awareness for operators.However, with careful design and implementation, these challenges can be mitigated.Our study also highlights the significance of communication and shared understanding among SAR team members.Effective communication and information flow between the operator, searchers, and the command center are crucial for maintaining situational awareness and achieving mission objectives.This finding highlights the need to prioritize clear and efficient communication channels when integrating drone swarms.Trust and reliability considerations emerged as important factors in the adaption of drone swarms in SAR operations.Building trust through technology exposure and experience, along with demonstrating system capability and reliability, is essential.SAR practitioners expressed concerns about the critical nature of SAR missions and the need for human judgement and decision-making in maintaining situational awareness.
We have identified several research avenues that should be holistically considered for future drone swarm design, development, and integration.First, future research should focus on developing adaptive and intelligent swarm control mechanisms to reduce the cognitive load on the operator.This could involve designing algorithms and automation features that allow drone swarms to autonomously adjust their formation, stay within specific altitude bounds, avoid obstacles, and adapt to environmental conditions like wind or rain.Such advancements could preserve the operator's means of control without increasing their mental workload.Second, investigating trust dynamics between operators and drone swarms is important.Future research should explore how to foster trust in swarm technology and its reliability in different domains, including SAR operations.This could include conducting studies to further understand how trust mediates between automation reliability and human reliance on automation [12] and swarm systems.Building trust through technology exposure and experience, as well as demonstrating the safety, capability, and safety of drone swarms will be essential for their successful work setting integration.Third, future research should strive to include prospective end users and stakeholders, such as SAR practitioners, in the design and validation processes of drone swarm systems and components, such as control methods or system visualizations.Involving these key stakeholders will ensure compatibility with existing work settings and organizational fit.Fourth, as drone swarms introduce new challenges and complexities, it is important to address legal and operational frameworks.Future research should explore necessary changes to regulatory and operational guidelines to accommodate the deployment of drone swarms.This includes addressing the issue of responsibility attribution.Finally, more research is needed to better understand the integration effects and organizational impact of integrating drone swarms in existing work settings.This could involve conducting field studies and simulations to assess how drone swarms affect operational procedures, teamwork dynamics, and the roles of SAR practitioners.By identifying and addressing potential barriers to integration, researchers can propose strategies to facilitate the seamless adoption of drone swarms in SAR organizations.
By taking a holistic perspective on these issues, researchers can advance the HSI field as it relates to SAR operations and pave the way for safe and effective integration of drone swarms in real-world settings.

Figure 1 :
Figure 1: The Mural scenario whiteboard used to facilitate the digital workshop.The icons in the left vertical panel represents (starting from the top ) mission commanders, dogs, the swarm operator, drones, search personnel, and initial possible target locations.Workshop participants drew figures to explain search strategies and inserted text boxes to indicate wind direction, etc.

Figure 2 :
Figure 2: The contextual activity template (CAT) model describes how the object-related processes (or functions) from the work domain analysis (in 3.1) in maps onto the six control situations extracted from the SAR mission description.The circle and whiskers indicates in which situations each function typically occurs.The dotted line boxes represents situations where a function would not typically occur, but possibly could.