Industrial Challenges in Secure Continuous Development

The intersection between security and continuous software engineering has been of great interest since the early years of the agile development movement, and it remains relevant as software development processes are more frequently guided by agility and the adoption of DevOps. Several authors have contributed studies about the framing of secure agile development and secure DevOps, motivating academic contributions to methods and practices, but also discussions around benefits and challenges. Especially the challenges captured also our interest since, for the last few years, we are conducting research on secure continuous software engineering from a more applied, practical perspective with the overarching aim to introduce solutions that can be adopted at scale. The short positioning at hands summarizes a relevant part of our endeavors in which we validated challenges with several practitioners of different roles. More than framing a set of challenges, we conclude by presenting four key research directions we identified for practitioners and researchers to delineate future work.


INTRODUCTION AND BACKGROUND
Combining secure development practices with continuous software development (CSD) workflows poses various challenges.How to cope with them has been in the scope of both researchers and practitioners alike for many years [5].In our perception, a certain Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.Copyrights for third-party components of this work must be honored.For all other uses, contact the owner/author(s).ICSE-SEIP '24, April 14-20, 2024, Lisbon, Portugal © 2024 Copyright held by the owner/author(s).ACM ISBN 979-8-4007-0501-4/24/04. https://doi.org/10.1145/3639477.3639736consensus seems to be that the integration of security practices is, at least, difficult.This is corroborated by various secondary studies [4,[7][8][9][10]12] and more practically oriented grey literature [3,11] reporting on the state of the art and related challenges.However, existing recommendations require too often more empirical evaluations and especially acceptance by practitioners considering their aim at solving practical engineering challenges [2,5].To the best of our knowledge, while CSD has been adopted in various interpretations by large enterprises (e.g.scaled agile), the question of which security-specific challenges and needs are relevant in the context of CSD requires further exploration specifically in terms of relevance according to practitioners' target (e.g.software product security, secure development process, security strategy).We consider this of particular importance to both practitioners as well as researchers developing plausible solutions to those challenges.In this paper, we report on our study results to provide a first answer to this very question.We extend existing discussions around security challenges in agile development and, in particular, DevOps, considering two perspectives: The experiences and expert opinions of individuals, and the perspective of engineering teams.We aim to analyze how groups of practitioners perceive which challenges in their specific context, as described next.
Study Design in a Nutshell.In brief, we conducted case studies in 3 different companies operating in highly regulated environments.
Initially, we partnered with security experts to produce a draft list of relevant challenges in the field combining pertinent practitioners and academic sources.Later, in 2019 and 2020, we held five workshops with security and CSD practitioners to refine and validate the list of challenges (see Figure 1).A exploratory workshop with security experts, who consult security for agile teams, delivered the final challenges.Subsequently, individuals of two focus groups reviewed and prioritized the challenges: subject experts (SE) in CSD with responsibility for security, and agile team members with cross-functional roles (AT).Our study participants chose a maximum of five challenges in CSD, that are of priority considering their common objective.Such an objective is either to improve the security of a specific software product or to implement strategies for secure agile development in their organization.The study reported here is part of our long-term investigation of our industrial partners with the goal of supporting an efficient integration of security practices in CSD.As we hope to foster the exchange and discussions around industrial challenges in CSD, we concentrate on reporting our distilled challenges and implications for further research.

INDUSTRIAL CHALLENGES
Our study yielded a list of prioritized challenges for security in CSD as depicted in Table 1.The challenges are grouped into five categories: • Continuous development: how to perform security activities like threat modeling, secure code review, security hardening continuously, and, enabling continuous experimentation for security purposes.• Value Stream: how to make security value visible to the customers through requirements, architecture or faster feedback of security vulnerabilities.• Efficiency: how to achieve security for product and in the process with less impact in resources or lead time.• Knowledge Transfer: how to educate teams to make effective security decisions and share knowledge.• CI/CD Pipelines: how to automate security in pipelines considering its protection and compliance.

FUTURE RESEARCH WORK
Considering these challenges at our industrial partner, we propose the following four major takeaways as relevant for organizations adopting CSD while pertaining to regulated security environments: (1) Visibility and assessment of security practices maturity for CSD based on applicable security standards allowing practitioners to break silos between engineering and security experts teams by providing transparency of the status of security practices that regulators will expect in the development process.This includes evaluating both human and automationbased security activities e.g.secure coding review by peerreviewed and static methods.Challenges: C1,C2, C4, C5, C7, C8.
(2) Implementation of continuous security feedback-loop referring to the effective management of security-related issues, that are identified through automatic vulnerability checking by CI/CD Pipelines.Challenges: C9, C12.(3) Continuous automatic security compliance assessing capabilities and limitations of automation in continuous security compliance, e.g. by providing delta analyses with particular security norms, and allowing for the development of sensible automation solutions.Challenges: C6, C7, C8, C14.(4) Improvement of security skills in agile/DevOps teams by training security champions and providing pertinent guidance on continuous security practices.This favors better decisionmaking about security risks in the team's products' continuous delivery.Challenges: C9, C10 influencing C11, C13, C15.
The relevance of the takeaways is further corroborated by our partner's decision to explicitly support them with dedicated internal research projects and doctoral studies (c.f.[1,6,13]).

Figure 1 :
Figure 1: Summary of Study Methods.Circles in grey indicate: number of workshops n, and participants P.

Table 1 :
Practitioner's Prioritized Challenges for Security in Continuous Software Development