Common LeSS Transformation Patterns

Various scaled agile frameworks have been developed to address the challenges of implementing agile methods in large-scale projects. Adoption of these frameworks is quite demanding. The paper aims to analyze selected case studies focused on the adoption of the Large Scaled Scrum (LeSS) framework through the usage of natural language processing. As a result, common patterns of LeSS transformations are presented, i.e., adoption patterns, success patterns, and challenges. So, the audience, i.e., agile coaches who work with the LeSS framework and potential LeSS adopters, can understand how they may approach the agile transformation and which mistakes to avoid.


INTRODUCTION
To thrive in the contemporary business landscape, organizations must be able to rapidly change the way of creating and delivering value to customers.Central to this adaptability is the pivotal role played by software systems.The methods employed in their development must facilitate collaboration, foster innovation, and enhance speed.Traditional waterfall methods, having failed to meet the demands of this new challenge, have given way to the emergence of agile methods.
Agile methods were officially introduced through a set of four core values and 12 principles outlined in the Agile Manifesto in 2001 [3].These principles contribute to Agile's ability to respond to changing requirements, deliver high-quality software, and foster collaboration in the development team and with stakeholders [9].According to the 15th State of agile report [7] the prevalence of agile methods surged from 37 percent in 2020 to 86 percent in 2021.
Although agile methods were originally designed for a usage in small, single team projects [4], they also become attractive for larger projects and companies [8].Usage of large-scale agile involves additional concerns in handling inter-team coordination and interfacing with other organizational units, such as human resources, marketing and sales, and product management.In addition, large-scale may cause users and other stakeholders to become distant from the development teams [8].Several agile scaling frameworks were developed to solve these issues like Scaled Agile Framework (SAFe) [35], Large Scaled Scrum (LeSS) [40], Diciplined Agile Delivery (DAD) [27], Scrum@Scale [37], Nexus [36], and others.However, introducing an agile approach to large-scale projects poses greater complexity [8] and implementation of large-scale agile methods has proven highly challenging The challenges and success factors of large-scale agile transformations are currently quite intensively researched [6,8,19,25,28,34,41].However, Uludag et al. [42] present the observation of adopting specific scaling frameworks in companies and the associated benefits and challenges as the most frequently stated research question on scaling agile frameworks.While some studies focusing specifically on the adoption of Scaled Agile Framework (SAFe) exist [20,28], publications focusing specifically on the adoption of Large Scale Scrum (LeSS) do not exist.To fill this gap, the authors defined the following research question: What are common patterns that can be recognized through the natural language processing (NLP) analysis of the LeSS transformation case studies?Combining definitions from machine learning and agile methodologies, a pattern, within the context of both realms, can be seen as a recognizable and repeatable structure or set of behaviors.In machine learning, patterns are data-centric, focusing on identifying and classifying regularities in data, while in agile, they are more process-oriented, providing solutions to recurring problems.Both share the commonality of leveraging past data to guide future decisions, actions, or strategies.For this research, the pattern is defined as aforementioned, with an additional criterion: it must be present in more than half of the analyzed cases.By presenting these patterns, future LeSS adopters gain a competitive edge and valuable insights on best practices and pitfalls to navigate for a successful and streamlined transformation.
The paper is based on research conducted within the master's thesis [15].The paper is organized as follows: The related work is presented in Section 2. The research method is described in Section 3. Results of the research are presented in the form of LeSS transformation patterns in Section 4. In Section 5 the results are discussed, and limitations of the research are stated.Lastly, the conclusion is presented in Section 6.

RELATED WORK
Large Scale Scrum (LeSS) was created by Bas Vodde and Craig Larman in 2013 based on their experiences working with large-scale product development.LeSS emphasizes adopting agile principles and best practices [23], laid down by small-scale agile frameworks, mostly Scrum and XP (Extreme programming).The LeSS provides two different large-scale Scrum frameworks [40], i.e., the basic LeSS that applies to up to eight teams (of eight people each) and the LeSS Huge that introduces additional scaling elements for the development of up to hundreds of developers.
A comprehensive review of the agile scaling frameworks highlighting differences and similarities is presented in [1].Scaled Agile Framework (SAFe) is the most used scaling agile framework based on the State of Agile survey [7].However, some practitioners consider SAFe too heavy and complex, and some even say that SAFe adds complexity to bureaucracy, evolving into "the new waterfall" [10].Moreover, SAFe strongly emphasizes process rather than people [32].On the other hand, LeSS is much more lightweight than SAFe [19] and emphasizes simplicity in scaling Scrum.
Transforming a company to become agile is a process.Each company addresses the transformation differently.The reasoning for an individual approach to transformation lies in the variety of specific business needs and, most importantly, matching the agile framework to organization processes [18].Whether a company focuses on a slow transformation or a "big bang" does not necessarily yield an easier time, as both approaches have their benefits and difficulties.Organization-wide agile transformation focuses not only on a change of approach to software development [26] but also on all aspects of the company, including the agile mindset.
Mundra [25] writes about the difficulties, narratives, and cultural aspects that must be considered when adopting agile on a large scale.Similar challenges are stated by Trippensee and Remane [41] who also add the need for correct tailoring of the selected method to the organisation's needs and provide mapping of relevant studies for future agile adopters to ease access to the sources.
Despite Dikert et al [8] published challenges and success factors for large-scale agile transformations as a result of a systematic literature review, no agile scaling framework was examined in this study.Although Kalenda et al. [19] have reviewed practices, challenges, and success factors for scaling agile focused mainly on SAFe and Less frameworks, they present them generally for all scaling agile frameworks.The same is valid for Conboy and Carrol [6], who identified challenges and success factors associated with implementing SAFe, Scrum-at-Scale, Spotify, LeSS, Nexus, and other mixed or customized large-scale agile frameworks.
Uludag et al. [42] identified challenges and success factors related to large-scale agile transformations in their systematic mapping study.Less framework, however, occurred only in 3 cases.Edison et al. [11] conducted a systematic literature review to compare the main large-scale agile methods: SAFe, LeSS, Scrum-at-Scale, DAD, and the Spotify model.They also present challenges and success factors associated with the use of large-scale agile methods.In this study, LeSS framework was used in 4 out of 34 cases.Most recent research investigating challenges in large-scale agile software development projects was based on a case study with two Swedish software companies [34], where only Scrum of Scrums and Scrum were used.The authors did not find any study focusing specifically on the LeSS transformation.

METHODS
The goal of the research is to analyze selected LeSS agile framework adoption case studies presented on the official LeSS framework website [40] with usage of natural language processing (NLP) and identify common LeSS transformation patterns.The analysis was conducted in the following phases that are then described in subsequent subsections:

Phase 1 Case Studies Selection
At first, case studies for the analysis were selected.The LeSS official web page [40] offers 32 unique case studies about LeSS transformation that present not only successful transformations but also challenged transformations.18 case studies are labelled as long being more than 10 pages, and 14 as short with less than 10 pages.The following selection criteria were defined to be applied to individual case studies: • SC1: Case study is longer than three pages.
• SC2: Case study contains a conclusion.
• SC3: Factors of success and/or failure are mentioned in the text.Then, two other criteria were applied to the group of case studies: • SC4: At least half of the case studies are long (more than ten pages).• SC5: At least half of the case studies are anonymized to provide an accurate view and enable complete transparency.Based on these criteria and the preliminary analysis, 11 out of the 32 case studies were selected, which are specified in the following list: • CS1: German Big Insurance -LeSS agile transformation in 2017-2018 after a Scrum adoption in 2016-2017.The case study describes adoption in a software development department of a large insurance company in Germany.It features the department redesign, introducing LeSS events and redefining ongoing processes.• CS2: Large Dutch Bank -Improvement of the previous Spotify model in accordance with the LeSS framework in an environment of a bank company.The product is developed across more than 20 teams, out of which all can release new increments every two weeks.The LeSS principles improve the overall Spotify architecture and processes of the company.• CS3: Solarwinds -An IT software company providing software for companies around the globe that turned to LeSS because of its inefficiency in terms of adding new features to the product.The adoption focused mainly on the key principles of LeSS.Thus, reducing the complexity and eliminating queues to work more efficiently.• CS4: Sys Store -A software development company that provides an online platform to try various software products before buying them.The German company adopted scrum in 2011, which LeSS replaced during the years 2015-2016.
The transformation was set to build cross-functional teams in a rigorous environment to prevent bugs and the constant need to fix the code after merging various branches.• CS5: Thales -A transformation of a business unit of a software development company that works in the aerospace industry, providing radar solutions for customers around the world.The company has approached LeSS to eliminate the "scrum-but" adoption in an environment where projects take years to finish.• CS6: Very Big Bank -A LeSS adoption on a full-scale company environment with thousands of employees and a traditional organizational structure with multiple layers and intermediaries.The LeSS transformation took place between 2015 -2016 and was set to fix the previously wrongly adopted Scrum.The LeSS dealt with the rigid structure, queues, and processes in the company.• CS7: Bwin.party -The biggest player in online gaming spread across the world that underwent a LeSS transformation to optimize the company product and development aspects and to unify the approach to development, which differed in different subsidiaries.The LeSS was set to reduce the T2M and enable faster deliveries to improve upon the customers' feedback and provide them with more features in less time.• CS8: Telecom Australia -A large telecommunication provider in Australia that adopted less in 2012 to jump starts its competitiveness with the new billing and ordering capabilities.
The adoption aimed to solve the underlying issues regarding the continuous queue creation and inefficiency of software development processes in the company and to enable outsourcing.• CS9: Tom Commerce -LeSS adoption in e-commerce product development aimed to solve the underlying issues of under-delivering, bad team morale, lack of quality and other aspects.• CS10: UBS -A long-term incremental adoption of LeSS principles and ideology that began in 2005 was slowly revealing and helping to fix fundamental issues the company had faced.Despite its financial domain, UBS was experimenting with trains and different forms of agile beforehand.However, only with LeSS the company managed to improve on its weaknesses slowly.• CS11: VeSecurity -A security management company that underwent a LeSS adoption to reduce the customers' negative feedback regarding the low quality and visibility of progress along the low customer satisfaction.The company adopted LeSS to reduce the expertise-driven teams and form crossfunctional teams that deliver the product more efficiently.
Characteristics of the case studies are depicted in Table 1.

Phase 2 NLP Analysis and Human Evaluation
Natural Language Processing (NLP) solutions are broadly used for the processing of semi-structured and unstructured data.The main benefit of NLP lies in conducting large-scale analyses, providing accurate and objective summaries, and gaining real, quick, and actionable insights [12].In many cases, the key benefit of NLP is that the algorithms are repeatable in the same way and under the same conditions.The rapid development of artificial intelligence and machine learning brings an innovative approach to the field of NLP.It opens new opportunities for these technologies, methods, and practices to be applied and processed.NLP was used to assist the human in evaluating the factual accuracy and similarities in the studies.With the knowledge of the essential limitations of the mentioned approach, it was beneficial to incorporate the human factor as a final step to double-check the common patterns and combine the two techniques for more specific results.So, the analysis was conducted in two steps.Firstly, the data-driven NLP analysis compared each paragraph with all other paragraphs and built a base for the second part, which involved a human evaluation of the NLP analysis results.Phase 2 was conducted in the following steps: were divided into paragraphs, labeled and numbered for tracking.Paragraphs with fewer than 80 characters were omitted to prioritize substantive content.These brief paragraphs typically serve as headings, leading to 99-100 percent similar sentences.This exclusion aimed to prevent bias in the results of NLP analysis.This procedure reduced the number of paragraphs from 3500 to approximately 2000, enhancing the overall analysis quality.• Step 2.3 Paragraph Comparison: During the NLP, the dimensional dense vector was calculated for every single paragraph and compared with all other paragraphs using the cosine technique to calculate the final similarity.The pretrained model all-mpnet-base-v2 was used [16].The allmpnet-base-v2 model is crucial for transforming paragraphs into dense vector representations.This model, leveraging MPNet's advanced pre-training method [24], encodes the semantic essence of the text into a 768-dimensional vector space [31].For each paragraph, the model generates a dense vector, effectively capturing its semantic information (the context of the paragraph).The generated vectors were compared using the cosine similarity technique, allowing for an effective measure of semantic similarity between paragraphs.

Phase 4 Human Evaluation of Individual Case Studies
Despite the benefits of the NLP, a human evaluation of each case study was needed to answer the research questions, and identify the reasons for the LeSS adoption, success factors and challenges faced during the transformation.The original organization structure, way of working, architecture, vendor management, people motivation, performance measurement, customer satisfaction, company efficiency, product quality, or economic conditions the companies from selected studies had faced before the transformation started were substantial for the adoption motivators' part.The previous experience with any agile transformation has also been taken into account.
The answers related to successes, failures, and challenges were derived from the described processes of the change and case study author's conclusions, progress evaluation, and further improvements planned or expected after study publishing with the focus on identified adoption motivators within the selected companies.

Phase 5 Patterns Derivation
Ultimately, success factors, failures, and challenges distilled from case studies were systematically compared, revealing common patterns defined by their frequency across the cases.If the factor occurred in more than half of the case studies, it became a pattern.The process of pattern derivation and all derived LeSS transformation patterns are presented in section 4.

ANALYSIS RESULTS: LESS TRANSFORMATION PATTERNS
In this section, the common LeSS transformation patterns that were extracted based on the previous NLP and human analyses are presented.Patterns are categorized into three groups: adoption patterns, success patterns and challenges.

Success Patterns
Table 3 shows the LeSS transformation success factors defined from the selected case studies.The success factors that occurred in more than half of the case studies were defined as common success patterns.Success pattern IDs are depicted in the first column of

Common Challenges
Table 4 shows the challenges encountered during the transformations in the selected case studies.The challenges that occurred in more than half of the case studies were defined as common challenges and are depicted in the first column of Table 4. Their numbering is again based on the frequency of occurrences.The most common challenge Different Company Culture (CP1) involves adapting to a new agile culture, which can be challenging in organizations used to traditional management styles.CP2: Lack of Leadership Support represents a critical challenge where the lack of support from leadership can significantly hinder the transformation process.Leadership support is essential for guiding and facilitating the change process effectively.CP3: Unclear Organizational Design expresses that the ambiguity in roles and organizational structure can create confusion and impede the effective implementation of LeSS.CP4: Productivity/Prioritization Issues represent challenges with effectively prioritizing tasks and maintaining productivity levels during the transition to LeSS.CP5: Lack of LeSS Principles Understanding means that team members do not fully understand LeSS principles, which may lead to implementation difficulties.

DISCUSSION
Adoption patterns provide insight into the motivations behind adopting LeSS in various organizations.They reflect a common theme of seeking greater agility, efficiency, and responsiveness in a rapidly changing business environment.Each pattern encapsulates a specific area of focus, guiding organizations toward successful agile transformation.As other studies focused on large-scale agile transformations that were examined in Section 2 do not specify reasons for adoption, identified LeSS transformation patterns were compared with the reasons for agile adoption stated in the 15th State of Agile survey [7].Most of the adoption patterns were also reflected in the survey findings, except AP1.The possible reason is that respondents of the 15th State of Agile survey were from companies implementing all agile methods, not only scaled agile ones.So, the significant reason for the adoption of scaled agile methods, and specifically the LeSS framework, is just breaking down the monolithic architecture.
The most occurring success patterns were SP1: Growth/Agile Mindset, SP2: Volunteering and a "Healthy" Atmosphere, and SP3: High Quality, Technological Debt Reduction.These success patterns provide future adopters with the best practices recommended during the LeSS agile transformation to smoothen up the adoption process and eliminate possible issues.Comparing these success patterns with success factors identified by Dikert et al. [8] Agile Mindset was reflected as the second most important success factor.SP2: Volunteering and a "Healthy" Atmosphere can be matched to the Team Autonomy success factor stated in [8].The most significant success factor identified in [19], i.e., Acquiring Knowledge, possibly matches SP11: Utilize Experienced Coaches and Individuals.On the other hand, the United View on Values and Practices, Tools and Infrastructure, and Careful Transformation success factors from [19] do not have corresponding counterparts in our study.SP3: High Quality, Technological Debt Reduction matches the fourth success factor, Solid Engineering Practices from [19].
The most occurring challenges were CP1: Different Company Culture, CP2: Lack of Adoption Support from the Leadership, and CP3: Unclear Organizational Design.Comparing these challenges to challenges stated in the 15th State of Agile survey [7], we found out that the challenges faced by companies across the globe, according to the survey report, closely correlate with the patterns derived in this research.The comparison with the study focusing on challenges of the SAFe transformation [28] does not provide correspondence except for the Organizational Politics challenge that possibly corresponds to CP3: Unclear Organizational Design.Higher matching was found between challenges identified in our study and challenges of large-scale development methods [11].However, Organizational Resistance to Change mentioned in other studies [7,11,28] did not appear among our LeSS transformation challenge patterns.On the other hand, we uncover patterns that are not depicted in other studies, such as Difficulties in Creating a Common Backlog, and the Lack of Adoption Understanding among the Employees.We are convinced that the challenges patterns provide future adopters with issues that should be considered.

Limitations and Validity
Conducted case study analysis may be prone to specific methodological risks regarding validity.In particular, publications and sources included in the analysis may be subject to publication bias.For this study, only LeSS adoption case studies presented on the official LeSS framework website [40] were selected, which may constitute publication bias [33] as maintainers of the website can affect the publication.Based on the preliminary analysis, it is necessary to admit that not only are successful transformations presented on the website, but case studies present transformation challenges.Moreover, the case studies were selected based on the selection criteria to mitigate this bias.
Another threat to validity occurs due to the dates on which the case studies were conducted.As there is a significant evolution between the technologies adopted between 2012 and 2023, the impact of LeSS framework adoption and its success factors and challenges may be different from the current ones.Therefore, the timeline when the case studies were conducted is highlighted in a structured description of the analyzed case studies in Table 1.
While models like all-mpnet-base-v2 have advanced NLP capabilities, NLP has inherent limitations.
• Contextual Understanding: NLP models may struggle with understanding context and nuances in language, particularly with idioms, sarcasm, or culturally specific references.The human evaluation of the NLP analysis was incorporated into the method to address these limitations.
The following cases can serve as an example of the result of these limitations.The Sys store case study had the highest number of success factors.Nevertheless, the transformation in the Sys store was unsuccessful.However, the case study shows many lessons learned and best practices for future adopters to follow, making it seem like a highly successful transformation for the computer.At the same time, according to the NLP, the UBS transformation is different from the other companies' transformation, but the human evaluation of the case studies shows the similarities. .

CONCLUSION
Companies must stay competitive in today's world.The situation is ever-changing, and companies must be able to adapt to the highly volatile markets and customers' needs.Since the early 2000s, there has been a notable surge in companies adopting agile frameworks.However, agile transformation is a demanding process.With the aim of helping companies undergo this process successfully, the goal of this paper was to derive common LeSS transformation patterns from the selected LeSS transformation case studies.Patterns, that were categorized into three groups: adoption patterns, success patterns, and challenges, were discussed and compared to similar findings from related sources.These success patterns and challenges offer crucial insights for organizations undertaking a LeSS transformation journey.The success patterns provide a set of best practices to emulate, while the common challenges present scenarios to avoid or prepare for, ensuring a smoother and more effective transition to agile practices.
The methodology presented in the paper, which combines advanced NLP techniques using the all-mpnet-base-v2 model with a structured human evaluation process, represents a versatile blueprint for future research across various domains.This approach can revolutionize how textual data are analyzed in sociology, psychology, and legal studies, enabling a deeper understanding of semantic similarities in diverse datasets.Its application can significantly enhance semantic analysis tasks such as sentiment analysis, topic modeling, and multilingual text understanding.Beyond text analysis, this methodology has potential implications in humancomputer interaction, refining the capabilities of AI systems like chatbots and virtual assistants for more nuanced and context-aware responses.
Moreover, incorporating agile principles through the involvement of a certified LeSS agile coach in the evaluation process sets a precedent for future research.This approach promotes a structured, objective, and iterative evaluation method, ensuring research findings' practical relevance and reliability.Additionally, this methodology provides a framework for addressing ethical concerns in AI, particularly in reducing biases and improving transparency in AI systems.As a comprehensive blueprint for large-scale text analysis, it is poised to influence various research and commercial contexts, especially in the era of big data.Furthermore, the methodology is an excellent educational tool, offering practical insights into integrating advanced technology with human oversight in research and AI development.Combining sophisticated NLP technology with a rigorous human evaluation process in the methodology offers a comprehensive model for future research, balancing technological advancement with ethical and practical considerations.

Table 1 :
• Step 2.4 Human Evaluation of NLP Analysis: The second author, a certified agile coach specializing in Scrum, Kanban, and LeSS agile frameworks, manually analyzed the top 5 unique paragraph pairs exhibiting the highest similarity.He set clear objectives for the evaluation, developed comprehensive criteria, and ensured an unbiased and structured Overview of Selected Case Studies (L for Length, A for Anonymized) The evaluator, trained in workshops organized by the agile coach Petr Novotný for Scrum and Craig Larmann for LeSS, assessed the paragraph pairs based on the developed criteria.Following the evaluation, he conducted feedback sessions, fostering iterative improvements in the methodology among the group of agile coaches at T-Mobile Czech.

Table 2
shows the common reasons for LeSS adoption in the companies.The case studies are identified with IDs that are depicted as column headers CS1 to CS11.If the reason for adoption was found during the NLP analysis, N is shown in the cell, whereas H represents the human analysis, and NH represents both NLP and human analyses.The reasons for adoption that occurred in more than half of the case studies were defined as common adoption patterns with IDs depicted in the first column of

Table 2
allows for quicker feedback incorporation and continuous product improvement.This approach aligns with agile principles of iterative development and frequent delivery, ensuring that products and services evolve continuously to meet customer needs.

Table 2 :
Reasons for LeSS Adoption N (NLP Analysis), H (Human Analysis), NH (NLP and Human Analyses)

Table 3 :
LeSS Transformation Success Factors N (NLP Analysis), H (Human Analysis), NH (NLP and Human Analyses)

Table 4 :
LeSS Transformation Challenges N (NLP Analysis), H (Human Analysis), NH (NLP and Human Analyses)

Table 3
Growth/Agile Mindset stresses the significance of nurturing an agile culture and mindset centered on continuous improvement and adaptability.It encourages a mindset shift in the organization toward embracing change, learning, and innovation.SP2 pattern shows that volunteering and a "healthy" atmosphere are significant factors in fostering a positive work environment where team members voluntarily participate and engage in the transformation process.The pattern highlights the importance of employee well-being and motivation in driving successful change.SP3: High Quality, Technological Debt Reduction pattern focuses on enhancing product quality and addressing accumulated technological debt to facilitate smoother and more efficient operations.It aims to build a solid technical foundation that supports agile practices and principles.The Connect the Business and Technology (SP4) pattern emphasizes aligning business objectives and technological capabilities.The goal is to guarantee that technology strategies and operations are tightly aligned with business objectives.The Promote Good Engineering Practices (SP5) pattern stresses the need for robust engineering practices to support agile transformation.It involves adopting best practices in software development to improve quality and agility praised by the frameworks.
• Training Data Bias: The models can inherit biases present in their training data, leading to skewed or unfair outcomes.•Complexity and Resources: Advanced NLP models often require significant computational resources for training and operation, which can be a barrier for widespread adoption.• Generalization: Models may not generalize well across different languages, domains, or types of data.• Interpretability: Understanding why a model made a specific decision can be challenging, leading to issues with transparency and trust.