Managing API Evolution in Microservice Architecture

Nowadays, many software systems are split into loosely coupled microservices only communicating via Application Programming Interfaces (APIs) to improve maintainability, scalability, and fault tolerance. However, the loose coupling between microservices provides no immediate feedback on breaking API changes, and consuming services break or exhibit unexpected behavior only after the first actual call to the changed API. Hence, development teams must actively identify and communicate all breaking changes to affected teams to stay compatible. This research addresses this problem with three contributions. First, we identified API evolution strategies and open challenges in practice with an explorative study. Based on the study findings, we formulated two open research directions for evolving publicly accessible APIs, i.e., REpresentational State Transfer (REST) APIs. As the second contribution, we will introduce a REST API change extraction approach to improve the change notification accuracy. We plan experiments on open-source projects to evaluate our approach's accuracy and compare it to openapi-diff for structural changes. Third, we plan to investigate methods for automating communication with affected teams, which will then improve the change notification reliability. Finally, we will evaluate the accuracy and reliability of our notifications with a user study.


INTRODUCTION
The Microservice Architecture (MSA) improves the overall maintainability, scalability, and fault-tolerance of a system by replacing a large application with many small and loosely coupled services [16,24,26].Each such microservice manages a subset of the overall system's responsibilities and individual development teams exclusively develop, maintain, and deploy a logical set of microservices.Typically, the MSA employs two means of communication [38]: event-driven for system-internal service communication, e.g., publish-subscribe, and request-response for communicating to external services, e.g., REpresentational State Transfer (REST) [12].In this paper, we use the term Application Programming Interface (API) for both, as both enable communication between microservices.The communication style in MSA is called "smart endpoints and dumb pipes" and allows the APIs to evolve independently [35].
However, as each microservice API's data structures and behaviors evolve independently, the API evolution process requires more synchronization efforts between the development teams than for a single monolithic code base [24,39].Changes in one API could result in unexpected behavior and potentially break consumer services, i.e., services interacting with the API [35].From the consumer perspective, such breaking changes are undetected at compile time and only manifest during an actual call to the changed API at runtime.Consumer-side contract tests do not mitigate this problem as the API test doubles do not automatically reflect the latest changes.Accordingly, development teams of consumer services rely on the teams providing an API to correctly notify them about all changes in advance.The provider teams can utilize regression testing [2,15] to detect API contract violations after introducing changes.Still, they must understand the actual API changes leading to these violations.We conducted preliminary discussions with practitioners regarding these challenges.In practice, they manually identify the API changes and notify other teams about them, slowing down the API evolution process and making it susceptible to human error.

RELATED WORK
Related works identified the communication and integration of microservices as major challenges in the MSA [1,32].Chen et al. [8] conducted a grey literature review on microservice API concerns in practice and identified concerns with API design, API versioning, and API testing.Wang et al. [34] studied REST API change discussions on Stack Overflow and concluded that REST APIs are more change-prone than Java library APIs and the Web Services Description Language (WSDL).Bogart et al. [3] found that developers maintain old API versions to prolong the transition period for consumers but the several separate versions increase the maintenance cost as a result.According to Zdun et al. [37], the microservice API evolution process still misses efficient communication and support for consumers affected by API changes.Similarly, Cerny et al. [6] proposed detecting incompatible API changes and communicating changes to consumer teams as open research challenges.Multiple works focused on detecting and classifying API changes in source code, e.g., in Java libraries [4,29] and the Android Platform [23].They automatically identify breaking source code API changes and accordingly update the calls [36] or the API documentation [21].Related works also investigated the evolution of WSDL [7,27], but only recently started to target REST APIs [20].To this end, related works did not discuss the underlying reasons for the API evolution challenges and how to solve them sustainably, and change extraction approaches did not sufficiently target REST API changes.

RESEARCH QUESTIONS
We hypothesize that companies currently do not follow a sustainable microservice API evolution strategy.Further, we hypothesize that automated approaches for reporting structural and behavioral breaking API changes and notifying affected consumers would improve the change notification accuracy and reliability, mitigating human errors and delays with currently used strategies.Hence, we define the following research questions (RQs) to address: RQ1 Which strategies do developers follow to introduce and communicate API changes in loosely coupled systems, and what challenges do they face?RQ2 How can structural and behavioral changes of REST APIs be automatically extracted based on the source code?RQ3 How can relevant REST API changes be automatically communicated to consumer teams affected by these changes?

METHODOLOGY AND CONTRIBUTIONS
This section focuses on the research methods to answer the three RQs and the expected contributions.

Catalogue of API evolution strategies and challenges
To answer RQ1, we conducted semi-structured interviews [17] with developers and architects working on developing loosely coupled systems exposing an API, e.g., REST or event-driven, for at least one year.We applied open coding [9] used in grounded theory [14] and analyzed the interview transcripts iteratively.The codebook began to stabilize after 12 interviews, and we did not encounter any new categories for the last three interviews, indicating theoretical saturation [33].To mitigate threats to validity, we applied investigator triangulation [5] and member checking [28].We are currently in the process of publishing the results.
In summary [22], we conducted 17 interviews with participants from 11 companies.We identified six microservice API evolution strategies in practice, which we formulated as best practices to follow.Further, we identified six challenges and formulated them as pitfalls when handling API evolution.We drew the relationships between the strategies and challenges and discovered two important problems in the evolution process of publicly accessible APIs, i.e., REST APIs.First, development teams informally communicate REST API changes to affected teams, e.g., through verbal meetings, instant messaging channels, or e-mails, and collaborate closely to avoid or quickly repair unnoticed breaking changes.Second, doubtful or unresponsive consumer teams not migrating to a new REST API version force the provider team to support and maintain outdated versions, continuously increasing technical debt and degrading the overall API design.Based on these two problems, we propose two research directions: a) automating the impact analysis of source code changes on the REST API and b) automating the communication of REST API changes to affected teams.

Automated REST API change extraction from source code changes
To answer RQ2, we plan to propose an approach for extracting REST API changes from source code changes and to evaluate the prototype on Java Spring Boot1 open-source projects.We selected Java Spring Boot as one of the most popular microservice frameworks offering high flexibility in defining REST APIs [8,10].Our approach will consist of a change extractor and a change classifier.The change extractor analyzes the OpenAPI specifications2 of two REST API versions, generates suitable intermediate models introduced by us, and extracts the structural changes between them using GumTree [11], a popular approach for differencing tree structures [13,19,25].To extract the behavioral changes, we will experiment with the limited behavioral description provided by the OpenAPI specification and supplement it with a lightweight control and data flow analysis of the source code, such as introduced by Hanam et al. [18].We will also evaluate the feasibility of describing detailed behavioral information by extending the OpenAPI specification.Next, we will investigate existing REST API change type taxonomies [30,34], currently focusing on structural changes, and will extend them with novel behavioral changes.The change classifier then applies our extended taxonomy on the extracted structural and behavioral changes and finally outputs the corresponding REST API changes.We plan experiments on existing open-source projects to evaluate our approach's applicability and change classification accuracy.Further, we will compare our approach to the state-ofthe-art openapi-diff 3 , which identifies structural changes between OpenAPI specifications, but does not detect behavioral changes.

Automated communication of REST API changes to affected teams
To answer RQ3, we will build on the findings from our study in Section 4.1 by automating REST API change notifications to affected teams via online collaboration platforms, such as GitHub and GitLab.We will provide GitHub Actions 4 and GitLab CI/CD templates 5 to automatically send REST API change notifications generated from source code with our approach in Section 4.2.We will explore two means of registering for notifications.Consumer teams can enter the relevant service repositories via a web interface or add a dependency file referencing them to their own repository.We will also investigate methods to register private repositories containing proprietary services.Further, we expect that different technical roles prefer different levels of detail and timeliness of notifications.Hence, we will experiment with different granularities and criticality levels of changes and different notification triggers, e.g., opened pull request, completed merge, or passed deployment pipeline.We plan to evaluate our approach with a user study [31].We will assess its reliability by comparing the timeliness and completeness of our REST API change notifications to manual notifications, and its helpfulness, as done by Frick et al. [13], in terms of understandability compared to manually written or verbal notifications.
In conclusion, the final prototype will incorporate the study insights from Section 4.1, the REST API change extraction approach from Section 4.2, and the automated notification approach.The user study will evaluate our hypothesis that such a tool improves the API change notification accuracy and reliability.
195 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.