Smart radio access selection and slice allocation for differentiated traffic management over 6G heterogeneous networks

The upcoming 6G systems will represent a complete paradigm shift for global communications, claiming a three-dimensional (3D) ecosystem with terrestrial and non-terrestrial networks (TNs-NTNs) to improve coverage and capacity and enable advanced applications with strict quality of service (QoS) and quality of experience (QoE) requirements. Addressing the critical research verticals toward the envisioned 2030 will require a seamless unicast/multicast/broadcast convergence and a native softwarized, disaggregated, and intelligent Radio Access Network (RAN) conception. In such a context, the network slicing paradigm is an appealing feature for enhanced differentiated traffic management. This research aims to efficiently manage radio access selection and slice allocation based on Machine Learning (ML) techniques inserted in the O-RAN framework. The project focuses on finding the best combination of access network and network slices to fulfill multiple users' requests and optimize resource usage over a 6G heterogeneous environment. Moreover, it addresses a load-balancing strategy to improve network performance and avoid overloading. The proposed algorithm is adapted to diverse network conditions, numerous service constraints, and several user types with different priorities and mobility behaviors. The proposal is evaluated through network-level simulations, focusing on effectively utilizing network resources and maximizing the QoS/QoE metrics.


INTRODUCTION
The envisioned 6G era will represent a complete paradigm shift for global communications, merging the physical, digital, and virtual worlds through immersive human interaction [1].A significant use case will be immersive and advanced experiencesharing communications, including extended reality (XR), holographic, and three-dimensional (3D) video delivery.Then, 6G will enable extreme communication applications such as autonomous driving, telesurgery, mixing robotic technologies, flexible manufacturing, and seamless interaction with immersive applications.Such a wave of multimedia and experience delivery will align with an upcoming connected everything era.6G massive communication implies a hyperconnected resilient infrastructure with many different end devices.
Addressing the challenging research verticals shaping the 6G context will require a compelling mix of enabling radio access technologies (RATs) such as the Multicast/Broadcast Services (MBS) support and softwarized/intelligent conceptions.One of the prominent features of future networks is the utilization of high-frequency bands, such as the millimeter-wave (mmWave) and terahertz (THz) bands.Path losses, signal penetration, and The current study of the International Mobile Telecommunications (IMT)-2030 group positions the enhanced ubiquitous coverage as one of the prominent use cases of 6G [5].In such context, non-terrestrial networks (NTNs) are considered essential for the success of 6G to provide true "global" coverage, overcoming the infrastructure limitations of current terrestrial networks (TNs) [4].Consequently, the cooperation among TNs, unmanned aerial vehicles (UAVs), high-altitude platforms (HAPs), and satellite constellations will conform to the 3D ecosystem of future wireless networks [6,7].
Managing this ultra-dense heterogeneous environment requires dynamic RAN deployment, utilizing a virtualized, disaggregated, and reconfigurable framework.In the last few years, the architecture proposed by the O-RAN Alliance has gained momentum [8].In this framework, the BSs are connected to RAN intelligent controllers (RICs) through open interfaces to perform control actions and policies (e.g., network selection, load balancing, and scheduling policies).The network selection is a critical process since multiple types of users request numerous services with tight QoS requirements simultaneously over a heterogeneous infrastructure.Multiple users compete for finite resources; therefore, an effective strategy is necessary to improve user satisfaction and optimize resource utilization [9].In addition, load balancing is crucial to ensure adequate on-demand resource management and avoid saturation.
On the other hand, 6G networks must incorporate Machine Learning (ML) solutions as a prominent feature to manage highly dynamic environments and make proactive decisions adapting the network to multiple scenarios.Specifically, Deep Reinforcement Learning (DRL) has been presented in recent literature as a valuable prospect thanks to the trial-and-error learning process [3].Furthermore, the Federated Learning (FL) paradigm and DRL combination (F-DRL) is a suitable solution, benefiting from collaborative ML training while preserving data privacy and considerably reducing communication overhead compared with traditional ML approaches (e.g., centralized solutions) [10].

RESEARCH OBJECTIVES AND SCOPE
Bearing the previous research overview, we intend to deal with the following identified challenges: (i) The TNs-NTNs cooperation to satisfy multiple service requests with an alwaysbest-connected paradigm; (ii) Integration of network selection and slice allocation tasks into the envisioned 6G O-RAN framework, satisfying critical requirements of numerous 6G advanced applications; (iii) Dynamic resource management based on network-human factors, combining QoS/QoE metrics; (iv) Unicast/multicast/broadcast shared delivery to improve network capacity and optimize resource utilization.
In such context, this Ph.D. research aims to dynamically manage RAN selection and slice allocation over the 6G ultradense heterogeneous environment.To reach the Ph.D. main goal and face the presented challenges, we define several specific objectives (SOs): SO-1: State-of-the-art analysis regarding the network selection and slice resource management solutions in 5G networks and SO-6: Redefine the optimization function of SO-3 based on a human-centric QoE approach, maximizing the Mean Opinion Score (MOS).SO-7: Validation of the proposal through rigorous networklevel simulations.Recreation of a heterogeneous TN-NTN environment with numerous service requests (e.g., AR, 360 0 video, IoT application), diverse user types, priorities, and mobility behaviors.Theoretical and empirical computational complexity and algorithm performance analysis regarding the state-of-the-art.Analysis of the unicast/multicast/broadcast service delivery's impact on the network performance.
Then, the research's main contributions can be summarized as follows: 1. We address the dynamic access network selection and slicing resource allocation process over a 6G heterogeneous environment to satisfy multiple concurrent user types with diverse mobility behaviors and service requirements.The algorithm decision relies on a combination of network-human factors to optimize QoS/QoE metrics.
2. The slicing paradigm supports the proposal to achieve differentiated unicast/multicast/broadcast service management, dynamically prioritize traffic under diverse network conditions (e.g., overloading), and meet the defined Service Level Agreement (SLA).
3. The proposal is based on F-DRL, which takes advantage of the 6G O-RAN framework, reduces communication overhead and computational complexity, and improves privacy compared to traditional ML approaches (e.g., centralized ML).
4. The solution employs CGT as a load-balancing mechanism during overload situations, facilitating resource adjustments to accommodate additional clients without abruptly compromising the active users' perception.
5. Comprehensive simulation results are obtained, recreating diverse 6G use cases like XR applications.The conducted tests are oriented to optimize resource utilization, considering multiple critical features such as throughput, delay, energy consumption, overloading, NS availability, SLA satisfaction, and MOS.

RELATED WORK
Montalban et al. [11] proposed a Multi-Attribute Decision-Making (MADM) solution to address multiple users' requests within a convergent broadband/broadcast architecture.However, Smart radio access selection and slice allocation for differentiated traffic management over 6G heterogeneous networks ACM MMSys'24 DS, April 2024, Bari, Italy this approach did not fully leverage the benefits of the NS paradigm to reallocate resources on demand, as demonstrated in [7], [12].Gonzalez et al. [12] employed MADM to manage the network selection process and attend to multiple service requests, dynamically adjusting NS resources over different traffic conditions.In [7], the authors focused on terrestrial-airborne cooperation employing MADM to serve various unicast/broadcast service requests effectively.The complexity of massive, dynamic, and heterogeneous 6G systems has rendered heuristic algorithms impractical [13].Consequently, recent papers have shifted their focus toward solving network selection and resource allocation tasks through ML.Although ML has a high cost of offline training, it can quickly make near-optimal decisions once trained.Moreover, ML solutions do not depend on accurate mathematical models, making them appealing for complex network scenarios [14].
Particularly, DRL increases attention due to its ability to deal with complex and dynamic environments [13].For example, in [15], the authors proposed a centralized Double Deep Q-Network (DDQN) algorithm aided by dynamic NS allocation to serve multiple service requests.Nevertheless, centralized schemes suffer scalability issues in real deployments, and only small networks can benefit from this strategy [9].The authors of [16] presented a distributed algorithm where each BS, based on its local information, shows its willingness to attend the service request.In general, distributed ML strategies reduce the overhead and complexity regarding centralized systems.However, the lack of a round-trip fashion between an aggregated unit and the agents limits the generated local models to only use the individual information without any benefit from peer data [10].In the case of [17], a DRL multi-agent solution is proposed, in which the BSs are the agents to reduce frequent handovers.The authors offered an effective combination of states from adjacent BSs, and their coefficients are jointly computed to improve results.Nevertheless, this solution does not inquire about privacy issues.
Recent studies have focused on F-DRL to overcome the limitations of centralized and distributed ML processes.The authors of [18] introduced a hybrid F-DRL approach for RAN slicing to enhance throughput and reduce handover costs.The authors of [19] employed a user-centric F-DRL algorithm to select the proper BS and resource blocks (RBs) for each requested service.However, this strategy raises concerns regarding the agents' selection during training, particularly in environments with diverse types of users, mobility patterns, and battery limitations.Additionally, the effectiveness of users' decisions relies on a higher-level entity that ultimately determines whether the user should accept the proposed BS association.This introduces complexity, mainly because multiple users compete for the same resources.
Rezazadeh et al. [20] presented an F-DRL algorithm inserted in the O-RAN architecture to exploit this disaggregated framework.According to the number of NSs at each BS, multiple parallel layers are deployed on the RIC to enhance local resource allocation.In [21], the authors presented an F-DRL solution where several BSs participate in training a common ML model to perform power allocation of their users.As a weak point, they did not consider the user association and handover procedures due to diverse mobility behaviors.
The papers [18]- [21] did not include either user priority differentiation or a heterogeneous RAN infrastructure (TN-NTN integration), which represents one of the future wireless networks' pillars [4], [22].The previously analyzed works, except [7] and [12], did not consider resource adjustments during overload situations.Furthermore, most of these papers, except [7] and [11], focus only on unicast capabilities without taking advantage of the MBS paradigm.
All the analyzed proposals are only based on QoS metrics without considering human factors.Differently, in [23], the authors proposed a heuristic algorithm for efficient multicast video delivery based on the QoS/QoE perspective.The QoE metric is obtained as a function of the quality video level and start-up delay.Zabetian et al. [24] employed QoS parameters and the results of subjective tests to train an ML model for estimating QoE.Both papers represent a good approach for resource allocation based on network-human factors.However, none of them considered the slicing paradigm for dynamic traffic management or user priority differentiation.Moreover, the analyzed use cases did not handle mmWave propagation over a heterogeneous environment or advanced applications with high data rates and low delay.
Compared with the previous works, this research proposal stands out as an integral solution inserted in the emerging O-RAN architecture.It addresses the challenge of dynamic network selection and slice allocation over a 6G heterogeneous environment to satisfy multiple users with diverse priorities, mobility patterns, and stringent unicast/multicast/broadcast service requirements.It is considered the cooperation among TNs-NTNs to improve network capacity and coverage.Furthermore, the proposed solution tackles overload situations, balancing resources among active users and accepting new clients while guaranteeing satisfactory QoS/QoE levels.Overall, the proposal provides a comprehensive solution that handles the complexities of network selection and slicing resource management in a heterogeneous environment employing mmWave bands.

CURRENT STATE AND FUTURE WORK
Fig. 1 shows the general system model of our current research state.The proposal aims to optimize the slicing resource utilization dynamically according to diverse network conditions, mobility patterns, user types, and service requests over a heterogeneous environment composed of terrestrial and airborne BSs.The ML solution is based on F-DRL and inserted in the novel O-RAN framework, where multiple ML local agents located in the BSs train an enhanced ML Global Model in the non-RT RIC.Then, the inference entities are in the BSs.The local agents' location and the F-DRL design must guarantee a timescale below 10 ms to enable future use cases with strict latency requirements, such as holographic, XR, and telesurgery applications.
The local agents collaborate to find the policy that maximizes the long-term QoS for all the users in the network.The ML model parameters obtained during training in the BSs ( ... ) are collected in the non-RT RIC (i.e., through the O1 interface) to compute an enhanced ML Global Model via the Federated Average (FedAvg) method.Next, the updated ML Global Model parameters ( ) are returned to the local agents, so knowledge earned by all the agents is leveraged for the individual action selection.This data exchange occurs in predefined intervals to reduce communication overhead.No user-related or safetycritical data is transmitted among BSs or to the non-RT RIC, enhancing privacy regarding centralized ML solutions.Furthermore, our proposal considers diverse service instances mapped into unicast and multicast NSs to exploit radio resources economically and efficiently and simultaneously serve many users.
To evaluate the performance of our proposal, we recreate a heterogeneous environment composed of several terrestrial BSs and UAVs acting as aerial BSs.The airborne nodes are opportunistically deployed in the grid to assist the terrestrial infrastructure during a temporal event by increasing coverage and network capacity [7,25].The path loss model applied for TNs is detailed in [26], whereas the model for UAV-BSs is found in [27].We consider numerous randomly distributed users with different mobility behaviors and priorities requesting one of the available multimedia services.
The above-described target scenario is recreated using an adhoc link-level simulator developed in Python [28].The simulator delivers the Signal-to-Interference Noise Ratio (SINR) and Channel Quality Indicator (CQI) values for all the links between users and BSs.Moreover, it provides the users' spatial distribution, throughput, delay, and energy consumption.The obtained data are inputs of the ML solution to optimize network resource allocation and guarantee an adequate BS selection process.
In future steps, we will consider:

CONCLUSIONS
The presented research project aims to dynamically manage RAN selection and slice allocation over the 6G ultra-dense heterogeneous environment.This short document presents the real-world problems and challenges that motivate our current Ph.D. research.We detail our work's objectives and analyze how our proposal can face some of the limitations in the state-of-theart.Finally, we describe the current state of our research, the general system model, and the future steps.
General system model based on the O-RAN architecture.
beyond.Definition of the principal theoretical concepts covered in the research.SO-2: Definition of the system model and problem formulation for addressing the network selection and slicing resource allocation tasks inserted into the 6G O-RAN framework.SO-3: Design a heuristic solution to conduct network selection and resource allocation tasks in a heterogeneous environment.Definition of the optimization function based on QoS metrics, such as throughput, delay, and energy consumption.SO-4: Design a cooperative game theory (CGT) solution to handle overload situations and optimize resource utilization.SO-5: Extend SO-3 by proposing an ML-based network selection and resource allocation algorithm (e.g., F-DRL).Definition of the training and inference entities' location, action, and state spaces, reward function, etc.