Deep Cooperation in ISAC System: Resource, Node and Infrastructure Perspectives

Abstract

With the mobile communication system evolving into 6th-generation (6G), the Internet of Everything (IoE) is becoming reality, which connects human, big data and intelligent machines to support the intelligent decision making, reconfiguring the traditional industries and human life. The applications of IoE require not only pure communication capability, but also high-accuracy and large-scale sensing capability. With the emerging integrated sensing and communication (ISAC) technique, exploiting the mobile communication system with multi-domain resources, multiple network elements, and large-scale infrastructures to realize cooperative sensing is a crucial approach to satisfy the requirements of high-accuracy and large-scale sensing in IoE. In this article, the deep cooperation in ISAC system including three perspectives is investigated. In the microscopic perspective, namely, within a single node, the cooperation at the resource-level is performed to improve sensing accuracy by fusing the sensing information carried in the time-frequency-space-code multi-domain resources. In the mesoscopic perspective, the sensing accuracy could be improved through the cooperation of multiple nodes including Base Station (BS), User Equipment (UE), and Reconfigurable Intelligence Surface (RIS), etc. In the macroscopic perspective, the massive number of infrastructures from the same operator or different operators could perform cooperative sensing to extend the sensing coverage and improve the sensing continuity. This article may provide a deep and comprehensive view on the cooperative sensing in ISAC system to enhance the performance of sensing, supporting the applications of IoE.

I Introduction

Internet of Things, Artificial Intelligence (AI), big data, and automation technologies are reconfiguring traditional industries, which are undergoing digital, networked, and intelligent transformation, opening the era of Internet of Everything (IoE). The elements in the applications of IoE are transferring from pure human to the symbiosis of human, big data, intelligent machines, and massive number of sensors, where the information processing includes sensing, communication, computation and decision-making, realizing the coupling of digital and physical spaces. These applications urgently need to be supported by new information infrastructures with the integration of sensing and communication. With the development of Integrated Sensing and Communication (ISAC) technique [1], the mobile communication system, as the crucial infrastructure to support the emerging IoE, is constantly breaking through the pure communication function and integrating radar sensing function. Notably, International Telecommunication Union (ITU) has identified ISAC as one of the scenarios of the 6th-generation (6G) mobile communication system [2].

The mobile communication system realizes radar sensing with ISAC technology by analyzing the echo of ISAC signal [3], thereby realizing target localization, environment reconstruction, etc. ISAC not only enhances the utilization of spectrum and hardware resources, but also realizes the coupling twin of digital and physical spaces and the mutual benefit of communication and sensing functions [1]. The Millimeter-wave (mmWave), Terahertz (THz), and massive Multiple Input Multiple Output (MIMO) technologies in mobile communication system are develo** rapidly, which guarantee the feasibility of ISAC technology [4]. However, in the applications of IoE such as the sensing of vehicles and Unmanned Aerial Vehicles (UAVs), the fading and path occlusion of ISAC signals degrade the signal quality drastically, which brings challenges to achieve high-accuracy, large-coverage, and continuous sensing. To this end, it is urgent to explore cooperative sensing approaches in the ISAC system to enhance sensing performance.

Cooperative sensing in ISAC system could be realized from the following three comprehensive perspectives according to the scope of cooperation.

1. In the microscopic perspective, i.e., within a single node, there still exists resource-level cooperation to improve sensing accuracy by fusing the sensing information carried in the time-frequency-space-code multi-domain resources.

2. In the mesoscopic perspective, the sensing accuracy could be improved through the cooperation of multiple nodes including Base Station (BS), User Equipment (UE), and Reconfigurable Intelligence Surface (RIS), etc.

3. In the macroscopic perspective, the large number of infrastructures from one operator or even multiple operators could be applied to extend the sensing area and improve the continuity of sensing.

However, realizing the deep cooperation of ISAC system in the perspectives of resource, node and infrastructure still faces the following challenges.

1. Resource-level Cooperation: The inconsistency of physical-layer parameters on fragmented multi-domain resources brings challenges to sensing information fusion. For example, the high and low frequency bands with different subcarrier spacings bring difficulty in the fusion of the sensing information in collected at different frequency bands.

2. Node-level Cooperation: The fusion of the sensing information from multiple nodes with non-synchronization or low-accuracy synchronization level in space, frequency and time domains is challenging.

3. Infrastructure-level Cooperation: The fast and continuous handover among multiple BSs from one operator or different operators is challenging.

Facing the above challenges, there are some related studies. Zhang et al. in [5] initially proposed the concept of Perceptive Mobile Network (PMN) and proposed a Remote Radio Units (RRUs) cooperative sensing scheme under the architecture of Centralized Radio Access Network (C-RAN) to improve sensing accuracy. In [6], Ji et al. proposed a broad concept of cooperative sensing in ISAC system, including multi-static, multi-band, and multi-source cooperation. Tong et al. in [7] further proposed the concept, algorithm and demonstration of multi-view cooperative sensing in ISAC system. [8, 9] proposed sensing information fusion algorithms with multi-BS cooperative sensing. Overall, the existing studies on cooperative sensing in ISAC system mainly focus on multiple BSs cooperative sensing. However, even in terms of node-level cooperative sensing, such as our previous work [9], the cooperation between BS and UE and the cooperation between BS and RIS are rarely studied. Besides, the multi-BS cooperation is not structurally classified. This article aims to provide a deep, comprehensive and concise view on cooperative sensing in ISAC system, namely, the cooperation in the perspectives of resource, node and infrastructure. The main contributions of this article are as follows.

1. In terms of resource-level cooperation, the echo signals in multiple antennas, multiple time slots or frames, multiple frequency bands and multiple code words could be fused to improve sensing accuracy. Meanwhile, the sensing information in multi-domain resources could be fused simultaneously to further enhance the sensing accuracy.

2. In terms of node-level cooperation, we have provided a concise classification and structural relation for multi-node cooperative sensing. Multiple macro BSs or multiple micro BSs could perform cooperative sensing. Since the micro BS is close to target and could be deployed on the high frequency band, the sensing accuracy of micro BS is higher compared with macro BS. Hence, multiple micro BSs cooperative sensing is a preferred scheme. However, multiple micro BSs have smaller overlapped area. When the target moves outside of the overlapped area of multiple micro BSs. The cooperation between macro and micro BSs is required. Meanwhile, the cooperation schemes among BS, UE and RIS are also studied in this article. Multi-node cooperation effectively improves sensing accuracy and continuity.

3. In terms of infrastructure-level cooperative sensing, the infrastructures from one or multiple operators, across the air, ground and space, could be applied to realize seamless target sensing. The network architecture supporting cooperative sensing, the moving target detection and tracking methods, and the large-scale sensing assisted digital twin construction, are investigated in the infrastructure-level cooperation.

It is worth noting that we have a related publication on cooperative sensing in ISAC system [10]. The research in [10] focuses mainly on the framework and key enabling technologies of multi-BS cooperative sensing, including unified performance metrics, interference management, ISAC signal design and optimization, and signal processing algorithms. However, [10] does not provide a structural classification of multi-BS cooperative sensing. Besides, a deep cooperation paradigm in ISAC system including resource-level cooperation, BS-UE cooperation, BS-RIS cooperation and infrastructure-level cooperation was not studied in [10].

Figure 1: Resource-level cooperation in multi-domain resources.

II Basic Concepts in ISAC System

In this section, the types of sensing, the performance metrics of sensing, the scenarios and requirements in ISAC system are briefly introduced.

II-A Types of Sensing

The radar sensing in ISAC system mainly includes active sensing and passive sensing. In active sensing, the transceiver detects the target by receiving the echo signal reflected by the target. In passive sensing, the receiver detects the target by receiving the signal emitted from the target or the echo signal from other transmitters reflected by the target [5]. The BS with sensing function realizes the coupling of digital space and physical space, which is a unified information infrastructure supporting IoE.

II-B Performance Metrics of Sensing

• Accuracy of detection: The accuracy of detection is measured by probabilities of detection and false alarm.

• Accuracy of parameter estimation: The accuracy of the estimation of distance and velocity of target is measured by Mean Square Error (MSE), Rooted MSE (RMSE) and Normalized MSE (NMSE), which measure the deviation between the estimation value and the real value.

• Sensing area: Sensing area measures the range of radar sensing, which is influenced by the fading of radar signal and the Radar Cross-Section (RCS) of target.

• Sensing continuity: Sensing continuity measures the performance of target tracking. When detecting the target in $n$ continuous time instants, the fraction of the time instants with sensing accuracy higher than a threshold to $n$ is defined as the sensing continuity.

II-C Scenarios and Requirements

• Cooperative Sensing of Vehicles: In the application of intelligent transportation, the lane-level sensing of vehicles needs to be realized. Thus, cooperative sensing is essential in this scenario to improve the sensing accuracy.

• Cooperative Sensing of UAVs: In the scenario of UAV sensing in the urban areas, the cooperative sensing is required to realize continuous sensing of UAVs with high maneuverability due to the blockage of buildings. Since the RCS of UAV is small, the cooperative sensing is required to improve the accuracy of small target sensing.

III Resource-level Cooperation

As shown in Fig. 1, within a single node, the resource-level cooperation could be realized to improve the sensing accuracy.

III-A Cooperation in Space-domain Resource

Resource cooperation in space-domain mainly applies MIMO to exploit the multi-path signal propagation. As shown in Fig. 1, the signals in multiple antennas could be fused to enhance the sensing performance, which is further classified into the sensing information fusion with uniform and non-uniform antenna array.

III-A1 Sensing information fusion over uniform antenna array

In single-node sensing scenario, the angle, distance and velocity of target need to be estimated for localization and trajectory prediction. With a uniform antenna array, the number of antennas is affecting the resolution of angle estimation for the target, which is realized using Multiple Signal Classification (MUSIC) method, Estimating Signal Parameter via Rotational Invariance Techniques (ESPRIT) method, etc [4]. On the other hand, the accuracy and resolution of distance estimation are related to bandwidth and Signal-to-Noise Ratio (SNR) of echo signal, and the accuracy and resolution of velocity estimation are related to sensing time and SNR. Since multiple antennas occupy the same time-frequency resources, the SNR of echo signal can be improved by fusing the sensing information on multi-antenna, further improving the performance of distance and velocity estimation, which is realized by phase compensation and correlation accumulation. Meanwhile, the time-frequency resources are extended by utilizing the data of multi-antenna, i.e., enlarging the bandwidth and extending sensing time, which is accomplished by splicing the data in each antenna with the challenge of determining the initial phase of the data in each antenna to perform splicing operation.

III-A2 Sensing information fusion over non-uniform antenna array

For non-uniform antenna arrays, such as sparse antenna arrays, optimization algorithms can be employed to achieve the performance of a full antenna array using only a portion of antenna resources. For the estimation of target’s angle, the virtual antenna array is obtained by hybrid cross-multiplication among antennas, so that the angle can be estimated using MUSIC method. In contrast to uniform antennas, hybrid cross-multiplication leads to a decrease in the SNR on multi-antenna, which reduces the SNR gain in the sensing information fusion of multi-antenna and further degrades the sensing performance. Hence, the trade-off between improving sensing performance of multi-antenna fusion and saving space-domain resources exists with the non-uniform antenna array.

III-B Cooperation in Time-domain Resource

The cooperation in time-domain, as shown in Fig. 1, is mainly used to improve the SNR of echo signal by time accumulation, further improving the estimation accuracy of distance and velocity. Assuming that ISAC system adopts Orthogonal Frequency Division Multiplexing (OFDM) signal, the multiple frames can be used for coherent or non-coherent accumulation to improve the SNR of echo signal and further improve the estimation accuracy of distance. Similarly, multiple frames can be used to improve the resolution of velocity estimation [11].

III-C Cooperation in Frequency-domain Resource

Figure 2: Multi-BS cooperation.

Cooperation in frequency-domain mainly involves the fusion of the multiple reference signals and the signal on multiple frequency bands, as shown in Fig. 1.

III-C1 Fusion over multiple reference signals

Reference signals, also known as pilot signals, can be applied in radar sensing [11], which include downlink and uplink reference signals. Downlink reference signals could be applied in downlink active or passive sensing. The uplink reference signals including the Demodulation Reference Signal (DMRS), Sounding Reference Signal (SRS), and so on, could be applied in uplink sensing. Since different reference signals occupy different resources in frequency domain, the various reference signals can be cooperatively used in radar sensing, achieving higher sensing performance than single reference signal. Since multiple reference signals do not fill the complete time-frequency resource blocks, the challenge of this research is the sidelobes deterioration caused by non-continuous resources in time-frequency domains. The sparsity of the targets when map** the data in the time-frequency domain to the Doppler-delay domain brings an opportunity for the application of Compressed Sensing (CS) technique into ISAC signal processing. The CS technique can be applied to recover sensing information at empty position in the time-frequency resource blocks. Therefore, the CS-based multiple reference signals cooperative sensing has significant performance improvement.

III-C2 Fusion over high and low frequency bands

The mmWave and THz frequency bands are gradually applied in future mobile communication system. Meanwhile, the sub-6 GHz frequency bands are essential in enhancing the coverage of mobile communication system. With the technique of Carrier Aggregation (CA), the high and low-frequency bands are combined to improve the performance of communication. Similarly, the cooperation of high and low-frequency bands can also improve the sensing performance, where the challenge is the inconsistency of physical layer parameters in the high and low-frequency bands during sensing information fusion. For example, when adopting OFDM as the ISAC signal, the different subcarrier spacings in the high and low-frequency bands brings challenge for sensing information fusion. This problem can be solved by reorganizing the channel information matrices of high and low-frequency bands to match the parameters of the corresponding positional elements [12]. In addition to the cooperation of high and low-frequency bands, there are additional cases of cooperation in frequency-domain, such as the fusion of the signals over the fragmented spectrum bands or unlicensed frequency bands.

III-D Cooperation in Code-domain Resource

With the cooperation in code-domain, communication and radar sensing functions occupy the same resources in time-frequency domains, which are differentiated by the code words, thereby obtaining higher resolution of distance and velocity estimation [13] because the resources in time-frequency domains are not diminishing with the code-division method compared with the traditional time-division or frequency-division method. In addition, the sensing information on different code words in the same time-frequency domains can be extracted and further fused to obtain high sensing accuracy.

III-E Cooperation in Multi-domain Resources

The cooperative sensing in multi-domain fuses the sensing information in multi-domain resources to achieve a higher sensing accuracy than the sensing method using single-domain resource. For example, using multiple antennas, multiple signals on different antennas may overlap in the time-frequency domains, so that different code words are applied to distinguish the multiple signals. Then, the multiple signals with different code words could be extracted and fused, thereby improving the sensing accuracy.

In urban environments, the sensing information in multi-path could be applied to improve the sensing accuracy with multi-domain resource cooperation. The echo signals in multi-path could be separated through multi-domain resource cooperation, which is further fused to realize high-precision target localization, high-continuity target tracking, and high-accuracy environmental reconstruction.

IV Node-level Cooperation

Figure 3: BS-UE cooperation.

In node-level cooperation, multiple nodes, including BS, UE, RIS, could cooperate to improve the sensing accuracy.

IV-A Multi-BS Cooperation

The cooperation between multiple BSs is categorized according to the coverage area of sensing, namely multiple micro BSs cooperative sensing, macro-micro BSs cooperative sensing, and multiple macro BSs cooperative sensing, where the sensing area is increasing in these three categories.

Fig. 2(a) shows multi-BS cooperative active sensing, i.e., when the target is located in the overlapped coverage area of multiple BSs, each BS performs sensing separately, and the sensing information fusion is performed by the fusion center, which could be one BS or the Mobile Edge Computing Server (MECS) [10]. Fig. 2(b) shows multi-BS cooperative passive sensing, i.e., the passive BS receives the echo signals of other BSs reflected by the target and performs sensing information fusion. Fig. 2(c) shows multi-BS cooperative active and passive sensing, i.e., the BS can both perform active sensing and receive the echo signals of other BSs reflected by the target in passive sensing, and finally fuse the sensing information of active sensing and passive sensing. The above three types of cooperative sensing requires that the target is located in the overlapped coverage area of multiple BSs. Since the sensing area of micro BS is smaller than that of macro BS, when the target moves out of the sensing area of micro BS, the macro-micro BSs cooperative sensing is performed, as shown in Fig. 2(d), which is investigated in Section IV-B.

IV-B Macro-micro BSs Cooperation

Compared with macro BS, micro BS has higher frequency band and smaller transmit power. Hence, the sensing area of micro BS is smaller than that of macro BS. When the target moves out of the overlapped sensing area of multiple micro BSs as shown in Fig. 2(d), the macro BS and micro BS could cooperate to improve the sensing accuracy and continuity, where the main challenge is the fusion of the sensing information from the micro BS working on high-frequency band and the macro BS working on low frequency band. To address the above challenge, the channel information matrices from macro and micro BSs are adjusted and fused with low synchronization accuracy, which achieves higher sensing accuracy compared with data-level sensing information fusion.

IV-C BS-UE Cooperation

The BS and UE cooperative sensing can be classified according to the sensing range. Generally, the downlink sensing range is smaller than the uplink communication range, and the uplink communication range is smaller than the downlink communication range, as shown in Fig. 3. When the target is located within the downlink sensing coverage, such as the location A in Fig. 3, the BS can simultaneously perform downlink sensing by receiving the echo signal of BS and uplink sensing by receiving the uplink reflected signal from UE. Then, the echo signals from downlink and uplink sensing are fused in the BS, which is the cooperative downlink and uplink sensing [14]. When the target is outside of the downlink sensing coverage and within the uplink communication coverage, such as the location B in Fig. 3, multiple UEs detect the target and upload the sensing information to the BS for sensing information fusion. When the target is outside of the uplink communication coverage and within the downlink communication coverage, such as the location C in Fig. 3, multiple UEs detect the target and fuse the sensing information by themselves, with the guidance of BS in resource allocation.

IV-D BS-RIS Cooperation

Figure 4: Infrastructure-level cooperation.

In practice, there may be blockage caused by the buildings between BS and target, which results in the absence of Line-of-Sight (LOS) links between BS and target and further results in miss detection. In this scenario, the BS-RIS cooperative sensing could be applied to detect the target [15]. As shown in Fig. 4, the BS transmits ISAC signal, which is reflected by the RIS and reaches the target. Then, the echo signal follows the opposite path and reaches the BS. Since the BS and the RIS are fixed, it is feasible to perform radar signal processing to estimate the distance and velocity of target.

V Infrastructure-level Cooperation

In the application of smart city, there are massive number of targets to be detected and tracked. For example, in the scenario of UAV sensing, due to the high maneuverability of UAV, the infrastructure-level cooperation is required to accurately detect and track UAV. Generally, the following techniques are required in the infrastructure-level cooperation.

V-1 Network architecture supporting cooperative sensing

In order to support ISAC enabled cooperative sensing, the network architecture need to be designed. The network elements include Remote Radio Unit (RRU), Building Base Band Unite (BBU), MECS, and core network, etc. The sensing information fusion centers are deployed in the MECS for node-level cooperative sensing and deployed in the core network for infrastructure-level cooperative sensing. Multiple RRUs detect target with multi-domain resources and fuse the sensing information in the MECS. The interfaces between BSs, information processing procedures and signaling interaction procedures among the network elements need to be designed to support cooperative sensing. The fusion center in the core network fuse the large amount of sensing information to build a full view of the digital twin of physical space.

In order to realize large-scale and seamless sensing, the Heterogeneous Networks (HetNets), including the BSs from different operators and the Access Points (APs) using IEEE 802 techniques, could cooperate to sense target and environment. In this case, the network architecture supporting HetNets cooperative sensing needs to be designed. A new plane to fuse the sensing information from HetNets needs to be constructed to support network interoperability. Then, the procedures of sensing information fusion with non-synchronization among HetNets faces great challenges. Hence, the space and time calibration among HetNets is necessary.

V-2 Moving target detection and tracking

When tracking the target with high maneuverability, the infrastructures from the space-air-ground integrated networks, as well as the infrastructures from single or multiple operators, could cooperatively detect and track the target. As shown in Fig. 4, multi-BS cooperation, as well as the cooperation between BS and UAV, could be performed to detect the vehicle and UAV with high maneuverability. Meanwhile, multi-satellite cooperation could be performed to detect the aircraft in a large-scale air space. In this scenario, the handover of target sensing by multiple BSs needs to be studied. The handover in target sensing studies the switching of multiple BSs belonging to the same operator or different operators to realize the continuous sensing of moving target due to the limited coverage of single BS. Besides, the detection methods of moving small target, and the machine learning techniques for target tracking need to be studied.

V-3 Large-scale sensing assisted digital twin construction

In the scenario of intelligent transportation, the massive amount of sensing information needs to be structurally stored and processed to build a digital twin to control the traffic flow. In this case, the construction of digital twin with large-scale sensing information fusion needs to be studied, which connects the digital and physical spaces. Besides, the close loop information flow including the procedures of sensing, communication, computing and control needs to be modeled and analyzed, whose performance metrics are further applied to optimize the control of physical space, such as the traffic flow control in the scenario of intelligent transportation.

VI Performance Evaluation

In this section, we provide the simulation results of resource-level and node-level cooperative sensing, verifying the advantages of cooperative sensing. Among the performance metrics mentioned in section II-B, the performance metric of RMSE is commonly used to characterize sensing accuracy.

VI-A Resource-level Cooperation

According to the 3rd Generation Partnership Project (3GPP) 38.211 standard, the high-frequency band with carrier frequency of 24 GHz and subcarrier spacing of 120 kHz, and the low-frequency band with carrier frequency of 5.9 GHz and subcarrier spacing of 30 kHz, are applied in the high and low frequency bands cooperative sensing [12]. As illustrated in Fig. 5, it is obvious that the RMSEs of distance and velocity estimation are lower and the convergence speed is faster with multiple frequency bands cooperative sensing than those results with single frequency band sensing. The reason is that the time-frequency resources are extended with multiple frequency bands cooperation.

(a) Range estimation.

(b) Velocity estimation.

Figure 5: RMSEs of multi-band sensing and single-band sensing.

(a) Range estimation.

(b) Velocity estimation.

(c) Localization performance with multi-BS cooperative active and passive sensing.

Figure 6: RMSEs of multi-BSs cooperative sensing.

VI-B Node-level Cooperation

As shown in Fig. 6, the multi-BS cooperative UAV sensing is simulated to verify the performance of node-level cooperative sensing. The number of BSs is 4, the subcarrier spacing is 240 kHz, and the carrier frequency is 24 GHz. Multi-BS cooperative active sensing is compared with single-BS sensing in terms of the performance of localization and velocity estimation for UAV. As shown in Fig. 6(a) and 6(b), the multi-BS cooperative sensing achieves more accurate localization estimation than single-BS sensing. Multi-BS cooperative active sensing achieves significantly better performance in velocity estimation than single-BS sensing. The same simulation parameters are applied in multi-BS cooperative active and passive sensing, where 1000 times Monte Carlo simulations are performed to calculate the RMSE of location estimation. It is revealed that the localization performance with multi-BS cooperative active and passive sensing is much better compared with single-BS sensing, as shown in Fig. 6(c).

VII Conclusion

In order to realize high-accuracy, large-scale and continuous sensing in ISAC system, this article provides a deep and comprehensive view on the cooperative sensing in ISAC system, including resource-level cooperative sensing, node-level cooperative sensing, and infrastructure-level cooperative sensing. In the resource-level cooperation, the sensing information in time-frequency-space-code domains is fused to improve sensing accuracy. In node-level cooperation, multi-node including BS, UE, and RIS could perform cooperative sensing to fuse the sensing information from multiple nodes, extending the sensing coverage and improving the sensing accuracy. In infrastructure-level cooperation, the massive number of infrastructures perform cooperative sensing to realize continuous sensing. The research in this paper may provide a research guideline for cooperative sensing in ISAC system, promoting the applications of IoE with the connection of digital and physical spaces.