A LARGE IPX PROVIDER¶

In this section, we describe the IPX-P’s underlying infrastructure, its functions and services, as well as the dataset we collect. The latter enable us to present a detailed view of the IPX-P’s real-world operations.

The IPX-P we dissect is a Tier-1 Internet Service Provider operating one of the largest backbone networks world-wide. The carrier operates an IPX platform that runs on top of its vast Multiprotocol Label Switching (MPLS) transit network. 2 The IPX-P infrastructure integrates more than 100 PoPs in 40+ countries with a particularly strong presence in America and Europe. In terms of network connectivity, the IPX-P offers two types of interfaces, namely the IPX Access for clients (service providers) and the IPX Exchange for peering with other IPX-Ps. The main mobile peering points the IPX-P uses are those in Singapore, Ashburn and Amsterdam. By peering with other large Tier-1 carriers (via peering points or direct private interconnects), the IPX-P extends its footprint to regions where it does not own infrastructure.

The IPX-P we analyze implements in a flexible manner the model of multi-service connectivity solution. In other words, any customer can choose the optimal set of services that best fits their requirements. Specifically, the IPX-P provides a set of functions across all the different layers [29], including IPX Transport, SCCP Signaling, Diameter Signaling or GTP Signaling. Based on a tailored bundle of functions, the IPX-P then supports services such as Data Roaming, M2M and other roaming value added services (e.g., Steering of Roaming, welcome SMS, sponsored roaming, Data and Financial Clearing).

Overall, the IPX-P’s customers are active in 19 countries and include MNOs, IoT/M2M service providers and cloud service providers. The majority of customers are MNOs that rely on the IPX-P for enabling data roaming for their end-users (≈75% of the customer base). IoT service providers also rely on IPX-Ps for their operations [19], and we note that these type of players constitute ≈20% of the customer base of the IPX-P we analyze.Any customer for the data roaming service would implicitly need to use both the SCCP and Diameter signaling functions, as well as the corresponding GTP signaling function, in order to allow the different network elements from the home and visited networks to interact. Given that IoT service providers usually rely on the communication services of one (or several) MNOs, they also require access to the same type of functions that enable data roaming. However, due to the immense load they put on the IPX-P’s platform, IoT providers usually have access to separate slices of the roaming platform. We detail these services and the core functions that support them next

在本节中，我们描述了IPX-P的底层基础设施、其功能与服务，以及我们收集的数据集。后者使我们能够呈现该IPX-P真实世界运营情况的详细视图。

我们剖析的这家IPX-P是一家一级互联网服务提供商（Tier-1 Internet Service Provider），运营着全球规模最大的骨干网络之一。该运营商的IPX平台运行于其庞大的多协议标签交换（MPLS）转接网络之上。该IPX-P的基础设施集成了遍布40多个国家的100多个接入点（PoPs），在美洲和欧洲尤具规模。在网络连接性方面，该IPX-P提供两类接口，即面向客户（服务提供商）的IPX接入（IPX Access）和用于与其他IPX-Ps对等互联的IPX交换（IPX Exchange）。该IPX-P使用的主要移动对等互联点位于新加坡、阿什本和阿姆斯特丹。通过与其他大型一级运营商对等互联（经由对等互联点或直接私有互联），该IPX-P将其覆盖范围扩展至其未拥有基础设施的区域。

我们分析的这家IPX-P以灵活的方式实现了多服务连接解决方案模型。换言之，任何客户都可以选择最符合其需求的最优服务组合

具体而言，该IPX-P提供了一系列跨越所有不同层面的功能[29]，包括:

IPX传输（IPX Transport）、SCCP信令（SCCP Signaling）、Diameter信令（Diameter Signaling）或GTP信令（GTP Signaling）

基于定制化的功能组合，该IPX-P进而支持诸如数据漫游（Data Roaming）、M2M以及其他漫游增值服务（例如，漫游导向（Steering of Roaming）、欢迎短信、赞助漫游、数据与财务清算（Data and Financial Clearing））

总体而言，该IPX-P的客户活跃于19个国家，包括移动网络运营商（MNOs）、物联网（IoT）/M2M服务提供商以及云服务提供商。大部分客户是MNOs，它们依赖该IPX-P为其终端用户提供数据漫游服务（约占客户群的75%）。物联网服务提供商的运营也依赖于IPX-Ps [19]，我们注意到这类参与者约占我们分析的这家IPX-P客户群的20%。任何使用数据漫游服务的客户都必然需要同时使用SCCP和Diameter信令功能，以及相应的GTP信令功能，以便归属地网络和拜访地网络的各种网元能够进行交互。鉴于物联网服务提供商通常依赖一家（或多家）MNOs的通信服务，它们也需要接入支持数据漫游的同类功能。然而，由于它们给IPX-P平台带来了巨大负载，物联网提供商通常会接入漫游平台的独立切片（专用部分）。接下来，我们将详细介绍这些服务及其支持的核心功能。

M2M

M2M（Machine to Machine）是指机器与机器之间的信息交流与传递，通过网络及机器设备通讯的传递与链接达到信息共享的概念

在5G网络技术背景下，M2M服务是指连接具有低延迟和低带宽特性的机器，如机器人、传感器和物联网设备

IPX-P Infrastructure and Monitoring¶

We monitor the IPX-P infrastructure that supports three core functions – SCCP Signaling, Diameter Signaling, GTP signaling (for the different RATs) – that enable two main services, namely Data Roaming and M2M service.

We capture in Figure 2 a schematic view on the manner in which we capture these corresponding datasets. We rely on a commercial software solution for capturing and analyzing in real time the raw signaling traffic, which we mirror from the signaling routers to a central collection point. In that central location, the commercial software re-builds the signaling dialogues between different core network elements in the visited and the home MNOs. We monitor for two representative periods before and during the COVID-19 emergency, from December 1st to December 14th 2019 and from July 10th to July 24th 2020, respectively. We integrate these periods to provide a longitudinal analysis of operations in the platform, and also control for any potential impact the mobility restriction to tackle the COVID-19 emergency may have had on the IPX-P’s operations. However, we mention that a deep analysis on the impact of measures nations imposed to tackle the COVID-19 pandemic is outside the scope of our work. Table 1 summarizes the datasets we use to characterize the operations of an IPX-P with a large international footprint.

SCCP Signaling: This function provides access to the IPX-P’s SS7 signaling network, satisfying the 2G/3G interconnection needs for international roaming of MNOs. The SCCP Signaling network of this particular IPX-P has a redundant configuration with four international Signaling Transfer Points (STPs) located in North America (Miami, Puerto Rico) and Europe (Frankfurt, Madrid).

To capture clients’ activity across this signaling platform, we monitor the Mobile Application Protocol (MAP) protocol, which supports end-user mobility and allows major network elements (e.g., the Home Location Registry (HLR), Visiting Location Registry (VLR) or the Mobile Switching Center (MSC)) use to communicate. Figure 2 shows that by sampling the traffic from the SCCP Signaling platform and mirroring it to a central location, we are able to reconstruct the SCCP dialogues between different network elements, and build the dataset we use in this paper. We use a commercial solution for the raw data processing towards rebuilding the SCCP records that correspond to procedures devices in international roaming trigger. We collect traffic corresponding to the following procedures of each device belonging to one of the IPX-P’s clients (outbound roaming) or to foreign devices that connect to the network of one of the IPX-P’s clients (inbound roaming): i) location management (update location, cancel location, purge mobile device); ii) authentication and security (send authentication information); iii) fault recovery.

Diameter Signaling: This function provides the Diameter signaling capabilities necessary to enable 4G roaming for customers. The infrastructure of this particular IPX-P includes four Diameter Routing Agents (DRAs) meant to forward Diameter messages and simplify interworking between different network elements. It is application-unaware and does not inspect the messages it receives. The service also integrates Diameter Proxy Agents (DPAs), which include the functionality of the DRAs and can additionally inspect and route Diameter messages based on different parameters. Finally, by leveraging the Hosted Diameter Edge Agent (DEA) service, the IPX-P offers a infrastructure-as-a-service functionality to help operators expedite the launch of LTE roaming services. Thus, operators can use the dedicated customer virtual DEA from the IPX-P instead of deploying their own infrastructure. The LTE Diameter service integrates other value added services, including Welcome SMS, Steering of Roaming or Sponsored Roaming.

To monitor the activity of the IPX-P’s customers, we monitor traffic across the geo-redundant signaling network with four DRAs located two in Europe (Frankfurt, Madrid) and two in North America (Miami, Boca Raton). The approach is similar to the case of SCCP Signaling we described above (Fig. 2). We collect traffic corresponding to events including Diameter Transactions.

Data Roaming: This service enables the IPX-P to connect MNOs with foreign roaming partners, to ensure the data transport required for data roaming in 2G/3G (Gn, Gp interfaces) and LTE (S8 interface). The data roaming service relies on the GTP function to build and manage tunnels between roaming partners, to transport data to and from end-users. Note that the service requires the use of the SCCP and the Diameter signaling functions.

For this paper, we collect statistics regarding the tunnels between the Serving GPRS Support Node (SGSN) and Gateway GPRS Support Node (GGSN) nodes for 2G/3G, and between Serving Gateway (SGW) and Packet Data Network Gateway (PGW) for LTE. The IPX-P we study deploys a commercial software solution for monitoring, which centralizes large amounts of data from the different network elements (namely, the SCCP, Diameter and GTP signaling points) that are part of the infrastructure they operate. We capture the Create/Delete Packet Data Protocol (PDP) context procedures that the devices trigger before/after a data communication, as well as metrics about the data sessions. Specifically, the monitoring solution generates one record for the Create Session Request/Response exchange and retains basic information, such as the tunnel ID. Additionally, the monitoring solution generates a record when a data session is completed, which captures statistics for the whole session, such as the total amount of bytes transferred or the RTT. Because of the high amount of traffic and processing that collecting and generating these statistics implies, we only collect this dataset for the inbound and outbound roamers for the IPX-P’s customers connecting to PoPs in only a few selected countries (i.e., Spain, US, Brazil, Argentina, Colombia, Peru, Costa Rica, Uruguay, Ecuador).

M2M Service: An M2M platform operating on top of the IPX-P allows to avoid the cost of establishing technical and commercial relationships with every local operator, and can offer more stable connectivity/coverage services to IoT providers. By leveraging data roaming and the basic functions this requires (e.g., SCCP signaling, Diameter signaling, IPX/GRX carrier), an M2M platform can directed all traffic from its IoT devices to a single home country, no matter where the device is located in the world. M2M platforms are being leveraged by a growing number of industries, from health to automotive and logistics.

We monitor the activity of one specific M2M platform that relies on a Spanish MNO and on the IPX-P we analyze to support its business. We separate from the above-mentioned signaling and data roaming datasets only the traffic corresponding to the IoT devices this M2M platform operates. For this, we use the unique identifiers (i.e., encrypted Mobile Station International Subscriber Directory Number (MSISDN)) assigned to each device of the M2M platform. This allows us to capture the performance of the IPX-P solution from the point of "things" using the same system.

研究对象：

监控IPX-P基础设施，该基础设施支持三个核心功能（SCCP信令、Diameter信令、GTP信令）和两项主要服务（数据漫游和M2M服务）

(1) 数据收集方法：

使用商业软件实时捕获和分析原始信令流量
将信令流量从路由器镜像到中央收集点
在中央位置重建不同核心网元素间的信令对话

(2) 研究时期：

监控COVID-19前后两个代表性时期（2019年12月1-14日 && 2020年7月10-24日）

(3) SCCP信令功能：

提供对SS7信令网络的访问，满足2G/3G国际漫游需求
具有冗余配置，在北美和欧洲设有四个国际信令转接点
监控MAP协议，捕获位置管理、认证安全和故障恢复等流量

(4) Diameter信令功能：

提供4G漫游所需的信令能力
包含DRA、DPA和托管DEA服务
集成欢迎短信、漫游引导等增值服务

(5) 数据漫游服务：

连接MNO与外国漫游合作伙伴
依靠GTP功能建立和管理隧道
收集2G/3G和LTE网络间隧道的统计数据

(6) M2M服务：

避免与每个本地运营商建立技术和商业关系的成本
为物联网提供商提供稳定的连接/覆盖服务
将物联网设备流量引导到单一归属国家

Ethical considerations¶

Data collection and retention at network middle-boxes are in accordance with the terms and conditions of the IPX-P and the local regulations, and only with the specific purpose of providing and managing the IPX service. The terms also include data processing for monitoring and reporting as allowed usages of collected data. Data processing only extracts aggregated information and we do we not have access to any personally identifiable information. We nevertheless consulted with the Institutional Review Board (IRB) office at our institution who confirmed that no IRB review was necessary as the study relies on the analysis of de-identified data.

在网络中间设备上的数据收集和保留符合IPX-P的条款和条件以及当地法规，且仅用于提供和管理IPX服务的特定目的。这些条款还包括将数据处理用于监控和报告作为收集数据的允许用途。数据处理仅提取聚合信息，我们不会访问任何可识别个人身份的信息。尽管如此，我们仍咨询了我们机构的伦理审查委员会(IRB)办公室，他们确认由于本研究依赖于对去标识化数据的分析，因此无需进行IRB审查。