Data Architecture

Vortex Data Broker

The infrastructure is designed as a multi-tier pipeline, ensuring high-speed delivery of data via the Vortex Data Broker. This architecture guarantees data persistence in the Cloud while maintaining low-latency streams for real-time operations.

Ingestion & Security

Data originates from Manufacturing Apps (Aero-MES et al) and Vendor Apps (FMS, De-Icing, etc). All raw data is intercepted at the L3.5 Vortex Proxy, which performs Deep Packet Inspection (DPI) and converts untrusted signals into standardized MQTT topics. This demarcation point marks the official start of the Digital Thread.

Transport & Protocols

The system utilizes a hierarchy of protocols optimized for each tier to balance latency and reliability:

• Edge-to-Site (L3.5 to L4):The MQTT over TLS for lightweight, real-time streaming from assets, and within local network.
• Site-to-Corporate (L4 to L5):The gRPC/AMQP Streams for guaranteed delivery and high-throughput backbone communication over WAN.

Broker Hierarchy & Normalization

The Vortex Site Hub acts as a regional aggregator, performing semantic normalization and real-time contextual mapping via MES lookups. The Vortex Corporate Hub serves as the central nervous system, routing enriched Digital Thread streams to global consumers and cloud services.

Storage Tiers (Hot & Cold Path)

Upon reaching the corporate level, the data flow splits to serve different operational needs:

• Hot Path: Direct streaming to TwinForge (L6) for sub-second synchronization of the digital twin and AI-driven feedback loops.
• Cold Path: Immutable archiving in the Aero-Lakehouse (L6) using open formats like Parquet/Delta for historical auditing and ML training.

Digital Highways

Definition & Scope of Digital Highways

A Digital Highway is a logically and physically isolated data plane within the Aero-Factory architecture, designed to transport a specific class of information with guaranteed performance, security, and behavioral characteristics.

Each Digital Highway is defined by the following three layers:

• Network Isolation Layer: Implemented via VLANs, VDOMs (e.g., FortiGate), and routing policies to ensure physical or virtual traffic separation and eliminate cross-traffic interference.
• Transport & Broker Layer: Configured via dedicated protocol selection (e.g., MQTT, gRPC, AMQP), broker partitioning, and QoS policies to meet latency and throughput requirements.
• Semantic Layer: Defined by the type of payload (e.g., telemetry, commands, media) and its role within the system (e.g., observation, control, analytics).

A Digital Highway is therefore not a single component, but an end-to-end configuration spanning network, transport, and data semantics, ensuring deterministic behavior and controlled interaction between system layers. It is the foundational communication framework of the Aero-Factory ecosystem. It is important to distinguish that a Digital Highway is not a collection of standalone, dedicated hardware/software components. Instead, it represents a specific architectural configuration of the Vortex Data Broker stack and physical network segments, delivering information distribution infrastructure for a particular payload.

Digital Highway Invariants

The following rules apply to all Digital Highways:

• Strict Isolation: Traffic from one Highway must not traverse or interfere with another without passing through a defined inter-highway gateway.
• No Semantic Overlap: Each Highway has a clearly defined purpose and must not carry payloads belonging to another Highway.
• Non-Bypassability: All data must flow through its designated Highway; direct cross-layer or cross-domain shortcuts are prohibited.
• Deterministic Routing: Each data flow must have a predictable and auditable path from source to destination.
• Independent Scalability: Each Highway must be scalable without impacting the performance of other Highways.

Design

Each Highway is a specialized "information corridor" designed to provide high-speed, reliable data exchange tailored to the unique requirements of different data types. By leveraging the modularity of the Vortex Data Broker, the system can isolate traffic, prioritize critical commands, and ensure that, for example, high-bandwidth streaming media does not interfere with low-latency control signals. All data streams are managed as high-speed Highways that logically isolate different types of traffic to prevent congestion and ensure operational security.

Highway Name	Direction	Payload Type	Role
Digital Thread	Edge → Cloud	Telemetry & State	Immutable record of what happened (JSON/MQTT).
Vision & Analytics	Edge → Cloud	CCTV / LiDAR / Media	High-bandwidth streaming for AI vision and security.
Orchestration	Bidirectional	Missions & Tasks	Downstream: Assigning work to assets. Upstream: Mission feedback.
Command & Control	Cloud → Edge	PLC Writes / C2	Direct tactical instructions to hardware.
Insight Loop	Cloud → Edge	AI Setpoints	Efficiency optimizations from TwinForge.
ERP	Bidirectional	Master Data / Schedules	Synchronization of ERP Master Data via DCC components.
Management	Bidirectional	Meta / Logs / Config	Fleet Provisioning and System Observability.

Digital Thread

Central to the architecture is the Digital Thread, one of Digital Highways. The Digital Thread is an immutable, time-sequenced record of all field events and asset states, ensuring 100% traceability from raw sensor telemetry to high-level business decisions. Unlike standard data models, the Digital Thread weaves together isolated events, such as fuel levels from FMS or de-icing results from De-Icing System, into a continuous operational narrative. This ensures full traceability and "cradle-to-grave" visibility for every flight production process within the Aero-Factory.

1. Raw Ingestion: Proprietary signals are captured from vendor assets as unverified raw values.
2. Standardization: The L3.5 Proxy validates the data and maps it to a standardized MQTT topic hierarchy.
3. Enrichment: The Vortex Site Hub binds telemetry to real-time flight context (Flight ID, Gate, etc.) via MES integration.
4. Distribution: Enriched threads are routed through the Corporate Hub for global consumption.
5. Persistence: Data is archived in the Aero-Lakehouse for full process traceability.

ERP

The ERP Highway is a dedicated integration corridor designed for the synchronization of organizational Master Data and administrative schedules. Unlike the Orchestration Highway, this channel is strictly prohibited from carrying mission-critical commands or machine tasks. It relies on Data Delivery Components (DDCs), abstractions of vendor-specific integration suites (e.g., SAP Integration Suite), distributed across the Purdue levels (L6 to L3) to maintain consistency within the ERP's own domain.

The ERP Highway manages organizational consistency through the following stages:

1. Master Data Distribution: Global ERP system (L5) pushes master tables, code lists, and asset registries down to lower-tier DDC components.
2. Independent Consistency: Each DDC component independently manages the synchronization and integrity of local ERP mirrors at L6, L5, L4 and L3.
3. Vortex Proxy Access (L3): To maintain strict OT isolation, the L3 DCC component does not communicate directly with higher tiers; instead, it receives data via a VPN Gateway through the Vortex Proxy (L3.5).
4. Resource Availability Sync: Operational schedules and resource availability are updated bidirectionally to ensure the ERP is aware of site-level constraints.
5. Schema Governance: While the ERP manages its own internal consistency, any data intersecting with the Aero-Factory must be eligible for conversion into the Unified Data Schema for cross-Highway visibility.

Impleentation example: To align with a multi-level ERP Highway, the SAP ecosystem utilizes SAP Integration Suite as the central orchestrator, deploying Edge Integration Cell (EIC) nodes as the local DDCs at Level 3 and 4 to ensure offline process continuity and local data persistence. These edge components maintain a stateful cache of master data and work orders, communicating upward via the SAP Cloud Connector, acting as the "Always-On" gateway, which establishes a secure, encrypted tunnel to the SAP Business Technology Platform (BTP) at Levels 5 and 6 as central and cloud DDC respectively. This allows for seamless bidirectional synchronization where tactical execution data is emitted from the OT layers as UDS-compatible payloads, while strategic planning and global analytics are consolidated at Level 6 within SAP Datasphere (part of Aero-Lakehouse), ensuring a compliant implementation of ERP Highway from the shop floor to the corporate boardroom.

Digital Highways Implementation Strategy

Implementation Principles

Data Planes

Digital Highways are implemented as isolated data planes that combine network segmentation, transport configuration, and semantic constraints. Each Highway is realized through coordinated enforcement across VLANs or virtual routing domains, broker-level routing, and strict access control. Message content shall not influence Highway routing once a session is established, but should be used for validation and rejection of non-conforming payloads. This ensures that different categories of traffic, such as telemetry, control commands, and media streams, remain separated in both behavior and infrastructure.

The default implementation model is based on UDS transported over MQTT, where messages are validated at the first trusted ingestion point to ensure alignment with the Highway assigned at connection time, based on integration contract, namespace, and payload semantics. Classification is derived from the integration contract, namespace, and payload semantics. Once a client establishes a connection to a specific Highway endpoint, all subsequent traffic is implicitly bound to that Highway's identifier at the session level. This allows the broker layer to perform deterministic routing toward the appropriate network segment and consumer group without inspecting individual message headers.

Listeners

Routing decisions are executed at the broker layer based on the ingress listener, ensuring compatibility with encrypted transport (e.g., TLS) and avoiding deep packet inspection. Network segmentation acts as an enforcement boundary rather than a decision-making mechanism. Once assigned at connection time, the Highway classification remains immutable for the duration of the session, guaranteeing consistent behavior across the system.

Certain data types, particularly high-bandwidth or continuous streams such as video or LiDAR, are not transported via MQTT. These are implemented as Hybrid Highways, where the primary data flow uses a native transport protocol within a dedicated network segment, while a parallel control and trace channel is maintained through UDS messages over MQTT. This dual-broadcasting approach preserves performance characteristics without sacrificing architectural consistency.

Publishers

Publishers are not bound to a specific Highway; however, they must publish only to topics conforming to the defined Unified Namespace and payload schema. Enforcement of Highway isolation is applied at the subscription and network levels. Publishers should normally perform ingest using Highway-agnostic sessions.

Hybrid Transport and Traceability

For all non-MQTT payloads, the architecture enforces the “Echo MQTT” principle and a designated Highway (such as "Vision & Analysis"), requiring that every data transfer be accompanied by corresponding UDS metadata events (aka., dual-broadcasting). These events provide a complete and consistent representation of system activity within that Highway, ensuring that no data flow remains outside the observable and governed environment.

Concretely, in the case of the Vision & Analytics Highway, Raw Media Streams are transmitted directly over isolated network paths and persisted in chunked form, typically as time-based segments. Upon creation of each segment, a UDS event is emitted containing a reference to the stored object (e.g., URI or storage key), along with temporal and contextual metadata. This enables both real-time consumption of live streams and deterministic retrieval for post-event analysis.

Apart from Raw Media Streams, typical non-MQTT payloads are LiDAR Point Clouds, High-Frequency Waveform Data, Large Binary Files etc.

Contract-Based Routing

The Routing Strategy is implemented at the MQTT Broker level, establishing a secure, network-enforced boundary between data streams by mapping logical "Highways" to physical or logical network interfaces. In this architecture, the MQTT Broker is configured as a multihomed gateway, hosting distinct listeners on specific IP addresses and ports that correspond to isolated network segments (VLANs).

By binding the "Digital Thread" Highway to a specific listener port and the "Vision & Analysis" Highway to another, the infrastructure utilizes Layer 3 firewalls to ensure that traffic from a particular network segment (data plane) is physically incapable of reaching an unauthorized listener. This creates a "Data DMZ" where Brokers act as the sole, controlled points of convergence for disparate security zones.

// Example Routing Configuration (Contract-Based)
{
  "highways": [
    {
      "name": "Digital Thread",
      "endpoints": [
        {
          "address": "mqtts://vortex-edge:8881",
          "topics": ["Aero/+/+/+/+/Raw"]
        },
        {
          "address": "mqtts://vortex-site-hub:8881",
          "topics": ["Aero/+/+/+/+/Raw"]
        },
        {
          "address": "mqtts://vortex-central-hub:8881",
          "topics": [
            "Aero/+/+/+/+/Raw", // exclusively for Aero-Lakehouse
            "Aero/+/+/+/+/Refined"
          ]
        },
        {
          "address": "mqtts://vortex-cloud-hub:8881",
          "topics": ["Aero/+/+/+/+/Refined"]
        }
      ]
    },
    {
      "name": "Vision & Analytics",
      "endpoints": [
        {
          "address": "mqtts://vortex-central-hub:8882",
          "topics": ["Aero/+/+/+/+/Stream"]
        }
      ]
    }
  ]
}

In order to ensure deterministic access to Digital Highways, client connectivity is governed through a Contract-Based Assignment principle. As part of its integration contract, each Highway is defined by corresponding connection endpoints, security requirements, and permitted scope within the Unified Namespace.

Clients establish connections using these predefined endpoints; Highway selection is resolved exclusively at connection time and is not dependent on message-level metadata or topic structures. This approach eliminates ambiguity, reduces processing overhead, and aligns access control with architectural intent, ensuring that each consumer operates exclusively within its designated data plane.

Initialy, splitting the seven identified Highways into two distinct data planes is a recommended architectural baseline that balances operational simplicity with robust network performance. This dual-plane approach (separating the high-bandwidth "Vision & Analytics" stream from the mission-critical telemetry and control streams) ensures that the "heavy" data does not contend for the same physical throughput or processing priority as the "precise" data. While this two-plane model serves as the standard implementation, the architecture is inherently modular; additional network segments (VLANs) or dedicated data planes can be introduced as needed to accommodate future scaling, ultra-low latency requirements, or increased security isolation for specific sub-systems.

In the example above, vortex-central-hub Broker is resolved (via DNS Resolving) into two distinct IP addresses (10.10.10.5 and 10.10.20.5), which mandates the specific VLAN through which the traffic will flow from and to that Broker. This physical routing is determined at the moment of connection; by selecting a listener port, the client is automatically steered into a pre-defined network segment.

Initially, all Highways except for the Vision & Analytics Highway utilize the same data plane (VLAN 10). This baseline configuration simplifies the management of telemetry, orchestration, and command traffic, while immediately isolating high-bandwidth media streams from critical operational signals. In the future, some events may force the upgrade of data planes, for example if C2 requires ultra-low latency isolation, AI pipelines explode in bandwidth or Security zones diverge (e.g., external partners).

This Contract-Based Routing ensures that network optimization, such as adjusting Quality of Service (QoS) or implementing Jumbo Frames, can be performed independently of the application logic. By utilizing the "Data DMZ" pattern, the Broker acts as a controlled convergence point, allowing the infrastructure team to harden or scale individual data planes without requiring changes to the software or the underlying UDS message structure.

More information about UDS, and "Raw" and "Refined" data types can be found in Data Model.

Next: Application Architecture