Let's talk networking. And AI. And infrastructure. And everything. Register for Connect 2026
Megaport Connect
A first-of-its-kind virtual summit built to help network engineers and IT leaders upskill, share ideas, and connect around the future of cloud and hybrid networking
This deep dive explains the end-to-end architecture of cloud on-ramps like AWS Direct Connect and Azure ExpressRoute.
Understand the differences between VGW, TGW, and Megaport-integrated setups. You’ll also explore routing policy, scale considerations, and how to avoid common bottlenecks in enterprise connectivity design.
Presented by: Chris Gunn, Solutions Architect at Megaport.
Chris Gunn, Solutions Architect at Megaport, presented this technical session as a deeper follow-on to introductory cloud on-ramp concepts. Rather than defining cloud connectivity at a beginner level, the webinar moves into the architectural mechanics behind AWS Direct Connect, Azure ExpressRoute, transit gateways, cloud WAN services, and provider-hosted connectivity models.
The central theme is that cloud on-ramps are not just a single circuit or service order. They involve a chain of physical connectivity, layer 2 handoff design, regional placement, layer 3 peering, gateway selection, and routing policy. Each cloud provider uses different terminology and has provider-specific implementation details, but the underlying design questions are consistent: where does the connection physically land, who owns which part of the path, what type of peering is required, and how should routes be distributed into cloud networks?
The session focuses primarily on AWS and Azure examples while noting that similar concepts exist across other cloud providers. It explains when direct and dedicated on-ramps make sense, when provider-hosted connections are simpler, how geography affects on-ramp placement, and how different gateway constructs such as Direct Connect gateways, virtual network gateways, transit gateways, Cloud WAN, and Azure Virtual WAN fit into enterprise connectivity designs.
Cloud on-ramp design begins below routing. Before BGP peerings, gateways, or route propagation can work, the physical and layer 2 path has to exist between the customer environment and the cloud provider’s on-ramp location.
The webinar frames this path using a logistics analogy:
This breakdown matters because ownership and operational responsibility change depending on the chosen connection model. A customer may own more of the physical and layer 2 path in a dedicated model, or rely more heavily on a provider-hosted model where the service provider has already built the on-ramp capacity into cloud providers.
Understanding this separation also helps prevent a common design mistake: treating “Direct Connect” or “ExpressRoute” as if it automatically describes the full end-to-end circuit. In practice, it describes only part of the path. The full design still needs to account for the physical handoff, provider network, VLAN presentation, cloud-side gateway, and routing relationships.
In a direct or dedicated connection model, the customer is effectively responsible for the layer 1 and layer 2 path into the cloud provider’s on-ramp. The customer may be physically present in the relevant data center, or may work with a network provider that can establish the necessary cross-connect, but the result is still a customer-owned connection into the cloud provider’s environment.
This model is typically ordered at larger port sizes, such as 10 Gbps up to 100 Gbps. The capacity belongs to the customer, and the connection is usually presented as a trunk toward the cloud provider. That trunking model allows multiple logical layer 3 peerings to be built over separate VLANs, subject to each provider’s rules and limitations.
Dedicated connectivity is especially important when the customer needs:
MACsec is a key distinction. Because MACsec encrypts layer 2 headers, including VLAN information, it is not suitable for provider-hosted services where a provider is using VLAN tags to separate multiple customers on shared infrastructure. In a dedicated model, the connection is the customer’s, so MACsec can be run from the customer’s equipment toward the cloud provider without encrypting other customers’ traffic.
The trade-off is operational complexity. Dedicated connectivity requires ordering and coordinating the physical connection, arranging the cross-connects, and handling more of the deployment work before the cloud-side service can be used.
Provider-hosted connectivity changes the ownership model. Instead of each customer ordering their own direct physical connection into every cloud provider, the provider has already deployed large on-ramp capacity, such as 100 Gbps links, into cloud providers. Individual customers consume a slice of that capacity.
In this model, customer connections are separated at layer 2, typically by VLAN. One VLAN might represent a 1 Gbps service for one customer, another might represent a 50 Mbps service for another customer, and another might represent a larger service such as 10 Gbps or, in some AWS cases, 25 Gbps.
The practical advantage is speed and simplicity. The customer only needs to connect into the provider’s network and order the hosted cloud connection. The provider has already handled much of the cloud on-ramp infrastructure, which can make deployment significantly faster than ordering a new dedicated physical cloud connection.
The trade-offs are mostly around control and feature availability:
For many customers, the recommendation is to use provider-hosted connectivity unless there is a specific reason to choose dedicated connectivity. The main reasons to choose dedicated are higher bandwidth requirements or MACsec requirements. Otherwise, hosted connectivity usually provides a faster and simpler path to the cloud.
Cloud providers associate on-ramp locations with regions, but the closest physical on-ramp is often still the best choice even when the target workload is in a different cloud region. The webinar uses the example of a customer in San Jose connecting to an AWS workload in US East 2. The customer does not necessarily need to ride a third-party provider network across the country to reach an on-ramp near US East 2.
Most cloud providers allow customers to use a local on-ramp within the same geopolitical region and then reach other cloud regions over the cloud provider’s own backbone. In AWS, a Direct Connect gateway can function as a broader resource for reaching regions within the relevant scope. In Azure, a standard ExpressRoute SKU can reach regions within the same geopolitical region, while connectivity outside that area may require a Premium SKU.
This has cost and performance implications. If a customer uses a provider network to carry traffic from one side of the country to the other before entering the cloud, the provider may charge for that transport. By contrast, once the customer enters through a suitable local on-ramp, the cloud provider’s backbone may carry the traffic to the target cloud region without the same additional provider middle-mile charge.
The general rule is to research the specific cloud provider’s regional rules, but do not assume the on-ramp must be physically located near the target cloud region. Often, the better design is to use the closest appropriate on-ramp and let the cloud provider’s network carry the traffic from there.
AWS Local Zones add an important exception to the normal region-based mental model. A Local Zone places compute closer to a metro area than the parent AWS region. The example in the session is Chicago, where local compute can avoid hairpinning traffic back through a more distant regional location when low latency is required inside the Chicago metro area.
The confusing part is that a Local Zone’s parent region may not be the same region associated with the nearest on-ramp locations. In the example, the Chicago Local Zone is associated with a parent region in US East, while nearby Direct Connect locations may appear under a different regional association.
The design guidance is to keep using the closest appropriate Direct Connect on-ramp, but associate the on-ramp directly with the Local Zone rather than with the parent region. If the connection is associated only with the parent region, traffic may hairpin back through that parent region and lose the latency benefit the Local Zone was meant to provide.
For workloads using Local Zones, the on-ramp decision should therefore consider both physical proximity and the cloud-side association target. The goal is not just to enter AWS locally, but to ensure the routing relationship lands on the Local Zone path rather than forcing traffic through a more distant parent region.
Once the physical and layer 2 connectivity is established, the design moves to layer 3 peering. The most common model is private peering, which connects the customer to private infrastructure-as-a-service resources such as virtual compute, storage, VPCs, and VNets.
Private peering commonly uses RFC 1918 addressing, such as 10.0.0.0/8 or 192.168.0.0/16 space. The customer establishes BGP with the relevant cloud-side gateway and exchanges routes to reach private cloud resources.
This is distinct from public peering, which connects to public cloud provider networks or SaaS services reachable through those public environments. Examples discussed include services such as Office 365, Salesforce, or other public cloud-facing services. Public peering is a valid use case, but it is less common than private connectivity into the customer’s cloud infrastructure.
Public peering also introduces different addressing requirements. A customer generally needs public IP addressing to peer with the cloud provider’s public network. In some cases, that may require requesting public IP addresses from the cloud provider. In other cases, the customer may want to advertise its own public IP space over BGP into the public peering relationship.
For the rest of the architecture discussion, the session focuses primarily on private peering because that is the dominant design pattern for connecting on-premises environments into customer-owned IaaS networks.
Gateways are the cloud-side layer 3 peering points where routes are exchanged and connectivity is associated with cloud networks. Every major provider has gateway constructs, but the names and exact capabilities differ.
In AWS, relevant gateway constructs include:
In Azure, the equivalent design often involves an ExpressRoute gateway or virtual network gateway associated with a VNet or a larger WAN construct.
These gateways participate in BGP routing and need ASN configuration. Azure and many other providers typically use a provider public ASN for BGP peering. AWS differs in that, for private peering, it allows private BGP ASNs. This is one of the points where cloud-specific behavior becomes important: the general design pattern is similar, but the exact ASN and gateway requirements are provider-dependent.
Associations are also critical. It is not enough to create the on-ramp and gateway. The relevant VPCs, VNets, WAN hubs, or other cloud network targets must be associated or attached so that routes can be shared into the intended environment.
Cloud on-ramps do not always have to land directly into a single VPC or VNet. In larger environments, customers may use a provider’s global WAN service to connect multiple regions and cloud networks. In those cases, the on-ramp can be connected into the cloud WAN architecture rather than first landing in an individual network and then relying on downstream connectivity.
In Azure, this can mean deploying ExpressRoute into a Virtual WAN design. The ExpressRoute gateway is associated with a Virtual WAN hub, and that hub provides connectivity onward into the rest of the Virtual WAN environment.
In AWS, a Direct Connect gateway can attach into AWS Cloud WAN. From there, the connection can be associated with Cloud WAN segments so routes can be distributed into the intended cloud WAN topology.
This pattern gives the on-premises environment a more direct path into the cloud provider’s WAN fabric. Instead of treating a single VPC or VNet as the entry point, the on-ramp becomes part of the larger cloud networking backbone that connects multiple cloud networks or regions.
The same design rule still applies: attachments and associations determine reachability. Whether the destination is a VPC, a VNet, a Virtual WAN hub, or a Cloud WAN segment, the architecture must explicitly connect the on-ramp gateway to the cloud-side network constructs that should receive and advertise routes.
AWS Transit Gateway is a distinct option for designs that need transitive routing across multiple VPCs or networks. Rather than connecting an on-ramp to a single VPC, Direct Connect can land into a Transit Gateway so that routes can be distributed to the VPCs and networks attached to it.
This is useful when the cloud architecture has multiple VPCs that need shared connectivity back to on-premises networks. A Transit Gateway acts as the central routing point, allowing the on-ramp to participate in a broader routed topology instead of requiring separate point-to-point designs for every VPC.
The key architectural distinction is scope. A virtual private gateway is tied more directly to a VPC-oriented connectivity model, while Transit Gateway supports a larger routing domain with multiple attachments. Selecting between these options depends on how many VPCs or networks need reachability, whether transitive routing is required, and how the organization wants to centralize route propagation and control.
Azure has a notable layer 2 capability that affects hosted connection design: Q-in-Q, or 802.1Q encapsulation inside another VLAN tag. This allows multiple peerings to be carried for one customer over a provider-hosted cloud connection.
In this model, the provider can wrap the customer’s connection in a service tag, or S-tag, identifying that the traffic belongs to a specific customer connection toward Azure. Inside that outer tag, customer tags, or C-tags, can represent different peerings.
For example:
This is an important exception to the general hosted-connection limitation around multiple peerings. In Azure designs, customers may be able to retain the operational simplicity of provider-hosted connectivity while still supporting multiple peering types over the same hosted relationship.
The broader lesson is that provider-hosted versus dedicated is not only a generic product choice. Provider-specific layer 2 and peering features can materially affect what architectures are possible without moving to a dedicated physical connection.
The webinar closes by comparing terminology across cloud providers. The naming differs, but the same major categories appear repeatedly:
The differences become important during implementation. AWS private peering can use private ASNs, while many other providers require peering with the provider’s public ASN. Azure has Q-in-Q behavior that affects hosted multi-peering designs. AWS has gateway choices such as Direct Connect gateway, virtual private gateway, Transit Gateway, and Cloud WAN attachments.
From an architecture perspective, the task is to map each provider’s terminology back to the same underlying questions: Is the connection dedicated or hosted? What bandwidth and encryption requirements apply? Is the peering public or private? Which gateway receives the connection? Which cloud networks or WAN segments should learn the routes?
Cloud on-ramps are best understood as an end-to-end architecture rather than a single service. The physical path, provider middle mile, layer 2 handoff, peering type, gateway selection, ASN behavior, and route associations all determine whether the design will meet the organization’s performance, scale, and operational requirements.
For many enterprises, provider-hosted connectivity offers the most practical starting point because the provider has already built the cloud-facing infrastructure. Dedicated connectivity becomes important when the design requires larger bandwidth, more control over the trunk, or MACsec encryption.
The most effective designs also account for cloud geography and gateway scope. A local on-ramp can often reach distant regions over the cloud provider backbone, but special cases such as AWS Local Zones require careful association to avoid unnecessary hairpinning. At layer 3, the choice between private peering, public peering, VPC/VNet gateways, transit gateways, and cloud WAN services determines how routes are shared and how broadly the connection extends across the cloud environment.
The practical takeaway is to design from requirements backward: identify the target workloads and regions, decide whether hosted or dedicated connectivity is appropriate, choose the correct peering type, and attach the on-ramp to the gateway or WAN construct that matches the intended routing domain.
With a Megaport Solutions Architect
We’ve helped hundreds of customers, just like you, to simplify their connectivity.
