Demystifying Direct Connect and Transit Gateway so you can better understand your AWS networking capabilities.
How AWS Virtual Private Cloud (VPC) Peering Options Have Evolved
Ever since AWS Direct Connect launched in 2012, organisations have been embracing dedicated connections to their cloud services – they gained the ability to improve their overall data transfers, increase their network performance, and enhance their data privacy when connecting to AWS. With the introduction of an array of new service features and enhancements, the options for peering with AWS have certainly evolved. Let’s take a look at how this has developed and how you can take advantage of some of the features available.
Launching AWS Direct Connect
Back when Direct Connect launched in 2012, connecting to AWS via Layer 2 with Megaport required a VXC that connected to a Virtual Interface (VIF) on the AWS side. At Layer 3, you were required to connect the L2 component to an L3 construct which required establishing a Virtual Private Gateway (VGW). A VGW is the routing target on AWS that connected the VXC through to each VPC. Each AWS VPC and Megaport VXC required 1:1 mapping including separate Border Gateway Protocol (BGP) sessions and VXCs/VIFs to each VGW.
Introducing Direct Connect Gateway
In 2017, AWS released Direct Connect Gateway (DXGW) which provided customers the ability to establish one BGP session for up to ten distinct VPCs within a single AWS account. The service matured over time from ten VPCs from a single account in a single AWS region, then expanded to support connectivity across AWS regions and, most recently, extended across multiple regions and multiple AWS accounts. DXGW also signalled the move from a single standard AWS ASN (7224 in most regions) to a customer defined ASN in the private ASN range (64512 to 65534). VGWs and DXGWs still exist as options for customers with Private VIF peerings today.
It was still possible to peer VPCs (even across accounts/regions) within AWS, however where a DXGW was used, it was only possible to peer from the VIF to the ‘next directly connected’ VPC, and not between VPCs that were not adjacently peered.
In this diagram, assume that VPC ‘A’ is linked directly to a VXC/VIF (via VGW or DXGW), and although VPC ‘B’ and ‘C’ are peered to VPC ‘A’, they may not communicate directly with the VIF. This is known as Transitive Routing. Some customers would run a virtual appliance (router) in VPC A to work around this constraint (informally known as a ‘Transit VPC’), however, overall design brought with it some complexities.
Transit Gateway: Topology and Overview
In late 2018, AWS launched Transit Gateway (TGW). A TGW allowed a ‘full mesh’ of routes to be passed between VPCs and VPN terminations which were not reliant on the VPC peering (with its transitive routing limitations) providing the ability to connect thousands (up to 5000) of VPCs and on-premises networks together, across multiple accounts to a single gateway.
Initially, a TGW was only accessible by customers to connect to associated VPCs via the AWS provided IPSEC VPN service. At launch, this was announced in the following regions: US East (Virginia), US East (Ohio), US West (Oregon), US West (Northern California), EU (Ireland), and AsiaPacific (Mumbai). It was further expanded to include the above as well as Canada (Central), EU (London), EU (Frankfurt), EU (Paris), Asia Pacific (Tokyo), Asia Pacific (Singapore), Asia Pacific (Seoul), and Asia Pacific (Sydney) regions (current as at end May 2019).
Transit Gateway for Direct Connect
Transit Gateway for Direct Connect support was recently announced on 30th April 2019 across four North American cloud regions (native Direct Connect termination INTO an existing TGW). This is currently supported with partners that offer dedicated and hosted connections across US East (N. Virginia), US East (Ohio), US West (N. California), and US West (Oregon) AWS Regions with additional regions enabled in the future. Note: While it is possible for customers in other regions to route traffic from Direct Connect to TGWs in those regions, this would potentially result in additional network latency which may impact performance.
Connecting with Transit Gateway via Megaport
When TGW via Direct Connect launched, network partners like Megaport connected customers to AWS via the following models: dedicated connections, hosted connections, and Hosted Virtual Interfaces (VIF). Megaport uses and is still supported by AWS via the Hosted VIF model. Hosted VIFs allow Megaport to provide you with flexibility and scalability when connecting to AWS which include scaling bandwidth in increments of 1Mbps from 1Mbps to 10Gbps VXCs where previously, partners delivering hosted connections were limited to a maximum of 500Mbps per VIF.
On 19th March 2019, AWS announced higher capacity for Direct Connect partners using the Hosted Connection model including a new program supporting higher speed options of 1, 2, 5, or 10 Gbps. The Megaport team are currently reviewing this model and working with AWS towards offering the hosted connection model to customers, over the coming months.
Factors to Consider
Please note there are some limitations with the recent release of Direct Connect TGW support. These include:
- The Hosted VIF model does not natively support Transit Gateway via Direct Connect.
- TGW support is reliant on hosted connection providers supporting a minimum of 1Gbps service offering, while most providers are in transition and can still only offer 500Mbps maximum on their hosted connection model.
- A customer moving to this model would be faced with an AWS imposed limit of one Private, one Public, one Transit VIF per AWS Direct Connect hosted connection.
- The availability of TGW, in terms of AWS regions, is widespread (accessible via VPN) however, the availability of TGW via Direct Connect is still somewhat limited to AWS US regions.
Workarounds and Interim Options
A customer who is using TGW today in most regions will be doing so via the AWS IPSEC VPN service with internet-delivered connections. The capability to add a TGW via Direct Connect (Private, Hosted VIF) is not currently available via Megaport.
Where a customer would like to promote their TGW to be accessible over a Direct Connect, it is possible (in all regions) to utilise a Public (Hosted) VIF across a Megaport VXC service. You would need to retain your IPSEC VPN connection but you can move traffic off your internet path and onto a Megaport VXC path instead. The AWS IPSEC VPN service via Public VIF will require you to use either your own public IP space to perform the public peering, or can utilise Megaport Cloud Router to provide the public peering IPs and NAT this back to across a VXC that uses RFC1918/private IP space. AWS TGWs will be available in all regions that AWS TGW is currently available in, without the associated TGW via DX dependency. With the above workaround in place, traffic will be end-to-end encrypted and also subject to a maximum throughput of 1.25Gbps (AWS VPN limitation).
We’ll be providing you with regular updates as we work towards offering the hosted connection model and Transit Gateway service with Megaport. Below are some handy links and further reading materials on both hosted connection and Transit Gateway. If you have any questions about these services, feel free to reach out to our team.
AWS Transit Gateway: what is it and how does it work?
Hosted Connection: New speed and support announcement
Matt Simpson, VP Cloud, Megaport
Jason Bordujenko, Head of Solutions, APAC