Skip to content

Commit e7a12d5

Browse files
mbaldessarimbaldessari
committed
Initial pass at ingress-bgp docs
1 parent c7466e1 commit e7a12d5

6 files changed

Lines changed: 474 additions & 0 deletions

File tree

‎content/patterns/ingress-mesh-bgp/_index.adoc‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
title: Ingress Mesh BGP
3+
date: 2025-01-26
4+
tier: sandbox
5+
summary: This pattern demonstrates multi-cluster service mesh networking using MetalLB with BGP and Red Hat Service Interconnect (Skupper) for cross-cluster connectivity.
6+
rh_products:
7+
- Red Hat OpenShift Container Platform
8+
- Red Hat Advanced Cluster Management
9+
- Red Hat Service Interconnect
10+
industries:
11+
- General
12+
- Telecommunications
13+
aliases: /ingress-mesh-bgp/
14+
# pattern_logo: ingress-mesh-bgp.png # TODO: Create pattern logo
15+
links:
16+
github: https://github.com/validatedpatterns/ingress-mesh-bgp
17+
install: imbgp-getting-started
18+
bugs: https://github.com/validatedpatterns/ingress-mesh-bgp/issues
19+
ci: ingressmeshbgp
20+
---
21+
:toc:
22+
:imagesdir: /images
23+
:_content-type: ASSEMBLY
24+
include::modules/comm-attributes.adoc[]
25+
26+
include::modules/imbgp-about.adoc[leveloffset=+1]
27+
28+
include::modules/imbgp-architecture.adoc[leveloffset=+1]
29+
30+
[id="next-steps_imbgp-index"]
31+
== Next steps
32+
33+
* link:imbgp-getting-started[Deploy the Ingress Mesh BGP pattern].

‎content/patterns/ingress-mesh-bgp/imbgp-getting-started.adoc‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
title: Getting started
3+
weight: 10
4+
aliases: /ingress-mesh-bgp/imbgp-getting-started/
5+
---
6+
7+
:toc:
8+
:imagesdir: /images
9+
:_content-type: ASSEMBLY
10+
include::modules/comm-attributes.adoc[]
11+
12+
include::modules/imbgp-deploying.adoc[leveloffset=1]
13+
14+
[id="next-steps_imbgp-getting-started"]
15+
== Next steps
16+
17+
After the pattern is deployed and working correctly, you can:
18+
19+
* Verify the BGP routing is functioning correctly by checking the routing table on the core router
20+
* Test the anycast service by accessing it from the client VM
21+
* Explore the Red Hat Service Interconnect (Skupper) console to see the cross-cluster connectivity

‎modules/imbgp-about.adoc‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
:_content-type: CONCEPT
2+
:imagesdir: ../../images
3+
4+
[id="about-ingress-mesh-bgp-pattern"]
5+
= About the Ingress Mesh BGP pattern
6+
7+
Use case::
8+
9+
* Deploy multi-cluster applications with unified ingress using BGP-based load balancing.
10+
* Enable anycast IP addressing for seamless failover between OpenShift clusters.
11+
* Connect services across clusters using Red Hat Service Interconnect (Skupper) for secure east-west traffic.
12+
* Demonstrate enterprise-grade BGP routing integration with Kubernetes/OpenShift environments.
13+
+
14+
[NOTE]
15+
====
16+
This pattern is designed for AWS environments and simulates a datacenter-like BGP network topology using EC2 instances as routing infrastructure.
17+
====
18+
19+
Background::
20+
Modern distributed applications often span multiple clusters for high availability, geographic distribution, or workload isolation. Traditional ingress solutions provide per-cluster access, but organizations need unified entry points that can intelligently route traffic across clusters.
21+
22+
This pattern addresses these requirements by combining:
23+
24+
* *MetalLB with BGP mode* - Provides load balancer services that advertise routes via BGP, enabling anycast addressing where the same IP is reachable through multiple clusters.
25+
* *Red Hat Service Interconnect (Skupper)* - Creates a secure application-layer virtual network connecting services across clusters without requiring VPN or special network configurations.
26+
* *FRRouting (FRR)* - Industry-standard routing software running on EC2 instances that acts as the BGP peering infrastructure, simulating top-of-rack (TOR) switches and core routers.
27+
28+
[id="about-solution-imbgp"]
29+
== About the solution
30+
31+
This pattern deploys a complete multi-cluster networking demonstration on AWS that includes:
32+
33+
* Two OpenShift clusters designated as "west" (hub) and "east" (spoke)
34+
* A simulated routing infrastructure with FRR-based routers
35+
* MetalLB configured in BGP mode on both clusters
36+
* Red Hat Service Interconnect linking services between clusters
37+
* A hello-world application deployed across both clusters demonstrating the connectivity
38+
39+
The solution uses ECMP (Equal-Cost Multi-Path) routing to distribute traffic across both clusters when accessing the anycast IP address.
40+
41+
.Benefits of the Ingress Mesh BGP pattern:
42+
43+
* *Unified service access* - Single IP address reaches services on multiple clusters
44+
* *Automatic failover* - BGP route withdrawal provides fast failover when a cluster becomes unavailable
45+
* *Secure cross-cluster communication* - Skupper encrypts all inter-cluster traffic using mutual TLS
46+
* *No network infrastructure changes* - Skupper works over existing networks without VPNs or firewall changes
47+
* *GitOps-driven deployment* - All components are deployed and managed through ArgoCD
48+
49+
[id="about-technology-imbgp"]
50+
== About the technology
51+
52+
The following technologies are used in this solution:
53+
54+
https://www.redhat.com/en/technologies/cloud-computing/openshift/try-it[Red Hat OpenShift Platform]::
55+
An enterprise-ready Kubernetes container platform built for an open hybrid cloud strategy. It provides a consistent application platform to manage hybrid cloud, public cloud, and edge deployments.
56+
57+
https://www.redhat.com/en/technologies/management/advanced-cluster-management[Red Hat Advanced Cluster Management for Kubernetes]::
58+
Controls clusters and applications from a single console, with built-in security policies. Extends the value of Red Hat OpenShift by deploying apps, managing multiple clusters, and enforcing policies across multiple clusters at scale.
59+
60+
https://www.redhat.com/en/technologies/cloud-computing/service-interconnect[Red Hat Service Interconnect]::
61+
Based on the open source Skupper project, Red Hat Service Interconnect enables secure communication between services across different environments without requiring VPN infrastructure or special firewall rules. It creates a virtual application network that works at Layer 7.
62+
63+
https://metallb.universe.tf/[MetalLB]::
64+
A load-balancer implementation for bare metal Kubernetes clusters, using standard routing protocols. In this pattern, MetalLB operates in BGP mode to advertise service IPs as routes to the upstream network infrastructure.
65+
66+
https://frrouting.org/[FRRouting (FRR)]::
67+
A free and open source Internet routing protocol suite for Linux and Unix platforms. It implements BGP, OSPF, RIP, and other protocols. In this pattern, FRR runs on EC2 instances to simulate datacenter routing infrastructure.
68+
69+
Kubernetes Gateway API::
70+
The next generation of Kubernetes Ingress, providing a more expressive and extensible API for managing traffic into and within a cluster. Red Hat Service Interconnect uses Gateway API for exposing services.

‎modules/imbgp-architecture.adoc‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
:_content-type: CONCEPT
2+
:imagesdir: ../../images
3+
4+
[id="architecture-ingress-mesh-bgp"]
5+
= Architecture
6+
7+
The Ingress Mesh BGP pattern demonstrates a multi-cluster networking architecture that combines BGP-based anycast ingress with service mesh connectivity. The architecture simulates an enterprise network topology on AWS.
8+
9+
[id="network-topology-imbgp"]
10+
== Network topology
11+
12+
The pattern creates the following network components on AWS:
13+
14+
.Network topology diagram
15+
image::ingress-mesh-bgp/network.svg[Network Topology,700]
16+
17+
=== VPCs and subnets
18+
19+
The pattern provisions several VPCs to simulate separate network segments:
20+
21+
* *Client-Core VPC (192.168.8.0/24)* - Contains the client VM and core router
22+
* *Core-West TOR VPC (192.168.12.0/24)* - Connects the core router to the west cluster's top-of-rack router
23+
* *Core-East TOR VPC (192.168.16.0/24)* - Connects the core router to the east cluster's top-of-rack router
24+
* *West Workers VPC (10.0.0.0/16)* - The west OpenShift cluster's VPC
25+
* *East Workers VPC (10.1.0.0/16)* - The east OpenShift cluster's VPC
26+
27+
=== Routing infrastructure
28+
29+
The pattern deploys EC2 instances running FRRouting to create a simulated datacenter network:
30+
31+
[cols="1,1,2"]
32+
|===
33+
|Component |ASN |Description
34+
35+
|Core Router
36+
|64666
37+
|Central router that peers with both TOR routers and advertises client network routes
38+
39+
|West TOR
40+
|64001
41+
|Top-of-rack router for the west cluster, peers with core and west OpenShift workers
42+
43+
|East TOR
44+
|64002
45+
|Top-of-rack router for the east cluster, peers with core and east OpenShift workers
46+
47+
|West OpenShift (MetalLB)
48+
|65001
49+
|MetalLB speakers on west cluster workers, peer with west TOR
50+
51+
|East OpenShift (MetalLB)
52+
|65002
53+
|MetalLB speakers on east cluster workers, peer with east TOR
54+
|===
55+
56+
=== Anycast addressing
57+
58+
Both clusters advertise the same anycast IP range (192.168.155.0/24) via BGP. When a client accesses an anycast IP:
59+
60+
. The core router receives BGP advertisements from both TOR routers for the anycast range
61+
. ECMP routing distributes traffic across both paths
62+
. Requests reach either the west or east cluster based on the routing decision
63+
. If one cluster becomes unavailable, BGP route withdrawal automatically redirects traffic to the remaining cluster
64+
65+
[id="cluster-components-imbgp"]
66+
== Cluster components
67+
68+
=== West cluster (Hub)
69+
70+
The west cluster acts as the management hub and includes:
71+
72+
* *Red Hat Advanced Cluster Management* - Manages the east cluster as a spoke
73+
* *HashiCorp Vault* - Centralized secrets management
74+
* *External Secrets Operator* - Synchronizes secrets from Vault to Kubernetes
75+
* *MetalLB* - Provides BGP-advertised load balancer services (ASN 65001)
76+
* *Red Hat Service Interconnect (Skupper)* - Hosts the Skupper site with link access enabled
77+
* *Hello-world application* - Frontend component of the demo application
78+
79+
=== East cluster (Spoke)
80+
81+
The east cluster is a managed spoke that includes:
82+
83+
* *External Secrets Operator* - Retrieves secrets from the hub's Vault
84+
* *MetalLB* - Provides BGP-advertised load balancer services (ASN 65002)
85+
* *Red Hat Service Interconnect (Skupper)* - Connects back to the west cluster's Skupper site
86+
* *Hello-world application* - Backend component of the demo application
87+
88+
[id="service-interconnect-imbgp"]
89+
== Red Hat Service Interconnect
90+
91+
Red Hat Service Interconnect (based on Skupper) creates a virtual application network between the clusters:
92+
93+
* The west cluster hosts a Skupper site with `linkAccess: default`, allowing other sites to connect
94+
* The east cluster establishes a link to the west cluster using a pre-shared access token
95+
* Services exposed through Skupper listeners become accessible across both clusters
96+
* All traffic between sites is encrypted using mutual TLS
97+
98+
The pattern uses the Skupper v2 API with the following components:
99+
100+
* *Site* - Defines the Skupper installation in each namespace
101+
* *Listener* - Exposes a service to the Skupper network
102+
* *Connector* - Connects a local workload to a Skupper-exposed service
103+
* *AccessGrant/AccessToken* - Manages secure connection between sites
104+
105+
[id="gitops-structure-imbgp"]
106+
== GitOps structure
107+
108+
The pattern follows the Validated Patterns framework:
109+
110+
----
111+
ingress-mesh-bgp/
112+
├── values-global.yaml # Global configuration
113+
├── values-west.yaml # West (hub) cluster configuration
114+
├── values-east.yaml # East (spoke) cluster configuration
115+
├── charts/
116+
│ ├── all/
117+
│ │ ├── hello-world/ # Demo application
118+
│ │ ├── metallb/ # MetalLB configuration
119+
│ │ └── rhsi/ # Skupper configuration for west
120+
│ └── east-site/
121+
│ └── rhsi-east/ # Skupper configuration for east
122+
└── ansible/
123+
└── playbooks/ # Infrastructure automation
124+
----
125+
126+
ArgoCD manages the deployment of all components, with Red Hat Advanced Cluster Management distributing configurations to the appropriate clusters.

0 commit comments

Comments
 (0)