Taicheng's Profile

Multi-NIC VMs

A common use case is to inspect bidirectional traffic between two VPC networks by leveraging a group of network virtual appliances, that is, multi-NIC VMs.

VPC networks named vpc-a and vpc-b, each with one subnet.

Each backend VM has two network interfaces, one attached to each VPC network (nic0 attached to VPC vpc-a, nic1 attached to VPC vpc-b).

The subnets, subnet-a and subnet-b, use the 10.13.1.0/24 and 10.15.1.0/24 primary IP address ranges, respectively, and they both reside in the us-central1 region.

Creating the Common Managed Instance Group (MIG)

Create a startup script that will install the iptables software on any backend VM. This script named config.sh will be passed to the gcloud compute instance-templates create command using the –metadata flag in the next step:

#!/bin/bash
# Enable IP forwarding:
echo 1 > /proc/sys/net/ipv4/ip_forward
echo "net.ipv4.ip_forward=1" > /etc/sysctl.d/20-iptables.conf
# Read VM network configuration:
md_vm="http://metadata.google.internal/computeMetadata/v1/instance/"
md_net="$md_vm/network-interfaces"
nic0_gw="$(curl $md_net/0/gateway -H "Metadata-Flavor:Google" )"
nic0_mask="$(curl $md_net/0/subnetmask -H "Metadata-Flavor:Google")"
nic0_addr="$(curl $md_net/0/ip -H "Metadata-Flavor:Google")"
nic0_id="$(ip addr show | grep $nic0_addr | awk '{print $NF}')"
nic1_gw="$(curl $md_net/1/gateway -H "Metadata-Flavor:Google")"
nic1_mask="$(curl $md_net/1/subnetmask -H "Metadata-Flavor:Google")"
nic1_addr="$(curl $md_net/1/ip -H "Metadata-Flavor:Google")"
nic1_id="$(ip addr show | grep $nic1_addr | awk '{print $NF}')"
# Source based policy routing for nic1
echo "100 rt-nic1" >> /etc/iproute2/rt_tables
sudo ip rule add pri 32000 from $nic1_gw/$nic1_mask table rt-nic1
sleep 1
sudo ip route add 35.191.0.0/16 via $nic1_gw dev $nic1_id table rt-nic1
sudo ip route add 130.211.0.0/22 via $nic1_gw dev $nic1_id table rt-nic1
# Use a web server to pass the health check for this example.
# You should use a more complete test in production.
sudo apt-get update
sudo apt-get install apache2 -y
sudo a2ensite default-ssl
sudo a2enmod ssl
echo "Example web page to pass health check" | sudo tee /var/www/html/index.html
sudo systemctl restart apache2

Create a common instance template named third-party-template-multinic, which will be used to create new VMs in both the vpc-a and vpc-b VPC networks, when an autoscaling event is triggered:

gcloud compute instance-templates create third-party-template-multinic \
    --region=us-central1 \
    --network-interface subnet=subnet-a,address="" \
    --network-interface subnet=subnet-b \
    --tags=allow-ssh,allow-health-check,my-network-tag \
    --image-family=debian-10 \
    --image-project=debian-cloud \
    --can-ip-forward \
    --metadata=startup-script="$(< config.sh)"

Create a common managed instance group named third-party-instance-group that will also be used by the two backend services, one in the vpc-a and the other one in the vpc-b VPC networks:

gcloud compute instance-groups managed create third-party-instance-group \
    --region=us-central1 \
    --template=third-party-template-multinic \
    --size=3

.

Create a new HTTP health check named hc-http-80 to test TCP connectivity to the VMs on port 80:

gcloud compute health-checks create http hc-http-80 \
    --region=us-central1 \
    --port=80

Use the previously created health check to create two internal backend services in the us-central1 region: one named backend-service-a in the vpc-a VPC, and the other one named backend-service-b in the vpc-b VPC:

gcloud compute backend-services create backend-service-a \
    --load-balancing-scheme=internal \
    --health-checks-region=us-central1 \
    --health-checks=hc-http-80 \
    --region=us-central1 \
    --network=vpc-a \
    --session-affinity=CLIENT_IP
gcloud compute backend-services create backend-service-b \
    --load-balancing-scheme=internal \
    --health-checks-region=us-central1 \
    --health-checks=hc-http-80 \
    --region=us-central1 \
    --network=vpc-b \
    --session-affinity=CLIENT_IP

Add to each of the two backend services the managed instance groups you created in step 3 (third-party-instance-group), which contains the third-party virtual appliances as backends:

gcloud compute backend-services add-backend backend-service-a \
    --instance-group=third-party-instance-group \
    --instance-group-region=us-central1 \
    --region=us-central1
gcloud compute backend-services add-backend backend-service-b \
    --instance-group=third-party-instance-group \
    --instance-group-region=us-central1 \
    --region=us-central1

Create two regional, internal forwarding rules: one associated with the subnet-a and the other one associated with the subnet-b. Connect each forwarding rule to its respective backend service, that is, backend-service-a and backend-service-b:

gcloud compute forwarding-rules create ilb-a \
    --load-balancing-scheme=internal \
    --ports=80 \
    --network=vpc-a \
    --subnet=subnet-a \
    --region=us-central1 \
    --backend-service=backend-service-a \
    --address=10.13.1.77
gcloud compute forwarding-rules create ilb-b \
    --load-balancing-scheme=internal \
    --ports=80 \
    --network=vpc-b \
    --subnet=subnet-b \
    --region=us-central1 \
    --backend-service=backend-service-b \
    --address=10.15.1.77

Creating the Custom Static Routes That Define the Load Balancers As the Next Hops:

gcloud compute routes create ilb-nhop-dest-10-15-1 \
    --network=vpc-a \
    --destination-range=10.15.1.0/24 \
    --next-hop-ilb=ilb-a \
    --next-hop-ilb-region=us-central1
gcloud compute routes create ilb-nhop-dest-10-13-1 \
    --network=vpc-b \
    --destination-range=10.13.1.0/24 \
    --next-hop-ilb=ilb-b \
    --next-hop-ilb-region=us-central1

Creating the First VM

gcloud compute instances create vm-a \
    --zone=us-central1-a \
    --image-family=debian-10 \
    --image-project=debian-cloud \
    --tags=allow-ssh \
    --subnet=subnet-a \
    --private-network-ip 10.13.1.70 \
    --metadata=startup-script='#! /bin/bash
    sudo apt-get update
    sudo apt-get install apache2 -y
    sudo a2ensite default-ssl
    sudo a2enmod ssl
    vm_hostname="$(curl -H "Metadata-Flavor:Google" \
    http://metadata.google.internal/computeMetadata/v1/instance/name)"
    echo "Page served from: $vm_hostname" | \
    sudo tee /var/www/html/index.html
    sudo systemctl restart apache2'

Creating the Second VM

gcloud compute instances create vm-b \
    --zone=us-central1-b \
    --image-family=debian-10 \
    --image-project=debian-cloud \
    --tags=allow-ssh \
    --subnet=subnet-b \
    --private-network-ip 10.15.1.70 \
    --metadata=startup-script='#! /bin/bash
    sudo apt-get update
    sudo apt-get install apache2 -y
    sudo a2ensite default-ssl
    sudo a2enmod ssl
    vm_hostname="$(curl -H "Metadata-Flavor:Google" \
    http://metadata.google.internal/computeMetadata/v1/instance/name)"
    echo "Page served from: $vm_hostname" | \
    sudo tee /var/www/html/index.html
    sudo systemctl restart apache2'

Verifying Load Balancer Health Status:

gcloud compute backend-services get-health backend-service-a \
    --region us-central1
gcloud compute backend-services get-health backend-service-b \
    --region us-central1

Testing Connectivity from the Testing VM:

gcloud compute ssh vm-a --zone=us-central1-a
curl http://10.15.1.70
exit

Testing Connectivity from the Production VM:

gcloud compute ssh vm-b --zone=us-central1-b
curl http://10.13.1.70
exit

Taicheng