Dual ISP Loadbalanced Network

Here's the condensed version of my pf.conf. I have two ISP connections--one cable, one DSL. I'm providing loadbalancing on outbound connections for users on my internal network, as well as a secondary connection to one of the web servers in my DMZ. That server actually sits behind another OpenBSD box running relayd as an https proxy, so you'll see rdr rules to that here. Mainly our external web services are only accessible through the DSL connection. I'm using multipath routing for handling multiple gateways as well as ifstated to handle failover in case one of the ISPs goes down.

Many thanks to hednod on the irc channel for suggesting I try reply-to combined with tagging to get the secondary connection to work. Without it, I could connect to the server from the outside world but not through locations outside my network on the same ISPs.

Note I've changed the external addresses.

pf.conf:

# pf.conf for loadbalancing with cable & DSL connections

# em0 = LAN (internal network) = 192.168.1.0/24
# em1 = WAN (Internet) = x.x.x.x with many aliases (DSL)
# em2 = WAN (Internet) = Cable connection (y.y.y.y)
# em3 = DMZ network = 192.168.20.0/24

# macros
int_if = "em0"
ext_if1 = "em1"  # DSL
ext_if2 = "em2"  # cable
dmz_if = "em3"
ext_gw1 = "x.x.x.n" #DSL gateway 
ext_gw2 = "y.y.y.n" # cable gateway 
dsl_ext_addrs = "x.x.x.x/28"  
int_network = "192.168.1.0/24"
dmz_network = "192.168.20.0/24"
cable_ext_addr = "y.y.y.y" 
router_tcp_services = "{ 113, 22 }"

server_dmz_addr = "192.168.20.2"
server_dsl_addr = "x.x.x.2"
server_ext_ports = { 80, 8080 }
server_int_ports = { 80, 8080, 22 }

# this is the relayd box between the server above and the rest of the DMZ
sslbox_dmz_addr = "192.168.20.3"
sslbox_ports = "{ 8080, 443, 22, 9090 }"

icmp_types = "echoreq"
priv_nets = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }"

# options:
set block-policy return

# scrub:
scrub in all


# nat/rdr ##############################################

# ftp proxy rules
nat-anchor "ftp-proxy/*"
rdr-anchor "ftp-proxy/*"

# nat rules for loadbalancing to internal network:
nat on $ext_if1 from $int_network to any -> ($ext_if1)
nat on $ext_if2 from $int_network to any -> ($ext_if2)

# nat for routing traffice from dmz to DSL connection
nat on $ext_if1 from $dmz_if:network to any -> ($ext_if1)

# nat rule for traffic from dmz to cable connection
nat on $ext_if2 from $dmz_if:network to any -> ($ext_if2)

# redirect from cable IP to sslbox for secondary connection to server
# (the sslbox uses relayd as an https proxy to offload encryption from the actual server)
# I'm using port 9090 since my cable ISP seems to be blocking access to 443
rdr log on $ext_if2 proto tcp from any to $cable_ext_addr port 9090 \
        -> $sslbox_dmz_addr port 8080 

# This redirects attempts to connect to the server through its cable address to the sslbox:
rdr pass on $int_if proto tcp from any to $cable_ext_addr port 9090 \
        -> $sslbox_dmz_addr port 8080

# redirects to the same server through its DSL address & ports
rdr pass on $ext_if1  proto tcp from any to $server_dsl_addr port \
        $server_ext_ports -> $server_dmz_addr
rdr pass on $int_if proto tcp from any to $server_dsl_addr port \
        $server_int_ports -> $server_dmz_addr

# rules to redirect to ssl box for encrypted connection
rdr pass on $ext_if1 proto tcp from any to $server_dsl_addr port \
        443 -> $sslbox_dmz_addr port 8080
rdr pass on $int_if proto tcp from any to $server_dmz_addr port \
        443 -> $sslbox_dmz_addr port 8080

## filter rules  #############################################

# Start with default block all rule:
block log all

# Open up the loopback:
pass quick on lo0 all

# Anti-spoofing rules for the WAN interfaces:
block drop in quick on { $ext_if1, $ext_if2 } from $priv_nets to any
block drop out quick on { $ext_if1, $ext_if2 } from any to $priv_nets
antispoof for $ext_if1 inet
antispoof for $ext_if2 inet 

# load-balancing on outgoing connection from internal network:

pass in on $int_if route-to \
     { ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
     proto tcp from $int_network to any flags S/SA modulate state
        
pass in on $int_if route-to \
     { ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
     proto { udp, icmp } from $int_network to any keep state

# The next two rules were essential for getting the cable connection to the web server to work.  Thanks hednod!
pass in quick log on $ext_if2 reply-to ( $ext_if2 $ext_gw2 ) proto tcp from any to $sslbox_dmz_addr port 8080 tag circ2
pass out quick log on $ext_if2 tagged circ2

# Accept connections on router's WAN interface for services running
# on the router itself.
pass in on $ext_if1 inet proto tcp from any to $dsl_ext_addrs \
    port $router_tcp_services flags S/SA keep state
pass in on $ext_if2 inet proto tcp from any to $cable_ext_addr \
    port $router_tcp_services flags S/SA keep state
pass in on $ext_if1 inet proto icmp all icmp-type $icmp_types
pass in on $ext_if2 inet proto icmp all icmp-type $icmp_types

# Pass all traffic in and out of LAN interface, except
# initially block all traffic between LAN and DMZ.
pass in on $int_if from $int_if:network to any 
pass out on $int_if from any to $int_if:network
block in on $int_if from $int_if:network to $dmz_if:network
block out on $int_if from $dmz_if:network to $int_if:network

# pass rules for loadbalancing to ensure traffic goes out the way it came:
pass out quick on $ext_if1 route-to ($ext_if2 $ext_gw2) from ($ext_if2) to any keep state
pass out quick on $ext_if2 route-to ($ext_if1 $ext_gw1) from ($ext_if1) to any keep state

# Allow traffic from some dmz servers to internal network
pass out quick on $int_if proto tcp from $server_dmz_addr to $int_if:network \
   port $server_int_ports flags S/SA keep state

pass out quick on $int_if proto tcp from $sslbox_dmz_addr to $int_if:network \
   port $sslbox_ports flags S/SA keep state

# Pass redirects into DMZ from LAN.
pass in on $int_if inet proto tcp from $int_if:network \
    to $server_dmz_addr port $koha_int_ports flags S/SA keep state

pass in on $int_if inet proto tcp from $int_if:network \
   to $sslbox_dmz_addr flags S/SA keep state

# Permit connections from router to selected ports on DMZ servers.
# This traffic passes through DMZ interface and can originate from the
# outside world or from LAN.

pass out on $dmz_if proto tcp from any to $server_dmz_addr \
    port $server_int_ports flags S/SA keep state

pass out on $dmz_if inet proto tcp from any to $sslbox_dmz_addr \
    port $sslbox_ports flags S/SA keep state

# rules for passing into dmz from internal network
pass in on $dmz_if proto tcp from $int_if:network to $server_dmz_addr \
     port $server_int_ports flags S/SA keep state

pass in on $dmz_if proto tcp from $int_if:network to $sslbox_dmz_addr \
    flags S/SA keep state

# Pass all inbound traffic from DMZ interface, keep state on connections.
pass in on $dmz_if proto tcp all modulate state flags S/SA
pass in on $dmz_if proto { udp, icmp } all keep state

# Pass outbound icmp echo requests through DMZ interface, keep state.
pass out on $dmz_if inet proto icmp all icmp-type $icmp_types keep state

# Pass all outbound traffic through WAN interfaces, keep state.
pass out quick on $ext_if1 proto tcp all modulate state flags S/SA
pass out quick on $ext_if1 proto { udp, icmp } all keep state
pass out quick on $ext_if2 proto tcp all modulate state flags S/SA
pass out quick on $ext_if2 proto { udp, icmp } all keep state













Looking for something?

Use the form below to search the wiki:

 

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!