[j-nsp] SRX650 cluster - ethernet switching issue

[Paulhamus, Jon]

Hello group -

I have a pair of SRX 650's running in a cluster -

My issue is that I have 2 trunk links on each firewall passing completely different VLAN's but when I enable any form of spanning tree, I'm seeing one of those links blocked (3 out of the 4 links get blocked by STP). I've tried rstp, stp and mstp - all with the same issue. The switches in use are 1- EX-4500, and 2 EX-4200's in a VC. I may have a config issue - or is this possibly a bug? Any help would be greatly appreciated!

Here is the relevant configuration:

ports 2/0/8 and 2/0/9 are my trunk links on each firewall
port 2/0/15 is the swfab on each firewall

------------------

ge-2/0/8 {
unit 0 {
family ethernet-switching {
port-mode trunk;
vlan {
members [ 200 201 250-260 ];
}
}
}
}
ge-2/0/9 {
unit 0 {
family ethernet-switching {
port-mode trunk;
vlan {
members [ 300 400 401 500 850 900 1701 1753 ];
}
}
}
}

ge-11/0/8 {
unit 0 {
family ethernet-switching {
port-mode trunk;
vlan {
members [ 200 201 250-260 ];
}
}
}
}
ge-11/0/9 {
unit 0 {
family ethernet-switching {
port-mode trunk;
vlan {
members [ 300 400 401 500 850 900 1701 1753 ];
}
}
}
}

swfab0 {
fabric-options {
member-interfaces {
ge-2/0/15;
}
}
}
swfab1 {
fabric-options {
member-interfaces {
ge-11/0/15;
}
}
}


----------------------

> show chassis cluster ethernet-switching status
Cluster ID: 1
Node Priority Status Preempt Manual failover
Redundancy group: 0 , Failover count: 1
node0 100 primary no no
node1 1 secondary no no
Redundancy group: 1 , Failover count: 1
node0 100 primary yes no
node1 1 secondary yes no
Ethernet switching status:
Probe state is UP. Both nodes are in single ethernet switching domain(s).


> show chassis cluster ethernet-switching interfaces
swfab0:
Name Status
ge-2/0/15 up
swfab1:
Name Status
ge-11/0/15 up
















Privileged and Confidential:
The information contained in this message and any attachments hereto is intended solely for the use of the individual or entity to which it was addressed, and may contain confidential or privileged information. If you have received this message in error, please notify the sender and delete the message. The unauthorized use, disclosure, duplication or alteration of this message is strictly forbidden. Although BLaST IU 17 has taken precautions to ensure no viruses are present in this communication, BLaST accepts no responsibility for any loss or damage arising from the use of this message or attachments. BLaST additionally accepts no responsibility for any non-business related content.

[Ben Dale]

Hi John,

>
> My issue is that I have 2 trunk links on each firewall passing completely different VLAN's but when I enable any form of spanning tree, I'm seeing one of those links blocked (3 out of the 4 links get blocked by STP). I've tried rstp, stp and mstp - all with the same issue.

This is expected behaviour. Neither RSTP nor STP are VLAN-aware, so they simply see a topology containing 3 bridges (SRX, EX, EX-VC) in a loop and block the port "furtherest" from the root bridge.

A simple fix would be VSTP (per-VLAN Spanning-Tree), but the SRX platform didn't support it last time I checked.

You can use MSTP can solve this issue by allowing multiple forwarding topologies, but it will require specific configuration all three devices - if you simply enable it with defaults, it will behave exactly the same way as RSTP.

Plenty of info on the specifics of MSTP can be found here:

http://www.juniper.net/techpubs/en_US/junos9.4/topics/example/spanning-trees-ex-series-mstp-configuring.html
http://kb.juniper.net/library/CUSTOMERSERVICE/technotes/8010065-001-EN.pdf

Good luck!

Ben

[Paulhamus, Jon]

Thank you Ben. I did configure MSTP and saw other issues with the config, but I don't believe that I tried VSTP. I'll give that a go this coming weekend. I appreciate your input!


------------------------





-----Original Message-----
From: Ben Dale [mailto:bdale at comlinx.com.au]
Sent: Monday, January 02, 2012 5:18 AM
To: Paulhamus, Jon
Cc: juniper-nsp at puck.nether.net
Subject: Re: [j-nsp] SRX650 cluster - ethernet switching issue

Hi John,

>
> My issue is that I have 2 trunk links on each firewall passing completely different VLAN's but when I enable any form of spanning tree, I'm seeing one of those links blocked (3 out of the 4 links get blocked by STP). I've tried rstp, stp and mstp - all with the same issue.

This is expected behaviour. Neither RSTP nor STP are VLAN-aware, so they simply see a topology containing 3 bridges (SRX, EX, EX-VC) in a loop and block the port "furtherest" from the root bridge.

A simple fix would be VSTP (per-VLAN Spanning-Tree), but the SRX platform didn't support it last time I checked.

You can use MSTP can solve this issue by allowing multiple forwarding topologies, but it will require specific configuration all three devices - if you simply enable it with defaults, it will behave exactly the same way as RSTP.

Plenty of info on the specifics of MSTP can be found here:

http://www.juniper.net/techpubs/en_US/junos9.4/topics/example/spanning-trees-ex-series-mstp-configuring.html
http://kb.juniper.net/library/CUSTOMERSERVICE/technotes/8010065-001-EN.pdf

Good luck!

Ben

[Павел Лунин]

BTW, never could understand people running L2 on srx650 coupled with a
normal switch. Especially in srx-cluster + ex-vc. What for?
03.01.2012 16:07 ???????????? "Paulhamus, Jon" <jpaulhamus at iu17.org>
???????:

> Thank you Ben. I did configure MSTP and saw other issues with the config,
> but I don't believe that I tried VSTP. I'll give that a go this coming
> weekend. I appreciate your input!
>
>
> ------------------------
>
>
>
>
>
> -----Original Message-----
> From: Ben Dale [mailto:bdale at comlinx.com.au]
> Sent: Monday, January 02, 2012 5:18 AM
> To: Paulhamus, Jon
> Cc: juniper-nsp at puck.nether.net
> Subject: Re: [j-nsp] SRX650 cluster - ethernet switching issue
>
> Hi John,
>
> >
> > My issue is that I have 2 trunk links on each firewall passing
> completely different VLAN's but when I enable any form of spanning tree,
> I'm seeing one of those links blocked (3 out of the 4 links get blocked by
> STP). I've tried rstp, stp and mstp - all with the same issue.
>
> This is expected behaviour. Neither RSTP nor STP are VLAN-aware, so they
> simply see a topology containing 3 bridges (SRX, EX, EX-VC) in a loop and
> block the port "furtherest" from the root bridge.
>
> A simple fix would be VSTP (per-VLAN Spanning-Tree), but the SRX platform
> didn't support it last time I checked.
>
> You can use MSTP can solve this issue by allowing multiple forwarding
> topologies, but it will require specific configuration all three devices -
> if you simply enable it with defaults, it will behave exactly the same way
> as RSTP.
>
> Plenty of info on the specifics of MSTP can be found here:
>
>
> http://www.juniper.net/techpubs/en_US/junos9.4/topics/example/spanning-trees-ex-series-mstp-configuring.html
> http://kb.juniper.net/library/CUSTOMERSERVICE/technotes/8010065-001-EN.pdf
>
> Good luck!
>
> Ben
>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>

[Paulhamus, Jon]

??

Why not? If you have more devices that need access to specific vlan zones on the SRX, and you're low on physical interfaces, why not use a switch. This can be extremely handy when bringing trunks into a VMWare server(s). I'm not sure what you're saying about especially in a cluster either - clustering of the firewalls is soley for redundancy in my situation.

If you think there are better options, I'm opened to recommendations.



From: ????? ????? [plunin at senetsy.ru]
Sent: Friday, January 06, 2012 5:45 AM
To: Paulhamus, Jon
Cc: Ben Dale; juniper-nsp at puck.nether.net
Subject: Re: [j-nsp] SRX650 cluster - ethernet switching issue


BTW, never could understand people running L2 on srx650 coupled with a normal switch. Especially in srx-cluster + ex-vc. What for?

03.01.2012 16:07 ???????????? "Paulhamus, Jon" <jpaulhamus at iu17.org<mailto:jpaulhamus at iu17.org>> ???????:
Thank you Ben. I did configure MSTP and saw other issues with the config, but I don't believe that I tried VSTP. I'll give that a go this coming weekend. I appreciate your input!


------------------------





-----Original Message-----
From: Ben Dale [mailto:bdale at comlinx.com.au<mailto:bdale at comlinx.com.au>]
Sent: Monday, January 02, 2012 5:18 AM
To: Paulhamus, Jon
Cc: juniper-nsp at puck.nether.net<mailto:juniper-nsp at puck.nether.net>
Subject: Re: [j-nsp] SRX650 cluster - ethernet switching issue

Hi John,

>
> My issue is that I have 2 trunk links on each firewall passing completely different VLAN's but when I enable any form of spanning tree, I'm seeing one of those links blocked (3 out of the 4 links get blocked by STP). I've tried rstp, stp and mstp - all with the same issue.

This is expected behaviour. Neither RSTP nor STP are VLAN-aware, so they simply see a topology containing 3 bridges (SRX, EX, EX-VC) in a loop and block the port "furtherest" from the root bridge.

A simple fix would be VSTP (per-VLAN Spanning-Tree), but the SRX platform didn't support it last time I checked.

You can use MSTP can solve this issue by allowing multiple forwarding topologies, but it will require specific configuration all three devices - if you simply enable it with defaults, it will behave exactly the same way as RSTP.

Plenty of info on the specifics of MSTP can be found here:

http://www.juniper.net/techpubs/en_US/junos9.4/topics/example/spanning-trees-ex-series-mstp-configuring.html
http://kb.juniper.net/library/CUSTOMERSERVICE/technotes/8010065-001-EN.pdf

Good luck!

Ben

_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net<mailto:juniper-nsp at puck.nether.net>
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Pavel Lunin]

Sorry, missed this reply because of the new year holidays.

>> BTW, never could understand people running L2 on srx650 coupled with a normal switch. Especially in srx-cluster + ex-vc. What for?
> Why not? If you have more devices that need access to specific vlan zones on the SRX, and you're low on physical interfaces, why not use a switch. This can be extremely handy when bringing trunks into a VMWare server(s).
>

When you build a FW cluster, you anyway must have a pair of supporting
switches in almost all sorts of design. Either each SRX connected to its
own switch (I prefer this) or full mesh (people like this but there is
no much sense, imho). So in terms of the number of physical ports, it
seems like this is not the SRX's job (in most cases). Although in case
of port deficit, this can be a kind of workaround, I agree.

> I'm not sure what you're saying about especially in a cluster either - clustering of the firewalls is soley for redundancy in my situation.

If you physically connect something to SRX in cluster mode, not to the
supporting switches, it becomes complicated to teach the device to
switch traffic to a different SRX node in case of a failure. Say, you
have SRX1 and SRX2 in cluster. SRX1 connected to SW1 and SRX2 connected
to SW2. SRX1 is primary for RG0 and RG1. Say, SW1 and SW2 form a VC. You
have some devices connected to the VC (most probably using LAGs) and
some devices connected to the SRXes itself (LAG is not supported here,
AFAIR). Let's say SW1 fails and SRX1-SW1 link does so as well. RG1
switches to the backup node SRX2. But how the directly connected device
will know it should forward traffic to the second node? In case it still
send it to SRX1, this will lead to h-shape forwarding through the swfab
link (not good). xSTP can help, but it moves the solution further from
best (I don't even want to think of what can happen with STP in case of
SRX's RG0 switchover). You anyway must run xSTP though, since you now
have two switch nodes instead of one.

Add here operational expenses of managing and troubleshooting switching
stuff of SRXs instead of just on VC and lack of some switching features.
I think, it's cheaper overall to just add port-capacity to the switch
cluster. So, while it does work in principle, as a design for a new
setup, I'd say, it's a bit clumsy.

[Paulhamus, Jon]

Thanks for the reply.

The case what not that I was connecting end points directly to the SRX, it's that I wanted 2 trunk links between each SRX node and the switch stack (there is only 1 switch stack), ?each of the 2 links supporting different VLAN's. So - 2 links from the primary node to the switch stack, and 2 links from the secondary node to the switch stack as well... this does not work with STP enabled as 3 of the 4 links get blocked by STP. I needed a setup like this as I'm pushing over 1Gb through each switch link.







-----Original Message-----
From: Pavel Lunin [mailto:plunin at senetsy.ru]
Sent: Monday, January 16, 2012 9:32 AM
To: Paulhamus, Jon
Cc: Ben Dale; juniper-nsp at puck.nether.net
Subject: Re: [j-nsp] SRX650 cluster - ethernet switching issue



Sorry, missed this reply because of the new year holidays.

>> BTW, never could understand people running L2 on srx650 coupled with a normal switch. Especially in srx-cluster + ex-vc. What for?
> Why not? If you have more devices that need access to specific vlan zones on the SRX, and you're low on physical interfaces, why not use a switch. This can be extremely handy when bringing trunks into a VMWare server(s).
>

When you build a FW cluster, you anyway must have a pair of supporting switches in almost all sorts of design. Either each SRX connected to its own switch (I prefer this) or full mesh (people like this but there is no much sense, imho). So in terms of the number of physical ports, it seems like this is not the SRX's job (in most cases). Although in case of port deficit, this can be a kind of workaround, I agree.

> I'm not sure what you're saying about especially in a cluster either - clustering of the firewalls is soley for redundancy in my situation.

If you physically connect something to SRX in cluster mode, not to the supporting switches, it becomes complicated to teach the device to switch traffic to a different SRX node in case of a failure. Say, you have SRX1 and SRX2 in cluster. SRX1 connected to SW1 and SRX2 connected to SW2. SRX1 is primary for RG0 and RG1. Say, SW1 and SW2 form a VC. You have some devices connected to the VC (most probably using LAGs) and some devices connected to the SRXes itself (LAG is not supported here, AFAIR). Let's say SW1 fails and SRX1-SW1 link does so as well. RG1 switches to the backup node SRX2. But how the directly connected device will know it should forward traffic to the second node? In case it still send it to SRX1, this will lead to h-shape forwarding through the swfab link (not good). xSTP can help, but it moves the solution further from best (I don't even want to think of what can happen with STP in case of SRX's RG0 switchover). You anyway must run xSTP though, since you now have two switch nodes instead of one.

Add here operational expenses of managing and troubleshooting switching stuff of SRXs instead of just on VC and lack of some switching features.
I think, it's cheaper overall to just add port-capacity to the switch cluster. So, while it does work in principle, as a design for a new setup, I'd say, it's a bit clumsy.

[Morgan McLean]

Is there any reason you aren't using reth groups? You can do an AE with
those.

Morgan

On Mon, Jan 16, 2012 at 10:44 AM, Paulhamus, Jon <jpaulhamus at iu17.org>wrote:

> Thanks for the reply.
>
> The case what not that I was connecting end points directly to the SRX,
> it's that I wanted 2 trunk links between each SRX node and the switch stack
> (there is only 1 switch stack), each of the 2 links supporting different
> VLAN's. So - 2 links from the primary node to the switch stack, and 2
> links from the secondary node to the switch stack as well... this does not
> work with STP enabled as 3 of the 4 links get blocked by STP. I needed a
> setup like this as I'm pushing over 1Gb through each switch link.
>
>
>
>
>
>
>
> -----Original Message-----
> From: Pavel Lunin [mailto:plunin at senetsy.ru]
> Sent: Monday, January 16, 2012 9:32 AM
> To: Paulhamus, Jon
> Cc: Ben Dale; juniper-nsp at puck.nether.net
> Subject: Re: [j-nsp] SRX650 cluster - ethernet switching issue
>
>
>
> Sorry, missed this reply because of the new year holidays.
>
> >> BTW, never could understand people running L2 on srx650 coupled with a
> normal switch. Especially in srx-cluster + ex-vc. What for?
> > Why not? If you have more devices that need access to specific vlan
> zones on the SRX, and you're low on physical interfaces, why not use a
> switch. This can be extremely handy when bringing trunks into a VMWare
> server(s).
> >
>
> When you build a FW cluster, you anyway must have a pair of supporting
> switches in almost all sorts of design. Either each SRX connected to its
> own switch (I prefer this) or full mesh (people like this but there is no
> much sense, imho). So in terms of the number of physical ports, it seems
> like this is not the SRX's job (in most cases). Although in case of port
> deficit, this can be a kind of workaround, I agree.
>
> > I'm not sure what you're saying about especially in a cluster either -
> clustering of the firewalls is soley for redundancy in my situation.
>
> If you physically connect something to SRX in cluster mode, not to the
> supporting switches, it becomes complicated to teach the device to switch
> traffic to a different SRX node in case of a failure. Say, you have SRX1
> and SRX2 in cluster. SRX1 connected to SW1 and SRX2 connected to SW2. SRX1
> is primary for RG0 and RG1. Say, SW1 and SW2 form a VC. You have some
> devices connected to the VC (most probably using LAGs) and some devices
> connected to the SRXes itself (LAG is not supported here, AFAIR). Let's say
> SW1 fails and SRX1-SW1 link does so as well. RG1 switches to the backup
> node SRX2. But how the directly connected device will know it should
> forward traffic to the second node? In case it still send it to SRX1, this
> will lead to h-shape forwarding through the swfab link (not good). xSTP can
> help, but it moves the solution further from best (I don't even want to
> think of what can happen with STP in case of SRX's RG0 switchover). You
> anyway must run xSTP though, since you now have two switch nodes instead of
> one.
>
> Add here operational expenses of managing and troubleshooting switching
> stuff of SRXs instead of just on VC and lack of some switching features.
> I think, it's cheaper overall to just add port-capacity to the switch
> cluster. So, while it does work in principle, as a design for a new setup,
> I'd say, it's a bit clumsy.
>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>

[Павел Лунин]

> The case what not that I was connecting end points directly to the SRX,
> it's that I wanted 2 trunk links between each SRX node and the switch stack
> (there is only 1 switch stack), each of the 2 links supporting different
> VLAN's. So - 2 links from the primary node to the switch stack, and 2
> links from the secondary node to the switch stack as well... this does not
> work with STP enabled as 3 of the 4 links get blocked by STP. I needed a
> setup like this as I'm pushing over 1Gb through each switch link.
>
>
My favorite design is to connect each SRX to its own (no cross-links)
VC-member using several physical links (as many as you need for
performance) and to use L3 on top of reth VLAN units, each of which
includes two physical links (primary and secondary). You can distribute
VLANs across reths manually, just as you would do for STP (if it worked).
At the SRX side you have several reth interfaces with units on them, at the
VC side you use interface-range for bundling ports connected to primary and
secondary SRX into a single config knob, which is really useful. Of course,
on SRXes, when you move a VLAN X from, say, reth0 to reth1, you also need
to replace reth0.X to reth1.X in the rest of config (zones, etc) but JUNOS
allows you to do so with a single command.

You can either bound all the reths into a single RG1 (always
active/passive) or put each reth into a dedicated RG (partial
active/active). You can even easily make different nodes act as master for
different RGs, though in most cases it will give nothing except pain in the
rear. I love the first approach (one RG1 for all reths) because it's much
simpler. RG1/2/3/etc failover is quite fast (sufficient for most
applications) so failing over all the reths to backup node in case only one
of the primary node links fails is not a very high price for the simplicity.

A small drawback of this design is h-shape switching, if you connect
downlink devices to the VC with LAGs. Traffic will come to the VC balanced
between the switches connected to active and backup SRXes. So the VC will
first forward the frames to the switch, connected to the active SRX via
internal VC link. But if we consider a VC of 2?EX4200 connected with two VC
cables, this is not much of a problem.

You can also add cross-links and bound four interfaces into a single
LAG/reth (a hybrid of the two) on SRX (two links on master are active, two
on backup are down), but JUNOS does not allow to bound two ae interfaces
into an interface range on the EX side. So for each reth on SRX you will
have two ae's on EX.This is hateful and prone to mistakes. On the other
hand this approach will give you the ability to balance traffic across two
links (SRX1?EX1 and SRX1?EX2) automatically with hashing (limited to two
links only) and additional speed of switching over LAG members in case of a
failure. When switches are connected not with 2?32G cables but something
else (1G llinks or 10G and it's an SRX High End cluster) this approach will
also save some resources because of avoiding interswitch forwarding, as
described above. Though in my experience it's not usually worth the 'two on
EX ? one on SRX' interface matching headache.

--
Regards,
Pavel

[Paulhamus, Jon]

Hi Pavel -

I believe that what you are describing is exactly what I'm looking for, but I did not realize this was the proper way to configure. Can you please provide a configuration example?

Thanks again for all of the input.


________________________________
From: ????? ????? [plunin at senetsy.ru]
Sent: Monday, January 16, 2012 3:58 PM
To: Paulhamus, Jon
Cc: Ben Dale; juniper-nsp at puck.nether.net
Subject: Re: [j-nsp] SRX650 cluster - ethernet switching issue


The case what not that I was connecting end points directly to the SRX, it's that I wanted 2 trunk links between each SRX node and the switch stack (there is only 1 switch stack), each of the 2 links supporting different VLAN's. So - 2 links from the primary node to the switch stack, and 2 links from the secondary node to the switch stack as well... this does not work with STP enabled as 3 of the 4 links get blocked by STP. I needed a setup like this as I'm pushing over 1Gb through each switch link.


My favorite design is to connect each SRX to its own (no cross-links) VC-member using several physical links (as many as you need for performance) and to use L3 on top of reth VLAN units, each of which includes two physical links (primary and secondary). You can distribute VLANs across reths manually, just as you would do for STP (if it worked). At the SRX side you have several reth interfaces with units on them, at the VC side you use interface-range for bundling ports connected to primary and secondary SRX into a single config knob, which is really useful. Of course, on SRXes, when you move a VLAN X from, say, reth0 to reth1, you also need to replace reth0.X to reth1.X in the rest of config (zones, etc) but JUNOS allows you to do so with a single command.

You can either bound all the reths into a single RG1 (always active/passive) or put each reth into a dedicated RG (partial active/active). You can even easily make different nodes act as master for different RGs, though in most cases it will give nothing except pain in the rear. I love the first approach (one RG1 for all reths) because it's much simpler. RG1/2/3/etc failover is quite fast (sufficient for most applications) so failing over all the reths to backup node in case only one of the primary node links fails is not a very high price for the simplicity.

A small drawback of this design is h-shape switching, if you connect downlink devices to the VC with LAGs. Traffic will come to the VC balanced between the switches connected to active and backup SRXes. So the VC will first forward the frames to the switch, connected to the active SRX via internal VC link. But if we consider a VC of 2?EX4200 connected with two VC cables, this is not much of a problem.

You can also add cross-links and bound four interfaces into a single LAG/reth (a hybrid of the two) on SRX (two links on master are active, two on backup are down), but JUNOS does not allow to bound two ae interfaces into an interface range on the EX side. So for each reth on SRX you will have two ae's on EX.This is hateful and prone to mistakes. On the other hand this approach will give you the ability to balance traffic across two links (SRX1?EX1 and SRX1?EX2) automatically with hashing (limited to two links only) and additional speed of switching over LAG members in case of a failure. When switches are connected not with 2?32G cables but something else (1G llinks or 10G and it's an SRX High End cluster) this approach will also save some resources because of avoiding interswitch forwarding, as described above. Though in my experience it's not usually worth the 'two on EX ? one on SRX' interface matching headache.

--
Regards,
Pavel

[Paulhamus, Jon]

Are you suggesting something aong these lines?

Interfaces for

ge-2/0/4 {
description "*** Connected to XXX***";
gigether-options {
redundant-parent reth4;

ge-11/0/4 {
description "*** Connected to XXX***";
gigether-options {
redundant-parent reth4;

ge-2/0/5 {
description "*** Connected to XXX***";
gigether-options {
redundant-parent reth5;

ge-11/0/5 {
description "*** Connected to XXX***";
gigether-options {
redundant-parent reth5;


set interfaces reth4 vlan-tagging
set interfaces reth4 description "*** Connected to XXX TRUNK #1 ***";
set interfaces reth4 redundant-ether-options redundancy-group 4
set interfaces reth4 unit 200 vlan-id 200
set interfaces reth4 unit 200 family inet address 192.168.200.0/24
set interfaces reth4 unit 201 vlan-id 201
set interfaces reth4 unit 201 family inet address 192.168.201.0/24
set interfaces reth4 unit 202 vlan-id 202
set interfaces reth4 unit 202 family inet address 192.168.202.0/24

set interfaces reth5 vlan-tagging
set interfaces reth5 description "*** Connected to XXX TRUNK #2 ***";
set interfaces reth5 redundant-ether-options redundancy-group 5
set interfaces reth5 unit 205 vlan-id 205
set interfaces reth5 unit 205 family inet address 192.168.205.0/24
set interfaces reth5 unit 206 vlan-id 206
set interfaces reth5 unit 206 family inet address 192.168.206.0/24
set interfaces reth5 unit 207 vlan-id 207
set interfaces reth5 unit 207 family inet address 192.168.207.0/24

Then, on the EX side - just simple trunk configuration permitting said VLAN's for those ports?


Thank you again.

________________________________
From: ????? ????? [plunin at senetsy.ru]
Sent: Monday, January 16, 2012 3:58 PM
To: Paulhamus, Jon
Cc: Ben Dale; juniper-nsp at puck.nether.net
Subject: Re: [j-nsp] SRX650 cluster - ethernet switching issue


The case what not that I was connecting end points directly to the SRX, it's that I wanted 2 trunk links between each SRX node and the switch stack (there is only 1 switch stack), each of the 2 links supporting different VLAN's. So - 2 links from the primary node to the switch stack, and 2 links from the secondary node to the switch stack as well... this does not work with STP enabled as 3 of the 4 links get blocked by STP. I needed a setup like this as I'm pushing over 1Gb through each switch link.


My favorite design is to connect each SRX to its own (no cross-links) VC-member using several physical links (as many as you need for performance) and to use L3 on top of reth VLAN units, each of which includes two physical links (primary and secondary). You can distribute VLANs across reths manually, just as you would do for STP (if it worked). At the SRX side you have several reth interfaces with units on them, at the VC side you use interface-range for bundling ports connected to primary and secondary SRX into a single config knob, which is really useful. Of course, on SRXes, when you move a VLAN X from, say, reth0 to reth1, you also need to replace reth0.X to reth1.X in the rest of config (zones, etc) but JUNOS allows you to do so with a single command.

You can either bound all the reths into a single RG1 (always active/passive) or put each reth into a dedicated RG (partial active/active). You can even easily make different nodes act as master for different RGs, though in most cases it will give nothing except pain in the rear. I love the first approach (one RG1 for all reths) because it's much simpler. RG1/2/3/etc failover is quite fast (sufficient for most applications) so failing over all the reths to backup node in case only one of the primary node links fails is not a very high price for the simplicity.

A small drawback of this design is h-shape switching, if you connect downlink devices to the VC with LAGs. Traffic will come to the VC balanced between the switches connected to active and backup SRXes. So the VC will first forward the frames to the switch, connected to the active SRX via internal VC link. But if we consider a VC of 2?EX4200 connected with two VC cables, this is not much of a problem.

You can also add cross-links and bound four interfaces into a single LAG/reth (a hybrid of the two) on SRX (two links on master are active, two on backup are down), but JUNOS does not allow to bound two ae interfaces into an interface range on the EX side. So for each reth on SRX you will have two ae's on EX.This is hateful and prone to mistakes. On the other hand this approach will give you the ability to balance traffic across two links (SRX1?EX1 and SRX1?EX2) automatically with hashing (limited to two links only) and additional speed of switching over LAG members in case of a failure. When switches are connected not with 2?32G cables but something else (1G llinks or 10G and it's an SRX High End cluster) this approach will also save some resources because of avoiding interswitch forwarding, as described above. Though in my experience it's not usually worth the 'two on EX ? one on SRX' interface matching headache.

--
Regards,
Pavel

[stasm]

On Mon, Jan 16, 2012 at 6:31 PM, Pavel Lunin <plunin at senetsy.ru> wrote:

......


> Either each SRX connected to its own switch (I prefer this) or full mesh
> (people like this but there is no much sense, imho). So in terms of the
> number of physical ports, it seems like this is not the SRX's job (in most
> cases). Although in case of port deficit, this can be a kind of workaround,
> I agree.
>
....
cross connection only for cases of double failures. in case of juniper's
devices it is a typical case for juniper ;)