[j-nsp] STP in spine leaf architecture

[Mehul gajjar]

Dear all,

Can anybody give me knowledge how Spanning tree behaviour in spine/leaf data center architecture where qfx as a spine and leaf too.

Sent from my iPad
_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Hugo Slabbert]

Post by Mehul gajjar
Dear all,
Can anybody give me knowledge how Spanning tree behaviour in spine/leaf
data center architecture where qfx as a spine and leaf too.

I haven't done spine/leaf networks with QFX, but in a general a spine &
leaf setup should be L3 for interswitch links, so any STP should be local
to a given switch. i.e. you may have STP domains on a leaf's local L2
domain, so on its edge ports / server-facing links, but that should not
extend upwards in the topology unless I'm missing something.

Here I'm just talking about a vanilla spine & leaf setup, not anything
Juniper-specific e.g. QFabric or VCF or whatnot.

If you're doing an overlay like VXLAN over top of the L3 underlay, then how
loops are managed would be specific to the overlay, with the addendum that
even if you manage to form a loop in the overlay somewhere/somehow, the
TTLs in the L3 underlay should still have looped frames in the overlay TTL
out.

I'd be curious about more specific details from folks running QFX in prod
in this type of setup.

[Thomas Bellman]

[...] in a general a spine & leaf setup should be L3 for interswitch
links, so any STP should be local to a given switch. [...]
Here I'm just talking about a vanilla spine & leaf setup, not anything
Juniper-specific e.g. QFabric or VCF or whatnot.

You can also build a spine & leaf setup using TRILL och Shortest Path
Bridging (SPB), in which case you have a single large layer 2-domain.
Not using Juniper equipment, though, since Juniper supports neither
TRILL nor SPB...

I'd be curious about more specific details from folks running QFX in
prod in this type of setup.

You are generally correct though. Configure your swithc-to-switch
links as L3 ports (i.e. 'interface ... unit ... family inet/inet6',
not 'family ethernet-switching'), and some routing protocol like
OSPF, IS-IS or BGP. BGP is fairly popular in datacenter settings,
but OSPF works fine as well, as should IS-IS.

Layer 2 domains should be kept to a single leaf switch, and thus you
don't need to run Spanning Tree at all. And definitely not on your
links between spines and leafs, since that would block all but one of
the uplinks, and give you all the pains of Spanning Tree without any
of the benefits. (You *might* want to run STP on your client ports and
configure them as edge ports with bpdu-block-on-edge, to protect against
someone misadvertently connecting two L2 client ports togethere.)

(I don't run a pure spine-and-leaf network myself. I am trying to
migrate towards one, but we still have several "impurities", and
have STP running in several places.)

[Hugo Slabbert]

Post by Thomas Bellman

[...] in a general a spine & leaf setup should be L3 for interswitch
links, so any STP should be local to a given switch. [...]
Here I'm just talking about a vanilla spine & leaf setup, not anything
Juniper-specific e.g. QFabric or VCF or whatnot.

You can also build a spine & leaf setup using TRILL och Shortest Path
Bridging (SPB), in which case you have a single large layer 2-domain.
Not using Juniper equipment, though, since Juniper supports neither
TRILL nor SPB...

A fair point; TRILL was only somewhat in the mix when we were evaluating
options, but vendor support was hit and miss. VXLAN ended up being a more
common and "vetted" solution for L2 across a spine & leaf setup.

Post by Thomas Bellman

I'd be curious about more specific details from folks running QFX in
prod in this type of setup.

You are generally correct though. Configure your swithc-to-switch
links as L3 ports (i.e. 'interface ... unit ... family inet/inet6',
not 'family ethernet-switching'), and some routing protocol like
OSPF, IS-IS or BGP. BGP is fairly popular in datacenter settings,
but OSPF works fine as well, as should IS-IS.
Layer 2 domains should be kept to a single leaf switch, and thus you
don't need to run Spanning Tree at all. And definitely not on your
links between spines and leafs, since that would block all but one of
the uplinks, and give you all the pains of Spanning Tree without any
of the benefits. (You *might* want to run STP on your client ports and
configure them as edge ports with bpdu-block-on-edge, to protect against
someone misadvertently connecting two L2 client ports togethere.)

Yep; that's our CYA config.

Post by Thomas Bellman
(I don't run a pure spine-and-leaf network myself. I am trying to
migrate towards one, but we still have several "impurities", and
have STP running in several places.)

We all still have lots of "dirty corners" in our networks ;)

[Vincent Bernat]

Post by Mehul gajjar
Can anybody give me knowledge how Spanning tree behaviour in
spine/leaf data center architecture where qfx as a spine and leaf too.

Hello,

There is no loop, so you don't need STP. You can still use it by placing
all leaf ports as edge.