[j-nsp] Router for full routes

[Dovid Bender]

Hi All,

In my 9-5 I work for an ITSP where we have two MX5's with
- iBGP
- two up steams with two BGP sessions each (one per routes)
- one upstream with one bgp session
- one bgp session where we get minimal routes (maybe 15 total)

I was told that convergence time on the MX5 would be horrible so we never
tried full routes. I am wondering what's the "lowest" model that can
support full routes without having an issue re-sorting the routes.

TIA.
_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Mark Tinka]

Post by Dovid Bender
Hi All,
In my 9-5 I work for an ITSP where we have two MX5's with
- iBGP
- two up steams with two BGP sessions each (one per routes)
- one upstream with one bgp session
- one bgp session where we get minimal routes (maybe 15 total)
I was told that convergence time on the MX5 would be horrible so we never
tried full routes. I am wondering what's the "lowest" model that can
support full routes without having an issue re-sorting the routes.

The MX80 has only 2GB of RAM. We have discontinued all our MX80 routers
over the past 12 months as the full table was causing them to lock-up
and reboot.

The MX104 has 4GB of RAM, so you have more headroom. But the CPU is just
as bad as in the MX80.

At this stage, I'd say the cheapest MX router you should go for that is
decent is the MX204.

Mark.
_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Jason Lixfeld]

Post by Mark Tinka
At this stage, I'd say the cheapest MX router you should go for that is
decent is the MX204.

Isn’t the MX204 RE more than decent? 8 core 1.6Ghz, 32GB DDR4 RE sounds like decent is an understatement, no?
_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://pu

[Mark Tinka]

Isn’t the MX204 RE more than decent? 8 core 1.6Ghz, 32GB DDR4 RE sounds like decent is an understatement, no?.

You have to play a little hard to get :-)...

Mark.

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailm

[Tim Jackson]

Yes. Calling it decent is an understatement. It's really quick. It's a Xeon
E5-2608Lv4.

Post by Jason Lixfeld

Post by Mark Tinka
At this stage, I'd say the cheapest MX router you should go for that is
decent is the MX204.

Isn’t the MX204 RE more than decent? 8 core 1.6Ghz, 32GB DDR4 RE sounds
like decent is an understatement, no?
_______________________________________________
https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https:/

[Chris Adams]

Post by Tim Jackson
Yes. Calling it decent is an understatement. It's really quick. It's a Xeon
E5-2608Lv4.

Yep. The RE VM "only" gets half the resources (so 4 cores and 16G RAM),
but that is plenty good! It also has dual NVMe SSDs for storage. When
I upgraded JUNOS from 17.4 to 18.1, I think it only took about 3 minutes
from "request system reboot" until I could SSH back in to the RE
management ethernet.

[Tim Jackson]

Yeah 16G for the RE + I think you actually get 5 cores in the Junos VM:

% sysctl -a | egrep -i 'hw.machine|hw.model|hw.ncpu'
hw.machine: amd64
hw.model: QEMU Virtual CPU version 1.7.2
hw.ncpu: 5
hw.machine_arch: amd64

It's really fast though. Great little box so far.

--
Tim

Post by Chris Adams

Post by Tim Jackson
Yes. Calling it decent is an understatement. It's really quick. It's a

Xeon

Post by Tim Jackson
E5-2608Lv4.

Yep. The RE VM "only" gets half the resources (so 4 cores and 16G RAM),
but that is plenty good! It also has dual NVMe SSDs for storage. When
I upgraded JUNOS from 17.4 to 18.1, I think it only took about 3 minutes
from "request system reboot" until I could SSH back in to the RE
management ethernet.
--
_______________________________________________
https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Jason Lixfeld]

So the rest is for guest VMs then?

Post by Tim Jackson
% sysctl -a | egrep -i 'hw.machine|hw.model|hw.ncpu'
hw.machine: amd64
hw.model: QEMU Virtual CPU version 1.7.2
hw.ncpu: 5
hw.machine_arch: amd64
It's really fast though. Great little box so far.
--
Tim

Post by Chris Adams

Post by Tim Jackson
Yes. Calling it decent is an understatement. It's really quick. It's a

Xeon

Post by Tim Jackson
E5-2608Lv4.

Yep. The RE VM "only" gets half the resources (so 4 cores and 16G RAM),
but that is plenty good! It also has dual NVMe SSDs for storage. When
I upgraded JUNOS from 17.4 to 18.1, I think it only took about 3 minutes
from "request system reboot" until I could SSH back in to the RE
management ethernet.
--
_______________________________________________
https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Tim Jackson]

I think the PFE ukern runs as a process in the hypervisor that uses another
core and a few G of ram:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4479 root 20 0 16.7g 16g 26m S 500 53.0 114757:25 qemu-system-x86
21332 root 20 0 3008m 263m 215m R 135 0.8 30488:26 J-UKERN

--
Tim

Post by Jason Lixfeld
So the rest is for guest VMs then?

Post by Tim Jackson
% sysctl -a | egrep -i 'hw.machine|hw.model|hw.ncpu'
hw.machine: amd64
hw.model: QEMU Virtual CPU version 1.7.2
hw.ncpu: 5
hw.machine_arch: amd64
It's really fast though. Great little box so far.
--
Tim

Post by Chris Adams

Post by Tim Jackson
Yes. Calling it decent is an understatement. It's really quick. It's a

Xeon

Post by Tim Jackson
E5-2608Lv4.

Yep. The RE VM "only" gets half the resources (so 4 cores and 16G RAM),
but that is plenty good! It also has dual NVMe SSDs for storage. When
I upgraded JUNOS from 17.4 to 18.1, I think it only took about 3 minutes
from "request system reboot" until I could SSH back in to the RE
management ethernet.
--
_______________________________________________
https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Mark Tinka]

Post by Chris Adams
Yep. The RE VM "only" gets half the resources (so 4 cores and 16G RAM),
but that is plenty good! It also has dual NVMe SSDs for storage. When
I upgraded JUNOS from 17.4 to 18.1, I think it only took about 3 minutes
from "request system reboot" until I could SSH back in to the RE
management ethernet.

That's one of the things I enjoy about running OS's in VM's - the reboot
is super quick!

FreeBSD booting in ESXi vs. on bare metal is a world apart!

Mark.
_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Rob Foehl]

Post by Mark Tinka
At this stage, I'd say the cheapest MX router you should go for that is
decent is the MX204.

Any thoughts on MX204s replacing ancient MX240s, assuming one can make the
interface mix work?

I'm looking at the replacement option vs. in-place upgrades of a mixed bag
of old RE/SCB/DPC/MPC parts... Seems like an obvious win in cases with
only a handful of 10G ports, less so otherwise.

-Rob
_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Mark Tinka]

Post by Rob Foehl
Any thoughts on MX204s replacing ancient MX240s, assuming one can make
the interface mix work?
I'm looking at the replacement option vs. in-place upgrades of a mixed
bag of old RE/SCB/DPC/MPC parts...  Seems like an obvious win in cases
with only a handful of 10G ports, less so otherwise.

We just did a round of upgrades where small edge PoP's on the MX104 were
moved to the MX480.

We also did another round of upgrades where peering and transit boxes on
the MX80 and MX104 were moved to the MX480.

This was all before the MX204 was even a commercial reality, so we
missed that boat.

Ideally, I'd be keen to run the MX204 in a peering or transit role, as
we don't really need the hardware redundancy, given we have exit PoP's
all over the world.

The MX204 is also ideal, for us, to run a 100Gbps Metro-E ring where
customers also need 10Gbps hand-off. It would also drive our existing
ASR920 Metro-E devices, eliminating the need to run the ring on DWDM.

But to your question, there is nothing ancient about the MX240. It's
just small. Look at your future needs and consider whether having those
2 line card slots running the latest-generation Trio chip will scale
better than migrating to the MX204, and that should answer your question.

Mark.
_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.net

[Rob Foehl]

But to your question, there is nothing ancient about the MX240. It's just
small. Look at your future needs and consider whether having those 2 line
card slots running the latest-generation Trio chip will scale better than
migrating to the MX204, and that should answer your question.

Thanks for the detailed reply, Mark.

By "ancient", I mean boxes still running RE-S-1300s, original SCBs, and
either DPCs or older MPC2s -- basically, everything EOL except the
chassis, and running a mix of 1G and 10G interfaces. The limited slot
count isn't much of an issue, especially with the possibility of moving to
at least MPC3Es with 10x10G MICs.

The REs are the biggest issue, stuck on old code and not nearly enough
memory. 1G interfaces are also a problem, but switches are cheap...

I do like the idea of the MX204 as an edge box, currently have some MX80s
in that role that wouldn't be missed.

-Rob
_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Mark Tinka]

Post by Rob Foehl
 
Thanks for the detailed reply, Mark.
By "ancient", I mean boxes still running RE-S-1300s, original SCBs,
and either DPCs or older MPC2s -- basically, everything EOL except the
chassis, and running a mix of 1G and 10G interfaces.  The limited slot
count isn't much of an issue, especially with the possibility of
moving to at least MPC3Es with 10x10G MICs.
The REs are the biggest issue, stuck on old code and not nearly enough
memory.  1G interfaces are also a problem, but switches are cheap...
I do like the idea of the MX204 as an edge box, currently have some
MX80s in that role that wouldn't be missed.

Fair point, bringing an MX240/480/960 chassis from 2009 up-to-scratch in
2018 could be costlier than going for an MX204. My suggestion would be
that if you want to retain the benefits of a chassis, dump the MX240 and
move to an MX480, and put in the new line cards and RE.

Otherwise, if you have a limited budget and need more bang for you $$
right away, the MX204 will be a better option. But keep in my mind that
this may or may not be good enough for your long term plans, depending
on your use-case.

Mark.
_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.net

[Gert Doering]

Hi,

Post by Dovid Bender
I was told that convergence time on the MX5 would be horrible so we never
tried full routes. I am wondering what's the "lowest" model that can
support full routes without having an issue re-sorting the routes.

How much throughput do you need?

MX150 might be an alternative... "no (real) forwarding hardware, but
fast CPU and lots of RAM"

gert

[Dovid Bender]

Not much at all 250 mbit.

Post by Gert Doering
Hi,

Post by Dovid Bender
I was told that convergence time on the MX5 would be horrible so we never
tried full routes. I am wondering what's the "lowest" model that can
support full routes without having an issue re-sorting the routes.

How much throughput do you need?
MX150 might be an alternative... "no (real) forwarding hardware, but
fast CPU and lots of RAM"
gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress
Gert Doering - Munich, Germany

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Mark Tinka]

Post by Gert Doering
How much throughput do you need?
MX150 might be an alternative... "no (real) forwarding hardware, but
fast CPU and lots of RAM"

In my mind, the MX150 is really the ideal scalable RR from Juniper for
folk that don't want to mess around building it themselves on their own
x86 servers.

Mark.

[Mark Tinka]

Post by Gert Doering
How much throughput do you need?
MX150 might be an alternative... "no (real) forwarding hardware, but
fast CPU and lots of RAM"

In my mind, the MX150 is really the ideal scalable RR from Juniper for
folk that don't want to mess around building it themselves on their own
x86 servers.

Mark.

[Josh Richesin]

What are people using now days for core routers with about 100 BGP sessions with about 10 of them being full routes? We have an MX104 that is actually doing it, however people are having issues and that is scaring me a bit. We are looking at upgrading it to 10G, but the cost is crazy as well.

Josh

Post by Gert Doering
How much throughput do you need?
MX150 might be an alternative... "no (real) forwarding hardware, but
fast CPU and lots of RAM"

In my mind, the MX150 is really the ideal scalable RR from Juniper for
folk that don't want to mess around building it themselves on their own
x86 servers.

Mark.


Confidentiality and Sensitive Information Notice: This message and all attachments are subject to the Electronic Communications Privacy Act, Title 18 U.S.C. and the sole property of Sureline Broadband. This email may contain information that is privileged, sensitive, confidential, or otherwise exempt from disclosure under applicable law, and may not be copied, printed, or forwarded without the express and sole permission of Sureline Broadband. If you are not the addressee or you have received this email in error, please delete this email, and contact us immediately at ***@surelinebroadband.com.
_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Mark Tinka]

Post by Josh Richesin
What are people using now days for core routers with about 100 BGP sessions with about 10 of them being full routes? We have an MX104 that is actually doing it, however people are having issues and that is scaring me a bit. We are looking at upgrading it to 10G, but the cost is crazy as well.

So our core is BGPv4-free, as traffic is all MPLS forwarded. We do,
however, hold the IPv6 BGP table in the core.

At any rate, that many number of full table sessions on the MX104 is
scary. I'd seriously consider another option sooner rather than later.

If your forwarding requirements are simple, you're likely to get away
with whatever fast, cheap Broadcom-based router Cisco, Arista or Juniper
are pushing out today. But, test, test, test. Broadcom chips for service
routers is tricky; but could be more forgiving for a simple core function.

Mark.
_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Tom Beecher]

Can confirm convergence time on the MX80 with even a single full table
session is extremely painful, and essentially not functional in a
production environment.

Post by Dovid Bender
Hi All,
In my 9-5 I work for an ITSP where we have two MX5's with
- iBGP
- two up steams with two BGP sessions each (one per routes)
- one upstream with one bgp session
- one bgp session where we get minimal routes (maybe 15 total)
I was told that convergence time on the MX5 would be horrible so we never
tried full routes. I am wondering what's the "lowest" model that can
support full routes without having an issue re-sorting the routes.
TIA.
_______________________________________________
https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[joel jaeggli]

Post by Tom Beecher
Can confirm convergence time on the MX80 with even a single full table
session is extremely painful, and essentially not functional in a
production environment.

Post by Dovid Bender
Hi All,
In my 9-5 I work for an ITSP where we have two MX5's with
- iBGP
- two up steams with two BGP sessions each (one per routes)
- one upstream with one bgp session
- one bgp session where we get minimal routes (maybe 15 total)
I was told that convergence time on the MX5 would be horrible so we never
tried full routes. I am wondering what's the "lowest" model that can
support full routes without having an issue re-sorting the routes.

 the 32 bit freescale  control-plane cpu is bit under-powered.

virtual-mx / mx150  as software devices are more than capable of doing
it. So are devices like the mx204, MX REs newer than the RE-2000 (which
is a bit long in the tooth / more than 10 years old a this point) also 
mid-range and higher SRXes

Post by Tom Beecher

Post by Dovid Bender
TIA.
_______________________________________________
https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.