[j-nsp] qfx5100 software upgrades and virtual-chassis

[Louis Kowolowski]

I currently have a 6 node VC of qfx5100. All are running 14.1X53-D43.7 and host software 13.2X51-D38. In discussions with JTAC, they claim that upgrading the host software to match the VM, it requires a reboot of *all* nodes in the VC at the same time.

Has anybody else had to deal with this? Are there any work-arounds? Taking the whole thing down is extremely awkward. Can we do a rolling upgrade (manually, I know ISSU/NSSU doesn't handle this) and stay operational? We are working on a plan to re-architect this into 2x 3 node VC and MC-LAG them together, but it would be nice to be able to fix this more short-term.

--
Louis Kowolowski ***@cryptomonkeys.org
Cryptomonkeys: http://www.cryptomonkeys.com/

Making life more interesting for people since 1977

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Chuck Anderson]

Logically, why couldn't you isolate one member at a time, do the upgrade, then rejoin it to the VC?

Post by Louis Kowolowski
I currently have a 6 node VC of qfx5100. All are running 14.1X53-D43.7 and host software 13.2X51-D38. In discussions with JTAC, they claim that upgrading the host software to match the VM, it requires a reboot of *all* nodes in the VC at the same time.
Has anybody else had to deal with this? Are there any work-arounds? Taking the whole thing down is extremely awkward. Can we do a rolling upgrade (manually, I know ISSU/NSSU doesn't handle this) and stay operational? We are working on a plan to re-architect this into 2x 3 node VC and MC-LAG them together, but it would be nice to be able to fix this more short-term.

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Louis Kowolowski]

It seems reasonable, and no worse than the situation now, with mis-matched versions. I don't see why the host software would have anything to do with VC, seems like it should just be handling VM operations, and doing hardware-passthrough...

Post by Chuck Anderson
Logically, why couldn't you isolate one member at a time, do the upgrade, then rejoin it to the VC?

Post by Louis Kowolowski
I currently have a 6 node VC of qfx5100. All are running 14.1X53-D43.7 and host software 13.2X51-D38. In discussions with JTAC, they claim that upgrading the host software to match the VM, it requires a reboot of *all* nodes in the VC at the same time.
Has anybody else had to deal with this? Are there any work-arounds? Taking the whole thing down is extremely awkward. Can we do a rolling upgrade (manually, I know ISSU/NSSU doesn't handle this) and stay operational? We are working on a plan to re-architect this into 2x 3 node VC and MC-LAG them together, but it would be nice to be able to fix this more short-term.

--
Louis Kowolowski ***@cryptomonkeys.org
Cryptomonkeys: http://www.cryptomonkeys.com/

Making life more interesting for people since 1977

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Nitzan Tzelniker]

If you cant afford taking down the whole VC do not work with VC
This is my philosophy with VC
Do MC-LAG or EVPN in these environment (even with VC just to increase the
number of ports )

Regarding the host they dont have to be the same unless there is a known
issue

From
https://www.juniper.net/documentation/en_US/junos/topics/task/installation/qfx-series-software-upgrading.html


However, pay attention to these notes regarding Junos OS and Host OS
versions:

-

The Junos OS and Host OS versions do not need to be the same.
-

During an ISSU, the Host OS cannot be upgraded.
-

Upgrading the Host OS is not required for every software upgrade, as
noted above.


Nitzan

Post by Louis Kowolowski
It seems reasonable, and no worse than the situation now, with mis-matched
versions. I don't see why the host software would have anything to do with
VC, seems like it should just be handling VM operations, and doing
hardware-passthrough...

Post by Chuck Anderson
Logically, why couldn't you isolate one member at a time, do the

upgrade, then rejoin it to the VC?

Post by Chuck Anderson

Post by Louis Kowolowski
I currently have a 6 node VC of qfx5100. All are running 14.1X53-D43.7

and host software 13.2X51-D38. In discussions with JTAC, they claim that
upgrading the host software to match the VM, it requires a reboot of *all*
nodes in the VC at the same time.

Post by Chuck Anderson

Post by Louis Kowolowski
Has anybody else had to deal with this? Are there any work-arounds?

Taking the whole thing down is extremely awkward. Can we do a rolling
upgrade (manually, I know ISSU/NSSU doesn't handle this) and stay
operational? We are working on a plan to re-architect this into 2x 3 node
VC and MC-LAG them together, but it would be nice to be able to fix this
more short-term.
--
http://www.cryptomonkeys.com/
Making life more interesting for people since 1977
_______________________________________________
https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Louis Kowolowski]

Post by Nitzan Tzelniker
If you cant afford taking down the whole VC do not work with VC
This is my philosophy with VC
Do MC-LAG or EVPN in these environment (even with VC just to increase the number of ports )

This is now my philosophy as well. Just working on cleaning up the mess w/as little impact as I can.

Post by Nitzan Tzelniker
Regarding the host they dont have to be the same unless there is a known issue
From https://www.juniper.net/documentation/en_US/junos/topics/task/installation/qfx-series-software-upgrading.html
• The Junos OS and Host OS versions do not need to be the same.
• During an ISSU, the Host OS cannot be upgraded.
• Upgrading the Host OS is not required for every software upgrade, as noted above.

Yes, and clarification from JTAC is that they need to be the same major version, which doesn't sound unreasonable.

--
Louis Kowolowski ***@cryptomonkeys.org
Cryptomonkeys: http://www.cryptomonkeys.com/

Making life more interesting for people since 1977

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.n

[Richard McGovern]

Louis, for reason (I assume some issue) is TAC saying that a host-upgrade solves something? I also assume from Junos SW point of view, you plan to stay with 14.1X53-D43, yes?

BTW, in the future solution to this is to perform Host Upgrade at same time as Junos Upgrade, using the force-host option - https://www.juniper.net/documentation/en_US/junos/topics/task/installation/qfx-series-software-upgrading-cli.html

Best of luck, Rich

Post by Nitzan Tzelniker
If you cant afford taking down the whole VC do not work with VC
This is my philosophy with VC
Do MC-LAG or EVPN in these environment (even with VC just to increase the number of ports )

This is now my philosophy as well. Just working on cleaning up the mess w/as little impact as I can.

Post by Nitzan Tzelniker
Regarding the host they dont have to be the same unless there is a known issue
From https://www.juniper.net/documentation/en_US/junos/topics/task/installation/qfx-series-software-upgrading.html
• The Junos OS and Host OS versions do not need to be the same.
• During an ISSU, the Host OS cannot be upgraded.
• Upgrading the Host OS is not required for every software upgrade, as noted above.

Yes, and clarification from JTAC is that they need to be the same major version, which doesn't sound unreasonable.

--
Louis Kowolowski ***@cryptomonkeys.org
Cryptomonkeys: https://urldefense.proofpoint.com/v2/url?u=http-3A__www.cryptomonkeys.com_&d=DwIFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=cViNvWbwxCvdnmDGDIbWYLiUsu8nisqLYXmd-x445bc&m=SwXGavYoXMX7_3ONE2EFDbM6qqRplmC0qoQHLrDr_HI&s=TRlgGH-vOckvzTgRz3rVHAdHuIz4wUvhYEmam8MKFas&e=

Making life more interesting for people since 1977




_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https:

[Louis Kowolowski]

Post by Richard McGovern
Louis, for reason (I assume some issue) is TAC saying that a host-upgrade solves something? I also assume from Junos SW point of view, you plan to stay with 14.1X53-D43, yes?

Yes (Supposedly). And Yes. I'm not convinced, but they won't troubleshoot the case further until I do.
SNMP counters and descriptions aren't being updated for AE interfaces >9. I troubleshot this to the actual switch. For example in bytes is 48, it doesn't change. If I look at the counters for the physical interface, they update as expected. Also, the >9 is wierd. AE0-9 work as expected. AE10-? don't update properly.

This is only for SNMP. If you look at the interface counters under 'show interface', they update as expected. The correct description is also shown. I tried restarting SNMP and it didn't make a difference.

Also worth noting is that this oddity/problem doesn't appear to impact the flow of traffic through the box, nor the ability of the AE interface to function in any way (such as LACP failover).

Post by Richard McGovern
BTW, in the future solution to this is to perform Host Upgrade at same time as Junos Upgrade, using the force-host option - https://www.juniper.net/documentation/en_US/junos/topics/task/installation/qfx-series-software-upgrading-cli.html

Yes, I've been made aware of that now. :-)

Thanks.

--
Louis Kowolowski ***@cryptomonkeys.org
Cryptomonkeys: http://www.cryptomonkeys.com/

Making life more interesting for people since 1977

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp