Network Management

 View Only
  • 1.  Flexfabric IRF to AOS-CX VSX Server connection issues?

    Posted Jan 21, 2026 05:26 AM

    Hi, 
    recently we attempted to exchange our old Flexfabric 5900 IRF stack with two new 8260 CX-Switches, rewrote the config, built the VSX stack, checked every interface twice to have the same config as the previous flexfabric, tested MCLAGs with both access switches and servers (since its the first time im installing a VSX instead of VSF or IRF) to make sure they work as they should, but upon exchanging the old Core with the new core, half of our servers ports (Servers are mainly ESXi, NIMBLE and EMC DataDomain, which were connected via Access Ports and not LAGs) were not working properly anymore and we cannot figure out why.

    this is the old flexfabric configuration for LAGs and the non-LAG Ports:

    interface Bridge-Aggregation1

    description to_Access_U1

    port link-type trunk

    port trunk permit vlan all

    link-aggregation mode dynamic

    interface Ten-GigabitEthernet1/0/3

     port link-mode bridge

     description RZ2-NIMBLE-1

     port access vlan 171

     flow-control receive enable

     stp edged-port


    and this is the port configuration on the new CX:

    interface lag 1 multi-chassis
        no shutdown
        description Access_U1
        no routing
        vlan trunk native 1
        vlan trunk allowed all
        lacp mode active
        lacp fallback

    interface 1/1/26
        no shutdown
        mtu 9000       (MTU on the flexfabric was configured via VLAN interface and not physical interface)
        flow-control rx override-negotiation
        description RZ2-NIMBLE-1
        no routing
        vlan access 171
        spanning-tree port-type admin-edge

    I had two calls with TAC about the config and they confirmed the config to be correct, the new CX core also reported all interfaces and MC-LAGs to be working without error, but, as said, servers had trouble reaching the network on some or all ports. in the example of the NIMBLE server, it was connected to both units with a total of 8 cables and 4 per unit for redundancy purposes with all of them as Edge/Access Ports.

    the only idea i have left what could cause those issues if it isnt on the network / Switch-config side, is there any change in Stack behaviour that can cause the 8 NIMBLE ports to not work as before anymore since theyre now connected to "two" logically seperate switches (except for MC-LAGs) and not to "one" logical switch anymore? did anyone else maybe run into a similar issue before? Do is the switch config fine or not (i can share more of that if needed for understanding) and is the issue with the configuration on the Server-Ports, or am I overlooking something else entirely?



    -------------------------------------------


  • 2.  RE: Flexfabric IRF to AOS-CX VSX Server connection issues?

    Posted Jan 21, 2026 11:41 AM

    Hi, it's not clear to me if a peer (say the HPE Nimble) is (or was) configured to use LAG (Links Aggregation Group) with LACP or not because this is expected IF on the VSX side you setup a corresponding VSX LAG in Active Mode (which is a Multi-Chassis LAG with LACP)...8 aggregated links (4 on VSX Primary and 4 on VSX Secondary) aren't an issue (di per sè) if on the peer side you are working with LAG (Bonding) with LACP and on the VSX side you're working with VSX LAG (which is normally operating in active mode = LACP mode).

    This is an example of VSX LAG:

    interface lag 1 multi-chassis
        description Peer-Server-A-Links-to-VSX-Primary
        no shutdown
        no routing
        vlan trunk native n1 <----------------- this is the "untagged" VLAN IDs (corresponding to the PVID Port VLAN ID)
        vlan trunk allowed n1,n2,n3,n4,etc <--- these are all the allowed VLAN IDs (PVID + all tagged ones needed), OK to se "all"
        lacp mode active

    With these member interfaces (on VSX Primary to receive links X,Y, corresponding member interfaces are on VSX Secondary to receive links V,W):

    interface 1/1/1
        description Server-A-ethX
        no shutdown
        mtu 9198
        flow-control rx
        lag 1


    interface 1/1/2
        description Server-A-ethY
        no shutdown
        mtu 9198
        flow-control rx
        lag 1

    -------------------------------------------



  • 3.  RE: Flexfabric IRF to AOS-CX VSX Server connection issues?

    Posted Jan 22, 2026 02:11 AM

    Hi, i mightve expressed myself wrong, basically all servers (including Nimble) were connected via 8 (4 per Unit) seperate, vlan access, STP admin-edge ports.
    my current thought is, that the FLexfabric IRF makes it look to the server like all 8 ports go to the same switch while the VSX looks like two different switches now with 4 ports each and thats the part causing issues.

    -------------------------------------------



  • 4.  RE: Flexfabric IRF to AOS-CX VSX Server connection issues?

    Posted Jan 22, 2026 03:53 AM

    Hello Daniel, your thoughts are correct: the IRF (which can be seen like a stack, more or less like the VSF) creates a single logical unit and there isn't the concept of Multi-Chassis (so, from the point of view of the IRF a LAG to a connected peer is still a "standard" LAG...AND...from the point of view of the peer the whole IRF - and each one of its members - looks like a single big switch = a single logical identity...so no matter where the peer's originating links land, they land on the same entity). VSX instead is a Cluster of (at maximum) two Switches and they appear as a single logical entity to peers only in terms of services (think about IP Routing with the Active Gateway concept, as example) and in terms of connectivity (with VSX LAGs = Multi-Chassis LAGs). If you have a peer with simple uplinks (like they were "independent" access ports not tied together with a LAG concept, as you said) to the VSX then you need (and you can) to land them on VSX Primary and VSX Secondary members (necessarily) without using any LAG concept on the VSX side (neither a LAG on a VSX member nor a VSX LAG which is spanning over the entire VSX Cluster <- because the peer is not using a LAG at all, in any case). Typically your scenario looks similar to an ESXi host (with vSS Virtual Standard Switch implemented, not vDS Virtual Distributed Switch) where the physical Host is uplinked to the VSX via various standalone independent physical links (carrying various tagged VLANs), so some links go to VSX Primary (to edge ports operating in Trunk mode <- due to tagged traffic) and some others go to VSX Secondary (to corresponding edge ports, operating in Trunk mode too <- due to tagged traffic). No VSX LAG, No "standard" LAG (neither static non-protocol nor LACP driven).

    -------------------------------------------



  • 5.  RE: Flexfabric IRF to AOS-CX VSX Server connection issues?

    Posted Jan 22, 2026 04:00 AM

    Hi,
    thats what we tried on the VSX with this config: 

    interface 1/1/26
        no shutdown
        mtu 9000       (MTU on the flexfabric was configured via VLAN interface and not physical interface)
        flow-control rx override-negotiation
        description RZ2-NIMBLE-1
        no routing
        vlan access 171
        spanning-tree port-type admin-edge

    Unfortunately, this didnt work and still caused issues. i dont know enough about server configuration, may we need to change something on the servers themselves so they know theyre talking to two differentr switches now?

    -------------------------------------------



  • 6.  RE: Flexfabric IRF to AOS-CX VSX Server connection issues?

    Posted Jan 22, 2026 04:26 AM
    Edited by DP-7f7ca3 Jan 22, 2026 07:54 AM

    Hi, that's an example related to what I wrote about the ESXi scenario (interface 1/1/33 on VSX Primary, no LAGs involved):

    interface 1/1/33
        description Server-xxxxxx-esxi01-s1p2-vmnic1-vSwitch2
        no shutdown
        mtu 9198
        flow-control rx
        no routing
        vlan trunk native 2
        vlan trunk allowed 2,1000,1100
        spanning-tree bpdu-guard
        spanning-tree port-type admin-edge
        spanning-tree tcn-guard
        loop-protect
        loop-protect vlan 2,1000,1100

    For simmetry a corresponding 1/1/33 interface on the VSX Secondary has an identical configuration (description changes because both the Slot and Port of the originating link change on the same server). As you can see, it's nothing fancy. In the above example I was forced to use the VLAN Id 2 as untagged and VLAN Id 1000 and 1100 as tagged. If you have all tagged VLANs the corresponding lines will become (example):

    interface 1/1/33
        description Server-xxxxxx-esxi01-s1p2-vmnic1-vSwitch2
        no shutdown
        mtu 9198
        flow-control rx
        no routing
        vlan trunk native 2 tag (NOTE the "tag", a little bit counter-intuitive since native generally means untagged)
        vlan trunk allowed 2,1000,1100
        spanning-tree bpdu-guard
        spanning-tree port-type admin-edge
        spanning-tree tcn-guard
        loop-protect
        loop-protect vlan 2,1000,1100

    The fact that that 1/1/33 operates in "trunk mode" is due to carrying more VLANs (more than one which in your case is the VLAN Id 171 untagged = PVID).

    With regard to this: "Unfortunately, this didnt work and still caused issues. i dont know enough about server configuration, may we need to change something on the servers themselves so they know theyre talking to two differentr switches now?" consider that, if your server wasn't aware before (of being connected to an IRF) - so its uplink interfaces were made to land independently to any group of IRF members (no matter exactly where, all on IRF member 1 or all on IRF member 2 or whatever other combination to distribute the load) then why to expect your sever should be made aware now on the (well formed) VSX Cluster? simply you should land those links on the same way but you need to configure the edge ports on each VSX member to act accordingly (see example above).

    Hope this helps.

    Edit 1: the MTU is also set at VLAN interface context level (with the comand ip mtu 9198), I mean where the Active Gateway feature (in my case) was set.

    Edit 2: about your flow-control other thread, I read (H3C, as example...but feel free to see HPE documents too) this:

    To avoid dropping packets on a link, you can enable generic flow control at both ends of the link. When traffic congestion occurs at the receiving end, the receiving end sends a flow control (Pause) frame to ask the sending end to suspend sending packets. Generic flow control includes the following types:

    • TxRx-mode generic flow control - Rnabled by using the flow-control command. With TxRx-mode generic flow control enabled, an interface can both send and receive flow control frames:
      • When congestion occurs, the interface sends a flow control frame to its peer.
      • When the interface receives a flow control frame from its peer, it suspends sending packets to its peer.
    • Rx-mode generic flow control - Enabled by using the flow-control receive enable command. With Rx-mode generic flow control enabled, an interface can receive flow control frames, but it cannot send flow control frames:
      • When congestion occurs, the interface cannot send flow control frames to its peer.
      • When the interface receives a flow control frame from its peer, it suspends sending packets to its peer.

    To handle unidirectional traffic congestion on a link, configure the flow-control receive enable command at one end and the flow-control command at the other end.

    To enable both ends of a link to handle traffic congestion, configure the flow-control command at both ends.

    Probably would be nice to see how that is set on the peer Server (HPE Nimble) in order to use the correct flow-control setting on the Aruba switches (I will probably remove the override-negotiation option from the flow-control rx override-negotiation configuration line as per your current configuration on ArubaOS-CX).

    -------------------------------------------