NPV and NPIV
May 8, 2012 70 Comments
The CCIE Data Center blueprint makes mention of NPV and NPIV, and Cisco UCS also makes heavy use of both topics, topics that many may be unfamiliar with most. This post (part of my CCIE Data Center prep series) will explain what they do, and how they’re different.
(Another great post on NPV/NPIV is from Scott Lowe, and can be found here. This is a slightly different approach to the same information.)
NPIV and NPV are among the two most ill-named of acronyms I’ve come across in IT, especially since they sound very similar, yet do two fairly different things. NPIV is an industry-wide term and is short for N_Port ID Virtualization, and NPV is a Cisco-specific term, and is short for N_Port Virtualization. Huh? Yeah, not only do they sound similar, but the names give very little indication as to what they do.
NPIV
First, let’s talk about NPIV. To understand NPIV, we need to look at what happens in a traditional Fibre Channel environment.
When a host plugs into a Fibre Channel switch, the host end is called an N_Port (Node Port), and the FC switch’s interface is called an F_Port (Fabric Port). The host has what’s known as a WWPN (World Wide Port Name, or pWWN), which is a 64-bit globally unique label very similar to a MAC address.
However, when a Fibre Channel host sends a Fibre Channel frame, that WWPN is no where in the header. Instead, the host does a Fabric Login, and obtains an FCID (somewhat analagous to an IP addres). The FCID is a 24-bit number, and when FC frames are sent in Fibre Channel, the FCID is what goes into the source and destination fields of the header.
Note that the first byte (08) of the FCID is the same domain ID as the FC switch that serviced the host’s FLOGI.
In regular Fibre Channel operations, only one FCID is given per physical port. That’s it. It’s a 1 to 1 relationship.
But what if you have an ESXi host, for example, with virtual fibre channel interfaces. For those virtual fibre channel interfaces to complete a fabric login (FLOGI), they’ll need their own FCIDs. Or, what if you don’t want to have a Fibre Channel switch (such as an edge or blade FC switch) go full Fibre Channel switch?
NPIV lets a FC switch give out multiple FCIDs on a single port. Simple as that.
The magic of NPIV: One F_Port gives out multiple FCIDs on (0x070100 to the ESXi host and 0x070200 and 0x070300 to the virtual machines)
NPV: Engage Cloak!
NPV is typically used on edge devices, such as a ToR Fibre Channel switch or a FC switch installed in a blade chassis/infrastruture. What does it do? I’m gonna lay some Star Trek on you.
NPV is a clocking device for a Fibre Chanel switch.
Wait, did you just compare Fibre Channel to a Sci-Fi technology?
How is NPV like a cloaking device? Essentially, an NPV enabled FC switch is hidden from the Fibre Channel fabric.
When a Fibre Channel switch joins a fabric, it’s assigned a Domain_ID, and participates in a number of fabric services. With this comes a bit of baggage, however. See, Fibre Channel isn’t just like Ethernet. A more accurate analogue to Fibre Channel would be Ethernet plus TCP/IP, plus DHCP, distributed 802.1X, etc. Lots of stuff is going on.
And partly because of that, switches from various vendors tend not to get a long, at least without enabling some sort of Interopability Mode. Without interopability mode, you can’t plug a Cisco MDS FC switch into say, a Brocade FC switch. And if you do use interopability mode and two different vendors in the same fabric, there are usually limitations imposed on both switches. Because of that, not many people build multi-vendor fabrics.
Easy enough. But what if you have a Cisco UCS deployment, or some other blade system, and your Fibre Channel switches from Brocade? As much as your Cisco rep would love to sell you a brand new MDS-based fabric, there’s a much easier way.
Engage the cloaking device.
(Note: NPV is a Cisco-specific term, while other vendors have NPV functionality but call it something else, like Brocade’s Access Gateway.) A switch in NPV mode is invisible to the Fibre Channel fabric. It doesn’t participate in fabric services, doesn’t get a domain ID, doesn’t do fabric logins or assign FCIDs. For all intents and purposes, it’s invisible to the fabric, i.e. cloaked. This simplifies deployments, saves on domain IDs, and lets you plug switches from one vendor into a switch of another vendor. Plug a Cisco UCS Fabric Interconnect into a Brocade FC switch? No problem. NPV. Got a Qlogic blade FC switch plugging into a Cisco MDS? No problem, run NPV on the Qlogic blade FC switch (and NPIV on the MDS).
The hosts perform fabric logins just like they normally would, but the NPV/cloaked switch passes FLOGIs up to the NPIV enabled port on the upstream switch. The FCID of the devices bears the the Domain ID of the upstream switch (and appears directly attached to the upstream switch).
The NPV enabled switch just proxies the FLOGIs up to the upstream switch. No fuss, no muss. Different vendors can interoperate, we save domain IDs, and it’s typically easier to administer.
TL;DR: NPIV allows multiple FLOGIs (and multiple FCIDs issued) from a single port. NPV hides the FC switch from the fabric.
The Data Center Overlords 
Pingback: NPV and NPIV | Storage CH Blog
Put in terms @networkingnerd likes, NPV is NAT for FC 😉
I’ve heard NPV/NAT comparison before, and as much as I like to torture Tom with tales of NAT, I’m not sure I like the comparison.
NAT is an active agent, changing source and/or destination addresses. With NPV, no addresses are changed. It’s more of a passive technique by forwarding requests on behalf of the N_Ports.
Besides, cloaking device sounds cooler than NAT 🙂
Nice blog post Tony. but analogy of NAT confuses with IVR NAT which really NATs FCID when frame sent from one fabric (VSAN) to another.
Great post. A couple of points.
1) In a “Layer 2” FC SAN, a WWPN is functionally equivalent to IP Address (not a MAC Address); and the 24 bit Address (FCID) is functionally equivalent to the MAC Address (not the IP Address). The 24 bit address is used by the FC switch to decide how to forward the frame (this is the same as a MAC in L2 Ethernet). In terms of routing frames, the WWPN only comes into play when a Layer 3 FC Router is used.
2) Your cloaking device analogy seems right on to me, and this can be good and bad. You clearly articulate the good points above. The bad has to do with having an invisible device in your FC SAN and not being able to understand exactly how hosts and storage ports are attached to it. I’ve been told by a customer that this can be problematic when trying to understand how much bandwidth is being consumed by each device, etc
Regards, Erik
Nitpicking, but FC forwarding is actually routing (L3 switching), not bridging (L2 switching). There is no equivalent to MAC address in FC, and that makes perfect sense because the links are P2P and thus don’t need L2 addresses. WWPN is functionally equivalent to host name (and only used when registering, like a hostname should be … think about client ID in DHCP requests in IP).
http://blog.ioshints.info/2011/07/is-fibre-channel-switching-bridging-or.html
BTW, you don’t have to trust me on this one – FC-SW-5 clearly talks about “FC router”, and it’s not describing inter-fabric routing.
Hi Ivan, nitpicking is fine, but I respectfully disagree. I’ve read your blog entry and I fail to see how you came to the conclusion that FC is somehow actually routing and not switching? A counter argument is that a router alters the DA and SA, an FC switch does not do this. There are other resources out there (take a look at google images and an FC/FCoE layering diagram). These images are not what I would call a technical resource but do help to visualize what I’m talking about.
FCoE is a completely different story and where I would agree that FC is a layer 3 protocol to Ethernet. This is because with FCoE, the FPMA and/or FCF-MAC change on an FCF by FCF (hop by hop) basis.
Regards, Erik
If we agree that FC is a layer-3 protocol, them I’m perfectly fine with that 😉 The rest is terminology confusion caused by overzealous marketing departments 20 years ago.
BTW, the counter-counter argument would be that a router does not alter IP addresses (like the FC switch does not alter FCIDs).
That analogy works too of course, but I like this way:
WWPNs are similar to MAC addresses because they’re globally unique, burnt into the HBA (usually), and have an OUI. All three are traits of a MAC address. (WWNNs don’t have a functional analogue in Ethernet).
FCIDs are like IP addresses, because they are the source and destination in FC frames, and each FC switch is more like a router, with a routing protocol (FSPF) figuring out where to send the FC frames. They’re also assigned via a mechanism not too different than DHCP, as DHCP often gives IPs after considering the MAC address.
In a NAT’d IP environment, addresses are not globally unique, just like an FC fabric the FCIDs are not going to be globally unique. Domain IDs are somewhat like a network prefix.
Zoning gets a little loser with the analogies, perhaps similar to private VLANs or 802.1X. Neither of those are commonly used in networking, so perhaps I’ll just call it zoning 🙂
Sometimes though, you have to cross boundaries, and that requires NAT. So IVR is more like NAT in that regard. More specifically, enterprise-to-enterprise NAT, like when two companies merge and they both use the same 10.x.x.x address space. It seems like IVR (though I’ve never configured it in production) can be just as much of a pain in the ass, too.
Of course in all of these analogies, there are ways that the analogue is fundamentally different. But this way of thinking helped me wrap my brain around the FC world, especially coming from the Ethernet/IP world.
Hi Tony, FC switching is not routing at L3. If it was, the L2 Addresses would change at every switch (what you and Ivan are referring to as a router) and these 24 bit addresses do not change unless you hit an IVR-NAT or FCR instance, at which point the associations between end devices have to be done by associating WWPNs to other WWPNs…
The three criteria associated with L2 MAC Addresses that you defined above (globally unique, burnt into the HBA (usually), and have an OUI), Would run into a problem when you consider that an FPMA, used in FCoE, shares none of these traits, yet it is a MAC Address and is considered to be a Layer 2 address.
Or put another way “THERE.. ARE.. FOUR.. LIGHTS!!”
Seriously, I think we (You, Ivan and I) are like three people looking at a prism from different angles and are arguing about the colors being seen.. At the end of the day it depends on your perspective.
Erik
Bonus points for the Picard reference 🙂
Reblogged this on CCIE #31104, what's next? and commented:
CCIE Data Center topics high level explination: NPV and NPIV.
I consider npv more of a proxy than a nat though whenever I discuss I always point out the ambiguity in any of these terms. Similarly I tend to think of the so called sanrouters as more like mat gateways.
As for l2/l3 im afraid im in the l3 camp. Fspf was based on ospf, as noted the wwn is burned in to the hba and the fcid is allocated by the login service. Fc is basically a network or directly attached layer 3 routers. Well it was before fcoe as now you can have a layer 2 domain in between.. Indeed to a network guy the fabric login service has similarities to dhcp and the name server to dynamic dns. .
Ok, actually let’s all be honest… None of these protocols was based on the official OSI model (which indeed was invented after networks) and so there will always be a breakdown in terminology in this discussion where everyone can claim to be right and everyone can claim the next guys is wrong. Most of the protocols in OSI terms are layer 1.x or 2.y or 3.z as they all bend the definitions…..
The fc guys may (and I say may) be right that within the strict fc definitions of fc layers that fc considers its forwarding to be layer 2. I suspect some of the similarities, differences, and confusion, comes for the fact that for both Ethernet and fc the current “switched” model was not the original model. A house of confusing cards was then built as NPV and IFR and e rest got put I there.
I’m a fan of let’s forget our differences and histories, recognize all the terms are overloaded and confused, and try to start with a clean sheet of paper. Clearly with fcoe you can now have l2 switching in between the devices that forward based othe fcid so today surely it makes more sense to think of fc forwarding as l3 ?
Of course that’s not the only area the two worlds struggle to agree ! At which point I need a drink 😉
By the way, I like in many senses the blog. However an N_Port Virtulizer is anything BUT invisible. By the way to nit pick NPV (unlike NPIV where the standard defined the acronym) as an acronym was never defined – indeed the standard has only a strange reference in one table to N_Port Virtulizers as a new device type. You have to be a mind reader to work out from the standards what was meant or read a bunch of peripheral ppt.
Anyway, these devices are very visible. They log in to the fabric, they declare their wwn’s, they are allocated one or more fcid, just like a host. What is invisible to the San fabric is they are only pretending to be hosts and in fact that they are acting on behalf of a bunch of real hosts. The term acting on behalf of is why I use the term “proxy” myself, sometimes ”
Personally I point out to people that these devices use the NPIV protocol as part of their pretense when they map FLOGI to FDISC.
I also point out that such a device can be fc to fc, fcoe to fc, fcoe to fcoe, or even fc to fcoe. The fact that they convert between types is why I now prefer to say proxy-gateway though I can also see reasons gateway is a bad term to use.
Another downside to these devices is that unlike an fc switch, or in fcoe a fip snooping bridge, the load balancing may well end up being less good (let’s say session based) as there is no standards based way to do exchange based routing. This like the easier point on monitoring, can make them a nuisance. In the fc to fc case it’s a necessary nuisance as embedded switches in blade servers broke San scale and caused a messy overlap. In the case of fcoe a fip snooping bridge is even more invisible when you want it to be, but if well but if well built can give/maintain all the monitoring you need.
Hi!
Great post.
Brocade’s name for this feature is ‘Access Gateway’ – Is Qlogic’s the same as Cisco’s (NPV)?
I don’t know if it’s still the case, but QLogic’s was called “Transparent Routing” at one point. That was about 2 years ago, and I don’t know if they’ve changed it since.
You know, as I’m bored on pto I decided to go back to the books and check… To me despite the many inconsistencies one can point out because neither fc nor ip actually follow or are based on the osi model I see no doubt but to say fc forwards at layer 3. One could pull out the exact definitions from osi, layer 3 is where your path is determined and forwarding is based on logical not physical addresses – that seems to fit both ip and fc nicely. Also being old enough to remember loop I can take the analogy of the low byte of the fcid being the address within a loop (within a subnet) and the high two bytes showing me where in the san overall I am (which port of which switch – ie on which subnet).
A little more fun – fc is British (fibRE) and so counts from 0 (ground floor is 0) and osi seems more American and so counts from 1 (ground floor is 1) :-). S an American floor 3 is a British floor 2 🙂
Osi. Fibre channel Ethernet/ip
Layer 1 Physical Fc-0 physical Ethernet
Layer 2 Data link Fc-1 data link Ethernet
Layer 3 Network Fc-2 Network Ip
Layer 4 Transport Fc-3 common services Tcp/udp
Layer 5 session Fc-4 Protocol mapping
Layer 6 presentation
Layer 7 application. Dhcp, dns
http://en.wikipedia.org/wiki/OSI_model
Simon, the lowest (outermost) level of addressing with FC is the FCID. This is the Address that is used to deliver frames from one N_Port to another. This functionality is provided by the MAC Address in the traditional networking space and is associated with Layer 2. When we have talked about Layer 3 in the SAN space, it has always been associated with a Fiber Channel Router (FCR / IVR-NAT). When you need to route between two N_Ports via a FCR/IVR-NAT, you use the WWPN to create the association and therefore it makes sense to associate it with the L3 Address in that context.
Feel free to point out the flaw(s) in the above logic.
Other than the fact that I incorrectly spelled Fibre 🙂
Erik, there are other networking protocols besides Ethernet, some of the better designed (or less broken). L2 protocol was always supposed to be used between ADJACENT devices and since initial Ethernet used thick coax cables, MAC addresses were used so that the devices connected to the same cable could talk to each other.
L2 addresses make no sense on P2P links and protocols that were designed for P2P links (HDLC, PPP) don’t use them. It’s my understanding FC uses only P2P links today.
Tangential note: actually serial link protocols do have L2 addresses in frame headers because almost every serial link protocol in use today uses the frame format from IBM’s SDLC which was a multi-access (multi-drop) protocol and thus needed physical addresses. However, those L2 addresses usually have fixed values (and are not used).
The fact that FCID is the first address in the packet header is thus irrelevant. Likewise IP was the first address in the SLIP header 😉
Finally, the reason everyone gets confused by the layering is the brokenness of Ethernet. A few idiots within DEC designed L2-only protocols (LAT and MOP), got stuck with the maximum-devices-per-cable limitation and pushed through the idea of a cable extender (transparent bridge). The rest is history … For more details, read this blog post:
http://blog.ioshints.info/2010/07/bridges-kludge-that-shouldnt-exist.html
… and if you don’t trust me, watch the video of Radia Perlman’s GoogleTalk (link in my blog post). I guess you can’t get more authoritative information than that.
Hi Ivan, a link between an N_Port (e.g., HBA) and an F_Port (e.g., switch port) is considered to be point-to-point. But a Fabric consists of multiple point-to-point connections that all reside in the same L2 space. When a switch receives a frame, typically it will validate that the specified source FCID (S_ID) is allowed to access the Destination FCID (D_ID) of the frame. This is actually how FC zoning is enforced today (sort of like a L2 ACL). My point is, not only do these L2 addresses make sense on an FC Point-to-Point link, they actually enable functionality that is critical to the proper operation of FC.
I’ll have to check out the video..
Erik
Erik, throughout our discussion you keep repeating “FCID = L2 address”. Until you step back and say “let’s examine whether it makes more sense to consider FCID to be L2 or L3 address” it doesn’t make sense to continue.
Ivan, it’s true, I keep repeating my original assertion that an FCID is an L2 address and providing additional supporting evidence for my position. FWIW, at several points in the thread I have discussed the implications of treating the FCID as a L3 Address and given reasons for why I think this doesn’t make sense.
Agreeing to disagree is acceptable to me..
As my final point on the matter, the terminology here is ambiguous and broken, the definitions of layer 2 and layer 3 are different in different protocols, few of the protocols fit exactly the osi model, and none of the protocols we are using here as comparison points either fit the osi model or differ in the same way. As such no matter that we are all learned experts we will always find places to differ in our views based on our own histories perspectives and biases.
Ethernet is both layer 1 and layer 2 depending on what one is referring too with switching being an l2 function, and ip is both layer 2 and layer 3 and indeed deliberately breaks the boundaries with routing being an l3 function. It is also important to note that ethernet and ip have no direct relationship with each other in most senses as they evolved in parallel which adds further confusion. I assume however we all agree that an Ethernet L2 broadcast domain (physical or virtual aka vlan) maps to an ip l2 subnet or ip l2 broadcast domain. In that context, fibre channel is both layer 2 and layer 3 just as is ip – ie it is a hierarchical addressing system but that still makes the fc San switch the equivalent of an ip router.
Now I completely agree that as point to point and loops went away, and San routing came along (and of course NO ONE implements IFR as defined in the standard with so called routers actually being nat devices) it became convenient in terms of general network design to think of a single fc San as a single l2 domain. However that was from a mental model perspective and not how the protocol actually works.
By the way, a small detail most people forget is that before fc San switching and zoning were developed, back in the late 90’s, brocade used to describe the zone as the equivalent of the vlan in Ethernet. I’m not using that to justify my position other than my position that in fc’s history mutually contradictory statements / points of reference / comparisons have been used. Erik could indeed use this against me to argue that that would play to his position of fc San switching being l2 in nature.
It is unfortunate that the development of FCoE adds to the confusion, because it both at one level puts ethernet as a layer below fc (implies fc switching is layer 3 in the same sense that we consider ip routing to be layer 3, with an ip subnet mapping to an l2 Ethernet broadcast domain) and maps vlan to a virtual San fabric (implying fc switching is layer 2). However any in depth analysis of the ways in which vendors actually ask people to use/deploy fcoe IMHO still leans towards my bias of fc switching being l3 in nature.
Like Ivan, I don’t think the order of the bytes in the frame have any particular significance particulalrly as fc was developed as a monolithic stack in some isolation from other things. The fact that people say layer 2 = frames layer 3 = packets is unhelpful to my arguement given fc uses the term frame (another point to erik). I still contend going back to my osi reference model that the wwn being a globally unique address burned in to the hba at manufacture and the fcid being a logical address allocated during initialization is more pertinent and more clearly highlighted in the definition of the layers.
However, ultimately it’s all academic as what matters is whether it works and how do people deploy it. Like other areas we could get in to religious wars about things that actually don’t matter to most people and as brook notes loose sight of the fact that any useful analogy when explaining a concept is good just so long as we remember it’s just a useful analogy. For myself, I always try hard to make it clear that I am making analogies to help people build a mental model and to point out to them the things that they need to concentrate on.
… and to add my final point on this matter – the only reason I care is because people moving from FC world to IP+Ethernet world (or vice versa) get the wrong expectations if they equate the way FC switching works with the way Ethernet transparent bridging works. Regardless of the layering opinions, FC-SW has more in common with IP than with Ethernet.
Ivan, in regards to the statement that “FC-SW has more in common with IP than with Ethernet.”, let’s evaluate your assertion by examining the following functions:
1. Login
2. Address assignment
3. Discovery
4. Frame Delivery
5. L3 Routing
1&2) FC requires a login to a fabric at which point an FCID is assigned to it. Login does not occur with either Ethernet or IP and while the assignment of a MAC Address or IP Address is not a requirement for either, there are cases where it’s done for both. As a result, the fact that an FCID is assigned to an N_Port cannot be used as an indication that the FCID is a L3 address.
3&4) Frames can be delivered from one node to another based on the source and destination FCID alone. In a layer 2 Ethernet network, this is usually accomplished via MAC Addresses once the DA of the end station is known. In most networks today, the discovery of the DA is accomplished via ARP an L3 protocol and ARP relies on the layer 3 address to accomplish this lookup. With FC, the D_IDs available for communication are typically discovered via the Name Server and the D_IDs that are returned in response to a Name Server query are typically limited by the WWPN to WWPN associations that have been defined by the SAN Administrator. Note, I’ve been arguing that the WWPN is a Layer 3 address.
4&5) FC fabrics use hierarchical Addresses and FSPF to build forwarding tables. While I agree that FSPF is based on OSPF and OSPF is widely used with IP, I do not agree that therefore FC is a L3 protocol. If this logic was sound, the same argument could be made about TRILL since it uses IS-IS.
Regards, Erik
* Fabric login = DHCP (which also uses client ID)
* Like FCIDs, IP addresses are not changed when packets traverse the network
* ARP is only needed on multi-access networks, not on P2P links. I can use the same argument as you do and describe how FC header matches perfectly IP-over-PPP header.
* DNS is used to discover IP addresses and dynDNS can be used to register dynamic (DHCP-assigned) IP addresses in DNS.
Finally, TRILL is tunneling of L2 frames across L3 backbone (TRILL), a fact that is glossed over by every vendor’s marketing department. Look at the TRILL headers – it’s very obvious where outer L2 header is, where TRILL L3 header is and where the payload L2 frame sits.
The whole argument we have comes from the gross misuse of term “switching” in the networking industry. Assume for the moment that you’ve never heard of “switching” (which means as much as cloud does) and that you know two forwarding paradigms: routing and transparent bridging. I did a thorough comparison between them a while ago:
http://blog.ioshints.info/2010/07/bridging-and-routing-is-there.html
http://blog.ioshints.info/2010/07/bridging-and-routing-part-ii.html
Please go through them and tell me which one of these two is closer to what you think FC is doing – and please don’t think about IP, MAC or SWITCHING while doing so.
Ivan, thanks for the links to your blog, I hadn’t had the chance to read them until just now. In regards to your invitation to “Please go through them and tell me which one of these two is closer to what you think FC is doing – and please don’t think about IP, MAC or SWITCHING while doing so.”
Since you are asking me to compare FC to IP and Ethernet, I’m not sure exactly what you mean by the last part of your request, but I’ll see what I can do. See below:
You state: “They (routers) drop datagrams sent to unknown destinations and tell the sending hosts they did so.
My response: Today, SCSI-FCP is performed over Class 3 FC (a.k.a. “Ship and Pray”) and there is no notification when frames are dropped. As a result FC is more like Ethernet in this regard.
You state: “In short, routing is “forwarding based on presumption of knowledge”, bridging is “forwarding by guessing”.”
My response: I disagree with this definition of bridging versus routing. When a DA is known to both the end station and the switch that receives the frames, does a bridge all of sudden become a router? Of course not, therefore I think this definition is invalid.
You state: “Loop detection. IP (and most other layer-3 protocols) has a hop count in its header. Ethernet header does not have a hop count (neither do most other layer-2 protocols).”
My response: FC does not does not have a hop count in its header and by your own words, this makes it more like a Layer-2 protocol.
You state: “Multicast. Routers stop multicast or broadcast packets unless they are configured to forward them.”
My response: FC was designed (in part) to carry IP and it supports broadcast. Whether or not individual products support multicast is another story.
You state: “Forwarding tables. IP routing tables are built by routers exchanging (somewhat) authoritative information: their connected subnets and their static routes.
My response: I agree that FC exchanges authoritative information (Link State Records) to be able to determine the optimal route from one device to another based on Domain ID. However, this doesn’t make FC a layer 3 protocol as your reference to TRILL (later in that same paragraph) clearly indicates.
You state: “Addressing. Layer-3 addresses are configurable and usually include some topology information, allowing the layer-3 routing to scale. Layer-2 addresses are supposed to be static (hardwired) and are (within a single network) randomly scattered around the network.”
My response: Hardwired? I can easily move a Layer-2 Address with vMotion (although I don’t believe it’s the default behavior) and I know I can move them with my KVM VMs. In addition, FPMAs used in FCoE are another example of how Layer-2 addresses can be assigned. With all of this in mind, I disagree with your definition of Layer-2 and Layer-3 addresses.
You state: “Spoofing. The “forwarding by learning” paradigm makes it extremely easy to spoof a bridged network: send frames with wrong source MAC address.
My response: With FC, due to the way that forwarding is done, spoofing is done using the WWPN.
You state: “Fragmentation. IP was designed to span a multitude of physical media with different characteristics and supports datagram fragmentation and path MTU discovery.
My response: FC does not support fragmentation and it much more likely to fail spectacularly (like Ethernet) if the frame size in use is not supported by an intermediary. We spent an inordinate amount of time discussing this topic in FC-BB-5.
You state: “Out-of-order packets. Out-of-order packets are a fact of life in any multipath topology (including any layer-3 network). Layer-3 protocols were thus designed to deal with them, either rearranging them (TCP) or dropping them (most UDP applications).
Protocols that pretend the hosts communicate on a shared cable tend to ignore the out-of-order problems; some protocols might even terminate the session when receiving one.“
My response: The main protocols that utilize FC (SCSI-FCP and FICON) do not handle out of order frames well. And FC in general seems more like Layer-2 in this regard.
PART 2:
You state: “Zero configuration. In their simplest incarnation, the bridges are plug-and-play devices”
My response: I wouldn’t recommend trying to run it production, but by default, many FC switches will operate without any user configuration. As a result, FC is more like Ethernet.
You state: “Equal-cost multipath. Routers can load-balance traffic between equal-cost paths across the network.”
My response: As both you and I pointed out earlier ECMP like functionality can be accomplished via TRILL and does not necessarily define the layer at which a protocol is operating.
In addition – I reviewed your other points and didn’t see anything that would act as a counter-point to the 11 points I’ve already made.
Regards, Erik
ok, a couple of points…
in order delivery – in a stable network both L2 ethernet switches and L3 ip routers – out of order in any significant level causes retransmission and congestion control and for that reason the way all L2 Ethernet Switches and L3 IP routers as well as TRILL Fabrics etc maintain i norder delivery for the things that they need to just as FC Switches do. In all cases the only time you get ooo is as a result of some change or disruption in the network and in all cases someone higher up the stack has to deal with the consequences. both IP and FC have some limited reorder capability.
TCP which is a layer 4 protocol fixes ooo by dropping and retransmitting so that layer 5 sees in order delivery. UDP drops and the higher level layers have to fix. In that sense UDP at layer 4 is the equivelent of Class-3 FC, and TCP at layer 4 is the equivelent of Class-4 FC. Given that the classes of service in the OSI model are actually layer 4 i now have to correct myself and say that FC layer 2 (counting from 0 so its 3rd layer) is the equivelent of layer 3 and 4 in IP and OSI terms.
I will just answer the “bridges know what to do” part:
* Layer-3 forwarding devices (formerly known as “routers”) have authoritative information that they use to forward the packets. If the destination address (or prefix) is not known, the packet is dropped and (optionally) the sender is informed.
* Layer-2 forwarding devices (formerly known as “transparent bridges”) have best-guess information based on MAC addresses gleaned from past traffic. If the destination MAC address is not known, the packet is flooded.
This is (in my personal opinion) the essence of the difference between routing and transparent bridging, and the root cause for better scalability of routing-based networks.
Simon, in practice, FC has no re-ordering capability. I know that some N_Ports claim the ability to re-order but in reality they only support frames arriving 2 or 3 spots out of order. Given that the types of changes that will actually cause out of order frames to occur would easily exceed the ability of the N_Ports to re-order, there is effectively no re-ordering..
Also, in regards to the way you are counting network layers. FC does not use the OSI model, so your point about FC-2 really being OSI layer 3 is basically invalid. Essentially FC-0 and FC-1 map into OSI layer 1 and FC-2 maps into OSI layer 2. This is pretty clear cut from my point of view.
Osi. Fibre channel Ethernet/ip
Layer 1 Physical Fc-0 physical Ethernet
Layer 2 Data link Fc-1 data link Ethernet
Layer 3 Network Fc-2 Network Ip
Neither FC nor ethernet nor ip uses the OSI model, however using something helps with a common frame of reference on language – a reference model is useful but as i noted all the protocols do strange things causing a lot of confusion.
Ethernet clearly has a very rich layer 2 and fc has a very sparse layer 2 but both put the bit/byte encoding there. As Iven notes, FC switching is like IP routing when IP is using a simple P2P protocol (HDLC, PPP) rather than ethernet links in which case IP would also have a very sparse L2 layer. Even if you contend that all of FC-0 and FC-1 are in ISO 1 that does not change the nature of FC-2 and in most senses i consider this to be layer 3 in the IP context. If you like (and there is no exact comparison) i would say FC-2 is 80% layer 3 in nature with some things that are layer 2 like and indeed some things that are layer 4 like in nature. i guess you will contend that it is 80% layer 2 like.
As technically it is hard if not impossible to do a line by line comparison i do not beleive we will ever agree and i dont know that it matters. i actually think that FC-2’s layer 3ness is actually one of fibre channels STRENGTHS and so something to be proud of. Ethernet would have been better if it was built this way instead of trying to reinvent new protocols to try and be more like FC. As a side note we see some dc’s going far more L3 rather than L2 in their deployment because of the strengths of L3 in multibox deployments.
regarding reorder
ip (routers or end nodes) do not reorder anything at layer 3, TCP at layer 4 doesnt ‘reorder’ either more than a small amount, it drops anthing ooo if there is any significant ooo and TCP retransmits. the net net is that ethernet switches and ip routers in steady state are carefully designed and configured to ensure that TCP traffic arrives in order. this is similar to FC networks in the old days at the session level and these days at the exchange level making sure that the packets go through the same route so that in steady state they arrive in order. all of this frankly is in the boardline between standards vs implementations. my point is that in reality for the ULP’s that need in order all three technologies are implemented in such a way to ensure actual in order so as to make sure that the ULP does not have to do anything complex. In other words discussing IOD/OOD as a characteristic of one layer or another is a red herring.
by the way, i am also forced to point out that a number of people in the FC industry know that the subject of whether FC is basically layer 2 or layer 3 switching is something of a political hot potatoe. ultimately it doesnt matter for most people. i could go through a number of the comments here pointing out what i consider are technical errors on statements on the nature of wwn and mac address, ip address and fcid, etc. but i can also think of points that could be made to support your position. i fully understand that we can find points on both sides of the arguement.
Hi Simon, as I indicated previously your application of the OSI model to FC is incorrect.
I agree that the topic of out of order frames is othogonal to the Layering discussion. I didn’t bring it up in the first place!!
If its not valid to compare with the osi model then it’s not valid to try to make a direct comparison of fc with either Ethernet switching or ip routing as each of them is a standard developed in isolation to meet its own goals. Ip being able to run over Ethernet is is some sense no more or less relevant than fcoe meaning fc can run over Ethernet and if anything confuses the conversation.
Simon, I know a few people in the FC industry as well, I can pretty much guarantee you that whether or not FC is an L2 or L3 protocol is not something that is currently being debated by anyone. This is the first controversy concerning the topic I’ve ever participated in.
I think having you go through and picking apart my arguments would be an ideal next step. I look forward to seeing it.
Given that people may not like OSI as a point of definition/reference i am going to use the following http://en.wikipedia.org/wiki/LAN_switching as a point of definition/reference. we all have to be cognizant of the fact that more and more there is deliberate blurring/violation of layers which can make it possible to have a perspective where in some senses a device can be considered to be at a different layer.
Based on this i am going to state that FC is layer 3 switching (which is slightly different from layer 3 routing and i was incorrect to use the term layer 3 routing previously) and NOT layer 2 switching. i do not believe that the order of entries in the packet/frame are relevant as the details of the packet format are fairly arbitrary. FC is clearly very very similar to an layer 3 ip switch where you have direct point to point connections.
Unfortunately as there is no authoritative definition of layer 2 and layer 3 that we can agree on i also have to accept that there will always be room for us to disagree on the definitions and so at what layer a given protocol works.
MAC Address and WWN are physical not logical addresses. Ethernet forwards based on the physical and in those cases such as VMWARE where strange stuff happens what you have is virtual representations of physical devices and a VMOTION looks to ethernet switches EXACTLY like a physical NIC being unplugged in one place and replugged in another place. layer 2 switching does not use the structure of the physical address to work out where to forward packets but instead the switch learns where devices are and builds up a forwarding table.
FC (before FCoE) has no equivalent of L2 ethernet switching – just as a bunch of IP switches/routers directly connected via a simple point to point protocol would not have either. FC Switches forward based upon a logical address (the FCID) where the value/structure of that logical address along with the logical address (domain id) of the FC switch allows the appropriate path for a packet to traverse to be determined. This is identical to the way IP forwarding works with an IP switch connected by simple point to point links.
FCF’s (FCoE Switches) operate exactly like IP Switches/Routers where they are using ethernet as the connection protocol between end devices and the FCF and between one FCF and the next. in both cases the layer 3 address (FCID or IP) stay the same and the layer 2 address (MAC) changes as you pass through each layer 3 forwarding device.
If one goes back to the history of where and how FC Switching was developed, where brocade in particular was developing and selling switches prior to the standards using its own silicon and software, and working on the standards, and where McDATA as it moved from ESCON to FC was licensing brocade silicon and software to make its first generation of devices, it is also very clear to me how those protocols and switches were derived and developed with significant content from the way IP worked with changes and enhancements to meet the DC requirements.
To me it is very hard to not say the closest analogy is FC forwarding is like IP forwarding, and if we define ethane forwarding as layer 2 and ip forwarding as layer 3 then FC forwarding is layer 3.
Simon I completely agree with one thing you said in your latest response:
“Unfortunately as there is no authoritative definition of layer 2 and layer 3 that we can agree on i also have to accept that there will always be room for us to disagree on the definitions and so at what layer a given protocol works.”
I’ve already provided counter arguments to your other points and to do so again wouldn’t be productive for the above reason.
Erik, it is perfectly fine to agree that we have different perspectives on what the definition is of layer 2 and layer 3 and my intention all the way through here has been to highlight that the definition effects the result and see if we can agree on a common definition. It is perfectly fine to agree that when looking at the functional behavior it depends which aspect of the function one looks at as to ones view and that this makes such a joint agreement hard.
Ultimately I think though you have not said this that you would agree with me that the first few bytes of both the Mac address and wwn define the vendor that made the interface and the remaining bytes are unique within that vendor and have no meaning. You correctly point out to me that these days Ethernet and fc have virtualized these as well in different ways. Of course fcoe in my view breaks layers in the way it creates it’s virtual Mac addresses so violating this definition.
Ultimately I think though you have not said this that you would agree with me that the ip address and the fcid directly identify your location in the network – that is their meaning and purpose, in the fcid case the high byte identifying the fc switch that you are connected too and historically the middle byte the port and low byte your address in the loop of that port (though these days only the use of the high byte is perhaps solid) and in the ip case the subnet you are on and so indirectly which router you are on and the low byte your identity within that subnet.
I think you did agree that ospf and fspf both do the same function but we also all agree that with trill and other things such protocols are applied sometimes in layer 2 and layer 3. I’m not sure whether you agree or not that fspf was directly derived from ospf but I understand your point that this is not relevant.
If my definition of layer 2 and 3 is that noted in paragraph 2 and 3 above and your is a different definition then as we both say we are simply disagreeing on the definition of layer 2 and layer 3.. It does not matter to me if we agree on the layer definitions as we all agree that these are fairly arbitrary though IMHO in a converged world we all have to be clear that different protocols have different definitions. Just so long as everyone agrees on the algorithms used in forwarding, route selection, etc, then we are all able to agree on how things work and so when they will or will not work and so agree on best practices and supportability.
In my experience we have always and continue to achieve the last critical point and that is what is most important to our respective and often joint customers. As such I have the highest respect in you in Emc and your work in testing qualifying and supporting customers deploying these technologies.
Simon, in regards to your paragraph 2, you’re basing your entire argument on the fact that WWPNs and MACs both use OUIs and then you go onto say that FCoE, which contradicts this assertion, is somhow a layering violation and is therefore an invalid counter argument against your point of view? Taking into account all of the points that have been made thus far, I think it’s just as likely that your characterization of FCoE merely violates your invalid layering definition.
In regards to your paragraph 3, we’ve already discussed this, so let’s boil it down. Are you prepared to argue that TRILL is layer 3 for the same reasons that you claim FC is?
If you want to make the argument that we are really trying to enhance our end users understanding through the use of imperfect analogies, consider the following:
When speaking about FC switching, I say Layer 2 because everything in a fabric (in the same 24 bit address space) can access everything else by default. Also, RSCNs almost make it seem like all 24 bit addresses are in the same “Broadcast Domain”. FC fabrics (that do not use zoning) and L2 Ethernet LANs also have about the same scalability limits for reasons that are similar enough to use for the sake of a high level comparison.
When speaking about FC Routing (which uses WWPN associations to establish connectivity), I say Layer 3 because the standard FC-IFR (Inter-Fabric Routing) and the implementations IVR (inter-VSAN Routing) and IR (Integrated Routing) all use the term ROUTING IN THEM!!! Also, fabric RSCNs are not forwarded by FC Routers which is analogous to what happens to broadcast frames when they hit an IP router.
How does your model make FC’s functionality easier to understand?
Erik,
No matter what terminology one wishes to use the following are simple facts:
1) the WWN is burned in to the HBA at the factory just as is the MAC address and has part identifying the vendor and part unique within that vendor like the MAC – indeed reading the FC standards and supporting documents i do see clear references and terminology that is the same as that used in the MAC standards
3) any modern additions are virtual instances of what was originally a physical identification and under almost all circumstances for both FC and Ethernet are not detectable as virtual rather than physical so they are virtual and not logical and cannot be used to justify a different position
3) the FCID is a logical address that directly identifies your location within the network just like the IP address
4) the IFR standards clearly refer to NAT (Network Address Translation) with as i recall the simple version referred to as IFR-NAT and the more complex version described as a strict superset of the former. anyway these were developed way after FC ‘switching’
5) IP broadcast and multicast do exist though of course some of that is restricted to within the subnet but then some FC stuff is restricted within the zone – in the late 90’s brocade even used to refer to zones being like VLANs
6) although FC switching does not have a hop count, it synthesizes the effect in part by implementing age discard, limiting the hop count, all to the requirement for max time to live across the fabric – frankly it would have been better if there had been some form of hop count
7) as already noted the other pieces map fairly cleanly to DHCP, DynDNS, FSPF etc etc
Ultimately, ethernet which is typically referred to as L2 has some L3 characteristics within the broader ethernet protocol set and IP which is typically referred to as L3 has some L2 characteristics within the broader protocol set. more recent technologies such as TRILL have further muddied the waters but TRILL is actually usually deployed as Ethernet tunneled in Ethernet (L2 in L2) just like mac-in-mac or q-in-q, and uses a few L3 constructs on the edge.
As such for an ethernet/ip guy trying to understand FC the IP analogy is the closest comparison point and the only one that gives a broadly consistent mental model to understand FCs’ top to bottom operation. It is also broadly consistent no matter FC or FCoE as i noted.
No matter the terminology FC is far more IP like than ethernet like and indeed if you gave any ethernet/ip person pure non marketing based training on FC and FCoE they will come to the conclusion that FC is doing layer 3 switching/routing, that FCoE adds layer 2 capabilities beyond the previous simple link level stuff, that IFR and SAN Routing is more like NAT than anything else, and so on.
Now both sides have to respect each others language both in its precision and in its ambiguity, in its rules and in its exceptions (and i was always taught the exception PROVES the rule) and both in the standards history and common practice terminology. Now ultimately no where in the FC standards does it say that the FC SAN Fabric is doing layer x switching it simply defines the behaviour. indeed as we have noted elsewhere ethernet uses both the term switch and bridge, and for IP as i noted you have both IP routers and layer 3 IP switches – as such we should not get tied up on the term bridge switch router or frame packet but look at the operational model. anyway FC clearly defines switching to be at the 3rd layer of its protocol stack and its only called FC-2 because it counts from 0.
You tweeted me to draw my attention to this blog and my view is simple clear and self consistent and the exceptions are comparatively rare to my case and i have openly drawn attention to those exceptions.
Now i can see that if you want to reverse WWN and FCID then you can construct a model which on some basis might be considered layer 2, personally i think its a flawed mental model but more importantly it is a model that has little to no comparison with the way that ethernet switches work and as such is not a useful mental model in discussion between FC/FCoE people and Ethernet people.
Finally, i think like you i came to FC in the very early days of FC Switches and before those switching standards were completed, before that i spent 15 years in the lan and wan space and was fully qualified in ethernet and ip protocols from multiple vendors.
i fully appreciate that this approach (and indeed agreement style and british humor) may irritate some FC people (though i say confidentially not all) however after a decade teaching FC, iSCSI, storage, first to WAN and LAN people in the days of FCIP and iFCP and again to these people in the new world of FCoE i am 100% confident as to the best way to teach an ethernet/ip person how FC actually works in terms of its networking characteristics at the asic firmware configuration and debugging levels.
if anyone wishes to explain to me how in full detail in terms of its full forwarding model FC works more like ethernet than ip i would be interested to hear it because i cannot see how to do so other than the marketing position of default global visibility and as my laptop can ping anywhere in the works that is not blocking me with a firewall even global visibility is not an L2 trait.
Simon, I’ve already addressed all of your “simple facts” in the previous comments.
I used a marketing description of L2/L3 because that was the only tact I hadn’t yet tried.
I have no intention of explaining how FC switching works in this comments section. Anyone interested in this topic should Google “Networked Storage Concepts and Protocols techbook”, download a copy and enjoy. The switches section that starts on page 147 (that I wrote 10 years ago by the way) is just as relevant today as it was then. My personal favorite is the Build Fabric (Fabric Configuration process) description.
Yes, the fact that my tweet brought you into this discussion is a bit ironic.
Since no new arguments have been made against my assertion that FC is L2, I’m pretty much done discussing this topic in this forum. We’ll just have to agree to disagree.
“Peace and long life”
We have both responded to each others points so further discussion does not add value. You are only disagreeing with me on names and relationships and not on how addresses are created or forwarding decisions made. If we have a discussion on how a storage subsystem works I will defer to you on the actual implementation details as Emc designs and builds such systems and I have never worked for a company that designs and manufactures said systems. Live long and prosper.
The location of the wwn and fcid does not effect how the addresses are created or consumed.
Simon, I found it amusing that you could go from stating:
“As such I have the highest respect in you in Emc (sic) and your work in testing qualifying and supporting customers deploying these technologies.”
To implying that I have no place discussing L2/L3 bridging and routing based on the company I work for:
“If we have a discussion on how a storage subsystem works I will defer to you on the actual implementation details as Emc (sic) designs and builds such systems and I have never worked for a company that designs and manufactures said systems.”
BTW, I didn’t say I was incapable of discussing FC switching architectures or how forwarding decisions are made; it’s just that I’ve already written about the topic at length (refer to the techbook) and I don’t wish to repeat that information here.
You said: “You are only disagreeing with me on names and relationships and not on how addresses are created or forwarding decisions made.”
For the fourth and (hopefully) final time, the way an address is created or assigned is not an absolute indication of the layer to which it belongs. Both IP Addresses and MAC Addresses can be (and are in practice) assigned dynamically or statically.
Also, when making forwarding decisions in FC, the address closest to the physical layer is the FCID, not the WWPN. I know you don’t think it matters, but consider this; outside of FLOGI/PLOGI and maybe a couple of NS registrations, FC Frames do not contain the WWPN and it can therefore not be a Layer 2 address. Can you explain why my reasoning is flawed with regards to this specific point?
In regards to your point about FC starting at FC-0 and therefore FC-2 is actually OSI Layer 3. My interpretation is that the FC model broke OSI layer 1 into two different layers. If you look at the description for OSI Layer 1, it is described as the physical layer and Layer 2 is described as the data link layer (Frames, data delivery, etc). If you refer to the FC Layer model, FC-0 is physical media and FC-1 talks about 8b/10b, encoding, etc. It isn’t until FC-2 that Frames and Addresses come into play and since FC-2 is the first layer that discusses frames and data delivery, this is consistent with my assertion that FC is a Layer 2 protocol.
Tony and Others,
First, I work for Brocade, and before that, I worked for McData Corp., so I am indeed old and grey 😉
I like this introduction to the book “Fibre Channel” by Alan F. Benner, McGraw-Hill, (c) 1996, as it aptly describes the trend toward technology unification underlying the development of the Fibre Channel standards. (It was my first Fibre Channel book, by the way). It makes clear why there are varying definitions (Ivan, Tony, Erik, Simon).
“Fibre Channel is part of a general trend toward increasing unification of different areas of the data communications industry. From a broad perspective, three different communities have historically been interested in data communications. These three communities have had fairly little interaction, and have largely used different technologies, terminologies and design languages, because the requirements of their operating environments were so different, but they are moving towards closer cooperations.
First is the telecommunication industry, which has traditionally been run by telephone companies. …
Second is the LAN (local area network) communication industry, which has been dominated by network companies closely tied to computer companies. …
Third is the channel, or storage industry, which, while not generally considered in communications terms, has many similar problems and design considerations …”
Such was the world view circa 1995 when Fibre Channel was emerging, and I find much the same market and technology drivers exist today almost 20 years later.
When a new “hybrid” technology emerges, those in various neighboring industries use the words and concepts they know best to understand it. That’s the first step, and many analogies are used to help make the new words and concepts clear. Later, as they become proficient, they abandon the analogies, and instead use the words invented with the new technology to describe it. I think the comments here are an excellent example of this way of learning new things.
BTW, I liked the post as it uses “good enough” analogies to help educate those interested in a new technology, or feature, so they can get a “first order approximation” of what it is. Good job.
One of the fun things in the IT industry is that we the experts need to be split personality split brain. At the same time we need to know exactly how the protocol and products actually work technically and we need need to be able to use good enough explanations comparisons and analogies even though they are often technically incorrect.
This reminds me of school where it seemed sometimes, particularly in physics, that every year my teacher would tell me that what we were fought the previous year was incorrect and that now we would be told how things really worked.
For some of us who have both been in the fibre channel industry since before the days of the modern San fabric, and who before that grew up with Ethernet and ip as these evolved from 10base2 and 10base5 through hubs and to switches, and who also were thought networking first principles at university, it is easy to know but often hard to explain all of the nuances.
Even going back to the standards (which are not the first principles of course) can be challenging as the standards are full of politics – something not unique to the networking industry of course.
I agree, there are two competing goals at work in the learning process: Accuracy versus relatability.
When learning a new skill/knowledge people tend to build on existing knowledge, and that requires taking existing knowledge and relating it to new knowledge. This always comes at the expense of some degree of accuracy.
So while comparing Fibre Channel to Ethernet/IP is not accurate, as they have many fundamental differences, it is useful to help build a greater understanding using pre-existing knowledge. The pedantic learning can come later, as a subject is better understood, you can add a much higher degree of specificity.
Pingback: Access Gateway – NPV – IPM | Home
I would like to say that, unlike NPIV, NPV is not a term defined in any standard. It’s a Cisco term that actually stands for N_Port Virtualizer/zation. Other vendors call it differently. At Brocade we call it Access Gateway mode. We should try to avoid using vendor-specific terms as if they were industry-standard terms.
Yup, my bad. I’m preparing an update post. I knew NPIV was industry standard term, and just assumed NPV was. I’ll make that distinction.
What’s wrong with my “ghost device”, eh? We can make it an industry standard. George will not oppose, nor will Brooke. Right guys? 😉
(BTW, remember it _does_ send FLOGI to initialize its own link first, then passes FDISCs for dudes behind it; let’s not give Ethernet people chance to mess things up, eh? 🙂
yes i agree with Juan, in fact the reference in the FC standards to N_Port Virtulizating Device is very thin – the only reference is to it being a new type of device and you have to look at the supporting docs/ppt form the standards meetings to work out that it is a device that internally does exactly what is intended.
all that said, although some times i dislike for many reasons using terms that come from specific vendors, in this case i actually think NPV is perhaps a term that we SHOULD formalize as the official acronym for such a device. unlike an earlier acronym from Cisco related to CEE/DCB i do not believe that they trademarked the term NPV and so it is fairly harmless APART form the fact that if you google you will almost exclusively find cisco related materials.
historically i used to refer to such devices as ‘NPIV Proxy Gateways’ to make the point that they were using the well known NPIV protocol from them to the switch to connect and then parodying the FC services form the switch forwards to the servers in order to gateway the servers in to the actual switch without themselves being a switch.
Simon, that was exactly my concern. If an influential blogger uses a vendor-specific term as if it were industry-standard, all the readers interested in finding more information about the topic will only stumble upon one vendor’s solution and walk away believing that is the only such solution in the market…
Juan,
In this case the FC vendors did a poor job in creating the term N_Port Virtualizer / ing device and not creating a sensible term for people to use in normal language. I’m not even sure if I should say N_Port Virtualizer or N_Port Virtualizing Device.
I’ve argued on your side in the past with my nemesis and he convinced me that NPV is a sensible reasonable abbreviation and so cannot be considered a vendor proprietry term.
As i noted before this is different to Data Center Ethernet where the abbreviation was trade marked and do could not be used by others.
Erik,
my comments about your position and that of your companies were to recognize that you have more knowledge of the subject of fibre channel than most people and that the work you do gives you deeper knowledge of fc networking than even many people that work for the switch companies many of whom do not have the long history that both of us have. if this was interpreted otherwise it was not my intention.
I have on more than one occasion tried to remove this from a conversation of layer 2 and layer 3 though sometimes i have stepped back in to it. The point I and others have made in the spirit of the original artical is that irrespective of layers and ones definition of them and indeed packet formats (both of which are often arbitrary) the simple bottom line is that the full complement of forwarding mechanisms used by the switch are similar to ip switch/routers and dissimilar than Ethernet. Understanding this is very useful for anyone who is crossing the boundaries between fc and Ethernet/ip. You seem to have not commented on this point which both ivan and I made and both of us have spent time in other aspects. In making this point I am not making any assertion wrt whether either should be considered as layer 2 or layer 3 and when discussing with people I usually remember to make the point that fc people consider this l2 switching.
It is true that I have other disagreements and these are related to the point you used when you invited me here but not related to the intent of the original artical. I believe these are very important points.
I and others do consider the Mac and wwn to be equivalents (ibm red books make that comparison and I have the same respect for their authors as I do for you). The official assignment of both is the same and whether you agree or not on those occasions it is assigned in other ways those ways have to simulate the effect of their original assignment mechanism. The Mac wwn issue is very much a secondary consideration. Doing google search on wwn IEEE and wwn Mac as well as reading the standards shows the similarities clearly.
I and others consider fcid and ip to be similar and both fundamentally logical in nature by which I mean directly stating your position within the network something which cannot be denied. Being logical in nature is saying something very different than whether the address is assigned or not assigned. This similarity between fc and ip is critical in their very common operation and so in cross training people.
Despite the name used I also state wry clearly that what fc calls routing is not what ip calls routing but rather has more in common with NAT – a point made by others I think and even referenced in the standard.
As fc was originally in a sense a monolithic stack in isolation from non fc protocols it has it’s own language and considerations. Despite IFCP and FCIP before FCoE frankly the cross protocol discussions were largely uninteresting. It was developed by a completely different standards body to Ethernet and ip. As ip and ethernet have very different heritages to each other and are owned by two separate standards bodies some things are more complex than fc, but some things like layering are far clearer.
All of this makes many of the discussions complex and makes full agreement impossible. Taking your (I think) prism analogy despite the other differences I do ask you to consider why many people find the fcid / ip, flogi / dhcp , name server / DNS, fspf / ospf comparisons useful ad appropriate. Good Ethernet and ip people, I’m sure like good fc people, know full well the ambiguities in their respective areas so lets not waist time on these points.
For anyone who wants to compare wwn and Mac http://en.wikipedia.org/wiki/World_Wide_Name is not bad and has pointers to both IEEE and t11 as well for further details.
http://en.wikipedia.org/wiki/Data_link_layer says…..
The data link layer is layer 2 of the seven-layer OSI model of computer networking. It corresponds to, or is part of the link layer of the TCP/IP reference model.
Delivery of frames by layer-2 devices is effected through the use of unambiguous hardware addresses. A frame’s header contains source and destination addresses that indicate which device originated the frame and which device is expected to receive and process it. In contrast to the hierarchical and routable addresses of the network layer, layer-2 addresses are flat, meaning that no part of the address can be used to identify the logical or physical group to which the address belongs.
Under that definition there is no way that fc switches can equate to layer 2 in osi terms as forwarding as the fcid violates the very fundamental rule noted here per my arguement at several points in this thread. Fc as I noted has a sparse or non existent layer 2 layer. Even the fc standards refer to IFR as nat.
By the way the wiki entry on osi layer 3 says….
In general, direct or strict comparisons between these models should be avoided, since the layering in TCP/IP is not a principal design criterion and the Internet Engineering Task Force (IETF) considers it “harmful”.[2]
which is a good way of saying lets not bother argueing 🙂 because IETF don’t believe in layer models at all……….
My lecturer at university when I learned networks back in the 80’s was very disparaging about networking standards and I have to agree with him……….
On that note I will not respond to further comments on this thread
Pingback: The NPIV/NPV Article: The Wolf 359 of the Internet | The Data Center Overlords
Pingback: NPV and NPIV – Gestalt IT
Pingback: NPV and NPIV – Gestalt IT
Pingback: Technology Short Take #24 - blog.scottlowe.org - The weblog of an IT pro specializing in virtualization, storage, and servers
Would I be correct if I explained an NPV enabled FC switch as a “pass through” device?
Hi Chris, I would avoid using the term “pass through” to describe an NPV gateway since “pass-thru” has a special meaning in the blade server space. Generally speaking, a pass-thru device presents the “internal” ports on a blade server as an external port on the pass-thru that can then be used directly by the customer. The net result is each internal port ends up with an external unshared interface. Outside of the bandwidth considerations, this is beneficial because the interoperability concerns introduced by any intermediary devices (e.g., blade server switch modules) are mostly eliminated. The downside to the pass-thru is that is will require more cables and access or aggregation layer switch ports to be consumed.
Pingback: NPG (NPIV Proxy Gateway) Mode Vs Full Fabric Mode in FCoE | Internet Protocols
Pingback: Fibre Channel in the Cloud: FCaaS | The Data Center Overlords
I sincerely thank you because I never have been able to fully understand this concept but, thanks to your easy explanations, now I got it
Fantastic post. Even in 2017.
Cheers
same as marco
I suppose by saying pass through, the writer was addressing the process rather port extension literally.