Hey all,
We have some really good questions come by our desks that require some really good answers. If you've worked with our hardware, you know that when it comes to Mac Networking we play a bigger role then most think.
This is my attempt to keep some of these questions aggregated here so that you can become more familiar with how Small Tree can help you sell more and help your clients to continue working faster and still remain just as creative!
1. A client is already using Ethernet to share files between editing/graphics workstations. What is the quantifiable gain from implementing a Jumbo frame LAN and segmenting via VLAN with a Small Tree switch?
Using AFP and assuming you have fast enough storage and enough memory on the server, the best you'll see over a normal network is 50MB/sec (over a single Gigabit port). With jumbo frames over the same single port, you will see 70MB/sec. We have proven this several times.
Memory on the server, storage bandwidth and filesystem fragmentation are the most common causes of lower performance than this. (Note that the storage on a client is typically capable of 40MB/sec, so if the file is being copied over, that's the best you'll see. If it's being read into memory or written from memory (as with Final Cut), then that's not a factor)
Performance scales well as you add more ports to this configuration. (IE 4 ports should achieve 280MB/sec provided you have enough clients pulling to use all 4)
2. Does trunking on the client side allow for more simultaneous streams of video. Example, a client is using Compressed HD footage for editing a reality TV series. Since a single stream of Compressed HD doesn't exceed 100MB per second. Will multiple GbE connections trunked allow for multiple streams of video to be accessed?
No. Many many customers make this assumption.
Here's an explanation:
TCP cannot tolerate out of order packets (all of those paths are exceptions and cause TCP to go through slow path code and ultimately re-enter slow startup mode). Hence, 802.3ad trunking cannot "split" traffic from a socket onto multiple ports. (consider that port A would receive packets 1, 3, 5, 7 and port B would receive 2, 4, 6, 8... All of these are "out of order" and would need to be reordered.. And that's a best case scenario. What if packet 3 gets dropped or is some other type of packet, or belongs to another socket that is also being "split"?)
802.3ad allows you to greater a larger pipe between the Server and its clients -- by combining several small pipes together between the server and the switch. It allows you to support a greater number of clients at the same time, making the network appear more responsive from the clients perspective. By enabling the Server to provide more bandwidth at the same time, makes a larger pipe between the server and the switch and thereby supporting more clients access to the server at the same time. This is an age old problem that goes back a long way.
If you have 10 clients and they are all gigabit, and you have one server and it's gigabit, how many clients does it take to saturate the server? The answer is theoretically 1, and realistically 3 or 4. With just 3 or 4 clients pushing or pulling, the server runs out of bandwidth. Even if you purchase two 500MB/sec RAIDS and stripe them out, you have saturated your gigabit port because your 3 or 4 clients are all pulling 30-40MB/sec.
One solution is to buy a bigger pipe for the server, but that's often very expensive and at times, a larger pipe simply doesn't exist (for example, if you went with 10Gb to the desktop today, you'd be stuck).
So the answer is 802.3ad. With that, you can create a larger pipe in the server. This allows you to match the servers network bandwidth to your storage bandwidth and scale up your client load more effectively and efficiently, without spending a ton of money on the next size pipe (10Gb).
3. What is the real-world utilization of GbE and 10GbE. Fibre Channel engineers always state how inefficient GbE is due to TCP/IP. I've heard claims that GbE maxes out at 60Mbs, is this true?
Gigabit will do 50MB/sec with standard frames, 70MB/sec with jumbo frames over AFP. (over a single port)
The problem with being more specific than this is that it changes based on a number of factors.
Our rule of thumb is that it takes 1Ghz of CPU to drive 1Gbit of network bandwidth. One a single processor power PC system, you can see the issue. We're going to eat most of the processor and there won't be much left over for Final Cut Pro.
On an 8 core Intel Mac Pro, I can assume one of the 8 cores is going to handle the network and the user probably won't even notice! In fact, that same core will still have 60-70% of its cycles remaining!
The main issue is AFP. It's just not very efficient, so the 50 and 70MB/sec numbers still seem to hold.
4. How does 10GbE compare to Fibre Channel in regards to price and performance? A side by side comparison of switches, HBAs, cable run lengths, etc... would be very informative and helpful in helping clients see why they should choose an Ethernet-based solution from Small Tree.
The jury is still out on this. All of our 10Gb testing has been with point to point setups and we simply haven't tested scaling yet. AFP seems to go around 170 MB/sec for writes and 110MB/sec for reads, but we're not sure if that's AFPs issue or ours just yet. (Myricom is not faster, so I suspect it's AFP). The one potential bright spot here is that our Intel cards support multiple threads reading from the card. So it's possible we'll be able to scale that number quite well (IE many clients can achieve the 110/170 number rather than just one).
We're actually setting up some scaling tests this week (3/15) and they will likely lead to some driver tweaks and revisions.
For the moment however, I think it's still safe to say 10Gb is awesome in the server (it provides better performance than a 6 port card link aggregated), but we just don't know enough about it to say it will work all the way out to the desktop.
5. How does the 802.3ad Link Aggregation technique provide improved performance to our clients given our application?
802.3ad Link Aggregation allows you to support multiple Ethernet connections [conversations] at the same time. This means that your supporting more clients applications in parallel, which means your server is providing more bandwidth at more of your clients any given moment, and that your clients can get their work done faster due to less waiting on the server.
*Updated 4/21 - I received some email comments about some confusion with questions #2 and #5 on this post. I've gone back and I've edited accordingly to hopefully eliminate some of the confusion.
Small Tree Communications Extreme Mac-Over Sign Up
