CET2176C - Server+ Lecture #15 - Designing Mission Critical Servers

Materials:
Lecture Only
Objectives:
The student will become familiar with:
The server planning phase in detail,
The server availability solutions,
Each technology's functions and capabilities,
The technical details of each technology,
Competency:

The student will become familiar with the server planning phase with an emphasis on high availability solutions including the various server hardware and physical framework technologies including their features, capabilities, limitations and be able to choose the best ones for a given server as well as be able to select the best solutions for an enterprise level network and be able to design, construct and install the server.

Server Planning Phase
  1. The basic details of the Server Planning Phase were fully outlined in Lecture #2 - The Server Project. Refer to this lecture as needed. In this module, two different servers fulfilling two different roles will be designed.

  2. The first server will be a high capacity, high network throughput file server. The anticipated requirements are:

    Storage Capacity: 6TB
    Storage Performance: Minimum 320MB/sec controller-to-drive interface, highest speed hard drives possible
    Storage Fault Tolerance: Minimum 2 controllers, each able to tolerate one drive failure
    Network throughput: Minimum 300MB/sec
  3. Based on this information alone, this file server will obviously be under very high workloads including delivery of large files to multiple high speed connections simultaneously. The system should be equipped under these circumstances with no less than 2 CPU's each a dual core or better and at least 4GB of RAM. The system will use two SCSI U320 controllers (capable of the minimum acceptable controller-to-drive interface speed) and 15000RPM hard drives (the fastest possible) The SCSI controllers will be capable of hardware level RAID 5 in compliance with the minimum fault tolerance expectations. The operating system will be Windows 2000 Server. The only role for the server will be to coordinate rights restrictions of the users to its files and folders with the domain controller. (Basically no roles other than file sharing.)

  4. Projected parts list

    QTY  ITEM                                                              COST   PRICE
     1 x Asus DSBV-D Motherboard w/onboard video.........................  $315    $350
     2 x Intel Xeon L5320 Quad-core 8MB on-die L2........................  $714    $800
     2 x LSI Logic 3202064B dual channel U320 PCI-X RAID controllers.....  $490    $800
     1 x Intel I940 4-port PCIe x4 Gigabit Ethernet NIC..................  $438    $440
    22 x Seagate ST3300655LW 300GB U320 15000RPM 68-pin LVD SCSI HDD.....$12980  $14300
     4 x Kingston DDR2-667 1GB Fully buffered ECC DIMM...................  $320    $400
     1 x 23-bay server case..............................................  $458    $500
     1 x iStar 1800W quad-redundant power supply......................... $1050   $1050
     1 x Cables-to-go #32568 8-port VGA/PS2 rackmount KVM................  $212    $220
                                                                         $16977  $18860
                                                                          LABOR:  $1880
                                                                           TOTAL:$20524
     NOTES: Server will use site's existing I/O devices
            OS and applications provided by the client
    

    Featured Components

    Asus motherboard with dual Socket J (LGA771) supports two quad core
    Xeon processors and features three PCI-X (64-bit/133MHz) slots

    LSI U320 PCI-X RAID controllers support up to 15
    hard drives perchannel in RAID-5 configuration

    Intel 4-port 1000Base-TX Gigabit Ethernet NIC for PCI-Express x4 slot
    The NIC supports 802.3ad Link Aggregation Control Protocol for using
    all four as a single connection to the network backbone (500MB/sec)

    iStar 1800W (quad 450W modules) redundant load balancing power supply

  5. The only way possible to achieve a network connectivty throughput of greater than 125MB/sec (1000Mbps) is either to use a 10G Ethernet controller (1.25GB/sec) or to use network adapters that are 802.3ad "link aggregation" compliant. Since a 10G Ethernet controller would have to attach to a 10G Ethernet connection on the switch which would if available be quite expensive, the 802.3ad NIC "teaming" technology is chosen instead. As long as the switch also has four 1000Base-TX ports that also support 802.3ad, then this NIC can exchange data with the switch at 125MB/sec x 4 = 500MB/sec.

  6. The only way that the two U320 SCSI controllers can attach to an expansion bus at their maximum data transfer rates is if the bus supports that DTR as well. PCI-X is a parallel bus, but it is 64-bits wide and runs at 133MHz yielding a DTR of 1066MB/sec. This supports the U320 controllers totaling 640MB/sec and the desired network connectivity throughput of 300MB/sec. The NIC actually supports 500MB/sec, while streaming data at this speed, the expansion bus can easily match this performance from the two hard drive controllers. However, it should be noted that in reality the PCI-X bus is somewhat overwhelmed by this system's I/O requirements. The solution would be to use PCI-Express devices.

  7. In comparison to the above design, if the controller-to-drive interface requirement is relaxed from SCSI U320 (320MB/sec) to SATA-II (300MB/sec) and the hard drive speed requirement is reduced to 7200RPM, then the system looks like this:

    QTY  ITEM                                                              COST   PRICE
     1 x Asus DSBV-D Motherboard w/onboard video.........................  $315    $350
     2 x Intel Xeon L5320 Quad-core 8MB on-die L2........................  $714    $800
     2 x Adaptec 4-port SATA-II PCI-X RAID controllers...................  $530    $800
     1 x Intel I940 4-port PCIe x4 Gigabit Ethernet NIC..................  $438    $440
     8 x Seagate ST31000340AS 1TB SATA-300 HDD........................... $2000   $2800
     4 x Kingston DDR2-667 1GB Fully buffered ECC DIMM...................  $320    $400
     1 x standard PC 8-bay tower case....................................  $150    $250
     1 x iStar 1800W quad-redundant power supply......................... $1050   $1050
     1 x Cables-to-go #32568 8-port VGA/PS2 rackmount KVM................  $212    $220
                                                                          $5257   $7110
                                                                          LABOR:  $ 710
                                                                           TOTAL: $7820
     NOTES: Server will use site's existing I/O devices
            OS and applications provided by the client
    
  8. The largest 15000 RPM hard drive (at the time of this writing - Apr. 2008) is 300GB. This forces the system to use each RAID-5 with 10 hard drives + 1 parity drive in order to have the 3000GB or 3TB required capacity (understood that these are hard drive manufacturers even billion "gigabytes") By switching to SATA-II the controllers are actually slightly more expensive, but the drives are much less expensive as well as far fewer in number. Clearly the sacrifice is worth reducing the number of hard drives (components with moving parts that will eventually fail) from 22 to 8 while reducing the total raw cost from almost $17,000 to about $5250! Note that the markups are about the same so it is possible to "white box" a powerful server for the customer and make a reasonable profit (roughly $3000 for these two versions of the file server) and be able to save the customer by making wise sacrifices close to $12,000.

  9. The second server is an application server that will be running SQL Server and service many employees simultaneously. While the storage requirements for this server are dramatically lower than the file server, the processing power requirements are far greater. The ideal goal of our application server is as many processor cores as possible and as much RAM as possible. The server will be fine with a single RAID-5 controller and 2TB of total storage capacity. The minimum network connectivity throughput to the network backbone is still 300MB/sec.

    QTY  ITEM                                                              COST   PRICE
     1 x Supermicro Quad Socket 604 Motherboard ......................... $1310   $1500
     4 x Intel Xeon X7350 Quad-core 8MB on-die L2........................ $9556  $10500
     1 x Adaptec 4-port SATA-II PCI-X RAID controllers...................  $265    $400
     1 x Intel I940 4-port PCIe x4 Gigabit Ethernet NIC..................  $438    $440
     3 x Seagate ST31000340AS 1TB SATA-300 HDD...........................  $750    $850
     4 x Kingston DDR2-667 4GB Fully buffered ECC DIMM................... $1100   $1300
     1 x standard PC 8-bay case..........................................  $150    $250
     1 x iStar 1800W quad-redundant power supply......................... $1050   $1050
                                                                         $14619  $16290
                                                                          LABOR:  $1630
                                                                           TOTAL:$17920
     NOTES: Server will use KVM of the file server, and site's existing I/O devices
            OS and applications provided by the client
    

    Featured Components

    Supermicro quad socket 604 motherboard supports Xeon X7350 Quad Core
    for a total of 16 processor cores and up to 192GB of ECC RAM

  10. This server has $10,000 in processors, an expenditure that will be difficult to avoid since it is one of the few "white box" (complete PC built from industry standard components, and has no name brand) systems that supports four quad-core xeon processors each with 8MB of L2 cache. There are less expensive quad-core processors but they are either only dual-CPU capable or have less L2 (equal or greater L3 but L3 is not as fast as L2) A system based on four quad-core AMD Opteron processors might be possible, but AMD often runs L2 at a fraction of core speed.

  11. It is important to remember OS limitations as well. While these systems are physically possible to design and build, the operating systems may either not have drivers for the devices (motherboard chipset is a particular concern) or the combination of device drivers may conflict. Especially with high-end server components where there is a very limited marketplace, there is not nearly the quantity of feedback to the manufacturers as there is with end-user components. It is more likely for the technician to assemble a unique combination of components that have conflicting issues and it will likely be necessary to change one or more components in order to get the system a combination of components whose drivers "behave." As such, a minimum of 10% markup on the parts is required to allow for making these changes during the prototype construction, installation, and post-installation phases to provide funds for purchasing alternative components when the vendor refuses to accept the opened item although many will allow exchanges which is all that is needed.

Fun work
  1. Design a comparable application server to the one above, but based on four AMD quad-core processors.

  2. The quad socket 604 motherboard above has a proprietary form factor which could cause serious problems with mounting it to a standard ATX case. Try to find a standard form factor quad Socket 604 motherboard and matching form factor case.

  3. For the system above, find a case with dual fail-over power supplies rather than using the quad-450W module power supply mentioned in the lecture.

  4. As mission critical servers both need to be attached to some form of surge protector and UPS, price the largest, measured by VA, value UPS that you can find. NOTE: some UPS'es are "stackable" in that more battery power can be added onto them. Find the largest "single box" UPS on the market.

Copyright©2000-2008 Brian Robinson ALL RIGHTS RESERVED