Difference between revisions of "Altix 350"

From Nekochan
Jump to: navigation, search
m
m (Added note about mixing bricks with diff CPU speeds)
 
(17 intermediate revisions by the same user not shown)
Line 4: Line 4:
 
[[Image:altix-350-32p-rack.jpg|thumbnail|A full rack of Altix 350 nodes and two routers]]
 
[[Image:altix-350-32p-rack.jpg|thumbnail|A full rack of Altix 350 nodes and two routers]]
  
This page captures information specific to the Altix 350 model. Please see the very detailed '''[[Altix]]''' series overview page.
+
This page captures information specific to the Altix 350 model. Please see the [[Altix]] series overview page for information about other models.
  
== General Information ==
+
SGI Introduced the Altix 350 model at LinuxWorld in New York on January 20th, 2004. This model was a midrange complement to the [[Altix 3000]] line introduced the previous year, and could be configured with as few as one Itanium 2 CPU in a single 2U module or "brick," to as many as 32 CPUs across 16 modules in a full 39U rack. Meanwhile the 350 provides the same single system image and software features found in the larger configurations, and runs the same operating systems and applications. The model was also offered as a cluster node using Ethernet and Infiniband interconnects.
  
SGI Introduced the Altix 350 model at LinuxWorld in New York on January 20th, 2004. This model was a midrange complement to the [[Altix 3000]] line introduced the previous year, and could be configured with as few as two Itanium 2 CPUs in a single 2U node or "brick," or as many as 32 CPUs in a full 42U rack. Meanwhile the 350 provides the same single system image and software features found in the larger configurations, and runs the same operating systems and applications. The model was also offered as a cluster node using Ethernet and Infiniband interconnects.
 
  
 
== System Architecture ==
 
== System Architecture ==
  
 +
The Altix 350 implements the IP57 processor, but follows the IP41 architecture from the earlier [[Altix 3000]] systems. A system can include from 1 to 32 processors in up to 16 modules or bricks, with one or two [NUMAlink] routers connecting them. Configurations up to 8 modules can be organized in a ring topology without a router.
  
== CPU Options ==
+
Note that CPU speeds must be matched within a given brick, but bricks containing different speed CPUs can be connected to each other when building or upgrading a system.
  
The Altix 350 was originally offered with 1.x GHz Itanium2 processors, but have been fitted with all subsequent variants up through the 1.6 GHz
+
=== Brick or Module Types ===
  
== Memory Subsystem ==
+
Altix 350 systems can be composed of four different modules.
 +
* Base Module
 +
* CMPX Expansion Module ("Cpu Memory Pci/pci-X")
 +
* CPU Expansion Module
 +
* NUMAlink 4 Router Module (often labeled "NL4R")
 +
These modules typically run on 110/220V AC power. Storage components were also often included in system configurations.
  
 +
Every Altix 350 system must have at least one base module, and any system with more than 8 modules must include at least one router.
  
== I/O Subsystem ==
+
==== ''Base Module'' ====
  
 +
===== CPU Options =====
  
== Add-On Options ==
+
Each Altix 350 Base Module will accommodate one or two Itanium 2 processors. The Altix 350 has been offered with a wide variety of Itanium2 processors. Examples have been seen with everything from early 1.0GHz units through the penultimate Madison parts at 1.6 GHz with 9MB of L3 cache, and several steps in between.
  
 +
{|class="wikitable"
 +
!Clock Speed
 +
!L3 Cache Size
 +
!S-Spec(s)
 +
!Part Number(s)
 +
|-
 +
|1.0 GHz
 +
|1.5 MB
 +
|'''SL754'''; SL67U, SL6P5
 +
|YA80543KC00115M; YK80542KC00115M
 +
|-
 +
|1.3 GHz
 +
|3.0 MB
 +
|'''SL6XD''', SL7SD, SL8CY
 +
|YA80543KC0133M
 +
|-
 +
|1.4 GHz
 +
|1.5, 3.0, or 4.0 MB
 +
|SL76K, SL7FP, SL6XE
 +
|YA80543KC01715M, YA80543KC0173M, YA80543KC0174M
 +
|-
 +
|1.5 GHz
 +
|4.0, 6.0 MB
 +
|'''SL7ED''', SL8CX; '''SL6XF'''
 +
|YA80543KC0214M; YA80543KC0216M
 +
|-
 +
|1.6 GHz
 +
|3.0, 6.0, 9.0 MB
 +
|SL7EC, SL7FQ, SL8CW; SL7EB, SL8CV; SL87H, SL8CU
 +
|YA80543KC0253M; YA80543KC0256M; YA80543KC0259M
 +
|-
 +
|}
  
== Hard Drive Carriers ==
+
All the processors shown above use Socket 611 (PAC611) and run at a bus speed of 400 MHz, and should be interchangeable. A bold S-spec indicates a processor that has been seen in a working Altix 350. For more details about each of these processors, check the listings at [http://www.cpu-world.com/CPUs/Itanium_2/index.html CPU World].
 +
 
 +
Intel's later dual-core [http://en.wikipedia.org/wiki/Montecito_(processor) Montecito] CPUs support the older bus signaling used on Madison CPUs, but at this time it is unconfirmed whether or not SGI ever released firmware support for these CPUs. In any case, such an application would be giving up many advantages of the Montecito design - DDR2 memory, PCIe support, and three times the FSB throughput (in a system designed for Madison).
 +
 
 +
===== Memory Subsystem =====
 +
 
 +
Like other first-generation Altix systems, the Altix 350 uses registered DDR DIMMs with ECC, rated at 266MHz or faster, which are commonly referred to as PC2100, PC2700, etc. Memory is organized in three banks of four DIMMs each, or twelve slots total, providing up to 24GB of RAM total per base module. DIMMs of 512MB, 1GB, and 2GB are officially supported; it is unclear as of this writing if 4GB parts will work.
 +
 
 +
There is an attempt to document suitable DIMM part numbers from different manufacturers in [http://forums.nekochan.net/viewtopic.php?f=3&t=16727949 this forum thread].
 +
 
 +
===== I/O Subsystem =====
 +
 
 +
Each Base module includes the following:
 +
 
 +
* Front panel LCD display and buttons
 +
* Two NUMAlink ports
 +
* L1 USB Type B connector
 +
* L1 DB9 serial console port
 +
 
 +
The Base module provides two 64 bit PCI/PCI-X buses with two slots each, or a total of four PCI/PCI-X slots per module; however one slot is used for the required IO9/IO10 card. These buses can accommodate cards running at 33, 66, 100 or 133MHz, though the bus will run at the speed of the slowest card installed.
 +
 
 +
Base modules can be fitted with an internal slim-line optical drive as well as two 3.5" Ultra160 [[SCSI]] hard drives, or two 3.5" SATA hard drives. The IO9 controller supports Ultra160 SCSI devices and includes an externally-accessible VHDCI port, while the IO10 controller supports SATA devices and includes a high density serial connector. Both controllers include a separate IDE interface to support optical drives.
 +
 
 +
Either IO controller provides one 1000baseT Ethernet interface.
 +
 
 +
The IO9 or IO10 must always be installed in Slot 1, the bottom slot in the base module.
 +
 
 +
====== Hard Drive Carriers ======
  
 
{{:Origin 350 Drive Carriers}}
 
{{:Origin 350 Drive Carriers}}
  
 +
==== ''CMPX Expansion Module'' ====
  
== Operating Systems ==
+
The "Cpu Memory Pci/pci-X" (CMPX) Expansion Module is very similar to the base module.
  
The Altix 350 is supported by both RedHat Advanced Server / Enterprise Linux, and SuSE Enterprise Linux.
+
===== CPU Options =====
  
 +
CPU options for the CMPX module are the same as for the Base module. The CMPX module may be fitted with 0, 1, or 2 CPUs.
 +
 +
===== Memory Options =====
 +
 +
Memory options for the CPMX module are the same as for the Base module. Memory may be omitted from this module according to the product datasheet.
 +
 +
===== I/O Subsystem =====
 +
 +
Each CMPX module includes the following:
 +
 +
* Front panel LCD display and buttons
 +
* Two NUMAlink ports
 +
* L1 USB Type B connector
 +
* L1 DB9 serial console port
 +
 +
Each CMPX modules provide two 64 bit PCI/PCI-X buses with two slots each, or a total of four PCI/PCI-X slots per module. These buses can accommodate cards running at 33, 66, 100 or 133MHz.
 +
 +
Note that the CPMX module does not support internal mass storage options.
 +
 +
==== ''CPU Expansion Module'' ====
 +
 +
The CPU Expansion module is similar in appearance to the Base or CMPX module.
 +
 +
===== CPU Options =====
 +
 +
CPU options for the CPU Expansion module are the same as for the Base module. The CPU Expansion module may be fitted with 1 or 2 CPUs.
 +
 +
===== Memory Options =====
 +
 +
Memory options for the CPU Expansion module are the same as for the Base module. Memory may not be omitted from this module according to the product datasheet.
 +
 +
===== I/O Subsystem =====
 +
 +
Each CPU Expansion module includes the following:
 +
 +
* Front panel LCD display and buttons
 +
* Two NUMAlink ports
 +
* L1 USB Type B connector
 +
* L1 DB9 serial console port
 +
 +
Note that the CPU Expansion module does not support internal mass storage or PCI/PCI-X options.
 +
 +
==== ''Router Module'' ====
 +
 +
The NUMAlink Router allows configurations of three to eight Base Compute Modules. It is a 2U module similar in appearance to the grey 2U Altix 350/3x00 modules with an hexagonal recess around the LCD display on the front bezel.
 +
 +
The Router Module includes the following features:
 +
 +
* Eight NUMAlink ports
 +
* An L1 controller with front-mounted LCD display
 +
* Two NUMAlink ports
 +
* One USB Type A connector for the L1 controller
 +
* One standard DB-9 serial port
 +
* One power supply
 +
 +
Each of the eight NUMAlink ports can be connected to a Base Compute, Compute Expansion,
 +
 +
=== Topologies ===
 +
 +
There are several different ways to organize the modules in an Altix 350 system.
 +
 +
# Simple ring
 +
# Single-plane
 +
# Dual-plane
 +
 +
=== Add-On Options ===
 +
 +
Many PCI/PCI-X cards are available with support for Linux on Itanium.
 +
 +
 +
== Operating System Support ==
 +
 +
=== Linux ===
 +
 +
The Altix family was originally supported by SGI's Advanced Linux Environment, but eventually support was added to stock SuSE Linux Enterprise Server (SLES) or RedHat Enterprise Linux (RHEL) distributions for Itanium. As of mid-2013 SGI seems to only support SLES on any of their Itanium platforms, while both may be supported on their Xeon-based systems depending on model.
 +
 +
Several Altix 350 owners have [http://forums.nekochan.net/viewtopic.php?f=8&t=16718710&start=15#p7342876 reported success] installing Debian Linux, and at least one has successfully built and installed [http://forums.nekochan.net/viewtopic.php?f=10&t=16727765#p7361257 Scientific Linux CERN 5.4 and CentOS 5.x through 5.9].
 +
 +
SGI offers the ProPack and SGI Foundation Software for Linux, which provides system management and performance monitoring tools beyond what ships with the base Linux distributions they support.
 +
 +
== Troubleshooting and Repair ==
 +
 +
Engineers from SGI [http://www.gelato.unsw.edu.au/archives/linux-ia64/0407/10303.html confirmed reports] that a bug in older PROM versions would not work with the Linux 2.6 kernel. PROM version 4.43 is known to be compatible with SLES 11 SP1.
 +
 +
=== Replacement Parts ===
 +
 +
The Altix 350 uses a common Delta Electronics DPS-500EB modular power supply. While the DPS-500EB E variant is most commonly reported by 350 owners, others have successfully substituted DPS-500EB A units. However there is a known failure mode for earlier versions of these PSUs when run 24/7 for extended periods. See [http://forums.nekochan.net/viewtopic.php?f=3&t=16725720 this thread] for details and a fix.
 +
 +
=== Mixed PROM Versions ===
 +
 +
If you have modules with different versions of the PROM connected to each other it could cause problems. In some cases it could even prevent your system from completing the power-on self test successfully and reaching the EFI Shell. Not to worry though, forum member [http://forums.nekochan.net/memberlist.php?mode=viewprofile&u=14157532 rosmaniac] found a solution and shared it in [http://forums.nekochan.net/viewtopic.php?f=14&t=16725868#p7352653 this thread]. This was the basis for [http://forums.nekochan.net/viewtopic.php?f=10&t=16727765 a how-to posting] in the [http://forums.nekochan.net/viewforum.php?f=10 Tips & Tricks] forum.
  
 
== Further Reading ==
 
== Further Reading ==
  
 +
Wiki articles:
 
* SGI [[Altix]]
 
* SGI [[Altix]]
 
* [[SGI Origin 350]]
 
* [[SGI Origin 350]]
Line 43: Line 202:
 
* [[NUMAlink router]]
 
* [[NUMAlink router]]
  
=== External References ===
+
Forum threads:
 +
* [http://forums.nekochan.net/viewtopic.php?t=16725651 Altix 350 PROM update]
 +
* [http://forums.nekochan.net/viewtopic.php?f=10&t=16727765 How to fix an Altix 350 with mixed PROM versions]
 +
* [http://forums.nekochan.net/viewtopic.php?f=8&t=16718710 Installing Debian on the Altix 350]
 +
* [http://forums.nekochan.net/viewtopic.php?f=10&t=16727812 Operating System Options for Altix/Prism Systems]
 +
* [http://forums.nekochan.net/viewtopic.php?f=3&t=16725720 Warning:Origin 350 PSU may fail during sustained powered-on!]
 +
* [http://forums.nekochan.net/viewtopic.php?f=3&t=16727949 Working RAM Part Numbers for 1st-gen Altix Systems]
  
 +
=== External References ===
 +
SGI:
 
* [http://techpubs.sgi.com/library/tpl/cgi-bin/download.cgi?coll=hdwr&db=bks&docnumber=007-4417-002 SGI Altix 350 Quick Start Guide]
 
* [http://techpubs.sgi.com/library/tpl/cgi-bin/download.cgi?coll=hdwr&db=bks&docnumber=007-4417-002 SGI Altix 350 Quick Start Guide]
 
* [http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi/srch9@Altix%20350/hdwr/bks/SGI_EndUser/books/Altix350_UG/sgi_html/index.html SGI Altix 350 System User's Guide]
 
* [http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi/srch9@Altix%20350/hdwr/bks/SGI_EndUser/books/Altix350_UG/sgi_html/index.html SGI Altix 350 System User's Guide]
* [http://en.wikipedia.org/wiki/Itanium Wikipedia page on Itanium]
 
 
* [http://www.sgi.com/products/remarketed/servers/altix350.html SGI remarketing page for Altix 350]
 
* [http://www.sgi.com/products/remarketed/servers/altix350.html SGI remarketing page for Altix 350]
 +
 +
Other:
 +
* [http://www.gelato.unsw.edu.au/archives/linux-ia64/0407/10303.html Early Altix PROMs may not support newer Linux kernels]
 +
* [http://en.wikipedia.org/wiki/Itanium Itanium page at Wikipedia]
 +
* [http://www.cpu-world.com/CPUs/Itanium_2/index.html Socket 611 Itanium 2 CPUs] at CPU-World.com
  
 
=== Press Releases ===
 
=== Press Releases ===

Latest revision as of 13:00, 8 July 2016

Industry Leading...
A single Altix 350 node
Half rack of Altix 350 nodes
A full rack of Altix 350 nodes and two routers

This page captures information specific to the Altix 350 model. Please see the Altix series overview page for information about other models.

SGI Introduced the Altix 350 model at LinuxWorld in New York on January 20th, 2004. This model was a midrange complement to the Altix 3000 line introduced the previous year, and could be configured with as few as one Itanium 2 CPU in a single 2U module or "brick," to as many as 32 CPUs across 16 modules in a full 39U rack. Meanwhile the 350 provides the same single system image and software features found in the larger configurations, and runs the same operating systems and applications. The model was also offered as a cluster node using Ethernet and Infiniband interconnects.


System Architecture

The Altix 350 implements the IP57 processor, but follows the IP41 architecture from the earlier Altix 3000 systems. A system can include from 1 to 32 processors in up to 16 modules or bricks, with one or two [NUMAlink] routers connecting them. Configurations up to 8 modules can be organized in a ring topology without a router.

Note that CPU speeds must be matched within a given brick, but bricks containing different speed CPUs can be connected to each other when building or upgrading a system.

Brick or Module Types

Altix 350 systems can be composed of four different modules.

  • Base Module
  • CMPX Expansion Module ("Cpu Memory Pci/pci-X")
  • CPU Expansion Module
  • NUMAlink 4 Router Module (often labeled "NL4R")

These modules typically run on 110/220V AC power. Storage components were also often included in system configurations.

Every Altix 350 system must have at least one base module, and any system with more than 8 modules must include at least one router.

Base Module

CPU Options

Each Altix 350 Base Module will accommodate one or two Itanium 2 processors. The Altix 350 has been offered with a wide variety of Itanium2 processors. Examples have been seen with everything from early 1.0GHz units through the penultimate Madison parts at 1.6 GHz with 9MB of L3 cache, and several steps in between.

Clock Speed L3 Cache Size S-Spec(s) Part Number(s)
1.0 GHz 1.5 MB SL754; SL67U, SL6P5 YA80543KC00115M; YK80542KC00115M
1.3 GHz 3.0 MB SL6XD, SL7SD, SL8CY YA80543KC0133M
1.4 GHz 1.5, 3.0, or 4.0 MB SL76K, SL7FP, SL6XE YA80543KC01715M, YA80543KC0173M, YA80543KC0174M
1.5 GHz 4.0, 6.0 MB SL7ED, SL8CX; SL6XF YA80543KC0214M; YA80543KC0216M
1.6 GHz 3.0, 6.0, 9.0 MB SL7EC, SL7FQ, SL8CW; SL7EB, SL8CV; SL87H, SL8CU YA80543KC0253M; YA80543KC0256M; YA80543KC0259M

All the processors shown above use Socket 611 (PAC611) and run at a bus speed of 400 MHz, and should be interchangeable. A bold S-spec indicates a processor that has been seen in a working Altix 350. For more details about each of these processors, check the listings at CPU World.

Intel's later dual-core Montecito CPUs support the older bus signaling used on Madison CPUs, but at this time it is unconfirmed whether or not SGI ever released firmware support for these CPUs. In any case, such an application would be giving up many advantages of the Montecito design - DDR2 memory, PCIe support, and three times the FSB throughput (in a system designed for Madison).

Memory Subsystem

Like other first-generation Altix systems, the Altix 350 uses registered DDR DIMMs with ECC, rated at 266MHz or faster, which are commonly referred to as PC2100, PC2700, etc. Memory is organized in three banks of four DIMMs each, or twelve slots total, providing up to 24GB of RAM total per base module. DIMMs of 512MB, 1GB, and 2GB are officially supported; it is unclear as of this writing if 4GB parts will work.

There is an attempt to document suitable DIMM part numbers from different manufacturers in this forum thread.

I/O Subsystem

Each Base module includes the following:

  • Front panel LCD display and buttons
  • Two NUMAlink ports
  • L1 USB Type B connector
  • L1 DB9 serial console port

The Base module provides two 64 bit PCI/PCI-X buses with two slots each, or a total of four PCI/PCI-X slots per module; however one slot is used for the required IO9/IO10 card. These buses can accommodate cards running at 33, 66, 100 or 133MHz, though the bus will run at the speed of the slowest card installed.

Base modules can be fitted with an internal slim-line optical drive as well as two 3.5" Ultra160 SCSI hard drives, or two 3.5" SATA hard drives. The IO9 controller supports Ultra160 SCSI devices and includes an externally-accessible VHDCI port, while the IO10 controller supports SATA devices and includes a high density serial connector. Both controllers include a separate IDE interface to support optical drives.

Either IO controller provides one 1000baseT Ethernet interface.

The IO9 or IO10 must always be installed in Slot 1, the bottom slot in the base module.

Hard Drive Carriers
Intel-sr2300-drive-sled.jpg

The Origin/Onyx 350, Prism, Altix 350, and Altix 450/4x00 all use the same drive carriers. These are standard parts used by a number of manufacturers, notably Intel's SC5200, SRSH4, SR1300/2300, and SR1400/2400 and Sun's v60x and v65x servers.

The units carry "Assy A65278-00x" on a sticker, where the "-00x" may end in any digit though "-005" seems common. This assembly number can be useful when searching for parts in the Internet. According to SR1300/SR2300 support documents at intel.com, the Intel accessory part number for the drive carrier is FXX2DRVCARBLK, UPC code "7 35858 14621 0" and "MM #" 835853. However note that there is some indication that Intel may recycle these accessory part numbers in the FXX- form between different models.

As of early 2011 these drive sleds are commonly available on eBay for US$10.

Forum Threads About Origin 350 Drive Carriers

CMPX Expansion Module

The "Cpu Memory Pci/pci-X" (CMPX) Expansion Module is very similar to the base module.

CPU Options

CPU options for the CMPX module are the same as for the Base module. The CMPX module may be fitted with 0, 1, or 2 CPUs.

Memory Options

Memory options for the CPMX module are the same as for the Base module. Memory may be omitted from this module according to the product datasheet.

I/O Subsystem

Each CMPX module includes the following:

  • Front panel LCD display and buttons
  • Two NUMAlink ports
  • L1 USB Type B connector
  • L1 DB9 serial console port

Each CMPX modules provide two 64 bit PCI/PCI-X buses with two slots each, or a total of four PCI/PCI-X slots per module. These buses can accommodate cards running at 33, 66, 100 or 133MHz.

Note that the CPMX module does not support internal mass storage options.

CPU Expansion Module

The CPU Expansion module is similar in appearance to the Base or CMPX module.

CPU Options

CPU options for the CPU Expansion module are the same as for the Base module. The CPU Expansion module may be fitted with 1 or 2 CPUs.

Memory Options

Memory options for the CPU Expansion module are the same as for the Base module. Memory may not be omitted from this module according to the product datasheet.

I/O Subsystem

Each CPU Expansion module includes the following:

  • Front panel LCD display and buttons
  • Two NUMAlink ports
  • L1 USB Type B connector
  • L1 DB9 serial console port

Note that the CPU Expansion module does not support internal mass storage or PCI/PCI-X options.

Router Module

The NUMAlink Router allows configurations of three to eight Base Compute Modules. It is a 2U module similar in appearance to the grey 2U Altix 350/3x00 modules with an hexagonal recess around the LCD display on the front bezel.

The Router Module includes the following features:

  • Eight NUMAlink ports
  • An L1 controller with front-mounted LCD display
  • Two NUMAlink ports
  • One USB Type A connector for the L1 controller
  • One standard DB-9 serial port
  • One power supply

Each of the eight NUMAlink ports can be connected to a Base Compute, Compute Expansion,

Topologies

There are several different ways to organize the modules in an Altix 350 system.

  1. Simple ring
  2. Single-plane
  3. Dual-plane

Add-On Options

Many PCI/PCI-X cards are available with support for Linux on Itanium.


Operating System Support

Linux

The Altix family was originally supported by SGI's Advanced Linux Environment, but eventually support was added to stock SuSE Linux Enterprise Server (SLES) or RedHat Enterprise Linux (RHEL) distributions for Itanium. As of mid-2013 SGI seems to only support SLES on any of their Itanium platforms, while both may be supported on their Xeon-based systems depending on model.

Several Altix 350 owners have reported success installing Debian Linux, and at least one has successfully built and installed Scientific Linux CERN 5.4 and CentOS 5.x through 5.9.

SGI offers the ProPack and SGI Foundation Software for Linux, which provides system management and performance monitoring tools beyond what ships with the base Linux distributions they support.

Troubleshooting and Repair

Engineers from SGI confirmed reports that a bug in older PROM versions would not work with the Linux 2.6 kernel. PROM version 4.43 is known to be compatible with SLES 11 SP1.

Replacement Parts

The Altix 350 uses a common Delta Electronics DPS-500EB modular power supply. While the DPS-500EB E variant is most commonly reported by 350 owners, others have successfully substituted DPS-500EB A units. However there is a known failure mode for earlier versions of these PSUs when run 24/7 for extended periods. See this thread for details and a fix.

Mixed PROM Versions

If you have modules with different versions of the PROM connected to each other it could cause problems. In some cases it could even prevent your system from completing the power-on self test successfully and reaching the EFI Shell. Not to worry though, forum member rosmaniac found a solution and shared it in this thread. This was the basis for a how-to posting in the Tips & Tricks forum.

Further Reading

Wiki articles:

Forum threads:

External References

SGI:

Other:

Press Releases