The new Mac Pro hasn't been released yet, but enough is known to figure out what the architecture looks like. In particular, the decision to use Thunderbolt ports instead of PCIe slots for expansion drives other design decisions that were made. In this post, I'll describe the architecture and how It can be deduced.
Apple set out to build a pro machine with only Thunderbolt for high-speed expansion. Thunderbolt ports on a computer are required to carry both data in the form of PCIe lanes and video as Displayport (Source 1, p. 7). This is ideal for a laptop connecting to, for example, the Apple Thunderbolt display, as a single cable carries video and data. It's not good for a server, where the preference would be for more high-speed I/O, but not as many Displayport connections. The choice of 6 Thunderbolt ports is interesting, as it requires that there be two GPUs to support video on the 6 Thunderbolt ports (plus the HDMI port, for 7 total). So, while Apple may tout the inclusion of 2 GPUs as intentional for GPU-based computation, it is also required if they decided 4 Thunderbolt ports wasn't enough.
The GPUs have been announced as AMD FirePro D300, D500, or D700. Since both the CPU and GPU support PCIe 3.0, we can assume the two GPUs use the 16-lane ports 2 and 3 on the Xeon CPU (Source 3).
The matching system chip for the Xeon E5-1600/2600 v5 is an intel C60x Platform Controller Hub (Source 2), which Apple is almost certainly using. It is connected via the 4-lane PCIe 2.0 port 0 of the CPU. The C60x includes Ethernet, USB, audio, clock, and all other internal I/O connections. It includes an internal PCIe switch, and all internal devices appear as PCIe devices. It also has 8 lanes of configurable PCIe that can connect to other devices, such as the SSD, WiFi and BlueTooth cards.
The choice of Thunderbolt 2 only leaves two possible parts, the intel DSL5520 (2-port) Thunderbolt controller or the 1-port DSL5320 (source 1, p. 11). The limited number of PCIe ports on the Xeon CPU dictate the use of the DSL5520. The DSL5520 can't use PCIe 3.0, so they use a PCIe 2.0-speed connection to the CPU. This makes some sense, because Thunderbolt uses PCIe 2.0 speeds.
With ports 0, 2, and 3 occupied, that leaves port 1 for 2 of the Thunderbolt controllers (Source 3 p. 17). Port 1 can be configured as 1 x8 port, or 2 x4 ports, which are called port 1 and port 1b, which is where two of the Thunderbolt controllers should be connected. The third Thunderbolt controller then must connect to the system chip, sharing bandwidth with all internal I/O.
So, with 6 thunderbolt ports, and other details released by Apple, there is little room for the architecture to deviate from this description. Certainly the HDMI port could be connected to either GPU, and there is some uncertainty about other connections, but this is what makes the most sense given the constraints.
In the next post, I'll go over some of the implication of the design.
Source 1: IDF 2013: Thunderbolt 2 Technology: New Capabilities and Features
Source 2: intel C600 series chipset
Source 3: Xeon E5 v2 datasheet
©2013 David Amis.