PCIe? USB? Sorting Out Two COM / SFF Design Decisions
Originally appeared in RTC magazine
Modern Computer-on-Module (COM) standards include multiple buses, either in a single connector or among several. Given these COMs’ plethora of bus interfacing options, which should your next design use?
Figure 1 – USB vs. PCI Express design variants
Among all the bus types available in most COM specifications, only USB and PCI Express (PCIe) are well supported, high-level buses, with plug-and-play and excellent interoperability. A PCIe or USB design guarantees it will work with any vendor’s COM solution. Additionally, all of the current and up-and-coming COMs support both of these buses, letting you reuse your schematic and produce the same device for complementary—or competing—COM standards. Having the facts will make it easier to choose which bus to use in your next embedded peripheral design.
PCIe and USB are each fully supplied with operating system level interfaces, both built into the OS and available from a variety of third parties. The buses are mature and well documented, down to the nitty gritty details of implementation for new peripherals. However, there remain real differences between them that will help you choose which bus is right for your new embedded peripheral.
Latency Provides the Most Contrast between USB and PCI Express
A single USB transaction takes at least 125 microseconds, and empirically usually consumes 250 microseconds, even in Linux. 125 microseconds is significant in computer-based systems, especially in the embedded world where human input is often a smaller portion of the system’s total operation. This means USB takes longer to issue control events, like setting up a mode of operation, or polling for a status change.
PCIe, on the other hand, can complete a single transaction in anywhere from a fraction of a microsecond to around two microseconds, depending on the system architecture. This low latency allows more control requests to be issued, or better timing to be achieved even when using relatively few requests. The difference in latency effectively means USB is less “real-time” than PCI Express. Many applications don’t need microsecond latencies, but only PCIe can deliver it when necessary.
Do not confuse latency with bandwidth however. Both buses have real-world data rates in excess of 40 Mbyte/s. PCIe, however, can achieve speeds at least an order of magnitude greater than USB, theoretically as much as 250 Mbyte/s for a single PCIe lane. PCIe lanes can also be grouped (used in parallel.) You’re probably familiar with the 16x PCIe slots normally used for video cards. Grouping PCIe lanes can achieve a near linear increase in bandwidth while retaining the same low latency.
PCIe, then, is both less latent and higher-bandwidth than USB even can be. So why should you even consider USB?
USB is easy and inexpensive to design. USB Bus interface chips are simple, relatively low pin-count devices, allowing fewer layers to be used in the design. USB 1.1 even supports two-layer boards! With only one signal pair to route, the designs are truly simple.
Many USB Bus peripheral interface chips include a microcontroller. These chips provide sub-processors to handle the overhead of USB communication, and the busywork associated with things like streaming to an IDE device, or any FIFO you attach. This microcontroller implements the relatively high-level communication protocol you design. This lets you create designs quickly, at a low cost, and achieve incredible flexibility by modifying firmware instead of hardware. This firmware can be updated over the USB connection “in-system,” usually without the customers even noticing.
Additionally, the USB interface chips are very low cost and have relatively few support chips required. This keeps the unit bill of materials (BoM) costs low, which keeps production costs down. PCI Express, on the other hand, is an expensive, involved, very high pin-count design. Dozens of high-speed differential signals may need to be routed, compared to a single pair for USB.
PCIe designs also require more support components: often 3.3, 2.5 and 1.8 volt references are needed. Also, active PCIe signals are so fast (up to 8 GHz) that most silicon can’t accept the serial stream directly, and it must be processed by a special chip such as a PHY. The output of this device must be fed to an FPGA or other silicon to interpret.
The PHY, the FPGA and the nonvolatile storage used to load the program for all the devices on board are often only available as BGA packaged devices. BGA requires advanced solder facilities, reduces serviceability, and usually requires at least six-layer designs, greatly increasing cost across the entire design and production cycle.
A single FPGA-based PCI Express design may have three to five times the parts cost for the bus interface alone—completely excluding your circuitry! When you factor in the increased PCB, assembly, and test costs, PCIe is truly an expensive proposition compared to USB. Even if you use a COTS PCIe-to-local bus bridge chip instead of FPGAs (greatly reducing your device’s flexibility), design and parts costs exceeds USB by a significant margin.
Score one for USB for being easier and less expensive to design, and for its lower parts cost throughout.
PCIe does bring some more ammunition to the battle: backward compatibility. PCIe devices can be compatible with existing “old-school” PCI or PCI-X devices. This compatibility is obviously not in the bus connector; they are radically different. Rather, the lowest level software that communicates with memory and registers on the peripheral is easy to make 99% compatible with any existing PCI device. You can usually reach 100% compatibility if you stretch a little. This low-level software communicates with the hardware through the “register map” and plug-and-play subsystems, and very few PCI designs are cross-compatible at this level.
By designing your new peripheral to use the same register map as an existing PCI board, you can offer a low-cost migration path for customers dealing with the PCI end-of-life cycle; only the peripheral need change, the mission-critical software can remain identical despite the hardware upgrade. Your customers moving from a PC-104+-based embedded system can upgrade to a Pico-ITXe system using Pico-IO peripherals—with radically different size and bus connectors—and neither you nor they should need to modify the application or driver software they are using.
USB also provides some compatibility, but of a very different nature. USB devices have high-level interfaces; unlike PCI and PCIe devices with complex register maps and memory locations, USB is treated more as a collection of control functionality combined with a few high-bandwidth streaming pipes. Instead of issuing a dozen or more assembly language OUT DX,AX instructions, you might issue a single usb_control_request().
This means USB is definitely not low-level compatible with any existing PCI device.
Additionally, USB was—and is—designed with forward compatibility in mind: USB 2.0 controllers support USB 1.1 peripherals, and USB 1.1 hubs support USB 2.0 devices. You can even plug a USB 2.0 device into a USB 3.0 hub without losing any functionality! You certainly can’t plug a PCI device into a PCIe slot—if you could there wouldn’t be an end-of-life problem in the first place—which is really a great selling point for USB: USB devices aren’t likely to become obsolete simply due to bus changes.
USB is very easy to future-proof; by designing a useful and flexible set of interface functions, all of your I/O designs can reuse the same high-level application or API, regardless of how many times you redesign the peripheral, or how many bits it handles. The high level application talks to a fairly high level control-request interface, and this control-request interface is shared among all of your USB devices.
Whether the forward-looking compatibility of USB or the backward compatibility of PCIe is more important for your design depends on the nature of your requirement. You should consider PCIe as your first choice if you are replacing a PCI device your customers use, especially if you don’t have the resources to modify your software. USB, though, will prevent this compatibility cycle from being an issue the next time you upgrade your system, and allows much greater compatibility within a product generation due to the higher-level nature of the interface.
So, the verdict is PCIe for compatibility with existing PCI products as summarized in Figure 1; USB for compatibility with multiple products in a single generation, and with products going forward. This multitude of pros and cons sums into several easy choices:
USB vs. PCI Express design variants
- If you must have the speed of PCIe, your decision is easy, choose PCIe.
- If you cannot modify the source code for an existing PCI-based embedded system or the customer’s software applications, or the expense of doing so is prohibitive, choose PCIe.
- If cost of design and production is a primary concern, USB rules the day.
- If your embedded device needs to support flexible peripheral options, USB virtually eliminates costs associated with swapping out I/O devices.
Designers must understand and weigh the benefits of each bus type. Cost, effort, bandwidth, latency and compatibility are different factors to consider, and depend on specific application needs. New form factors are constantly being created, and ACCES will continue to develop products across the many bus types to target virtually every application. Explore the possibilities!