The difference between multi-core and multi-processing
When discussing the shift to multi-core, I often hear people ask why multi-core, which is relatively new, is so different from multi-processing, which has been with us for decades.
First, let’s start with the basics. Multi-processing simply means putting multiple processors in one system. Symmetric Multi-Processing, or SMP, implies that all of these processors are identical, also known as a homogeneous system. SMP systems have been around in the x86 world for a very long time, and there are software systems that take advantage of SMP well.
From a technical standpoint, the difference between multi-core and SMP is relatively benign. In an SMP system, each processor plugs into a different socket, and multiple processors are connected through some kind of bus. In a multi-core processor, the “core” logic of a processor is replicated multiple times on the same chip. Multiple cores may share data through some on chip logic or shared caches. Multiple cores are presented to applications at the OS level exactly the same way as multiple processors in an SMP system. Furthermore, you can mix the two together, e.g. by having an 8-core system with two processors, each containing four cores.
So, why is programming multi-core considered so much more of a problem than programming SMP systems? It’s not because of some fundamental technical difference between multi-core and SMP. It’s because of the reason why these technologies exist.
SMP systems have always been a good way to gain additional performance without having to scale out to multiple machines for applications that can take advantage of them. For example, web servers that need to process many transactions at once, all of which are mostly independent, can easily take advantage of SMP (and multi-core!) by splitting those transactions into separate threads. This means that, if you needed this additional performance (say, because your site suddenly got much more popular), you could make the choice of adding additional processors to a system to scale up.
The emphasis here, is on the word “choice.” Going SMP is a choice you could make. Five to ten years ago, whether or not you chose to scale up with SMP, you could also rely on the fact that individual processors by themselves were getting faster and faster every year. Software that wasn’t as easily distributed across multiple processors could simply wait for processors to get faster, and software would generally be designed, in terms of new features, to keep up with the performance increases being made on single processors.
This trend has stopped. Single processors are no longer increasing considerably in speed. Clock rates are not going up every year like they used to. This isn’t because processor vendors stopped investing effort into making their processors faster, it’s due to physical barriers relating to power and heat being reached in terms of what can be done to increase the performance of a serially executing processor. Multi-core is the answer to this. Putting twice as many cores on the same processor is far more energy efficient than trying to double clock speeds. So, processors have been “going multi-core”: first two, then four, soon six and eight. Cores will keep increasing, eventually reaching counts of hundreds or thousands per processor.
That brings me to the fundamentally important difference between SMP and multi-core. SMP is a choice, that some applications chose to (and still choose to!) take advantage of. Multi-core is a given, that all applications must react to in order to continue to scale up performance.
For an entirely serial application this will mean big changes now. Software such as ours attempts to make such change easy and get the most out of multi-core, hopefully turning a big problem into a huge opportunity. Even applications that currently scale on SMP systems may be bit by the multi-core shift however. Returning to the example of the web server that can scale across multiple processors or cores by dividing up transactions, consider what happens if each one of those transactions has to do some sort of video transcoding operation. As bandwidth increases and HD video becomes common, resolutions increase to the point where a single processor core may no longer be able to fulfill a single request in time. Traditionally, increased clock speeds and other single-processor advances might have covered this. Without these changes though, there’s only one solution: the video processing part of the application must use multiple cores in parallel. This is just one illustration of how even SMP-capable systems may require significant changes to keep up with the shift to multi-core.
In summary, the technical differences between SMP and multi-core are almost irrelevant. The important difference is their reasons to exist, and the slowdown in performance increases in single cores. This is relevant to any performance-aware application, whether it takes advantage of SMP already or not.
May 30th, 2008 at 11:57 am
I agree with you on the basic idea of your post: today’s performance gains only through multi-core computing. However, the idea of having 100-1000 identical cores in the future seems to be highly improbable. Let’s analyse it: In the year X set in future, let there be a processor with 1000 identical cores. Now let’s replace e.g. 10 of these cores with specialized cores. Why? Because let us suppose that it gives certain applications (call them ‘A’) used in the year X a big speed boost. Other applications now have to run at most at 99% of their original speed [= (1000-10)/1000]. Who cares that those other applications run at 99% speed when applications ‘A’ run let’s say 5% faster? Nobody. Therefore, homogeneous multi-core systems are in fact a transitory stage in the evolution of computing, the final stage being processors and systems with the right mixture of *computing blocks*. Complexity will win.
June 2nd, 2008 at 4:02 pm
Anonymous: Absolutely - I didn’t mean to imply anything at all about heterogeneous vs homogeneous architectures in my posting (apart from pointing out that the “S” in “SMP” implies homogeneous processors at some level). Heterogeneous processor designs are an interesting alternative to homogeneous designs, and we’re likely to see more of them in the future. Exactly what kind of processor architecture will “win out” in the next decade isn’t clear yet.
All of my points apply to heterogeneous computing, maybe even moreso. Software must adapt to multi-core to keep scaling in performance. If heterogeneous cores become commonplace, software will have to adopt to these as well. Our platform already targets heterogeneous designs such as the Cell Broadband Engine and combinations of CPUs and GPUs today, and our aim is to make it possible to write code that scales well “no matter what crazy hardware you throw at it” :) - multi-core, many-core, homogeneous, heterogeneous, you name it.