What is Parallel Matlab?

Parallel Matlab

The Parallel Matlab project is a research project that naturally arose from work in developing the Parallel Problems Server. It is a collection of Matlab 5 objects, Matlab m-files and Matlab MEX programs that allow for the mostly transparent integration of Matlab as a front end for the Parallel Problems Server. This collection of programs is known variously as Parallel Matlab, PPMatlab, MITMatlab and Matlab*p. For the sake of consistency, we will refer to it here at Matlab*p.

Why Matlab? The choice of Matlab was influenced by several factors. First, Matlab is the de facto standard for scientific computing. It enjoys wide use in both industry and academia. In the machine learning and vision community, for example, algorithms are often written as Matlab scripts and made freely available. In the scientific computing community, algorithms are often first prototyped in Matlab before being rewritten in highly-optimized form using a language such as Fortran.

P, Transparency and Software Reuse

Matlab*p brings the benefits of interactivity to parallel supercomputing for large problems. The user types standard MATLAB commands on a parallel machine and obtains the speed and memory benefits of parallel computation. Because users have their own MATLAB codes and MATLAB itself is built from scripts, the software reuse benefits of this approach can not be overstated.

One key to the reuse of standard MATLAB software is the simple but innovative P notation. This notation uses Matlab 5's object oriented features as one method of creating PPServer objects automatically. For example, a user need only type a=ones(1000*P,1000) or b=rand(1000,1000*P) to obtain two 1000-by-1000 matrices distributed in parallel. The reader can guess the use of P: it indicates that a is distributed by rows and b by columns.

Within MATLAB, a and b are simply handles to special distributed types that exist on the PPServer. Any further Matlab references to these variables (e.g. via such commands as eig, svd, inv, *, +, -) will be recognized as a call to the PPServer rather than as a traditional MATLAB command.

The primary purpose of P is to indicate a parallel dimension, making it easier for the user to create new distributed objects. There are other more subtle benefits of this approach, however. The reader might gain some insight by considering a few experimental inputs and outputs:

Input: P P*10 P+10 [P 4 5*P]

Output: 1p 10p 11p [1p 4 5p]

Observe that P behaves just like the integer 1. Therefore, P can be used as a natural way of propagating parallelism through the dimension variables rather than the arrays themselves. A little thought and the reader may be convinced that many Matlab functions may now work in parallel simply by calling the functions with a P argument. For example, note the code for the standard Matlab function, hilb(), which computes a Hilbert matrix:

function H=hilb(n)

J=1:n;J=J(ones(n,1),:);
I=J';
E=ones(n,n);
H=E./(I+J-1);

Note that hilb(100) will create the 100-by-100 Hilbert matrix in Matlab, but that hilb(10000*P) will create the 10000-by-10000 Hilbert matrix on the PPServer. This is accomplished by virtue of the fact that P will be propagated via the n parameter.

In practice, we have found that most of Higham's matrix toolbox works in parallel without any modification to the original code. This did not happen by accident. It is possible because we have implemented Matlab's fancy indexing capabilities, provided P, and allowed size(a) to return P-integers. Thus clever choices in P's definition and influence were crucial towards our goal of software reuse.

Resources

We believe that Matlab*p is the only one of the many competing "parallel Matlab" projects that obtains its capability through the combination of 1) object oriented Matlab scripts and 2) a parallel server to distributed processors.

For hardware, any collection of parallel machines using MPI (the standard multi-processor substrate available on various unix platforms) is a viable platform for Matlab*p. Each Matlab*p requires one copy of traditional Matlab.

A number of research and class projects have been implemented with Matlab*p and the applications that we can support grow every day.

function	H=hilb(n)
	J=1:n;J=J(ones(n,1),:); I=J'; E=ones(n,n); H=E./(I+J-1);