Berkeley Lab Scientific Computing Seminar

Date:

Thursday, September 21, 2006

Time:

1:00pm-2:00pm

Location:

Building 50F-1647

Seminar Speaker:

Yijian Wang
Department of Electrical and Computer Engineering
Northeastern University
http://www.ece.neu.edu/students/yiwang/

Title:

Accelerating I/O-Bound Parallel Scientific Applications

Abstract:

In the area of high performance parallel computation, there is a growing need to process very large, complex datasets. When executing these workloads on cluster-based systems, performance cannot scale by simply increasing the number of compute nodes. To effectively exploit parallel resources, we need to parallelize file I/O. The potential impact of exploiting parallel I/O grows as the gap between CPU and disk speeds continues to increase. While parallel I/O middleware systems (e.g., MPI I/O) provide users with environments where large datasets can be shared among multiple distributed processes, the performance of I/O-bound applications depends heavily on how the data is accessed and where the data is physically located on disk. I/O operations need to be parallelized both at the application level and at the disk level.

In this talk, we present an I/O partitioning scheme that utilizes both local and centralized I/O storage within a cluster to improve I/O parallelism and scalability. We characterize file access patterns by studying I/O profiles. We propose and evaluate a new file-partitioning algorithm to optimize data layout in cluster I/O subsystems. We also present a workload balancing mechanism that targets on heterogeneous clusters and tunes workload assignments to adapt the underlying storage devices. By migrating I/O workloads from heavily loaded nodes to lightly loaded nodes, we are able to balance I/O workloads and achieve higher parallel I/O throughput. Experimental results show that by utilizing our optimization approaches, I/O performance can be improved significantly.

Sponsor of Seminar:

David Skinner

Scientific Computing

Contact Esmond G. Ng EGNg@lbl.gov