Parallel Programming for Scientific Computing
PhD-level course, 5 hp
Overview: The course focuses on providing knowledge on and practical experience of using parallel programming models for implementation of computational methods. We will cover a set of different models, exposing a span of paradigms for parallel programming and providing a background for performing parallel programming in different PhD research projects at TDB. The course will include hands-on experiments and the examination is performed by solving assignments.
Prerequisites: It is assumed that the students have basic knowledge and experience of standard parallel programming techniques (MPI, OpenMP), at a level corresponding to what is provided by the undergraduate course Programming of Parallel Computers, 10 hp. This course is given in the first part of spring 2014 and students with insufficient (or old) background in these techniques can take part in selected lectures covering them.
Format: The course starts with two introductory lectures where different parallel paradigms are described and exemplified and the student groups are formed. The main part of the course consists of events where each programming model/tool is presented in a short lecture by a "champion", followed by hand-on work by the students (assisted by the champions) and a seminar where the results are presented. The assignments can be done individually or in groups of two. The course will end with an overview lecture on current development of parallel computers and implications for parallel programming. Also, a summary of the parallel computing landscape in Sweden and Europe will be presented, together with information on how different types of systems can be accessed.
In the assignments the emphasis is on analysis of
- Locality - how the locality is explored
- Communication - how the communication is handled
- Threads management - how the threads are handled
The following models/languages will be covered in the course (champions also listed):
- CUDA (Dimitar)
- OpenCL (Dimitar)
- Pthreads (Marcus)
- UPC (Jarmo)
- Cilk (Marcus)
- Chunks and Tasks (Elias, to be confirmed)
- SuperGlue (Marcus)
- Matlab parallel toolbox (Jarmo)
Examination: To pass the course, each student must complete the assignments for at least two programming tools. A student-tool mapping will be set up at the second lecture. An assignment is completed by presenting the results orally at one of the student seminars and by writing a short report. Also, participation in the lectures and student seminars is needed (of course, there might be special cases where some student may have to be somewhere else).
Assignments: The report should contains performance analysis of vector-vector updates (x=x + alpha y); dot product (alpha=x^T y); dense matrix-matrix multiplication; sparse matrix-vector multiplication; special route/algorithm (for Cilk, SuperGlue, Chunks and Tasks).
Teachers: Dimitar Lukarski (responsible for the course), Marcus Holm, Jarmo Rantakokko, Sverker Holmgren
Students and assignments/groups:
|date and time||room||topics||teachers|
|January 21, 13-15||2345||Introduction||Dimitar|
|January 22, 10-12||2345||Introduction cont., forming of groups||Sverker|
|January 24, 10-12||1245||Intro for Cuda and OpenCL||Dimitar|
|January 27, 10-12||1212||Intro for pthreads and UPC||Marcus, Jarmo|
|January 31, 10-12||2245||Intro for cilk and SuperGlue||Marcus|
|February 3, 13-15||2345||Intro for Chunks and Tasks and parallel Matlab||Elias(?) and Jarmo|
|March 4, 13-17||1213||Student seminar 1: Cuda, OpenCL, pthreads and UPC|
|March 11, 13-17||1213||Student seminar 2: cilk, SuperGlue, Chunks and Tasks, parallel Matlab|
|March 13, 13-15||1213||Summary lecture, deadline for student reports||Sverker|