The r st step in developing a parallel algorithm is to decompose the problem into tasks that can be executed concurrently A given problem may be docomposed into tasks in many different ways. Tasks may be of same, different, or even interminate sizes. A decomposition can be illustrated in the form of a directed

Constructing a Parallel Algorithm •identify portions of work that can be performed concurrently •map concurrent portions of work onto multiple processes running in parallel •distribute a program’s input, output, and intermediate data •manage accesses to shared data: avoid conflicts •synchronize the processes at stages of the

Typical steps for constructing a parallel algorithm — identify what pieces of work can be performed concurrently — partition and map work onto independent processors

Our goal today is to primarily discuss how to develop such parallel formulations. Of course, there will always be examples of “parallel algorithms” that were not derived from serial algorithms. Maximize concurrency and reduce overheads due to parallelization! Maximize potential speedup! Tasks can be of different size. Finding concurrent tasks...

Frequently used patterns for parallel applications: o Single Program Multiple Data - SPMD o Embarrassingly Parallel o Master / Slave o Work Pool o Divide and Conquer o Pipeline o Competition

Algorithms in which several operations may be executed simultaneously are referred to as parallel algorithms. In general, a parallel algorithm can be defined as a set of processes or tasks that may be executed simultaneously and may communicate with each other in order to solve a …

basic principles in MIMD algorithm design is to analyze the computations to be performed and determine the parallelism, meaning the dependency graph of the computation.

Parallel Algorithm Models • Data parallel – Each task performs similar operations on different data – Typically statically map tasks to processes • Task graph – Use task dependency graph to promote locality or reduce interactions • Master-slave – One or more master processes generating tasks – Allocate tasks to slave processes

Last two lectures: Algorithms and Concurrency •Introduction to Parallel Algorithms –Tasks and decomposition –Processes and mapping •Decomposition Techniques –Recursive decomposition (divide-conquer) –Data decomposition (input, output, input+output, intermediate) •Terms and …

Design Techniques in Parallel Algorithms - Explore various design techniques for parallel algorithms, including divide and conquer, dynamic programming, and more to improve computational efficiency.