It’s Parallel Programming 101 • Why can’t we advance from here? • Almost all of the patterns discussed are either intuitive, or covered in introductory courses. • Do these patterns capture most of parallel programs today? • Can these patterns be used to exploit full parallelism?

Parallel Design Patterns¶ There are multiple levels of parallel design patterns that can be applied to a program. At the highest level, algorithmic strategy patterns are strategies for decomposing a problem in its most abstract form.

Examine the catalogs of parallel design patterns for structures which match the dependencies in your application. Use the templates and examples for the appropriate patterns to structure and implement your code.

Structure the UEs (units of execution) of a parallel computation, for better manageability and easier integration with the core computation.

Each pattern has a name, providing a vocabulary for discussing solutions Helps with software reusability, malleability, and modularity Written in prescribed format to allow the reader to quickly understand the solution and its context Otherwise, too difficult for programmers, and software will not fully exploit parallel hardware

There are a number of parallel code patterns that are closely related to the system or hardware that a program is being written for and the software library used to enable parallelism, or concurrent execution.

• Order task pattern: identifying order constraints among task groups. – Control dependency: Find the task group that creates it – Data dependency: temporal order for producer and consumer relationship

Control Pattern: Fork-Join •Forks control flow into multiple parallel flows and joins later •OpenMP’s parallel construct •Cilk Plus-style spawn and sync •Is a join and a barrier same? CS 698L Swarnendu Biswas

By introducing a structured workflow a problem is converted into an efficient en effective parallel program. This conversion contains four stages: Finding Concurrency, Algorithm Structures, Supporting Structures and Implementation Mechanisms.

Patterns for Parallel Programming. Which decomposition more natural to start with, data or tasks? Task decomposition with 3 CPUs. Is the design good enough?