Introduction to Parallel Computing: Motivating Parallelism, Scope of Parallel Computing, Parallel

Programming Platforms: Implicit Parallelism: Trends in Microprocessor Architectures, Limitations of

Memory System Performance, Dichotomy of Parallel Computing Platforms, Physical Organization of
Parallel Platforms, Communication Costs in Parallel Machines, Routing Mechanisms for Interconnection
Networks, Impact of Process-Processor Mapping and Mapping Techniques.

1. Describe the Concept of Communication Costs and Process-Processor Mapping in Parallel

Computing.

2. What is the scope of parallel computing in addressing computational

challenges?

3. Discuss the advantages and limitations of implicit parallelism in programming models.

4. Discuss strategies for minimizing communication overhead in parallel computing.

5. Evaluate the role of mapping in optimizing resource utilization and load balancing.

6. Explain the significance of process-processor mapping in parallel computing.

7. Discuss the recent trends observed in microprocessor architectures with respect to implicit

parallelism.

Module :2

Principles of Parallel Algorithm Design: Preliminaries, Decomposition Techniques, Characteristics of

Tasks and Interactions, Mapping Techniques for Load Balancing, Methods for Containing Interaction

Overheads, Parallel Algorithm Models Basic Communication Operations: One-to All Broadcast and All-to-

One Reduction, All to-All Broadcast and Reduction, All-Reduce and Prefix-Sum Operations, Scatter and

Gather, All-to All Personalized Communication, Circular Shift ,Improving the Speed of Some

Communication Operations.

Preliminaries and Parallel Algorithm Design (5 Marks):

Define the key preliminaries in parallel algorithm design as outlined in the book.

Discuss the significance of understanding decomposition techniques in the context of parallel algorithm

design.

Explain how characteristics of tasks and interactions play a role in formulating effective parallel

algorithms.

Decomposition Techniques (5 Marks):

Compare and contrast different decomposition techniques mentioned in the book.

Discuss the advantages and limitations of task decomposition and data decomposition.

Provide examples of real-world problems where each decomposition technique is most suitable.

Load Balancing and Mapping Techniques (5 Marks):

Explain the importance of load balancing in parallel computing.

Discuss mapping techniques for load balancing as presented in the book.

Evaluate the effectiveness of different mapping techniques in optimizing parallel algorithm

performance.

Interaction Overheads (5 Marks):

Define interaction overheads in the context of parallel algorithm design.

Discuss methods outlined in the book for containing interaction overheads.

Provide examples of scenarios where interaction overheads significantly impact algorithm performance.

Parallel Algorithm Models (5 Marks):

Outline the parallel algorithm models discussed in the book.

Discuss the suitability of each model for different types of parallel computing problems.

Explore how parallel algorithm models influence the design and implementation of algorithms.

Basic Communication Operations (5 Marks):

Explain the concept of basic communication operations in parallel computing.

Provide a detailed overview of one-to-all broadcast and all-to-one reduction operations.

Discuss the practical significance of these basic communication operations in parallel algorithms.

All-to-All Communication Operations (5 Marks):

Explore the characteristics and applications of all-to-all broadcast and reduction operations.

Explain the significance of all-reduce and prefix-sum operations in parallel computing.

Discuss scenarios where these operations are essential for efficient algorithm execution.

Scatter and Gather Operations (5 Marks):

Define scatter and gather operations in the context of parallel computing.

Discuss how these operations are utilized in parallel algorithm design.

Provide examples of algorithms that benefit from the use of scatter and gather operations.

All-to-All Personalized Communication (5 Marks):

Explain the concept of all-to-all personalized communication as presented in the book.

Discuss scenarios where this communication operation is particularly useful.

Evaluate the trade-offs and considerations in implementing all-to-all personalized communication.

Circular Shift and Communication Speed Improvement (5 Marks):

Define circular shift as a communication operation in parallel computing.

Discuss methods outlined in the book for improving the speed of certain communication operations.

Explore practical examples where circular shift and speed improvement techniques are applied for

performance gains.

Module -3:

Analytical Modeling of Parallel Programs: Sources of Overhead in Parallel Programs, Performance

Metrics for Parallel Systems, The Effect of Granularity on Performance, Scalability of Parallel Systems.

Minimum Execution Time and Minimum Cost-Optimal Execution Time, Asymptotic Analysis of Parallel

Programs Section 5.7. Other Scalability Metrics, Programming Using the Message-Passing Paradigm:

Principles of Message-Passing Programming, The Building Blocks: Send and Receive Operations, MPI: the
Message Passing Interface, Topologies and Embedding, Overlapping Communication with Computation,

Collective Communication and Computation Operations, Groups and Communicators

Sources of Overhead in Parallel Programs (5 Marks):

Define the sources of overhead in parallel programs, as discussed in the book.

Discuss how each source of overhead contributes to the overall execution time of parallel programs.

Provide examples of scenarios where minimizing specific overheads is critical for optimal performance.

Performance Metrics for Parallel Systems (5 Marks):

Enumerate the performance metrics for parallel systems presented in the book.

Discuss the significance of each metric in evaluating the efficiency of parallel programs.

Explore how different metrics may be prioritized based on the nature of the parallel application.

Effect of Granularity on Performance (5 Marks):

Explain the concept of granularity in parallel computing.

Discuss the effect of granularity on the performance of parallel programs.

Provide examples illustrating the impact of fine and coarse granularity on different types of applications.

Scalability of Parallel Systems (5 Marks):

Define scalability in the context of parallel systems.

Discuss the factors that contribute to or limit the scalability of parallel programs.

Explore methods and strategies outlined in the book for achieving scalable parallel systems.

Minimum Execution Time and Cost-Optimal Execution Time (5 Marks):

Differentiate between minimum execution time and cost-optimal execution time.

Discuss the trade-offs involved in minimizing both execution time and cost in parallel computing.

Provide examples of scenarios where achieving the minimum execution time may not lead to cost

optimality.

Asymptotic Analysis of Parallel Programs (5 Marks):

Explain the concept of asymptotic analysis in the context of parallel programs.

Discuss how asymptotic analysis helps in understanding the scalability of parallel algorithms.

Provide examples where asymptotic analysis is particularly useful in predicting performance trends.

Other Scalability Metrics (5 Marks):

Explore additional scalability metrics beyond those specifically mentioned in the book.

Discuss the importance and relevance of these alternative scalability metrics.

Analyze how considering multiple scalability metrics can offer a more comprehensive performance

evaluation.

Principles of Message-Passing Programming (5 Marks):

Outline the principles of message-passing programming as discussed in the book.

Discuss the advantages and challenges associated with the message-passing paradigm.

Provide examples of scenarios where message-passing programming is preferable over other paradigms.

MPI: The Message Passing Interface (5 Marks):

Explain the significance of MPI in message-passing programming.

Discuss the building blocks of MPI, including send and receive operations.

Evaluate how MPI facilitates efficient communication in parallel programs.

Overlapping Communication with Computation (5 Marks):

Discuss the concept of overlapping communication with computation in parallel programming.

Explore strategies outlined in the book for achieving overlap and minimizing idle time.

Provide examples of algorithms or applications where overlapping communication with computation is

particularly advantageous.
Module-4:

Programming Shared Address Space Platforms: Thread Basics, Why Threads? The POSIX Thread API,

Thread Basics: Creation and Termination, Synchronization Primitives in P threads, Controlling Thread and

Synchronization Attributes, Thread Cancellation, 08 Composite Synchronization Constructs, Tips for

Designing Asynchronous Programs, OpenMP: a Standard for Directive Based Parallel Programming

Dense Matrix Algorithms: Matrix-Vector Multiplication, Matrix - Matrix Multiplication, Solving a System

of Linear Equations Sorting: Issues in Sorting on Parallel Computers, Sorting Networks, Bubble Sort and

its Variants, Quick sort, Bucket and Sample Sort.

Thread Basics and Importance (5 Marks):

Define the basics of threads in the context of parallel programming.

Discuss the significance of using threads and their advantages in shared address space platforms.

Explain scenarios where employing threads is particularly beneficial for parallel programming.

POSIX Thread API (5 Marks):

Provide an overview of the POSIX Thread API for parallel programming.

Discuss the key features and functions offered by the POSIX Thread API.

Explore scenarios where the POSIX Thread API is commonly used in parallel applications.

Thread Creation and Termination (5 Marks):

Explain the fundamental concepts of thread creation and termination.

Discuss considerations and best practices for managing the lifecycle of threads.

Provide examples of scenarios where proper thread creation and termination are critical for program

execution.

Synchronization Primitives in Pthreads (5 Marks):

Outline the synchronization primitives available in Pthreads.

Discuss the purpose and usage of these primitives in ensuring thread safety.

Explore scenarios where specific synchronization primitives are more suitable than others.

Controlling Thread and Synchronization Attributes (5 Marks):

Discuss how thread and synchronization attributes are controlled in parallel programming.

Explain the significance of customizing thread attributes based on application requirements.

Provide examples of scenarios where fine-tuning thread attributes is crucial for optimal performance.

Thread Cancellation (5 Marks):

Define thread cancellation and its role in parallel programming.

Discuss the challenges and considerations associated with thread cancellation.

Provide examples of scenarios where thread cancellation is useful or should be avoided.

Composite Synchronization Constructs (5 Marks):

Explore composite synchronization constructs for managing complex synchronization scenarios.

Discuss how combining synchronization primitives can address specific challenges in parallel

programming.

Provide examples of applications or algorithms that benefit from composite synchronization constructs.

Tips for Designing Asynchronous Programs (5 Marks):

Provide tips for designing asynchronous programs in shared address space platforms.

Discuss common pitfalls and challenges in asynchronous programming.

Explore strategies for achieving efficient and reliable asynchronous parallel programs.

OpenMP: Directive-Based Parallel Programming (5 Marks):

Explain the concept of OpenMP and its role in directive-based parallel programming.

Discuss the key directives and features provided by OpenMP.

Explore scenarios where OpenMP is a suitable choice for parallelizing applications.

Dense Matrix Algorithms and Sorting (5 Marks):

Discuss the challenges and optimizations in dense matrix algorithms, focusing on matrix-vector

multiplication and matrix-matrix multiplication.

Explore issues related to sorting on parallel computers and various sorting algorithms.

Provide examples of applications or domains where dense matrix algorithms and sorting on parallel

computers are commonly applied.

Module-5:

Graph Algorithms: Definitions and Representation, Minimum Spanning Tree: Prim's Algorithm, Single-

Source Shortest Paths: Dijkstra's Algorithm, All-Pairs Shortest Paths, Transitive Closure, Connected

Components, Algorithms for Sparse Graphs, Search Algorithms for Discrete Optimization Problems:

Definitions and Examples, Sequential Search Algorithms, Search Overhead Factor, Parallel Depth-First

Search, Parallel Best-First Search, Speedup, A anomalies in Parallel Search Algorithms.

Graph Algorithms and Representation (5 Marks):

Define graph algorithms and discuss the various representations of graphs.

Explore the advantages and limitations of different graph representations.

Provide examples of scenarios where specific graph representations are more suitable.

Prim's Algorithm for Minimum Spanning Tree (5 Marks):

Explain the concept of Minimum Spanning Tree (MST) in graph theory.

Discuss the principles and steps involved in Prim's algorithm for finding an MST.

Explore scenarios where Prim's algorithm is efficient and suitable for parallelization.

Dijkstra's Algorithm for Single-Source Shortest Paths (5 Marks):

Define the problem of finding single-source shortest paths in a graph.

Discuss the principles and steps involved in Dijkstra's algorithm.

Explore considerations for parallelizing Dijkstra's algorithm in high-performance computing

environments.

All-Pairs Shortest Paths and Transitive Closure (5 Marks):

Discuss the significance of finding all-pairs shortest paths in a graph.

Explain algorithms for computing all-pairs shortest paths and transitive closure.

Explore the computational challenges in parallelizing algorithms for these graph problems.

Connected Components and Algorithms for Sparse Graphs (5 Marks):

Define connected components in the context of graph algorithms.

Discuss algorithms for identifying connected components in graphs.

Explore considerations for parallelizing algorithms for sparse graphs.

Search Algorithms for Discrete Optimization Problems (5 Marks):

Define discrete optimization problems and their significance.

Discuss sequential search algorithms for discrete optimization problems.

Explore the trade-offs and challenges in parallelizing search algorithms for discrete optimization.

Parallel Depth-First Search and Best-First Search (5 Marks):

Explain the principles of parallel depth-first search in graph algorithms.

Discuss the considerations for parallelizing best-first search.

Provide examples of scenarios where parallel depth-first search and best-first search are

advantageous.

Speedup and Anomalies in Parallel Search Algorithms (5 Marks):

Define speedup in the context of parallel algorithms.

Discuss how speedup is measured and evaluated in the context of parallel search algorithms.

Explore anomalies that may occur in parallel search algorithms and strategies for addressing them.