Parallel and Distributed Computing

Learning Outcomes

Subject Specific Intellectual and Research Skills

Having successfully completed this module you will be able to:

• Identify data races in shared data structures and protect them using threading primitives
• Analyse programs to identify opportunities for introducing concurrency and parallelism
• Decompose a high-level software system into loosely coupled portions

Knowledge and Understanding

Having successfully completed this module, you will be able to demonstrate knowledge and understanding of:

• Explain the strengths and weaknesses of packet oriented and connection oriented systems
• Describe common internet networking protocols
• Analytically provide upper bounds on acceleration due to parallelism

Transferable and Generic Skills

Having successfully completed this module you will be able to:

• Explain how network encryption can improve system security
• Describe the pros and cons of distributed systems for system reliability
• Identify the environmental impact of parallel computing systems

Subject Specific Practical Skills

Having successfully completed this module you will be able to:

• Use standard networking protocols to communicate between software on two different machines
• Use GPU parallelism to accelerate a simple iteration space
• Use multi-threading primitives to parallelise a shared-memory program

Networking concepts:

• Connection vs connectionless approaches
• Streams vs routed packets
• Reliable/unreliable delivery
• Addresses and naming

Practical networking:

• IP networks (main protocols and services, _not_ the deep implementation)
• Sockets
• Clients and servers
• HW to SW channels

Parallel programming concepts:

• Multiple threads
• Race conditions and atomics
• Parallel computing using iteration spaces
• Introduction to GPUs

Distributed computing:

• Combining networks and parallel processing
• Load balancing
• MPI
• Map-reduce
• Efficiency and power consumption of distributed compute

• Lectures
• Guided self-study
• Labs as part of the AICE Lab Programme which will cover practical aspects

Summative

This is how we’ll formally assess what you have learned in this module.