EXC: Extreme Computing

Welcome to Extreme Computing

Learning Outcomes

On successful completion of this course, you should be able to: 

1. demonstrate knowledge of the need for extreme computing by providing motivating examples of the scale of problems only computing at an extreme scale can solve (e.g., problems motivated by the use of large datasets and complex computation).

2. demonstrate knowledge of the problems associated with computing at an extreme scale, such as the need for multi-tier systems and programming models.

3. demonstrate knowledge of the infrastructure necessary for computing at an extreme scale through enumerating different file system designs, virtualisation techniques, replication, fault-tolerance paradigms, and alternative system designs.

4. demonstrate knowledge of data structures that can be used to efficiently process large datasets; and cluster-based algorithms for data processing, machine learning, and low latency processing through the use of distributed programming paradigms.

5. demonstrate knowledge of large-scale distributed deep learning systems for the purposes of solving machine learning problems of extreme scale.

The course is to be conceptually split into three main areas, with each area not necessarily accounting for an equal portion of the syllabus. The three areas and the material covered in each area are as follows:

* Background: Motivation for new computing paradigms; introduction and differences between cloud and cluster computing; scaling and performance.

* Infrastructure: Distributed file systems; multi-tier systems, virtualisation; replication; fault tolerance; concurrent programming; web services.

* Data structures and algorithms: decentralised data structures; programming frameworks; design and implementation of Map/Reduce programs; dealing with massive amounts of data; case studies from natural language processing, data processing, machine and deep learning; and computation over infinite streams.

The course will also deal with the legal, social, ethical, and professional issues involved in remotely storing data in cloud deployments and will also deal with potential solutions to these problems