Randomized Algorithms

RA: Course Materials

mcorey — Wed, 02 Aug 2023 11:55:37 +0000

RA: Course Materials mcorey Wed, 02/08/2023 - 12:55

Lecture Recordings All lecture recordings should be accessed via Learn; you will need to log in using your EASE account. (Learn provides you with access to any lecture recordings available for this course. You will need to select the "lecture recording" link once, before you can access any direct links to a lecture recording.)

Lectures slides All lecture slides can be found in the subsection Schedule of this Drupal page, inside the corresponding timetable box.

Tutorial sheets and solutions All tutorial sheets and solutions can be found in the subsection Tutorials of this Drupal page, inside the corresponding timetable box.

Past exam papers can be found in this UoE page. Academic years before 22/23 were run by different lecturers and questions style and topics can be different.

License

RA: Course Contacts

mcorey — Wed, 02 Aug 2023 11:50:39 +0000

RA: Course Contacts mcorey Wed, 02/08/2023 - 12:50

Raul Garcia-Patron Sanchez - course lecturer and course organizer
Email: rgarcia3@ed.ac.uk
Raul is member of the Quantum Software Lab. His research is mostly on quantum computation and a bit (mostly in the past) quantum communication. He is also course organized of Introduction to Quantum Computating.

Kousha Etessami - course lecturer
Email: kousha@ed.ac.uk

Kousha is a professor in LFCS, School of Informatics. His research is in theoretical computer science. Specifically, in algorithms and computational complexity theory, algorithmic game theory, analysis of probabilistic systems, Markov decision processes and stochastic games, and logic and automata theory.

Graham Freifeld - course tutor
Email: g.freifeld@sms.ed.ac.uk

Graham is a PhD student in Informatics, working on (randomized) algorithms and computational complexity.

License

RA: Resource List

mcorey — Wed, 02 Aug 2023 11:50:08 +0000

RA: Resource List mcorey Wed, 02/08/2023 - 12:50

All reading referenced in lecture slides are chapters and sections in the main textbook for this course, which is:
M. Mitzenmacher and E. Upfal, Probability and Computing: Randomized Algorithms and Probabilistic Analysis, 2nd edition , Cambridge University Press, 2017.

For background material on basic counting and discrete probability, you can visit Kousha Etessami's lecture notes (lectures 16-18, and lectures 25-29) for the old Discrete Mathematics course.

There are many excellent introductory textbooks on probability theory. We can recommend the following:

Sheldon Ross, A First Course in Probability, 8th Edition, Pearson, 2010. (A good, gentle but rigorous, introduction, including some background on counting.)
G. Grimmett and D. Stirzaker, Probability and Random Processes, 4th edition, Oxford University Press, 2020. (A more advanced text, covering much more, but also much more dense.)

Other reference texts on randomized algorithms, besides the main textbook for this course by Mitzenmacher and Upfal, include: R. Motwani and P. Raghavan, Randomized Algorithms, Cambridge University Press, 1995.

Some reference books specifically on Markov chains include:

J. R. Norris, Markov Chains, Cambridge University Press, 1998.
D. A. Levin and Y. Peres, Markov Chains and Mixing Times, 2nd Edition, American Mathematical Society, 2017.

An excellent classic reference book on the Probabilistic Method is:

N. Alon and J. H. Spencer, The Probabilistic Method, 4th Edition, Wiley, 2016.

License

RA: Assessment

mcorey — Wed, 02 Aug 2023 11:48:51 +0000

RA: Assessment mcorey Wed, 02/08/2023 - 12:48

All assessments should be accessed via Learn; you will need to log in using your EASE account.

License

RA: Tutorials

flittlet — Tue, 18 Jul 2023 14:58:38 +0000

RA: Tutorials flittlet Tue, 18/07/2023 - 15:58

The information below is still from the prior year 2022, so student can make a more informed course selection decision at the beginning of the semester. It will be updated accordingly during the course.

Week	Date	Tutorial#	Tutor	Topic
3	2-Oct-2023	1	Graham Freifeld	Randomized algorithm for checking equality of succinctly represented integers Tutorial sheet 1 . Solutions for tutorial sheet 1.
4	9-Oct-2023	2	Graham Freifeld	Discrete probability and algorithms Tutorial sheet 2 . Solutions for tutorial sheet 2.
5	16-Oct-2023	3	Graham Freifeld	Chernoff bounds and balls and bins Tutorial Sheet 3 . Solutions for tutorial sheet 3.
6	23-Oct-2023	4	Graham Freifeld	The probabilistic method Tutorial Sheet 4 . Solutions for tutorial sheet 4.
7	30-Oct-2023	5	Graham Freifeld	Markov Chains Tutorial Sheet 5 Solution tutorial 5
8	6-Nov-2023	6	Graham Freifeld	Markov Chain Monte Carlo, Metropolis, MCMC Tutorial 6 Solution tutorial 6
9	13-Nov-2023	7	Graham Freifeld	Metropolis, Glauber and Gibbs sampling Tutorial 7 Solution tutorial 7
10	20-Nov-2023	8	Graham Freifeld	Total variation distance and coupling of Markov chains Tutorial 8 Solution tutorial 8

License

RA: Schedule

flittlet — Tue, 18 Jul 2023 14:58:38 +0000

RA: Schedule flittlet Tue, 18/07/2023 - 15:58

Week	Date	Lecture#	Lecturer	Topic	Reading
1	18-Sep-2023	1	Kousha	Introduction: Testing polynomial identities	Chapter 1
1	20-Sep-2023	2	Kousha	Introduction II: verifying Matrix multiplication and Karger's min-cut algorithm	Chapter 1
2	25-Sep-2023	3	Kousha	Discrete Probability	Chapter 1
2	27-Sep-2023	4	Kousha	Discrete Probability II	Chapter 2 and 3
3	02-Oct-2023	5	Kousha	(1/2)-approximation for Max-Cut, de-randomization via conditional expectation	Chapter 2 and 3
3	04-Oct-2023	6	Kousha	Chernoff Bounds and applications I	Chapter 4
4	09-Oct-2023	7	Kousha	Chernoff Bounds and applications II	Chapter 4
4	11-Oct-2023	8	Kousha	Birthday paradox, and Balls in Bins	Chapter 5
5	16-Oct-2023	9	Kousha	The Probabilistic Method I	Chapter 6
5	18-Oct-2023	10	Kousha	The Probabilistic Method II	Chapter 6
6	23-Oct-2023	11	Raul	Markov Chains Basics	Chapter 7
6	25-Oct-2023	12	Raul	Markov Chains II: 2-SAT Randomized Algorithm	Chapter 7
7	30-Oct-2023	13	Raul	Monte Carlo Method and DNF	Chapter 7
7	1-Nov-2023	14	Raul	Markov Chain Monte Carlo and Approximate counting	Chapter 7
8	06-Nov-2023	15	Raul	Metropolis and Glauber	Chapter 10
8	08-Nov-2023	16	Raul	Total variation distance and coupling	Chapter 10
9	13-Nov-2023	17	Raul	Path Coupling	Chapter 11
9	15-Nov-2023	18	Raul	Ising Models and Simulated Annealing	Chapter 11
10	20-Nov-2023	19	Kousha	The Lovasz Local Lemma	Chapter 6
10	22-Nov-2023	20	Kousha/Raul	Revision session Markov chains

License

Randomized Algorithms

flittlet — Tue, 18 Jul 2023 14:58:37 +0000

Randomized Algorithms flittlet Tue, 18/07/2023 - 15:58

Welcome to Randomized Algorithms

The Lecturers for this course are Prof. Kousha Etessami and Dr. Raul Garcia-Patron.

One of the remarkable developments in Computer Science over the past 30 years has been the realization that the ability of computers to use randomness can lead sometimes to algorithms that are more efficient, conceptually simpler and more elegant that their best-known deterministic counterparts. Randomization has by now become a ubiquitous tool in computation, from applied physics to machine learning.

This course will survey several of the most widely used techniques in this context, illustrating them with examples taken from algorithms, such as Monte Carlo or Gibbs sampling, and combinatorics. Our goal is to provide a solid background in the key ideas used in the design and analysis of randomized algorithms and probabilistic processes.

The required textbook for the course is "Probability and Computing: Randomized Algorithms and Probabilistic Analysis" by Mitzenmacher and Upfal.

Learning Outcomes

On successful completion of this course, you should be able to:

Understand and apply fundamental tools in discrete probability (e.g. expectation, concentration inequalities, the probabilistic method, random walks) toward the design and analysis of randomized algorithms.
Understand randomized algorithms for selected combinatorial and graph problems.
Be able to analyze expected running time and error probabilities of randomized algorithms.
Understand the fundamentals of Markov chains and their algorithmic applications.
Apply Monte Carlo methods such as MCMC to some discrete algorithmic problems.

Course Outline

This course is about randomness as a resource in algorithms and computation. The course introduces basic mathematical models and techniques and applies them to the design and analysis of various randomized algorithms. We will also cover a variety of applications of probabilistic ideas and randomization in several areas of computer science.

-Introduction, review of discrete probability, and elementary examples including randomized algorithms for checking identities, matrix multiplication verification, minimum cut in graphs.
-Discrete Random Variables, Moments, Deviations and Tail Inequalities; applications, including the coupon collector problem.
-Chernoff bounds and applications: random sampling and estimation of discrete distributions. The birthday paradox and applications.
-The Probabilistic Method: random graphs and threshold phenomena. Max-cut approximation. Lovasz Local Lemma and application to boolean satisfiability.
-Random Walks and Markov Chains: hitting and cover times; stationary distributions, random walks on undirected graphs.
-The Monte Carlo Method; applications including sampling and approximate counting, the markov chain monte carlo method, the Metropolis algorithm.
-Coupling of Markov Chains, mixing time, and applications, including card shuffling and sampling of graph colourings and independent sets.

License