CS 8803
Differentiable and Probabilistic Programming Languages
Spring 2025
Announcements:
  • [1/6] Send your preference of 5 papers to present to the instructor by 1/8 10pm ET.
  • [1/6] Welcome to CS8803!

    • Instructor:
    Jacob Laurel,
    Office: 2332 Klaus
    Office Hours: Mon/Wed 4:00 - 5:00 PM
    Email: jlaurel6@gatech.edu

    Lectures:
    Time: Mon/Wed 5:00 - 6:15 PM
    Location: Ford Environmental Sci & Tech L1175

    Course Summary:
    CS 8803 is a graduate course on both Differentiable and Probabilistic Programming Languages. We will study how mathematical ideas of differentiation and statistical inference can be automated at the level of the programming language. We will then explore how one can statically analyze these programming languages to reason about the mathematical properties satisfied by the differentiable or probabilistic program. We will additionally study how these techniques can be applied towards problems in a variety of research areas such as machine learning, graphics, and scientific computing. This course also strives to study how these ideas cut across the full computational stack, hence we will cover papers ranging from high-level algorithms all the way down to the architectural level. Students are expected to attend lecture, ask questions, present papers and complete a course project. The full syllabus can be found here.  

    References:
    • Differentiable Programming and Automatic Differentiation
      • Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation by A. Griewank and A. Walther, SIAM, 2008.
    • Probabilistic Programming and Bayesian Inference
      • The Design and Implementation of Probabilistic Programming Languages by N. D. Goodman and A. Stuhlmüller, Online book
      • An Introduction to Probabilistic Programming by J. Van de Meent, B. Paige, H. Yang, and F. Wood, Online book

    Assignments:

    Grading:
    Grading breakdown:

    Schedule:

    Date Topic Paper Presenter and Optional Reading
    1/6 Introduction
    1/8 Introduction to Probabilistic Programming
    1/13 MCMC and Rejection Sampling
    • Jacob
    1/15 Program Analysis for Probabilistic Programs
    • Jacob
    1/20 MLK Day - No Class
    1/22 Testing Probabilistic Programs
    1/27 Verification of Probabilistic Programs
    • Chengqian
    1/29 Program Analysis of Probabilistic Programs
    • Kevin
    • Reading:
    2/3 Probabilistic Programming and Graphics
    2/5 Embedded Systems Probabilistic Programming
    • Jacob
    2/10 Probabilistic Programming and HCI
    • Shih-Hsiang
    2/12 Probabilistic Programming and Scientific Computing
    • Shih-Hsiang, Chengqian
    2/17 Introduction to Automatic Differentiation: Forward and Reverse Mode
    • Jacob
    2/19 Gradient Based Probabilistic Inference
    2/24 Higher-Order Automatic Differentiation
    2/26 Non-Differentiability in AD
    3/3 Checkpointing in AD
    • Jianxing
    3/5 Project Proposal Presentations
    • Alan, Steven, Shih-Hsiang, Chengqian, Elton, Zoya & Srikar
    • N/A
    3/10 Project Proposal Presentations
    • Ethan, Jianxing, Alec
    • N/A
    3/12 AD for Quantum Computing
    • Ethan, Zoya
    3/17 Spring Break - No Class
    • Spring Break - No Class
    • Spring Break - No Class
    3/19 Spring Break - No Class
    • Spring Break - No Class
    • Spring Break - No Class
    3/24 What can possibly go wrong?
    • Elton, Jacob
    3/26 AD for Hamiltonian MCMC
    • Alec
    3/31 Compilers for AD
    • Alan
    4/2 AD and Parallel Computing
    • Srikar
    4/7 AD for Graphics
    • Srikar, Alec
    4/9 Software Testing of AD
    • Huizhong, Jianxing
    4/14 AD for Explainable ML
    • Zoya
    4/16 Final Presentations
    • Ethan, Shih-Hsiang, Alec, Chengqian, Huizhong
    4/21 Final Presentation
    • Elton, Zoya, Srikar, Jianxing, Alan

    Misc:
    If you have any concern, please send an email to the instructor. This webpage template was taken from Qirun Zhang.