The lectures consist of basic courses and advanced courses.
In basic lectures, we aimed to help learners completely understand the computer system architecture that supports deep learning, and learn the system design under the full life cycle of deep learning through practical problems.
In advanced lectures, we introduced cutting-edge systems and artificial intelligence research work, including AI for Systems and Systems for AI, to help learners better find and define meaningful research questions.
| Course No. | Lecture Name | Remarks | Download |
| 1 | Introduction | Overview and system/AI basics | PPT Show |
| 2 | System perspective of Systems for AI | Systems for AI: a historic view; Fundamentals of neural networks; Fundamentals of Systems for AI | PPT Show |
| 3 | Computation frameworks for DNN | Backprop and AD, Tensor, DAG, Execution graph. Papers and systems: PyTorch, TensorFlow |
PPT Show |
| 4 | Computer architecture for Matrix computation | Matrix computation, CPU/SIMD, GPGPU, ASIC/TPU Papers and systems: Blas, TPU |
PPT Show |
| 5 | Distributed training algorithms | Data parallelism, model parallelism, distributed SGD Papers and systems: PipeDream |
PPT Show |
| 6 | Distributed training systems | MPI, parameter servers, all-reduce, RDMA Papers and systems: Horovod |
PPT Show |
| 7 | Scheduling and resource management system | Running dnn job on cluster: container, resource allocation, scheduling Papers and systems: Kubeflow, OpenPAI,Gandiva, HiveD |
PPT Show |
| 8 | Inference systems | Efficiency, latency, throughput, and deployment Papers and systems: TensorRT, TensorflowLite, ONNX |
PPT Show |
| Course No. | Course Name | Remarks | |
| 9 | Computation graph compilation and optimization | IR, sub-graph pattern match, Matrix multiplication and memory optimization Papers and systems: XLA, MLIR, TVM, NNFusion |
PPT Show |
| 10 | Efficiency via compression and sparsity | Model compression, Sparsity, Pruning | PPT Show |
| 11 | AutoML systems | Hyper parameter tuning, NAS Papers and systems: Hyperband, SMAC, ENAS, AutoKeras, NNI |
PPT Show |
| 12 | Reinforcement learning systems | Theory of RL, systems for RL Papers and systems: AC3, RLlib, AlphaZero |
PPT Show |
| 13 | Security and Privacy | Federated learning, security, privacy Papers and systems: DeepFake |
PPT Show |
| 14 | AI for systems | AI for traditional systems problems, for system algorithms Papers and systems: Learned Indexes, Learned query path |
PPT Show |