CS 7643: Deep Learning

Overview

Deep learning is a sub-field of machine learning that focuses on learning complex, hierarchical feature representations from raw data. The dominant method for achieving this, artificial neural networks, has revolutionized the processing of data (e.g. images, videos, text, and audio) as well as decision-making tasks (e.g. game-playing). Its success has enabled a tremendous amount of practical commercial applications and has had significant impact on society.

In this course, students will learn the fundamental principles, underlying mathematics, and implementation details of deep learning. This includes the concepts and methods used to optimize these highly parameterized models (gradient descent and backpropagation, and more generally computation graphs), the modules that make them up (linear, convolution, and pooling layers, activation functions, etc.), and common neural network architectures (convolutional neural networks, recurrent neural networks, etc.). Applications ranging from computer vision to natural language processing, and decision-making (reinforcement learning) will be demonstrated. Through in-depth programming assignments, students will learn how to implement these fundamental building blocks as well as how to put them together using a popular deep learning library, PyTorch. In the final project, students will apply what they have learned to real-world scenarios by exploring these concepts with a problem that they are passionate about.

CS 7643, originally created at Georgia Tech five years ago, was rebuilt with the support of Facebook for on-campus students in Spring 2020.

Course Goals

• Describe the major differences between deep learning and other types of machine learning algorithms.
• Explain the fundamental methods involved in deep learning, including the underlying optimization concepts (gradient descent and backpropagation), typical modules they consist of, and how they can be combined to solve real-world problems.
• Differentiate between the major types of neural network architectures (multi-layered perceptions, convolutional neural networks, recurrent neural networks, etc.) and what types of problems each is appropriate for.
• Select or design neural network architectures for new data problems based on their requirements and problem characteristics, and analyze their performance.
• Describe some of the latest research being conducted in the field and open problems that are yet to be solved.

Sample Syllabus

Spring 2024 syllabus (PDF)
Spring 2023 syllabus (PDF)
Fall 2022 syllabus (PDF)

Note: Sample syllabi are provided for informational purposes only. For the most up-to-date information, consult the official course documentation.

Before Taking This Class...

Suggested Background Knowledge

It is recommended that students have a strong mathematical background (linear algebra, calculus especially taking partial derivatives, and probabilities & statistics) and at least an introductory course in Machine Learning (e.g. equivalent to CS 7641). This should not be your first ML class, and self-study (e.g. online Coursera/Udacity courses) do not count. Strong programming skills (specifically Python) are necessary to complete the assignments.

Technical Requirements and Software

• Computer Hardware and Software
  • High-speed Internet connection
  • Laptop or desktop computer with a minimum of a 2 GHz processor and 4 GB of RAM
  • CUDA compatible GPU is helpful for assignments but not necessary
  • UNIX-like OS experience is recommended (Linux/iOS)
  • Windows/Linux for PC computers OR Mac iOS for Apple computers
  • Complete Microsoft Office Suite or comparable and ability to use Adobe PDF software (install, download, open and convert)
  • Mozilla Firefox, Chrome browser, and/or Safari browsers (Chrome required for on-boarding quiz)
• Canvas
  • This class will use Canvas to deliver course materials to online students. ALL course materials and quiz/discussion assessments will take place on this platform. Gradescope will be used for submission of assignments and the project.
• Proctoring Information
  • In order to verify the identity of all GT online students, all online students are required to complete the onboarding quiz that uses Honorlock. Honorlock is utilized for student identity verification and to ensure academic integrity. Honorlock provides student identity verification via facial and ID photos. You may also be asked to scan the room around you. The onboarding quiz will be a practice quiz that will not affect your grade in the course. You can take the onboarding quiz as many times as you want. All potential violations are reviewed by a human. The Honorlock support team is available 24/7. While Honorlock will not require you to create an account, download software, or schedule an appointment in advance, you will need Google Chrome and download the Honorlock Chrome Extension. Information on how to access Honorlock and additional resources are provided below. You can also access Honorlock support at https://honorlock.com/support/.

Academic Integrity

All Georgia Tech students are expected to uphold the Georgia Tech Academic Honor Code. This course may impose additional academic integrity stipulations; consult the official course documentation for more information.