EN.601.454/654 Augmented reality (Spring 2022)


  • 05/12 The class is over now. Thank you so much for all your good work! Enjoy the summer!
  • 04/01 Kickoff presentation scores are released on Blackboard. Great work, everyone! Good luck with the project implementation. Remember to meet your mentor(s) weekly and keep track on your plan.
    03/19 Project team assignment is posted on Piazza. Please reach out to your mentor at your earliest convenience to discuss your ideas about the project!
  • 03/16 Day 16 slide is posted. Remember to register your final project team and let us know your top three preferences by 3/18 11:59 pm.
  • 03/09 Mock exam solution and quiz 14 are released.
  • 03/08 Research assignment now dues on 3/10 by 11:59 pm.
  • 03/07 Day 13 slide and quiz 13 are posted.
  • 03/02 Day 12 slide, quiz 12 and mock exam are posted.
  • 02/28 Day 11 slide and quiz 11 are posted.
  • 02/26 Research Assignment team registration is closed. You should have received your team info.
  • 02/23 Day 10 slide, quiz 10, and Research Assignment are released.
  • 02/21 Day 9 slide and quiz 9 are posted.
  • 02/16 Day 8 slide, quiz 8, and Assignment 3 are released.
  • 02/14 Quiz 7 is released. Schedule is updated. The guest lecture is moved to 3/9 and the Unity tutorial will happen in class on 3/2. Furthermore, final exam schedule is out. We plan to use the makeup slot (5/13 afternoon) for the project demo instead of the assigned evening slot. Details to be confirmed.
  • 02/13 Day 7 slide is posted.
  • 02/09 Quiz 6 and Assignment 2 are released.
  • 02/07 Quiz 5 is released.
  • 02/06 Day 5 slide is posted.
  • 02/02 Released quiz 4 and updated schedule. Please note, Assignment 1 is now due on Wed instead of Mon. Also, we will stay in Hodson 316. Please ignore the location change on SIS. Please fill in your availability for the optional Unity tutorial session here. We encourage you to join the tutorial session and learn how to set up the environment for your project development. If you are familiar with Unity, you may skip the tutorial.
  • 01/31 Released quiz 3 and Assignment 1.
  • 01/30 Updated Research Assignment’s deadline.
  • 01/28 Updated TAs OHs and posted day 3 slide.
  • 01/26 Quiz 2 is posted. There was a typo in quiz 1’s deadline. It has been fixed. Please remember to finish both quiz 1/2 and the exit tickets before the class on Monday. The schedule and grade breakdown are updated.
  • 01/25 Day 2 slides are posted and CAs information updated.
  • 01/24 Assignment 0, day 1 quiz and slide are posted on Blackboard.
  • 12/20 Course webpage established. Please fill in the OH survey.

Prof. Nassir Navab and Prof. Ehsan Azimi

Time and Venue:
In-person: Mons and Weds from 8:30- 9:45 AM EST at Hodson 316.

Zoom: 997 6845 5195. Registration details on Blackboard.

Recommended Course Background:
EN.601.220: Intermediate Programming or equivalents
EN.601.226: Data Structures or equivalents
AS.110.201/212: Linear Algebra or equivalents

Reference Textbooks:

  • Augmented reality: principles and practiceDieter Schmalstieg, Tobias Höllerer. Boston: Addison-Wesley [2016] (You can borrow it or access it online with our library resource: link)
  • Multiple View Geometry in Computer VisionRichard Hartley, Andrew Zisserman. New York: Cambridge University Press [2003] (You can borrow it or access it online with our library resource: link)

Four to five written/programming assignments. a research assignment, quizzes, a midterm exam, and a final project.  Difficulties vary depending on the level of the course (454/654). The grading breakdown is 33+15+7+15+30.

Grading breakdown:

  • Written/Programming Assignments & Participation: 33% (1+10+10+10+2, w/ bonus, capped at 33)
  • Research assignment: 15% (7% for review and 8% for video)
  • Quizzes & Exit Tickets: 7% (1% each. weekly before Spring Break, to be posted on Blackboard, scope: everything the lecturer is going to say in the lectures that week. Attend the lectures will always help you learn more) Note that, in order to receive your quiz score, you have to fill in the exit ticket that day.
  • Midterm exam: 15%
  • Final project: 30% (5% for kickoff, 8% for implementation, 7% for summary, and 10% for demo and/or poster presentation)

Note 1: Important dates and times are highlighted in the schedule. Please mark them down in your calendar and plan your semester schedule ahead. We do not accept late submission and you should expect no extension for the assignments. Late submission results in 0 scores.

Note 2: If your project is medical related and you are interested (we strongly encourage you), after this course you should follow up with your mentor(s) and write up your project as an article for AE-CAI.

Note 3: Homeworks and projects will be submitted on Blackboard. For discussion, we use Piazza (access code is posted on Blackboard).

Office Hours:
Prof. Nassir Navab and Prof. Ehsan Azimi: by appointment

Before Spring Break:
Wei-Lun Huang: Thursday, 4:30 pm to 6:00 pm on Zoom (link is posted on Blackboard).

Sing Chun Lee: Friday, 8:30 am to 10:00 am at Malone 216.

Yihao Liu: Friday, 5:45 pm to 7:15 pm at Malone 216.

After Spring Break: Meet your tutors regularly by appointment!

Course Materials: They are available on Blackboard

Tentative Schedule (topics and speakers may change):

11/24 (Mon)Introduction to AR & Course LogisticsNassir Navab,
Ehsan Azimi
Assignment 0 (Submission Logistics) – Release
21/26 (Wed)Introduction, Transformations and Coordinate SystemsNassir Navab 
31/31 (Mon)Transformations, Parameter EstimationNassir NavabAssignment 0 – Due: 1/31, 08:30 am (before class)

Assignment 1 (Transformations, Parameter Estimation) – Release
42/02 (Wed)Parameter EstimationNassir Navab 
52/07 (Mon)Camera Models and Projection MatricesNassir Navab
62/09 (Wed)Camera CalibrationNassir Navab Assignment 1 – Due: 2/09, 08:30am (before class)

Assignment 2 (Camera Models, Camera Calibration) – Release
72/14 (Mon)Head-mounted Displays, Calibration and Applications Ehsan Azimi
82/16 (Wed)Medical Augmented Reality Nassir Navab Assignment 2 – Due: 2/16, 08:30am (before class)

Assignment 3 (HMD, Perception) – Release
92/21 (Mon)Visual System and PerceptionAlejandro Martin-Gomez
102/23 (Wed)Sonic Interaction Design in the Context of Medical Intervention Sasan Matinfar Assignment 3 – Due: 2/23, 08:30 am (before class)

Research Assignment – Release

Check the zoom link posted on Piazza.
112/28 (Mon)Design Thinking, User Interaction, and Input DevicesEhsan Azimi
123/02 (Wed)Tutorial Session: Introduction to UnitySing Chun LeeMock Exam – Release

This is a hands-on session. Remember to bring your laptop with you. Also, please follow the installation guideline (posted on Blackboard) to install Unity before class. It takes almost an hour for installation.
133/07 (Mon)Managing Spatial Relations in Augmented Reality ApplicationsNassir Navab

143/09 (Wed)Guest Lecture from Siemens HealthineersAli KamenResearch Assignment – Due 3/09, 11: 59 pm Due 3/10, 11:59 pm

Mock Exam Solution – Release
153/14 (Mon)Midterm ExamIn-class midterm Exam: 3/14
163/16 (Wed)Projects IntroductionNassir Navab, Ehsan AzimiProject team enrollment – Due 3/18 (Fri) 11:59 pm
173/21 (Mon)No ClassesSpring Break
183/22 (Wed)No ClassesSpring Break
193/28 (Mon)Project DevelopmentMeet your mentor(s) and prepare your kickoff presentation
203/30 (Wed)Kickoff PresentationPresentation submission due: 3/29 11:59 pm
In-class presentation: 3/30
214/04 (Mon)Project DevelopmentMeet your mentor(s) and make progress on your AR project
224/06 (Wed)Project DevelopmentMeet your mentor(s) and make progress on your AR project
234/11 (Mon)Project DevelopmentMeet your mentor(s) and make progress on your AR project
244/13 (Wed)Project DevelopmentMeet your mentor(s) and make progress on your AR project
254/18 (Mon)Storytelling.
Preparing your Project Demo
Meet your mentor(s) and finalize your AR project
264/20 (Wed)Preparing your Project DemoMeet your mentor(s) and finalize your demo
274/25 (Mon)Finalize your project and demoMeet your mentor(s) and finalize your demo
284/27 (Wed)Finalize your project and demoMeet your mentor(s) and finalize your demo
295/9 (Mon)Final Project SubmissionSubmission Deadline
305/11 (Wed)Final Project DemoTo be confirmed. Tentative time: 6 pm – 9 pm and venue: Hackerman B08.