Nonlinear Optimization

Semester: 2024 SS
Class type: Lecture
Languages: English
Teachers: Evelyn Herberg
Assistants: Mario Marić, Viktor Stein
Contact: Send e-mail
Websites for this class: MÜSLI, Moodle, heiCO(1100121502, 1100121503)
Exam Dates: 2024-08-06, second exam TBA

Course Information

The information on this website will be updated continuously.

Description

This class provides a detailed overview of algorithms for unconstrained and constrained smooth, nonlinear optimization problems in finite dimensions. It will emphasize the interplay of optimization and numerical linear algebra. Optimality conditions and associated constraint qualifications for constrained problems will be presented and discussed. Moreover, the class will give an introduction to algorithmic differentiation (AD).

Prerequisites

Skills from Analysis I and II (MA1/ MA2), Lineare Algebra I (MA4) as well as basic optimization skills are highly recommended.

Concept and Components

This class is comprised of:

  • Detailed lecture notes
  • 2x 2h lectures per week
  • Exercise sheets with corresponding answer sheets
  • Tutorials for co-working on exercise sheets
  • An exam following the end of the lecture period

Dates and Schedule

Information on the exercise labs can be found in MÜSLI.

Please also refer to the class’ schedule.

Exercises

Exercises will be centered around one exercise sheet per week containing both theoretical/analytical questions and numerical/implementation exercises. Default programming language – in the sense that it will be used in our answer sheets and we will be providing support for it – will be Python. Feel free to work with another programming laguage that is suitable for implementing the optimization algorithms if you are self sufficient working with it.

In the exercise tutorials (see MÜSLI), you will have the oportunity to work on the exercise sheets with your peers and with support from our tutors. You can get the most out of their time if you show up having already taken a first look at the exercises and bringing your own questions and discussion topics to the meetings. Feel free to ask questions on the lecture notes or the lectures as well.

For successful participation in this semester’s exercises, you will be asked to submit your exercises weekly, see exams. To evaluate your success on the exercise sheets, please compare your work with the answer sheets we will provide and, of course, feel free to talk to your tutor in the labs.

You may work on the exercises in groups of up to five students.

Please submit your work via Moodle in the following format:

The name of every group member must be represented in the file name. The members are separated by an underscore “_”. Each member is specified in the format “firstname(s)-surname(s)”. Should you have multiple surnames or first names, simply join them without any separator. The listing of the members is followed by an underscore and a tag for the current sheet in the format exsheetnumber. The sheet number must have two digits. If the exercise only requires a PDF, then a simple file upload suffices. Should the exercise require multiple files, such as some code files, etc., bundle them into a zip archive. The name of the archive must follow the conventions specified above. Example:

firstname(s)1-lastname(s)1_firstname(s)2-lastname(s)2_ex01.zip

Mario Maric and Viktor Stein von Kamienski are group members and want to hand in a PDF and a Jupyter notebook for sheet number 4. They archive both files into a zip and call it:

Mario-Maric_Viktor-Steinvonkamienski_ex04.zip

We will use this naming scheme to identify and track the groups. The order of the members, the capitalization of the names, or how many people uploaded the file is irrelevant. Just make sure that the spelling of the members does not change between different assignments. A failure to adhere to this convention will result in a fail on the assignment.

Class Registration

To register for this class, please register for an exercise group at MÜSLI. Please also register in case you are interested in participating in this class but not in the exercises. This will allow us to get an idea of class demand.

Attention(!): Registration for the class does not automatically include registration for an examination. More information on the exams can be found in the section exams.

Exams

We will be offering exams at the end of the semester and during the following summer break.

You may attend up to two exams. It is also possible to not register for the first exam and only attend the second exam. If you pass the exam, you can not take another exam for this course. In case you pass neither of the two offered exams, you have to wait until this course is offered again for the next attempt. It is up to the respective lecturer, whether your admission to the exam will remain valid in following semesters.

It is important that you register in HeiCo, so that we can officially submit your exam results.

Exam Format

The exams will be written with a duration of 120 minutes.

Relevant material for the exam consists of the lecture notes and exercises (including numerical exercises) that are provided for this course.

The following parts of the lecture notes are excluded from the exam:

  • Proofs that were not discussed in the lecture or exercises,
  • Paragraph 3.4,
  • Paragraph 3.5,
  • Modified Armijo backtracking,
  • Cubic interpolation in Wolfe-Powell line search,
  • Paragraph 9.4,
  • Paragraph 12.5,
  • Chapter 5 (will be included in second exam),
  • Appendix B.

Exam Admission Rules

If you want to take the exam with us, we require that you are registered in Müsli and that you meet at least one of the following criteria:

  1. You have been admitted to an exam for this class in a previous semester and let us know via E-Mail when and by whom you have been admitted by 2024-06-30.
  2. You have successfully participated in this semester’s exercise program in the sense that:
    • On at least 12 out of 13 exercise sheets, you have worked on every exercise to a reasonable degree. “To a reasonable degree” means that you have at least sketched an approach of how to solve the excercise and made a connection to the material of the lecture.

Exam Dates

The first exam will take place on 6th of August 2024 from 10:00 to 12:00 in Seminar room A and B. Please be there a few minutes early to ensure a timely start of the exam.

The date of the second exam will be announced.

Exam Registration

You can register in MÜSLI. Registration for the first exam is open until 30th of July 2024.

Cancellation Policy

You may cancel your registration for the exam up to 7 days before the scheduled date of the exam without the need for a reason. If you do not take the exam at the scheduled time without having cancelled at least 7 days ahead of time, you will be required to submit a medical certificate for the scheduled time of the exam, otherwise the central exam office will be notified about the missed exam by us.

In case you registered for an exam, but fail to meet the requirements installed to be admissible for the exam to begin with, your registration is not valid and will not be counted as a failed attempt.

Exam Aids

Every participant may bring a cheat sheet, i.e. one DinA4 sheet with your own notes. You can use both sides and either write or print your notes. You have to put your name on the cheat sheet and it is not allowed to photo-copy another student’s cheat sheet.

Course Material

Lecture Notes

Exercises

Literature

This class is self contained and does not rely on outside literature. If you would like to read up additionally, we can however recommend the following literature: