Inhalt

Abstract:

As machine learning (ML) matures from a theoretical discipline into a core engineering competency, the focus shifts from understanding simple models to mastering complex architectures and industrial-grade deployment. This course provides a rigorous exploration of deep learning architectures across diverse data modalities. As systems evolve from isolated components to deeply interconnected systems, the ability to interpret unstructured visual data (images), analyze high-velocity sensor streams (time series), and optimize relational system topologies (graphs) is essential for the next generation of engineers.

The curriculum examines specialized techniques for images—including classification, object detection, and segmentation—as well as sequence modeling for time series and relational analysis for graph structures. We discuss inspiring use cases such as precision agriculture (identifying crops and weeds in image data), systems health monitoring (detecting road surface conditions from time-series data), and molecular modeling (predicting solubility using graph data). Complementing these architectural foundations, the course introduces MLOps to emphasize the transition from isolated models to robust, reproducible production pipelines. Through this integrated approach, students get to know the specialized toolkit required to operationalize deep learning across complex engineering domains.

Gliederung:

The course is structured to guide students through specialized architectures for different data types and the systematic management of ML systems:

1. The ML lifecycle & workflow: We revisit the CRISP-DM workflow through an advanced lens, focusing on resource constraints and the nuances of data preparation, such as one-hot encoding and sophisticated imputation methods.
2. Deep learning for computer vision: We move beyond basic CNNs to explore state-of-the-art architectures, including residual networks (ResNet), vision transformers (ViT), and MLP-Mixers, alongside specialized tasks like object detection (Faster R-CNN) and semantic segmentation (U-Net).
3. Sequence modeling & time series: We investigate 1D convolutions and recurrent neural networks (RNNs) for sequential sensor data, addressing the vanishing gradient problem through gated architectures like long short-term memory (LSTM) and gated recurrent units (GRU).
4. Geometric deep learning: We introduce graph neural networks (GNNs) to handle non-Euclidean data, covering vertex/edge attributes, message passing, and graph attention for applications in molecular analysis and infrastructure networks.
5. Multimodal learning: We explore the fusion of disparate data sources (text, audio, and video) using early, late, and cross-modality strategies, with a focus on attention-based data fusion.
6. MLOps & technical debt: we conclude with a focus on the infrastructure required for scalable, repeatable, and observable machine learning systems in production environments.
   
   

Detaillierter Inhalt:

This course is delivered through advanced video modules that transition from specific architectural "building blocks" to full-scale system integration. The curriculum begins with deep-dive sessions on spatial data, analyzing how convolutions, pooling, and padding function within modern classification and detection frameworks. It then shifts to temporal data, providing a theoretical treatment of sequence modeling and the internal gating mechanisms of LSTMs and GRUs. The final technical phase introduces graph-based learning, enabling students to model systems where relationships between entities are as important as the entities themselves. To bridge the gap between experimental code and industrial software, the course also provides a comprehensive overview of MLOps. This includes managing technical debt, implementing CI/CD pipelines, and utilizing metadata stores for experiment tracking.

A central component of the learning experience is the practical application of theory through two major hands-on exercises:

• Transfer learning for quality inspection: Students develop an image classification system for a miniature truck assembly line. Using pre-trained convolutional neural networks, participants implement transfer learning to identify correct assembly configurations (varying by truck type and color) from a limited dataset, simulating high-efficiency quality control in manufacturing.
• Predictive maintenance for machining: Utilizing the NASA Ames milling dataset, students implement recurrent neural networks (RNNs) as a means of forecasting tool condition during machining operations. The exercise focuses on processing multi-sensor time-series data (vibration, current, and acoustic emissions) to predict tool wear, enabling proactive maintenance and preventing expensive tool breakage.
  
  

Technical proficiency is concluded by learning how to deploy these models effectively onto web servers, mobile devices, and industrial edge hardware.

Although this course can also be done standalone, some prior knowledge of machine learning frameworks and computer programming from introductory courses is strongly recommended.

Lern-/Qualifikationsziele:

Upon successful completion of this course, students will be able to:

• Implement spatial data solutions using 2D CNNs and transfer learning for advanced image classification tasks, while understanding the mechanics of detection and segmentation frameworks.
• Apply temporal modeling techniques using 1D convolutions and gated recurrent architectures (LSTM/GRU) to address sequence forecasting and vanishing gradient issues in engineering sensor data.
• Understand relational topologies by exploring the principles of graph neural networks (GNNs), including message passing and graph attention for non-Euclidean data structures.
• Identify multimodal fusion strategies using early, late, and cross-modality attention mechanisms to integrate disparate data sources like text, audio, and video.
• Explain MLOps infrastructure requirements, including the role of feature stores, metadata stores, and automated CI/CD pipelines in managing technical debt and production reliability.
• Select optimal deployment strategies for industrial AI, balancing latency and hardware constraints across web, mobile, and edge environments.
  
  

Lehrveranstaltungstyp:

Virtuelle Vorlesung

Interaktionsformen mit Betreuer/in:

E-Mail

Interaktionsformen mit Mitlernenden:

Forum

Kursdemo:

zur Kursdemo

Nutzung

Kurs ist konzipiert für:

Master’s degree

Maschinenbau; Mechatronik; Medizintechnik; Wirtschaftsingenieurwesen; International Production Engineering and Management; Elektrotechnik, Elektronik und Informationstechnik; Informations- und Kommunikationstechnik; Energietechnik; Materialwissenschaft und Werkstofftechnik; Elektro- und Informationstechnik; Medical Engineering and Data Science; Erneuerbare Energien und Energiemanagement

Formale Voraussetzungen:

Course enrolment via the Virtuelle Hochschule Bayern (vhb)

Erforderliche Vorkenntnisse:

Although this course can also be done standalone, some prior knowledge of machine learning frameworks and computer programming from introductory courses is strongly recommended.

Hinweise zur Nutzung:

-

Kursumsetzung (verwendete Medien):

Course consists of lecture modules containing videos with self-assessment questions, along with multiple hands-on exercises

Erforderliche Technik:

A computer able to run a web browser and, optionally, a Python development environment (a mobile device should also work but may not provide the optimal experience)

Nutzungsentgelte:

für andere Personen als (reguläre) Studenten der vhb Trägerhochschulen nach Maßgabe der Benutzungs- und Entgeltordnung der vhb

Rechte hinsichtlich des Kursmaterials:

-

Verantwortlich

Anbieterhochschule:

Uni München (LMU)

Anbieter:

Prof. Dr. Björn Eskofier

Autoren:

Jörg Franke

Nico Hanenkamp

Björn Eskofier

Betreuer:

Thomas Altstidl

Prüfung

Online course exam for students

Art der Prüfung:

schriftlicher Leistungsnachweis (Klausur)

Bemerkung:

The examination is online and the date is announced in the course.

Prüfer:

Prof. Dr. Björn Eskofier

Prüfungsanmeldung erforderlich:

ja

Anmeldeverfahren:

Registration for the exam takes place directly in the course.

Prüfungsanmeldefrist:

–

Prüfungsabmeldefrist:

–

Kapazität:

–

Prüfungsdatum:

–

Prüfungszeitraum:

–

Prüfungsdauer:

60 Minuten

Prüfungsort:

–

Zuständiges Prüfungsamt:

Examination office of the students' home university

Zugelassene Hilfsmittel:

None

Formale Voraussetzungen für die Prüfungsteilnahme:

Course enrolment via the Virtuelle Hochschule Bayern (vhb) and timely registration for the exam directly in the course

Inhaltliche Voraussetzungen für die Prüfungsteilnahme:

Course content

Zertifikat:

Ja (Proof of performance (graded))

Anerkennung:

–