上課地點:學新118

“Music Information Research” (MIR) is an interdisciplinary research field that concerns with the analysis, retrieval, processing, and generation of musical content or information. Researchers involved in MIR may have a background in signal processing, machine learning, information retrieval, human-computer interaction, musicology, psychoacoustics, psychology, or some combination of these. In this course, we are mainly interested in the application of machine learning, in particular deep learning, to address music related problems. Specifically, the course is divided to two parts: analysis and generation. The first part is about the analysis of musical audio signals, covering topics such as feature extraction and representation learning for musical audio, music audio classification, melody extraction, automatic music transcription, and musical source separation. The second part is about the generation of musical material, including symbolic-domain MIDI or tablatures, and audio-domain music signals such as singing voices and instrumental music. This would involve deep generative models such as generative adversarial networks (GANs), variational autoencoders (VAE), Transformers, and diffusion models. Here is a tentative schedule of the course: W1. Introduction to the course W2. Fundamentals & Music representation W3. Analysis I (timbre): Automatic music classification and representation learning (HW1: to be announced) W4. Generation I: Source separation W5. Generation II: GAN & Vocoders W6. Generation III: Synthesis of notes and loops (HW2: to be announced) W7. Analysis II (pitch): Music transcription, Melody extraction, and Chord Recognition W8. Generation IV: Symbolic MIDI generation W9. Generation V: Symbolic MIDI generation: Advanced Topics (HW3: to be announced) W10. Generation VI: Singing voice generation W11. Generation VII: Text-to-music generation W12. Proposal of ideas of final projects W13. Generation VIII: Differentiable DSP models and automatic mixing W14. Miscellaneous Topics W15. Break W16. Oral presentation of final projects

1. Understanding of different aspects of music: timbre, rhythm, pitch, harmony, and structure, and the use of domain knowledge for corresponding music signal analysis tasks. 2. Understanding of and hands-on experiences with deep learning techniques to music audio signal analysis 3. Understanding of and hands-on experiences with deep generative models for both musical audio and text-like music data such as MIDI 4. A taste of the fun of research

I would assume that students taking this course to * have good background in machine learning and mathematics (e.g., have taken courses such as Machine Learning, Deep Learning, Signals and Systems, Digital Signal Processing, Linear Algebra, Probability and Statistics) * have good coding experience in python and a deep learning framework such as PyTorch * have great interest in music

Jakub M. Tomcza, Deep Generative Modeling. 978-3-030-93158-2. Springer, 2022.