Treffer: Kymatio: Deep Learning MeetsWavelet Theory for Music Signal Processing.

We present a tutorial on MIR with the open-source Kymatio (Andreux et al., 2020) toolkit for analysis and synthesis of music signals and timbre with differentiable computing. Kymatio is a Python package for applications at the intersection of deep learning and wavelet scattering. Its latest release (v0.4) provides an implementation of the joint time--frequency scattering transform (JTFS), which is an idealisation of a neurophysiological model that is commonly known in musical timbre perception research: the spectrotemporal receptive field (STRF) (Patil et al., 2012). In the MIR research, scattering transforms have demonstrated effectiveness in musical instrument classification (Vahidi et al., 2022), neural audio synthesis (Andreux et al., 2018), playing technique recognition and similarity (Lostanlen et al., 2021), acoustic modelling (Lostanlen et al., 2020), synthesizer parameter estimation and objective audio similarity (Vahidi et al., 2023, Lostanlen et al., 2023). The Kymatio ecosystem will be introduced with examples in MIR: • Wavelet transform and scattering introduction (including constant-Q transform, scattering transforms, joint time--frequency scattering transforms, and visualizations) • MIR with scattering: music classification and segmentation • A perceptual distance objective for gradient descent • Generative evaluation of audio representations (GEAR) (Lostanlen et al., 2023) A comprehensive overview of Kymatio's frontend user interface will be given, with examples of extensibility of the core routines and filterbank construction. We ask our participants to have some prior knowledge in: • Python and NumPy programming (familiarity with Pytorch is a bonus, but not essential) • Spectrogram visualization • Computer-generated sounds No prior knowledge of wavelet or scattering transforms is expected. References • Andreux, M., Angles, T., Exarchakisgeo, G., Leonardu, R., Rochette, G., Thiry, L., . . . & Eickenberg, M. (2020). Kymatio: Scattering transforms in python. The Journal of Machine Learning Research, 21(1), 2256-2261. • Andreux, M., & Mallat, S. (2018, September). Music Generation and Transformation with Moment Matching- Scattering Inverse Networks. In ISMIR (pp. 327-333). • Lostanlen, V., El-Hajj, C., Rossignol, M., Lafay, G., Andén, J., & Lagrange, M. (2021). Time--frequency scattering accurately models auditory similarities between instrumental playing techniques. EURASIP Journal on Audio, Speech, and Music Processing, 2021(1), 1-21. • Lostanlen, V., Cohen-Hadria, A., & Bello, J. P. (2020). One or two components? the scattering transform answers. arXiv preprint arXiv:2003.01037. • Lostanlen, V., Yan, L., & Yang, X. (2023). From HEAR to GEAR: Generative Evaluation of Audio Representations. Proceedings of Machine Learning Research, (166), 48-64. • Muradeli, J., Vahidi, C., Wang, C., Han, H., Lostanlen, V., Lagrange, M., & Fazekas, G. (2022, September). Differentiable Time-Frequency Scattering On GPU. In Digital Audio Effects Conference (DAFx). • Vahidi, C., Han, H., Wang, C., Lagrange, M., Fazekas, G., & Lostanlen, V. (2023). Mesostructures: Beyond spectrogram loss in differentiable time-frequency analysis. arXiv preprint arXiv:2301.10183. [ABSTRACT FROM AUTHOR]

Copyright of International Society for Music Information Retrieval Conference Proceedings is the property of Ubiquity Press and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)