from Hacker News

TuneNN: A transformer-based network model for pitch detection

by CMLab on 12/19/23, 12:27 PM with 39 comments

• by joonatan on 12/19/23, 6:58 PM

  Could someone fill me in why would machine learning be necessary for pitch detection? Isn't it something that could just be solved with FFT or it's a much more complicated task?

• by CMLab on 12/19/23, 12:27 PM

  A transformer-based network model, pitch tracking for musical instruments.

  The timbre of musical notes is the result of various combinations and transformations of harmonic relationships, harmonic strengths and weaknesses, instrument resonant peaks, and structural resonant peaks over time.

  It utilizes the transformer-based tuneNN network model for abstract timbre modeling, supporting tuning for 12+ instrument types.

• by rrherr on 12/19/23, 4:53 PM

  How does the accuracy of this compare to CREPE?

  https://github.com/marl/crepe

  https://github.com/maxrmorrison/torchcrepe

  Does anyone know what the current state of the art is, within the Music Information Retrieval community?

• by bravura on 12/19/23, 7:08 PM

  What's the license?

  What are your thoughts on PESTO which learns pitch-prediction very well with a small network, and uses a self-supervised objective?

  https://arxiv.org/abs/2309.02265

  https://github.com/SonyCSLParis/pesto

• by vessenes on 12/19/23, 4:11 PM

  This is cool! The very best software-based tuning tech out there is probably in piano tuning apps; they cost hundreds of dollars+ and are specifically made to report on harmonics and other piano nuances.

  Do you have any comparisons against other pitch detection tech? Accuracy? Delay/Responsiveness? I assume it's much more compute work than a handcoded FFT type pitch detector.

  I think it's possible this would find utilization in the piano world if the output offers something new / something that can analyze what a piano tuning maestro can hear and make it accessible to a mid-tier tuner.

• by squidsoup on 12/19/23, 7:45 PM

  It might be worth pointing out that the banjo model is for a four string banjo, given a five string banjo is the more common instrument.

• by filterfiber on 12/19/23, 7:39 PM

  Does anyone know where I should look if I want to detect specific sounds? Like a smoke alarm, food bowl dispenser (its very distinct), cat meowing, 3d printer collision, that sort of thing?

• by ranting-moth on 12/19/23, 4:54 PM

  To the dev: the tuner gives me an incredibly high error window with the following message. It doesn't prompt to access the mic (I think that's related). Ubuntu/KDE/Firefox:

  An error occurred running the Unity content on this page. See your browser JavaScript console for more info. The error was: TypeError: 'microphone' (value of 'name' member of PermissionDescriptor) is not a valid value for enumeration PermissionName. checkPermission@https://aifasttune.com/public/web/microphone/microphone.js:3... _Microphone_checkPermission@https://aifasttune.com/public/web/Build/web.framework.js:10:... @https://aifasttune.com/public/web/Build/web.wasm:wasm-functi... @https://aifasttune.com/public/web/Build/web.wasm:wasm-functi... @https://aifasttune.com/public/web/Build/web.wasm:wasm-functi... @https://aifasttune.com/public/web/Build/web.wasm:wasm-functi... @https://aifasttune.com/public/web/Build/web.wasm:wasm-functi... @https://aifasttune.com/public/web/Build/web.wasm:wasm-functi... invoke_iiii@https://aifasttune.com/public/web/Build/web.framework.js:10:... @https://aifasttune.com/public/web/Build/web.wasm:wasm-functi... @https://aifasttune.com/public/web/Build/web.wasm:wasm-functi... @https://aifasttune.com/public/web/Build/web.wasm:wasm-functi... @https://aifasttune.com/public/web/Build/web.wasm:wasm-functi... @https://aifasttune.com/public/web/Build/web.wasm:wasm-functi... @https://aifasttune.com/public/web/Build/web.wasm:wasm-functi... @https://aifasttune.com/public/web/Build/web.wasm:wasm-functi... unityFramework/Module._SendMessageString@https://aifasttune.com/public/web/Build/web.framework.js:10:... ccall@https://aifasttune.com/public/web/Build/web.framework.js:10:... SendMessage@https://aifasttune.com/public/web/Build/web.framework.js:10:... SendMessage@https://aifasttune.com/public/web/Build/web.loader.js:1:3343 loadURL@https://aifasttune.com/public/web/game/fastGameController.js... i@https://aifasttune.com/assets/index-64322640.js:1:777 setup/<@https://aifasttune.com/assets/index-64322640.js:1:611