by jaflo on 6/26/23, 3:12 PM with 161 comments
The cool thing is that after the song is analyzed on the server, the client can recompute and preview the results completely client-side through an implementation that uses Web Workers and WebAssembly. The audio previewing uses Tone.js. I am thinking of writing up some more details about the implementation in the future.
I'm still working on a way to explain this easily, but I like the idea of carrying over the concept of content-aware fill from images to audio.
Please let me know if you have any comments or questions!
by Philip-J-Fry on 6/26/23, 4:53 PM
I think it was https://eternalbox.dev/ since that's all I can find on Google. But that site is down.
by josuepeq on 6/26/23, 4:44 PM
But for today’s music, shortening the 2010s/2020 already shorter lengths would mean a song might not be more than a minute in length. On average, full unedited tracks today end up being a bit shorter than they used to be, solely due to the economics of streaming. Rather than paying for the content second by second, it is done by paying per track play. The result is a lot of 2 minute tracks, which were produced with the “verse” parts getting jammed together into the “chorus” with no break in vocals, which also uses pitch adjustments, “the “bridge” is an afterthought that is terrible, or more recently, nonexistent……… Instrumental solo? Anyone? Bueller?
Music is no longer anticipated, budgeted for, and purchased on launch day with great fanfare. We have grown accustomed to the idea that we should have everything available at our fingertips, and as a consequence of this we get exactly what we pay for.
by lelanthran on 6/26/23, 7:30 PM
I uploaded Wuthering Heights[1], selected just the trailing guitar solo (3:09 to just before the fade-out starts), and got a bunch of seemless sounding 3m tracks of just that guitar shredding away.
All the ones it generated were perfectly cut and pasted, with one exception in track 2 where I could tell where it was cut/pasted.
Can't wait to try it with the guitar at the end of Brothers in Arms (Dire Straits), or the flute solo in Locomotive Breath.
If anyone is looking for a good use of AI, I wouldn't mind a webpage that lets me say "complete this solo that was abruptly cut short" and get something great.
[1] This one is especially sad because the sound engineer/producer later lamented that he faded the solo out so quickly because the player was still improving like mad and what he had left after the fadeout was apparently better than what he had done while recording.
by cheschire on 6/26/23, 4:39 PM
by mmh0000 on 6/26/23, 5:35 PM
I like to have multiple hour continuations of songs which I use to help me fall asleep. In the past I've made my own, but I'm terrible at audio editing. It is a ton of work and, for me, really hard to get right.
After a little bit of experimenting with Mofi, it seems to do a very good job and selecting when to repeat a sections.
[edit]
Here's my first attempt for a 30 minute song:
https://mofi.loud.red/edit/ebbf4b410181aa767152945cbb6a2d679...
by mmh0000 on 6/26/23, 10:06 PM
1] A lot of the heavy lifting is done client side, which is excellent; I have a super fast processor! But, watching the CPU utilization, it seems to spike the CPU for a few milliseconds, then waits for a second or two and, spikes the CPU again, then repeats. I suspect there may be some significant performance improvements to be had on client recomputes.
2] While it is doing recomputes, it would be great to get a progress bar or, at least, a message saying, "This will take a long time, be patient." It took me multiple attempts to determine that the service was indeed working, just that my client was taking its sweet time recomputing.
by spaceman_2020 on 6/26/23, 3:50 PM
by diggum on 6/26/23, 9:37 PM
by Solvency on 6/26/23, 4:06 PM
by zedadex on 7/6/23, 2:32 PM
With Eternal Jukebox I was able to emulate it by setting the jump percent very high manually at the right time (but had to be paying attention + had to get lucky with which branch it took), whereas here I'll see if I can get it going by calculating what the cut I want would set the song length to, and requesting that length and fewest cuts. (Edit: didn't work)
Sometimes the simplest features are the easiest to overlook but most useful
by MatthewWilkes on 6/26/23, 3:49 PM
by sergiotapia on 6/26/23, 5:21 PM
He just butchers an otherwise perfect song!
Will report back with results.
Edit: Results are great! https://voca.ro/13ar1g88LSKK
by lcnPylGDnU4H9OF on 6/26/23, 10:27 PM
> On repeat: Make an extended version of your song's favorite part by choosing the catchy part and seamlessly repeating part of it!
John Mulaney's "Best Meal I Ever Had" story[1] had me wanting to do this for Tom Jones' What's New Pussycat so I could play that as an inside joke with some friends. This would definitely have made that easier. (Ultimately, I had a version of the song which started in the middle and "ended" right before it "started" so the effect could be mimicked if the file was played on loop. This had a short silence manifest as the media player "loaded a new song" from its perspective; didn't have that issue in Audacity.)
[0] https://www.pokemon.com/us/pokedex/loudred
[1] https://www.youtube.com/watch?v=Tv1l1eUhN-E (Usually called "The Salt and Pepper Diner")
by sd9 on 6/26/23, 4:16 PM
by codetrotter on 6/26/23, 3:55 PM
This is perfect for TikTok videos!
Will definitely be using your tool. Very cool!
> I am thinking of writing up some more details about the implementation in the future.
Please do :D
by personjerry on 6/26/23, 6:10 PM
Is the project open-source?
by stu2010 on 6/26/23, 4:51 PM
by jedberg on 6/26/23, 6:18 PM
at 00:00 a 00:16 segment (00:00–00:16)
at 00:16 a 00:51 segment (02:05–02:57)
at 01:07 a 02:31 segment (01:34–04:05)
at 03:39 a 00:11 segment (04:03–04:14)
by quickthrower2 on 6/26/23, 4:34 PM
by redox99 on 6/26/23, 4:01 PM
by ConnorMooneyhan on 6/26/23, 4:50 PM
by superasn on 6/27/23, 7:06 AM
Though it's just anecdotal but I have seen that the quickest way to enter into the zone is to play repeating loops at 140 bpm.
Unfortunately I haven't really found many of these as most of the tracks I love are non looping. I think this can take care of it. Kudos on making this!
by kunalgupta on 6/26/23, 4:37 PM
by LorenDB on 6/27/23, 3:42 PM
by nicexe on 6/27/23, 7:15 AM
I set it to avoid the segment that the key changes and to favor the 2 solo segments.
It doesn't seem like its doing a good job. Only the first 5 seconds of the intro made it and then is just repeating over and over the big solo. In the end, a very small part of the short solo makes it in as an outro.
Maybe it works better at shortening songs or maybe changing duration closer to the original duration.
by pradn on 6/26/23, 5:32 PM
A less rosy scenario is like Spotify generating music to get around having to pay artists.
by edonhametsu on 6/27/23, 5:37 AM
For example, ESR 102 does not have the feature and it breaks the site.
by personjerry on 6/26/23, 5:10 PM
by empyrrhicist on 6/27/23, 2:05 AM
by cyberax on 6/26/23, 5:50 PM
It needs to be generalized to video content, but the task is a bit easier.
by danhau on 6/26/23, 9:31 PM
by benkaiser on 6/27/23, 1:51 AM
by emmanueloga_ on 6/26/23, 7:31 PM
Found a bunch of cool related links here [1].
1: https://en.wikipedia.org/wiki/Music_information_retrieval
by personjerry on 6/26/23, 9:22 PM
by Workaccount2 on 6/26/23, 5:35 PM
There are a few songs I love, but some of the best parts seem to be "buried" under other instruments.
by EwanG on 6/26/23, 7:31 PM
by JonathanFly on 6/27/23, 5:14 AM
by jonathantf2 on 6/26/23, 8:22 PM
by boffinAudio on 6/27/23, 6:57 AM
by wpietri on 6/26/23, 6:04 PM
by SillyUsername on 6/27/23, 7:43 PM
by smokeydoe on 6/26/23, 4:42 PM
by KolmogorovComp on 6/26/23, 4:14 PM
by EGreg on 6/27/23, 6:48 AM
by netvarun on 6/26/23, 11:57 PM
by JohnFen on 6/26/23, 5:42 PM
by yumikai on 6/27/23, 5:04 PM
by omegaworks on 6/26/23, 6:11 PM
by hkab on 6/27/23, 1:47 AM
by yanuzay on 6/27/23, 8:50 AM
by peterdigital on 6/27/23, 1:14 AM
by Pr0ject217 on 6/26/23, 11:18 PM
by sundaeofshock on 6/26/23, 4:11 PM
by 71a54xd on 6/26/23, 4:19 PM
by xadoc on 6/27/23, 5:59 AM
by IndigoIncognito on 6/27/23, 9:01 PM
I wouldn't mind giving money to services like this
by frankfrank13 on 6/26/23, 8:58 PM
by okinawamist on 6/27/23, 5:13 PM
by objectspace on 6/27/23, 7:33 PM
by ljlolel on 6/26/23, 3:47 PM