from Hacker News

Show HN: A little side project, a watercolor art generator

by nfriend on 11/11/22, 2:32 PM with 71 comments

Hi HN - this is a little side project I threw together. Some implementation details: image processing is all done with headless GIMP (running inside a Docker container) through its built-in Python API. It's _very_ slow (about 50 seconds/image), and currently it processes exactly one image at a time. The website is built with NextJS; payments are processed by Stripe.

I've had the best results with pictures of houses, although certain photos of people or nature can look neat, too. (For example: https://brushify.art/s/ruYmQWk, original photo from https://en.wikipedia.org/wiki/Pillars_of_Creation.) The effect obscures the edges of the photo, so images with plenty of margin around the subject work best.

Something I'd like to play around with is swapping the GIMP script for an AI-based process (maybe using something like Stable Diffusion?), with the goal of generating images that look more handmade (something like these: https://www.etsy.com/ca/search?q=watercolor+house). I have exactly zero AI experience though, so there would be a bit of a learning curve.

Would love any thoughts or critiques!

----

edit: remove unrelated details

  • by rikroots on 11/11/22, 5:17 PM

    Creating a watercolor filter which can be applied to an image dragged into a website <canvas> element is on my list of Fun Things To Do Over The Holidays - but having previously looked at the the mechanics that need to feed into the filter ... well it looks like I'd be jumping down a rabbit hole of complex maths and generative art approaches[1][2]. It's a complex problem space and I'm not sure I have the brain power and tenacity to do a good job of it.

    [1] - Tyler Hobbs (2017) - a guide to simulating watercolor paint with generative art - https://tylerxhobbs.com/essays/2017/a-generative-approach-to...

    [2] - Curtis|Anderson|Seims|Fleischery|Salesin (undated) - Computer-Generated Watercolor - https://grail.cs.washington.edu/projects/watercolor/paper_sm...

  • by kuu on 11/11/22, 3:23 PM

    My recommendation: Add examples on the main page, so the new users don't need to overload your site.

    Otherwise the process is the following:

    I enter the page -> Upload image -> See 4h of queue for getting my result -> I close the tab, leaving there a pending task.

  • by synapticpaint on 11/11/22, 4:33 PM

    It sounds like your app turns photos into a watercolor style image? You can definitely do this faster with Stable Diffusion (~6s per image before optimization).

    Here's how I would approach it: train a dreambooth model on watercolor style images, then run image-to-image using that model.

    For examples of what dreambooth models can do see: https://synapticpaint.com/dreambooth/info/ (sample images here generated using a "modern disney" style model).

    If you need help getting this set up feel free to email me! This stuff is probably not harder than getting gimp to run in a container.

  • by nfriend on 11/11/22, 3:07 PM

    Sorry all for the long wait times! Was not expecting much interest. At the very least, I need to look into removing photos from the queue when people exit the page.

  • by tfsh on 11/11/22, 4:29 PM

    The example looks cool, but I'm currently in an 11 hour queue :(

    Any chance of porting this to the web using WASM? I've used ImageMagick in the browser before, I've never used Gimp, but if there's enough overlap you could use the former. That's of course assuming two things:

    1) you have the time

    2) you're happy to port the closed source, source code to the client

    Both of which are perfectly fine to answer with a "no" :)

    Also, I think it would be prudent to terminate the request if the client instance is destroyed. Right now I assume there's a bunch of requests being processed for users who have closed the tab.

  • by wingworks on 11/11/22, 10:37 PM

    The results are pretty cool, though it's a shame it cropped the edges. https://brushify.art/result/aa4e68ae-d9de-4673-830e-f88032ec...

  • by chrischen on 11/12/22, 12:13 AM

    Our watercolor filter produces pretty good results (the Flower Market filter which also happens to be the example shown): https://www.instapainting.com/assets

    You can implement with something like this and simply train it against a watercolor image: https://github.com/yusuketomoto/chainer-fast-neuralstyle

    Haven’t tried it with stable diffusion but you’d probably have more control and better results with a CNN like the one I linked.

  • by andai on 11/11/22, 7:44 PM

    Here's a suggestion, port it to run in the browser. Free parallelism, because every user brings their own hardware :) So it scales infinitely, for free.

  • by jasonjmcghee on 11/11/22, 4:29 PM

    Fortunately OP, experience with AI is generally unrelated to success with Stable Diffusion / DALLE.

    What you want to gain is experience with prompt engineering for these tools.

    Here's a good resource https://openart.ai/promptbook

  • by bambax on 11/11/22, 4:23 PM

    > I'd like to play around with is swapping the GIMP script for an AI-based process

    Dall-e offers an API with some limitations (max 4MB square PNG, content filtering): https://beta.openai.com/docs/guides/images/usage

    Photopea.com has a watercolor filter, in JS, all client side. It's not very good ATM so there's certainly room for improvement. Doing it all client side would solve the queue problem.

  • by gregoriol on 11/11/22, 3:29 PM

    If you can make it with GIMP, you could probably look into OpenGL shaders to do it directly yourself: the processing could be faster and would provide you with more flexibility and coding fun times. I don't know of server-side libraries for that, but on iOS we have MetalPetal which provides some filtering features like ones in Gimp/Photoshop.

    Have fun!

  • by ikkjo on 11/12/22, 11:36 AM

    Would you be willing to share some details on the GIMP implementation? Are you actually generating brush strokes and "paint" the image or is it more filter based? The background effect looks like brush strokes to me, but are the details done in the same way?

  • by justchad on 11/11/22, 3:16 PM

    Nice, this is really cool! I don't see myself waiting for my photo to process as it's a ~4 hour wait but the example image is cool.

    Like you said I bet using Stable Diffusion would speed this up dramatically but who knows if you'll get the same effect on your images.

  • by _448 on 11/11/22, 3:20 PM

    In addition to upload/drop image, you could also provide an option to enter an URL of online image. That will make it easy for users to try the project very quickly. Then provide an ephemeral preview URL to see the output.

  • by yuvalkarmi on 11/11/22, 2:54 PM

    This is super cool! Didn't know you could run GIMP in headless mode.

  • by seshagiric on 11/12/22, 12:14 AM

    I understand it's a side project but you do have a paid offering. So I am curious what are you doing with the images after they are processed?

  • by asmosoinio on 11/11/22, 7:53 PM

    Site seems to be down - could you post the example output on some image sharing site?

    Seeing example output would be interesting, even if service is hugged to death.

  • by hollowdene on 11/11/22, 2:55 PM

    Can't see myself paying $5.99 for one of these, but it's nice. I appreciated the fun messages in the progress bar.

    PS: Congrats on the little one. :D

  • by iruoy on 11/11/22, 2:59 PM

    It's not exactly a hug of death, but a wait time of 90m is a bit long. I hope that people that leave the queue won't be processed.

  • by _dan on 11/11/22, 4:21 PM

    Perhaps put the wait time on the homepage - If I'd known you were running at an 8 hour delay I wouldn't have added to it!

  • by pknerd on 11/11/22, 3:45 PM

    The message on the page:

    It's a busy day! There are currently 512 people in front of you. Please keep this tab open!

    Estimated wait time: 7 hours, 7 minutes

  • by keizo on 11/12/22, 2:56 AM

    Super cool! Cnc painter next and make a business making custom cards for people!

  • by stoobs on 11/11/22, 3:38 PM

    Estimated wait time: 6 hours, 33 minutes

    Oops, sorry OP for all the traffic :D

  • by the_arun on 11/11/22, 8:23 PM

    There should be a smart phone app for this already, right?

  • by diego_moita on 11/11/22, 10:11 PM

    I just loved the messages while it is processing!

  • by Kiro on 11/11/22, 7:13 PM

    Why NextJS for this?

  • by Kaotique on 11/11/22, 2:53 PM

    Looks really nice!

  • by notyourav on 11/11/22, 4:56 PM

    14h wait time

  • by tristanbvk on 11/11/22, 2:55 PM

    Very vool

  • by hackmiester on 11/11/22, 4:45 PM

    I'm going to go against the grain here. If this were my project I would try to spin up render machines dynamically in response to load. (Similarly to how gitlab CI runners can be spun up automatically, for instance.) There is no need to replace a working technology stack to make it faster. 1 minute is a reasonable wait time, and if your wait time goes over 5 minutes, then spin up some more workers.

    Plus we have to be real and say, is the only reason the queue so long, because you let a bunch of nerds on HN add to the queue for free? I'm guessing the answer is, "probably." No need to engineer a fix for a problem that will typically not exist.

  • by XCSme on 11/11/22, 5:01 PM

    I hope the result is worth the wait!

    > There are currently 1069 people in front of you. Please keep this tab open!

    > Estimated wait time: 14 hours, 51 minutes

  • by moffkalast on 11/11/22, 3:04 PM

    > Estimated wait time: 2 hours, 35 minutes

    Bruh moment.

    I suppose it would make more sense to WASM the implementation and run it clientside?

  • by ale42 on 11/11/22, 4:08 PM

    Now: Estimated wait time: 9 hours, 49 minutes...

    The hug is getting tighter ;-)

  • by xchip on 11/11/22, 2:47 PM

    Why do we need to know you did this while on paternity leave?