from Hacker News

Show HN: Visual Sudoku solver in the browser

by taiters_ on 8/20/24, 12:49 PM with 16 comments

Hello!

I recently wanted to learn a bit about computer vision. Initially, I wanted to build something which could solve a jigsaw puzzle, but figured I should start with something (much) simpler, so I've built this instead.

This is a visual Sudoku solver which runs in the browser. It works by using OpenCV to identify and process the Sudoku grid, passing this to a simple ML model to identify the digits, and then solving the puzzle with a backtracking algorithm. The ML model was trained on the TMNIST data set using a model built with Keras, also a completely new area to me.

It's far from perfect, and doesn't like non perfectly lit or overly warped puzzles, but the main goal here was learning, which I did, a lot.

As this was primarily a learning project, I've tried to document my approach as much as possible, which can be found in this Python notebook:

https://github.com/Taiters/sudoku-solver/blob/main/notebooks...

I used Python while exploring OpenCV and training the models etc, and eventually ported this over to web (OpenCV.js + Tensorflow.js) to get something I could actually share with people.

Feel free to have a dig around the source or play around with the solver!

Github: https://github.com/Taiters/sudoku-solver

  • by giza182 on 8/20/24, 1:17 PM

    I think it would be super helpful to have an example puzzle there so one can quickly try it out without having to find a sudoku puzzle elsewhere

  • by iamflimflam1 on 8/20/24, 3:48 PM

    I made a similar thing a while back - works really well.

    https://www.atomic14.com/2020/07/25/browser-ar-sudoku.html

    You can see it in action here: https://sudoku.cmgresearch.com/

    (assuming it still works - haven't tried it for a while).

  • by hermannj314 on 8/20/24, 1:57 PM

    I undertook this exact project last year but stopped after writing a solver and playing with OpenCV to do edge detection. I also bought a Sudoku puzzle book for making a data set to work with.

    So of course I am glad to see this and thank you for writing this up. Now I'm reinvigorated to throw some time at this project again.

    As an aside, a great link I encountered during my project regarding building the fastest Sudoku Solver (conversation): https://codegolf.stackexchange.com/questions/190727/the-fast...

  • by Sanctor on 8/21/24, 1:24 PM

    Here are a couple attempts trying to solve an NYT hard sudoku puzzle: https://ibb.co/gtJghBP https://ibb.co/9nvHT1P

    1) The half transparent green font color is really hard to read over a white background.

    2) It fails to recongize already filled in squares and tries to change them. The source is computer generated and the image is pretty clear. It seems like a best case scenario for digit recongnition, nevertheless it fails.

    3) It constantly re-evaluates while I'm trying to hold the puzzle up to the camera with varying degrees of success. Too fast for me to get a screen grab when I see a successful attempt among all the failures.

    Cool project though! I was excited to try it :)

  • by NKosmatos on 8/20/24, 1:41 PM

    Hey, thanks for sharing this! I was thinking of creating a visual minesweeper solver as a web app and for sure I’m going to check your notebook ;-)

    ps1. As commented, I don’t have a sudoku available to try this. ps2. Yes, it’s cheating but we’re doing it for learning.

  • by zeograd on 8/20/24, 1:36 PM

    I love the idea but couldn't get it to work.

    When I click the snapshot button from the /viewer URI, I end up on /image and can only download the photo I took or go back.

    What is the intended workflow?

  • by piyushtechsavy on 8/21/24, 12:25 PM

    I would be good if you can also allow upload of an image with camera.