from Hacker News

Exploring Databases Visually

by dmfay on 4/4/21, 11:19 PM with 27 comments

  • by chunkyks on 4/5/21, 12:08 AM

    I implemented a DB visualisation tool like this for sqlite3, also using graphviz. It's called, unimaginatively, sqlite3todot [1]

    The couple tricks it has that I really like are a) it shows the table itself [which isn't uncommon], but also b) has the ability to group up tables, which for me turns out to be a devastatingly killer feature that no other tool has.

    When I'm working on a schema [especially one I've not seen before], one of the things that drives me insane is not being able to say "these six go together. Group them". Then you can simply look at arrows that enter or leave that grouping for your next exploration step. By the time I'm done, anything not-categorised stands out [it's simply in the space between groups]; any boxes that have an excess of arrows between them can mean my mental model is wrong [or the schema has some nonoptimalities].

    There's an example here; it only has two groupings ["input" and "output"], but it immediately gets the point across: https://icculus.org/~chunky/stuff/sqlite3_example/sqlitesche...

    [1] https://github.com/chunky/sqlite3todot

  • by simonw on 4/5/21, 3:51 AM

    I've been doing something similar to this using an Observable notebook - this one joins the sqlite_master table against the pragma_foreign_key_list() SQLite function and uses the returned data to show a DOT diagram:

    https://observablehq.com/@simonw/datasette-table-diagram

    It defaults to showing the tables from my GitHub example database but you can paste in the URL to another Datasette to see that one visualized instead - try https://latest.datasette.io/fixtures for example

  • by tracyhenry on 4/5/21, 3:26 AM

    As part of my research, I'm also building a new UI for exploring a database using data visualizations and an ER diagram. Here is a half-baked demo: http://mondial.kyrixdemo.live/

    Hopefully the UI is self-explanatory - if not try clicking on visual objects to perform drill down, or search something in the top righthand corner.

    The underlying dataset is a public DBMS called MONDIAL: https://www.dbis.informatik.uni-goettingen.de/Mondial/mondia...

  • by meesterdude on 4/5/21, 2:45 AM

    I'm building a 3D database that lets you explore graph and relational data that you've created. For research or creating your own mind palaces - you can freely join & filter collections of information. But it's more like a 3D excel of sorts.
  • by pm90 on 4/5/21, 12:22 AM

    Isn't this what schemaspy does? http://schemaspy.org/
  • by _wldu on 4/5/21, 1:09 AM

    I love graphs (it was my favorite topic in algorithms) and I think they can be applied to many subjects to help us understand and visualize problems and systems better. In particular, I think graphs could really help network security monitoring efforts to quickly identify anomalies.

    Also, graphviz is really cool.

  • by JoelJacobson on 4/5/21, 1:20 PM

    Foreign Keys are great!

    Too bad they were not invented until SQL-89, and could thus not be considered when the JOIN syntax we all know was designed and formalized by ANSI in SQL-86.

    Imagine if we could simply give the name of a foreign key, as an argument to JOIN, letting the query planner look up the foreign key columns, to do the join on.

    The default names for foreign keys are not very user-friendly, due to the SQL standard unfortunate decision to require the foreign key name to be unique per namespace, and not just per table, which would have made more sense.

    In PostgreSQL, foreign keys just need to be unique per table, so there we could give them names such as "customer" instead of "rental_customer_id_fkey".

    Imagine if we instead of:

      SELECT
        language.name,
        COUNT(*)
      FROM rental
      JOIN customer
        ON customer.customer_id = rental.customer_id
      JOIN address
        ON address.address_id = customer.address_id
      JOIN city
        ON city.city_id = address.city_id
      JOIN country
        ON country.country_id = city.country_id
      JOIN inventory
        ON inventory.inventory_id = rental.inventory_id
      JOIN film
        ON film.film_id = inventory.film_id
      JOIN language
        ON language.language_id = film.language_id
      WHERE country.country = 'Sweden'
      GROUP BY language.name
    
    Would write something like:

      SELECT
        language.name,
        COUNT(*)
      FROM rental
      JOIN rental->rental_customer_id_fkey AS customer
      JOIN customer->customer_address_id_fkey AS address
      JOIN address->address_city_id_fkey AS city
      JOIN city->city_country_id_fkey AS country
      JOIN rental->rental_inventory_id_fkey AS inventory
      JOIN inventory->inventory_film_id_fkey AS film
      JOIN film->film_language_id_fkey AS language
      WHERE country.country = 'Sweden'
      GROUP BY language.name
    
    After renaming the foreign keys:

      ALTER TABLE rental RENAME CONSTRAINT rental_customer_id_fkey TO customer;
      ALTER TABLE rental RENAME CONSTRAINT rental_inventory_id_fkey TO inventory;
      ALTER TABLE customer RENAME CONSTRAINT customer_address_id_fkey TO address;
      ALTER TABLE address RENAME CONSTRAINT address_city_id_fkey TO city;
      ALTER TABLE city RENAME CONSTRAINT city_country_id_fkey TO country;
      ALTER TABLE inventory RENAME CONSTRAINT inventory_film_id_fkey TO film;
      ALTER TABLE film RENAME CONSTRAINT film_language_id_fkey TO language;
    
    We could write this as:

      SELECT
        language.name,
        COUNT(*)
      FROM rental
      JOIN rental->customer
      JOIN customer->address
      JOIN address->city
      JOIN city->country
      JOIN rental->inventory
      JOIN inventory->film
      JOIN film->language
      WHERE country.country = 'Sweden'
      GROUP BY language.name
    
    And if allowing such a "foreign key operator" to be chained, we could write:

      SELECT
        language.name,
        COUNT(*)
      FROM rental
      JOIN rental->customer->address->city->country
      JOIN rental->inventory->film->language
      WHERE country.country = 'Sweden'
      GROUP BY language.name
    
    This is similar to "4.9 Reference types" in the SQL standard ISO/IEC 9075-2:2016(E), but it wouldn't require a separate REF column, it would merely use the existing foreign keys which we already have in well designed proper database schemas. We would just need to give them better names.

    Q: How would we name foreign keys if there are two going to the same table?

    A: Imagine having a "users" table with two columns "child_user_id" and "parent_user_id", both referencing "users". The foreign keys on such columns could simply be named "child" and "parent".