from Hacker News

Show HN: Flatmap – a new tool to make vector tiles from OpenStreetMap data fast

by zylepe on 10/25/21, 11:52 AM with 28 comments

  • by zylepe on 10/25/21, 11:53 AM

    Creator here. I maintain https://onthegomap.com as a side project. I originally used Google Maps API but after their price hike switched to OpenStreetMap data. I self-host GraphHopper for routing and tried to self-host OpenMapTiles but gave up and used Stadia Maps since it would have taken over 100 days [1] to generate a map of the world myself. Stadia Maps has been great and I highly recommend them, but the problem continued to nag me, so earlier this year I started prototyping a new way to generate vector tiles faster. The idea worked and ended up being able to generate a ~100GB planet.mbtiles file in as little as 59 minutes on a single c5ad.16xlarge instance with 64 cpus and 128GB RAM (although smaller machines appear to be more economical).

    I’m open-sourcing Flatmap today. Let me know what you think, or if you have any suggestions!

    [1] https://github.com/openmaptiles/openmaptiles/issues/654#issu...

  • by stevage on 10/25/21, 9:00 PM

    Looks great! Potentially a game changer for projects that need a basemap for a smallish area (like one country), get a fair bit of traffic, but don't have budget for ongoing tile subscriptions to Mapbox or MapTiler.

  • by tony_cannistra on 10/25/21, 8:15 PM

    This is really sweet, awesome work. I really love onthegomap too.

    I maintain an openmaptiles pipeline, and we've got our planet runtimes down to about ~4 days on AWS.

    Would be excited to see if we could use this tool, but we really heavily depend on PostGIS data manipulation to ensure that the data we want ends up in our output tiles. How difficult do you think migrating this SQL to your tool would be? It seems like you can support modifying the data fairly heavily, both before and after MBtile creation.

  • by RicoElectrico on 10/25/21, 1:01 PM

    Nice :)

    There's an ongoing effort to research vector tile solutions for openstreetmap.org. I wonder how your software stack compares to tilemaker, which has seen some optimization lately.

    https://github.com/openstreetmap/operations/issues/565

  • by lalaland1125 on 10/25/21, 7:50 PM

    Nice tool OP, but I think we as a community should try to avoid creating tools that use already common names.

    Googling "flatmap" gives a bunch of bad results. There is a value in having unique names.

  • by MattBlissett on 10/25/21, 11:01 PM

    This looks great!

    I hacked around with OpenMapTiles to generate vector tiles in Arctic, Antarctic and WGS84 projections (the latter with 2×1 tiles at zoom 0) [1]. The time spent waiting for PostGIS to do reprojections etc made development very slow, and I've put off updating the tiles.

    This tool is reprojects from WGS84 to Web Mercator while reading the tiles, so optimistically changing that would be much of what's needed.

    [1] See the map at the bottom of gbif.org, there's a button to change the projection

  • by willcodeforfoo on 10/25/21, 8:08 PM

    Thanks so much for this tool! Looks like one could generate a planet with less than $5 on Digital Ocean.

  • by epaulson on 10/25/21, 10:03 PM

    A bit off-topic, but do people who host their own map tiles worry that other apps will use/abuse their tileserver? I would worry that I'd come in one Monday morning and discover that I've got a $20K bandwidth bill because someone else got popular.

  • by ryantgtg on 10/25/21, 7:54 PM

    We self-host OpenMapTiles, but are super lazy about replacing them with updated tiles (and we don’t love the watermark). I’m really glad that there are options for generating them ourselves! Starred, and will give it a try soon.

  • by durkie on 10/26/21, 1:57 AM

    at that cost it almost feels like something internet archive or aws open data could provide on a regular basis. the cost of pre-made vector tiles is huge, and making them yourself has been a huge ordeal before this...

  • by jmnicolas on 10/25/21, 10:08 PM

    > at least 1.5x as much free RAM as the input .osm.pbf file size

    Ouch! Is there any plan to reduce the amount of ram or it's impossible?

    I'm currently building a tile server with Europe's map, the PBF is 28GB, I have 32GB ram.

  • by tschesnok on 10/25/21, 10:26 PM

    This is cool. I will dig in deeper and take a look. I'm working on a spherical display.. so maps with fast zoom will be a must.

  • by reichardt on 10/25/21, 7:36 PM

    Fantastic work! Are the vector tiles compatible with OpenMapTiles styles?

  • by tantalor on 10/25/21, 7:08 PM

    Clever name... but not exactly unique.