go-pmtiles

I agree, there are tradeoffs to using static storage - the intended audience for PMTiles is those that prefer using static sites instead of administering a Linux server. I would be interested to see a comparison of Btrfs + nginx serving latency, vs `pmtiles serve` from https://github.com/protomaps/go-pmtiles on a PMTiles archive on disk. That would be a more direct comparison. I think there's potentially some interesting use case for tiles in Btrfs volumes and incremental updates, which I haven't tackled in PMTiles yet!

Most of the development for pmtiles has happened in the last 2 years, including a maturing server implementation (http://github.com/protomaps/go-pmtiles) but some key parts are still missing like the ability to decode an archive in native mobile applications. sqlite (mbtiles) already has ~10+ years of integration into the mapping ecosystem so that still works better if you want to move tilesets around to desktop applications and mobile devices. The situation Paul is addressing is one unique to OpenStreetMap itself, which is minute-level updates of a global scale tileset. This is a use case pmtiles is designed explicitly not to address, where using a database is a better fit.

I just used their pmtiles tool to grab a map of just the area around Half Moon Bay, south of San Francisco. I grabbed the latest macOS Go binary from https://github.com/protomaps/go-pmtiles/releases I found a rough bounding box using http://bboxfinder.com/#37.373977,-122.593346,37.570977,-122.... Then I ran this: pmtiles extract https://build.protomaps.com/20231023.pmtiles hmb.pmtiles \ --bbox=-122.593346,37.373977,-122.400055,37.570977 The result was a 2MB hmb.pmtiles file which, when dragged onto the "drop zone" on https://maps.protomaps.com/ showed me a detailed zoomable map, with full OpenStreetMap metadata (airport polygons etc) - all the way down to individual building outlines level. All that in 2MB! I'm really impressed. My file is here: https://gist.githubusercontent.com/simonw/3c1a5358ca8f782f13...

Yes, PMTiles is a tradeoff that isn't appropriate for transactional use cases. SQLite is pretty good for that already. There is a throughput limit on S3 files of approximately 5500 GETs/sec per key. Bare archives on S3 is an appropriate choice for small-scale, zero maintenance deployments. If your application demands any thing close to that level of throughput, you're probably either: * Serving individual tiles over the internet: you should use the CDN integration http://protomaps.com/docs/cdn ; most tile requests will be cached and only misses will interact with the S3 bottleneck. * Bulk accessing a spatial subset of tiles: You shouldn't be requesting HTTP GETs for single tiles, but instead entire subsets of tiles with a single Range request made possible by the internal Hilbert curve ordering. This is still WIP here: https://github.com/

There isn't a tool to do that right now. It could be a fit in either https://github.com/protomaps/go-pmtiles or https://github.com/protomaps/PMTiles/tree/main/python - the Go program is faster and more production ready at this point. I imagine if folders are working for you the quantity of tiles doesn't number into the millions, so the Python program might be sufficient. Feel free to open an issue.

I don't think so. There's this: https://github.com/protomaps/go-pmtiles to get from MBTiles and there's a few xyz folder tree tools to get mbtiles from that structure (e.g., https://pypi.org/project/mbutil/). So you can get there, but I don't know of a good straight-line right now.