bdon

Hi, thanks for the mention!

The Mapbox repo has an older version of tippecanoe; there's a newer version actively being improved by Felt at http://github.com/felt/tippecanoe , and they've also funded my addition of the Protomaps format (PMTiles) as a direct output. No conversion step required anymore!

Author here - the catch is that the data you store and serve is usually not as rich and complete as Google's, of course.

The solution as described in the post is for serving tile archives, and where you get the tile archive is up to you - I have a commercial offering (and am working on an open source one too), and there's various other FOSS projects for tile creation.

The renderer used in the linked article (MapLibre) has an internal tile cache used when you return to a previously loaded area of the map.

For the non-tiled parts of the PMTiles file, like metadata and the directories that store Z/X/Y to offset information, those are LRU cached by the PMTiles implementation. The most widely deployed right now is for TypeScript/Node. The "client" can either be web browser or inside a runtime like Lambda/Cloudflare Workers. On those the cache will persist across invocations using the ephemeral memory of the serverless function.

A level deeper, browsers vary in how they cache Range requests; IIRC Firefox does not cache while others will treat Range as if it was any other HTTP header.

Yes, PMTiles is a tradeoff that isn't appropriate for transactional use cases. SQLite is pretty good for that already.

There is a throughput limit on S3 files of approximately 5500 GETs/sec per key. Bare archives on S3 is an appropriate choice for small-scale, zero maintenance deployments. If your application demands any thing close to that level of throughput, you're probably either:

* Serving individual tiles over the internet: you should use the CDN integration http://protomaps.com/docs/cdn ; most tile requests will be cached and only misses will interact with the S3 bottleneck.

* Bulk accessing a spatial subset of tiles: You shouldn't be requesting HTTP GETs for single tiles, but instead entire subsets of tiles with a single Range request made possible by the internal Hilbert curve ordering. This is still WIP here: https://github.com/protomaps/go-pmtiles/issues/31

If you have access to SQLite in your desktop application then there isn't any big wins from using PMTiles over MBTiles; PMTiles does have some special design around tile-deduplication but this is possible in MBTiles too by using table views.

There isn't a tool to do that right now. It could be a fit in either https://github.com/protomaps/go-pmtiles or https://github.com/protomaps/PMTiles/tree/main/python - the Go program is faster and more production ready at this point. I imagine if folders are working for you the quantity of tiles doesn't number into the millions, so the Python program might be sufficient.

Feel free to open an issue.

The limitation is of the demo, which uses a publicly accessible asset on cloud storage.

The near-term goal for Protomaps to support z15, which is beyond what most OSM-based solutions provide; there's over a billion addressable tiles for just this one zoom level, and the Protomaps format (PMTiles) is designed with these use cases in mind.

I'll find a better way to communicate this on the site, thanks.

I'm the author of a few of the tools mentioned in this post!

A convenient new development is instead of using tippecanoe -> go-pmtiles to create PMTiles archives, you can now output .pmtiles directly:

tippecanoe -o bks2.pmtiles mainroad.geojson ...

This is available in Tippecanoe (https://github.com/felt/tippecanoe) v2.17 and later.

Thanks to Felt (https://felt.com) for supporting this open source work.

Hi, Protomaps author here (not author of the linked article) - can you link the area you're seeing the rendering issues? Most demos are built using OpenStreetMap data, which should be decent for NZ, so curious what the root cause is here.

(EDIT: if you're referring to the landing page demo, it only displays data to a limited zoom level (10), so I've fixed that to not display the map too far beyond that. Thanks!)

This is interesting!

* Is there a way to access a GOL file from a different language?

* Can a GOL be updated in-place with replication diffs?

SVG will become sluggish past a few thousand features, which is typical for viewing an urban area on the map.

If you want to display infinite fractional zoom levels, WebGL is mandatory. That's what most consumers will expect from a high fidelity map application, because that's the way Google and Apple Maps work.

The barriers to adopting vector-everywhere are social and commercial, not technical.

There are a couple great public raster services like osm.org's default style and http://maps.stamen.com. These are 100% free to use, so they get used everywhere, but incur significant expense to the organizations running (paying) for them.

There aren't equivalent solutions in vector-land yet... I wrote a bit about this previously: https://protomaps.com/blog/free-tier-maps

Because Leaflet is an extensible library for building map UIs using the browser, using DOM elements as grid cells, and using SVG in some cases for overlay geometry and user interaction.

Things that are mandatory for map display, like:

- Label priority and positioning

- Display of internationalized, RTL, and Bidi text

- Geometry rasterization and styling

Are outside the scope of Leaflet's core design and can be implemented by plugins, one of which I have linked in a sibling comment.

Well, I wrote one already for leaflet ;)

https://github.com/protomaps/protomaps.js

TBH, the blocker is not demand for vector rendering but the technical difficulty and performance constraints of doing rasterization in the browser.

A few reasons:

1) Some of the most popular map library frontends like Leaflet support only raster tile layers out of the box.

2) Quality vector generalization (feature-dropping) is much more difficult than raster generalization.

3) A large fraction of use cases just need a "good enough" basemap without those extra features.

For the next revision of the JS decoder I'm including `fflate` as a dependency so clients can decompress gzipped tile data using JavaScript.

https://github.com/protomaps/PMTiles/blob/master/js/package....

The COMTiles design has a very neat idea for batching requests directly on the client due to its ordered layout; this should also be possible in PMTiles spec v3 which introduces an optional Hilbert ordering on disk, as well as index compression and optimizations for run-length-encoding of repetitive tiles. I'm hoping to have this ready and implemented in clients by the FOSS4G conference (late August)

The end goal of PMTiles is similar to SQLite-over-HTTP: SQLite is designed for access on local disk, and might make many requests for a single query, each incurring a HTTP roundtrip (could take hundreds of milliseconds). PMTiles is designed from the ground up for HTTP, so attempts to minimize the # of requests to 3 or less for any tile.

The extra conversion step is something I'm hoping to get rid of by building PMTiles writers into tiling tools directly.

* Yes, the serverless part is a CDN accelerator for any tiled data, as long as you can get your own map data into PMTiles format. I have tested with azure functions but it's not as mature as other platforms yet.

If your several map apps are part of a single organization, it should be fine. I'm happy to discuss this more if you email me at brandon@protomaps.com.

Yep, it's inspired by COG as well as MBTiles, hence the name. Key differences are:

* like MBTiles it's agnostic to what individual tiles are. You can store vector data as SVG, Protobuf, raster PNGs, JPGs or even raw digital elevation models.

* It's not backwards compatible in the same way Cloud Optimized GeoTIFF is.

* It has a recursive index structure to avoid needing to load in huge indexes at once for large datasets (hundreds of millions of tiles).

* It is designed for remote HTTP range reads. If your application can instead directly access a filesytem, like on a mobile phone, MBTiles, which uses SQLite, is a more established solution.

The specification is open source on GitHub https://github.com/protomaps/PMTiles and is still evolving in response to user needs. I've created direct client support for Leaflet and MapLibre, and plan to work on OpenLayers next.

The tiles in the PMTiles on static storage are already pre-generated. However, most storage solutions work in a single region, and don't offer great latency guarantees.

Lambda/Workers performs the computation to populate the edge cache with some vendor specific optimizations, which gets data closer to end users. There's also space to do more advanced filtering or data combination at lambda time that I'm exploring.

Agreed that the pure serverless pricing model may not be the best for all use cases. Future options I'm looking at are Fly.io and Cloud Run, I can help evaluate these for customers. There's a lot of competition and emerging products in this space.

The stack I describe in the post is only for map tiles - Map tiles are a good fit for CDNs because the input space is small (just Z/X/Y coordinates on a square grid) and thus very cacheable.

Geocoding is a very different problem because the input space - human language - is much, much larger, and answering queries quickly to support features like autocomplete really requires a server with hot data in memory.

One of my favorite projects in this space is Pelias https://pelias.io which is an open source auto-completing geocoder based on OSM plus other open data. It's backed by a great team that also runs a business: Geocode Earth https://geocode.earth

I've tested Backblaze B2 as a storage option, and I found that with the level of traffic a typical map backend incurs, there were enough 503 errors to cause problems. This is an intentional trade-off of the B2 design that allows them to set an aggressive price point: https://www.backblaze.com/blog/b2-503-500-server-error/

I'm constantly evaluating all of the different deployment combinations and new features on cloud providers, so part of the service I'm offering is advice for your specific provider and this workload.

Author of post here. One of the cool possibilities here is that Planetiler output is just another dataset, provided it's been converted to the S3-friendly PMTiles archive format (utility here: http://github.com/protomaps/PMTiles)

While Google Maps and MapBox let you customize the appearance of vector maps, you can only modify and remove data that already exists at a zoom level. A self-hosted solution allow products like yours to have 100% custom basemap datasets specific to the application, and serve overlay datasets (like those made with tippecanoe) through the same system.

> What is your definition of End Product here? Eg if we have an app and website under one brand is that two end products or one?

One brand with web + mobile app would be one product. To be more explicit, a software development consultancy with multiple clients would need one license per client.

For updates, you are 100% in control of the data once it is on your S3. This means that it won't change from under you, change pricing, or disappear. This also means you'll have to explicitly copy updates from upstream, and I'm figuring out the right cadence for that (likely ~quarterly)

React users can take a look at this component called Pigeon Maps to accomplish a similar user experience in more React DOM compatible way:

https://pigeon-maps.js.org

Though you miss out on the wide range of Leaflet plugins!

* Leaflet is based completely on the web platform. It achieves a zoomable slippy map experience by animating DOM elements - usually image tiles.

* MapLibre GL JS is a BSD licensed fork of Mapbox GL JS version 1. Instead of consuming bitmap image tiles, it consumes tiles corresponding to the same coordinate extents as the above tiles, but containing tagged geometry features. So instead of sending a PNG image you're sending what is essentially an SVG, but in binary format and with some domain-specific compression.

* The GL JS renderers also rasterize those vector tiles using WebGL, which means that all the geometry data exists in GPU memory and can be scaled, rotated, and recolored etc in realtime efficiently, and also implement their own limited text rendering so even labels can be rotated in realtime.

Which vector-based rendering did you use? I'm working on a new attempt at that here, though not WebGL-based:

http://github.com/protomaps/protomaps.js

so it would be valuable to know if it can get closer to what you wanted to accomplish.

A cursory search of media published in the last month [1][2] shows those can be used to productively communicate in Chinese languages. You can argue that it is outside of what is prescribed in a table, but if not Pinyin, then what writing system is being used here?

[1] http://finance.eastmoney.com/a/202107151999911843.html [2] https://www.marieclaire.com.tw/lifestyle/art/58762