Show HN: Self-Hosted Maps Stack | Hacker News Reader

Show HN: Self-Hosted Maps Stack

maps.ellenhp.me

213 pointsellenhp66 comments

Over the past week I built a project to let people self-host an entire maps stack so they don't have to send data to the big G. Right now it includes a base map, geocoder and directions server. Currently only bicycle directions are supported, and I'm only hosting tiles for Seattle because I'm unemployed and can't afford to host data for the whole planet. Check it out!

https://maps.ellenhp.me/

https://github.com/ellenhp/headway

Ultimately I'd like to really focus on transit routing (not implemented) but I'll probably throw a driving mode in there too.

Open articleView on HN

What's the advantage compared to using https://www.openstreetmap.org/ and Organic Maps, or self-hosting them? (the website is https://github.com/openstreetmap/openstreetmap-website and the tile server is https://github.com/gravitystorm/openstreetmap-carto/)

Mostly just that you can self-host this stack and keep your location information private (the way it should be). If you're fine sharing your current and future whereabouts with people then there's no need to self-host so you're free to pick whatever service is the nicest to use. For most people that's going to be Google or Apple maps, but there's definitely a niche for a project like this.

Compared to self-hosting an existing project, it is much easier to bring up a new Headway instance:

`make Seattle.up` will bring up a new instance on 8080.

OrganicMaps and OsmAnd operate entirely on local map data and don't send your information to remote servers.

Didn't realize that about Organic Maps. It also appears to be open-source which is cool. Is its geocoder any good? Search in OsmAnd is painfully slow and not tolerant of typos at all. Headway uses photon as a geocoder autocompletion system which is incredible and makes the experience miles better than I've ever had with OsmAnd.

I love the idea of offline maps but I've never used an offline maps app with a decent search experience. I also aim to make Headway very transit focused which will allow it to serve real-time data about transit vehicle locations from GTFS realtime feeds.

edit: I just tried out organic maps and it is GOOD. Way better than OsmAnd. I still want to keep working on Headway but this will be useful in the meantime.

Looks like it uses OpenStreetMap data. Please ensure attribution is displayed as per the license terms.

See https://www.openstreetmap.org/copyright

They already do that.

It is hidden under the little ⓘ in the bottom right corner. That is a very meagre interpretation of the attribution guidelines. To help combat the idea that a map must be made by some megacorp to be viable, visibility of OpenStreetMap is important.

ellenhp: Why not start with the attribution toggled open, or some alternative way to highlight the map data's provenance?

edit: Attribution is now shown toggled open at page load, resolving this issue.

This seems resolved. Attribution toggles open when its loaded now.

Huh, it does. That was impressively fast.

Where you viewing it on a small window/screen the first time? This uses mapbox or maplibre for map viewing and it defaults to collapsing the attribution on small screens: https://docs.mapbox.com/mapbox-gl-js/api/markers/#attributio...

No, nearly full screen on a desktop.

Please open an issue with upstream (maplibre-gl-js) if you're not happy with the attribution that it provides by default. On my end I need to do better to make sure that it's never covered up. I'm really bad at CSS so that's going to be a fun time but obviously needs to be done. There's an issue open to track it now.

Seems to be hit and miss on mobile web (I can trigger the 'i' to show but don't see it in all views).

Currently travelling so will investigate further when somewhere appropriate (may be cached on my phone, something specific to mobile web, etc).

I'd really love feedback and suggestions by the way! I have a fair amount of experience in maps but I've never just built an entire maps web app from scratch so I'm kind of stumbling through a lot of it, especially the web frontend stuff.

It'd be really great to be able to know what's in the docker black box. It'd be even better to be able to know what the moving parts are for those of us that do self hosting sysadmin style. My VMs are too small for docker... but i'm sure the underlying stack would run without issue.

maps.ellenhp.me has only a IPv4 address and it seems to be hosted on digitalocean, which supports IPv6.

would love to have IPv6 support. It should be on default by now, if there aren't any issues hindering support.

For a full multimodal routing engine including realtime data for transit, you should have a look at OpenTripPlanner version 2 (if you haven't already): https://www.opentripplanner.org/

For a single solution to geocoding and autocompletion, there's Pelias: https://pelias.io/

If you only want routes without multimodal you can use Graphhopper https://www.graphhopper.com/

Or alternatively OSRM? https://github.com/Project-OSRM/

Pelias is pretty difficult to use with the kind of automated config I'm hoping to have for this project but I might give it another go.

OTP is actually really fantastic and it might be a better choice than valhalla's multimodal especially if it can handle GTFS realtime. Thank you for the reminder that it exists!!!

Nice! I was going to point out the project I’ve been working on [1] to reduce the cost but I see you’ve already found it :-) Feel free to reach out it you run into any issues!

[1] https://github.com/onthegomap/planetiler

Planetiler is amazing! Thank you for your work.

Am I missing something, or does it only cost about $2.27 per month to store all the tiles in Amazon S3? They're definitely not the cheapest game in town.

(Your build example says the tiles are 99GB, and their normal tier is $0.023/GB)

I guess the part of the cost that people actually care about is the bandwidth. How do you suggest dealing with that part of the problem?

Planetiler isn't for hosting tiles, but generating them. For a planet-scale tile generation job it's the difference between hundreds or thousands of dollars in VPS fees and being able to generate them locally on a modestly equipped desktop. Planetiler does drastically reduce cost in that respect.

Another option if you want to offload the tiles from embedded in your docker image to static file hosting would be to convert the mbtiles output to pmtiles [1]. Then just upload the pmtiles file to s3 or similar and use the maplibre plug-in to read tiles directly from that large file using byte range requests without a tile server.

There’s a python utility to convert mbtiles to pmtiles, but I’m also planning to add native pmtiles output to planetiler soon [2].

[1] https://github.com/protomaps/PMTiles

[2] https://github.com/onthegomap/planetiler/issues/98

This is great! If you're interested, I'd love to have you at a GIS/mapping-focused retreat I'm hosting at https://www.gradientretreat.com/ this summer. You can reach out privately at avi@avibryant.com.

Is there any way to write commands where I could give my own time weighting to certain types of roads for route finding?

Google maps, organic maps, osmand+, I still have to double check what crazy suggestion it has made. Single track gravel roads over a perfectly safe alternative a lot of the time.

I have the avoid unpaved roads options ticked... it would be great to have some machine learning magic take a look at a street view picture and decide if there is grass growing up the middle of the road and add a weighted chance of needing to reverse 200 metres when there is other traffic

Very nice!

I haven’t looked into the code yet, but I’m especially interested in the routing bit.

As a cyclist I’m almost always disappointed by google, apple, and Valhalla. The cycling directions from these systems are usually better than nothing, but almost always have one or two dangerous or ridiculous segments.

I’ve never tried to solve it myself, only day dreamed.

Any insights?

You might want to look into https://cycle.travel/ (my site!), https://brouter.de/brouter-web, Komoot or RideWithGPS - all do OpenStreetMap-powered bike directions.

Getting bike routing right is really challenging, partly because often there genuinely isn’t a good route, and partly because standards and infrastructure (and OSM mapping practice!) differ from country to country. I love the challenge and the feedback from users when it works well, but it’s a massive task.

I just did a quick test of cycle.travel using the last couple trips I took, and it’s pretty good!

It’s anecdotal, but definitely better than what I’m seeing with Val halla / google / apple.

Can you share any details about how your route finding works?

> As a cyclist I’m almost always disappointed by google, apple, and Valhalla

Have you tried the OSRM bike config? (The one you find in https://github.com/Project-OSRM/osrm-backend/blob/master/pro...)

Headway uses Valhalla. Most of what I'm doing with this project is the config to bring up the entire stack easily.

I self host OSM by using OSMAnd+ on my phone and downloading the maps of the areas I move to. I'm not particularly interested in directions so I don't know if they work well. It's missing all the satellite and street view layers and they are the main reason I often go to Google Maps.

OSMAnd/OSM data varies a good bit by region. In the US, missing addresses are often the only significant issue with directions (and for OSMAnd, no live traffic data).

> I'm not particularly interested in directions

What's your use case?

I use it build new tracks for cycling and record them. I explore the map (the OSM road type layer and Google satellite view), place markers about 10 meters after the intersections on the road I must take, follow them. I use my old and smaller phone on a phone holder on the handlebar. The screen is almost always turned off. I turn it on when I know I'm about to need to know where to go. I usually remember the places the next time I cycle on the same road.

Navigation when driving my car: not every month, not many times per year. After all I know the roads, I don't need somebody to tell me where to turn. When I do, it's usually the last km, when the chances to miss an intersection are higher. On main roads or close to home, not so much.

You can enable the "Microsoft Earth" layer for sattelite imagery in OsmAnd, which is (at least in my region) even better than Google's. I have it set as the "underlay map", which means it's drawn on top of OSM background elements, but below roads, buildings and markings. It also gives you a slider right on the map view to adjust the opacity of the OSM elements so you can see just sattelite imagery when you need it or even set it to somewhere in between.

As for Street View, while I haven't looked into getting it into OsmAnd, Mapillary has an official plugin and while its coverage isn't really comparable, it does often have newer images and often saves me an app switch if I just need to see anything in a general area.

Thank you. I didn't know about those layers. Very useful!

I did know about Mapillary. I'll check if its coverage got better.

I’m working on an app that will extensively use a variety of maps. Did you directly interact with OSM (it’s like a big local DB of map tiles, right?) or did you use any important helper libraries? What are the best resources you used to learn about building city level maps like this?

Lots of it is stuff that I've picked up over the years. I work(ed) in the maps industry for the last 4 years or so, until I was laid off about a week and a half ago. I do interact with OSM a little bit in the docker image generation but the servers (geocoder, tileserver, routing server) all pre-process their data to turn it into a format that's useful for each server's specific purpose. So none of the HTTP requests that my demo server is responding to right now actually directly touch any OSM data.

Most of the OSM data processing is done in the Makefile, but beware that it was very hastily written so it's not a perfect example:

https://github.com/ellenhp/headway/blob/main/Makefile

Headway is a great name for wayfinding! Looking forward to this evolving.

How much money to add transit directions ?

I would absolutely love if this supported an API to get directions. I've worked on a few projects with public transit directions and I'd love to use a common base instead.

I'm planning on tackling transit this weekend!

God Speed, feel free to put up a donation page. I'll definitely donate !

I think this was the push I needed to apply for Github Sponsors. Keep an eye on the project though, I'd really love feedback from people passionate about transit! Feel free to email me too if you have ideas of what the UX should be. ellen.h.poe@gmail.com

In the chance you're open for cryptocurrency donations/support, throw in a BTC, XMR, and/or ETH address and you should see some coming

(Some of us can't or won't use GH sponsors, paypal etc)

FWIW I'm super hyped about transit/walking/biking/driving directions as well.

Minimize left turns? :D

Neat, where are you going to source your data from?

The plan is to use this dataset [1] and do a bounding-box intersection with the chosen city to automatically download appropriate timetables when the user runs `make CityName.up`

[1] https://database.mobilitydata.org/

Is it possible to use alternative map data source?

I'm looking for a way to generate topographical maps...

Or may be you can suggest any alternative project for that?

These things are way more difficult than they should be, really. As evidenced by the effort needed to put this together.

Been wanting to do similar but stopped once I saw just how much data hosting the entire world map takes.

Some interesting work is being done to make this affordable (storing a single large planet file in S3 to limit upload costs and using byte-range offset requests to fetch individual tiles) [1]

[1] https://protomaps.com/docs/pmtiles/

Hello from a fellow map geek building Shademap[1] in Bothell. Excited to see where your project goes.

[1] https://shademap.app

Email me. I'll give you a server to further development.

All the good open source tile servers have pivoted into commercial businesses, and I'd like to support the development of a project that stays open.

That sounds wonderful! This project is primarily focused on people who want to self-host a google maps replacement, but self-hosting isn't for everyone so I think it would be really cool to host an instance that would work for most people. I'd love for it to grow beyond that though, because lots of people need tiles for static maps or other uses. I will need to re-architect the docker config a bit before it will work at planet scale, because right now mbtiles are bundled with the tileserver image.

What about openmaptiles.org?

They don't host anything IIRC. But most of the FOSS maps ecosystem is based around their schema. I know Planetiler uses it at least.

They turned into MapTiler, a commercial service.

I wrote one years ago, I just put it in a gist:

https://gist.github.com/raggi/d5c8cad1d3c7b25c6748d8322d3c14...

happy to share experiences, etc.

I guess I should mention, I have Go code for gtfs too, if it's of interest I can extract that too and get it up somewhere.

My friend Alexander created an easy docker file[1] for an OSM tile server. I use it for nearly two years without problems.

[1] https://github.com/Overv/openstreetmap-tile-server

Fully open:

- https://github.com/systemed/tilemaker (for generating tiles)

- https://github.com/systemed/mod_mbtiles (for serving them)

tilemaker is similar to planetiler which OP is using (it's more configurable but also more memory-hungry).

One might not need tiles for personal uses, though. If you're not intent on scaling to the level of Google, a tile server seems almost like an overkill. And on some platforms, such as Windows, UMN MapServer is massively easier to install in my experience (not to mention its flexibility).

I know you're focused on self-hosting websites on the internet, but I have another, closely related problem.

I have a degoogled Android phone, and location services are one of my bigger frustrations with it. The phone has more than enough room to store the entire planet's maps locally, which means some microg-style replacement for google's location services should be able to compute location name from GPS coordinates, and also pull up map tiles without hitting the network. Both those features are super flaky, especially if the network is down.

This is particularly frustrating because I have Organic Maps installed, and it has already downloaded the necessary databases!

OSMand can run entirely offline with OpenStreetMap data files. It downloads a base (country-wide-ish), state/region map, and then optionally smaller delta updates published between more significant updates.

Reply on news.ycombinator.com