Dataset/style design 3.0

[mmertsock]

I’ve been thinking for a while about simplifying the Mapbox dataset and style schemas. I have some concrete thoughts on what a new dataset and style schema will look like, but I need to take some time to write that up. Might be a little while before I can post something, but meanwhile I wanted to start a general discussion on the path forward, with these goals in mind:

• Streamline the workflow for adding and editing map data
• Eliminate some clutter and unused/outdated features
• Facilitate development of the next versions of the apps for mobile and web platforms

Overall, I would prefer to have a simpler map if it means it’s updated more frequently. 80% of the small details in these maps/apps are probably not being used by 80% of users.

Some more specific ideas follow. Forgive me if some of this gets into the weeds a bit, I’m coming back to this after taking a year or so off of the project so my thoughts aren’t too coherent yet.

Simpler data schema and editing workflows

The Mapbox dataset was originally generated using a series of scripts that migrated data from the old JSON+GPX file formats. A lot of the data is sort of “denormalized”; for example every trail segment feature has a lot of properties like “trailsroc-color”, “trailsroc-name”, etc. This was done so because of the way Mapbox Styles work: a style renders a given map feature, using the properties present on that individual feature. There was no way to have, e.g., a trail or park name stored in a single shared feature that you could then reference when styling all of the trail segments or points of interest related to that trail/park. Perhaps that has changed with the past couple years of Mapbox updates.

When I first implemented the Mapbox maps, there was no way to do indirection. There have been plenty of enhancements to Mapbox since then, so perhaps there are ways to consolidate some of this data into a single feature, and then look up that feature by ID or something in the style properties, so that you don’t have to duplicate a bunch of properties across dozens of trail segments. This would reduce the amount of work required to add and edit map data.

Some concrete ways to simplify the editing workflow:

• Eliminate some data and styling entirely. For example the discussion of trail colors below would simplify trail segments and should make it easier to add/maintain trail data. There are probably other things that have proven to not be sufficiently interesting/useful data and we could eliminate?
• Last I tried, editing the actual coordinates of trails and other line data in Mapbox Studio was inefficient. When adding new data, I still preferred to edit GPX files in an external tool (Adze) and then run a script to convert to the correct GeoJSON format and import into Mapbox. Is it still painful to edit large amounts of trail data in Mapbox?
• Even if we don’t find a perfect data schema that makes everything easy to edit/obvious to understand, perhaps just getting some clear documentation and tools/scripts will help

Trail segment data and styling

Trail data is still largely an evolution of the original 1.0 iOS app, which was based on the Monroe County maps. A “trail” is a collection of trail segments, with a name and color. The names and colors are based largely on the Monroe County map data, e.g. the East Esker Trail at Mendon Ponds, which almost everybody knows as the “green loop”. Ideally the colors match the actual blaze colors. This coloring features prominently in the map design and user experience. However:

• Blaze colors change over time, and not every park or trail system has consistent blazes or any at all. I’m certain the current map data is out of date or inaccurate for some of the parks
• Many rail trails, etc., have no real blaze color (e.g. Lehigh Valley). The color on the map is arbitrarily chosen, the only purpose it’s serving is to identify what segments are part of that trail and distinguish it from nearby trails
• Some trails don’t have a color blaze but do have markers using a logo, e.g. Black Diamond and I think Auburn trails
• There are a lot of unmarked trails that are currently just gray lines with called “unnamed trail” on the map

Is this focus on color still useful? Perhaps we could have the base styling — color, line width, dash pattern, etc. — signify something else: surface type, a hierarchy of major/minor trails, etc. I think this would communicate more useful information to the public, and would be easier to keep it accurate.

For the subset of trails that do have reliable blazing, we could use “symbols” in Mapbox to show this separate from the line styling. This would show up as a pattern of squares or other shapes with an appropriate color; it may even be possible to use custom SVG symbols for the rail trails that are marked with logos.

When a user selects a trail segment on the map or chooses a trail from a list/search view, we could highlight all the segments of whatever named trail or trail system it’s part of, if any.

The current data schema assumes a pretty rigid 1:N relationship of Park/Trail System to Trails, and then 1:N of Trails to Trail Segments. Perhaps we flatten/decouple these relationships, which I think would greatly simplify things, make it easier to handle special cases, and give some room for new functionality. I have some concrete thoughts on this that I’ll write up later.

Other map features to change/eliminate

Thoughts?

New functionality?

Anything new that we should consider supporting?

[mcbeth]

Most of the comments we get are about blaze colors. I probably find that the most useful part of the whole app, so while updates are a pain, that may be the most important part of what we do. If we want to represent that some other way, then the ideal is still that the color is immediately obvious.

Once I get around to getting Dryer in, everything is going to be Green/Blue/Black :)

I really liked when I got to the point that MapBox was just the place I uploaded files to rather than having to deal with their ever-changing editor.

I tolerate GPX as the core datasource, and would be quite hesitant to switch to MapBox's editor (GUIs and Data Lock-in are evil). I'd also buy in to us storing everything as GeoJSON, or maybe GeoPackage. I like the text based formats for their scrutability and classic UNIX hackability (GeoPackages I think fail that test, but allow more expressive features without buying into the SHP file nightmare).

[mmertsock]

Most of the comments we get are about blaze colors. I probably find that the most useful part of the whole app, so while updates are a pain, that may be the most important part of what we do. If we want to represent that some other way, then the ideal is still that the color is immediately obvious.

Ok, it's useful to know that. So the goal could be more consistent presentation of color information. I think the biggest problem in that regard is the assignment of arbitrary colors to trails that have no actual color blazes. But making everything that default grey color would cause the more important no-color trails (such as rail trails) to become indistinct from small side paths. So perhaps we:

1. Remove the abritrary color assignments from no-color trails
2. Introduce another form of visual hierarchy to distinguish major trunk/loop/through/named trails from secondary trails. Wider vs narrower line width could be a good choice. Can also use this to set a different zoom level threshold for visibility, so the major trails persist longer as you zoom out, and medium zoom levels are less cluttered
3. Then later we can consider other trail identification enhancements such as logo markers

Step 2 would likely require a new geojson attribute to render this correctly in the Mapbox style. We could bulk-initialize the existing data by using a heuristic, e.g. a good start would be to set trailsroc-hierarchy = 1 for anything that has a trailsroc-name attribute, and set it to 2 for everything else. And then manually edit as needed.

I really liked when I got to the point that MapBox was just the place I uploaded files to rather than having to deal with their ever-changing editor.

I agree about avoiding reliance on the Mapbox Studio editor. Most of my editing (though clearly not all) has been offline followed by bulk import of geojson content into the Mapbox dataset. And it looks like you've worked out a workflow for all of that, which solves one of my concerns, of my personal workflow and scripts not being useful or portable for others.

[mcbeth]

I think your path forward makes sense. It may be worthwhile to consider how we would want to handle special routes (10k loop in Mendon, race routes, etc) with these changes. As for the hierarchy... It may be worthwhile to also hit everything with the grey as a 2 to start.

Workflow:
I've something that works for me, the main thing I'm relying on from you is the make_geojson.rb and geojson-merge. The second makes me very cranky, as anything Node related makes me very cranky, but that may be a "get off my lawn" problem. QGIS doesn't like editing GPX (loads just fine), so I've been converting to GeoPackage and editing there, then reexporting, which is silly, but works for now. I got some comfort with QGIS, since it can use Mapbox as a background, then I can just overlay other tracks to draw crappy programmer-art maps for the #TrailsRoc races.