Adding Mapbox vector tile basemaps

Hi!

First, an update on the code. @langstonsmith, the new MapFragment interface is now on the master branch, and it is in use by the newly consolidated GeoTraceActivity and GeoShapeActivity. So I think it is mature enough to be ready for new implementations. There are existing Google Maps and OSM implementations (GoogleMapFragment and OsmMapFragment) that you can use as examples if you'd like to start building a MapboxMapFragment implementation.

Note that the configuration settings to select the basemap are handled by MapHelper, which has not yet been refactored. My plan is to move the functionality of MapHelper into each of the appropriate MapFragment implementations, so my advice to you is to keep your changes to MapHelper minimal (e.g. just hardcode a vector tile source for now) and focus on implementing MapFragment. Feel free to reach out to me any time if you have questions.

Second, I just want to clarify the question of online vs. offline tiles. @langstonsmith, is your plan to implement online vector tiles only, or will there be support for both online and offline vector tiles? What would be involved in making it possible to display offline vector tiles that are packaged and preloaded onto the mobile device's filesystem in advance, as @paul.uithol describes in his comment?

@paul.uithol, can you elaborate a little on what you meant in your first bullet point, "defining a 'map view' that can use configurable vector and raster sources"? At the moment, ODK Collect already has configuration settings such that:

  • You can choose the Google map or OSM map
  • You can choose which standard basemap style you want (e.g. satellite, terrain, etc.)
  • You can choose a raster tile file from the device's filesystem (an mbtiles file)

If @langstonsmith then adds a Mapbox implementation, we'd have a third option in addition to Google and OSM, and presumably there would be some configuration setting for choosing the vector tile set to use as the basemap.

Does that meet your requirements, or can you help me understand how what you have in mind would be different from this?

Thanks!

3 Likes

Awesome. Amazing update @zestyping. Thanks.

I will try to look at the master branch in the next couple of days and see what can be done to add Mapbox. My mind is definitely thinking about online tiles as a start.

Using Mapbox offline vector tiles largely revolves around declaring and downloading certain desired region(s). The region could be declared ahead of time or chosen by a user. The Offline Plugin for Android helps with this.

Anyways. More to come.

2 Likes

Hi @langstonsmith, how's it going?

@paul.uithol, any further thoughts on the questions in my last comment above?

@zestyping, busy couple of weeks of work travel. I re-based my branch with master to pull in the #2465 refactor and all of the other changes. Lots of conflicts but I've cleaned them up at this point :slightly_smiling_face:

There are existing Google Maps and OSM implementations (GoogleMapFragment and OsmMapFragment) that you can use as examples if you'd like to start building a MapboxMapFragment implementation

so my advice to you is to keep your changes to MapHelper minimal (e.g. just hardcode a vector tile source for now) and focus on implementing MapFragment

The Mapbox Maps SDK already has a SupportMapFragment :muscle: The state of my pr is that I've created a MapboxMapFragment that:

  • extends SupportMapFragment
  • implements the MapFragment interface

Many more tweaks are needed so that data is shown properly on a Mapbox map, depending on the specific widget. But yea. My pr now has your changes and things still look possible/promising.

I'm still not at a point where I want to open the pull request, but here's a link to a collection of GIFs which show the current state of my work. Vector tiles are working well :ok_hand:

I'm still looking into how offline can be supported via Mapbox and what the Mapbox Maps SDK can handle (related ticket). Seems that the current MBTile format that Mapbox spits out, still can't be processed by the app. I'm planning to ask some colleagues some clarifying questions, because this is new territory for me.

In my opinion, some things that have come up in this thread, such as some of @paul.uithol 's suggestions, are future feature requests, rather than blockers for my specific additions.

I will definitely have Marena continue running point on Mapbox access token and what makes sense for the ODK community.

@zestyping the mapping interfaces are great! Based on map UI interaction and device location listeners, it seems that I've hooked Mapbox into everything correctly.

Happy to keep the questions and conversations going here, but I'm feeling like I'm coming into the homestretch of the pr. Continued offline investigations, checkstyle cleanup, and testing, seem like the final 3 remaining things.

4 Likes

Wow, these screen video GIFs look fantastic! Beautiful work!

I'm pleased to hear that it seems like the MapFragment interface is working out for you—did you have to make any changes to get it to play well with your implementation classes?

I also noticed GeoPoint among your GIFs, which is the one class I hadn't yet migrated to use the new MapFragment interface. (I have some work in progress on that, but haven't pushed that branch yet.) How did you do that—did you do your own migration of GeoPoint to use the new interface?

Not many. Making tweaks based on an already-working copy of the GoogleMapFragment was a great way to start :slightly_smiling_face: As I said above, the Mapbox Maps SDK already has a SupportMapFragment, so most of the work was adding in our own LocationEngine stuff. Other work was doing simple package name swapping. For example, swapping the Google Maps Polyline class for the Mapbox Polyline class/package.

I also noticed GeoPoint among your GIFs, which is the one class I hadn't yet migrated to use the new MapFragment interface. (I have some work in progress on that, but haven't pushed that branch yet.) How did you do that—did you do your own migration of GeoPoint to use the new interface?

Yea, I just copied the Google Maps GeoPointMapActivity and made adjustments for a Mapbox equivalent. There aren't a ton of methods, so it didn't take very long.

1 Like

Hi everyone - very exciting to see how this is moving! I have a proposal for how to handle the question of access tokens.

For the official ODK Collect app (available in the Play Store)

  • We set up an ODK Mapbox account with @yanokwa @zestyping and use the token from that account for the Play Store app. This token isn't publicly published to Github and won't carry over to any clone/fork of the ODK Collect repo. It is only for the Play Store app.
  • All the official ODK Collect app traffic goes on one account that Mapbox can support with an annual coupon (or special plan if need be), akin to Google's approach of supporting ODK's use of their APIs.

For forks of ODK Collect

  • Devs are asked to create their own Mapbox account/token to use in their fork
  • We include instructions on how to contact the Mapbox Community team if the fork is for a non-profit or the Mapbox Sales team if for a commercial project with high volume.
  • I'm thinking we add something like the following to ODK Collect's developer page:

Mapbox APIs - If you want to use Mapbox services within your fork of ODK Collect, you will need to create a Mapbox account in order to have a Mapbox access token. Opening a Mapbox account on the 'Pay-As-You-Go' plan does not require a credit card. Mapbox provides free API usage up to the monthly thresholds documented at https://www.mapbox.com/pricing/ (e.g. below 50,000 monthly active users is free). If your usage exceeds any of these thresholds, you will receive an automatic email with instructions on how to add a credit card to pay for the services. Services will remain live until the end of the 30-day billing term, after which point the account will be deactivated and requires a credit card to reactivate.

Once you have created a Mapbox account, find your access token as described here and paste your access token in --- [instructions to be defined by @langstonsmith ]---. Please be sure not to commit your personal access token to a branch that you will submit a pull request for.

Learn more about Mapbox access tokens here and Mapbox terms of service here.

If you are creating a fork of ODK Collect for a non-profit organization or positive impact project, you can contact the Mapbox Community team with any questions about using Mapbox. If you are creating a fork of ODK Collect for a commercial enterprise and have questions about pricing or terms, you can contact the Mapbox Sales team.

Does anyone have concerns about this approach?

Great seeing the updates and progress here @zestyping @langstonsmith @Marena.

@Marena Agreed with the proposal for having a single account for the official ODK Collect app and then giving dev's options for fork's or working with a fork of ODK Collect. I don't see any issues with this approach at the moment.

@zestyping Following up to your question about @paul.uithol's comment. Yes, I think that's in line with what he was talking about. Being able to configure based on different online or offline options is something we'd be looking for. Some of the other items, like editing, sound like are beyond the scope of this thread and are starting to get towards this new thread: Load existing data points in to a geopoint widget in Collect

Haven't forgotten about this. @zestyping, it seems as though you continue to make great progress on map-related work in the app: https://github.com/opendatakit/collect/pulls/zestyping

I'm thinking about holding off on trying to finish my own work until your major refactoring is complete...

Hey @langstonsmith!

I'd say we're done with the major refactoring; the MapFragment interface is pretty stable now. Of the pull requests you see there, only #2776 touches MapFragment; the rest are behaviour changes above the MapFragment level, which should not affect your work.

Here's the latest MapFragment interface, which is just pending QA; I expect it to be on master by next week: https://github.com/opendatakit/collect/blob/568823acbc2a6027863063aa09cb4b988bc38424/collect_app/src/main/java/org/odk/collect/android/map/MapFragment.java

Can you help me understand what steps you have left to do to finish your work? If there are any concerns that are holding you back, I'd like to address them promptly so you can proceed.

1 Like

Ah, I see now @zestyping . Your other prs branch off of #2776 :slightly_smiling_face:

Still wrestling with some funky Fragment lifecycle stuff on the Mapbox side. Will also kinda' wait until #2776 lands so that I can pull the changes into my branch.

1 Like

There's a new related discussion here: Geo: Using the Mapbox SDK for Android

Hey, thanks for Geo: Using the Mapbox SDK for Android .

I'm reeeeealllly close to opening my pr. Here's my work https://github.com/opendatakit/collect/compare/master...langsmith:ls-adding-mapbox-maps?expand=1

I need to finish hooking the Mapbox Annotation Plugin for Android into some of the logic related to marker clearing, line dragging, etc.

Adding Mapbox vector tile basemaps wasn't mentioned in today's Geo: Using the Mapbox SDK for Android ticket. Might be good to cross reference them.

1 Like

Great! Thanks very much—I'm looking at your work now.

Hey @langstonsmith,

I've merged the latest master with your branch and resolved all the conflicts. There were also some inconsistencies in spacing and indentation introduced by your IDE, and it looks like there was a global search-and-replace that went awry (changing "marker" into "symbolOptions" throughout); so I reverted these. The result, squashed into one commit on top of master, is ping/langston-squashed.

I then went through and removed all the changes that weren't related to the Mapbox SDK. I noticed that you did a number of other cleanups (which were nice!), but for now I'd like to cleanly separate out just the changes that are related to this feature (we can do your other cleanups in separate commits, later). The result is ping/langston-minimal.

I'll continue working from the ping/langston-minimal branch. If you can, please continue your work by starting your own branch based on ping/langston-minimal, so it will be easier to merge our changes together.

@langstonsmith On second thought, if you're thinking of proceeding further, please reach out to me and let's coordinate first so that we don't do conflicting work. Sorry for any confusion, and thanks!

Ok @zestyping, thanks for all your recent work on top of mine.

I saw that you opened 3 tickets at https://github.com/mapbox/mapbox-plugins-android/issues :+1: . As you can probably tell by this point, my code is/was a bit byzantine because our Annotation Plugin needs some improvements to easily support the behavior that the ODK app is looking for. If the app didn't need marker/line dragging, it'd be easier to just use the Maps SDK's sources/layers/Features to draw the desired data. The Annotation Plugin makes dragging much easier, which is why I've instituted it.

Hey thanks, @langstonsmith!

Yeah, now that I've dug into it, I'm understanding the API and the issues much better. I can't figure out how to make dragging workable, though—the symbols are draggable, but every drag also triggers a map click, so extra points keep getting added. I filed another issue, maybe we can discuss it there?

Collect v1.22 added the Mapbox engine and several beautiful basemap styles hosted by Mapbox.

Collect v1.23 will add offline tile support for the Mapbox engine and a standalone mapping settings screen. Please see ODK Collect v1.23 Beta to learn more, try it out and offer feedback!

1 Like