Mapbox SDK integration

[zestyping]

Hi!

This thread is for us to discuss implementation details of integrating the Mapbox SDK into ODK Collect.

Current efforts have led to a branch that supports points, traces, and shapes. It works in that you can place points and vertices, but vertices cannot be moved by dragging.

We've run into a few limitations of the Mapbox SDK, which we've now filed as issues:

• #934: Allow custom data attached to symbols
• #935: Symbols disappear when they collide with other symbols
• #936: Adding symbols is slow
• #937: Start symbol dragging without also triggering a map click.

@langstonsmith, what's the best way to proceed on these? @Marena, any thoughts?

Thanks!

[Marena]

Thank you for filing these, @zestyping - I'll see what I can do internally. @langstonsmith your best bet for technical advice.

Are these all blockers to the integration, or just things that it would be good to figure out or improve?

[zestyping]

Thanks very much, Marena!

They do each affect the integration differently — good thing you asked!

#937 is the biggest blocker — it means we can't move markers by dragging them. There are a few ways I could think of addressing this, so I hope I can discuss it with whoever is looking at fixing it.

#935 is a smaller blocker, as it makes the user experience closing.

#936 is tolerable but it would be really good to at least understand the nature of the issue. If we're lucky, it may go away when #935 is addressed.

#934 is a programmer convenience. We can work around it; it just makes the Mapbox implementation messier than the others.

[zestyping]

Hi @Marena and everyone, just wanted to give a quick update on progress here.

Thanks to Tobrun, #935 is solved—there was just another API call that I didn't realize was related.

After that information helped me understand the root cause of #935, I decided to attach custom data to the symbols in a different way. It's still a bit of a workaround, but it's fine, so #934 is not a blocker.

That change helped make it possible to come up with a workaround for #937. I think #937 should still be addressed, but the workaround does the job! (Our code now maintains a local flag to keep track of whether a drag was taking place, and checks that flag in both onMapClick and onMapLongClick, ignoring the click if a drag is in progress.)

And as mentioned before, #936 is a nice-to-have. (Addressing #935 did not impact #936.)

This means we have a working implementation of Mapbox SDK that is essentially feature-complete! Thanks to @langstonsmith for discussing these and helping me out, and building most of the Mapbox implementation on which this is built.

There is one issue left, which is that the Mapbox SDK crashes (in a way that looks deliberate) when I initialize it with an empty string as the access token. It wasn't doing this before in my simpler testing, so I need to figure out what is different about the situation when it is being used by ODK Collect. We need the Mapbox SDK to work with an empty access token and show the OSM tiles as a fallback. @langstonsmith has offered to look into this (thank you!)—Langston, when you have any updates, let's use this forum thread to discuss the issue.

So, the remaining work I have to do is to fix this crash and to change the Settings screens to give options for the Mapbox map, and we also need to decide on a deprecation path for OSMdroid. If all that goes smoothly, we may be able to release it soon, perhaps in the next release of ODK Collect!

[Marena]

Amazing work @zestyping @langstonsmith and Tobrun - thank you for the update. Fingers crossed we can get those last pieces sorted in time for the next release!

[langstonsmith]

@zestyping

There is one issue left, which is that the Mapbox SDK crashes (in a way that looks deliberate) when I initialize it with an empty string as the access token. It wasn't doing this before in my simpler testing, so I need to figure out what is different about the situation when it is being used by ODK Collect.

@zestyping, I downloaded, opened, and installed ODK with your fork/branch (https://github.com/opendatakit/collect/compare/master...zestyping:ping/mapbox) . On an emulated device, I wasn't able to reproduce the error you were seeing. What happens specifically? Does the app crash for you? When I do Mapbox.getInstance(Collect.getInstance(), ""); in the MapboxMapFragment, I get /org.odk.collect.android E/Mbgl: {collect.android}[Setup]: loading style failed: HTTP status code 401 in the log cat. The app doesn't outright crash on me though. Should I be trying something different? Want me to try on a physical device? Should I open a particular GPS widget?

You should know that the Maps SDK includes a method to set the access token at runtime in case it needs to be updated/changed before a Mapbox API call is made. Mapbox.setAccessToken(); Perhaps that can be used in some way for the setup and flow that you want.

Also, you might want to replace the dev-config.xml writing with mention of the secret.properties setup.

[zestyping]

Here's what happens in the log:

05-17 09:23:19.357 325-325/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
05-17 09:23:19.357 325-325/? A/DEBUG: Build fingerprint: 'samsung/grandpplteub/grandpplte:6.0.1/MMB29T/G532MUMU1ASA1:user/release-keys'
05-17 09:23:19.357 325-325/? A/DEBUG: Revision: '5'
05-17 09:23:19.357 325-325/? A/DEBUG: ABI: 'arm'
05-17 09:23:19.357 325-325/? A/DEBUG: pid: 18440, tid: 19135, name: DefaultFileSour >>> org.odk.collect.android <<<
05-17 09:23:19.357 325-325/? A/DEBUG: signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
05-17 09:23:19.417 325-325/? A/DEBUG: Abort message: '/usr/local/google/buildbot/src/android/ndk-release-r19/external/libcxx/../../external/libcxxabi/src/abort_message.cpp:73: abort_message: assertion "terminating with uncaught exception of type std::runtime_error: You must provide a Mapbox API access token for Mapbox tile sources" failed'
05-17 09:23:19.417 325-325/? A/DEBUG: r0 00000000 r1 00004abf r2 00000006 r3 97bab978
05-17 09:23:19.417 325-325/? A/DEBUG: r4 97bab980 r5 97bab930 r6 00000002 r7 0000010c
05-17 09:23:19.417 325-325/? A/DEBUG: r8 98324475 r9 97bab714 sl b6cb9ec0 fp 97bab5b0
05-17 09:23:19.417 325-325/? A/DEBUG: ip 00000006 sp 97baaf38 lr b6c7ede5 pc b6c811d4 cpsr 400e0010
05-17 09:23:19.437 325-325/? A/DEBUG: backtrace:
05-17 09:23:19.437 325-325/? A/DEBUG: #00 pc 000431d4 /system/lib/libc.so (tgkill+12)
05-17 09:23:19.437 325-325/? A/DEBUG: #01 pc 00040de1 /system/lib/libc.so (pthread_kill+32)
05-17 09:23:19.437 325-325/? A/DEBUG: #02 pc 0001c7e7 /system/lib/libc.so (raise+10)
05-17 09:23:19.437 325-325/? A/DEBUG: #03 pc 00019a65 /system/lib/libc.so (__libc_android_abort+34)
05-17 09:23:19.437 325-325/? A/DEBUG: #04 pc 00017600 /system/lib/libc.so (abort+4)
05-17 09:23:19.437 325-325/? A/DEBUG: #05 pc 0001b3fb /system/lib/libc.so (__libc_fatal+16)
05-17 09:23:19.437 325-325/? A/DEBUG: #06 pc 00019aed /system/lib/libc.so (__assert2+20)
05-17 09:23:19.437 325-325/? A/DEBUG: #07 pc 00277509 /data/app/org.odk.collect.android-1/lib/arm/libmapbox-gl.so
05-17 09:23:19.437 325-325/? A/DEBUG: #08 pc 002775b1 /data/app/org.odk.collect.android-1/lib/arm/libmapbox-gl.so
05-17 09:23:19.437 325-325/? A/DEBUG: #09 pc 002761a5 /data/app/org.odk.collect.android-1/lib/arm/libmapbox-gl.so
05-17 09:23:19.437 325-325/? A/DEBUG: #10 pc 00275bfb /data/app/org.odk.collect.android-1/lib/arm/libmapbox-gl.so
05-17 09:23:19.437 325-325/? A/DEBUG: #11 pc 00275bc3 /data/app/org.odk.collect.android-1/lib/arm/libmapbox-gl.so
05-17 09:23:19.437 325-325/? A/DEBUG: #12 pc 00204f05 /data/app/org.odk.collect.android-1/lib/arm/libmapbox-gl.so
05-17 09:23:19.437 325-325/? A/DEBUG: #13 pc 001f4943 /data/app/org.odk.collect.android-1/lib/arm/libmapbox-gl.so
05-17 09:23:19.437 325-325/? A/DEBUG: #14 pc 001f4cef /data/app/org.odk.collect.android-1/lib/arm/libmapbox-gl.so
05-17 09:23:19.437 325-325/? A/DEBUG: #15 pc 00051c17 /data/app/org.odk.collect.android-1/lib/arm/libmapbox-gl.so
05-17 09:23:19.437 325-325/? A/DEBUG: #16 pc 0002b20f /data/app/org.odk.collect.android-1/lib/arm/libmapbox-gl.so
05-17 09:23:19.437 325-325/? A/DEBUG: #17 pc 0002b34b /data/app/org.odk.collect.android-1/lib/arm/libmapbox-gl.so
05-17 09:23:19.437 325-325/? A/DEBUG: #18 pc 0002ac85 /data/app/org.odk.collect.android-1/lib/arm/libmapbox-gl.so
05-17 09:23:19.437 325-325/? A/DEBUG: #19 pc 001f3b31 /data/app/org.odk.collect.android-1/lib/arm/libmapbox-gl.so
05-17 09:23:19.437 325-325/? A/DEBUG: #20 pc 000406e3 /system/lib/libc.so (_ZL15__pthread_startPv+30)
05-17 09:23:19.437 325-325/? A/DEBUG: #21 pc 0001a0e9 /system/lib/libc.so (__start_thread+6)


The crash comes directly from native code, so it cannot be caught as a Java exception; and this is all posted at the ASSERT log level, even more severe than ERROR. It looks deliberate, based on the call to abort_message:

assertion "terminating with uncaught exception of type std::runtime_error: You must provide a Mapbox API access token for Mapbox tile sources" failed'

I have found a workaround, though. If I supply "pk." as the access token instead of "", this crash doesn't occur.

[zestyping]

Better workaround: if I make sure that the style JSON doesn't reference any Mapbox-sourced layers, it also doesn't crash.

[langstonsmith]

Ok, thanks for the info. I will look into this but will continue considering this a blocker .

[zestyping]

I don't think it's a blocker any more. As long as we are careful not to request any Mapbox-sourced layers, we should be fine.

[zestyping]

Just as a heads-up @Marena and @langstonsmith—the Mapbox integration work is proceeding in https://github.com/opendatakit/collect/pull/3074.