Help: Distance between geopoints and display (GPS)

Hi all,

I have been working on a solution in ODK and have hit a roadblock. I'll
explain the basic concept of what I'm trying to do and then the research
I've done to try to figure it out. The basic concept is I would like to be
able to hit the "Record Location" (Geopoint) in an ODK survey, then on the
next page display the distance to some previously recorded locations
(pulling the previous locations in via pulldata function). So for example,
if we want to see where the nearest bank is to our current location, I
would have the bank's geocoordinates in a separate csv, pull in the bank's
coordinates into the current ODK Collect survey using pulldata, then
calculate the distance between the newly recorded point (record location,
geopoint) and the bank (this is where I get hung up).

Questions:

  1. Is there a way to get the geopoint into numerically separated latitude
    and longitude fields and use them in calculate functions? If so I'm pretty
    sure I could figure the programming out from there.

I know the following:

  1. I can calculate distances between two geopoints (lat, long) using the
    haversine formula. I looked into the javarosa support and it looks like
    sin/cos/tan has been added in so using haversine is possible. Even if
    sin/cos/tan support wasn't, the distances we are trying to calculate are
    short (max 15KM) and they don't need to be perfect so I could in theory use
    the pythagorean theorem to get a simple distance estimate (just drop the
    curvature of the earth, so the calculation would be slightly less
    accurate).

  2. I know that SurveyCTO has the capability to do distance-between() added.
    Unfortunately I work with microfinance institutions in the Philippines that
    already run our combination of ODK/Kobo Toolbox and it would be both time
    and cost prohibitive to migrate them to a paid version on SurveyCTO. If
    possible I'd like to do this in regular ODK Collect. (Don't get me wrong
    though, SurveyCTO looks awesome and I appreciate the features they have
    added support for in regular ODK, seems like a lot of the new features come
    from CTO now).

  3. I know that even if I have a list of, for example, multiple banks, if I
    figured out how to get haversine distances as numerics, I could use the
    min() function on a whole list of banks to get just the closest one to the
    current geopoint and display only that one. It would be a long form (I
    program in XLSform) as it would have to calculate each distance
    individually, but it would be worth it for me to program if it were doable.

  4. The only work around I have figured out so far is having ODK display the
    geopoint coordinate and having the field staff type in the numbers into
    integer fields. I've tried to separate out lat/long, destring it, and
    various other attempts on calculate functions to no avail. I can get it to
    display though. It would be a hinderance (and slow the field staff down) to
    have them type this in, and it brings in the possibility of human error,
    neither being ideal.

Any help would be greatly appreciated.

Thanks!

Neal

Neal,

In brief outline, here is how we are dealing with a similar issue. We wish to visit a stratified random sample of wells to monitor their condition. We need to compare the location of a well being visited with the location that was recorded for a specific well on a previous visit, in order to confirm that this is the correct well. The two GPS readings should be "close", differing only due to lack of perfect GPS accuracy, not standing exactly at the same spot each time, etc.

  1. Record the GPS coordinates in the usual way. The coordinates are stored by ODK Collect as a string variable, with spaces between the latitude, longitude, altitude, and accuracy).
  2. Use the substr() function to extract the latitude and longitude strings, then convert them to numerical values. Since apparently the length of the strings can vary somewhat, use nested if functions to find the separating spaces.
  3. Use pulldata() to retrieve the previously recorded coordinates from a preloaded csv file.
  4. Use the pythagorian theorem to calculate the distance between the two sets of coordinates. If it is the same well, the calculated "distance" should be small, in the tens of meters max. Given these small distances, and not needing great precision for this purpose, we ignore the earth's curvature and use an average value of latitude to convert the longitude difference from degrees to meters.

If you are interested, I would be glad to share the relevant lines of code from the xlsform spreadsheet.

All the best,

Hayden Boyd

··· On 3/10/2016 11:37 PM, Neal Barsch wrote:

Hi all,

I have been working on a solution in ODK and have hit a roadblock. I'll explain the basic concept of what I'm trying to do and then the research I've done to try to figure it out. The basic concept is I would like to be able to hit the "Record Location" (Geopoint) in an ODK survey, then on the next page display the distance to some previously recorded locations (pulling the previous locations in via pulldata function). So for example, if we want to see where the nearest bank is to our current location, I would have the bank's geocoordinates in a separate csv, pull in the bank's coordinates into the current ODK Collect survey using pulldata, then calculate the distance between the newly recorded point (record location, geopoint) and the bank (this is where I get hung up).

Questions:

  1. Is there a way to get the geopoint into numerically separated latitude and longitude fields and use them in calculate functions? If so I'm pretty sure I could figure the programming out from there.

I know the following:

  1. I can calculate distances between two geopoints (lat, long) using the haversine formula. I looked into the javarosa support and it looks like sin/cos/tan has been added in so using haversine is possible. Even if sin/cos/tan support wasn't, the distances we are trying to calculate are short (max 15KM) and they don't need to be perfect so I could in theory use the pythagorean theorem to get a simple distance estimate (just drop the curvature of the earth, so the calculation would be slightly less accurate).
  1. I know that SurveyCTO has the capability to do distance-between() added. Unfortunately I work with microfinance institutions in the Philippines that already run our combination of ODK/Kobo Toolbox and it would be both time and cost prohibitive to migrate them to a paid version on SurveyCTO. If possible I'd like to do this in regular ODK Collect. (Don't get me wrong though, SurveyCTO looks awesome and I appreciate the features they have added support for in regular ODK, seems like a lot of the new features come from CTO now).
  1. I know that even if I have a list of, for example, multiple banks, if I figured out how to get haversine distances as numerics, I could use the min() function on a whole list of banks to get just the closest one to the current geopoint and display only that one. It would be a long form (I program in XLSform) as it would have to calculate each distance individually, but it would be worth it for me to program if it were doable.
  1. The only work around I have figured out so far is having ODK display the geopoint coordinate and having the field staff type in the numbers into integer fields. I've tried to separate out lat/long, destring it, and various other attempts on calculate functions to no avail. I can get it to display though. It would be a hinderance (and slow the field staff down) to have them type this in, and it brings in the possibility of human error, neither being ideal.

Any help would be greatly appreciated.

Thanks!

Neal

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

1 Like

Hayden,

This sounds almost perfect for what I'm trying to do. If you wouldn't mind
sharing the code that would be fantastic! I tried to de-string the GPS
codes but couldn't figure out a way to do so, but it sounds like you have.

Thanks very much!

Neal

··· On Saturday, March 12, 2016 at 6:21:44 AM UTC+8, Hayden Boyd wrote: > > Neal, > > In brief outline, here is how we are dealing with a similar issue. We > wish to visit a stratified random sample of wells to monitor their > condition. We need to compare the location of a well being visited with > the location that was recorded for a specific well on a previous visit, in > order to confirm that this is the correct well. The two GPS readings > should be "close", differing only due to lack of perfect GPS accuracy, not > standing exactly at the same spot each time, etc. > > 1. Record the GPS coordinates in the usual way. The coordinates are > stored by ODK Collect as a string variable, with spaces between the > latitude, longitude, altitude, and accuracy). > 2. Use the substr() function to extract the latitude and longitude > strings, then convert them to numerical values. Since apparently the length > of the strings can vary somewhat, use nested if functions to find the > separating spaces. > 3. Use pulldata() to retrieve the previously recorded coordinates from a > preloaded csv file. > 4. Use the pythagorian theorem to calculate the distance between the two > sets of coordinates. If it is the same well, the calculated "distance" > should be small, in the tens of meters max. Given these small distances, > and not needing great precision for this purpose, we ignore the earth's > curvature and use an average value of latitude to convert the longitude > difference from degrees to meters. > > If you are interested, I would be glad to share the relevant lines of code > from the xlsform spreadsheet. > > All the best, > > Hayden Boyd > > On 3/10/2016 11:37 PM, Neal Barsch wrote: > > Hi all, > > I have been working on a solution in ODK and have hit a roadblock. I'll > explain the basic concept of what I'm trying to do and then the research > I've done to try to figure it out. The basic concept is I would like to be > able to hit the "Record Location" (Geopoint) in an ODK survey, then on the > next page display the distance to some previously recorded locations > (pulling the previous locations in via pulldata function). So for example, > if we want to see where the nearest bank is to our current location, I > would have the bank's geocoordinates in a separate csv, pull in the bank's > coordinates into the current ODK Collect survey using pulldata, then > calculate the distance between the newly recorded point (record location, > geopoint) and the bank (this is where I get hung up). > > Questions: > > 1. Is there a way to get the geopoint into numerically separated latitude > and longitude fields and use them in calculate functions? If so I'm pretty > sure I could figure the programming out from there. > > I know the following: > > 1. I can calculate distances between two geopoints (lat, long) using the > haversine formula. I looked into the javarosa support and it looks like > sin/cos/tan has been added in so using haversine is possible. Even if > sin/cos/tan support wasn't, the distances we are trying to calculate are > short (max 15KM) and they don't need to be perfect so I could in theory use > the pythagorean theorem to get a simple distance estimate (just drop the > curvature of the earth, so the calculation would be slightly less > accurate). > > 2. I know that SurveyCTO has the capability to do distance-between() > added. Unfortunately I work with microfinance institutions in the > Philippines that already run our combination of ODK/Kobo Toolbox and it > would be both time and cost prohibitive to migrate them to a paid version > on SurveyCTO. If possible I'd like to do this in regular ODK Collect. > (Don't get me wrong though, SurveyCTO looks awesome and I appreciate the > features they have added support for in regular ODK, seems like a lot of > the new features come from CTO now). > > 3. I know that even if I have a list of, for example, multiple banks, if I > figured out how to get haversine distances as numerics, I could use the > min() function on a whole list of banks to get just the closest one to the > current geopoint and display only that one. It would be a long form (I > program in XLSform) as it would have to calculate each distance > individually, but it would be worth it for me to program if it were doable. > > 4. The only work around I have figured out so far is having ODK display > the geopoint coordinate and having the field staff type in the numbers into > integer fields. I've tried to separate out lat/long, destring it, and > various other attempts on calculate functions to no avail. I can get it to > display though. It would be a hinderance (and slow the field staff down) to > have them type this in, and it brings in the possibility of human error, > neither being ideal. > > Any help would be greatly appreciated. > > Thanks! > > Neal > > -- > -- > Post: opend...@googlegroups.com > Unsubscribe: opendatakit...@googlegroups.com > Options: http://groups.google.com/group/opendatakit?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "ODK Community" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to opendatakit...@googlegroups.com . > For more options, visit https://groups.google.com/d/optout. > > >

A clever way to extract the lat/long/alt/acc from a geopoint is to observe
that the representation for a multiple-choice selection is as a
space-separated list of values, as is the geopoint coordinate.

So you can use the selected-at() function to work with geopoints...

latitude (degrees) = number(selected-at(${geofield},0))
longitude (degrees) = number(selected-at(${geofield},1))
altitude (meters) = number(selected-at(${geofield},2))
accuracy (meters) = number(selected-at(${geofield},3))

Saves a bunch of string manipulation.

··· On Sun, Mar 13, 2016 at 8:56 PM, Neal Barsch wrote:

Hayden,

This sounds almost perfect for what I'm trying to do. If you wouldn't mind
sharing the code that would be fantastic! I tried to de-string the GPS
codes but couldn't figure out a way to do so, but it sounds like you have.

Thanks very much!

Neal

On Saturday, March 12, 2016 at 6:21:44 AM UTC+8, Hayden Boyd wrote:

Neal,

In brief outline, here is how we are dealing with a similar issue. We
wish to visit a stratified random sample of wells to monitor their
condition. We need to compare the location of a well being visited with
the location that was recorded for a specific well on a previous visit, in
order to confirm that this is the correct well. The two GPS readings
should be "close", differing only due to lack of perfect GPS accuracy, not
standing exactly at the same spot each time, etc.

  1. Record the GPS coordinates in the usual way. The coordinates are
    stored by ODK Collect as a string variable, with spaces between the
    latitude, longitude, altitude, and accuracy).
  2. Use the substr() function to extract the latitude and longitude
    strings, then convert them to numerical values. Since apparently the length
    of the strings can vary somewhat, use nested if functions to find the
    separating spaces.
  3. Use pulldata() to retrieve the previously recorded coordinates from a
    preloaded csv file.
  4. Use the pythagorian theorem to calculate the distance between the two
    sets of coordinates. If it is the same well, the calculated "distance"
    should be small, in the tens of meters max. Given these small distances,
    and not needing great precision for this purpose, we ignore the earth's
    curvature and use an average value of latitude to convert the longitude
    difference from degrees to meters.

If you are interested, I would be glad to share the relevant lines of
code from the xlsform spreadsheet.

All the best,

Hayden Boyd

On 3/10/2016 11:37 PM, Neal Barsch wrote:

Hi all,

I have been working on a solution in ODK and have hit a roadblock. I'll
explain the basic concept of what I'm trying to do and then the research
I've done to try to figure it out. The basic concept is I would like to be
able to hit the "Record Location" (Geopoint) in an ODK survey, then on the
next page display the distance to some previously recorded locations
(pulling the previous locations in via pulldata function). So for example,
if we want to see where the nearest bank is to our current location, I
would have the bank's geocoordinates in a separate csv, pull in the bank's
coordinates into the current ODK Collect survey using pulldata, then
calculate the distance between the newly recorded point (record location,
geopoint) and the bank (this is where I get hung up).

Questions:

  1. Is there a way to get the geopoint into numerically separated latitude
    and longitude fields and use them in calculate functions? If so I'm pretty
    sure I could figure the programming out from there.

I know the following:

  1. I can calculate distances between two geopoints (lat, long) using the
    haversine formula. I looked into the javarosa support and it looks like
    sin/cos/tan has been added in so using haversine is possible. Even if
    sin/cos/tan support wasn't, the distances we are trying to calculate are
    short (max 15KM) and they don't need to be perfect so I could in theory use
    the pythagorean theorem to get a simple distance estimate (just drop the
    curvature of the earth, so the calculation would be slightly less
    accurate).

  2. I know that SurveyCTO has the capability to do distance-between()
    added. Unfortunately I work with microfinance institutions in the
    Philippines that already run our combination of ODK/Kobo Toolbox and it
    would be both time and cost prohibitive to migrate them to a paid version
    on SurveyCTO. If possible I'd like to do this in regular ODK Collect.
    (Don't get me wrong though, SurveyCTO looks awesome and I appreciate the
    features they have added support for in regular ODK, seems like a lot of
    the new features come from CTO now).

  3. I know that even if I have a list of, for example, multiple banks, if
    I figured out how to get haversine distances as numerics, I could use the
    min() function on a whole list of banks to get just the closest one to the
    current geopoint and display only that one. It would be a long form (I
    program in XLSform) as it would have to calculate each distance
    individually, but it would be worth it for me to program if it were doable.

  4. The only work around I have figured out so far is having ODK display
    the geopoint coordinate and having the field staff type in the numbers into
    integer fields. I've tried to separate out lat/long, destring it, and
    various other attempts on calculate functions to no avail. I can get it to
    display though. It would be a hinderance (and slow the field staff down) to
    have them type this in, and it brings in the possibility of human error,
    neither being ideal.

Any help would be greatly appreciated.

Thanks!

Neal

--

Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Hi Mr. Hayden,
Could you please share the xlsform calculations for this? That's exactly what I need for my public street lights form!

Hi @brunoseabra
you are asking for a form from 2016 so I guess chances are small. Probably it will be better for you if you create a new topic and describe your case.

Hello brunoseabra, the attached xls form snippet shows how we calculate the distance from the current location to a previously-recorded location. We want to be sure that we are at the well to be surveyed, not at another well that might be in the general area. GPS is the current reading. Using the suggestion from Mitch Sundt, we parse the reading into NewLat and NewLong coordinants. OldLat and OldLong are the coordinates from the previous survey, which we pull from a csv media file. We also pull Distance_calc.xlsx (22.8 KB) cosOldLat to account for longitude lines becoming closer together farther from the equator. We calculate the distance in meters using the pythagorean theorem. I hope this is useful. Regards, Hayden Boyd

2 Likes

FYI you can now just use the ODK distance() function to accomplish this; see Calculate distance between geopoints

1 Like

Hi @Hayden_Boyd , sorry for contacting you this much late, but I have a case where I use the point you mentionne in the Distance_calc.xlsx but I don't know how to get the cosOldLat. Can we have a screen of the media file from with data are being pulled ?

Hello Safari Mupe,

Apologies for this delayed response. My first try via gmail did not go through.

As you already know, there is no way for ODK Collect to calculate cosOldLat on the Android device. That is easily solved, however. When I use my computer to prepare the csv media file with data from the previous well surveys, I calculate the cosine of the latitude and add it to the csv file. Then, ODK Collect can read the pre-calculated cosOldLat value from the csv file, along with the latitude, longitude, and other data from the previous survey of the target well.

I hope this is helpful. Let me know if you have any questions or would like more information

All the best, Hayden Boyd

1 Like