ODK Aggregate Pricing and Cloud Storage along with frequent Fusion Table read write and query

Hi Team,

I want to know the pricing for odk aggregate app engine.

  1. Our survey will capture nearly about 1 million data with 50 fields in
    one form.
  2. The survey will take 90 days.
  3. Each submission will capture one image of size 200k
  4. We need the data to be exported to google fusion table.
  5. We need to access the fusion table frequently as to query the data daily
    which could further be exportd as kml network link.
  6. Please let me know the pricing and storge requirements for read write
    query.

Thanks
Neeraj

All the Open Data Kit tools are all free and open-source.

Your costs will be for setting up and managing the servers on which ODK
Aggregate run.

For non-technical groups, we recommend using Google App Engine for this
purpose because Google will then handle everything -- once you have set up
the server.

See:
https://opendatakit.org/use/aggregate/deployment-planning/#App_Engine_Usage_Fees
for information about AppEngine expenses.

For a survey of your size and limited duration, it should not cost very
much.

If this is your first use of the ODK tools, we strongly recommend that you
do a small (e.g., 10-device, 3-day) data collection and analysis pilot to
gain familiarity with how to train your data collectors in the use of the
tools, to make sure you are collecting the information you need for your
analysis of the larger study, and to set up and verify the publishing into
fusion tables does what you want it to do.

··· On Mon, May 23, 2016 at 6:07 AM, Neeraj Dewangan wrote:

Hi Team,

I want to know the pricing for odk aggregate app engine.

  1. Our survey will capture nearly about 1 million data with 50 fields in
    one form.
  2. The survey will take 90 days.
  3. Each submission will capture one image of size 200k
  4. We need the data to be exported to google fusion table.
  5. We need to access the fusion table frequently as to query the data
    daily which could further be exportd as kml network link.
  6. Please let me know the pricing and storge requirements for read write
    query.

Thanks
Neeraj

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Thanks Mitch,

Greatly appreciate your reply.
I have been using the odk aggregate for data collection in offline mode and
downloading the data with odk briefcase.
And than manually uploading the csv and media to fusion table.
But few days back I set up the auth2 service account with odk aggregate
app engine to export the data to fusion table.
I got an error for regarding quota limit, even though the data was manually
uploaded and downloaded from fusion table.

  1. Is there any limit for fusion table export from odk agggregate server.
  2. And is there any limit for fusion table download/upload or maximum
    number of records in fusion table

My project ID for cgsird2012.appspot.com is ID: cgsird2012 (
# 333502449148)

Please let me know the type of server setting for my data collection so
that in future quota limit will not occur.

Thanks & Regards

Neeraj

··· On Monday, May 23, 2016 at 6:37:25 PM UTC+5:30, Neeraj Dewangan wrote: > > Hi Team, > > I want to know the pricing for odk aggregate app engine. > 1. Our survey will capture nearly about 1 million data with 50 fields in > one form. > 2. The survey will take 90 days. > 3. Each submission will capture one image of size 200k > 4. We need the data to be exported to google fusion table. > 5. We need to access the fusion table frequently as to query the data > daily which could further be exportd as kml network link. > 6. Please let me know the pricing and storge requirements for read write > query. > > Thanks > Neeraj >

Google provides a small amount of free usage of AppEngine. Beyond that, you
will need to set up billing.

If you had already submitted many rows (1000's) into ODK Aggregate, when
you set up publishing of all of that existing data and all future
(streaming) data into Fusion Tables, it is quite possible for the burst of
activity to publish that accumulated set of rows into Fusion Tables can
exceed your daily quota on AppEngine.

Note that there are different quota limits for different Google products --
including Fusion Tables.

··· ================== Fusion tables does have some limitations on the quantity of data that can be pumped into it from ODK Aggregate in a day. There is also a per-minute limit, which ODK Aggregate automatically handles. For larger data sets, there is also a overall size limit.

https://developers.google.com/fusiontables/docs/v1/using#KML

The default request quota limits are the following:

  • 25,000 requests per day per API project, where reads count as one
    request and writes count as five requests.
  • 30 write requests per minute per table

You may request more quota at the Google APIs Console
https://code.google.com/apis/console under the "Quotas" tab.

The storage quota limits are:

  • 250 MB per table
  • 1 GB total among all your tables

============
There appear to be some limits w.r.t. the Google Maps integration within
Fusion Tables:
https://developers.google.com/fusiontables/docs/v1/using#Geo
Geographic design constraints

It's useful to keep in mind these limits:

  • Only the first 350,000 rows of data in a table are mapped or included
    in spatial query results.*
    Therefore:
    • Maps made with Fusion Tables Layers in the Google Maps API or with
      the Fusion Tables web application will only show data from within this
      first 350,000 rows. If you apply a filter to a table larger than
      350K rows
      and the filter matches data in rows after the first 350K, these rows will
      not appear on the map.
    • Queries with spatial predicates only return data from within this
      first 350K rows. If you apply a SELECT query with a spatial
      predicate to a
      very large table and the query matches data in rows after the first 350K,
      these rows are not returned.
  • When importing or inserting data, remember:
    • The total size of the data sent in one API call cannot exceed 1MB.
    • API calls to write data should not exceed 0.5 qps per table.
    • A cell of data in Fusion Tables supports a maximum of 1 million
      characters; it may sometimes be necessary to reduce the precision of
      coordinates or simplify polygon or line descriptions.
    • The maximum number of vertices supported per table is 5 million.*
  • When looking at the map, you may notice:
    • The ten largest-area components of a multi-geometry are shown.
    • When zoomed far out, map tiles may exceed the 500 feature-per-tile
      limit, and features will be dropped from the map.*
    • When zoomed farther out, tables with more than 500 features will
      replace lines or polygons with dots.*

On Mon, May 23, 2016 at 10:34 AM, Neeraj Dewangan <neeraj.dewangan@gmail.com wrote:

Thanks Mitch,

Greatly appreciate your reply.
I have been using the odk aggregate for data collection in offline mode
and downloading the data with odk briefcase.
And than manually uploading the csv and media to fusion table.
But few days back I set up the auth2 service account with odk aggregate
app engine to export the data to fusion table.
I got an error for regarding quota limit, even though the data was
manually uploaded and downloaded from fusion table.

  1. Is there any limit for fusion table export from odk agggregate server.
  2. And is there any limit for fusion table download/upload or maximum
    number of records in fusion table

My project ID for cgsird2012.appspot.com is ID: cgsird2012 (
# 333502449148)

Please let me know the type of server setting for my data collection so
that in future quota limit will not occur.

Thanks & Regards

Neeraj

On Monday, May 23, 2016 at 6:37:25 PM UTC+5:30, Neeraj Dewangan wrote:

Hi Team,

I want to know the pricing for odk aggregate app engine.

  1. Our survey will capture nearly about 1 million data with 50 fields in
    one form.
  2. The survey will take 90 days.
  3. Each submission will capture one image of size 200k
  4. We need the data to be exported to google fusion table.
  5. We need to access the fusion table frequently as to query the data
    daily which could further be exportd as kml network link.
  6. Please let me know the pricing and storge requirements for read write
    query.

Thanks
Neeraj

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Thanks for the valuable information.

··· On May 23, 2016 11:57 PM, "Mitch Sundt" wrote:

Google provides a small amount of free usage of AppEngine. Beyond that,
you will need to set up billing.

If you had already submitted many rows (1000's) into ODK Aggregate, when
you set up publishing of all of that existing data and all future
(streaming) data into Fusion Tables, it is quite possible for the burst of
activity to publish that accumulated set of rows into Fusion Tables can
exceed your daily quota on AppEngine.

Note that there are different quota limits for different Google products
-- including Fusion Tables.

==================
Fusion tables does have some limitations on the quantity of data that can
be pumped into it from ODK Aggregate in a day. There is also a per-minute
limit, which ODK Aggregate automatically handles. For larger data sets,
there is also a overall size limit.

https://developers.google.com/fusiontables/docs/v1/using#KML

The default request quota limits are the following:

  • 25,000 requests per day per API project, where reads count as one
    request and writes count as five requests.
  • 30 write requests per minute per table

You may request more quota at the Google APIs Console
https://code.google.com/apis/console under the "Quotas" tab.

The storage quota limits are:

  • 250 MB per table
  • 1 GB total among all your tables

============
There appear to be some limits w.r.t. the Google Maps integration within
Fusion Tables:
https://developers.google.com/fusiontables/docs/v1/using#Geo
Geographic design constraints

It's useful to keep in mind these limits:

  • Only the first 350,000 rows of data in a table are mapped or
    included in spatial query results.*
    Therefore:
    • Maps made with Fusion Tables Layers in the Google Maps API or
      with the Fusion Tables web application will only show data from within this
      first 350,000 rows. If you apply a filter to a table larger than 350K rows
      and the filter matches data in rows after the first 350K, these rows will
      not appear on the map.
    • Queries with spatial predicates only return data from within this
      first 350K rows. If you apply a SELECT query with a spatial predicate to a
      very large table and the query matches data in rows after the first 350K,
      these rows are not returned.
  • When importing or inserting data, remember:
    • The total size of the data sent in one API call cannot exceed 1MB.
    • API calls to write data should not exceed 0.5 qps per table.
    • A cell of data in Fusion Tables supports a maximum of 1 million
      characters; it may sometimes be necessary to reduce the precision of
      coordinates or simplify polygon or line descriptions.
    • The maximum number of vertices supported per table is 5 million.*
  • When looking at the map, you may notice:
    • The ten largest-area components of a multi-geometry are shown.
    • When zoomed far out, map tiles may exceed the 500
      feature-per-tile limit, and features will be dropped from the map.*
    • When zoomed farther out, tables with more than 500 features will
      replace lines or polygons with dots.*

On Mon, May 23, 2016 at 10:34 AM, Neeraj Dewangan < neeraj.dewangan@gmail.com> wrote:

Thanks Mitch,

Greatly appreciate your reply.
I have been using the odk aggregate for data collection in offline mode
and downloading the data with odk briefcase.
And than manually uploading the csv and media to fusion table.
But few days back I set up the auth2 service account with odk aggregate
app engine to export the data to fusion table.
I got an error for regarding quota limit, even though the data was
manually uploaded and downloaded from fusion table.

  1. Is there any limit for fusion table export from odk agggregate server.
  2. And is there any limit for fusion table download/upload or maximum
    number of records in fusion table

My project ID for cgsird2012.appspot.com is ID: cgsird2012 (
# 333502449148)

Please let me know the type of server setting for my data collection so
that in future quota limit will not occur.

Thanks & Regards

Neeraj

On Monday, May 23, 2016 at 6:37:25 PM UTC+5:30, Neeraj Dewangan wrote:

Hi Team,

I want to know the pricing for odk aggregate app engine.

  1. Our survey will capture nearly about 1 million data with 50 fields in
    one form.
  2. The survey will take 90 days.
  3. Each submission will capture one image of size 200k
  4. We need the data to be exported to google fusion table.
  5. We need to access the fusion table frequently as to query the data
    daily which could further be exportd as kml network link.
  6. Please let me know the pricing and storge requirements for read write
    query.

Thanks
Neeraj

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/UGpF9D0lA-0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.