Here is a strange one...
I have a users uploading data, and I can see the data in the Google
Docs link. However, when I download the csv file, the new data is not
there??? Any ideas.
Thanks!
Alex
Running 0.9.8
···
On Oct 14, 3:42 pm, Alex Pavluck wrote: > Here is a strange one... > > I have a users uploading data, and I can see the data in the Google > Docs link. However, when I download the csv file, the new data is not > there??? Any ideas. > > Thanks! > AlexServer: Google App Engine running 0.9.8
Checked again today and the csv file has 200 less records than the
external link to Google Docs. All the missing data were uploaded
yesterday. Has anyone seen this before? Is there anything to do to
troubleshoot this? Is there anything that I should learn to discern
from the App Engine logs?
This is really concerning because if I didn't have the Google Docs
link I would think that the server wasn't receiving the data!
Thanks!
Alex
···
On Oct 14, 3:42 pm, Alex Pavluck wrote: > Running 0.9.8 > > On Oct 14, 3:42 pm, Alex Pavluck wrote: > > > > > > > > > Here is a strange one... > > > I have a users uploading data, and I can see the data in the Google > > Docs link. However, when I download the csv file, the new data is not > > there??? Any ideas. > > > Thanks! > > AlexLast bit of additional information....
I downloaded briefcase and it is still fetching instances and those
greatly out number the download csv. Should the instance count match
the number of submissions? This would make sense because the team
claims to have uploaded over a 1,000 more records than we thought we
had on the server. Why is it that everything can see the submissions
but the csv doesn't?
Thanks,
Alex
···
On Oct 15, 8:02 am, Alex Pavluck wrote: > Server: Google App Engine running 0.9.8 > > Checked again today and the csv file has 200 less records than the > external link to Google Docs. All the missing data were uploaded > yesterday. Has anyone seen this before? Is there anything to do to > troubleshoot this? Is there anything that I should learn to discern > from the App Engine logs? > > This is really concerning because if I didn't have the Google Docs > link I would think that the server wasn't receiving the data! > > Thanks! > Alex > > On Oct 14, 3:42 pm, Alex Pavluck wrote: > > > > > > > > > Running 0.9.8 > > > On Oct 14, 3:42 pm, Alex Pavluck wrote: > > > > Here is a strange one... > > > > I have a users uploading data, and I can see the data in the Google > > > Docs link. However, when I download the csv file, the new data is not > > > there??? Any ideas. > > > > Thanks! > > > Alexhey alex,
this issue has been filed at
http://code.google.com/p/opendatakit/issues/detail?id=375.
this is the first i've heard of something like this, and it sounds
like there is a bug somewhere in csv generation that we need to track
down.
yaw
···
On Sat, Oct 15, 2011 at 05:23, Alex Pavluck wrote: > Last bit of additional information.... > > I downloaded briefcase and it is still fetching instances and those > greatly out number the download csv. Should the instance count match > the number of submissions? This would make sense because the team > claims to have uploaded over a 1,000 more records than we thought we > had on the server. Why is it that everything can see the submissions > but the csv doesn't? > > Thanks, > Alex > > > > On Oct 15, 8:02 am, Alex Pavluck wrote: >> Server: Google App Engine running 0.9.8 >> >> Checked again today and the csv file has 200 less records than the >> external link to Google Docs. All the missing data were uploaded >> yesterday. Has anyone seen this before? Is there anything to do to >> troubleshoot this? Is there anything that I should learn to discern >> from the App Engine logs? >> >> This is really concerning because if I didn't have the Google Docs >> link I would think that the server wasn't receiving the data! >> >> Thanks! >> Alex >> >> On Oct 14, 3:42 pm, Alex Pavluck wrote: >> >> >> >> >> >> >> >> > Running 0.9.8 >> >> > On Oct 14, 3:42 pm, Alex Pavluck wrote: >> >> > > Here is a strange one... >> >> > > I have a users uploading data, and I can see the data in the Google >> > > Docs link. However, when I download the csv file, the new data is not >> > > there??? Any ideas. >> >> > > Thanks! >> > > Alex > > -- > Post: opendatakit@googlegroups.com > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > Options: http://groups.google.com/group/opendatakit?hl=en >Thanks, Yaw. Let me know if you need access to the server or the logs to
help you find the issue.
Is there any other way to download the data?
Thanks again!
Alex
Alex Pavluck, MPH
Research Informatics Analyst
Lymphatic Filariasis Support Center
The Task Force for Global Health
325 Swanton Way, Decatur, GA 30030
cell: (404)647-4904 Fax: (404)371-1138
apavluck@taskforce.org
···
On Sun, Oct 16, 2011 at 4:16 PM, Yaw Anokwa wrote:hey alex,
this issue has been filed at
http://code.google.com/p/opendatakit/issues/detail?id=375.this is the first i've heard of something like this, and it sounds
like there is a bug somewhere in csv generation that we need to track
down.yaw
On Sat, Oct 15, 2011 at 05:23, Alex Pavluck apavluck@gmail.com wrote:
Last bit of additional information....
I downloaded briefcase and it is still fetching instances and those
greatly out number the download csv. Should the instance count match
the number of submissions? This would make sense because the team
claims to have uploaded over a 1,000 more records than we thought we
had on the server. Why is it that everything can see the submissions
but the csv doesn't?Thanks,
AlexOn Oct 15, 8:02 am, Alex Pavluck apavl...@gmail.com wrote:
Server: Google App Engine running 0.9.8
Checked again today and the csv file has 200 less records than the
external link to Google Docs. All the missing data were uploaded
yesterday. Has anyone seen this before? Is there anything to do to
troubleshoot this? Is there anything that I should learn to discern
from the App Engine logs?This is really concerning because if I didn't have the Google Docs
link I would think that the server wasn't receiving the data!Thanks!
AlexOn Oct 14, 3:42 pm, Alex Pavluck apavl...@gmail.com wrote:
Running 0.9.8
On Oct 14, 3:42 pm, Alex Pavluck apavl...@gmail.com wrote:
Here is a strange one...
I have a users uploading data, and I can see the data in the Google
Docs link. However, when I download the csv file, the new data is
not
there??? Any ideas.Thanks!
Alex--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
unfortunately, the ways you've tried are the only options we have
available. mitch will take a look at your issue as soon as he can and
we'll get things sorted for you.
···
On Sun, Oct 16, 2011 at 14:54, Alex Pavluck wrote: > Thanks, Yaw. Let me know if you need access to the server or the logs to > help you find the issue. > Is there any other way to download the data? > Thanks again! > Alex > > Alex Pavluck, MPH > Research Informatics Analyst > Lymphatic Filariasis Support Center > The Task Force for Global Health > 325 Swanton Way, Decatur, GA 30030 > cell: (404)647-4904 Fax: (404)371-1138 > apavluck@taskforce.org > > > On Sun, Oct 16, 2011 at 4:16 PM, Yaw Anokwa wrote: >> >> hey alex, >> >> this issue has been filed at >> http://code.google.com/p/opendatakit/issues/detail?id=375. >> >> this is the first i've heard of something like this, and it sounds >> like there is a bug somewhere in csv generation that we need to track >> down. >> >> yaw >> >> On Sat, Oct 15, 2011 at 05:23, Alex Pavluck wrote: >> > Last bit of additional information.... >> > >> > I downloaded briefcase and it is still fetching instances and those >> > greatly out number the download csv. Should the instance count match >> > the number of submissions? This would make sense because the team >> > claims to have uploaded over a 1,000 more records than we thought we >> > had on the server. Why is it that everything can see the submissions >> > but the csv doesn't? >> > >> > Thanks, >> > Alex >> > >> > >> > >> > On Oct 15, 8:02 am, Alex Pavluck wrote: >> >> Server: Google App Engine running 0.9.8 >> >> >> >> Checked again today and the csv file has 200 less records than the >> >> external link to Google Docs. All the missing data were uploaded >> >> yesterday. Has anyone seen this before? Is there anything to do to >> >> troubleshoot this? Is there anything that I should learn to discern >> >> from the App Engine logs? >> >> >> >> This is really concerning because if I didn't have the Google Docs >> >> link I would think that the server wasn't receiving the data! >> >> >> >> Thanks! >> >> Alex >> >> >> >> On Oct 14, 3:42 pm, Alex Pavluck wrote: >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > Running 0.9.8 >> >> >> >> > On Oct 14, 3:42 pm, Alex Pavluck wrote: >> >> >> >> > > Here is a strange one... >> >> >> >> > > I have a users uploading data, and I can see the data in the Google >> >> > > Docs link. However, when I download the csv file, the new data is >> >> > > not >> >> > > there??? Any ideas. >> >> >> >> > > Thanks! >> >> > > Alex >> > >> > -- >> > Post: opendatakit@googlegroups.com >> > Unsubscribe: opendatakit+unsubscribe@googlegroups.com >> > Options: http://groups.google.com/group/opendatakit?hl=en >> > >> >> -- >> Post: opendatakit@googlegroups.com >> Unsubscribe: opendatakit+unsubscribe@googlegroups.com >> Options: http://groups.google.com/group/opendatakit?hl=en > > -- > Post: opendatakit@googlegroups.com > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > Options: http://groups.google.com/group/opendatakit?hl=en >Thanks. This is happening to other surveys as well which are on other
App Engine databases. I think the issue might have started with the
0.9.8 upgrade. Please let me know if you need any logs or other
pieces of data to assist with this issue.
Thanks,
Alex
···
On Oct 16, 6:21 pm, Yaw Anokwa wrote: > unfortunately, the ways you've tried are the only options we have > available. mitch will take a look at your issue as soon as he can and > we'll get things sorted for you. > > > > > > > > On Sun, Oct 16, 2011 at 14:54, Alex Pavluck wrote: > > Thanks, Yaw. Let me know if you need access to the server or the logs to > > help you find the issue. > > Is there any other way to download the data? > > Thanks again! > > Alex > > > Alex Pavluck, MPH > > Research Informatics Analyst > > Lymphatic Filariasis Support Center > > The Task Force for Global Health > > 325 Swanton Way, Decatur, GA 30030 > > cell: (404)647-4904 Fax: (404)371-1138 > > apavl...@taskforce.org > > > On Sun, Oct 16, 2011 at 4:16 PM, Yaw Anokwa wrote: > > >> hey alex, > > >> this issue has been filed at > >>http://code.google.com/p/opendatakit/issues/detail?id=375. > > >> this is the first i've heard of something like this, and it sounds > >> like there is a bug somewhere in csv generation that we need to track > >> down. > > >> yaw > > >> On Sat, Oct 15, 2011 at 05:23, Alex Pavluck wrote: > >> > Last bit of additional information.... > > >> > I downloaded briefcase and it is still fetching instances and those > >> > greatly out number the download csv. Should the instance count match > >> > the number of submissions? This would make sense because the team > >> > claims to have uploaded over a 1,000 more records than we thought we > >> > had on the server. Why is it that everything can see the submissions > >> > but the csv doesn't? > > >> > Thanks, > >> > Alex > > >> > On Oct 15, 8:02 am, Alex Pavluck wrote: > >> >> Server: Google App Engine running 0.9.8 > > >> >> Checked again today and the csv file has 200 less records than the > >> >> external link to Google Docs. All the missing data were uploaded > >> >> yesterday. Has anyone seen this before? Is there anything to do to > >> >> troubleshoot this? Is there anything that I should learn to discern > >> >> from the App Engine logs? > > >> >> This is really concerning because if I didn't have the Google Docs > >> >> link I would think that the server wasn't receiving the data! > > >> >> Thanks! > >> >> Alex > > >> >> On Oct 14, 3:42 pm, Alex Pavluck wrote: > > >> >> > Running 0.9.8 > > >> >> > On Oct 14, 3:42 pm, Alex Pavluck wrote: > > >> >> > > Here is a strange one... > > >> >> > > I have a users uploading data, and I can see the data in the Google > >> >> > > Docs link. However, when I download the csv file, the new data is > >> >> > > not > >> >> > > there??? Any ideas. > > >> >> > > Thanks! > >> >> > > Alex > > >> > -- > >> > Post: opendatakit@googlegroups.com > >> > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > >> > Options:http://groups.google.com/group/opendatakit?hl=en > > >> -- > >> Post: opendatakit@googlegroups.com > >> Unsubscribe: opendatakit+unsubscribe@googlegroups.com > >> Options:http://groups.google.com/group/opendatakit?hl=en > > > -- > > Post: opendatakit@googlegroups.com > > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > > Options:http://groups.google.com/group/opendatakit?hl=enI tried another thing....On the topic of I have data in a 0.9.8 server
but I can't download it all as csv, Google Docs says I have more
records, and using Briefcase says I have even more.
I created another App Engine database running the newest version of
Aggregate and I used briefcase to migrate from the 0.9.8 database to
the new one. It was successful and I can see the data on the server!
However, I have 3,095 records in one form and when I export to csv it
only gives me the first 500 records. What am I missing to pull the
whole database?
Thanks,
Alex
···
On Oct 17, 9:33 am, Alex Pavluck wrote: > Thanks. This is happening to other surveys as well which are on other > App Engine databases. I think the issue might have started with the > 0.9.8 upgrade. Please let me know if you need any logs or other > pieces of data to assist with this issue. > > Thanks, > Alex > > On Oct 16, 6:21 pm, Yaw Anokwa wrote: > > > > > > > > > unfortunately, the ways you've tried are the only options we have > > available. mitch will take a look at your issue as soon as he can and > > we'll get things sorted for you. > > > On Sun, Oct 16, 2011 at 14:54, Alex Pavluck wrote: > > > Thanks, Yaw. Let me know if you need access to the server or the logs to > > > help you find the issue. > > > Is there any other way to download the data? > > > Thanks again! > > > Alex > > > > Alex Pavluck, MPH > > > Research Informatics Analyst > > > Lymphatic Filariasis Support Center > > > The Task Force for Global Health > > > 325 Swanton Way, Decatur, GA 30030 > > > cell: (404)647-4904 Fax: (404)371-1138 > > > apavl...@taskforce.org > > > > On Sun, Oct 16, 2011 at 4:16 PM, Yaw Anokwa wrote: > > > >> hey alex, > > > >> this issue has been filed at > > >>http://code.google.com/p/opendatakit/issues/detail?id=375. > > > >> this is the first i've heard of something like this, and it sounds > > >> like there is a bug somewhere in csv generation that we need to track > > >> down. > > > >> yaw > > > >> On Sat, Oct 15, 2011 at 05:23, Alex Pavluck wrote: > > >> > Last bit of additional information.... > > > >> > I downloaded briefcase and it is still fetching instances and those > > >> > greatly out number the download csv. Should the instance count match > > >> > the number of submissions? This would make sense because the team > > >> > claims to have uploaded over a 1,000 more records than we thought we > > >> > had on the server. Why is it that everything can see the submissions > > >> > but the csv doesn't? > > > >> > Thanks, > > >> > Alex > > > >> > On Oct 15, 8:02 am, Alex Pavluck wrote: > > >> >> Server: Google App Engine running 0.9.8 > > > >> >> Checked again today and the csv file has 200 less records than the > > >> >> external link to Google Docs. All the missing data were uploaded > > >> >> yesterday. Has anyone seen this before? Is there anything to do to > > >> >> troubleshoot this? Is there anything that I should learn to discern > > >> >> from the App Engine logs? > > > >> >> This is really concerning because if I didn't have the Google Docs > > >> >> link I would think that the server wasn't receiving the data! > > > >> >> Thanks! > > >> >> Alex > > > >> >> On Oct 14, 3:42 pm, Alex Pavluck wrote: > > > >> >> > Running 0.9.8 > > > >> >> > On Oct 14, 3:42 pm, Alex Pavluck wrote: > > > >> >> > > Here is a strange one... > > > >> >> > > I have a users uploading data, and I can see the data in the Google > > >> >> > > Docs link. However, when I download the csv file, the new data is > > >> >> > > not > > >> >> > > there??? Any ideas. > > > >> >> > > Thanks! > > >> >> > > Alex > > > >> > -- > > >> > Post: opendatakit@googlegroups.com > > >> > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > > >> > Options:http://groups.google.com/group/opendatakit?hl=en > > > >> -- > > >> Post: opendatakit@googlegroups.com > > >> Unsubscribe: opendatakit+unsubscribe@googlegroups.com > > >> Options:http://groups.google.com/group/opendatakit?hl=en > > > > -- > > > Post: opendatakit@googlegroups.com > > > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > > > Options:http://groups.google.com/group/opendatakit?hl=enBecause of historic limits on App Engine database queries the old
0.9.x series did have limits to the number of records it could put in
a CSV.
Aggregate 1.0 RC2 should not have such limits. As Yaw said we will get
to the issue as soon as we can. Please keep the issue updated with
information.
http://code.google.com/p/opendatakit/issues/detail?id=375.
Waylon
···
On Mon, Oct 17, 2011 at 8:46 AM, Alex Pavluck wrote: > I tried another thing....On the topic of I have data in a 0.9.8 server > but I can't download it all as csv, Google Docs says I have more > records, and using Briefcase says I have even more. > > I created another App Engine database running the newest version of > Aggregate and I used briefcase to migrate from the 0.9.8 database to > the new one. It was successful and I can see the data on the server! > However, I have 3,095 records in one form and when I export to csv it > only gives me the first 500 records. What am I missing to pull the > whole database? > > Thanks, > Alex > > > > On Oct 17, 9:33 am, Alex Pavluck wrote: >> Thanks. This is happening to other surveys as well which are on other >> App Engine databases. I think the issue might have started with the >> 0.9.8 upgrade. Please let me know if you need any logs or other >> pieces of data to assist with this issue. >> >> Thanks, >> Alex >> >> On Oct 16, 6:21 pm, Yaw Anokwa wrote: >> >> >> >> >> >> >> >> > unfortunately, the ways you've tried are the only options we have >> > available. mitch will take a look at your issue as soon as he can and >> > we'll get things sorted for you. >> >> > On Sun, Oct 16, 2011 at 14:54, Alex Pavluck wrote: >> > > Thanks, Yaw. Let me know if you need access to the server or the logs to >> > > help you find the issue. >> > > Is there any other way to download the data? >> > > Thanks again! >> > > Alex >> >> > > Alex Pavluck, MPH >> > > Research Informatics Analyst >> > > Lymphatic Filariasis Support Center >> > > The Task Force for Global Health >> > > 325 Swanton Way, Decatur, GA 30030 >> > > cell: (404)647-4904 Fax: (404)371-1138 >> > > apavl...@taskforce.org >> >> > > On Sun, Oct 16, 2011 at 4:16 PM, Yaw Anokwa wrote: >> >> > >> hey alex, >> >> > >> this issue has been filed at >> > >>http://code.google.com/p/opendatakit/issues/detail?id=375. >> >> > >> this is the first i've heard of something like this, and it sounds >> > >> like there is a bug somewhere in csv generation that we need to track >> > >> down. >> >> > >> yaw >> >> > >> On Sat, Oct 15, 2011 at 05:23, Alex Pavluck wrote: >> > >> > Last bit of additional information.... >> >> > >> > I downloaded briefcase and it is still fetching instances and those >> > >> > greatly out number the download csv. Should the instance count match >> > >> > the number of submissions? This would make sense because the team >> > >> > claims to have uploaded over a 1,000 more records than we thought we >> > >> > had on the server. Why is it that everything can see the submissions >> > >> > but the csv doesn't? >> >> > >> > Thanks, >> > >> > Alex >> >> > >> > On Oct 15, 8:02 am, Alex Pavluck wrote: >> > >> >> Server: Google App Engine running 0.9.8 >> >> > >> >> Checked again today and the csv file has 200 less records than the >> > >> >> external link to Google Docs. All the missing data were uploaded >> > >> >> yesterday. Has anyone seen this before? Is there anything to do to >> > >> >> troubleshoot this? Is there anything that I should learn to discern >> > >> >> from the App Engine logs? >> >> > >> >> This is really concerning because if I didn't have the Google Docs >> > >> >> link I would think that the server wasn't receiving the data! >> >> > >> >> Thanks! >> > >> >> Alex >> >> > >> >> On Oct 14, 3:42 pm, Alex Pavluck wrote: >> >> > >> >> > Running 0.9.8 >> >> > >> >> > On Oct 14, 3:42 pm, Alex Pavluck wrote: >> >> > >> >> > > Here is a strange one... >> >> > >> >> > > I have a users uploading data, and I can see the data in the Google >> > >> >> > > Docs link. However, when I download the csv file, the new data is >> > >> >> > > not >> > >> >> > > there??? Any ideas. >> >> > >> >> > > Thanks! >> > >> >> > > Alex >> >> > >> > -- >> > >> > Post: opendatakit@googlegroups.com >> > >> > Unsubscribe: opendatakit+unsubscribe@googlegroups.com >> > >> > Options:http://groups.google.com/group/opendatakit?hl=en >> >> > >> -- >> > >> Post: opendatakit@googlegroups.com >> > >> Unsubscribe: opendatakit+unsubscribe@googlegroups.com >> > >> Options:http://groups.google.com/group/opendatakit?hl=en >> >> > > -- >> > > Post: opendatakit@googlegroups.com >> > > Unsubscribe: opendatakit+unsubscribe@googlegroups.com >> > > Options:http://groups.google.com/group/opendatakit?hl=en > > -- > Post: opendatakit@googlegroups.com > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > Options: http://groups.google.com/group/opendatakit?hl=en >I am running the newest version of Aggregate and there is no way that I can
see to download more than 500 records. The server doesn't show a record
count so there is no way to download them one set of 500 at a time.
Thanks,
Alex
Alex Pavluck, MPH
Research Informatics Analyst
Lymphatic Filariasis Support Center
The Task Force for Global Health
325 Swanton Way, Decatur, GA 30030
cell: (404)647-4904 Fax: (404)371-1138
apavluck@taskforce.org
···
On Mon, Oct 17, 2011 at 12:31 PM, W. Brunette wrote:Because of historic limits on App Engine database queries the old
0.9.x series did have limits to the number of records it could put in
a CSV.Aggregate 1.0 RC2 should not have such limits. As Yaw said we will get
to the issue as soon as we can. Please keep the issue updated with
information.
http://code.google.com/p/opendatakit/issues/detail?id=375.Waylon
On Mon, Oct 17, 2011 at 8:46 AM, Alex Pavluck apavluck@gmail.com wrote:
I tried another thing....On the topic of I have data in a 0.9.8 server
but I can't download it all as csv, Google Docs says I have more
records, and using Briefcase says I have even more.I created another App Engine database running the newest version of
Aggregate and I used briefcase to migrate from the 0.9.8 database to
the new one. It was successful and I can see the data on the server!
However, I have 3,095 records in one form and when I export to csv it
only gives me the first 500 records. What am I missing to pull the
whole database?Thanks,
AlexOn Oct 17, 9:33 am, Alex Pavluck apavl...@gmail.com wrote:
Thanks. This is happening to other surveys as well which are on other
App Engine databases. I think the issue might have started with the
0.9.8 upgrade. Please let me know if you need any logs or other
pieces of data to assist with this issue.Thanks,
AlexOn Oct 16, 6:21 pm, Yaw Anokwa yano...@gmail.com wrote:
unfortunately, the ways you've tried are the only options we have
available. mitch will take a look at your issue as soon as he can and
we'll get things sorted for you.On Sun, Oct 16, 2011 at 14:54, Alex Pavluck apavl...@taskforce.org wrote:
Thanks, Yaw. Let me know if you need access to the server or the
logs to
help you find the issue.
Is there any other way to download the data?
Thanks again!
AlexAlex Pavluck, MPH
Research Informatics Analyst
Lymphatic Filariasis Support Center
The Task Force for Global Health
325 Swanton Way, Decatur, GA 30030
cell: (404)647-4904 Fax: (404)371-1138
apavl...@taskforce.orgOn Sun, Oct 16, 2011 at 4:16 PM, Yaw Anokwa yano...@gmail.com wrote:
hey alex,
this issue has been filed at
http://code.google.com/p/opendatakit/issues/detail?id=375.this is the first i've heard of something like this, and it sounds
like there is a bug somewhere in csv generation that we need to
track
down.yaw
On Sat, Oct 15, 2011 at 05:23, Alex Pavluck apavl...@gmail.com wrote:
Last bit of additional information....
I downloaded briefcase and it is still fetching instances and
those
greatly out number the download csv. Should the instance count
match
the number of submissions? This would make sense because the
team
claims to have uploaded over a 1,000 more records than we thought
we
had on the server. Why is it that everything can see the
submissions
but the csv doesn't?Thanks,
AlexOn Oct 15, 8:02 am, Alex Pavluck apavl...@gmail.com wrote:
Server: Google App Engine running 0.9.8
Checked again today and the csv file has 200 less records than
the
external link to Google Docs. All the missing data were
uploaded
yesterday. Has anyone seen this before? Is there anything to do
to
troubleshoot this? Is there anything that I should learn to
discern
from the App Engine logs?This is really concerning because if I didn't have the Google
Docs
link I would think that the server wasn't receiving the data!Thanks!
AlexOn Oct 14, 3:42 pm, Alex Pavluck apavl...@gmail.com wrote:
Running 0.9.8
On Oct 14, 3:42 pm, Alex Pavluck apavl...@gmail.com wrote:
Here is a strange one...
I have a users uploading data, and I can see the data in the
Docs link. However, when I download the csv file, the new
data is
not
there??? Any ideas.Thanks!
Alex--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options:http://groups.google.com/group/opendatakit?hl=en--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options:http://groups.google.com/group/opendatakit?hl=en--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options:http://groups.google.com/group/opendatakit?hl=en--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
hey alex,
the reason the server doesn't show a record count is due to how google
app engine is designed -- it never actually knows how may records it
has.
anyway, really sorry that things aren't working for you. mitch and
waylon are working the issue and we hope to have a solution posted to
http://code.google.com/p/opendatakit/issues/detail?id=375 soon.
yaw
···
On Mon, Oct 17, 2011 at 10:22, Alex Pavluck wrote: > I am running the newest version of Aggregate and there is no way that I can > see to download more than 500 records. The server doesn't show a record > count so there is no way to download them one set of 500 at a time. > Thanks, > Alex > > > Alex Pavluck, MPH > Research Informatics Analyst > Lymphatic Filariasis Support Center > The Task Force for Global Health > 325 Swanton Way, Decatur, GA 30030 > cell: (404)647-4904 Fax: (404)371-1138 > apavluck@taskforce.org > > > On Mon, Oct 17, 2011 at 12:31 PM, W. Brunette wrote: >> >> Because of historic limits on App Engine database queries the old >> 0.9.x series did have limits to the number of records it could put in >> a CSV. >> >> Aggregate 1.0 RC2 should not have such limits. As Yaw said we will get >> to the issue as soon as we can. Please keep the issue updated with >> information. >> http://code.google.com/p/opendatakit/issues/detail?id=375. >> >> Waylon >> >> On Mon, Oct 17, 2011 at 8:46 AM, Alex Pavluck wrote: >> > I tried another thing....On the topic of I have data in a 0.9.8 server >> > but I can't download it all as csv, Google Docs says I have more >> > records, and using Briefcase says I have even more. >> > >> > I created another App Engine database running the newest version of >> > Aggregate and I used briefcase to migrate from the 0.9.8 database to >> > the new one. It was successful and I can see the data on the server! >> > However, I have 3,095 records in one form and when I export to csv it >> > only gives me the first 500 records. What am I missing to pull the >> > whole database? >> > >> > Thanks, >> > Alex >> > >> > >> > >> > On Oct 17, 9:33 am, Alex Pavluck wrote: >> >> Thanks. This is happening to other surveys as well which are on other >> >> App Engine databases. I think the issue might have started with the >> >> 0.9.8 upgrade. Please let me know if you need any logs or other >> >> pieces of data to assist with this issue. >> >> >> >> Thanks, >> >> Alex >> >> >> >> On Oct 16, 6:21 pm, Yaw Anokwa wrote: >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > unfortunately, the ways you've tried are the only options we have >> >> > available. mitch will take a look at your issue as soon as he can and >> >> > we'll get things sorted for you. >> >> >> >> > On Sun, Oct 16, 2011 at 14:54, Alex Pavluck wrote: >> >> > > Thanks, Yaw. Let me know if you need access to the server or the >> >> > > logs to >> >> > > help you find the issue. >> >> > > Is there any other way to download the data? >> >> > > Thanks again! >> >> > > Alex >> >> >> >> > > Alex Pavluck, MPH >> >> > > Research Informatics Analyst >> >> > > Lymphatic Filariasis Support Center >> >> > > The Task Force for Global Health >> >> > > 325 Swanton Way, Decatur, GA 30030 >> >> > > cell: (404)647-4904 Fax: (404)371-1138 >> >> > > apavl...@taskforce.org >> >> >> >> > > On Sun, Oct 16, 2011 at 4:16 PM, Yaw Anokwa wrote: >> >> >> >> > >> hey alex, >> >> >> >> > >> this issue has been filed at >> >> > >>http://code.google.com/p/opendatakit/issues/detail?id=375. >> >> >> >> > >> this is the first i've heard of something like this, and it sounds >> >> > >> like there is a bug somewhere in csv generation that we need to >> >> > >> track >> >> > >> down. >> >> >> >> > >> yaw >> >> >> >> > >> On Sat, Oct 15, 2011 at 05:23, Alex Pavluck wrote: >> >> > >> > Last bit of additional information.... >> >> >> >> > >> > I downloaded briefcase and it is still fetching instances and >> >> > >> > those >> >> > >> > greatly out number the download csv. Should the instance count >> >> > >> > match >> >> > >> > the number of submissions? This would make sense because the >> >> > >> > team >> >> > >> > claims to have uploaded over a 1,000 more records than we >> >> > >> > thought we >> >> > >> > had on the server. Why is it that everything can see the >> >> > >> > submissions >> >> > >> > but the csv doesn't? >> >> >> >> > >> > Thanks, >> >> > >> > Alex >> >> >> >> > >> > On Oct 15, 8:02 am, Alex Pavluck wrote: >> >> > >> >> Server: Google App Engine running 0.9.8 >> >> >> >> > >> >> Checked again today and the csv file has 200 less records than >> >> > >> >> the >> >> > >> >> external link to Google Docs. All the missing data were >> >> > >> >> uploaded >> >> > >> >> yesterday. Has anyone seen this before? Is there anything to >> >> > >> >> do to >> >> > >> >> troubleshoot this? Is there anything that I should learn to >> >> > >> >> discern >> >> > >> >> from the App Engine logs? >> >> >> >> > >> >> This is really concerning because if I didn't have the Google >> >> > >> >> Docs >> >> > >> >> link I would think that the server wasn't receiving the data! >> >> >> >> > >> >> Thanks! >> >> > >> >> Alex >> >> >> >> > >> >> On Oct 14, 3:42 pm, Alex Pavluck wrote: >> >> >> >> > >> >> > Running 0.9.8 >> >> >> >> > >> >> > On Oct 14, 3:42 pm, Alex Pavluck wrote: >> >> >> >> > >> >> > > Here is a strange one... >> >> >> >> > >> >> > > I have a users uploading data, and I can see the data in >> >> > >> >> > > the Google >> >> > >> >> > > Docs link. However, when I download the csv file, the new >> >> > >> >> > > data is >> >> > >> >> > > not >> >> > >> >> > > there??? Any ideas. >> >> >> >> > >> >> > > Thanks! >> >> > >> >> > > Alex >> >> >> >> > >> > -- >> >> > >> > Post: opendatakit@googlegroups.com >> >> > >> > Unsubscribe: opendatakit+unsubscribe@googlegroups.com >> >> > >> > Options:http://groups.google.com/group/opendatakit?hl=en >> >> >> >> > >> -- >> >> > >> Post: opendatakit@googlegroups.com >> >> > >> Unsubscribe: opendatakit+unsubscribe@googlegroups.com >> >> > >> Options:http://groups.google.com/group/opendatakit?hl=en >> >> >> >> > > -- >> >> > > Post: opendatakit@googlegroups.com >> >> > > Unsubscribe: opendatakit+unsubscribe@googlegroups.com >> >> > > Options:http://groups.google.com/group/opendatakit?hl=en >> > >> > -- >> > Post: opendatakit@googlegroups.com >> > Unsubscribe: opendatakit+unsubscribe@googlegroups.com >> > Options: http://groups.google.com/group/opendatakit?hl=en >> > >> >> -- >> Post: opendatakit@googlegroups.com >> Unsubscribe: opendatakit+unsubscribe@googlegroups.com >> Options: http://groups.google.com/group/opendatakit?hl=en > > -- > Post: opendatakit@googlegroups.com > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > Options: http://groups.google.com/group/opendatakit?hl=en >Thanks, ODK team! I really appreciate your attention to your ODK
system.
Just as a review,
I am running a server with 0.9.8 Agg and it has more records than
either the link to Google Docs shows or the .csv download has in the
download file. I was able to access all the records by downloading
Agg v1 yesterday and migrating to this new server. I used briefcase
to migrate those data from 0.9.8 to 1.0 and all records were fetched
and uploaded to 1.0. The issue now is that I can see all the data on
1.0 but when I try to download them I can only receive 500 records (I
have over 3,000). This is never going to be enough for any of our
surveys.
After you get the download issue fixed I am hopeful you can provide me
with some suggestions as to the best way to go forward with data
collection (ie - have the phones continue to use 1.5 collect and
report to 0.9.8, or update the phones to 1.7 collect and have them
report to 1.0, or continue to use 1.5 collect and report to 0.9.8 and
then migrate the data to the 1.0)
Thanks again!
Alex
···
On Oct 18, 12:58 am, Yaw Anokwa wrote: > hey alex, > > the reason the server doesn't show a record count is due to how google > app engine is designed -- it never actually knows how may records it > has. > > anyway, really sorry that things aren't working for you. mitch and > waylon are working the issue and we hope to have a solution posted tohttp://code.google.com/p/opendatakit/issues/detail?id=375soon. > > yaw > > > > > > > > On Mon, Oct 17, 2011 at 10:22, Alex Pavluck wrote: > > I am running the newest version of Aggregate and there is no way that I can > > see to download more than 500 records. The server doesn't show a record > > count so there is no way to download them one set of 500 at a time. > > Thanks, > > Alex > > > Alex Pavluck, MPH > > Research Informatics Analyst > > Lymphatic Filariasis Support Center > > The Task Force for Global Health > > 325 Swanton Way, Decatur, GA 30030 > > cell: (404)647-4904 Fax: (404)371-1138 > > apavl...@taskforce.org > > > On Mon, Oct 17, 2011 at 12:31 PM, W. Brunette wrote: > > >> Because of historic limits on App Engine database queries the old > >> 0.9.x series did have limits to the number of records it could put in > >> a CSV. > > >> Aggregate 1.0 RC2 should not have such limits. As Yaw said we will get > >> to the issue as soon as we can. Please keep the issue updated with > >> information. > >>http://code.google.com/p/opendatakit/issues/detail?id=375. > > >> Waylon > > >> On Mon, Oct 17, 2011 at 8:46 AM, Alex Pavluck wrote: > >> > I tried another thing....On the topic of I have data in a 0.9.8 server > >> > but I can't download it all as csv, Google Docs says I have more > >> > records, and using Briefcase says I have even more. > > >> > I created another App Engine database running the newest version of > >> > Aggregate and I used briefcase to migrate from the 0.9.8 database to > >> > the new one. It was successful and I can see the data on the server! > >> > However, I have 3,095 records in one form and when I export to csv it > >> > only gives me the first 500 records. What am I missing to pull the > >> > whole database? > > >> > Thanks, > >> > Alex > > >> > On Oct 17, 9:33 am, Alex Pavluck wrote: > >> >> Thanks. This is happening to other surveys as well which are on other > >> >> App Engine databases. I think the issue might have started with the > >> >> 0.9.8 upgrade. Please let me know if you need any logs or other > >> >> pieces of data to assist with this issue. > > >> >> Thanks, > >> >> Alex > > >> >> On Oct 16, 6:21 pm, Yaw Anokwa wrote: > > >> >> > unfortunately, the ways you've tried are the only options we have > >> >> > available. mitch will take a look at your issue as soon as he can and > >> >> > we'll get things sorted for you. > > >> >> > On Sun, Oct 16, 2011 at 14:54, Alex Pavluck wrote: > >> >> > > Thanks, Yaw. Let me know if you need access to the server or the > >> >> > > logs to > >> >> > > help you find the issue. > >> >> > > Is there any other way to download the data? > >> >> > > Thanks again! > >> >> > > Alex > > >> >> > > Alex Pavluck, MPH > >> >> > > Research Informatics Analyst > >> >> > > Lymphatic Filariasis Support Center > >> >> > > The Task Force for Global Health > >> >> > > 325 Swanton Way, Decatur, GA 30030 > >> >> > > cell: (404)647-4904 Fax: (404)371-1138 > >> >> > > apavl...@taskforce.org > > >> >> > > On Sun, Oct 16, 2011 at 4:16 PM, Yaw Anokwa wrote: > > >> >> > >> hey alex, > > >> >> > >> this issue has been filed at > >> >> > >>http://code.google.com/p/opendatakit/issues/detail?id=375. > > >> >> > >> this is the first i've heard of something like this, and it sounds > >> >> > >> like there is a bug somewhere in csv generation that we need to > >> >> > >> track > >> >> > >> down. > > >> >> > >> yaw > > >> >> > >> On Sat, Oct 15, 2011 at 05:23, Alex Pavluck wrote: > >> >> > >> > Last bit of additional information.... > > >> >> > >> > I downloaded briefcase and it is still fetching instances and > >> >> > >> > those > >> >> > >> > greatly out number the download csv. Should the instance count > >> >> > >> > match > >> >> > >> > the number of submissions? This would make sense because the > >> >> > >> > team > >> >> > >> > claims to have uploaded over a 1,000 more records than we > >> >> > >> > thought we > >> >> > >> > had on the server. Why is it that everything can see the > >> >> > >> > submissions > >> >> > >> > but the csv doesn't? > > >> >> > >> > Thanks, > >> >> > >> > Alex > > >> >> > >> > On Oct 15, 8:02 am, Alex Pavluck wrote: > >> >> > >> >> Server: Google App Engine running 0.9.8 > > >> >> > >> >> Checked again today and the csv file has 200 less records than > >> >> > >> >> the > >> >> > >> >> external link to Google Docs. All the missing data were > >> >> > >> >> uploaded > >> >> > >> >> yesterday. Has anyone seen this before? Is there anything to > >> >> > >> >> do to > >> >> > >> >> troubleshoot this? Is there anything that I should learn to > >> >> > >> >> discern > >> >> > >> >> from the App Engine logs? > > >> >> > >> >> This is really concerning because if I didn't have the Google > >> >> > >> >> Docs > >> >> > >> >> link I would think that the server wasn't receiving the data! > > >> >> > >> >> Thanks! > >> >> > >> >> Alex > > >> >> > >> >> On Oct 14, 3:42 pm, Alex Pavluck wrote: > > >> >> > >> >> > Running 0.9.8 > > >> >> > >> >> > On Oct 14, 3:42 pm, Alex Pavluck wrote: > > >> >> > >> >> > > Here is a strange one... > > >> >> > >> >> > > I have a users uploading data, and I can see the data in > >> >> > >> >> > > the Google > >> >> > >> >> > > Docs link. However, when I download the csv file, the new > >> >> > >> >> > > data is > >> >> > >> >> > > not > >> >> > >> >> > > there??? Any ideas. > > >> >> > >> >> > > Thanks! > >> >> > >> >> > > Alex > > >> >> > >> > -- > >> >> > >> > Post: opendatakit@googlegroups.com > >> >> > >> > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > >> >> > >> > Options:http://groups.google.com/group/opendatakit?hl=en > > >> >> > >> -- > >> >> > >> Post: opendatakit@googlegroups.com > >> >> > >> Unsubscribe: opendatakit+unsubscribe@googlegroups.com > >> >> > >> Options:http://groups.google.com/group/opendatakit?hl=en > > >> >> > > -- > >> >> > > Post: opendatakit@googlegroups.com > >> >> > > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > >> >> > > Options:http://groups.google.com/group/opendatakit?hl=en > > >> > -- > >> > Post: opendatakit@googlegroups.com > >> > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > >> > Options:http://groups.google.com/group/opendatakit?hl=en > > >> -- > >> Post: opendatakit@googlegroups.com > >> Unsubscribe: opendatakit+unsubscribe@googlegroups.com > >> Options:http://groups.google.com/group/opendatakit?hl=en > > > -- > > Post: opendatakit@googlegroups.com > > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > > Options:http://groups.google.com/group/opendatakit?hl=enAlex,
We are tracking this issue on our issue tracking system. Specifically,
number 375. http://code.google.com/p/opendatakit/issues/detail?id=375.
Since Yaw created it for you, you won't get the updates unless you
star the issue or you can just navigate to the issue page and check
everyday but getting email notifications are easier
Once this issue is fixed or if Mitch needs more information he we will
be updating the issue so please do monitor it.
Cheers,
Waylon