Inflated image size - quality/compression

1. What is the problem? Be very detailed.
I really hope I've missed something simple here, but I can't seem to get reasonable image sizes unless I really reduce the max-pixels value, which isn't practical.

Image quality ex ODK Collect appears to have zero jpg compression and actually inflates image file size.
Resized images (to ~40% area) are similar in size to the original file, unresized images are nearly double the original size.

Image quality ex marked up ODK Collect is dramatically reduced in size however, close to a 75% irfanview jpg. (see 4c vs 4e)

Externally saving a sample image at 100% jpg quality is barely larger than the no resize Collect image (5a vs 1b & 1d) or actually smaller (5b vs 2c)

Externally saving the images at 75% jpg quality results in around a ~5x reduction in size from ODK or ~2.5x reduction from device camera.

Increases data volume & time to upload/download, consumes 2-5x DB space unnecessarily.

2. What app or server are you using and on what device and operating system? Include version numbers.
ODK Central 1.3.3 (client 1.3.2)
ODK Collect 2021.3.2, Android 11, Samsung Galaxy Tab S6

3. What you have you tried to fix the problem?
Reading documentation
Adjusting device settings
Testing different combinations of max-pixels and capture vs select image (see (#5))

4. What steps can we take to reproduce the problem?
Capture a photo and compare file sizes

5. Anything else we should know or have? If you have a test form or screenshots or logs, attach below.

Compression
Previously the default camera could be changed to one that has options for capture resolution and JPG quality (eg Open Camera) but this is not possible anymore and the only options on recent Samsung devices are HEIF (not available in Collect) or JPG, no resolution/quality options. But it doesn't seem that this matters, as even selecting a gallery image (eg 1a) can result in a file size that's similar (with a 60% shrink 1c) or larger (no resize 1d & 1d).

Testing with Galaxy Tab S6, sensor resolution 4128*3096:

  1. 'Choose Image'
    1a. Samsung camera capture outside Collect, original size 4.12mb
    1b. 'choose image' using above photo, max-pixels=4128, 7.1mb
    1c. 'choose image' using above photo, max-pixels=2560, 3.8mb
    1d. 'choose image' using above photo, no max-pixels, 7.1mb
    1e. 'choose image' using above photo, select annotate image but don't markup, just save & close, gives 2134x1600 (1600 screen pixel width) image, 0.5mb

  2. 'Take Picture' of as close to same scene as above photo as possible,
    2a. max-pixels=4128, 5.1mb
    2b. max-pixels=2560, 3.1mb
    2c. no max-pixels, 8.8mb

  3. Open Camera, capture as close to above scenes as possible
    3a 4128x3096Q75: 1.2mb
    3b 4128x3096Q100: 5.6mb
    3c 1440x1080Q75: 0.2mb

  4. Irfanview save as jpg, quality 75:
    4a : 1a 4.12 --> 1.6mb
    4b : 1b 7.1 --> 1.6mb
    4c : 1c 3.8 --> 0.7mb
    4d : 1d 7.1 --> 1.6mb
    4e : 1e 0.5 --> 0.5mb
    4f : 2a 5.1 --> 1.3mb
    4g : 2b 3.7 --> 0.7mb
    4h : 2c 8.8 --> 1.6mb

  5. Irfanview save as jpg, quality 100:
    5a : 1a 4.12 --> 7.5mb
    5b : 2c 8760 --> 8739kb

Annotation

Annotated image resolution appears to be a limit of device screen resolution since at least 2012. I have chosen a high res device to work around this, but if it is eventually resolved it would open up the use of cheaper and physically smaller devices without sacrificing annotated image quality (or having to capture an image without annotation and also capture one that allows annotation).

I don't know if this is even possible or probable, and I can work with a ~2MP annotated image, but thought I'd tack it onto this post as a side note.

1 Like

Hi @ahblake
Thanks for creating this topic and describing the issue. I've never noticed it and I've never heard about it. I will take a look at it when I'm free.

1 Like

Thanks,
It's quite strange, and I would have thought people would notice the very large image sizes before, so I was hoping that it was a simple error on my end.

Let me know if you would like me to provide any further details or testing.

I only have an SM-T865 Tab S6 and an SM-G973 S10 to hand but can ask some other people to complete a test form on other devices.

From the image rotation beta testing on a completely different device with selectable capture resolution, the filesizes are also excessive

Thanks I was able to reproduce the issue.
Now it is reported in our repository and we will try to fix it https://github.com/getodk/collect/issues/4978

From the image rotation beta testing on a completely different device with selectable capture resolution, the filesizes are also excessive

@ahblake
Do I understand correctly that in v2022.1 the problem does not occur if you don't specify max pixels. The size should be just the same (original). Otherwise it might occur what of course doesn't make sense because that option was introduced to produce smaller files and now it might have the opposite result (fewer pixels but bigger size). Could you confirm because this is what I noticed?

I'll test a few combinations for you this weekend.

So, permutations of: no annotate / capture image & select image/ max-pixels & no max-pixels, are there any other things to include?

I'm mostly interested in taking picture and choosing picture so the two options that you have when you use Image Widget. Those are the most common cases I think and from my investigation the problem now (in v2022.1) should not occur if you don't specify max-pixels in settings or on a form level.

If there are any other cases where the problem occurs please let us know.

Ok, is it worth including the project image size limit or is that basically the same as max-pixels?

What is that? Is there any option like that in Central?

Sorry, I was unclear, in Collect you can go to project settings \ form management \ image size and select from camera original or XS S M L.

Using collect settings and defining max-pixels on a form level in xls has the same result so it doesn't matter.

Ok;
ODK Collect 2022.1.0, Galaxy Tab S6 (SM-T865), held in a single location so the images are as close to the same content/lighting as possible.

Downloaded from Central

  1. Capture image, no max-pixels : 8572kB, 4128x3096
  2. Choose image, no max-pixels : 3349kB, 4128x3096
  3. Capture image, max-pixels = 4000 : 6916kB, 4000x3000
  4. Choose image, max-pixels = 4000 : 6974kB, 4000x3000
  5. The 'Choose image' source above taken in tablet camera app : 3349kB, 4128x3096

The above images, resaved at Q75 in irfanview

  1. 1788kB
  2. 1504kB
  3. 1543kB
  4. 1466kB
  5. 1504kB

Looks like capturing the image results in a huge file even exceeding a 100% quality save out of irfanview, and choosing an image and then applying `max-pixels' also results in a huge file.

And even for the unaltered chosen image, the size can still be halved (on this tablet at least) by lowering to 75% quality. Resaving the 'choose-image' source at varying qualities:

  • Q100 : 6335kB
  • Q95: 3463kB
  • Unaltered: 3349kB
  • Q90 : 2460kB
  • Q75: 1504kB

Thanks @ahblake

so to sum up: what I said about using max-pixels (that it might produce bigger files) is true. Then it looks like choosing images (without max-pixels) does not change file size but capturing does?

This is strange and I can't reproduce it. Maybe your camera app works with different settings if you open it via Collect vs directly?

On current Samsung Android builds there are no (that I can find) camera settings available to the user that affect compression or resolution in the default camera app. It's limited to HEIF : yes/no

I'll try the same tests on some other hardware I can get access to and see what happens for the capture image, no max-pixels case.

Ok I'm going to ask our QA team to check their devices too.
@kkrawczyk123 @Szymon_Rujner please check if there is any difference in image file size if you take photo directly opening your camera app vs capturing image via ODK Collect using normal Image Widget

I was able to reproduce the issue on Xiaomi Redmi 9T with Android 10, with the default settings, the photo captured by ODK Collect was a few MB larger in size than the same photo taken by camera app.

The same issue was not reproduced on Pixel 3a with Android 11

1 Like

Can you both check if you have the same quality selected in settings (camera settings not in Collect) when you start the camera app directly vs from Collect. Different camera apps might work in different ways for example on my device I can set standard quality if I open the camera app directly and high quality if I do the same via Collect (those settings will be remembered if I do that once) then the result will be similar to what you experienced - bigger images captured via Collect but it's not a bug.

Current stock Samsung OS camera doesn't allow you to change image resolution or quality. There's no way for me to create a 7-9MB image file in the stock camera on the Tab S6 or S10

The camera in Lineage OS on the S5 does allow you to change them, I have tested at High and Standard.

I get a different result, changing resolution and quality in the stock camera app directly on the S5 is saved, and the settings are carried over to ODK, likewise changing these settings in the camera app loaded via ODK are preserved and carried over to the stock camera app. They don't have separate settings maintained

Also, it doesn't remember which side camera was last used in ODK on all my devices, it always defaults to the last used camera in the stock camera app opened directly.

Some more image tests below, it looks like when max-pixels reduces an image's dimensions the filesize increases dramatically on all my devices, but Capture Image without max-pixels producing a massive file size is peculiar only to the Galaxy Tab S6 of the devices I have (and also the only device I am actually using ODK with)

Galaxy S10 / Android 11 One UI 3.1
ODK Collect 2022.1.0

  1. Capture image, no max-pixels : 2889kB, 4032x3024
  2. Choose image, no max-pixels : 2877kB, 4032x3024
  3. Capture image, max-pixels = 4000 : 6341kB, 4000x3000
  4. Choose image, max-pixels = 4000 : 6295kB, 4000x3000
  5. The 'Choose image' source above taken in tablet camera app : 2877kB, 4032x3024

The above images, resaved at Q75 in irfanview

  • 1254kB
  • 1249kB
  • 1288kB
  • 1247kB
  • 1249kB

Galaxy S5 / Android 11 Lineage
ODK Collect 2022.1.0

Camera settings 15.9MP, Standard quality.

  1. Capture image, no max-pixels : 1553kB, 5312x2988
  2. Choose image, no max-pixels : 1563kB, 5312x2988
  3. Capture image, max-pixels = 4000 : 4170kB, 4000x2250
  4. Choose image, max-pixels = 4000 : 4170kB, 4000x2250
  5. The 'Choose image' source above taken in handset camera app : 1563kB, 5312x2988

The above images, resaved at Q75 in irfanview

  • 1072kB
  • 1078kB
  • 695kB
  • 695kB
  • 1078kB

Galaxy S5 / Android 11 Lineage
ODK Collect 2022.1.0

Camera settings 15.9MP, High quality.

  1. Capture image, no max-pixels : 4675kB, 5312x2988
  2. Choose image, no max-pixels : 4715kB, 5312x2988
  3. Capture image, max-pixels = 4000 : 4514kB, 4000x2250
  4. Choose image, max-pixels = 4000 : 4516kB, 4000x2250
  5. The 'Choose image' source above taken in handset camera app : 4715kB, 5312x2988

The above images, resaved at Q75 in irfanview

  • 1102kB
  • 1124kB
  • 730kB
  • 736kB
  • 1124kB

Galaxy S5 / Android 11 Lineage
ODK Collect 2022.1.0

Camera settings 6MP, Standard quality.

  1. Capture image, no max-pixels : 788kB, 3264x1836
  2. Choose image, no max-pixels : 788kB, 3264x1836
  3. Capture image, max-pixels = 4000 : 783kB, 3264x1836
  4. Choose image, max-pixels = 4000 : 788kB, 3264x1836
  5. The 'Choose image' source above taken in handset camera app : 788kB, 3264x1836

The above images, resaved at Q75 in irfanview

  • 582kB
  • 582kB
  • 578kB
  • 582kB
  • 582kB

Ok so it looks like some camera apps might use different qualities when used directly vs when called via an external app (like ODK Collect). There is nothing we can do in this case (if those apps do not allow us to change it in their settings). To handle cases like that we should use scaling (max-pixels on a form level or in app settings). Of course there is a bug and even if scaling down is used you might end up with a bigger file but we have a pull request to address this issue: https://github.com/getodk/collect/pull/5004 so it should be fixed soon.