Inaccurate GPS data or missing GPS data after upload

Backlight 4 & 5 Support
#1

@Ben
Hi, now I have problems with correct GPS data after upload with Backlight Publisher.

IPhone XS, IOS 14.01.
Location service is enabled. For Safari - exact location, for photos - exact location.
I did some pictures. The gps data in the photos are correct, the position in maps is correct.
When I upload an image from my iphone with backlight publisher and safari browser, the gps data is gone.
If I upload the same image from my iPad out of photostream, the gps data in available.
But - when I click on the map symbol, the wrong position is shown.
I have tested this some times. Maybe I did soething wrong, maybe it is an IOS problem.

Any ideas?

Rainer

Edit:

Just talked with a friend. He found, that in the gps position data, the seconds are not considered after upload. Only the degree and minutes are correct.
As I wrote, in the file on my iphone, all gps data is correct and the position on the map also.

#2

Ben,
I’m seeing the same thing. Seconds are being stripped when uploading from my iPhone 6 iOS 12.4.8

Comparative albums here:
https://backlight-2-100.barbeephoto.com/test/

It would be interesting to find out if this is also happening with other brands of phones. Anyone out there want to give it a try?

#3

No solution?

Rainer

#4

I’ve taken the weekend off so haven’t looked into it yet.

#5

Preliminary findings:

  • There seem to be different ways of handling seconds in GPS coordinates. There is a fair bit of manual parsing to convert these into a decimal value. I’ve found that at least in photos from iOS the seconds were being dropped. I’ve coded a fix for this, which we’ll release soon.

  • I’m getting mixed results for the GPS data in iOS photos. It’s in some of the uploaded JPEGs but not others. The JPEGs with missing GPS data also have the bulk of the EXIF metadata missing. This missing data may be getting lost when Uppy (the library that uploaded JPEGs) creates files for upload. I need to spend more time looking into this.

@Rainer_Goergen, are you finding that GPS data is getting lost on all iOS uploads or only some?

#6

Further research shows that iOS strips most metadata, including GPS data, upon uploading through Safari (and presumably in all webkit-based browsers, which means all browsers in iOS. Chrome exhibits the same behaviour). There is no option to prevent this. I believe that the one image that I managed to upload with GPS data was through my laptop instead.

For now, if you need GPS data for images uploaded from iOS, your only bet is to indirectly upload them from a Mac or PC. There are technical solutions such as creating a Safari App Extension, or creating a native iOS app, but we don’t have the resources to work on these - iOS development is very time-consuming and we don’t have the skill set.

#7

@Ben

GPS data is getting lost on all iOS uploads.

But it is curious, that an upload with iPad OS from photostream kills only the GPS seconds.
I will do some more tests today.

Rainer

Edit:

Tried now again with MacPro and OS High Sierra.
Import from iPhone to Photo App, iPhone share with iMessage to Mac …

All images in the Photo App on my MacPro contain correct GPS data, showing correct on maps.
After upload with Backlight Publisher, the seconds are gone.

The same with Mozilla Firefox.

Same images upload with Lightroom Publisher - all uploads are with correct GPS data.

So there must be a bug somewhere with Backlight Publisher not with IOS or Safari.

Rainer

#8

Thanks for your further investigation. As above in my preliminary findings, there is an issue on Backlight’s handling of seconds. I will push an update after further testing.

#9

Hi @Rainer_Goergen, I have pushed an update that may fix the handling of seconds in your photos.
To update, can you visit Backlight Modules, and on the line for module-publisher click Reinstall.
Can you try that and re-upload an image that was previously losing seconds during upload?

#10

@Ben

Hi, tried it, after reinstall, clearing cache and new upload from my iPhone,
all GPS data is deleted again.

—-

When I upload the images with my iPad from photostream, the correct GPS data exists.

Rainer

#11

Hi Rainer, can you confirm that the fix improves GPS accuracy for new photostream uploads?

The results for iOS are as expected. No fix for this will be possible without writing a native app or plugin.

#12

Just sent you an email with photos and link.

The photos without GPS data are from my iPhone, the photos with GPS data upload from iPad via photostream - all correct.

Rainer

#13

Hi Rainer, thanks for the link. That all looks good. It’s unfortunate that there’s no setting to enable GPS to be included with Safari uploads. After all, photos from phones are more likely to have GPS details than photos from dedicated cameras.

#14

@Ben

But why is photo stream ok? The same photos as on my iPhone?

I make a photo with my iPhone. The photo is stored in camera roll and photo stream.
When I upload the photo from photo roll via Safari, all GPS data is deleted, when I upload the same photo from photo stream, all data is correct. When I store the same photo from photo stream again into photo roll and upload it via Safari again to my album, all GPS data is now also correct. There must be a bug in backlight publisher.

Rainer

#15

I installed the fix Ben mentioned earlier and am now able to upload from my iPhone with GPS data intact.
The phone images are stored on the cloud so I’m not sure if that means they get downloaded to the phone and then sent up to the album through the Backlight Publisher.
Either way, it seems to be working seamlessly for me.

#16

“But why is photo stream ok? The same photos as on my iPhone”

Because iOS’s security model strips GPS data from uploads made directly with Safari. It’s a security feature to prevent random website finding out where you are or have been. Think about children with iPhones. Apple clearly deems apps that have been screened by Apple and intentionally installed to be more trustworthy than random websites.

“There must be a bug in backlight publisher.”

No. This behaviour has been clearly explained. There is plenty of information on the Internet that backs up the above. Feel free to look through the Exif handling code in Backlight and report a bug.

#17

Thank you Ben, it was only a random test where I found this peculiarity.
So in future I will upload such photos only via photo stream or after upload to my NAS and then via Mac.

Thank you once more, Rainer

#18

Hi @Rainer_Goergen, we’ve made some further discoveries. The availability of EXIF data from the camera roll in Safari depends on a couple of things:

  1. The Formats setting in the Camera settings (available under Camera > Formats in your iOS settings). If set to High Efficiency, which is the default, then EXIF data is largely removed when uploading from Safari. If set to Most Compatible then the EXIF data is left intact, including GPS data.

That to me looks like a bug in iOS. If privacy was an issue then the GPS data should not be available with either setting. If privacy isn’t an issue then the GPS data should be available in both settings.

For iPhones that do not provide this setting, the EXIF data is intact.

Changing this setting does not have any effect on photos that are already in your camera roll.

  1. Apparently, if you edit photos in the camera roll, then the EXIF data may be intact, even if the Formats is set to High Efficiency. I have not had any luck with this, though I have one photo that was taken with High Efficiency that does manage to upload with GPS data. I have no idea why this one is fine, but other photos I have edited aren’t.

I’ve put together a basic test script to see exactly what metadata is in the file upon upload. This test script takes away any further code that may have been interfering with EXIF, however I have found the results the same with this script and Backlight’s uploader. To use the script, create a file on your server with the following contents then open the link to your uploaded file on an iPhone and anywhere else you want to try.

<form enctype="multipart/form-data" action="./exif.php" method="POST">
	<p><input type="file" name="file" value="upload" /></p>
	<p><input type="submit" name="submit" value="Upload"/></p>
</form>

<?php
if (isset($_POST) && count($_POST) && isset($_FILES['file']['tmp_name'])) {
	$exifData = exif_read_data($_FILES['file']['tmp_name'], 'IFD0');
	die('EXIF Data: <pre>'.print_r($exifData, true).'</pre>');
} 
?>

I’ve uploaded this to my server and can email you the link if you’d like to try it.

#19

@Ben
This was also my supposition with high efficiency format. I never could use this format with my iPhone 6s+. Since 3 weeks I have a new iPhone XS max, where this format is standard. I have now changed this to max. compatibility.

Yes send me the link for testing.

Rainer

#20

Hi @Rainer_Goergen, the thing is that HEIC files do include the exif data including GPS details, and iOS is producing JPEG files automatically from those files, and those files do have an EXIF data component. But that only includes information that could otherwise be determined by looking at the file itself, such as dimensions. So it looks like Apple has everything in place but fails to populate all of the other available EXIF details in the generated JPEGs. It’s not a shortcoming of the HEIC format but Apple’s handling of it when JPEGs are produced.