question: red timestamps?

[muellerto]

Last week I copied some files from a camera (I mean JPG pictures) onto my disk, and then I had them in the long view of clifm. Most of the files had coloring as expected but about ten files (all at the end of the list) had red timestamps. Why? They all came from the same SD card and have been made with the same camera firmware, they all had the same day, only the time was different - so what has been wrong with these timestamps?

[leo-arch]

Hi @muellerto. Have these files timestamps in the future by any chance? If not, could you post an example image?

[muellerto]

They all were made on the same day and the camera has been set up correctly some days ago, so the difference can only be seconds. So I can't imagine that there were some in future, must check this. I'll try to make a screen shot (at home). The color was bright red, looked like #FF0000.

[muellerto]

I guess something else: UTC and local time. When my timestamps from the camera are local times (I have UTC + 1h + one additional hour DST) and your props_now is UTC then we get a difference of two hours. This could then indeed lead to future timestamps. And that all these files are at the end of the list (this means: the youngest files) is also an indication for that.

Would be interesting which timestamps are in the EXIF data. I do a research about this.

I have probably no option for the timestamps on the camera. The firmware makes the timestamps and I must live with them. I also can easily live with them, it was just a question how this can happen. OK, I make pictures of the future :)

[leo-arch]

A difference between UTC and local time is a possible explanation. Indeed, props_now is generated using time(2), whose description says:

time() returns the time as the number of seconds since the Epoch, 1970-01-01 00:00:00 +0000 (UTC).

But I'm not sure.

[leo-arch]

I guess this explains the difference in your files timestamp colors: those taken in the last 2 hours (1 if not in DST) will be seen by clifm as future timestamps. I guess after 2 hours the red color will be gone (because the system time has already reached the files time).

[muellerto]

Think so. I try to reproduce this, then we can be sure.

If this is true, the guys of the camera firmware (Canon) made an error storing their files in the file system of the SD card. But hey, timestamps in a file system are always illustrious. On Windows you can use three applications and you get three different timestamps for the same file. Even cmd.exe (dir command) and the Explorer can differ in hours depending on the concrete timezone and DST. It's ridiculous.

[leo-arch]

I'm not sure the guys at Canon made a mistake, but it is still an interesting issue.

As a workaround, you can make the second field (wrong times) of DateShades match the third one (most recent times), so that these future timestamps will use the same color as the most recent ones. Something like this:

DateShades="2,231,231,253,250,247,244"

But this is just a workaround hiding a real issue. Maybe there's some way to subtract (or add) the timezone difference regarding UTC from the file timestamp to later compare it to the system current time (which is in UTC).

[muellerto]

It is like we discussed this morning :) For clifm the timestamps are in future. Looks like that:

It's indeed ridiculous:

• I took the last two pictures indeed at 16:43 (Middle Europe Standard Time + 1h DST), the camera knows that
• the camera says the pictures are from 17:43 (it struggles on it's own damned timestamps)
• when I copy the files onto my Linux machine ls, lsd and erdtree say the files are from 19:43! Three hours difference - how come???
• clifm says 19:43
• feh says 17:43
• phototonic says 19:43, this is shown in a view of all the EXIF data
• the raw EXIF data timestamp is 17:43

I guess one source of this mess is that the camera itself already thinks wrong. I'll check this ...

[leo-arch]

It's a mess, certainly. Of course, the true timestamp is 16:43. Maybe your camera ignores DST when showing the time. As to 19:43, this info comes from the stat struct, which takes the data from the file metadata written by the software in the filesystem.

Be it as it may, the truth is that even if we read the EXIF data, the files would still be in the future! If the camera do things wrong in the first place, there's nothing we can do, since we rely on the files metadata.

[leo-arch]

After some research, this is what I got: timestamps (both the current machine time and the file's time) are (or should be) stored in UTC. It is only afterwards, if a program wants to display this info in human readable format, that UTC time is converted to the localtime of the running machine (note that the original localtime is usually not preserved, only its UTC format).

This means that time(2), which we use to get the current time, and stat(2), used to get the file's time, should return times in UTC, and thereby, our get_color_age() function should be comparing UTC times. If this is the case, why do you have files in the future? The only thing that comes to my mind right now is that your camera is storing timestamps in (some) localtime and not in UTC (later clifm, or ls or whatever, assuming UTC times, convert these times back to the machine's localtime, in your case, adding 2/3 extra hours).

EDIT: The EXIF timestamp is metadata embedded within image files and is independent from the timestamp actually written into the filesystem. This explains the different timestamps shown by image viewers (which usually consult EXIF data) and file managers, which by contrast consult the filesystem metadata.

[muellerto]

Yesterday I saw that the camera has two general modes:

• it can use local time of a configurable time zone + DST for everything, in this case also the file timestamps are in local time
• it can use UTC (without any time zone or DST) for everything, in this case the file timestamps are also strict UTC ... AND THEN they remain correct when I copy the files somewhere else

I think I'll run the camera now on UTC. The only disadvantage is that the camera display shows also a UTC time. OK, this is bearable.

I think time() and stat() (and all the other functions and C++ classes based on them) take just what they get from the file system. When the file system has a UTC timestamp they retrieve UTC, when the file system has a local timestamp then they return local. This is because the file timestamp is just an integer value. It can even be mixed from file to file. And a file system with it's functions can't guarantee that the file timestamps are all in a concrete format, functions like utime() take just seconds or nanoseconds (from 1970) as parameters, there's never a time zone or DST and that it therefore must be UTC is just an assumption. Many people don't care about this, I remember long discussions with workmates and superiors not even understanding the problem.

EXIF has properties containing a time zone offset and more, but only since version 2.31 (2016). But the file creator software can ignore filling them when it writes a file. So the EXIF timestamp has in many cases the same problems like a file timestamp.

[leo-arch]

Thanks for the info @muellerto! This was a really interesting discussion. Please mark it as solved if it applies.

[muellerto]

And I thought I have a usable tool for taking pictures of the future ...

[leo-arch]

Almost.