Skip to content

[JOSS Review] Metadata import should not rely on timestamps from filesystem

Hi 👋

when going through your examples and README I noticed that the metadata that you extract from data files relies on timestamps from the filesystem (modification time/access time). This can create problems once data files are moved around or in other ways altered.

I was wondering:

Can you not get the time information from the data files' contents directly? The data files in your examples do seem to include time information in their contents: The RGA files feature a column "Timestamp" and the XPS files have a metadata section "[Info 1]" with keys "Date" and "Time".

Are there specific data formats that you support that do not include basic date and time information? Even in that case, you could still read in the time info from the formats that include it and have a fallback that relies on file timestamps if the data file does not include this info.

When you solely rely on filesystem metadata, you need your users to be careful with their files. I could imagine that this could easily lead to problems.

What are your thought on that?

Edited by Nils Lüttschwager
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information