Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How popular should a file format be, to be added here? #278

Closed
Luapulu opened this issue Nov 3, 2020 · 5 comments
Closed

How popular should a file format be, to be added here? #278

Luapulu opened this issue Nov 3, 2020 · 5 comments

Comments

@Luapulu
Copy link

Luapulu commented Nov 3, 2020

I have a loader package for a format used in a little corner of physics. I'd guess there might be tens of people using this format at most. Nonetheless, I've followed FileIO conventions while writing it so I could easily add it here.

How popular should a file format be, to be added here?

@timholy
Copy link
Member

timholy commented Nov 3, 2020

My view: if it has nontrivial magic bytes, it's presumably unique so it won't get in the way of anything else. In that case, add it without hesitation.

If there are no magic bytes, and if the extension is not unique or is too short to be plausibly unique, then popularity becomes an important consideration.

@Luapulu
Copy link
Author

Luapulu commented Nov 3, 2020

There aren't magic bytes but every line ends in "physiDet" and the first line has to contain three integers. (It's a human readable delimited-ish format) The ending is ".root.hits" so conceivably there may be some other simulation / data analysis software based on ROOT, that saves to a ".root.hits" file format.

Example

@timholy
Copy link
Member

timholy commented Nov 3, 2020

Interesting. 👍 for it being easily recognizable. 👎 for the fact that it needs a custom detector, and the more of those we have to run the slower the general thing becomes. hits seems like a pretty unique extension, though. I forget (and don't have time to check) if we only run detectors on extensions that match.

@Luapulu
Copy link
Author

Luapulu commented Nov 4, 2020

src/query.jl#L72-L90 seems to suggest, that if only one format exists for an extension and it has no magic bytes, it will get loaded as is without further checking.

@Luapulu
Copy link
Author

Luapulu commented Nov 4, 2020

Ok, quick correction on my specific file format: The line ending may be different. First line still has three integers, but following lines may vary in terms of the ending string. For my use case this hasn't made a difference, but if this is to be added here I'd have to generalise my code first.

I'll let things mature a bit and may be come back to this later if makes sense to add rarely used formats here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants