Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maybe handle new DAQ storing strings as array of objects of type bytes #500

Open
D4vidH4mm3r opened this issue Mar 15, 2024 · 3 comments
Open

Comments

@D4vidH4mm3r
Copy link

It was observed that the new data aggregator version stores strings in the RUN section (specifically, classId came up) as arrays of one element of type bytes. This means that using get_run_value returns something like array([b'some string']) instead of a simple string.
In case this behavior is here to stay, it would be nice to have get_run_value handle getting and decoding the actual string automatically. Would of course need some backwards compatibility; see the hotfix documented in the EXtra-redu repository.

@egorsobolev
Copy link
Member

@steffenhauf asks: would you like to keep the new format, or should we make sure that strings in the run section are scalars again?

I would roll back. I think, it makes sense to store string scalars in the same way as other scalars. What do you think?

@philsmt
Copy link
Contributor

philsmt commented Mar 16, 2024

I agree and would prefer rolling back:

  1. Conceptually, the point of RUN values is that they represent a single value per data collection and are hence scalar with respect to the train axis. While there are already fake arrays in METADATA, we can fortunately hide most of these.

  2. Trivial file format changes like these are frustrating for users. Unless there is a strong reason for this change, we should really avoid them. While EXtra-data is used by most users, it's not obligatory and we pretend to offer a stable file format.

@dgoeries
Copy link
Member

This should be fixed. We made a mistake

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants