-
Notifications
You must be signed in to change notification settings - Fork 135
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
allow ordered dict export to yaml #198
Conversation
Great idea Scott - I'll review this later today |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good other than a very minor typo. I suppose this opens us up to moving the DCM and regression YAML methods to OrderedDicts instead of relying on the list in this module. Are you planning to work on that too, or just using this for something external?
Fun sidenote: Python 3.6 now has order preserved in default dictionaries, which probably won't be fully relevant for us for many years, but good to be aware of.
urbansim/utils/tests/test_yamlio.py
Outdated
@@ -146,3 +147,53 @@ def test_frame_to_yaml_safe(): | |||
'col2': {0: 'a', 1: 'b', 2: 'c'}} | |||
y = yaml.dump(d, default_flow_style=False) | |||
assert_dfs_equal(pd.DataFrame(yaml.load(y)), df) | |||
|
|||
|
|||
def test_orderered_dict(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a minor typo here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was planning on using this for DCM updates. It's really just cosmetic, but I wanted a little more control over the look of the yaml.
One thing to note is that constructing a pandas DataFrame with an ordered dict will allow for controlling the column order, but not necessarily the row/index order. However, I think in most cases this can be handled with some re-indexing from the dictionary keys:
pd.DataFrame.from_dict(my_odict, orient='index').reindex(my_odict.keys()).T
Currently, in utils.yamlio, dictionaries exported via convert_to_yaml use a hard-coded list to control the ordering of keys in the output file. This has two drawbacks (1) the list has to be updated when new parameters are added and (2) only the outer dictionaries ordering can be controlled, the ordering of inner dictionaries will not follow the order list.
This PR adds support for exporting ordered dictionaries to yaml in utils.yamlio. For example, the following:
Will have the following yaml:
Note that if a ordered dictionary is provided (as in the current example), its ordering will override the hard-code ordering the yamlio file.