New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Parse Vensim/Stella models into MIRA template models #272

Merged

bgyori merged 50 commits into gyorilab:main from nanglo123:add-vensim-digestion

Jan 30, 2024

+1,331 −20

Contributor

nanglo123 commented Jan 17, 2024 •

edited

Loading

This PR makes use of the pysd library to parse Vensim and Stella models represented by the .mdl and .stmx, .xmile, or .itmx file extensions repsectively and turns them into an equivalent MIRA template model.

Currently, only a limited number of vensim models are able to be successfully parsed. This module cannot parse vensim models that incorporate series data, MACROs, conditional statements (IF-ELSE), and vensim specific functions (e.g. ZIDZ, PULSE). Additionally, pysd cannot parse Stella models with the .itmx extension and its dependency: parsimonious errors when processing certain stella models with valid file extensions (.stmx or .xmile).

Additionally, for now, we cannot process Vensim models with flows that contain multiple input or output states.

bgyori force-pushed the add-vensim-digestion branch from f3c932b to 20352b6 Compare

January 23, 2024 16:37

bgyori reviewed

View reviewed changes

mira/sources/vensim.py Outdated Show resolved Hide resolved

bgyori reviewed

View reviewed changes

mira/sources/vensim.py Outdated Show resolved Hide resolved

bgyori reviewed

View reviewed changes

mira/sources/vensim.py Outdated Show resolved Hide resolved

bgyori reviewed

View reviewed changes

mira/sources/vensim.py Outdated Show resolved Hide resolved

bgyori reviewed

View reviewed changes

mira/sources/vensim.py Outdated Show resolved Hide resolved

bgyori reviewed

View reviewed changes

mira/sources/vensim.py Outdated Show resolved Hide resolved

nanglo123 added 23 commits

January 24, 2024 13:37


          Initial vensim commit,downloading file and reading bytes

341c402


          Create initial implementation of vensim digestion with while loop

eaf06c6


          Add handling for local mdl files

4f61f93


          Add handling for non-integral multi-line expressions and add more com…

f9b9284

…ments


          Handle variable names that cannot be parsed by sympy


          Handle multi-line comments

8b42fe3


          Remove print debugging statements and unused flag variable

75bb46e


          Handle multi-line expressions in a more robust way

a67bbc5


          Add method handling multi-line expressions into first pass of mdl fil…

078a0d1

…e to convert variable names


          Import pysd library and read vensim file

314827b


          Create template model from information extract from pysd except for t…

a77e083

…emplates


          Add processing for observables

362d4a1


          Add initial implementation of automated processing of templates for s…

ebb289c

…ir.mdl example


          Remove addition of initials to template model and make demo notebook …

dd09287

…of Vensim SIR Template Model


          Change notebook stratify on new age stratum and rerun it

c90bb0c


          Change file path for sir.mdl file

87c5ce5


          Load sir.mdl file with url rather than relying on local copy

cee229f


          Update notebook to compare parameters and rate laws between stockflow…

877bc08

… and vensim models and add initials to vensim tm


          Re-order cells of notebook

95d2760


          Format rate law printing in a neater way

d6591bd


          Rerun cell

fb747ca


          Change vensim notebook based off suggestions

a4651af


          Use temporary file to store data retrieved from url when creating a v…

ff05a5f

…ensim model

nanglo123 added 6 commits

January 24, 2024 13:37


          Add unit handling for parameters and initials that may be missing units

59cd0a6


          Add updated comments

7f6a6f8


          Delete example directory and add example sir mdl to module docstring

f083a6f


          Add pysd to list of required installs in setup.cfg

0c03b68


          Add initial values and remove initial expression from parameter

7e2dd13


          Add option for processing local files and add API functions for vensim

71d0fc5

nanglo123 force-pushed the add-vensim-digestion branch from 845d77f to 71d0fc5 Compare

January 24, 2024 18:37

nanglo123 added 13 commits

January 24, 2024 19:29


          Standardize units

967d91a


          Update docstrings for modules and add vensim and stella module to sou…

2218c7f

…rces rst


          Change url type in docstring and add initial test file for system dyn…

f7039ca

…amics


          Generalize logic for finding controllers

d35697a


          Extract initial state values from model state property rather than si…

3b41a4f

…mulating the model to avoid additional dependencies being imported


          Add support for processing stella models through file or url, refacto…

97fad13

…r code such that some software specific preprocessing is done before calling generic method for parsing model


          Update docstring for stella and vensim file

ef3423a


          change variable name and remove pass statement from test_system_dynamics

2afa0b5


          Extend system dynamics notebook by ingesting SIR stella model and cha…

3015c97

…nge demo notebook name.


          Delete original Vensim notebook due to renaming issue

06154af


          Refactor stella code to account for order of operations, clean up par…

b1764f7

…sing code text processing


          Remove pass statement from model parsing file

bffd3f0


          Rename variables and dictionary fields for easier readability

3f873ea

bgyori reviewed

View reviewed changes

mira/sources/system_dynamics/parse_pysd_model.py Outdated Show resolved Hide resolved

bgyori reviewed

View reviewed changes

docs/source/sources.rst Show resolved Hide resolved

nanglo123 added 3 commits

January 30, 2024 13:12


          Rename pysd parsing module, add it to sources and change code to refl…

3361fcf

…ect renamed module


          Pass a sympy object into SympyExprStr wrapper to avoid error in io mo…

d680664

…dule, remove main method from stockflow sources and change display name for variables in stockflow modeling


          Further add to system dynamics test to test for correctness and add e…

f207291

…nd to end test

bgyori changed the title ~~Parse Vensim models into MIRA template models~~ Parse Vensim/Stella models into MIRA template models

nanglo123 and others added 2 commits

January 30, 2024 15:06


          Add extra barline under module name in sources.rst

3b22826


          Add all to module level

5d74509

bgyori approved these changes

View reviewed changes

Member

bgyori left a comment

Looks great, thanks!

bgyori merged commit 7d41dcf into gyorilab:main

2 checks passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet