New data loading merge#322
Closed
cormacmlynch wants to merge 26 commits into
Closed
Conversation
First pass at implementing a new way to store and read in data files. This method stores variables as CSV files (one for each variable). Users are expected to edit CSV files directly. CSVs are then read into the model, removing the xlsx->csv step. Polars is also used in place of Pandas for speed gains (not clear yet how much the gain in speed is due to this swap). Also included is a script (convert_inputs_to_new_format.py) which can be used to convert old style files to the new format. Model output has been tested for Tr, P, Fr, H. Shares results the same as before.
no longer needed as part of new data loading
Removing some old text left over from prototyping
Agent-Logs-Url: https://github.com/cpmodel/FTT_StandAlone/sessions/c7808ca3-18df-4bbb-b204-6c66dc5ff412 Co-authored-by: Femkemilene <26096675+Femkemilene@users.noreply.github.com>
# Conflicts: # SourceCode/model_class.py # SourceCode/support/input_functions.py Co-authored-by: Femkemilene <26096675+Femkemilene@users.noreply.github.com>
More clear file name
ftt_gmp_main solve function runs successfully when called.
Reflects the removal of the masterfiles folder. Instead looks at S0 folder. Maybe we need to find a more robust way to do this?
These had stopped working with the new data loading changes.
This reverts commit 5421ab2.
Previous commit reverted because linking the progress bar in the GUI seems to significantly slow down the runtime. Adding a brief message for when inputs are being loaded.
…ata-loading-merge
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merging the most recent edits in the GMP branch with the newer data loading method. This will allow us to edit/create data in csv format that should reduce merge conflicts and make changes much clearer.