Designing maps for gridded precip effect on forcing thresholds

[ChristinaB]

• review DHSVM spatially variable output options (see link below and suggested three soil related outputs)

• make document on setting up config file for printing maps (optional)

• review state variable list with erkan (before or after draft results)

• talk to Bart Dennis and Mark about copying some documentation from old website - Christina - pyDHSVM

• 1. run script to myconvert all the files in all the folders (ends with a big long ascii file)

• 2. postprocessing scripts to python -- see .m code uploaded with example from Chehalis (ends with an ascii file for each date)

• 3. pseudo code to postprocess from ascii dhsvm output to Landlab or matplotlib (easy visualization - future uses attaching variables to rastermodelgrid)

• 4. use gridclimdict dataframe as input to Jim's exc prob code in OGH

• pyDHSVM unit test

[ChristinaB]

Maps
Maps are complete fields of a particular variable (state or flux) at a particular time. These can be output periodically, but take up a lot of space for any sizeable application.
In the Configuration file you need to use the ID of the variable that you would like to output. Consult the list of variable IDs to find out which ID to use. The files are in BINARY or NETCDF format.

list of variable IDs

[ChristinaB]

Go to the end of config file and find an example like this. Edit for the dates you want water table depth - for example

################ MODEL MAPS ####################################################
Number of Map Variables = 3 # Number of different variables for
# which you want to output maps

For each of the variables make a block like the one that follows, varying
the number of the variable (n = 1, .. , Number of Map Variables)

Map Variable 1 = 503 # 503= water table depth ID of the variable to output
Map Layer 1 = 1 # If the variable exists for a number
# of layers, specify the layers here
# with the top layer = 1
Number of Maps 1 = 8 # Number of maps you would like to
Map Date 1 1 = 08/01/1987-00 # output for this variable
Map Date 2 1 = 08/01/1988-00 # Vary the first number from
Map Date 3 1 = 08/01/1989-00
Map Date 4 1 = 08/01/1990-00
Map Date 5 1 = 08/01/1991-00
Map Date 6 1 = 08/01/1992-00
Map Date 7 1 = 08/01/1993-00
Map Date 8 1 = 08/01/1994-00

[ChristinaB]

Try these

501	Soil.Moist	Soil Moisture Content	-	Soil moisture for layer %d	NC_FLOAT	TRUE
502	Soil.Perc	Percolation	m/timestep	Percolation	NC_FLOAT	TRUE
503	Soil.TableDepth	Water Table Depth	m below surface	Depth of water table	NC_FLOAT	FALSE

[ChristinaB]

Steps for converting DHSVM mapping output into a useful visualization

1. DHSVM outputs a binary file that is the size of the DEM grid #columns wide #rows long for n=1 map.
   For multiple timesteps (t) for each map (m), the binary file will be size #columns wide: (#rows longt)_m
   The number of columns will never change, the #rows are appended based on the # MODEL MAPS # section in the config file.
   This binary file needs to be converted to an ascii the size of #columns wide: (#rows longt) for each map.

2. From a unix machine, use this script with input giving the name of the new folder that will hold the processed data (mapDHSVMascii_outputfolder)

Beginner

Change directory to output folder containing Map.Snow.Iwq.bin

./myconvert float ascii Map.Snow.Iwq.bin Map.Snow.Iwq.asc 5404 386

Output prints .asc to the same folder

Advanced

Loop through multiple folders using command line
myconvert script sits in the execution directory holding multiple folders -
1= DHSVMoutput_3hr
2= DHSVMoutput_1day

mkdir DHSVMoutput_ascii
foreach F(DHSVMoutput_)
echo $F
./myconvert float ascii $F/Map.Snow.Iwq.bin $F/Map.Snow.Iwq.asc 5404 386
./myconvert float ascii $F/Map.Soil.TableDepth.bin $F/Map.Soil.TableDepth.asc 5404 386
./myconvert float ascii $F/Map.Snow.Swq.bin $F/Map.Snow.Swq.asc 5018 386
mkdir DHSVMoutput_ascii/$F
cp $F/.asc DHSVMoutput_ascii/$F
end

Most Advanced

Loop through multiple folders using a script

Execute:

!process_me.scr mapDHSVMascii_outputfolder

What process_me.scr looks like:

#!/bin/csh
set FULL_BAS_DIR = $1
mkdir $FULL_BAS_DIR

foreach F(headeroffolders*)
echo $F
./myconvert float ascii $F/Map.Snow.Iwq.bin $F/Map.Snow.Iwq.asc 5404 386
./myconvert float ascii $F/Map.Soil.TableDepth.bin $F/Map.Soil.TableDepth.asc 5404 386
./myconvert float ascii $F/Map.Snow.Swq.bin $F/Map.Snow.Swq.asc 5018 386

mkdir $FULL_BAS_DIR/$F
cp $F/*.asc $FULL_BAS_DIR/$F

end

[ChristinaB]

See Project Log for Paper

https://github.com/Freshwater-Initiative/pyDHSVM/wiki/PREEVENTS-hydromet-flood-forcing-sensitvity-on-geomorpholgy---Jeff-Keck-Paper-%231-Project-Log

[ChristinaB]

• Update config file for 5 state variables

• Update config file for corresponding timesteps (every day during the flood) in all model setup folder

• Run all models to get map outputs

• Calculate row length for the myconvert calculation

• Test beginner one line script

• Update Advanced code for five variables

[ChristinaB]

After breaking ascii file into individual dates
concatenate two files (in unix, with awk, with grep)

cat file1.txt file2.txt > new.txt
cat header.txt snow_11090600.asc > snow_11090600_map.asc

[ChristinaB]

Mapping pipeline

1. Update config file
2. Run model
3. Run script convert binary to ascii
4. TO DO write script to break ascii into parts by date (there be a better way to do this in awk)
5. Add header with lat/long info to the top of each ascii
6. Open ascii into map for viz or movie or figure

[ChristinaB]

Set up pyDHSVM
fork
git clone of your fork
do work
git push
pull request to FWI
go away
sync
https://gist.github.com/CristinaSolana/1885435
before working again

[ChristinaB]

9/17/2018

(1) worked on transitioning over to python as my main scripting tool.

(2) created a description of the application of the forcing data temporal resolution effect on model ouput paper.

(3) began working on converting some of my matlab scripts to python.

Make priority list of code tasks and writing tasks for
The Paper!

Update pyDHSVM on Github

[ChristinaB]

9/21/2018 Christina + Jeff Meeting

To visualize using PNNL data and OGH, see instructions on the Freshwater Observatory repo
Freshwater-Initiative/Observatory#32

[ChristinaB]

Data on rain and snow thresholds. Jennings, K. S., Winchell, T. S., Livneh, B., & Molotch, N. P. (2018). Spatial variation of the rain–snow temperature threshold across the Northern Hemisphere. Nature communications, 9(1), 1148.
We are using this citation as justification for searching for rain - snow threshold band of 6 degrees.

[ChristinaB]

Kienzle, S. W. A new temperature based method to separate rain and snow. Hydrol. Process. 22, 5067–5085 (2008).

Here is another citation on 50% rain-snow threshold