DHSVM groundwater output #1

ChristinaB · 2018-02-20T16:57:12Z

The output for the Skagit model is uploaded in the Code

ChristinaB · 2018-02-20T17:16:56Z

ChristinaB · 2019-04-04T21:58:41Z

Not using 1915-2010 data - using SeYeun's model time periods instead.

ChristinaB · 2019-04-04T22:01:09Z

Write up methods for DHSVM
Draft figure for Figure 3

ChristinaB · 2019-04-05T18:07:59Z

Number of Rows = 1020 # Number of rows
Number of Columns = 916 # Number of columns
depth to groundwater
CNRM has 62 years/maps for depth to groundwater = 63240 rows in the binary file
./myconvert float ascii Map.Soil.TableDepth.bin Map.Soil.TableDepth.asc 63240 916
CNRM45: Error reading input: Row: 59160 Column: 0 - > 58 years only printed?
CNRM85: Error reading input: Row: 60180 Column: 0 - > 59 years only printed?

snow
CNRM has 61 years/maps for snow = 62220 rows in the binary file
./myconvert float ascii Map.Snow.Swq.bin Map.Snow.Swq.asc 62220 916
CNRM45: Error reading input: Row: 58140 Column: 0 - > 57 years only printed?
CNRM85: Error reading input: Row: 58140 Column: 0 - > 57 years only printed?

ice
CNRM has 61 years/maps for ice = 62220 rows in the binary file
./myconvert float ascii Map.Snow.Iwq.bin Map.Snow.Iwq.asc 62220 916
CNRM45: Error reading input: Row: 59160 Column: 0 - > 58 years only printed?
CNRM85: Error reading input: Row: 58140 Column: 0 - > 57 years only printed?

ChristinaB · 2019-04-05T20:26:07Z

#SBATCH --time=300:00:00
in the SLURM submissin needs to be extended to 400 hours or rerun the last 10 years

ChristinaB · 2019-04-05T20:27:57Z

Draft matlab code for processing and checking that depth to water table values work as expected. See
C:\Users\cband\Skagit\SCLlandsliding\DHSVMoutputs_20190405\output_SCLlandslide_bclivlow_G_CNRM-CM5__rcp45
nrow=1020;
[nrowtotal, ncol]=size(watertabledata)
mapcount=nrowtotal/nrow;

%Print Livneh data
cd C:\Users\cband\Skagit\SCLlandsliding\DHSVMoutputs_20190405\output_SCLlandslide_bclivlow_G_CNRM-CM5__rcp45

watertabledata=load('Map.Soil.TableDepth.asc');
snowdata=load('Map.Snow.Swq.asc');
icedata=load('Map.Snow.Swq.asc');

dateswatertableCNRM45=load('dates-watertableCNRM45.txt');

% WATER TABLE DEPTH check length of data read in
i=1
datafilename=[{strcat('watertabledepth',num2str(i),'-',num2str(dateswatertableCNRM45(i)))}];
for i=2:mapcount
datafilename=[datafilename {strcat('watertabledepth',num2str(i),'-',num2str(dateswatertableCNRM45(i)))}];
end

GCM='CNRM-CM5__rcp45';

%Print depth to groundwater data
%for i=1:2
dlmwrite(strcat(char(GCM),'',char(datafilename(1)),'.asc'),watertabledata(1:nrow,:),'delimiter',' ','precision','%.1f');
dlmwrite(strcat(char(GCM),'',char(datafilename(2)),'.asc'),watertabledata(nrow+1:nrow*2,:),'delimiter',' ','precision','%.1f');

for i=3:mapcount
dlmwrite(strcat(char(GCM),'_',char(datafilename(i)),'.asc'),watertabledata(nrow*(i-1)+1:nrow*i,:),'delimiter',' ','precision','%.1f');
end

% SNOW & ICE check length of data read in
i=1
datafilename=[{strcat('snowdata',num2str(i),'-',num2str(dateswatertableCNRM45(i)))}];
datafilename=[{strcat('icedata',num2str(i),'-',num2str(dateswatertableCNRM45(i)))}];
for i=2:mapcount
datafilename=[datafilename {strcat('snowdata',num2str(i),'-',num2str(dateswatertableCNRM45(i)))}];
datafilename=[datafilename {strcat('icedata',num2str(i),'-',num2str(dateswatertableCNRM45(i)))}];
end

GCM='CNRM-CM5__rcp45';

%Print depth to groundwater data
%for i=1:2
dlmwrite(strcat(char(GCM),'',char(datafilename(1)),'.asc'),watertabledata(1:nrow,:),'delimiter',' ','precision','%.1f');
dlmwrite(strcat(char(GCM),'',char(datafilename(2)),'.asc'),watertabledata(nrow+1:nrow2,:),'delimiter',' ','precision','%.1f');
dlmwrite(strcat(char(GCM),'',char(datafilename(1)),'.asc'),snowdata(1:nrow,:),'delimiter',' ','precision','%.1f');
dlmwrite(strcat(char(GCM),'',char(datafilename(2)),'.asc'),snowdata(nrow+1:nrow2,:),'delimiter',' ','precision','%.1f');
dlmwrite(strcat(char(GCM),'',char(datafilename(1)),'.asc'),icedata(1:nrow,:),'delimiter',' ','precision','%.1f');
dlmwrite(strcat(char(GCM),'',char(datafilename(2)),'.asc'),icedata(nrow+1:nrow*2,:),'delimiter',' ','precision','%.1f');

for i=3:mapcount
dlmwrite(strcat(char(GCM),'',char(datafilename(i)),'.asc'),snowdata(nrow*(i-1)+1:nrow*i,:),'delimiter',' ','precision','%.1f');
dlmwrite(strcat(char(GCM),'',char(datafilename(i)),'.asc'),icedata(nrow*(i-1)+1:nrow*i,:),'delimiter',' ','precision','%.1f');
end

SaveAsciiRaster(varname, header)

ChristinaB · 2019-04-05T23:57:38Z

testingoutputs_20190405.xlsx

It works!!

RondaStrauch · 2019-04-08T02:59:28Z

@ChristinaB What does this mean in English? What worked?

ChristinaB · 2019-04-16T23:06:34Z

It works! the saturation events from this GCM shift from peaks in June to peaks in winter (relatively speaking over 30 year intervals).

ChristinaB · 2019-04-16T23:13:26Z

To Do by May 2, 2019

Check that outputs finished completely
Convert from binary to ascii using script
Move to Hydroshare for safe keeping. Three tarballs of ascii files (dgw, ice, snow).
Design workflow task in pyDHSVM.

Steps:
Read in tarball

Read in header

Loop through files

Clip to model extent

Calculate a probability distribution of each time period at each grid cell (150 m).

Attach to landlab raster model grid

Use the same distribution at 30 m resolution

Coupling complete! Test input to landlab landsliding component.

Updating before adding new sat extent files for the Skagit

Updating to Master

ChristinaB · 2019-05-02T22:32:35Z

Here is code for processing binary grid outputs into ascii files

Number of Rows = 1020 # Number of rows
Number of Columns = 916 # Number of columns

Historic 44 year for depth to groundwater, 43 years for ice and snow
./myconvert float ascii Map.Soil.TableDepth.bin Map.Soil.TableDepth.asc 44880 916
./myconvert float ascii Map.Snow.Swq.bin Map.Snow.Swq.asc 43860 916
./myconvert float ascii Map.Snow.Iwq.bin Map.Snow.Iwq.asc 43860 916

depth to groundwater
CNRM has 62 years/maps for depth to groundwater = 63240 rows in the binary file
./myconvert float ascii Map.Soil.TableDepth.bin Map.Soil.TableDepth.asc 63240 916
./myconvert float ascii Map.Snow.Swq.bin Map.Snow.Swq.asc 62220 916
./myconvert float ascii Map.Snow.Iwq.bin Map.Snow.Iwq.asc 62220 916

ChristinaB · 2019-05-02T23:35:15Z

@NCristea
The saturation extent dates are on Github, and depth to water table DHSVM outputs on HydroShare. Download dtw_DHSVMoutput_20190502.tar

ChristinaB · 2019-05-08T02:00:09Z

@NCristea Hyak shuts down on May 14, so it won't let me start a two week run that won't finish in time. I am running for the next 150 hours, which may get a good way into the new runs. But we need to just develop the code to work from Map.Depth.X with inputs of rows, columns, and number of maps, and convert into ascii with names by date (see above). The final results won't be ready until end of May.

ChristinaB · 2019-05-16T19:14:58Z

https://github.com/Freshwater-Initiative/OGH_group_tutorial/blob/master/Observatory_usecase7_xmapLandlab.ipynb

See oxl.calculateUTMbounds() and oxl.mappingfileToRaster() functions in https://github.com/Freshwater-Initiative/Observatory/blob/master/ogh/ogh_xarray_landlab.py

ChristinaB · 2019-05-16T20:51:36Z

@NCristea you can download a zipped file of raster and ascii files and headers at this link: SCL Domain

Extent	Resolution/Grid Cell Size	xllcorner	yllcorner	ncols	nrows
Observatory Climate Forcings SCL extent	6 km	591315	5354895	13	13
Skagit Hydrology & Climate Change Study Domain	150 meter	535665	5311395	916	1020
Seattle City Light Skagit Study Domain - Hydrology	150 meter	591315	5354895	545	544
Seattle City Light Skagit Study Domain - Landslide/Fire	30 meter	591315	5354895	2725	2720

ChristinaB · 2019-11-29T22:26:55Z

https://github.com/Freshwater-Initiative/SkagitLandslideHazards/blob/master/dhsvm_config_files/INPUT.skagitSCLlandslide.historic1960.bclivlow.G

Let's use this list to verify the dates in NetCDF DHSVM output/Landlab input using the Skagit input config files and Skagit data on HydroShare in GIS folder. https://www.hydroshare.org/resource/70d746c7da584ae6bd2f88deb5a4c188/

ChristinaB · 2019-11-29T22:45:17Z

Time 1: dates from Sauk extent model run dtw_DHSVMoutputs_20190405.tar
Model Runs Labeled 20190405 match the config files on my computer are dated March 8. This is what is on Github in comment above.

Time 2: dates from Skagit extent model run 20190502. also correspond. This file is on Hyak.

Time 3: re-compressing and sending outputs from Hyak processed in May --dtw_DHSVMoutputs_20190925.tar

ChristinaB · 2019-12-02T22:56:01Z

To Do - Run model for dates from Sauk extent (done - spatially representative for rain and rain-snow dominated area), Skagit extent (will bias toward snow melt days for max saturated event), GoodellCreek extent (will be specific to the fire; rain-snow focus). Is the landslide model sensitive? Does all of this matter? Or is Queen Slope the Master of the Slippery Future Universe?

ChristinaB · 2019-12-02T22:59:40Z

@NCristea
We flipped the Y axis of the xarray netcdf.
data = xr.open_dataset('dtw_historic_with_dates.nc')
flippy = data.reindex(y=list(reversed(data.y)))
this won't plot flipped in xarray plotting because it is too smart and fixes the flipped UTM coordinates in the map. But investigating the sequence shows the correct Y sequence is flipped. The first Y coordinate should be 5354910. We have this working and validated it using the Landlab grid building/netcdf read writing.

ChristinaB · 2020-01-16T21:36:41Z

Skagit extent skagit historic dates

ChristinaB · 2020-01-30T03:52:48Z

Here is code for processing binary grid outputs into ascii files for Skagit Model Skagit List of Max Annual Dates - continuous 1960-2011 and from 2006-2099 for future data

Number of Rows = 1020 # Number of rows
Number of Columns = 916 # Number of columns

Future 95 year for depth to groundwater, 61 years for ice and snow
./myconvert float ascii Map.Soil.TableDepth.bin Map.Soil.TableDepth.asc 96900 916
./myconvert float ascii Map.Snow.Swq.bin Map.Snow.Swq.asc 62220 916
./myconvert float ascii Map.Snow.Iwq.bin Map.Snow.Iwq.asc 62220 916

Historic 52 year for depth to groundwater, 43 years for ice and snow
./myconvert float ascii Map.Soil.TableDepth.bin Map.Soil.TableDepth.asc 53040 916
./myconvert float ascii Map.Snow.Swq.bin Map.Snow.Swq.asc 43860 916
./myconvert float ascii Map.Snow.Iwq.bin Map.Snow.Iwq.asc 43860 916

ChristinaB · 2020-01-30T04:09:33Z

Script for converting binary to ascii

foreach F(output_*)
cd $F
./myconvert float ascii Map.Soil.TableDepth.bin Map.Soil.TableDepth.asc 96900 916
./myconvert float ascii Map.Snow.Swq.bin Map.Snow.Swq.asc 62220 916
./myconvert float ascii Map.Snow.Iwq.bin Map.Snow.Iwq.asc 62220 916
cd ..
end

ChristinaB · 2020-03-19T22:59:14Z

run finished Skagit historic run script to ascii
Add tar file to: https://www.hydroshare.org/resource/01b486f301864828ba2cd9ab7ac77c4e/ delete old files
Delete https://www.hydroshare.org/resource/70d746c7da584ae6bd2f88deb5a4c188/
Test code here https://www.hydroshare.org/resource/4cac25933f6448409cab97b293129b4f/

ChristinaB · 2020-04-20T21:13:02Z

foreach F(output_SCLlandslide_bc*)
cp $F/Map.Soil.TableDepth.asc ../dtw_20200320/$F
cp $F/Map.Snow.Swq.asc ../ice_20200320/$F
cp $F/Map.Snow.Iwq.asc ../snow_20200320/$F
end

ChristinaB · 2020-04-20T21:33:23Z

Finish zipping on Hyak.
Organize tar files
Move all Skagit DHSVM model files to a new Skagit on HydroShare model instance

ChristinaB · 2020-04-20T21:41:37Z

review contract deliverables and move outputs of SCL to https://www.hydroshare.org/resource/70d746c7da584ae6bd2f88deb5a4c188/

RondaStrauch mentioned this issue Aug 20, 2018

What is a 'storm'? #5

Open

1 task

ChristinaB changed the title ~~DHSVM groundwater output planning~~ DHSVM groundwater output Apr 4, 2019

ChristinaB self-assigned this Apr 4, 2019

ChristinaB added methods results labels Apr 4, 2019

ChristinaB assigned NCristea Apr 30, 2019

NCristea added a commit that referenced this issue May 2, 2019

Merge pull request #1 from ChristinaB/master

c242a02

Updating before adding new sat extent files for the Skagit

ChristinaB added a commit that referenced this issue May 2, 2019

Merge pull request #1 from Freshwater-Initiative/master

2f27896

Updating to Master

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DHSVM groundwater output #1

DHSVM groundwater output #1

ChristinaB commented Feb 20, 2018 •

edited

Loading

ChristinaB commented Feb 20, 2018 •

edited

Loading

ChristinaB commented Apr 4, 2019

ChristinaB commented Apr 4, 2019

ChristinaB commented Apr 5, 2019 •

edited

Loading

ChristinaB commented Apr 5, 2019

ChristinaB commented Apr 5, 2019 •

edited

Loading

ChristinaB commented Apr 5, 2019

RondaStrauch commented Apr 8, 2019

ChristinaB commented Apr 16, 2019

ChristinaB commented Apr 16, 2019 •

edited

Loading

ChristinaB commented May 2, 2019 •

edited

Loading

ChristinaB commented May 2, 2019

ChristinaB commented May 8, 2019

ChristinaB commented May 16, 2019

ChristinaB commented May 16, 2019

ChristinaB commented Nov 29, 2019

ChristinaB commented Nov 29, 2019 •

edited

Loading

ChristinaB commented Dec 2, 2019

ChristinaB commented Dec 2, 2019

ChristinaB commented Jan 16, 2020

ChristinaB commented Jan 30, 2020 •

edited

Loading

ChristinaB commented Jan 30, 2020 •

edited

Loading

ChristinaB commented Mar 19, 2020 •

edited

Loading

ChristinaB commented Apr 20, 2020

ChristinaB commented Apr 20, 2020

ChristinaB commented Apr 20, 2020 •

edited

Loading

DHSVM groundwater output #1

DHSVM groundwater output #1

Comments

ChristinaB commented Feb 20, 2018 • edited Loading

ChristinaB commented Feb 20, 2018 • edited Loading

ChristinaB commented Apr 4, 2019

ChristinaB commented Apr 4, 2019

ChristinaB commented Apr 5, 2019 • edited Loading

ChristinaB commented Apr 5, 2019

ChristinaB commented Apr 5, 2019 • edited Loading

ChristinaB commented Apr 5, 2019

RondaStrauch commented Apr 8, 2019

ChristinaB commented Apr 16, 2019

ChristinaB commented Apr 16, 2019 • edited Loading

ChristinaB commented May 2, 2019 • edited Loading

ChristinaB commented May 2, 2019

ChristinaB commented May 8, 2019

ChristinaB commented May 16, 2019

ChristinaB commented May 16, 2019

ChristinaB commented Nov 29, 2019

ChristinaB commented Nov 29, 2019 • edited Loading

ChristinaB commented Dec 2, 2019

ChristinaB commented Dec 2, 2019

ChristinaB commented Jan 16, 2020

ChristinaB commented Jan 30, 2020 • edited Loading

ChristinaB commented Jan 30, 2020 • edited Loading

Script for converting binary to ascii

ChristinaB commented Mar 19, 2020 • edited Loading

ChristinaB commented Apr 20, 2020

ChristinaB commented Apr 20, 2020

ChristinaB commented Apr 20, 2020 • edited Loading

ChristinaB commented Feb 20, 2018 •

edited

Loading

ChristinaB commented Feb 20, 2018 •

edited

Loading

ChristinaB commented Apr 5, 2019 •

edited

Loading

ChristinaB commented Apr 5, 2019 •

edited

Loading

ChristinaB commented Apr 16, 2019 •

edited

Loading

ChristinaB commented May 2, 2019 •

edited

Loading

ChristinaB commented Nov 29, 2019 •

edited

Loading

ChristinaB commented Jan 30, 2020 •

edited

Loading

ChristinaB commented Jan 30, 2020 •

edited

Loading

ChristinaB commented Mar 19, 2020 •

edited

Loading

ChristinaB commented Apr 20, 2020 •

edited

Loading