Calculating AGWS Using HSPF Methods

[ilonah22]

Revisiting AGWS calculations

From #31

• For modeled output add a plot of AGWET vs. AGWS:
  • Also, can we revisit the notion of calculating AGWS for USGS gage flows based on computed AGWRC and AGWO
  • Can we then plot these for the USGS gages?
  • Also note, the estimation of AGWS based on AGWO estimated from USGS gage flow is a fundamental output/goal of this project, so, making progress here is going to be a massive accomplishment.

Current Branch: https://github.com/HARPgroup/baseflow_storage/tree/ih_model_calcs

Current equation using information from the HSPF user manual:

$AGWS = \frac{AGWO}{KGW (1 + KVARY \cdot GWVS)}$ and $KGW = 1 - AGWRC^{\left(\frac{hr/int}{ 24}\right)}$.

In order to use AGWRC, we have to assume that $KVARY*GWVS =0$, so the final equation I used is

$AGWS = \frac{AGWO}{1-AGWRC}$

Current script for calcs and plots: SF_event_summary and calc_storage

Updates From 1/20 Meeting

• Supported hypothesis that ET does now have major impact on flow
• Look at specific events with low flow
• Try other gages
• Flow vs. change in storage?
• Create separate step/function for storage calc at given time
  • Use new function in Will's app
• Check storage vs. time to make sure it decreases over event

[ilonah22]

Last week Rob mentioned overlaying the AGWS results using model and USGS flow data as inputs.

Full Scale:

Zoomed to show majority of data:

As we saw last week, the model data has a distinct linear pattern throughout the data. Overall however, the two datasets seem to agree quite well.

Next, I thought it might be interesting to look at a few case studies to compare the values a little more directly

Event 99 is the event which has the high calculated AGWS values that we noticed last week.

I just browsed some of the vents without using any kind of calculated to compare the model and USGS flow calculated values. I could see some potential benefit in creating a metric to compare them. This might allow us to quantify model performance.

Hydrographs for events 84 and 99:

Left side: hydrograph using flow in watershed inches from USGS data
Right side: flow in inches using model AGWO

[rburghol]

@ilonah22 -- I see that you already got the flows in here, that is super great. (Hey @COBrogan check this out) And also, you put them in inches, which is doubly helpful.

• An obvious next step would be to superimpose them on the graphs of AGWS :).
• And, since no good deed should go unpunished, having a key to translate inches to cfs could be helpful.
• I think that comparing these plots is a little more challenging, as the storage plots are in magnitude order sorted by modeled AGWS, so, provided that there is no recharge happening during the events, the AGWS charts would be in reverse order of the time series flow (i.e., AGWS would be at it's highest at the beginning of the period).
  • Can you confirm that what I am saying about the different ordering is correct?

Anyhow, I wanted to think through the mass balance here a bit more. So what I see is (for the purpose of gaming out the mass balance):

• Event 84:
  • Model runs out of water, but the gage shows small flow values still occurring (this is where the conversion to CFS could be helpful, I assume that 0.004 inches is a very small flow, but how small?).
  • beginning (max) AGWS for USGS is about 0.32 inches, so, over 23 days, flows range from 0.008 to 0.004, so lets estimate an average of $0.006 * 23$ = 0.138, so, our ending storage should be $0.32 - 0.138 = 0.182$, and I'll eyeball the final storage as about 0.16, so that puts our error/missing losses at 0.02 over 23 days, or 0.0008 per day. All in all this tells me that we are in a pretty gnarly drought!
  • Also note, the 0.0008 remainder or error/ET is well within the range of modeled AGWET
  • Also, the change from 0.008 to 0.004 in AGWO over a period of 23-24 days, corresponds to an AGWRC of between 0.965 ad 0.977 on the baseflow recession curves below:
    Image 1: Theoretical baseflow recession curves for example AGWRC values from 0.99 to 0.90. Number of days for 50% reduction in flow to occur are indicated along the plot line.

• Event 99:
  • the model obviously has at least one rainfall day occurring during the event, so we'll say "not baseflow", so our detection goofed it! (to the question of does this change when we use the dataset from @benhdye and @willprokopik revised methodology)
  • The observed does not have the same rainfall signature, but dang, that baseflow storage estimate is gigantic...25 inches at a time when the model tells us that we are in the lower ranges of simlated AGWS, 0.25 inches is probably one of the lowest values that model simulates in February. This emphasizes the need to have the full dates printed on these images somewhere.
• Questions/suggestions:
  • Which watershed is this? Coote's?
  • How about using date as the x-axis here? You could also keep the modeled AGWS as the x, just a thought (maybe both tell us something)
  • Is this the new method that trims out storm flows at the beginning?
  • Keep going on this. Let's get those dates in there, and perhaps expand the universe and/or use the updated event dataset.
  • Also, I should note that I definitely concur with your assessment voiced in the work session that the "AGWS vs Two Inputs" chart looks qualitatively very, very good.
  • Which event is the super high outliers (circled in black below)? Is that 99? Is it the same event for both model and observed?
  • Descriptive statistics for the events will be extremely helpful here, $R^2, p, mean(AGWRC)$

[ilonah22]

I wanted to answer a few of your quicker questions from last week:

• This data is all for Mount Jackson.
• This is not using the trimmed data, I didn't want to start using that data until Ben and Will had finalized the method. If you all think it would be better to look at the trimmed data, I can start working with that.
• The super high event is 99 and it is the same event for modeled and observed.
• For this watershed the conversion factor for in to cfs is ~13,660, so the range for event 84 is about 55-109 cfs and for 99 it's 204-260 cfs

Here the superimposed AGWS and AGWO plots you mentioned using model data:

Here are the same plots using USGS flow and the AGWS calculated using those flows:

84 - It looks like the final storage that you predicted of 0.182 matches the data
99 - Definitely days of rainfall that the storage values are catching
101 - This is the event that we thought looked pretty recession-like from some of last week's plots, and it still looks pretty good for recession because of the gradual decrease in storage
124 - This is a new event I found that I thought might be interesting because of the obvious days with rain in the middle of the event

Mass Balance on Active Groundwater Storage
Using this diagram as a guide I just wanted to get an overall view of the mass balance on the active groundwater storage:

Here is the mass balance using only the model variables:
$AGWS = AGWI+AGWLI-AGWET-AGWO$

I know that lest week we had talked about making sure that AGWO roughly matched AGWS-AGWET, should I continue using only those factors, since having limited variables makes it easier to translate model and USGS data? Or should I use some of these other variables (AGWI and AGWLI)?

Then, I took a look in the model to see where AGWET is coming from so that we may try to partially replicate it for the USGS data.

Unfortunately, I don't know how well we will be able to use this formula on the USGS flow data, since there is a model parameter and the groundwater elevation, which I am not sure how we would get that from USGS data.

[rburghol]

$AGWS = AGWI+AGWLI-AGWET-AGWO$

Where is the beginning value of $AGWS$ in this equation? I think that it is critical to understand that AGWS is a persistent store of water, and thus often it is the most important part of the determination of tomorrows AGWS is today/s AGWS. Also note that under baseflow conditions we know that AGWI and (and AGWLI, what is this? Lateral inflow?).

I know that lest week we had talked about making sure that AGWO roughly matched AGWS-AGWET

This is definitely not my hypothesis, and the reason I wanted you to go back to the model equations is to answer this question from the issue #31:

• Honestly, I think that this above image demonstrates that:
  • AGWET is less than 0.005 in/day under the majority of simulated days
  • AGWET has a small effect on AGWO, except the very few times when AGWET has any noticeable impact on AGWS on a day to day basis. This occurs only when AGWS is very close to AGWET
  • AGWET can have a larger effect on AGWS over a long period as it accumulates day after day, but only if it is persistently high during that period.
• AGWO is dependent on AGWS, by the equations that you show above. Several factors affect AGWS, but during droughts the inflow parameters go to zero, and thus only AGWO and AGWET impact AGWS. I need you to demonstrate mathematically, and graphically, when AGWET does have a large effect, and when it does not, so that we can quantify and then move past the uncertainty.
• Your image from a few weeks back, shown below, demonstrates that AGWET is very tiny compared to

Unfortunately, I don't know how well we will be able to use this formula on the USGS flow data, since there is a model parameter and the groundwater elevation, which I am not sure how we would get that from USGS data.

We don't need to apply that equation or simulate it. We need to know the following:

• AGWET is, in the model, left over ET that is not used up in the surface simulation (plant ET, UZET).
• AGWET + Error is just the remainder of the calculation that I made for you above

[ilonah22]

As we discussed in our meeting, I remade some of the plots that we have been looking at to see how they compare:

First, I made a new version of this original AGWS (from model) and the two calculated AGWSes:

Some notable differences are the range of values. It appears that many of the observations at higher original AGWS values have been removed by Ben's trimming function.

Next, I looked at two of the specific events which we have been looking at:

The AGWS for both for these events looks slightly less like a stormflow pattern.

Moving Forward with Mass Balance

Considering what we talked about last week (AGWI and AGWLI ~ 0), the mass balance equation we can use to validate our data is $AGWO = AGWS - AGWET$

I am thinking that and easy way to start will be to use this equation and model data as inputs to get an expected AGWS value, then compare that the AGWS values we have been calculating using our AGWRCs.

[rburghol]

@ilonah22

the mass balance equation we can use to validate our data is $AGWO = AGWS - AGWET$

I believe that this is not the equation. But if it is, show how you can go from the equation in the top of our issue $AGWS = \frac{AGWO}{1-AGWRC}$ to the one you propose here.

[ilonah22]

Different AGWS Formulas

Based on our discussion last Tuesday, I was thinking that the mass balance we are trying to create is going to be used mainly to check that our method of calculating AGWRC is valid. So the first equation, using AGWRC (the user manual version) is the "experimental" equation and the mass balance version is the "control".

If you had something else in mind using the original equation, let me know, but for this week I went along with this.

Simplified from model user manual:

$AGWS = \frac{AGWO}{1-AGWRC}$

Mass balance from model diagram:

$AGWS = AGWO + AGWET$

For some reason, I have been getting a ton of error messages while trying to download any model data, so all I had to work with was ET and AGWRC. This is the error message I was getting:

In file(file, "rt") :
  URL 'https://deq1.bse.vt.edu:81/p6/out/land/subsheds2/pwater/forN51171_pwater.csv': status was 'SSL connect error'

Since I wasn't able to get any AGWO data, I just put in a superficial avg. value from the plots above of 0.001 in, just to try and get a sense of the comparison.

Unfortunately, we cannot really do anything with this information since it's not coming from real outflow data. However, the roughly similar ranges could be a good sign that once we use actual data the values may match pretty well. I would love to get the model data working again if possible.

[ilonah22]

Finding Error

beginning (max) AGWS for USGS is about 0.32 inches, so, over 23 days, flows range from 0.008 to 0.004, so lets estimate an average of 0.006 ∗ 23 = 0.138, so, our ending storage should be 0.32 − 0.138 = 0.182, and I'll eyeball the final storage as about 0.16, so that puts our error/missing losses at 0.02 over 23 days, or 0.0008 per day.

Using the method described in this comment, I think this equation should work to help us find error and compare it to AGWET:

$AGWS_0 - (mean(AGWO)*ndays) = AGWS_f + error$

[COBrogan]

Thanks @ilonah22 . I think we can simplify this equation to be a little more straghtforward because we can just use the sum of AGWO from the USGS data as $\Sigma(AGWO) = mean(AGWO) * ndays$. So:
$AGWS_0 - \Sigma(AGWO) = AGWS_f + error$

[ilonah22]

Results from Error Calculations

Using this formula I calculated the error:
$AGWS_0 - \Sigma(AGWO) = AGWS_f + error$

This resulted in a majority of errors less than zero; the range of values was (-0.57, 0.11). Actually, a vast majority of the errors were negative:

Some possible sources of this error, other than ET, are the AGWI and AGWLI values that we are not including. However, I wouldn't expect these variables to make up 0.6 in. of difference, especially during recession events.

Plot or Error vs. ET

To confirm that the error is roughly equivalent to ET and see what conditions might affect that relationship, I made a plot of the calculated error vs. ET. This should, in theory be a 1:1 line, or somewhere close.

I initially included the negative numbers for error, just to see how the plot looked before zooming in on positive values only.

Next, I zoomed in on positive values only, to see if there was a direct relationship.

There does seem to be a linear relationship not too far from 1:1. However there are quite a few occurrences of AGWET being higher than the error. I guess this could also be a result of those extra variables not being included. I think I might check, just for my own understanding, what happens to the relationship if I include AGWI and AGWLI.

[rburghol]

@ilonah22 -- this is very interesting. I like both y'all value chart and the non-negative chart. I will ask a clarifying question or two:

• Can you first define what it is to be a negative error versus a positive error?
• You say there are several incidents where AGWET is greater than the error, can you dive into some of those and show us what's happening? Quantify your observation
• Can a wrong estimate of AGWRC cause the errors that we see?
• can you provide a description of how AGWET could influence this error?
• When you mention AGWI, I think that's possible, but I would like to first quantify what conditions produce that, ie, define the behavior that you expect to see.

@COBrogan @julieshortridge

[COBrogan]

Can you first define what it is to be a negative error versus a positive error?

Does a negative error indicated our AGWRC estimate is too low? I want to make sure I have a clear picture of what this means for understanding our coefficient estimates because...

Can a wrong estimate of AGWRC cause the errors that we see?

I'm very interested in this question as it leads to the follow-up: can this error analysis help us refine the AGWRC that we ultimately select for our model? If error is often negative, what does that mean for our approach so far?

[ilonah22]

11/7 Notes

Positive vs. negative error

As we discussed today, a negative value for error means that the final storage is higher than expected. Meaning AGWO for the event is greater than the change in storage.

Model data vs. USGS data

I made an error when making these plots for this week and used exclusively model data for all of my calculations. As a result this means that these values cannot be of much use in examining our methods of calculating AGWRC.

My next step will be to perform the same calculations using the correct data.

Other notes on data and plots

The results and plots have been made using Mount Jackson trimmed data

The script I am currently using is mass_balance.R