Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DA notebook #15

Open
2 of 6 tasks
jgomezdans opened this issue Feb 25, 2022 · 16 comments
Open
2 of 6 tasks

DA notebook #15

jgomezdans opened this issue Feb 25, 2022 · 16 comments
Assignees
@jgomezdans

Comments

@jgomezdans
Copy link
Collaborator

jgomezdans commented Feb 25, 2022 •
edited
Loading

  • Model exploration with parameters ("sliders")
  • Automatically fit the LAI ("DA button")
  • Plot yield distribution for all fields w/boxplots
  • Per pixel LAI exploration
    • Show yield
    • Show parameter spatial distribution
@jgomezdans jgomezdans self-assigned this Feb 25, 2022
jgomezdans pushed a commit that referenced this issue Feb 28, 2022
Create ensembles
117d711 
Address #15 #18
@jgomezdans
Copy link
Collaborator Author

Need to update GUI and test "DA button" with big safe ensemble Tuesday.

@jgomezdans
Copy link
Collaborator Author

I have tested the DA button, but my results are very different to yours @Ma-hy
image

Looking into your ensemble fitting code, @MarcYin noticed that you calculate the wrong LAI value:

np.nanmean(np.transpose(obs['mean_bios_all'][:,4,:])*obs['mean_bio_scales_all'][:,4], axis=1)

the LAI should be calculated as

f.f.mean_bios_all[:, 4, :].mean(axis=0)

We need to discuss this tomorrow morning 9am. I've sent an invite through.

jgomezdans pushed a commit that referenced this issue Mar 1, 2022
Added assimilate function
b08bda9 
The function works OK with the ensembles that have been generated
and are available.
Addresses  #15
@profLewis
Copy link
Contributor

profLewis commented Mar 2, 2022 via email

@profLewis
Copy link
Contributor

profLewis commented Mar 2, 2022 via email

@Ma-hy
Copy link
Contributor

Ma-hy commented Mar 2, 2022

Tried assimilating correct LAI, like below
image

@Ma-hy
Copy link
Contributor

Ma-hy commented Mar 2, 2022

Compared with the wrong LAI calculation
image

@jgomezdans
Copy link
Collaborator Author

Your new thing is the same as my plot above: only a small variation between 2200-2500 kg/ha. Plot them with the same axes.

@Ma-hy
Copy link
Contributor

Ma-hy commented Mar 2, 2022

image
Yeah, the slope is quite flat, but keeps high R value, by the way, the pixel-by-pixel assimilation shows a bit larger variation.

@profLewis
Copy link
Contributor

profLewis commented Mar 2, 2022 via email

@jgomezdans
Copy link
Collaborator Author

I'm currently filtering by the empirical relationship and will have some results in a bit. the ensembles from Hongyuan didn't cover empirical relationship line, so have changed things a bit to get to that point.

@jgomezdans
Copy link
Collaborator Author

OK, so after fitting the empirical line, and changing a more sensible set of parameters, here's something more hopeful. Results first:

  • Fit to the LAI is fair. Remember, small ensemble (made smaller by filtering, around 650 ensemble members)
    image
  • Observed vs predicted yield: Pretty good correlation, slope around 0.45
    image
  • Parameters vs obs yield. Reasonable trends (see for example CVO), but there's a lot of internal correlation here
    image
  • Correlation matrix shows huge correlations, so we may be able to further simplify the number of crop parameters
    image

I'm doing this on a small ensemble, so I selected the 5 best solutions (for the correlation matrix, I selected the best 20). In general, this looks reasonable, and I think we can maybe expect better results with bigger ensembles that allow better fitting to LAI, and possibly reducing the input parameters space (not too bothered about that).

This was referenced Mar 2, 2022
Open
Merged
@MarcYin
Copy link
Contributor

MarcYin commented Mar 3, 2022

Assimilation code is working OK for fitting LAI etc, but using the DA parameters to forward model observations doesn't produce the same as the ensemble. Likely to be a discrepancy between ensemble generation code and notebook code.

Screenshot 2022-03-03 at 19 35 08
s

@jgomezdans
Copy link
Collaborator Author

Mostly there, bits missing are showing the per pixel DA results.
Additionally, @Ma-hy has found a new smaller set of parameters for the model, that are maybe worthwhile exploring, but we'll see in discussion.

@profLewis
Copy link
Contributor

profLewis commented Mar 3, 2022 via email

@jgomezdans
Copy link
Collaborator Author

jgomezdans commented Mar 3, 2022 via email

@profLewis
Copy link
Contributor

profLewis commented Mar 3, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jgomezdans jgomezdans

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jgomezdans @profLewis @Ma-hy @MarcYin