About tutorial of odil

[HarondFinch]

Thank you for sharing the code!
l am wondering that if you could add a simple illustrative example (a jupyter notebook maybe) to show the basic usage/tutorial of odil, l find it a little confusing reading codes related to API of odil.

[pkarnakov]

So far the code in examples/basic is the simplest illustration, it actually does have a notebook which may possibly even run in Colab. I can try to write some tutorial, did you have any particular features or questions in mind?

[kangbeicai]

So far the code in examples/basic is the simplest illustration, it actually does have a notebook which may possibly even run in Colab. I can try to write some tutorial, did you have any particular features or questions in mind?

I have a question which can't be solved well by PINNs.
Solve a transient diffusion equation where the time is normalized to t/t_max. t_max is a unknown quantity which should be solved like an inversed problem. The initial u(x,0) can be sin(x),where 0 < x < pi. u(pi/2, t_max) is a known quantity and is used to constrain the solution. When I used PINNs, t_max can't be solved very well when u(pi/2, t_max) / u(pi/2,0) is small ( like 0.01). Can you try to solve it with odil?

[pkarnakov]

I have added an example heat_tmax in this pull request: #5
It seems to work. The reference tmax=4.5 so the ratio is 0.011.

[kangbeicai]

I have added an example heat_tmax in this pull request: #5 It seems to work. The reference tmax=4.5 so the ratio is 0.011.

Thank you for your example. I have tried it, ODIL exactly works better than PINNS in terms of speed, but I still have some questions.

1. I have noticed that the initial profile is always a little difference with the reference, why not use a hard constraint like PINNs？ I guess using a hard constraint in ODIL is much easier than in PINNs?
2. Since I used hard constraints in PINNs, the initial boundary can be fitted perfectly. The exact problem I met before is that the profile at tmax can't matched very well. The relative error may be higher than 10%, but such error can't be recognized in current plots since only significant absolute error can be seen in such plots. When I worked on PINNs, I realized that relative error cannot be reduced to a satisfactory level since the loss is designed to measure the total absolute error. Do you think this problem of relative error can be solved by ODIL?

[pkarnakov]

The exact solution was evaluated on a wrong set of points, which should be node-based in time. Here's an updated plot.
The initial profile is imposed as a hard constraint:

    u = mod.concatenate([extra.u_init[None, :], u[1:]], axis=0)

I think the relative error can be made small too on a sufficiently fine mesh.

[kangbeicai]

I hvae made a few more attempts, and compared the profile at tmax. It's working better than I ever expected, even tmax = 10 can be predicted well after 90000 epoch, where the ratio is lower than 1e-4.


However, when I change optimizer to adam, it can't predicted the problem ( where tmax = 4.5) very well even using 40000 epoch.

And I try to solve this problem in deepxde with L-BFGS-B, but the it stop early.
So, I think that the effectiveness of ODIL is attributed to its use of sparse matrices, which can be stably solved with the L-BFGS-B algorithm. Am I right?

[kangbeicai]

When I change optimizer to newton, it fails to run. Are there any other code that need to be changed?

Running Newton optimizer

epoch=00000
residual: eqn:5.6127, imp:0.011106
tmax=1
memory: 390 MiB
walltime: 1.863 s
walltime+callback: 1.864 s
u_00000.png
Traceback (most recent call last):
  File "e:\python\ODIL\odil-main\examples\heat_tmax\heat_tmax.py", line 222, in <module>
    main()
  File "e:\python\ODIL\odil-main\examples\heat_tmax\heat_tmax.py", line 200, in main
    odil.optimize(args, args.optimizer, problem, state, callback)
  File "D:\miniconda\Lib\site-packages\odil\util.py", line 316, in optimize
    return optimize_newton(args, problem, state, callback, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\miniconda\Lib\site-packages\odil\util.py", line 244, in optimize_newton
    vector, matrix = problem.linearize(state)
                     ^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\miniconda\Lib\site-packages\odil\core.py", line 1321, in linearize
    m = modsp.csr_array(garray)
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\miniconda\Lib\site-packages\scipy\sparse\_compressed.py", line 92, in __init__
    coo = self._coo_container(arg1, dtype=dtype)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\miniconda\Lib\site-packages\scipy\sparse\_coo.py", line 84, in __init__
    self._shape = check_shape(M.shape, allow_1d=is_array)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\miniconda\Lib\site-packages\scipy\sparse\_sputils.py", line 332, in check_shape
    raise ValueError('shape must be a 1- or 2-tuple of positive '
ValueError: shape must be a 1- or 2-tuple of positive integers

[slitvinov]

@kangbeicai I can reproduce this issue. We’ll look into it and track it separately as #7.

[pkarnakov]

Here's the results with Newton's method, after the fix in #7 (comment)
Newton's method requires a different treatment of the shifted fields, tf.roll is not sufficient as we need to track them separately. Also I had to initialize the temperature with the initial profile instead of zeros, otherwise it would diverge.

[pkarnakov]

However, when I change optimizer to adam, it can't predicted the problem ( where tmax = 4.5) very well even using 40000 epoch.

Using a learning rate schedule should help to achieve higher accuracy with Adam. There is no direct support for it in ODIL, but to achieve that I normally just run odil.optimize() several times in a loop, each time multiplying args.lr by e.g. 0.1.