[EigenSolvers] Add FEAST with MKL support to EigenSolversApplication #6482

qaumann · 2020-02-28T13:46:28Z

If the EigenSolversApplication is compiled with MKL support, FEAST can now also be included to solve generalized eigenvalue problems. With this FEAST interface, it is possible to find either the lowest or highest eigenvalues for real symmetric problems or some eigenvalues inside a specified range for real and complex symmetric problems. The support can also be extended to general matrices or polynomial problems, if required.

The interface to FEAST in EigenSolversApplication differs from the one in ExternalSolversApplication in the following points:

Here, FEAST 4.0 is used. With the new version, it is possible to look for the highest / lowest eigenvalues in a specified range
The builtin MKL support is used for the matrix computations, so the complicated reverse communication interface is not required

There is one small change in the core, which makes the Solve function for complex eigensystems available to Python.

Tests are provided and the readme is updated. I tested it on Linux and will also try to run it on Windows. This is still a draft, so tell me what you think. I am happy for suggestions.

oberbichler · 2020-03-03T10:43:15Z

For the eigenvalue solver I have also tried FEAST. It turned out that (at least the version in MKL) is very sensitive to the choice of parameters. Especially the the search range was problematic. Do you also have this problem?

qaumann · 2020-03-03T11:34:14Z

I also experienced this, but would not call it a problem. I like the feature, that you can limit your eigenvales to a specific range very much and this is also important for some methods I am using. The only parameter that can really influence your results is the subspace size, but there is a functionality in FEAST which estimates a suitable size. It would be no big problem to expose this also in Kratos, e.g. as a process or a InitializeSolutionStep.

In the new FEAST 4, you can, however, also look for extremal eigenvalues, e.g. the lowest in your defined search interval, which would start at zero for structural problems. This is the same functionality as the EigensystemSolver, if I am not mistaken.

Regarding Windows compilation, I managed to compile FEAST, but linking it with all required Fortran libraries is something that needs further investigation and currently I do not have enough time for this. I hope this is no showstopper here.

I also discusses with @philbucher if we should include FEAST's source code in the repository, like it is done in ExternalSolversApplication. Then the compilation of the library can be switched on and off directly from the configure file.

philbucher · 2020-03-03T19:30:28Z

To me it would be ok adding the feast code into the repo
We could do so in a separate PR, since it also would involve the Comittee

How big is it?
The license is MIT right?

oberbichler · 2020-03-04T07:26:56Z

It was a problem because it was not possible to find a general setup to find the smallest eigenvalues for our benchmarks. The new functions are definitely an enrichment.

About the fortran-stuff: AFAIK it is not so easy to integrate fortran in Windows with the default toolchain. But I think we can start the FEAST integration as an exclusive Linux feature.

BTW: Is there any connection to Eigen?

qaumann · 2020-03-04T16:29:16Z

The only connection is MKL. And that I thought, EigenSolversApplication is an application for solving the eigenvalue problem :)

oberbichler · 2020-03-05T06:08:48Z

Was not the intention of the application but with the MKL dependency it makes sense to but it here.

qaumann · 2020-03-13T12:45:06Z

@philbucher I added the FEAST library to external_libraries. It weighs 2.5mb and is licensed under BSD. Compilation can now be included by -DUSE_FEAST4 on Linux. After some days of trying, I gave up on the Windows compilation, as the linking from the Fortran library to MKL is not too trivial. If anyone has experience with this, I would love to learn more about it.

applications/EigenSolversApplication/custom_python/eigen_solvers_python_application.cpp

kratos/python_scripts/eigen_solver_factory.py

philbucher · 2020-03-31T10:34:30Z

I created #6645 since I didn't want to pollute this branch

I noticed that the tests test_complex_symmetric_gev and test_real_symmetric_gev take multiple seconds on my machine. Same for you?

Other than that this is ready, I will discuss it in the committee

qaumann · 2020-03-31T12:19:38Z

I noticed that the tests test_complex_symmetric_gev and test_real_symmetric_gev take multiple seconds on my machine. Same for you?

Other than that this is ready, I will discuss it in the committee

Thank you very much for the review.

On my machine the tests run for around 0.7 seconds each (in release). This is because I have to copy each column of the eigenvector matrix to a new vector, as the ublas space (at least its Python interface) does not support matrix-matrix-multiplication. If it takes too long, I can adapt the tests.

philbucher · 2020-03-31T12:29:42Z

On my machine the tests run for around 0.7 seconds each (in release). This is because I have to copy each column of the eigenvector matrix to a new vector, as the ublas space (at least its Python interface) does not support matrix-matrix-multiplication. If it takes too long, I can adapt the tests.

hm if it ok in the CI then I guess it is fine
If you really want you could try to expose this and check but not necessary, up to you

…ast-improvements Improvements for [EigenSolvers] Add FEAST with MKL support to EigenSolversApplication #6482

qaumann · 2020-03-31T12:35:14Z

If you really want you could try to expose this and check but not necessary, up to you

I would rather not in the scope of this PR. Another problem is, that it's a multiplication of sparse and dense matrices. I will change the matrix used in the test, if the CI complains.+

And regarding enable_if, I would stick to using it in CreateFEAST, if there is nothing severe against it.

philbucher · 2020-03-31T16:38:03Z

@KratosMultiphysics/technical-committee this PR adds the latest version of FEAST in the EigenSolversApplication.
An older version of FEAST is also in the ExternalSolversApplication, hence I think adding this should not be a problem for anyone
As @qaumann mentioned above the added files sizes are 2.5MB which I think is ok

If you agree then we don't have to use meeting-time for this ;)

philbucher · 2020-04-03T11:54:33Z

you need to merge master, there are some conflicts

…east

qaumann · 2020-04-03T13:54:32Z

the conflict should be resolved

RiccardoRossi · 2020-04-03T14:13:38Z

@KratosMultiphysics/technical-committee proposes the following
1 - accept this PR (and try to remove the old feast from the ExternalSolversApplication)
2 - in a following PR change the name EigenSolversApplication->LinearSolversApplication
3 - add, in a followup PR the (optional) compilation of SuperLU to the LinearSolversApplication
4 - move to legacy the ExternalSolversApplication

IMPORTANT: by default, everything has to compile without BLAS (mkl, or any blas) and without fortran. This means that in the tests we CANNOT use anything depending on those features

philbucher

@qaumann I think this is ready, please feel free to merge
I made a small comment but this can be a future PR

It would be also nice if you could do the following two things:

Port the tests in StructuralMechanics that use the FEAST of the ExternalSolversApp to use the solvers of the EigenSolversApp?
If that is done remove Feast from the ExternalSolversApp as requested by @KratosMultiphysics/technical-committee

Nice work overall and thanks for the patience 👍

philbucher · 2020-04-05T17:31:56Z

applications/EigenSolversApplication/custom_solvers/feast_eigensystem_solver.h

+    template<>
+    Parameters SettingsHelper<double>::GetDefaultParameters()
+    {
+        return Parameters(R"({


@qaumann I suggest you implement / clean up in a follow up PR

qaumann · 2020-04-07T06:20:35Z

@philbucher Thanks again for the review and the comments, I will address them now.

qaumann added 15 commits February 19, 2020 18:03

test feast eigensolvers with mkl

200b734

make cmake work

d4466b3

include as C

f3b425b

make feast work

9f61647

fix eigenvectors (row/column storage)

cca0d1c

use externally compiled feast4

98eda40

add valuetype typedef

be165b7

add complex eigensolver

4ce4958

add complex feast eigensolver

177af95

provide warnings and errors

c55f1ac

add tests for feast eigensolver

1efd553

add eigen feast to factory

32ad78c

update readme

072ba7c

minor

8c6c6d2

minor

539a969

qaumann added the Applications label Feb 28, 2020

qaumann requested review from oberbichler and philbucher February 28, 2020 13:46

fix cmake error message

a26efd5

skip tests if FEAST is not available

8997252

qaumann added 2 commits March 12, 2020 17:05

disabling windows support

7aa650f

add FEAST as external library

299ea82

qaumann marked this pull request as ready for review March 16, 2020 09:13

oberbichler reviewed Mar 30, 2020

View reviewed changes

applications/EigenSolversApplication/custom_python/eigen_solvers_python_application.cpp Outdated Show resolved Hide resolved

oberbichler reviewed Mar 30, 2020

View reviewed changes

kratos/python_scripts/eigen_solver_factory.py Outdated Show resolved Hide resolved

qaumann and others added 4 commits March 31, 2020 09:37

remove argument

f946037

remove hasattr

3e5a8af

created SettingsHelper

917dce7

added CreateFortranIndices

bbec5fd

Merge pull request #6645 from KratosMultiphysics/eigen-solvers/add-fe…

c8ae718

…ast-improvements Improvements for [EigenSolvers] Add FEAST with MKL support to EigenSolversApplication #6482

Merge remote-tracking branch 'origin/master' into eigen-solvers/add-f…

df08b7a

…east

philbucher approved these changes Apr 5, 2020

View reviewed changes

philbucher mentioned this pull request Apr 6, 2020

[Python Wheels] Add dockerfiles #6633

Merged

qaumann merged commit 79eef12 into master Apr 7, 2020

qaumann deleted the eigen-solvers/add-feast branch April 7, 2020 06:21

This was referenced Apr 7, 2020

[StructuralMechanics] Remove ExternalSolversApplication from the tests #6721

Merged

[External solvers] Remove old FEAST version #6725

Merged

philbucher mentioned this pull request Apr 8, 2020

[Core][Py-LinearSolverFactory] clean deprecated code #6741

Merged

oberbichler mentioned this pull request May 7, 2020

Rename EigenSolversApplication to LinearSolversApplication #6888

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[EigenSolvers] Add FEAST with MKL support to EigenSolversApplication #6482

[EigenSolvers] Add FEAST with MKL support to EigenSolversApplication #6482

qaumann commented Feb 28, 2020

oberbichler commented Mar 3, 2020

qaumann commented Mar 3, 2020

philbucher commented Mar 3, 2020

oberbichler commented Mar 4, 2020

qaumann commented Mar 4, 2020

oberbichler commented Mar 5, 2020

qaumann commented Mar 13, 2020

philbucher commented Mar 31, 2020

qaumann commented Mar 31, 2020

philbucher commented Mar 31, 2020

qaumann commented Mar 31, 2020 •

edited

Loading

philbucher commented Mar 31, 2020

philbucher commented Apr 3, 2020

qaumann commented Apr 3, 2020

RiccardoRossi commented Apr 3, 2020

philbucher left a comment

philbucher Apr 5, 2020

qaumann commented Apr 7, 2020

[EigenSolvers] Add FEAST with MKL support to EigenSolversApplication #6482

[EigenSolvers] Add FEAST with MKL support to EigenSolversApplication #6482

Conversation

qaumann commented Feb 28, 2020

oberbichler commented Mar 3, 2020

qaumann commented Mar 3, 2020

philbucher commented Mar 3, 2020

oberbichler commented Mar 4, 2020

qaumann commented Mar 4, 2020

oberbichler commented Mar 5, 2020

qaumann commented Mar 13, 2020

philbucher commented Mar 31, 2020

qaumann commented Mar 31, 2020

philbucher commented Mar 31, 2020

qaumann commented Mar 31, 2020 • edited Loading

philbucher commented Mar 31, 2020

philbucher commented Apr 3, 2020

qaumann commented Apr 3, 2020

RiccardoRossi commented Apr 3, 2020

philbucher left a comment

Choose a reason for hiding this comment

philbucher Apr 5, 2020

Choose a reason for hiding this comment

qaumann commented Apr 7, 2020

qaumann commented Mar 31, 2020 •

edited

Loading