Version 1.6 release notes and doc updates

cgi-estonia-space · May 15, 2023 · 5dcfbaf · 5dcfbaf
1 parent 3669742
commit 5dcfbaf
Show file tree

Hide file tree

Showing 19 changed files with 98 additions and 55 deletions.
diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -2,7 +2,7 @@ cmake_minimum_required(VERSION 3.20 FATAL_ERROR)
 project(ALUs
         LANGUAGES CXX CUDA
         DESCRIPTION "Parallel ALUs for geospatial data processing"
-        VERSION 1.5.0
+        VERSION 1.6.0
         )
 
 set(CMAKE_CUDA_STANDARD 17)

diff --git a/README.md b/README.md
@@ -12,7 +12,8 @@ through [ESA's GSTP programme](https://www.esa.int/Enabling_Support/Space_Engine
 
 Developed by [CGI Estonia](https://www.cgi.com/ee/et).
 
-## For appetizer -> [performance test](PERFORMANCE.md) and [further ESA SNAP comparison](docs/SNAP_COMPARISON.md)
+## [Quick performance overview](PERFORMANCE.md)
+For further comprehensive evaluation see [Wiki](https://github.com/cgi-estonia-space/ALUs/wiki) 
 
 # Out of the box usage
 
@@ -82,10 +83,13 @@ It can be used in conjunction with binaries to easily execute code. Read the [in
 
 # Minimum/Recommended requirements
 
+For specific, check each processor's README.
+
+Below are rough figures:
 * NVIDIA GPU device compute capability 6.0 (Pascal) or higher
 * 2(minimum)/4(recommended) GB of available device memory (some ALUs can manage with less)
 * High speed (NVMe) SSD to benefit from the computation speedups
-* 4 GB of avalable RAM to enable better caching/input-output
+* 4 GB of extra RAM to enable better caching/input-output (GDAL raster IO)
 
 # Contributing
 

diff --git a/RELEASE.md b/RELEASE.md
@@ -1,3 +1,25 @@
+# Release 1.6.0
+
+## Breaking changes
+
+## Known Caveats
+* Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP.
+  Currently it is unresolved what is the correct way. See examples [A](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/coh_missing_pixels.png) and [B](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/beirut_iw1_b6_coastal.png).
+  See posts about the issues/bugs - [tile size affecting results](https://forum.step.esa.int/t/tile-size-is-affecting-coregistration-results/32193/4)
+  and [no-data value interpretation](https://forum.step.esa.int/t/coregistration-no-data-value/35304/2).
+* Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see [slideshow](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/GPU-GSTP-MPR-0008.pdf)
+
+## Major Features and Improvements
+* Basic S1 metadata in the produced results for `alus-coh`, `alus-cal` and `alus-coht` - [PR item](https://github.com/cgi-estonia-space/ALUs/pull/29)
+* Polynomial estimation for subtract flat earth phase for each burst in coherence opeation - [PR item](https://github.com/cgi-estonia-space/ALUs/pull/26)
+* Jupyter notebook and related dependencies updates - [PR item](https://github.com/cgi-estonia-space/ALUs/pull/25)
+* GRD support for `alus-cal` - [PR item](https://github.com/cgi-estonia-space/ALUs/pull/31)
+
+## Bug Fixes and Other Changes
+
+## Thanks to our Contributors
+
+
 # Release 1.5.0
 
 ## Breaking changes
@@ -6,10 +28,10 @@
 
 ## Known Caveats
 * Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP.
-  Currently it is unresolved what is the correct way. See examples [A](docs/coh_missing_pixels.png) and [B](docs/beirut_iw1_b6_coastal.png).
+  Currently it is unresolved what is the correct way. See examples [A](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/coh_missing_pixels.png) and [B](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/beirut_iw1_b6_coastal.png).
   See posts about the issues/bugs - [tile size affecting results](https://forum.step.esa.int/t/tile-size-is-affecting-coregistration-results/32193/4)
   and [no-data value interpretation](https://forum.step.esa.int/t/coregistration-no-data-value/35304/2).
-* Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see [slideshow](docs/GPU-GSTP-MPR-0008.pdf)
+* Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see [slideshow](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/GPU-GSTP-MPR-0008.pdf)
 * Processing scenes with Copernicus DEM 30m COG has not well defined behavior for some specific cases.
   See workaround in ALUs - https://github.com/cgi-estonia-space/ALUs/wiki/DEM-handling-functionality-for-Sentinel-1-processors#copernicus-dem-cog-30m
   SNAP processing discrepancies - https://forum.step.esa.int/t/copernicus-dem-complications-when-coregistering-s1/38659/3
@@ -32,10 +54,10 @@
 
 ## Known Caveats
 * Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP.
-  Currently it is unresolved what is the correct way. See examples [A](docs/coh_missing_pixels.png) and [B](docs/beirut_iw1_b6_coastal.png).
+  Currently it is unresolved what is the correct way. See examples [A](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/coh_missing_pixels.png) and [B](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/beirut_iw1_b6_coastal.png).
   See posts about the issues/bugs - [tile size affecting results](https://forum.step.esa.int/t/tile-size-is-affecting-coregistration-results/32193/4)
   and [no-data value interpretation](https://forum.step.esa.int/t/coregistration-no-data-value/35304/2).
-* Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see [slideshow](docs/GPU-GSTP-MPR-0008.pdf)
+* Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see [slideshow](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/GPU-GSTP-MPR-0008.pdf)
 * There are two code paths to calculate terrain corrected outputs' dimensions and geocoordinates
   ([see description](https://forum.step.esa.int/t/range-doppler-terrain-correction-raster-size-and-coordinates-inconsistency/35977)). Since ALUs
   implements the SNAP UI way the end results probably differ when compared to SNAP GPT outputs.
@@ -55,10 +77,10 @@
 
 ## Known Caveats
 * Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP.
-  Currently it is unresolved what is the correct way. See examples [A](docs/coh_missing_pixels.png) and [B](docs/beirut_iw1_b6_coastal.png).
+  Currently it is unresolved what is the correct way. See examples [A](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/coh_missing_pixels.png) and [B](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/beirut_iw1_b6_coastal.png).
   See posts about the issues/bugs - [tile size affecting results](https://forum.step.esa.int/t/tile-size-is-affecting-coregistration-results/32193/4)
   and [no-data value interpretation](https://forum.step.esa.int/t/coregistration-no-data-value/35304/2).
-* Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see [slideshow](docs/GPU-GSTP-MPR-0008.pdf)
+* Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see [slideshow](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/GPU-GSTP-MPR-0008.pdf)
 * There are two code paths to calculate terrain corrected outputs' dimensions and geocoordinates
   ([see description](https://forum.step.esa.int/t/range-doppler-terrain-correction-raster-size-and-coordinates-inconsistency/35977)). Since ALUs
   implements the SNAP UI way the end results probably differ when compared to SNAP GPT outputs.
@@ -77,10 +99,10 @@
 
 ## Known Caveats
 * Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP.
-  Currently it is unresolved what is the correct way. See examples [A](docs/coh_missing_pixels.png) and [B](docs/beirut_iw1_b6_coastal.png).
+  Currently it is unresolved what is the correct way. See examples [A](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/coh_missing_pixels.png) and [B](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/beirut_iw1_b6_coastal.png).
   See posts about the issues/bugs - [tile size affecting results](https://forum.step.esa.int/t/tile-size-is-affecting-coregistration-results/32193/4) 
   and [no-data value interpretation](https://forum.step.esa.int/t/coregistration-no-data-value/35304/2).
-* Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see [slideshow](docs/GPU-GSTP-MPR-0008.pdf)
+* Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see [slideshow](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/GPU-GSTP-MPR-0008.pdf)
 * There are two code paths to calculate terrain corrected outputs' dimensions and geocoordinates
   ([see description](https://forum.step.esa.int/t/range-doppler-terrain-correction-raster-size-and-coordinates-inconsistency/35977)). Since ALUs
   implements the SNAP UI way the end results probably differ when compared to SNAP GPT outputs.
@@ -104,10 +126,10 @@
 
 ## Known Caveats
 * Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP.
-  Currently it is unresolved what is the correct way. See examples [A](docs/coh_missing_pixels.png) and [B](docs/beirut_iw1_b6_coastal.png).
+  Currently it is unresolved what is the correct way. See examples [A](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/coh_missing_pixels.png) and [B](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/beirut_iw1_b6_coastal.png).
   See posts about the issues/bugs - [tile size affecting results](https://forum.step.esa.int/t/tile-size-is-affecting-coregistration-results/32193/4) 
   and [no-data value interpretation](https://forum.step.esa.int/t/coregistration-no-data-value/35304/2).
-* Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see [slideshow](docs/GPU-GSTP-MPR-0008.pdf)
+* Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see [slideshow](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/GPU-GSTP-MPR-0008.pdf)
 
 ## Major Features and Improvements
 * `--no_mask_cor` implemented to disable masking during the coregister step for coherence estimation. Disabling avoids 
@@ -137,9 +159,9 @@ This is the first release with refactored architecture where each processor is a
 
 ## Known Caveats
 * Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP.
-  Currently it is unresolved what is the correct way. See examples [A](docs/coh_missing_pixels.png) and [B](docs/beirut_iw1_b6_coastal.png).
+  Currently it is unresolved what is the correct way. See examples [A](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/coh_missing_pixels.png) and [B](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/beirut_iw1_b6_coastal.png).
   See posts about the issues/bugs - [tile size affecting results](https://forum.step.esa.int/t/tile-size-is-affecting-coregistration-results/32193/4) and [no-data value interpretation](https://forum.step.esa.int/t/coregistration-no-data-value/35304/2).
-* Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see [slideshow](docs/GPU-GSTP-MPR-0008.pdf)
+* Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see [slideshow](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/GPU-GSTP-MPR-0008.pdf)
 
 ## Major Features and Improvements
 * There is a Jupyter notebook, which enables to more easily assign processing parameters, inputs and it automatically downloads needed auxiliary files background. 
@@ -164,7 +186,7 @@ This will be the last release with the current architecture (algorithms as share
 
 * Auxiliary files must be separately downloaded and supplied via CLI arguments.
 * Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines.
-  Currently investigation is ongoing to find out the exact reason. See examples [A](docs/coh_missing_pixels.png) and [B](docs/beirut_iw1_b6_coastal.png).
+  Currently investigation is ongoing to find out the exact reason. See examples [A](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/coh_missing_pixels.png) and [B](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/beirut_iw1_b6_coastal.png).
 
 ## Major Features and Improvements
 
@@ -184,7 +206,7 @@ This will be the last release with the current architecture (algorithms as share
 
 * Auxiliary files must be separately downloaded and supplied via CLI arguments
 * Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines.
-  Currently investigation is ongoing to find out the exact reason. See examples [A](docs/coh_missing_pixels.png) and [B](docs/beirut_iw1_b6_coastal.png)
+  Currently investigation is ongoing to find out the exact reason. See examples [A](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/coh_missing_pixels.png) and [B](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/beirut_iw1_b6_coastal.png)
 
 ## Major Features and Improvements
 

diff --git a/VERSION b/VERSION
@@ -1,3 +1,3 @@
 #define VERSION_MAJOR 1
-#define VERSION_MINOR 5
+#define VERSION_MINOR 6
 #define VERSION_PATCH 0
diff --git a/algs/calibration-routine/README.md b/algs/calibration-routine/README.md
@@ -26,7 +26,7 @@ Processing steps below.
 
 ```
 ALUs - Calibration
-Version 1.4.0
+Version 1.6.0
 
 Arguments:
 
@@ -37,16 +37,18 @@ Arguments:
                              same folder as final outcome). NOTE - this may 
                              decrease performance. Default OFF.
   --sw arg                   Subswath for which the calibration will be 
-                             performed, one of the following - IW1;IW2;IW3.
+                             performed, one of the following - IW1;IW2;IW3. Not
+                             used for GRD.
   -p [ --polarisation ] arg  Polarisation for which the calibration will be 
                              performed - VV;VH
   --bi1 arg                  First burst index - starting at '1', leave 
-                             unspecified for whole subswath
+                             unspecified for whole subswath and GRD
   --bi2 arg                  Last burst index - starting at '1', leave 
-                             unspecified for whole subswath
+                             unspecified for whole subswath and GRD
   -a [ --aoi ] arg           Area Of Interest WKT polygon. Or shapefile (.shp) 
                              consisting similar geometry.Overrules first and 
-                             last burst indexes and subswath selections.
+                             last burst indexes and subswath selections. Not 
+                             effective for GRD.
   -t [ --type ] arg          Type of calibration to be performed, one of the 
                              following - sigma;beta;gamma;dn
   --dem arg                  DEM file(s). SRTM3 and Copernicus DEM 30m COG are 
@@ -59,10 +61,15 @@ Arguments:
 
 
 https://github.com/cgi-estonia-space/ALUs
+
 ```
 
 ## Performance
 
+Please see the latest and always updated comprehensive evaluation in the [Wiki](https://github.com/cgi-estonia-space/ALUs/wiki/Calibration-routine-evaluation)
+
+**Following contents below might be deprecated**
+
 Reference laptop computer details:  
 CPU: Intel i7 10750h </br>
 RAM: 32GB </br>

diff --git a/algs/coherence-estimation-routine/README.md b/algs/coherence-estimation-routine/README.md
@@ -18,6 +18,11 @@ Coherence estimation routine consists of:
 ### alus-coh
 
 ```
+ALUs - Coherence estimation routine
+Version 1.6.0
+
+Arguments:
+
   -h [ --help ]                         Print help
   -r [ --in_ref ] arg                   Reference scene's input SAFE dataset 
                                         (zipped or unpacked)
@@ -43,9 +48,11 @@ Coherence estimation routine consists of:
   -p [ --polarisation ] arg             Polarisation for which coherence 
                                         estimation will be performed - VV;VH
   --sw arg                              Reference scene's subswath
-  -a [ --aoi ] arg                      Area Of Interest WKT polygon, overrules
-                                        first and last burst indexes
-  --dem arg                             DEM file(s). SRTM3 and Copernicus DEM
+  -a [ --aoi ] arg                      Area Of Interest WKT polygon. Or 
+                                        shapefile (.shp) consisting similar 
+                                        geometry.Overrules first and last burst
+                                        indexes and subswath selections.
+  --dem arg                             DEM file(s). SRTM3 and Copernicus DEM 
                                         30m COG are currently supported.
   --no_mask_cor                         Do not mask out areas without elevation
                                         in coregistration
@@ -77,6 +84,8 @@ Coherence estimation routine consists of:
                                         be used for processing
 
 
+https://github.com/cgi-estonia-space/ALUs
+
 ```
 
 ### alus-coht
@@ -127,6 +136,10 @@ Coherence estimation routine consists of:
 
 ## Performance
 
+Please see the latest and always updated comprehensive evaluation in the [Wiki](https://github.com/cgi-estonia-space/ALUs/wiki/Coherence-estimation-evaluation)
+
+**Following contents below might be deprecated**
+
 Reference laptop computer details:  
 CPU: Intel i7 10750h </br>
 RAM: 32GB </br>
@@ -201,10 +214,7 @@ Band 1 Block=14224x1 Type=Float32, ColorInterp=Gray
 Legend</br>
 ![relative_error_legend](../../docs/relative_error_legend.png)
 
-Full scene</br>
-![full_scene](../../docs/coh_relative_error_colored.png)
-
-Orbit state vector related, [see cause](../../docs/GPU-GSTP-MPR-0008.pdf)
+Orbit state vector related, [see cause](https://alus-goods-set.s3.eu-central-1.amazonaws.com/alus_repo_docs/GPU-GSTP-MPR-0008.pdf)
 ![spots](../../docs/coh_relative_error_spots.png)
 
-For Copernicus DEM such comparison is not done because of the pending issues found in the SNAP - https://forum.step.esa.int/t/copernicus-dem-complications-when-coregistering-s1/38659/2
+For Copernicus DEM such comparison is not done because of the pending issues found in the SNAP - https://forum.step.esa.int/t/copernicus-dem-complications-when-coregistering-s1/38659/2