Skip to content

Settings of the main processor

Jump to bottom
Adam Povey edited this page Feb 15, 2022 · 2 revisions

Settings of the ORAC main processor

The primary executable of ORAC (in the src folder) is controlled using a driver file, which we recommend generating with the pyorac scripts though you can generate them manually, to populate the Ctrl structure (which is an argument of virtually every routine in the code). The driver file is interpreted by a bison/flex lexical parser within read_driver(), where we also define the default values. This document summarises meaning/implications of all variables within the Ctrl structure and their default values (within the code, not within pyorac).

Contents of the state vector

  • approach: The retrieval assumptions to use with respect to radiative transfer. Default is determined from the LUT name. Choices are (and these names will appear throughout where they influence default values):
    • AppCld1L: single-layer cloud
    • AppCld2L: two-layer cloud
    • AppAerOx: dual-view aerosol over a BRDF surface parameterisation
    • AppAerSw: dual-view aerosol over a Swansea surface parameterisation
    • AppAerO1: single-view aerosol
  • class: Different retrieval assumptions to use with respect to particle. Default is determined from the LUT name. Choices are:
    • ClsCldWat: liquid water cloud
    • CldCldIce: frozen water cloud
    • ClsAshEyj: volcanic ash
    • ClsAerOx: aerosol over a BRDF surface parameterisation
    • ClsAerSw: aerosol over a Swansea surface parameterisation
  • class2: Like class but for the second cloud layer.
  • AP: The method used to select the a priori value of state vector elements. Choices and defaults are,
    • SelmMeas: Deduce the value from the measurements
    • SelmAux: Draw the value from external information,
      • ITs: Use the bottom level of SPixel%RTM%T with uncertainty AUXErrTsLand or AUXErrTsSea as defined here
      • ISP: Use SPixel%Surface%Sw_p with uncertainty SPixel%Surface%Sw_p_var
      • ISS: Use SPixel%Surface%Sw_s with uncertainty `SPixel%Surface%Sw_s_var
      • IRs: Use SPixel%Surface%Rs or SPixel%Surface%Rs2 with SPixel%Surface%SRs or SPixel%Surface%SRs2, as appropriate
    • SelmPrev: Use the value of the last successful retrieval
    • SelmCtrl: Use Ctrl%XB with Ctrl%Sx (this option overrides the others where they fail for some reason)
    • ITau/ITau2: SelmCtrl
    • IRe/IRe2: SelmCtrl
    • IPc/IPc2: SelmCtrl
    • IFr: SelmMeas
    • IFr2: SelmCtrl
    • ITs: SelmAux
    • ISS/ISP/ISG: SelmCtrl
    • IRs: SelmAux
  • FG: The method used to select the first guess value of state vector elements. Choices are the same as AP, except SelmCtrl uses Ctrl%X0 rather than Ctrl%XB. Defaults are,
    • ITau/ITau2: SelmCtrl
    • IRe/IRe2: SelmCtrl
    • IPc: SelmMeas
    • IPc2: SelmCtrl
    • IFr/IFr2: SelmCtrl
    • ITs: SelmAux
    • ISS/ISP/ISG: SelmCtrl
    • IRs: SelmAux
  • XB: Assumed a priori values. Defaults are,
    • For a two-layer cloud,
      • ITau: 0.1
      • IRe: 12 for ClsCldWat, 0.7 for ClsAshEyj, and 30 otherwise
      • IPc: 245
      • ITau2: 0.8
      • IRe2: 30 for ClsCldIce, 0.7 for ClsAshEyj, and 12 otherwise
      • IPc2: 800
    • Otherwise,
      • ITau: 0.18 for ClsAshEyj, -0.3 for ClsAerSw, -1.5 for ClsAerOx, and -0.8 otherwise
      • IRe: 30 for ClsCldIce, 12 for ClsCldWat, 0.7 for ClsAshEyj, and -0.07 otherwise
      • IPc: 400 for ClsCldIce, 600 for ClsAshEyj, and 900 otherwise
    • IFr/IFr2: 1
    • ITs: 300
    • ISS: 0.1
    • ISP/ISG: 0.3
    • IRs: 0.01
  • X0: Assumed first guess values. Defaults are the same as XB.
  • SX: Assumed variances. Defaults are,
    • For a two-layer cloud,
      • ITau/ITau2: 2 for aerosol and 1e8 otherwise
      • IRe: 0.5 for aerosol and 1e8 otherwise
      • IPc/IPc2: 200
      • IRe2: 0.5 for aerosol and 4 otherwise otherwise
    • Otherwise,
      • ITau: 0.18 for ClsAshEyj, -0.3 for ClsAerSw, -1.5 for ClsAerOx, and -0.8 otherwise
      • IRe: 30 for ClsCldIce, 12 for ClsCldWat, 0.7 for ClsAshEyj, and -0.07 otherwise
      • IPc: 400 for ClsCldIce, 600 for ClsAshEyj, and 900 otherwise
    • IFr/IFr2: 1e8
    • ITs: 1e8
    • ISS: 1.0
    • ISP: 0.01 for the nadir view and 0.5 otherwise
    • ISG: 0.1
    • IRs: 1e8
  • Sy: Assumed measurement covariance. Default is,
    • Thermal channel: 2.0
    • Otherwise: 0.05
  • Nx: The number of state vector elements to be retrieved (in the ideal circumstance of all expected inputs being valid; there is complicated logic to work out what to do if not). Setting this is very important if manually setting Ctrl%X. Default values depend on the channels provided.
  • X: dim(?,3), the state vector elements to retrieve for IDay, ITwilight, and INight pixels (respectively along the second dimension). Defaults are,
    • AppAerOx
      • Day) ITau, IRe, IRs(IRho_DD) for all solar channels in the first view
      • Twilight or Night) None
    • AppAerSw
      • Day) ITau, IRe, ISS for all solar channels, ISP for all views
      • Twilight or Night) None
    • AppAerO1
      • Day) ITau, IRe, IRs(IRho_DD) for all solar channels in the first view
      • Twilight or Night) None
    • AppCld1L
      • Day) ITau, IRe, IPc, ITs
      • Twilight) IPc, ITs
      • Night)
        • do_new_night_retrieval) ITau, IRe, IPc, ITs
        • !do_new_night_retrieval) IPc, ITs
    • AppCld2L
      • Day) ITau, IRe, IPc, ITau2, IRe2, IPc2, ITs
      • Twilight) IPc, IPc2, ITs
      • Night)
        • do_new_night_retrieval) ITau, IRe, IPc, ITau2, IRe2, IPc2, ITs
        • !do_new_night_retrieval) IPc, IPc2, ITs
  • Nxj: The number of state vector elements that aren't retrieved but should be considered within the Jacobian calculation. Default values depend on the channels provided.
  • XJ: dim(?,3), the state vector elements that aren't retrieved but should be considered within the Jacobian calculation. Default values are,
    • AppAerOx/AppAerSw/AppAerO1: None
    • AppCld1L/AppCld2L: IRs(IRho_DD) for all solar channels
    • AppCld

Generic settings

  • LUTClass: name of the LUT to use.
  • verbose: Print retrieval's progress to terminal. Default true.
  • use_ann_phase: Use MSI_Data%ann_phase as the pixel type rather than MSI_Data%cldtype (later used by %Ctrl%types_to_process). Default true.
  • do_new_night_retrieval: For night pixels under AppCld1L, use cloud_indexing_log_night() rather than cloud_indexing_logic(), which generally attempts to retrieve ITau+IRe. Default true.
  • do_CTX_correction: Calculate the cloud-depth-corrected height values (for comparison to CALIOP). Default true.
  • CTP_correction_limit: The largest acceptable adjustment to cloud top pressure when calculating the CALIOP adjustment. Default is 100.
  • process_cloudy_only: All pixels with MSI_Data%cldmask == 0 are omitted from processing. Default true.
  • process_aerosol_only: All pixels with MSI_Data%cldmask /= 0 are omitted from processing. Default false.
  • force_nighttime_retrieval: All pixels are considered to be night, which largely means that the presence of visible channels is not considered in the logic for selecting a state vector. Default false.
  • Run_ID: Not used. Default 'none'.
  • MaxSolZen: The solar zenith angle between day and twilight. Default 75.
  • MaxSatZen: The maximum satellite zenith angle processed. Default 90.
  • MinRelAzi: The minimum relative azimuth processed (to avoid sunglint). Default 0.
  • Sunset: The solar zenith angle between twilight and night. Default 90.
  • i_equation_form: The visible forward model equation to use (see here). Default is 1 for dual-view BRDF aerosol retrievals, 0 for the Swansea aerosol retrieval, and 3 otherwise.
  • LUTIntSelm: Interpolation method used with the LUTs. Default is LUTIntMethBicubic. Choices are,
    • LUTIntMethBicubic: bicubic interpolation, which is highly recommended unless computing cost is a substantial limitation
    • LUTIntMethLinear: linear interpolation, which tends to cluster retrievals near LUT points
  • Cloudtype: Selects which model of RS%Coreg and RS%Homog to use. Default is 2 for aerosol and 1 otherwise. The meaning of these values is somewhat ambiguous.
  • Max_SDAD: The maximum acceptable distance (in pixels) between a point and the last successful retrieval to use that as a first guess (in X_SDAD()). Default is `0.
  • sabotage_inputs: A testing mode that systematically replaces input variances with fill_value until all possible combinations of channel failure are considered. Default false.
  • surfaces to skip: Pixels with this value of land-sea flag will be omitted from processing. Default is 0. Choices are ILand and ISea.
  • second_aot_ch: ORAC number for the channel at which the second wavelength of AOD is output. Default is 3.
  • Ntypes_to_process: The number of elements in Ctrl%types_to_process. Important to specify when overriding types_to_process.
  • types_to_process: Pixels with values of MSI_Data%Type not in this array will be omitted from processing. Default is,
    • For use_ann_phase and AppCld1L and ClsCldWat, just LIQUID
    • For use_ann_phase and AppCld1L and ClsCldIce, just ICE
    • For !use_ann_phase and AppCld1L and ClsCldWat and !process_cloudy_only, [FOG_TYPE, WATER_TYPE, SUPERCOOLED_TYPE]
    • For !use_ann_phase and AppCld1L and ClsCldWat and process_cloudy_only, [FOG_TYPE, WATER_TYPE, SUPERCOOLED_TYPE, CLEAR_TYPE, PROB_CLEAR_TYPE]
    • For !use_ann_phase and AppCld1L and ClsCldIce and !process_cloudy_only, [OPAQUE_ICE_TYPE, CIRRUS_TYPE, OVERLAP_TYPE, PROB_OPAQUE_ICE_TYPE]
    • For !use_ann_phase and AppCld1L and ClsCldIce and process_cloudy_only, [OPAQUE_ICE_TYPE, CIRRUS_TYPE, OVERLAP_TYPE, PROB_OPAQUE_ICE_TYPE, CLEAR_TYPE, PROB_CLEAR_TYPE]
    • For use_ann_phase and AppCld2L, [FOG_TYPE, WATER_TYPE, SUPERCOOLED_TYPE, SWITCHED_TO_ICE_TYPE, OPAQUE_ICE_TYPE, CIRRUS_TYPE, OVERLAP_TYPE, PROB_OPAQUE_ICE_TYPE, SWITCHED_TO_WATER_TYPE, CLEAR_TYPE, PROB_CLEAR_TYPE]
    • For !use_ann_phase and AppCld2L, just OVERLAP_TYPE
    • Otherwise, all defined types

Further variables are determined by the code, but some of those may be overridden for experimental retrievals.

  • ReChans: As different wavelengths penetrate different depths into a cloud, there is no single effective radius value for a cloud. To resolve this ambiguity, ORAC only processes one channel considered to provide effective radius information. This defines, in order of the preference, the ORAC ID of channels that can be used for such information.
  • r_e_chans: The ORAC ID of all channels that could reasonably be considered to possess effective radius information.
  • tau_chans: The retrieval logic may desire a minimum number of channels provide information on optical depth in order to attempt retrieval of ITau. This lists the ORAC ID of channels that can be used for such information.
  • ir_chans: The retrieval logic may desire a minimum number of channels provide information on cloud top temperature in order to attempt retrieval of IPc. This lists the ORAC ID of channels that can be used for such information.

Ctrl%FID

Names of the input and output files. A user can set,

  • Data_Dir: Location of the input files from ORAC's preprocessor.
  • Filename: The root name of the input files from ORAC's preprocessor.
  • Out_Dir: Destination of the output files from this processing.
  • SAD_Dir: Location of the text LUTs for the first cloud layer.
  • SAD_Dir2: Location of the text LUTs for the second cloud layer.
  • NCDF_LUT_Filename: Location of the netCDF LUT for the first cloud layer.
  • NCDF_LUT_Filename2: Location of the netCDF LUT for the second cloud layer.

The ten input files (config, MSI, LWRTM, SWRTM, PRTM, LS, CF, geo, loc, alb) are derived from the first two and the two output files (L2_primary, L2_secondary)

Ctrl%Ind

Indexing arrays for the datasets. These are largely determined from the input files and not set by the user.

  • X0/X1: First and last pixel to process across track (counting from 1 in whatever field is provided by the preprocessor).
  • Y0/Y1: First and last pixel to process along track (counting from 1 in whatever field is provided by the preprocessor).

Ctrl%RS

Options for the surface reflectance.

  • RsSelm: source of the "assumed" surface reflectance. Default SelmAux. Choices are:
    • SelmAux: Use MSI_Data%alb, %rho_0v (from the input files)
    • SelmCtrl: Use Ctrl%RS%B
  • SrsSelm: source of the "assumed" surface reflectance covariance matrix. Default is SelmCtrl for aerosol and SelmMeas otherwise. Choices are:
    • SelmAux: Use MSI_Data%rho_dd_unc and %rho_dd_cor for the uncertainty and correlation, respectively. Where present, additional constant values snow_unc, veg_unc, bare_unc, svd_unc are added to that (in quadrature).
    • SelmMeas: Use Ctrl%RS%Sb * Rs for the uncertainty and Ctrl%RS%Cb for the correlation.
    • SelmCtrl: Use Ctrl%Sx for the uncertainty and Ctrl%RS%Cb for the correlation.
  • allow_a_default_surface: When the above fail (e.g. because the input data is missing for that pixel), use Ctrl%RS%B rather than a fill value. Default true.
  • add_fractional: Add Ctrl%RS%Sb * Rho_DD to the surface uncertainty. Default true for aerosol and false otherwise.
  • use_full_brdf: when true, use a three-term representation of surface bidirectional reflectance; when false, use a Lambertian surface. Default is false for ClsAerSw and true otherwise.
  • read_full_brdf: read all BRDF terms from input files. Default is false for ClsAerSw and true otherwise.
  • Cb: Correlation of surface reflectance values between wavelengths. Default is 0.98 for AppAerO1, 0.4 for other aerosol retrievals, and 0.2 otherwise.
  • diagonal_SRs: neglect off-diagonal elements of the surface reflectance covariance. Default is true for dual-view aerosol and false otherwise.
  • B: dim(NSolar, 2), assumed surface reflectance. Default is only set of channels similar to AATSR Ch1-4 to
    • [0.01, 0.005, 0.0001, 0] over sea and [0.2, 0.15, 0.10, 0.01] over land for aerosol retrievals or
    • [0.05, 0.02, 0.01, 0.01] over sea and [0.15, 0.10, 0.01, 0.01] over land otherwise.
  • Sb: dim(NSolar, 2), assumed surface reflectance fractional uncertainty. Default is
    • 0 over sea and 2e-4 over land for AppAerOx and
    • 0.2 otherwise.

EqMPN

Options for the measurement covariance.

  • SySelm: source of the measurement covariance matrix. Default SelmAux. Choices are,
    • SelmMeas: use MSI_Data%SD from the input files
    • SelmAux: use SAD_Chan%Thermal%NEBT or SAD_Chan%Solar%NEDR from the old LUTs or SAD_Chan%Solar%SNR from the new LUTs.
    • SelmCtrl: use Ctrl%Sy
  • Homog: add homogeneity noise to the covariance as explained here. Default false for aerosol and true otherwise.
  • Coreg: add coregistration noise to the covariance as explained here. Default false for aerosol and true otherwise.

Invpar

Parameters of the Levenberg-Marquadt iteration.

  • ConvTest: Once convergence is achieved, do an additional iteration using a purely Gauss-Newton step and only accept the solution if the magnitude of cost change is less than 1. Default true for aerosol and false otherwise.
  • MqStart: The Marquardt parameter (\gamma_0 in Rodgers Eq.5.34; alpha in the code) is initialised to Hessian * MqStart. Default 0.001.
  • MqStep: Step in Marquardt paremeter between iterations (multiplicative for rejected steps, divisive for accepted steps). Default 10.
  • MaxIter: The maximum number of iterations. Default 25 for aerosol and 40 otherwise.
  • Ccj: A retrieval has converged when the change in cost reduces by <= Ccj * Ny in an iteration. Default 0.001 for AppAerSw and 0.05 otherwise.
  • Xscale: The state vector is multiplied by these values before linear algebra is done to better condition the matrices. Defaults are,
    • ITau/ITau2: 1 for AppAerSw and 10 otherwise
    • IRe/IRa2: 10 for AppAerOx and 1 otherwise
    • IFr/IFr2: 1000
    • IRs: 100 for AppAerO1, 1000 for AppAerOx, and 1 otherwise
    • All others: 1
  • XLLim: Lower limit for the state vector. Defaults are,
    • ITau/ITau2: -2.0 for aerosol in the first layer and -3.0 otherwise
    • IRe: 0.01 for ClsAshEyj, -2.0 for aerosol, and 0.1 otherwise
    • IRe2: 0.1
    • IPc/IPc2: 10
    • IFr: 0
    • ITs: 200
    • IRs: 0 for cloud and 1e-5 otherwise
    • ISP/ISG: 1e-5
  • XULim: Upper limit for the state vector. Defaults are,
    • ITau/ITau2: 0.7 for aerosol and 2.408 otherwise
    • IRe/IRe2: 100 for CldCldIce, 35 for ClsCldWat, 20 for ClsAshEyj, and 1 otherwise
    • IPc: 1200
    • IFr: 1
    • ITs: 320 for AATSR or ATSR-2 and 400 otherwise
    • IRs: 1
    • ISP: 100
    • ISG: 1
  • Pc_dmz: The smallest acceptable distance between two cloud layers. Default is 50.
  • always_take_GN: Always output the solution of a successful ConvTest, even if it is worse. Default true for aerosol and false otherwise.
  • dont_iter_convtest: When doing the ConvTest, don't increase the iteration count. Default true for aerosol and false otherwise.
  • disable_Ss: Ignore the covariance of inactivate state vector elements. Default true for aerosol and false otherwise.

QC

Empirical limits beyond which quality control flags are set.

  • MaxJ: The maximum acceptable cost. Default is 3 for aerosol and 100 otherwise.
  • MaxDoFN: The maximum acceptable number of degrees of freedom for noise. Default is 2.
  • MaxElevation: The maximum acceptable surface elevation. Default is 1500.
Clone this wiki locally