Skip to content

Intermediary files

Adam Povey edited this page Feb 19, 2019 · 1 revision

The files required to run the main ORAC processor are,

# Default name Descriptive name Type Purpose
1 User_defined.txt Driver file ASCII Specifies data files, relevant folders, and channels to consider
2 INST_ChX.sad Channel specification file ASCII Defines the principal characteristics of a particular instrument channel
3 INST_CLASS_LUT_ChX.sav LUT file ASCII Look-up table LUT of cloud radiative properties for particle model CLASS
4 User_defined.alb.nc Albedo file NCDF Surface albedo data by pixel
5 User_defined.clf.nc Cloud flag file NCDF Cloud flagging bit map by pixel
6 User_defined.geo.nc Geometetry file NCDF Satellite/solar/relative viewing angles by pixel
7 User_defined.loc.nc Geolocation file NCDF Lat/lon by pixel
8 User_defined.lsf.nc Land-sea file NCDF Land-sea flag by pixel
9 User_defined.lwrtm.nc LW-RTM file NCDF Atmospheric RT results in the longwave
10 User_defined.msi.nc Imagery file NCDF Multi-spectral imagery data by pixel
11 User_defined.prtm.nc RTM profile file NCDF Profile data (T,p) used in RTM calculations
12 User_defined.swrtm.nc SW-RTM file NCDF Atmospheric RT results in the shortwave
13 User_defined.config.nc Configuration file NCDF Summarises the channel configuration

1. Driver file

This is the driver file for the ORAC system, outlined here.

1b. How to select with channel to use for effective radius

Using MODIS as an example the default value for Ctrl%ReChans is

Ctrl%ReChans=20,6,7,5

Ctrl%ReChans must have 4 elements based on the length of
Ctrl%r_e_chans=5,6,7,20 (the set of candidate Re channels).

If 5,6,7,20 are given to ORAC (with the bits for all Re channel set in the channel bit mask in the driver file) then it will choose one Re channel based on availability and decreasing priority in the Ctrl%ReChans list. So if 20 is available then 20 is used or if 20 and 6 are not available then 7 is used. If for example you don't want 6 to ever be used then use

Ctrl%ReChans = 20,0,7,5

For Aqua you might want to use

Ctrl%ReChans = 20,7,6,5

If you want more control and don't want ORAC to chose just one effective radius channel then use

Ctrl%ReChans = 0,0,0,0

which basically turns this feature off. In which case it will use whatever is set in the channel bit mask even if more than one Re channel is set.

2. Channel specification file

Each channel for each instrument has an associated file giving its principal characteristics. The file name is constructed as in the section title using the instrument code and the channel identifier. The identifier convention is that ascending number corresponds to increasing wavelength. However, this need not be strictly adhered to. The only requirement is that all channels with solar contributions are numbered below those with only thermal contributions. (For example, the order [Ch1=0.67, Ch2=0.87, Ch3=11] is conventional, [Ch1=0.87, Ch2=0.67, Ch3=11] is acceptable, while [Ch1=0.67, Ch2=11, Ch3=0.87] is not acceptable.) Modifications to these files should be limited to the system administrator. A sample specification file for the ATSR-2 3.7um channel is given below.

ATSR-2_Ch5.sad     % Channel characterisation file
3.7 um             % Descriptor               SAD_CHAN()%Desc
Ch5                % File ID                  SAD_CHAN()%Fileid
2715.30            % Central wavenmuber       SAD_CHAN()%Wvn
1                  % Thermal source flag      SAD_CHAN()%Thermal%Flag
238452.0           % Planck coefficient 1     SAD_CHAN()%Thermal%B1
3885.87            % Planck coefficient 2     SAD_CHAN()%Thermal%B2
-2.041717          % Planck coefficient 3     SAD_CHAN()%Thermal%T1 

1.005042           % Planck coefficient 4     SAD_CHAN()%Thermal%T2
.5,.5,1.,.5,.5     % Thermal noise Eqv Homog  SAD_CHAN()%Thermal%NeHomog
.15,.15,1.,.25,.25 % Thermal noise Eqv Coreg  SAD_CHAN()%Thermal%NeCoreg
0.28               % Thermal NeBT             SAD_CHAN()%Thermal%NeBT
1                  % Solar source flag        SAD_CHAN()%Solar%Flag
5.39,0.177         % Solar constant           SAD_CHAN()%Solar%F0,F1
.5,.5,1.,.5,.5     % Solar noise Eqv Homog    SAD_CHAN()%Solar%NeHomog
.15,.15,1.,.25,.25 % Solar noise Eqv Coreg    SAD_CHAN()%Solar%NeCoreg
0.0                % Solar NeFR               SAD_CHAN()%Solar%NeFR
0.01 1.0           % 'typical' sea/land refl  SAD_CHAN()%Solar%Rs(2)

This channel is chosen as an example because it has both solar and thermal components and hence the characterisation file is more complete than most other channels. If a source (thermal or solar) is not present, then the detail following the flag for that source is omitted; the file below is for the ATSR-2 0.67um channel which has only solar sources.

ATSR-2_Ch2.sad     % Channel characterisation file
0.67 um            % Descriptor               SAD_CHAN()%Desc
Ch2                % File ID                  SAD_CHAN()%Fileid
14925.4            % Central wavenmuber       SAD_CHAN()%Wvn
0                  % Thermal source flag      SAD_CHAN()%Thermal%Flag
1                  % Solar source flag        SAD_CHAN()%Solar%Flag
21.46, 0.703       % Solar constant           SAD_CHAN()%Solar%F0,F1
.5, .75,2.,1.,.75  % Solar noise Eqv Homog    SAD_CHAN()%Solar%NeHomog
1.5,2., 3.,1.,1.   % Solar noise Eqv Coreg    SAD_CHAN()%Solar%NeCoreg
0.0058             % Solar NedR               SAD_CHAN()%Solar%NedR
0.1 5.0            % 'typical' sea/land refl  SAD_CHAN()%Solar%Rs(2)

3. LUT file

LUT file names are constructed from the instrument, channel, cloud class and radiative properties codes. Thus the water cloud bidirectional reflectance file for the SEVIRI 0.8um channel is named SEVIRI-1_WAT_RBD_Ch2.sad. Files are ASCII formatted for portability. Each file has header information describing its contents and allowing for some consistency checks between it and the requested parameters. The header contents are described in the following table; see following sections for full details.

LUT Description Solar Thermal Wavelength Tau Sat zen Sun zen Rel azi Re
Needed for ? Header information present ?
grid descriptors (nX, dX, X)
Rbd Bidirection reflectance * * * * * * *
Tb Beam direct transmission * * * * *
Tbd + Tfbd Beam diffuse transmission + flux * * *
Td + Tfd Diffuse transmission + flux * * * * * *
Rd + Rfd Diffuse reflection + flux * * * * * *
Em Emission * * * * *

The exact contents of the LUT files depend on which grid descriptors apply to the LUT in question (see table above). However, the general structure is the same in each case:

  • Line 1: Wavelength (f8.2). Store in SAD_LUT()%wavelength
  • Line 2: Grid descriptor 1: number of values (integer), grid step size for this descriptor (float)
  • Line 3: Grid descriptor 1: grid descriptor values
  • Line 4: Grid descriptor 2: as above
  • Line 5 etc as above for all relevant grid descriptors
  • Line n to end of file: LUT values (f8.3); stored in SAD_LUT( )%LUT(x, y, z, ...)

For example, the RBD LUT files contain the following information:

Wavelength (f8.2). Store in ''SAD_LUT()%wavelength''
no. of tau (i2), delta_Tau (f8.2). Check against ''SAD_CloudClass()%nTau''),
Tau values (nTau * f8.3). First (1) stored in ''SAD_LUT()%Grid%!MinTau'', last (nTau) stored in ''%!MaxTau''
No. of Satzen values, dSatzen i4, f8.2 check nsat against ''SAD_CloudClass()%nSat'', dsat stored in ''SAD_LUT()%Grid%dSatzen'', nsat stored in ''SAD_LUT()%Grid%nSatzen''
Satzen values (nSat * f8.2) satz(1) stored in ''SAD_LUT()%Grid%!MinSatzen'', satz(nsat) stored in !MaxSatzen
No. of Sunzen values, dSunzen (i4, f8.2) check nsun against ''SAD_CloudCLass()%nSun'', dsun stored in ''SAD_LUT()%dSunzen'', nsun stored in ''SAD_LUT()%nSunzen''
Sunzen values (nsun* f8.2) sunz (1) stored in ''SAD_LUT()%Grid%!MinSunzen'', sunz(nsun) in !MaxSunzen
No. of Relazi values, dRelazi (i4, f8.2) check nazi against ''SAD_CloudCLass()%nAzi'', dRelazi stored in ''SAD_LUT()%Grid%dRelazi'',
Relazi values (nazi * f8.2) azi(1) stored in ''SAD_LUT()%Grid%!MinRelazi'', azi(nazi) stored in ''!MaxRelazi'' No. of Re values, dRe (i4, f8.2) check nr against ''SAD_CloudCLass()%nRe'', dr stored in ''SAD_LUT()%Grid%nRe'',
Re values (nre * f8.2), Re(1) stored in ''SAD_LUT()%Grid%!MinRe'', re(nre) stored in ''!MaxRelazi''
RBD (f8.3) stored in ''SAD_LUT()%Solar%Rbd'' (chan_index, 1:ntau, 1:nsat, 1:nsun,1:nazi,1:nr) (written left indices first)

4. Albedo file

Provides the surface albedo and emissivity information. The variables in the file are: alb_abs_ch_numbers, emis_abs_ch_numbers, alb_data, emis_data, rho_0d_data, rho_0v_data, rho_dd_data, rho_dv_data.

5. Cloud flag file

Cloud flags are assumed to be provided in a file separately to the MSI data. For ATSR-2 data, pre-processor code will generate a cloud flag file. See the section on image segmentation above. The variables in the file are: ann_phase, ann_phase_uncertainty, cccot_pre, cflag, cldmask, cldmask_uncertainty, cldtype, cphcot.

6. Geometry file

ATSR MSI data includes geometric information, but in order to retain the functionality required for other data sources, it is assumed that a separate pixel-based geometry file will be available. The variables in the file are: solzen, satzen, solaz, sataz, relazi.

7. Geolocation file

Latitude and longitude for each pixel, taken from the geolocation considerations of the original satellite data. The variables in the file are: lat, lon.

8. Land-sea file

ORAC will not obtain the underlying surface character for a particular pixel but will assume a pixel-based map is available via this file. For ATSR-2 processing, this will obtained by the pre-processor code. See the section on image segmentation below. The variables in the file are: dem, lsflag, lusflag, nisemask.

9. LW-RTM file

The longwave radiative transfer file supplies the atmospheric channel transmittances and radiance terms (see the ATBD) for the channels with thermal emission sources. The source of this file for ORAC prototyping and testing will be the Eumetsat RTM model. See the !Read_Lw_RTM module description for details of the Lw RTM file format and contents. The variables in the file are: lw_channel_abs_ids, lw_channel_instr_ids, lw_channel_wvl, emiss_lw, tac_lw, tbc_lw, rbs_up_lw, rac_up_lw, rac_down_lw.

10. Imagery file

To avoid the difficulties of ingesting multiple data formats from different imagers within the retrieval itself, this is managed within the pre-processing to produce this file summarising only the desired spectral information in a format appropriate to ORAC. The variables in the file are: msi_instr_ch_numbers, msi_abs_ch_wl, msi_ch_swflag, msi_ch_lwflag, msi_ch_view, time_data, msi_data, sd_data.

11. RTM profile file

This file contains the level pressure and temperature information that was used in the RTM models. See the Read_Lw_RTM module description for details of the RTM profile file format and contents. The variables in the file are: lon_rtm, lat_rtm, hprofile_rtm, o3profile_rtm, pprofile_rtm, qprofile_rtm, tprofile_rtm.

12. SW-RTM file

The shortwave radiative transfer file supplies the atmospheric channel transmittances and radiance terms (see the ATBD) for the channels without thermal emission sources. The source of this file for ORAC prototyping and testing will be a modified LOWTRAN7 RTM model. See the Read_Sw_RTM module description for details of the Sw RTM file format and contents. The variables in the file are: sw_channel_abs_ids, sw_channel_instr_ids, sw_channel_wvl, tac_sw, tbc_sw.

13. Configuration file

Provides dimension and arrays needed to initialise the reading of all the other files. The variables in the file are: alb_abs_ch_numbers, emis_abs_ch_numbers, msi_abs_ch_wl, msi_ch_lwflag, msi_ch_swflag, msi_ch_view, msi_instr_ch_numbers.