Pipeline Scripts
This section is designed to outline the key procedures of each of the pipeline scripts.
This script features the main preprocessing and cleaning of the data before the ICA process begins. Data is flagged for high standard deviation by channel to remove comically bad channels. The data is then re-referenced to the average site, and high/low pass filtered. Channel neighbour correlations are then calculated to find odd channels and bridged channels. The most redundant channel is also flagged to reduce the rank of the data. The data is average re-referenced without any flagged channels and channel correlations are calculated by time and flagged. The file is then saved containing the marks structure, then saved again with all of the mentioned flags purged from the data in order to prime it for ICA.
| Variable | Default in Config File | Description |
|---|---|---|
| [in_path] | [] | Relative path to input data files assuming cd = work_path |
| [montage_info] | Montage information to fit channel locations on a standard surface. This can be a transformations matrix (e.g. [-.14,-22,-48,-.07,0,-1.57,1080,1260,1240]), or a file containing a transformation matrix (e.g. [batch_dfn]_transmat.mat), or a BIDS channel location tsv file (assuming that during BIDS init the channels were coregistered to the standard surface). | |
| [staging_script] | derivatives/lossless/code/scripts/staging_script.m | Script file path/name that creates marks based on events in the raw data file. |
| [recur_sec] | 1 | Recurrence (sec) for artefact detection epoching |
| [limit_sec] | [0 1] | Limits (sec) for artifact detection epoching |
| [sd_t_meth] | q | Time based mark. Marks periods of time with high standard deviation across channels for rejection. See Vised-Marks wiki, here |
| [sd_t_vals] | [.3 .7] | Quantile size to determine critical distance. See documentation link above. |
| [sd_t_o] | 6 | Factor to multiply against critical distance to determine outliers. See documentation link above. |
| [sd_t_f_meth] | fixed | See Vised-Marks wiki, here |
| [sd_t_f_vals] | Leave empty for fixed. See above documentation for other methods. | |
| [sd_t_f_o] | 0.2 | Fixed specific criteria. Interpretation is that 20% of channels must agree that time period is an outlier to be rejected. |
| [sd_t_pad] | 1 | Number of seconds to pad onto each side of the above flag time flag |
| [sd_ch_meth] | q | Channels based mark. Marks channels with high standard deviation across time for rejection. See Vised-Marks wiki, here |
| [sd_ch_vals] | [.3 .7] | Quantile size to determine critical distance. See documentation link above. |
| [sd_ch_o] | 6 | Factor to multiply against critical distance to determine outliers. See documentation link above. |
| [sd_ch_f_meth] | fixed | See Vised-Marks wiki, here |
| [sd_ch_f_vals] | Leave empty for fixed. See above documentation for other methods. | |
| [sd_ch_f_o] | 0.2 | Fixed criteria flagging. Interpretation is that a channel must have outlier behaviour 20% of the time to be rejected. See documentation link above. |
| [ref_loc_file] | derivatives/lossless/code/misc/standard_1020_bucanl19.elc | Name of file containing the reference locations (including the relative path) |
| [low_bound_hz] | 1 | Lower bound of the filter bass-band |
| [high_bound_hz] | Upper bound of the filter bass-band | |
| [save_f_res] | 1 | 1 if you want to save the filter residuals |
| [n_nbr_ch] | 3 | Number of channels to use in nearest neighbour r calculation (for channel criteria) |
| [r_ch_meth] | q | Method used for flagging low r channels. See here |
| [r_ch_vals] | [.3 .7] | Percentage trim for confidence intervals during low r channel standard deviation criteria (e.g. [.3 .7]) |
| [r_ch_o] | 6 | Factor to multiply against critical distance to determine outliers. See documentation link above. |
| [r_ch_f_meth] | fixed | Fixed method used for flagging low r channels (should be 'fixed') |
| [r_ch_f_vals] | Leave empty for fixed. See above documentation for other methods. | |
| [r_ch_f_o] | 0.2 | Flagging specific criteria. Interpretation is that a channel must be unlike other channels greater than 20% of the time to be rejected. |
| [bridge_trim] | 40 | Percentage trim for z calculation of bridged channels (e.g. 40 = 20% top and bottom) |
| [bridge_z] | 6 | z threshold for flagging channel during bridging criteria |
| [n_nbr_t] | 3 | Number of channels to use in nearest neighbour r calculation (for epoch criteria) |
| [r_t_meth] | q | See Vised-Marks wiki, here |
| [r_t_vals] | [.3 .7] | Quantile size to determine critical distance. See documentation link above. |
| [r_t_o] | 6 | Factor to multiply against critical distance to determine outliers. See documentation link above. |
| [r_t_f_meth] | fixed | Fixed method used for flagging low r epochs (should be 'fixed') |
| [r_t_f_vals] | Leave empty for fixed. See above documentation for other methods. | |
| [r_t_f_o] | 0.2 | Flagging specific criteria. Interpretation is that a channel is bridged if highly like other channels 20% of the time. |
| [min_gap_ms] | 2000 | Minimum time (ms) to allow between periods marked for rejection. These short periods will be marked as mark_gap. |
| [out_path] | [] | Relative path to output data files assuming cd = work_path |
| [amica_param_file] | derivatives/lossless/code/misc/amica15_default.param | template amicadefs.param file to modify |
| [amica_threads_s02] | 8 | number of threads to use for running s02 Amica script |
The Amica script is run once initially, and then three more times in parallel after the component artefact rejection to ensure that the Amica procedure is replicable if run multiple times on the same dataset.
Versions
- amica_init (s02)
- amica_A,B,C (s04a, s04b, s04c)
| Variable | Default in Config File | Description |
|---|---|---|
| [dep_path] | derivatives/lossless/code/dependencies/eeglab_asr_amica/plugins/amica | Relative path to Amica executable program assuming cd = work_path |
| [in_path] | [] | Relative path to input data files assuming cd = work_path |
| [amica_ver] | amica15 | Name of the Amica executable program version |
This script loads the results of the preliminary ICA and uses the standard deviation of the components to flag and remove time that does not fit the model. This will help make the next iteration of Amica more stable. The log likelihood of the model is also ingested for future reference. It then prepares the required parameter files for the parrallel Amicas (s04a, s04b, s04c).
| Variable | Default in Config | Description |
|---|---|---|
| [in_path] | derivatives/lossless | Path to input and output data files assuming cd = work_path |
| [recur_sec] | 1 | Recurrence (sec) for artifact detection epoching |
| [limit_sec] | [0 1] | Limits (sec) for artifact detection epoching |
| [sd_t_meth] | q | Time based mark. This mark is meant to reject periods of time where components display behaviour unlike themselves. See Vised-Marks wiki, here |
| [sd_t_vals] | [.3 .7] | Quantile size to determine critical distance. See documentation link above. |
| [sd_t_o] | 6 | Factor to multiply against critical distance to determine outliers. See documentation link above. |
| [sd_t_f_meth] | fixed | Fixed method used for flagging epochs (should be 'fixed'). See Vised-Marks wiki, here . |
| [sd_t_f_vals] | Should be left empty for fixed methodology. | |
| [sd_t_f_o] | 0.2 | Flagging specific criteria. Can be interpreted such that 20% of components must all agree that a period of time is artefactual for it to be removed. |
| [out_path] | derivatives/lossless | Relative path to output data files assuming cd = work_path |
| [min_gap_ms] | 2000 | Minimum time (ms) to allow between periods marked for rejection |
| [amica_param_file] | derivatives/lossless/code/misc/amica_default.param | Template amicadefs.param file to modify |
| [amica_threads_s04] | 8 | number of threads to use for running s04a, s04b, s04c amica scripts |
This script combines the results of Amica A,B and C. It takes this information and using ICAtest it determines which components were replicated across runs. Additionally, time sections with high standard deviation, beta, theta and alpha power are marked across time. Other processing includes: dipole fitting, ICLabel classification, and BIDS exporting.
| Variables | Default in Config | Description |
|---|---|---|
| [in_path] | derivatives/lossless | Relative path to input data files assuming cd = work_path |
| [recur_sec] | 1 | Recurrence (sec) for the current artifact detection epoching |
| [limit_sec] | [0 1] | Limits (sec) for the current artifact detection epoching |
| [sd_t_meth] | q | Time based mark. This mark is meant to reject periods of time where components display behaviour unlike themselves. See Vised-Marks wiki, here |
| [sd_t_vals] | [.3 .7] | Quantile size to determine critical distance. See documentation link above. |
| [sd_t_o] | 6 | Factor to multiply against critical distance to determine outliers. See documentation link above. |
| [sd_t_f_meth] | fixed | Fixed method used for flagging epochs (should be 'fixed'). See Vised-Marks wiki, here . |
| [sd_t_f_vals] | Should be left empty for fixed methodology. | |
| [sd_t_f_o] | 0.2 | Flagging specific criteria. Can be interpreted such that 20% of components must all agree that a period of time is artefactual for it to be removed. |
| [min_gap_ms] | 2000 | Minimum time (ms) to allow between periods marked for rejection. |
| [icr_crit] | 0.05 | False discovery rate of component replicablilty, measure of sensitivity. See ISCTest. |
| [dip_vol] | derivatives/lossless/code/misc/standard_vol.mat | file path/name of the standard volume file used in dipole fitting |
| [dip_mri] | derivatives/lossless/code/misc/standard_mri.mat | file path/name of the standard MRI file used in dipole fitting |
| [dip_elc] | derivatives/lossless/code/misc/standard_1020.elc | file path/name of the standard electrode file used in dipole fitting |
| [dip_transmat] | [0 0 0 0 0 0 1 1 1] | Transformation matrix to apply pre dipole fit. |
| [cor_transmat] | [0,0,0,0,0,1.57,1,1,1] | Transformation matrix to apply post dipole fit. |
| [cor_nosedir] | +X | Nose direction of montage to apply between dipole fitting to the standard head and performing ICLabel to the EEGLAB default orientation (e.g. +Y) |
| [out_path] | derivatives/lossless | Relative path to output data files assuming cd = work_path |