[Feature] multiple methods of outlier identification

[danibene]

Description

Allow for different methods of outlier identification based on standardization or percentiles.

Proposed Changes

• Add new outlier identification method based on percentiles: _find_outliers_percentile(), defaults to identifying observations below Q1 - 1.5 * IQR or above Q3 + 1.5 * IQR as outliers, e.g. as in this paper
• Add argument method in find_outliers() calling _find_outliers_standardize() or _find_outliers_percentile()
• Pass keyword arguments to standardize() e.g. robust=True

Checklist

• I have read the CONTRIBUTING file.
• My PR is targetted at the dev branch (and not towards the master branch).
• I ran the CODE CHECKS on the files I added or modified and fixed the errors.
  • Just in case someone else runs into this: I had an import error with black at first, but it’s all good now
  • Also I decided to ignore the following pylint output since it concerned a line that I did not add (from ..stats import standardize)

    ************* Module find_outliers
    find_outliers.py:4:0: E0402: Attempted relative import beyond top-level package (relative-beyond-top-level)
    

• I have added the newly added features to News.rst (if applicable)
  • Though I am confused about which version the changes should correspond to: for the previous PR, I listed the features under 0.1.6 in the News file, but I saw that they were part of release 0.2.0. Does that mean I should move the previous changes to 0.2.0 and this PR to 0.2.1?

[codeclimate]

Code Climate has analyzed commit d79318f and detected 0 issues on this pull request.

View more on Code Climate.

[codecov-commenter]

Codecov Report

Merging #647 (3074bca) into dev (b123c7b) will decrease coverage by 0.08%.
The diff coverage is 3.44%.

@@            Coverage Diff             @@
##              dev     #647      +/-   ##
==========================================
- Coverage   53.64%   53.55%   -0.09%     
==========================================
  Files         269      269              
  Lines       12010    12032      +22     
==========================================
+ Hits         6443     6444       +1     
- Misses       5567     5588      +21     

Impacted Files	Coverage Δ
neurokit2/misc/find_outliers.py	11.42% <3.44%> (-19.35%)	⬇️
neurokit2/eda/eda_eventrelated.py	100.00% <0.00%> (+2.04%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b123c7b...3074bca. Read the comment docs.

[DominiqueMakowski]

Thanks @danibene, sorry I was on holidays, I will review this and merge asap :)

[DominiqueMakowski]

@danibene I checked and somewhat simplified the function, as I was not sure about all the various arguments (such as the range multiplier and the difference between the percentile range and threshold).

I tried to unify the arguments so that everything is controlled using exclude, and added examples. Can you check and let me know if there is any particular behaviour that is impossible to achieve using the current implementation? thanks

[danibene]

Thanks @DominiqueMakowski ! It looks a lot cleaner now :-)

One small comment about lines 92-93, since exclude is not an array/list/tuple, there should be no indexing right?

            right = np.percentile(data, (1 - (exclude[1] / 2)) * 100)
            left = np.percentile(data, (exclude[0] / 2) * 100)

Without the extra arguments (e.g. the range multiplier), we can only have the threshold based directly on the percentile rather than, for example, 1.5*interquartile range + the 75th percentile (as in the "quartiles" method in MATLAB's isoutlier).

[DominiqueMakowski]

there should be no indexing right?

good catch!

rather than, for example, 1.5*interquartile range + the 75th percentile (as in the "quartiles" method in MATLAB's isoutlier).

mmh I'd say for now it's better to keep the function fairly generic; this method seems to be somewhat very matlab-esque and oddly specific. If users want to mimic this behaviour they can probably do it on their side, as it's not too hard to do.

If that's okay with you I'll merge :)

[danibene]

this method seems to be somewhat very matlab-esque and oddly specific. If users want to mimic this behaviour they can probably do it on their side, as it's not too hard to do.

I have seen the 1.5*IQR rule outside of MATLAB (e.g. https://online.stat.psu.edu/stat200/lesson/3/3.2), but keeping things simple for now makes sense to me.

If that's okay with you I'll merge :)

Yes please do! Thank you for your help with this :-)

[DominiqueMakowski]

I have seen the 1.5*IQR rule outside of MATLAB

Thanks, I didn't know that. Well if in the future we see that there's demand for this method we can always think about adding it in :)

Merging now, thanks again!