DLC notebooks 21 and 22 #772
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
edeno
reviewed
Jan 17, 2024
There was a problem hiding this comment.
Hi @emreybroyles , thanks for all your work on this. The numbering of the notebooks has recently changed. Can you change it from 05 and 06 to 21 and 22 respectively?
I'm not sure how the datajoint-python submodule was added in here but that should probably be removed as well as the change to gitignore.
Were there any changes to the DLC code that you had to make or was it only the notebooks that changed?
edeno
reviewed
Jan 18, 2024
|
Thanks for putting these together @emreybroyles - Can you confirm that you used the |
4 tasks
CBroz1
approved these changes
Jan 19, 2024
There was a problem hiding this comment.
Thanks @emreybroyles! Looks good
Out of caution, I generally comment out any lines that run deletes in case someone hits 'run all' (would hate for someone to delete your data!), but I can patch that on my end
edeno
added a commit
that referenced
this pull request
Jan 20, 2024
* Docs fixes: tidy decoding docstrings, change hatch version get, add inits * Pytest revamp (#743) * WIP: Pull from old stash, resolve conflicts * Pytest WIP. Position centriod fix. Centralize device prompt logic * Add tests for all tables in * WIP: Improve coverage behav, dio * WIP: Add coverage, see details: - Add `return_fig` param to plotting helper functions to permit tests - `common_filter` - `common_interval` - Add coverage for ~1/2 of `common` - `common_behav` - `common_device` - `common_ephys` - `common_filter` - `common_interval` - with helper funcs tested seperately - `common_lab` - `common_nwbfile` - partial * WIP pytest common 2nd half, start lfp * WIP lfp tests, ahead of fetch upstream * Add lfp pipeline tests * Run pre-commit checks * Fix bug * Unpin position_tools for CI * Change download data dir * Change download data dir 2 * Fix teardown. Coverage 67% * Update changelog * logger.warn -> logger.warning * Minor decoding fixes (#769) * Add non-local detector and remove replay_trajectory_classification * Reorganize * Fix formatting and imports * Update .gitignore * Remove because of circular import * Fix name of parameter * Handle case where ther is only one interval * Fix settings * Handle single interval * from_unit_dict does not exist in 0.98.2 of spike interface * Simplify call * Update for SpikeSorting merge table and add spyglass mixin * Fix dependencies * Fix merge conflict * Update src/spyglass/decoding/v1/clusterless.py Co-authored-by: Chris Brozdowski <[email protected]> * Update src/spyglass/decoding/v1/clusterless.py Co-authored-by: Chris Brozdowski <[email protected]> * Update src/spyglass/decoding/v1/clusterless.py Co-authored-by: Chris Brozdowski <[email protected]> * Update src/spyglass/decoding/v1/clusterless.py Co-authored-by: Chris Brozdowski <[email protected]> * Apply suggestions from code review Co-authored-by: Chris Brozdowski <[email protected]> * Remove unused imports and format * Add saving of waveform features * Don't store electrodes, full waveforms, waveform mean * Fix spike times and add convenience method * Add spike location and some formatting * Remove circular import * Fix dict expansion * Initial working clusterless pipeline * Add position group * Rename classifier to decoding * Handle encoding and decoding intervals * Put old files under v0, try/except for old decoding package * Rename visualization and remove from v0 v0 visualization is redundant with visualization * Place parameters and position group in core.py * Add sorted spikes decoding * Add objects to init for convenience * Remove unused imports * Fix fetching of spike times * Insert into merge table * Update CHANGELOG.md * Function for removing decoding outputs not in DecodingOutput * Fix name * Add draft of tutorials and rearrange notebooks * Fix config loading * Add 1D decoding and some notes on estimate_parameters kwarg * Update 43_Decoding_SortedSpikes.ipynb * Remove old decoding notebook * Save initial conditions and discrete transitions * Apply suggestions from code review Co-authored-by: Chris Brozdowski <[email protected]> * Be more specific with import error * Remove unneeded comments * Remove incorrect dimension name * Project merge_id from SpikeSortingOutput for clarity * Update src/spyglass/decoding/v0/clusterless.py Co-authored-by: Chris Brozdowski <[email protected]> * Update src/spyglass/decoding/v0/clusterless.py Co-authored-by: Chris Brozdowski <[email protected]> * Update src/spyglass/decoding/v0/clusterless.py Co-authored-by: Chris Brozdowski <[email protected]> * Fix linting * Update notebooks * Ignore .pem * Add session as a primary key for Groups * Add some helper methods * Update notebooks * Update README.md * Update pyscripts * Update 42_Decoding_Clusterless.ipynb * Update CHANGELOG.md * Add fetch and insert * Simplify class conversion * Do the dictionary conversion of class for the user * Update CHANGELOG.md * Update .gitignore * Use methods in populate * Avoid fetching interval range if not needed * Generalize finding class from modules * Use args/kwargs * Simplify tuple unpacking * Make decoding kwargs nullable * Add function for get_recording and get_sorting to the spikesorting merge table * make decoding waveform features agnostic to spikesorting source * Fix spelling * Use fetch1_dataframe for position * Use self instead of class * Update src/spyglass/decoding/v1/sorted_spikes.py Co-authored-by: Samuel Bray <[email protected]> * Be more careful about populating select keys * Make more readable/remove unused imports * Save classifier * Clean up saved model paths * add function load_linear_position_info * Update src/spyglass/decoding/v1/sorted_spikes.py Co-authored-by: Samuel Bray <[email protected]> * Update 41_Extracting_Clusterless_Waveform_Features.py * Update docstring * Apply suggestions from code review Co-authored-by: Chris Brozdowski <[email protected]> * Update src/spyglass/decoding/v1/clusterless.py Co-authored-by: Chris Brozdowski <[email protected]> * Update src/spyglass/decoding/v1/clusterless.py Co-authored-by: Chris Brozdowski <[email protected]> * Fix linting * Fix syntax * Rename variable to avoid confusion * Restrict UnitWaveformFeaturesGroup and SortedSpikesGroup * Concatenate linear position and position dataframes * Static methods don't require instantiating class * Avoid merge restrict * Add version to defaults * Remove unused import * Fix classifier path * Add dry run * Remove non-default * Handle permissions and file not found * Keep position info within encoding/decoding interval * Add methods to get the spike_times, spike_indicators, firing rate * Fix docstring to match default * Implement function rather than import * Remove unused broken imports * Add decoding cleanup * Fix import * Put old vis code back * Fix import * Add draft helper functions * Limit options on input * Fix logic * Fix where the key is passed * Update notebooks * Host main visualizations in non_local_detector repo * Update notebooks/py_scripts/41_Extracting_Clusterless_Waveform_Features.py Co-authored-by: Chris Brozdowski <[email protected]> * Update src/spyglass/spikesorting/merge.py Co-authored-by: Chris Brozdowski <[email protected]> * Update src/spyglass/decoding/decoding_merge.py Co-authored-by: Chris Brozdowski <[email protected]> * Revert "Limit options on input" This reverts commit 386714c. * Use f-string for version * Add useful imports to the top level This would have to change a bit if there were multiple versions of the pipeline. * Make source class a hidden attribute * Update CHANGELOG.md --------- Co-authored-by: Chris Brozdowski <[email protected]> Co-authored-by: Sam Bray <[email protected]> * DLC notebooks 21 and 22 (#772) * add envs to bashrc; multi cam addition * 12/11/23 using TackEpoch to define interval_names * 12/11/23 remove comment * del smooth duration * jan 10 again * DLC noteboks 5 and 6 * 5 and 6 * fix ignore and 21 * Removed submodule * removed .gitignore * DLC notebooks * Docs fixes: tidy decoding docstrings, change hatch version get, add inits * Jupysync, blackify, purge old .py nbs * Docs updates for new notebooks * Update changelog * Spell fix, pre-commit --------- Co-authored-by: Eric Denovellis <[email protected]> Co-authored-by: Sam Bray <[email protected]> Co-authored-by: emreybroyles <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR adds DLC pipeline notebooks 21 and 22 to be more accessible and unambiguous for everyone. The new notebook 21 is essentially the previous notebooks 21-23 together in one notebook, with extra explanatory parts for user ease, where pose estimation tables are populated one-by-one, for one epoch at a time. The new 22 is the same as 21, except it can populate pose estimation tables in a loop, for multiple epochs over a day, given the camera and nwb name of choice.
notebooks/21_DLC.ipynb: updated DLC notebooknotebooks/22_DLC_Loop.ipynb: updated DLC Loop notebookChecklist:
CITATION.cffCHANGELOG.md