Multi-modal acquisition #171

lawhead · 2021-09-08T23:53:53Z

Overview

Re-architected the data acquisition module in support of multi-modal acquisition.

Ticket

https://www.pivotaltracker.com/story/show/178699590
https://www.pivotaltracker.com/story/show/178875719

Contributions

Created a new lightweight data acquisition client for reading from LSL data sources. Multiple clients can be created to read from different sources (ex. EEG and eye tracker).
Separated out the responsibilities for recording and querying data.
Deprecated MarkerWriter
Deprecated support for TCP connections.
Renamed supported_device to preconfigured_device.
Added a Clock abstraction, which by default uses the pylsl local_clock.
Refactored data queries to use timestamps rather than sample number.
Added code to demonstrate functionality and usage.

Test

Run all unit tests
Run Calibration task -> offline_analysis -> Copy Phrase task.
Confirm timing data.

…s for DeviceSpec sample_rate and support for an IRREGULAR_RATE for devices such as buttons and switches; renamed supported_devices to preconfigured_devices; demos and other prep work for multi-modal acquisition

…cumentation and type annotations

… streams; initial implementation of LSL-specific acquisition client

…ta_for_decision before refactoring

…er integration; context manager support; added the ability to name the raw data file. Simplified interface of marker writer to reflect current usage. Updated the acquisition helper to initialize the LslClient.

… that wraps lsl_local_clock. Updated all code to use the new internal clock.

…ing timestamps rather than sample numbers. Modified acquisition client to record only the configured data, rather than all detected LSL data. Synced the recorded data offset with the client.

tab-cmd

This is a nice change! I added some comments. Running locally, I see the TRG column is still present but has sensor values when it was 0.0 before? I see that TRG_device_stream used to have those values. Is this due to how we mock channels for serving?

tab-cmd · 2021-09-09T17:28:27Z

bcipy/acquisition/datastream/lsl_server.py


        if include_meta:
+            # TODO: different types of metadata depending on the content type


Let's make a ticket for this! I don't know how much we use the metadata, but it seems useful

https://www.pivotaltracker.com/story/show/179542651

tab-cmd · 2021-09-09T17:30:32Z

bcipy/acquisition/demo/demo_eye_tracker_server.py

+                      content_type='Gaze')
+
+
+def eye_tracker_data_generator(display_x=1920, display_y=1080):


We should move this somewhere... datastream/generator?

I added a new package for acquisition/datastream/mock and moved the eye_tracker_server.py and switch.py here.

tab-cmd · 2021-09-09T17:31:16Z

bcipy/acquisition/demo/demo_eeg_with_switch.py

+        log_to_stdout()
+
+    # Generic LSL device with 16 channels.
+    eeg_device = DeviceSpec(name="LSL_demo",


tab-cmd · 2021-09-09T17:32:05Z

bcipy/acquisition/demo/demo_lsl_server.py

+log = logging.getLogger(__name__)
+
+
+def custom_generator(device_spec: DeviceSpec, low=-1000, high=1000):


move to wherever the other eye tracker generator was moved!

Actually I think this one can be removed at this point.

tab-cmd · 2021-09-09T17:33:01Z

bcipy/acquisition/demo/demo_switch.py

+                        background_color='white',
+                        action=self.switch.click)
+
+    # @override


some cleanup on this file

tab-cmd · 2021-09-09T17:34:47Z

bcipy/acquisition/errors.py

@@ -0,0 +1,3 @@
+class InvalidClockError(Exception):


Other modules use the name exceptions.py. It may be good to change for consistency, but errors is descriptive enough

tab-cmd · 2021-09-09T17:40:51Z

bcipy/acquisition/protocols/lsl/lsl_recorder.py

+        self.max_chunk_size = 1024
+
+        # seconds to sleep between data pulls from LSL
+        self.sleep_seconds = 0.2


Do you think these values might change? chunk size and sleep seconds?

I created a ticket to explore this in more detail (see comment below). The max_chunk_size here is the default value that's used if you call pull_chunk without any parameters. The user can set this to a different value after the LslRecordingThread is instantiated (and before it starts recording). There is definitely a relationship between sleep seconds and max_chunk_size, though, and we may want to do some validation checks before setting.

tab-cmd · 2021-09-09T17:41:24Z

bcipy/acquisition/protocols/lsl/lsl_recorder.py

+        # and capture a timestamp, then use that timestamp to determine how
+        # long we slept since receiving the call and set the max_samples
+        # parameter accordingly.
+        # TODO: do we need to use max_chunk_size here


pull out to a spike?

https://www.pivotaltracker.com/story/show/179544118

tab-cmd · 2021-09-09T17:41:59Z

bcipy/acquisition/protocols/lsl/lsl_recorder.py

+        inlet.close_stream()
+        self._cleanup()
+
+# pylint: disable=import-outside-toplevel


This can be moved closer to the import

Thanks, good catch. Weird that this was here.

tab-cmd · 2021-09-09T17:45:49Z

bcipy/helpers/clock.py

+    - start_at_zero : If `True` the value returned by `getTime` is
+    relative to when the clock was started or reset. Default if `False`.
+    - get_time : optional function called to get the current time. The
+    default value is the LabStreamingLayer (LSL) local_clock.


We might indicate why it's the default...

lawhead added 19 commits February 3, 2021 17:45

Refactored DeviceSpec to take a data_type parameter; Added validation…

1795849

…s for DeviceSpec sample_rate and support for an IRREGULAR_RATE for devices such as buttons and switches; renamed supported_devices to preconfigured_devices; demos and other prep work for multi-modal acquisition

Added class method to connectors for connection_method; cleaned up do…

183d2ff

…cumentation and type annotations

Added a multi_modal_client and corresponding demo

8c54a12

Acquisition refactoring; added an LSL Recorder for recording all data…

c98c760

… streams; initial implementation of LSL-specific acquisition client

Add initial get_data query to lsl_client; simplify code in process_da…

745a490

…ta_for_decision before refactoring

Fixed type issues in lsl_client

cb5f1f9

Merged latest from 1.5.1

6f62170

Refactored lsl_recorder to use raw_data module; unit tests and cleanup

b1cb8a3

LslClient feature parity with current acquisition client; marker writ…

a1fa675

…er integration; context manager support; added the ability to name the raw data file. Simplified interface of marker writer to reflect current usage. Updated the acquisition helper to initialize the LslClient.

Work on offsets for lsl client

4550e72

Created a Clock object with the same interface as psychopy core.Clock…

c8f30a9

… that wraps lsl_local_clock. Updated all code to use the new internal clock.

Removed unused multi-modal approaches

17a1dec

Added unit tests for clock

672108c

Removed support for TCP connection in acquisition helper

1c69c4c

Removed marker writer from acquisition clients and display code

39016ae

Re-wrote process_data_for_decision to query the acquisition client us…

e637291

…ing timestamps rather than sample numbers. Modified acquisition client to record only the configured data, rather than all detected LSL data. Synced the recorded data offset with the client.

Merged latest from 1.5.1

ce78d66

Added better logging for acquisition; updated multi-modal demos

306fb66

Fixed unit tests; better documentation; fixed edge case for data queries

18ae6d5

lawhead requested a review from tab-cmd September 8, 2021 23:53

lawhead added 2 commits September 9, 2021 07:33

Fixed linting errors

31d7cf5

Address tests failing in CI

1403fe8

tab-cmd requested changes Sep 9, 2021

View reviewed changes

lawhead added 2 commits September 10, 2021 15:38

Refinements and cleanup

ef60f81

Adjusted test tolerance

6c29ff0

tab-cmd approved these changes Sep 15, 2021

View reviewed changes

lawhead merged commit 4b53064 into 1.5.1 Sep 16, 2021

tab-cmd deleted the multi-modal-acq branch January 24, 2022 19:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multi-modal acquisition #171

Multi-modal acquisition #171

lawhead commented Sep 8, 2021

tab-cmd left a comment

tab-cmd Sep 9, 2021

lawhead Sep 10, 2021

tab-cmd Sep 9, 2021

lawhead Sep 10, 2021

tab-cmd Sep 9, 2021

tab-cmd Sep 9, 2021

lawhead Sep 10, 2021

tab-cmd Sep 9, 2021

tab-cmd Sep 9, 2021

tab-cmd Sep 9, 2021

lawhead Sep 10, 2021

tab-cmd Sep 9, 2021

lawhead Sep 10, 2021

tab-cmd Sep 9, 2021

lawhead Sep 10, 2021

tab-cmd Sep 9, 2021


		if include_meta:
		# TODO: different types of metadata depending on the content type

		content_type='Gaze')


		def eye_tracker_data_generator(display_x=1920, display_y=1080):

		log = logging.getLogger(__name__)


		def custom_generator(device_spec: DeviceSpec, low=-1000, high=1000):

Multi-modal acquisition #171

Multi-modal acquisition #171

Conversation

lawhead commented Sep 8, 2021

Overview

Ticket

Contributions

Test

tab-cmd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment