Skip to content

Commit f07d131

Browse files
authored
Merge pull request #13 from ttngu207/main
Mostly code cleanup - formatting
2 parents 983d61a + ea1547f commit f07d131

File tree

13 files changed

+307
-140
lines changed

13 files changed

+307
-140
lines changed

Background.md

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# Array Electrophysiology Element
2+
## Description of modality, user population
3+
4+
Neuropixels probes were developed by a collaboration between HHMI Janelia, industry partners, and others (Jun et al., Nature 2017).
5+
Since its initial release in October 2018, 300 labs have ordered 1200 probes.
6+
Since the rollout of Neuropixels 2.0 in October 2020, IMEC has been shipping 100+ probes monthly (correspondence with Tim Harris).
7+
8+
Neuropixels probes offer 960 electrode sites along a 10mm long shank,
9+
with 384 recordable channels per probe that can record hundreds of units spanning multiple brain regions
10+
(Neuropixels 2.0 version is a 4-shank probe with 1280 electrode sites per shank).
11+
Such large recording capacity have offered tremendous opportunities for the field of neurophysiology research,
12+
yet this is accompanied by an equally great challenge in terms of data and computation management.
13+
14+
## Acquisition tools
15+
The typical instrumentation used for data acquisition is the Neuropixels probe and headstage interfacing with a PXIe acquisition module (https://www.neuropixels.org/control-system).
16+
Two main acquisition softwares are used for Neuropixels:
17+
+ SpikeGLX - developed by Bill Karsh and Tim Harris at HHMI/Janelia
18+
+ OpenEphys - developed by Joshua Siegle at the Allen Institute.
19+
20+
These save the data into specific directory structure and file-naming convention as custom binary formats (e.g. “.bin”, “.dat”). Meta data are stored as separate files in xml or text format.
21+
22+
## Preprocessing tools
23+
The preprocessing pipeline includes bandpass filtering for LFP extraction, bandpass filtering for spike sorting, spike sorting, and manual curation of the spike sorting results, and calculation of quality control metrics. In trial-based experiments, the spike trains are aligned and separated into trials. Standard processing may include PSTH computation aligned to trial onset or other events, and often grouped by different trial types.
24+
Neuroscience groups have traditionally developed custom home-made toolchains.
25+
26+
In recent years, several leaders have been emerging as de facto standards with significant community uptake:
27+
+ Kilosort
28+
+ JRClust
29+
+ MountainSort
30+
+ SpyKING CIRCUS
31+
32+
Kilosort provides most automation and has gained significant popularity, being adopted as one of the key spike sorting methods in the majority of the teams/collaborations we have worked with. As part of Year-1 U24 effort, we provide support for data ingestion of spike sorting results from Kilosort.
33+
Further effort will be devoted for the ingestion support of other spike sorting methods.
34+
On this end, a framework for unifying existing spike sorting methods,
35+
named [SpikeInterface](https://github.com/SpikeInterface/spikeinterface),
36+
has been developed by Alessio Buccino, et al.
37+
SpikeInterface provides a convenient Python-based wrapper to invoke, extract, compare spike sorting results from different sorting algorithms.
38+
39+
## Precursor projects and interviews
40+
41+
Over the past few years, several labs have developed DataJoint-based data management and processing pipelines for Neuropixels probes.
42+
Our team collaborated with several of them during their projects.
43+
Additionally, we interviewed these teams to understand their experiment workflow, pipeline design, associated tools, and interfaces.
44+
These teams include:
45+
46+
+ International Brain Lab - https://internationalbrainlab.org - https://github.com/int-brain-lab/IBL-pipeline
47+
+ Mesoscale Activity Project (HHMI Janelia) - https://github.com/mesoscale-activity-map - https://github.com/mesoscale-activity-map/map-ephys
48+
+ Moser Group (private repository)
49+
50+
## Pipeline Development
51+
Through our interviews and direct collaboration on the precursor projects,
52+
we identified the common motifs to create the Array Electrophysiology Element with the repository hosted at https://github.com/datajoint/element-array-ephys.
53+
54+
Major features of the Array Electrophysiology Element include:
55+
+ Pipeline architecture defining:
56+
+ Probe, electrode configuration compatible with Neuropixels probes and generalizable to other types of probes (e.g. tetrodes) - supporting both chronic and acute probe insertion mode
57+
+ Probe-insertion, ephys-recordings, LFP extraction, clusterings, curations, sorted units and the associated data (e.g. spikes, waveforms, etc.)
58+
+ Store/track/manage different curations of the spike sorting results
59+
+ Ingestion support for data acquired with SpikeGLX and OpenEphys acquisition systems
60+
+ Ingestion support for spike sorting outputs from Kilosort
61+
+ Sample data and complete test suite for quality assurance
62+
63+
Incorporation of SpikeInterface into the Array Electrophysiology Element will be on DataJoint Elements development roadmap.
64+
Dr. Loren Frank has led a development effort of a DataJoint pipeline with SpikeInterface framework and NeurodataWithoutBorders format integrated (https://github.com/LorenFrankLab/nwb_datajoint).
65+
66+
## Alpha release: Validation sites
67+
We engaged several labs to adopt and validate the Array Electrophysiology Element during the alpha testing phase. These include:
68+
+ Andreas Tolias Lab (BCM)
69+
+ BrainCoGs (Princeton Neuroscience Institute)
70+
+ Brody Lab (Princeton)
71+
72+
## Beta release
73+
As the validation progresses, we expect to produce a beta version of the workflow for users to adopt independently by May 1, 2021.

CONTRIBUTING.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Contribution Guidelines
2+
3+
This project follows the [DataJoint Contribution Guidelines](https://docs.datajoint.io/python/community/02-Contribute.html). Please reference the link for more full details.

LICENSE

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
MIT License
22

3-
Copyright (c) 2020 DataJoint NEURO
3+
Copyright (c) 2021 DataJoint NEURO
44

55
Permission is hereby granted, free of charge, to any person obtaining a copy
66
of this software and associated documentation files (the "Software"), to deal

README.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
1-
# DataJoint Element - Canonical Electrophysiology
2-
DataJoint Element for electrophysiology.
3-
This repository features DataJoint pipeline design for extracellular electrophysiology,
1+
# DataJoint Element - Array Electrophysiology Element
2+
DataJoint Element for array electrophysiology.
3+
4+
This repository features DataJoint pipeline design for extracellular array electrophysiology,
45
with ***Neuropixels*** probe and ***kilosort*** spike sorting method.
56

67
The pipeline presented here is not a complete pipeline by itself, but rather a modular
@@ -10,11 +11,13 @@ This modular pipeline element can be flexibly attached downstream
1011
to any particular design of experiment session, thus assembling a fully functional
1112
ephys pipeline.
1213

14+
See [Background](Background.md) for the background information and development timeline.
15+
1316
## The Pipeline Architecture
1417

1518
![ephys pipeline diagram](images/attached_ephys_element.svg)
1619

17-
As the diagram depicts, the ephys element starts immediately downstream from ***Session***,
20+
As the diagram depicts, the array ephys element starts immediately downstream from ***Session***,
1821
and also requires some notion of ***Location*** as a dependency for ***InsertionLocation***.
1922

2023
### The design of probe
@@ -52,4 +55,4 @@ This ephys element features automatic ingestion for spike sorting results from t
5255

5356
## Usage
5457

55-
See [this project](https://github.com/vathes/canonical-full-ephys-pipeline) for an example usage of this ephys pipeline element.
58+
See [this project](https://github.com/datajoint/workflow-array-ephys) for an example usage of this Array Electrophysiology Element.
File renamed without changes.

0 commit comments

Comments
 (0)