Merge branch 'main' of https://github.com/datajoint/element-array-ephys

Author: CBroz1

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,39 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: 'bug'
+assignees: ''
+
+---
+
+## Bug Report
+
+### Description
+
+A clear and concise description of what is the overall operation that is intended to be
+performed that resulted in an error.
+
+### Reproducibility
+Include:
+- OS (WIN | MACOS | Linux)
+- DataJoint Element Version
+- MySQL Version
+- MySQL Deployment Strategy (local-native | local-docker | remote)
+- Minimum number of steps to reliably reproduce the issue
+- Complete error stack as a result of evaluating the above steps
+
+### Expected Behavior
+A clear and concise description of what you expected to happen.
+
+### Screenshots
+If applicable, add screenshots to help explain your problem.
+
+### Additional Research and Context
+Add any additional research or context that was conducted in creating this report.
+
+For example:
+- Related GitHub issues and PR's either within this repository or in other relevant
+  repositories.
+- Specific links to specific lines or a focus within source code.
+- Relevant summary of Maintainers development meetings, milestones, projects, etc.

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,5 @@
+blank_issues_enabled: false
+contact_links:
+  - name: DataJoint Contribution Guideline
+    url: https://docs.datajoint.org/python/community/02-Contribute.html
+    about: Please make sure to review the DataJoint Contribution Guidelines

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,57 @@
+---
+name: Feature request
+about: Suggest an idea for a new feature
+title: ''
+labels: 'enhancement'
+assignees: ''
+
+---
+
+## Feature Request
+
+### Problem
+
+A clear and concise description how this idea has manifested and the context. Elaborate
+on the need for this feature and/or what could be improved. Ex. I'm always frustrated
+when [...]
+
+### Requirements
+
+A clear and concise description of the requirements to satisfy the new feature. Detail
+what you expect from a successful implementation of the feature. Ex. When using this
+feature, it should [...]
+
+### Justification
+
+Provide the key benefits in making this a supported feature. Ex. Adding support for this
+feature would ensure [...]
+
+### Alternative Considerations
+
+Do you currently have a work-around for this? Provide any alternative solutions or
+features you've considered.
+
+### Related Errors
+Add any errors as a direct result of not exposing this feature.
+
+Please include steps to reproduce provided errors as follows:
+- OS (WIN | MACOS | Linux)
+- DataJoint Element Version
+- MySQL Version
+- MySQL Deployment Strategy (local-native | local-docker | remote)
+- Minimum number of steps to reliably reproduce the issue
+- Complete error stack as a result of evaluating the above steps
+
+### Screenshots
+If applicable, add screenshots to help explain your feature.
+
+### Additional Research and Context
+Add any additional research or context that was conducted in creating this feature request.
+
+For example:
+- Related GitHub issues and PR's either within this repository or in other relevant
+  repositories.
+- Specific links to specific lines or a focus within source code.
+- Relevant summary of Maintainers development meetings, milestones, projects, etc.
+- Any additional supplemental web references or links that would further justify this
+  feature request.

.github/workflows/u24_element_release_call.yml

@@ -17,6 +17,7 @@ jobs:
     secrets:
       TWINE_USERNAME: ${{secrets.TWINE_TEST_USERNAME}}
       TWINE_PASSWORD: ${{secrets.TWINE_TEST_PASSWORD}}
+      GOOGLE_ANALYTICS_KEY: ${{secrets.GOOGLE_ANALYTICS_KEY}}
   call_u24_elements_release_alpine:
     if: >-
       github.event.workflow_run.conclusion == 'success' && github.repository_owner == 'datajoint' && !contains(github.event.workflow_run.head_branch, 'test')
@@ -26,3 +27,5 @@ jobs:
     secrets:
       TWINE_USERNAME: ${{secrets.TWINE_USERNAME}}
       TWINE_PASSWORD: ${{secrets.TWINE_PASSWORD}}
+      GOOGLE_ANALYTICS_KEY: ${{secrets.GOOGLE_ANALYTICS_KEY}}
+

.gitignore

@@ -118,4 +118,11 @@ dj_local_conf_old.json
 docker-compose.y*ml
 
 # include
-!docs/docker-compose.yaml
+!docs/docker-compose.yaml
+
+# vscode settings
+.vscode
+*.code-workspace
+
+# exports/notes
+temp*

.pre-commit-config.yaml

@@ -0,0 +1,56 @@
+default_stages: [commit, push]
+exclude: (^.github/|^docs/|^images/)
+
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.4.0
+    hooks:
+      - id: trailing-whitespace
+      - id: end-of-file-fixer
+      - id: check-yaml
+      - id: check-added-large-files # prevent giant files from being committed
+      - id: requirements-txt-fixer
+      - id: mixed-line-ending
+        args: ["--fix=lf"]
+        description: Forces to replace line ending by the UNIX 'lf' character.
+
+  # black
+  - repo: https://github.com/psf/black
+    rev: 22.12.0
+    hooks:
+      - id: black
+      - id: black-jupyter
+        args:
+          - --line-length=88
+
+  # isort
+  - repo: https://github.com/pycqa/isort
+    rev: 5.11.2
+    hooks:
+      - id: isort
+        args: ["--profile", "black"]
+        description: Sorts imports in an alphabetical order
+
+  # flake8
+  - repo: https://github.com/pycqa/flake8
+    rev: 4.0.1
+    hooks:
+      - id: flake8
+        args: # arguments to configure flake8
+          # making isort line length compatible with black
+          - "--max-line-length=88"
+          - "--max-complexity=18"
+          - "--select=B,C,E,F,W,T4,B9"
+
+          # these are errors that will be ignored by flake8
+          # https://www.flake8rules.com/rules/{code}.html
+          - "--ignore=E203,E501,W503,W605"
+          # E203 - Colons should not have any space before them.
+          #        Needed for list indexing
+          # E501 - Line lengths are recommended to be no greater than 79 characters.
+          #        Needed as we conform to 88
+          # W503 - Line breaks should occur after the binary operator.
+          #        Needed because not compatible with black
+          # W605 - a backslash-character pair that is not a valid escape sequence now
+          #        generates a DeprecationWarning. This will eventually become a SyntaxError.
+          #        Needed because we use \d as an escape sequence

CHANGELOG.md

@@ -3,8 +3,25 @@
 Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and
  [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) convention.
 
-## [0.1.5] - 2022-07-11
-
+## [0.2.1] - 2022-01-06
+
++ Add - `build_electrode_layouts` function in `probe.py` to compute the electrode layout for all types of probes.
++ Update - parameterize run_CatGT step from parameters retrieved from `ClusteringParamSet` table
++ Update - clustering step, update duration for "median_subtraction" step
++ Bugfix - handles single probe recording in "Neuropix-PXI" format
++ Update - safeguard in creating/inserting probe types upon probe activation
++ Add - quality control metric dashboard
++ Update & fix docstrings
++ Update - `ephys_report.UnitLevelReport` to add `ephys.ClusterQualityLabel` as a foreign key reference
++ Add - `.pre-commit-config.yaml`
+
+## [0.2.0] - 2022-10-28
+
++ Add - New schema `ephys_report` to compute and store figures from results
++ Add - Widget to display figures
++ Add - Add `ephys_no_curation` and routines to trigger spike-sorting analysis
+  using Kilosort (2.0, 2.5)
++ Add - mkdocs for Element Documentation
 + Add - New `QualityMetrics` table to store clusters' and waveforms' metrics after the spike sorting analysis.
 
 ## [0.1.4] - 2022-07-11
@@ -27,11 +44,16 @@ Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and
 
 + Update - Rename module for acute probe insertions from `ephys.py` to `ephys_acute.py`.
 + Add - Module for pre-clustering steps (`ephys_precluster.py`), which is built off of `ephys_acute.py`.
-+ Add - Module for chronic probe insertions (`ephys_chronic.py`).  
++ Add - Module for chronic probe insertions (`ephys_chronic.py`).
 + Bugfix - Missing `fileTimeSecs` key in SpikeGLX meta file.
 + Update - Move common functions to `element-interface` package.
 + Add - NWB export function
 
+## [0.1.0b4] - 2021-11-29
+
++ Add - Processing with Kilosort and pyKilosort for Open Ephys and SpikeGLX
+
+
 ## [0.1.0b0] - 2021-05-07
 
 + Update - First beta release
@@ -41,12 +63,16 @@ Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and
 + Add - GitHub Action release process
 + Add - `probe` and `ephys` elements
 + Add - Readers for: `SpikeGLX`, `Open Ephys`, `Kilosort`
-+ Add - Probe table supporting: Neuropixels probes 1.0 - 3A, 1.0 - 3B, 2.0 - SS, 2.0 - MS
-
-[0.1.4]: https://github.com/datajoint/element-array-ephys/compare/0.1.3...0.1.4
-[0.1.3]: https://github.com/datajoint/element-array-ephys/compare/0.1.2...0.1.3
-[0.1.2]: https://github.com/datajoint/element-array-ephys/compare/0.1.1...0.1.2
-[0.1.1]: https://github.com/datajoint/element-array-ephys/compare/0.1.0...0.1.1
-[0.1.0]: https://github.com/datajoint/element-array-ephys/compare/0.1.0b0...0.1.0
-[0.1.0b0]: https://github.com/datajoint/element-array-ephys/compare/0.1.0a5...0.1.0b0
++ Add - Probe table supporting: Neuropixels probes 1.0 - 3A, 1.0 - 3B, 2.0 - SS,
+  2.0 - MS
+
+[0.2.1]: https://github.com/datajoint/element-array-ephys/releases/tag/0.2.1
+[0.2.0]: https://github.com/datajoint/element-array-ephys/releases/tag/0.2.0
+[0.1.4]: https://github.com/datajoint/element-array-ephys/releases/tag/0.1.4
+[0.1.3]: https://github.com/datajoint/element-array-ephys/releases/tag/0.1.3
+[0.1.2]: https://github.com/datajoint/element-array-ephys/releases/tag/0.1.2
+[0.1.1]: https://github.com/datajoint/element-array-ephys/releases/tag/0.1.1
+[0.1.0]: https://github.com/datajoint/element-array-ephys/releases/tag/0.1.0
+[0.1.0b4]: https://github.com/datajoint/element-array-ephys/releases/tag/0.1.0b4
+[0.1.0b0]: https://github.com/datajoint/element-array-ephys/releases/tag/0.1.0b0
 [0.1.0a5]: https://github.com/datajoint/element-array-ephys/releases/tag/0.1.0a5

CODE_OF_CONDUCT.md

@@ -0,0 +1,133 @@
+
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, caste, color, religion, or sexual
+identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the overall
+  community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or advances of
+  any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email address,
+  without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+[Support@DataJoint.com](mailto:support@datajoint.com).
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series of
+actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or permanent
+ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior, harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within the
+community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.1, available at
+[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
+
+Community Impact Guidelines were inspired by
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
+[https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations