Release notes

Spyking CIRCUS 0.8

This is the 0.8 release of the SpyKING CIRCUS, a new approach to the problem of spike sorting. The code is based on a smart clustering with sub sampling, and a greedy template matching approach, such that it can resolve the problem of overlapping spikes. The publication about the software is available at


The software can be used with command line, or a dedicated GUI


The code may still evolve. Even if results are or should be correct, we can expect some more optimizations in a near future, based on feedbacks obtained on multiple datasets. If you spot some problems with the results, please be in touch with


Code and documentation contributions (ordered by the number of commits):

  • Pierre Yger
  • Marcel Stimberg
  • Baptiste Lebfevre
  • Christophe Gardella
  • Olivier Marre
  • Cyrille Rossant
  • Joze Guzman
  • Ariel Burman
  • Ben Acland

Release 1.0.2

  • possibility for the amplitudes [a_min, a_max] to depend on time
  • median is now removed per shanks
  • common ground syntax slightly changed, to allow one ground per shank
  • fix a bug when collect_all and dense templates
  • fix if no templates are found
  • improvements of the smart search
  • add the option to collect normalized MSE during fitting. False by default
  • fix the rhd wrapper
  • divide and conquer assigment now based on barycenters instead of simple extremas
  • exit the clustering if too many centroids are found (sign of bad channels)
  • fixes in the meta merging GUI (RPV and dip)
  • optimizations for the second component, less double counting
  • fix to use at least 1 CPU
  • better estimation of amplitudes for axonal spikes
  • enhance the estimation of amplitudes by proper alignement

Release 1.0.0

  • prevent the use of negative indices for channels
  • fix if no templates are found
  • fix if the dead file is empty
  • fix for recent versions of MPI (if dead times used)
  • fix if dead times are not sorted or overlapping
  • add the auto_cluster param in [data] to force gobal_tmp if needed
  • fix when no cluster are found on some electrodes
  • fix in the MATLAB Gui if no spikes are found
  • support for the maxwell file format (MaxOne and MaxTwo)
  • optimizations for faster fitting
  • templates are densified dring fitting if not enough sparse (faster)

Release 0.9.9

  • fix for shanks (because of optimization in 0.9.8)
  • fix for clusters (if global tmp is not created)
  • fix for recent versions of MPI (shared memory issues)
  • still speeding up the fitting procedure, as a final bottleneck
  • fix in the smart search and chunks exploration

Release 0.9.8

  • fix a bug while filtering HDF5 file with overwrite set to False
  • fix a bug for windows and Intel MPI
  • speeding up the fitting procedure
  • reducing the memory footprint while optimizing amplitudes for large number of templates
  • changing the way of saving overlaps, making use of internal symmetry. Lot of memory saved

Release 0.9.7

  • fix a bug in the preview mode
  • fix a bug while converting with export_all set to True
  • fix a rare bug when both peaks are detect in clustering with smart search
  • fix a bug if removing reference channel after filtering has been already done
  • fix a bug while converting with export_all
  • fix a bug in the filtering introduced in 0.9.6 (last chunk not filtered)
  • fix a possible bug in smart search with dynamic bins
  • enhance the robustness of the whitening for very large arrays
  • speeding up the fitting procedure
  • enhancing the non-selection of noisy snippets, and thus clustering
  • option to dynamically adapt cc_merge for large number of electrodes
  • remove putative mixtures based on variance, speeding up drastically CC estimation

Release 0.9.6

  • fixes in the smart search (not all rare cases were covered in 0.9.5)
  • fix a bug if multi file is activated with very small chunks
  • speeding up the estimation of the templates: less snippets, closer to centroids
  • speeding up the estimation of the amplitudes: less noise snippets
  • speeding up isolation step during the smart search
  • number of bins is adapted during the smart search as function of noise levels
  • add the possibility to hide the status bars (for SpikeInterface logs)

Release 0.9.5

  • speeding up the optimization of the amplitudes with MPI
  • speeding up the processing of numpy datafiles (SpikeInterface)
  • speeding up the smart search step (pre-generation of random numbers)
  • speeding up the clustering step
  • fix a bug while filtering in the preview mode introduced in 0.9.2
  • speeding up the fitting step

Release 0.9.4

  • speeding up the optimization of the amplitudes with MPI
  • speeding up the processing of numpy datafiles (SpikeInterface)
  • speeding up the smart search step (pre-generation of random numbers)

Release 0.9.2

  • speeding up the algorithm
  • fixing a bug in the clustering while assigining labels
  • better detection of noise snippets discarded during clustering
  • cosmetic changes in the sanity plots (clusters)
  • better handling of overlapping chunks while filtering, removing filtering artefacts
  • templates are restricted within shanks
  • optimization of the amplitudes once all templates have been found
  • export of a purity value, for phy, to assess how good a cluster is (between 0 and 1)
  • display the purity value in MATLAB
  • fix a (silent) bug in the supports introduced in 0.9.0, preventing mixture removal
  • nb_chances is automatically adapted during the fitting procedure
  • drifts are now automatically handled by the meta merging procedure
  • enhancement in the automatic merging of drifts

Release 0.9.1

  • Minor bug fixes in spyking-circus-launcher
  • fix a bug in the amplitude display. Values were shuffled when several CPU were used
  • add the option to ignore second component [clustering]->two_components

Release 0.9.0

  • can now fit spikes below detection threshold (with spike_thresh_min)
  • templates are now estimated without any spatial restrictions
  • display a warning if N_t is not optimally chosen

Release 0.8.9

  • fix a small bug in the smart search, introduced while refactoring in 0.8.7

Release 0.8.8

  • fix a regression introduced in 0.8.7 for non contiguous probe indices

Release 0.8.7

  • new methods to detect the peaks, more robust when low thresholds are fixed
  • more accurate fitting procedure, slightly slower
  • minor bug fixes
  • addition of a sparsity_limit parameter in the meta merging GUI, to remove noise more precisely
  • new parameter file is properly copied
  • enhancement of the smoothing/alignement procedure, more accurate estimation of noisy templates
  • better estimation of the amplitudes boundaries used during fitting
  • optimization while removing mixtures and important bug fixes
  • fix a bug in the thresholding method
  • minor updates to get more refined spikes during whitening and clustering
  • tests with SpikeInterface, showing clear increase in performance
  • some cleaning in the parameter file
  • default value for cc_merge is now 0.95, since merging functions are more robust
  • noisy templates are removed by default while meta merging with a lower threshold (0.75)
  • speeding up whitening and clustering steps

Release 0.8.6

  • Export from manual sorting with MATLAB to phy is now possible
  • Modification to pass SpikeSorters test suite

Release 0.8.5

  • fix a bug while removing noisy templates in meta merging
  • refactoring of the meta merging GUI, addition of bhatta distances
  • meta merging more robust for non stationary recordings
  • enhance logging if parameters are missing and/or not defined
  • can now display the electrode labels in preview GUI
  • detects if a wrong install of MPI is present (linking with mpi4py)
  • conda install overwrites the old parameter file
  • raw dispay of the MUA in the result GUI (to be improved)
  • display an error if not all nodes on a cluster can read the datafiles
  • fix a bug for thresholding method using dead times

Release 0.8.4

  • fix if no spikes are found on some electrodes
  • fix as mean/median-pca methods were broken (albeit not used)
  • fix to prevent a rare crash while loading too sparse overlaps
  • fix a bug with the new dip method in python 2.7
  • add the thresholding method to extract only MUA activity (requested by users)
  • channel lists in probe files can be non sorted
  • memory usage is dynamically adapted to reduce memory footprint
  • hdf5 and npy file format can now work with 3D arrays (x, y, time) or (time, x, y)
  • fix a bug if basis for pos and neg spikes have different sizes
  • add some docstrings (thanks to Jose Guzman)
  • sparse export for phy is now the default
  • comments can now be added in the trigger/dead times files
  • 4096 channels can now run on a single machine, with low memory consumption
  • basic support for 3d probes, without any visualization
  • more robust to saturating channels with nan_to_num
  • cc_merge set to 1 automatically if templates on few channels are detected
  • fix a bug if only one artefact type is given
  • fix a bug if only 2 spikes are found on a single electrode
  • former parameters sim_same_elec and dip_threshold renamed into merge_method and merge_param
  • sanity plots for local merges can now be produced during clustering (debug_plots in [clustering])

Release 0.8.3

  • automatic suppression, during meta merging, of noisy templates (for SpikeToolKit/Forest)
  • during the phy export, we can automatically pre-assign labels to neurons
  • fix a bug when converting to phy with dead channels
  • fix a bug when converting to phy with file formats without data_offset
  • speedup the estimation of the amplitude distribution
  • minor fixes for clusters
  • smoothing of the templates thanks to Savitzky-Golay filtering
  • fix a bug when launching GUIs for file format without data offset
  • can now work with scipy 1.3 and statsmodels 0.10
  • isolation mode is improved, set as default and leading to better performance
  • reducing overclustering with the Hartigan dip-test of unimodality
  • can now set the number of dimensions for local PCA (10 by default)

Release 0.8.2

  • add a docker file to build the software
  • add support for shanks in phy 2.0
  • add support for deconverting in the qt launcher
  • do not create a Qt App if merging in auto mode
  • waveforms are convolved with a Hanning window to boost PCA
  • oversampling in now adapted as function of the sampling rate
  • reduction of I/O while oversampling
  • speed improvement with undersampling while cleaning the dictionary
  • automation of the software for SpikeForest/SpikeToolkit benchmarks
  • merging is now included in the default pipeline
  • normalization of the metrics in the meta merging GUI

Release 0.8.0

  • major improvement in the clustering. No more max_clusters parameters
  • much faster clustering (thanks to Ruben Herzog)
  • added the statsmodels library as a required dependency
  • enhancement of the smart search mode
  • enhancement of the bicubic spline interpolation
  • fix a typo when using dead times and the collect mode
  • fix a minor bug when small amount of spikes are found during smart search
  • fix a bug in the wrapper for BRW files
  • support for phy 2.0 and phylib
  • remove the strongly time shifted templates
  • additing of a wrapper for MDA file format
  • amplitudes for unfitted spikes is now 1 when exporting to phy
  • default install is now qt5, to work with phy 2.0

Release 0.7.6

  • cosmetic changes in the GUI
  • adding a deconverting method to switch back from phy to MATLAB
  • support for the lags between templates in the MATLAB GUI
  • warn user if data are corrupted because of interrupted filtering
  • reduction of the size for saved clusters
  • display the file name in the header
  • fix a nasty bug allowing spikes at the border of chunks to be fitted even during dead periods

Release 0.7.5

  • fix a bug for MPICH when large dictionaries.
  • fix a bug for numpy files when used with new numpy versions
  • add the possibility to subtract one channel as a reference channel from others
  • native support for blackrock files (only .ns5 tested so far)
  • simplifications in the parameter file
  • fix for display of progress bars with tqdm
  • addition of a multi-folders mode for openephys
  • hide GPU support for now, as this is not actively maintained and optimized
  • fix in the MATLAB GUI for float32 data
  • fix the broken log files
  • default cpu number is now half the available cores

Release 0.7.4

  • fix a regression with spline interpolation, more investigation needed

Release 0.7.0

  • fix a possible rounding bug if triggers are given in ms
  • artefacts are computed as medians and not means over the signal
  • can turn off shared memory if needed
  • a particular pattern can be specified for neuralynx files
  • fix bugs with output_dir, as everything was not saved in the folder
  • add a circus-folders script to process virtually files within several folders as a single recording
  • add a circus-artefacts script to concatenate artefact files before using stream mode
  • multi-files mode is now enabled for Neuralynx data
  • fixes for conversion of old dataset with python GUI
  • smooth exit if fitting with 0 templates (thanks to Alex Gonzalez)
  • enhance the bicubic spline interpolation for oversampling
  • spike times are now saved as uint32 for long recordings

Release 0.6.7

  • optimizations for clusters (auto blosc and network bandwith)
  • addition of a dead_channels option in the [detection] section, as requested
  • prevent user to remove median with only 1 channel
  • fix for parallel writes in HDF5 files
  • hide h5py FutureWarning

Release 0.6.6

  • fix for matplotlib 2.2.2
  • fix a bug when loading merged data with phy GUI
  • faster support for native MCD file with pyMCStream
  • more robust whitening for large arrays with numerous overlaps
  • add an experimental mode to refine coreset (isolated spikes)
  • put merging units in Hz^2 in the merging GUI
  • add a HDF5 compression mode to greatly reduce disk usage for very large probe
  • add a Blosc compression mode to save bandwith for clusters
  • fix a display bug in the merging GUI when performing multiple passes

Release 0.6.5

  • reduce memory consumption for mixture removal with shared memory
  • made an explicit parameter cc_mixtures for mixture removal in the [clustering] section
  • Minor fixes in the MATLAB GUI
  • fix in the exact times shown during preview if second is specified
  • prevent errors if filter is False and overwrite is False

Release 0.6.4

  • fix a bug in the BEER for windows platforms, enhancing robustness to mpi data types
  • speed up the software when using ignore_dead_times
  • ensure backward compatibility with hdf5 version for MATLAB
  • fix a rare bug in clustering, when no spikes are found on electrodes
  • fix a bug in the MATLAB GUI when reloading saved results, skipping overlap fixes

Release 0.6.3

  • fix a bug if the parameter file have tabulations characters
  • add a tab to edit parameters directly in the launcher GUI
  • fix dtype offset for int32 and int64
  • minor optimizations for computations of overlaps
  • explicit message displayed on screen if filtering has already been performed
  • can specify a distinct folder for output results with output_dir parameter
  • fix a bug when launching phy GUI for datafiles without data_offset parameter (HDF5)
  • fix a memory leak when using dead_times
  • fix a bug for BRW and python3
  • fix a bug in the BEER
  • pin HDF5 to 1.8.18 versions, as MATLAB is not working well with 1.10
  • fix a bug when relaunching code and overwrite is False
  • fix a bug when peak detection is set on both with only one channel

Release 0.6.2

  • fix for openephys and new python syntax
  • fix in the handling of parameters
  • fix a bug on windows with unclosed hdf5 files
  • fix a bug during converting with multi CPU on windows
  • minor optimization in the fitting procedure
  • support for qt5 (and backward compatibility with qt4 as long as phy is using Qt4)

Release 0.6.1

  • fix for similarities and merged output from the GUIs
  • fix for Python 3 and HDF5
  • fix for Python 3 and launcher GUI
  • fix for maxlag in the merging GUI
  • optimization in the merging GUI for pairs suggestion
  • addition of an auto_mode for meta merging, to suppress manual curation
  • various fixes in the docs
  • fix a bug when closing temporary files on windows
  • allow spaces in names of probe files
  • collect_all should take dead times into account
  • patch to read INTAN 2.0 files
  • fix in the MATLAB GUI when splitting neurons
  • fix in the MATLAB GUI when selecting individual amplitudes

Release 0.6.0

  • fix an IMPORTANT BUG in the similarities exported for phy/MATLAB, affect the suggestions in the GUI
  • improvements in the neuralynx wrapper
  • add the possibility to exclude some portions of the recordings from the analysis (see documentation)
  • fix a small bug in MS-MPI (Windows only) when shared memory is activated and emtpy arrays are present

Release 0.5.9

  • The validating step can now accept custom spikes as inputs
  • Change the default frequency for filtering to 300Hz instead of 500Hz

Release 0.5.8

  • fix a bug for int indices in some file wrappers (python 3.xx) (thanks to Ben Acland)
  • fix a bug in the preview gui to write threshold
  • fix a bug for some paths in Windows (thanks to Albert Miklos)
  • add a wrapper for NeuraLynx (.ncs) file format
  • fix a bug in the installation of the MATLAB GUI
  • fix a bug to see results in MATLAB GUI with only 1 channel
  • fix a bug to convert data to phy with only positive peaks
  • add builds for python 3.6
  • optimizations in file wrappers
  • fix a bug for MCS headers in multifiles, if not all with same sizes
  • add the possibility (with a flag) to turn off parallel HDF5 if needed
  • fix a bug with latest version of HDF5, related to flush issues during clustering

Release 0.5.7

  • Change the strsplit name in the MATLAB GUI
  • Fix a bug in the numpy wrapper
  • Fix a bug in the artefact removal (numpy 1.12), thanks to Chris Wilson
  • Fixes in the matlab GUI to ease a refitting procedure, thanks to Chris Wilson
  • Overlaps are recomputed if size of templates has changed (for refitting)
  • Addition of a “second” argument for a better control of the preview mode
  • Fix when using the phy GUI and the multi-file mode.
  • Add a file wrapper for INTAN (RHD) file format

Release 0.5.6

  • Fix in the smart_search when only few spikes are found
  • Fix a bug in density estimation when only few spikes are found

Release 0.5.5

  • Improvement in the smart_select option given various datasets
  • Fix a regression for the clustering introduced in 0.5.2

Release 0.5.2

  • fix for the MATLAB GUI
  • smart_select can now be used [experimental]
  • fix for non 0: DISPLAY
  • cosmetic changes in the clustering plots
  • ordering of the channels in the openephys wrapper
  • fix for rates in the MATLAB GUI
  • artefacts can now be given in ms or in timesteps with the trig_unit parameter

Release 0.5rc

  • fix a bug when exporting for phy in dense mode
  • compatibility with numpy 1.12
  • fix a regression with artefact removal
  • fix a display bug in the thresholds while previewing with a non unitary gain
  • fix a bug when filtering in multi-files mode (overwrite False, various t_starts)
  • fix a bug when filtering in multi-files mode (overwrite True)
  • fix a bug if matlab gui (overwrite False)
  • fix the gathering method, not working anymore
  • smarter selection of the centroids, leading to more clusters with the smart_select option
  • addition of a How to cite section, with listed publications

Release 0.5b9

  • switch from progressbar2 to tqdm, for speed and practical issues
  • optimization of the ressources by preventing numpy to use multithreading with BLAS
  • fix MPI issues appearing sometimes during the fitting procedure
  • fix a bug in the preview mode for OpenEphys files
  • slightly more robust handling of openephys files, thanks to Ben Acland
  • remove the dependency to mpi4py channel on osx, as it was crashing
  • fix a bug in circus-multi when using extensions

Release 0.5b8

  • fix a bug in the MATLAB GUI in the BestElec while saving
  • more consistency with “both” peak detection mode. Twice more waveforms are always collect during whitening/clustering
  • sparse export for phy is now available
  • addition of a dir_path parameter to be compatible with new phy
  • fix a bug in the meta merging GUI when only one template left

Release 0.5b7

  • fix a bug while converting data to phy with a non unitary gain
  • fix a bug in the merging gui with some version of numpy, forcing ucast
  • fix a bug if no spikes are detected while constructing the basis
  • Optimization if both positive and negative peaks are detected
  • fix a bug with the preview mode, while displaying non float32 data

Release 0.5b6

  • fix a bug while launching the MATLAB GUI

Release 0.5b3

  • code is now hosted on GitHub
  • various cosmetic changes in the terminal
  • addition of a garbage collector mode, to collect also all unfitted spikes, per channel
  • complete restructuration of the I/O such that the code can now handle multiple file formats
  • internal refactoring to ease interaction with new file formats and readibility
  • because of the file format, slight restructuration of the parameter files
  • N_t and radius have been moved to the [detection] section, more consistent
  • addition of an explicit file_format parameter in the [data] section
  • every file format may have its own parameters, see documentation for details (or –info)
  • can now work natively with open ephys data files (.openephys)
  • can now work natively with MCD data files (.mcd) [using neuroshare]
  • can now work natively with Kwik (KWD) data files (.kwd)
  • can now work natively with NeuroDataWithoutBorders files (.nwb)
  • can now work natively with NiX files (.nix)
  • can now work natively with any HDF5-like structure data files (.h5)
  • can now work natively with Arf data files (.arf)
  • can now work natively with 3Brain data files (.brw)
  • can now work natively with Numpy arrays (.npy)
  • can now work natively with all file format supported by NeuroShare (plexon, blackrock, mcd, …)
  • can still work natively with raw binary files with/without headers :)
  • faster IO for raw binary files
  • refactoring of the exports during multi-file/preview/benchmark: everything is now handled in raw binary
  • fix a bug with the size of the safety time parameter during whitening and clustering
  • all the interactions with the parameters are now done in the circus/shared/ file
  • all the interactions with the probe are now done in the circus/shared/ file
  • all the messages are now handled in circus/shared/
  • more robust and explicit logging system
  • more robust checking of the parameters
  • display the electrode number in the preview/result GUI
  • setting up a continuous integration workflow to test all conda packages with appveyor and travis automatically
  • cuda support is now turned off by default, for smoother install procedures (GPU yet do not bring much)
  • file format can be streamed. Over several files (former multi-file mode), but also within the same file
  • several cosmetic changes in the default parameter file
  • clustering:smart_search and merging:correct_lag are now True by default
  • fix a minor bug in the smart search, biasing the estimation of densities
  • fix a bug with the masks and the smart-search: improving results
  • addition of an overwrite parameter. Note that any t_start/t_stop infos are lost
  • if using streams, or internal t_start, output times are on the same time axis than the datafile
  • more robust parameter checking

Release 0.4.3

  • cosmetic changes in the terminal
  • suggest to reduce chunk sizes for high density probes (N_e > 500) to save memory
  • fix a once-in-a-while bug in the smart-search

Release 0.4.2

  • fix a bug in the test suite
  • fix a bug in python GUI for non integer thresholds
  • fix a bug with output strings in python3
  • fix a bug to kill processes in windows from the launcher
  • fix graphical issues in the launcher and python3
  • colors are now present also in python3
  • finer control of the amplitudes with the dispersion parameter
  • finer control of the cut off frequencies during the filtering
  • the smart search mode is now back, with a simple True/False flag. Use it for long or noisy recordings
  • optimizations in the smart search mode, now implementing a rejection method based on amplitudes
  • show the mean amplitude over time in the MATLAB GUI
  • MATLAB is automatically closed when closing the MATLAB GUI
  • mean rate is now displayed in the MATLAB GUI, for new datasets only
  • spike times are now saved as uint32, for new datasets only
  • various fixes in the docs
  • improvements when peak detection is set on “both”
  • message about cc_merge for low density probes
  • message about smart search for long recordings
  • various cosmetic changes
  • add a conda app for anaconda navigator

Release 0.4.1

  • fix a bug for converting millions of PCs to phy, getting rid of MPI limitation to int32
  • fix bugs with install on Windows 10, forcing int64 while default is int32 even on 64bits platforms
  • improved errors messages if wrong MCS headers are used
  • Various cosmetic changes

Release 0.4

First realease of the software