Fast and user-friendly annotation and segmentation of cryo-electron tomography data using convolutional neural networks
This repository comprises a standalone version of Ais, the segmentation editor for cryoET. For the version integrated into the correlative microscopy data processing suite scNodes, see the scNodes repository.
A timelapse video of the full workflow, from reconstructed tomograms to segmented volumes showing membranes, ribosomes, mitochondrial granules, and microtubuli, is available on our YouTube channel.
Contact: mlast@mrc-lmb.cam.ac.uk
Ais works on Windows and Linux machines but not on MacOS. Install as follows:
conda create --name ais
conda activate ais
conda install python==3.9
conda install pip
pip install git+https://github.com/bionanopatterning/Ais
Then run using either of the following commands:
ais
ais-cryoet
Compatibility between Python, tensorflow, and CUDA versions can be an issue. The following combination was used during development and is known to work:
Python 3.9
Tensorflow 2.8.0
CUDA 11.8
cuDNN 8.6
protobuf 3.20.0
The software will work without CUDA, but only on the CPU. This is much slower but still reasonably interactive if the tomograms aren't too big (in XY). We do recommend installing CUDA and cuDNN in order for tensorflow to be able to use the GPU. See: https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html for help installing CUDA and cuDNN.
Open any number of tomograms and define any features you want to eventually segment. Step one is to annotate - very sparsely - in order to prepare training data for your networks. Usually, 5 to 10 minutes of annotation is enough to get started. After that, train an initial net, and check what needs further improvement.
annotation.mp4
All steps other than annotation can be done in the CLI. When training in the GUI, though, you can keep an eye on the network output as the training progresses. Not as fast as it would be in the CLI, but a nice way to see how the network is learning.
training_run.mp4
When networks still need a bit of improvement before you're ready to segment your data, model assisted annotation helps you quickly polish the training data. Screen the output of a model, copy it to the training annotations, and edit any mistakes that need fixing.
model_assisted_annotation.mp4
Before using model-assisted annotation, turn on flood mode to use the active contouring brush. Then play around with the filters and sensitivity, and watch the brush snap to the edges of your features automatically.
active_contouring.mp4
models.mp4
After segmenting your data, you can visualize the results the Ais rendering tab. You can also set up picking jobs here, or if you've already ran 'ais pick' in the terminal you can inspect the resulting particle coordinates in the context of the tomograms.
rendering.mp4
Ais integrates with Pom, a tool to present large cryoET datasets as searchable databases. Use Pom to organise the data, and Ais to mine it.
pom_database.mp4
If you're using Ais often, or are setting up a large new project where you plan to segment many different features, the feature library is a useful way to organise your work. Name and style a feature once, then automatically grab those settings whenever you create a new annotation for that feature.
feature_library.mp4
