Segment Anything Model Applied to CryoET

Background

CryoET

I don’t want to get too into the details here, but to briefly give you a bit of background, CryoET (Cryogenic Electron Tomography) is the method by which we achieve near-atomic resolution of how bacteria interact with their environment. Why is this important? Well, take Legionnaires’ disease, for example. The bacteria called Legionella are engulfed by white blood cells that protect the body. Unfortunately, a cascade of proteins is released that manipulate the cell, allowing the bacteria to reproduce instead of being destroyed. Without images of this process via CryoET, we would have no idea what is happening, let alone develop any means of combating the bacteria.

A bottleneck in the process of getting this near-atomic resolution of macromolecular complexes (the things that do the cool stuff in bacteria) is finding them in the tomogram (3d image of bacteria). Tomograms have a very low signal-to-noise ratio not only making them difficult to see, but requiring that you collect a lot of them and averaging them to get a good idea of what they look like.

Typically this process is done by hand. Lets try and do it with a machine!

Here is a picture so you can see how bad the signal to noise ratio really is.

SAM (Segment Anything Model)

The Segment Anything Model (SAM) is an AI model developed by Meta (formerly Facebook) that is designed for image segmentation tasks. Image segmentation involves dividing an image into different segments or regions, typically by identifying and separating objects within the image.

SAM is particularly powerful because it is a general-purpose model that can segment objects in any type of image, regardless of the object category, without needing task-specific training.

Plan

WORK IN PROGRESS