Many surgeries can be broken down into a series of surgical steps, performed in a pre-determined order to accomplish the desired result of the surgery. Each of the surgical steps may correspond with one or more surgical activities. For example, a surgical step wherein two anatomical objects are reconnected may involve the surgical activity of suturing the objects with one another.
Medical professionals often must rely on each other to detect any errors or provide guidance during a typical surgical procedure. However, outside of an educational setting, it is not common for a sufficiently skilled/educated professional to be available to simply monitor the progress of a surgical procedure. Accordingly, there is a need for an automated system and method which can monitor a surgical procedure and provide guidance for professionals conducting the procedure.
Machine understanding of surgical procedures requires methods for surgical video parsing, identifying and segmenting objects such as anatomical structures and surgical instruments from a sequence of images. Traditional approaches begin by specifying an object model, such as template matching, constellations, bags of features, or shape models.
Researchers at Roswell Park Comprehensive Cancer Center have developed an intelligent, computer vision-based system that can provide for identification, analysis, and prediction of surgical activity during minimally invasive or robot-assisted surgery (RAS). This intelligent vision-based technology for minimally invasive and RAS procedures will provide for a Situation and Awareness-based intelligence guided surgical system (SASS) with capabilities that include surgical assistance, surgical guidance, robot control, and safety and assessment of surgeries. The level of granularity that is addressed is formalized by surgical image, real-time surgical activity, safe surgical navigation, and ultimately automation. The proposed system is capable of identifying surgical activity and surgical steps from an observation of a sequence of surgical images during surgical procedure, from both single channel and multi-channel stereo video images. Identification of the surgical images is performed by conventional computer vision-based approaches such as feature extraction, template matching, supervised or unsupervised classification, and spectral methods. SASS queries a KnowledgeBase comprising expert and machine labeled surgical images, activities, steps, and procedures for making predictions of surgical steps from an observed sequence of surgical images.
