This paper addresses class-incremental semantic segmentation of laparoscopic images, aiming to train a segmentation model to recognize new anatomical classes without forgetting prior ones, independent of the initial dataset. Our approach structures the training into \(T+1\) steps, starting from an initial step followed by \(T\) incremental step. In each step \(t\), the model, denoted by \(\mathcal ^t\), is trained on a dataset \(\mathcal ^t\) with \(\textbf^t\) as the input image and \(\textbf^t \) as its corresponding ground-truth segmentation map. Moreover, to prevent catastrophic forgetting, we utilize a diffusion model to generate a set of synthetic images, denoted by \(\'_0\}_^N\), where N represents the total number of synthetic images generated. For training at step \(t\) (\(t>0\)), the training set is the concatenation of two image sets: the new generated images \(\'_0\}_^N\) and the current step’s images \(\mathcal ^t\), thus enhancing the model’s ability to learn new and retain old classes. An overview of each incremental step is shown in Fig. 2.

In our approach, the primary task is to synthesize additional laparoscopic images \(\^_0\}\) to diversify our training dataset for incremental learning. We utilize the denoising diffusion probabilistic model (DDPM) [26], a generative model that synthesizes realistic images by gradually denoising. Specifically, the objective of the DDPM training phase is to develop an accurate model of the noise characteristics embedded in the data. This learning is facilitated by an iterative process that begins by sampling a real laparoscopic image \( \textbf_0 \). During training, a diffusion step \( s \in \ \) is selected, and noise \( \varvec \sim \mathcal (\textbf, \textbf) \) is added. The model parameters are refined by calculating the discrepancy between the predicted noise and the injected noise:

$$\begin \nabla _\theta \left\| \varvec - \varvec_\theta \left( \sqrt_s} \textbf_0 + \sqrt_s} \varvec, s\right) \right\| ^2, \end$$

(1)

where \( \alpha _s = 1 - \beta _s \) indicating the proportion of the original signal retained at step \( s \), \( \beta _s \) is a variance schedule for each diffusion step, and \( \bar_s \) is the cumulative product of \( \alpha _s \) from step 1 to \( s \). The neural network \( \varvec_\theta \), parameterized by \( \theta \), is trained to predict the noise added at each step, crucial for accurately reversing noise during sampling.

The sampling phase aims to synthesize realistic laparoscopic images by reversing the learned noise distributions. Starting with a noise image \( \textbf^_S \sim \mathcal (\textbf, \textbf) \), the model reconstructs cleaner images in reverse:

$$\begin \textbf^_ = \frac} \left( \textbf^_s - \frac_s}} \varvec_\theta (\textbf^_s, s) \right) + \sigma _s \textbf, \end$$

(2)

where \( \textbf \sim \mathcal (\textbf, \textbf) \) if \( s > 1 \), otherwise \( \textbf = \textbf \), and \( \sigma _s \) represents the standard deviation determined by the noise schedule. This reverse diffusion process continues until \(\textbf^_0\), the final synthetic laparoscopic image, is synthesized, thereby enriching the dataset to improve the robustness and generalization of the model in class-incremental learning scenarios, particularly by increasing data diversity, balancing class representation, and enhancing training effectiveness.

To enhance class-incremental learning for laparoscopic image segmentation, we employ dense alignment distillation on all aspects (DADA) method from IDEC [17] as the distillation network backbone. This method efficiently distills knowledge across both intermediate layers and output logits, ensuring accurate pixel classification. The inputs are processed concurrently by the static previous model, \(\mathcal ^\), and the trainable current model, \(\mathcal ^t\), with atrous spatial pyramid pooling generating context-rich embeddings for effective feature distillation.

The DADA method evaluates similarities between the intermediate features embedding \(\^_l\}_\) from \(\mathcal ^\) and \(\^_l\}_\) from \(\mathcal ^t\), aiding \(\mathcal ^t\) in inheriting and refining its predecessor’s features. Knowledge distillation is implemented using a weighted loss across selected layers and output logits:

$$\begin \mathcal _ = \sum _ \varvec_l \cdot d(\textbf_l^, \textbf_l^t) \end$$

(3)

where \(L\) represents the intermediate and output layers of network \(M\) involved in distillation, \(\varvec_l\) the layer weights, and \(d(\cdot ,\cdot )\) the Euclidean distance.

To enhance the model’s ability to distinguish features based on their relevance to specific classes and their source (real or synthetic), we employ an image-level contrastive learning strategy. This strategy utilizes encoded features from both current and previous models. Specifically, for any laparoscopic image \(\textbf\), we obtain its encoded features \(\textbf^ = \mathcal ^t(\textbf)\) from the current model \(\mathcal ^t\), and \(\textbf^ = \mathcal ^(\textbf)\) from the previous model \(\mathcal ^\). Similarly, for a synthetically generated image \(\textbf^\), its features are \(\textbf^} = \mathcal ^t(\textbf^)\) and \(\textbf^} = \mathcal ^(\textbf^)\).

These features are projected into a new feature space using a projection head \(p(\cdot )\), which facilitates effective feature discrimination. In this space, positive pairs consist of features extracted from the same input image by both models \(M^\) and \(M^t\). Negative pairs are constructed by pairing features from a real image processed by current model with features from a synthetic image processed by the current or the previous model.

The contrastive loss for a single positive pair \(\left( \textbf^, \textbf^\right) \) is defined as follows:

$$\begin&\mathcal _\left( \textbf^, \textbf^\right) \nonumber \\&\ = -\log \frac^) \cdot p(\textbf^) / \tau \right) }^) p(\textbf^) / \tau \right) \sum _^ \in \^}, \textbf^}\}} \exp \left( p(\textbf^) p(\textbf^) / \tau \right) } \end$$

(4)

where \(\tau \) is a temperature scaling parameter that adjusts the sharpness of the distribution, facilitating the differentiation between similar and dissimilar feature pairs. This approach ensures that features from the same image are more similar to each other than to features from different images, thus enhancing the model’s discriminative capabilities in identifying relevant features from laparoscopic images.

The integrated loss function, \(\mathcal _}\), supports CISS by combining three losses:

$$\begin \mathcal _} = \mathcal _} + \mathcal _} + \mathcal _}, \end$$

(5)

where \(\mathcal _}\) represents the weighted cross-entropy loss, adjusted for class frequency to tackle class imbalance, as in previous work [25]. In the initial training phase, only \(\mathcal _}\) is applied. During incremental learning steps, \(\mathcal _}\) remains actively employed and is augmented by the integration of pseudo-labels from the previous model with current ground-truth annotations. This integration, applied to both generated and real images, leverages high-confidence data to significantly enhance the model’s segmentation accuracy. \(\mathcal _}\) measures the knowledge distilled from the previous to the current model, ensuring retention of learned features. Meanwhile, \(\mathcal _}\) enhances the model’s capability to distinguish between diverse feature representations across both new and previously learned classes.