Authors: Oleksandr Husak, ...

Affiliation: NCT/DKFZ

Description: ## Model Architecture and Approach
Our implementation leverages a class-conditional diffusion model with differential privacy.

### Core Components
- Denoising Neural Network: A multi-layer architecture with residual connections
- Adaptive Group Normalization (AdaGN): Custom normalization that dynamically adjusts based on both temporal and class conditioning signals
- Embedding Systems
- Sinusoidal time embeddings for diffusion timestep encoding
- Learned class embeddings for condition-guided generation
- Combined embedding projections to integrate multiple conditioning signals

## Diffusion Process Design
- Noise Schedule: Implements the improved cosine beta schedule from (DDPM++), providing several advantages:
- Smoother noise progression throughout the diffusion process
- Better sampling stability during the generative process
- Forward Process: Gradually adds noise according to the schedule, transforming real data into pure noise
- Reverse Process: Iteratively denoises from random noise to synthetic samples guided by class conditioning

## Privacy Preservation Mechanisms
- Differential Privacy Integration: Implemented via Opacus
- Privacy-Utility Tradeoff: Optimized through multi-objective hyperparameter tuning (MOASHA algorithm)

## Training Methodology
- Class-Weighted Loss Function: Addresses class imbalance using inverse square-root weighting to prevent over-representation of majority classes while stabilizing training
- Learning Rate Scheduling: Tested various schedulers (OneCycleLR, ReduceLROnPlateau, etc.) and selected ReduceLROnPlateau.

## Evaluation Framework
- Validation during training:
- Real-to-synthetic validation (train on real, test on synthetic)
- Synthetic-to-real validation (train on synthetic, test on real)

## Implementation Details
- PyTorch Lightning: Structured training and evaluation loops
- Hyperparameter Optimization: Multi-objective tuning balancing privacy budget (epsilon) and model utility (val_loss)

Authors: Jules Kreuer, Sofiane Ouaari

Affiliation: Methods in Medical Informatics - University of Tübingen

Description: The model implements a diffusion-based generative approach to the provided gene expression data with privacy considerations. At its core, the architecture uses a diffusion model with residual linear blocks with group normalisation.
The diffusion process follows a path where Gaussian noise is progressively added to the data according to different noise scheduling schemes (linear, cosine or power-based). During training, the model learns to predict and remove this noise. For reverse sampling, the model iteratively denoises random Gaussian samples guided by class conditioning to generate synthetic cancer data.
Key technical features include sinusoidal position embeddings for time-step encoding, attention blocks for capturing complex relationships within the data, and a privacy approach that combines differential privacy (by adding calibrated noise to gradients) and strong regularisation.

Additional improvements come from the implementation of early stopping, learning rate scheduling via OneCycleLR.
We explored attention and post-processing techniques such as outlier clipping, but discarded them as too computationally expensive and not useful enough.
The model training process includes gradient clipping to improve numerical stability.

Authors: Shane Menzies, Sikha Pentyala, Daniil Filienko, Jineta Banerjee, Martine De Cock

Affiliation: University of Washington Tacoma

Description: We adopt the Private-PGM method implemented by Chen, et al. in "Towards Biologically Plausible and Private Gene Expression Data Generation" (PETS2024), available from https://github.com/MarieOestreich/PRO-GENE-GEN, with parameters tuned for RNA-Seq data, using quantile binning into 4 bins per feature (gene), and differential privacy budget epsilon = 7. Experiments were run on the TACC Frontera / NAIRR Pilot.