Background: The Allure of Synthetic Data in Healthcare

Many teams turn to generative models such as MedGAN, StyleGAN‑Medical, or diffusion‑based pipelines to create large volumes of synthetic radiographs, CT slices, or pathology slides. The apparent advantages—privacy preservation, unlimited scalability, and rapid iteration—make the approach tempting for research groups that lack access to diverse patient cohorts.

However, synthetic data is not a neutral canvas. The underlying generator inherits the statistical quirks, demographic imbalances, and labeling errors of its training set. When those hidden patterns propagate into downstream diagnostic classifiers, they can produce systematic mis‑diagnoses that are difficult to trace back to the source.

Setting Up a Minimal Synthetic Imaging Pipeline

The following example uses the torch and torchvision libraries to train a lightweight GAN on a public chest‑X‑ray dataset (NIH ChestX‑Ray14). The goal is not to achieve state‑of‑the‑art image fidelity but to illustrate how bias can be introduced early in the generation stage. 

import torch
import torch.nn as nn
import torchvision.transforms as T
from torchvision.datasets import ImageFolder
from torch.utils.data import DataLoader

# Simple convolutional generator
class Generator(nn.Module):
    def __init__(self, nz=100, nc=1, ngf=64):
        super(Generator, self).__init__()
        self.main = nn.Sequential(
            nn.ConvTranspose2d(nz, ngf * 8, 4, 1, 0, bias=False),
            nn.BatchNorm2d(ngf * 8),
            nn.ReLU(True),
            nn.ConvTranspose2d(ngf * 8, ngf * 4, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(True),
            nn.ConvTranspose2d(ngf * 4, ngf * 2, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ngf * 2),
            nn.ReLU(True),
            nn.ConvTranspose2d(ngf * 2, nc, 4, 2, 1, bias=False),
            nn.Tanh()
        )

    def forward(self, input):
        return self.main(input)

# Load a tiny subset for demonstration
transform = T.Compose([T.Resize(64), T.CenterCrop(64), T.ToTensor(),
                       T.Normalize([0.5], [0.5])])
dataset = ImageFolder('chestxray_subset/', transform=transform)
loader = DataLoader(dataset, batch_size=32, shuffle=True)

After a few epochs, we can generate a batch of synthetic images and store them for later evaluation. Note the use of a fixed random seed to make the experiment reproducible. 

# Fix seed for reproducibility
torch.manual_seed(42)

gen = Generator()
z = torch.randn(32, 100, 1, 1)  # latent vectors
synthetic_batch = gen(z).detach()

# Save to disk
for i, img in enumerate(synthetic_batch):
    img_path = f'synthetic_images/img_{i}.png'
    T.ToPILImage()(img.mul(0.5).add(0.5)).save(img_path)

At this stage many practitioners assume the synthetic set is ready for downstream training. The next sections reveal why that assumption can be dangerous.

Detecting Hidden Demographic Skew

The NIH dataset contains metadata about patient age, sex, and disease label. When we train the generator on a subset that over‑represents older male patients, the synthetic images will reflect the same distribution, even though the pixel‑level realism looks convincing. 

# Load metadata (CSV with columns: filename, age, sex, label)
import pandas as pd
meta = pd.read_csv('chestxray_subset/metadata.csv')

# Simple function to compute age distribution
def age_histogram(df):
    return df['age'].hist(bins=range(0, 100, 10))

# Compare real vs synthetic age distributions
real_ages = meta['age']
synthetic_ages = []  # placeholder: we assign ages based on nearest neighbor

# Nearest‑neighbor assignment (illustrative)
from sklearn.metrics import pairwise_distances
real_images = [T.ToTensor()(ImageFolder.load_image(p)) for p in meta['filename']]
synthetic_images = [img.squeeze().numpy() for img in synthetic_batch]

dist = pairwise_distances(synthetic_images, real_images, metric='euclidean')
nearest = dist.argmin(axis=1)
synthetic_ages = real_ages.iloc[nearest].values

import matplotlib.pyplot as plt
plt.figure(figsize=(10,4))
plt.subplot(1,2,1)
plt.title('Real Age Distribution')
age_histogram(meta)

plt.subplot(1,2,2)
plt.title('Synthetic Age Distribution')
pd.Series(synthetic_ages).hist(bins=range(0,100,10))
plt.show()

When the synthetic histogram shows a pronounced peak around the 70‑80 year range, it signals a demographic bias that will be inherited by any classifier trained on these images. Regulatory frameworks such as the EU AI Act require demonstrable fairness across protected groups; hidden skew violates those mandates.

Evaluating Clinical Performance on a Balanced Test Set

To surface the impact of bias, we train a lightweight ResNet‑18 classifier on the synthetic data and evaluate it against a balanced, manually curated test set. 

from torchvision.models import resnet18
model = resnet18(pretrained=False, num_classes=14)  # 14 disease classes
criterion = nn.BCEWithLogitsLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)

# Training loop (very short for demo)
for epoch in range(3):
    for imgs, _ in loader:  # loader would need to point to synthetic images
        optimizer.zero_grad()
        outputs = model(imgs)
        loss = criterion(outputs, torch.zeros_like(outputs))  # dummy targets
        loss.backward()
        optimizer.step()
    print(f'Epoch {epoch} loss: {loss.item():.4f}')

After training, we compute per‑group AUC scores on the balanced test set. A significant drop in performance for younger patients or females indicates that the synthetic training set failed to capture critical variance. 

# Pseudo‑code for per‑group AUC
from sklearn.metrics import roc_auc_score

def evaluate(model, test_loader, groups):
    model.eval()
    all_preds, all_labels = [], []
    for imgs, labels in test_loader:
        with torch.no_grad():
            preds = model(imgs)
        all_preds.append(preds.cpu())
        all_labels.append(labels.cpu())
    preds = torch.cat(all_preds).numpy()
    labels = torch.cat(all_labels).numpy()
    results = {}
    for group_name, idx in groups.items():
        group_preds = preds[idx]
        group_labels = labels[idx]
        results[group_name] = roc_auc_score(group_labels, group_preds, average='macro')
    return results

# Example group indices (constructed from test metadata)
group_indices = {
    'young_female': [...],
    'old_male': [...],
    # ...
}
auc_by_group = evaluate(model, balanced_test_loader, group_indices)
print(auc_by_group)

If the AUC for “young_female” is markedly lower than for “old_male”, the synthetic pipeline has introduced a hidden liability that could lead to mis‑diagnosis in real clinical settings.

Mitigation Strategies Without Discarding Synthetic Data

Completely abandoning synthetic data is rarely practical. Instead, adopt the following safeguards: