Semantic Image Synthesis with SPADE (GauGAN) - Tensorflow

Simple Tensorflow implementation of "Semantic Image Synthesis with Spatially-Adaptive Normalization" (CVPR 2019 Oral)

Project page | Paper | Pytorch code

Requirements

• scipy == 1.2.0
  • The latest version is not available. imsave is deprecated.
• tqdm
• numpy
• pillow
• opencv-python
• tensorflow-gpu
• keras

Preparation

• YOUR DATASET
  • Image
  • Segmentation map
    • Don't worry. I do one-hot encoding of segmentation map automatically (whether color or gray)
• CelebAMask-HQ
  • Download CelebAMask-HQ (pre-processed by Junho Kim)
    • The original was taken from here

Pretrained model

• Will be soon
  • The pytoch author has changed the network architecture. So I'm re-training again.

Usage

├── dataset
   └── YOUR_DATASET_NAME
       ├── image
           ├── 000001.jpg 
           ├── 000002.png
           └── ...
       ├── segmap
           ├── 000001.jpg
           ├── 000002.png
           └── ...
       ├── segmap_test
           ├── a.jpg 
           ├── b.png
           └── ...
       ├── segmap_label.txt (Automatically created) 
       
├── guide.jpg (example for guided image translation task)

Train

> python main.py --dataset spade_celebA --img_ch 3 --segmap_ch 3 --phase train 

Random test

> python main.py --dataset spade_celebA --segmap_ch 3 --phase random

Guide test

> python main.py --dataset spade_celebA --img_ch 3 --segmap_ch 3 --phase guide --guide_img ./guide_img.png

Our Results

Loss grpah

CityScape

CelebA-HQ (Style Manipulation)

---

CelebA-HQ (Random Manipulation)

---

How about the Least-Square loss ?

CelebA-HQ (Style Manipulation)

---

CelebA-HQ (Random Manipulation)

---

Architecture

Generator	Image Encoder	Discriminator	All-in-one

SPADE architecture

SPADE	SPADE Residual Block

Author

Junho Kim