Unconditional Generation

• Image Generation
  
• Sentence Generation
  

Conditional Generation

• We will control what to generate (e.g. 给定文字产生对应图像，给定图像产生另一张图像 (风格转换)…)

Generator outputs a complex distribution

• The data we want to generate has a distribution P d a t a ( x ) P_{data}(x) Pdata​(x); A generator G G G is a network. The network defines a probability distribution P G ( x ) P_G(x) PG​(x).
  • It is difficult to compute P G ( x ) P_G(x) PG​(x). We can only sample from the distribution. ⇒ \Rightarrow ⇒ Hard to measure the closeness between P G ( x ) P_G(x) PG​(x) and P d a t a ( x ) P_{data}(x) Pdata​(x)
    

Generator

• Generator: a neural network (NN), or a function
  • Input: a vector; Each dimension of input vector may represent some characteristics
  • Output: a high dimensional vector (image / sentence) ⇒ \Rightarrow ⇒ a Complex Distribution
    

Discriminator

• Discriminator: a neural network (NN), or a function
  • Input: a high dimensional vector (image / sentence)
  • Output: a scalar (Larger value means real, smaller value means fake)
    

Generator and Discriminator

• 首先我们需要准备一个由真实图片组成的数据集，然后我们的 Generator v1 由向量生成了一堆图片，但由于一开始 Generator v1 的参数是随机初始化的，它生成的图片实际上就是一堆随意的输出。此时我们就可以训练 Discriminator v1，使它分辨出哪张图片是生成器生成的，哪张图片是真实的；在训练完 Discriminator v1 后，我们转而训练 Generator v1，使它生成的图片能尽量骗过 Discriminator v1 (生成使 Discriminator v1 输出得分高的图片)，这样就得到了 Generator v2…
• 这样不断地重复，不断得到更好的 Generator 和 Discriminator…
  

This is where the term “adversarial” comes from.

Algorithm

• Initialize generator and discriminator
• In each training iteration:
  • Step 1: Fix generator G G G, and update discriminator D D D (Discriminator learns to assign high scores to real objects and low scores to generated objects)
    
  • Step 2: Fix discriminator D D D, and update generator G G G (Generator learns to “fool” the discriminator)
    • How to implement? 可以把 Generator 和 Discriminator 组合起来，看作一整个网络。我们只需要让最后网络输出的数值越大越好。同时注意，我们在进行参数更新时只调整前几个对应 Generator 的 hidden layer 的参数
      
• 算法的数学描述：
  

Note: input 的 vector 采样自某个分布 (Uniform distribution, Gaussian distribution…); 具体这些 vector 是几维的可能是一个需要调整的超参

Structured Learning / Prediction

• Output is composed of components with dependency (e.g. output a sequence, a matrix, a graph, a tree …)

Why Structured Learning Challenging?

• One-shot / Zero-shot Learning:
  • In classification, each class has some examples.
  • In structured learning, If you consider each possible output as a “class”, since the output space is huge, most “classes” do not have any training data. So machine has to create new stuff during testing.
• Machine has to learn to do planning.
  • Machine generates objects component-by-component, but it should have a big picture in its mind. (Because the output components have dependancy, they should be considered globally.)

Structured Learning Approach

• 在常规的监督学习中，我们可以收集一个数据集，样本为服从某个分布的向量，标签为对应的图片。我们直接用该数据集训练网络即可 (这里有个难点: 如何确定每张图片对应的向量？ → \rightarrow → 在人为确定每张图片对应的向量时，应该使比较相似的两张图片对应的向量也比较相似；e.g. 用向量的第一个分量表示输出图片代表的数字，第二个分量表示倾斜的程度)
  
• 还有一个方法可以更方便的标注出每张图片对应的向量：Encoder in auto-encoder provides the code
  

• Encoder: Compact representation of the input object
• Decoder: Reconstruct the original object
• Train: 将 Encoder 和 Decoder 组合起来，希望输入和输出尽量相似；这里注意到，其实 Decoder 就是我们想要的 Generator !
  

Decoder as a generator

• 问题: Training data 是有限的，难以保证 Decoder 的质量
  

• 解决方法：Variational Auto-encoder (VAE)

• It is easier to catch the relation between the components by top-down evaluation
  

How to learn the discriminator?

• I only have some real images ⇒ \Rightarrow ⇒ Discriminator only learns to output “1” (real)
  
• Discriminator training needs some negative examples (Quality of negative examples is critical)
  

How to generate realistic negative examples? - General Algorithm

• Given a set of positive examples, randomly generate a set of negative examples.
• In each iteration
  • Learn a discriminator D D D that can discriminate positive and negative examples.
    
  • Generate negative examples by discriminator D D D
    x ^ = arg max ⁡ x ∈ X D ( x ) \hat x=\argmax_{x\in\mathcal X}D(x) x^=x∈Xargmax​D(x)
• 因此，关键就是要解 arg max ⁡ \argmax argmax 问题 (最简单的方法无疑是枚举所有可能的 x x x，但这种方法开销太大)。在 GAN 中，Generator 实际上就是用来解 arg max ⁡ \argmax argmax 问题并以此生成 negative examples 的

e.g. 去掉语音中的杂音

• 下面的语音用 spectrum 表示，因此可以直接套用图像处理的网络架构
  
• Conditional GAN
  

• Generator: 给 Generator 看一段影片，让它预测影片接下来发生的事情
  

Unsupervised Conditional Generation

• Transform an object from one domain to another without paired data (e.g. style transfer; 我们只有一堆风景照和一堆艺术画，但风景照和艺术画之间并不是两两对应的)
  
  

• Problem: ignore input (Discriminator 只负责判别画是否属于艺术画，因此 Generator 可能学会只输出某些艺术画，使得输出的画与输入的照片完全无关)
  • The issue can be avoided by network design. Simpler generator makes the input and output more closely related. (shallow network 不太受这个问题的影响，可以直接 train)

相比于 Direct Transformation，Projection to Common Space 可以支持更大程度的转换

Target

Training

• 利用 Auto-Encoder 的思想，相当于 train 两个 Auto-Encoder (分别为图中的红色箭头和蓝色箭头所示)
  
• 如果只 learn auto-encoder，decoder output 的 image 会很模糊，因此还可以再加上 Discriminator，这就相当于 train 两个 VAE-GAN
  

Problem

• Because we train two auto-encoders separately, the images with the same attribute may not project to the same position in the latent space.
  • latent space: 隐空间；隐空间的作用是为了找到 模式 (pattern) 而学习数据特征并且简化数据表示

• Couple GAN [Ming-Yu Liu, et al., NIPS, 2016]; UNIT [Ming-Yu Liu, et al., NIPS, 2017]

• 使两个 Encoder 和 Decoder 共享参数 (如下图虚线所示)：Encoder 共享后面几个 layer 的参数，Decoder 共享前面几个 layer 的参数
  • 最极端的情况是共享所有参数，这样 Encoder 还需要读入一个 flag 表示图片位于哪个 domain
    

• Domain Discriminator: The domain discriminator forces the output of E N X EN_X ENX​ and E N Y EN_Y ENY​ have the same distribution. [Guillaume Lample, et al., NIPS, 2017]
  • input: latent vector; output: 判断 latent vector 属于哪个 domain
    

• ComboGAN [Asha Anoosheh, et al., arXiv, 017]

类似 CycleGAN

• Used in DTN [Yaniv Taigman, et al., ICLR, 2017] and XGAN [Amélie Royer, et al., arXiv, 2017]

• 计算 latent vector 的相似度 ⇒ \Rightarrow ⇒ 语义上的相似度
  

• Given a data distribution P d a t a ( x ) P_{data}(x) Pdata​(x)
• We have a distribution P G ( x ; θ ) P_G(x;\theta) PG​(x;θ) parameterized by θ \theta θ
  • We want to find θ \theta θ such that P G ( x ; θ ) P_G(x;\theta) PG​(x;θ) close to P d a t a ( x ) P_{data}(x) Pdata​(x)

Maximum Likelihood Estimation = Minimize KL Divergence

• Sample { x 1 , x 2 , . . . , x m } \{x^1,x^2,...,x^m\} {x1,x2,...,xm} from P d a t a ( x ) P_{data}(x) Pdata​(x). We can compute P G ( x i ; θ ) P_G(x^i;\theta) PG​(xi;θ).
• Likelihood of generating the samples:
  L = ∏ i = 1 m P G ( x i ; θ ) L=\prod_{i=1}^m P_G(x^i;\theta) L=i=1∏m​PG​(xi;θ)
• Find θ ∗ \theta^* θ∗ maximizing the likelihood (also minimizes the KL divergence):
  θ ∗ = arg max ⁡ θ ∏ i = 1 m P G ( x i ; θ ) = arg max ⁡ θ ∑ i = 1 m log ⁡ P G ( x i ; θ ) ≈ arg max ⁡ θ E x ∼ P d a t a [ log ⁡ P G ( x ; θ ) ] = arg max ⁡ θ ∫ x P d a t a ( x ) log ⁡ P G ( x ; θ ) d x = arg max ⁡ θ [ ∫ x P d a t a ( x ) log ⁡ P G ( x ; θ ) d x − ∫ x P d a t a ( x ) log ⁡ P d a t a ( x ) d x ] = arg min ⁡ θ ∫ x P d a t a ( x ) log ⁡ P d a t a ( x ) P G ( x ; θ ) d x = arg min ⁡ θ K L ( P d a t a ( x ) ∣ ∣ P G ( x ; θ ) ) \begin{aligned}\theta^*&=\argmax_\theta\prod_{i=1}^m P_G(x^i;\theta)=\argmax_\theta\sum_{i=1}^m \log P_G(x^i;\theta) \\&\approx \argmax_\theta E_{x\sim P_{data}}[ \log P_G(x;\theta)] \\&=\argmax_\theta \int_xP_{data}(x)\log P_G(x;\theta)dx \\&=\argmax_\theta[ \int_xP_{data}(x)\log P_G(x;\theta)dx-\int_x P_{data}(x)\log P_{data}(x)dx] \\&=\argmin_\theta \int_xP_{data}(x)\log\frac{P_{data}(x)}{P_G(x;\theta)}dx \\&=\argmin_\theta KL(P_{data}(x)||P_G(x;\theta)) \end{aligned} θ∗​=θargmax​i=1∏m​PG​(xi;θ)=θargmax​i=1∑m​logPG​(xi;θ)≈θargmax​Ex∼Pdata​​[logPG​(x;θ)]=θargmax​∫x​Pdata​(x)logPG​(x;θ)dx=θargmax​[∫x​Pdata​(x)logPG​(x;θ)dx−∫x​Pdata​(x)logPdata​(x)dx]=θargmin​∫x​Pdata​(x)logPG​(x;θ)Pdata​(x)​dx=θargmin​KL(Pdata​(x)∣∣PG​(x;θ))​

Generator is a NN

• Generated distribution:
  P G ( x ) = ∫ z P p r i o r ( z ) I G ( z ) = x d x P_G(x)=\int_z P_{prior}(z)\mathbb I_{G(z)=x}dx PG​(x)=∫z​Pprior​(z)IG(z)=x​dx
  • Difficult to compute the likelihood; Hard to learn by maximum likelihood ⇒ \Rightarrow ⇒ GAN can solve it!

P p r i o r ( z ) P_{prior}(z) Pprior​(z) can be a Gaussian distribution.

• A generator G G G is a network. The network defines a probability distribution P G ( x ; θ ) P_G(x;\theta) PG​(x;θ)
  

Our objective
G ∗ = arg min ⁡ G D i v ( P G , P d a t a ) G^*=\argmin_GDiv(P_G,P_{data}) G∗=Gargmin​Div(PG​,Pdata​)

How to compute the divergence? - Sampling is good enough ……

• Although we do not know the distributions of P G P_G PG​ and P d a t a P_{data} Pdata​, we can sample from them.
  

Discriminator D D D evaluates the “difference” between P G ( x ) P_G(x) PG​(x) and P d a t a ( x ) P_{data}(x) Pdata​(x)

• Example Objective Function for D D D ( G G G is fixed):
  
  • 须在 NN 后加 sigmoid 来保证 log ⁡ \log log 内的值有意义
• Training: Using the example objective function is exactly the same as training a binary classifier (i.e. minimize the cross entropy error)
  
• The maximum objective value is related to JS divergence.
  • intuition: small divergence ⇒ \Rightarrow ⇒ hard to discriminate (cannot make objective large); large divergence ⇒ \Rightarrow ⇒ easy to discriminate

max ⁡ D V ( G , D ) \max_DV(G,D) maxD​V(G,D)

• Given G G G, what is the optimal D ∗ D^* D∗ maximizing (Assume that D ( x ) D(x) D(x) can be any function)
  
• Given x x x, the optimal D ∗ D^* D∗ maximizing (Since D ( x ) D(x) D(x) can be any function)
  i.e. Find D ∗ D^* D∗ maximizing: f ( D ) = a log ⁡ ( D ) + b log ⁡ ( 1 − D ) f(D)=a\log(D)+b\log(1-D) f(D)=alog(D)+blog(1−D) (a concave function)
  注意到， D ∗ D^* D∗ 的输出值在 0 和 1 之间，也符合在 D ( x ) D(x) D(x) 之后加 sigmoid 的做法
• 下面我们就可以把 D ∗ D^* D∗ 代入 V ( G , D ) V(G,D) V(G,D)，得到 max ⁡ D V ( G , D ) \max_DV(G,D) maxD​V(G,D)，发现它与 Jensen-Shannon divergence 相关; 也就是说，我们将 train D D D 后得到的 D ∗ D^* D∗ 代入 V V V (objective function) 就可以得到 JS divergence
  max ⁡ D V ( G , D ) = V ( G , D ∗ ) = E x ∼ P data  [ log ⁡ P data  ( x ) P data  ( x ) + P G ( x ) ] + E x ∼ P G [ log ⁡ P G ( x ) P data  ( x ) + P G ( x ) ] = ∫ x P d a t a ( x ) log ⁡ P data  ( x ) P data  ( x ) + P G ( x ) d x + ∫ x P G ( x ) log ⁡ P G ( x ) P data  ( x ) + P G ( x ) d x = ∫ x P d a t a ( x ) log ⁡ 1 2 P data  ( x ) ( P data  ( x ) + P G ( x ) ) / 2 d x + ∫ x P G ( x ) log ⁡ 1 2 P G ( x ) ( P data  ( x ) + P G ( x ) ) / 2 d x = ∫ x P d a t a ( x ) [ log ⁡ P data  ( x ) ( P data  ( x ) + P G ( x ) ) / 2 − log ⁡ 2 ] d x + ∫ x P G ( x ) [ log ⁡ P G ( x ) ( P data  ( x ) + P G ( x ) ) / 2 − log ⁡ 2 ] d x = − 2 log ⁡ 2 + ∫ x P d a t a ( x ) log ⁡ P data  ( x ) ( P data  ( x ) + P G ( x ) ) / 2 d x + ∫ x P G ( x ) log ⁡ P G ( x ) ( P data  ( x ) + P G ( x ) ) / 2 d x = − 2 log ⁡ 2 + K L ( P data  ∥ P data  + P G 2 ) + K L ( P G ∥ P data  + P G 2 ) = − 2 log ⁡ 2 + 2 J S D ( P data  ∥ P G ) Jensen-Shannon divergence \begin{aligned} &\max _{D} V(G, D)=V\left(G, D^{*}\right) \\ =&E_{x \sim P_{\text {data }}}\left[\log \frac{P_{\text {data }}(x)}{P_{\text {data }}(x)+P_{G}(x)}\right]+E_{x \sim P_{\text {G}}}\left[\log \frac{P_{\text {G}}(x)}{P_{\text {data }}(x)+P_{G}(x)}\right] \\=&\int_{x} P_{d a t a}(x) \log \frac{P_{\text {data }}(x)}{P_{\text {data }}(x)+P_{G}(x)} d x+\int_{x} P_{G}(x) \log \frac{P_{\text {G}}(x)}{P_{\text {data }}(x)+P_{G}(x)} d x \\=&\int_{x} P_{d a t a}(x) \log \frac{\frac{1}{2}P_{\text {data }}(x)}{(P_{\text {data }}(x)+P_{G}(x))/2} d x+\int_{x} P_{G}(x) \log \frac{\frac{1}{2}P_{\text {G}}(x)}{(P_{\text {data }}(x)+P_{G}(x))/2} d x \\=&\int_{x} P_{d a t a}(x)[ \log \frac{P_{\text {data }}(x)}{(P_{\text {data }}(x)+P_{G}(x))/2} -\log2]d x+\int_{x} P_{G}(x) [\log \frac{P_{\text {G}}(x)}{(P_{\text {data }}(x)+P_{G}(x))/2}-\log2] d x \\=&-2\log2+\int_{x} P_{d a t a}(x) \log \frac{P_{\text {data }}(x)}{(P_{\text {data }}(x)+P_{G}(x))/2} d x+\int_{x} P_{G}(x) \log \frac{P_{\text {G}}(x)}{(P_{\text {data }}(x)+P_{G}(x))/2} d x \\=&-2 \log 2+\mathrm{KL}\left(P_{\text {data }} \| \frac{P_{\text {data }}+P_{G}}{2}\right)+\mathrm{KL}\left(P_{G} \| \frac{P_{\text {data }}+P_{G}}{2}\right) \\ =&-2 \log 2+2 J S D\left(P_{\text {data }} \| P_{G}\right) \quad\text{{Jensen-Shannon divergence}} \end{aligned} =======​Dmax​V(G,D)=V(G,D∗)Ex∼Pdata ​​[logPdata ​(x)+PG​(x)Pdata ​(x)​]+Ex∼PG​​[logPdata ​(x)+PG​(x)PG​(x)​]∫x​Pdata​(x)logPdata ​(x)+PG​(x)Pdata ​(x)​dx+∫x​PG​(x)logPdata ​(x)+PG​(x)PG​(x)​dx∫x​Pdata​(x)log(Pdata ​(x)+PG​(x))/221​Pdata ​(x)​dx+∫x​PG​(x)log(Pdata ​(x)+PG​(x))/221​PG​(x)​dx∫x​Pdata​(x)[log(Pdata ​(x)+PG​(x))/2Pdata ​(x)​−log2]dx+∫x​PG​(x)[log(Pdata ​(x)+PG​(x))/2PG​(x)​−log2]dx−2log2+∫x​Pdata​(x)log(Pdata ​(x)+PG​(x))/2Pdata ​(x)​dx+∫x​PG​(x)log(Pdata ​(x)+PG​(x))/2PG​(x)​dx−2log2+KL(Pdata ​∥2Pdata ​+PG​​)+KL(PG​∥2Pdata ​+PG​​)−2log2+2JSD(Pdata ​∥PG​)Jensen-Shannon divergence​

JS divergence ∈ [ 0 , log ⁡ 2 ] \in[0,\log2] ∈[0,log2]

Our objective
G ∗ = arg min ⁡ G D i v ( P G , P d a t a ) = arg min ⁡ G   max ⁡ D V ( G , D ) G^*=\argmin_GDiv(P_G,P_{data})=\argmin_G\ \max _{D} V(G, D) G∗=Gargmin​Div(PG​,Pdata​)=Gargmin​ Dmax​V(G,D)

How to find G ∗ G^* G∗?

• (1) Initialize generator and discriminator
• (2) In each training iteration:
  • Step 1: Fix generator G G G, and update discriminator D D D ⇒ \Rightarrow ⇒ Given a generator G G G, max ⁡ D V ( G , D ) \max _{D} V(G, D) maxD​V(G,D) evaluate the “difference” between P G P_G PG​ and P d a t a P_{data} Pdata​
  • Step 2: Fix discriminator D D D, and update generator G G G ⇒ \Rightarrow ⇒ Pick the G G G defining P G P_G PG​ most similar to P d a t a P_{data} Pdata​
    

Notation

• L ( G ) = max ⁡ D V ( G , D ) L(G)=\max_DV(G,D) L(G)=maxD​V(G,D); i.e. loss function for generator
  

Algorithm

• Given G 0 G_0 G0​
• Find D 0 ∗ D_0^* D0∗​ maximizing V ( G 0 , D ) V(G_0,D) V(G0​,D) (Using Gradient Ascent): V ( G 0 , D 0 ∗ ) V(G_0,D_0^*) V(G0​,D0∗​) is the JS divergence between P d a t a ( x ) P_{data}(x) Pdata​(x) and P G 0 ( x ) P_{G_0}(x) PG0​​(x)
• Obtain G 1 G_1 G1​: Decrease JS divergence (?)
  
• Find D 1 ∗ D_1^* D1∗​ maximizing V ( G 1 , D ) V(G_1,D) V(G1​,D): V ( G 1 , D 1 ∗ ) V(G_1,D_1^*) V(G1​,D1∗​) is the JS divergence between P d a t a ( x ) P_{data}(x) Pdata​(x) and P G 1 ( x ) P_{G_1}(x) PG1​​(x)
• Obtain G 2 G_2 G2​: Decrease JS divergence (?)
  
• …

Decrease JS divergence (?)

• 注意到，我们上面在 Algorithm 中注明了，在 train Generator 时 (对 L ( G ) L(G) L(G) 作梯度下降) 未必会使 JS divergence 减少。原因是当 Generator 改变时，用同一个 Discriminator 计算出的 V ( G , D ) V(G,D) V(G,D) 就不是在衡量 JS divergence 了
  
• 那么为什么我们说对 L ( G ) L(G) L(G) 作梯度下降可以看作减少 JS divergence 呢？这是因为我们新增了假设： D 0 ∗ ≈ D 1 ∗ D_0^*\approx D^*_1 D0∗​≈D1∗​
  • 该假设要求我们: Don’t update G G G too much

In practice, how to compute m a x D V ( G , D ) max_DV(G,D) maxD​V(G,D)

• We can use sampling to approximate expectation
• Sample { x 1 , x 2 , . . . , x m } \{x^1,x^2,...,x^m\} {x1,x2,...,xm} from P d a t a ( x ) P_{data}(x) Pdata​(x), sample { x ~ 1 , x ~ 2 , . . . , x ~ m } \{\tilde x^1,\tilde x^2,...,\tilde x^m\} {x~1,x~2,...,x~m} from generator P G ( x ) P_G(x) PG​(x)
  

Cross entropy error: − y log ⁡ ( y ~ ) − ( 1 − y ) log ⁡ ( 1 − y ~ ) -y \log (\tilde{y})-(1-y) \log (1-\widetilde{y}) −ylog(y~​)−(1−y)log(1−y ​)

Summary

• train Discriminator 是为了衡量 JS divergence，因此理论上我们想要让每个 iteration 中都将 Discriminator 训练至收敛。但实际上我们只需进行 k k k 次 Gradient Ascent 得到 JS divergence 的一个大致的 lower bound 即可，不必训练 D D D 至收敛 (即使我们训练至收敛，仍然可能收敛至 local minima 或者由于 D D D 的表现能力有限，无法到达 global minima) (在更极端的情况下，在 train D D D 时可以只更新 1 次参数，也可以得到不错的效果)
• 注意到之前关于 Decrease JS divergence (?) 的讨论中作出的假设。为了维持这个假设，更新 Generator 参数时不能使其更新幅度过大，因此我们在每个 iteration 中只对 G G G 的参数进行 1 次梯度下降
• 注意到，在 train G G G 时，由于 D D D 的参数固定，因此 V ~ \tilde V V~ 的第一项与 G G G 无关，在训练时只需将 V ~ \tilde V V~ 的第二项作为优化目标即可

Objective Function for Generator in Real Implementation

• Minimax GAN (MMGAN): 在开始训练 Generator 时， D ( x ) D(x) D(x) 会比较小，代表 Generator 生成的图片无法骗过 Discriminator，而此时 log ⁡ ( 1 − D ( x ) ) \log(1-D(x)) log(1−D(x)) 微分很小，训练会变得很慢
  
• Non-saturating GAN (NSGAN): 为了改善上面的缺点，可以将 log ⁡ ( 1 − D ( x ) ) \log(1-D(x)) log(1−D(x)) 替换为 − log ⁡ ( D ( x ) ) -\log(D(x)) −log(D(x))。它们的趋势一样，但在开始时训练速度更快 (没有理论保证)
  
  • Real implementation: label x x x from PG as positive
    

f-divergence

• P P P and Q Q Q are two distributions. p ( x ) p(x) p(x) and q ( x ) q(x) q(x) are the probability of sampling x x x.
  • f f f is convex; f ( 1 ) = 0 f(1) = 0 f(1)=0
    
  • D f ( P ∣ ∣ Q ) D_f(P||Q) Df​(P∣∣Q) evaluates the difference of P P P and Q Q Q

If P P P and Q Q Q are the same distributions, D f ( P ∣ ∣ Q ) D_f(P||Q) Df​(P∣∣Q) has the smallest value, which is 0

• If p ( x ) = q ( x ) p(x)=q(x) p(x)=q(x) for all x x x
  

KL divergence: f ( x ) = x log ⁡ x f(x)=x\log x f(x)=xlogx

Reverse KL divergence: f ( x ) = − log ⁡ x f(x)=-\log x f(x)=−logx

Chi Square divergence: f ( x ) = ( x − 1 ) 2 f(x)=(x-1)^2 f(x)=(x−1)2

Every convex function f f f has a conjugate function f ∗ f^* f∗ ( ( f ∗ ) ∗ = f (f^*)^*=f (f∗)∗=f)

• 从下图中可以看出， f ∗ ( t ) f^*(t) f∗(t) 一定是 convex function
  

f ∗ ( t ) f^*(t) f∗(t) 就是在 t t t 取定值时，找到 x x x 使得 x t − f ( x ) xt-f(x) xt−f(x) 最大。如上图所示，我们事先画出所有固定 x x x 后关于 t t t 的 x t − f ( x ) xt-f(x) xt−f(x) 直线 (假设有 3 个可能的 x x x)， f ∗ ( t 1 ) f^*(t_1) f∗(t1​) 就是 t = t 1 t=t_1 t=t1​ 时 3 条直线的最大值，即图中红线所示曲线

• e.g. 当 f ( x ) = x log ⁡ x f(x)=x\log x f(x)=xlogx 时 (KL divergence)，我们可以计算出 f ∗ ( t ) = e t − 1 f^*(t)=e^{t-1} f∗(t)=et−1
  
  

• f ( x ) f(x) f(x) 与 f ∗ ( x ) f^*(x) f∗(x) 互为 Fenchel Conjugate Function
  
• 因此可以在计算 f f f divergence 时，将 f ( x ) f(x) f(x) 用其 Fenchel Conjugate Function 替代
  
• D D D is a function, whose input is x x x, and output is t t t；我们可以求得 D f ( P ∣ ∣ Q ) D_f(P||Q) Df​(P∣∣Q) 的 lower bound
  
• 我们可以通过找最大的 lower bound 来让其逼近 D f ( P ∣ ∣ Q ) D_f(P||Q) Df​(P∣∣Q)
  
• 至此，我们就得到了 D f ( P d a t a ∣ ∣ P G ) D_f(P_{data}||P_G) Df​(Pdata​∣∣PG​) 的表达式
  
• 进一步可以写出 G ∗ G^* G∗ 的表达式: (Original GAN has different V ( G , D ) V(G,D) V(G,D))
  
• 现在我们可以根据我们想要 minimize 的 f f f divergence，找出其 f ∗ f^* f∗，然后就能求得 V ( G , D ) V(G,D) V(G,D)，进而训练 GAN 来最小化改 f divergence 了！
  

• 下面我们来看， f f f divergence 到底是想要解决什么问题呢？

Mode Collapse

• Mode Collapse: 在 train GAN 的时候，real data 的 distribution 很大，但 generated data 的 distribution 却很小
  • e.g. 如下图所示，在做图像生成时，输出的图片来来回回就那几张
    

Mode Dropping

• Mode Dropping: real data 的 distribution 可能有多个 mode，但 generated data 确涵盖了其中一部分 mode。表面看起来 generated data 能会觉得还不错，而且多样性也够，但其实产生出来的数据只有真实数据的一部分
  

Why?

• 之所以会发生 Mode Collapse 和 Mode Dropping 直观上还是比较容易理解的：当 Generator 学会产生某种图片以后，它发现这种图片总能骗过 Discriminator，于是它就一直生成这种图片
• Dive deeper: Flaw in Optimization? (just a guess…): 当 P d a t a > 0 , P G = 0 P_{data}>0, P_G=0 Pdata​>0,PG​=0 时，KL divergence → ∞ \rightarrow\infty →∞，因此最小化 KL divergence 可能会使 P G P_G PG​ 尽可能覆盖所有 P d a t a P_{data} Pdata​，不会出现 Mode collapse 但最后生成图片的质量不会太高；而如果最小化 Reverse KL，当 P G > 0 , P d a t a = 0 P_{G}>0, P_{data}=0 PG​>0,Pdata​=0 时，Reverse KL divergence → ∞ \rightarrow\infty →∞，因此 Generator 可能会变得相当保守，进而出现 Mode collapse
  • 但实验结果证明，选择不同的 divergence 并不能有效缓解 Mode Collapse 或 Mode Dropping
    

Ensemble

• 可以通过集成学习来有效避免 Mode Collapse 和 Mode Dropping。例如我们要产生 25 张图片，那么我们就可以训练 25 个 GAN，每个 GAN 各生成 1 张图片。这样即使每个 GAN 都遇到了 Mode Collapse 或 Mode Dropping 的问题，最后生成的 25 张图片也会是不太一样的 (如果只生产一张图片，那么我们可以随机选择一个 Generator 进行生成)
  

• In most cases, P G P_G PG​ and P d a t a P_{data} Pdata​ are not overlapped. - Why?
  • (1) The nature of data: Both P G P_G PG​ and P d a t a P_{data} Pdata​ are low-dim manifold in high-dim space. (高维空间中的低维流形; e.g. 将一个 2 维平面折到 3 维平面中) The overlap can be ignored.
  • (2) Sampling: Even though P G P_G PG​ and P d a t a P_{data} Pdata​ have overlap. If you do not have enough sampling …… (采样出的点没有交集)