花书《深度学习》全书章节摘要及笔记整理

写在前面:一直以来,极市得到了许许多多开发者的关注和支持,极市的成长离不开各位开发者的见证,为此我们特开设极市开发者祝愿帖,期待听到您真实的心声和建议~φ (> ω<*) :

极市开发者祝愿活动 (有奖回馈)

由三位大佬 Ian Goodfellow、Yoshua Bengio 和 Aaron Courville 撰写的《深度学习》,又名“花书”,是深度学习领域奠基性的经典教材,被誉为深度学习的“圣经”。全书内容非常全面而深度,多达800页左右,本文分享一个关于花书各章摘要的github项目,不仅有摘要笔记整理,部分章节还配备代码和详细博客介绍,值得收藏学习!

作者:Aman Dalmia、Ameya Godbole
项目地址:Deep-Learning-Book-Chapter-Summaries(这么棒的项目,欢迎给作者star~)

Deep-Learning-Book-Chapter-Summaries

This repository provides a summary for each chapter of the Deep Learning book by Ian Goodfellow, Yoshua Bengio and Aaron Courville and attempts to explain some of the concepts in greater detail. Some of the tougher chapters have blog post(s) dedicated to them which can be found on http://medium.com/inveterate-learner.

Chapters

  • Part I: Applied Math and Machine Learning Basics
    • Chapter 2: Linear Algebra [chapter]
    • Chapter 3: Probability and Information Theory [chapter]
    • Chapter 4: Numerical Computation [chapter]
    • Chapter 5: Machine Learning Basics [chapter]

  • Part II: Modern Practical Deep Networks
    • Chapter 6: Deep Feedforward Networks [chapter]
    • Chapter 7: Regularization for Deep Learning [chapter]
    • Chapter 8: Optimization for Training Deep Models [chapter]
    • Chapter 9: Convolutional Networks [chapter]
    • Chapter 10: Sequence Modeling: Recurrent and Recursive Nets [chapter]
    • Chapter 11: Practical Methodology [chapter]
    • Chapter 12: Applications [chapter]

  • Part III: Deep Learning Research

    • Chapter 13: Linear Factor Models [chapter]
    • Chapter 14: Autoencoders [chapter]
    • Chapter 15: Representation Learning [chapter]
    • Chapter 16: Structured Probabilistic Models for Deep Learning [chapter]
    • Chapter 17: Monte Carlo Methods [chapter]
    • Chapter 18: Confronting the Partition Function [chapter]
    • Chapter 19: Approximate Inference [chapter]
    • Chapter 20: Deep Generative Models [chapter]



    笔记的形式是 .ipynb,便于在 Jupyter Notebook 上打开和观看。例如,我们来看一下第二章线性代数的笔记。
    file

可见,Jupyter 笔记不仅包含了知识点的总结,也有相关代码。再来看第九章的卷积网络部分,配备了一些完整的图片处理代码。

import numpy as np
from scipy import signal
from scipy import misc
import matplotlib.pyplot as plt

# %matplotlib inline

img = misc.ascent()
kernel = np.random.randn(5,5)
# kernel = np.array([[0,-10,0,10,0],[-10,-30,0,30,10],[0,-10,0,10,0]])

img = img.astype(np.float32)/255
orig_in = img

offsetx = offsety = 20
shift_in = np.zeros(orig_in.shape)
shift_in[offsetx:,offsety:] = img[:-offsetx,:-offsety]

rot_in = misc.imrotate(img, 90)
scale_in = misc.imresize(orig_in, 1.5)

output1 = signal.convolve2d(orig_in, kernel, mode='same')
output2 = signal.convolve2d(shift_in, kernel, mode='same')
output3 = signal.convolve2d(rot_in, kernel, mode='same')
output4 = signal.convolve2d(scale_in, kernel, mode='same')
fig, axes = plt.subplots(2, 4, figsize=(14, 7))
ax_orig = axes[0,0]
ax_shift = axes[0,1]
ax_rot = axes[0,2]
ax_scale = axes[0,3]

diff_orig = axes[1,0]
diff_shift = axes[1,1]
diff_rot = axes[1,2]
diff_scale = axes[1,3]

ax_orig.imshow(output1, cmap='gray')
ax_orig.set_title('Original')
ax_shift.imshow(output2, cmap='gray')
ax_shift.set_title('Shifted')
ax_rot.imshow(output3, cmap='gray')
ax_rot.set_title('Rotated')
ax_scale.imshow(output4, cmap='gray')
ax_scale.set_title('Scaled')

def shift(arr, offset):
output = np.zeros(arr.shape)
output[offset:, offset:] = arr[:-offset,:-offset]
return output

def rotate(arr, angle):
return misc.imrotate(arr, angle)

def resize(arr, scale):
return misc.imresize(arr, scale)

diff_orig.hist(np.ravel(output1),bins=100)
diff_orig.set_title('Output histogram')
diff_shift.hist(np.ravel(np.abs(output2-shift(output1, 20))),bins=100)
diff_shift.set_title('Shift histogram difference')
diff_rot.hist(np.ravel(np.abs(output3-rotate(output1, 10))),bins=100)
diff_rot.set_title('Rotate histogram difference')
diff_scale.hist(np.ravel(np.abs(output4-resize(output1, 1.5))),bins=100)
diff_scale.set_title('Scale histogram difference')

ax_orig.set_xticks([])
ax_shift.set_xticks([])
ax_rot.set_xticks([])
ax_scale.set_xticks([])

ax_orig.set_yticks([])
ax_shift.set_yticks([])
ax_rot.set_yticks([])
ax_scale.set_yticks([])

plt.tight_layout()
# plt.show()
plt.savefig('images/conv_equivariance.png')



对于池化层的代码示例:

import numpy as np
np.random.seed(101)

from scipy import signal
from scipy import misc
import matplotlib.pyplot as plt
%matplotlib inline

img = misc.ascent()
img = img.astype(np.float32)/255

# The image is more interesting here
orig_in = img[-200:,-300:-100]
offsetx = offsety = 15
shift_in = img[-200-offsetx:-offsetx,-300-offsety:-100-offsety]
kernel1 = np.random.randn(5,5)
kernel2 = np.random.randn(5,5)
kernel3 = np.random.randn(5,5)

def sigmoid(arr):
# Lazy implementation of sigmoid activation
return 1./(1 + np.exp(-arr))

def maxpool(arr, poolsize, stride):
# Lazy looping implementation of maxpool
output_shape = np.floor((np.array(arr.shape)-poolsize)/stride)+1
output_shape = output_shape.astype(np.int32)
output = np.zeros(output_shape)
for x in range(output_shape[0]):
for y in range(output_shape[1]):
output[x,y] = np.max(arr[x*stride:x*stride+poolsize,y*stride:y*stride+poolsize])
return output

output1_1 = signal.convolve2d(orig_in, kernel1, mode='valid')
pool1_1 = maxpool(output1_1, 2, 2)
actv1_1 = sigmoid(pool1_1)
output1_2 = signal.convolve2d(actv1_1, kernel2, mode='valid')
pool1_2 = maxpool(output1_2, 2, 2)
actv1_2 = sigmoid(pool1_2)
output1_3 = signal.convolve2d(actv1_2, kernel3, mode='valid')
pool1_3 = maxpool(output1_3, 2, 2)

output2_1 = signal.convolve2d(shift_in, kernel1, mode='valid')
pool2_1 = maxpool(output2_1, 2, 2)
actv2_1 = sigmoid(pool2_1)
output2_2 = signal.convolve2d(actv2_1, kernel2, mode='valid')
pool2_2 = maxpool(output2_2, 2, 2)
actv2_2 = sigmoid(pool2_2)
output2_3 = signal.convolve2d(actv2_2, kernel3, mode='valid')
pool2_3 = maxpool(output2_3, 2, 2)

fig, axes = plt.subplots(4, 3, figsize=(10, 10))

k1, k2, k3 = axes[0,:]
p1_1, p1_2, p1_3 = axes[1,:]
p2_1, p2_2, p2_3 = axes[2,:]
h1, h2, h3 = axes[3,:]

k1.imshow(kernel1, cmap='gray')
k1.set_title('kernel1')
k2.imshow(kernel2, cmap='gray')
k2.set_title('kernel2')
k3.imshow(kernel3, cmap='gray')
k3.set_title('kernel3')
k1.set_xticks([])
k2.set_xticks([])
k3.set_xticks([])
k1.set_yticks([])
k2.set_yticks([])
k3.set_yticks([])

p1_1.imshow(pool1_1, cmap='gray')
p1_1.set_title('pool1_1')
p1_2.imshow(pool1_2, cmap='gray')
p1_2.set_title('pool1_2')
p1_3.imshow(pool1_3, cmap='gray')
p1_3.set_title('pool1_3')
p1_1.set_xticks([])
p1_2.set_xticks([])
p1_3.set_xticks([])
p1_1.set_yticks([])
p1_2.set_yticks([])
p1_3.set_yticks([])

p2_1.imshow(pool2_1, cmap='gray')
p2_1.set_title('pool2_1')
p2_2.imshow(pool2_2, cmap='gray')
p2_2.set_title('pool2_2')
p2_3.imshow(pool2_3, cmap='gray')
p2_3.set_title('pool2_3')
p2_1.set_xticks([])
p2_2.set_xticks([])
p2_3.set_xticks([])
p2_1.set_yticks([])
p2_2.set_yticks([])
p2_3.set_yticks([])

h1.hist(np.ravel(np.abs(pool1_1-pool2_1)),bins=100)
h1.set_title('Pool 1 diff')
h2.hist(np.ravel(np.abs(pool1_2-pool2_2)),bins=100)
h2.set_title('Pool 2 diff')
h3.hist(np.ravel(np.abs(pool1_3-pool2_3)),bins=100)
h3.set_title('Pool 3 diff')

plt.tight_layout()
# plt.show()
plt.savefig('images/pool_invariance.png')



参考:
红色石头:https://zhuanlan.zhihu.com/p/59999718




推荐阅读
机器学习必备宝典-《统计学习方法》的 python 代码实现及课件
统计学习方法 (笔记, 代码, notebook, 参考文献, Errata)

CV干货 微信公众号: 极市平台
ID: extrememart