在生成圖片時(shí)，神經(jīng)網(wǎng)絡(luò)是凍結(jié)的，也就是網(wǎng)絡(luò)的權(quán)重不再更新，只需要更新輸入的圖片。常用的預(yù)訓(xùn)練卷積網(wǎng)絡(luò)包括Google的Inception、VGG網(wǎng)絡(luò)和ResNet網(wǎng)絡(luò)等。
?DeepDream簡介
DeepDream是一種藝術(shù)性的圖像修改技術(shù)，主要是基于訓(xùn)練好的卷積神經(jīng)網(wǎng)絡(luò)CNN進(jìn)行圖片的生成。
在生成圖片時(shí)，神經(jīng)網(wǎng)絡(luò)是凍結(jié)的，也就是網(wǎng)絡(luò)的權(quán)重不再更新，只需要更新輸入的圖片。常用的預(yù)訓(xùn)練卷積網(wǎng)絡(luò)包括Google的Inception、VGG網(wǎng)絡(luò)和ResNet網(wǎng)絡(luò)等。
DeePDream的基本步驟：
- 獲取輸入圖片
- 將圖片輸入網(wǎng)絡(luò)，得到所希望可視化的神經(jīng)元的輸出值
- 計(jì)算神經(jīng)元輸出值對(duì)圖片各像素的梯度
- 使用梯度下降不斷更新圖片
重復(fù)第2、3、4步，直到滿足所設(shè)定的條件
下面是使用Keras實(shí)現(xiàn)DeepDream的大致過程：
用Keras實(shí)現(xiàn)DeepDream
獲取測(cè)試圖片
In [1]:
# ---------------
from tensorflow import keras
import matplotlib.pyplot as plt
%matplotlib inline
base_image_path = keras.utils.get_file(
"coast.jpg",
origin="https://img-datasets.s3.amazonaws.com/coast.jpg")
plt.axis("off")
plt.imshow(keras.utils.load_img(base_image_path))
plt.show()
上面是Keras自帶的一張海岸線的圖片。下面就是對(duì)這張圖進(jìn)行變化。
準(zhǔn)備預(yù)訓(xùn)練模型InceptionV3
In [2]:
# 使用Inception V3實(shí)現(xiàn)
from keras.applications import inception_v3
# 使用預(yù)訓(xùn)練的ImageNet權(quán)重來加載模型
model = inception_v3.InceptionV3(weights="imagenet", # 構(gòu)建不包含全連接層的Inceptino
include_top=False)
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/inception_v3/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5
87916544/87910968 [==============================] - 74s 1us/step
87924736/87910968 [==============================] - 74s 1us/step
In [3]:
設(shè)置DeepDream配置
In [4]:
# 層的名稱 + 系數(shù)：該層對(duì)需要最大化的損失的貢獻(xiàn)大小
layer_settings = {"mixed4":1.0,
"mixed5":1.5,
"mixed6":2.0,
"mixed7":2.5}
outputs_dict = dict(
[
(layer.name, layer.output) # 層的名字 + 該層的輸出
for layer in [model.get_layer(name) for name in layer_settings.keys()]
]
)
outputs_dict
Out[4]:
{'mixed4': <KerasTensor: shape=(None, None, None, 768) dtype=float32 (created by layer 'mixed4')>,
'mixed5': <KerasTensor: shape=(None, None, None, 768) dtype=float32 (created by layer 'mixed5')>,
'mixed6': <KerasTensor: shape=(None, None, None, 768) dtype=float32 (created by layer 'mixed6')>,
'mixed7': <KerasTensor: shape=(None, None, None, 768) dtype=float32 (created by layer 'mixed7')>}In [5]:
# 特征提取
feature_extractor = keras.Model(inputs=model.inputs, outputs=outputs_dict)
feature_extractor
Out[5]:
<keras.engine.functional.Functional at 0x15b5ff0d0>
計(jì)算損失
In [6]:
def compute_loss(image):
features = feature_extractor(image) # 特征提取
loss = tf.zeros(shape=()) # 損失初始化
for name in features.keys(): # 遍歷層
coeff = layer_settings[name] # 某個(gè)層的系數(shù)
activation = features[name] # 某個(gè)層的激活函數(shù)
#為了避免出現(xiàn)邊界偽影，損失中僅包含非邊界的像素
loss += coeff * tf.reduce_mean(tf.square(activation[:, 2:-2, 2:-2, :])) # 將該層的L2范數(shù)添加到loss中；
return loss
梯度上升過程
In [7]:
import tensorflow as tf
@tf.function
def gradient_ascent_step(image, lr): # lr--->learning_rate 學(xué)習(xí)率
with tf.GradientTape() as tape:
tape.watch(image)
loss = compute_loss(image) # 調(diào)用計(jì)算損失方法
grads = tape.gradient(loss, image) # 梯度更新
grads = tf.math.l2_normalize(grads)
image += lr * grads
return loss, image
def gradient_ascent_loop(image, iterations, lr, max_loss=None):
for i in range(iterations):
loss, image = gradient_ascent_step(image, lr)
if max_loss is not None and loss > max_loss:
break
print(f"第{i}步的損失值是{loss:.2f}")
return image
圖片生成
np.expand_dims用法（個(gè)人添加）
In [8]:
import numpy as np
array = np.array([[1,2,3],
[4,5,6]]
)
array
Out[8]:
array([[1, 2, 3],
[4, 5, 6]])
In [9]:
Out[9]:
In [10]:
array1 = np.expand_dims(array,axis=0)
array1
Out[10]:
array([[[1, 2, 3],
[4, 5, 6]]])
In [11]:
Out[11]:
In [12]:
array2 = np.expand_dims(array,axis=1)
array2
Out[12]:
array([[[1, 2, 3]],
[[4, 5, 6]]])
In [13]:
Out[13]:
In [14]:
array3 = np.expand_dims(array,axis=-1)
array3
Out[14]:
array([[[1],
[2],
[3]],
[[4],
[5],
[6]]])
In [15]:
Out[15]:
np.clip功能（個(gè)人添加）
np.clip(
array,
min(array),
max(array),
out=None):
In [16]:
array = np.array([1,2,3,4,5,6])
np.clip(array, 2, 5) # 輸出長度和原數(shù)組相同
Out[16]:
array([2, 2, 3, 4, 5, 5])
In [17]:
array = np.arange(18).reshape((6,3))
array
Out[17]:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17]])
In [18]:
Out[18]:
array([[ 5, 5, 5],
[ 5, 5, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 14],
[15, 15, 15]])
參數(shù)設(shè)置
In [19]:
step = 20. # 梯度上升的步長
num_octave = 3 # 運(yùn)行梯度上升的尺度個(gè)數(shù)
octave_scale = 1.4 # 兩個(gè)尺度間的比例大小
iterations = 30 # 在每個(gè)尺度上運(yùn)行梯度上升的步數(shù)
max_loss = 15. # 損失值若大于15，則中斷梯度上升過程
圖片預(yù)處理
In [20]:
import numpy as np
def preprocess_image(image_path): # 預(yù)處理
img = keras.utils.load_img(image_path) # 導(dǎo)入圖片
img = keras.utils.img_to_array(img) # 轉(zhuǎn)成數(shù)組
img = np.expand_dims(img, axis=0) # 增加數(shù)組維度；見上面解釋(x,y) ---->(1,x,y)
img = keras.applications.inception_v3.preprocess_input(img)
return img
def deprocess_image(img): # 圖片壓縮處理
img = img.reshape((img.shape[1], img.shape[2], 3))
img /= 2.0
img += 0.5
img *= 255.
# np.clip：截?cái)喙δ埽ＷC數(shù)組中的取值在0-255之間
img = np.clip(img, 0, 255).astype("uint8")
return img
生成圖片
In [21]:
# step = 20. # 梯度上升的步長
# num_octave = 3 # 運(yùn)行梯度上升的尺度個(gè)數(shù)
# octave_scale = 1.4 # 兩個(gè)尺度間的比例大小
# iterations = 30 # 在每個(gè)尺度上運(yùn)行梯度上升的步數(shù)
# max_loss = 15.0 # 損失值若大于15，則中斷梯度上升過程
original_img = preprocess_image(base_image_path) # 預(yù)處理函數(shù)
original_shape = original_img.shape[1:3]
print(original_img.shape) # 四維圖像
print(original_shape) # 第2和3維度的值
(1, 900, 1200, 3)
(900, 1200)
In [22]:
successive_shapes = [original_shape]
for i in range(1, num_octave):
shape = tuple([int(dim / (octave_scale ** i)) for dim in original_shape])
successive_shapes.append(shape)
successive_shapes = successive_shapes[::-1] # 翻轉(zhuǎn)
shrunk_original_img = tf.image.resize(original_img, successive_shapes[0])
img = tf.identity(original_img)
for i, shape in enumerate(successive_shapes):
print(f"Processing octave {i} with shape {shape}")
# resize
img = tf.image.resize(img, shape)
img = gradient_ascent_loop( # 梯度上升函數(shù)調(diào)用
img,
iteratinotallow=iterations,
lr=step,
max_loss=max_loss
)
# resize
upscaled_shrunk_original_img = tf.image.resize(shrunk_original_img, shape)
same_size_original = tf.image.resize(original_img, shape)
lost_detail = same_size_original - upscaled_shrunk_original_img
img += lost_detail
shrunk_original_img = tf.image.resize(original_img, shape)
keras.utils.save_img("dream.png", deprocess_image(img.numpy()))
結(jié)果為：
Processing octave 0 with shape (459, 612)
第0步的損失值是0.80
第1步的損失值是1.07
第2步的損失值是1.44
第3步的損失值是1.82
......
第26步的損失值是11.44
第27步的損失值是11.72
第28步的損失值是12.03
第29步的損失值是12.49
同時(shí)在本地生成了新圖片，看下效果：
再看一眼原圖：相對(duì)比之下，新圖有點(diǎn)夢(mèng)幻的味道！
