TensorFlow 文檔有以下示例，可以說(shuō)明當(dāng)訓(xùn)練集太大而無(wú)法放入內(nèi)存時(shí)，如何創(chuàng)建批量生成器以將訓(xùn)練集批量提供給模型：from skimage.io import imreadfrom skimage.transform import resizeimport tensorflow as tfimport numpy as npimport math# Here, `x_set` is list of path to the images# and `y_set` are the associated classes.class CIFAR10Sequence(tf.keras.utils.Sequence): def __init__(self, x_set, y_set, batch_size): self.x, self.y = x_set, y_set self.batch_size = batch_size def __len__(self): return math.ceil(len(self.x) / self.batch_size) def __getitem__(self, idx): batch_x = self.x[idx * self.batch_size:(idx + 1) * self.batch_size] batch_y = self.y[idx * self.batch_size:(idx + 1) * self.batch_size] return np.array([ resize(imread(file_name), (200, 200)) for file_name in batch_x]), np.array(batch_y)我的目的是通過(guò)將每個(gè)圖像旋轉(zhuǎn) 3 倍 90° 來(lái)進(jìn)一步增加訓(xùn)練集的多樣性。在訓(xùn)練過(guò)程的每個(gè) Epoch 中，模型將首先輸入“0° 訓(xùn)練集”，然后分別輸入 90°、180° 和 270° 旋轉(zhuǎn)集。如何修改前面的代碼以在CIFAR10Sequence()數(shù)據(jù)生成器中執(zhí)行此操作？請(qǐng)不要使用tf.keras.preprocessing.image.ImageDataGenerator()，以免答案失去對(duì)其他類型不同性質(zhì)的類似問(wèn)題的普遍性。注意：這個(gè)想法是在模型被輸入時(shí)“實(shí)時(shí)”創(chuàng)建新數(shù)據(jù)，而不是（提前）創(chuàng)建并在磁盤(pán)上存儲(chǔ)一個(gè)比稍后使用的原始訓(xùn)練集更大的新的增強(qiáng)訓(xùn)練集（也在批次）在模型的訓(xùn)練過(guò)程中。
1 回答
米脂
TA貢獻(xiàn)1836條經(jīng)驗(yàn) 獲得超3個(gè)贊
使用自定義Callback并掛鉤到on_epoch_end. 每個(gè)紀(jì)元結(jié)束后更改數(shù)據(jù)迭代器對(duì)象的角度。
示例（內(nèi)聯(lián)記錄）
from skimage.io import imread
from skimage.transform import resize, rotate
import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from keras.utils import Sequence
from keras.models import Sequential
from keras.layers import Conv2D, Activation, Flatten, Dense
# Model architecture (dummy)
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=(15, 15, 4)))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
# Data iterator
class CIFAR10Sequence(Sequence):
def __init__(self, filenames, labels, batch_size):
self.filenames, self.labels = filenames, labels
self.batch_size = batch_size
self.angles = [0,90,180,270]
self.current_angle_idx = 0
# Method to loop throught the available angles
def change_angle(self):
self.current_angle_idx += 1
if self.current_angle_idx >= len(self.angles):
self.current_angle_idx = 0
def __len__(self):
return int(np.ceil(len(self.filenames) / float(self.batch_size)))
# read, resize and rotate the image and return a batch of images
def __getitem__(self, idx):
angle = self.angles[self.current_angle_idx]
print (f"Rotating Angle: {angle}")
batch_x = self.filenames[idx * self.batch_size:(idx + 1) * self.batch_size]
batch_y = self.labels[idx * self.batch_size:(idx + 1) * self.batch_size]
return np.array([
rotate(resize(imread(filename), (15, 15)), angle)
for filename in batch_x]), np.array(batch_y)
# Custom call back to hook into on epoch end
class CustomCallback(keras.callbacks.Callback):
def __init__(self, sequence):
self.sequence = sequence
# after end of each epoch change the rotation for next epoch
def on_epoch_end(self, epoch, logs=None):
self.sequence.change_angle()
# Create data reader
sequence = CIFAR10Sequence(["f1.PNG"]*10, [0, 1]*5, 8)
# fit the model and hook in the custom call back
model.fit(sequence, epochs=10, callbacks=[CustomCallback(sequence)])
輸出：
Rotating Angle: 0
Epoch 1/10
Rotating Angle: 0
Rotating Angle: 0
2/2 [==============================] - 2s 755ms/step - loss: 1.0153 - accuracy: 0.5000
Epoch 2/10
Rotating Angle: 90
Rotating Angle: 90
2/2 [==============================] - 0s 190ms/step - loss: 0.6975 - accuracy: 0.5000
Epoch 3/10
Rotating Angle: 180
Rotating Angle: 180
2/2 [==============================] - 2s 772ms/step - loss: 0.6931 - accuracy: 0.5000
Epoch 4/10
Rotating Angle: 270
Rotating Angle: 270
2/2 [==============================] - 0s 197ms/step - loss: 0.6931 - accuracy: 0.5000
Epoch 5/10
Rotating Angle: 0
Rotating Angle: 0
2/2 [==============================] - 0s 189ms/step - loss: 0.6931 - accuracy: 0.5000
Epoch 6/10
Rotating Angle: 90
Rotating Angle: 90
2/2 [==============================] - 2s 757ms/step - loss: 0.6932 - accuracy: 0.5000
Epoch 7/10
Rotating Angle: 180
Rotating Angle: 180
2/2 [==============================] - 2s 757ms/step - loss: 0.6931 - accuracy: 0.5000
Epoch 8/10
Rotating Angle: 270
Rotating Angle: 270
2/2 [==============================] - 2s 761ms/step - loss: 0.6932 - accuracy: 0.5000
Epoch 9/10
Rotating Angle: 0
Rotating Angle: 0
2/2 [==============================] - 1s 744ms/step - loss: 0.6932 - accuracy: 0.5000
Epoch 10/10
Rotating Angle: 90
Rotating Angle: 90
2/2 [==============================] - 0s 192ms/step - loss: 0.6931 - accuracy: 0.5000
<tensorflow.python.keras.callbacks.History at 0x7fcbdf8bcdd8>
添加回答
舉報(bào)
0/150
提交
取消