The basic tutorial of Keras for R is provided by keras here, which simple and fast to get started. But very soon, I realize this basic tutorial won’t meet my need any more, when I want to train larger dataset. And this is the tutorial I’m going to discuss about keras generators, callbacks and tensorboard.
Keras Installation
If you haven’t got your keras in R, just follow the steps at below:
devtools::install_github("rstudio/keras")
library(keras)
install_keras()
MNIST handwriting recognition
This is the code copied from the basic keras tutorial for MNIST handwriting recognition. I’m going to change based on this.
library(keras)
# donwnload dataset
mnist x_train y_train x_test y_test
# reshape
x_train x_test # rescale
x_train x_test
# one-hot encoding the label
y_train y_test
# define the computational graph
model %
layer_dense(units = 256, activation = 'relu', input_shape = c(784)) %>%
layer_dropout(rate = 0.4) %>%
layer_dense(units = 128, activation = 'relu') %>%
layer_dropout(rate = 0.3) %>%
layer_dense(units = 10, activation = 'softmax')
# compile model
model %>% compile(
loss = 'categorical_crossentropy',
optimizer = optimizer_rmsprop(),
metrics = c('accuracy'))
# model training
history % fit(
x_train, y_train,
epochs = 30, batch_size = 128,
validation_split = 0.2
)
Generator
Generators are useful tools to build the training pipelines, such as functions like data sampling and preprocessing before feeding into model training.
A particular advantage of generator is the case that training data is too large to fit into memory. We can prepare the training data files in parts and use generator to read data, preprocess and eventually feed into model for training.generator in R is to define a function within a function, which is slightly different with python and syntax is record in this website.
In this example, we can define a simple sampling generator, which serves as pre-processing pipelines – linking raw data to our expected data format by doing sampling, re-scaling, re-shaping and one-hot encoding.
sampling_generator function() {
# sample a batch
inSample x.sample # re-scale
x.sample # re-shape
x.sample # one-hot encoding
y.sample # return format (feature, label)
return(list(x.sample, y.sample))
}
}
Callback
Callbacks are used to control the training process, such as saving model, early stop and reducing learning rate.
The full list of callbacks are show here.
callbacks # record training history in csv file callback_csv_logger(format(Sys.time(),'model/log/mnist_%Y%m%d_%H%M.csv')), # early stop if eval_loss not improving for 5 epoches callback_early_stopping(patience = 5), # logdir for tensorboard callback_tensorboard(format(Sys.time(),'model/tensorboard/mnist_%Y%m%d_%H%M')))
Tensorboard
Tensorboard is the UI view to compare different models as well as the model structure visualization. To launch your tensorboard, type this in your terminal:
tensorboard --logdir=/model/tensorboard/
after that, tensorboard will be available at http://localhost:6006.
We will notice that, there are no data captured in tensorboard, because we haven’t run any model pointing to specified tensorboard directory.
Training with generators & callbacks
We just need to specify the callbacks in the fit_generator and set the steps_per_epoch in each epoch, which is usually number of rows / batch_size.
valid.set hist % fit_generator( sampling_generator(x_train,y_train,batch_size = 128), callbacks = callbacks, steps_per_epoch = 500, epochs = 50, verbose = 1, validation_data = valid.set, validation_steps = 1)
We now can see the our training history compared with keras example training. Our training stops early at epoch-11, because eval_loss becomes stagnant.
The code of keras_model is here.
The code of my_model is here.
Chaoran's Data Story 