Overview / Usage

Hyperparameter search is one of the key areas in developing a good deep neural network architecture and training scheme. This project aims to streamline the searching of hyperparameters while continuing training in the mean time. The way the project works is: In the beginning, several instances of training is commenced with randomly selected hyperparameters. After a predefined amount of training is completed, the trained networks are evaluated on validation data. For the next iteration, the new hyperparameters are selected based on the performance of the previous ones. The training is restarted with the new set of hyperparameters; however, the weights of the best performing networks are used in the next iterations if the architectures dictated by the hyperparameters allow so. In this way, the search for the best hyperparameters are performed simultaneously with the training.

Methodology / Approach

The project is created using Python and Intel optimized version of TensorFlow. Intel AI DevCloud is the platform used for simultaneous hyperparameter search and training.

Technologies Used

Intel Optimized TensorFlow
Intel AI DevCloud