Hyperparameter Optimization for deep-learning models predictive of anti-cancer drug responses
Rylie Weaver, Rohan Gnanaolivu, Rajeev Jain, Chen Wang, Oleksandyr Narykov
- Abstract
- Artificial intelligence (AI) and deep learning (DL) have emerged as powerful tools for analyzing complex molecular and genetic data, particularly in cancer therapeutic research. Every DL model has hyper-parameters that control the high-level learning behaviors and neural network (NN) architectures, consequently influencing model’s performance. Despite central roles of hyper-parameters to DL models, the selection of best hyper-parameters is very computationally costly, as each set of hyper-parameter configuration necessitates the training and evaluation of an individual DL instance. To fully exploit DL’s predictive potentials, this study is designed to examine a hyper-parameter optimization (HPO) framework for DL models in predicting anti-cancer drug response, consisting of the process of finding the optimal values of DL hyper-parameters that maximizes performance while minimizing computational cost compared to a random search.
The inputs of the developed HPO framework include DL models containerized in a singularity instance, specified hyper-parameters, as well as value ranges for constraining optimization searching space. Within the HPO framework, we devised a genetic algorithm (GA), which firstly initializes a population of hyper-parameter configurations inside the hyperparameter space, and then simulates evolution of configurations through iterations of mutation, mating, and selection. According to a fitness function corresponding to the DL model's validation loss, we designed the entire GA process to iteratively improve configurations and choose the best HPO solution over multiple evolution generations.
As the use-case evaluation, our developed HPO framework is applied to DrugCell and DeepTTA DL models. DrugCell is an interpretable method that constructs a visible NN following gene ontology relationships and predicts anti-cancer drug responses by incorporating biochemical properties and structures of compounds. DeepTTA’s focus is a creation of robust drug representations, which is achieved via transformer architecture applied to Explainable Substructure Partition Fingerprints (ESPF). This method also utilizes gene expression data from tumors and creates a separate embedding for biological samples. Then both drug and tumor representations are used for regression purposes. We examined a set of critical hyper-parameters of these models, which are learning rate, batch size, optimizer, and dropout using training-testing-validation schema on the CCLE and GDSC cancer cell line / drug datasets. We achieved improvement in the concordance of predicted CCL-drug response versus measured ones with HPO compared to the default settings and a random search in both models. Through the landscape analysis of HPO results, we identified that the learning rate and batch size were the most influential hyper-parameter for performance out of the ones chosen. Our study demonstrates the importance of HPO and robustness of GA in finding optimal hyperparameters for anti-cancer drug research, as well as some hyperparameter configuration guidance for researchers with similar models. Moreover, after improvement with HPO, the interpretability and effectiveness of our models give further insights into the effectiveness of cancer drugs and the mechanisms behind cancer-drug prediction. - Presented by
- Rylie Weaver <rylieweaver9@gmail.com>
- Institution
- Argonne National Laboratory, Mayo Clinic
- Hashtags
- #machinelearning #HyperparameterOptimization #cancer #deeplearing
Chat with PresenterAvailable November 12th 3:00-4:00 MST
