TY - GEN
T1 - An Autonomic Workflow Performance Manager for Weather Research and Forecast Workflows
AU - Gu, Shuqing
AU - Yao, Likai
AU - Tunc, Cihan
AU - Akoglu, Ali
AU - Hariri, Salim
AU - Ritchie, Elizabeth
N1 - Funding Information:
This work is partly supported by the Air Force Office of Scientific Research (AFOSR) Dynamic Data-Driven Application Systems (DDDAS) award number FA95550-12-1-0241, National Science Foundation research projects NSF IIP-0758579, SES-1314631 and DUE-1303362, and Thomson Reuters in the framework of the Partner University Fund (PUF) project (PUF is a program of the French Embassy in the United States and the FACE Foundation and is supported by American donors and the French government).
Publisher Copyright:
© 2016 IEEE.
PY - 2016/12/5
Y1 - 2016/12/5
N2 - Parameter selection is a critical task in scientific workflows in order to maintain the accuracy of the simulation in an environment where physical conditions change dynamically such as in the case of weather research and forecast (WRF) simulations. Considering the large number of simulation parameters, the size of the configuration search space becomes prohibitive for rapidly evaluating and identifying the parameter configuration that leads to most accurate prediction. We present an autonomic workflow performance manager that can automatically manage model initialization and workflow execution for a given resource allocation. We model the configuration selection of WRF workflow using Apache Storm and automate the process of model initialization, configuration and execution. We reduce the timescale of the configuration search workflow by a factor of 10x by using 20 threads when compared to serial workflow execution as it is typically performed by domain scientists.
AB - Parameter selection is a critical task in scientific workflows in order to maintain the accuracy of the simulation in an environment where physical conditions change dynamically such as in the case of weather research and forecast (WRF) simulations. Considering the large number of simulation parameters, the size of the configuration search space becomes prohibitive for rapidly evaluating and identifying the parameter configuration that leads to most accurate prediction. We present an autonomic workflow performance manager that can automatically manage model initialization and workflow execution for a given resource allocation. We model the configuration selection of WRF workflow using Apache Storm and automate the process of model initialization, configuration and execution. We reduce the timescale of the configuration search workflow by a factor of 10x by using 20 threads when compared to serial workflow execution as it is typically performed by domain scientists.
KW - Apache Storm
KW - Automated Workflow Management
KW - Model Parameter Sweeping
KW - Weather Research and Forecast
UR - http://www.scopus.com/inward/record.url?scp=85010303869&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85010303869&partnerID=8YFLogxK
U2 - 10.1109/ICCAC.2016.23
DO - 10.1109/ICCAC.2016.23
M3 - Conference contribution
AN - SCOPUS:85010303869
T3 - Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016
SP - 111
EP - 114
BT - Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016
A2 - Gupta, Indranil
A2 - Diao, Yixin
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016
Y2 - 12 September 2016 through 16 September 2016
ER -