HIERARCHICAL DATAFLOW MODEL WITH AUTOMATED FILE MANAGEMENT FOR ENGINEERING AND SCIENTIFIC APPLICATIONS
Abstract
Solving modern scientific and engineering problems typically implies using multiple task-specific software applications and often a complex sequence of computations must be performed. Adopted approach to achieve the required level of automation is to use one of the many available scientific and engineering workflow systems, which can be based on different workflow models. This paper introduces a workflow model targeted to provide natural automation and distributed execution of complex iterative computation processes, where the calculation chain contains multiple task-specific software applications which exchange files during the process.
The proposed workflow model addresses a wide range of applications and targets complex cases when a single iteration of a top-level process may contain multiple nested execution loops. Typical requirements to process automation are considered as well: execution isolation, data re-use and caching, parallel execution, data provenance tracking.
About the Authors
A. M. Nazarenko
DATADVANCE
Russian Federation
Moscow
Russian Federation
Moscow
N. O. Perestoronin
DATADVANCE
Russian Federation
Moscow
Russian Federation
Moscow
A. A. Prokhorov
DATADVANCE;
Institute for information transmission problems RAS (Kharkevich Institute)
Russian Federation
Moscow
Russian Federation
Moscow
References
1. Afanasiev A., Sukhoroslov O., Voloshinov V. MathCloud: Publication and Reuse of Scientific Applications as RESTful Web Services. In 12th International Conference "Parallel Computing Technologies" (PaCT 2013), 2013, vol. 1662, pp. 394–408. Springer.
2. Altintas I., Berkley C., Jaeger E., Jones M., Ludascher B., Mock S. Kepler: an extensible system for design and execution of scientific workflows. Proceedings. 16th International Conference on Scientific and Statistical Database Management, 2004, (August 2015):423–424, 2004.
3. Burnaev E., Gubarev F., Morozov S., Prokhorov A., Khominich D. PSE/MACROS: Software Environment for Process Integration, Data Mining and Design Optimization. The internet representation of scientific editions of FSUE "VIMI" (The All-Russian Research Institute for Interindustry Information - a Federal Informational and Analytical Center of the Defense Industry, a Federal State Unitary Enterprise), (4):41–50, 2013.
4. Churches D., Gombas G., Harrison A., Maassen J., Robinson C., Shields M., Taylor I., Wang I. Programming scientific and distributed workflow with Triana services. Concurrency Computation Practice and Experience, 18(10):1021–1037, 2006.
5. Elmroth E., Hernández F., Tordsson J. Three fundamental dimensions of scientific workflow interoperability: Model of computation, language, and execution environment. Future Generation Computer Systems, 26(2):245–256, February 2010.
6. Johnson G. LabVIEW graphical programming. Tata McGraw-Hill Education, 1997.
7. Johnston W., Hanna J., Millar R. Advances in dataflow programming languages. ACM Computing Surveys, 36(1):1–34, March 2004.
8. Malone B., Papay M. Modelcenter: an integration environment for simulation based design. In Simulation Interoperability Workshop, 1999.
9. Martins J., Lambe A. Multidisciplinary Design Optimization: A Survey of Architectures. AIAA Journal, 51(9):2049–2075, 2013.
10. Seider D., Fischer P.M., Litz M., Schreiber A., Gerndt A. Open Source Software Framework for Applications in Aeronautics and Space. In 2012 IEEE Aerospace Conference, pages 1–11, 2012.
11. Wikipedia. Modefrontier — wikipedia, the free encyclopedia, 2015. [Online; accessed 15 August 2015].
12. Wikipedia. Optimus platform — wikipedia, the free encyclopedia, 2015. [Online; accessed 15 August 2015].
13. Wolstencroft K., Haines R., Fellows D., Williams A., Withers D., Owen S., SoilandReyes S., Dunlop I., Nenadic A., Fisher P., Bhagat J., Belhajjame K., Bacall F., Hardisty A., de la Hidalga A., Vargas M., Sufi S., Goble C. The Taverna workflow suite: designing and executing workflows of Web Services on the desktop, web or in the cloud. Nucleic acids research, 41(Web Server issue), 2013.
14. Zhao Y. A Model of Computation with Push and Pull Processing. Master’s report, technical memorandum ucb/erl m03/51, University of California at Berkeley, 2003.
Review
For citations:
Nazarenko A.M., Perestoronin N.O., Prokhorov A.A. HIERARCHICAL DATAFLOW MODEL WITH AUTOMATED FILE MANAGEMENT FOR ENGINEERING AND SCIENTIFIC APPLICATIONS. Civil Aviation High Technologies. 2016;19(5):161-172. (In Russ.)
Views:
733
Email this article 
