Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources

In the current and future industry and society, there will be an increasing number of systems storing and processing large amounts of data. This is the next frontier for innovation, competition and productivity with ongoing large initiatives both in the EU and the US. Areas where processing of large amounts of unstructured data is applied include medicine, meteorology, genomics, connectomics, physics, biology, environmental research, Internet search, finance and governmental informatics. Finding structure and information in these data sets does obvious demand massive computational resources, but it imposes also huge demands on I/O and communication in order to yield answers to nearly arbitrary questions in subjective real-time. If these questions are posed by machines, or if a human’s question covers interconnected data sets, there are furthermore dependencies between processing steps. In a large computing cluster like a grid or a cloud, an important challenge is thus to execute the many concurrent computations in an efficient and dynamic manner where available resources, processing, communication, dependencies and timeliness must be taken into account when mapping tasks to processing cores.

As such, the aim of the EONS research project is to perform basic research in the area of system and tools support for both, parallel programming and parallel processing, in the context of future distributed large-scale heterogeneous systems. EONS will develop concepts and mechanisms that enable the development of software for these next-generation big-data applications. This is achieved by solving fundamental challenges for the dispatching, division, scheduling and identification of tasks that can run correctly in parallel in a shared distributed system of heterogeneous computing resources in complex topologies.

The following points are investigated by EONS: