On the locality of Java 8 streams in real-time big data applications

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Typical Big Data frameworks do not consider the architecture of the servers that make up the cluster. However, these computers are increasingly heterogeneous and are based on a ccNUMA architecture. In such architectures, main memory access times differ depending on the core on which access is requested. Hence, as well as locality of data access throughout a cluster of servers, locality of memory access within individual servers can have an impact on performance. Java is a commonly-used language for Big Data applications (through the popularity of Hadoop) and the newlyreleased Java 8 introduces streams to simplify data-parallel programming. However, this paper argues that there are no built-in parallel stream sources that can efficiently operate on very large datasets and take data locality into account. This paper details recent work from the JUNIPER project, an EU Framework 7 Project, which is investigating how the Java 8 platform (augmented by the Real-Time Specification for Java) can be used for real-time Big Data applications. JUNIPER introduces architecture-aware stream sources which are suitable for Big Data systems and which preserve locality of data. Our results show that when reading data from disk, thread affinity can seriously degrade the performance of standard Java streams, but JUNIPER's architecture-aware streams maintain their performance.

Original languageEnglish
Title of host publicationJTRES '14
Subtitle of host publicationProceedings of the 12th International Workshop on Java Technologies for Real-time and Embedded Systems
PublisherACM
Pages20-28
Number of pages9
ISBN (Electronic)978-1-4503-2813-5
DOIs
Publication statusPublished - 13 Oct 2014
Event12th International Workshop on Java Technologies for Real-Time and Embedded Systems, JTRES 2014 - Niagara Falls, United States
Duration: 13 Oct 201414 Oct 2014

Publication series

NameACM International Conference Proceeding Series
PublisherACM

Conference

Conference12th International Workshop on Java Technologies for Real-Time and Embedded Systems, JTRES 2014
Country/TerritoryUnited States
CityNiagara Falls
Period13/10/1414/10/14

Cite this