Huge data but small programs: visualization design via multiple embedded DSLs

D.J. Duke; R. Borgo; C. Runciman; M. Wallace

doi:10.1007/978-3-540-92995-6_3

Huge data but small programs: visualization design via multiple embedded DSLs

D.J. Duke, R. Borgo, C. Runciman, M. Wallace

Computer Science

Research output: Contribution to conference › Paper › peer-review

Abstract

Although applications of functional programming are diverse, most examples deal with modest amounts of data -- no more than a few megabytes. This paper describes how Haskell has been used to address a challenging visualization problem, involving 200 time steps from an astrophysics simulation of early star formation. Each individual step involves nearly 40 million samples, and uncompressed the complete dataset is nearly a terabyte. Our solution makes novel and extensive use of domain specific languages to specify data resources, rendering abstractions, and most significantly, the desired visualization. The result is a powerful framework for multi-field visualization, including the use of animation to explore the evolution of data over time. This approach represents a significant departure from standard practices in visualization, and has applications well beyond the original problem. That our solution consists of less than 4.5K lines of code is itself a notable result. This paper motivates and describes the overall architecture of our solution, and technical features of the DSLs that are used in place of the traditional visualization pipeline. We conclude with thoughts on how functional technologies may find even broader use within other branches of visualization.

Original language	English
Pages	31-45
Number of pages	14
DOIs	https://doi.org/10.1007/978-3-540-92995-6_3
Publication status	Published - 2009

Bibliographical note

Access to Document

10.1007/978-3-540-92995-6_3

http://dx.doi.org/10.1007/978-3-540-92995-6_3

Cite this

@conference{d73b148a78d740a58dca2ee30fec4d1e,

title = "Huge data but small programs: visualization design via multiple embedded DSLs",

abstract = "Although applications of functional programming are diverse, most examples deal with modest amounts of data -- no more than a few megabytes. This paper describes how Haskell has been used to address a challenging visualization problem, involving 200 time steps from an astrophysics simulation of early star formation. Each individual step involves nearly 40 million samples, and uncompressed the complete dataset is nearly a terabyte. Our solution makes novel and extensive use of domain specific languages to specify data resources, rendering abstractions, and most significantly, the desired visualization. The result is a powerful framework for multi-field visualization, including the use of animation to explore the evolution of data over time. This approach represents a significant departure from standard practices in visualization, and has applications well beyond the original problem. That our solution consists of less than 4.5K lines of code is itself a notable result. This paper motivates and describes the overall architecture of our solution, and technical features of the DSLs that are used in place of the traditional visualization pipeline. We conclude with thoughts on how functional technologies may find even broader use within other branches of visualization.",

author = "D.J. Duke and R. Borgo and C. Runciman and M. Wallace",

note = "{\textcopyright} 2009 Springer. This is an author produced version of a paper published in 'Lecture Notes in Computer Science'. Volume 5418. Uploaded in accordance with the publisher's self-archiving policy. ",

year = "2009",

doi = "10.1007/978-3-540-92995-6_3",

language = "English",

pages = "31--45",

}

TY - CONF

T1 - Huge data but small programs: visualization design via multiple embedded DSLs

AU - Duke, D.J.

AU - Borgo, R.

AU - Runciman, C.

AU - Wallace, M.

PY - 2009

Y1 - 2009

N2 - Although applications of functional programming are diverse, most examples deal with modest amounts of data -- no more than a few megabytes. This paper describes how Haskell has been used to address a challenging visualization problem, involving 200 time steps from an astrophysics simulation of early star formation. Each individual step involves nearly 40 million samples, and uncompressed the complete dataset is nearly a terabyte. Our solution makes novel and extensive use of domain specific languages to specify data resources, rendering abstractions, and most significantly, the desired visualization. The result is a powerful framework for multi-field visualization, including the use of animation to explore the evolution of data over time. This approach represents a significant departure from standard practices in visualization, and has applications well beyond the original problem. That our solution consists of less than 4.5K lines of code is itself a notable result. This paper motivates and describes the overall architecture of our solution, and technical features of the DSLs that are used in place of the traditional visualization pipeline. We conclude with thoughts on how functional technologies may find even broader use within other branches of visualization.

AB - Although applications of functional programming are diverse, most examples deal with modest amounts of data -- no more than a few megabytes. This paper describes how Haskell has been used to address a challenging visualization problem, involving 200 time steps from an astrophysics simulation of early star formation. Each individual step involves nearly 40 million samples, and uncompressed the complete dataset is nearly a terabyte. Our solution makes novel and extensive use of domain specific languages to specify data resources, rendering abstractions, and most significantly, the desired visualization. The result is a powerful framework for multi-field visualization, including the use of animation to explore the evolution of data over time. This approach represents a significant departure from standard practices in visualization, and has applications well beyond the original problem. That our solution consists of less than 4.5K lines of code is itself a notable result. This paper motivates and describes the overall architecture of our solution, and technical features of the DSLs that are used in place of the traditional visualization pipeline. We conclude with thoughts on how functional technologies may find even broader use within other branches of visualization.

UR - http://www.scopus.com/inward/record.url?scp=70350656154&partnerID=8YFLogxK

U2 - 10.1007/978-3-540-92995-6_3

DO - 10.1007/978-3-540-92995-6_3

M3 - Paper

SP - 31

EP - 45

ER -

Huge data but small programs: visualization design via multiple embedded DSLs

Abstract

Bibliographical note

Access to Document

Other files and links

Cite this