JX | The Cooperative Computing Lab

The JX Expression Language is an extension of the JSON data description language. It combines familiar expression operators, function calls, external data, and ordinary JSON contents to yield a powerful data querying and manipulation language. JX is used throughout the CCTools to manage and query unstructured data.

For example, JX expressions can be used to describe jobs in a workflow:

{
    "command" : "collect.exe"
    "inputs" :  [ "input."+i+".txt" ]
    "outputs" : [ "output."+i+".txt" ]
} for i in range(1,100)

Or to write LINQ-style queries on remote data:

fetch(url).select(type=="wq_master").select(tasks_submitted>100).project([name,tasks_running+tasks_waiting])

Related Publications

Flexible Partitioning of Scientific Workflows Using the JX Workflow Language

Tim Shaffer, Nathaniel Kremer-Herman, and Douglas Thain

In Practice and Experience in Advanced Research Computing (PEARC), 2019

doi: 10.1145/3332186.3338100

DOI Bib PDF

@inproceedings{jx-pearc19,
  author = {Shaffer, Tim and Kremer-Herman, Nathaniel and Thain, Douglas},
  title = {{Flexible Partitioning of Scientific Workflows Using the JX Workflow Language}},
  booktitle = {{Practice and Experience in Advanced Research Computing (PEARC)}},
  year = {2019},
  note = {{doi: 10.1145/3332186.3338100}},
  doi = {10.1145/3332186.3338100},
  cclpaperid = {961},
  keywords = {makeflow, jx},
}

Poster: A First Look at the JX Workflow Language

Tim Shaffer, Kyle M.D. Sweeney, Nathaniel Kremer-Herman, and Douglas Thain

In IEEE International Conference on e-Science, 2018

doi: 10.1109/eScience.2018.00094

DOI Bib PDF

@inproceedings{jx-escience18,
  author = {Shaffer, Tim and Sweeney, Kyle M.D. and Kremer-Herman, Nathaniel and Thain, Douglas},
  title = {{Poster: A First Look at the JX Workflow Language}},
  booktitle = {{IEEE International Conference on e-Science}},
  year = {2018},
  note = {{doi: 10.1109/eScience.2018.00094}},
  doi = {10.1109/eScience.2018.00094},
  cclpaperid = {954},
  keywords = {makeflow, jx},
}