Ecologist-centered tools for easily describing and running agent-based simulations, executable either locally with parallelization or out to distribution across hundreds of machines.
Focused on vegetation, this platform runs on JVM or WebAssembly (via TeaVM). It allows for fluent modeling of organisms, disturbances, and management interventions. This open source project also supports stochastic mechanics and the use of external resources like outside geotiffs or COGs. Using a highly readable domain specific language crafted just for ecologists, Josh makes it easy to quickly describe ecological systems and run those simulations with highly performant computational machinery with minimal fuss in installation. This suite of tools also allows for running simulations in the browser, local performant parallelized execution to take advantage of a single machine's resources, and large scale distributed processing all without changing a single line of code.
If you have a browser, you can use these tools without any installation required. When you are ready to scale up, this software can execute either directly on your machine, in a containerized environment, across potentially many machines via JoshCloud, or on your own infrastructure.
Simply send your browser to editor.joshsim.org. There, you can build simulations using the Josh Language all without installing anything new on your machine. Your work is completely private and your simulations never leave your computer unless you ask them to. Computation runs right within your browser using WebAssembly! However, when you are ready to push forward, you can either download your work and continue using your local computer or take advantage of our infrastructure to run your simulations across many machines all with the click of a button!
For distributed usage across many machines with absolutely zero setup, the project maintains JoshCloud which provides access via an API key. This service is currently provided to trusted partners in preview. Simply provide your API via the web-based editor running on your machine or at editor.joshsim.org.
The ability to use local tools to submit simulations that will run across our infrastructure via the CLI are also coming soon.
The easiest way to get started locally is to simply get yourself a copy of open source Java and the latest release of Josh. Then, fire up the command line. First, write your code to .josh files. Then, execute locally like so:
$ java -jar joshsim.jar run simulation.josh
While COGs, geotiffs, and netCDF files can be provided directly, the preferred approach is to provide a jshd file which preprocesses these geospatial inputs for speed:
$ java -jar joshsim.jar preprocess simulation.josh MySimulation data.nc variable units output.jshd
$ java -jar joshsim.jar run simulation.josh --data output.jshd
Available commands include:
validatefor checking syntax:$ java -jar joshsim.jar validate simulation.joshrunfor executing simulations:$ java -jar joshsim.jar run simulation.joshserverfor starting a local web interface:$ java -jar joshsim.jar serverpreprocessfor creating optimized jshd files:$ java -jar joshsim.jar preprocess simulation.josh MySimulation data.nc variable units output.jshd
Run the jar without any command specified to get further help documentation.
You can run the local UI through joshsim. Execute:
$ java -jar joshsim.jar server
This will start a local web server which makes the UI available via your browser where you can work in private. The local server supports a sandbox mode that limits access to only the code and jshd files provided, with no network access otherwise.
Containerization through Docker and Development Containers can help you move your work from one computer to the next with ease. Please see our Dockerfile and devcontainer.json.
Distributing workloads is easy. Simply deploy either our jar file or container to a serverless solution like Lambda / CloudRun or submit on your own cluster via Kubernetes.
Distribution follows a leader / worker node architecture. A leader node will make a request to worker nodes through a URL in which API keys are passed. You may use a load balancer to distribute requests across workers. Note that the server runs both the leader and worker endpoints. Therefore, if desired, nodes may operate in both capacities. In other words, requests to workers and leaders can be at the same URL.
The Josh server supports HTTP2 for efficient communication. You can send Josh code and jshd precomputed data over the network in the request body. Results are returned via HTTP2 streaming back to the user client.
Here's an example of running the server with distribution:
$ java -jar joshsim.jar server --worker-url your-server-url.com/runReplicate
See also our example Dockerfile.
When running in server mode, some additional security mechanisms are in place.
Josh includes a sandbox that limits access to only code and jshd files (with no other network access) when running in server mode. This ensures security and privacy. Note that the JoshCloud community infrastructure offering also runs with the sandbox. If the sandbox is not desired, use local execution from the command line.
The server command will look for API keys in the JOSH_API_KEYS environment variable. If it is empty or not specified, all requests are allowed. If a value is given, a comma separated list of valid API keys is expected. If this feature is in use, requests without a valid API key will be rejected.
Josh uses a domain-specific language designed specifically for ecological modeling. Here's a basic hello world example to get you started:
start unit year
alias years
alias yr
alias yrs
end unit
start simulation Main
grid.size = 1000 m
grid.low = 33.7 degrees latitude, -115.4 degrees longitude
grid.high = 34.0 degrees latitude, -116.4 degrees longitude
grid.patch = "Default"
steps.low = 0 count
steps.high = 10 count
exportFiles.patch = "memory://editor/patches"
end simulation
start patch Default
ForeverTree.init = create 10 count of ForeverTree
export.averageAge.step = mean(ForeverTree.age)
export.averageHeight.step = mean(ForeverTree.height)
end patch
start organism ForeverTree
age.init = 0 year
age.step = prior.age + 1 year
height.init = 0 meters
height.step = prior.height + sample uniform from 0 meters to 1 meters
end organism
This example creates a simple simulation with a "ForeverTree" organism - trees that grow in height over time. Each tree starts at age 0 and height 0, then grows one year older and gains random height each timestep.
The exportFiles.patch = "memory://editor/patches" line configures exports to be sent to the IDE's memory for interactive visualization rather than being saved to disk files. This memory export target allows users to explore and visualize simulation results directly within the web-based editor without creating output files.
For more comprehensive tutorials and guides, visit https://joshsim.org/guide.html.
The Python interface (joshpy) is coming soon but not yet released.
We have options available for different kinds of development workflows.
We provide a standard Dockerfile and devcontainer.json. Replit files are also provided (.replit and replit.nix).
In addition to a development container we provide instructions for general local setup to aid various environments.
- Get yourself a copy of open source Java.
- Install the Gradle build system.
- Build the project with
gradle fatJar.
This will produce your home-built copy of Josh at build/libs/joshsim-fat.jar. If you want to develop for the web interface, also install vanilla JS production dependencies with editor/third_party/install_deps.sh. This also requires the WASM build via bash editor/war/get_from_jar.sh from the repo root. This can be packaged into the jar itself with gradle fatJar afterwards.
For Josh itself, please use the Google Java Style Guide. We aim for approximately 80% test coverage and all non-test public members having JavaDoc.
For the web interface, please use the Google JavaScript Style Guide with all public members having JSDoc. Note that we use vanilla JavaScript which must be able to run directly in browser so code is not run through webpack or similar prior to deployment. Very limited production dependencies are simply included via minified JS. We do not consider production use of CDNs to be acceptable for privacy reasons.
We require that our automated tests and checks pass prior to merging commits.
We offer tests at two levels. First, the Java source behind Josh can be tested via JUnit and Gradle:
$ gradle test
Josh scripts can also be validated for syntax errors and tested if unit tests are provided:
$ java -jar joshsim.jar validate script.josh
$ java -jar joshsim.jar test script.josh
To check the default examples, execute bash examples/validate.sh and bash examples/test.sh.
Deployment instructions are provided both inside and outside of CI / CD.
To deploy a new version of Josh, simply merge to main to generate and deploy a new joshsim.jar and web editor resources (including joshsim.wasm) via SFTP and GitHub Actions.
If deploying outside of our CI / CD systems, only the jar is required to run Josh.
$ gradle fatJar
Simply use the output found under build/libs/joshsim-fat.jar. If also deploying the web editor:
$ cd web; bash support/package.sh
Simply serve static files from web/build like so:
$ cd web/build; python -m http.server
Any static hosting solution such as Nginx or Jetty can be used.
This is an open source project of the Schmidt Center for Data Science and Environment at UC Berkeley.
Released under the BSD-3-Clause License. See LICENSE for more information.
We use the following open source technologies:
- ACE Editor for the code editing interface under BSD-3.
- ANTLR4 for parsing the Josh domain-specific language under BSD-3.
- Apache Commons Collections for enhanced collections under Apache v2.
- Apache Commons CSV for CSV handling under Apache v2.
- Apache SIS for coordinate system transformations and COG support under Apache v2.
- Checkstyle under LGPL.
- D3 for data visualization under ISC.
- GeoTools for geospatial data processing under LGPL.
- Gradle under Apache v2.
- JTS Topology Suite for geometry handling under EDL.
- JUnit under EPL v2.
- Math.js for mathematical expressions under Apache v2.
- Minio Java SDK under Apache v2.
- Mockito for testing under MIT.
- Picocli for command line parsing under Apache v2.
- Popper.js for tooltip positioning under MIT.
- Public Sans font under OFL-1.1.
- SLF4J for logging under MIT.
- Spotless for code formatting under Apache v2.
- Tabby for tab interface management under MIT.
- TeaVM for WebAssembly compilation under Apache v2.
- Tippy.js for tooltips under MIT.
- UCAR NetCDF for NetCDF support under BSD-3.
- Undertow for the local web server under Apache v2.
We recommend Temurin.