The New Sysimage Service in JuliaHub

The sysimage service is a new feature available in JuliaHub, which makes working with sysimages built by PackageCompiler.jl simple and accessible.

What is a sysimage?

A sysimage is a bundle of packages of the environment it is built from. The result of creating one is a file which you can use when you start a Julia process. Using a sysimage will allow you to get immediate access to the packages that you have bundled up without needing to download or precompile them, so it is a good idea to give it a try if your workflows can benefit from such improvements. For more information on sysimages please visit the PackageCompiler.jl documentation.

The Sysimage Service

Traditionally working with sysimages requires quite a few steps to get started. First you need to build the sysimage, store it and rebuild it whenever your environment changes. For standard workflows you only need to attach the sysimage to your Julia process, but for multi-process setups you also need to take care of the sysimage distribution and relocatability. This gets especially difficult when working with cloud based compute resources.

With JuliaHub, when you request a sysimage for your job, the Sysimage Service will spawn a build based on your environment (Manifest.toml file) and automatically attach it to your job. The next time you use the same environment the sysimage will come from the cache saving you some time you would otherwise spend on managing and rebuilding the sysimage.

When launching distributed jobs you don’t need to worry about anything - JuliaHub takes care of distributing the sysimage to all your requested worker processes and because they run in the exact same environment you don’t have to deal with relocatability issues.

Your sysimages managed by JuliaHub are associated with the environment you provide, so you don't have to think about keeping your images in sync or accidentally using the wrong sysimage. When your environment changes, JuliaHub will detect that and automatically build and cache a new sysimage for you.

When to use it

You should consider using the Sysimage Service when:

1. Repeatedly running jobs with an environment that takes a noticeable amount of time to instantiate (to save time on each run)
2. Running distributed jobs with many processes (to save instantiate time on each process and mitigate any instantiate related worker startup issues)

How to use it

Currently the Sysimage Service is only available for standard Julia batch jobs and package based jobs such as My Applications or Registered Applications.

You can use the feature when submitting jobs through the website, JuliaHubClient.jl and the Julia IDE interface (JuliaHub.jl support is pending). 

Look out for a checkbox with a Sysimage build label or a submission parameter called sysimage_build.

When you submit a job with a sysimage build for the first time you should see an additional build job in your list as well as your main job. Once the sysimage build finishes your main job will start and when it’s done it should be displayed like on the image below.

Next time you run a job with the same environment you will not see an additional build job. The sysimages are only built when your environment changes. Your previous builds stay in the cache, so you don’t need to worry about losing or managing them.

Please refer to our documentation for more information. Should you want to use JuliaHub, just go to juliahub.com and start free today!