clodl: self-contained dynamic libraries

clodl computes the closure of a shared object. That is, given
an executable or shared library, it returns a single self-contained
file packing all dependencies. Think of the result as a poor man’s
container image. Compared to containers:

closures do not provide isolation (e.g. separate process,
network, filesystem namespaces),
but closures do allow for deploying to other machines without
concerns about missing dependencies.

Clodl can be used to build binary closures or library closures.

A binary closure is made from an executable or a shared library
defining symbol main and can be executed. In practice, the binary
closure is a zip file appended to a script that uncompresses the file
to a temporary folder and has main invoked.

A library closure is a zip file containing the shared libraries in
the closure, and provides one or more top-level libraries which depends on all of
the others. When the closure is uncompressed, these top-level libraries
can be loaded into the address space of an existing process.

Executing a closure in the address space of an existing process
enables lightweight high-speed interop between the closure and the
rest of the process. The closure can natively invoke any function in
the process without marshalling/unmarshalling any arguments, and vice
versa.

Example of binary closure

clodl is implemented as a set
of Bazel build rules. It integrates with your
Bazel build system, e.g. as follows:

cc_binary(
  name = "hello-cc",
  srcs = ["main.c"],
  deps = ...
)
binary_closure(
  name = "hello-closure-bin",
  src = "hello-cc",
)

With Haskell:

haskell_binary(
    name = "hello-hs",
    srcs = ["src/test/haskell/hello/Main.hs"],
    ...
)
binary_closure(
  name = "hello-closure-bin",
  src = "hello-hs",
)

The test BUILD file has complete examples.

Example of library closure

clodl is useful for “jarifying” native binaries. Provided shim Java
code, closures can be packed inside a JAR and then loaded at runtime
into the JVM. This makes JAR’s an alternative packaging format to
publish and deploy native binaries.

cc_binary(
  name = "libhello.so",
  srcs = ["main.c"],
  linkshared = 1,
  deps = ...
)
library_closure(
  name = "hello-closure",
  srcs = ["libhello.so"],
)
java_binary(
  name = "hello-jar",
  classpath_resources = [":hello-closure"],
  main_class = ...,
  srcs = ...,
  runtime_deps = ...,
)

Importing clodl

In the WORKSPACE file:

http_archive(
    name = "io_tweag_clodl",
    sha256 = "1181131b0fc111a1f16f0532605e9835e308ac5bc278b62f825adb0844ff7590",
    strip_prefix = "clodl-0a7a2f93f4043a2db623f7d820578e3baea228d1",
    urls = ["https://github.com/tweag/clodl/archive/0a7a2f93f4043a2db623f7d820578e3baea228d1.tar.gz"],
)
...

In BUILD files:

load(
    "@io_tweag_clodl//clodl:clodl.bzl",
    "binary_closure",
    "library_closure",
)
...

Building it

Requirements:

The Bazel build tool;
the Nix package manager;
in Linux, the scanelf tool from the pax-utils package and the patchelf tool;
in OSX, otool and install_name_tool.

To build and test:

$ bazel build //...
$ (cd tests; bazel run hello-java)

License

clodl is free software, and may be redistributed under the terms
specified in the LICENSE file.

About

clodl is maintained by Tweag I/O.

Have questions? Need help? Tweet at
@tweagio.