Conda is a platform-neutral packaging system. The BioConda channel is of particular interest for bioinformatics users, as it provides access to an enormous library of ready-to-use bioinformatics applications, which can be otherwise notoriously difficult to install. One of the problems solved by Conda, and BioConda, is the dependency hell which results from trying to install several applications, which have requirements for incompatible versions of their dependencies, for example different versions of samtools.

Read more

A BioInformatics pipeline may require invocation of some low-level programs like samtools hundreds or thousands of times. If these processes are run as BioContainers, using rkt for example, this means creation of very many rkt pods, adding up to a significant overhead both in space and time. Can we do better? In fact we can. For the purpose of running a containerized application in this context, all that matters is which container image it is, and which user it runs as.

Read more

Commonly, BioInformatics pipelines are built by piping output between applications. If each application is itself a BioContainer, and we are running them using rkt, this means we need to be able to pipe output between rkt pods. How can we do this? Rkt supports running multiple applications within a pod, so has no direct concept of pod standard input and standard output. By default, application output goes to the systemd journal within the pod, which appears on the standard output of the pod itself.

Read more

The ability to easily run containerized applications using rkt is quite intoxicating, and quickly leads on to an exploration of what applications are readily available. Docker Hub and Quay are worth exploring. Among those applications on Docker Hub are Julia and Ruby. Noting the slightly non-standard security requirements needed by the Julia interpreter, we can configure rktrunner to run Julia using this configuration file snippet: [options.common] image = [ "--seccomp", "mode=retain,@docker/default-whitelist,mbind", ] [alias.

Read more

Authenticating web proxies are an inconvenient fact of life, in that some applications simply will not run correctly behind one. However, many can be made to work simply by setting the http_proxy and possibly also the https_proxy environment variables. The rest of this post assumes the user is in fact behind such a proxy, and for the sake of simplicity, discusses only http_proxy. Such statements should be taken to also refer to https_proxy in environments where that is in fact required.

Read more

Bioinformatics users are a demanding bunch. They need to be able to run a large variety of applications on powerful servers. These servers may be administered by a small team of sysadmins, who quickly become a bottleneck in packaging and installing applications for the local users. Using application packaging such as RPM can make the task manageable, but there is still a per-application overhead on the sysadmin, and with a typical site using perhaps in excess of 300 applications, this becomes unmanageable.

Read more