Wednesday, March 26, 2014

rebar and repository organisation

Introduction

In Erlang ecosystem the most popular build tool seems to be rebar. It is quite easy to start using. If you have an Erlang application with directory structure organised according to OTP requirements, you just launch rebar compile in the root directory of app and it compiles.
Whenever you need a dependency from another application which is not in OTP, you create a rebar.config file with following content
{deps, [
  {dep_name, dep_version, path_to_dep}
]}.
Most likely path_to_dep would be VCS url, for example,
  {git, "git://github.com/rebar/rebar.git"}.
But what shall we do when we need to create one more own Erlang application? Let's overview possible code organisation strategies.

Repository per application

The most popular repository organisation used with rebar is a separate repository for each application. It provides a nice separation of concerns, great support from rebar and it is quite easy.
A new repository is created, source code is placed. Then algorithms for 3rd-party dependencies is repeated - reference to the new application is set up in rebar.config of original application.
Not everything is so bright with this approach. Even if you have only 3rd-party dependencies you need to take care of versioning, because depending just only on branch (especially on one under active development) could bring instability into the system you are working on. To solve this problem people specify a release branch as dependency (in case of git it is 'tag'). Rebar supports it from the box, even erlang application's version from .app file is supported (see dep_version in rebar.config), but it is not extensively used as it is the second versioning mechanism to take care of.
During development of system which consists of multiple applications it is very common that changes are required in several components for one feature. So feature branch is created in each repository, and in order to keep consistency during development this branch is specified in rebar.config.
{deps, [
    {app1, ".*",
        {git, "git_url/app1.git",
            {branch, "feature12"}}},
    {app2, ".*",
        {git, "git_url/app2.git",
            {branch, "feature12"}}}
]}.
You need to take care of rebar.config before or after merging to the development branch. With growing number of repositories code management could start to be a pain, which could be reduced with automation of routine tasks.


Single repository

An alternative to writing scripts for VCS handling is keeping everything in a single repository. The idea is to place each Erlang application into separate directory in the repository. As a result directory structure looks like this:
repo_dir
|-app1
|-app2
|-app3
|-rebar.config
In each directory there is a standard OTP application structure. The only change required is putting
{deps_dir, ".."}.
into each app's rebar.config and
{deps_dir, "."}.
into rebar.config in root directory. Dependencies from the same repository are specified only by name:
{deps, [app2]}.
The entire project could be found here.
All rebar commands such as compile, eunit and clean could be launched in each application's directory separately or in root. It is convenient, for example, to launch unit tests in all apps once change is made with rebar eunit in top level directory.
Keeping things in a single repository allows to do an "atomic commit", when single merge adds feature into development or integration branch. You can create a release (tag in git) with a single command as well. In over words it allows to have a snapshot of entire system easily.
It is quite easy to move modules from one application to another.
Of course there is a drawback with single repository. It does not allow to manage access for developers in flexible way. For example, company decides to opensource one of applications, so that rest of them are still private. It could only be achieved by moving the code into a separate repository. The same problem occurs if access must be granted only to the part of system (not for all applications).

To SumUp

I would recommend to start with single repository for all Erlang applications and move some of them to separate repositories only for a good reason.