Working in many different languages across multiple projects means it’s often a
pain to remember how to build all the different components. How to run all
the ad-hoc automation tasks. Or even what the build command is for this
programming language ecosystem. For example, Clojure projects use
boot. A Java project could use
gradle. Python will use
setup.py, while Django will use
If you primarily use only one programming language and you know its build tool inside and out, then you can stop reading here. This article is primarily for the lazy polyglot who wants to stop remembering all these disparate details.
Scenario One: One Project, Many Commands
In this scenario, you’re mostly working on one main app all day. Your company’s monolith, for example.
Let’s say you’re working in a Django project. To run the Django unit tests you run:
DJANGO_SETTINGS_MODULE=myapp.settings.local_test python manage.py test
Now, I’m lazy. I don’t want to have to remember these two commands, their environment variables, or their arguments. What I normally do in this situation is write a simple Makefile.
test: test-py test-js test-py: DJANGO_SETTINGS_MODULE=myapp.settings.local_test REUSE_DB=1 ./manage.py test test-js: node_modules/.bin/gulp test
Now when I come back to this project, I only have to remember to run
test, and all my tests get run.
(In my real Makefile I use .PHONY targets to tell Make that these names don’t correspond to actual files.)
For this example, I only created a test target, but the common targets I like to
dist. These targets create a common
protocol I can depend on. They also serve as a form of executable documentation for how
to do one of these tasks.
Makefiles can get pretty hairy, pretty quickly, so the minute a task starts getting hard to read or maintain, I’ll write a separate script to perform the task, then just delegate to the external script within the Makefile.
Why Make? Because it’s ubiquitous and it’s easy to install.
Scenario Two: Many Projects, Many Commands
In this scenario, you’re working on many different projects. Maybe one microservice in your ecosystem is written in Erlang, the next is in Java, and the next is in Go.
The strategy I prefer here is to use what GitHub Engineering calls, “Scripts to Rule Them All”.
The “Scripts to Rule Them All” technique is essentially creating a consistent
set of scripts in each project. For example, in each project’s directory there
would be a
script folder with the following scripts:
script/bootstrap- Installs / updates all dependencies.
script/setup- Sets up a project to be used.
script/test- Runs the tests.
script/cibuild- Invoked by continuous integration servers to run tests.
script/server- Starts the app.
As you can see, like the Makefile from Scenario One, we’re in essence creating
a protocol. Whenever you check out a project, you know you can depend on
script/setup to get started, and
script/test to run your tests.
These scripts can call each other to compose functionality. For example,
script/cibuild, will probably call
script/test to actually run the tests.
This is a powerful pattern, and I encourage you to read the entire GitHub Engineering article to learn more.
Achievement Unlocked: Easier Onboarding
New team members just have to learn the protocol. After that they can get started quickly on any of the projects.
Achievement Unlocked: Automation
If you adopt a project build protocol, then you can use it to leverage automation. Imagine how easy it would be to configure a continuous integration server if all your projects used the same build steps.
- Check out the project source code.
Imagine how much easier it would be to automate deployments when all your apps use the same protocol to start up?
- Check out the project source code.
Achievement Unlocked: Less Documentation Required
Most projects have a README that describes how to set up a developer’s environment to run the tests and start the server. Go install these packages, start the database service, set some environment variables, etc.
If you adopt a project build protocol you don’t have to document these steps explicitly all the time. Delegate to the protocol instead. The build scripts are executable documentation. That means you should give variables good names, and add useful comments like you would your production code.
Every project has mundane details you have to keep track of: how to build it, test it, and run it. And every programming language has its own build ecosystem. Don’t get bogged down in these details, but abstract them away. I’ve illustrated two ways to do it. I bet you can think of more. What’s important is that you and your team agree on a protocol and stick to it. Once you do, you’ve got one less thing to think about.