Today we’re excited to announced the release of Builder 3.0, which contains improvements based on our experience using Builder in more than 100 projects over the past 10 months.
Builder solves the problem of multiple projects needing the same dependencies, configuration, and npm tasks. For example, every time we create a new repository for a React component, we can use a Builder “archetype” that automatically provides our standard Babel, eslint, and webpack settings. We love using npm scripts for our tasks, so the archetype also provides a base set of tasks for things like building, testing, running a development server, etc.
It does more than that, but you should check out the Builder homepage to read more. Let’s talk about this release!
What’s New
The focus of version 3.0 was making Builder more production-ready. Here are a few changes we made along those lines.
spawn vs. exec
Node’s child_process module offers two ways to launch a child process: spawn and exec. spawn accepts an executable to run and an array of arguments to pass it, while exec takes a single command string, exactly as you’d type it on the command line.
If you think about what tasks look like in an npm config, they’re all complete command strings:
```json { "scripts": { "build": "babel --out-dir lib src", "clean": "rimraf lib", "prepublish": "npm run clean && npm run build", "test": "karma start" } } ```
At first glance, these strings can’t be passed as commands to spawn without parsing them, because spawn needs the arguments separated into an array. We didn’t want to get into the business of parsing shell commands, so we reached for exec.
We got by with this initial choice for a while, but eventually noticed issues. exec buffers output, meaning it captures all of the command’s output into a string. This means you need to worry about buffer size and tell exec what you want the maximum to be. To make this capturing work, it also means that your commands are writing their output to a pipe rather than the true stdout and stderr file descriptors. This shouldn’t really make a difference, but as we’ll see later, it does.
To get around the output buffering issue while still avoiding any command parsing, we copied the way npm (and exec itself) runs commands:
```javascript child_process.spawn(["sh", ["-c", "command string here"]]) // on Windows: child_process.spawn(["cmd", ["/d /s /c", "command string here"]]) ```
This is exactly what exec does: it just passes the full command string along to the shell. But by taking matters into our own hands and using spawn, we also get better control over output buffering. Specifically, we can use the stdio: "inherit" option to allow child processes to write directly to stdout and stderr, without us doing any buffering in between. Much cleaner!
Pipes and process.exit
When we were still using exec to launch processes, we noticed that some tasks would consistently have their output cut off. (This especially stood out for certain programs that used ANSI formatting codes, where getting cut off meant that the formatting was never reset. The remaining output from completely unrelated tasks was getting colored and underlined!) Was it due to the aforementioned maximum buffer size? This particular command wasn’t writing anywhere near that limit, so something else must be going on…
We found that some Node programs use the following pattern (simplified here):
```javascript console.log("Lint error: missing semicolon"); ... console.log("Lint failed!"); process.exit(1); ```
This looks fine, and will appear to work under most circumstances. But it turns out there’s a known gotcha here. If you log a bunch of data, then call process.exit, and your process is piping its output to a parent Node process instead of writing to the standard output stream, then the output will be cut off! Anything after the first 8192 bytes (in our testing) won’t make it to the parent’s end of the pipe in time.
Many programs never consider how they were launched or where their output is actually going, so this bug is easy to miss when you’re writing a Node CLI program. If you’re writing such a script and use process.exit, you can fix this issue like so:
```javascript function safeExit(code) { process.on("exit", function() { process.exit(code); }); } console.log("Lint failed!"); safeExit(1); ```
This pattern will let the event loop and Node process end naturally, giving it time to flush its output instead of terminating immediately. Then when it’s ready to exit, it will still use the exit code you provided. (This was an issue with some programs that Builder was running, but we also updated Builder itself to use this pattern, in case another process wants to spawn Builder.)
Switching to spawn with stdio: "inherit" also fixed the programs we were running that had this issue, since they were no longer writing to a pipe.
Better Windows support
For 3.0, we added AppVeyor to our continuous integration (CI) setup so that our test suite runs on Windows. This was tricky; since Builder is a task runner, our tests need to set environment variables, launch real processes, and capture output – all things that can (and do) differ across different platforms.
Running on Windows revealed a few bugs that were previously undiscovered. For example, Windows has case-insensitive environment variables. If you set the PATH environment variable in a Node process, it is internally transformed to set Path on Windows. Builder now accounts for this behavior.
Some small niceties
We’ve made the help and version commands nicer to use by changing their default log level to be less noisy. Now you’ll see only the usage info, task list, and version number by default – unadultered by the internal task administrivia that we used to show.
Try it out
Check out the latest version of Builder like so:
```bash npm install builder ```
And follow along or contribute here: github.com/FormidableLabs/builder
Thanks for reading!
