Mark Needham

Thoughts on Software Development

node.js: child_process.exec not returning all results

with one comment

I’ve been playing around with some node.js code to get each of the commits from our git repository but noticed that it didn’t seem to be returning me all the results.

I had the following code:

var exec = require('child_process').exec;
 
var gitRepository = '/some/local/path';
 
exec('cd ' + gitRepository + ' && git log --pretty=format:"%H | %ad | %s%d" --date=raw ', function(error, stdout, stderror) {
  var commits = stdout.split("\n");
 
  // do some stuff with commits
});

We have around 2000 commits in the repository but I was only getting back 1600 of them when I checked the length of commits.

Eventually I decided to print out what was in error and got the following message:

error: Error: maxBuffer exceeded.

Going back to the documentation revealed my mistake:

maxBuffer specifies the largest amount of data allowed on stdout or stderr – if this value is exceeded then the child process is killed.

The default options are

{ encoding: ‘utf8’,
timeout: 0,
maxBuffer: 200*1024,
killSignal: ‘SIGTERM’,
cwd: null,
env: null }

The limit is 2048000 which is around about the number of bytes being returned when I get to 1600 commits.

Changing the code to increase the buffer sorts it out:

var exec = require('child_process').exec;
 
var gitRepository = '/some/local/path';
 
exec('cd ' + gitRepository + ' && git log --pretty=format:"%H | %ad | %s%d" --date=raw ', {maxBuffer: 500*1024}, function(error, stdout, stderror) {
  var commits = stdout.split("\n");
 
  // do some stuff with commits
});
Be Sociable, Share!

Written by Mark Needham

September 22nd, 2011 at 7:55 pm

Posted in Javascript

Tagged with ,

  • Jos

    Add the same problem … ┬áthe easiest solution is to output the result to /dev/null