Monday, July 16, 2012

Checking Progress on Asynchronous Jobs

I recently discovered the wc command in *nix environments, which is a handy solution for when I run asynchronously processed jobs.  I often need to run ensembles of simulations.  If I process them serially, and put an output command somewhere in the script, I can immediately see how far along I am by checking a log file.  But if the data is processed asynchronously, it can be a bit tougher to see what's going on.

Running the following on my log file tells me the number of lines so far:
wc -l log.out
Hence, if the log file has n lines, and I have a total of N cases, then, assuming each case produces a single output statement, n/N % of my job is done.

No comments:

Post a Comment