Recipe: Use a Ruby script as a Drake step

You want to write a Ruby script that knows how to take a specified input file, and produce a specified output file. You want to wire up this Ruby script as a step in Drake. This script may be large and complicated, so you don't want to use the [ruby] protocol for inline Ruby.

Here's one approach. First, create your workflow file with the step for your Ruby script (we'll create the Ruby script in just a bit):

workflow.d:

;
; Runs myruby.rb, giving it the input file as the
; first argument. stdout is directed to our output file.
;
out.csv <- in.csv [shell]
  ruby myruby.rb $INPUT > $OUTPUT

Next, let's make sure we have some sample input data. Create the in.csv file:

in.csv:

Artem,Boytsov,artem
Aaron,Crow,aaron
Alvin,Chyan,alvin
Maverick,Lou,maverick
Vinnie,Pepi,vinnie
Will,Lao,will

Okay, time to produce some output, The Ruby Way! Let's add a new column called "LineNumber", and fill it in with the correct line number of each line:

myruby.rb:

#
# Reads each line from the input CSV file (specified by first argument),
# appends a line number and writes the new line to stdout.
#

line_num=0
infile=ARGV[0]

File.open(infile, 'r').each do |line|
  puts "#{line.strip()},#{line_num += 1}"
end

Okay, we're ready to go. Run Drake!

$ drake

out.csv after running Drake:

Artem,Boytsov,artem,1
Aaron,Crow,aaron,2
Alvin,Chyan,alvin,3
Maverick,Lou,maverick,4
Vinnie,Pepi,vinnie,5
Will,Lao,will,6