Simple Clustering with RSRuby

Clustering is a technique for grouping objects which are similar. For example, given the following data instances:

A:	10,	20,	30,	40
B:	40,	30,	20,	10
C:	11,	21,	29,	39
D:	39,	29,	21,	11

We can see that A and C are 'closer' than B and D, so A and C can be grouped together, and B and D form a separate group.

R supports a wide variety of clustering techniques, all of which require us to pass a data frame object to the R interpreter. This post describes a simple way of generating a data frame, passing it to a clustering algorithm, and reading back some of the results.

Creating a Data Frame

The data frame is an R-specific data structure for holding data and its interpretation. For clustering, we need to describe a number of instances (the rows within a standard table) based on their attributes (the columns). For example, the following shows the first two instances from a weather dataset:

instance	outlook	temperature	humidity	windy
1	0	85	85	0
2	0	80	90	1

To define this in RSRuby, we need to set up a conversion tool, to handle the conversion from a hashmap to R's data frame.

require 'rsruby'
require 'rsruby/dataframe'

r = RSRuby.instance

r.class_table['data.frame'] = lambda {|x| DataFrame.new(x)}
RSRuby.set_default_mode(RSRuby::CLASS_CONVERSION)

The weather dataset can be defined and set up as follows:

outlooks = [0,0,1,2,2,2,1,0,0,2,0,1,1,2]
temps = [85,80,83,70,68,65,64,72,69,75,75,72,81,71]
humidities = [85,90,86,96,80,70,65,95,70,80,70,90,75,91]
winds = [0,1,0,0,0,1,1,0,0,0,1,1,0,1]

weather_data = {}
weather_data['outlook'] = outlooks
weather_data['temperature'] = temps
weather_data['humidity'] = humidities
weather_data['windy'] = winds

example = r.as_data_frame(:x => weather_data)

Of course, the definition of the hashmap can be simplified, but I like this format as a template for constructing the data frames from other sources of data. Note that the hashmap must be provided as the value of the :x key when constructing the data frame.

Clustering

To use any of the cluster techniques, we must include the 'cluster' library:

r.library("cluster")

A simple form of clustering algorithm is called clara. This takes a data frame and a number of clusters, and then returns a set of results which describe the data in terms of those clusters; the result is returned as a hashmap for Ruby, and the cluster results can be retrieved using the "clustering" key:

clara_result = r.clara(example,2)

14.times do |i|
  puts "Data: #{clara_result["data"][i].join(",")} Cluster: #{clara_result["clustering"][i]}"
end

A sample output is:

Data: 0,85,85,0 Cluster: 1
Data: 0,80,90,1 Cluster: 2
Data: 1,83,86,0 Cluster: 1
Data: 2,70,96,0 Cluster: 2
Data: 2,68,80,0 Cluster: 1
Data: 2,65,70,1 Cluster: 1
Data: 1,64,65,1 Cluster: 1
Data: 0,72,95,0 Cluster: 2
Data: 0,69,70,0 Cluster: 1
Data: 2,75,80,0 Cluster: 1
Data: 0,75,70,1 Cluster: 1
Data: 1,72,90,1 Cluster: 2
Data: 1,81,75,0 Cluster: 1
Data: 2,71,91,1 Cluster: 2

Interacting with the Command Line from Ruby

One of the strengths of Ruby is as a scripting language, that is, a language for managing control of other programs. A typical setting is to provide our script with some parameters which are then used to set up and run other programs. For example, we could control the number of times the external program is called, or collect the results and store them in specific files. This kind of automation can help speed up our work considerably, and also lets us think of our research at a higher level of abstraction: i.e. just create a pool of training data as big as is needed, rather than having to laboriously construct every item in the pool by hand. Ruby is also highly suited to this task, as many of the tasks, such as parsing input arguments or formatting strings for output display, are easier in Ruby than in, for example, C.

To achieve this level of control, we first need some way for our Ruby script to handle command-line arguments provided to it. Second we need to be able to construct and run commands from within Ruby just as if we had typed them at the command line.

For example, I am working on the analysis of what are known as k-SAT landscapes: the idea is to construct a boolean function, such as x_1 & x_2 | x_1 & x_3, which contains a given number, m, of clauses, each clause with k terms. The clauses are constructed from n different variables, and there should be N of the landscapes for any given combination of n, m and k. I have a C program which will take n, m, and k as input arguments and print the landscape to standard output. I want to automate the construction of the N landscapes, and capture the output into files named in the format: 'results-n-m-k-i.txt', where i will range from 0 to N-1. This is ideal for automation, as the construction of any landscape can take up to a day.

Basic Techniques

When you start a Ruby program, using: 'ruby program.rb X ...' you can provide command-line arguments as 'X ...'. These arguments are placed into the array ARGV, as strings. (Warning for C programmers: ARGV[0] is the first argument, not the name of the program!) It is a simple matter to read each item out of ARGV, convert to a suitable number format, and place the result into an internal variable: to convert the first parameter to an integer and store in 'x', write: x = ARGV[0].to_i

Ruby strings make it easy to interpolate values from variables using the #{} syntax, which interprets anything inside the brackets and puts the result into the string. For example:

5.times do |i|
  puts "The value is #{i}"
end

Will print:

The value is 0
The value is 1
The value is 2
The value is 3
The value is 4

Finally, to call an external program, and pass arguments to it, Ruby provides convenient backticks, ` `. Anything between the backticks will first be interpreted as a string (so variable values can be added in, as above) and then passed to the shell for execution, just as if you had typed the command within the backticks on the command line. Thus, `wc #{filename}` will call the word-count program on the file named within the 'filename' variable.

My final code is below. Notice the check for the number of input arguments, and output of a help message if there are not enough; this is always useful, as I very quickly forget what those input arguments should be. The script can be run easily using:

ruby create-samples.rb 20 50 3 20

# Automate the construction of landscapes
#
# ruby create-samples.rb n m k N
#
#      where n = number of variables
#            m = number of clauses
#            k = number of variables per clause
#            N = number of formulae
#
# Results for formulae constructed with names: results-n-m-k-i.txt
#      where i runs from 0 to N-1

if ARGV.length != 4
  puts <<-HELPMESSAGE
Automate the construction of landscapes
Copyright (c) Peter Lane, 2008.

ruby create-samples.rb n m k N

     where n = number of variables
           m = number of clauses
           k = number of variables per clause
           N = number of formulae

Results for formulae constructed with names: results-n-m-k-i.txt
     where i runs from 0 to N-1
HELPMESSAGE
else
  n = ARGV[0].to_i
  m = ARGV[1].to_i
  k = ARGV[2].to_i
  N = ARGV[3].to_i
  N.times do |i|
    puts "Creating file #{i} for n=#{n}, m=#{m}, k=#{k}"
    `../makels #{n} #{m} #{k} > results-#{n}-#{m}-#{k}-#{i}.txt`
    sleep 1.0 # wait at least a second, to allow for C's randomisation!
  end
end