In the last blog post we introduced described the need for a testing tool for Linux and middleware installations. After some Internet reasearch and a small test, we concluded RSpec would be a good fit. In this blog post, we will dive into the way we use RSpec to build the specifications and tests needed for your systems.
Before we dive into RSpec, lets first describe our environment. Most of our applications use at least the combination of an Oracle database and a WebLogic JEE server. Sometimes the applications are also built using Tibco products. Normally, all these middleware functions are installed on separate distinct systems. This means one or more systems for the Oracle database. One or more systems for the WebLogic JEE server, and one or more systems for Tibco products. All these systems have interrelated settings. For example, the Weblogic server needs a connection to the Oracle database to get it’s data. This group of interrelated systems, we call a platform. We build specifications and tests for a complete platform. So the specification contains all settings and tests for a set of 3 or more systems.
What’s this RSpec thing?
RSpec is a tool based on the Behaviour Driven Development(BDD) software development process. Wikipedia says:
At the heart of BDD is a rethinking of the approach to unit testing and acceptance testing that North came up with while dealing with these issues. For example, he proposes that unit test names be whole sentences starting with the word “should” and should be written in order of business value.
Heart of the matter is, you write specifications, and you accompany them with a test to validate the specification. Throughout this blog post, the terms specification (and spec) and tests are both used and mean (about) the same. If you would like to know more about the RSpec core, I recommend checking out the web site and to read the book The RSpec Book: Behaviour-Driven Development with RSpec, Cucumber, and Friends
The top spec
The code below shows what we call, a top level spec. As you can see, we use all the normal RSpec syntax.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
If you read through the code, you can see, the top level spec, contains the specifications for all the systems in this platform. We have a
dbhost. The spec for this system starts at line 8. The other system in the platform is the
wlshost. Again: together, we call them a platform. To run the spec on either system, you can use the regular RSpec command:
:on_host => host, takes care that only the right set of spec’s and tests are run on the system. If we enter this command on node
wlshost, only line 15 trough to line 28 are run. On the other hand, if we run the command on host
dbhost, only lines 13 until 15 are run. If you enter the command on any other system, nothings happens.
Running in different environments
One of the design goals of our testing setup is that we want to be able to run the same set of tests in all of our environments. This means the same tests run in development, test, acceptance and even in production. To accomplish this, we make heavy use of RSpec’s metadata.
1 2 3 4
:on_host => host and the
:with_ip => '10.0.0.1' are user defined metadata elements. Later in this blog post, I will show how we actually use this metadata to make the individual tests are unaffected by the environment the run in.
If you have more than a couple of environment specific settings, setting them all in the
describe block, would become quite large and cumbersome to read and understand. Therefore, we introduced the
include_context "running in our
development network in line 5. In this line, we include a specific context or environment.
In the shared_contexts, you can specify all sorts of metadata parameters. Because they are named, you can select which one you need. Right now, we use the one that works in our development network. The shared contexts can be written in another file. Here is the one we used:
1 2 3 4 5 6 7 8 9 10
shared_context, we can specify all specific setting for an environment. What’s even better, we can share it between different top level specs. So the top level spec’s running in this environment, can include this shared context and have all nescecarry settings. It’s also very DRY If you change your dns servers, there’s only one place you have to change this value.
shared_context idiom is RSpec standard. The
meta_for is something we added. You can specify any legal ruby variable name on the left and any ruby type on the right side.
structure of the spec’s
Let’s get back to the actual specification. Again we make heavy use of a standard RSpec feature. The shared examples. Using
it_behaves_like, we can call a set of specifications.
1 2 3
We have a convention to structure the tests in three levels.
- The hardware
- The type of function of the system. E.g. a database or a JEE server
- The extra additions we need to get a specific application running on it.
The real stuff
All the elements we talked about this far, are mostly stuff we need to structure the set of tests. But what does the real stuff look like? Here is part of the real stuff.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
This is part of the specifications for the base Linux system. It describes the LDAP and DNS settings. Here, you see the basic RSpec structure. We use
describe to structure a big set of specifications and tests into smaller units. The actual specifications are done using the
it commands. Our aim is to write communicable text in the
it statement. We have noticed that besides a good test tool, RSpec really enabled us to communicate about what we did and why we did it.
System.files["/etc/ldap.conf"]["idle_timelimit"].should eq "870" is the actual test. Also in this code, we place a high value on communication. Even someone who doesn’t know anything about RSpec and ruby, but knows about LDAP, is able to understand that we test if the
/etc/ldap.conf is set to 870.
What does that meta thing do?
I told you before that we make heavy use of Rspec’s user defined metadata. We define the information either in the describe block or we can describe them in the shared_context’s. But in the test, is where we actually use them. Let’s look at one in detail.
1 2 3
This specification text contains a call to the
meta method with the parameter
:dns_servers. This call looks into the meta information that’s available and retrieves the value for
:dns_servers. If you check back to the
shared_context, you can see, that it translates to
['188.8.131.52', '192.168.42.155']. So when this specification is run, the description of the specification will become:
In the test itself, the
*meta(:dns_servers) will also be translated to the array with the two string values.
meta method is not standard RSpec. In standard RSpec, there is a difference between meta information you can use on the description level and the meta information you can use in the test. We felt that it would be useful to use them equally in both the description and the test. So we build our own extension to the RSpec meta information.
The real real stuff
I said before that I would show you the real stuff. But that was a small lie. The
System class you see in the example is an abstraction. An abstraction we use to keep the spec’s and tests at that level very readable and with a high communication value. Did I tell you the value we put on communication ;–). But NOW I’,m going to show you the real stuff. Here…. without further ado, is the
1 2 3 4 5 6 7 8 9
System class, we use the Facter gem.
Facter is a lightweight program that gathers basic node information about the hardware and operating system. Facter is especially useful for retrieving things like operating system names, hardware characteristics, IP addresses, MAC addresses, and SSH keys.
With facter, we retrieve all sorts of information from the operating system and middleware in a way that we can easily use it in our spec’s (Or in a puppet manifest). Depending on the kind of information, it returns an integer, an array, a string or even a hash. Returning a hash helps in making the higher level tests very easy to read. Let me give you an example. Let’s look at the test below:
1 2 3
It calls the
files method on the
System class. We’ve seen the files method calling the
system_files fact. The fact returns the following information:
1 2 3 4 5 6
That’s why it is easy to get the
Next time, I’m going to tell you a bit about the work we did to make it easier to build custom types and resources for Puppet. Stay tuned.