When virtual testbeds are better than physical

April 26th, 2007

We are sometimes asked why a development or operations or support group would employ a virtual testbed like Replicate’s. Wouldn’t they insist on a physical testbed? In fact, our customers have come to prefer the virtual testbed. The reason takes a little explaining.

In our experience, the biggest fidelity problems in testing come from variations in the state of the system during testing. Ideally, testers use one server for each production server, and they reimage each machine before each test session. But there is rarely enough hardware available for this, and reimaging machines requires many steps and a lot of time. So most tests and operation rehearsals take place with the testbed that is at hand. It may involve one server that combines the functions of two or three, or it may be in a state that was left over at the end of the previous test. This can introduce false problems, and cover up problems that would become visible if the testing was done from a multi-machine consistent state.

Replicate’s fast and automated setup allows testers to control the most important variable, the state of the machines and their software. Multiple machines are always on tap, and they always start in a known state loaded from a library that contains any number of stored configurations. This eliminates the biggest sources of unknowns. We think the unknowns introduced by virtualization are extremely small by comparison.

And as a bonus, testing is done more thoroughly and more often. Operational upgrades can be repeatedly rehearsed with enough machines and realistic configurations on tap. Post-installation support issues can reproduced more often and with much less effort. Virtual testbeds can be valuable through an application’s full life cycle, in a way that physical ones can’t match.

Entry Filed under: Software testing