Software Developer and Performance Engineer
Performance
Software Performance
Managing Testing Data
May 15th
Performance testing has an almost insatiable thirst for data. When executing thousands of tests to put load on servers, each of the tests needs data that is given to the server. In my experience, I have noticed 3 types of data that are needed.
- Unchecked – Values that must be filled out but for the most part are not checked by the server. These values can be randomly generated, selected from a list, or simply entered as fixed data that doesn’t change from test to test. Managing this type of data is very simple and requires relatively little work.
- Checked – Values that are checked by the server and must match for a test to execute properly. These values are different based on a test type, but may be the same every time the test is run. A great way to deal with these types of values is to look them up by test type or name, and a description of the value. Properties files, spreadsheets, or some other simple mechanism can easily accommodate this type of data. The data can still be random, but it must be grouped together so that random selections choose all the pieces of data that go together.
- Consumable – Values which are consumable cannot be used again and are lost once they are used. These values are the hardest to manage because not only do they behave like checked data, they disappear and new data is needed to execute a second test. When trying to put a server under heavy load, lots of unique data will be needed. The best way to manage this kind of data is with a database. The sheer volume of data, and the fact that it needs to be marked as used is best handled by a tool made for the purpose.
Security Just Keeps Getting In My Way
May 12th
One of the most difficult things to deal with when performance testing is security. It is designed to stop the very things we need to do to generate server load. Basic security protects a system from robot attacks, multiple logins, invalid permissions, and more. Many of the techniques used in performance testing are similar to attacks that attempt to compromise a system.
How do you deal with security then? I usually ask that it be turned off on the systems I am going to test. A lot of time and effort can be wasted trying to make an application play by the rules. For example, if a system allows only one login at a time and you need to simulate 1000 clients, you will need to have 1000 users defined. Worse, you will need to manage the use of all those logins across the machines being used to run the performance test. It becomes a data management nightmare and can cause endless frustration.
Some security rules only allow one instance of an application to connect to the server from an IP address. This makes it impossible to use a tool that spawns multiple threads or processes to simulate multiple clients.Now it might be impossible to turn security off or there may be rules against it. If that is the case then there are not a lot of options. But if it is at all possible, I suggest that you have security turned off while doing performance tests and avoid all of these problems entirely.
