I have just started a new Performance Test engagement on a major upgrade to an HR system and as with 90% of my engagements the non-functional application requirements are contradictory. An example is these requirements:
- The system must support 5,000 concurrent users
- The system must support 1-page requests per second
They look innocent enough, however, one of the contracted values is out by a factor of over 150. In this Post I will create some simple performance profiles and recalculate the values.
In this HR system there are three types of users:
- Normal employees (about 27,000 or 90% of logged in users)
- Manager employees (about 2,500 or about 8.5% of logged in users)
- HR administrators (about 500 or about 1.5% of logged in users)
After consulting the product owner and the business analyst we have determined:
A good representation of a typical employee: Login screen, submit login, main screen, select employee details, select home, select leave request, submit leave, select timesheet, select submit timesheet, and log out
A good representation of a typical manger Employee (Same as normal employees) plus leave approval: Select leave request, approve leave request select timesheet, approve timesheet and approve new employee
A good representation of a typical HR administrator (Same as manager employees except approve new employee) plus add employee: select new employee and Submit new employee
Calculating the average number of page impressions per user:
- Normal employees will on average have 10-page impressions and are about 90% 0f the user base (0.9*10)
- Manager employees will on average have 15-page impressions are about 8.5% of the user base (0.085 * 15)
- HR administrator will on average have 16-page impressions and are about 1.5% of the user base (0.015 * 16)
The average number of page impressions per user is about 10.515 – I will use 11 page impressions per user interaction.
A good average user page impressions rate is one page per 30 seconds (Some will be faster and others slower, for example, a typical user entering login details will be less than 10 seconds and others will be longer, for example, a typical user entering timesheets will take minutes). Using 30 seconds for the average user page impression gives an average site visit time of around about 5 and a half minutes or 330 seconds (important for script pacing calculations).
To meet the 5,000 concurrent user requirement the system will need to support just over 15 (5,000/330) user logins per second or the system will need to support 167 (5,000/30) transactions per second. This significantly exceeds the original non-functional requirement of 1-page impression per second.
Going the other way to support the 5,000 concurrent users at one page impression per second would have the average time a user will need to stay on a page is just over 83 minutes or the average user will spend over 15 hours a week submitting their timesheets and leave requests.
As an HR system has a captive audience (All 30,000 employees are required to use the system on a weekly basis) and there are deadlines (timesheets need to be submitted by a given date/time) – 5,000 is a good estimation of the expected number of concurrent user sessions. To align the page impression rate will need to be increased from 1 per second to 167 to align with the expected number of concurrent sessions. I have scheduled a meeting for tomorrow morning with my customer and software vendor to discuss aligning the non-functional requirements.