Most of the people will say the "response time" are the best way to determine whether a web application is slow or not. As it turns out, the whole concept of response time as ultimate metric to determine whether an app is slow or not is wrong. This blog post will try to explain why many developers usually falls into this kind of misconception.
First thing first, lets define what is "slow"?
In the context of web application, there are many ways this could be expressed by different people:
"If I clicked a button in a web page, and it took more than 5 seconds to reload or go to the next page, it is slow"
or
"When I visit a particular link and it displays the contents immediately, then its not slow"
or
"If I hit the play button on the video player of the clip I wanted to watch and it played immediately, then its now slow"
or
"When I send traffic like 1000 users simultaneously and the page loads fine still around 4 seconds, then it is not slow"
You'll notice in those common examples the following observations:
Slowness of an application is a relative value depending on the users goal
In the case of web application, it is relative to what the user is trying to achieve. Interestingly, when a certain activity like "transferring funds via online banking" or "streaming a high resolution video", users perception about slowness of an application is different compare to simple activity like "browsing the facebook activity" or "watching a short clip about neighbor's cat".
Users tend to be more patient when a certain process is something that of value for them. Like for example when streaming a high resolution video, even though it would take a while for it to load they would consider it "fast" if it played like lets say 10 seconds after hitting the play button. That is in comparison to just watching a clip about a cat but takes 2-3 seconds to load.
You cannot rely on a single test to determine the slowness of an application
When a certain web application is reported to be slow by 2 out of 100 users, did you consider the web application as slow? . No right? first instinct is, you will check what is wrong with those 2 users. There are many factors that could chime in once your web app is being hit by thousands of traffic, there goes the scalability concerns, concurrency, hardware issues. It boils down to this principle -- slowness cannot be measured by just doing a single test, you need to gather higher samples and compute its central tendencies (e.g. mean, average , mode etc).
It's not about the response time, it's about user satisfaction
Imagine this - you have a script which the sole purpose is to ping the web server if it responds within 3 seconds. When you check the logs, everything is fine, all requests is within 3 seconds some are even less than that. But then when you check the site itself, it is full of 404 page and 500 (Internal Server Error). At this point, you will realize that even though the web server "technically" is responding fast to web requests, you should somehow check if those responses are something users would like to receive. Thus, it is actually being user satisfaction.
There would be many other ways and scenarios like above which could affect how do you define the slowness of a web application. That very same issue for example makes it very hard for us (developers) to compare two different web applications that is using different technologies and process (like for example Java vs PHP, RoR vs Laravel, ASP.NET vs RoR) even using the most accurate bench marking tools.
Is there any hope we could have a "global" comparison metric so that it actually measures accurately the slowness of a web application?
Yes there is!
Enter Apdex (short for Application Performance Index).
Apdex is an open standard developed by an alliance of companies that defines a standardized method to report, benchmark, and track application performance. It basically aims to provide a solution to the above-mentioned issues we are encountering in our everyday struggle to measure our web applications performance.
In Apdex, things are simple: you just have number on a uniform scale of 0-to-1 wherein
0 = no users satisfied
1 = all users satisfied
At the very basic level, it is centered on why you are measuring things anyway : end-users satisfaction.
The index translates many individual response times, measured at the user-task level, into a single number. A Task is an individual interaction with the system, within a larger process. Task response time is defined as the elapsed time between when a user does something (mouse click, hits enter or return, etc) and when the system (client, network, servers) responds such that the user can proceed with the process. This is the time during which the human is waiting for the system. These individual waiting periods are what define the "responsiveness" of the application to the user.
How it Works
The index is based on three zones of application responsiveness:
- Satisfied: The user is fully productive. This represents the time value (T seconds) below which users are not impeded by application response time.
- Tolerating: The user notices performance lagging within responses greater than T, but continues the process.
- Frustrated: Performance with a response time greater than F seconds is unacceptable, and users may abandon the process.
The Apdex formula is the number of satisfied samples plus half of the tolerating samples plus none of the frustrated samples, divided by all the samples:
So it is easy to see how this ratio is always directly related to users' perceptions of satisfactory application responsiveness. To understand the full meaning of the ratio, it is always presented as a decimal value with a sub-script representing the target time T. For example, if there are 100 samples with a target time of 3 seconds, where 60 are below 3 seconds, 30 are between 3 and 12 seconds, and the remaining 10 are above 12 seconds, the Apdex is:
More detailed explanation about the standard can be found in the official Apdex site.
There you have it. That is the "correct way" of measuring slowness of a web application.
But how do I actually implement this concept in my web application?
There are tools that considers Apdex standard to measure application performance, like for example NewRelic's APM is using Apdex heavily to give reports.
Though you can implement this standard as well by yourself, I recommend checking out some tools first like NewRelic to get a hold of how to use the apdex standard in a real-world applications.
I hope you learned something new today, so next time someone asked you how "slow" a web application is? well, simply give them the Apdex of your web app and give him/her the link to this article :)
Did you find this useful?
I'm always happy to help! You can show your support and appreciation by Buying me a coffee (I love coffee!).