EXPERT RESPONSE
Before addressing your specific questions about OpenSTA, I think it's worth addressing this in a more general manner. OpenSTA is without a doubt the best open source software for generating load against most Web sites. It may not be as "pretty" as the expensive tools, doesn't integrate directly with other "pretty" tools and is only for Web sites, but within those bounds I find it to be competitive with the commercial tools and I like it better than several of them.
That point is important because if you are having difficulty understanding OpenSTA's test results, odds are good that you'd have the same difficulty across the board. Remember, even though most people call these "performance testing" tools, they aren't. They are "load generation and data collection" tools. They aren't analysis tools, they aren't reporting tools, they don't know what "good" is, and once you get reasonably good with one of these tools, it's quite easy to create a test that the team will accept as valid to generate just about whatever "results" they want to see.
The fact is that even though most people refer to the data at the conclusion of load generation, it is virtually raw data. Folks call that data "test results" because they call the load generation a "test" and the data is the "result" of executing the "test." Don't confuse this with a "test result" which is a conclusion about whether or not the Web site (in this case) demonstrated the desired characteristics under some generated load. As a performance tester, it is your job to figure that part out and no matter what the marketing material may say, no tool on the market can do that part for you.
So, with that out of the way, let's take a look at your specific questions.
- Is there any feature available to analyze the results generated with the optimum condition results?
I'm not sure what optimum condition results are, but the answer is no. If you want to do a side-by-side comparison of data, you'll want to do it by exporting the data and manipulating it in a spreadsheet. In fact, all of the top performance testers I've ever talked to list their top two tools for performance testing as 1) their brain and 2) their favorite spreadsheet program.
- Most of the information generated are in graphical form wherein not much of detail can be understood. I tried looking for details in help file but could not find much on results generated. I want you to please provide me an understanding on graphs generated so that I can work further.
The graphs really aren't intended for much more than a quick scan to see if things look like you expect them to look (i.e. "Is this data worthy of further analysis?") Everyone has their own favorite graphs (personally, I like scatter charts, trend diagrams and sparklines.) If the automatic graphs don't work for you, just build your own in that spreadsheet program we were discussing in question 1. This is actually what development team recommends in the F.A.Q. -- which also contains the best answer to this question, that you can refer to here.
- I want information on Processor and Memory consumption during peek hours of testing (i.e CPU & Memory consumption when simulated with large number of users).
I presume you mean that you want OpenSTA to collect that for you. You can create a Collector that will allow you to capture SNMP or PerfMon statistics -- assuming you know the name and/or IP address of the target server and the server and network security features are configured to allow you to do so.
Even with the expensive tools it's almost never that simple. Your servers generally need to be running on the right platform (and/or you have to be able special monitoring software on them), be on the same subnet as the load generator, and have special security features disabled. In all my years of doing performance testing on hundreds of applications, I've only been involved with all this coming together twice.
Honestly, it is much easier to simply get an output log from whatever resource monitoring software your developers or administrators are using that includes timestamps and resource levels and add that to your results analysis spreadsheet (see question 1).
- HTTP Errors versus Active Users Graph Report is not visible along with the listing of various other reports.
Are you saying that you can only see one report at a time? That sounds like a bug to me. See if the OpenSTA repair tool here helps with that. Either way, if you've exported your data to the spreadsheet program (see question 1), you can view it whenever, however and wherever you like.
- Can graph details be converted to readable summarized information?
Sure they can. Using -- you guessed it -- your favorite spreadsheet program (see question 1).
- What is the meaning of query value?
I'm assuming that you're referring to the Collectors. OpenSTA calls the specification of the data you want a Collector to capture a "query." Along the same lines, it refers to the answer to that query (the data the Collector captures) as the "query value(s)."
More information:
|
