Blackanvil Blogspot in 2006 06 Shootout Load Runner vs Grind

8/22/2019 Blackanvil Blogspot in 2006 06 Shootout Load Runner vs Grind

1/33

pdfcrowd comopen in browser PRO version Are you a developer? Try out the HTML to PDF API

Discussions on scalability -- load, stress, and performance testing.

The Black Anvil

W e d n e s d a y , J u n e 1 4 , 20 0 6

Shootout: Load Runner vs The Grinder vs ApacheJMeter

I recently needed to recommend a tool to use for a s calability testing project, and I was in

the fortunate situation of having some time to survey the field, and to look into the top

contenders in greater depth. From an original list of over 40 candidates, I selected three

finalists in the open-source and commercial categories. I then took some time to look at

them in detail, to determine which tool to recommend for the ongoing scale testing effort.

Since I have seen several questions about how these tools compare to each other on variousmailing lists, I 'm sharing my findings here in the hopes that others will find them useful.

My three finalists were Load Runner, from Mercury Interactive; JMeter, from the Apache

foundation, and The Grinder, an open-source project hosted on SourceForge.

1 INTRODUCTION

0Share More Next Blog Create Blog Sign In
http://googleads.g.doubleclick.net/aclk?sa=l&ai=C7SVs5rb3UazBJqfQ0AHHl4G4B_Dn9OMDgPS9lDvAjbcBEAEgpqG1BFDgp7SM-P____8BYMkGoAHgtefmA8gBA6gDAcgD0QSqBK8BT9Cl_xftV6pEE6XCS_EQ7ZfMvh4Q6UJUax8XPWUhvXTJPOztZxI9v_YM2-VTHIvbkaA4ZaYTzxirVzmI4qSq26A8i7lppiPAW1narRIuIg5yip2DESpPSmR3k7COzQFr1SzEE35MlqR2S4Mg1utHzFZPpa2qOhas-ZprwUGZCmsyZvo3Udk5l_rdVJaBEl4ZHcSSxwUoyNMlufE8akmTkGnboyISdHleNIMFe4DI4KAGA4AHiMqYGQ&num=1&sig=AOD64_0X09ISyzksVSHguyn7gmZlbLt7aw&client=ca-pub-0757442049305497&adurl=http://www.telerik.com/automated-testing-tools/%3Futm_content%3DZQXXAZHAPFhttp://blackanvil.blogspot.in/http://blackanvil.blogspot.in/http://blackanvil.blogspot.in/http://www.blogger.com/next-blog?navBar=true&blogID=27391056http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://blackanvil.blogspot.in/http://googleads.g.doubleclick.net/aclk?sa=l&ai=C7SVs5rb3UazBJqfQ0AHHl4G4B_Dn9OMDgPS9lDvAjbcBEAEgpqG1BFDgp7SM-P____8BYMkGoAHgtefmA8gBA6gDAcgD0QSqBK8BT9Cl_xftV6pEE6XCS_EQ7ZfMvh4Q6UJUax8XPWUhvXTJPOztZxI9v_YM2-VTHIvbkaA4ZaYTzxirVzmI4qSq26A8i7lppiPAW1narRIuIg5yip2DESpPSmR3k7COzQFr1SzEE35MlqR2S4Mg1utHzFZPpa2qOhas-ZprwUGZCmsyZvo3Udk5l_rdVJaBEl4ZHcSSxwUoyNMlufE8akmTkGnboyISdHleNIMFe4DI4KAGA4AHiMqYGQ&num=1&sig=AOD64_0X09ISyzksVSHguyn7gmZlbLt7aw&client=ca-pub-0757442049305497&adurl=http://www.telerik.com/automated-testing-tools/%3Futm_content%3DZQXXAZHAPFhttp://www.blogger.com/next-blog?navBar=true&blogID=27391056http://www.blogger.com/home#createhttp://www.blogger.com/http://www.blogger.com/


2/33


I found that I could use any of them and get a reasonably good amount of scale test

coverage. Each tool has unique things it does very well, so in that sense, there is no wrong

answer. Conversely, each of the tools I considered have unique deficiencies that will

impede or block one or more of the scenarios in our test plan. So there is no right answer

either any option selected will be something of a trade-off.

Based on this research, I recommended The Grinder as the tool to go forward with. It has asimple, clean UI that clearly shows what is going on without trying to do too much, and

offers great power and simplicity with its unique Jython-based scripting approach. Jython

allows complex scripts to be developed much more rapidly than in more formal languages

like Java, yet it can access any Java library or class easily, allowing us to re-use elements of

our existing work.

Mercury's Load Runner had a largely attractive feature set, but I ultimately disqualified it

due to shortcomings in these make-or-break areas:

Very high price to license the software.

Generating unlimited load is not permitted. With the amount of load our license

allows, I will be unable to effectively test important clustered server

configurations, as well as many of our surge scenarios.

Very weak server monitoring for Solaris environments. No support for monitoring

Solaris 10.

JMeter was initially seen as an attractive contender, with its easy, UI-based script

development, as well as script management and deployment features. It's UI is feature-rich

and this product has the Apache branding. It was ultimately brought down by the bugginessof it's UI though, as several of it's key monitors gave incorrect information or simply didn't

work at all.

2 SUMMARY OF RESULTS

Join this site

w ith Google Friend Connect

Members (1)

Already a member? Sign in

Followers

2011 (2)

2010 (1)

2008 (1)

2006 (4)

Blog Archive
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://googleads.g.doubleclick.net/aclk?sa=l&ai=C7SVs5rb3UazBJqfQ0AHHl4G4B_Dn9OMDgPS9lDvAjbcBEAEgpqG1BFDgp7SM-P____8BYMkGoAHgtefmA8gBA6gDAcgD0QSqBK8BT9Cl_xftV6pEE6XCS_EQ7ZfMvh4Q6UJUax8XPWUhvXTJPOztZxI9v_YM2-VTHIvbkaA4ZaYTzxirVzmI4qSq26A8i7lppiPAW1narRIuIg5yip2DESpPSmR3k7COzQFr1SzEE35MlqR2S4Mg1utHzFZPpa2qOhas-ZprwUGZCmsyZvo3Udk5l_rdVJaBEl4ZHcSSxwUoyNMlufE8akmTkGnboyISdHleNIMFe4DI4KAGA4AHiMqYGQ&num=1&sig=AOD64_0X09ISyzksVSHguyn7gmZlbLt7aw&client=ca-pub-0757442049305497&adurl=http://www.telerik.com/automated-testing-tools/%3Futm_content%3DZQXXAZHAPFhttp://void%280%29/http://blackanvil.blogspot.in/search?updated-min=2011-01-01T00:00:00-08:00&updated-max=2012-01-01T00:00:00-08:00&max-results=2http://void%280%29/http://blackanvil.blogspot.in/search?updated-min=2010-01-01T00:00:00-08:00&updated-max=2011-01-01T00:00:00-08:00&max-results=1http://void%280%29/http://blackanvil.blogspot.in/search?updated-min=2008-01-01T00:00:00-08:00&updated-max=2009-01-01T00:00:00-08:00&max-results=1http://void%280%29/http://blackanvil.blogspot.in/search?updated-min=2006-01-01T00:00:00-08:00&updated-max=2007-01-01T00:00:00-08:00&max-results=4http://void%280%29/http://blackanvil.blogspot.in/2006_11_01_archive.html


3/33


All the items in the tables below are discussed in greater detail in the following sections.

These tables are to give a quick overview

There are several features that are key to any scale testing effort. Items in this table are

key to our efforts. Not having any of these will seriously impact our ability to generate

complete scale test coverage.

Item Load Runner JMeter The Grinder

Solaris Monitoring - neutral neutral

Unlimited Load generation - + +

Supports IP spoofing + - +

Large download performance + - - *

* Multiple workarounds are being investigated, including calling native (libcurl) code for the

most intensive downloads.

Items in this section are not make-or-break to our test effort, but will go a long way to

making the test effort more effective.


Server monitoring mixed neutral neutral

3 Comparison Tables

3.1 Critical Items

3.2 Non-Critical Items

3.2.1 General

June (1)

Shootout: Load Runner vs The Grindervs Apache JMe...

May (1)

Travis Bear

View my complete profile

About Me
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://void%280%29/http://blackanvil.blogspot.in/2006_11_01_archive.htmlhttp://void%280%29/http://blackanvil.blogspot.in/2006_06_01_archive.htmlhttp://blackanvil.blogspot.in/2006/06/shootout-load-runner-vs-grinder-vs.htmlhttp://void%280%29/http://blackanvil.blogspot.in/2006_05_01_archive.htmlhttp://www.blogger.com/profile/04646417737587545733http://www.blogger.com/profile/04646417737587545733


4/33


Batch Mode - + +

Ease - Installation - + +

Ease Script Authoring + + mixed

Ease Running Tests neutral + neutral

Results Reporting + - -

Agent Management + + -

Cross Platform - + +

Cost - + +

Technical Level + + -

Stability/Bugginess neutral - neutral


Transaction power + neutral +

Custom protocols + + +

Out-of-the-box protocols + neutral -

Transaction aggregation + - +

3.2.2 Agents
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdf


5/33


Scalability of Agent + neutral neutral

Slow sockets + neutral -

External libs usable + + +

Load Scheduling + + -

Ease of porting JCS - neutral +


Scalability of Controller neutral - +

Real-time test monitoring + - neutral

Real-time load adjustment + - -

Script management + + -

Script Development Environment + + -

3.2.3 Controller


6/33


Mercury is extremely strong in this area for Windows testing. Unfortunately, it is very weak

in unix/Solaris. For windows hosts, Load Runner uses the native performance counters

available in perfmon. This allows monitoring myriad information from the OS, as well asmetrics from individual applications (such as IIS) that make their information available to

perfmon.

For Solaris hosts, Load Runner is restricted to the performance counters available via

rpc.rstatd. This means some very basic information on CPU and memory use, but not much

else. Note that Load Runner does not currently support any kind of performance monitoring

on Solaris 10.

JMeter has no monitoring built in. Thus, wrapper scripts are required to synchronize testdata with external perf monitoring data. This is the approach I used to great effect with our

previous test harness. The advantage of this method is I can monitor (and graph!) any

information the OS makes available to us. Since the amount of data to us is quite large, this

is a powerful technique.

The same wrapper-based approach would be required here as I detailed above for JMeter.

This is a make-or-break item. There are many scenarios I just can't cover if I can only open

a few thousand socket connections to the server.

Load runner restricts the number of vusers you can run. Even large amounts of money only

allow a licence for a modest number of users. Historically, the rate for 10,000 HTTP vusers

has been $250,000. However, on a per agent basis, load is generated very efficiently so it

may take less hardware to generate the same amount of load. (But for the money you spend

4 GENERAL

4.1 Server Monitoring -- Windows, Solaris, etc.

4.1.1 Load Runner

4.1.2 JMeter

4.1.3 The Grinder

4.2 Can generate unlimited load

4.2.1 Load Runner


7/33


on the load runner license, you could buy a LOT of load generation hardware!)

Since this is Free/Open Source, you may run as many agents as you have hardware to put

them on. You can add more and more load virtually forever, as long as you have more

hardware to run additional agents on. However, in specific unicast scenarios, such as

repeatedly downloading very large files (like PIPEDSCHEDULE), the ability of agents to

generate load falls off abruptly due to memory issues.

In this matter the Grinder's story is the same as JMeter. The limit is only the number of

Agents. The Grinder suffers the same lack of ability to effectively download large files as

JMeter. A workaround that uses native code (libcurl) to send requests is being investigated.

No. Hands-free runs can be scheduled with the scheduler, but multiple specific scenarioscannot be launched from the command line. This may be adequate for single tests; it's not

clear how this would work if a series of automated tests was desired.

Yes, the ability to do this is supported out of the box. However, it can only be run from a

single agent; the distributed testing mechanism requires the UI. So for automated nightly

benchmarks it may be ok, but for push-to-failure testing where much load is required, the UI

is needed. It would presumably be possible to have a wrapper script launch JMeter in batch

mode at the same time on multiple agents. This would achieve arbitrary levels of load, but

would not have valid data for collective statistics like total transactions per second, total

transactions, etc.

As with JMeter, a single agent can be run from the command line. See JMeter comments,

above.

4.2.2 JMeter

4.2.3 The Grinder

4.3 Can run in batch (non-interactive) mode

4.3.1 Load Runner

4.3.2 JMeter

4.3.3 The Grinder

4.4 Ease of Use


8/33


Installation takes a ton of time, a lot of disk space, and a very specific version of Windows.

But it's as simple as running a windows installer, followed by 3 or 4 product updaters.

For HTTP tests, Load Runner is strong in this category, with it's browser recorder and icon-

based scripts.

The UI of the controller is complex and a bit daunting. There is great power in the UI if you

can find it.

Be sure Sun's JRE is installed. Unpack the tar file. Simple.

Very quick. Start up the console, a few clicks of the mouse, and you are ready to generate

load. Add thread group, add a sampler, and you have the basics. Throw in an assertion or

two on your sampler to validate server responses.

Both distributed and local tests can be s tarted form the UI. A menu shows the available

agents, and grays out the ones that are already busy. Standalone tests can be started fromthe command line. JMeter wins this category hands down.

Installation is as simple as installing java, and unpacking a tar file.

4.4.1 Load Runner

4.4.1.1 Installation

4.4.1.2 Setting up Simple tests

4.4.1.3 Running Tests

4.4.2 JMeter




4.4.3 The Grinder




9/33

df d mi b PRO i Are you a developer? Try out the HTML to PDF API

Setting up tests, even simple tests, requires writing Jython code. So developer experience is

important. A proxy script recorder is included to simplify this. In addition, there are many

useful example scripts included to help you get started.

Involves configuring a Grinder.properties file, manually starting an agent process, manually

starting the console, then telling the test to run from within the console UI.

Having key graphs generated at the conclusion of a scale run, such as load over time, server

CPU, transactions per second, etc, can save a lot of tedium, since manually generating these

graphs from log files is quite time consuming.

Load runner has an excellent integrated analysis tool that can dynamically generate graphs

on any of the myriad performance counters available to it. The downfall of this approach is

that there are only a small number of performance metrics it can gather on Solaris. And

while I can gather additional server metrics using sar, vmstat, dtrace, iostat, mpstat, etc.,

integrating this information in to the load runner framework will be difficult at best.

JMeter does not gather anyserver-side performance metrics. But it can generate a limited

number of client-side graphs while the test is in progress. These graphs can be saved after

the tes is over. Fortunately, all the test data is written in a standard format. So it probably

makes more sense to generate all the desired graphs via shell scripts during post-

processing. This is the same approach I used with our previous test harness.

Like with JMeter, there are no graphs generated out of the box, but with the standard-

format log files, scripted post-production is reasonably straightforward, giving us a powerful

and flexible view on the results.


4.5 Results Reporting

4.5.1 Auto-generated?

4.5.1.1 Load Runner

4.5.1.2 JMeter

4.5.1.3 The Grinder

4.5.2 Analysis tools?

4.5.2.1 Load Runner


10/33

df di b PRO i Are you a developer? Try out the HTML to PDF API

Yes very powerful tool for doing analysis after a run. An infinite number of customized

graphs can be generated. These graphs can be exported into an html report.

Nothing included. I would want to transfer over some of the scripts used in our previous test

harness, or write a simple tool that dumps test data into a DB for post-analysis.

Nothing included. See the JMeter comments, above.

This works well in Load Runner; each agent can run as a service or an application,

simplifying management. Test scripts are auto-deployed to agents.

JMeter is good here. Each agent is a server that the controller can connect to at will in real-

time. Test scripts are automatically sent to each agent, centralizing management.

Grinder is the weakest here. The properties files that define how much load to apply, must

be manually deployed to all agents. A wrapper shell script like the one used by our previous

test harness could address this by always deploying the Jython scripts to the agents before

each run.

Not really. A subset of the complete agent functionality can be had for agents running on

Linux or Solaris. Non-windows agents run each vuser as a process rather than a thread,

reducing the amount of load an agent can produce. The controller and VUGen both are

Windows-only. And Load Runner is poor at measuring non-Windows server statistics.

4.5.2.2 JMeter

4.5.2.3 The Grinder

4.6 Simplicity of Agent management

4.6.1 Load Runner

4.6.2 JMeter

4.6.3 The Grinder

4.7 Tool is cross-platform4.7.1 Load Runner

4.7.2 JMeter


11/33

df di b PRO i A d l ? T t th HTML t PDF API

Yes. Java/Swing app is platform-agnostic.

Yes. This app is based on Java, Swing, and Jython. Like JMeter, it will run anywhere you can

set up a JVM.

Expect to pay in the low to mid six-figures for a license allowing any kind of robust load-

generation capacity. But that's not all, there are high ongoing support costs as well. For the

same kind of money I could get over 100 powerful machines to use as scale agents, as well

as associated network switches, cabling, etc.

Free. (Apache License)

Free. (Grinder License)

Load Runner has the widest audience of all these tools; perhaps not surprising given its

maturity as a commercial product. It's browser-recording and icon-based script development

give it the lowest technical barriers to entry of any of the three products. A QA engineer

with modest technical background and little to no coding sk ills can still be productive with

tool. And it's ability to load Windows .dll's and other libraries give it a power and flexibility

useful to developers and other more advanced users.

JMeter does not require developer skills to perform basic tests in any of the protocols it

support out of the box. A form-driven UI allows the user to design their own scenario. This

scenario is then auto-deployed to all agents during test initialization.

4.7.3 The Grinder

4.8 Cost

4.8.1 Load Runner

4.8.2 JMeter

4.8.3 The Grinder

4.9 Intended audience/technical level

4.9.1 Load Runner

4.9.2 JMeter

4.9.3 The Grinder


12/33df di b PRO i A d l ? T t th HTML t PDF API

While it's possible that a regular QA engineer could be used to run the console and perform

some testing, the tool is really more aimed at developers. This is the only tool of the three

that did not include any kind of icon-based or UI-based script development. At a minimum,

users will need to know how to write Python/Jython code to create simple test scripts, and

the ability to write custom Java classes may be required as well, depending on the scenario.

The controller crashes occasionally under heavy load, but this is infrequent and largely

manageable. Other than this, the product seems robust enough.

JMeter fares poorly in this area.

TODO

I found no issues with the Grinder, other than the previously-mentioned memory issue with

large file downloads.

4.10 Stability/Bugginess

4.10.1 Load Runner

4.10.2 JMeter

4.10.3 The Grinder


13/33df di b PRO i

Are you a developer? Try out the HTML to PDF API

Any arbitrary criteria can be set to define if a transaction passes. This includes but is not

limited to response time, contents of response body, response code, or just about anythingelse.

In JMeter, samplers generate your test requests. You can add a wide variety of assertion

types to any of your samplers. These will allow you assert on response code, match regular

expressions against the response body, assert on the size or md5sum of the response.

As with Load Runner, pass/fail criteria has merely to be defined within the test script.Criteria can be whatever you want.

Yes if you get away from the icon-based view in Vugen and go to the code level, you can

wrap anything you want in a transaction to get timing information, pass/fail data, etc.

Yes done through plugins.

Yes an API exists to easily wrap any Java or Jython method in a transaction.

5 AGENTS

5.1 Power of transactions

5.1.1 how flexible on what can be passed/failed?

5.1.1.1 Load Runner

5.1.1.2 JMeter

5.1.1.3 The Grinder

5.1.2 user-defined transaction/statistic types?

5.1.2.1 Load Runner

5.1.2.2 JMeter

5.1.2.3 The Grinder

5.2 Other Protocols

5.2.1 Which protocols are supported out of the box?

5 2 1 1 L d R


14/33df di b PRO iAre you a developer? Try out the HTML to PDF API

This varies by the type of license purchased, with each protocol having a separate cost and a

separate limit for the number of allowable VUsers. The potential number of protocols is

extremely high, including Java, ODBC, FTP, HTTP, and others.

Supports several protocols out of the box:

jdbc

http

ftp

jndi

The Grinder only supports HTTP out of the box.

Yes. There are multiple ways to do this. You can implement your own protocol handler in a

.dll or in Load Runner's pseudo-c. Then you can invoke this handler from any type of VUser

that you have a license for. Alternately, unless your protocol is something uncommon, you

can probably buy a pre-existing implementation of your protocol, and licenses for VUsers to

run this protocol.

Yes. An external Java plugin that supports your protocol must be added in to JMeter to

support this.

Any protocol can be tested with the Grinder. An HTTP plugin is included. In other cases, you

will create a separate Java class that implements a handler for your protocol. In your test

5.2.1.1 Load Runner

5.2.1.2 JMeter

5.2.1.3 The Grinder

5.2.2 Can transactions wrap custom (non-http) protocols? Can

transactions wrap multiple (http or other) requests to the server?

5.2.2.1 Load Runner

5.2.2.2 JMeter

5.2.2.3 The Grinder

i t ill thi J l i G i d t t bj t Y t l i d/i k d


15/33df di b PRO iAre you a developer? Try out the HTML to PDF API

script, you will wrap this Java class in a Grinder test object. Your protocol is used/invoked

by calling any method you want from your java class v ia the tes t wrapper. The wrapper will

pass/fail the transaction based on response time.

This default behavior can be overridden with additional code in your Jython script. For

example, after invoking your protocol method, you could inspect the state of your Java

object and pass/fail the transaction based on information there.

I have typically seen libraries like Apache's HTTPClient max out the CPU to 100% when it's

conducting high-bandwidth, large fi le downloads. The library supports high bandwidth use

and many transactions per second just fine, but has issues with repeated large file

downloads.

Per-agent load generation capacity is strong. Licensing constraints may limit actual load

generated.

With the exception of the high-bandwidth case, per agent capacity is good.

Runs out of memory when repeatedly downloading large documents in many threads.

Currently, there does not seem to be a workaround inside The Grinder itself. However, with

my previous test harness I was able to work around this same issue by calling native code,

and there is reas on to believe that approach may work here as well.

Assuming a large range of valid IP addresses assigned to the agent machines, does the test

harness support binding outgoing requests to arbitrary IP addresses? The ability to support

this is critical for out test effort. If all broker requests come in from the same IP address,

the broker thrashes unrealistically as it continually updates customer settings.

5.3 Capacity of single agent to generate load, particularly in

high-bandwidth scenarios

5.3.1 Load Runner

5.3.2 JMeter

5.3.3 The Grinder

5.4 Can support IP spoofing

5.4.1 Load Runner

Y ( li k i di 1)


16/33

Are you a developer? Try out the HTML to PDF API

Yes. (see link in appendix 1)

JMeter is weak here. There is a new mechanism (not yet released but available in nightly

builds) where outbound requests can round-robin on a predetermined list of local IPs. This is

not good enough for Fat Client simulation.

The local IP address to bind the outbound request to can be specified in the Jython scripts.

This is just what I need.

Load Runner supports this out of the box.

JMeter does not support this out of the box, but there is a slow socket implementation in

the wild, written for the Apache HTTP Client (which JMeter uses), that should be possible to

drop in fairly easi ly.

The Grinder does not support this. It may be possible with additional code hacking, but the

path for this is not clear. Their third-party HTTP implementation means writing a custom

solution may be challenging. Perhaps it would be possible using JNI and libCurl, although the

author of the libCurl binding suggest there may be a memory leak in the C layer.

Windows .dll's may be loaded. Home-made libraries written in Load Runner's pseudo-C libs

work fine as well. Additionally, function libraries can be embedded directly in the v irtual user

5.4.2 JMeter

5.4.3 The Grinder

5.5 Can support variable connection speed/bandwidth

throttling

5.5.1 Load Runner

5.5.2 JMeter

5.5.3 The Grinder

5.6 Can run arbitrary logic and external libraries within

agent

5.6.1 Load Runner

script


17/33

pdfcrowd.comopen in browser PRO version Are you a developer? Try out the HTML to PDF API

script.

External Java libraries can be accessed via the plugin architecture.

The Grinder offers lots of flexibility for loading and executing third party libraries. With

Jython, any Java code may be called, and most python code may be run unchanged. And

there is a decent collection of example scripts that comes with the Grinder distribution.

Load Runner has a powerful, UI-based scheduling tool which allows you great flexibility to

schedule arbitrary amounts of load over time. Load can be incrementally stepped up and

stepped down, by single threads or entire groups. There is a graphical schedule builder that

can generate schedules of arbitrary complexity.

JMeter has UI-based scheduling that allows per-thread startup delays, as well as runs that

start in the future. JMeter tests can run forever, for a specified time interval, or for a

specified number of iterations for each thread.

No per-thread ramp-in. No generic scheduling tool. Primitive per-process (instead of per-

thread) scheduling is possible but use of this feature probably reduces an Agent's maximum

load-generation capacity, as the overhead of running a new process is far greater than theoverhead of creating a new thread.

5.6.2 JMeter

5.6.3 The Grinder

5.7 Scheduling

5.7.1 Load Runner

5.7.2 JMeter

5.7.3 The Grinder

6 CONTROLLER


18/33


Load runner probably handles as much or more real-time data as any product out there. But

they do it effectively. If you give the controller a beefy box to run on, you should have no

problems.

Limited. The amount of transaction monitors you can have running is configurable. If more

that one or two are going and the agents are producing a lot of transaction data, the UI

takes all the CPU, bogs down and becomes unusable.

The grinder does very well here, probably better than Load Runner. By design, the agents

only send a limited amount of real-time data back to the controller during a tes t run. And

the sampling period is adjustable with a big friendly slider. This is a handy feature I didn't

fully appreciate at first if the network bandwidth numbers are updating too fast, it's hard

to see how many digits are in the number before it updates again. But with the slider, you

can lock that number down for enough time to really consider it.

Load Runner features very strong real-time monitoring in the controller. Client side graphs,

such as total transactions per second, errors per second, can be displayed next to server

side graphs like CPU use and disk activity. The user can drag and drop from a list of dozens

of graph types.

Basic, table-based monitoring similar to what is in our previous test harness works properly.

Other monitors threw null pointer exceptions.

6 CONTROLLER

6.1 Ability of Controller to handle high volume of agent data

6.1.1 Load Runner

6.1.2 JMeter

6.1.3 The Grinder

6.2 Real-Time Monitoring (Controller)

6.2.1 Load Runner

6.2.2 JMeter

6.2.3 The Grinder

The Grinder is good here. It has simple, sliding performance graphs for all transactions in


19/33


The Grinder is good here. It has simple, sliding performance graphs for all transactions in

one tab. These graphs are similar to what you see in the Windows Task Manager, where

performance metrics older than a given amount of time slide off the left side of the graph.

In addition, as in our previous test harness or JMeter, there is numeric data that periodically

updates in a table.

Sometimes while a test is in progress, you want to make adjustments. Increase the load.

Decrease the load. Bring another agent online.

Load Runner wrote the book on this topic, with its highly-flexible ability to start and stop

load in the middle of a test, with individual agents, groups of agents, or the entire set of

agents.

JMeter has the ability to interactively start and stop load on an agent-by-agent basis. It

cannot interactively be done at the per-thread level, but agents and thread groups can haveschedulers assigned to them.

The Grinder console does not have the ability to dynamically adjust the levels of load being

generated by the agents. Coupled with its lack of a scheduler, this makes the Grinder the

least flexible of the three tools when it comes to interactively setting load levels.

Yes.

Yes.

Yes.

6.3 Real-Time Load Adjustment

6.3.1 Load Runner

6.3.2 JMeter

6.3.3 The Grinder

6.4 Controller-side script management/deployment

6.4.1 Load Runner

6.4.2 JMeter

6.4.3 The Grinder

6 5 Can write simple scripts in the UI?


20/33


Load Runner comes with a powerful script-development tool, VUGen. This gives the test

developer the option of developing icon-based test scripts, as well as the traditional code-

view development environment. In addition. Load Runner can record web browser sessions

to auto-generate scripts based on the recorded actions.

Scripts are based on XML. They can be written in your preferred text editor, or created in an

icon-based UI in the controller window. I found this feature to be both easy to use and

surprisingly flexible. There is also a recorder feature to let you interactively create your

scripts.

The Grinder is the weakest of the three here. It does have a TCP Proxy feature that can

record browser sessions into Jython scripts. But there is no integrated graphical

environment for script development

6.5 Can write simple scripts in the UI?

6.5.1 Load Runner

6.5.2 JMeter

6.5.3 The Grinder

7 CONCLUSION


21/33


I selected The Grinder due to several make-or-break issues. However, each tool has unique

strengths and weaknesses. Which tool is ultimately best for you depends on a number of

things, such as:

Does you budget allow for an expenditure ranging from several tens to hundreds

of thousands of dollars?

Will you be testing in a windows-only environment?

What is the technical level of your scale testers?

Both of the open source projects have merits, but neither one is ideal. My approach will be

to work with the Grinder development team to resolve the most serious offenders.

7 CONCLUSION


22/33


Load Runner system requirements (controller must be on Windows!)

http://www.mercury.com/us/products/performance-center/loadrunner/requirements.html

Linux/Solaris server monitoring (weak)

http://www.mercury.com/us/products/performance-center/loadrunner/monitors/unix.html

JMeter home page

http://jakarta.apache.org/jmeter/

JMeter Manual

http://jakarta.apache.org/jmeter/usermanual/index.html

The Grinder home page

http://grinder.sourceforge.net/

The Grinder Manual

http://grinder.sourceforge.net/g3/getting-started.html

Windows IP address multi homing

http://support.microsoft.com/kb/q149426/

8 Appendix 1 Additional information
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.mercury.com/us/products/performance-center/loadrunner/requirements.htmlhttp://www.mercury.com/us/products/performance-center/loadrunner/monitors/unix.htmlhttp://jakarta.apache.org/jmeter/http://jakarta.apache.org/jmeter/usermanual/index.htmlhttp://grinder.sourceforge.net/http://grinder.sourceforge.net/g3/getting-started.htmlhttp://support.microsoft.com/kb/q149426/


23/33


These are some of the distinguishing features of each product:

Cool with Load Runner

highly developed, mature product

strong support

It is complex, but feature-rich

Problems w/ Load Runner

Extreme cost, both up front and ongoing

Limited load generation capacity based on license/key.

Limited ability to monitor server stats outside of windows.

Cool w/ Grinder

Jython scripting means rapid script development

Jython simplifies coding complex tasks

Good real-time feedback in the UI in most tabs.

Sockets based agent/controller communications. Trouble-free in our testing.

Problems w/ Grinder:

(Since this original article was posted, many of these issues have been addressed. See the

blog entry titled "The Grinder: Addressing the Warts.")

no scheduling; load is all-or-nothing

no slow sockets, no prospects for easily fixing this

Memory failures in a few large file download scenarios.

9 Appendix 2 Distinguishing features

Cool w/ JMeter:
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://blackanvil.blogspot.com/2006/11/grinder-addressing-warts.html


24/33


Posted by Travis Bear at 11:34 PM

Less technical expertise required

Overall more slick or polished feel availability of startup scripts, more

utility in the UI.

Problems w/ JMeter:

Limited feedback in the UI when the test is running

Memory and CPU issues when downloading very large files

The UI is buggy. Big pieces, including monitors, just don't work. Many Null

Pointer Exceptions in the log, etc.

23

Recommend this on Google

17 comments:

Eran Witkon said...

Very interesting review of these tools.

I was wondering why did you pick these tools in specific and did not review other

tools such as WebLOAD. Obviously, I am not objective but using your criteria

RadViews WebLOAD tool is giving a good fight here and should be counted as well.

Eran WitkonVP R&D

Radview - The Smart Choice in Web Application Testing

Mobile: +972-544-600677

mail: [email protected]

http://www.radview.com

4:34 AM

Travis Bear said...

I originally wrote this review for an internal audience at Siemens, since we needed
http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.blogger.com/profile/04646417737587545733http://blackanvil.blogspot.in/2006/06/shootout-load-runner-vs-grinder-vs.htmlhttp://www.blogger.com/email-post.g?blogID=27391056&postID=115035332067654999http://www.blogger.com/share-post.g?blogID=27391056&postID=115035332067654999&target=emailhttp://www.blogger.com/share-post.g?blogID=27391056&postID=115035332067654999&target=bloghttp://www.blogger.com/share-post.g?blogID=27391056&postID=115035332067654999&target=twitterhttp://www.blogger.com/share-post.g?blogID=27391056&postID=115035332067654999&target=facebookhttp://www.blogger.com/profile/01705860445460422801http://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1162730040000#c116273004887473695http://www.blogger.com/profile/04646417737587545733http://www.blogger.com/profile/04646417737587545733


25/33


to pick a new tool. I was already somewhat familiar with Load Runner, Silk performer,

and JMeter before I started. I did a day or so on Google getting a large list of other

test tool candidates, and then researching the candidates in greater detail. I started

with about 40 tools which i narrowed down to 3.

Based on the white paper at your web site, it appears that Web Load is a tool for

testing Microsoft web servers. While I have nothing against Microsoft, the solution

we are developing at Siemens is based on Solaris and Java. All our load generation

hardware is running Linux. So it appeared that your product would not be a good fit

with our specific environment. True?

If I have the wrong idea, I'd be interested to know more -- how does it monitor Linux

and Solaris system resources such as CPU, disk activity, etc., what kind of graphs can

be generated, etc., etc.

1:53 PM

Eran Witkon said...

Thanks for your feedback.1) First lets start with the fact that Siemens in Europe are a good client of ours so

you might get some inside information regarding our offering.

2) WebLOAD is a multi platform load testing tools targeted for any RIA application. I

will double chec k our white papers to see why you got that impression.

3) WebLOAD engine run on all platforms including Linux and Solaris (the execution

console and IDE runs on windows only).

4) WebLoad can test any Java/J2EE based applications, is certified by BEA WebLogic.

5) We are adding JMX support for our coming Q1 release.

6) WebLOAD support different performance measurements see table below:

Current PMM support

BEA WebLogicIBM WebSphere

Microsoft Active Server Pages

Microsoft ASP .NET

Oracle

Microsoft SQL Server

Windows Platform

Unix platforms using SNMP

Unix platforms using RSTATD

Unix UC-Davis

Windows .NET Framework

Microsoft IIS
http://www.blogger.com/profile/04646417737587545733http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1162763580000#c116276359031556443http://www.blogger.com/profile/01705860445460422801


26/33


Microsoft IIS

Sun One (former iplanet 6)

Sun One (former iplanet 4)

Netscape LDAP

General Performance M onitor

General SNMP

General Unix

7) WebLOAD includes 3 groups of statistics that provide a thorough view of the client

side experience:

The first group includes more than 30 pre-defined c lient side statistics (counters,timers and per-time unit statistics) which brings a very detailed view of HTTP

protoco l level statistics. This group includes metrics such as hits per sec ond,

connect time, process time, page time, round time and many more.

The second group inc ludes automatic transactions which report timers, on any

page and HTTP error that was part of the session. In this group you can find the

time it took to receive each page in the session, the number of times the session

rece ived spec ific error such as 404, etc.

The third group includes user-defined counters, timers and transactions. In

WebLOAD the user can add custom timers and counters to the test script. The user

can also wrap parts of the script with a transaction block and collect statistics on

that blocks (e.g. "how much time it takes to insert a new employee to theapplication")

In addition to the client side statistics WebLOAD can collect a wide range of server

side statistics using its PMM mechanism as described above.

All WebLOAD statistics are collected on-line and can be viewed while the session is

running and anytime afterwards. The statistics can be displayed as graphs or textual

information. All statistics can be exported to various formats such as Excel, tabbed

file, HTML.

For more information please refer to WebLOAD's user guide "WebLOAD statistcs

reports".

Eran Witkon

VP R&D

Radview - The Smart Choice in Web Application Testing

Mobile: +972-544-600677

mail: [email protected]



27/33



4:09 AM

Anonymous said...

I enjoyed reading your Shootout article but wanted to point out a potential

discrepancy.

You state that LoadRunner only supports monitoring via rstatd and has no supportfor Solaris 10. It is my understanding that the only monitoring support JMeter and

Grinder have is also through rstatd on any unix server. This leads me to a different

conc lusion regarding your 3 candidates and their support for monitoring. All three

require rstat -d running on the Unix servers and if you want more than the basic

counters supported by LR you will need to write scripts no matter which of these

tools you select. I would rate LoadRunner, via the Controller or the LR Analysis

application, superior to the suppor t for either Grider or Jmeter.

I also feel that your conclusions make it sound like your evaluation disqualifies LR for

some show stopping functional reasons, but then goes on to say that 2 of the

reasons are licensing and licensing. Certainly, a tool that costs $100k is not going towork on a minimal budget. However, many of LR features are far superior to both

Jmeter and Grinder. Most of these features are designed to save time in the

preparation, execution and analysis of performance tests.

In our consulting work we are evaluating 3 Open Source tools and our preliminary

findings show the monitoring and analysis side to be the most glaring area of

functionality not handled by Open Source solutions. It is my experience (for the past

14 years) that LR saves countless hours on collating and analyzing results.

So for the budget conscious Open Sourc e tools allow for a lower entry point into

performance testing. However, you will spend more time creating, debugging, andmuch more time collecting, collating and analyzing data with the current Open

Source solutions. Any analysis of commercial grade tools vs. Open Source tools

should include a total cost of ownership line item to present the whole picture.

12:41 PM

Travis Bear said...

Monitoring and analysis is one area where Load Runner is very strong, and I gave it a

good rating in this area. But it's only strong if your target server runs Windows. Since
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1164629340000#c116462939910322202http://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1167943260000#c116794331842883275http://www.blogger.com/profile/04646417737587545733


28/33


g g y g y g

our project runs on Solaris, we have to live with the limitations of rstatd, or roll own

monitoring. And since rstatd is so limiting, we have had much better by logging to

disk the performance metrics available through vmstat, sar, iostat, and prstat.

I hope this analysis is not seen as a dis on Load Runner -- if you've got the cash and

your server runs Windows, I know from experience that it is a flexible, powerful

solution. We opted for the Grinder due to licensing and due to Load Runner's

weakness outside the Windows environment.

Speaking a bit more about server monitoring: JMeter and The Grinder do not do ANY

performance monitoring -- either through rstatd or any other mechanism, so if you

go down that road, then there is no avoiding that you will have to do the monitoring

yourself.

The server monitoring solution we came up with in our shop is a little crude, but it's

fast, it's general-purpose, and it works. I'll be making it available under an open-

source license when I have some time. The idea is that we use a wrapper script that:

* connects to the target server(s) via ssh

*starts logging perf data (vmstat, iostat, sar, and prstat) to disk on the remote server

* starts the load generation UI, blocks until the UI exits,

* ssh back to the target server, runs awk scripts to convert the various log files into

.csv files

* donwloads the .csv files

* graphs the .csv files with an automated tool such as ploticus or JFreeChart.

A similar approach to this would probably work against Windows servers as well, just

replacing the solaris tools mentioned above with whatever perfmon counters you are

interested in.

2:09 AM

Boris said...

very interesting review. I am also Siemens employee in Germany and back in 2000 my

department made decision to purchase LoadRunner.

Just one remark about LoadRunner test automation: Load Runner is offering OLE
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1168510140000#c8337103088940986393http://www.blogger.com/profile/04720452481223374086


29/33


g

automation of controller and it's agents what would theoretically enable automated

performance test with this tool

4:52 AM

Matt said...

Great review, and an interesting read.

I have used LR for many years now, but have recently moved on to Grinder and

JMeter.

I think LR (and the like) is a great consultants or IT Department tool. It has a low

learning curve and is suited for QA's and non-developers. I think it is also very good

for short-term Performance Projects.

However, the reason why we have implemented Grinder and JMeter here is not just

to do with cost.

Everything we are developing is HTTP - which of course helps, but also we areimplementing automated performance tests using these tools after nightly builds.

And of course we can extend the functionality of these harnesses to perform

scheduled, adhoc, and multi scenario based Performance Tests with ease.

It's true we have put quite a bit of effort in auto-manipulating and graphing output of

these tools, as well as scripting Linux native performance tools.

But the outcome of this is more statistical information and system metrics then LR or

any other tool could give us.

Another interesting listen is - http://video.google.co.uk/videoplay?docid=-6891978643577501895&q=google+benchmark+testing

Oh.. a quick grinder fix we found when wanting to increment thread ramp up:-

Add a parameter to your properties file and alter your c ode to:-

grinder.sleep(grinder.threadID *

int(grinder.properties.getProperty("app.processIncrementInterval")))

6:18 AM
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1171457520000#c8130360267970987821http://www.blogger.com/profile/12835570088352901164http://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1175001480000#c6883352126616067245http://www.blogger.com/profile/01812927346511062657


30/33


Fabrizio said...

Very interesting review of these tools. Just 2 things:

- I think it would be more interesting doing a comparison viewing also other Mercury

products you can use with LoadRunner like M ercury Diagnostic for J2EE

- It's important you can create a ramp in any testing product for a better testing

strategies; for me this is a mustptjuh

2:17 PM

Travis Bear said...

The Grinder does support ramping in load over time (as Matt points out, above), if

that's what you're talking about.

I'm not sure about the other M ercury produc ts. My expertise is in scale testing, so

I'm probably not the best c hoice to review Mercury Diagnostic, or other ancillary

stuff.

5:01 PM

Pimp Daddy Roland said...

Nice work mate, you've hit some of the metrics I'm interested in and saved me some

serious time, cheers!

Roland

2:02 PM

saurabh mishra said...

Is VMSTAT Better then Rstatd ... can we rstatd only with monitoring tool...?

8:46 AM

Oliver Erlewein said...

Hi

Good article.
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.blogger.com/profile/01812927346511062657http://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1182201420000#c7109669777025945417http://www.blogger.com/profile/04646417737587545733http://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1182384060000#c721596521861094566http://www.blogger.com/profile/13984960208320458639http://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1195509720000#c219069191006729477http://www.blogger.com/profile/04553841006877954895http://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1204044360000#c3123312828739196926http://www.blogger.com/profile/16862713267611644928


31/33


We've dec ised on JMeter for our projects and have spent our money on hardware

and scripting the distribution and charting of graphs. All in all we're still miles away

from what LR would have cost us.

Nice side effect is that I can give every customer the whole installation after the

project ends. No licensing problems.

Our projects are usually quite small so we don't need all the furly stuff LR offers. It

would be like editing a TXT file with Word.

I find the most effort is spent in finding & fixing the issues so anything that gets me

to that point is OK. I suppose grinder would have worked just as well for us.

Monitoring is a problem and I think I will take some of the stuff from the comments

and have a go at it.

If my company will allow it I plan to release the number crunching parts as Open

Source sometime soon so that anybody can use it. It is currently based on

processing the JTL files with GnuPlot and GhostScript (PDF output). It runs on Linux

and Cygwin.

Cheers

Oliver

New Zealand

2:57 PM

Andrey Isaev said...

A good addition to The grinder would be using it together with GrinderStone: Eclipse

plugins that allows to debug created scripts and thus achieve better quality ofscripts. This will add some point for The Grinder to compete with LoadRunner

powers.

You may visit GrinderStone home page here:

http://code.google.com/p/grinderstone/

-------

Andrey Isaev

Andruschuk Borislav

1:31 AM
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1214431020000#c8096485853289672696http://www.blogger.com/profile/11634801021071940869http://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1222763460000#c6341575241196329431


32/33


Post a Comment

Yura Kupitman said...

I agreed with the point that this review is very interesting. I had no personal

experience with WebLOAD, but the impression as well feedbacks from my colleagues

are really good. I'm just doubt that nobody mentioned slamd

http://www.slamd.com/index.shtml project from SUN. It is very impressive framework

that supports distributed load as well.

4:34 AM

dime said...

In your post you have mentioned that it can run in batch more. Can you provide me

a documentation using which I can automate the process everyday..

2:44 AM

Travis Bear said...

Dime, this means running the Grinder with no console. Just start the agent (from the

command line) and let it run. That's all there is to it. If you need specific details, this

is covered in the grinder documentation.

-Travis

8:44 AM

Dzmitry said...

I do not think that it is a big problem, that JMeter has no good reporting tool out-of-

box. Fortunately, there are some solution, how to workaround it. One of them is to

use ,

jmeter cloud

that has rather well-developed tools for analyzing results.

2:36 AM

C t Li k

Links to this post
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://www.blogger.com/comment.g?blogID=27391056&postID=115035332067654999http://www.blogger.com/profile/09812405962247744914http://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1253187288989#c3660247683183485963http://www.blogger.com/profile/16525535053117095845http://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1262861056063#c4644579243609736145http://www.blogger.com/profile/04646417737587545733http://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1262882683574#c2770594145165923581http://www.blogger.com/profile/16398673109757954251http://blazemeter.com/http://blackanvil.blogspot.com/2006/06/shootout-load-runner-vs-grinder-vs.html?showComment=1360319785791#c3166107515826019999http://www.blogger.com/blog-this.g


33/33


Newer Post Older PostHome

Subscribe to: Post Comments (Atom)

Create a Link

Simple template. Powered by Blogger.
http://pdfcrowd.com/http://pdfcrowd.com/redirect/?url=http%3a%2f%2fblackanvil.blogspot.in%2f2006%2f06%2fshootout-load-runner-vs-grinder-vs.html&id=ma-130730085152-0e501bb4http://pdfcrowd.com/customize/http://pdfcrowd.com/html-to-pdf-api/?ref=pdfhttp://blackanvil.blogspot.in/2006/11/grinder-addressing-warts.htmlhttp://blackanvil.blogspot.in/2006/05/its-all-about-scalability.htmlhttp://blackanvil.blogspot.in/http://blackanvil.blogspot.com/feeds/115035332067654999/comments/defaulthttp://www.blogger.com/blog-this.ghttp://googleads.g.doubleclick.net/aclk?sa=l&ai=CzNKw5rb3UYeoFPLW0AHOoYGgDZKE9-YB0qy_kSnAjbcBEAEgpqG1BFDoxL3R_f____8BYMkGoAGcmcH_A8gBAqgDAcgD0ASqBKsBT9B_i5-ozeU-AZ2P6XY54McEFGYJppCdABVyi5iLR8ISOLqhRj_xtDiDrmivh388qAhVtWnIgZfbVDqBmFCBJ4H9R5_jOcZpMzDmEi0VgUzb0Ru9YcrAaMWPHPj7HZIxmrFkbc0VrHk-QFldjnj4v9BFY_O3mXn1-l3uDDxFN9WZaTWmdc65m_1ebk6d8M3Z1PdluIpSRAPaVQJWO3wjwdwINJyjRY4eEuoDoAYCgAfM5j4&num=1&sig=AOD64_176TTS1UJL7KyrUh2pFSwwm8eIew&client=ca-pub-0757442049305497&adurl=http://www.paessler.com/webstress/%3Fsource%3Dadwords%26campaign%3Dwebstress_image_ads_US%26adgroup%3Dwebserver_benchmark_testing%26adnum%3Dgif_webstress_limit_300x239_011%26placement%3Dblackanvil.blogspot.inhttp://www.blogger.com/

Blackanvil Blogspot in 2006 06 Shootout Load Runner vs Grind

Documents

Transcript of Blackanvil Blogspot in 2006 06 Shootout Load Runner vs Grind