First a bit of history: Netscape Messaging Server includes Mailstone to allow customers to test their mail server installations before public roll out. Mailstone is a benchmark framework for testing mail protocols like POP, IMAP, and SMTP. However, all the new directions for Mailstone are in protocols other than mail. So we changed the name to mstone (think multi-stone) and made the source available so people can develop new tests.
Mstone gives you a lot of detailed information. It does not spit out a standardized "POPmark" number or anything like that. You could standardize a test sequence and combine the detailed numbers using a standard formula, but that's a task for people with great patience.
In order to simulate high load levels and to randomize accesses, mstone can run on multiple test client machines and the results automatically combined when the test is complete. Each client can have multiple processes and multiple threads (limited only by the OS and hardware). Simultaneous connection counts of 100,000 have be performed and higher levels are possible. For long lasting protocols like IMAP and WMAP, this is important for properly stressing the servers.
Mstone currently runs on recent versions of: Linux, Solaris, AIX, OSF, and HPUX. Any OS with POSIX threads support should be an easy port (mostly of the build system). The test client machines can each be running different operating systems. Common utilities like perl, gnuplot, and gd are needed to run mstone.
Tests are defined through workload files and command line arguments. Everything is designed to be repeatable and easily sequenced. The results are available in HTML, plain text, and as a spreadsheet import file (CSV).
The 4.2 release is the best tested. SendMail contributed a large number of updates (thanks!). When these are fully integrated, version 5.0 will be released. See NEWS.txt