Performance Benchmarks
Here are the results of several performance benchmarking tests run on ten different Common Lisp implementations running on a MacBook Pro, a dual P4 3.0 GHz, a P3 1.1 GHz, a G3 400 MHz and an UltrasparcIIe 500 MHz. The tests run include

  • the well-known Gabriel benchmarks
  • hashtable exercising
  • READ-LINE exercising
  • mathematical functions: factorial, fibonacci, ackermann's number
  • some bignum intensive code from Bruno Haible
  • various aspects of CLOS: time taken to compile and execute DEFCLASS forms which create a class hierarchy, instance creation time, time taken to compile and execute DEFMETHOD forms, execution of method combinations, both with and without :after methods.
  • various operations on arrays, bitvectors and strings.
  • Odd Benchmarks that test various features of CL against each other

Except for the CLOS code, timings do not include compilation time. The garbage collector is run before each test to try to make the timings more repeatable. Remember that the only real benchmark is your application: this code is only representative of real-life programs to a limited extent. For certain targets, we assume that the times reported by GET-INTERNAL-RUN-TIME and GET-INTERNAL-REAL-TIME are accurate.

The code for these performance benchmarks can be downloaded from http://www.chez.com/emarsden/downloads/cl-bench.tar.gz. (For some reason this seems to error out as a direct link but not when accessed from the following page: http://www.chez.com/emarsden/downloads/.) A Makefile and a Bourne shell are used to run the benchmarks. A Windows user may not have those installed or wish to use them; in that case, create a .bat file in the toplevel cl-bench directory. Examples:

rem Clozure Common Lisp 1.6 32-bit ran successfully
set lisp=C:\ccl16\wx86cl.exe
set opt=--batch --no-init
set optimize=(declaim (optimize (speed 3) (space 1) (safety 0) (debug 0) (compilation-speed 0)))
if exist files/optimize.txt goto got_olisp
cd files
echo %optimize% > optimize.txt
for %%i in (*.lisp) do copy optimize.txt+%%i %%i_o
rename *.lisp_o *.olisp
cd ..
:got_olisp
%lisp% %opt% --load sysdep/setup-openmcl.lisp --load do-compilation-script.lisp --eval "(quit)"
%lisp% %opt% --load sysdep/setup-openmcl.lisp --load do-execute-script.lisp --eval "(quit)"

rem Steel Bank Common Lisp 1.0.47 ran successfully
set lisp="C:\Program Files\Steel Bank Common Lisp\1.0.47\sbcl.exe"
rem Heap size must be 1GB or STRING-CONCAT will exhaust it.
set opt=--dynamic-space-size 1000
set optimize=(declaim (optimize (speed 3) (space 1) (safety 0) (debug 0) (compilation-speed 0)))
if exist files/optimize.txt goto got_olisp
cd files
echo %optimize% > optimize.txt
for %%i in (*.lisp) do copy optimize.txt+%%i %%i_o
rename *.lisp_o *.olisp
cd ..
:got_olisp
rem --script is required.  --load blah --eval "(quit)" will not work.
%lisp% %opt% --load sysdep/setup-sbcl.lisp --script do-compilation-script.lisp
%lisp% %opt% --load sysdep/setup-sbcl.lisp --script do-execute-script.lisp

Not all Windows CLs can run cl-bench. As of 15 April 2011:

  • Allegro CL 8.2 Express Edition will not run cl-bench because the heap size is restricted.
  • Lispworks Personal Edition 6.0.1 does not have a console available. Attempts to run cl-bench manually through the IDE failed because the listener doesn't appear to understand hierarchical directories. Also the Personal Edition has a heap size restriction so it will probably fail anyways.
  • The CLISP 2.49 Windows native binary does not have a POSIX module. cl-bench will fail with an error 'there is no package named "OS"'. Also, watch out when installing this binary. The CLISP Windows binary installer will completely destroy your system %PATH%, if you check the "add CLISP to your path" option. If you notice that your system path has been destroyed before you reboot, you may be able to recover an older copy of it by starting regedit and looking under HKLM\System\ControlSetXXX\Control\Session Manager\Environment\Path, where "XXX" is a number such as "004". Otherwise once you reboot you are SOL.
  • The CLISP 2.48 Cygwin package provides a POSIX layer, so most benchmarks do run. However, STRING-CONCAT fails because a desired string length of 6553600 exceeds the supported maximum length. This test can be skipped by adding #-(or clisp) before the STRING-CONCAT test in do-execute-script.lisp.
Here is an example of the type of results you can obtain, for x86 and SPARC platforms. To interpret these numbers: the number in the Reference column is a number of seconds taken to execute that test. The other numbers are the relative performance of that implementation, so for example a number of 1.3 means 30% slower than the reference implementation. A number of -1 means that the corresponding test was not run on that implementation, either because it does not support the given feature (certain types of method combination for CLISP, for instance), or because of implementation limits (heap size limit on LispWorks Personal, for example).


+-- MAC-Pro 2x2.66 GHz Dual Core Xeon
Benchmark                 Reference  ECL 8  SBCL   CMU C  CLISP
-----------------------------------------------------------------------
COMPILER                 [      0.65]   2.92   1.76   2.05   2.09
WALK-LIST/SEQ            [      0.03]   2.79   0.51   0.82   0.49
WALK-LIST/MESS           [      0.02]   5.70   1.06   0.99   5.60
BOYER                    [      7.97]   0.11   0.29   0.65   4.50
BROWSE                   [      0.47]   1.00   0.43   0.98   3.00
DDERIV                   [      0.44]   1.86   0.36   1.31   4.24
DERIV                    [      0.48]   1.79   0.50   1.49   4.33
DESTRUCTIVE              [      0.29]   2.59   0.60   1.18   4.71
DIV2-TEST-1              [      0.71]   1.39   0.34   1.52   4.25
DIV2-TEST-2              [      0.79]   1.67   0.56   1.65   4.66
FFT                      [      0.06]  13.75   0.50   0.73  53.54
FRPOLY/FIXNUM            [      0.48]   2.03   0.50   0.69   7.20
FRPOLY/BIGNUM            [      1.00]   0.62   0.22   0.53   1.64
FRPOLY/FLOAT             [      0.70]   1.99   0.46   0.90   5.51
PUZZLE                   [      0.34]   8.24   0.55   0.39  22.65
TAK                      [      0.18]   1.62   1.51   1.48  15.63
CTAK                     [      0.41]   4.36   0.53   0.44   7.68
TRTAK                    [      0.18]   1.62   1.50   1.48  15.58
TAKL                     [      0.49]   0.53   0.65   0.81   6.51
STAK                     [      0.43]   1.10   0.92   0.73   6.79
FPRINT/UGLY              [      1.64]   1.18   0.44   0.40   0.74
FPRINT/PRETTY            [      1.90]   7.87   0.89   1.55   1.30
TRAVERSE                 [      0.53]   1.37   1.29   1.27  11.41
TRIANGLE                 [      0.50]   8.86   1.15   1.10  24.55
RICHARDS                 [      1.52]   0.32   0.29   0.23   5.64
FACTORIAL                [      0.84]   0.57   0.20   0.68   1.31
FIB                      [      0.10]   8.17   3.00   2.92  19.99
FIB-RATIO                [      0.04]   4.89   1.07   0.97   1.38
ACKERMANN                [      0.81]   9.89   2.67   2.99  -1.00
MANDELBROT/COMPLEX       [      0.28]   5.96   1.30   3.76   6.02
MANDELBROT/DFLOAT        [      0.13]   1.05   0.13   0.21  10.04
MRG32K3A                 [      5.70]   0.92   0.09   0.08   3.70
CRC40                    [      1.04]  46.68   7.61  17.49  73.57
BIGNUM/ELEM-100-1000     [      0.78]   0.09   0.15   0.22   0.05
BIGNUM/ELEM-1000-100     [      5.14]   0.04   0.05   0.07   0.02
BIGNUM/ELEM-10000-1      [      5.54]   0.05   0.05   0.07   0.02
BIGNUM/PARI-100-10       [      0.48]   0.02   0.04   0.08   0.04
BIGNUM/PARI-200-5        [     13.98]   0.00   0.00   0.02   0.00
PI-DECIMAL/SMALL         [      2.07]   2.09   0.39   1.38   0.72
PI-DECIMAL/BIG           [      2.58]   2.34   0.29   0.86   0.35
PI-ATAN                  [      2.27]   0.40   0.31   0.57   0.80
PI-RATIOS                [      5.00]   0.26   0.24   0.52   0.12
HASH-STRINGS             [      3.57]   0.22   0.04   0.04   0.31
HASH-INTEGERS            [      1.97]   0.58   0.14   0.37   0.89
SLURP-LINES              [      3.85]   3.07   0.49   1.48   1.53
BOEHM-GC                 [      9.18]   0.50   0.09   0.26   1.17
DEFLATE-FILE             [      0.29]   3.18   0.82   1.23   9.43
1D-ARRAYS                [      0.02]  16.75   1.45   2.41  32.32
2D-ARRAYS                [      0.51]  11.36   0.53   1.13  32.18
3D-ARRAYS                [      1.26]  22.31   0.68   1.19  28.69
BITVECTORS               [     16.87]   0.42   0.03   0.08   0.32
BENCH-STRINGS            [      1.70]   4.93   0.24   0.39   8.41
fill-strings/adjustable  [     20.59]   0.69   0.24   0.55   1.37
STRING-CONCAT            [     27.50]   2.21   0.89   3.76  -1.00
SEARCH-SEQUENCE          [      1.80]   3.28   0.06   1.09   3.77
CLOS/defclass            [      0.17]   4.23   4.48  16.02   2.71
CLOS/defmethod           [      0.11]   2.92  29.41  43.53   1.11
CLOS/instantiate         [      3.22]  38.92   1.25   2.27   0.89
CLOS/simple-instantiate  [      3.44]  19.83   0.04   0.11   0.57
CLOS/methodcalls         [      2.61]   1.88   0.29   0.89   1.43
CLOS/method+after        [      1.41]   2.74   1.43   3.46   1.59
CLOS/complex-methods     [      0.51]   6.37   2.40   0.08  -1.00
EQL-SPECIALIZED-FIB      [      0.45]   7.46   0.44   0.54   2.40
Reference time in first column is in seconds; other columns are relative
Reference implementation: Clozure Common Lisp Version 1.2-r11313M  (DarwinX8664)
Impl ECL 8: ECL 8.9.0 (CVS 2008-06-19 17:09)
Impl SBCL : SBCL 1.0.22.17
Impl CMU C: CMU Common Lisp CVS Head 2008-11-08 00:35:08 (19E)
Impl CLISP: CLISP 2.47+ (2008-10-24) (built 3435106496) (memory 3435106574)


,---- MacBook Pro, Intel Core 2 Duo @2.33 GHz ---
|
| Benchmark                 Reference  OpenMCL  CLISP    ECL   ABCL
| -----------------------------------------------------------------
| COMPILER                 [      1.21]   0.85   0.95   4.39  -1.00
| LOAD-FASL                [      0.34]   1.34   1.39  -1.00  -1.00
| SUM-PERMUTATIONS         [      1.09]   4.22   2.04  -1.00   1.73
| WALK-LIST/SEQ            [      0.01]   2.67   1.05  26.39  -1.00
| WALK-LIST/MESS           [      0.03]   0.72   4.89  12.00  -1.00
| BOYER                    [      2.58]   4.50  13.45   0.36   1.06
| BROWSE                   [      0.31]   2.35   4.13   1.60   0.72
| DDERIV                   [      0.20]   2.40  10.85   6.50   2.06
| DERIV                    [      0.30]   1.73   8.19   4.61   1.98
| DESTRUCTIVE              [      0.19]   1.63   7.66   4.55   2.68
| DIV2-TEST-1              [      0.30]   2.49  11.88   4.01   0.97
| DIV2-TEST-2              [      0.53]   1.57   7.97   3.67   0.83
| FFT                      [      0.03]   2.27 102.68  24.24  24.82
| FRPOLY/FIXNUM            [      0.26]   2.38  13.09   4.17   5.37
| FRPOLY/BIGNUM            [      0.27]   5.50   6.69   2.94   2.73
| FRPOLY/FLOAT             [      0.35]   2.88  11.03   2.97   3.04
| PUZZLE                   [      0.15]   2.64  48.36  25.11  82.40
| TAK                      [      0.30]   0.95  10.16   7.84   4.50
| CTAK                     [      0.21]   2.24  15.66  18.82 940.77
| TRTAK                    [      0.31]   0.94  10.24   7.66   3.68
| TAKL                     [      0.37]   1.57   9.65   1.64   2.61
| STAK                     [      0.35]   1.37   7.92   3.12   5.60
| FPRINT/UGLY              [      0.95]   2.28   1.42   1.19   2.66
| FPRINT/PRETTY            [      1.84]   1.35   1.27  31.65   8.18
| TRAVERSE                 [      0.75]   0.88   9.91   1.92   6.01
| TRIANGLE                 [      0.65]   0.98  20.47   9.78  24.10
| RICHARDS                 [      0.44]   2.85  22.54   2.89  42.59
| FACTORIAL                [      0.20]   6.84   7.45   2.67   2.28
| FIB                      [      0.32]   0.44   5.82   2.42   1.08
| FIB-RATIO                [      0.04]   1.83   1.68   7.50   8.73
| ACKERMANN                [      2.43]   0.42  -1.00   3.95   1.57
| MANDELBROT/COMPLEX       [      0.34]   1.14   6.34   2.22   3.32
| MANDELBROT/DFLOAT        [      0.02]  10.35  96.39   4.90  13.94
| MRG32K3A                 [      0.54] 186.18  51.36   5.11  11.00
| CRC40                    [      9.17]   0.13  10.04   6.62   2.29
| BIGNUM/ELEM-100-1000     [      0.13]  10.02   0.32   1.03   2.85
| BIGNUM/ELEM-1000-100     [      0.29]  29.53   0.36   1.42   2.67
| BIGNUM/ELEM-10000-1      [      0.32]  28.86   0.31   2.36   2.70
| BIGNUM/PARI-100-10       [      0.02]  37.14   0.59   1.52   2.82
| BIGNUM/PARI-200-5        [      0.08] 305.08   1.02   1.30   2.23
| PI-DECIMAL/SMALL         [      0.96]   3.36   1.96   9.86  43.01
| PI-DECIMAL/BIG           [      0.86]   4.89   1.29  16.51  93.86
| PI-ATAN                  [      0.85]   3.90   2.92   1.80   2.19
| PI-RATIOS                [      1.41]   5.98   0.53   1.91   3.36
| HASH-STRINGS             [      0.23]   5.54   4.61   4.68   4.49
| HASH-INTEGERS            [      0.51]   5.76   2.75   2.26   1.80
| SLURP-LINES              [      1.98]   2.59   3.22   2.77   1.43
| BOEHM-GC                 [      1.16]  13.17  12.17   6.97   5.67
| DEFLATE-FILE             [      0.37]   1.40   8.13   2.96   4.11
| 1D-ARRAYS                [      0.06]   0.76   9.54   6.35   2.21
| 2D-ARRAYS                [      0.32]   7.55  50.23  17.14  33.41
| 3D-ARRAYS                [      0.99]   5.77  35.60  43.07  36.99
| BITVECTORS               [      0.60]  -1.00   9.47  11.02   1.09
| BENCH-STRINGS            [      0.45]   5.91   1.34  32.56   7.43
| fill-strings/adjustable  [     13.64]   2.53   2.01   1.30   0.70
| STRING-CONCAT            [     37.05]   1.01  -1.00   1.17   1.71
| SEARCH-SEQUENCE          [      0.12]  17.46  60.84  83.05  20.36
| CLOS/defclass            [      0.66]   0.35   0.68   2.81  15.94
| CLOS/defmethod           [      0.82]   0.11   0.13   2.51   7.91
| CLOS/instantiate         [      5.41]   0.97   0.65 121.47   2.57
| CLOS/simple-instantiate  [      0.16]  82.43  13.24 663.62  69.45
| CLOS/methodcalls         [      0.73]   6.08   6.62  10.67  11.16
| CLOS/method+after        [      0.78]   2.87   3.44  34.92  14.77
| CLOS/complex-methods     [      1.48]   0.46  -1.00   2.70   3.16
| EQL-SPECIALIZED-FIB      [      0.23]   2.67   4.43  13.52  -1.00
|
| Reference time in first column is in seconds; other columns are relative
| Reference implementation: SBCL 1.0.2 (binary download)
| Impl OpenMCL: OpenMCL Version 1.1-pre-061231 (DarwinX8664)
| Impl ECL: ECL 0.9i (GCC 4.0.1 -O2 -march=nocona -fomit-frame-pointer)
| Impl CLISP: CLISP 2.41 (2006-10-13) (built 3379682400) (memory 3379682557)
| Impl ABCL: Armed Bear Common Lisp 0.0.9 (Apple Java 1.5.0_06 -server -Xss128M -Xms1G -Xmx1G -Xrs)
| === Test machine ===
|    Machine-type: X86
|    Machine-version: MacBook Pro, Intel Core 2 Duo @2.33 GHz, 3 GB, Mac OS X 10.4.8 
`----


,---- dual Pentium 4, 3.0 GHz, 1000 MB RAM, 512 KB cache ---
| 
| Benchmark               Reference  CMUCL CMUCL  SBCL    ACL     ACL  GNUCL   ECL-S  CLISP Poplog     ABCL    ABCL
|                         CMUCL 18e    19a 19aP4 0.8.13   6.2     5.0  2.6.3    0.9d 2.33.2  15.53   J1.4.2  J1.5b2
| -----------------------------------------------------------------------------------------------------------------
| COMPILER                 [  1.98]   1.15  1.05  1.49   0.60    0.90   0.08    3.50   0.69   0.01    20.23   19.35
| LOAD-FASL                [  0.18]   1.16  1.12  2.79   1.70    1.19   0.45    2.61   2.19  11.14    24.47   23.08
| SUM-PERMUTATIONS         [  2.05]   1.13  1.11  1.52   0.84    1.52   1.24   -1.00   1.51   1.63     6.39    5.51
| WALK-LIST/SEQ            [  0.03]   1.00  0.96  1.00   1.07    1.07   1.43    4.29   1.00  10.36     3.93    3.82
| WALK-LIST/MESS           [  0.16]   1.31  0.47  1.82  -1.00   -1.00   2.06    0.75   1.57  -1.00    -1.00   -1.00
| BOYER                    [  5.51]   0.99  0.97  0.86   1.14    1.05   2.86    4.23  12.43   7.73     5.65    4.22
| BROWSE                   [  0.58]   1.04  0.97  0.96   0.44    1.33  12.85    2.50   6.27   1.74     4.86    3.43
| DDERIV                   [  0.56]   0.97  0.98  0.91   0.54    1.11   3.97    6.97  12.55   2.30     7.98    6.38
| DERIV                    [  0.62]   1.02  1.01  0.93   0.47    0.41   3.65    6.37  12.88   2.11     6.99    5.36
| DESTRUCTIVE              [  0.51]   0.97  1.00  0.89   0.33    0.33   3.66    3.72   7.75   2.39     8.89    5.11
| DIV2-TEST-1              [  1.30]   0.97  0.94  0.71   0.15    0.18   2.66    4.54  10.66   1.64     4.18    1.99
| DIV2-TEST-2              [  1.41]   0.96  0.94  0.84   0.33    0.33   2.89    4.42  10.49   1.34     1.61    0.64
| FFT                      [  0.06]   1.03  1.02  0.87   0.94    1.09  55.00   42.03  65.58 250.16    63.09   23.39
| FRPOLY/FIXNUM            [  0.54]   0.99  1.00  0.87   1.39    2.00   2.52    4.69  12.59   3.04    13.01    5.85
| FRPOLY/BIGNUM            [  0.62]   0.98  1.02  0.84   1.49    2.20   1.68    2.84   7.81   1.60     2.06    1.57
| FRPOLY/FLOAT             [  0.94]   0.88  0.90  0.76   1.09    2.76   1.35    2.70  10.75   2.19     1.77    1.27
| PUZZLE                   [  0.17]   0.99  1.00  1.10   4.31    1.90   6.21   46.26  71.59  14.14   109.75   61.32
| TAK                      [  0.36]   1.02  1.00  1.00   0.62    0.78   1.71    7.17  13.94   2.97    10.29    6.06
| CTAK                     [  0.33]   1.20  1.19  0.94   3.71    3.34   4.53    6.90  11.24  25.02  1461.22 1128.68
| TRTAK                    [  0.36]   1.01  0.99  0.99   0.61    0.78   1.67    7.13  13.64   2.95     7.44    3.35
| TAKL                     [  0.37]   1.05  1.00  0.99   0.85    0.98   0.87    4.78  20.55   5.03     4.37    4.08
| STAK                     [  0.47]   1.12  0.99  1.05   5.35    2.54   1.67   -1.00   7.64   7.34    15.64    7.32
| FPRINT/UGLY              [  0.68]   1.08  1.06  1.63   3.30   13.72   1.23    1.95   1.70   8.58    21.18   21.04
| FPRINT/PRETTY            [  2.23]   0.95  0.94  1.37   2.30    4.40   0.39    1.22   3.34   6.77   216.94  199.28
| TRAVERSE                 [  0.80]   1.00  1.00  4.21   0.56    0.64   2.23    7.21  14.10   5.29    20.60   15.60
| TRIANGLE                 [  0.60]   1.04  1.07  1.01   1.48    0.68   2.09   18.39  30.20  10.58    17.64   12.62
| RICHARDS                 [  0.46]   1.05  1.03  1.01   4.43    3.21   1.19   10.72  30.24  12.08    28.02   24.34
| FACTORIAL                [  0.33]   0.98  0.97  1.33   3.15    3.03   9.79    4.71  20.27   2.40     3.55    2.16
| FIB                      [  0.44]   1.01  1.00  1.01   0.25    0.27   4.77    2.24   6.44   1.10     5.55    2.26
| FIB-RATIO                [  0.34]   0.99  0.99  1.01  11.25   15.95  30.55    0.99   0.11   2.94     3.24    1.69
| ACKERMANN                [  5.63]   1.00  1.01  1.00   0.53    0.64  22.28    2.06   6.69   1.92     4.33    2.05
| MANDELBROT/COMPLEX       [  9.13]   0.97  0.97  1.01   1.80    5.36   1.93    2.24  10.58   0.77     0.75    0.32
| MANDELBROT/DFLOAT        [  5.26]   0.99  0.98  0.96   0.92    5.55   2.51    3.42  13.09   1.50     1.58    0.76
| MRG32K3A                 [  0.78]   1.00  1.00  1.03   5.93  183.82   5.45    7.13 118.85   7.38    10.43    8.45
| CRC40                    [ 19.30]   1.01  1.11  0.95   4.02    4.27  13.52    8.91  12.63   2.26     4.40    2.23
| BIGNUM/ELEM-100-1000     [  0.50]   1.01  1.02  1.17   1.59    1.88   0.77    0.22   0.10   2.40     1.56    1.46
| BIGNUM/ELEM-1000-100     [  2.74]   1.01  1.00  1.40   1.66    1.68   0.92    0.05   0.07   3.08     0.43    0.35
| BIGNUM/ELEM-10000-1      [  3.00]   1.01  1.00  1.38   4.77    4.80   0.79    0.05   0.06  19.89     0.48    0.37
| BIGNUM/PARI-100-10       [  1.28]   0.50  0.49  0.69   0.05    0.07   0.02    0.02   0.01   0.02     0.19    0.15
| BIGNUM/PARI-200-5        [ 15.90]   0.49  0.49  0.76   0.02    0.02   0.01    0.00   0.00   0.01     0.02    0.02
| PI-DECIMAL/SMALL         [ 25.10]   1.01  1.00  1.42  14.89   15.31   8.44    0.27   0.24   3.31     1.97    1.79
| PI-DECIMAL/BIG           [ 53.32]   1.00  1.00  1.55  15.63   15.74   6.07   -1.00   0.05   3.84     1.90    1.58
| PI-ATAN                  [  1.38]   1.01  0.99  1.29   2.97    2.98   5.80   -1.00   6.89   1.16     2.40    2.38
| PI-RATIOS                [  4.51]   0.99  1.00  1.11   5.31    5.42   3.91    0.58   0.33   2.01     1.54    1.44
| SLURP-LINES              [  0.89]   0.96  0.98  1.02   0.57    7.09   2.85    1.63   4.69   3.00    23.99   21.94
| HASH-STRINGS             [  0.47]   0.84  0.78  0.81  13.63 4218.26 251.42   50.04   5.81  35.71     7.68    6.02
| HASH-INTEGERS            [  0.84]   0.99  0.94  1.21   3.13    4.89   0.58    1.49   6.31  11.79     5.04    4.08
| BOEHM-GC                 [  1.89]   1.01  0.97  1.16   1.37    3.82   3.89    7.87  18.34   2.22    12.13    7.14
| DEFLATE-FILE             [  0.45]   1.30  1.30  1.22   1.07   24.97   3.27    9.55   8.87   7.80    16.87   11.25
| 1D-ARRAYS                [  0.07]   1.01  1.03  1.49   2.60    8.22   4.11    6.30  12.14  18.63     4.79    3.29
| 2D-ARRAYS                [  0.87]   1.01  1.01  0.63  10.25    9.77  22.62   21.31  26.39  14.16    46.67   31.88
| 3D-ARRAYS                [  2.91]   1.00  1.00  0.77   7.88    7.62   8.91   16.19  16.69   7.88    36.01   26.28
| BITVECTORS               [  0.63]   0.97  0.95  1.60   1.69    1.74  10.16   13.12  33.12  -1.00  2334.29  822.13
| BENCH-STRINGS            [  2.54]   1.02  0.99  0.14   3.88   16.87   4.30    5.78   0.47  33.55     9.67    4.18
| fill-strings/adjustable  [ 19.24]   1.01  1.00  1.00   1.47   -1.00   0.90   15.19   4.62 104.94     1.22    0.91
| STRING-CONCAT            [ 55.08]   1.07  1.03  0.50   0.60   -1.00  26.24   -1.00  10.29  -1.00     2.23    1.66
| SEARCH-SEQUENCE          [  3.03]   1.00  1.00  0.05   0.73    4.07   2.28    3.29   3.58   3.61    11.74    9.28
| CLOS/defclass            [  3.46]   1.08  1.07  0.63   0.05    0.11  -1.00    0.47   0.12   0.02     3.92    3.75
| CLOS/defmethod           [  7.27]   1.03  1.05  0.85   0.03    0.05  -1.00    0.17   0.01   0.00     1.34    1.27
| CLOS/instantiate         [  8.03]   1.13  1.09  0.98   0.26    0.86  -1.00  281.76   1.36   1.55    28.19   27.07
| CLOS/simple-instantiate  [  0.33]   0.99  0.99  0.96   0.84    0.57  -1.00   51.78  18.96 112.65   692.94  635.93
| CLOS/methodcalls         [  1.61]   0.97  0.97  1.23   3.25    3.01  -1.00   50.49  10.92   9.61    97.55   86.93
| CLOS/complex-methods     [  0.06]   1.00  0.97  7.44   4.92    5.76  -1.00 1963.22  -1.00  -1.00  1799.75 1600.25
| EQL-SPECIALIZED-FIB      [  0.32]   1.02  1.01  1.06   1.19   10.87  -1.00    7.16   5.30   9.06 10336.55 9321.19
| 
| Reference time in first column is in seconds; other columns are relative
| Reference implementation: CMU Common Lisp 18e CVS Head 2003-09-14 16:41:12 (binary rel.)
| Impl CMUCL:  CMU Common Lisp 19a-pre3 (binary rel. from ftp.linux.org.uk/pub/lisp/cmucl)
| Impl CMUCL:  CMU Common Lisp 19a-pre3 Pentium4 (built with CFLAGS="-march=pentium4")
| Impl SBCL:   SBCL 0.8.13 (bootstrapped with CMUCL 18e)
| Impl ACL6.2: International Allegro CL Enterprise Edition 6.2 [Linux/X86] (binary:)
| Impl ACL 5:  Allegro CL Trial Edition 5.0 [Linux/X86] (no heap limit! binary only:)
| Impl GNUCL:  Kyoto Common Lisp GCL 2.6.3 ANSI (gcc 3.3.1 -march=pentium4)
| Impl ECL-S:  ECL 0.9d CVS Head 2004-07-27 (aka ECL Spain, gcc 3.3.1 -march=pentium4)
| Impl CLISP:  CLISP 2.33.2 (2004-06-02) (not optimized for pentium4 because of make failure)
| Impl Poplog: Sussex Poplog 15.53e Common Lisp 2.0 (using "easy to install version" haha)
| Impl ABCL14: Armed Bear Common Lisp 0.0.3.16+ CVS Head 2004-08-01 / Sun Java 1.4.2-b28
| Impl ABCL15: Armed Bear Common Lisp 0.0.3.16+ CVS Head 2004-08-01 / Sun Java 1.5.0-beta2-b51
|              Java invocation for ABCL: java -server -Xss64M -Xmx256M -Xrs
| === Test machine ===
|    Machine-type: X86
|    Machine-version: dual Intel Pentium 4 at 3.00GHz, 1000 MB RAM, 512 KB cache
|    Linux 2.4.21-199-smp4G i686 i386 GNU/Linux (SuSE 9.0)
| (declaim (optimize (speed 3) (space 1) (safety 0) (debug 0) (compilation-speed 0)))
`----


,---- Pentium III@1.1GHz, 512MB RAM ---
|
| Benchmark                 Reference   18d   18dsm  CLISP   SBCL  Poplog
| -----------------------------------------------------------------------
| BOYER                          1.50   0.73   0.74   5.61   1.01   3.03
| BROWSE                         0.61   0.74   0.74   2.54   1.26   2.85
| DDerviv                        0.66   0.52   0.53   2.14   1.15   1.62
| Deriv                          1.29   0.50   0.48   2.05   1.10   1.26
| DESTRUCTIVE                    1.05   0.91   0.95   3.17   1.13   3.38
| DIV2-TEST-1                    1.09   0.51   0.51   1.66   1.07   1.29
| DIV2-TEST-2                    0.54   0.52   0.61   1.69   1.09   1.31
| FFT                            0.19   1.00   1.00  37.00   1.21  91.21
| FRPOLY/FIXNUM                  0.54   0.87   0.89   6.85   1.09   3.00
| FRPOLY/BIGNUM                  1.30   0.73   0.74   2.44   1.08   1.39
| FRPOLY/FLOAT                   0.54   0.76   0.78   4.41   1.06   2.11
| PUZZLE                         0.68   0.99   0.99  25.09   9.35  27.93
| TAK                            0.79   0.97   0.97   9.08   1.25   3.10
| CTAK                           0.73   1.00   0.97   6.49   1.05   4.84
| TRTAK                          0.79   0.97   0.97   9.13   1.25   3.11
| TAKL                           1.37   1.01   1.00   7.70   1.27   4.02
| STAK                           0.96   1.02   1.02   7.07   1.08   1.50
| FPRINT                         1.21   0.92   0.82   1.06   2.28   4.17
| TRAVERSE                       2.00   0.94   0.99  11.45   1.50   5.42
| TRIANGLE                       1.70   1.01   1.01  17.58   1.28  23.19
| RICHARDS                       0.53   1.00   0.98  21.26   1.00   6.47
| FACTORIAL                      0.37   0.54   0.57   0.81   1.14   0.62
| FIB                            0.33   1.00   1.00   5.30   1.06   1.18
| RATIOS                         2.24   0.96   0.96   0.12   0.96   1.81
| BIGNUM/ELEM-100-1000           1.06   0.92   0.97   0.08   0.97   2.27
| BIGNUM/ELEM-1000-100           4.99   0.99   0.99   0.05   0.93   2.14
| BIGNUM/ELEM-10000-1            5.19   1.01   1.00   0.05   0.92  12.90
| BIGNUM/PARI-100-10             1.21   0.99   1.02   0.02   0.95   0.04
| BIGNUM/PARI-200-5             14.30   0.99   1.00   0.01   0.92   0.01
| HASH-STRINGS                   1.15   0.84   0.69   3.25   0.99  30.89
| HASH-INTEGERS                  0.44   0.80   0.66   2.75   1.27   6.95
| BOEHM-GC                       6.41   0.47   0.46   4.10   1.12   1.53
| DEFLATE                        1.16   1.00   1.02   6.34   1.52   5.53
| CLOS/defclass                  1.75   0.90   0.92   0.53   2.53   0.06
| CLOS/defmethod                 8.73   0.77   0.71   0.02   2.17   0.00
| CLOS/instantiate               6.42   0.57   0.54   0.96   0.99   3.57
| CLOS/methodcalls               8.17   0.42   0.44   1.01   0.90   3.93
| CLOS/method+after             10.76   0.84   0.77   0.47   2.42   2.01
| CLOS/complex-methods           0.29   1.07   1.14  -1.00   6.17  -1.00
| 1D-ARRAYS                      2.44   0.81   0.79   5.68   1.39  15.75
| 2D-ARRAYS                     11.28   0.99   0.89   3.45   4.39   6.04
| BITVECTORS                     2.01   0.82   0.81   0.60   0.98  -1.00
| FILL-STRINGS                   6.83   0.88   0.82   1.16   3.77  27.38
| fill-strings/adjustable       14.18   0.99   0.87   1.36   1.73 118.49
| BENCH-STRING-CONCAT            5.79   1.88   1.62  -1.00   2.75  -1.00
| 
| Reference implementation: CMU Common Lisp 18c
| Impl   18d: CMU Common Lisp 18d-pre, level-1 built 2002-04-08 on melbourne
| Impl 18dsm: CMU Common Lisp 18d-pre-small, level-1 built 2002-04-08 on sydney
| Impl CLISP: CLISP 2.27.2 (released 2001-10-05) (built 3222523548) (memory 3222523674)
| Impl SBCL : SBCL 0.7.1.22 (from debian testing)
| Impl Poplog: Sussex Poplog Common Lisp 2.0
| Linux melbourne 2.4.17 #2 SMP Wed Feb 6 11:35:03 CET 2002 i686 unknown
|  (declaim (optimize (speed 3) (space 0) (safety 1) (debug 0) (compilation-speed 0)))
`----


,---- 400MHz G3, 1024KB L2 cache, 192 MB RAM --
|
| Benchmark                 Reference  CLISP   SBCL
| --------------------------------------------------
| BOYER                          7.15   3.57   0.36
| BROWSE                         2.42   2.51   0.61
| DDerviv                        4.00   2.62   0.29
| Deriv                          6.91   2.83   0.28
| DESTRUCTIVE                    3.31   4.42   0.69
| DIV2-TEST-1                    5.06   3.26   0.28
| DIV2-TEST-2                    2.54   3.27   0.26
| FFT                           16.63   1.27   0.05
| FRPOLY/FIXNUM                  2.89   4.29   0.40
| FRPOLY/BIGNUM                  5.75   2.22   0.46
| FRPOLY/FLOAT                   3.53   2.43   0.34
| PUZZLE                        19.53   2.37   1.06
| TAK                            2.08  12.04   1.45
| CTAK                           3.21   6.04   0.76
| TRTAK                          2.08  12.05   1.45
| TAKL                           6.94   5.36   0.67
| STAK                           2.86   7.01   0.94
| FPRINT                         5.38   1.33   1.75
| TRAVERSE                       7.81  10.37   1.20
| TRIANGLE                       7.59  11.73   1.04
| RICHARDS                       2.37  17.46   0.72
| FACTORIAL                      2.30   1.82   0.30
| FIB                            0.45  12.87   1.44
| BIGNUM/ELEM-100-1000           2.33   0.17   0.99
| BIGNUM/ELEM-1000-100           9.50   0.18   0.61
| BIGNUM/ELEM-10000-1           18.52   0.09   0.33
| BIGNUM/PARI-100-10             1.88   0.10   1.85
| BIGNUM/PARI-200-5             17.84   0.05   3.35
| HASH-STRINGS                   5.60   1.49  14.50
| HASH-INTEGERS                  2.12   1.28   0.72
| BOEHM-GC                      58.63   1.82   0.22
| DEFLATE                        4.53   4.78   1.36
| CLOS/defclass                  0.75   4.76  14.77
| CLOS/defmethod                 0.31   2.87 150.39
| CLOS/instantiate              31.41   0.83   0.54
| CLOS/methodcalls              29.87   1.34   0.50
| CLOS/method+after             12.61   1.69   6.09
| CLOS/complex-methods           2.71  -1.00   1.55
| 1D-ARRAYS                      8.36   5.42   1.49
| BITVECTORS                    11.30   0.36   0.29
| FILL-STRINGS                  32.03   0.75   2.84
| fill-strings/adjustable       35.86   1.87   2.78
|
| Reference implementation: OpenMCL Version (Beta: Linux) 0.10.1
| Impl CLISP: CLISP 2.28 (released 2002-03-03) (built 3227525942) (memory
+3227526439)
| Impl SBCL : SBCL 0.7.2
`----


,---- UltraSPARCIIe at 500MHz, 640MB RAM, SunOS 5.8 ---
|
| Benchmark                 Reference  CMU C  CLISP
| -----------------------------------------------------
| BOYER                          3.98   0.91   8.03
| BROWSE                         1.72   0.91   2.85
| DDerviv                        2.02   0.75   3.21
| Deriv                          3.63   0.81   3.13
| DESTRUCTIVE                    3.11   1.01   4.18
| DIV2-TEST-1                    2.19   0.83   3.92
| DIV2-TEST-2                    1.12   0.82   3.85
| FFT                            0.74   1.03  28.86
| FRPOLY/FIXNUM                  1.87   1.01   7.89
| FRPOLY/BIGNUM                  4.59   1.29   3.07
| FRPOLY/FLOAT                   1.65   0.96   5.68
| PUZZLE                         2.07   0.95  30.62
| CTAK                           2.74   1.01   9.04
| TAK                            1.84   1.00  14.08
| RTAK                           1.84   1.01  13.95
| TAKL                           3.37   1.01  11.63
| STAK                           2.32   1.01   8.87
| FPRINT                         4.17   1.02   1.12
| TRAVERSE                       5.84   0.99  13.74
| TRIANGLE                       5.53   0.86  15.57
| CASCOR                        10.53   0.73  52.81
| RICHARDS                       2.35   0.94  22.46
| FACTORIAL                      1.46   1.48   2.88
| FIB                            0.94   0.99   6.71
| BIGNUM/ELEM-100-1000           2.80   1.24   0.28
| BIGNUM/ELEM-1000-100          10.14   1.19   0.44
| BIGNUM/ELEM-10000-1           11.38   1.35   0.41
| BIGNUM/PARI-100-10             2.76   1.15   0.09
| BIGNUM/PARI-200-5             27.19   1.06   0.05
| READ-LINE                      3.39   1.06   1.19
| HASH-STRINGS                   5.42   1.20   2.19
| HASH-INTEGERS                  1.61   0.76   2.00
| BOEHM-GC                      19.97   0.76   4.14
| CLOS/defclass                  4.78   1.01   0.81
| CLOS/defmethod                27.61   0.89   0.03
| CLOS/instantiate              20.93   0.85   1.28
| CLOS/methodcalls              23.62   1.08   1.94
| CLOS/method+after             33.70   1.07   0.78
| CLOS/complex-methods           1.41   0.92  -1.00
| 1D-ARRAYS                     10.77   0.92   3.51
| 2D-ARRAYS                     56.66   1.40   2.61
| BITVECTORS                     5.35   0.86   0.42
| FILL-STRINGS                  18.88   1.07   0.97
| fill-strings/adjustable       45.09   1.46   1.41
| BENCH-STRING-CONCAT           48.10   0.90  -1.00
| 
| Reference implementation: CMU Common Lisp 18c, Built 2000-11-27
| Impl CMU C: CMU Common Lisp 18d-pre, level-1 built 2001-12-12 on liszt
| Impl CLISP: CLISP 2.27.2 (released 2001-10-05) (built on moustacho)
| SunOS eagles 5.8 Generic_108528-10 sun4u sparc SUNW,Sun-Blade-100
`----

Note that results prior to 2002-01 were a little bogus, since the optimization settings were not being set correctly across implementations, so default optimization levels were used.