This benchmark was devised in order to investigate the balance between Input and Output (I/O) time and calculation time in a simple program which captures the essence of a telephone company billing application. It does not attempt to cover a wide spectrum of decimal operations and so is not suitable for benchmarking decimal implementations; modern applications often need much greater precision and also alignment, division, conversion, and rounding operations which do not appear in this benchmark.

In summary, the benchmark reads a large input file containing a suitably distributed list of telephone call durations (each in seconds). For each call, a charging rate is chosen and the price calculated and rounded to hundreths. One or two taxes are applied (depending on the type of call) and the total cost is converted to a character string and written to an output file. Running totals of the total cost and taxes are kept; these are displayed at the end of the benchmark for verification.

Full details and a discussion of the design parameters are included in The ‘telco’ benchmark specification. Links to the sample data files may be found below.

The following table gives some initial results, derived using the sample benchmark programs linked from the table. Comments and corrections are invited.

Initial results

In this table, each implementation is shown on a separate row. The columns are:

1. The language used in the implementation.
2. A link to the sample source code for the benchmark.
3. The percentange of the benchmark time which is spent in the decimal arithmetic calculations (including roundings and conversions). For measurements in this column the input and output files are on a local disk.
4. As the third column, except that the input and output files are on a different machine (connected via fast Ethernet). This scenario gives a ‘worst case’ I/O time (and hence the ‘best case’ calculation percentage). A database implementation would be between the two cases.
5. Implementation-specific notes.

General Notes:

1. All timings used the best of three measurements.
2. The local machine for all tests except cTelco754r was an IBM Intellistation Model 6889 (450MHz Pentium II, SCSI disks, 256MB RAM). For the cTelco754r test, a Shuttle X (3GHz, Pentium 4, ATA disk, 1GB) was used.
3. For the Ethernet I/O tests, the data files were held on a Thinkpad T21 (800MHz Pentium III, IDE disk, 256MB RAM), connected by a 100MHz Ethernet link via one hub.
4. The I/O time (local I/O) averaged over all three implementations on the Intellistation was 2.75 microseconds per number.
5. The tax calculations were between 35% and 50% of the total calculation time, depending on the implementation.

Other results

• William M. Klein has implemented the specification in COBOL, as discussed in this comp.lang.pl1 thread.

• Glenn Someone has implemented the specification in Turbo Pascal 7.0, available in: comp.lang.pascal.borland
  The timings shown below are on the test file expon180.1e6b, run on a Celeron 433Mhz PC.

Data files

Two data collections are available for use with the benchmark source files; one is a short test file comprising ten numbers; the other contains one million numbers (7.63MBytes). Each file is available in three formats: 8-byte packed decimal, 8-byte binary, and the decimal64 format from the IEEE 754r draft (one number in every eight bytes in all three cases).

All files were generated with an exponential distribution with mean set to 180; for the larger files the actual mean is 180.2 and the maximum value is 2641.

Here are the links to the files. Be sure to use ‘Save As’ or ‘Save Target As’ for the test files (as well as for the zipped files) to avoid line-ending problems due to browser/platform behaviour.

	Packed decimal	Binary	decimal64
10-number test file	telco.test	telco.testb	telco.testr
Million-number file (zipped)	expon180.1e6	expon180.1e6b	expon180.1e6r

Here is a worked example (using the numbers from the telco.test files) to aid verification of new implementations of the benchmark:

    Time Rate | Price   Btax   Dtax  | Output
  ------------+----------------------+--------
     308   L  |  0.40   0.02         |  0.42
      76   L  |  0.10   0.00         |  0.10
      68   L  |  0.09   0.00         |  0.09
     323   D  |  2.89   0.19   0.09  |  3.17
      82   L  |  0.11   0.00         |  0.11
     329   D  |  2.94   0.19   0.10  |  3.23
      14   L  |  0.02   0.00         |  0.02
     101   D  |  0.90   0.06   0.03  |  0.99
     522   L  |  0.68   0.04         |  0.72
      50   L  |  0.06   0.00         |  0.06
  ------------+----------------------+--------
     Totals:  |  8.19   0.50   0.22  |  8.91
  

Note: Not all rounding cases are covered in the small test file. A round-down-to-even case is covered in the last record (50 × 0.0013 → 0.065 which is truncated to 0.06 because 6 is even), but a round-up-to-even case is not covered, for example 550 × 0.0013 → 0.715 which should be rounded up to 0.72. The latter example can be found as the 5873^rd entry in the main test file (at offset hex 16F0).