Decimal Testcases, version 2.44
Copyright (c) IBM Corporation, 2009. All rights reserved. ©
7 Apr 2009
[previous | contents | next]

Testcase file format

This section describes the format of the testcase files. These are distributed as plain text files with a file name that identifies the group of tests contained in the file and a file extension (if appropriate) of .decTest. File names will not have embedded blanks.

The files are encoded using one byte per character, using 7-bit ASCII encoding. These may be converted to Unicode by treating them as UTF-8-encoded files or by directly converting each 7-bit character to Unicode by prefixing nine 0 bits.


Testcase syntax

Each testcase file consists of one or more lines (the line delimiter mechanism may vary, depending on the operating system). Within each line, control characters (those with encodings in the range 0 through 31) are not used.

Each line is treated as a sequence of tokens, delimited by the start of the line, spaces between tokens, or (after the final token) the end of the line. There may be more than one space between tokens, and spaces may also appear before the first token on a line and after the last token on a line. Tokens may also be quoted, to include spaces (see below).

If the first two characters of a token are two hyphens (--) the token indicates the start of a comment. The two-hyphen sequence and any characters that follow it, up to the end of the line on which the sequence occurs, are ignored (that is, they are treated as commentary).

The lines in a file may be:

empty
Lines with no characters, or only space characters. These lines are treated as commentary and are ignored.

comments
Lines whose first token starts a comment. These are also ignored.

directives
Lines which are used to control the testcase environment in some way, for example to change the working precision. These lines have two tokens and are of the form:

  keyword: value

where the case-independent keyword describes the purpose of the directive, and the value is a parameter associated with the directive. The possible keywords and their values are described below.

tests
Lines which describe a specific test. These lines have at least five tokens, and are of the form:

  id operation operand1 operand2 operand3 -> result conditions

where the named tokens are as follows:

id
A short name which identifies the test. It is case-independent and unique in the file. In the current testcases it comprises three or four alphabetic characters followed by three or four digits (for example, divx101).

operation
A case-independent keyword which describes the operation to be carried out for this test (for example, divide).

operand1
The first (or only) operand required for the operation. The token may be quoted (see below). If it is not quoted then its value is the sequence of characters comprising the token, taken exactly as written.

If the value of a token includes an octothorpe character (#, also called the hash or pound sign), the operand is a specific format-dependent representation, as described below.

Otherwise, the value of the token is treated as a string, which should be converted to a number using the to-number conversion of the arithmetic specification.

operand2
An optional second operand, if required for the operation. If present, it has the same syntax and follows the same rules as operand1.

operand3
An optional third operand, if required for the operation. If present, it has the same syntax and follows the same rules as operand1 and there must be an operand2.

result
This defines the result of carrying out the operation on the operand or operands, and may also be quoted. It will either be the string form of a valid number or a format-dependent representation (see below), or the question mark character (?) which indicates that the result is undefined. The latter is only used in tests for the X3.274 subset of the specification, for error results.

conditions
Zero or more tokens each of which is the case-independent name of a condition set by the operation. If no condition is set then no condition tokens will be present.

Quoted tokens

Any operand or result token may be quoted. That is, it may begin with a delimiter which is a single or double quote character and it is then only ended by a matching quote (which must be present and on the same line). A quote which matches the starting quote may be included inside a quoted operand by doubling up the quote; in this case it does not end the token.

The content of the quoted token (after the delimiters have been removed and any doubled delimiter quotes have been reduced to a single instance) forms the value of the token. The value may contain any 7-bit ASCII characters (other than the control characters, whose encodings are in the range 0 through 31), including spaces or the comment start sequence.

If the value of an operand token is not a valid number or format-dependent representation (that is, its syntax is not valid) then the token must be quoted.[1]  Result tokens must always have correct syntax.

Processing of a test case line

When the value of an operand token is converted to a number before use, using the to-number conversion, it is subject to the values set by directives, except that the precision directive is only used when the operation is toSci, toEng, or apply. For other operations, sufficient precision is used so that rounding of the operand is avoided.[2]  The conversion of operands to numbers may set flags, such as the Rounded or Inexact flags, if the ‘perfect’ value of the operand is outside the bounds set by the directives.

In contrast, the value of a result token is effectively converted to a number without constraints – no flags will be set by this process. The ‘perfect’ result number is compared with the result of the operation on the operand(s) and must match exactly if the test is to succeed.

Note that in all cases the actual processing by a test case interpreter need only act as though the steps described here are carried out.

Format-dependent representations

When the value of an operand or result token includes the octothorpe character (#, also called the hash or pound sign), the operand or result is derived from or creates a specific decimal format.

The operand or result token must have one of the following syntaxes:

Example

Here is an example of a small testcase file, comprising some commentary, directives which set the version and context, and some tests.
  -- simple.decTest
  -- Testcase for some simple operations.
  Version: 2.44
  
  Precision:   9
  Rounding:    half_up
  MaxExponent: 999
  MinExponent: -999
  
  simp001  add       1 1 -> 2   -- can we get this right?
  simp002  multiply  2 2 -> 4
  simp003  divide    1 3 -> 0.333333333  Inexact Rounded
  simp004  divide    1 0 -> NaN Division_by_zero
  simp005  toSci  ’1..2’ -> NaN Conversion_syntax

Note: Tokens and lines do not have a defined length limit, however the current testcases are limited to a maximum token length of 1050 characters and a maximum line length of 4000 characters.


Directives

Directives are used to control the testcase environment in some way. Each has a keyword (which is immediately followed by a colon) and a value.

The first four directives are required; no tests can be run without these settings being specified (that is, there is no default value for these settings). Once set, each setting remains in force until a new directive with the same keyword is encountered; the setting is then replaced by the new value.
Keyword Value
precision An unsigned positive integer. Its value is used to set the precision in the context for the following tests.

If the setting exceeds the maximum precision that can be handled by an implementation then following tests should be skipped (until the setting is suitably reduced or the end of the file is reached).

rounding A word which describes the rounding mode to set in the context for the following tests. It is case-independent and will be one of:

  ceiling
  down
  floor
  half_down
  half_even
  half_up
  up
  05up

If an unsupported rounding mode is set, then following tests should be skipped (until the setting is changed to a supported mode or the end of the file is reached).

maxexponent An unsigned integer which may be zero or positive. This value describes the value of the adjusted exponent beyond which overflow will be raised. The following tests will indicate an overflow condition if the adjusted exponent exceeds this setting.

Implementations, in general, will have fixed maximum exponent limits, which may not match the setting in the testcase:

  • If the setting of maxexponent is larger than can be handled, then following tests should be skipped (until the setting is suitably reduced or the end of the file is reached).
  • If the setting of maxexponent is smaller than can be enforced, then following tests which indicate an overflow condition should be skipped (until the setting is suitably changed or the end of the file is reached).
minexponent An unsigned integer which may be zero or negative. This value describes the value of the adjusted exponent below which underflow will be raised. The following tests will indicate an underflow condition if the adjusted exponent is less than this setting. (Note that if extended is set, smaller exponents down to minexponentprecision + 1 are possible because subnormal values are allowed.)

Implementations, in general, will have fixed minimum exponent limits, which may not match the setting in the testcase:

  • If the setting of minexponent is smaller than can be handled, then following tests should be skipped (until the setting is suitably reduced or the end of the file is reached).
  • If the setting of minexponent is smaller than can be enforced, then following tests which indicate an underflow or subnormal condition should be skipped (until the setting is suitably changed or the end of the file is reached).

The next three directives are optional:
Keyword Value
version A number which describes the version of the testcases which follow. This may be up to five digits, or four digits with an embedded decimal point. For example:

  version: 2.40

The meaning of the version number is not defined, except that later versions of testcases should have a larger version number.

extended Either 0 or 1. This directive indicates the level of arithmetic needed for the following tests.

When set to 1 (the default), numbers whose value is zero may have non-zero sign and exponent, operations may result in subnormal values, extra checking is performed on the length of operands, and the results of operations are defined after errors (they may be 0, infinite, or NaN values). For example:

  extended: 1 -- enable extended values
  div0 divide -1 0 -> -Infinity Division_by_zero

When set to 0, only the X3.274 subset of the arithmetic is required, where the sign of a zero value result is always 0, subnormal values raise underflow, and some other differences are expected.

If an implementation does not support testcases as selected by the extended setting then following tests should be skipped (until the extended setting is changed to an acceptable value or the end of the file is reached).

clamp Either 0 or 1. This directive indicates whether explicit exponent clamping is applied.

If 0 (the default), the only clamping applies to zero results, which will have maximum and minimum exponents as described under maxexponent and minexponent above.

If 1, a restricted exponent range (as used in certain concrete representations) applies: the maximum exponent is reduced to maxexponentprecision + 1. This will clamp zeros at a lower value and may cause the coefficient and exponent of certain normal values to be ‘folded down’.

If an implementation does not support testcases with the clamp option as selected by the clamp setting then following tests should be skipped (until the clamp setting is changed to an acceptable value or the end of the file is reached).

The final directive allows testcase groups (files) to themselves be grouped together in a hierarchy:
Keyword Value
dectest A word specifying the file name (without extension) of another testcase file to be processed at this point. For example, a testcase which simply runs the testcases for the three division operations might read:

  -- divides.decTest -- Test divisions
  dectest: divide
  dectest: divideint
  dectest: remainder

Note that this directive is not an ‘include’; the current settings are not inherited by the file to be processed – that file must be processed in exactly the same way as if it were the only testcase being run.


Operations

Each test line identifies an operation by means of a case-independent keyword, which is always the second token of the line. The following operations are defined.
Keyword Definition
abs If the operand is negative, this is minus; otherwise it is plus.
add The two operands are added together using add.
and The two logical operands are anded together using and.
apply This operation applies the constraints of the directives to the operand; the result must then match in precision and value.
canonical The operand is converted to an canonical encoding, if necessary (the result should be a format-dependent representation).
class The class of the operand is tested; the result is one of the strings defined for class.
compare The operands are compared using compare.
comparesig The operands are compared using compare-signal.
comparetotal The operands are compared using compare-total.
comparetotalmag The operands are compared using compare-total-magnitude.
copy The operand is copied to the result.
copyabs The operand is copied to the result using copy-abs.
copynegate The operand is copied to the result using copy-negate.
copysign The first operand is copied to the result with the sign of the second, using copy-sign.
divide The first operand is divided by the second, using divide.
divideint The first operand is divided by the second to give an integer result, using divide-integer.
exp e is raised to the power of the operand, using exp.
fma The three operands are combined, using fma.
invert The logical operand is inverted using invert.
ln, log10 The logarithm of the operand in base e or 10 is computed, using ln or log10.
logb The exponent of the operand is extracted, using logb.
max, min The operands are compared using compare and the larger or smaller, respectively, is returned.
maxmag, minmag The magnitudes of the operands are compared using compare and the larger or smaller, respectively, is returned.
minus The operand is subtracted from zero, using minus.
multiply The operands are multiplied together using multiply.
nextminus The next value less than the operand is computed using next-minus.
nextplus The next value greater than the operand is computed using next-plus.
nexttoward The next value to the first operand in the direction of the second is computed using next-toward.
Keyword Definition
or The two logical operands are ored together using or.
plus The operand is added to zero, using plus.
power The first operand is raised to power of the second, using power.
quantize The first operand is quantized so that its exponent is set to that of the second operand, using quantize.
reduce Trailing zeros are removed, using reduce (previously named normalize).
remainder The first operand (the dividend) is divided by the second (the divisor) to give a remainder after integer division, using remainder.
remaindernear The first operand (the dividend) is divided by the second (the divisor) to give a remainder after division to the nearest integer, using remainder-near (IEEE remainder).
rescale The first operand is rescaled so that its exponent is set to the value of the second operand, using rescale.
rotate The coefficient of the first operand is rotated by the number of digits given by the second operand, using rotate.
samequantum The exponents of the operands are compared for equality.
scaleb The exponent of the first operand is adjusted by a value given by the second operand, using scaleb.
shift The coefficient of the first operand is shifted by the number of digits given by the second operand, using shift.
squareroot The square root of the operand is computed, using square-root.
subtract The second operand is subtracted from the first, using subtract.
toEng The operand is converted to a string using to-engineering-string.
tointegral The operand removes any fraction, using round-to-integral-value.
tointegralx The operand removes any fraction, using round-to-integral-exact; it may result in Inexact.
toSci The operand is converted to a string using to-scientific-string.
trim Insignificant fractional zeros are removed, using trim.
xor The two logical operands are exclusive-ored together using xor.


Conditions

Each test may cause zero or more conditions to be raised. The case-independent names of these conditions (if any) are listed following the result token of each test.

Only those conditions occuring during the tested operation are listed unless the operation is toSci, toEng, or apply. For these operations, conditions raised during the conversion of the operand are included (this allows the testing of conversions in both directions).

The following condition names are defined, together with the name used for it in the arithmetic specification and the IEEE 754 exception which would be raised by the condition.
Condition Specification name IEEE exception
clamped Clamped (no equivalent)
conversion_syntax Conversion syntax Invalid operation
division_by_zero Division by zero Division by zero
division_impossible Division impossible Invalid operation
division_undefined Division undefined Invalid operation
inexact Inexact Inexact
insufficient_storage Insufficient storage Invalid operation
invalid_context Invalid context Invalid operation
invalid_operation Invalid operation Invalid operation
lost_digits Lost digits (no equivalent)
overflow Overflow Overflow
rounded Rounded (no equivalent)
subnormal Subnormal (no equivalent)
underflow Underflow Underflow

Notes:

  1. The condition names are simply the names from the arithmetic specification, with spaces changed to underscores so each forms a single token.
  2. The inexact, rounded, and subnormal conditions are included in the testcases, even when extended is 0, to aid analysis and debugging. (Underflow implies all three.)
    The rounded condition indicates that an operand or the result of a test has had one or more zero or non-zero digits removed by rounding. That is, the number of digits in the coefficient of the result is fewer than in the coefficient of the ‘ideal’ result.
    In contrast, the inexact condition indicates only that non-zero trailing digits were removed (that is, the result would compare unequal to the ideal result).
  3. Similarly, the clamped condition, which can only occur if extended is 1, is included. This indicates when a zero is clamped to the maximum or minimum adjusted or representable exponent, or when a normal number is ‘folded-down’ in order to fit in a specific encoding.
  4. The lost_digits condition can only occur if extended is 0.
  5. The insufficient_storage condition is not a predictable condition and so will not appear in any testcases. It is listed here as a reminder that some implementations could raise this condition for some tests.

Footnotes:
[1] This rule allows future extensions to the syntax of tests.
[2] This rule allows the testing of rounding in the toSci, toEng, and apply operations, and also permits the testing of the lost_digits condition in the other operations.

[previous | contents | next]