Skip to content
Snippets Groups Projects
Select Git revision
  • master default
  • proposed-rewrite-strategy-for-fixtures-with-assertions
  • travisbuddy
  • _syntax
  • update-expressions
  • features-cleanup
  • v8-with-all-flags
  • promise-constructor-override
  • jugglinmike-refactor-for-parsers-directive
  • donprinthandle
  • remove-issafeinteger-followup
  • bigint-fix-minvalue
  • revert-1176-1051
  • ljharb-finally
  • revert-1141-ArrayIteratorPrototype-next
  • revert-1150-revert-1064-class-fields
  • dev
  • es5-tests
18 results
Blame Permalink
After you've reviewed these contribution guidelines, you'll be all set to contribute to this project.
CONTRIBUTING.md 18.36 KiB

Test262 Authoring Guidelines

Source Material

Test Case Names

Test cases should be created in files that are named to identify the feature or API that's being tested.

The names should use alphanumeric characters and ., -, _. Otherwise, there is no strict naming convention, but the file names should be human readable, helpful and, ideally, consistent within a single directory. For examples:

  • Math.fround handling of Infinity: test/built-ins/Math/fround/Math.fround_Infinity.js
  • Array.prototype.find use with Proxy: test/built-ins/Array/prototype/find/Array.prototype.find_callable-Proxy-1.js
  • arguments implements an iterator interface: test/language/arguments-object/iterator-interface.js

See the following directory trees for further recommended examples:

Note The project is currently transitioning from a naming system based on specification section numbers. There remains a substantial number of tests that conform to this outdated convention; contributors should ignore that approach when introducing new tests and instead encode this information using the esid frontmatter tag.

Test Case Style

A test file has three sections: Copyright, Frontmatter, and Body. A test looks roughly like this:

// Copyright (C) 2015 [Contributor Name]. All rights reserved.
// This code is governed by the BSD license found in the LICENSE file.

/*---
 description: brief description
 info: >
   verbose test description, multiple lines OK.
   (info typically contains relevant, direct quote from ECMAScript)
---*/

[Test Code]

Copyright

The copyright block must be the first section of the test. The copyright block must use // style comments.

Frontmatter

The Test262 frontmatter is a string of YAML enclosed by the comment start tag /*--- and end tag ---*/. There must be exactly one Frontmatter per test.

Test262 supports the following tags:

The following tags are deprecated, but exist in old tests:

description

description: [string]

This is one of two required frontmatter tags. The description should be a short, one-line description of the purpose of this testcase. We suggested that the description be kept to less than 100 characters, but clarity is preferred over brevity.

Eg: Insert <LS> between chunks of one string

info

info: [multiline string]

This allows a long, free-form comment. The comment is almost always a direct quote from ECMAScript. It is used to indicate the observable being tested within the file.

For example:

/*---
esid: sec-weakset.prototype.has
description: Throws TypeError if `this` is not Object.
info: |
  WeakSet.prototype.has ( value )

  1. Let S be the this value.
  2. If Type(S) is not Object, throw a TypeError exception.
---*/

Note: Adding more context than the direct quote from ECMAScript should rarely be necessary. If you must add context to the quote, use the JavaScript single line comment syntax.

negative

negative: [dictionary containing phase and type]

This means the test is expected to throw an error of the given type. If no error is thrown, a test failure is reported.

  • type- If an error is thrown, it is implicitly converted to a string. In order for the test to pass, this value must match the name of the error constructor.
  • phase - Negative tests whose phase value is "parse" must produce the specified error prior to executing code. The value "resolution" indicates that the error is expected to result while performing ES2015 module resolution. The value "runtime" dictates that the error is expected to be produced as a result of executing the test code.

For best practices on how to use the negative tag please see Handling Errors and Negative Test Cases, below.

For example:

negative:
  phase: parse
  type: ReferenceError

esid

esid: [spec-id]

This tag is required for all new feature tests. This tag identifies the hash ID from the portion of the ECMAScript draft which is most recent to the date the test was added. It represents the anchors on the generated HTML version of the specs. E.g.: esid: sec-typedarray-length. This tag might be used to replace a es6id or further.

When writing a new test for a Stage 3+ spec not yet published on the draft, the pending value can be used while a hash ID is not available.

includes

includes: [file-list]

This tag names a list of helper files that will be included in the test environment prior to running the test. Filenames must include the .js extension.

The helper files are found in the harness/ directory. When some code is used repeatedly across a group of tests, a new helper function (or group of helpers) can be defined. Helpers increase test complexity, so they should be created and used sparingly.

timeout

timeout: [integer]

This tag specifies the number of milliseconds to wait before the test runner declares an asynchronous test to have timed out. It has no effect on synchronous tests.

Test authors should not use this tag except as a last resort. Each runner is allowed to provide its own default timeout, and the user may be permitted to override this in order to account for unusually fast or slow hardware, network delays, etc.

author

author: [string]

This tag is used to identify the author of a test case.

flags

flags: [list]

This tag is for boolean properties associated with the test.

  • onlyStrict - only run the test in strict mode
  • noStrict - only run the test in "sloppy" mode
  • module - interpret the source text as module code
  • raw - execute the test without any modification (no helpers will be available); necessary to test the behavior of directive prologue; implies noStrict
  • async - defer interpretation of test results until after the invocation of the global $DONE function
  • generated - informative flag used to denote test files that were created procedurally using the project's test generation tool; refer to the section titled "Procedurally-generated tests" for more information on this process
  • CanBlockIsFalse - only run the test when the CanBlock property of the Agent Record executing the test file is false
  • CanBlockIsTrue - only run the test when the CanBlock property of the Agent Record executing the test file is true

features

features: [list]

Some tests require the use of language features that are not directly described by the test file's location in the directory structure. These features should be specified with this tag. See the features.txt file for a complete list of available values. This tag is required for new tests written for new features, but contributions will not be "blocked" if the tag is missing from frontmatter. The committing maintainer is required to ensure that the tag is present and contains the correct feature names; this can be done in an follow up commit.