- Feb 19, 2016
-
-
Mike Pennisi authored
Document the console runner's "deprecated" status. Move the documentation to a less central location in the project's file hierarchy.
-
Mike Pennisi authored
This reverts commit 7ae29d49.
-
Mike Pennisi authored
This reverts commit 21781289.
-
Mike Pennisi authored
This reverts commit b791cc4f.
-
- Feb 10, 2016
-
-
Mike Pennisi authored
When executing multiple tests in parallel, each "child" thread would write to the process's standard output buffer immediately upon test completion. Because thread execution order and instruction interleaving is non-deterministic, this made it possible for characters to be emitted out-of-order. When extended to support multiple concurrent threads, the runner was outfitted with a "log lock" dedicated to sharing access to the output file (when applicable). Re-use this lock when writing to standard out, ensuring proper ordering of test result messages.
-
Mike Pennisi authored
A recent extension to the test runner introduced support for running tests in parallel using multi-threading. Following this, the runner would incorrectly emit the "final report" before all individual test results. In order to emit the "final report" at the end of the output stream, the parent thread would initialize all children and wait for availability of a "log lock" shared by all children. According to the documentation on the "threading" module's Lock object [1]: > When more than one thread is blocked in acquire() waiting for the state > to turn to unlocked, only one thread proceeds when a release() call > resets the state to unlocked; which one of the waiting threads proceeds > is not defined, and may vary across implementations. This means the primitive cannot be used by the parent thread to reliably detect completion of all child threads. Update the parent to maintain a reference for each child thread, and to explicitly wait for every child thread to complete before emitting the final result. [1] https://docs.python.org/2/library/threading.html#lock-objects
-
- Feb 08, 2016
-
-
Till Schneidereit authored
Adds a `-j`/`--workers-count` parameter to `tools/packaging/test262.py`, defaulting to `[number of cores] - 1`. Speeds up running the test suite by about ~3x on my 4-core machine, with the SpiderMonkey shell. This could certainly be optimized more by just appending test results to per-thread lists and merging them at the end, but it's better than nothing.
-
- Jul 17, 2015
-
-
André Bargull authored
-
André Bargull authored
-
- Jul 07, 2015
-
-
Mike Pennisi authored
Some tests involving the directive prologue are invalidated by source text transformations that insert executable code in the beginning of the script. Implement a `raw` flag that allows these tests to opt-out of this transformation. Update the relevant tests to use this flag (and remove references to globals only available when code is injected). Update the Python runner accordingly: - Do not run tests marked as "raw" in strict mode - Reject invalid test configurations Update the browser runner accordingly: - Do not modify the script body of tests marked as "raw"
-
- Jul 06, 2015
-
-
Erik Arvidsson authored
-
- Jul 01, 2015
-
-
Mike Pennisi authored
The "monkeyYaml" parser is intended to serve as a lightweight fallback to Python's standard YAML parser in contexts where the latter is not available. Any intentionally-simplified implementation will necessarily exhibit non-standard behavior for different input, so not all input accepted by the standard parser will be accepted by "monkeyYaml". If loaded exclusively in fallback situations, these edge cases can only be identified (and debugged) in the environments that require the fallback. This has allowed developers to unknowingly author tests that cause errors. Update the test runner to use "monkeyYaml" in all cases, ensuring more consistent behavior across contexts and precluding this class of regression.
-
Erik Arvidsson authored
Fixes #345
-
- Jun 26, 2015
-
-
Mike Pennisi authored
Some JavaScript source files are only relevant in the context of the Test262 website. They should not be explicitly included by individual tests, so their presence in the `harness/` directory alongside "include" files is misleading. Move the scripts to a location within the `website/` directory to better-reflect their intended use. Update the relevant HTML templates with the new locations.
-
Mike Pennisi authored
Although test files once expressed dependencies on external files using a global `$INCLUDE` function, that pattern was removed in favor of declarative meta-data [1]. Remove the associated logic from the Python runner and the browser. [1] See commit d4354d14.
-
- Jun 18, 2015
-
-
Mike Pennisi authored
Since the Python runner was updated to include `assert.js` in all tests unconditionally, a number of tests have been written that implicitly rely on its presence. The browser runner does not currently provide this file's contents to these tests, so they fail unconditionally. Update the browser runner to inject that file's contents into every test context. Note: the existing approach to file retrieval (namely loading via synchronous XHR requests) is inefficient and deprecated in some browsers. It is honored here for the sake of consistency and to minimize the changeset necessary to fix the browser runner.
-
- Jun 10, 2015
-
-
Erik Arvidsson authored
monkeyYaml didn't split lines correctly leading to \r in resulting values. Fixes #295
-
- May 25, 2015
-
-
Mike Pennisi authored
Test262 now accepts community contributions. Update the copy on the website to reflect this.
-
- May 14, 2015
-
-
Brian Terlson authored
-
- Mar 30, 2015
-
-
smikes authored
additional test, tests should have unique names improve python style thx to arv
-
- Feb 27, 2015
-
-
Gregory Brail authored
Jenkins. Wrap XML output correctly and post-process failure messages to avoid invalid XML characters.
-
- Dec 09, 2014
-
-
Brian Terlson authored
-
- Dec 07, 2014
-
-
Brian Terlson authored
-
- Dec 02, 2014
-
-
smikes authored
-
- Dec 01, 2014
-
-
Domenic Denicola authored
Presumably since Python is whitespace-sensitive, this is actually quite important?
-
Domenic Denicola authored
-
- Nov 12, 2014
-
-
smikes authored
add test to repro #114 fix issue #114 factor out PercentFormat function new fn write; start factoring summary output fix tests to match consolidate "Failed Tests" message add test for successful run consolidate all_succeeded message consolidate summary line consolidate ntests failing negative tests squeeze whitespace
-
- Oct 24, 2014
- Aug 28, 2014
-
-
Sam Mikes authored
parseTestRecord.py: - recover from ImportError when YAML not defined - use monkeyYaml as backup monkeyYaml.py: - parser for subset of YAML used in test262 frontmatter test_monkeyYaml.py: - unit tests
-
- Aug 12, 2014
-
-
Sam Mikes authored
pilot directory for replacing flags: [negative] with negative: errorname ch07-7.2,7.3: add error names ch07-rest: insert error names rest of ch07 ch08: add expected error name ch11: add error name to negative ch12: put error name in negative ch13: add error name to negative only one test, did it manually ch15: add error name to negative: ch07: add error name to negative these tests used flags: \n - negative and so were not caught by the earlier naive grep for flags: [negative] --- changes suggested by @anba bestPractice - remove added NotEarlyError ch07,ch11 - expect ReferenceError (req'd by ES6) correct test descriptions to expect ReferenceError only enforce parsing as block statement by adding `;` remove needless include of $FAIL.js effectively, a rebase of all changes onto master Merge branch 'negative-addErrorName-work' into negative-addErrorName-bestPractice Merge branch 'saved-bestPractice-negative' into negative-addErrorName-work Conflicts: test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T2.js test/suite/ch12/12.1/S12.1_A4_T1.js test/suite/ch12/12.5/S12.5_A2.js incorporate changes made on PR branch re-remove added NotEarlyError Merge branch 'negative-addErrorName-work' into negative-addErrorName-bestPractice Conflicts: test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T2.js test/suite/ch12/12.1/S12.1_A4_T1.js test/suite/ch12/12.5/S12.5_A2.js ch15: correct negative regexes ch10: avoid use of not-NotEarlyError regex ch14 error regexps ch13 error regexps ch12: negative error matching Merge branch 'console-runner-checkError' into negative-addErrorName-work test262.py: check negative tests with regex implement checking of negative tests => negative: SyntaxError means that /SyntaxError/ must match stderr or test is reported as failure Fixes #78 expect real errors bestPractice: supply error names to negative pilot directory for replacing flags: [negative] with negative: errorname ch07-7.2,7.3: add error names ch07-rest: insert error names rest of ch07 ch08: add expected error name ch11: add error name to negative ch12: put error name in negative ch13: add error name to negative only one test, did it manually ch15: add error name to negative: ch07: add error name to negative these tests used flags: \n - negative and so were not caught by the earlier naive grep for flags: [negative] --- changes suggested by @anba bestPractice - remove added NotEarlyError ch07,ch11 - expect ReferenceError (req'd by ES6) correct test descriptions to expect ReferenceError only enforce parsing as block statement by adding `;` remove needless include of $FAIL.js Merge branch 'negative-addErrorName-work' into negative-addErrorName-bestPractice resolved Conflicts: test/suite/ch07/7.9/S7.9_A5.7_T1.js test/suite/ch11/11.13/11.13.1/S11.13.1_A2.1_T3.js test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T1.js test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T10.js test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T11.js test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T2.js test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T3.js test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T4.js test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T5.js test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T6.js test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T7.js test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T8.js test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T9.js test/suite/ch11/11.3/11.3.1/S11.3.1_A2.1_T3.js test/suite/ch11/11.3/11.3.2/S11.3.2_A2.1_T3.js test/suite/ch11/11.4/11.4.4/S11.4.4_A2.1_T3.js test/suite/ch11/11.4/11.4.5/S11.4.5_A2.1_T3.js test/suite/ch12/12.5/S12.5_A2.js test/suite/ch12/12.6/12.6.3/S12.6.3_A4_T2.js negative: second half of @anba notes correct test descriptions to expect ReferenceError only enforce parse as block statement by adding `;` remove needless include of $FAIL.js ch12: specify type negative: @anba fixes bestPractice - remove added NotEarlyError ch07,ch11 - expect ReferenceError (req'd by ES6) bestPractice: supply error names to negative pilot directory for replacing flags: [negative] with negative: errorname ch07-7.2,7.3: add error names ch07-rest: insert error names rest of ch07 ch08: add expected error name ch11: add error name to negative ch12: put error name in negative ch13: add error name to negative only one test, did it manually ch15: add error name to negative: ch07: add error name to negative these tests used flags: \n - negative and so were not caught by the earlier naive grep for flags: [negative] expect a SyntaxError ch07: add error name to negative these tests used flags: \n - negative and so were not caught by the earlier naive grep for flags: [negative] ch15: add error name to negative: ch13: add error name to negative only one test, did it manually ch12: put error name in negative ch11: add error name to negative ch08: add expected error name ch07-rest: insert error names rest of ch07 ch07-7.2,7.3: add error names negative-errorname: pilot pilot directory for replacing flags: [negative] with negative: errorname bestPractice: supply error names to negative bestPractice: supply error names to negative pilot directory for replacing flags: [negative] with negative: errorname ch07-7.2,7.3: add error names ch07-rest: insert error names rest of ch07 ch08: add expected error name ch11: add error name to negative ch12: put error name in negative ch13: add error name to negative only one test, did it manually ch15: add error name to negative: ch07: add error name to negative these tests used flags: \n - negative and so were not caught by the earlier naive grep for flags: [negative] bestPractice: supply error names to negative negative-errorname: pilot pilot directory for replacing flags: [negative] with negative: errorname ch07-7.2,7.3: add error names ch07-rest: insert error names rest of ch07 ch08: add expected error name ch11: add error name to negative ch12: put error name in negative ch13: add error name to negative only one test, did it manually ch15: add error name to negative: ch07: add error name to negative these tests used flags: \n - negative and so were not caught by the earlier naive grep for flags: [negative] expect a SyntaxError negative: @anba fixes bestPractice - remove added NotEarlyError ch07,ch11 - expect ReferenceError (req'd by ES6) ch12: specify type negative: second half of @anba notes correct test descriptions to expect ReferenceError only enforce parse as block statement by adding `;` remove needless include of $FAIL.js expect real errors ch12: negative error matching ch13 error regexps ch14 error regexps add "description" header back remove now-unused includes remove needless include
-
- Aug 02, 2014
-
-
Sam Mikes authored
sta.js: slight change to Test262Error() semantics; message property now always set (default "") make $ERROR a var set $ERROR to function $ERROR so it can be overridden if needed remove 2009 copyright in favor of 2012 copyright REVERTED: remove never-used fn testFailed testFailed actually used by $FAIL remove obsolete fn $INCLUDE ed.js: remove commented-out obsolete code test262.py: remove always-included harness file that provides no functions used by any extant test add comment line to nonstrict cth.js: define `print` for node, cscript use cth (console test harness) to define functions for console runner. V8, Spidermonkey (js) and JavaScriptCore (jsc) provide a function `print`. Provide a default `print` for node and cscript set print_handle to 'print' by default (can still override) for cscript: wrap tests in try/catch so we get syntax errors
-
- Jul 30, 2014
-
-
Sam Mikes authored
parseTestRecord: add support for YAML frontmatter parseTestRecord: initial unit test for test record parser parseTestRecord: refactor for testing factor old parsing; add YAML parsing runner: support "includes" from YAML frontmatter support frontmatter "includes" in python runner use test.includes if present instead of scanning test code with regex harness: factor individual functions out into files tools: handle YAML errors tolerate missing keys in dictionary (flags, includes) report filename when empty frontmatter block new option --list-includes to test262.py harness: factor helper functions into separate files sth: remove extra close-paren (syntax error) test_common: TDD; failing parse of YAML common: use parseTestRecord (YAML-aware)
-
- Jul 21, 2014
-
-
Sam Mikes authored
test262.py: only supply async helper scripts when test is async sth.js: factor out function isAsyncTest() timer.js: improve workaround for async tests when Promise is defined but setTimeout is noot timer.js emulates setTimeout using Promise by doing a busy loop that checks if `timeout` milliseconds have elapsed. Modified check to (timeLeft > 0) instead of (!timeLeft) to prevent infinite loop when check does not happen to run at precise millisecond timeout expires. Because test262.py did not support the $INCLUDE directive, some helper scripts were added to every test -- notably testIntl, timer, and donePrintHandle Now that $INCLUDE is supported, these can be dropped, speeding overall test run time
-
- Jul 18, 2014
-
-
smikes authored
packager.py cannot run due to syntax errors in a few script files packagerConfig: use git instead of hg test262: use pop instead of delete to avoid throw if property missing S22.1.2.1_T*: fix docString header comment: s/b /** S22.1.2.1_T3: fix docString header comment: s/b /**, fix end of docstring * / => */
-
Sam Mikes authored
doneprintHandle.js: make $DONE accept any falsy argument as meaning 'pass' PromiseHelper.js: checkSequence: new helper fn for async tests .gitignore: port .hgignore to .gitignore syntax test262.py: support $INCLUDE directive in python test runner S25.4.4.1*: tests to cover Section 25.4.4.1, Promise.all( iterable ) A1.1: Promise.all is callable A1.2: Promise.all expects 1 argument A2.1: Promise.all([]) is a Promise A2.2: Promise.all([]) is resolved immediately A2.3: Promise.all([]) is resolved with a new empty array A3.1: Promise.all expects an iterable argument
-
- Jul 15, 2014
-
-
NikSurya authored
-
- Mar 02, 2014
-
-
Yusuke Suzuki authored
-
- Jun 10, 2013
-
-
Prashanth Srinivasan authored
-
- May 10, 2013
-
-
Dan Schaffer authored
-