Skip to content
Snippets Groups Projects
  1. Feb 12, 2016
    • Mike Pennisi's avatar
      Make asynchronous test configuration explicit · 23d56620
      Mike Pennisi authored
      For asynchronous tests, the contract between test file and test runner
      is implicit: runners are expected to inspect the source code for
      references to a global `$DONE` identifier.
      
      Promote a more explicit contract between test file and test runner by
      introducing a new frontmatter "tag", `async`. This brings asynchronous
      test configuration in-line with other configuration mechanisms and also
      provides a more natural means of test filtering.
      
      The modifications to test files was made programatically using the
      `grep` and `sed` utilities:
      
          $ grep "\$DONE" test/ -r --files-with-match --null | \
              xargs -0 sed -i 's/^\(flags:\s*\)\[/\1[async, /g'
          $ grep "\$DONE" test/ -rl --null | \
              xargs -0 grep -E '^flags:' --files-without-match --null | \
              xargs -0 sed -i 's/^---\*\//flags: [async]\n---*\//'
      23d56620
  2. Feb 08, 2016
    • Till Schneidereit's avatar
      Enable parallel test execution in console runner · 7ae29d49
      Till Schneidereit authored
      Adds a `-j`/`--workers-count` parameter to `tools/packaging/test262.py`, defaulting to `[number of cores] - 1`.
      
      Speeds up running the test suite by about ~3x on my 4-core machine, with the SpiderMonkey shell. This could certainly be optimized more by just appending test results to per-thread lists and merging them at the end, but it's better than nothing.
      7ae29d49
  3. Jul 07, 2015
    • Mike Pennisi's avatar
      Implement `raw` flag · ab7617de
      Mike Pennisi authored
      Some tests involving the directive prologue are invalidated by source
      text transformations that insert executable code in the beginning of the
      script. Implement a `raw` flag that allows these tests to opt-out of
      this transformation. Update the relevant tests to use this flag (and
      remove references to globals only available when code is injected).
      
      Update the Python runner accordingly:
      
      - Do not run tests marked as "raw" in strict mode
      - Reject invalid test configurations
      
      Update the browser runner accordingly:
      
      - Do not modify the script body of tests marked as "raw"
      ab7617de
  4. Jun 26, 2015
    • Mike Pennisi's avatar
      Remove support for legacy `$INCLUDE` syntax · 86c7e272
      Mike Pennisi authored
      Although test files once expressed dependencies on external files using
      a global `$INCLUDE` function, that pattern was removed in favor of
      declarative meta-data [1].
      
      Remove the associated logic from the Python runner and the browser.
      
      [1] See commit d4354d14.
      86c7e272
  5. May 14, 2015
  6. Feb 27, 2015
  7. Dec 07, 2014
  8. Dec 02, 2014
  9. Dec 01, 2014
  10. Nov 12, 2014
    • smikes's avatar
      add some unit tests · 64093f79
      smikes authored
      add test to repro #114
      fix issue #114
      factor out PercentFormat function
      new fn write; start factoring summary output
      fix tests to match
      consolidate "Failed Tests" message
      add test for successful run
      consolidate all_succeeded message
      consolidate summary line
      consolidate ntests
      failing negative tests
      squeeze whitespace
      64093f79
  11. Oct 24, 2014
  12. Aug 12, 2014
    • Sam Mikes's avatar
      bestPractice: supply error names to negative · 6b7e5d77
      Sam Mikes authored
      pilot directory for replacing flags: [negative] with
      negative: errorname
      
      ch07-7.2,7.3: add error names
      ch07-rest: insert error names rest of ch07
      ch08: add expected error name
      ch11: add error name to negative
      ch12: put error name in negative
      
      ch13: add error name to negative
      only one test, did it manually
      
      ch15: add error name to negative:
      
      ch07: add error name to negative
      
      these tests used flags: \n - negative and so
      were not caught by the earlier naive grep for flags: [negative]
      
      --- changes suggested by @anba
      
      bestPractice - remove added NotEarlyError
      ch07,ch11 - expect ReferenceError (req'd by ES6)
      correct test descriptions to expect ReferenceError only
      enforce parsing as block statement by adding `;`
      remove needless include of $FAIL.js
      
      effectively, a rebase of all changes onto master
      
          Merge branch 'negative-addErrorName-work' into negative-addErrorName-bestPractice
      
          Merge branch 'saved-bestPractice-negative' into negative-addErrorName-work
      
          Conflicts:
          	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T2.js
          	test/suite/ch12/12.1/S12.1_A4_T1.js
          	test/suite/ch12/12.5/S12.5_A2.js
      
          incorporate changes made on PR branch
      
          re-remove added NotEarlyError
      
          Merge branch 'negative-addErrorName-work' into negative-addErrorName-bestPractice
      
          Conflicts:
          	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T2.js
          	test/suite/ch12/12.1/S12.1_A4_T1.js
          	test/suite/ch12/12.5/S12.5_A2.js
      
          ch15: correct negative regexes
      
          ch10: avoid use of not-NotEarlyError regex
      
          ch14 error regexps
      
          ch13 error regexps
      
          ch12: negative error matching
      
          Merge branch 'console-runner-checkError' into negative-addErrorName-work
      
          test262.py: check negative tests with regex
      
          implement checking of negative tests =>
      
          negative: SyntaxError
      
          means that /SyntaxError/ must match stderr
          or test is reported as failure
      
          Fixes #78
      
          expect real errors
      
          bestPractice: supply error names to negative
      
          pilot directory for replacing flags: [negative] with
          negative: errorname
      
          ch07-7.2,7.3: add error names
          ch07-rest: insert error names rest of ch07
          ch08: add expected error name
          ch11: add error name to negative
          ch12: put error name in negative
      
          ch13: add error name to negative
          only one test, did it manually
      
          ch15: add error name to negative:
      
          ch07: add error name to negative
      
          these tests used flags: \n - negative and so
          were not caught by the earlier naive grep for flags: [negative]
      
          --- changes suggested by @anba
      
          bestPractice - remove added NotEarlyError
          ch07,ch11 - expect ReferenceError (req'd by ES6)
          correct test descriptions to expect ReferenceError only
          enforce parsing as block statement by adding `;`
          remove needless include of $FAIL.js
      
          Merge branch 'negative-addErrorName-work' into negative-addErrorName-bestPractice
      
          resolved Conflicts:
          	test/suite/ch07/7.9/S7.9_A5.7_T1.js
          	test/suite/ch11/11.13/11.13.1/S11.13.1_A2.1_T3.js
          	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T1.js
          	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T10.js
          	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T11.js
          	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T2.js
          	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T3.js
          	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T4.js
          	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T5.js
          	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T6.js
          	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T7.js
          	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T8.js
          	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T9.js
          	test/suite/ch11/11.3/11.3.1/S11.3.1_A2.1_T3.js
          	test/suite/ch11/11.3/11.3.2/S11.3.2_A2.1_T3.js
          	test/suite/ch11/11.4/11.4.4/S11.4.4_A2.1_T3.js
          	test/suite/ch11/11.4/11.4.5/S11.4.5_A2.1_T3.js
          	test/suite/ch12/12.5/S12.5_A2.js
          	test/suite/ch12/12.6/12.6.3/S12.6.3_A4_T2.js
      
          negative: second half of @anba notes
      
          correct test descriptions to expect ReferenceError only
          enforce parse as block statement by adding `;`
          remove needless include of $FAIL.js
      
          ch12: specify type
      
          negative: @anba fixes
      
          bestPractice - remove added NotEarlyError
          ch07,ch11 - expect ReferenceError (req'd by ES6)
      
          bestPractice: supply error names to negative
      
          pilot directory for replacing flags: [negative] with
          negative: errorname
      
          ch07-7.2,7.3: add error names
          ch07-rest: insert error names rest of ch07
          ch08: add expected error name
          ch11: add error name to negative
          ch12: put error name in negative
      
          ch13: add error name to negative
          only one test, did it manually
      
          ch15: add error name to negative:
      
          ch07: add error name to negative
      
          these tests used flags: \n - negative and so
          were not caught by the earlier naive grep for flags: [negative]
      
          expect a SyntaxError
      
          ch07: add error name to negative
      
          these tests used flags: \n - negative and so
          were not caught by the earlier naive grep for flags: [negative]
      
          ch15: add error name to negative:
      
          ch13: add error name to negative
      
          only one test, did it manually
      
          ch12: put error name in negative
      
          ch11: add error name to negative
      
          ch08: add expected error name
          ch07-rest: insert error names rest of ch07
      
          ch07-7.2,7.3: add error names
      
          negative-errorname: pilot
      
          pilot directory for replacing flags: [negative] with
          negative: errorname
      
          bestPractice: supply error names to negative
      
      bestPractice: supply error names to negative
      
      pilot directory for replacing flags: [negative] with
      negative: errorname
      
      ch07-7.2,7.3: add error names
      ch07-rest: insert error names rest of ch07
      ch08: add expected error name
      ch11: add error name to negative
      ch12: put error name in negative
      
      ch13: add error name to negative
      only one test, did it manually
      
      ch15: add error name to negative:
      
      ch07: add error name to negative
      
      these tests used flags: \n - negative and so
      were not caught by the earlier naive grep for flags: [negative]
      
      bestPractice: supply error names to negative
      
      negative-errorname: pilot
      
      pilot directory for replacing flags: [negative] with
      negative: errorname
      
      ch07-7.2,7.3: add error names
      
      ch07-rest: insert error names rest of ch07
      
      ch08: add expected error name
      
      ch11: add error name to negative
      
      ch12: put error name in negative
      
      ch13: add error name to negative
      
      only one test, did it manually
      
      ch15: add error name to negative:
      
      ch07: add error name to negative
      
      these tests used flags: \n - negative and so
      were not caught by the earlier naive grep for flags: [negative]
      
      expect a SyntaxError
      
      negative: @anba fixes
      
      bestPractice - remove added NotEarlyError
      ch07,ch11 - expect ReferenceError (req'd by ES6)
      
      ch12: specify type
      
      negative: second half of @anba notes
      
      correct test descriptions to expect ReferenceError only
      enforce parse as block statement by adding `;`
      remove needless include of $FAIL.js
      
      expect real errors
      
      ch12: negative error matching
      
      ch13 error regexps
      
      ch14 error regexps
      
      add "description" header back
      
      remove now-unused includes
      
      remove needless include
      6b7e5d77
  13. Aug 02, 2014
    • Sam Mikes's avatar
      harness: remove unused code · 4205a1da
      Sam Mikes authored
      sta.js:
      slight change to Test262Error() semantics; message property now always set (default "")
      make $ERROR a var
      set $ERROR to function $ERROR so it can be overridden if needed
      remove 2009 copyright in favor of 2012 copyright
      REVERTED: remove never-used fn testFailed
      testFailed actually used by $FAIL
      remove obsolete fn $INCLUDE
      
      ed.js:
      remove commented-out obsolete code
      
      test262.py:
      remove always-included harness file that provides no functions used by any extant test
      add comment line to nonstrict
      
      cth.js: define `print` for node, cscript
      
      use cth (console test harness) to define functions
      for console runner.  V8, Spidermonkey (js) and JavaScriptCore (jsc)
      provide a function `print`.  Provide a default `print` for node
      and cscript
      
      set print_handle to 'print' by default (can still override)
      for cscript: wrap tests in try/catch so we get syntax errors
      4205a1da
  14. Jul 30, 2014
    • Sam Mikes's avatar
      tools, harness: support new YAML frontmatter · c33bf0e0
      Sam Mikes authored
      parseTestRecord: add support for YAML frontmatter
      parseTestRecord: initial unit test for test record parser
      parseTestRecord: refactor for testing
      
      factor old parsing; add YAML parsing
      
      runner: support "includes" from YAML frontmatter
      
      support frontmatter "includes" in python runner
      use test.includes if present instead of scanning test code with regex
      
      harness: factor individual functions out into files
      
      tools: handle YAML errors
      
      tolerate missing keys in dictionary (flags, includes)
      report filename when empty frontmatter block
      new option --list-includes to test262.py
      
      harness: factor helper functions into separate files
      
      sth: remove extra close-paren (syntax error)
      
      test_common: TDD; failing parse of YAML
      
      common: use parseTestRecord (YAML-aware)
      c33bf0e0
  15. Jul 21, 2014
    • Sam Mikes's avatar
      test262.py: only include helper scripts when needed · d4a3479a
      Sam Mikes authored
      test262.py: only supply async helper scripts when test is async
      sth.js: factor out function isAsyncTest()
      timer.js: improve workaround for async tests when Promise is defined but setTimeout is noot
      
      timer.js emulates setTimeout using Promise by doing a busy loop that checks
      if `timeout` milliseconds have elapsed.  Modified check to (timeLeft > 0) instead
      of (!timeLeft) to prevent infinite loop when check does not happen to run
      at precise millisecond timeout expires.
      
      Because test262.py did not support the $INCLUDE directive, some helper
      scripts were added to every test -- notably testIntl, timer, and donePrintHandle
      Now that $INCLUDE is supported, these can be dropped, speeding overall test run time
      d4a3479a
  16. Jul 18, 2014
    • smikes's avatar
      packager: make packager runnable · 7032af4d
      smikes authored
      packager.py cannot run due to syntax errors in a few script files
      
      packagerConfig: use git instead of hg
      test262: use pop instead of delete to avoid throw if property missing
      S22.1.2.1_T*: fix docString header comment: s/b /**
      S22.1.2.1_T3: fix docString header comment: s/b /**, fix end of docstring * / => */
      7032af4d
    • Sam Mikes's avatar
      async,promises: initial tests of Promises · 7e07cc13
      Sam Mikes authored
      doneprintHandle.js: make $DONE accept any falsy argument as meaning 'pass'
      PromiseHelper.js: checkSequence: new helper fn for async tests
      .gitignore: port .hgignore to .gitignore syntax
      test262.py: support $INCLUDE directive in python test runner
      
      S25.4.4.1*: tests to cover Section 25.4.4.1, Promise.all( iterable )
      A1.1: Promise.all is callable
      A1.2: Promise.all expects 1 argument
      A2.1: Promise.all([]) is a Promise
      A2.2: Promise.all([]) is resolved immediately
      A2.3: Promise.all([]) is resolved with a new empty array
      A3.1: Promise.all expects an iterable argument
      7e07cc13
  17. Jul 15, 2014
  18. Mar 02, 2014
  19. Jun 10, 2013
  20. May 10, 2013
  21. Aug 27, 2012
  22. Sep 30, 2011
  23. Sep 23, 2011
  24. Sep 14, 2011
  25. Sep 12, 2011
  26. Sep 11, 2011
Loading