Yeti automates browser testing.
Yeti automates tests written for various test frameworks. Yeti scales from your dev box (where it works by itself) to your CI system (where it launches browsers with Selenium) without changing your existing tests.
You can use any of these test frameworks with Yeti.
npm install -g yetiYeti requires Node.js, which provides the npm command for installation.
You can download Node.js source or pre-built
installers from their website.
Just run Yeti with the HTML files containing your tests.
$ yeti test/*.html
Creating a Hub at http://localhost:9000
Waiting for agents to connect at http://localhost:9000.
When ready, press Enter to begin testing.Point your browsers at that URL, then come back and press Enter.
[Open some browsers...]
Agent connected: Safari (6.0) / Mac OS
Agent connected: Chrome (22.0.1221.0) / Mac OS
[Come back, press Enter]
✔ Testing started on Safari (6.0) / Mac OS, Chrome (22.0.1221.0) / Mac OS
✔ Agent completed: Safari (6.0) / Mac OS
✔ Agent completed: Chrome (22.0.1221.0) / Mac OS
504 tests passed! (9.1 seconds)
$Yeti exits automatically when all tests complete. If test failures occur, Yeti will exit with a non-zero status code.
0 indicates testing was completed and all tests passed.1 indicates an error occurred that prevented testing from running to completion.3 indicated testing was completed but some tests failed.These codes are useful for using Yeti programmatically. The codes 0 and 3 indicate testing completion.
If Yeti ended in code 1, you may wish to retry the command to workaround transient errors. This is useful
when using Yeti to launch browsers in CI, which sometimes may fail due to temporary Selenium issues.
Yeti can output machine-readable JUnit XML suitable for use in [Jenkins][] with the --junit option.
$ yeti --junit test/*.html > yeti.xmlYeti will output XML on stdout and status messages on stderr.
When using Yeti several times in the same Jenkins job, it's useful to label tests with a prefix
to distinguish between different Yeti runs after Jenkins merges the reports together. You can
assign this prefix with the --name option.
$ yeti --name stable --junit test/stable/*.html > stable.xml
$ yeti --name flaky --junit test/flaky/*.html > flaky.xmlYeti can generate a code coverage report with the -c or --coverage option.
$ yeti -c
Found 1 file to test.
Agent connected: Chrome (34.0.1825.4) / Mac OS from 127.0.0.1
✓ Testing started on Chrome (34.0.1825.4) / Mac OS
✓ Agent completed: Chrome (34.0.1825.4) / Mac OS
Coverage summary:
Statements : 96.02% ( 1110/1156 )
Branches : 88.53% ( 772/872 )
Functions : 97.45% ( 229/235 )
Lines : 96.27% ( 1110/1153 )
✓ 397 tests passed! (7 seconds)Use the --coverage-report option to generate reports written to ./coverage
or another directory defined by --coverage-dir. Values for --coverage-report include:
lcov for LCOV and HTML reports.lcovonly for LCOV without HTML.html for HTML only.json for Istanbul JSON.summary for the default summary shown above.Yeti instruments all JavaScript files automatically with [Istanbul][]. If you'd prefer to instrument
your JavaScript code with Istanbul as a part of a build step, you can pass --no-instrument
to skip all instrumenting. Yeti will still generate code coverage reports if used with
the --coverage option.
If you'd like to exclude some JavaScript files from instrumenting, you can pass multiple
--instrument-exclude options defining [minimatch][] patterns to exclude.
$ yeti -c --instrument-exclude "**/vendor/**" --instrument-exclude "**.nocover.js"To prevent needing to specify these verbose options every time, you can put a
coverageOptions object in your project's .yeti.json file containing configuration
that is only used with --coverage.
$ cd yui3/src/app
$ cat ../../.yeti.json
{
"basedir": ".",
"glob": "**/tests/unit/*.html",
"coverageOptions": {
"instrument": false,
"query": "filter=coverage"
}
}
$ yeti -c # will run with --no-instrument --query "filter=coverage"
Found 1 file to test.
Agent connected: Chrome (34.0.1825.4) / Mac OS from 127.0.0.1
✓ Testing started on Chrome (34.0.1825.4) / Mac OS
✓ Agent completed: Chrome (34.0.1825.4) / Mac OS
Coverage summary:
Statements : 96.02% ( 1110/1156 )
Branches : 88.53% ( 772/872 )
Functions : 97.45% ( 229/235 )
Lines : 96.27% ( 1110/1153 )
✓ 397 tests passed! (7 seconds)Yeti provides server-side AJAX routes with [echoecho][ee]. Your test can [make relative HTTP requests][ee-usage] to test your code against server-side HTTP GET, POST, PUT, DELETE, OPTIONS, GET with delay, JSON or JSONP responses via POST, or any HTTP status code.
Example supported routes:
echo/status/500 returns a 500 response.echo/delay/3 returns a 200 response after 3 seconds.echo/jsonp?callback=foo returns a JSONP response with the given
POST data wrapped in a call to foo.Note these routes are intentionally relative paths. See the [echoecho README][ee-readme] for more details.
Yeti will move on to the next test if a test takes longer than 5 minutes (300 seconds).
You can adjust this interval with the --timeout option.
This will run Yeti with a 120 second timeout:
$ yeti --timeout 120 test.htmlThere isn't a general timeout setting. Yeti actively pings browsers about every 2-5 seconds and disconnects them if they fail to respond to a ping three times.
You can specify query string parameters to add to your test URLs. This can be used to pass information to your tests that control its behavior.
This will append ?filter=coverage to your tests, which is used by the tests
for the [YUI Library][YUI] to trigger loading instrumented code.
$ yeti --query 'filter=coverage' test/*.htmlYeti will report an uncaught exceptions as Script Errors.
Yeti enforces [No-Quirks Mode][] in your tests because it may impact DOM-related APIs. [Add a DOCTYPE][doctype] to your test document to fix this.
When combined with [localtunnel][], mobile testing is simple. If you're not dealing with sensitive information, startup your Yeti Hub and then run:
$ localtunnel 9000
Port 9000 is now publicly accessible from http://3z48.localtunnel.com ...You can then visit that URL on your mobile (or any other) device and have it run new tests.
To save time, start a Yeti Hub.
$ yeti --server
Yeti Hub listening on port 9000.Point browsers at your local Yeti on port 9000. Now, you're ready to run tests without having to reconnect browsers each time.
Starting Yeti in another terminal will connect to that Hub instead of starting a new one and will begin testing immediately if browsers are already connected.
$ yeti test/*.html
Connected to http://localhost:9000
Agent connected: Chrome (22.0.1221.0) / Mac OS
Agent connected: Safari (6.0) / Mac OS
✔ Testing started on Chrome (22.0.1221.0) / Mac OS, Safari (6.0) / Mac OS
✔ Agent completed: Chrome (22.0.1221.0) / Mac OS
✔ Agent completed: Safari (6.0) / Mac OS
504 tests passed! (11.5 seconds)
$Your Yeti Hub can be shared with other developers.
First, I'll start a Hub on test.yeti.cx on port 80.
$ yeti --server --port 80Go ahead and point a few browsers there.
Now, others can connect to it from their computer like so:
$ yeti --hub http://test.yeti.cx/ test/*.html
Connected to http://test.yeti.cx/
Waiting for agents to connect at http://test.yeti.cx/.
When ready, press Enter to begin testing.Your pwd and your test file will be served through the Hub. Like magic.
[Hit Enter]
Agent connected: Chrome (22.0.1221.0) / Mac OS
Agent connected: Safari (6.0) / Mac OS
✔ Testing started on Chrome (22.0.1221.0) / Mac OS, Safari (6.0) / Mac OS
✔ Agent completed: Safari (6.0) / Mac OS
✔ Agent completed: Chrome (22.0.1221.0) / Mac OS
504 tests passed! (8.7 seconds)This makes it really simple to setup an ad-hoc testing lab shared with your team.
You can specify the wd-url option to connect Yeti to a Selenium 2 Hub using the
WebDriver protocol. Specifying one or more caps options will cause Yeti to launch
browsers for the given capabilities over WebDriver.
For example, launching 2 Firefox browsers to test multiple files:
yeti --hub http://example.com --wd-url http://localhost:4444 \
--caps "browserName=firefox;version=26;platform=Windows XP" \
--caps "browserName=firefox;version=26;platform=Windows XP" test/*.htmlOr launching an iPhone simulator on Sauce Labs:
yeti --caps "platform=OS X 10.8;version=7;device-orientation=portrait;app=safari;device=iPhone Simulator" test.htmlHere's a breakdown of all available CLI options.
false or specify --no-hub to override a configuration file.debug or info) Print debugging information.wd-url.You may use place JSON in a .yeti.json file to set project or user specific configuration.
Yeti will look for .yeti.json in these places:
Here is an example .yeti.json for the YUI project, which is placed in the repository root:
{
"hub": "http://test.yeti.cx/",
"basedir": ".",
"glob": "**/tests/unit/*.html"
}Here is the breakdown of these settings:
.yeti.json lives is
permitted to serve files to the Yeti Hub.These settings let YUI developers simply run yeti inside of the project directory
to run tests. Since all tests in the project match the glob pattern, the yeti
command works for specific components as well as for the entire project.
This configuration can be overridden on the command line. For example, to ignore the
hub setting, you can run Yeti with --no-hub.
You can require("yeti") inside your application to script Yeti for your own use.
For API documentation:
make html toNo open issues yet, or sync has not completed.