ember-cli-visual-acceptance

Create baseline images and test for CSS regression during standard Ember tests using html2Canvas and ResembleJS

Installation

ember install ember-cli-visual-acceptance

Configuration

Custom Image upload

There are many cases where you don't want to use Imgur (such as repo's on Github enterprise). So in your ember-cli-build.js you have the ability to utilize your own upload function.

upload: function (image, index,  req, options) {
  let result = 'foobar-image-url'
  console.log('We using me for images w/ result: ', result)
  return result
}

The return value should be the url where the image is hosted. The parameters are as follows:

• image: The base64 string of the image to upload
• index: The index of the current image that relates to (Diff, Current, Baseline)
• req: the request object from the middleware
• options: the visualAcceptanceOptions in ember-cli-build.js

Location of saved images

You can specify the location where the saved images will be stored (do not store under tests directory). Include the following in your ember-cli-build.js:

visualAcceptanceOptions: {
  imageDirectory: '<folder name>'
}

Browsers - NightmareJS(Electron) html2canvas vs. PhantomJS render callback

NightmareJS(Electron)

NightmareJS is a new addition to ember-cli-visual-acceptance. NightmareJS is run on top of Electron, using Electron to take the capture. This capture should provide the most accurate representation of your app.

Notes

• If the image capture is an empty file or distorted image you should try extending the viewport in vendor/nightmarejs-launcher.js the viewport currently set to .viewport(3000, 4000) which should be ample room (but the more tests you have the longer the page will grow). The distortion/emptiness of the image is caused by the captured area not being visible on Electron.

PhantomJS - SlimerJS

PhantomJS and SlimerJS can both be used with this tool to capture images.

Personally I prefer SlimerJS. As PhantomJS's webkit version is behind the latest Safari's webkit. While SlimerJS uses the same version of Gecko as the latest Firefox.

Warning

With certain repositories I've had trouble with SlimerJS having segmentation faults on both Linux and Mac. I've yet to resolve this issue. So I have re-included html2Canvas work.

Html2Canvas

Html2Canvas is used when a browser does not have the function window.callPhantom (Only PhantomJS and SlimerJS have this defined). Html2Canvas is still in beta and as result you will see some issues. Html2Canvas relies on Canvas drawing support. I find Chrome has the best Canvas drawing support (miles ahead of their competitors), while Firefox has the second best Canvas drawing support.

SVGs

Html2Canvas has difficulties rendering SVGs (more so in Firefox than in Chrome). As a result I have added a new expermental functionality that attempts to render the svgs better. You can use this experimental feature by setting experimentalSvgs to true (Example: capture('svg-experimental', null, null, null, true))

Target browser and OS version

You can specify the exact version of the browser as well as the OS the test will be run against.

- install `ember-cli-visual-acceptance`
- run `ember test -s` (this will launch a browser)
- open the console and type `window.ui`

To target specific versions, you can edit ember-cli-build.js as follows:

visualAcceptanceOptions: {
  targetBrowsers: [{
    browser: "Chrome",
    os: "Mac OS X",
    osversion: "10.11.2",
    version: "49.0.2623.112"
  }]
}

Also, rather then specifying exact version for the browser and the OS versions ,you can append >= as follows:

visualAcceptanceOptions: {
  targetBrowsers: [{
    browser: "Chrome",
    os: "Mac OS X",
    osversion: ">=10.11.2",
    version: ">=49.0.2623.112"
  }]
}

You can also omit the osversion and version if not needed.

API

capture (imageName, options)

Name	Type	Default	Description
imageName	string	required	Name of the image you wish to save
options	object	{}	Object that holds all the options for the capture
options.width	number	null	Define the width of the canvas in pixels. If null, renders with full width of the targetElement.
options.height	number	null	Define the height of the canvas in pixels. If null, renders with full height of the targetElement.
options.misMatchPercentageMargin	float	0.00	The maximum percentage ResembleJs is allowed to misMatch.
options.targetElement	HTMLElement	ember-testing-container	DOM element to capture (Most likely want to set height and width to null, so we don't overwrite the element's height and width )
options.experimentalSvgs	boolean	false	Set to true in order try experimental rendering of svgs using html2canvas.
options.assert	object	undefined	Only use to pass in Qunit assert object.

Usage

• Configure the systems and browsers that will capture images in your ember-cli-build.js as described above in the Configuration section.

  • Different systems and browsers produce different images
  • To prevent false positives images are only captured against specific targets
  • The results of each target are stored in separate directories and are only compared against the same target

• Create your tests (ex: tests/integration/components/frost-button-test.js)

it ('primary small button', function(done) {
  this.render(hbs`
    {{frost-button
      priority='primary'
      size='small'
      text='Text'
    }}`)
  capture('primary-small-button', done)
})

• Run ember test -s

Baseline

• The first capture will automatically become the baseline image.
• To create a new baseline, run the following:

ember new-baseline

Failure example

From ember test:

Integration: FrostSelectComponent selects the hovered item when enter is pressed
    ✘ Image is above mismatch threshold.: expected false to be true
        AssertionError: Image is above mismatch threshold.: expected false to be true

Then a new <nameOfImage>-fail.png will show up in your visual-acceptance directory. Visual differences are shown in pink. More info about visual diffs can be found here. ember-cli-visual-acceptance only uses the .scaleToSameSize() option for ResembleJS

Example Usage

Without done() callback

it('supports placeholder', function () {
  const $input = this.$('.frost-select input')
  expect($input.attr('placeholder')).to.eql('Select something already')
  return capture('placeholder')
})

With done() callback

it('selects the hovered item when enter is pressed', function (done) {
  keyUp(dropDown, 40)
  keyUp(dropDown, 13)

  Ember.run.later(() => {
    let dropDownInput = this.$('.frost-select input')
    let value = dropDownInput.val()
    expect(value).to.eql(props.data[0].label)
    capture('Boston', done, { width: 1920, height: 1080, misMatchPercentageMargin: 5.00})
  })
})

Working with Mirage

ember-cli-visual-acceptance makes api calls to it's own testem middleware. So in order for the tests to work you must have this.passthrough(), or list the paths explicitly in this.passthrough('/image','/passed','/fail','/report','/istargetbrowser','/should-assert') in your mirage config.js.

Svg4everybody and <use> tags

Currently there is no canvas support for svg <use> tags. As a result visual acceptance will produce a blank canvas in the place of those svgs.

Ember-Frost-Core

If your svgs are coming from ember-frost-core you can utilize the inline svg rendering config option in your tests/dummy/config/environment.js. If using visual acceptance in an integration test you must import the svg-use-polyfill instance initializer and run it beforeEach test. You can import the intializer via import {initialize as initializeSvgUse} from 'ember-frost-core/instance-initializers/svg-use-polyfill' and make use of it in the beforeEach hook by doing

beforeEach(function () {
  initializeSvgUse()
})

Setting up Travis

The details to setup Travis can be found here. Once complete visual-acceptance will be able to attach reports to your Pull Requests.

Linting

ESLint

Add the below line to the top of your test file:

/* global capture */

as seen in the sample integration test.

JSHint

Add the below line to the top of your test file:

/* globals capture */