Use responsive screenshots + diffs to ensure site looks "correct" prior to deployment

[jeffposnick]

Some combination of pageres for generating screenshots of pages for various viewport sizes and an image diff tool (blink-diff, maybe?) to determine whether anything has changed from our current set of "golden" images could help prevent us from deploying changes that inadvertently affect site layout.

(There's also the automated-screenshot-diff package, but I don't think that gives us the responsive screenshot aspect, which is important.)

[jeffposnick]

I've been looking into this again and wanted to provide an update:

• phantomjs-based approaches are a no-go, as the current release is based on a WebKit that's not compatible with Polymer.
• I'm working on an approach that uses selenium-webdriver. Because this relies on a local instance of Chrome (or another browser), I don't know that it will be possible to integrate with our current CI infrastructure.
• To deal with the initial "I/O" animation on initial page load I'm introducing a sleep(), but it would be nicer to wait for a specific event or for a style to change to trigger the screenshot. I'm looking into what would be the most appropriate condition to wait for.
• The experiment FAB's animation can lead to some false positives, but I think I can mask out the region from the image diff comparison.
• The current selenium Chrome driver won't do full-page screenshots, so multiple screenshots with automated scrolling in between each one will be required.

My plan is to create a new gulp command that could be manually run prior to merging a PR to get some peace of mind regarding unexpected layout changes. It would take in two git branch identifiers as parameters and automate screenshots of all the pages at 400px, 900px, and 1200px viewport widths, and produce output images indicating what, if anything, has changed on the pages for manual verification.

[ebidel]

In #350....

• The load animation now fires io-logo-animation-done when it's done.
• There's also an animation for when the page transition is al done.

[jeffposnick]

I wanted to track some individual aspects of this:

• Support for Chrome.
• Support for Firefox.
• Support for Safari.
• Support for Internet Explorer (10, 11).
• Support for comparing full pages (not just initial viewport).
• Support for comparing arbitrary branches via command line parameters.
• Support for using git stash to compare uncommitted changes with master, and then restore them.
• Support for running "in the cloud".
• Support for triggering screenshots following specific interactions, like button presses.
• Support for running against a subset of pages and resolutions via command line parameters.