Shim CSS Mixins in terms of CSS Custom Properties (#3587)

* check for native css variables

* keep custom properties, still unpack mixins

* make `updateStyles` (modulo @apply) work with native custom properties.
make `getComputedStyleValue` work with native custom properties.

* modifications

* sigh

* fix tests in firefox safari and chrome 49

* add back missing styleutil code in x-styling

* support all the fallback cases with nested parens

* fix rules with property and mixin definitions for native

* First draft of @apply shim

```css
--foo: {
  color: red;
  border: 2px solid black;
}
@apply --foo;
```
becomes
```css
--foo-color: red;
--foo-border: 2px solid black;
color: var(--foo-color);
border: var(--foo-border);
```

* step 3

* Process stylesheets in custom-styles

move apply-shim to lib
Fix some tests

* always add semicolon in flattenMixin

* reset order back to master

* step 4

* Temporary ordering fixup for native @apply shim.

* better step 5

* make custom-style support @apply shim.

* * make @apply regex match globally
* fix custom-style test's late registrations.

* Collect default property values between consumption and application

No more global for defaults
No need to process keyframe rules

* stupid stateful regexes

* Handle realiasing mixins (step 6)

Fix up shim var syntax `var(--a, --b)` to `var(--a, var(--b))`
Disable native variables in Safari 9.1 until https://bugs.webkit.org/show_bug.cgi?id=155782 is fixed

* Add more tests for apply shim

Test new `@apply --foo` syntax
Test aliasing mixins with var
Test fixing bad var syntax `var(--a, --b)` -> `var(--a, var(--b))`

* fix :root for apply shim

* add StyleTransformer smoke test.

* Move regexes from style-properties to style-util for easier sharing

Clean up dependencies in apply-shim
Remove lazyRegister hack

* Move check for CSS Custom Properties to settings lib

Enabled only if support is detected and `lazyRegister` setting is used.

Override with `Polymer = {useNativeCSSProperties: true}` before
polymer is loaded.

* add apply shim to smoke test

* actually let uses set useNativeCSSProperties false

* SCOPE_SELECTORS needs to work with built selectors for custom-style and elements

* No need to decorate styles for apply shim

* apply mixins and gather defaults incrementally

* Fix @apply consumption to incrementally process rule text

When >1 mixin is consumed, correctly use properties produced by last
mixin as fallbacks for current mixin

LOTS more comments, more descriptive comment names

Micro-opts

* MORE APPLY SHIM TESTING

Actually test apply shim (needs lazyRegister), and test with shadow dom

* Make sure @apply without parens works in property shim as expected

* Simpler tracking of mixin properties

Address naming feedback
Use property map in apply map to keep better track of properties to set
to 'initial' when mixins are redefined

* Support builds of CSS

PolymerBuild supports one property, css, which can be 'shady' or
'shadow'.

'shadow' is a "universal" build, which will work in shadydom or
shadowdom.

'shady' will only work in shadydom.

* patch a few spots that custom property shim needs to know about the build

* still need to transform the selectors if a shadow build was the source

* support tests with build to choose the right property

* make `importHref` avoid re-importing already loaded resources.

* One more spot a shady build will break custom property shim

:root will be calculated to `hostname > *.hostname` by the build

* A few more allowances for the builds

* mark elements and custom-styles as built, no globals

* use propertyDataFromStyles for :host and :root

Fixes #3610 `:host(tag-name)`

* make tests pass by hacking `propertyDataFromStyles`, needs refactoring.

* Search for properties in :host and :root rules at the same time

- Remove caching, does not apply with @media may invalidate
- custom-style will forward css-build state to the rules for property
lookup

* Don't expect shady built styles to be in head

Can't be sharded like that :(
Fix polyfill shadowdom and style-defaults with matchesSelector on <html> element
Fix url test with getPropertyValue when quotes are not present (the
string is escaped with '/')

* Fixes #3637. Normalizes attached timing between Shady and Shadow DOM under native Custom Elements.

* Fixes #3638. Avoid spamming document.head with already loaded link elements when importHref is called repeatedly with the same url.

* actually listen for the error event (unclear how to test error without server support)

* fix typo

* fix lint errors

* fix bad merge conflict

* Safari 9.1.1 is still busted, drop minor version check for AppleWebKit

* only apply statically shimmed styles if the element has cssText (this optimization can be made because elements always have a placeholder comment node to indicate style position).

* * slight optimization: cache cssBuild info on element.
* avoid decorating elements whenever there is a css build (of any type)

* revert dom scoping change and add clarifying comment about why this is needed when there is a non-shady build.

* formatting

* custom-style: avoid applying shimmed custom properties when native custom properties are in use.

* avoid shimming styles under shady dom when there is a shady css build.

* custom-style: when native custom properties are in use and no build is available, avoid applying shimmed properties and only update the style to reflect apply shim changes.

* avoid work in the presence of a css build.

* add test for :host(element-name)

* correct custom-style under shadow build when using shady and custom properties.

* fix tests to not rely on order in className

* Support custom-style with css-build status in HTMLImports polyfill

- Copy css-build status to cloned style in main document
- Move css-build status out of ast
  - Instead forward a reference to the style in the ast walker callback

* Clean up logic in custom-style _apply

More easy to see that no work happens if using a targeted css build with
native custom properties

* Loop over all property names ever used for a mixin

Add test to show that redefining a mixin works as expected

* Bail early if rule does not have properties

checking for native css custom properties is not needed in the property
shim anymore

* Comments.

* test that invalid @media rules do *not* apply via the custom properties shim.

* Use `_-_` as seperator for apply-shim created variables

Differentiates apply-shim variables more obviously from user variables
#3587 (comment)

* Fix custom-style test with new separator

* fix a few more tests for built styles

* Revert "Fixes #3637. Normalizes attached timing between Shady and Shadow DOM under native Custom Elements."

This reverts commit 9d272e0.

* [ci skip] PolymerBuild global has been removed

src/lib/apply-shim.html

@@ -0,0 +1,247 @@
+<!--
+@license
+Copyright (c) 2014 The Polymer Project Authors. All rights reserved.
+This code may only be used under the BSD style license found at http://polymer.github.io/LICENSE.txt
+The complete set of authors may be found at http://polymer.github.io/AUTHORS.txt
+The complete set of contributors may be found at http://polymer.github.io/CONTRIBUTORS.txt
+Code distributed by Google as part of the polymer project is also
+subject to an additional IP rights grant found at http://polymer.github.io/PATENTS.txt
+-->
+<link rel="import" href="style-util.html">
+<script>
+/**
+ * The apply shim simulates the behavior of `@apply` proposed at
+ * https://tabatkins.github.io/specs/css-apply-rule/.
+ * The approach is to convert a property like this:
+ *
+ *    --foo: {color: red; background: blue;}
+ *
+ * to this:
+ *
+ *    --foo_-_color: red;
+ *    --foo_-_background: blue;
+ *
+ * Then where `@apply --foo` is used, that is converted to:
+ *
+ *    color: var(--foo_-_color);
+ *    background: var(--foo_-_background);
+ *
+ * This approach generally works but there are some issues and limitations.
+ * Consider, for example, that somewhere *between* where `--foo` is set and used,
+ * another element sets it to:
+ *
+ *    --foo: { border: 2px solid red; }
+ *
+ * We must now ensure that the color and background from the previous setting
+ * do not apply. This is accomplished by changing the property set to this:
+ *
+ *    --foo_-_border: 2px solid red;
+ *    --foo_-_color: initial;
+ *    --foo_-_background: initial;
+ *
+ * This works but introduces one new issue.
+ * Consider this setup at the point where the `@apply` is used:
+ *
+ *    background: orange;
+ *    @apply --foo;
+ *
+ * In this case the background will be unset (initial) rather than the desired
+ * `orange`. We address this by altering the property set to use a fallback
+ * value like this:
+ *
+ *    color: var(--foo_-_color);
+ *    background: var(--foo_-_background, orange);
+ *    border: var(--foo_-_border);
+ *
+ * Note that the default is retained in the property set and the `background` is
+ * the desired `orange`. This leads us to a limitation.
+ *
+ * Limitation 1:
+
+ * Only properties in the rule where the `@apply`
+ * is used are considered as default values.
+ * If another rule matches the element and sets `background` with
+ * less specificity than the rule in which `@apply` appears,
+ * the `background` will not be set.
+ *
+ * Limitation 2:
+ *
+ * When using Polymer's `updateStyles` api, new properties may not be set for
+ * `@apply` properties.
+
+*/
+Polymer.ApplyShim = (function(){
+  'use strict';
+
+  var styleUtil = Polymer.StyleUtil;
+
+  var MIXIN_MATCH = styleUtil.rx.MIXIN_MATCH;
+  var VAR_ASSIGN = styleUtil.rx.VAR_ASSIGN;
+  var VAR_MATCH = styleUtil.rx.VAR_MATCH;
+  var APPLY_NAME_CLEAN = /;\s*/m;
+
+  // separator used between mixin-name and mixin-property-name when producing properties
+  // NOTE: plain '-' may cause collisions in user styles
+  var MIXIN_VAR_SEP = '_-_';
+
+  // map of mixin to property names
+  // --foo: {border: 2px} -> (--foo, ['border'])
+  var mixinMap = {};
+
+  function mapSet(name, prop) {
+    name = name.trim();
+    mixinMap[name] = prop;
+  }
+
+  function mapGet(name) {
+    name = name.trim();
+    return mixinMap[name];
+  }
+
+  // "parse" a mixin definition into a map of properties and values
+  // cssTextToMap('border: 2px solid black') -> ('border', '2px solid black')
+  function cssTextToMap(text) {
+    var props = text.split(';');
+    var out = {};
+    for (var i = 0, p, sp; i < props.length; i++) {
+      p = props[i];
+      if (p) {
+        sp = p.split(':');
+        // ignore lines that aren't definitions like @media
+        if (sp.length > 1) {
+          // some properties may have ':' in the value, like data urls
+          out[sp[0].trim()] = sp.slice(1).join(':');
+        }
+      }
+    }
+    return out;
+  }
+
+  function produceCssProperties(matchText, propertyName, valueProperty, valueMixin) {
+    // handle case where property value is a mixin
+    if (valueProperty) {
+      VAR_MATCH.lastIndex = 0;
+      var m = VAR_MATCH.exec(valueProperty);
+      if (m) {
+        var value = m[2];
+        if (mapGet(value)){
+          valueMixin = '@apply ' + value + ';';
+        }
+      }
+    }
+    if (!valueMixin) {
+      return matchText;
+    }
+    var mixinAsProperties = consumeCssProperties(valueMixin);
+    var prefix = matchText.slice(0, matchText.indexOf('--'));
+    var mixinValues = cssTextToMap(mixinAsProperties);
+    var oldProperties = mapGet(propertyName);
+    var combinedProps = mixinValues;
+    if (oldProperties) {
+      // NOTE: since we use mixin, the map of properties is updated here
+      // and this is what we want.
+      combinedProps = Polymer.Base.mixin(oldProperties, mixinValues);
+    } else {
+      mapSet(propertyName, combinedProps);
+    }
+    var out = [];
+    var p, v;
+    // set variables defined by current mixin
+    for (p in combinedProps) {
+      v = mixinValues[p];
+      // if property not defined by current mixin, set initial
+      if (v === undefined) {
+        v = 'initial';
+      }
+      out.push(propertyName + MIXIN_VAR_SEP + p + ': ' + v);
+    }
+    return prefix + out.join('; ') + ';';
+  }
+
+  // fix shim'd var syntax
+  // var(--a, --b) -> var(--a, var(--b));
+  function fixVars(matchText, prefix, value, fallback) {
+    // if fallback doesn't exist, or isn't a broken variable, abort
+    if (!fallback || fallback.indexOf('--') !== 0) {
+      return matchText;
+    }
+    return [prefix, 'var(', value, ', var(', fallback, '));'].join('');
+  }
+
+  // produce variable consumption at the site of mixin consumption
+  // @apply --foo; -> for all props (${propname}: var(--foo_-_${propname}, ${fallback[propname]}}))
+  // Example:
+  // border: var(--foo_-_border); padding: var(--foo_-_padding, 2px)
+  function atApplyToCssProperties(mixinName, fallbacks) {
+    mixinName = mixinName.replace(APPLY_NAME_CLEAN, '');
+    var vars = [];
+    var mixinProperties = mapGet(mixinName);
+    if (mixinProperties) {
+      var p, parts, f;
+      for (p in mixinProperties) {
+        f = fallbacks && fallbacks[p];
+        parts = [p, ': var(', mixinName, MIXIN_VAR_SEP, p];
+        if (f) {
+          parts.push(',', f);
+        }
+        parts.push(')');
+        vars.push(parts.join(''));
+      }
+    }
+    return vars.join('; ');
+  }
+
+  // replace mixin consumption with variable consumption
+  function consumeCssProperties(text) {
+    var m;
+    // loop over text until all mixins with defintions have been applied
+    while((m = MIXIN_MATCH.exec(text))) {
+      var matchText = m[0];
+      var mixinName = m[1];
+      var idx = m.index;
+      // collect properties before apply to be "defaults" if mixin might override them
+      // match includes a "prefix", so find the start and end positions of @apply
+      var applyPos = idx + matchText.indexOf('@apply');
+      var afterApplyPos = idx + matchText.length;
+      // find props defined before this @apply
+      var textBeforeApply = text.slice(0, applyPos);
+      var textAfterApply = text.slice(afterApplyPos);
+      var defaults = cssTextToMap(textBeforeApply);
+      var replacement = atApplyToCssProperties(mixinName, defaults);
+      // use regex match position to replace mixin, keep linear processing time
+      text = [textBeforeApply, replacement, textAfterApply].join('');
+      // move regex search to _after_ replacement
+      MIXIN_MATCH.lastIndex = idx + replacement.length;
+    }
+    return text;
+  }
+
+  var ApplyShim = {
+    _map: mixinMap,
+    _separator: MIXIN_VAR_SEP,
+    transform: function(styles) {
+      styleUtil.forRulesInStyles(styles, this._boundTransformRule);
+    },
+    transformRule: function(rule) {
+      rule.cssText = this.transformCssText(rule.parsedCssText);
+      // :root was only used for variable assignment in property shim,
+      // but generates invalid selectors with real properties.
+      // replace with `:host > *`, which serves the same effect
+      if (rule.selector === ':root') {
+        rule.selector = ':host > *';
+      }
+    },
+    transformCssText: function(cssText) {
+      // fix shim variables
+      cssText = cssText.replace(VAR_MATCH, fixVars);
+      // produce variables
+      cssText = cssText.replace(VAR_ASSIGN, produceCssProperties);
+      // consume mixins
+      return consumeCssProperties(cssText);
+    }
+  };
+
+  ApplyShim._boundTransformRule = ApplyShim.transformRule.bind(ApplyShim);
+  return ApplyShim;
+})();
+</script>

src/lib/css-parse.html

@@ -112,7 +112,7 @@
       var cssText = '';
       if (node.cssText || node.rules) {
         var r$ = node.rules;
-        if (r$ && (preserveProperties || !this._hasMixinRules(r$))) {
+        if (r$ && !this._hasMixinRules(r$)) {
           for (var i=0, l=r$.length, r; (i<l) && (r=r$[i]); i++) {
             cssText = this.stringify(r, preserveProperties, cssText);
           }
@@ -175,7 +175,7 @@
       port: /@import[^;]*;/gim,
       customProp: /(?:^[^;\-\s}]+)?--[^;{}]*?:[^{};]*?(?:[;\n]|$)/gim,
       mixinProp:  /(?:^[^;\-\s}]+)?--[^;{}]*?:[^{};]*?{[^}]*?}(?:[;\n]|$)?/gim,
-      mixinApply: /@apply[\s]*\([^)]*?\)[\s]*(?:[;\n]|$)?/gim,
+      mixinApply: /@apply\s*\(?[^);]*\)?\s*(?:[;\n]|$)?/gim,
       varApply: /[^;:]*?:[^;]*?var\([^;]*\)(?:[;\n]|$)?/gim,
       keyframesRule: /^@[^\s]*keyframes/,
       multipleSpaces: /\s+/g