Layout Instability API

[skobes-chromium]

Request for Mozilla Position on an Emerging Web Specification

• Specification Title: Layout Instability API
• Specification or proposal URL: https://wicg.github.io/layout-instability/
• Caniuse.com URL (optional):
• Bugzilla URL (optional):
• Mozillians who can provide input (optional):

Other information

Explainer: https://github.com/WICG/layout-instability/blob/master/README.md
TAG review: w3ctag/design-reviews#393

[dbaron]

cc @dholbert @heycam @emilio @mstange

[dholbert]

The feature seems reasonable to me, in general.

I filed some spec issues and posted some pull requests, mostly for editorial stuff. My spec-issues included two more theoretical concerns that perhaps need some consideration (to be discussed on the issues themselves rather than here, probably):

• Is visibilitychange really the right signal for when to aggregate layout shift? WICG/layout-instability#50 RE whether the spec's suggested aggregation-time (at visibilitychange) is in fact a good idea.
• Define how much data should be buffered (also: does 'buffered' impose a substantial cost on the whole web here?) WICG/layout-instability#52 RE what buffered should do (and whether it's problematic for it to do anything at all)

[dholbert]

I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1651528 on a hypothetical/eventual Firefox implementation of this, BTW.

[dholbert]

I filed one more spec issue on something that could influence our (hypothetical) implementation strategy:
WICG/layout-instability#61

[dholbert]

I've posted a pull request #425 to mark this spec as "worth prototyping".

There are some pieces that still need clarity, which I've filed spec issues for[1], and the editors have been responsive on those & hopefully we'll have (or can propose) answers when prototyping.

Assuming reasonable resolutions to those issues, the only piece I'm unsure about is whether this can be implemented efficiently such that it's ~zero-cost for sites that don't use the feature, given the requirements of buffered[2]. That's something we'll know more about after prototyping the feature, so I don't think that should block this assessment. But, that issue could hypothetically change our assessment down the line, if prototyping suggests that the feature imposes a measurable cost for our rendering engine. (The cost may vary between rendering engines, too; I believe Blink's LayoutNG might cache the results of earlier layouts, or at least that was part of the design at one point -- that might make this feature "free" for them, in a way that it is not free for Firefox. On the other hand, maybe the cost will still be negligible; I'm not sure.)

[1] Things that need clarity:
WICG/layout-instability#66
WICG/layout-instability#65
WICG/layout-instability#62
WICG/layout-instability#61
WICG/layout-instability#57
WICG/layout-instability#53

[2] Note about cost: WICG/layout-instability#52 (comment)

[emilio]

I'm a bit late to this but I'm a bit concerned about both the complexity vs. value of it...

I think there are some legit reasons for shifting stuff on the screen (like animations and such). From a quick glance at the spec there's nothing that would prevent something like a bouncing animation from reporting a lot of layout shift right?

Just something bouncing on screen is not a great example, but something like a parallax effect where stuff moves into view when you scroll seems like a very common thing to do in landing pages and so on, and it seems that'd get counted as layout shift (specially since scroll events are intentionally not excluding inputs, looks like).

[emilio]

I guess the intention of the 3 pixel shift is to mitigate some of it? But still fast animations would report a lot of shift I think.

[dholbert]

@emilio does this snippet of the explainer address your concerns? (see the sections with titles "Shifting Nodes" and "Transform Changes")

(Perhaps those explainer sections need to be better captured in the spec... I am noticing that e.g. the word "transform" never appears in the spec right now.)

[emilio]

@dholbert it does, to some extent, though:

• There's no reference to transform on the spec... Probably should be more explicit if all the computations are supposed to be done without transforms applied? Also, does this mean that we need to keep around the scrollable overflow area but without transforms?

• Scroll input is intentionally out of the "recent input" stuff:

Excluding inputs generally include mousedown, keydown, and pointerdown. However, an event whose only effect is to begin or update a fling or scroll gesture is not an excluding input.

[dholbert]

I've now filed WICG/layout-instability#66 on the transform stuff and added it to my list above.

RE scroll input, I think the spec's snippet about "an Element P" (in section 2.2) is meant to be what excludes scrolling from being considered as layout shift. Though if the site uses additional transform effects for this sort of scrolling, then I think you're correct that it might be counted as shift, per WICG/layout-instability#66 (comment) (but that might be a spec bug & not the spec editors' intended outcome).

[MatsPalmgren]

I'm concerned about both the complexity and performance penalty vs. value of this feature. I'm also concerned about the privacy implications of the data collection this enables. If I understand correctly, this API is exposed to non-privileged web content, but that seems unnecessary if we only want to visualize these metrics in Firefox devtools for example.

I'm fine with standardizing private (not exposed to web-content) APIs that devtools can use to present comparable performance metrics to web developers across browsers, but I'm opposed to adding more ways for mass-surveillance companies like Google and Facebook to collect privacy-sensitive data.

[MatsPalmgren]

I'm also concerned about the long term consequences for the web if performance metrics like this are incorporated into SEO scoring and such. It seems to me this would act as a forcing function that will lead to more centralization of resources (such as downloadable fonts), which I think is undesirable.

[Bas-moz]

I'm concerned about both the complexity and performance penalty vs. value of this feature. I'm also concerned about the privacy implications of the data collection this enables. If I understand correctly, this API is exposed to non-privileged web content, but that seems unnecessary if we only want to visualize these metrics in Firefox devtools for example.

I'm fine with standardizing private (not exposed to web-content) APIs that devtools can use to present comparable performance metrics to web developers across browsers, but I'm opposed to adding more ways for mass-surveillance companies like Google and Facebook to collect privacy-sensitive data.

The intent here is not meant for this API to mainly serve Dev Tools. The idea is that this API serves as a RUM that can be used by websites as telemetry to track the experience actual users are having on their site. See also Google's post here: https://web.dev/vitals/

I haven't yet thought of any 'real' privacy concerns with this particular feature but I'd be very interested in hearing them.

As to performance metrics being incorporated in SEO, I'm not particularly excited by that myself, but that is not really under our control. The best we can do in my opinion is make sure that if performance metrics are going to be incorporated in SEO, they're the best possible metrics we can make them.

[emilio]

I'm also concerned about the long term consequences for the web if performance metrics like this are incorporated into SEO scoring and such. It seems to me this would act as a forcing function that will lead to more centralization of resources (such as downloadable fonts), which I think is undesirable.

I chatted with mats and I think this argument is not valid because of cache partitioning, fwiw.

But I have some other comments, which make me lean towards Mats' argument... What is the benefit for a website having access to this data outside of analysis of page loads which can be performed on devtools / with different network speeds, etc? Is the idea to monitor this data across time to measure the "health" of the website?

I'm also concerned about legit user actions (like zooming in and out, resizing or such) which are going to cause a lot of shift aren't they? And they wouldn't be covered by the spec. How do you avoid these or help discard them?

[Bas-moz]

I'm also concerned about the long term consequences for the web if performance metrics like this are incorporated into SEO scoring and such. It seems to me this would act as a forcing function that will lead to more centralization of resources (such as downloadable fonts), which I think is undesirable.

I chatted with mats and I think this argument is not valid because of cache partitioning, fwiw.

But I have some other comments, which make me lean towards Mats' argument... What is the benefit for a website having access to this data outside of analysis of page loads which can be performed on devtools / with different network speeds, etc? Is the idea to monitor this data across time to measure the "health" of the website?

I'm also concerned about legit user actions (like zooming in and out, resizing or such) which are going to cause a lot of shift aren't they? And they wouldn't be covered by the spec. How do you avoid these or help discard them?

The reality of performance testing on the web shows us that what shifts you actually get are wildly, and I do mean wildly different on a per device/per user basis. Many things affect this like the order in which resources come in on a particular device (which isn't just a consequence of end-user network speed but many other factors). The speed at which the browser runs certain work and therefor the scheduling of things. You can design an impossibly large testing matrix to try and account for this, but the reality is that that is incredibly hard to do, even for large organizations, and possible more important, impossible to do for smaller content creators.

Therefore, yes, if you're a website trying to monitor this data for your users (across time as well as for a single point in time), local testing is not a great option.

If pageload were a much more deterministic process I might be more sympathetic to that argument, but both the data sites are getting from Google's API as the results of our own performance testing show us that it isn't. I certainly see value in exposing this as a RUM for content creators.

[Bas-moz]

I'm also concerned about legit user actions (like zooming in and out, resizing or such) which are going to cause a lot of shift aren't they? And they wouldn't be covered by the spec. How do you avoid these or help discard them?

If I understand you correctly I believe those concerns are meant to be addressed by the hadRecentInput/LastInputTime attributes. See also section 2.4 and 5.1 in the spec.

[emilio]

Those inputs aren't always accessible to the website usually, right? for example if you change zoom with your browser UI, the content tab has no information about any event...

[emilio]

Anyhow, it's not a huge deal, just something that I think at least should be better defined...

[emilio]

Ah, here's the sentence I missed on the spec:

An excluding input is [...], or any event which directly changes the size of the viewport.

That makes sense and I guess covers the two cases I thought about :)