Skip to content

Commit 3492ff6

Browse files
authored
feat: static variable analysis (#770)
* feat: static variable analysis * Accept any iterable from `children`, `arguments`, etc. * Test analysis of standard tags * Use `TagToken.tokenizer` instead of creating a new one * Test analysis of netsted tags * Group variables by their root value * Test analysis of nested globals and locals * Analyze included and rendered templates WIP * Use existing tokenizer when constructing `Hash` * Improve test coverage * Analyze variables from `layout` and `block` tags * Test analysis of Jekyll style includes * Handle variables that start with a nested variable * Async analysis * Test non-standard tag end to end * Implement convenience analysis methods on the `Liquid` class * More analysis convenience methods * Accept string or template array * Draft static analysis docs * Deduplicate variables names * Fix isolated scope global variable map * Coerce variables to strings instead of extending String * Private map instead of extending Map * Fix e2e test * Tentatively implement analysis of aliased variables * Fix nested variable segments array * Update docs sidebar
1 parent 35a8442 commit 3492ff6

38 files changed

+2520
-37
lines changed

docs/source/_data/sidebar.yml

+1
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ tutorials:
2020
operators: operators.html
2121
truth: truthy-and-falsy.html
2222
dos: dos.html
23+
static_analysis: static-analysis.html
2324
miscellaneous:
2425
migration9: migrate-to-9.html
2526
changelog: changelog.html
+286
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,286 @@
1+
---
2+
title: Static Template Analysis
3+
---
4+
5+
{% since %}v10.20.0{% endsince %}
6+
7+
{% note info Sync and Async %}
8+
There are synchronous and asynchronous versions of each of the methods demonstrated on this page. See the [Liquid API](liquid-api) for a complete reference.
9+
{% endnote %}
10+
11+
## Variables
12+
13+
Retrieve the names of variables used in a template with `Liquid.variables(template)`. It returns an array of strings, one string for each distinct variable, without its properties.
14+
15+
```javascript
16+
import { Liquid } from 'liquidjs'
17+
18+
const engine = new Liquid()
19+
20+
const template = engine.parse(`\
21+
<p>
22+
{% assign title = user.title | capitalize %}
23+
{{ title }} {{ user.first_name | default: user.name }} {{ user.last_name }}
24+
{% if user.address %}
25+
{{ user.address.line1 }}
26+
{% else %}
27+
{{ user.email_addresses[0] }}
28+
{% for email in user.email_addresses %}
29+
- {{ email }}
30+
{% endfor %}
31+
{% endif %}
32+
{{ a[b.c].d }}
33+
<p>
34+
`)
35+
36+
console.log(engine.variablesSync(template))
37+
```
38+
39+
**Output**
40+
41+
```javascript
42+
[ 'user', 'title', 'email', 'a', 'b' ]
43+
```
44+
45+
Notice that variables from tag and filter arguments are included, as well as nested variables like `b` in the example. Alternatively, use `Liquid.fullVariables(template)` to get a list of variables including their properties as strings.
46+
47+
```javascript
48+
// continued from above
49+
engine.fullVariables(template).then(console.log)
50+
```
51+
52+
**Output**
53+
54+
```javascript
55+
[
56+
'user.title',
57+
'user.first_name',
58+
'user.name',
59+
'user.last_name',
60+
'user.address',
61+
'user.address.line1',
62+
'user.email_addresses[0]',
63+
'user.email_addresses',
64+
'title',
65+
'email',
66+
'a[b.c].d',
67+
'b.c'
68+
]
69+
```
70+
71+
Or use `Liquid.variableSegments(template)` to get an array of strings and numbers that make up each variable's path.
72+
73+
```javascript
74+
// continued from above
75+
engine.variableSegments(template).then(console.log)
76+
```
77+
78+
**Output**
79+
80+
```javascript
81+
[
82+
[ 'user', 'title' ],
83+
[ 'user', 'first_name' ],
84+
[ 'user', 'name' ],
85+
[ 'user', 'last_name' ],
86+
[ 'user', 'address' ],
87+
[ 'user', 'address', 'line1' ],
88+
[ 'user', 'email_addresses', 0 ],
89+
[ 'user', 'email_addresses' ],
90+
[ 'title' ],
91+
[ 'email' ],
92+
[ 'a', [ 'b', 'c' ], 'd' ],
93+
[ 'b', 'c' ]
94+
]
95+
```
96+
97+
### Global Variables
98+
99+
Notice, in the examples above, that `title` and `email` are included in the results. Often you'll want to exclude names that are in scope from `{% assign %}` tags, and temporary variables like those introduced by a `{% for %}` tag.
100+
101+
To get names that are expected to be _global_, that is, provided by application developers rather than template authors, use the `globalVariables`, `globalFullVariables` or `globalVariableSegments` methods (or their synchronous equivalents) of a `Liquid` class instance.
102+
103+
```javascript
104+
// continued from above
105+
engine.globalVariableSegments(template).then(console.log)
106+
```
107+
108+
**Output**
109+
110+
```javascript
111+
[
112+
[ 'user', 'title' ],
113+
[ 'user', 'first_name' ],
114+
[ 'user', 'name' ],
115+
[ 'user', 'last_name' ],
116+
[ 'user', 'address' ],
117+
[ 'user', 'address', 'line1' ],
118+
[ 'user', 'email_addresses', 0 ],
119+
[ 'user', 'email_addresses' ],
120+
[ 'a', [ 'b', 'c' ], 'd' ],
121+
[ 'b', 'c' ]
122+
]
123+
```
124+
125+
### Partial Templates
126+
127+
By default, LiquidJS will try to load and analyze any included and rendered templates too.
128+
129+
```javascript
130+
import { Liquid } from 'liquidjs'
131+
132+
const footer = `\
133+
<footer>
134+
<p>&copy; {{ "now" | date: "%Y" }} {{ site_name }}</p>
135+
<p>{{ site_description }}</p>
136+
</footer>`
137+
138+
const engine = new Liquid({ templates: { footer } })
139+
140+
const template = engine.parse(`\
141+
<body>
142+
<h1>Hi, {{ you | default: 'World' }}!</h1>
143+
{% assign some = 'thing' %}
144+
{% include 'footer' %}
145+
</body>
146+
`)
147+
148+
engine.globalVariables(template).then(console.log)
149+
```
150+
151+
**Output**
152+
153+
```javascript
154+
[ 'you', 'site_name', 'site_description' ]
155+
```
156+
157+
You can disable analysis of partial templates by setting the `partials` options to `false`.
158+
159+
```javascript
160+
// continue from above
161+
engine.globalVariables(template, { partials: false }).then(console.log)
162+
```
163+
164+
**Output**
165+
166+
```javascript
167+
[ 'you' ]
168+
```
169+
170+
If an `{% include %}` tag uses a dynamic template name (one that can't be determined without rendering the template) it will be ignored, even if `partials` is set to `true`.
171+
172+
### Advanced Usage
173+
174+
The examples so far all use convenience methods of the `Liquid` class, intended to cover the most common use cases. Instead, you can work with [analysis results](static-analysis-interface) directly, which expose the row, column and file name for every occurrence of each variable.
175+
176+
This is an example of an object returned from `Liquid.analyze()`, passing it the template from the [Partial Template](#partial-templates) section above.
177+
178+
```javascript
179+
{
180+
variables: {
181+
you: [
182+
[String (Variable): 'you'] {
183+
segments: [ 'you' ],
184+
location: { row: 2, col: 14, file: undefined }
185+
}
186+
],
187+
site_name: [
188+
[String (Variable): 'site_name'] {
189+
segments: [ 'site_name' ],
190+
location: { row: 2, col: 41, file: 'footer' }
191+
}
192+
],
193+
site_description: [
194+
[String (Variable): 'site_description'] {
195+
segments: [ 'site_description' ],
196+
location: { row: 3, col: 9, file: 'footer' }
197+
}
198+
]
199+
},
200+
globals: {
201+
you: [
202+
[String (Variable): 'you'] {
203+
segments: [ 'you' ],
204+
location: { row: 2, col: 14, file: undefined }
205+
}
206+
],
207+
site_name: [
208+
[String (Variable): 'site_name'] {
209+
segments: [ 'site_name' ],
210+
location: { row: 2, col: 41, file: 'footer' }
211+
}
212+
],
213+
site_description: [
214+
[String (Variable): 'site_description'] {
215+
segments: [ 'site_description' ],
216+
location: { row: 3, col: 9, file: 'footer' }
217+
}
218+
]
219+
},
220+
locals: {
221+
some: [
222+
[String (Variable): 'some'] {
223+
segments: [ 'some' ],
224+
location: { row: 3, col: 13, file: undefined }
225+
}
226+
]
227+
}
228+
}
229+
```
230+
231+
### Analyzing Custom Tags
232+
233+
For static analysis to include results from custom tags, those tags must implement some additional methods defined on the [Template interface]( /api/interfaces/Template.html). LiquidJS will use the information returned from these methods to traverse the template and report variable usage.
234+
235+
Not all methods are required, depending in the kind of tag. If it's a block with a start tag, end tag and any amount of Liquid markup in between, it will need to implement the [`children()`](/api/interfaces/Template.html#children) method. `children()` is defined as a generator, so that we can use it in synchronous and asynchronous contexts, just like `render()`. It should return HTML content, output statements and tags that are child nodes of the current tag.
236+
237+
The [`blockScope()`](/api/interfaces/Template.html#blockScope) method is responsible for telling LiquidJS which names will be in scope for the duration of the tag's block. Some of these names could depend on the tag's arguments, and some will be fixed, like `forloop` from the `{% for %}` tag.
238+
239+
Whether a tag is an inline tag or a block tag, if it accepts arguments it should implement [`arguments()`](/api/interfaces/Template.html#arguments), which is responsible for returning the tag's arguments as a sequence of [`Value`](/api/classes/Value.html) instances or tokens of type [`ValueToken`](/api/types/ValueToken.html).
240+
241+
This example demonstrates these methods for a block tag. See LiquidJS's [built-in tags](built-in) for more examples.
242+
243+
```javascript
244+
import { Liquid, Tag, Hash } from 'liquidjs'
245+
246+
class ExampleTag extends Tag {
247+
args
248+
templates
249+
250+
constructor (token, remainTokens, liquid, parser) {
251+
super(token, remainTokens, liquid)
252+
this.args = new Hash(token.tokenizer)
253+
this.templates = []
254+
255+
const stream = parser.parseStream(remainTokens)
256+
.on('tag:endexample', () => { stream.stop() })
257+
.on('template', (tpl) => this.templates.push(tpl))
258+
.on('end', () => { throw new Error(`tag ${token.getText()} not closed`) })
259+
260+
stream.start()
261+
}
262+
263+
* render (ctx, emitter) {
264+
const scope = (yield this.args.render(ctx))
265+
ctx.push(scope)
266+
yield this.liquid.renderer.renderTemplates(this.templates, ctx, emitter)
267+
ctx.pop()
268+
}
269+
270+
* children () {
271+
return this.templates
272+
}
273+
274+
* arguments () {
275+
yield * Object.values(this.args.hash).filter((el) => el !== undefined)
276+
}
277+
278+
blockScope () {
279+
return Object.keys(this.args.hash)
280+
}
281+
}
282+
```
283+
284+
[liquid-api]: /api/classes/Liquid.html
285+
[static-analysis-interface]: /api/interfaces/StaticAnalysis.html
286+
[built-in]: https://github.com/harttle/liquidjs/tree/master/src/tags

docs/themes/navy/languages/en.yml

+1
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ sidebar:
5252
operators: Operators
5353
truth: Truthy and Falsy
5454
dos: DoS
55+
static_analysis: Static Analysis
5556

5657
miscellaneous: Miscellaneous
5758
migration9: 'Migrate to LiquidJS 9'

src/index.ts

+2-2
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,9 @@ export { Drop } from './drop'
66
export { Emitter } from './emitters'
77
export { defaultOperators, Operators, evalToken, evalQuotedToken, Expression, isFalsy, isTruthy } from './render'
88
export { Context, Scope } from './context'
9-
export { Value, Hash, Template, FilterImplOptions, Tag, Filter, Output } from './template'
9+
export { Value, Hash, Template, FilterImplOptions, Tag, Filter, Output, Variable, VariableLocation, VariableSegments, Variables, StaticAnalysis, StaticAnalysisOptions, analyze, analyzeSync, Arguments, PartialScope } from './template'
1010
export { Token, TopLevelToken, TagToken, ValueToken } from './tokens'
11-
export { TokenKind, Tokenizer, ParseStream } from './parser'
11+
export { TokenKind, Tokenizer, ParseStream, Parser } from './parser'
1212
export { filters } from './filters'
1313
export * from './tags'
1414
export { defaultOptions, LiquidOptions } from './liquid-options'

0 commit comments

Comments
 (0)