-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add cattrs to unstructure/structure and allow (de)serialization with json or similar #181
Conversation
this way we don't get an extra {"data": {"foo": ...}}, but just {"foo": ...}, as 'data' is the only attribute/key for DataStore anyway. reuse bytes encodings logic in misc, move lib specific private methods to lib.py module to avoid circular import. Check if converter already supports natively bytes, in which case avoid the encoding/decoding to base85 string
hm gotta figure out py37 compat (next week) |
and it's a tiny bit faster than checking any(isinstance(a.type, str)...)
@madig I think this is ready for another review pass, at your leisure. Thanks |
…ut it the cattr.preconf.orjson converter does stuff we don't need/use: - it serializes bytes as base85 (but we use base64 anyway as more widely compatible) - it coerces mapping keys to str (but all our mappings use str anyway) - has handler for datetimes (but we don't use any) - coerces sets to lists (but we use no sets)
By the way, all this is generic and not tied to json or any specific serialization format. Here's how it'd work with msgpack for example. The only difference below is, instead of using ufoLib2's default_converter, we create a new GenConverter, register our custom structure/unstructure hooks, and allow_bytes=True (for msgpack can encode those natively, unlike the our default_converter which is geared toward json and we need to encode bytes as base64). import io
import cattr
import ufoLib2
import ufoLib2.converters
import msgpack
conv = cattr.GenConverter(omit_if_default=True)
ufoLib2.converters.register_hooks(conv, allow_bytes=True)
font = ufoLib2.Font.open("...ufo")
buf = io.BytesIO()
msgpack.pack(conv.unstructure(font), buf)
buf.seek(0)
font2 = conv.structure(msgpack.unpack(buf), Font)
assert font == font2 |
after write() the _reader is set to None, if one calls unlazify after that, they'd get an assertion error.
avoid repeating the word 'default'..
AFAIK, this is the most unused of all glyph attributes, it's also marked optional in the UFO spec. fontTools.ufoLib happily skips it when it is None. Having all Glyph objects carry an unused image object seems like a waste. If one wants to use images, one can set one.
…mit_if_default' global option A value equal to None is pratically equal to it not being there, so there's no point in serializing those even if Converter has omit_if_default=False. Add tests for different omit_if_default values
The LayerSet does not seem to get unstructured? import json
import ufoLib2
import ufoLib2.converters
u = ufoLib2.Font.open("tests/data/MutatorSansBoldCondensed.ufo/")
d = ufoLib2.converters.default_converter.unstructure(u)
j = json.dumps(d)
|
that is very weird, the tests pass, and I added more that explicitly test json (de)serialization. It must be a local setup issue of yours. |
It was a fluke, sorry for the noise. Something different: How should other libraries handle serialized dumps? Consider e.g. intermediate build steps in a pipeline, where currently a Designspace might reference normal UFOs but should reference JSON dumps instead. Should e.g. designspaceLib be made aware of the new format or should this rather be done elsewhere? |
we are not designing any "new format" here. This is not tied to json particularly either. Clients of ufoLib2 can do whatever they like with this, this is just enabling them to more easily (de)construct ufoLib2 classes to something that can be (de)serialized. We can figure those thing out elsewhere |
Right, so we use it externally to the established libraries, where tooling has to deliberately make use of it. |
This reverts commit 2ab50c0.
This is to continue supporting font.kerning = {(a, b): -10, ...} even after we typed Font.kerning as Kerning dict subclass to customize (un)structuring via cattrs in #181
this allows to do things like
Still WIP, as I need to add tests.Added tests