feat: ras data library

[miguelpeixe]

All Submissions:

• Have you followed the Newspack Contributing guideline?
• Does your code follow the WordPress' coding standards and VIP Go coding standards?
• Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Changes proposed in this Pull Request:

A reader data library using localStorage. Along with the library implementation, this PR also tweaks the content gate's frontend metering strategy to use it.

Reader data

Gets and sets reader data using localStorage. Each item is a localStorage entry prefixed with np_reader_ and a stringified JSON value.

newspackRAS.push( ras => {
  const reader = ras.store.get( 'reader' ); // { "email": "john@doe.com", "authenticated": true }
} );

Updating a value emits a data event:

newspackRAS.push( ras => {
  ras.on( 'data', ( { detail: { key, value } } ) => {
    console.log( { key, value } );
  } );

  ras.store.set( 'isDonor', true ); // Logs { key: 'isDonor', value: true }
  ras.store.delete( 'isDonor' ); // Logs { key: 'isDonor', value: undefined }
} );

Data sync

By default, every data update will attempt to sync with the database as user meta. For anonymous sessions, an internal unsynced array will store the keys that are pending synchronization until there's an authenticated session that allows the data push.

Items can also skip synchronization on .set():

newspackRAS.push( ras => {
  ras.store.set( 'foo', 'bar', false ); // Will not sync
  ras.store.set( 'foo', 'bar', true ); // Syncs
  ras.store.set( 'foo', 'bar' ); // Syncs
} );

For authenticated readers, synced items will rehydrate the store on page load, which allows for systems that rely on the store to have consistent reader data across different sessions.

Backend handling of synced data

Using the synchronization strategy, data can be originated and managed on the backend:

The following will update the reader localStorage store on the next page load:

$user_id = 123;
$key     = 'isDonor';
$value   = true;
\Newspack\Reader_Data::update_item( $user_id, $key, wp_json_encode( $value ) );

Note that the value should always be stringified JSON, so make sure to always use wp_json_encode(). Even for strings so 'test' becomes the json compliant '"test"'

You can also delete and get store items:

$user_id = 123;
$key     = 'isDonor';

$value = \Newspack\Reader_Data::get_data( $user_id, $key );

\Newspack\Reader_Data::delete_item( $user_id, $key );

Reader activity

The store supports "collections", which will be used for managing reader activity with a schema that integrates with data events.

The collection can hold up to 1,000 items and each lives for 30 days.

The following code:

newspackRAS.push( [ 'donate', { 'amount': 5 } ] );

1. Updates the np_reader_activity localStorage entry to include:

   { "action": "donate", "data": { "amount": 5 }, "timestamp": 123456789 }
2. Emits an activity event:

   newspackRAS.push( ras => {
     ras.on( 'activity', ( { detail: { action, data, timestamp } ) => {
       console.log( 'Reader activity', { action, data, timestamp } );
     }
   } );

Activities can be fetched with:

newspackRAS.push( ras => {
  const donations = ras.getActivities( 'donate' );
  console.log( 'Donations', donations );

  const allActivities = ras.getActivities();
  console.log( 'All activities', allActivities );
} ); 

The next step is to have proper integration with data events: a new activity can dispatch a data event on the backend and a data event can dispatch a reader activity to the frontend.

How to test the changes in this Pull Request:

1. Confirm the new tests have good coverage
2. In a new session authenticate with a reader account
3. Open devtools and confirm the localStorage contains the reader entry
4. Using the devtools console, update a data item: newspackReaderActivation.store.set( 'foo', 'bar' );
5. Confirm the value is added to the store and also quickly added and removed from np_reader__unsynced (the queue polling strategy picked it up the synced it to the backend)
6. Visit your database and confirm the user meta is added as newspack_reader_data_item_foo
7. Manually remove the item from localStorage, refresh the page, and confirm it's rehydrated back
8. Remove using the API: newspackReaderActivation.store.delete( 'foo' );
9. Refresh the page and confirm the value is not added and it has been removed as user meta
10. Start a fresh anonymous session and update several values:

    newspackReaderActivation.store.set( 'foo', 'baz' );
    newspackReaderActivation.store.set( 'boolean', true );
    newspackReaderActivation.store.set( 'array', [ 1, 2, 3 ] );
    
11. Confirm the values have been added to localStorage and the np_reader__unsynced item contains [ 'foo', 'boolean', 'array' ]
12. Authenticate to that same reader account and navigate to any page (so the API token is available)
13. Confirm the unsynced items are cleared and the user meta is updated with the stringified JSON values
14. foo user meta value should update from bar to baz
15. Test pushing an activity: newspackRAS.push( [ 'test-activity', { foo: 'bar' } ] );
16. Confirm the np_reader_activity entry is updated to include the activity with a timestamp property

Other information:

• Have you added an explanation of what your changes do and why you'd like us to include them?
• Have you written new tests for your changes, as applicable?
• Have you successfully ran tests with your changes locally?

[leogermani]

I've played around a little and everything works as expected. I have one maybe-blocker comment and another non-blocking suggestion.

The maybe blocker is that this won't work well in Multi-sites running on subfolders, because the local storage is tied to the root of the domain. I know this is not a pressing requirement for us, but maybe easier to address it now than in the future if the need arises.

And the suggestions for the api:

• what do you think of renaming dispatch to dispatchActivity to make it easier to understand what it does
• also, why not create a similar wrapper for store.set and store.delete? also with specific names like setData for example.
• Then we can maybe reconsider if we need to include the store to the the main object, and leave it only for internal usage. wdyt?

[miguelpeixe]

The maybe blocker is that this won't work well in Multi-sites running on subfolders, because the local storage is tied to the root of the domain. I know this is not a pressing requirement for us, but maybe easier to address it now than in the future if the need arises.

Nice catch! We can make the store prefix a filterable localized value, so each site can have its own prefix and store preserved. WDYT? It's already an isolated config value, so making that change is not a big lift.

what do you think of renaming dispatch to dispatchActivity to make it easier to understand what it does

Good call! Done in 13e0672

also, why not create a similar wrapper for store.set and store.delete? also with specific names like setData for example.
Then we can maybe reconsider if we need to include the store to the the main object, and leave it only for internal usage. wdyt?

I'm not sure, the wrapper for "activity" exists because it's a specific use of the store's "collections". I don't see a reason to justify wrapping the store's get/set/delete.

[miguelpeixe]

@leogermani implemented the filterable store prefix in a26ca63

[leogermani]

Cool,

I think that instead of filtering the prefix, you could simply inform the blog id with get_current_blog_id(). That should do it.

I don't see a reason to justify wrapping the store's get/set/delete.

The only reason I see is to simplify the API. So the developer only needs to interact with one thing and the methods have self-explanatory names setData instead of store.set for example.

[miguelpeixe]

I think that instead of filtering the prefix, you could simply inform the blog id with get_current_blog_id(). That should do it.

Hub & Spoke is not MS, is it?

The only reason I see is to simplify the API. So the developer only needs to interact with one thing and the methods have self-explanatory names setData instead of store.set for example.

I actually like being able to pass the store object. You can see how it's implemented in metering.js.

[leogermani]

Hub & Spoke is not MS, is it?

It's not.

I actually like being able to pass the store object. You can see how it's implemented in metering.js.

Ok! If you think that's a good approach I'm totally fine with it! I can help writing some docs later on as this matures

[miguelpeixe]

Thanks!

I added the get_current_blog_id() in 71175ab. It's good to have native support for MS!

[matticbot]

🎉 This PR is included in version 2.1.0-alpha.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

[matticbot]

🎉 This PR is included in version 2.1.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀