typescript sync generator - cache race conditions for tsc-XXX.hash files

[cogwirrel]

Current Behavior

The nx sync command can intermittently fail on projects which make use of the nx typescript plugin, since multiple parallel writes can corrupt the tsc-XXXX.hash files.

For example:

 NX   Failed to process project graph.

     - Error: ValueExpected in /tmp/e2e-KmT87d/npm/e2e-test/.nx/workspace-data/tsc-8717437280021111401.hash at 1:1
  > 1 | 
      | ^
  
      at parseJson (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/utils/json.js:29:15)
      at readJsonFile (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/utils/fileutils.js:32:37)
      at readTargetsCache (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/@nx/js/src/plugins/typescript/plugin.js:18:37)
      at exports.createNodesV2 (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/@nx/js/src/plugins/typescript/plugin.js:38:30)
      at LoadedNxPlugin.createNodes (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/project-graph/plugins/loaded-nx-plugin.js:26:65)
      at LoadedNxPlugin.createNodes.<computed> (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/project-graph/plugins/loaded-nx-plugin.js:36:34)
      at createNodes (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/project-graph/plugins/isolation/plugin-worker.js:72:63)
      at consumeMessage (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/project-graph/plugins/isolation/messaging.js:42:32)
      at /tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/project-graph/plugins/isolation/plugin-worker.js:30:47
      at Socket.<anonymous> (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/utils/consume-messages-from-socket.js:13:17)

I believe this is an issue where there are multiple entries for the @nx/js/typescript plugin in nx.json (with the same options but different include/exclude patterns), and when plugins run in a worker (ie CI=true) they can conflict.

Expected Behavior

No corrupted cached files.

GitHub Repo

https://github.com/cogwirrel/nx-ts-plugin-race-repro

Steps to Reproduce

In the above repo, pnpm i to install then run CI=true pnpm nx sync

I've added enough packages and plugin configs to consistently hit the bug on my machine.

Nx Report

$ pnpm nx report

 NX   Report complete - copy this into the issue template

Node           : 20.18.0
OS             : darwin-arm64
Native Target  : aarch64-macos
pnpm           : 10.4.1

nx             : 20.4.6
@nx/js         : 20.4.6
@nx/workspace  : 20.4.6
@nx/devkit     : 20.4.6
typescript     : 5.7.3
---------------------------------------
Registered Plugins:
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
@nx/js/typescript
---------------------------------------
⚠️ Unable to construct project graph.
Failed to process project graph.
     - Error: EndOfFileExpected in /Users/jacsteve/personal/ts-plugin-race/.nx/workspace-data/tsc-13991073429437061962.hash at 2706:4
    2704 |     }
    2705 |   }
  > 2706 | }  "7956094541842478791_packages/p21/tsconfig.json": {
         |    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    2707 |     "targets": {
    2708 |       "typecheck": {
    2709 |         "dependsOn": [
  ...

Failure Logs

NX   Failed to process project graph.

     - Error: ValueExpected in /tmp/e2e-KmT87d/npm/e2e-test/.nx/workspace-data/tsc-8717437280021111401.hash at 1:1
  > 1 | 
      | ^
  
      at parseJson (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/utils/json.js:29:15)
      at readJsonFile (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/utils/fileutils.js:32:37)
      at readTargetsCache (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/@nx/js/src/plugins/typescript/plugin.js:18:37)
      at exports.createNodesV2 (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/@nx/js/src/plugins/typescript/plugin.js:38:30)
      at LoadedNxPlugin.createNodes (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/project-graph/plugins/loaded-nx-plugin.js:26:65)
      at LoadedNxPlugin.createNodes.<computed> (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/project-graph/plugins/loaded-nx-plugin.js:36:34)
      at createNodes (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/project-graph/plugins/isolation/plugin-worker.js:72:63)
      at consumeMessage (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/project-graph/plugins/isolation/messaging.js:42:32)
      at /tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/project-graph/plugins/isolation/plugin-worker.js:30:47
      at Socket.<anonymous> (/tmp/e2e-KmT87d/npm/e2e-test/node_modules/nx/src/utils/consume-messages-from-socket.js:13:17)

Package Manager Version

pnpm 10.4.1

Operating System

• macOS
• Linux
• Windows
• Other (Please specify)

Additional Information

I found an old PR here where some writes to the cache were made more atomic: #12920

However the typescript plugin just writes without any locking etc:

nx/packages/js/src/plugins/typescript/plugin.ts

Lines 107 to 109 in 1407152

	function writeToCache<T extends object>(cachePath: string, data: T) {
	writeJsonFile(cachePath, data, { spaces: 0 });
	}

Another possible solution might be to include the include/exclude in the options hash:

nx/packages/js/src/plugins/typescript/plugin.ts

Lines 125 to 129 in 1407152

	const optionsHash = hashObject(options);
	const targetsCachePath = join(
	workspaceDataDirectory,
	`tsc-${optionsHash}.hash`
	);