Decouple DSA from Fedora

[jcoyne]

• Stop relying on the Fedora produced ActiveMQ messages for indexing
  • Switch to using RabbitMQ
  • This also would affect the workflow-server too. We can possibly cut down on indexing by only indexing the first/last step of a workflow and on error.
• Rewrite the export to purl/stacks so that it doesn't rely on the XML datastreams (using cocina instead)
• Figure out what the interaction between DSA and preservation looks like. I believe that preservation depends on contentMetadata.xml being present. Can this be switched to use Cocina? Do we rewrite all the existing objects with cocina?
  • https://github.com/sul-dlss/preservation_catalog/blob/2dc297dd416f5264902fabb919805cac666a26db/app/services/moab_storage_service.rb#L14
  • Update content_diff endpoint to accept Cocina structural metadata sul-dlss/preservation_catalog#1756
  • The preservation_catalog endpoint must be able to handle old objects that have contentMetadata.xml
  • Investigate using the moab manifest for this.
  • MOAB does not require contentMetadata.xml but the gem does. Investigate: https://github.com/sul-dlss/moab-versioning/search?l=Ruby&q=contentMetadata
  • This looks like we require versionMetadata.xml: https://github.com/sul-dlss/moab-versioning/blob/1cfe59cfb4f9fea7a1cc35b936a8f5a7ddd94029/lib/moab/deposit_bag_validator.rb#L54
  • Investigate if we need versionMetadata.xml in the MOAB.
  • Look in preservation_robots
• Update common-accessioning to pass cocina metadata rather than using the legacy APIs.
  • Can any common-accessioning steps be removed?
  • rights https://github.com/sul-dlss/common-accessioning/blob/0f44e8603f421f23b12fd15b0fffe8efe6d488c8/lib/robots/dor_repo/accession/rights_metadata.rb#L18
  • description https://github.com/sul-dlss/common-accessioning/blob/0f44e8603f421f23b12fd15b0fffe8efe6d488c8/lib/robots/dor_repo/accession/descriptive_metadata.rb#L19
  • provenance https://github.com/sul-dlss/common-accessioning/blob/0f44e8603f421f23b12fd15b0fffe8efe6d488c8/lib/robots/dor_repo/accession/provenance_metadata.rb#L14
  • content https://github.com/sul-dlss/common-accessioning/blob/0f44e8603f421f23b12fd15b0fffe8efe6d488c8/lib/robots/dor_repo/accession/content_metadata.rb#L25
• Update hydra_etd to pass cocina metadata rather than using the legacy APIs. https://github.com/sul-dlss/hydra_etd/blob/6bae7466a84d97e2f18accf64f90c793a1fa0251/app/jobs/other_metadata_job.rb#L51-L61
• Update gis-robot-suite to pass cocina metadata rather than using the legacy APIs. https://github.com/sul-dlss/gis-robot-suite/blob/6d0802f2d68b1613339303dd8fb41c2d082153dc/lib/robots/dor_repo/gis_assembly/load_geo_metadata.rb#L35-L40
• Update argo to pass cocina metadata rather than using the legacy APIs.
  • https://github.com/sul-dlss/argo/blob/f728b0922975739ede99bfcb86d7cf82d611d209/app/controllers/datastreams_controller.rb#L72
  • https://github.com/sul-dlss/argo/blob/f728b0922975739ede99bfcb86d7cf82d611d209/app/services/apply_mods_metadata.rb#L82
• Update the argo UI so that datastreams aren't featured.
  • Vivian suggested to Justin (2021-10-13) that all of these changes need to be made: https://projects.invisionapp.com/share/ZR1072MLCMEW#/screens/444556663_Feb_2021_Remodel Do they? This could take a substantial amount of time.
• Begin writing Cocina metadata to a metadata store (presumably postgresql). Serve it from the metadata store if it is found there, else fallback to Fedora.
• assembly robots should transform stubContentMetadata.xml to cocina.json

[andrewjbtw]

I think fully replacing the events datastream may belong on this list. But I think most of what's left to do there is really for the UI to display in Argo.

Also, goobi sends in a stubContentMetadata.xml. I don't think goobi uses the legacy API itself because it kicks of assemblyWF, but if we're going away from contentMetadata we should have goobi send in structural metadata in another form. We might even be able to get it to use the SDR API. This will have to be coordinated with the vendor.

[jcoyne]

We also need to replace free-jqgrid in Argo as it is no longer maintained. And has some bugs that prevent us from upgrading to the latest version of rails that uses esbuild: free-jqgrid/jqGrid#502 Probably not necessary at this point.

[jcoyne]

Moved this to https://docs.google.com/document/d/1apN7YmdMLhlOp4XU3AbycNpvn5Kr0emEVzMo7xj0clA/edit#