Any hint / doc / guide / recipe / tutorial for graphql (with graphql-ruby gem)?

[ghost]

Any hint / doc / guide / recipe / tutorial for graphql (with graphql-ruby gem)?

I found nothing here (in the issues) and nothing here: http://shrinerb.com.

Why?

[monorkin]

Hi! At the moment there is no official way to upload files using GraphQL (as far as I am aware).

You have three options how to solve this problem:

1. Presigned upload URLs
   This is a common method for handling file upload directly from the client to the file storage server. Shrine already supports this feature.

   To get it working just follow the guide, and update your client to use the presigned URL for file upload.

   This method is not suitable for applications where the server needs to use the uploaded file immediately (eg. to create different versions of the file, send it right away to another client, ...). But it lowers your server's bandwidth and CPU usage. Also note, if you are bound to AWS, you can create a lambda that will trigger image processing and ping your server to notify it about the upload.

2. Base64 endoced
   This method is the simplest of all that I will mention here, but it is also the worst to use in a production (high load) environment.

   Your client can encode the image to a Base64 encoded string and send it as a field in a mutation.

   While on the surface this seems harmless, since the image has to be uploaded to the server in one way or another, this approach has many unwanted side-effects. The biggest issue is memory consumption. Since the image is now part of your request's JSON body it will be parsed as a string, a very large string in fact, which will drive your memory consumption up. Just to be clear, if you upload ten 5MiB images in one mutation you will have a hash that's at least 50MiB in memory.

   If you do decide to use this upload method, then Shrine's got you covered - use the data_uri plugin. After you add it to your uploader you just need to do the following in your resolver:

   class CreateUserResolver
     def call(_object, arguments, _context)
       user = User.new
       # ...
       user.avatar_data_uri = arguments[:encoded_avatar_image]
       # ...
       user.save
     end

   So, Shrine will add a #{attachment_name}_data_uri method which can be assigned a Base64 encoded image.

3. Multipart upload
   This is the normal way of uploading files (that's what happens if you submit a form with a file field in it). Though when working with GraphQL it gets a bit weird since the spec isn't finalized (this isn't the official way of doing file uploads, but it's the closest thing there is to an official implementation).

   Since GraphQL requests are only JSON requests, instead of the normal GraphQL fields you usually get (query, variables and operationName) you will receive operations which will contain the raw JSON of your GraphQL request, map which will contain the raw JSON of the file mappings, and numbered fields e.g. "0", "1", ... which will contain the uploaded files. The map field contains the index of the files as keys and the values are arrays with JSON pointers where the file occurs in the variables field.

   Example params:

   {
     "operations"=>"{ \"query\": \"mutation($files: [Upload!]!) { multipleUpload(files: $files) { id } }\", \"variables\": { \"files\": [null, null] } }", 
     "map"=>"{ \"0\": [\"variables.files.0\"], \"1\": [\"variables.files.1\"] }", 
     "0"=>#<ActionDispatch::Http::UploadedFile:0x000055fb90c6dbd0 @tempfile=#<Tempfile:/tmp/RackMultipart20180131-31-6zkb3i.txt>, @original_filename="b.txt", @content_type="text/plain", @headers="Content-Disposition: form-data; name=\"0\"; filename=\"b.txt\"\r\nContent-Type: text/plain\r\n">, 
     "1"=>#<ActionDispatch::Http::UploadedFile:0x000055fb90c6da18 @tempfile=#<Tempfile:/tmp/RackMultipart20180131-31-13i0pio.txt>, @original_filename="c.txt", @content_type="text/plain", @headers="Content-Disposition: form-data; name=\"1\"; filename=\"c.txt\"\r\nContent-Type: text/plain\r\n">
   }

   To get this to work in Ruby you will need to parse the operations part. Then parse the map part. Iterate through the map, take the file with the index of each key of map and replace the value of variables pointed to by the JSON path with the file. Then you can call the GraphQL schema as normal, and you will have access to the uploaded files in your resolvers. At that point, you can use shrine as normal and just assign the file to your model's attribute (or use an uploader directly).

   There is a reference implementation. And there is a request spec. And you can follow the discussion about the proposal here.

---

My suggestion would be to use method 1. if you can. It's the best solution that's standardized and widely used. If you can't use that method then I would suggest you take a risk and try method 3..

Personally, I've used method 2. and it's shown to be problematic in production. But easy to implement. If you do decide to use method 2. then please filter the image filed in your logs, else each file uploaded will also be persisted in your logs (if you upload a 5MiB image your log will increase 5MiB in size) and try to limit the client's body size to something reasonable (do this check in your proxy, clients are unreliable by nature).

---

Hope this gets you going in the right direction :)

[hmistry]

@stankec Thank you for chiming in and providing a good writeup on the available options in using Shrine with GraphQL.

[ghost]

@stankec, thanks a lot for you amazing answer.

I think there is another way, but maybe I'm wrong.

I think I can have in my single page application client (Apollo, Relay, ember-graphql, whatever) a form to send with graphql normally and an upload component (detached from that main form) that upload to my Shrine endpoint. After the upload I can take metadata (from the Shrine's response) and fullfill an hidden field of my graphql form.

Am I totally wrong? Where am I wrong?

[monorkin]

@johnunclesam you are correct. You can do that and it's no different from the approach outlined in method 1..

---

A pre-signed URL doesn't necessarily have to go to S3, it can go to your server, DigitalOcean Spaces, Google Cloud Storage, etc. The point of the approach is "out-of-order upload", that is, that you upload a file first and then send it's location to the server to store it.

Be aware of the downsides of this approach. If you don't pre-sign URLs anybody will be able to leave a file on your server and retrieve it at any time - this can cause legal issues since people can effectively use your server as a file repository for illegal materials. This isn't completely solved by pre-signing - pre-signing only solves the "anytime" and "anybody" part since to upload or retrieve a file your server has to be contacted first to generate a URL, and at that point, you can determine to e.g. decline the request to non-logged-in users.

Another issue is "shopping cart collection". In a regular upload (multipart) if a client aborts the form submission the file is "lost" in the sense that it wasn't stored on the server. But with pre-signed URLs, a client can abort form submission after they have uploaded a file. To remove those orphaned files you will have to create a script that periodically removes them (e.g. non-attached images older than 24 hours). You can also ignore this issue all together but it may become costly if you are on a service like S3 or it could crash your app if the server's disk gets filled up.

[monorkin]

Hey! @johnunclesam @hmistry can we mark this issue as resolved?

[hmistry]

@stankec I don't have privileges to close it. Either @johnunclesam or @janko-m will have to do it. @janko-m is busy at the moment but don't worry, he'll get around to it when he gets the time. He is pretty good about taking care of that. 😃

[monorkin]

@janko-m I will trade beer for closed issues :D 🍻

[janko]

@stankec Thank you so much for providing such detailed explanation for using Shrine with GraphQL ❤️. You definitely get a beer for that 🍺

[dmitry]

Found this guide via search, would be great to move it somewhere to the documentation, so the others could find it much easier than me :)

[janko]

I'm not sure whether this belongs in Shrine documentation, because it's about hooking up Shrine with something else. And there are a lot of things you could hook up Shrine to: JSON-API, gRPC, CKEditor, Trix etc.

I would rather that someone writes an article about it, and then we can add a link on https://shrinerb.com. I'm hardly the person to do it, though, because I've never used GraphQL. I'm also too busy developing and documenting Shrine features by themselves.

[monorkin]

@janko-m here's the article https://blog.stanko.io/graphql-file-upload-with-shrine-45fa26463c68

[dmitry]

Another method: inject the uploaded file in the execution context.

[monorkin]

@dmitry can you explain this further?

[dmitry]

@monorkin in the controller's action you can pass file from the params to the graphql execution method through the :context and catch it inside a mutation resolver method.

      result = Schema.execute(
        params[:query],
        variables: ensure_hash(params[:variables]),
        context: {file: params[:file]}
      )

[monorkin]

@dmitry This is not that different from the multipart upload method. In face the only difference is that you packed your data in a custom location.

I didn't cover what to do after the file gets to your server because there isn't an universally best solution - how you pack your data depends on your use-case. I only covered how to get your data from your client to your server, since this is the unstandardized part of GraphQL.