List

GraphQL and Rails beyond HTTP APIs

GraphQL and Rails beyond HTTP APIs

by Gui Vieira

In the presentation titled "GraphQL and Rails beyond HTTP APIs," Gui Vieira, a software developer at Shopify, explores the versatile applications of GraphQL, extending well beyond traditional HTTP APIs. The session, held at RailsConf 2022, delves into how GraphQL can serve as a secure and structured data layer in Rails projects, discussing several innovative use cases and implementation strategies.

Key Points Discussed:

- Introduction to GraphQL: Vieira begins by addressing the common question of how GraphQL compares to REST. He notes that while it's often seen as a replacement, GraphQL offers much more functionality that transcends the limitations of HTTP APIs.

- Case Study - Space Trips API: Using a fictional API for space tourism, Vieira demonstrates various queries to book flights and manage passengers, emphasizing how GraphQL queries return predictable structures that enhance developer confidence.

- Execution Contexts: He explains that GraphQL is agnostic to HTTP, allowing it to be executed in diverse environments like background jobs or even from database-stored queries, enabling a wide range of use cases.

- Webhooks: The presentation discusses how organizations can improve event-driven workflows using GraphQL webhooks. For example, an organization called "Orbital" automates passenger tagging processes post-flight, showcasing how GraphQL allows clients to specify the exact data they need in webhook responses, ultimately simplifying operations.

- WebAssembly Integration: Vieira introduces WebAssembly to replace the need for maintaining server endpoints for webhooks, facilitating function execution in a secure environment without excessive networking overhead.

- Bulk Operations for Data Export: The topic extends to handling large-scale data exports through GraphQL's bulk operations feature. This allows requests to be processed as background jobs, significantly reducing the strain on both the client and server.

- Comparison of GraphQL and REST: The conclusion emphasizes that GraphQL is not just an HTTP API alternative; it presents a wealth of capabilities, making it vital to explore its full potential.

Conclusions and Takeaways:

- GraphQL provides a flexible framework for various data handling needs in Rails applications.

- Beyond HTTP, GraphQL supports real-time data communication, minimizes unnecessary API calls, and allows for effective data query customization.

- Developers are encouraged to dive deeper into GraphQL to discover its extensive functionalities and adapt them to their project requirements.

Are you considering building a GraphQL API for your Rails project or already have one? Do you know GraphQL can be leveraged beyond HTTP APIs?

We will explore how GraphQL does not depend on HTTP and can be used as a secure and structured data layer for Rails projects. You will learn to deliver real-time GGraphQL through Websockets, Webhooks containing all the data you need, provide data for WebAssembly code and parallelize queries exporting large amounts of data. Every Rails project needs consistent access to data and GraphQL brings solutions beyond the typical HTTP API.

RailsConf 2022

00:00:00.900 foreign
00:00:15.020 thanks for being here either in person
00:00:17.940 or watching online
00:00:20.460 my name is Viera you can call me gee I'm
00:00:24.180 a software developer at Shopify working
00:00:27.119 at the API patterns team
00:00:29.640 and I live in Vancouver Canada
00:00:34.079 I'll make some assumptions that may or
00:00:36.239 may not be related to
00:00:38.340 maybe graphql is something new to you
00:00:41.579 anyone to learn more about it and I'll
00:00:45.360 break some graphql queries now
00:00:47.780 especially at the first half of the
00:00:49.739 presentation so hopefully everyone can
00:00:52.460 enjoy the presentation
00:00:55.680 maybe you want to use graphql in a
00:00:58.020 project
00:00:58.980 you can even see how graphql would look
00:01:01.379 like there but you haven't implemented
00:01:04.260 yet
00:01:06.119 or you already maintaining a graphical
00:01:08.340 project but you want to exchange it
00:01:13.260 I've been out of these positions and
00:01:15.600 there's something common to all of them
00:01:18.119 it's a question and this question can
00:01:20.759 can come from yourself
00:01:23.040 a teammate a manager or from the
00:01:26.520 internet
00:01:28.979 graphical versus rest which being there
00:01:32.580 reading and discussing how graphql
00:01:34.920 Compares with rest multiple times
00:01:38.820 is this a fair comparison are we
00:01:41.400 comparing Apples to Apples
00:01:44.460 is graphical just a replacement for West
00:01:48.420 so let's explore what graphql can do
00:01:52.020 Beyond HTTP apis and then we'll release
00:01:55.500 this question at the end
00:02:00.540 so let's view that API here and explore
00:02:03.899 it
00:02:06.000 I present your space trips this is a
00:02:08.280 fictional API for this the space tourism
00:02:12.840 industry with this API
00:02:17.040 forever websites can book space flight
00:02:19.680 tickets manage the passengers which are
00:02:22.620 the customers and companies operating
00:02:25.080 space flights can manage the flights
00:02:28.319 change schedules and also manager manage
00:02:31.440 the passengers
00:02:34.260 let's explore some frames that we can do
00:02:36.360 here
00:02:37.500 we can start with a discovery that will
00:02:40.860 return the schedule flights so the
00:02:43.500 upcoming flights
00:02:44.879 we start with the flight field
00:02:48.739 and we pass the schedule status
00:02:53.879 so this kind of status may be um just uh
00:02:58.140 removes the flights that were canceled
00:03:01.440 or
00:03:03.239 um
00:03:04.080 happen in the past
00:03:07.560 and this field returns a connection so
00:03:11.459 Connection in graphql
00:03:13.040 it's a conventional it's actually a
00:03:15.840 specification that's very common to
00:03:17.760 graphical apis and it's a way for you
00:03:20.760 for us to manage arrays of objects and
00:03:24.180 have features such as pagination cursors
00:03:27.440 and other more
00:03:31.140 and we have the nodes object here so
00:03:33.659 they know the objects basically
00:03:34.860 represents what you are returning so we
00:03:37.620 are returning an array of flights and
00:03:39.599 for each flight we want information such
00:03:41.640 as ing the company that is operating the
00:03:46.200 flight
00:03:47.599 the ticket availability and departure
00:03:50.760 and arrival times
00:03:54.720 okay so we are ready to make our first
00:03:57.780 graphql
00:03:59.340 request to our EPA
00:04:02.400 so that's the response and returns to
00:04:04.980 flights one flight from space Y and
00:04:09.360 another flight from Blue beginning so
00:04:12.599 our space flight companies here
00:04:17.040 um let's get more information about the
00:04:19.380 spacewire flight
00:04:22.820 we can start by
00:04:27.600 um sorry actually before this I want to
00:04:32.100 um
00:04:33.180 showed that the response mirrors the
00:04:36.000 query
00:04:37.199 so the same way the query starts with
00:04:40.680 flights
00:04:41.759 the response mirrors this same structure
00:04:44.400 so we have flights then nodes and then
00:04:48.240 the fields for each flight
00:04:51.180 the structure of a graphical response is
00:04:54.180 very predictable and this is very
00:04:55.979 important because
00:04:57.419 it makes you write very confident code
00:05:00.660 you don't need to check for all kinds of
00:05:04.080 scenarios
00:05:06.840 now we will do another graphical query
00:05:10.199 and we want to hit return the passengers
00:05:12.600 for the spacewire flight and for this we
00:05:15.360 start with the IG
00:05:18.360 we got the flight ID
00:05:20.160 and we call the flight field we pass the
00:05:23.340 HG
00:05:25.199 and for this flight we want to pre-read
00:05:28.320 the passengers
00:05:29.460 and passengers is already it's also a
00:05:34.080 connection a collection field
00:05:36.840 and for each passenger we want to return
00:05:39.900 the name the email and the text
00:05:45.840 okay
00:05:47.160 that's nice but can we make this query
00:05:50.160 more reusable
00:05:53.100 yeah we can use variables and extract
00:05:56.180 hard-coded values into variables so
00:05:59.039 instead of changing the entire query we
00:06:01.680 just change the variables
00:06:04.199 and we start this by wrapping the query
00:06:07.680 in our operation
00:06:09.539 the operation name can be anything it's
00:06:12.120 already user provided and it's a way for
00:06:15.720 clients API consumers to organize their
00:06:19.860 queries give a descriptive name
00:06:24.000 and then we declare the variable name
00:06:26.520 and the type so we have flight ID that's
00:06:29.160 our variable name and the type is ID
00:06:33.900 now we replace our hard-coded value with
00:06:36.780 the variable name and instead of passing
00:06:39.360 the variable in the query we pass
00:06:41.520 separately in a Json object
00:06:44.699 now we're ready to run this query
00:06:48.500 and the response has two passengers for
00:06:52.259 display
00:06:55.199 but what's happening behind the scenes
00:06:57.900 here
00:07:00.539 the graphical client
00:07:02.340 gets your query and converts into intra
00:07:05.220 Json payload so it makes a post request
00:07:07.680 to your graphical endpoint
00:07:10.560 and for the Json payload it has a query
00:07:14.520 key with just a string with your query
00:07:18.479 and the variables key with an object
00:07:21.000 with the variable names and value
00:07:24.000 so that's what happened
00:07:26.599 that what graphical clients do with your
00:07:29.819 your query
00:07:32.699 and on the server side we have a
00:07:35.280 controller here that calls the schema
00:07:39.240 and executes the query and for the
00:07:42.599 execution we need to provide the query
00:07:45.240 name sorry the query
00:07:47.479 the the variables and also the context
00:07:52.740 context here is very important
00:07:56.280 context defined by the application so
00:07:58.919 you can put any information there and we
00:08:02.340 usually put information that's very
00:08:04.099 useful for executing the query
00:08:07.620 here we are passing the current user so
00:08:10.740 we can use the current user information
00:08:12.300 for authorization purposes
00:08:15.360 query in variables are provided by the
00:08:17.880 user and context is provided by the app
00:08:20.879 so we have total control over the
00:08:23.039 context
00:08:24.599 and the result of the execution is
00:08:27.300 realized to Json to create adjacent
00:08:29.940 response
00:08:33.539 let's replace
00:08:35.339 all values with our literals here and as
00:08:39.360 you can see
00:08:41.279 query is just a string variable
00:08:44.240 variables is just a hash
00:08:47.300 and the response is also just a hash
00:08:50.580 that we can encode to Json for the
00:08:53.820 response
00:08:55.260 so there's nothing here that is related
00:08:58.320 to http
00:09:00.480 there are no HTTP methods paths query
00:09:05.640 strings headers nothing here
00:09:09.120 so
00:09:10.680 graphql doesn't depend on HTTP we can
00:09:13.920 execute in different contexts
00:09:16.080 and we say that graphql is transparent
00:09:19.560 layer agonostic
00:09:22.320 the same way you can execute graphql in
00:09:25.860 a controller
00:09:27.600 you can also execute graphql in a
00:09:30.240 background job
00:09:31.500 or even
00:09:33.600 execute graphql queries that are stored
00:09:36.480 in the database
00:09:38.279 so this opens a lot of possibilities for
00:09:40.680 graphql
00:09:42.360 because
00:09:43.860 graphical has a schema we have the
00:09:46.500 context that we have total control for
00:09:48.779 authorization for anything that we need
00:09:50.899 the responses are very predictable they
00:09:54.600 need graphical needs very simple data
00:09:58.260 structures to run and the response is
00:10:00.480 very predictable so there's a lot of
00:10:02.519 potential here
00:10:03.839 for for usage
00:10:06.140 besides executing a controller
00:10:12.839 web hooks
00:10:16.440 we have a new company here this is
00:10:18.779 orbital
00:10:19.920 also a fictional company and they are a
00:10:22.320 website that's sells space flight
00:10:25.800 tickets
00:10:27.660 orbital uses the space trips API to book
00:10:31.800 those tickets and to manage managing
00:10:34.260 their customers
00:10:37.500 and they want to improve the way they
00:10:40.500 manage their customers which are the
00:10:43.019 passengers
00:10:44.760 when I when a flight ends they want to
00:10:48.120 add the astronaut tag to each passenger
00:10:51.240 they've booked for this flight
00:10:54.959 so there's an event here that triggers
00:10:57.600 this flow
00:10:58.920 and for these days they need to
00:11:00.899 subscribe to a live hook
00:11:03.060 so this is a Graphica mutation
00:11:05.480 and that subscribes to the webhook
00:11:08.779 and subscribes to the flight finished
00:11:13.399 topic so every time a flight finishes it
00:11:17.160 will trigger a web hook
00:11:19.920 and army town needs to set up a web
00:11:21.899 server to receive those web hooks so
00:11:24.420 they set up a web server they expose the
00:11:27.360 inch point and they also need to
00:11:28.980 maintain this web server to make sure
00:11:30.959 they don't miss any web hook
00:11:36.180 flight ends and they receive the first
00:11:39.480 Red Hook
00:11:41.160 and the first webhook has a lot of
00:11:43.260 information about the flight
00:11:46.019 but no information about the passengers
00:11:49.380 anyone they need the passenger
00:11:51.120 information track the tags to them
00:11:54.300 so
00:11:55.680 without the passenger and data they
00:11:57.660 can't add the tags
00:12:00.000 well they need to make an API request to
00:12:03.600 get the data they actually need
00:12:06.899 so early so then
00:12:09.779 um
00:12:10.920 let me go back here
00:12:16.140 so they get the flight Ag and they
00:12:18.720 create a new query they pass the the
00:12:22.260 flight edgy here
00:12:23.880 and
00:12:25.620 they are query for the passengers IGS
00:12:29.519 now the response has the data they
00:12:31.680 actually need here
00:12:37.079 and now they finally can add the
00:12:40.100 astronaut tags to those passengers so
00:12:44.100 it's a lot of work and they get a web
00:12:47.279 hook that doesn't have the information
00:12:49.200 they actually need
00:12:52.800 and that's the the whole workflow they
00:12:55.320 receive a web hook they make an API
00:12:57.720 request just to get the passenger ideas
00:12:59.399 and then the add the text that they need
00:13:06.600 can we improve this is it possible to
00:13:09.720 let API clients choose the webhook
00:13:13.920 payload
00:13:16.680 yeah we can do this with graphql because
00:13:19.079 with graphql we Define queries
00:13:23.279 on this every hook subscription they are
00:13:26.160 creating a graph by Weber Hawk and they
00:13:28.740 pass a graphical query
00:13:30.720 with the information they need
00:13:33.000 the client provides the query
00:13:35.579 and the survey provides the variable
00:13:38.639 so we have the flight ID variable here
00:13:40.680 and this is the variable we will be
00:13:43.079 provided by the server for the flight
00:13:45.540 that just painted
00:13:49.980 so they subscribe to the web hook
00:13:54.500 and on this on the server side we have a
00:13:59.940 background job
00:14:01.019 that used to just change serialize a
00:14:04.920 simple web hook
00:14:06.120 but now our background job executes
00:14:09.120 graphql
00:14:10.320 it gets the information from the web
00:14:12.120 hook subscription
00:14:13.740 and runs graphql in the background job
00:14:17.820 and at the end it sends the
00:14:21.540 the the result of that query back to
00:14:24.720 orbital
00:14:32.639 okay
00:14:33.959 so now we have the the new web hook
00:14:37.980 and the web hook only has the
00:14:40.320 information that orbital needs nothing
00:14:43.560 more nothing less
00:14:45.839 in this whole flow here that needed an
00:14:49.320 extra request
00:14:51.360 now don't need the extra request anymore
00:14:54.060 they have all the information they need
00:14:56.279 on the webhook payload and they they can
00:14:59.220 just add the srl tags to the passengers
00:15:06.000 and this is very good because instead of
00:15:08.519 having a one size fits-all approach
00:15:11.459 when you let clients repair clients to
00:15:15.480 define the information they need this
00:15:18.180 adds a lot of value to them make their
00:15:20.220 lives easier very easy to maintain and
00:15:23.459 also it's more efficient we are not
00:15:25.380 making a necessary API requests here
00:15:31.320 let's do something different now
00:15:35.880 we have this flow and it's working fine
00:15:41.040 but orbital needs to set to maintain our
00:15:44.579 web server just to receive the web hook
00:15:47.639 they need to make security updates they
00:15:51.000 need to monitor they need to deal with
00:15:53.959 networking issues they need to handle
00:15:56.579 missing Lobby hooks
00:15:59.060 if they have any kind of outage
00:16:03.240 but is it possible to
00:16:06.899 exchange to move all this logic that
00:16:10.560 receives the Bible book and and makes an
00:16:14.040 API request to the server
00:16:17.160 can the server execute code that was
00:16:19.980 provided by the client
00:16:24.240 yeah we can do this with a webassembly
00:16:27.540 so why am I saying they allows us to
00:16:31.620 write code in different languages
00:16:34.560 compile just call this code into a
00:16:38.399 package that can run in a very safe
00:16:41.699 environment
00:16:43.620 and with this flow we don't need all
00:16:46.740 this network
00:16:48.600 um
00:16:49.800 coming from and to our server
00:16:54.240 and orbital won't need to maintain a web
00:16:57.300 server anymore
00:17:01.680 yeah in the same way uh it's just a very
00:17:06.419 simple logic going on on the orbital so
00:17:09.360 it's very easy to write some code for
00:17:12.419 this
00:17:14.160 several languages compile to webassembly
00:17:17.579 and because webassembly support is
00:17:19.919 coming soon to Ruby we can use Ruby for
00:17:23.100 this
00:17:24.179 and we write a simple method here called
00:17:27.720 perform
00:17:28.980 in the argument that this method
00:17:30.660 receives is the web hook that we just
00:17:33.240 defined so we just defined the web hook
00:17:36.179 so we know
00:17:37.620 what's uh
00:17:39.660 the the argument here is very
00:17:41.340 predictable because that's the graphql
00:17:43.559 response
00:17:46.860 that's the our argument for our function
00:17:51.360 and
00:17:53.340 the the function the method
00:17:56.840 extracts the passenger IGS here
00:18:02.340 and then runs the edge flight sorry add
00:18:05.640 tag mutation passing the passenger
00:18:09.539 passenger ig's variables there
00:18:15.179 now orbital compiles discouraged
00:18:17.460 webassembly uploads to space trips and
00:18:21.660 gets
00:18:23.760 an ID for this script
00:18:27.360 and now
00:18:28.919 for this new web hook instead of passing
00:18:33.000 a URL argument
00:18:35.700 orbital passes
00:18:38.820 I script IG
00:18:40.799 so instead of delivering to an HTTP
00:18:44.100 endpoint it executes the script
00:18:50.340 under server side
00:18:52.080 that's how the background job
00:18:54.299 for maybe hooks looks like now
00:18:58.860 it's true
00:19:00.740 executes the the query that they just
00:19:03.980 they have subscribed to
00:19:06.780 and they it loads the webassembly script
00:19:11.880 and it provides us it provides some
00:19:13.799 context
00:19:15.000 so this content is very important
00:19:18.059 to create
00:19:19.280 a safe environment to run this script
00:19:23.640 and what it's doing is for whatever
00:19:27.500 graphql require equipment is made here
00:19:30.600 it adds the current user contacts that
00:19:33.419 we need for authorization
00:19:36.059 so the execution here will be
00:19:38.640 very similar to the execution we had for
00:19:41.820 example in the controllers
00:19:45.059 and then he executes the script in this
00:19:47.760 context
00:19:50.520 now the whole flow is within the space
00:19:55.080 trips servers we don't need the back and
00:19:58.620 forth from Network
00:20:00.559 requests we don't need to to handle
00:20:04.799 missing web Hooks and orbital doesn't
00:20:08.460 need to maintain this web server just to
00:20:12.120 receive web hooks do some logic and
00:20:14.280 create
00:20:15.059 a mutation
00:20:21.419 another nice use case here
00:20:24.179 book operations
00:20:27.539 orbital now has a new requirement
00:20:30.720 they need to export all their passenger
00:20:33.539 data choose synchronize with some
00:20:36.179 external systems
00:20:40.020 and they have 5 000 passengers 5 000
00:20:43.440 customers and this number grows every
00:20:46.080 day
00:20:49.200 and it is a very common use case a lot
00:20:52.140 of API clients use apis to
00:20:55.620 synchronize data they often iterate over
00:20:59.220 pages and pages of Records they do this
00:21:02.940 a lot
00:21:05.340 they do this for marketing they do this
00:21:08.160 for accounting data analytics you name
00:21:11.220 it
00:21:13.919 but on the other hand servers need to
00:21:16.919 protect the stability there
00:21:19.320 they need to ensure
00:21:21.919 response time and for this usually they
00:21:25.740 create pagination limits and also Rich
00:21:28.260 damaging
00:21:32.460 and in a regular use cases with this
00:21:36.600 case they make a query
00:21:40.140 with a cursor and whenever the has next
00:21:44.340 page returns true it means they need to
00:21:48.240 keep iterating and to restraint
00:21:51.659 this is very inefficient because orbital
00:21:55.200 has 5 000 customers and the paginational
00:21:58.620 image is 50. so they need to make 100
00:22:02.400 requests to get all the passenger data
00:22:06.059 in this number will only grow
00:22:10.200 this is this is very inefficient not
00:22:13.020 only for the client but also for the
00:22:14.880 server
00:22:17.400 and if they want to export
00:22:20.820 data for example
00:22:24.000 the flights for each passenger now they
00:22:27.240 have nested connections they have nested
00:22:30.059 paginations
00:22:31.260 this gets unsustainable very quickly
00:22:36.539 so is there a way to export large amount
00:22:39.900 large amounts of data
00:22:41.760 while keeping the server
00:22:45.240 stable with a good response without
00:22:48.419 overwhelming the server and also doing
00:22:51.000 this in a in a very efficient way
00:22:54.840 for this we can use web hooks sorry for
00:22:57.900 this we can use book operations
00:23:01.020 so book operations are queries that runs
00:23:04.740 as background jobs instead of
00:23:08.240 running the query once we can split this
00:23:12.960 query into smaller jobs and run them
00:23:16.080 then in parallel whenever possible
00:23:20.960 and the way it works is
00:23:26.460 the pagination will be handled by the
00:23:28.919 server
00:23:29.820 so orbital here is just removed all
00:23:34.580 information about pagination so we don't
00:23:37.919 see first 50 on the query anymore
00:23:43.260 and then they create a mutation children
00:23:47.659 Discovery as a book operation
00:23:51.000 and they they pass the the query that
00:23:55.260 they want to run
00:23:57.720 and now this query will be run as a
00:24:00.120 background job
00:24:01.679 on the server
00:24:03.299 and the server
00:24:04.980 parses the query detects the connection
00:24:07.760 and splits this job into smaller and
00:24:11.340 smaller jobs that eventually will be
00:24:14.460 assembled together
00:24:17.520 and be available to the user and because
00:24:20.700 this runs as background jobs they can
00:24:23.520 run in parallel you have a lot of
00:24:25.380 control of the pace they run they don't
00:24:29.220 affect the overall API performance
00:24:34.860 and when the book operation is finished
00:24:39.240 the client in this case orbital can
00:24:41.700 download the
00:24:43.500 the query result as a Json file
00:24:49.260 so this removes a lot of complexity for
00:24:53.460 exporting large amounts of data from the
00:24:57.000 client makes it more efficient also on
00:25:00.419 the server side and it's it's a very
00:25:04.140 good feature for everyone to adopt
00:25:11.100 we have book operations like Shopify if
00:25:13.500 you want to try you can go to the admin
00:25:16.679 API documentation and experiment with
00:25:19.679 book operations
00:25:25.260 all right so we export some use cases of
00:25:28.320 graphql Beyond httpis we explored web
00:25:32.100 Hooks webassembly and book operations
00:25:35.039 but there are many more use cases for
00:25:37.860 this we can use for web sockets
00:25:40.700 event messaging you name it
00:25:48.480 so back to our original question graphql
00:25:52.799 versus rest
00:25:54.960 graph fail is not just a replacement for
00:25:57.960 rest
00:25:59.220 there are a lot of possibilities for
00:26:01.140 graphql
00:26:02.460 so when you're comparing graphical to
00:26:04.980 rest you are comparing rest to the tip
00:26:08.400 of the graphical weisenberg
00:26:10.440 and there's a lot to explore there
00:26:13.679 so I invite you to explore graphql
00:26:16.980 a lot there's a lot to to discover then
00:26:20.640 and
00:26:22.140 thank you