Summarized using AI

Symmetric API Testing

Aditya Mukerjee • June 25, 2016 • Earth • Talk

The video titled 'Symmetric API Testing' features Aditya Mukerjee, a risk engineer at Stripe, discussing the importance of API design and testing, particularly focusing on the concept of symmetric API testing. Mukerjee begins by highlighting the evolution of API communication from an asymmetric model to one that promotes symmetry, where both clients and servers communicate using the same format and protocols. This approach simplifies development and testing, improving overall compatibility and stability.

Key points discussed include:
- The need for designing API endpoints right from the start, ensuring clarity in communication between clients and servers.
- The significance of developing API clients alongside server code to gain insights into user experiences and interactions.
- The variety of testing methodologies, especially the importance of integration tests which validate user experience and application interconnectivity, contrasting with unit tests that operate in isolation.
- Risks associated with relying solely on unit tests, emphasizing the need for thorough integration testing, illustrated by a cautionary tale from NASA about a satellite docking failure due to overlooked environmental factors.
- Examples of how to conduct symmetric API testing, including practical methodologies using libraries like Anaconda, which automate processes like rate limiting.
- Techniques for capturing and reusing actual API responses to test applications without changing source code for every endpoint, thus promoting efficiency.
- The critical role of interfaces in ensuring that both sides of the API communicate effectively, detailing how they can be designed to promote flexibility and robustness across multiple programming languages.
- Finally, emphasizing the necessity of reflecting on interface contracts for symmetry and consistency in applications, with a reminder that these principles apply across various data formats including JSON and XML.
In conclusion, Mukerjee stresses the importance of building and testing APIs symmetrically, which leads to improved functionality and reliability, while also allowing for easier maintenance and adaptation to changes in technology or client-server interactions.

Symmetric API Testing
Aditya Mukerjee • June 25, 2016 • Earth • Talk

Symmetric API Testing by Aditya Mukerjee

When implementing REST API servers and clients, testing is critical. Symmetric API testing is a design pattern that ensures compatibility between client and server with transparent tests. Testing APIs symmetrically reduces build times and code complexity, while simultaneously improving API stability.

Help us caption & translate this video!

http://amara.org/v/PssK/

GORUCO 2016

00:00:15.730 Good morning, everyone! It's great to see so many people here, especially so early on a Saturday morning. My name is Aditya Mukerjee, and I am a risk engineer at Stripe.
00:00:21.050 Stripe is the payments backbone for many companies, including several of our sponsors here today, such as Lyft, Kickstarter, Slack, and UNICEF, among many others. On the risk team, I spend my days defending our users and customers—people like you—from online fraud.
00:00:39.440 If you're interested in hearing some battle stories or tales of what it's like on the dark web defending against fraudsters, I'm more than happy to talk with you about that later. But today, I'm here to talk about API design and testing.
00:00:52.430 Let's take a trip back to 2006. Ten years ago, Facebook had just opened itself up to high schoolers and college students, and people were freaking out about this. You may even have had a MySpace page; I would like to know how many people had a MySpace ten years ago.
00:01:10.580 Okay, good! I was afraid that only ten people would raise their hands, which would mean most people in this room were liars, because, let’s admit it, we all had MySpaces back then. Ten years ago, the best practices of web development dictated that you would do something like this: write a web server that would respond to HTTP requests by talking to your database, doing some computation, and then spinning back some HTML that the browser would render.
00:01:30.590 If you wanted to allow someone to write a native client for your application, the native client would use some raw format to communicate, likely XML at the time. It would still talk to the server, which would communicate with the database, perform some more computation, and then spit back data in that raw format. This system is fundamentally flawed; it is asymmetric.
00:01:56.659 Different clients are using different mediums to interact with the server to achieve fundamentally the same result. Fortunately, at some point, we realized that this was silly, and we consolidated our approaches. This is what the state looks like in 2016: we now understand that the browser is a special type of client, but it's still just a client. It communicates with the server using the exact same format and protocol as any other client, whether that's a desktop client or a mobile client.
00:02:33.170 Now, the server doesn’t even need to know what kind of client it’s communicating with, and this is beautiful. This is what we call symmetry in API design. When writing symmetric APIs, there are a few points you should keep in mind.
00:03:05.780 First, you want to design your API endpoints first. You don’t need to go overboard with this, but it’s crucial to make sure that your interfaces are clearly defined right from the start. Even if you're just prototyping and don't intend to make backward compatibility promises, you need to clearly outline how your client and server will communicate.
00:03:19.599 Designing your API first will force you to consider the key functional components of your application. If you've ever taken courses on UX, UI, or product design, you will have learned about user stories and storyboarding; this is essentially a lightweight way of achieving that. The interface defines how your users will interact with your product.
00:04:14.260 Finally, you always want to develop your API server and client together, even if someone else thinks they will provide the client for you. You should always write at least one client for every API you are building. If you do not, you won't fully understand your users' experiences, as you wouldn't have gone through the process yourself.
00:04:40.030 As for testing, there are several different ways that you might want to test APIs. I’ve polled many people, and no one can quite agree on the exact differences between integration tests and functional tests. I spoke to about ten people and got twenty different answers, but fortunately, for the purposes of this talk, it doesn't really matter too much.
00:05:04.479 We're just discussing degrees of granularity versus interconnection. For this talk, I’m going to use integration tests as a catch-all term to mean anything that tests more than a single application function, anything that examines how multiple application functions work together, or anything that relies on non-application state, such as a database call.
00:05:51.530 Integration tests provide several significant benefits. First of all, they give us the behavior that our users actually experience. It’s one thing to test all the ways our components should work together in isolation, which is what our unit tests provide, but we also want to test how our code actually functions in practice.
00:06:32.850 Integration tests can also provide information about changes that may occur in the external environment. For example, if you upgrade your database and it now uses a different format for communication, running only your unit tests might not indicate that your client or server suddenly broke. However, the same characteristics that make integration tests powerful can also render them fragile and unreliable.
00:07:21.090 They rely on external systems, which is beneficial because they can catch errors due to changes in those external systems. Still, it poses a challenge when tests fail, as it becomes unclear whether this is due to something outside our project scope or responsibility, or something we can control.
00:07:54.229 There's a great story I'd like to share about this. Some years back, NASA was building a satellite that needed to dock with the ISS. They conducted thorough testing at ground level to ensure that the probe would extend the correct distance from the body of the satellite. However, when they sent it into space, they forgot that space is a vacuum.
00:08:06.200 As a result, the force needed to project something a certain distance from the satellite body is slightly less in a vacuum than with atmospheric pressure, leading to a failure of the entire project. This illustrates the crucial need for thorough integration testing; although they performed unit tests at ground level, they neglected the integration tests that would account for environmental conditions.
00:08:54.199 Most projects will utilize a combination of unit tests, integration tests, and functional tests, which necessitates duplicating a lot of code. While this is beneficial for overall coverage, it ultimately increases the volume of code that must be maintained. More code means a greater chance of introducing bugs into our tests.
00:09:06.650 Let’s look at an example: we can consider Anaconda, the Twitter client library for Go. Anaconda was one of the first Go client libraries to return native structs, which you can think of as native objects. It was also the first Go library to handle rate limiting behind the scenes.
00:09:45.990 Most client libraries require you to handle errors related to rate limiting manually, informing you that you need to slow down. However, Anaconda manages this behind the scenes; it determines when you should retry and automatically replays the request for you.
00:10:37.969 I am using Anaconda as an example since it is open-source; you can explore it later if you’re interested. Importantly, for symmetric API testing, you want to test the client and server simultaneously. Even though I wrote the library, I did not write the Twitter API to go with it.
00:11:25.270 As I mentioned, the examples I use are in Go, but these design patterns are language-agnostic and can be applied in any programming language. A common approach to testing is to stub out responses. For instance, here’s an example JSON response from Twitter’s documentation for retrieving user information, alongside a corresponding struct that represents that response.
00:12:24.290 We can marshal the JSON into a struct to clearly illustrate the correspondence between the JSON response and the struct. For now, we will write this out manually, allowing us to mock these JSON responses in our tests using the native types. But what happens if our response structure changes?
00:13:26.900 Our tests will still refer to the old stub responses and won’t know about the new structure the server is providing. For example, Twitter used to return a string to indicate whether a tweet was censored in a specific country. At some point, they changed this to an array of strings, allowing tweets to be censored in multiple countries simultaneously.
00:14:26.570 This issue only occurs in production or when running tests against the live API. You may think this doesn't matter if you’re working in a dynamically typed language, but it can lead to errors. For instance, if you're writing JavaScript and use .length on a field you think is a string but is actually an array, the result will be semantically valid but not what you need.
00:15:19.890 Wouldn’t it be ideal to combine the reliability of unit tests with the completeness of integration tests without having to write additional code? Unfortunately, we can achieve this! Let’s focus on the client side of the tests.
00:16:08.070 We start by creating a local server. The first few lines of our testing code set it up, telling the client to query the local server instead of the actual API. We can construct the endpoint and hard-code the example response we want to return for our unit tests.
00:17:07.690 Instead of hard-coding responses for every controller, we can intercept live responses, record them, and then replay them. This process allows us to easily handle all HTTP requests dispatched through one source, which is crucial for simplicity.
00:17:54.720 What this essentially means is we don’t have to add response recording to every single endpoint or function separately. We modify it just once, drastically reducing the barrier to testing every API endpoint for structural integrity. There’s no reason to ship untested code when this process is so straightforward.
00:18:51.720 Thus, instead of creating separate handler functions for each endpoint, we can store the recorded responses in a directory dedicated to those JSON responses. For example, for the endpoint /user/lookup, we can save responses in a directory called JSON/user/lookup.json.
00:19:47.430 You may wonder why we named it JSON rather than responses. This choice is intentional, as we want to test symmetrically—the same data types that the client reads in the responses are also used by the server.
00:20:39.059 The production server processes a native type, computes, marshals it into JSON, and sends it over the wire. Why not share? Instead of manually writing that struct definition, we can use Go's capabilities to generate struct definitions from valid JSON.
00:21:34.650 For systems interacting with databases, even schema-less databases can have implicit schemas based on how you're interacting with your models. Therefore, it's beneficial to explicitly define your contracts.
00:22:45.000 To maintain consistency, we can use Go generate to update all struct definitions based on recorded JSON in our project directory. This ensures that our interface is preserved, and we know any changes will reflect in our code and tests.
00:23:35.859 This principle holds true across other programming languages as well. While Go generate caters specifically to Go, you can use various build systems to achieve similar effects in other environments. I want to briefly emphasis the importance of interfaces, as they are key to symmetric API testing.
00:24:20.220 An interface serves as a shared boundary that two components use to exchange information. In Go, any type can define an interface by implementing a specific method signature, allowing different types to satisfy the same interface.
00:25:16.970 This means you can write code against interfaces without knowing the specific types involved, which promotes flexibility and code that documents itself. Essentially, you want to ensure that your interfaces are designed thoughtfully as they facilitate symmetry in your application.
00:26:11.660 Interfacing guarantees that identical validity rules apply to both sides of the interface. You don't need interfaces built into the language to implement these patterns, and even in dynamically typed languages, you can infer interfaces through implementation.
00:27:12.480 It's important that you think about interfaces at the beginning of your coding and tooling processes, just as you would with testing and building.
00:27:58.470 We’re just scratching the surface with what interface types can achieve. If you’re interested in advanced uses of Go interfaces, I presented at Go for Khan in India and Dubai this past February, and I encourage you to check it out.
00:28:48.899 Until now, we've focused on testing just the client, but these same techniques can be applied to server logic. We record actual client requests and feed them directly to the server handlers. For example, the endpoint we’re testing could simply print a string based on a request’s input.
00:29:27.530 We are not making an actual HTTP request; instead, we create a request value representing what the server would see. This means we can test the handler directly without any network activity. When it comes to what we want to test on recorded responses, the obvious focus would be on state changes like database updates.
00:30:20.550 For instance, when testing a signup endpoint, we want to ensure a new user is created in the database, and also validate the server's response—ensuring the user object’s JSON representation is returned correctly.
00:31:16.700 Both the client and server interact with the same recordings for validation, ensuring consistency across the application. This calls for a symmetry layer that defines the contract for how these components communicate.
00:32:00.650 This principle of symmetry applies not only to JSON but also to various interchange formats, including Protocol Buffers or XML. It’s important to test the actual behavior of the client and server rather than an idealized model.
00:32:45.590 In summary, we have a few guiding principles for API testing. We have tests that compose and degrade gracefully, similar to how APIs should function. You should strive to write at least one client for every API and lower the barrier for writing tests to ensure the highest coverage.
00:33:36.589 Ultimately, remember that interfaces are powerful and are key to achieving symmetry within your applications. The strength of your API comes from the robust contracts you establish, either implicitly or explicitly.
Explore all talks recorded at GORUCO 2016
+5