Contract Driven Development – Deploying your MicroServices independently without integration testing

Welcome everyone to the contract driven development session by Joel and Hari. Just a quick introduction from my side. So Hari Krishnan is a founder and CEO of Polarizer Technologies. He is a polygot, full stack developer, architecture consultant, XP coach and the trainer. And Joy is an engineer by heart. He’s a senior consultant. He loves building things. And apart from that, when he’s not coding or not working, he is also a great musician. So, without any further delay, over to you, Joy. Let’s get started. Thanks Karthik.

 

And welcome to all of you to this talk. Let me ask my colleague Hari to give a quick introduction about himself first.

 

Sure. Thanks Joel. So I’m Hari Krishnan. I’m a consultant and a coach. I help both unicorn startups and large enterprises with their transformation activities. I love contributing to the community. I speak at quite a few of these conferences and I volunteer. Also, my interests include distributed systems and high performance application architecture. So that’s quickly about myself. Over to you, Joel.

 

Thank you. Hari and I have about 19 years in this industry. At present, I am a consultant and coach and my focus areas tend to be agile software development, software quality practices, and contract testing. Well, let’s jump into the session today with a demo. I’m going to show you some code. This is an ecommerce service. I won’t go through the code for this too much. Let’s quickly take a look at its open API specification. Even if you haven’t seen an API specification before, I’m sure you’ll all recognize an API when you see it. This is the Slash Products API, which takes an ID in the path. The ID has been defined as a number, as you can see here. And this essentially means that this API could be product ten, products 20, so on and so forth. And when you fire a Get on this API, it’s going to return some product details and the product details, the exact shape of it is defined in this file. But you can see an example. There’s a name type with Gadget inventory ID. So this is how the API should look. Let’s try to run some tests on this API.

 

I’m going to just quickly run these tests. Let’s give it a second start up. And here we go. Tests are running naturally. We are starting the spring application up here so that we can hit it with the tests. And you have some tests that have run twelve tests. As you can see from this name here, this is a set of contract tests. So let’s see what this actually means. There’s a request that has gone out, right? I was saying we just started the application, obviously so that we could hit it. The request has gone out. It’s a Get request to the application to Slash Product ten. And the application has responded with a 200.

 

And it has a payload here, right? And this payload is familiar. I just showed this to you a few moments ago in the contract. And as you can see, it’s a contract valid payload. This is exactly the way the specification defines it to be. And this test therefore passes, which is good. Here we have another request. This goes to post to product ten. The payload this time is an actual product content and we get a 200. And this is about updating product details. You can see the title on the left hand side. So there are ten more, essentially. And these are contract tests.

 

We’ll talk more about them. But essentially what we are trying to say is, is this API implemented in the correct fashion according to the specification that I showed you in the YAML file earlier? What I’d like to show you next is the code for this, right? Where is the code here that generated these twelve tests? And the answer is this is it. There’s nothing more. This code Snippet is all you need.

 

And all that we are really doing here is defining a few details about where the service is, what port is it starting up on, which IP is it starting upon? And we are using this very cool open source tool called Specmatic. And we simply extend from this class and Specmatic takes it over and then reads the contract and generates the test for you. And you have nothing to do. This is all free, essentially. Specmatic generates a test for you for free. There is no other test code aside from what you see on screen. So that’s contract tests. Hari will talk a lot more about this. But before I move on, I have one more thing to show you. I just uncomment this line and says here’s pragmatic generative tests. So now we take this to the next level. We simply set the flag to true run the tests. And just from that we have a lot more tests. And in fact, this time we think failures too. In fact, if you remember, we had twelve tests passing earlier. Now we have 16 and we have 26 failing tests as well. What happened?

 

What has changed? Generator tests. Basically, what we’re saying is not only are we looking at the contract now, we are giving it a much more thorough test. We are generating even more tests. There are positive tests and we are even generating negative tests. So let’s take an example of what the negative test looks like. Let’s see. Why is this test failed? It’s negative, right? You see the word negative over here? Why is it failed? We’re saying key zero is missing at the map, right? No such element.

 

There’s an exception. Is that really the core of the problem? So let’s scroll down past the exception. Let’s scroll to the actual request and see what has happened. We’ve seen this before, right? In one of the cases earlier, post to slash product slash Ten, we’ve seen this payload as well. We are trying to update a product, details of a product. This is new. So as per the contract, the ID is supposed to be a number. The contract does not allow this to be anything other than a number. It does not allow it to be null, for example. And since the contract makes this explicit, SPECT just knows that there could be a potential problem here. And so it says, let’s pass a null and see what happens. And it passes a null. The application obviously tries to read that as a zero. There is no product calls with the ID. Zero throws an exception there, scroll right back down and the exception is not handled. So we throw a 500 and right there is a problem. Because this is a poor way to handle a bad request, you should be throwing a 400 series forex, series error, in fact, in this case, ideally of four to two, which is unprocessable entity.

 

In Http, there are another 25 tests like this. And if you make all these tests pass, you have basically bulletproof your application. So that’s 26 plus 16, that’s around 42 tests generated for you from the contract, completely for free. No code other than what I see on the screen.

 

We’ll show you how to use Specmatic.

 

To do this and other kinds of things, very interesting things as this talk progresses. But now, before we move forward, I think we need to take a step back and set context, understand what the problem is that we are trying to solve, and then we start getting deeper into how we are going to solve it. And for that, I’d hand it over to you, Hari.

 

All right, so let’s quickly get into the context of why this talk and what we’re trying to achieve here. Before that, wasn’t that a pretty awesome demo by Joel, practically taking open API specifications and being able to generate tests for free? That’s really interesting, given that we had to write no code whatsoever. Why is that all important and why is this all the more relevant, given the widespread adoption of microservices? And that’s what we’re going to take a look at now. So, to set the context, let’s imagine that we’re building a mobile application which requests product details from a service, and then the service responds with those details, and the mobile app displays the same fairly straightforward. Right now, the application requesting the data is the consumer and the service responding with the data, let’s call it the provider. With that terminology out of the way, let’s think about how we’d go about building this consumer. Right? Let’s say I am the mobile application developer and I have a dependency on the actual provider. One way for me to build this application for the mobile app is to wait for the provider to become available, and only after it is available, I’ll use it as a reference and build it right now this is the sequential style of development and not really productive in terms of pushing features out.

 

So what is the typical widely accepted solution to this? I could stand up a mock server to emulate the provider so I can make independent progress on consumer application development. Right? This looks good on paper. However, there’s a fundamental issue here, right? The mock that I’m hand rolling in order to sort of emulate the provider need not be truly representative of the actual provider. Now that is a big problem, right? How is that a big problem? Let’s take a look. Now, as a consumer application developer, I may be wrongly assuming that I could send a string for the product ID while the actual service is expecting an integer. And likewise the provider may be returning a name and an SKU while I am wrongly assuming again that it’s going to give me back the name and the price. What does that lead to? When I deploy the consumer application, the mobile app alongside the real provider, integration is going to be broken. And that’s bad, right? And what’s worse, such issues, we know we cannot find it on our local machine because obviously, like what we saw, we are dependent on a handrolled mock. So we are not really getting any feedback from the real provider.

 

However, this same situation continues even in the continuous integration environment, right? Because there as well we are using a handroid mock, possibly correct. And for the provider it is not a very different story. Provider also does not have an emulation of the consumer, so even the provider application engineers may be developing this application in isolation, right? So the first instance where you realize such an issue exists is when you deploy both these components to environments such as integration testing and then you realize there is a compatibility issue. Now, this is a double whammy of an issue, right? Because not only does it compromise your integration testing environment, it also blocks your path to production, which means you have unhappy users. Now, this is not a good place to be. There’s more bad news. So the heat map at the bottom kind of represents the cost of fixing issues alongside the timeline and the later in the environment you’re finding them. So an issue found very much on the left in your local environment is very quick to fix, right. When you compare it to this compatibility issue which is being found out only in integration or worse, in production, it’s going to cost you in terms of user experience and also it’s going to cost you in terms of resolution time, which is not a good idea.

 

So what would we like to do? Integration testing, like we all know, is not a panica to any of this problem statement, right? What we’d like to do is to be able to shift left the identification of such compatibility issues and thereby try and eliminate integration testing altogether. Now, is this even possible to find out compatibility issues without integration testing? Let’s find out. I’ll hand it over to Joel to show us the same. Over to you, Joel.

 

All right. Thanks, Hari. Well, as Hari said, we are going to now learn how to shift left to make sure that we can identify these compatibility issues right on our laptops without having to get into integration testing. So, stepping back, a lot of the issues that we have seen previously between consumer and provider are perhaps they are caused by a communication gap. It might be because they are documenting the APIs using something like Word or Excel or even worse, perhaps word of mouth. So would it not be much better, much nicer, if we could capture every detail of the API, including the headers and the request, the types, the payloads, the keys, the data types? Is the key optional? Is the key mandatory? Is it nullable? Is it a string?

 

So on and so forth.

 

All of that rich detail. If we could capture that somehow in an open, industry accepted, industry standard, widely understood specification format, what if we could do that? The good news is these formats exist. And for rest, the most widely used one is open API. And you can put all this down in an open API specification. And if the consumer and provider actually have the specification, then they can get on the same page about what the form of the API will be. And that should go a long way towards improving their integration and reducing integration issues. Or would it? Because an API specification is good, it’s a step up, but it’s still a description. It doesn’t actually force you to do anything. So it doesn’t really force the provider to implement the API as per the specification. And it doesn’t really force the consumer to have the right expectations about the shape of the API, which should be as per the specification. So we need something more than that. We need to go beyond we need an executable contract. We need a contract to enforce the API specifications. Now, let’s talk about that first. The good news is API specifications is something that we do have.

 

It contains all the details that need to be enforced. Everything is there. What if we can turn the API specifications themselves into executable contracts? Would that not actually solve the problem? Right. And that’s exactly what we are going to show you. How do you turn API specifications into executable contracts, both for the provider and the consumer, to ensure that the two of them continue to match the contract? And how does that work? The tool that we demonstrated at the start of the stock, as I said, is an open source tool called Specmatic. Specmatic will take the open API specification and for the consumer it generates a contract as stub, or what we also call smart marks. And using this, it is possible for the consumer to make sure that their expectations about the form of the API matches and is faithful to the contract. And thus the API, the consumer can.

 

Stay in sync with the contract.

 

Now, having done this on the consumer side, we need something for the provider side as well. For that, specmatic again takes the Open API specification, the same Open API specification, and runs contract as test. With contract as test, it becomes possible for the provider to faithfully simulate the kinds of requests that a consumer would send as per the contract. And then when it returns a response. As I’ve shown you earlier, the contract test actually checks whether the response will be understood by a consumer. And so this is reciprocal. If a consumer stays in sync with the contract using contract as stub, and if the provider stays in sync with the contract using contract as test, now they will integrate when they get into an environment. In fact, not only that, it becomes possible for them to work independently, because there is no need now for a consumer to wait for the provider to be ready to start work. The consumer can just use the contract and use a contract as stub and start off. They have something they can invoke. The provider doesn’t need for the consumer to be ready to be sure that they’re going to integrate.

 

They can just start off with the contract tests and know that that’s going to be a faithful simulation of how actual consumers that are contracted during will work with the provider. So let me actually get now deeper into the consumer side of the story. How does the consumer emulate the provider using Smartmocks? And for this piece, I am going to run a small exercise. And I think this will be helpful because it will get you actually hands on and you can see for yourself by actually learning, by doing. You can hurry. I would also like you to post these details on the chat. You can download the Open API specification sample from here. You can import this file into postman as a collection. This is one of the nice things about having a specification format is that as it’s a format, it’s machine readable and suddenly it means that your tools, other tools, can understand it. Then let’s download and set up schematic from here. Just put the two of them in the same directory. This will help you to make things a little more smooth. Then you see you into the download location and run this command.

 

This is a java tool. So by any chance, if you don’t have Java, don’t worry about it. I will be doing all of this on screen. If you have Java and would like to follow along, you can follow along from here.

 

All right. Okay. Yeah, I guess Joel, you can.

 

So before we begin the exercise, let’s take a quick look at the specification. And now I think after seeing the demo, which I did at the beginning, this will be a little familiar. We see the same products API here, we see the Get and this time the response is in line. So that means when you fire a Get request against Products, what we are expecting to see is this response in return. And the response should contain a JSON object with name and SKU, right? Both of them are strings. So the first thing that we are going to do is start this up as a stub. For that we are going to say Java minus jar Specmatic jar stub products API. Yava. Let’s run this and see what we see. Now my output might look a little different from yours, but the most important thing is that you get this line at the end. Stop server is running on http NEIP 9000 control C to Stop, right? And once you have that and this command is posted as well in the chat, so I think you should be able to run it from there. Once you have this, the next thing you can do is step into Postman and you can do the same thing with Curl.

 

But I’m going to do the demo using Postman because I think most people should have it. We’re going to import this contract, right? Let’s start off with that. Let’s import the contract. Boom. Postman understands it, which is awesome. Import. Now we have a sample products API. We have get products and postman is set up. So essentially let’s change this base URL. I’m going to change it to keep things simple. Local Post 9000 to slash Products ID colon ID. Here is Postman syntax for a variable. The variable is declared below path is the path of variables and Postman has generated something random. Let’s put in a value of our own and let’s hit send, see what that gives us. Get a response with some random values. What’s happening here? So basically, Specmatic has understood products one. It knows what that means because this is in the specification, but we have not yet told it what to do with Slash Product One. And so it helpfully goes back to the specification, looks at the format of the response, generates a response and returns it. And this is basically a randomly generated response with the right keys just to prove that it’s possible to tell Specmatic what to do.

 

Send Five and returns batteries with this is a preset value. I told Specmatic how to respond to Five. That’s something that we will see later, a little bit later. Has everyone followed with me so far? Has everyone got to this point? After this I’m going to take it onto something a little further, basically. So I’m not going to ask everyone to follow along, but I just like to see that you’ve reached this point and you know how to set up Specmatic and invoke it and pass it a contract.

 

Okay, it’s all great. Okay, nice. Thanks for the feedback.

 

Yeah, go ahead.

 

Thanks.

 

So I’m going to take it further now. I’m going to do a lot more demos, and in the interests of time, I’m just going to show you how this works. But I think what you can always do is get your hands on an open API specification. And these things are all very easy to try out. We will share with you the documentation for doing this later. But for now, I’m going to kill this and I’m going to switch to another window with a slightly more involved contract, where we are going to try out a few more things. So you can see this is a very similar contract. In fact, this piece is exactly the same. And I’ve added on a small API here. Here. The second API is for adding a product. So slash products, you post an object to it and you add a product. So let’s see what we can do. Right, we’re going to say Specmatic stub products API. Right? You’ve seen this command before. This time I’m not saying Java minus Java, because I’ve set up Specmatic as an alias. So the command just works and we start the stub off.

 

And now we try five. And you see the randomized response. Because this time I’ve started up a new instance of Specmatic and I haven’t told Specmatic what to do with the value five. Well, let’s do that. Now we’ll see how we do this. So first I create a directory. All right. Inside the directory, I’ll create a file. Call it stop JC inside this file. So Specmatic is reacting and it knows that there’s something wrong with the file, so it hasn’t loaded it inside the file. I will put this content kept in a snippet there. And I will just change this to BBC one, two, three. Save it. I’ll just give this quick restart. There you go. Specmatic has read and accepted this file. Let’s fire a request here. Specmatic now returns batteries ABC One to three. All right, so first question is, what if I want to stub out multiple things? How do I do that? Let’s say that it’s not only batteries, but also torches. So we’ll call this torches. We’ll rename this to batteries in torches. I guess we should have a different product ID. Make that ten torches. Save it. Five reacts with batteries, ten reacts with torches.

 

Of course, we forgot to change the SKU, right? It’s ABC One to three. And if I send it with five, it still says ABC One to Three. And I think for the purpose of this test situation, we really don’t care what the value is. So I don’t want to have to think about it. I just say string here. I’m just telling Specmatic, you return some random value, leave me out of it. You figure it out. I don’t care what you return, just get me a string back. Right, and let’s see now what happens when I make a request to five. Specmatic returned a random value. And if I do ten, Specmatic returns another random value. Right? And this way I can get a load of my head. Let Specmatic do the work that I don’t want to do. And this makes things a little easier. Maybe instead of torches this time, we’ll call it notebooks. And this would be 15. This will be notebooks. And voila. Give that a let’s see what happens when you do 15. Should just work. There you go. Notebooks returned as well. Of course. Now comes the next interesting point.

 

What if we tell Specmatic to stop out? The whole point of doing this was we should not be able to stop out something incorrect. So let’s try to do that, see what happens. We know SKU is supposed to be a string, not a number. So let’s try to stop on a number. Give Spec. There you go. Specmatic immediately tells you response body, SKU. So these are the breadcrumbs. The breadcrumbs basically give you pinpointed feedback about where the problem is. And this is exactly what the problem is. Contract, expected string. But stub contain ten, which is a number, which is fair. Interestingly, we had set this before we made this change. Now Specmatic no more recognizes 15 because it’s rejected it, right? Specmatic will not accept a stub that does not match the contract. So enough of the response. Let’s take this to the request. Let’s say we want to we had an API to create something, right? So let’s see how that would work. We’d say create towel. The create API was a post, as I recall. So there was a post to slash products. The body basically goes in the request this time. This time we are saying towel.

 

We are saying PQR XYZ. And what should the body be here? We’ll have returns with the ten or something. Let’s save it. Let’s see what Specmatic thinks. Maybe we need to give that a restart. Oh, this is interesting. It looks like we got the body wrong. So we are seeing response body, SKU. Contract, expected string, but stop, contain ten. So essentially I stopped this out incorrectly. Let me just quickly check the contract. The contract basically says we return an ID, right? So obviously that was wrong. Going to turn an ID. Let’s make that ten. Let’s hope that we got it right this time. Specmatic will take a second restart. Oh, sorry. This was supposed to be a string. Oops. Oh. The problem is somewhere else. The problem is somewhere else. My bad. I think this is one of the other expectations which we had which we had set up incorrectly. Yeah, we didn’t revert that. And so now we have this problem here. Right. I think we should be all good now. But this is interesting. This is exactly the kind of problem that spectrum is designed to solve, right? Everything is loaded now successfully.

 

Nothing was allowed to fall. Nothing was allowed to fall through the cracks. I missed one thing here. I got it there. Specmatic got this one, I fixed that. I missed this. Specmatic got that. Nothing is allowed to fall through the crack. So now we post products, towel, PQR, exercise. Let’s just take this body and post it, right? I’m going to create a new request here. Let’s post it to http localhost 9000 products. We post the body, we say raw, we say JSON post this. Right? And we send Specmatic accepted, of course, down to the same question. We might not really care about string. We send that that is accepted as well. And once again, let’s try to answer the obvious. What happens if we stub something out here? We know what error we get, right? What happens if we try to do that here as well? Specmatic doesn’t accept it, right? Specmatic won’t even accept the request. Now, this time, the error is not coming from the stub. This time the error is coming from the request. The SKU was incorrect. The SKU was the wrong type. This doesn’t even reach the stub. Specmatic just tells us that the request was wrong.

 

Okay? Now, we’ve spent a good amount of time looking at spectic and even seeing literally on the fly how Mind mistakes during the demo were caught by Specmatic, right? Which is, I think, pretty cool. But we should now talk about workflow tests in a real, more of a real scenario. What if we were to post to one API some data, get back some ID, and then we have to pass that to the stub, right? The thing is, you might have multiple applications. You might have an application which you’re testing, whose ID you get back, and you need to pass that to another ID, to another API, which you need to stop out. And that basically means you’re getting the ID from some application on the flight during the test. And if you’re doing that, what are you going to put in the file? Right? We’ve seen that everything that goes into a stub file has to be known ahead of time. But if I’m going to get an ID in the test from some application and the ID is coming live, I can’t predict what that ID is going to be. But maybe I want to validate that ID.

 

Maybe I want to validate that the request that goes out to the stub has the right ID which just got generated. What do I put in my file? Obviously, the answer is I can’t put anything in the file because I can’t predict it ahead of time. Right? This much is obvious, and obviously I’ve shown you the spectratic restarts. But a restart is not a valid way of running a test. It’s very unpredictable. What we really need to do is we need to be able to tell Specmatic after it has loaded here. We’ve seen how to tell Specmatic before. It loads, right? And when we make a change to the file, it reloads everything. But now we need to tell a running instance of Specmatic what to expect and how to respond. So let’s see how we do that. I’ve shown you let’s come back to this window. I’ve shown you how to stop this out, right? We know how this looks in a file. Well, it’s pretty simple. Specmatic has an API called Set Expectations. Expectations is basically what you tell Specmatic to expect. And when it gets that request, what does it respond with?

 

So we are setting expectations with Specmatic. We are saying when you receive post slash products, this time to something different. Let’s call this a sponge for whatever reason, and we say, PQR, one, two, three, return a response 20. We send this out to the Specmatic and Specmatic returns at 200. This is important because right here when Specmatic response, you know whether your expectations are correct or not. So, to start with, let’s actually see how well this worked. Right? We say sponge and we say PQR, one, two, three. And we get a response back. Oh, sorry, I had a typo there. We get a response back, which is ID 20, which is great. We take this take the next step here. How exactly would spectic react if we try to set something incorrect? So we set an SKU ID, which is a number you’ve seen this before. You know what to expect. But let’s go the whole way. Here you go. Request body SKU. So, basically, Specmatic never lets it go at every stage, be it an expectation, be it a request, be it a response in the expectation, Specmatic will flag it and it will not miss it.

 

And that means you get the early feedback right on your laptop. You don’t need to wait to write a line of code before this happens. So this is how you get early feedback. This is how you can stop something or dynamically. And now we need to actually fit this into a test, right? Let’s try to take all of this and put it together and see how does this actually work with a test. Before we go there, and I show you how that works, let’s quickly review the anatomy of a component test. You’ll typically have a test. You will have, of course, a system under test. And then, since we are stubbing of the API before integration, we have Specmatic. We have a Specmatic stub, which has been given a contract. Now, every test has typically three phases arrange, act, and assert, right? In the arrange phase, in this case, the test will tell spectic what to expect and what to return dynamically. In many cases, Specmatic will validate this with the contract and will return the necessary feedback. We’ll see how this works later. Once this is done, the test will now act. The act section is where the test invokes the consumer.

 

This could be a mobile client. In this diagram, it could be another microservice. Anything that consumes another API is a consumer. The consumer will now hit the API, which in this case is Specmatic. Specmatic will respond. The consumer responds to the test. The test now runs a set of assertions on the response to check whether everything passed or not. And with that as background, let’s just quickly take a look at how the actual test looks. This is how it is. This is the arranged section I just described. This is the URL you have seen before. This is the request format you have seen before. Right. Again, we are stubbing out something like Slash products, gadget, et cetera. This is the response. This is a framework called Karate. Okay. The test is written in Karate. And the key thing is this line on line 36, which I’ve highlighted, is an assertion. We are asserting that Specmatic returns 200. If Specmatic has to reject the API, it will return of 400. And this assertion will break the test right here. You are not even going to waste the time to run the rest of it because you know right here that what the test expects of the downstream API is incorrect.

 

But assuming that Specmatic does accept this expectation and then moves ahead, the test then makes the request to the system under test. This API is the system being tested. And this is a test that is testing a microservice. This microservice basically receives a call. It calls Specmatic downstream Specmatic response, microservice response here. And now the test assets on the response to the microservice. Right. And this is a real world. This is how it would work in the real world on an actual test. Framework doesn’t matter. Specmatic is framework and language agnostic. This is karate. You could do this with anything else, selenium or whatever other framework you name. Right? All right. This has been a deep dive into how it is possible for the consumer to stay in sync with the contract by validating their expectations about how the API is going to look. This is good so far. Now we need to take a turn to the other side and see how it works for the provider. And for that, I’ll turn it over to Hari. All your side.

 

Thanks, Joel. That’s pretty awesome for the deep dive into the consumer side. So now let’s look at the provider side of the story. Right? So if you recollect the image earlier that Joel was sharing of Specmatic, trying to keep the equation balanced on both sides. So for the consumer, you do all the stubbing and the mocking for the provider, we need to do something called contract test. Right. And this is the initial teaser that we did with the free test we were generating. But now I’d like to do something a little bit more interesting. And I’d like to give it a spin in terms of trying to understand what we’re trying to do here. So given you have a specification and you have a system under test, we could generate tests and that much you have already seen. We could use specmatic to leverage the specification as tests and verify that the system is indeed adhering to the specification or not. Now, if this is the case, there is a hypothetical scenario we can think about. What if the provider code does not exist at all and all you have is the specification, right? Then there is something more interesting I can do, which is I have tests, I have no code, which means I could do test first development, right?

 

Which is something on the lines of test driven development. So let’s actually try that out. Would it not be a fun activity to check? So I have a blank application here, which is more like a Springboard Kotlin based application, which I just directly created by going to start spring IO. And it’s an empty shell, right. And this is a command which is essentially going to run the test based on the products API YAML file, which is the specification for this particular app. It’s got one path here for products and it is a fairly straightforward specification. It has only one operation, it does the get. And then for the response you will get back the product details with the name and the SKU. That’s pretty much all we are trying to achieve. So what I’m going to do is start off by directly running the spectrumatic test command. And I’m going to say the app is located at this location, which is localhost port 80 80. Let’s see what happens. Okay, obviously it’s going to fail because we did expect that, right? There is no application to actually respond. So you got a connection refused. So what I’m going to do now is very quickly just start off this app here and let’s run it.

 

And as soon as the app gets going, I’m going to kick off that command again. So the app is running on port 80 80. So now I go back to my terminal terminal and I run the same command again. Now this time around you have a failure, but it’s not like connection refused. You got a 404 and that’s again fairly obvious because the application is running on port 80. But then you don’t have an endpoint to actually support that product ID, right? So why don’t we go ahead and flesh out that piece of the code. So what I’m going to do is quickly paste in some snippet of code here, which I have, and say for this path product ID, like any good developer would do, I’m going to return Hello World to start off with, why not? This is a perfectly valid endpoint that I’m adding to my code. And this time around I don’t want to keep going back to restarting the app and then going to the command line and then running it. It’s getting a little repetitive, right? What you could also do with Specmatic is essentially have contract tests here, right?

 

Essentially. So what I’m going to do here is I have this contract test which extends from Specmatic JUnit support. And for this capability to be available, I have included Specmatic JUnit support into my gradle build file. So I’m going to show you that why I wanted to show you this specifically is I want to highlight that the JUnit support, the Spectrum support, is being added as a test implementation capability only. So which means Specmatic out and out is not going to be part of your production deliverable at all. It’s only during your test infrastructure. Right. So I’ve added this support and on the contract test side, the only pieces of information I have to do is extend this. And then what I was earlier doing on the command line here is to provide the coordinates of the application, which is localhost import 80. 80. It’s very same things I’m going to do here with the system properties during the setup stage. And I’m also starting off the Spring app. And then in the tear down, I close the Spring app. That’s pretty much all. It’s very basic plumbing, no other code. Right. Now, instead of writing executing this command line command, I’m going to instead execute the contract test.

 

Let’s see what happens. So last time we saw the 404 because the URL itself was not supported by the application. Now, this time around, let’s see what is the failure. So you got a test failure now and this time it’s not a 404, it’s still a 200. So why did the test fail? Again, fairly obvious, because I returned a hello world. But the specification file says that we are expecting an object with name and SKU. Right. So obviously I didn’t do a good job. I need to go back and do the next incremental step, which is I’m going to add in the data object here, which is the product itself. Very quickly, let me paste that. Snip it in. And I’m also going to return the actual product itself, which the test is expecting based off of the API specification. Okay. And I quickly kick it off again. Will it pass now? Yes. No, maybe pull down in the chat hooray. We got the green. It’s always a joy, right, when you go from red to green when you are doing test driven development sort of approach. Yeah. So this is fun. We’ve gotten to this point.

 

But then this is not really how the real applications would look like, right? Because you have to think about test data management now. So let’s take a look at that. Now, if you look at the log here for this request, specmatic randomly generated some ID 887 and sent it. And because we’re always hardcoding and responding with product details, the test is always going to pass. So that’s not a real very watertight test. What is realistic is your database might have only one product or test data. You have like two, three rows, and anything else which comes in, it’s not going to be able to fetch. So I’m going to simulate that situation by saying if the product ID is not equal to two. So let’s say that’s the only ID that I have in my test data, right? I’m going to throw a runtime exception. So I’m not entirely flushing out all the repository layer and the service layer and all the TDD layers here. But just for the purpose of this demonstration, I’m quickly trying to say that this line here, line number 14, is sort of representing your repository, which says, I don’t have any other test data.

 

Now let’s kick off this test again and see what happens. Will it pass? Will it fail? And if it fails, what will it fail for? That’s the big question. Okay, there is a null, which is interesting. Let’s scroll down to see what happened. Okay. Oh my. This is a 500 internal server error. And that’s not good news, right? Like ideally we have to handle it. But then first order of business is go from a failing test to a passing test and then we’ll get back to this 500 problem. So what is the issue here? Specmatic is repeatedly sending some random IDs. Last time it sent some other number. This time it is sending 650. But in my test data, I only have ID two. So how do I tell Specmatic that you need to send two and just some random number? So what I am going to do is leverage examples inside of open APS specification. So I’m going to say value two for this particular parameter called ID. And also on the response side, I’m going to say that the name and SKU are these two values. I’ll save that in and then I’m going to kick off the test.

 

That’s the only change I’ve done here. Now, will it pass? I’m going to keep asking you this question, so it’s going to be interesting if you can post in the chat, what are you guessing? So we could almost try padding on this problem, right? Okay. So this time it passed again. It’s a good feeling. We went from red to green to red to green. So we have a good rhythm going here in terms of the TDD. So why did this work? That’s the big question. If you look at what I did, I gave a very specific name to this example on the request site called 200 underscore, okay? And I used the same name down here in the response for the response came out also. Now this is not a capability that’s coming out of open API. Open API has request and then multiple responses, right? I could define 200, 400, 404 to two, so on. But the issue is which request will generate which sort of a response code is not necessarily possible to define within plain open API. So what we’ve done with Specmatic is use this naming convention based mechanism to stitch together examples to say, hey, this kind of request produces this kind of a response.

 

And that’s how Specmatical is able to send it to and glean that, yeah, I got back a book and this SKU, so it makes sense. And that’s how this test passed. Okay, now let’s get back to the original issue, right? Which is it was throwing a 500 and which is not the right thing to do, which means now I need to define how the application should behave in case of a 404 if the ID is not found. So I’m going to quickly paste one more example in which makes sense, right, and I’m going to give it a meaningful name such as 404 not found. And by now you also know the drill because I need to say I need to have a 404 response here, right? I’m going to paste that in also right after here, I’m going to paste in the error schema itself. I’m going to say for the error, I need the timestamp, the status and the path where it happened. So I put in all those good stuff and make sure this is going to work. Obviously, I cannot stop here. I also need to give the example for this situation.

 

So I’m going to put that in right here. I’m going to say this time around, I name it according to the same name in the request. But I’m not really defining what the timestamp should be, what the status should be, because these values will keep changing, right. I cannot hard code it into the example. So which is why I’m doing the same thing which Joel did earlier, right? Which is Specmatic is able to figure out if it is just a string or it can pattern match a data type or it can match a very specific value. So I’m going to say I don’t really care as long as these four attributes for that in the response. Now with that, I’ve modified the specification. So let me run the test again and see what happens. Now, notice how my behavior itself has changed, right? Like practically this time, I am not changing the code, I’m changing the specification first. So I’m almost becoming a spec first developer, right? Like or a test first developer in this case. So in this scenario, what happened? We had one test pass and one test fail. And which was the test that failed?

 

Obviously, the one with the 404, right. It sent a zero. It was expecting a 404, but then it got a 500. Now we need to fix that problem. Now this is easy in springboard. That’s not a very difficult problem. I just need to define an error for 404. I’m going to do that right here. Right? I’m going to define imported. Why is that a problem? I guess that’s a string. Still no problem. Okay. Should be that. Yeah, this can be a little bit of a problem here. And then once we’re done that, instead of throwing a runtime exception this time around, I’ll throw a very specific exception that I’m doing. This is just Springboard way of doing it. But then if you are using any other stack, you could do whatever makes sense there. Now, with that, I’m going to run the app again. I mean, I’m going to run the test again. And will it pass now? And this is a lot of fun, right? Because you’re not really trying to just randomly just sit and write code and figure out whether this is going to work or not. Now, I’m practically writing just enough code to fit the bill for the specification.

 

And every time I make a change, I do make the change in the specification first. Which means I’m thinking about the API design first. Look how it forced me, right? Because I did not have a 404 in the first place. Right? Now, because I did something in the app and I realized I had not designed the error code. So I came over here, I designed my error schema, then I added a four or four response and then built the app. So it’s a nice little feedback loop for me to make sure that I’m pretty much in line with the specification and also building to the specification. And nowhere have I gone off the actual track that I need to be on. And just to call out, I have used Springboot for this demonstration. But then again, specmatic is both language and platform agnostic. It’s just an executable like I showed you earlier. I can run it from command line as well, but just for the purpose of convenience. And since I have it, I’m using the JUnit support for contract testing. Now, this is what we called a traceable at approach, wherein we use the initial specification more like an acceptance criteria and use that to flesh out the initial pieces of the code itself, right?

 

And then from there on, of course, you will need to further flush out this application by putting in your API test and then actually driving the logic forward and the further layers. But this locks in your API signature and that’s the real important part that you need to take away from this exercise. So with that, I will get back to the deck. And one last bit which I want to leave you with on the traceable at Approach is think about API specifications as your done criteria, right? One of the done criteria for your API itself. If you have that, then this sort of an approach to use it as a test for building out your application is not just about testing your application, it’s actually designing your APIs better. So that’s the important piece. So contract testing can often get taken for a testing approach, but if you think about it, it’s largely about API design. Just like TDD itself is not about testing and it’s more about design. Okay? So with that, I will hand it over to Joel to talk about this very, very interesting topic called contract versus contract testing. Over to you, Joel.

 

So let’s quickly review what we’ve seen so far before moving ahead. We have seen how consumers can stay in sync with the specification by using contract as stub to validate their expectations of how the provider API is going to work. We have seen how the provider API can stay in sync with the same specification by using contract tests so that they know exactly how consumers would send requests and they can make sure that their responses will be easy to consume for the consumers. And so, as long as both are doing their due diligence, we know that they will integrate and that they can also both start development independently, securing the knowledge that they are staying in sync with the contract and thereby with each other. So this is good so far. I think we made a lot of progress and seen a lot of things since the start of the stock. The story isn’t over yet, because we haven’t yet seen what happens when we change the contract. Right? Is it possible that there could be some compatibility problem introduced when someone changes a contract? And to understand that, let’s briefly take a step back and understand what we mean by backward compatibility.

 

Essentially, let’s take the example of some provider API. Let’s say you make a change to the provider. This is now an updated provider. You’ve made some changes and you deploy it into some environment. The consumers have not been changed yet. Now the consumer makes some requests to the provider, which is now updated. And all of a sudden the response from the provider is no longer understood by the consumer. What does this mean? It means the provider is no more compatible with the consumer. And we would also say the provider is backward incompatible. Why backward? Because the provider has changed and moved ahead. The consumer has not. So this is overall what we mean by backward compatibility. You could say the provider is now backward incompatible, or you could say the change to the providers are backward incompatible change. And now let’s bring this into the world of contracts. Pop quiz. What if we make changes to the contract in the request? Let’s say you add a mandatory required field. Is this considered backward compatible in the world of contracts? What that means is what we’re really asking is if I make a change to the contract and the provider implements this change, will the new provider still be compatible with existing consumers that have not changed?

 

Right. And if that is not going to be the case, then we consider the changes to the contract to be backward incompatible changes. So to repeat my question in the request if we add a mandatory or required field, is this a backward compatible change? I’m going to open my chat window and I’d like you to post your responses. You can just say yes or no. I’ll just give it about 1015 seconds and just post your thoughts there. Right. Backward compatible. I’ll ask others. I’ll give it a little longer in case anyone else wishes to post as well.

 

Santor says yes. How do the others feel?

 

I’ll leave you to monitor the chat. Hurry and let me know when to go ahead.

 

Sure. Joel, we have one or two responses now. Just waiting for the rest. Just take a guess. This is a very straightforward quiz. We don’t have any curveballs here. The curveballs are yet to come. Yes, no, maybe. Okay. Santosh Kumar says yes. Prashant says no. So we have a fair mixed. Joel, you address the audience on why this is or this is not compatible.

 

Okay? So I think now the time has come to ask for Spec matte’s opinion as well. Let me just quickly adjust this so that I can see over the zoom bar. So let’s make this change here. I have the current contract as it stands today and I am going to make the change in another file. So let’s take this as an example. You’ve seen this API before? I think you’re all familiar with it. We have two files. One is with the current contract, which remains unchanged so that we can compare it. And then there’s another file with the new contract where we are going to make the change. Currently, the two are identical. You have the products API. You’ve seen this before. We are using this to create a product. You are passing a JSON object to it name and SKU. At this stage, SKU is not mandatory. The question was, what if we add a mandatory field? We’ll add a mandatory field, see what happens. We are now going to say Specmatic compare. We’ll say products API current, compare it with products API new and see what the result is. The answer seems to be that this is not a backward compatible change.

 

Why is that? Let’s reason this out for a second. What has happened here? New contract expects key named SKU in the request. But it is missing from the old contract. This means that if a provider had implemented this contract a compliant provider would now expect SKU to definitely be there because it’s mandatory. But if you put this in an environment with existing consumers where SKU was not mandatory then an unchanged consumer may not send SKU. And if it does not send SKU, this would not show up in the request. And this might turn into some exception when the provider looks for it. Maybe in Java that might become a null reference exception in JavaScript that might become undefined. There are various ways people, languages represent a key that does not exist. This key would not exist and would break the provider. And hence this is considered a backward incompatible change. What I’d like to point out before moving ahead is that because I ran this command on my laptop locally, I did not even have to implement a single line of provider code or a single line of consumer code. I didn’t have to get any errors, be it contract test errors, be it integrated system errors.

 

There were no errors at all. I got this just by making a change to the contract and comparing the old with the new. And I got it within seconds, no other code written. So let’s take another example. In the request, if I change an optional nullable to optional non nullable, this what do you think? This is actually also a relatively easy one. What do you think? I’ll ask Hari to monitor the comments and let me know.

 

Sure, I pasted the same question in the chat so you can stare at it for a little longer. Because there are two aspects to it. Optional nullable, optional non nullable.

 

So there’s a truth table. Now you have to think through.

 

Just take a wild guess. It’s okay. I think sometimes it’s also a good idea to just trust your intuition on all of these things. No. Okay, we have no, it’s backward incompatible. Okay, Joel, I guess you can go ahead and break the mystery.

 

So let us try this out. We will revert this back so that it’s the same as before. Now we remove nullable, right? We are going from now SKU is optional. We are going from optional nullable to optional compulsory. Let’s compare the two for those who thought it was backward incompatible. That is correct. The wheels turn a little bit longer for this one, though, because of the fact that there are multiple factors to consider. And I could see that it took a little longer for the first response to come in. And the reason here is that there was a string in the new contract. It was nullable in the old contract. And that means that a new provider which implements this change is only expecting a string, whereas old consumers may be sending null as well. And that could break a provider because they are going to expect a string and get a null right here again, there was no need to write code. And you got this feedback without writing any code in the consumer or provider. You got it for free. Let’s take a third example. What if I changed a schema component that is referenced somewhere in the request and response?

 

So maybe it’s two or three levels deep. Maybe it is there in multiple parts of the hierarchy. Maybe it is there in multiple files. Maybe there are remote references, right? And I want to show you an example of that before we move ahead. What do I mean by this? Something like this? You have a contract here that has several different APIs. I’ll just show you what it looks like. There’s cart inventory, product storage, storage, et cetera. This is an ecommerce contract. It’s a little more involved, a little more real world. There are a lot of different APIs in here, and all of these are different. Let’s say that the change we want to implement is a compliance request. Let’s say that state is no longer nullable. Now we take nullable off. What is the effect? What is the effect of this change to the address field? If I look at the address field, it is in various areas, basically inside the component section. But it is not to be found directly in the request for responses. And so we are going to have to go one by one and figure out where it is. And the first one might be inventory response.

 

Is this a problem? Is it even here? Even here at all? Warehouse info. Okay, warehouse info contains it. Now you think through this and so on, right? I’m going to have to think this through for request and response across multiple different APIs. Maybe I follow through for response. But somewhere if I scroll down, squirrel away devilishly is something in the request. Now I have this one request to think about as well, but somewhere deep, I might have missed it. Complex contracts are like that. Real world contracts are like that. It’s very easy to miss stuff. What’s the guarantee that you’re going to get it right? And this contract, once you start implementation, it’s too late. We want to actually get this feedback even before a single line of developer code is written, right? Application code is written before a developer actually picks this contract up to implement. And a check like this is basically so easy and cheap to run and gives you the feedback right here even before you start development. Why would you not do it right? It’s super easy, super cheap. In fact, you can even integrate this with your CI builds and make sure that backward, incompatible contracts never even make it out the door.

 

In case you miss it on your own laptop, it hits CI. How that works, I think Hari might probably talk a little bit about later. But this is a very powerful tool that makes sure that even the contract so now we have safeguarded the consumer, we have safeguarded the provider, and this way we safeguard the contract as well. A little bit about how we do that. Contract versus contract, right? What? What did we say? Hari made a reference to this at the start. This is a patent pending technique that I’m talking about with Specmatic uses. It turns out that if you start up version two of the contract as a stub, which represents the new provider, and you start up version one of the contract and run it as tests which represent old consumers. If version one’s contract tests pass, it means that the contract changes in version two are backward compatible. Because all the requests generated by version one contract were understood by version two stub. And all the responses that came back from version two stub were understood by version one’s contract. This is a very naive explanation. We’ve done a lot more work under the hood to make this bulletproof tent fast, but this runs within seconds, as you’ve seen, for all contracts.

 

Now, with that, having covered backward compatibility, I’ll hand it over to Hari to talk about the next topic.

 

All right, so quickly going into my slides, so we’ve seen three major concepts, right? Which is contract as test, contract as stub, and contract versus contract testing. And there’s an overarching theme that you would have noticed. It’s a completely no code approach so far, especially the backward compatibility. Just to highlight what Joel was saying, which is all you had to do to experiment with an API change is practically just go about making changes to the specification and then experiment away. And you pretty much get feedback onto whether this is going to be a compatible change or not, right? That’s the overall theme which we have been keeping in mind while we build Specmatic out as a tool. Now, with that, I would like to call your attention to this topic called Central Contract Repo. And why is this so important? And this big question about are we on the same page? And what is the topic on which we need to be on the same page? And why do we need to treat contracts as code? So, okay, let’s take a look at the situation. Now, despite all the hard work we’ve done so far, it’s very much likely that let’s say I am the provider application developer.

 

I made a change to the provider code and then I forget updating the contract or I update the contract, I do all the due diligence there, but then I forget to send the contract over to someone over an email, or upload it to the documentation website or something like that, right? And on the consumer side, I may not be on the most current version of the Open API specification. I could have potentially not have downloaded it. Or maybe I just missed the email from the provider application team. Now, that means both this consumer and provider application teams may be working off of multiple versions of the same specification in their own sources of truth. Which means you are back to square one. We have broken integration. Now, that’s not a pretty picture. What we need is a single source of truth when it comes to the Open API specification, because that’s when you can really leverage the power of having a common agreement or a contract, right? That’s when we started thinking about which is the right place to keep this contracts and where else would you keep it? It is code, right? Contracts are code.

 

Like open APS specification is a YAML file. Why would you keep it anywhere else? The ideal place for it to be is a repository, a version control system. In our case we use Get. We call it the central contract repo. Now if your specification needs to get to the central contract repo, it needs to pass a path of rigor, right? So that could be done through pull request or merge request process. The initial step could be a Linter wherein you just verify that the specification is in line with your organization’s style guide for how you are supposed to build your APS specifications and whatnot. There are tons of tools out there. The ones which we are using right now is something called Stoplight Spectral. It’s a pretty cool tool. And once you’re done with the basic Linting, then comes the important piece which is the backward compatibility testing which Joel just demonstrated. That is really powerful because that’s going to let you figure out if the changes that you are making is a compatible or backward breaking change or not. And only then you can move on to the next step which could be a manual review if at all necessary.

 

And then you can merge the change into the git repository itself. Now, you may ask a question what if the second stage of this pull request process comes out as it’s not backward compatible? What do we do then? It means it’s a signal for a version bump, right? So what we are following as a versioning strategy for our contracts is a semantic versioning. But then again, it’s not the only way of doing it. Why we find it useful is because it makes sense in the context of what we are operating. So if it is a major version upgrade because it’s a backward incompatible change, right, in that case we might go from 10 to 20. And if it’s a backward compatible change but there’s still a behavior change, in that case we might just go from one 10 to one one, which is just a minor version upgrade. Patch version is reserved for only a structural change. So for example, if I’m extracting a mix in schema from the central Open API and pulling it out for reuse purposes only, the structure is changing and the behavior is remaining absolutely the same. In that case we might just have a notional patch upgrade.

 

So this is the strategy we’ve been following. Now, once that is done and you have the central contract rapport with all the contracts living there, the next immediate question is how do I pull it down to the local environment? How does the consumer application engineer pull it down to his or her laptop? And likewise for the provider. Now, that’s where this file called specmatic JSON comes in and that’s a config file that you would have seen earlier that I was using and joel was also using for the teaser. And that’s the one which is going to let specmatic know where to pull the contracts from and what to do with it. And by which I mean I’d like to quickly showcase what this JSON file looks like. So this is the Specmatic JSON config file. It has at the top the coordinates to your repository itself, where your specifications reside, ideally should be only one, right? And then you have two sections. You have test and you have stub. Now, a contract like you’ve already seen, can be leveraged both as a test or as a stub. And it depends on which context you are operating in.

 

If you are operating in the context of a consumer, the contract may be leveraged as a stub. And likewise in the context of a provider, the same contract may be leveraged as a test. So let’s actually take an example of this. So this is central contract repo that we are maintaining as an example. And if you notice here, we have just a few contracts sitting here. And the naming convention for the folder structure is more like a typical package naming convention that you would follow in Java or C Sharp or any other programming language. Because it just makes sense, right, to organize your contracts in the same hierarchy as you would any other code because contract is code. Now, once it is in here, let’s say I have this UI application which is the consumer which needs to pull these contracts and need to emulate the provider. So in this case, I reference the contract sitting there under the Stub category. Notice how this API order v one YAML is used as a stub here. Now, this is the consumer application right now for the provider. This is the provider for the same consumer that you saw earlier here.

 

I won’t be using it as a stub because I need to emulate the consumer. So in this case, the same YAML file is being referenced under the test category of the Specmatic JSON and that’s how you are able to pull it. Since it is Git and Specmatic has been configured with Git, it is always able to pull the latest version, be it your local laptop or be it a CI environment or any other environment it’s running on, there is no question of the contract not being up to date. So that’s the important piece. Now with that out of the way, let’s actually piece all of these things together, right? Contract as test, contract as stub, a contract versus contract. And then you have central contract repository. Now with these four items, how does it all fit into your CI pipeline itself? How do you embrace CDD in the CI? Now you know that the central contract repo has the open API files and Specmatic can pull from there and it can make it available as contract as a server for your local environment of the consumer and also contract as test for the local environment of the provider.

 

This part we’re seeing. Let’s actually take a look at how this pans out for the CI environment. For the consumer in the CI, after you have run the unit test for the consumer, when you’re component testing, you don’t have to look for another tool for stubbing out the provider. You could pretty much leverage the same contractor stub server that you’re using with Specmatic on your local machine on the CI also because it’s just an executable, like I already mentioned, so it can run in any environment. And for the provider side, once you are done with the unit testing, we recommend you run the contract test first before you run your component testing. The reason being it is important that you verify the signature first before you verify the logic. The explanation there is if the signature itself is broken, there’s no point in really testing the logic, right? So that’s the reason for the sequencing. And since at this point you have adhered to the specification on both the consumer and the provider, both in the local and the CI for each of their environments, you can confidently deploy to integration testing and you know for sure it’s going to be compatible with each other, which means you have an unblocked path to production.

 

And from the heat map point of view, which we started this whole slide deck with, you are very much in the green, right? And you are very much in the left. So we are able to successfully shift left the identification of compatibility issues to the local and the CI, thereby keeping your higher environments always viable for testing such as workflow and whatnot, instead of always being stuck with integration testing. Now that’s how Specmatic is able to help you leverage API specifications as executable contracts so that you can make independent progress on both building your consumer and your provider applications and confidently deploy them independently, while you can be sure that they are going to play along well with each other when deployed together in a higher environment. I’d like to thank all of you for joining and being a very patient audience. We’d love to hear your feedback. And this is an open source project, so feel free to try it out. And if you find any issues or need help, do file bugs and we’d be more than happy to look at it. Thank you.

 

Okay, thank you so much everyone, for attending this session. Special thanks to you, Aria. The session was really very well planned. The demos and everything were really well. So thank you so much for everything.