End-To-End Testing with Cypress

Captions provided by White Coat Captioning (https://whitecoatcaptioning.com/). Communication Access Realtime Translation (CART) is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings.

JASON: Hello everybody and welcome to another episode of Learn With Jason where we're going to learn something new in 90 minutes. Today on the show we have Dr. Gleb Bahmutov. Thank you so much for joining us!

GLEB: Hi, Jason. It's good to be here.

JASON: I'm super excited to have you on the show. I remember we met for the first time way back at a testing contest.

GLEB: Assert JS.

JASON: That's right and I've been following Cypress ever since and I have been blown away how much more approachable it uses to write tests using something like Cypress. Came from IBM where you wrote a lot of big mocha assertion libraries and everything was hidden in Jenkins and you didn't know what was going on, right? And using Cypress always kind of feels like a breath of fresh air. So let's -- before we type about Cypress actually, I would love to talk about you. For those of us who are not familiar with your work, do you mind giving us a background?

GLEB: My background is right here.

I will not give you a background. Finding that virtual background in high def resolution was a hassle. Jason, it was a pleasure to actually watch you talk. I came to software quality in graduate. I've den network layers, I've done computer graphics with my degree -- that was my fault In every step in school, in my career, I've always loved something I could understand, come back later, modify something. How do I test with, how do you factor so progressively after a career in computer vision, I joined a company. Where I was working to run inside the browser. I was like this web thing actually can do everything, right? One of the things I love about Cypress but everything is in Javascript mode. And our program C++, every language probably. But having a single language full stack and all the tools implemented gives me tremendous power. I could literally fix everything. You don't like something in Cypress, go ahead and change it. Maybe write a plugin. Maybe change your task. It's all the same language. The knowledge, the tools. So after a while I joined other startups and when I started using Cypress in beta program four years ago, I had same feeling that you probably had right now, it's approachable. It makes writing a task a joy. After a while we talked about ideas and I joined Cypress, been for three years at the company, could not be happier. It's absolutely incredible writing open source tools for developers. I was incredible, lucky

JASON: Very, very, cool. Hows the quality? I'm getting weird notices over here. I'm worried I'm dropping too many frames I want to check that I don't need to kick my router real quick. Did Chad even hear us? Did we get lost? Give me just a second. I'm going to unplug and replug the Ethernet cable. I've got it on a dual connection but I think my Ethernet cable gave out. One second.

GLEB: Absolutely.

JASON: Okay. Hopefully that should fix it. Please keep me in informed if that starts to lag again. Also you may have noticed the buffer is back. I'm in my little felt box. Because I got a bunch of crap about my audio sounding like fazie because I'm in a open hard floor room so I'm back in my box.
So for those of us who are new to testing, so I talked a little about how amazing Cypress has felt to use in comparison to other testing tools. But for a lot of us I think we look at testing as something we have to do, not something we want to do. It feels like being told to eat your vegetables. And I wanted to talk about like how you get past that. Like what are you seeing in teams or how are you helping encourage teams to move past that feeling of, like, uh, testing is a thing we have to do before we can ship and starting to see it as such a power-up in the work flow?

GLEB: I think people change their attitude towards testing when they see it not as time synch instead as a tool for productivity and a tool for advancement and a tool to deliver the features faster and advance their careers. So just to give you an example, many people think I'll work on a feature and allocate one day for testing at the end. And what usually happens? Well, it falls off. You either rush it or you never do any testing and what do you have? You have a feature that you're not sure works, right? Or works incorrectly. Or the tests are incredibly brittle. Because you rush and you're trying to just write them at the end. And to me when you understand writing the tests and writing the feature has to go hand in hand. Once you reach a little core, a little nucleus of functionality now you're safe and now you build the second step and you build the first one. I don't say always doing testing in development. I just say split the big feature into smaller chunks and keep testing in parallel. When you say it's hard to understand, it's hard to test the architecture in a way but it's hard to understand. But it means you should be refactor your source code to make it more testable. The long-term payoff when you have to come back one year, half year later, you'll be able to add a feature and fix a bug.

JASON: I feel like that's the thing that has been the biggest the part that seemed to click for me, I wrote testable code, I walked away and came back months later. This was just me, I wasn't on a team. I came back I was like what does this code do and I ran the tests. When I made changes I was confident that I wasn't accidentally breaking something in a file that I hadn't thought about in nine months because my tests all passed. And I think that's where I really started to understand like holy crappy should be doing this all the time. Whether it's just me, whether it's a whole team. And I am 100% guilty of not testing all the time. If you look at learnwithJason.dev, it's not tested at all. But what I notice is when I do it, I'm always faster on those projects.

GLEB: Absolutely. It might slow you down at first. Like the first day. If you have to bring tools, maybe learn something new to start testing right away. But it definitely spits you up ten times and the more you go with passing time, the more boost it gives you, day by day. And especially -- forecast your own code. Would you want to take over someone's code that doesn't have any test? We all say greenfield project. I can take over project and it would be a brownfield of abandoned code, unknown quality, unknown features, unknown mine fields, or I can take over a lush garden with features, tests, and I know if I refactor something it will keep growing and blooming. I'm sorry for all these vegetable analogies.

JASON: Whenever we can talk about food in any sense here, I'm pretty happy.

Thank you for the subfeel. I think Robert subscribed before we even started. Roberto. Married With Children. Your background has been recognized.

GLEB: Busted.

JASON: Let's go ahead. Let's dive in and build something. I think what would be fun to do today and something we like to do on the show is to start with an empty folder and see what we can do. Let me switch over into programming mode. We're going to be using this is the Cypress website so let me throw that over here. And so let's yeah. Let's dig in. If you wanted to start a new project, where would you start? Let --

GLEB: How do you usually start? Do you start with a repo? I usually start with an empty folder and empty repository so GitHub all the time.

JASON: Same here. We'll do Cypress testing and we'll move into that and I'm going to get into it. I'll get create and we'll make the repo on GitHub right now.

GLEB: Is that like a GitHub CLI that you use?

JASON: This is the hub CLI. I need to switch over to GitHub CLI, I just. Done it yet. Now that I've got that, just open it.

GLEB: Do you want to empty it?

GLEB: What's the teach command.

JASON: It's an alias that strips away some of the tool tips and some of the extra UI so it's a more simplified bigger font, that kind of stuff. So then I think that's all we're going need for now. I guess I can make sure -- from here, we're set. We've got ourselves an empty folder. So I think maybe what we want to do is start is we can just have a like a site folder. Or we can call this -- we'll call it like public.

GLEB: Right. So it's like a static site.

JASON: Yeah. I meant to call this file and we'll give ourselves a folder called public. All right. And inside here I can like make a basic --

GLEB: Did you share the code when you create the file -- it creates a folder if it's correct.

JASON: Yeah, it's amazing. So you can just do like folder and it will just create all those folders for you which is absolutely magical. My problem is sometimes I try to click buttons and I can never tell the difference between these two and I click the wrong one. We're going to do Cypress testing on Jamstack site. Okay.
I've heard a lot of times you should write the test before you write the code. Is Cypress the kind of -- is what we're doing today something you would consider being test-driven development or how would you go about doing this?

GLEB: You can do both. Why don't we start test-driven development. Why don't we write an end-to-end test first.

JASON: Cool.

GLEB: You have to install Cypress, right?

JASON: Let's do it. I'm going to MPM install Cypress.

GLEB: As a dev dependency.

JASON: So dash D. Do I need anything else?

GLEB: Not right now, I think.

JASON: Perfect.

GLEB: And you will get the latest version of Cypress, 4.6.0 that just came out yesterday.

JASON: Excellent. So this is something that I've always kind of -- so we're downloading the Cypress node module and then it also pulls down a version of Cypress, and this gets stored locally, is that right?

GLEB: Yes, it is stored in your user profile so you only download 4.6.0 once and it's shared between all your projects but uses that version of Cypress.

JASON: That's to make it faster so you don't have to do the full download on every project.

GLEB: You saw how long it took. You really want to cache all the tools locally on CI. That's important.

JASON: So now we've got Cypress. Here's Cypress and now we actually, we don't need -- we can make this into a thing. So would I just run like --

GLEB: You would do Cypress run.

JASON: Okay. So now let's make this work.

GLEB: Before we do that, Cypress needs a spec file. It needs to know what are you testing. I suggest you run MPX Cypress open which is interactive mode where you will see Cypress running open. Cypress open is the first time you want to run it, that's what you would use.

JASON: Okay. So chat, have any of you used Cypress before? And actually here's the better question. Let us know if you have done testing at all. What are you doing for testing and what have you -- what have you used? What tools are you using? Okay. So here is -- we've got just examples.

GLEB: The first time you run it it shows here's what you can do. The most important thing, you see the Chrome button on the top right corner of Cypress?

JASON: Uh-huh.

GLEB: That's where you can select the browser. It looks around you have Firefox, you have Chrome already installed locally. So I can run those. So you can pick whatever browser you want.

JASON: Very cool. Yeah, that's awesome. And we're seeing, yeah, the chat. This is for a lot of folks in the chat today, this is their first time writing test. A handful of people have tried Cypress. A lot of jest. Jest is good. Good stuff.

GLEB: We can definitely even delete the folder so it doesn't accidentally run because we want to write a test for our own page so Cypress integration and it's called example.

JASON: Delete examples.

GLEB: Probably under integration you probably want to start a new file. And call it test spec or whatever you prefer.

JASON: Does dash or dot?

GLEB: It doesn't matter. Cypress will consider every integration.

JASON: That makes sense. And now it automatically picked that up too which is really cool.

GLEB: And if you run it now it will say could not find any specs or any tests in that file.

JASON: This is so cool.

GLEB: It already shows you hey, this will be probably something on the right. And command log on the left.

JASON: Yeah. This is really, really cool. Because like -- and what I love about this, and we'll see more as we go. But check out here on the left, this is like a Cypress window.

GLEB: Yes.

JASON: And this is like eventually will be our website. So like now it's not testing where we have to just trust that the computers are doing what we expected them to do. We get to watch it happen.

GLEB: Let's watch it happen, Jason.

JASON: Let's watch it happen.

GLEB: Let's write the test.

JASON: We're going to write our first test. Before we do that let's talk about what our folders mean. We've got four folders that came in with Cypress here and we've got a Cypress JSON and our Cypress JSON is just empty if we want to modify.

GLEB: That's where you can set global settings, maybe basic settings. Make you want mass spec files. Right now you don't need to touch them.

JASON: So then we have probably the like we can talk about -- so this index file, it looks like doesn't do anything -- if we type config, dot. It usually comes with Cypress JSON file. But this is where you can modify. But plugins file allows you to change the behavior of Cypress in many ways. Change for configure, register event handlers. It's a file that you don't have to touch by default until you probably use Cypress for a month.

JASON: So we know we don't need to worry about that one. We won't change it, close it out of there. Not that one. This one, plugins. And then in support --

GLEB: Cypress allows you, it has a big API of commands, like click, type, everything is bundled in. You don't have to install anything to start. But let's say you want to add your own commands or you want to bring third party commands like testing library. We need to add code.

GLEB: So you can install additional packages, additional plugins for Cypress and load them inside the support file. Literally what will happen is each file will be loaded before each spec. This file will be concatenated to run before your spec file so you can log additional things.

JASON: Got it. Okay, cool. I'm reloading this page because I want -- it looks like our bot stopped cooperating. Let's see if it's going to cooperate now. There we go. It's back. Now we should be able to play the game, too. There it is. Look at it. By the way, this little floppy thing is a gift from Cassidy Williams. It is completely ridiculous. Did it quit on us already? I've got to figure out what's going on with that. I think I bugged it. Whatever.
So let's focus up. So we are going to -- we've got our commands but we're not going to use those today, I don't think.

GLEB: No. And the last folder is fixtures, right? That's where you can throw all your test fixtures that you want to load and use during the test.

GLEB: We have a special command loading fixtures. You can use them to stop network requests by just saying load the fixture and use it as a network response so you don't have to run a server, for example.

JASON: Got it. Perfect. We might need that today but let's start by just we're going to write this test. We've got our test spec and it's an empty folder. So if we want to start, what's our first step?

GLEB: We use mocha test runner under the hood. We would start with a function called it and you give an argument for it and then the name of a test. Loads or something. We're describing the page in this case.

JASON: Loads the home page. And then we pass function.

GLEB: Yes. And that's where all Cypress commands will go. All commands in Cypress are chained over the global site. Like lowercase CY site. If you write cy, let's say visit. So if you want to visit the URL, and you can say index HTML. We only have index HTML, you don't have to do the slash. It will read it I believe.

JASON: I get it, okay.

GLEB: If you hover over site or visit, there is no information.

JASON: Right.

GLEB: So can you copy with the reference line comment from the plugins file that you showed before? Whole line. If you copy, this is the best thing about modern text editors. You tell them load type for Cypress in your modules. Just hovering.

JASON: I think I turned that off. Now we can see here all these types when we do that.

GLEB: Every command comes with a little snippet, an assertion and a link to the page.

JASON: This is nice.

GLEB: So you can see how we're supposed to use something.

JASON: That's really, really nice. So now we're visiting the home page. But this isn't really a test, but I think we can run it, right?

GLEB: Wait, wait. You already have Chrome open, right? In a second window. You just have to find it.

JASON: Okay. So failed trying to load our index.

GLEB: Okay, one second let's look at the error message. Here's one thing that I want to show you. See how it shows you where it failed? Can you click on the blue link right above it? And say visual studio code. Yeah.

JASON: It opened my other non -- this is me with my special profile. For you this will just open the right way.

GLEB: But you see how it knew where the failing command was.

JASON: That's so cool.

GLEB: You go right there.

JASON: This is really nice. Okay. So that's amazing. So we just do that, okay. So then if I want to fix this, because it looks like it did look in the right place.

GLEB: Is it looking at the right place?

JASON: I think so.

GLEB: Is it? Is it?

JASON: Let's find out. It's not. Because I put it in public.

GLEB: Exactly. So I think the error message should give you plenty of information. This is one of the frustrating parts of testing is that you get an error stack sometimes and you have no idea where the error failed. Like what is it trying to do and how do you fix it. So we were working really hard right now on error improvements. We always had errors but here's what you did, here's how to fix this. So hopefully it's easier to figure out.

JASON: Yeah. What a powerful -- and it's such a level up too to have that sort of thing. Okay.
So now that we're visiting -- how do I tell it where to look for my -- Well, why don't you say public slash HTML. And if you save it --

JASON: Hey, look at that.

GLEB: It's washing your files for you. Second monitor, you work with your source code, specs, and just watch it pass.

JASON: This will be kind of squishy but we can make it work. Maybe I can make this even a bit smaller.

GLEB: Now you can actually work on the index HTML, right?

JASON: Okay. Let's open index.html and we'll add a thing and say this page is tested by Cypress. Okay. So then I've saved it. Now is this going to hot reload or anything?

GLEB: No. Because it's not reloading on your source files. It only auto loads on your desk files. There are plugins that will auto load by watching your folder by stock it just watches -- now a great command but will text if a text is there is contains. And you can give part of a text with your co-peers.

JASON: Look at that. Okay. But also check this out. Now our tests are passing. We now know that we are loading the right page.

GLEB: And notice when you hover on contains in a command log right here, it shows you which element. You know you're not finding a wrong element accidentally.

JASON: That's super cool. That's nice. And this is where I think it starts to be cool is we had like very minor hurdles to clear here before we were able to immediately start seeing this work. Which I feel was not the case when I was trying to set up Jenkins end-to-end testing. Jenkins is powerful. There's a reason it's the enterprise CI. But it is not fast. The setup on Jenkins is not quick.

GLEB: I have to say -- I switched jobs maybe six times in my career and every time you're doing a job, you have to -- I don't want to say prove yourself. But maybe set yourself on the right path. And I was looking back at my career like what did I do to set myself right. And at one company I install century, I brought crash reporting analytics they didn't have it and immediately we found thousands of errors and we start fixing them of course. And my stock professionally went up because all of a sudden this new thing became available. And I was thinking lately that if you starting a new job, you should do two things. First, work hard. Just because you have so much to learn. If you work hard at least initially for a couple of weeks, you will have greater protection. But second, because everyone, many people don't like testing. If you come in and you write good end-to-end tests, you take care of such a big problem that nobody wants to touch, and all of a sudden you'll be something who sold this huge thing. I strongly believe that if you engineer working in Javascript, working with application, write a few end-to-end tests and create a whole CI pipeline it will give you so many cookie points it's not even funny.

JASON: I want that on a T-shirt. Work hard and write tests on a T-shirt.

GLEB: That's a good T-shirt.

JASON: Is this great. So now we have -- I've got my spec file and my index.html. So I want to do -- let's say we want to add another page. So I want to -- let's see. Read more on the create an about page.

GLEB: Yeah. All we're doing is making a static site generator here, right?

JASON: Uh-huh. Let me make this wrap. If I can remember how to do that. So now when we save, if I go in here, we now have -- this is really tiny text but it just says you can read more on the about page. So that means I need to create an about page. And so I can just -- let's go here. I'll save this as public, about.html. And then we'll do the same thing here. Actually let me just copy this one over. We'll just say about. And we can say -- we can go back to the home page. This is the other thing. This is a browser. This is just a normal browser.

GLEB: Yeah, open dev tools. Whatever you normally do.

JASON: We can go in here, we can inspect play around and mess with the styles.

GLEB: One other thing. If you switch to console here in dev tools and click on any command in Cypress command leg on the left, anything will print there so you'll see outline the details.

JASON: So it shows what we found, how many we found. That's powerful. That's really powerful. Question in the chat about other browsers? We have a drop-down for that. Check this out. I can go over here and I can say let's run fire fox.

GLEB: We support any Chrome-based browser plus Firefox. Whatever you have preinstalled, it's there. By default Cypress comes bundled with electron. It used to be a big problem because it was behind in our installation but now it's pretty much up to date with stable Chrome. I consider both --

JASON: This is really nice. Can Cypress test the app in several browsers at the same time?

GLEB: If you're running Cypress in several containers. If you think about even -- you can run Cypress headlessly in parallel. But driving browser at full speed with extra overhead that Cypress does because it drops every command probably will tax your machine and at some point you will see oh, I slowed down now. The process will just crawl. But if you run it on CI, spin multiple containers, no problems.

JASON: Yeah. When we run Cypress run, can we tell it to use multiple browsers?

GLEB: Yes, you can say Cypress run browser Firefox and it can be in Firefox or you can use Chrome -- headless. What you want to do.

GLEB: By the way, why don't we expand this test and after we check the about page, when we can show Cypress run locally. Because you will see a nice vice there.

JASON: Yeah let's do it. I've got my test spec here. And so I want to links to -- let's see if I can do this from my own memory.

GLEB: Jason can I say something about this?

JASON: Of course.

GLEB: When you write uni test it's nice whenever test is short. Arrange at sort. You only check one thing. Because when you run a uni test and it fails usually it's in the terminal because the failure itself you know which is broken because there's one assertion. With Cypress I don't think it's the best strategy. I think your test should be like user story. So you can actually add more commands. Because you will see in a second why you will not have problem understanding if something fails, even in a longer end-to-end test.

JASON: Okay.

GLEB: Here I would say test the whole site. Right now it's quite small. So why write multiple tests.

JASON: Okay. So then I want to -- let's see what I can do. Here's a click. So what does click do?

GLEB: You don't have intellisense on hover.

JASON: I think it's because I'm in this -- I go here, this is what I want to see. And it's down at the bottom now. That's really frustrating. Maybe if I make this bigger it will work again? This is what I want. So I need the cy.get my button and then I can click it. I can do that. I'm going to get my anchor. We've only got one of those. But I'm already seeing something that I would want to control here is I would need to know how to get this, right? So I would have like a class of like about link.

GLEB: Some people use data attributes specifically for testing. Like data testing. It's up to you.

GLEB: You would do data testing. Let's data test for now.

JASON: No other.

GLEB: Save your test spec so we actually see the updated page. It doesn't really matter right now I want to show you something else you might appreciate. Right now this is failing because we don't have about link so it shows you failure. If you go to the browser and you see this target next to the URL, scroll a little bit higher, in the I frame. A command R on this page.

JASON: Here it is.

GLEB: Right now when it fails.

JASON: My window is so small.

GLEB: Yeah, this one. You have a target button just click on it. And now kind of hover over the link.

JASON: Oh, look at that.

GLEB: Copy it and paste in your file. So we have this selected playground that allows you to pick based on the preference.

JASON: And now we can see it's already doing what we expected.

GLEB: Exactly, yes.

JASON: Okay. So that's really slick. So we've got that. And then we would do the same thing, right? Like cy get H1.contains.

GLEB: You can combine and say H1 text. It will take two arguments. By default it thinks it's just text.

JASON: Like this?

GLEB: Yes.

JASON: That's even easier. It makes me want to refactor this one too. Okay. So that's all doing what I want.

GLEB: Now you have a full test. And I wanted to show Cypress run if you don't mind. So let's change contains on line 9 to something that's wrong. Or change like make it fail, pretty much.

JASON: Okay.

GLEB: Yeah. It tries to find maybe your text hasn't updated yet.

JASON: Okay. That's wrong now. So now I want to quit this.

GLEB: Yes, quit it.

JASON: And just run my test?

GLEB: Yes. Instead of Cypress open graphical UI, this runs headlessly by default. This is what you do on your CI. I found what spec, it's running it right now. Says the same error. But now you just scroll up you'll see additional things right. After the error, and it's pretty much the same.

JASON: So here's the name of the test we wrote. We know that one is failing. Timed out retrying, expected to find the content wrong within the selector H1 and it never did. That's very clear. And then we can see here we've got one test. 4 seconds. What's this? Okay. So inside of our Cypress screen shots.

GLEB: The name of a spec file in the test.

JASON: Let's go look at this. Screen shots. Holy crap.

GLEB: That's why I said write longer tests even if you fail somewhere in the middle you will see exactly why. You don't have to split a test artificially to keep them short.

JASON: Yeah. That's really cool. Brad you asked us to do something again and I'm not sure what you were asking us to do again. So if you want to call out what you were looking for, we can do it again.
And cell 92, we are about to show, I think, how to get into some asynch functions, right?

GLEB: Before we do that, Jason. Look at your screen shots. You will have one more link you will like.

JASON: Whoa. Videos. I would like to open this in the browser.

GLEB: Probably internal.

JASON: Let me open up a finder window. That's cool.

GLEB: Every platform by default, you don't have to install anything else, it will generate you a video by default. Think CI test fails, debugging, just look at the video. Find previously passing test run and look at the video just like side by side, where did they diverge, where did they go wrong.

JASON: Got it. So we got asked to do the selector playground. When we run the expect we end up out here. And then anywhere that we are, let's see. I think it's because my window is really tiny.

JASON: It's so weird that it does that.

GLEB: It might be weird nowadays but it's incompatible. Let's do this I'm going to tell the spec to stop doing this part. And now we've got -- I think it's the navigation for whatever reason is making that disappear. Then we've got this target button and if we click on anything, it shows us what we would actually do --

GLEB: Best practices for selecting elements and we've implemented it in the selector. If not use unique class and so on. And you can modify it by writing a plugin and loading it in Cypress plugins index files.

JASON: Really handy stuff. That's pretty powerful. I wonder we're navigating to not a server that's what's causing it to do the thing?

GLEB: It might be we just got a report of it in 4.6.on and we have not looked at it.

JASON: Cool. Yeah, this is like -- this is working, it's doing what we want. It checks for this but if we fix this test, it reruns. Now we get the test we want. And if we run it in CI we'll get our successful test, I hope.

GLEB: People ask how long does it wait for default. You can adjust it per command or globally. Why is it 4 seconds? Because 3 seconds is too short and 5 seconds is way too long so obviously it has to be 4 seconds.

JASON: Absolutely. Okay. So now we've got basic tests here. We've set up a site. That site is being tested. We start out on the home page, we make sure the home page is showing the data we expect. Then we go to the about page, and we make sure that that page is showing what we expect. So I think the next thing I would love to see is what do we do when we have Javascript? We want to load in some Javascript. I'm going to go to our index page and I'm going to write a really simple div ID. We'll make that empty by default and down there we're going to add some script and let's do a call to -- what's an open API? A Rick and Morty API. Here's an API I know works and we're going to use the REST version so we can go nice and fast. Any character and I believe if we just make a call to any character we'll get all of them. So let's do this.

GLEB: You want to lower the character --

JASON: We could do something that actually maybe I can get a single character. Let's do that. Let's get one character so I'm going to fetch this character. Trying to think -- we can just do that. Then we will get the response as JSON and if something goes wrong you can do that.

JASON: What I can do then the promise based API and inside we'll say Document .query selector Javascript. That's a bad class name. Needs something easier to type.

GLEB: Recording live Jason, you're doing great.

JASON: For now what I think we can do is let's do it like this. We'll say inner text equals JSON. Let me make this a little bit wider so we can see what's happening. So what we're doing is hitting the Rick and Morty API. We know we're getting JSON back so we're making sure we parse that and we're setting the inside of the loaded data -- we'll do null 2 so it actually for mats like an object we can read a little better. And if I go back to Cypress, NPX Cypress open. What we should see is pretty much immediately take us over to the about page and here's our character. That all loaded. And gave us what we wanted. And that request is going to take time. It's not immediately available and it could, on a slow day, that might take a full second to load or something.

GLEB: This is what causes -- and fail CS.

JASON: Let's do this. We'll make a couple of little things. We'll create like the name will be a document create element. H2 and we can set the inner text to be response.name and then we'll set the image will be Document .create element image and then we'll set the image dot source to be response.image and we'll set image dot all the to be response dot name. I need to add both of these do Document . The container will be this one here. And we'll container append child name and container.append child image. And I believe that will give us what we actually want so if I save this and we'll see that to see if I can write code on the first try. Nope!

Append child parameter one is not of type known

GLEB: You can open dev tools to see. Yeah. Not of type node. Maybe it's because I did this.
Computer, why? Why are you like this?

JASON: I think it might have been that. Let's try this again. So now we're actually loading some data. And then let's imagine this would be -- this is a page that would be loading a random character. So maybe we're getting like one of three, and I don't always know which one we're going to get. If you were going to do that, how would we do like a generic test? Thinking about -- I'm thinking about testing the contains of the H1. If I'm on a site -- go ahead.

GLEB: I was thinking we could do a lot of things. I think in this case let's just see if element exists, right, and see if this new element, this is an image or an H1?

JASON: It's going to be an H2 with a name and an image tag.

GLEB: Let's just say first we have to click and then we'll see what element.

JASON: We were going to do this on click, right? Let's do it like that. Add a button and say. . . And when we come down here, then we can say document we'll add an event listener for click and we'll fire off load character. So now if I run this when I click we get our Morty.

GLEB: Exactly. We have to interact with the page just like a human.

JASON: Exactly. Perfect. So let me see how much I've learned about Cypress here. So I'm going to cy get button click. Then I want to cy -- is there like a cy exists or would I still do get?

GLEB: You would do get.

JASON: And then I'm just making sure is that it? That's enough to see if it exists?

GLEB: Well, it's almost right. So in this case have just command, visit, contain, site contains. All those commands, they have belt an assertion. Visit will fail if the page doesn't respond which we saw. Site contains will fail if a text is not found. Site.get and click will fail if an element is not clickable or visible or covered by something else. I think in this case we really want to add an assertion. We're saying side get and we want to add an assertion in this case it would be dot should and then an argument string exists.

JASON: Okay. I'm going to do the same thing for the image. And we'll say should exist. Let's save it and see if it works.
It's so easy! That blows my mind. Having tried to do this in not Cypress, and fail miserably, this is just beautiful.

GLEB: I will take credit on behalf of the whole Cypress team. We're up to 35 people. Every time we see a happy human being using Cypress, it's like the best feeling for us.

JASON: Yeah. So there are built-in assertions for contains, but not for get. If I leave this off, will it --

GLEB: Let's try doing that. I don't remember -- you can have assertion. Get should not exist or should not be visible. In this case, yes you're right. It has it built-in assertion. You click a new button.

JASON: Right I wanted to see if it would fail if it didn't have the things. If I add this then it should pass. It does. So this is really just us being like extra super sure.

GLEB: Yes, a lot of times those divs exist but they're just hidden. So they will pass. So think some framework. You're trying to check if model is there. Well, get model will be there, it just isn't visible. It's nice to be able to say get should exists, get should be visible and making an assertion in this case explicit helps you make test more readable and someone else will understand what's the meaning of it.

JASON: I do like this. Because this to me feels like I can read this. It reads like a human wrote this test and I like that. And this is also -- what you we just did we wrote a note to our future selves. If this page working we can load the page and navigate to about. You do that by visiting the index page. You can check to see if we're looking at the index page. We get this link and click it. Then we check to make sure we're on the about page. Then we click the button and we should see a heading to and an image. And obviously we're using top level selectors here because this is not a complex app, but like if we had this is character image, that this was load character, these start to become much more descriptive. In fact we should probably do that so that they do make sense. So let's give this a name of or a class of load character. Because that seems like a reasonable thing to do. And down here we'll change that as well to load character. And then for these we'll just do a name dot class list add character name and copy paste this down here.
Then we should be able to -- these start to become much more descriptive as we go. And we can see like the character name and the character image are visible and when you hover over character name it shows us the name. Over the image it shows us the image. It really feels like a note to future developers.

GLEB: Yeah. Jason, let's make it more surprising. You said on button click we load the character. Why don't we play with it and instead of load character we'll add little function that says time out and load character one second later. Because that's a common use case. Nothing is that fast in reality.

JASON: So we will set a timeout and we'll load the character after, let's say 2500 milliseconds. So now we're going to simulate a really slow network request. Two and a half seconds to get this data back. So let's save. Waiting, waiting, still worked. Lovely.

GLEB: Build in assertion retries in this case what happens was you had a get command followed by assertion. The command will try getting the button until all the assertions attach. You can attach multiple all pass and then it goes to the next command. So it's pretty sweet, I would say.

JASON: It's really nice. Like it really is nice to use.
There is a question about alerts. Can you detect if an alert is shown?

GLEB: Yes. We have an augmentation alerts you can start but it's always stopped you will see an alert message in a command log.

JASON: There's a few other questions here. How do you change the order in which tests run? Is there a way to group them in test suites?

GLEB: Yes. This uses mocha under the hood so you can have like describe suite and when individual blocks, you search for describe and you will see an example. But you can group them. You can have a hook before each test or all test and you can nest them full power of what you want to do -- the only thing in our best practice guide is make each test Indiana of other test. Don't rely on one test to run first. Leave some data there and the second test will continue. Because you won't be able to run just the second test when you're debugging it, for example.

JASON: Something that I think is really, really important to keep in mind is like, as a general software practice, creating tight dependencies like that is usually a problem. But the reason that people get frustrated with testing is when you do things where your test becomes its own application that you have to maintain on top of the application you're maintaining, that's a problem. Like, testing should make your application faster to build. If every time you make a change in your application you also have to rewrite a bunch of tests, you've probably written -- you've probably written complex testing that didn't need to be that complex. You want to simplify and just test like one thing or one workflow at a time instead of like, you know, if you've got a user creation flow, that user creation flow shouldn't be related to the other things that are done by a logged in user. That logged in user should be stopped so it always succeed. That sort of stuff so you can rely on your app being -- like each test should assume that everything than what you're testing went properly.

GLEB: Right. You said it before, right? It's nice when you test, not implementation details, but the outside. In this case we didn't look how the site was implemented. We never had to hook into it. We literally can write whole site now with no interaction. What the user sees.

JASON: There are a few more questions in here and these are all good once. Eric asks can we put a time limit. So we can -- there's a setting if we want to change how long we wait, right?

GLEB: Yes. It could be global or per each command. You can let's say like on line 13, we can say inside will give an argument which is a second object and we can say timeout and the timeout of like 1,000 milliseconds. See, it failed because one second passed.

JASON: Yeah. But if we need it to be like super long like 10 seconds because we know our API is slow, we can do something like that.

GLEB: Yes. You can set it up to be 1 billion years and you hope the test passes after that.

JASON: Is it possible to test if an alert is shown in the pipeline when deploying. I mean, we we're kind of doing that when we run the Cypress run, right? Like we are -- this is run in the pipeline. So if anything goes wrong, we would just fail our test.

GLEB: Exactly, yes.

JASON: But maybe something to point out that might be important here is that Cypress is testing the site. Cypress is not testing the build pipeline. So you're not using Cypress to listen to your running build and say like oh the build exceeded this many kilobytes or something. That's a different thing. Cypress is for the built website that's actually created and useable. There's a couple of other good questions here. I want to make sure I don't miss anybody.
Is this all about testing with Javascript? This channel is about lots of things. Today it's about testing with Javascript.
There's a question about cy.get within. I don't know anything about that. So maybe that would be a fun one to play with.

GLEB: Yes. It's a little bit side tracking Netlify. I'll tell you what that is. Right now it start looking for the whole body of a page. But imagine you have a complex page and a form inside. And you want to fill a bunch of fields in. You can say cy get form.within and now all your commands will be scoped to the form all the fields there. It kind of changes the root of your command starts so you can concentrate on part of a page.

JASON: To rephrase that in a way that makes sense to me. When we run Document .create element, we are attaching to the entire HTML document.

GLEB: Yes.

JASON: But when I'm in name, or if we've got this container, if I run container.query selector, I'm only searching inside of this particular domino. So what you're doing is the same thing within is just scoping it to one domino that we only look inside of that.

GLEB: Let's try that. Because you have loaded data container. In our data we can assertions -- yes. So let's say cy get loader container.

JASON: Loaded data.

GLEB: And then we say within.

JASON: Do I just straight up that's it?

GLEB: Yes. And it takes -- it takes a callback and inside that callback you want to move your --

JASON: Oh, I get it.

GLEB: But now those selectors don't have to be unique on a page. They just have to be unique within that container.

JASON: Let's test that. I'm going to do an H2 out here as well.

GLEB: Yeah. Okay. So then what I'm going to do to make sure, let's also add one on the H2 -- actually, I can chain these, right? So I can say it should exist but I want it to not contain. Is it like that?

GLEB: No. Just say and instead of not contain. So and is should and when arguments to call and a string not dot contained. Everything you tried we just concatenate into one stream.

JASON: So what I want to check is we're getting a generic H2. So what I want to see come through in Cypress is that we're only matching once because there's only one H2 inside of this loaded data and I want to make sure it's not matching the outer text of this one here. So let's save this. Okay. Is there any way to test set interval or set timeout? That's exactly what we're doing with this about.HTML.

GLEB: Even more. You can stop those. We have site clock which overwrites those so you can control time. So you don't have to wait for 2 seconds. You can fast forward the 2 seconds or whatever.

JASON: That's cool.

GLEB: It makes the test faster, I would say.

JASON: That's really nice. There we go. And we made sure we only matched the one H2. So it's within our loaded data and we get the H2. So there we go. That's handy. Okay.
So then I think the next thing we want to do, just to make sure we don't run out of time here, is how do we get this up on the Internet?

GLEB: Well, we mail it by snail mail I think.

JASON: Okay. So a quick question here. Do we need to -- I imagine we probably want to ignore the videos and stuff like that.

GLEB: Yes. If you could ignore them that would be perfect.

JASON: Let me ignore Cypress screen shots. And Cypress videos. Okay. That's good. And then I'm going to just add everything else, make sure we only got what we wanted. I'm happy with that. And we can push, so I just have to type push in the future. And then I want to -- we can just deploy this to Netlify, right? Let's try it.

JASON: I'm going to create a Netlify site. We'll call this Cypress testing. All right. And our build command, there is none so I'm going to say no build command. And our directory to deploy is public.

GLEB: I never use Netlify CI like this to connect the site. I always use web application UI. And I do use Netlify locally. It's cool to watch you.

JASON: That command. Especially for me where I'm building a bunch of demos all the time, it saves me so much time not having to go to the website and click my buttons, I create my git repo and click the init. Does it support type script? Yes it absolutely does support type script. That's how we're getting the references here.

GLEB: We should type script definitions for every command for a long time. We now actually ship TS node. So if you write your specs using type script it should just work by default. It will be built in. Or is built in.

JASON: Okay. So there's our site. It's up and running. But it didn't like -- our test didn't run or anything. Nothing happened.

GLEB: Nothing, right? So we need to bring Netlify plug in Cypress. So this is enabled on your account if I understand correctly.

JASON: Let me go back and talk through this. I am on my Cypress testing. There's this build plugins beta link up here. I'm going to click this and chose enable the site I want here. I'm going to enable Cypress testing and now I'm going to -- we already know which one we want so I'm going to save that and then I'm going to go back to my site. And nothing really has changed yet. So we need to make a couple of changes locally. The first of which is I'm going to create a Netlify.toml. We don't need much in here, but I'm going to specify a plugin. So the package is -- what is it? Netlify plugin Cypress?

GLEB: Netlify plugin Cypress, yes.

JASON: Do I need any other config?

GLEB: I believe no. But you have to install it using MPM. You have to add it as a dev dependency.

JASON: Okay. Netlify plugin Cypress.

GLEB: Yes. I have my fingers crossed.

JASON: Me too, fingers and toes. I know for sure one thing is going to break because we, in our tests, we're looking for like public index, which isn't actually what's going to be live. What will be live, because this public site is going to get --

GLEB: Right, right. So what I think we should do instead of this folder just a slash. Instead of public, slash. Just remove index HTML.

JASON: Oh, just a slash. Okay. And I also need to fix in here, because the way that I set these up is not -- let's set this up to be the root and we'll set this one up to be slash about. That should be all good. That probably would have worked anyways but let's make it into an actual website.

GLEB: It's always tricky going from static site to HTTP server.

JASON: That's something I'm curious about. I notice that Cypress is starting a local host when it's doing these tests. Can I tell it to use the public folder locally so I can run these tests locally and in CI in.

GLEB: No. But you can install any HTP server and start it, right?

JASON: Okay. Let's do this part first and then we'll get it fixed. So let's --

GLEB: Why did --

JASON: Because we changed.

GLEB: Oh, it just changed.

JASON: Yeah, yeah. We'll get commit, add Cypress to Netlify build we'll push. You can combine Cypress with a tool called. . . If I plug in visual diff where did I put it. If you install this plugin apple tools gives you a comparison of what it used to look like, this is what it looks like now. Are you sure this is what it meant to do? Let's see here. There we go. Here'S the build running. Our Cypress stuff is installed. It uses electron. And we get our test passing. So everything worked the way we expected. That's super awesome. And it didn't take any magic from us. We just had to --

GLEB: This is the interesting thing, right? How did it know what to test? Which URL to load? This is a plugin into Netlify build system. The Netlify system knows what it's doing. It nodes the build command it knows the source files. Then it calls our file because it has registered on build hook. Netlify build system says here's the directory. So our plugin spins HTP server, serves the folder, sends the base URL in Cypress, runs the test, shuts down, shuts down the HTP server, everything is good no errors. That's it.

JASON: That is -- I mean this is like really, really powerful stuff, right? Because what we've done is we've taken something that like if you were trying to load puppeteer or what's the other one I always forget the name of?

GLEB: PlayWright, selenium.

JASON: That's the one I was thinking of. Every fifth try it wouldn't work. Like what happened? How is that a thing. So what I like about Cypress is that it's clear that the thing you're focusing on was making it understandable for developers. Because the error messages are clear. The workflow is clear. You're writing in borderline human language. It's a much more approachable experience than others I've had when trying to write these type of end-to-end testing and the fact that it just works when you put it into CI like this is really, really nice.

GLEB: Thank you.

JASON: So let's try to answer a couple more questions. We've only got a few minutes left. Can Cypress get any Javascript errors that may be happening in the browser during test execution.

GLEB: It will fail a test if you have an exception an actual error you're not catching in your application by default. You can set a callback and I can in other those, but by default any error on your side will stop the test. You don't want broken sites to be deployed.

JASON: Are we able to say something like I expect the site to throw a console error?

GLEB: Yes. So you can say callback, inspect the or message and say no it's fine I expect this to happen.

JASON: Cool, very cool. And so the follow-up question is, is Cypress able to get application server issues. The answer to that is no, right? It's just a website.

GLEB: It's just a website. It has no idea who's serving the page.

JASON: I guess if the server was returning headers you could read the headers out of the requests because it's just Javascript but that seems maybe don't do that.

GLEB: Probably not a good idea. At that point what do you do? Should the test ignore some errors? It probably is the wrong place.

JASON: Yeah. Is there a recorder tool for quick test recording right in the browser? I think that's what it's doing when you do run, right? It's going to make a video?

GLEB: I think that person meant can you interact your website inside Cypress I frame and instead of just a selector playground, record your interaction so it makes like a chunk of Cypress commands that you can copy and paste in your spec file. People have done it. There are recorders. I'm not happy with any of them. Because it's easy to record your clicks and actions but how do you record oh, I expected this Morty to appear, right?

JASON: Yeah.

GLEB: How do you record your assertions and that's what nobody sold. So it's faster to write it using this code editor when trying to record it correctly.

JASON: Yes. Question about whether the recording of this will be available. It absolutely will. This will be posted on learnwithJason.dev tomorrow and it will also be available on YouTube.
What is Cypress? You're going to have to watch the whole thing. We've gone all the way through it. It's super fun. Let's see here.
Aren't you overtesting what should already be in the unit test? I don't have any unit tests. I don't intend to write anyway. So what I like about this, and you can correct me if I'm wrong, Gleb, but what I feel about this is like this end-to-end test gets me 95% of what I was going to get with a unit test and it's easier to write.

GLEB: Probably easier. But the second thing, think about the user. You are testing your complete website and your complete stack. If you have a unit test for like load character but then your HTML structure is wrong and it's not connecting the button what good is unit test? It doesn't give you any confidence in a real system you're testing the whole thing.

JASON: The way that I've talked about it, unit tests are -- they're a specialized kind of test. When you have something complicated that's like a function that needs to behave a certain way, a unit test makes sense. But for like general stuff, like I clicked a button or I'm generating some HTML, unit tests make less sense because they're not showing the hold. As you just said, you're testing an isolated piece that could be completely broken but still pass the unit test.

GLEB: Exactly. It could be broken for so many reasons and the users don't care why it's broken. They'll just say it's broken.

JASON: Yeah. So is there a demo for using fixtures and tests. We don't have time for that today unfortunately. I think there's probably additional fun stuff we could do in the future with mocks and fixtures, but for today is there somewhere you can recommend?

GLEB: Go to our documentation. We invest the engineer's time in updating the docs. They're always up to date.

JASON: We should be testing the site using the real API. I think you should use the real API when you can. You want to mimic the user's actual experience not like your design experience. Also building and maintaining a mock API when you don't need to, is a pain. It's another piece of the app that needs to be maintained and kept up to date. If the live API changes and you forget to update your mock API, if you can rely on your APIs it's a good thing to do.
What else here? More people noticing your background. It really is a good one. Can I use some dot config files absolutely you can. That is what this Cypress.JSON is for. I can set the base URL here and that kind of stuff?

GLEB: And you can control it and read additional things, and change things you can pass additional environment variables. You can have different config files and pass different file names. You can totally create different configs.

JASON: Check out this plugins guide. There's a whole lot you can do and the Cypress docs are really good. Go check that link, look at the getting started and all that stuff.
Can you step into tests one by one

GLEB: This is a good one. Yeah. Do you mind doing NPX Cypress open again? Now inside your spec file, right at the beginning, after visit like right after the visit you can put cy.pause.
Have to change because we didn't set any base here. Check this out. Right? You pause the test and now you can play it or -- on the right where step to the next one. That one. Now you're going one by one.

JASON: Beautiful.

GLEB: The best thing, this is the cool thing. Cypress runs itself in the same environment, and same browser window. When it pauses, and stops, it pauses for application as well. Because like you're running in the same event loop. So you can really understand the environment and your application because you paused it.

JASON: So what I'm going to do real quick to make this work is I want to have a dev and we'll say serve public. And that allows me to run and it opened in the wrong window but we'll pull it over here. And now we can actually use our site locally. Okay. So how can I get Cypress -- let's just make that work, real quick. How can I get Cypress to use this?

GLEB: Go to Cypress JSON file and at base URL and then HTP with local host 5,000 I believe. And site visit should be just a slash.

JASON: We'll go here to just the slash.

GLEB: This will mimic what Netlify ran completely.

JASON: How do I get it to run that dev command?

GLEB: Okay. You can use two terminals or I have a utility Jason I know you know called start server and test. So you can do NPM start, server and test.

JASON: I should probably install that as a dev dependency.

GLEB: So once it finishes then, add another command saying like dev or local, I don't know.

JASON: Yeah. We'll call this local test do it this way. I like this format. Test local. And we'll do start server and test.

GLEB: Right. And now first command is whatever script you want to start server with. In our case dev, right?

JASON: Just dev and test like this?

GLEB: You want to wait for the port to respond. In this case it's 5000 because it's local host space 5000. And so it happens, it will run your NPM wait for 500 to respond and it will run the test and when everything is done shut down everything. So now you see start server, run Cypress run when it's done shut down. I use this all the time. I use it to do Cypress open so I can open Cypress with a single command.

JASON: Beautiful. Okay. So now we've got it running. And with that, unfortunately, we are out of time. Gleb, thank you so much for joining us today. This was amazing. Where should people go if they want to follow up with you online?

GLEB: Gleb.dev is my shortest domain name.

JASON: I know there were more questions. Follow Gleb on Twitter and make sure you ask questions there. Go check out Cypress, go check out the Cypress build plugin if you want to learn more, you can just search Netlify build plugins it will be the first thing that comes up here. Definitely go check that out, give them a try and let us know what you build. Gleb, thank you so much. Also, thank you to White Coat Captioning for providing the live captioning today. Thank you to Santa, to Fauna and to Netlify we picked up a new sponsor today. They make the captioning possible. Still looking for sponsors. So if you know a company that is looking to make the web more accessible hit me up because this stuff is expensive. With that we're going to raid somebody. Thank you so much. Chat we will see you next time later this week we have -- wait, what day is it? Today's Thursday. So we've got next Tuesday we've got -- next Monday. On memorial day, it's a holiday we thought it would be fun. Jem Young is going to come on it's going to be a blast the. Please come out hang out we will see you next week. Thanks y'all.