List

Mutation Testing with Mutant

Mutation Testing with Mutant

by Erik Michaels-Ober

In the video titled "Mutation Testing with Mutant" presented by Erik Michaels-Ober at RailsConf 2014, the speaker explores the concept of mutation testing as a tool for improving code quality by verifying the effectiveness of tests. The discussion begins with a metaphor comparing programmers to carpenters, emphasizing that just as carpenters rely on the right tools, programmers need effective development tools. The key theme revolves around mutation testing, which operates by modifying code and observing whether tests appropriately fail or pass in response to those changes. Key points covered in the talk include:

  • Importance of Tools: The talk outlines various programming tools such as code editors, debuggers, and code profilers, which enhance programming capabilities. Specifically, he highlights how editors prevent bugs through features like syntax highlighting and auto-completion.
  • Testing vs. Code Quality: While traditional testing is often considered a foolproof way to ensure code quality, Michaels-Ober argues that tests themselves can be flawed. He critiques the notion that 100% code coverage guarantees bug-free code, clarifying that flawed tests can provide a false sense of security.
  • Challenge of Measuring Test Effectiveness: To measure the effectiveness of tests, the speaker presents mutation testing as a solution to the problem of 'who watches the watchers.' Mutation testing creates small modifications to the code (mutants) and tests whether existing tests can catch these changes.
  • Mutation Testing Mechanics: The speaker elaborates on how mutation testing works through examples, explaining that if a test passes after a modification (mutant) is applied, it indicates that either some test cases are missing or existing tests are not rigorous enough.
  • Live Coding Demonstration: In the latter part of the talk, Michaels-Ober conducts live coding to demonstrate mutation testing's implementation using the Mutant library. He showcases how to write tests for a class representing planets and discusses different strategies to ensure comprehensive test coverage via mutation testing techniques.

Ultimately, the conclusion of the talk reinforces the idea that mutation testing serves as an essential tool for developers by offering a method to improve their tests, leading to higher code quality and fewer bugs. The speaker encourages viewers to experiment with mutation testing frameworks like Mutant to enhance their testing strategies.

Tests are a good tool to verify your code but how do you verify your tests? Mutation testing is a technique to evaluate the quality of your tests by programmatically mutating (i.e. making a series of small modifications to) your code and ensuring that the tests no longer pass. This talk will attempt to demonstrate the value of mutation testing and show you how to use Mutant to improve the quality of your tests and your code.

Erik Michaels-Ober is a programmer.

Help us caption & translate this video!

http://amara.org/v/FG1f/

RailsConf 2014

00:00:16.960 okay is the mic live yeah we're good okay um hi everybody welcome thank you for
00:00:23.439 coming um so uh this is gonna be a talk about tools
00:00:32.000 and um there's this common expression that says that a carpenter is only as
00:00:38.480 good as his or her tools um i'm not a carpenter but that that makes a lot of sense to me
00:00:44.559 um if your hammer is made out of feathers uh you're not gonna be able to build very much
00:00:51.039 and i really think the same thing is true for programmers um i know that that is
00:00:57.680 true the tools that we use really uh enable us to do our job and
00:01:04.559 we use so many tools it's easy to sort of take for granted uh the tools that we have and the tools that we do use
00:01:11.200 and so i think it's worth sort of thinking about the tools that we have and how they help us improve as a programmer and
00:01:18.640 thinking about what new tools we can use in this case i'll be talking specifically
00:01:24.000 about mutation testing and how that as a tool can really help us all improve as programmers help
00:01:30.079 us write better tests but um i think i just want to sort of take some time to reflect and set a
00:01:36.960 little bit of a context for the tools that we use every day and sort of i think take for granted a bit
00:01:44.320 so uh the first one is an editor um and it seems like a
00:01:50.479 very simple tool right you just type in text and shows up on the screen but it's incredibly sophisticated if
00:01:57.200 you've ever tried to write a text editor if you've ever read the source code of a text editor most text editors are like millions of
00:02:03.040 lines of code to implement what seems like a relatively simple thing and they help us they provide us with
00:02:09.520 things like syntax highlighting auto completion and this directly helps
00:02:14.640 us write better programs right we avoid bugs we'll realize a bug in our editor before we
00:02:20.319 before we deploy it to production before we even run tests we'll find a bug in our editor because our editor tells
00:02:26.400 us about it um this is an early version of vim so
00:02:36.560 um it can it can be really easy to forget sort of what these tools used to look like right this is how people used
00:02:41.760 to write code um and these look more like the sort of tools from the wood shop than the tools
00:02:46.879 that we're used to using so this is an early punch card machine the photo was taken
00:02:53.920 in the uh in the computer history museum in mountain view california and i can tell you for a fact that i
00:03:00.800 would not be a programmer today if this is how we still had to write programs and i suspect that many of you would not
00:03:06.640 be programmers um if this was sort of the state of the art in how it was done
00:03:11.840 and so i i think like i want to make the case that sort of both the quality and the quantity of
00:03:18.239 software uh would be much worse than it is today if not for sort of the continued evolution
00:03:24.319 of of our tools another tool i use every day is an
00:03:29.840 interactive debugger so um sort of allows you to step through your code
00:03:35.280 line by line and better understand how it works you can kind of get inside the code right i'm not going to spend too much
00:03:42.319 time talking about debuggers sort of public a service announcement uh
00:03:48.000 next week uh not next week this week next thursday this thursday um in this same room i
00:03:54.000 believe uh is a great talk on debugger driven development with pry so if you're interested in hearing more
00:04:01.040 about that you should go to that so what do we do when our code is slow
00:04:08.400 what's the tool for that right we have profilers that tell us where time is being spent
00:04:14.319 when we execute our code and i wouldn't even know how to start
00:04:19.519 optimizing a program if i didn't have a profile right profiler i would be a terrible optimizer without a profiler
00:04:26.000 i guess i would like start putting in uh you know t equals time dot now and then like at
00:04:32.479 the end of whatever i wanted to measure i would subtract the current time from the start time but that's crazy
00:04:39.759 like instrumenting your entire code that way is uh yeah like i wouldn't really know
00:04:45.840 how to optimize code without a profiler i wouldn't be as good at it none of us would
00:04:51.120 and um another sort of tool that is very prevalent in
00:04:57.600 the in the ruby community is testing um this is an example of someone who should have done more testing
00:05:06.400 i'll show that again all right so um i think this is a good
00:05:13.919 illustration of how testing can save you right test so that you find out before you
00:05:21.120 sort of run it in production okay enough of that um so i i'm actually going to make the
00:05:27.759 case that in the ruby ruby toolbox or maybe in the rubyists toolbox uh tests are sort of like the hammer
00:05:34.080 right like this is the thing we turn to all the time for all sorts of things we use them to prevent regressions
00:05:41.199 we use them to specify behavior and we actually use them to drive development dhh doesn't do this but
00:05:49.680 many others do and find it useful so um if we write tests then we have
00:05:56.639 perfect code right if we have tests that verify that our code does what it's supposed to do
00:06:02.319 then at the end of the day we have perfect code correct not correct um this is the fundamental
00:06:10.639 logical flaw with testing right you have some code and
00:06:16.240 uh you know that code can have bugs so you say i have an idea let's write some tests but tests are
00:06:22.880 just more code and we know that code has bugs so we're screwed
00:06:29.600 what's that test your tests right so um we're getting there patience
00:06:37.199 um so like one tool that people use to sort of measure the effectiveness of their tests is code
00:06:43.280 coverage and um it's sort of a metric that's
00:06:49.039 designed to tell you whether your your tests do what they're supposed to do but i'll show you uh in a moment why i
00:06:56.080 think it's a really flawed metric and why it sort of can give you a false sense of security right um
00:07:02.560 a lot of people think that they have 100 code coverage um and that means like their code is
00:07:07.919 perfect and bug free or if they reach that level then their code will be perfect and bug free um but this is not true right like this
00:07:14.639 guy thinks he's covered and he's not um and code coverage is
00:07:19.680 actually like it's something that was built into ruby right like in ruby 193 this is something that like we as a
00:07:25.360 programming community said like we want to have um and i'm not against it like i think it's good but um
00:07:31.280 i i do think it can give you a false sense of security right i thought this was a funny tweet
00:07:38.880 so you can have 100 code coverage and still have completely bug written
00:07:44.319 code
00:07:49.520 so so is there hope for us right like how do we how do we test our tests
00:07:56.000 it's sort of this problem of like who will watch the watchers right who do we who can we trust
00:08:01.039 if we can't trust our tests how why why are we even writing them and i'm going to try to make the case
00:08:07.599 that mutation testing is the sort of solution to this problem
00:08:13.680 so just like everything else like an editor like an interactive debugger like a
00:08:19.120 profiler like tests mutation testing is a tool
00:08:24.879 the basic idea behind it is that it takes your tests and it runs them against your code and they should pass
00:08:31.120 um and if they do pass then what it does is it takes your code and it makes a modification to your code
00:08:36.880 it actually changes your code at runtime and then it runs your tests against again against a modified version of your
00:08:43.760 code and the idea is that when that code is modified the test that previously passed
00:08:49.760 should now fail right so the thing the your modified code is called a
00:08:54.880 mutant and the idea is that if that test fails you kill the mutant right the mutant dies but if that mutant
00:09:01.040 survives then that means there's something wrong with your tests there might not be something wrong with your code but there's certainly
00:09:07.279 something wrong with your tests either you have a bug in your tests you have missing tests your tests are are either overspecified
00:09:14.800 or underspecified so this is a technique it's very helpful for sort of answering the question
00:09:20.800 what tests should i write which i think is a question that many of us struggle with it's certainly
00:09:25.839 something that beginners struggle with when they're starting a program like how do i how do i write tests what what
00:09:32.560 do i test right and then there's also this question of like how do i know when i'm done
00:09:38.399 how do i know when the code is sufficiently tested and um i think these are actually hard
00:09:43.839 questions to ask uh or hard questions to answer and uh mutation
00:09:49.279 mutation testing provides a quantitative answer to those questions you can say
00:09:54.959 with confidence that this code has 100 mutation coverage
00:10:03.600 so um just to sort of give an example
00:10:08.800 here is some code and an assertion about the code so i have a method foo
00:10:16.079 it takes an argument whose default is true and the actual method body for foo
00:10:22.480 is either return that argument or fail and my assertion says
00:10:31.279 assert nothing raised if i call the method foo without passing in any parameters and so
00:10:38.720 uh without passing in any arguments to the arg parameter rather and so what uh
00:10:47.279 you know this test will pass right um you call foo arg is true and it sort of
00:10:53.920 short circuits right it sees arg it sees the or and this test passes
00:11:01.040 so maybe you think this is a good test maybe you think you're done writing your tests but you are not
00:11:08.079 and a mutant of that code a small modification a sort of unit modification of that code
00:11:13.600 might look like this and basically what it did is it just sort of took that or
00:11:18.800 fail and removed it and the idea is that if you do that at least one of your tests should now
00:11:25.279 that was passing before should now fail one of your tests over that code for that foo method should now fail
00:11:31.680 and if it does not then you are not testing your code sufficiently
00:11:38.079 so uh this is called a statement deletion mutation there are various other types of
00:11:43.519 mutations so uh for example there are mutations that would take that default parameter
00:11:49.760 and change it from true to false or from true to nil right which would also cause
00:11:56.560 failure in this case um there's another mutation that will take the or and change it to an and right
00:12:03.519 so anytime there's sort of uh a unit in your code it takes greater than signs
00:12:08.560 and changes them to less than or equal to signs etc right it takes ifs and changes them
00:12:14.320 to unless it will take whole expressions and negate them and make sure that your tests fail
00:12:19.360 when the when the negation of a statement is uh when the method returns the negation
00:12:25.680 of the statement instead of the statement right so that's that's sort of the core idea behind mutation testing
00:12:32.560 and so you end up sort of writing these tests to cover all these cases that um and then you sort of know when
00:12:39.440 you're done right like you know when all of your tests uh when your code is fully mutation covered
00:12:46.399 uh this is another tweet this one from katrina owen and it's sort of this idea it's kind of
00:12:53.760 like both horrifying and satisfying at the same time but if you sort of add more granular tests you'll find more bugs
00:13:00.480 and in many cases mutant which is a mutation testing framework will find those bugs for you
00:13:06.399 right that's the tool okay so i promised there would be live coding this is sort of the introduction is over and now we will
00:13:14.240 write some code hopefully
00:13:20.079 i'm just going to switch to mirror displays
00:13:26.720 command f1
00:13:32.639 that is a pro tip that's great you're a pro i clearly am not okay cool
00:13:41.120 um cool and a new version of mutant was like just released a few minutes ago advance of this
00:13:49.279 uh presentation i am not uh the author of mutant uh it's a great library by marcus sherp
00:13:56.000 um and i encourage you all to check it out uh version 0.5.11 hot off the presses
00:14:02.959 so um this is some code so like the the sort of thrust behind
00:14:09.440 this live coding demo is i will not be live coding code i will be live coding tests because the idea is not to
00:14:16.079 like mutant doesn't verify that your code is correct it verifies that your tests are correct so you still need to write tests
00:14:21.839 right test verify that your code is correct mutant verifies that your test is correct so this is the code and it's pretty um
00:14:28.399 pretty simple but we'll sort of walk through it line by line just to make sure everyone has a good
00:14:33.760 understanding of it and so uh there's this module that represents the universe the entire
00:14:40.480 universe and inside of the universe we have planets and that's what this class is all about
00:14:46.240 um it's a pretty simple planet it takes a radius and an area as parameters
00:14:53.680 uh when it's constructed and stores those in instance variables uh the radius is the mean radius of the
00:15:01.519 planet uh in kilometers and the area is sort of the surface area uh of the planet in square kilometers
00:15:10.079 and then there's one sort of interesting method uh one public method
00:15:15.519 spherical and uh spherical will return true if uh if the planet
00:15:23.680 is a perfect sphere or within a particular tolerance of that so the idea is we calculate the
00:15:29.759 approximate area using 4 pi r squared which is the formula to
00:15:35.440 calculate the area of a sphere and if the uh
00:15:42.720 area sort of matches that then we know it's a sphere we know it's spherical this method returns true
00:15:50.000 and if uh if that's not true then the planet is not spherical it's either oblate like the earth or
00:15:56.880 prolate and um then this method will return false
00:16:02.000 so yeah we just sort of calculate the approximate area and then we have this range private method that just generates
00:16:08.480 a range we need sort of a tolerance the idea is you don't want it to be
00:16:14.160 too precise because we're dealing with pi so pi is um i mean in actuality it's a
00:16:21.680 non-terminating number in ruby uh it has like 10 digits of precision or something like that right
00:16:28.079 like the constant math pi but the idea is that like if it's close enough to a sphere within a
00:16:34.240 particular tolerance then we'll just call it round basically and so we generate this range which is
00:16:41.440 sort of the approximate area that we've calculated based on the radius plus or minus the tolerance and we see
00:16:49.120 if the area falls within those bounds does everyone understand this code i think it is pretty
00:16:54.480 simple i tried to make it fit on one screen i'm on slide
00:16:59.839 yeah okay so um if everyone understands it i want to
00:17:05.120 take a little bit of a poll this is kind of like the interactive part of the talk and you have to like everyone has to
00:17:10.400 participate that's the that's the rule everyone i people like to sort of sit by the sidelines and not
00:17:16.240 commit but you have to commit i'll be really angry if you don't
00:17:21.760 you don't want to see me angry so um how many tests do you think you need to
00:17:29.679 fully cover this code to cover the public methods the the
00:17:35.760 spherical method right so that it's sort of fully exercised who thinks you need zero tests show of
00:17:42.320 hands anybody no good
00:17:47.840 i agree you can't cover code without tests so it's good you've been paying some
00:17:52.960 attention who thinks you can do it with one test maybe sort of the happy path
00:17:58.320 right you write a test that says uh you know you expect some planet to be
00:18:04.960 spherical given a radius and an area and it is
00:18:10.480 all good who thinks that's sufficient
00:18:17.039 nobody so you can actually get c0 100 c0 code coverage of this entire
00:18:24.320 class with one test with one spec right you won't have 100 mutation
00:18:30.559 coverage but i will show you in a minute you can have 100 c0 code coverage despite the fact that
00:18:36.240 nobody in this room thinks that that is sufficient to cover this code so i will prove it to
00:18:41.679 you but you all intuitively know this to be the case and yet we all idolize this c0 code coverage
00:18:48.799 metric as if it means something and really it should it's a false sense of security
00:18:53.919 right you're the guy with the umbrella and the hurricane and the umbrella is like destroyed and inside out
00:19:00.880 okay so um how many people think you can do it with two tests
00:19:07.039 okay uh somebody who's raising your hand this gentleman in the front what are the two tests that you would write just sort
00:19:13.280 of roughly maybe the happy path and what other uh
00:19:18.880 one that's spherical and one that's not okay i think that's good uh how many people think you would need
00:19:24.480 three to do it okay maybe gentleman there who thinks you need three what's the third you
00:19:29.600 would write
00:19:41.919 say it again a value for tolerance it will blow up the computation how would you blow up the computation
00:19:55.039 passing in a string okay um great and what would you expect the
00:20:02.000 result to be like what would your expectation what would you assert like i pass in a string and i expect
00:20:10.559 an exception okay and if you didn't get an exception then that would be a problem okay um
00:20:18.240 okay uh who thinks four will do it nobody thinks four will do it
00:20:25.760 a few people do uh yeah what's what additional tests will
00:20:32.320 you add
00:20:38.000 great i really like this so the comment was that you're testing a range and there's sort of two sides there's
00:20:43.760 the i'm on the low end of the range and i'm included and i'm on the high end of the range so it would be there's two of those
00:20:52.080 right one for the low end and one for the high end exactly so it's sort of the happy path the thing is spherical the sad path the
00:20:59.200 thing is not spherical and both sides of the range i like that good um how many people think five
00:21:07.760 five or more how's that five or more okay lots of hands for five or more so um
00:21:15.520 according to mutant which is also software therefore imperfect um you can you can test this
00:21:22.159 with four and it will not handle things like you should like it sort of assumes that the radius
00:21:28.960 and area are valid right like you you can although actually maybe that's well we
00:21:35.520 can try it it's a live coding thing so let's just do it and see what happens but thank you for participating in that
00:21:41.280 i think it's an interesting exercise but um yeah basically like newton says the answer to
00:21:47.120 this question is four right it's basically the happy path the sad path and both sides of the range so
00:21:54.640 um yeah let's let's sort of show how that works
00:22:02.880 okay so um i'm going to start by just making a gem file as you do so that let me i can just
00:22:10.480 sort of show it's a very simple uh
00:22:15.760 layout so far i have a lib directory which contains universe.rb which you've all seen and a spec
00:22:23.360 directory which is empty so very little up my sleeve at this point
00:22:30.400 i'm just going to make a gem file as you
00:22:38.840 do and at this point i'm just going to add r spec because i'm starting to write some tests and i'm going to add mutant
00:22:48.240 okay so and we'll bundle install
00:22:58.480 ah cool it just installed that new version of of uh mutant that was just released moments ago good
00:23:05.600 uh let me just see what ruby version i'm on okay that should be fine so um
00:23:13.360 uh let's write some specs so we have the spec directory let's write uh planet spec dot rb
00:23:20.159 and we'll uh require our spec and we'll require our planet file i'll
00:23:26.320 just use require relative for that rather than messing with the load path or anything so that's up a directory in
00:23:31.840 lib and i think it's called universe
00:23:37.600 and uh now let's start writing our specs right so we're just going to describe
00:23:43.520 our uh planet in our universe module
00:23:51.440 and um so let's create a subject
00:23:59.039 which is just going to be our planet that's like the main thing that we're going to be testing here
00:24:04.080 and it's initialized with a radius and an area i believe in that order yep
00:24:12.320 cool so um let's create a context and
00:24:19.679 let's do the happy path first because that was kind of like we all agreed that the first test we should write was the
00:24:25.200 happy path so in this case venus is actually the happy path
00:24:30.960 venus is pretty darn close to the spherical um so in this case uh
00:24:38.559 we'll define the radius to be
00:24:58.840 cool uh and i think i said it's in uh meters yeah
00:25:04.480 so it'll be that and then the area will be
00:25:19.120 um let's see wikipedia
00:25:25.039 okay so the service area is what is that 460 million um
00:25:32.960 which is okay but actually like i would like a more precise number
00:25:38.240 because like i don't want to crank up our tolerance to some ridiculous value to make this true so i actually found a
00:25:44.799 more precise number than the one that's on wikipedia which is this so it's 460 million
00:25:51.360 264 thousand 740 which is you know pretty round numbers still but it's more
00:25:57.360 precise than the one on wikipedia and uh now we'll have our assertions so
00:26:02.880 we'll just say it's spherical venous is spherical
00:26:08.720 we expect our subject to be spherical
00:26:17.360 good is everyone satisfied do it like if people see bugs call them out like does this look like a good happy
00:26:22.960 path test yes this will pass good
00:26:29.520 uh let's run it uh yeah that should work
00:26:39.039 cool it passed hooray um let's do something else let's open up
00:26:44.559 our gem file again and add simple cuv to measure the c0 code coverage
00:26:53.840 and i guess here we can just say acquire simple cuff
00:26:59.520 simple curve start and so now if we run our specs again
00:27:04.799 we'll get a little coverage report
00:27:16.320 so for those who aren't that familiar with simple cuff basically it looks to make sure that you're
00:27:23.919 that every line of code is executed and if you test the happy path it totally is right the class the module is loaded the
00:27:32.240 class is loaded this constant is set we initialize
00:27:37.360 um we initialize a planet i can turn on lines
00:27:44.720 we initialize a planet on line nine we invoke the spherical method on line 15
00:27:52.720 in the assertion and that invokes the range method so we have you can actually see every
00:28:00.000 line of code is executed precisely one time so we have we're not over
00:28:05.520 testing we're not under testing we have perfect 100 c0 code coverage but we all agreed
00:28:11.679 that this was completely insufficient so ship it right okay
00:28:21.200 um i'm gonna delete this simple cub stuff because it's garbage
00:28:28.480 okay so um let's write some more tests so a planet that's not spherical is
00:28:36.880 no that's my name thank you
00:28:44.159 is our home the earth um radius of the earth
00:28:56.840 cool
00:29:07.919 i guess we could say 0.1 doesn't really matter and oops
00:29:14.960 what's the area
00:29:27.679 cool so in square kilometers it's 510 and uh 510 million rather
00:29:36.240 so we again we could like try to find a number that's more precise but we actually like the whole point of this test
00:29:41.440 is to test a planet that is an oblate spheroid not an actual sphere and so in this case
00:29:48.000 we want to so like it's fine that the numbers are not within the default tolerance um and so yeah basically we want to say
00:29:56.480 like it is oblate not spherical
00:30:06.240 so in this case we expect our subject not to be spherical
00:30:17.440 cool look good let's run it
00:30:24.720 cool our tests pass so uh this is like maybe your normal
00:30:30.960 workflow you would do this a few of you would stop at this point i think there were probably as many hands for like i would stop at two
00:30:37.440 or probably more tests for like i would stop at two than i would stop at four um or three but let me show uh
00:30:44.799 let me show what mutant does let me show sort of how this mutation testing stuff works so um you're going to say bundle exec
00:30:54.480 or i have it alias to be i can spell it out
00:30:59.760 so this is the mutant command line and it takes a bunch of arguments so uh you have to give it a lib for the
00:31:07.120 sort of lib directory that you're testing so that it knows to add that to the load path and then um you give it a require so
00:31:15.519 it's gonna require some specific library in this case the universe library
00:31:20.960 that you wrote um and then you can say like i want to test everything in universe so you can say
00:31:26.320 like with wild cards like colon colon universe star i can make that a little smaller so it fits on one
00:31:32.720 line um or you can say like i want to test specifically the planet
00:31:37.840 class or you can say like i want to test a particular method so you can say like i want to test spherical
00:31:44.240 something like that right but we want to test the whole planet class oh and you also there's an option
00:31:50.399 to say use our spec so it knows what test framework to run this is important because it's testing your
00:31:55.440 tests and i'm getting some sort of an error ah
00:32:01.200 i'm missing mutant r spec in my gem file that is easy to fix right so
00:32:07.840 uh our spec used to be built in this has changed recently so basically
00:32:13.039 there are other libraries there's like plug-in libraries so if you want to write if you if you use some
00:32:18.799 crazy test framework you can just write a gem that adds mutant support for that test framework so this happens to be the
00:32:25.200 one for rspec but you can use one for test unit or anything else so i will bi is just a shortcut for a
00:32:32.559 bundle install now we'll do this cool
00:32:37.840 so what it is doing you're like what this is crazy we only wrote two tests why are there all those little green
00:32:44.080 dots and f's flying by so basically what's happening is
00:32:49.679 um we it's taking our two tests and
00:32:56.000 it's running through these various mutations in this case it made 83 mutations to our code
00:33:02.080 based on what we used right like so depending on like if you use an and it
00:33:08.080 will convert it to an or but if you don't use that you can't you do that mutation um so in this case there was 83
00:33:14.480 mutations uh 83 sort of mutants and 82 of those mutants were killed
00:33:19.679 so um there in this case was one that was not and you get this really
00:33:25.760 cool output uh diff output so it basically says this is the mutation we did that was not
00:33:32.159 killed we took uh what is it line 24
00:33:38.720 what is that is there a comment we took uh line 25 right this range method
00:33:46.480 and we deleted the code that you wrote and we mutated in this way we got rid of that minus t
00:33:53.519 and it turned out that even after we made that mutation all of your tests still passed
00:34:01.039 actually maybe it would be helpful like i can show with earth so before we do earth
00:34:09.280 this is what the mutation output would look like right so i just want to give you a sense of like all the different mutations and
00:34:15.200 kind of how they work and what the output looks like so if we don't have the sort of unhappy path where it
00:34:20.960 returns false these are the various mutations it runs so there was this one which we saw earlier where it removes the minus t
00:34:27.359 from the range and it still passes because we're sort of in the top half of that range
00:34:33.040 there's this other one where it gets rid of the n so the beginning part of the range and
00:34:38.079 it just puts in t there uh here it actually gets rid of that
00:34:43.599 call to dot cover and it turns out that because the range returns true and you
00:34:49.119 haven't put in a thing that says it should return false um that this also passes
00:34:54.480 right so in this case you're just returning the range but that is truthy and so this uh
00:35:01.440 this test fails if you wanted to write a more precise test instead of saying uh no i guess that's
00:35:07.119 right so in this case it's just gonna check whether uh that method is truthy or falsy and in
00:35:13.520 this case it's truthy if it just returns the range right and you're not testing that it would ever be falsy
00:35:21.280 also if you just return the instance variable area so if you basically throw away everything except that last argument to
00:35:28.560 the cover method this turns out to also um like you have no test that covers this right
00:35:35.680 and actually you can delete that whole line and the previous line approximate area
00:35:40.960 um like you get the same result right like the fact that you have an approximate area and that is truthy and that you are only
00:35:46.160 testing that this method returns a truthy value means that this test will pass
00:35:52.079 so um i just wanted to show that i can bring
00:35:57.920 this back cool so now we're in a place where
00:36:04.560 oops okay so our tests will pass and
00:36:11.520 we have one mutant that we need to kill so does anyone have an idea for how to
00:36:17.040 kill this mutant
00:36:30.240 so uh the suggestion was to pass in a zero tolerance so let's try that so uh should i just um should we make up
00:36:38.240 a planet or how do you want to do that we could do mars maybe
00:36:44.079 ah venus shouldn't be spherical if the tolerance is zero so that's true so we can sort of change
00:36:51.040 this one to be it is spherical given the default tolerance
00:36:57.280 that's what that tests right it's spherical ish i like that
00:37:04.320 um but is not perfectly spherical
00:37:13.040 and so uh here we use expect this not to be spherical given a
00:37:18.400 tolerance of zero yeah so let's first run that test
00:37:26.320 cool so that passes it is not perfectly spherical and it is spherical-ish um
00:37:34.079 we didn't break that test okay so now let's uh do the same thing with our mutant command
00:37:44.640 so the mutant still lives wipe
00:37:50.800 so to make this fail what we need to do is we need to pass in a tolerance that
00:37:56.320 falls in the bottom half of the range so in this case venus is
00:38:01.520 slightly the area of venus is slightly above the perfect
00:38:07.880 spherosphericism or whatever right um it's not uh
00:38:13.040 it's on the high end of the range so what we need to do is we need to find a planet that is actually uh
00:38:19.760 on the low end of the range right where it's less it's uh spherical
00:38:25.920 but within the tolerance but it's uh yeah on the low end of the range
00:38:30.960 make sense so um
00:38:39.040 yeah i don't know like what we could do to test like we could i don't want to necessarily like look up more planets
00:38:44.560 and their radiuses but we could do something like this so um this is uh sorry that's not earth
00:38:52.480 this is like um rubinius5 i like that thank you for the
00:38:59.520 suggestion from the audience and rubinius5 uh let's sort of make it
00:39:05.040 easy for ourselves so we'll say the radius is
00:39:10.480 0.5 right so if we put that in our formula uh 0.5
00:39:17.599 squared is a quarter and then um a quarter when sort of cancels out the
00:39:25.359 multiple by four you you're dividing by four basically so the we know that the actual
00:39:32.400 area should be pi so then we can just say something like let the area
00:39:37.760 be uh math pi um and we want it to fall uh
00:39:45.680 we want the area to be below the range right so we want it to be like math pi minus like some amount that
00:39:51.760 falls within the tolerance or whatever right makes sense and then
00:39:56.800 we expect that this is uh going to be spherical
00:40:04.400 ish within the default tolerance
00:40:10.079 cool
00:40:18.560 okay so let's run that specs pass and have we killed the last
00:40:26.000 mutant
00:40:32.160 nice we have yeah so
00:41:01.760 you