Nội dung

calling context, and improves testability. The second case tests the server supports CAPA (capabilities), UIDL (unique identifiers), and CRAM (challenge/response authentication). The capability list is unordered so we check the list for UIDL then CRAM or the reverse. Bivio::Test allows us to specify a Regexp instance (qr//) as the expected value. The case passes if the expected regular expression matches the actual return, which is serialized by Data::Dumper. 13.6 Validate Using Implementation Knowledge foreach my $mode (qw(BEST APOP CRAM-MD5 PASS)) { $pop3 = Mail::POP3Client->new(%$cfg, AUTH_MODE => $mode); is_deeply([$pop3->Body(1)], $body_lines); is($pop3->Close, 1); } $pop3 = Mail::POP3Client->new(%$cfg, AUTH_MODE => ’BAD-MODE’); like($pop3->Message, qr/BAD-MODE/); is($pop3->State, ’AUTHORIZATION’); is($pop3->Close, 1); $pop3 = Mail::POP3Client->new( %$cfg, AUTH_MODE => ’BEST’, PASSWORD => ’BAD-PASSWORD’); like($pop3->Message, qr/PASS failed/); is($pop3->State, ’AUTHORIZATION’); is($pop3->Close, 1); $pop3 = Mail::POP3Client->new( %$cfg, AUTH_MODE => ’APOP’, PASSWORD => ’BAD-PASSWORD’); like($pop3->Message, qr/APOP failed/); is($pop3->Close, 1); Once we have validated the server’s capabilities, we test the authentication interface. Mail::POP3Client defaults to AUTH MODE BEST, but we test each mode explictly here. The other cases test the default mode. To be sure authentication was successful, we download the body of the first message and compare it with the value we sent. POP3 authentication implies Copyright c 2004 Robert Nagler All rights reserved nagler@extremeperl.org 115 authorization to access your messages. We only know we are authorized if we can access the mail user’s data. In BEST mode the implementation tries all authentication modes with PASS as the last resort. We use knowledge of the implementation to validate that PASS is the last mode tried. The Message method returns PASS failed, which gives the caller information about which AUTH MODE was used. The test doesn’t know the details of the conversation between the server and client, so it assumes the implementation doesn’t have two defects (using PASS when it shouldn’t and returning incorrect Message values). We’ll see in Mock Objects how to address this issue without such assumptions. The authentication conformance cases are incomplete, because there might be a defect in the authentication method selection logic. We’d like know if we specify APOP that Mail::POP3Client doesn’t try PASS first. The last case group in this section attempts to test this, and uses the knowledge that Message returns APOP failed when APOP fails. Again, it’s unlikely Message will return the wrong error message. 13.7 Distinguish Error Cases Uniquely sub _is_match { my($actual, $expect) = @_; return ref($expect) eq ’Regexp’ ? like(ref($actual) ? join(’’, @$actual) : $actual, $expect) : is_deeply($actual, $expect); } $pop3 = Mail::POP3Client->new(%$cfg); foreach my $params ( [Body => $body_lines], [Head => qr/\Q$subject/], [HeadAndBody => qr/\Q$subject\E.*\Q$body_lines->[0]/s], ) { my($method, $expect) = @$params; _is_match([$pop3->$method(1)], $expect); is($pop3->Message(’’), ’’); is_deeply([$pop3->$method(999)], []); like($pop3->Message, qr/No such message|Bad message number/i); } Copyright c 2004 Robert Nagler All rights reserved nagler@extremeperl.org 116 The Body method returns the message body, Head returns the message head, and HeadAndBody returns the entire message. We assume that 999 is a valid message number and that there aren’t 999 messages in the mailbox. Body returns an empty array when a message is not found. Should Body return something else or die in the deviance case? I think so. Otherwise, an empty message body is indistinguishable from a message which isn’t found. The deviance test identifies this design issue. That’s one reason why deviance tests are so important. To workaround this problem, we clear the last error Message saved in the Mail::POP3Client instance before calling the download method. We then validate that Message is set (non-blank) after the call. The test case turned out to be successful unexpectedly. It detected a defect in Message: You can’t clear an existing Message. This is a side-effect of the current test, but a defect nonetheless. One advantage of validating the results of every call is that you get bonuses like this without trying. 13.8 Avoid Context Sensitive Returns foreach my $params ( [Body => $body], [Head => qr/\Q$subject/], [HeadAndBody => qr/\Q$subject\E.*\Q$body/s], ) { my($method, $expect) = @$params; _is_match(scalar($pop3->$method(1)), $expect); is(scalar($pop3->$method(999)), undef); } When Body, Head, and HeadAndBody are invoked in a scalar context, the result is a single string, and undef is returned on errors, which simplifies deviance testing. (Note that Bivio::Test distinguishes undef from [undef]. The former ignores the result, and the latter expects a single-valued result of undef.) Bivio::Test invokes methods in a list context by default. Setting want scalar forces a scalar context. This feature was added to test nonCopyright c 2004 Robert Nagler All rights reserved nagler@extremeperl.org 117 bOP classes like Mail::POP3Client. In bOP, methods are invocation context insensitive. Context sensitive returns like Body are problematic.4 We use wantarray to ensure methods that return lists behave identically in scalar and list contexts. In general, we avoid list returns, and return array references instead. 13.9 Use IO::Scalar for Files foreach my $params ( [BodyToFile => $body], [HeadAndBodyToFile => qr/\Q$subject\E.*\Q$body/s], ) { my($method, $expect) = @$params; my($buf) = ’’; is($pop3->$method(IO::Scalar->new(\$buf), 1), 1); _is_match($buf, $expect); } BodyToFile and HeadAndBodyToFile accept a file glob to write the message parts. This API design is easily testable with the use of IO::Scalar, an in-memory file object. It avoids file naming and disk clean up issues. We create the IO::Scalar instance in compute params, which Bivio::Test calls before each method invocation. check return validates that the method returned true, and then calls actual return to set the return value to the contents of the IO::Scalar instance. It’s convenient to let Bivio::Test perform the structural comparison for us. 13.10 Perturb One Parameter per Deviance Case foreach my $method (qw(BodyToFile HeadAndBodyToFile)) { is($pop3->$method(IO::Scalar->new(\(’’)), 999), 0); my($handle) = IO::File->new(’> /dev/null’); $handle->close; is($pop3->$method($handle, 1), 0); } 4 The book Effective Perl Programming by Joseph Hall discusses the issues with wantarray and list contexts in detail. Copyright c 2004 Robert Nagler All rights reserved nagler@extremeperl.org 118 We test an invalid message number and a closed file handle5 in two separate deviance cases. You shouldn’t perturb two unrelated parameters in the same deviance case, because you won’t know which parameter causes the error. The second case uses a one-time compute params closure in place of a list of parameters. Idioms like this simplify the programmer’s job. Subject matter oriented programs use idioms to eliminate repetitious boilerplate that obscures the subject matter. At the same time, idioms create a barrier to understanding for outsiders. The myriad Bivio::Test may seem overwhelming at first. For the test-first programmer, Bivio::Test clears away the clutter so you can see the API in action. 13.11 Relate Results When You Need To foreach my $method (qw(Uidl List ListArray)) { my($first) = ($pop3->$method())[$method eq ’List’ ? 0 : 1]; ok($first); is_deeply([$pop3->$method(1)], [$first]); is_deeply([$pop3->$method(999)], []); } Uidl (Unique ID List), List, and ListArray return lists of information about messages. Uidl and ListArray lists are indexed by message number (starting at one, so the zeroeth element is always undef). The values of these lists are the message’s unique ID and size, respectively. List returns a list of unparsed lines with the zeroeth being the first line. All three methods also accept a single message number as a parameter, and return the corresponding value. There’s also a scalar return case which I didn’t include for brevity in the book. The first case retrieves the entire list, and saves the value for the first message. As a sanity check, we make sure the value is non-zero (true). This is all we can guarantee about the value in all three cases. 5 We use IO::File instead of IO::Scalar, because IO::Scalar does not check if the instance is closed when Mail::POP3Client calls print. Copyright c 2004 Robert Nagler All rights reserved nagler@extremeperl.org 119 The second case requests the value for the first message from the POP3 server, and validates this value agrees with the value saved from the list case. The one-time check return closure defers the evaluation of $ SAVE until after the list case sets it. We cross-validate the results, because the expected values are unpredictable. Unique IDs are server specific, and message sizes include the head, which also is server specific. By relating two results, we are ensuring two different execution paths end in the same result. We assume the implementation is reasonable, and isn’t trying to trick the test. These are safe assumptions in XP, since the programmers write both the test and implementation. 13.12 Order Dependencies to Minimize Test Length my($count) = $pop3->Count(); ok($count >= 1); is($pop3->Delete(1), 1); is($pop3->Delete(999), 0); $pop3->Reset; is($pop3->Close, 1); $pop3->Connect; is($pop3->Count, $count); # Clear mailbox, which also cleans up aborted or bad test runs foreach my $i (1 .. $count) { $pop3->Delete($i); }; is($pop3->Close, 1); $pop3->Connect; is($pop3->Count, 0); is($pop3->Close, 1); We put the destructive cases (Delete) near the end. The prior tests all need a message in the mailbox. If we tested delete first, we’d have to resend a message to test the retrieval and list methods. The case ordering reduces test length and complexity. Note that we cannot guarantee anything about Count except that is at least one. A prior test run may have aborted prematurely and left another Copyright c 2004 Robert Nagler All rights reserved nagler@extremeperl.org 120 message in the test mailbox. What we do know is that if we Delete all messages from one to Count, the mailbox should be empty. The second half of this case group tests this behavior. The empty mailbox case is important to test, too. By deleting all messages and trying to login, we’ll see how Mail::POP3Client behaves in the this case. Yet another reason to delete all messages is to reset the mailbox to a known state, so the next test run starts with a clean slate. This selfmaintaining property is important for tests that access persistent data. Rerun the entire test twice in a row, and the second run should always be correct. The POP3 protocol doesn’t remove messages when Delete is called. The messages are marked for deletion, and the server deletes them on successful Close. Reset clears any deletion marks. We cross-validate the first Count result with the second to verify Reset does what it is supposed to do. 13.13 Consistent APIs Ease Testing $pop3 = Mail::POP3Client->new; is($pop3->State, ’DEAD’); is($pop3->Alive, ’’); is($pop3->Host($cfg->{HOST}), $cfg->{HOST}); is($pop3->Host, $cfg->{HOST}); $pop3->Connect; is($pop3->Alive, 1); is($pop3->State, ’AUTHORIZATION’); is($pop3->User($cfg->{USER}), $cfg->{USER}); is($pop3->User, $cfg->{USER}); is($pop3->Pass($cfg->{PASSWORD}), $cfg->{PASSWORD}); is($pop3->Pass, $cfg->{PASSWORD}); is($pop3->Login, 0); is($pop3->State, ’TRANSACTION’); is($pop3->Alive, 1); is($pop3->Close, 1); is($pop3->Alive, ’’); is($pop3->Close, 0); $pop3 = Mail::POP3Client->new; $pop3->Connect; Copyright c 2004 Robert Nagler All rights reserved nagler@extremeperl.org 121 is($pop3->Alive, ’’); is($pop3->Login, 0); is($pop3->State, ’DEAD’); This section not only tests the accessors, but also documents the State and Alive transitions after calls to Connect and Login. There’s a minor design issue to discuss. The accessor Pass does not match its corresponding named parameter, PASSWORD, like the Host and User do. The lack of uniformity makes using a map function for the accessor tests cumbersome, so we didn’t bother. Also the non-uniform return values between Alive and Close is clear. While the empty list and zero (0) are both false in Perl, it makes testing for exact results more difficult than it needs to be. 13.14 Inject Failures $pop3 = Mail::POP3Client->new(%$cfg); is($pop3->POPStat, 0); $pop3->Socket->close; is($pop3->POPStat, -1); is($pop3->Close, 0); The final (tada!) case group injects a failure before a normal operation. Mail::POP3Client exports the socket that it uses. This makes failure injection easy, because we simply close the socket before the next call to POPStat. Subsequent calls should fail. We assume error handling is centralized in the implementation, so we don’t repeat all the previous tests with injected failures. That’s a big assumption, and for Mail::POP3Client it isn’t true. Rather than adding more cases to this test, we’ll revisit the issue of shared error handling in Refactoring. Failure injection is an important technique to test error handling. It is in a different class from deviance testing, which tests the API. Instead, we use extra-API entry points. It’s like coming in through the back door without knockin’. It ain’t so polite but it’s sometimes necessary. It’s also hard to do Copyright c 2004 Robert Nagler All rights reserved nagler@extremeperl.org 122 if there ain’t no backdoor as there is in Mail::POP3Client. 13.15 Mock Objects Mock objects allow you to inject failures and to test alternative execution paths by creating doors where they don’t normally exist. Test::MockObject6 allows you to replace subroutines and methods on the fly for any class or package. You can manipulate calls to and return values from these faked entry points. Here’s a simple test that forces CRAM-MD5 authentication: use strict; use Test::More; use Test::MockObject; BEGIN { plan(tests => 3); } my($socket) = Test::MockObject->new; $socket->fake_module(’IO::Socket::INET’); $socket->fake_new(’IO::Socket::INET’); $socket->set_true(’autoflush’) ->set_false(’connected’) ->set_series(getline => map({"$_\r\n"} # Replace this line with ’+OK POP3 ’ for APOP ’+OK POP3’, ’+OK Capability list follows:’, # Remove this line to disable CRAM-MD5 ’SASL CRAM-MD5 LOGIN’, ’.’, ’+ abcd’, ’+OK Mailbox open’, ’+OK 33 419’, ))->mock(print => sub { my(undef, @args) = @_; die(’invalid operation: ’, @args) if grep(/(PASS|APOP)/i, join(’’, @args)); return 1; 6 Version 0.9 used here is available at: http://search.cpan.org/author/CHROMATIC/TestMockObject-0.09/ Copyright c 2004 Robert Nagler All rights reserved nagler@extremeperl.org 123 }); use_ok(’Mail::POP3Client’); my($pop3) = Mail::POP3Client->new( HOST => ’x’, USER => ’x’, PASSWORD => ’keep-secret’ ); is($pop3->State, ’TRANSACTION’); is($pop3->Count, 33); In BEST authentication mode, Mail::POP3Client tries APOP, CRAM-MD5, and PASS. This test makes sure that if the server doesn’t support APOP that CRAM-MD5 is used and PASS is not used. Most POP3 servers always support APOP and CRAM-MD5 and you usually can’t enable one without the other. Since Mail::POP3Client always tries APOP first, this test allows us to test the CRAM-MD5 fallback logic without finding a server that conforms to this unique case. We use the Test::MockObject instance to fake the IO::Socket::INET class, which Mail::POP3Client uses to talk to the server. The faking happens before Mail::POP3Client imports the faked module so that the real IO::Socket::INET doesn’t load. The first three methods mocked are: new, autoflush, and connected. The mock new returns $socket, the mock object. We set autoflush to always returns true. connected is set to return false, so Mail::POP3Client doesn’t try to close the socket when its DESTROY is called. We fake the return results of getline with the server responses Mail::POP3Client expects to see when it tries to connect and login. To reduce coupling between the test and implementation, keep the list of mock routines short. You can do this by trial and error, because Test::MockObject lets you know when a routine that isn’t mocked has been called. The mock print asserts that neither APOP nor PASS is attempted by Connect. By editing the lines as recommend by the comments, you can inject failures to see that the test and Mail::POP3Client works. There’s a lot more to Test::MockObject than I can present here. It can make a seemingly impossible testing job almost trivial. Copyright c 2004 Robert Nagler All rights reserved nagler@extremeperl.org 124