Programming Lab Assignment 6

CMPS 12L
Introduction to Programming
Lab Assignment 6
Our goal in this lab assignment is to test a slightly buggy (but different) version of the LetterHome
program from Lab Assignment 5. This time however, we will not be debugging the program, but simply
testing it. The distinction is that in testing, we seek to reveal the existence of bugs, whereas debugging is
meant to eliminate them once they have been identified. Debugging is a process of correction, while
testing is a process of evaluation, and may be subjective to some degree. To debug, one must have access
to the program source code, but this need not be the case during testing. A software tester may have only
the program’s specification (i.e. a complete description of the program’s intended operation) to go by.
Software testing may be defined as the process used to verify the correctness, completeness, security, and
quality of programs. This process generally includes, but is not limited to, a set of program executions on
strategically chosen inputs, with the intent of finding errors. How these inputs are chosen often depends
on who is doing the testing and why. Typically, when a programmer sets out to test his or her own work,
a collection of inputs is chosen that will exercise every statement in the program, by following every
possible logical pathway program execution can take. This method of testing is called branch coverage
and it belongs to a set of testing regimes known as white box testing, since one must see the source code
to design such inputs. You should carry out this form of testing on all of your programs.
Often though, software is tested by the user, or by someone working on the user’s behalf. In this setting
the source code is usually proprietary information belonging to the developer, and is therefore not
available for inspection. The testing regimes used in this scenario are known collectively as black box
testing. In this case, the tester must choose a set of program inputs designed to verify all program
specifications. Ideally, one checks that the program behaves as intended in all conceivable circumstances.
Unfortunately this is usually impossible since the entire set of possible program inputs is very large, if not
infinite. This is where subjectivity may come in. One must focus on those inputs that are most closely
related to the program’s critical functions, and which are most likely to arise in practice.
In this assignment, you will perform black box testing on the LetterHome program. The specification for
this program is the general template for the letter body, given in the comment block of the file
LetterHome.java in the Examples/lab5 section of the class website. Furthermore, it is specified that all
program inputs are integers, and that a properly formatted input file consists of any number of lines, each
containing two integers separated by a space. If the sentence code (the first integer in a pair) is out of
range, the program is required to print the error message “x is not a valid sentence code”, where x is the
integer. If the modifier code (the second integer) is out of range, the program is required to print the word
“ERROR” in place of the corresponding modifier. Note that these are not errors in the program, but
specifications as to how the program must react to bad input. Note also that it is not considered bad input
for the student to request a negative number of dollars in sentence 4. The program is not required to
behave in any particular way on files that are wrongly formatted, so your suite of test inputs need not
include files containing doubles, non-numeric strings, or similar types of wrong input.
Since this is black box testing, you should pretend that you have not actually seen the source code file
LetterHome.java, given in lab5. In fact, the source has been changed in some critical ways, of which
you are unaware, so you don’t really have to pretend. You will not be given the altered source, but
instead, you will have only the corresponding executable object file LetterHome.class to work with.
You should copy this file from the directory /afs/cats.ucsc.edu/users/f/ptantalo/public/ to
2
your own account space. Begin by creating a directory within your cs12A directory called lab6, then cd
to that directory and do
% cp /afs/cats.ucsc.edu/users/f/ptantalo/public/LetterHome.class .
Be sure to include that final dot, which means “the directory I am now in”. This command copies the
object file LetterHome.class to your lab6 directory, giving it the same name. Be sure not to copy this
file to your lab5 directory, or wherever you keep your corrected object file from that assignment, since
the cp command will overwrite any file having the same name. The point is that LetterHome in this
assignment is a different program than LetterHome in lab5.
Once you have the executable, design an input file (or files) to thoroughly test each aspect of the
specifications (recall these are defined by the comment block at the beginning of LetterHome.java from
lab5, and by the above remarks.) Notice that even for this simple program, the set of possible inputs is
infinite, since the number of dollars requested by the student in sentence 4 could be any integer (even a
negative one) . This poses a problem for you, the tester, since for all you know, the program will crash
when the dollar amount 6732 is entered, and for no other amount. You must use your best judgment in
selecting these test inputs. Your test file(s) should elicit a number of errors from the program, which are
defined to be any departure from the above specifications. Look closely at the output since these could be
just minor grammatical errors. Create a text file called errors containing a numbered list of the errors
you found in the executable LeterHome.class. Include a short one or two sentence description of each
error. Your description should in no way refer to the source file for the program, since after all, you don’t
(really) have that. Instead, just say in what way the specifications were violated.
Submit your file errors, along with any test files you created for this assignment to the lab6 submit
directory. You may find this to be one of the easiest labs to complete, but it will take some time to do it
carefully, so please do not wait until the last minute to begin.