tcl.qa Definition/General, qa Definition/General

tcl.qa

Command tcl.qa Definition/General
Applicable release versions: AP
Category General (155)
Description
Syntax
Options
Example
Purpose
Related qa

qa

Command qa Definition/General
Applicable release versions: AP
Category General (155)
Description tests system functions and command and contains an option that is used to report bugs and suggest enhancements to Pick Systems.

How to write and run test programs are explained in this section. The guidelines for submitting reports to Pick Systems are also included.

QA Menu

To logo onto the QA accout, type the following:

to qa <return>

The Quality Control Master Menu displays. Option 14, Requests Entry, is used to report bugs and request enhancements.


Software Modification Suggestion

Option 14, Requests entry, on the Quality Control Master Menu calls the macro req. This option is used to enter or update a software modification/enhancement request. When this option is selected, the following information is prompted for.

report.by Name of person submitting request.
report.date The date the request is made.
company Company reporting the request operating system version.
o/s Operating system version.
model Type of machine request is reported from.
manufacturer Manufacturer of the hardware.
os.serial Serial number of the operating system prompting the request.
mfg.serial Serial number of the hardware.
type e)nhancement or f)ix.
description Description of request.
priority Internally assigned priority of the request.
offline.docs Document reference number for external file references.
tp.ref Test program id (tps item-id)
component Part of the o/s the request deals with.


Standards for Writing Test Programs

The file tps contains Pick/BASIC programs that test many of the features of the system. These test programs are, in most cases control programs which call other system facilities using the execute command. When a test program is written, it follows an essentially simple format.

x=2
if x=2 then
print 'passed'
end else print 'failed'

To write your own test program to be submitted to Pick Systems, follow the procedures listed below:

Each test program is self-contained and should produce an output message appropriate to the results of the test. If the string 'fail' or 'error' is located anywhere in the body of the message, the test driver program assumes the test failed. If no failure is detected, the message is checked for the presence of the string 'pass' and, if found, the test has passed.

A typical test program approach is to write a Pick/BASIC program which calls another system facility which operates on items and then compares the result to the previous result.

Account = QA
File name = TPS

The format of the tps is standard for the first 5 lines.

line 1 * short description
line 2 * a = automatic or i = interactive
line 3 * Enter the release on which the bug occured
line 4 * Today's date
line 5 * Name of test writer
line 6 Start coding

Use the print statement in FlashBASIC or Pick/BASIC rather than the crt statement for 'pass' or 'fail' message. If there are multiple failure modes, then print the program line number along with the message.

Make the test as self-contained as possible: i.e., create the files necessary to duplicate problem, create and write data to files, compile, catalog, etc.

Use the following format for files created by a test program:

file.id = '%':system(19):'.testid'
execute 'create-file ':file.id:' 1 1 (x'

Create with the test program using the x option in case the program terminates before the files are deleted within the test program.

Reproduce the problem with as few lines of code as possible.

Reproduce the problem with the smallest data set necessary to reproduce.

At the end of a test program, delete all files and items that have been created. If the test program alters anything else in the system, please change everything back to its original state.

Do not use the md as a scratch file in tests, unless it is absolutely necessary for procs, macros, or cataloged programs.

Create your own scratch file.


Test Driver

Individual test programs are selected by using the Pick/BASIC program rt. rt first checks to see if a select list is present. If so, soruses the names of the test programs to run the program from tps. If no select list has been specified, rt checks to see if any specific items have been specified following the verb rt item.list. Otherwise it runs all tests with the string *a* in the second attribute of the program in the test program file.

The first time the driver runs the selected tests, one at a time, from the first to the last. After running all the tests once, they run randomly and in definitely until stopped by type a character on the terminal running the test, or if rt is running as a phantom, a message may be sent to that line using the message command.

Each time rt selects a program to run, it updates the item in the tps.run file, adding the pib number of the process running the test and its start time. After the test is run, the item is further updated with the results of the test and the pib running information is deleted.


TPS.RUN File

Each time the driver runs a test program it updates the tps.run file. Each item-id in the file is the name of the test and the body of the item contains detail and summary information about the running of the tests.

15 runs Number of times the test is run
23 pass Number of times the test passed
17 fast Fastest run time of the test
18 slow Slowest run time of the test
19 avg Average test time in seconds (exponentially smoothed)
25 cpu Total CPU time used by this test in milli-seconds
29 num Number of times corresponding result was returned by the test
31 results Multi-valued for each unique result generated by the test

The source to the rt program and several tps is located in various files on the qa account.
Syntax
Options
Example
Purpose
Related tcl.qa