Both FRAD manufacturers and end users can benefit from a standard method of testing and evaluation.

Testing Criterion

• FRADs should be totally non-intrusive, i.e. the rest of the SNA network should not be aware they are being used.
• FRADs should not have any affect on SNA network or device functionality.
• FRADs should not have an adverse affect on network performance.

The test cases to be used for FRAD SNA testing should have the following characteristics:

• Tests should be publicly or commercially available.
• Test results should be completely and easily reproducible.
• Tests should provide reports containing results and performance measurements.
• Tests should provide scaleable performance.
• Tests should contain problem determination facilities.
• Tests should be easy to operate.
• Tests should be thoroughly documented and validated.
• Tests should be 'black box' and non-intrusive.
• Tests should provide comprehensive 'coverage.'
• Tests should be both 'sunny' and 'rainy' day.
• Test should be operable with SDLC, Token Ring, Ethernet, and X.25 QLLC.
• Hardware requirements should be minimal.
• Tests should be able to run with or without FRADs for comparison purposes.

• The SNA client must provide the same SNA/SDLC layer verification test results (test run, passed, and failed counts) as when the Implementation Under Test (IUT) FRAD is not present (i.e. the results with without the FRADs must be the same).
  
• The IUT must allow the same 3270/5250 functionality and processing of commands, orders, and attributes in the same manner as when directly attached to the SNA client.
• The IUT must execute the ACT Load/Performance tests with at least 70% of the performance of a directly attached benchmark 3270 Emulation Program with 4 LUs..
• The IUT must not stop or lock-up (need operator intervention) during any test.
• Any exceptions noted in the ACT Summary Report will be reviewed to determine their impact on the reliability, functionality, and performance of the IUT in a network environment.
  

ACT runs a subset of our Industry Standard SNA Development Test Facility (SDTF) tests to evaluate the non-intrusiveness, functionality, and performance of the FRAD to be tested in an SNA network environment.

A written report containing a management summary and detailed results of the tests is provided.

The method used to perform the tests and evaluation is as follows:

• Run all tests directly attached to a 3270/5250 emulator.
• Run all tests with the FRAD installed between the 3270/5250 emulator and SDTF. (Note: Only LU2 display devices will be used for all tests).

TESTING IS NOT TRAINING!!!

The idea is not to train a product to work, but to test it in a manner to insure that it conforms to SNA network formats and protocols.

The following IBM manuals contain the details of 3x74 and SNA and serve as the basis for the test cases to be run:

• SNA Format and Protocol
• SDLC Concepts
• LAN Technical Reference
• 3174 Functional Description
• 3174 Data Stream Programmers Reference.
• 5494 Functions Reference

The test cases are not designed to emulate the behavior of specific host application subsystems, but to test the functions defined in the above manuals with a high level of test coverage.

Testing Methodology

The methodology of testing an SNA FRAD implementation is as follows:

Attach SDTF and the SNA client directly and run the following tests:

• Test the layers of the SNA engine for conformance using sunny and rainy day tests.
• Test LU Type 2 for functionality.
• Stress test LU Type 2 to determine levels of performance.

Once the soundness of the underlying SNA engine has been established, the Presentation Services components of the LUs are tested. Testing of the SNA layers should precede from bottom (DLC) to top (DFC). However, equivalent test results should also be obtained running top to bottom to demonstrate that the order of execution of the tests produces identical results.

Testing of the LUs is centered around correct processing of the 3270 data streams. This functionality testing is centered controlled testing using a single LU. Once the soundness and functionality of a single LU has been established Multi-LU stress/performance tests are conducted.

The testing of the LUs can precede in any order and the individual test libraries and tests may also be run in any order. Several different orderings should be used to verify equivalent results are produced.

Stress testing of LU Type 2 should then be conducted using multiple LUs.

A diagram illustrating this methodology is shown below:

The individual test library names are shown within the boxes above. LU1/3 tests are not used for FRADs.