## 1  **Test Methodology**

<h1> Simple Web Server Test Plan</h1>

<h2>Objective </h2>
To specify a range of test, and the success metrics, required to qualify the release of the functionality provided by the Simple Web Server Tool ( https://bugs.openjdk.java.net/browse/JDK-8260510)

The range of testing will cover functional tests, reliability tests, stress (stability), scale, and security tests. Additionally,  elementary performance analysis and code coverage analysis will be carried.

<h2> Testing Scope</h2>

<h3> What's in scope </h3>

 - Functional Tests, reliability tests, stress and stability tests, scale tests, interoperability tests and security test. The latter taking guidance from the OWASP framework - Top ten web security issues https://owasp.org/www-project-top-ten/ and test guide  https://owasp.org/www-project-web-security-testing-guide/

- API test. The SWS exposes internal APIs, as such a full range of API tests will be developed similar to JCK level testing


<h3>What's out of scope </h3>

 - Formal JCK testing. However, there will be equivalent API level tests.


## 2  **Test Inventory**

<h2>Regression Tests Inventory </h2>

<h3>1. Functional tests </h3>

**Component Functional test cases**
|  tests | Count |
|-----------------------|------|
|???? |  |
|???? | |


**Command line test cases**
|  tests | Count |
|-----------------------|------|
|???? |  |
|???? | |

<h3>2. API tests</h3>

The SWST has an API, and this should be extensively tested for functional correctness and operational rubustness.

**JCK equivalent tests**
| tests | Count |
|-----------------------|------|
| ???? |?? |
| ???? |?? |

<h3>3. SPI tests</h3>

**SPI  tests**
| tests | Count |
|-----------------------|------|
| ???? |?? |
| ???? |?? |

<h3>4. HTTP Response Conformance tests</h3>
Establish that the server returns the correct HTTP responses for relevant 

**HTTP Resposne tests**
| tests | Count |
|-----------------------|------|
| ???? |?? |
| ???? |?? |

<h3>5. HTML Response Conformance tests</h3>

Validate that any HTML response bodies are well formed and correct
HTML validation

**HTML Response Validation tests**
| tests | Count |
|-----------------------|------|
| ???? |?? |
| ???? |?? |

<h3>6. HTML Response WCAG Conformance tests</h3>

Validate that any HTML response bodies are valid as per WCAG (Web content accessibility guidelines - e.g. screenreaders)

**HTML Response Validation tests**
| tests | Count |
|-----------------------|------|
| ???? |?? |
| ???? |?? |


<h2>Other Test Inventory </h2>

<h2>Stability, Reliability and Scale Tests </h2>

*Reliability is defined as the probability that a product, system, or service will perform its intended function adequately for a specified period of time, or will operate in a defined environment without failure.*

*Stability is a complex term but for the purposes of this use case can be defined as reliability at a given scale*

The main objective is to establish that the SWST performs reliably and stable under known conditions. Also, that a know set of metrics are establish in terms of scale, performance and throughput. In performing reliably it should be able to provide continuous service of a defined number of days, and should not exhibit any appreciable or excessive resource retention, in particular excessive memory growth. Tests to establish the number of connections and number of requests per second the SWST will handle

**Reliability tests scenarios**
| tests | Count |
|-----------------------|------|
| ??? | ??? |

**Stability tests scenarios**
| tests | Count |
|-----------------------|------|
| ??? | ??? |

**Scale Metrics tests**
| tests | Count |
|-----------------------|------|
| ??? | ??? |


<h2>Performance Metrics Tests </h2>

**Stress tests**
| tests | Count |
|-----------------------|------|
| ??? | ??? |

**Request Performance Metrics tests**
| tests | Count |
|-----------------------|------|
| ??? | ??? |


<h2>Microbenchmark Tests </h2>

**TBD**
| tests | Count |
|-----------------------|------|
| ??? | ??? |



<h2>Interoperability Tests </h2>

<h3>TBD</h3>

 - Java HttpClient 
 - Apache HttpClient 
 - curl 
 - Basic Common Browser testing

<h3>4. Others</h3>

investigate basic security and vulnerability tests - OWASP Zed Attack Proxy tests

**OWASP ZAP Tests**
| tests | Count |
|-----------------------|------|
| ??? | ??? |



<h2> Success criteria </h2>

<h3> Regression Tests </h3>

|Criteria|Planned|Status |Result |Comments|
|---|---|---|----|----|
| Public API coverage | *100%*  | 100% | | |
| Block coverage of new code | *??%*  | ??% |  |  |
| Test stability | SBR runs and ATR |  | | *make sure there are no intermittent failures. All good in ATR and SBR runs* |
| Open test issues | <?? | |  | *makes sure all the tests are ready* |
| Pass Rate  | 100%  | | | |
|  |  | | |  |




<h2> Test Configurations </h2>
N/A