Functional test for HTTPS

[krizhanovsky]

Need to develop functional tests for following HTTPS aspects. Consider original mbedTLS test suite for porting to implement the validations.

TLS tests

• stress test for new HTTPS connections establishing
• Test data consistency transfered through Tempesta (e.g. compare hash of an original file and received on client side). Responses and requests passed to Tempesta must have different sizes.
• Exceeding processing limits in handshake messages, e.g. more than TTLS_HS_CS_MAX_SZ cipher suites, other limits also must be tested
• Test certificates and private keys for RSA and EC ciphers and SHA256 and SHA512 hash. Unsupported certs (e.g. signed with SHA1) must be correctly declined by Tempesta FW (add a note to the Wiki)
• Try to establish SSL {1,2,3}.0 and TLS 1.{0,1} and verify that the connections are correctly terminated
• correct blocking of TLS messages (handshake, data, alerts, change cipher spec) with incorrect header values (type, version, length etc.) on all TLS session phases (not only handshakes)
• TLS false start (TLS: False Start #834, ALPN is required on client side). (The RFC seems dead, so we're not going to implement it)
• per-vhost certificates handling and SNI extensions for different vhosts listening on the same IP (Per-vhost TLS configuration #715)
• TLS extensions described in RFC 6066 (not all of them are supported), including wrong extension size and types in ClientHello
• TLS ticket-based (RFC 5077) sessions resumption (we don't support old-fashion session resumption by session ID) (The functionality isn't implemented yet, so the test was moved to TLS sessions resumption #1054.)
• HSTS, depends on HTTP: transform custom HTTP headers #511 since Strict-Transport-Security header must be added and Custom HTTP redirects #856. Test the new header together with a redirection. (The full support of HSTS requires HTTP redirects (Custom HTTP redirects #856) while simple header additional is already tested in vhost/test_add_hdr.py, so there is nothing to do at this time and I just added the test requirement to Custom HTTP redirects #856.)
• Some attacks from RFC 7457 are good to test (regardless most of them are outdated)
• stale certificates with overdue less than 1-minute don't pass validation.
• dynamic optimal TLS record size: egress TLS records have size as min(send_buffer, cwnd, rwnd) and a received decrypts a TLS record from Tempesta after single transmission.
• send TLS records by 1 byte to test the FSM robustness.
• Frankencert fuzzing (see the original paper) including SSL {1,2,3}.0 emulation. (We have not certificates validation logic).
• send file using sendfile or from cache (TLS: Memory corruption #1217)

Consider to use TLSfuzzer, thc-ssl-dos, and ssl-dos tools for long running CI tests.

Deproxy HTTPS support

Nowadays most HTTP transactions are over TLS and there are almost no plain-text HTTP any more, so please update existing tests (e.g. for web cache) to use HTTPS as well as plain text. HTTPS mode must be default for the tests hereafter.

The effort for HTTPS deproxy was made in https://github.com/tempesta-tech/tempesta-test/compare/vlad-https , but the solution seems too complicated and error prone (see https://github.com/tempesta-tech/tempesta-test/pull/96/files#r293164966). Probably it makes sense to use httplib.HTTPSConnection with ready HTTP message abstractions for deproxy rather than ssl module plus our home brew HTTP messages.

[krizhanovsky]

Done with tempesta-tech/tempesta-test#103