1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
|
## Limitations
* When **forking** within an application it is not valid for a cryptographic
operation to be started in the parent process, and completed in the child
process.
* Only **one level of forking is permitted**, if a child process forks again then
the Intel® QAT OpenSSL\* Engine will not be available in that forked
process.
* **Event driven mode** of polling operation is not supported in the FreeBSD
Operating system or in the qatlib RPM.
* QAT Engine doesn't support **ENCRYPT_THEN_MAC**(default) mode of operation meaning
when Encrypt then MAC is negotiated for symmetric ciphers say AES-CBC, the requests will not
get offloaded via QAT_HW, instead uses OpenSSL SW. Disable ENCRYPT_THEN_MAC with the flag
`SSL_OP_NO_ENCRYPT_THEN_MAC` programmatically using SSL_CTX_set_options() to offload
symmetric chained ciphers via QAT_HW. Please note disabling ENCRYPT_THEN_MAC has security
implications.
* OpenSSL 1.1.1n and OpenSSL 3.0.2 introduced misleading error message(undefined symbol: **EVP_PKEY_get_base_id**)
during engine load which can be ignored as it is not a real failure. This is later fixed in
OpenSSL\* 1.1.1o and OpenSSL\* 3.0.3 release.
* QAT Engine built for OpenSSL3.0 is only compatible with dependent libraries also linked with OpenSSL3.0
libraries due to [OpenSSL#17112](https://github.com/openssl/openssl/pull/17112). Same applies for OpenSSL 1.1.1.
* HKDF based on SM3 is not supported in QAT_HW, The request will fallback to OpenSSL software if
fallback been enabled otherwise failures are observed.
* There is a limitation with thread specific USDM: *memory allocated in one thread
should be freed only by the thread which allocates it*. When the QAT driver is configured
with `--enable-icp-thread-specific-usdm`, and when QAT_engine is used as the default
OpenSSL engine, it is required that OPENSSL_init_ssl() be called from the same thread that
calls OPENSSL_cleanup(). Incorrect cleanup can lead to a segmentation fault (segfault).
Also, memory allocated in a thread is freed automatically when the thread exits/terminates,
even if the user does not explicitly free the memory.
* SVM mode is not supported with BoringSSL library and KPT mode.
* AES-CCM ciphers are not enabled in OpenSSL by default. Need to enable it manually using the openssl.cnf
section as below
```
openssl_conf = cipher_conf
[cipher_conf]
ssl_conf = cipher_sect
[cipher_sect]
system_default = system_cipher_sect
[system_cipher_sect]
Cipherstring = ALL
Ciphersuites = TLS_AES_128_CCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256
```
* FreeBSD qatlib Header files are not installed to the install path due to a known issue in the driver.
Header needs to be manually copied in the default path (/usr/local) as below or to the
install path specified during driver installation.
```
# Create destination directory from the QATlib 23.09 top directory:
./install-sh -c -d '/usr/local/include/qat'
# Install Header Files:
/usr/bin/install -c -m 644 quickassist/include/cpa.h quickassist/include/cpa_dev.h quickassist/include/cpa_types.h quickassist/include/lac/cpa_cy_common.h quickassist/include/lac/cpa_cy_dh.h quickassist/include/lac/cpa_cy_drbg.h quickassist/include/lac/cpa_cy_dsa.h quickassist/include/lac/cpa_cy_ecdh.h quickassist/include/lac/cpa_cy_ecdsa.h quickassist/include/lac/cpa_cy_ecsm2.h quickassist/include/lac/cpa_cy_ec.h quickassist/include/lac/cpa_cy_im.h quickassist/include/lac/cpa_cy_key.h quickassist/include/lac/cpa_cy_kpt.h quickassist/include/lac/cpa_cy_ln.h quickassist/include/lac/cpa_cy_nrbg.h quickassist/include/lac/cpa_cy_prime.h quickassist/include/lac/cpa_cy_rsa.h quickassist/include/lac/cpa_cy_sym_dp.h quickassist/include/lac/cpa_cy_sym.h quickassist/include/dc/cpa_dc.h quickassist/include/dc/cpa_dc_dp.h quickassist/include/dc/cpa_dc_chain.h quickassist/lookaside/access_layer/include/icp_sal_poll.h quickassist/lookaside/access_layer/include/icp_sal_user.h quickassist/lookaside/access_layer/include/icp_sal.h quickassist/lookaside/access_layer/include/icp_sal_versions.h quickassist/utilities/libusdm_drv/qae_mem.h /usr/local/include/qat
```
* HKDF infolen > 80 is not supported due to QAT driver limitation.
* Symmetric keys are not protected by [Key Protection Technology](#qat_hw_kpt.md).
* QAT Engine does not process the plaintext if the length is not a multiple of AES_BLOCK_SIZE for
chained cipher AES-CBC-HMAC-SHA when built with OpenSSL v3 and above. No padding would be added
to the plaintext as specified in RFC 5652 or RFC 5246.
## Known Issues
### Functional
* AES-CBC-HMAC-SHA chained ciphers does not support **pipeline feature** when built with
OpenSSL 3.0 as the corresponding support is not available in OpenSSL 3.0 -
[OpenSSL#18298](https://github.com/openssl/openssl/issues/18298)
* There is an issue in **sshd** daemon application when using the QAT for default openssl.
sshd looks to be closing the file descriptors associated with QAT engine and driver after
initialising openssl. Similar issue was present which prevents the ability to ssh out of
the system using the QAT engine in versions of the ssh application before OpenSSH 8.7.
The issue has been fixed with this commit [c9f7bba](https://github.com/openssh/openssh-portable/commit/c9f7bba2e6f70b7ac1f5ea190d890cb5162ce127)
This update can be applied to sshd to work-around the issue.
* Known issue with QAT_SW SM2 in `ntls` mode since QAT_SW SM2 doesn't have plain sign and
verify operation support in engine. Disable QAT_SW SM2 to workaround the issue with ntls.
No issues with TLS mode since it uses digestsign and digestverify which is supported.
* Known issue in Software fallback with OpenSSL3.0 Engine(only) when disabled via co-existence
algo bitmap for algorithms PRF, HKDF, SM2 & SM3. QAT_HW PRF and QAT_HW HKDF are
not accelerated in OpenSSL 3.0 engine due to the issue [OpenSSL#21622](https://github.com/openssl/openssl/issues/21622)
* Known issue in Co-existence mode with QAT provider on OpenSSL 3.2 and above during QAT_SW offload
when QAT_HW modules are not present.
* Known build issue with the latest commit of BoringSSL; hence, IPP Crypto 2021.10 should be used
for the QAT engine with BoringSSL (use the BoringSSL commit mentioned in the Software requirements section).
### Performance
* There is known performance scaling issue (performance drop with threads >32)
with ECDSA Ciphers in the QAT Software acceleration using multithread mode
in the Haproxy application. This issue is not observed when using RSA ciphers
or in multi-process mode.
* SM3 is disabled by default due to performance drop observed in **multithread scenario**
for all ciphers suites due to the additional locks at engine_table_select introduced by
engine digest registration in OpenSSL - [OpenSSL#18509](https://github.com/openssl/openssl/issues/18509)
* In Co-Existence mode, Performance is lower than QAT_SW only when process number >=64 due
to known issue.
* Note regarding multithreaded performance with OpenSSL/*: In some cases, using QAT_Engine with
OpenSSL at higher thread counts can produce *worse* performance, due to issues in the way OpenSSL
handles higher thread counts. Check for `native_queued_spin_lock_slowpath()` consuming CPU process
idle time, and see the OpenSSL GitHub issues and web articles below.
- Performance bottleneck with locks in engine_table_select() function - [OpenSSL#18509](https://github.com/openssl/openssl/issues/18509)
- 3.0 performance degraded due to locking - [OpenSSL#20286](https://github.com/openssl/openssl/issues/20286)
- https://serverfault.com/questions/919552/why-having-more-and-faster-cores-makes-my-multithreaded-software-slower
- https://superuser.com/questions/1737747/high-system-cpu-usage-on-linux
* Nginx Handshake Performance in OpenSSL3.0 is slightly slower compared to OpenSSL 1.1.1. The same
behaviour is observed in OpenSSL_SW as well [OpenSSL#21833](https://github.com/openssl/openssl/issues/21833)
* Performance scaling is not linear in QAT2.0 supported platforms for ECDSA and Chacha-Poly algorithms.
* Performance drop observed with ECDSAP256 algorithm in the OpenSSL speed tests with FreeBSD 14 intree driver.
* Performance drop observed in QAT Engine with [async-nginx](https://github.com/intel/asynch_mode_nginx/tree/master) on FreeBSD OS with asymmetric and symmetric ciphers.
* BoringSSL on FreeBSD OS is validated functionally with limited performance validation on Nginx QUIC POC.
|
