Skip to content

Commit 96eb150

Browse files
ahardin-rhahardin-rh
authored
Merge pull request #13254 from ahardin-rh/routing-optimization-4-0
Added routing optimization content to 4.0
2 parents b8e0a32 + ab12162 commit 96eb150

File tree

5 files changed

+168
-9
lines changed

5 files changed

+168
-9
lines changed

‎_topic_map.yml

Lines changed: 2 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -270,13 +270,8 @@ Topics:
270270
File: network-optimization
271271
- Name: Optimizing storage
272272
File: optimizing-storage
273-
#---
274-
#Name: Storage
275-
#Dir: storage
276-
#Distros: openshift-*
277-
#Topics:
278-
#- Name: Persistent Storage Using AWS
279-
# File: persistent-storage-aws
273+
- Name: Routing optimization
274+
File: routing-optimization
280275
---
281276
Name: Nodes
282277
Dir: nodes

‎backup_and_restore/PLACEHOLDER

Lines changed: 0 additions & 2 deletions
This file was deleted.

‎modules/baseline-router-performance.adoc

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
[id='baseline-router-performance-{context}']
2+
= Baseline router performance
3+
4+
The {product-title} router is the ingress point for all external traffic
5+
destined for {product-title} services.
6+
7+
When evaluating a single HAProxy router performance in terms of
8+
HTTP requests handled per second, the performance varies depending
9+
on many factors. In particular:
10+
11+
* HTTP keep-alive/close mode
12+
13+
* route type
14+
15+
* TLS session resumption client support
16+
17+
* number of concurrent connections per target route
18+
19+
* number of target routes
20+
21+
* back end server page size
22+
23+
* underlying infrastructure (network/SDN solution, CPU, and so on)
24+
25+
////
26+
COMMENTING OUT METRICS UNTIL WE GET NEW DATA
27+
While performance in your specific environment will vary, Red Hat lab
28+
tests on a public cloud instance of size 4 vCPU/16GB RAM, a single
29+
HAProxy router handling 100 routes terminated by backends serving
30+
1kB static pages is able to handle the following number of transactions
31+
per second.
32+
33+
In HTTP keep-alive mode scenarios:
34+
35+
[cols="3",options="header"]
36+
|===
37+
|*Encryption* |*ROUTER_THREADS unset*|*ROUTER_THREADS=4*
38+
|none |23681|24327
39+
|edge |14981|22768
40+
|passthrough |34358|34331
41+
|re-encrypt |13288|24605
42+
|===
43+
44+
In HTTP close (no keep-alive) scenarios:
45+
46+
[cols="3",options="header"]
47+
|===
48+
|*Encryption* |*ROUTER_THREADS unset*|*ROUTER_THREADS=4*
49+
|none |3245|4527
50+
|edge |1910|3043
51+
|passthrough |3408|3922
52+
|re-encrypt |1333|2239
53+
|===
54+
55+
TLS session resumption was used for encrypted routes. With HTTP
56+
keep-alive, a single HAProxy router is capable of saturating 1 Gbit
57+
NIC at page sizes as small as 8 kB.
58+
59+
When running on bare metal with modern processors, you can expect roughly
60+
twice the performance of the public cloud instance above. This
61+
overhead is introduced by the virtualization layer in place on public clouds and
62+
holds mostly true for private cloud-based virtualization as well. The following
63+
table is a guide on how many applications to use behind the router:
64+
65+
[cols="2,4",options="header"]
66+
|===
67+
|*Number of applications* |*Application type*
68+
|5-10 |static file/web server or caching proxy
69+
|100-1000 |applications generating dynamic content
70+
71+
|===
72+
73+
In general, HAProxy can support routes for 5 to 1000 applications, depending on
74+
the technology in use. Router performance might be limited by the
75+
capabilities and performance of the applications behind it, such as language or
76+
static versus dynamic content.
77+
////
78+
79+
Router sharding should be used to serve more routes towards applications and
80+
help horizontally scale the routing tier.

‎modules/router-performance-optimizations.adoc

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
[id='router-performance-optimizations-{context}']
2+
= Router performance optimizations
3+
4+
*Setting the maximum number of connections*
5+
6+
One of the most important tunable parameters for HAProxy scalability is the
7+
`maxconn` parameter, which sets the maximum per-process number of concurrent
8+
connections to a given number. Adjust this parameter by editing the
9+
`ROUTER_MAX_CONNECTIONS` environment variable in the {product-title} HAProxy
10+
router's deployment configuration file.
11+
12+
[NOTE]
13+
====
14+
A connection includes the front end and internal back end. This counts as two
15+
connections. Be sure to set `ROUTER_MAX_CONNECTIONS` to double than the number
16+
of connections you intend to create.
17+
====
18+
19+
*CPU and interrupt affinity*
20+
21+
In {product-title}, the HAProxy router runs as a single process. The
22+
{product-title} HAProxy router typically performs better on a system with fewer
23+
but high frequency cores, rather than on an symmetric multiprocessing (SMP)
24+
system with a high number of lower frequency cores.
25+
26+
Pinning the HAProxy process to one CPU core and the network interrupts to
27+
another CPU core tends to increase network performance. Having processes and
28+
interrupts on the same non-uniform memory access (NUMA) node helps avoid memory
29+
accesses by ensuring a shared L3 cache. However, this level of control is
30+
generally not possible on a public cloud environment. On bare metal hosts,
31+
`irqbalance` automatically handles peripheral component interconnect (PCI)
32+
locality and NUMA affinity for interrupt request lines (IRQs). On a cloud
33+
environment, this level of information is generally not provided to the
34+
operating system.
35+
36+
CPU pinning is performed either by `taskset` or by using HAProxy's `cpu-map`
37+
parameter. This directive takes two arguments: the process ID and the CPU core
38+
ID. For example, to pin HAProxy process `1` onto CPU core `0`, add the following
39+
line to the global section of HAProxy's configuration file:
40+
41+
----
42+
cpu-map 1 0
43+
----
44+
45+
*Increasing the number of threads*
46+
47+
The HAProxy router comes with support for multithreading in {product-title}. On
48+
a multiple CPU core system, increasing the number of threads can help the
49+
performance, especially when terminating SSL on the router.
50+
51+
*Impacts of buffer increases*
52+
53+
The {product-title} HAProxy router request buffer configuration limits the size
54+
of headers in incoming requests and responses from applications. The HAProxy
55+
parameter `tune.bufsize` can be increased to allow processing of larger headers
56+
and to allow applications with very large cookies to work, such as those
57+
accepted by load balancers provided by many public cloud providers. However,
58+
this affects the total memory use, especially when large numbers of connections
59+
are open. With very large numbers of open connections, the memory usage will be
60+
nearly proportionate to the increase of this tunable parameter.
61+
62+
*Optimizations for HAProxy reloads*
63+
64+
Long-lasting connections, such as WebSocket connections, combined with
65+
long client/server HAProxy timeouts and short HAProxy
66+
reload intervals, can cause instantiation of many HAProxy processes.
67+
These processes must handle old connections, which were started
68+
before the HAProxy configuration reload. A large number of these processes is
69+
undesirable, as it will exert unnecessary load on the system and can
70+
lead to issues, such as out of memory conditions.
71+
72+
Router environment variables affecting this
73+
behavior are `ROUTER_DEFAULT_TUNNEL_TIMEOUT`, `ROUTER_DEFAULT_CLIENT_TIMEOUT`,
74+
`ROUTER_DEFAULT_SERVER_TIMEOUT`, and `RELOAD_INTERVAL` in particular.

‎scalability_and_performance/routing-optimization.adoc

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
[id='routing-optimization']
2+
= Routing optimization
3+
include::modules/common-attributes.adoc[]
4+
:context: routing-optimization
5+
6+
toc::[]
7+
8+
The {product-title} HAProxy router scales to optimize performance.
9+
10+
include::modules/baseline-router-performance.adoc[leveloffset=+1]
11+
12+
include::modules/router-performance-optimizations.adoc[leveloffset=+1]

0 commit comments

Comments
 (0)