Skip to content

Commit df783ec

Browse files
mfocuzMaxim Tyukovarthurpalves
authored
article completed(transfered from Medium) (#21)
* article completed(transfered from Medium) * Update 2022-01-10-hacking-netflix-eureka.md * chore: adjustments to text and teaser --------- Co-authored-by: Maxim Tyukov <maximty@backbase.com> Co-authored-by: Arthur Alves <arthuralves.p@gmail.com>
1 parent a079a42 commit df783ec

14 files changed

+174
-0
lines changed

_data/authors.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,3 +69,8 @@ Manisha Arya:
6969
name : "Manisha Arya"
7070
avatar : "assets/images/avatars/manisha_arya.jpg"
7171
bio : "Senior Frontend Engineer - R&D"
72+
73+
Maxim Tyukov:
74+
name : "Maxim Tyukov"
75+
avatar : "assets/images/avatars/mt.jpg"
76+
bio : "Principal Application Security Engineer - R&D"
Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
---
2+
title: "Hacking Netflix Eureka"
3+
excerpt: "Found explosed Eureka server and don't know what to do?"
4+
tags: Eureka Security
5+
authors:
6+
- Maxim Tyukov
7+
header:
8+
teaser: /assets/images/post/hacking-netflix-eureka/hacking-netflix-eureka.png
9+
teaser_alt: Hacking Netflix Eureka
10+
category: Security
11+
---
12+
13+
![](/assets/images/post/hacking-netflix-eureka/hacking-netflix-eureka.png)
14+
15+
Eureka is a service registry used to make micro-services to find each other. What can go wrong if the registry server becomes available from the internet? What if a malicious user registers his new micro-service?
16+
17+
It looks like the registry can become a point of failure from a security point of view. So that means we should properly secure it. But unfortunately, it is not always the case.
18+
19+
In this short research, you will look at Eureka from the attacker’s perspective and learn how to exploit SSRF and hijack internal infra’s or victims’ traffic. Both attack vectors are performed on the application layer. That means we can easily mitigate it on network and transport layers. We will talk about defense at the end.
20+
21+
## *Whose Registry is this?*
22+
23+
The official description for Netflix Eureka, as per their [GitHub repository](https://github.com/Netflix/eureka):
24+
25+
*"Eureka is a RESTful (Representational State Transfer) service that is primarily used in the AWS cloud for the purpose of discovery, load balancing and failover of middle-tier servers. It plays a critical role in Netflix mid-tier infra."*
26+
27+
## What is out there
28+
29+
Talking to the pentester’s community, it seems quite common to encounter the registry service “outside”, which is a common potential security misconfiguration.
30+
31+
Now, checking how many Eureka servers can be found by Google (you probably can improve this dork):
32+
![Google dork](/assets/images/post/hacking-netflix-eureka/google-dork.png)
33+
34+
35+
## Find me a microservice!
36+
37+
Let’s start with some basics to understand how it works based on test lab example:
38+
39+
![find me microservice](/assets/images/post/hacking-netflix-eureka/find-me-microservice.png)
40+
41+
Suppose we have the following:
42+
43+
1. Two services on the backend inside the local network, **webservice** and **secretservice**
44+
2. Spring Cloud Gateway (further referred to as **gateway**) configured to route all HTTP requests to **webservice**
45+
3. There is no HTTP route from the **gateway** to **secretservice**
46+
4. Registry **eureka server** is in DMZ (suppose that it is exposed by mistake)
47+
5. and user/attacker is on the internet.
48+
49+
On startup, each micro-service will send a REGISTER request to the Eureka server (and later will send a STATUS update every N seconds - 30s by default). Based on this information the **registry** knows how many instances of a particular micro-service are currently up and running and their addresses (address can be both DNS or IP).
50+
51+
But the registry itself is not a proxy. It is “passive” storage and contains info about currently running instances of particular services. They call it a mid-layer proxy.
52+
53+
To balance and proxy HTTP requests, we use **[Spring Cloud Gateway](https://spring.io/projects/spring-cloud-gateway)**. Gateway collects the same information from the Eureka server to know where it should balance requests. In the test lab, the gateway handles only one route to **webservice**:
54+
55+
```
56+
.route("webservice",
57+
r -> r.path("/**")
58+
.uri("lb://WEBSERVICE"))
59+
```
60+
61+
That means every path `"/\*\*"` (which is not registered on **gateway** separately) will be balanced ( `"lb://WEBSERVICE"` ) to the micro-service with the name **webservice**. There is no HTTP route to **secretservice**, so it is not supposed, by design, that the user will communicate with secretservice directly.
62+
63+
```
64+
@RequestMapping(value = "/")
65+
public class InternalService {
66+
67+
@RequestMapping(value = "/flag")
68+
public String available() {
69+
return "FLAG{EUREKA_PWNED}";
70+
}
71+
}
72+
```
73+
74+
Here **webservice**, on its turn, communicates with **secretservice** internally with the following code:
75+
76+
```
77+
Application application = eurekaClient.getApplication("secretservice");
78+
InstanceInfo instanceInfo = application.getInstances().get(0);
79+
String url = "http://" + instanceInfo.getIPAddr() + ":" + instanceInfo.getPort() + "/list/";
80+
Collection<User> users = restTemplate.getForObject(url, Collection.class);
81+
```
82+
83+
Eureka client keeps all info collected from the Eureka server in memory and updates it once per N seconds (i.e.: 30s by default).
84+
85+
## Exploitation
86+
87+
The main idea for attack vectors is changing HTTP routing with some profit for exploitation.
88+
89+
Getting access to the registry **we can register fake service** and try to access this service through **Spring Cloud Gateway**. After the fake service is registered, the **gateway** begins to balance requests to this service if we register the right service name.
90+
91+
At this point, multiple security issues arise.
92+
93+
* If an attacker registers a new instance of an existing micro-service and maps it to some internal micro-service, which doesn’t have a public HTTP route, the attacker can send HTTP requests to this service, which is about SSRF (Server Side Request Forgery)
94+
* **OR**, if the attacker registers his malicious service outside, both gateway and internal services will send requests to fake service outside, which means it is possible to hijack victim's traffic (+exploit XSS) or hijack internal services connections.
95+
96+
97+
### Attack Vector 1: Server Side Request Forgery
98+
99+
![Attack vector 1](/assets/images/post/hacking-netflix-eureka/attack-vector1.png)
100+
101+
For successful exploitation, we need to find a service mapped to root URL `“/”` (which is not really a requirement but provides more freedom in path walking). For the lab environment, *Spring Cloud Gateway* already maps `“/”` to *webservice*.
102+
103+
At first, find the address of the targeted micro-service to get access to (**secretservice**), you can find it on the registry:
104+
![regisrty before attack](/assets/images/post/hacking-netflix-eureka/registry-before-attack.png)
105+
106+
Then register new microservice with name **webservice** but with `_address:port_` of **secretservice** (to do this change `ipAddr`, `hostName` and port parameters in eureka registration request):
107+
108+
![registry-after-ssrf-attack](/assets/images/post/hacking-netflix-eureka/registry-after-ssrf-attack.png)
109+
110+
Now there are 2 instances of **webservice**. After **gateway** synced information from the **registry**, it begins to balance attacker requests between 2 instances of **webservice**. And due to one of the instances points to **secretservice** now, we bypass route restriction on HTTP layer and successfully perform SSRF attack.
111+
112+
In case of a successful attack, you will see 2 different pages during multiple requests, 404 error (because there is no route for `_/flag_` on legit **webservice**) and `"FLAG{EUREKA_PWNED}"`:
113+
![white label error](/assets/images/post/hacking-netflix-eureka/whitelabel-error.png)
114+
115+
![flag pwned](/assets/images/post/hacking-netflix-eureka/flag-pwned.png)
116+
117+
### Attack Vector 2: Traffic Hijack and XSS
118+
119+
![attack vector 2](/assets/images/post/hacking-netflix-eureka/attack-vector2.png)
120+
121+
In another scenario, what if an attacker will register an instance of **webservice** and point it to the malicious server? In this case, the **gateway** will begin to route HTTP traffic to the attacker-controlled server. This leads to secrets leakage (i.e.: `token` or `sessionId`) and XSS. For the browser, because the communication with the malicious server is performed in the backchannel, it looks like an official site (TLS on Gateway doesn’t help).
122+
123+
![xss](/assets/images/post/hacking-netflix-eureka/xss.png)
124+
125+
We can do the same trick by hijacking internal services calls, leading to information leaking or even more critical vulnerabilities.
126+
127+
## How to Communicate with Eureka
128+
129+
We'll attempt to communicate with Eureka with the following request
130+
131+
```
132+
POST /eureka/apps/WEBSERVICE HTTP/1.1
133+
Accept: application/json, application/*+json
134+
Accept-Encoding: gzip
135+
Content-Type: application/json
136+
User-Agent: Java/11.0.10
137+
Host: 127.0.0.1:8088
138+
Connection: keep-alive
139+
Content-Length: 1015
140+
141+
{"instance":{"instanceId":"host.docker.internal:webservice:8082","app":"WEBSERVICE","appGroupName":null,"ipAddr":"192.168.2.1","sid":"na","homePageUrl":"http://host.docker.internal:8082/","statusPageUrl":"http://host.docker.internal:8082/actuator/info","healthCheckUrl":"http://host.docker.internal:8082/actuator/health","secureHealthCheckUrl":null,"vipAddress":"webservice","secureVipAddress":"webservice","countryId":1,"dataCenterInfo":{"@class":"com.netflix.appinfo.InstanceInfo$DefaultDataCenterInfo","name":"MyOwn"},"hostName":"host.docker.internal","status":"UP","overriddenStatus":"UNKNOWN","leaseInfo":{"renewalIntervalInSecs":30,"durationInSecs":90,"registrationTimestamp":0,"lastRenewalTimestamp":0,"evictionTimestamp":0,"serviceUpTimestamp":0},"isCoordinatingDiscoveryServer":false,"lastUpdatedTimestamp":1630906180645,"lastDirtyTimestamp":1630906182808,"actionType":null,"asgName":null,"port":{"$":8082,"@enabled":"true"},"securePort":{"$":443,"@enabled":"false"},"metadata":{"management.port":"8082"}}}
142+
```
143+
144+
**However, this might fail.** If the protocol has changed somehow since this publication, the easiest way to extract the correct request is to run simple eureka client+server locally and use **Wireshark** (which takes ~10mins).
145+
146+
Check any tutorial about registry, e.g. [service-registration-and-discovery](https://spring.io/guides/gs/service-registration-and-discovery/)
147+
148+
## Defense is boring, but we have to…
149+
150+
Bear in mind that keeping Eureka **not** exposed to the internet must not be considered the main defense because it happens.
151+
152+
Security has to be implemented with a multi-layer approach, which guarantees that if one of the layers fails due to 0day or misconfiguration, other layers still keep you safe.
153+
154+
### Keep it safe on Application Layer
155+
156+
Eureka supports basic HTTP authentication; use it. Yes, basic authentication is not the best practice, and you have to share one password between multiple micro-services. But it still saves you some time after the attacker finds an exposed registry.
157+
158+
### Keep it safe on Presentation Layer
159+
160+
**mTLS** between micro-services will help against registering malicious service outside (against traffic hijack).
161+
162+
### Keep it safe on Transport/Network Layers
163+
164+
And, of course, micro-segmentation and strict firewall rules. There is no need to have network connectivity between the **gateway** and **secretservice** - it will safe from SSRF. As well as an uncontrolled outbound network connection to public networks must be prohibited - it will save from traffic hijacking (and from reverse shell :p).
165+
166+
Thanks for reading and stay safe!
167+
168+
169+

assets/images/avatars/mt.jpg

79.3 KB
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading

0 commit comments

Comments
 (0)