Skip to content
This repository was archived by the owner on Jan 29, 2025. It is now read-only.

Commit 88ad770

Browse files
togashidmtogashidm
committed
Update docs
1 parent 91282ff commit 88ad770

File tree

2 files changed

+43
-78
lines changed

2 files changed

+43
-78
lines changed

‎README.md

Lines changed: 21 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -20,51 +20,36 @@ The [extender](/extender) package at the top-level of this repo can be used to q
2020

2121
### Enabling a scheduler extender
2222

23-
Scheduler extenders are enabled by providing a scheduling policy to the default Kubernetes scheduler. An example policy looks like:
23+
Scheduler extenders are enabled by providing a scheduling configuration file to the default Kubernetes scheduler. An example of a configuration file:
2424

2525
````
26-
apiVersion: v1
27-
kind: ConfigMap
28-
metadata:
29-
name: scheduler-extender-policy
30-
namespace: kube-system
31-
data:
32-
policy.cfg: |
33-
{
34-
"kind" : "Policy",
35-
"apiVersion" : "v1",
36-
"extenders" : [
37-
{
38-
"urlPrefix": "https://tas-service.default.svc.cluster.local:9001",
39-
"apiVersion": "v1",
40-
"prioritizeVerb": "scheduler/prioritize",
41-
"filterVerb": "scheduler/filter",
42-
"weight": 1,
43-
"enableHttps": true,
44-
"managedResources": [
45-
{
46-
"name": "telemetry/scheduling",
47-
"ignoredByScheduler": true
48-
}
49-
],
50-
"ignorable": true,
51-
"tlsConfig": {
52-
"insecure": false,
53-
"certFile": "/host/certs/client.crt",
54-
"keyFile" : "/host/certs/client.key"
55-
}
56-
}
57-
]
58-
}
26+
apiVersion: kubescheduler.config.k8s.io/v1beta2
27+
kind: KubeSchedulerConfiguration
28+
clientConnection:
29+
kubeconfig: /etc/kubernetes/scheduler.conf
30+
extenders:
31+
- urlPrefix: "https://tas-service.default.svc.cluster.local:9001"
32+
prioritizeVerb: "scheduler/prioritize"
33+
filterVerb: "scheduler/filter"
34+
weight: 1
35+
enableHTTPS: true
36+
managedResources:
37+
- name: "telemetry/scheduling"
38+
ignoredByScheduler: true
39+
ignorable: true
40+
tlsConfig:
41+
insecure: false
42+
certFile: "/host/certs/client.crt"
43+
keyFile: "/host/certs/client.key"
5944
6045
````
6146

6247
There are a number of options available to us under the "extenders" configuration object. Some of these fields - such as setting the urlPrefix, filterVerb and prioritizeVerb are necessary to point the Kubernetes scheduler to our scheduling service, while other sections deal the TLS configuration of mutual TLS. The remaining fields tune the behavior of the scheduler: managedResource is used to specify which pods should be scheduled using this service, in this case pods which request the dummy resource telemetry/scheduling, ignorable tells the scheduler what to do if it can't reach our extender and weight sets the relative influence our extender has on prioritization decisions.
6348

64-
With a policy like the above as part of the Kubernetes scheduler configuration the identified webhook becomes part of the scheduling process.
49+
With a configuration like the above as part of the Kubernetes scheduler configuration the identified webhook becomes part of the scheduling process.
6550

6651
To read more about scheduler extenders see the [official docs](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/scheduler_extender.md).
67-
52+
6853
## Adding a new extender to Platform Aware Scheduling
6954
Platform Aware Scheduling is a single repo designed to host multiple hardware enabling Kubernetes Scheduler Extenders. A new scheduler can be added with an issue and pull request.
7055

‎telemetry-aware-scheduling/README.md

Lines changed: 22 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -46,50 +46,30 @@ If this pipeline isn't set up, and node level metrics aren't exposed through it,
4646
Note: a shell script that shows these steps can be found [here](deploy/extender-configuration). This script should be seen as a guide only, and will not work on most Kubernetes installations.
4747

4848
The extender configuration files can be found under deploy/extender-configuration.
49-
TAS Scheduler Extender needs to be registered with the Kubernetes Scheduler. In order to do this a configmap should be created like the below:
49+
TAS Scheduler Extender needs to be registered with the Kubernetes Scheduler. In order to do this a configuration file should be created like one the below:
5050
````
51-
apiVersion: v1
52-
kind: ConfigMap
53-
metadata:
54-
name: scheduler-extender-policy
55-
namespace: kube-system
56-
data:
57-
policy.cfg: |
58-
{
59-
"kind" : "Policy",
60-
"apiVersion" : "v1",
61-
"extenders" : [
62-
{
63-
"urlPrefix": "https://tas-service.default.svc.cluster.local:9001",
64-
"apiVersion": "v1",
65-
"prioritizeVerb": "scheduler/prioritize",
66-
"filterVerb": "scheduler/filter",
67-
"weight": 1,
68-
"enableHttps": true,
69-
"managedResources": [
70-
{
71-
"name": "telemetry/scheduling",
72-
"ignoredByScheduler": true
73-
}
74-
],
75-
"ignorable": true,
76-
"tlsConfig": {
77-
"insecure": false,
78-
"certFile": "/host/certs/client.crt",
79-
"keyFile" : "/host/certs/client.key"
80-
}
81-
}
82-
]
83-
}
51+
apiVersion: kubescheduler.config.k8s.io/v1beta2
52+
kind: KubeSchedulerConfiguration
53+
clientConnection:
54+
kubeconfig: /etc/kubernetes/scheduler.conf
55+
extenders:
56+
- urlPrefix: "https://tas-service.default.svc.cluster.local:9001"
57+
prioritizeVerb: "scheduler/prioritize"
58+
filterVerb: "scheduler/filter"
59+
weight: 1
60+
enableHTTPS: true
61+
managedResources:
62+
- name: "telemetry/scheduling"
63+
ignoredByScheduler: true
64+
ignorable: true
65+
tlsConfig:
66+
insecure: false
67+
certFile: "/host/certs/client.crt"
68+
keyFile: "/host/certs/client.key"
8469
8570
````
86-
This file can be found [in the deploy folder](deploy/extender-configuration/scheduler-extender-configmap.yaml). This configmap can be created with ``kubectl apply -f ./deploy/scheduler-extender-configmap.yaml``
87-
The scheduler requires flags passed to it in order to know the location of this config map. The flags are:
88-
````
89-
- --policy-configmap=scheduler-extender-policy
90-
- --policy-configmap-namespace=kube-system
91-
````
92-
71+
This file can be found [in the deploy folder](deploy/extender-configuration/scheduler-config.yaml). The API version of the file is updated by executing a [shell script](deploy/extender-configuration/configure-scheduler.sh).
72+
Note that k8s, from version 1.22 onwards, will no longer accept a scheduling policy to be passed as a flag to the kube-scheduler. The shell script will make sure the scheduler is set-up according to its version: scheduling by policy or configuration file.
9373
If scheduler is running as a service these can be added as flags to the binary. If scheduler is running as a container - as in kubeadm - these args can be passed in the deployment file.
9474
Note: For Kubeadm set ups some additional steps may be needed.
9575
1) Add the ability to get configmaps to the kubeadm scheduler config map. (A cluster role binding for this is at deploy/extender-configuration/configmap-getter.yaml)
@@ -98,7 +78,7 @@ Note: For Kubeadm set ups some additional steps may be needed.
9878
After these steps the scheduler extender should be registered with the Kubernetes Scheduler.
9979

10080
#### Deploy TAS
101-
Telemetry Aware Scheduling uses go modules. It requires Go 1.13+ with modules enabled in order to build. TAS has been tested with Kubernetes 1.14+. TAS was tested on Intel® Server Board S2600WF-Based Systems (Wolf Pass).
81+
Telemetry Aware Scheduling uses go modules. It requires Go 1.16+ with modules enabled in order to build. TAS has been tested with Kubernetes 1.20+. TAS was tested on Intel® Server Board S2600WF-Based Systems (Wolf Pass).
10282
A yaml file for TAS is contained in the deploy folder along with its service and RBAC roles and permissions.
10383

10484
**Note:** If run without the unsafe flag ([described in the table below](#tas-scheduler-extender)) a secret called extender-secret will need to be created with the cert and key for the TLS endpoint.

0 commit comments

Comments
 (0)