doc/user: Add metrics and service-monitor docs (#719)

* doc/user/metrics: Add metrics pkg docs

Add documentation for metrics and serviceMonitor.

Author: lilic

doc/user-guide.md

@@ -407,6 +407,10 @@ To implement complex deletion logic, you can add a finalizer to your Custom Reso
 deleted until you remove the finalizer (ie, after your cleanup logic has successfully run). For more information, see the 
 [official Kubernetes documentation on finalizers](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#finalizers).
 
+### Metrics
+
+To learn about how metrics work in the Operator SDK read the [metrics section][metrics_doc] of the user documentation.
+
 ## Leader election
 
 During the lifecycle of an operator it's possible that there may be more than 1 instance running at any given time e.g when rolling out an upgrade for the operator.
@@ -468,6 +472,7 @@ func main() {
 When the operator is not running in a cluster, the Manager will return an error on starting since it can't detect the operator's namespace in order to create the configmap for leader election. You can override this namespace by setting the Manager's `LeaderElectionNamespace` option.
 
 
+
 [pod_eviction_timeout]: https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/#options
 [manager_options]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/manager#Options
 [lease_split_brain]: https://github.com/kubernetes/client-go/blob/30b06a83d67458700a5378239df6b96948cb9160/tools/leaderelection/leaderelection.go#L21-L24
@@ -492,3 +497,4 @@ When the operator is not running in a cluster, the Manager will return an error
 [controller-go-doc]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg#hdr-Controller
 [request-go-doc]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/reconcile#Request
 [result_go_doc]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/reconcile#Result
+[metrics_doc]: ./user/metrics/README.md

doc/user/metrics/README.md

@@ -0,0 +1,51 @@
+# Monitoring with Prometheus
+
+[Prometheus][prometheus] is an open-source systems monitoring and alerting toolkit. Below is the overview of the different helpers that exist in operator-sdk to help setup metrics in the generated operator.
+
+## Metrics in operator-sdk
+
+The `func ExposeMetricsPort(ctx context.Context, port int32) (*v1.Service, error)` function exposes general metrics about the running program. These metrics are inherited from controller-runtime. This helper function creates a [Service][service] object with the metrics port exposed, which can then be accessed by Prometheus. The Service object is [garbage collected][gc] when the leader pod's root owner is deleted.
+
+By default, the metrics are served on `0.0.0.0:8383/metrics`. To modify the port the metrics are exposed on, change the `var metricsPort int32 = 8383` variable in the `cmd/manager/main.go` file of the generated operator.
+
+### Usage:
+
+```go
+    import(
+        "github.com/operator-framework/operator-sdk/pkg/metrics"
+        "sigs.k8s.io/controller-runtime/pkg/manager"
+    )
+
+    func main() {
+
+        ...
+
+        // Change the below variables to serve metrics on different host or port.
+        var metricsHost = "0.0.0.0"
+        var metricsPort int32 = 8383
+
+        // Pass metrics address to controller-runtime manager
+        mgr, err := manager.New(cfg, manager.Options{
+            Namespace:          namespace,
+            MetricsBindAddress: fmt.Sprintf("%s:%d", metricsHost, metricsPort),
+        })
+        
+        ...
+
+        // Create Service object to expose the metrics port.
+        _, err = metrics.ExposeMetricsPort(ctx, metricsPort)
+        if err != nil {
+            // handle error
+            log.Info(err.Error())
+        }
+
+        ...
+    }
+```
+
+*Note:* The above example is already present in `cmd/manager/main.go` in all the operators generated with operator-sdk.
+
+[prometheus]: https://prometheus.io/
+[service]: https://kubernetes.io/docs/concepts/services-networking/service/
+[gc]: https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/#owners-and-dependents
+

doc/user/metrics/service-monitor.md

@@ -0,0 +1,44 @@
+## Using the ServiceMonitor prometheus-operator CRD
+
+[prometheus-operator][prom-operator] is an operator that creates, configures, and manages Prometheus clusters atop Kubernetes.
+
+`ServiceMonitor` is a CustomResource of the prometheus-operator, which discovers the `Endpoints` in `Service` objects and configures Prometheus to monitor those pods. See the prometheus-operator [documentation][service-monitor] to learn more about `ServiceMonitor`.
+
+The `GenerateServiceMonitor` function takes a `Service` object and generates a `ServiceMonitor` resource based on it. To add `Service` target discovery of your created monitoring `Service` you can use the `metrics.CreateServiceMonitor()` helper function, which accepts the newly created `Service`.
+
+### Prerequisites:
+
+- [prometheus-operator][prom-quickstart] needs to be deployed in the cluster.
+
+### Usage example:
+
+```go
+    import(
+        "k8s.io/api/core/v1"
+        "github.com/operator-framework/operator-sdk/pkg/metrics"
+    )
+
+    func main() {    
+        
+        ...
+
+        // Populate below with the Service(s) for which you want to create ServiceMonitors.
+        services := []*v1.Service{}
+
+        // Create one `ServiceMonitor` per application per namespace.
+        // Change below value to name of the Namespace you want the `ServiceMonitor` to be created in.
+        ns := "default"
+        
+        // Pass the Service(s) to the helper function, which in turn returns the array of `ServiceMonitor` objects.
+        serviceMonitors, err := metrics.CreateServiceMonitors(restConfig, ns, services)
+        if err != nil {
+            // handle error here
+        }
+
+        ...
+    }
+```
+
+[prom-operator]: https://github.com/coreos/prometheus-operator
+[service-monitor]: https://github.com/coreos/prometheus-operator/blob/7a25bf6b6bb2347dacb235659b73bc210117acc7/Documentation/design.md#servicemonitor
+[prom-quickstart]: https://github.com/coreos/prometheus-operator/tree/master/contrib/kube-prometheus#quickstart