📖 refactor external type docs to be up to date

jakobmoellerdev · seh · camilamacedo86 · jakobmoellerdev · commit f208114b914b · 2024-01-30T12:43:47.000+01:00
Co-authored-by: Steven E. Harris &lt;seh@panix.com&gt;
Co-authored-by: Camila Macedo &lt;7708031+camilamacedo86@users.noreply.github.com&gt;
Co-authored-by: Bryce Palmer &lt;bpalmer@redhat.com&gt;
diff --git a/docs/book/src/reference/submodule-layouts.md b/docs/book/src/reference/submodule-layouts.md
@@ -110,7 +110,7 @@ YOUR_GO_PATH/test-operator imports
 ```
 
 The reason for this is that you may have not pushed your modules into the VCS yet and resolving the main module will fail as it can no longer
-directly access the API types as package but only as module.
+directly access the API types as a package but only as a module.
 
 To solve this issue, we will have to tell the go tooling to properly `replace` the API module with a local reference to your path.
 
@@ -120,8 +120,8 @@ You can do this with 2 different approaches: go modules and go workspaces.
 
 For go modules, you will edit the main `go.mod` file of your project and issue a replace directive.
 
-You can do this by editing the go.mod with
-
+You can do this by editing the `go.mod` with
+``
 ```shell
 go mod edit -require YOUR_GO_PATH/test-operator/api/v1alpha1@v0.0.0 # Only if you didn't already resolve the module
 go mod edit -replace YOUR_GO_PATH/test-operator/api/v1alpha1@v0.0.0=./api/v1alpha1
@@ -148,7 +148,7 @@ go mod tidy
 
 For go workspaces, you will not edit the `go.mod` files yourself, but rely on the workspace support in go.
 
-To initialize a workspace for your project, run ´go work init` in the project root.
+To initialize a workspace for your project, run `go work init` in the project root.
 
 Now let us include both modules in our workspace:
 ```shell
diff --git a/docs/book/src/reference/using_an_external_type.md b/docs/book/src/reference/using_an_external_type.md
@@ -6,95 +6,157 @@ There are several different external types that may be referenced when writing a
 * CRDs that are created and installed in another project.
 * A custom API defined via the aggregation layer, served by an extension API server for which the primary API server acts as a proxy.
 
-Currently Kubebuilder handles the first two—CRDs and Core Resources—seamlessly. You must scaffold the latter two—External CRDs and APIs created via aggregation—manually.
+Currently, kubebuilder handles the first two, CRDs and Core Resources, seamlessly. You must scaffold the latter two, External CRDs and APIs created via aggregation, manually.
 
 In order to use a Kubernetes Custom Resource that has been defined in another project
 you will need to have several items of information.
 * The Domain of the CR
 * The Group under the Domain 
-* The Go import path of the CR Type definition.
+* The Go import path of the CR Type definition
+* The Custom Resource Type you want to depend on.
 
 The Domain and Group variables have been discussed in other parts of the documentation.  The import path would be located in the project that installs the CR.
-
+The Custom Resource Type is usually a Go Type of the same name as the CustomResourceDefinition in kubernetes, e.g. for a `Pod` there will be a type `Pod` in the `v1` group.
+For Kubernetes Core Types, the domain can be omitted.
+``
 This document uses `my` and `their` prefixes as a naming convention for repos, groups, and types to clearly distinguish between your own project and the external one you are referencing.
-Note that by default, multigroup APIs are no longer included. To enable them again, see [the guide on multigroup API migration](https://book.kubebuilder.io/migration/multi-group.html).
 
-Example external API Aggregation directory structure
-```
-github.com
-    ├── theiruser
-        ├── theirproject
-            ├── apis
-                ├── theirgroup
-                   ├── doc.go`
-                   ├── install
-                   │   ├── install.go
-                   ├── v1alpha1
-                   │   ├── doc.go
-                   │   ├── register.go
-                   │   ├── types.go
-                   │   ├── zz_generated.deepcopy.go
-```
+In our example we will assume the following external API Type:
+
+`github.com/theiruser/theirproject` is another kubebuilder project on whose CRD we want to depend and extend on.
+Thus, it contains a `go.mod` in its repository root. The import path for the go types would be `github.com/theiruser/theirproject/api/theirgroup/v1alpha1`.
+
+The Domain of the CR is `theirs.com`, the Group is `theirgroup` and the kind and go type would be `ExternalType`.
+
+If there is an interest to have multiple Controllers running in different Groups (e.g. because one is an owned CRD and one is an external Type), please first
+reconfigure the Project to use a multi-group layout as described in the [Multi-Group documentation](../migration/multi-group.md).
+
+### Prerequisites
 
-In the case above the import path would be `github.com/theiruser/theirproject/apis/theirgroup/v1alpha1`
+The following guide assumes that you have already created a project using `kubebuilder init` in a directory in the GOPATH. Please reference the [Getting Started Guide](../getting-started.md) for more information.
 
+Note that if you did not pass `--domain` to `kubebuilder init` you will need to modify it for the individual api types as the default is `my.domain`, not `theirs.com`.
+Similarly, if you intend to use your own domain, please configure your own domain with `kubebuilder init` and do not use `theirs.com for the domain.
 
-### Add a controller
+### Add a controller for the external Type
+
+Run the command `create api` to scaffold only the controller to manage the external type:
 
-be sure to answer no when it asks if you would like to create an api? [Y/n]
 ```shell
-kubebuilder create api --group mygroup --version $APIVERSION --kind MyKind
+kubebuilder create api --group <theirgroup> --version v1alpha1 --kind <ExternalTypeKind> --controller --resource=false
+```
+
+Note that the `resource` argument is set to false, as we are not attempting to create our own CustomResourceDefinition,
+but instead rely on an external one.
+
+This will result in a `PROJECT` entry with the default domain of the `PROJECT` (`my.domain` if not specified in `kubebuilder init`).
+For use of other domains, such as `theirs.com`, one will have to manually adjust the `PROJECT` file with the correct domain for the entry:
+
+<aside class="note">
+If you are looking to create Controllers to manage Kubernetes Core types (i.e. Deployments/Pods)y
+you do not need to update the PROJECT file or register the Schema in the manager. All Core Types are registered by default. The Kubebuilder CLI will add the required values to the PROJECT file, but you still need to perform changes to the RBAC markers manually to ensure that the Rules will be generated accordingly.
+</aside>
+
+file: PROJECT
+```
+domain: my.domain
+layout:
+- go.kubebuilder.io/v4
+projectName: testkube
+repo: example.com
+resources:
+- controller: true
+  domain: my.domain ## <- Replace the domain with theirs.com domain
+  group: mygroup
+  kind: ExternalType
+  version: v1alpha1
+version: "3"
 ```
 
-## Edit the API files.
+At the same time, the generated RBAC manifests need to be adjusted:
+
+file: internal/controller/externaltype_controller.go
+```go
+// ExternalTypeReconciler reconciles a ExternalType object
+type ExternalTypeReconciler struct {
+	client.Client
+	Scheme *runtime.Scheme
+}
+
+// external types can be added like this
+//+kubebuilder:rbac:groups=theirgroup.theirs.com,resources=externaltypes,verbs=get;list;watch;create;update;patch;delete
+//+kubebuilder:rbac:groups=theirgroup.theirs.com,resources=externaltypes/status,verbs=get;update;patch
+//+kubebuilder:rbac:groups=theirgroup.theirs.com,resources=externaltypes/finalizers,verbs=update
+// core types can be added like this
+//+kubebuilder:rbac:groups=core,resources=pods,verbs=get;list;watch;create;update;patch;delete
+//+kubebuilder:rbac:groups=core,resources=pods/status,verbs=get;list;watch;create;update;patch;delete
+//+kubebuilder:rbac:groups=core,resources=pods/finalizers,verbs=update
+```
 
 ### Register your Types
 
-Edit the following file to the pkg/apis directory to append their `AddToScheme` to your `AddToSchemes`:
+<aside class="note">
+Note that this is only valid for external types and not the kubernetes core types.
+Core types such as pods or nodes are registered by default in the scheme.
+</aside>
+
+Edit the following lines to the main.go file to register the external types:
 
-file: pkg/apis/mytype_addtoscheme.go
+file: cmd/main.go
 ```go
 package apis
 
 import (
-	mygroupv1alpha1 "github.com/myuser/myrepo/apis/mygroup/v1alpha1"
 	theirgroupv1alpha1 "github.com/theiruser/theirproject/apis/theirgroup/v1alpha1"
 )
 
 func init() {
-	// Register the types with the Scheme so the components can map objects 
-	// to GroupVersionKinds and back
-	AddToSchemes = append(
-	  AddToSchemes, 
-	  mygroupv1alpha1.SchemeBuilder.AddToScheme,
-	  theirgroupv1alpha1.SchemeBuilder.AddToScheme,
-	)
-}
-
+	utilruntime.Must(clientgoscheme.AddToScheme(scheme))
+	utilruntime.Must(theirgroupv1alpha1.AddToScheme(scheme)) // this contains the external API types
+	//+kubebuilder:scaffold:scheme
+} 
 ```
 
-## Edit the Controller files
+## Edit the Controller `SetupWithManager` function
 
-### Use the correct imports for your API
+### Use the correct imports for your API and uncomment the controlled resource
 
-file: pkg/controllers/mytype_controller.go
+file: internal/controllers/externaltype_controllers.go
 ```go
 package controllers
 
 import (
-	mygroupv1alpha1 "github.com/myuser/myrepo/apis/mygroup/v1alpha1"
 	theirgroupv1alpha1 "github.com/theiruser/theirproject/apis/theirgroup/v1alpha1"
 )
+
+//...
+
+// SetupWithManager sets up the controller with the Manager.
+func (r *ExternalTypeReconciler) SetupWithManager(mgr ctrl.Manager) error {
+	return ctrl.NewControllerManagedBy(mgr).
+		For(&theirgroupv1alpha1.ExternalType{}).
+		Complete(r)
+}
+
 ```
 
-Note that core resources may simply be imported by depending on the API's from upstream Kubernetes:
+Note that core resources may simply be imported by depending on the API's from upstream Kubernetes and do not need additional `AddToScheme` registrations:
 
+file: internal/controllers/externaltype_controllers.go
 ```go
 package controllers
 // contains core resources like Deployment
 import (
    v1 "k8s.io/api/apps/v1"
 )
+
+
+// SetupWithManager sets up the controller with the Manager.
+func (r *ExternalTypeReconciler) SetupWithManager(mgr ctrl.Manager) error {
+	return ctrl.NewControllerManagedBy(mgr).
+		For(&v1.Pod{}).
+		Complete(r)
+}
 ```
 
 ### Update dependencies
@@ -103,47 +165,103 @@ import (
 go mod tidy
 ```
 
+### Generate RBACs with updated Groups and Resources
 
-### Verifying API Availability in the Cluster
-
-Since we are now using external types, it is best-practice to verify the existance of the API in the cluster.
-You can use the manager's client to verify API Existance before starting the controllers through the manager.
+```
+make manifests
+``` 
 
 ## Prepare for testing
 
-#### Register your resource in the Scheme
+### Register your resource in the Scheme
 
-Edit the `CRDDirectoryPaths` in your test suite by appending the path to their CRDs:
+Edit the `CRDDirectoryPaths` in your test suite and add the correct `AddToScheme` entry during suite initialization:
 
-file pkg/controllers/my_kind_controller_suite_test.go
+file: internal/controllers/suite_test.go
 ```go
+package controller
+
+import (
+	"fmt"
+	"path/filepath"
+	"runtime"
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+
+	"k8s.io/client-go/kubernetes/scheme"
+	"k8s.io/client-go/rest"
+	"sigs.k8s.io/controller-runtime/pkg/client"
+	"sigs.k8s.io/controller-runtime/pkg/envtest"
+	logf "sigs.k8s.io/controller-runtime/pkg/log"
+	"sigs.k8s.io/controller-runtime/pkg/log/zap"
+	//+kubebuilder:scaffold:imports
+	theirgroupv1alpha1 "github.com/theiruser/theirproject/apis/theirgroup/v1alpha1"
+)
+
 var cfg *rest.Config
+var k8sClient client.Client
+var testEnv *envtest.Environment
 
-func TestMain(m *testing.M) {
-	// Get a config to talk to the apiserver
-	t := &envtest.Environment{
-		Config:             cfg,
-		CRDDirectoryPaths:  []string{
-		  filepath.Join("..", "..", "..", "config", "crds"),
-		  filepath.Join("..", "..", "..", "vendor", "github.com", "theiruser", "theirproject", "config", "crds"),
-        },
-		UseExistingCluster: true,
-	}
+func TestControllers(t *testing.T) {
+	RegisterFailHandler(Fail)
 
-	apis.AddToScheme(scheme.Scheme)
+	RunSpecs(t, "Controller Suite")
+}
 
-	var err error
-	if cfg, err = t.Start(); err != nil {
-		log.Fatal(err)
+
+var _ = BeforeSuite(func() {
+	//...
+	By("bootstrapping test environment")
+	testEnv = &envtest.Environment{
+		CRDDirectoryPaths:     []string{
+			// if you are using vendoring and rely on a kubebuilder based project, you can simply rely on the vendored config directory 
+			filepath.Join("..", "..", "..", "vendor", "github.com", "theiruser", "theirproject", "config", "crds"),
+			// otherwise you can simply download the CRD from any source and place it within the config/crd/bases directory,
+			filepath.Join("..", "..", "config", "crd", "bases"),
+		},
+		ErrorIfCRDPathMissing: false,
+
+		// The BinaryAssetsDirectory is only required if you want to run the tests directly
+		// without call the makefile target test. If not informed it will look for the
+		// default path defined in controller-runtime which is /usr/local/kubebuilder/.
+		// Note that you must have the required binaries setup under the bin directory to perform
+		// the tests directly. When we run make test it will be setup and used automatically.
+		BinaryAssetsDirectory: filepath.Join("..", "..", "bin", "k8s",
+			fmt.Sprintf("1.28.3-%s-%s", runtime.GOOS, runtime.GOARCH)),
 	}
+	
+	var err error
+	// cfg is defined in this file globally.
+	cfg, err = testEnv.Start()
+	Expect(err).NotTo(HaveOccurred())
+	Expect(cfg).NotTo(BeNil())
 
-	code := m.Run()
-	t.Stop()
-	os.Exit(code)
-}
+	//+kubebuilder:scaffold:scheme
+    Expect(theirgroupv1alpha1.AddToScheme(scheme.Scheme)).To(Succeed())
+
+	k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme})
+	Expect(err).NotTo(HaveOccurred())
+	Expect(k8sClient).NotTo(BeNil())
+
+
+})
 
 ```
 
+### Verifying API Availability in the Cluster
+
+Since we are now using external types, you will now have to rely on them being installed into the cluster.
+If the APIs are not available at the time the manager starts, all informers listening to the non-available types
+will fail, causing the manager to exit with an error similar to
+
+```
+failed to get informer from cache       {"error": "Timeout: failed waiting for *v1alpha1.ExternalType Informer to sync"}
+```
+
+This will signal that the API Server is not yet ready to serve the external types.
+
 ## Helpful Tips
 
 ### Locate your domain and group variables
@@ -152,6 +270,6 @@ The following kubectl commands may be useful
 
 ```shell
 kubectl api-resources --verbs=list -o name
-kubectl api-resources --verbs=list -o name | grep mydomain.com
+kubectl api-resources --verbs=list -o name | grep my.domain
 ```
 
