What is a Custom Resource

In Kubernetes, a custom resource (CR) is an extension mechanism that allows users to define their own API resources and controllers. This allows users to introduce new abstractions and workflows into their Kubernetes cluster that are tailored to their specific needs.

Custom resources are created using the Kubernetes API machinery and can be accessed, manipulated, and managed like any other resource in the cluster. Custom resources can be used to represent any kind of object, such as a database, a message queue, or a custom application component.

To use a custom resource, a user typically defines a Custom Resource Definition (CRD) that specifies the structure and behavior of the resource. The CRD includes a schema that defines the fields and types of the resource, as well as any custom validation rules or default values.

Once a CRD is defined, users can create instances of the custom resource using the kubectl command-line tool or through Kubernetes client libraries. Users can also create controllers that watch for changes to custom resources and take action based on those changes, such as provisioning new resources or updating existing ones.

Why do you need a Custom Resource in Kubernetes?

Custom resources are needed in Kubernetes for a few reasons:

1. Extend Kubernetes functionality: By introducing custom resources, users can extend the functionality of Kubernetes beyond the built-in resources. This allows users to represent custom objects in a way that makes sense for their specific use cases.

2. Better abstraction: Custom resources provide a higher level of abstraction than using lower-level primitives, like pods or services. This abstraction allows users to define custom objects that can encapsulate complex business logic, making it easier to manage and maintain Kubernetes resources.

3. Standardize workflows: Custom resources can be used to standardize workflows across multiple teams or applications. By defining a custom resource, users can enforce consistent naming conventions, validation rules, and other policies across all instances of the resource.

4. Increase automation: Custom resources can be used to increase automation within Kubernetes. Users can define custom controllers that watch for changes to custom resources and take automated actions based on those changes. For example, a custom resource controller could automatically provision new resources when a certain threshold is reached.

Overall, custom resources provide a powerful mechanism for extending Kubernetes and tailoring it to specific use cases. They allow users to represent complex objects in a way that makes sense for their applications, while still benefiting from the scalability, flexibility, and automation of Kubernetes.

What is a Custom Resource Definition (CRD)

A Custom Resource Definition (CRD) is a Kubernetes API extension that allows users to create their own custom resources in Kubernetes. A CRD defines a new kind of custom resource that can be used like any other Kubernetes resource, such as pods or services.

A CRD is essentially a schema that defines the structure of the custom resource, including its fields, validation rules, and default values. Once a CRD is created, users can create instances of the custom resource that conforms to the schema.

A CRD is defined using a YAML or JSON file that describes the structure of the custom resource. The file includes a few required fields, such as the name of the custom resource and its API version, as well as optional fields for defining the schema of the custom resource.

Here's an example of a simple CRD definition for a custom resource called "Foo":

yamlCopy codeapiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: foos.example.com
spec:
  group: example.com
  version: v1alpha1
  scope: Namespaced
  names:
    plural: foos
    singular: foo
    kind: Foo
  validation:
    openAPIV3Schema:
      type: object
      properties:
        name:
          type: string
        size:
          type: integer

In this example, the CRD defines a custom resource called "Foo" with two fields: "name" and "size". The CRD also specifies that the custom resource should be created in a namespace, and it includes a validation schema that ensures that all instances of the custom resource include the required fields and have the correct data types.

Once a CRD is defined, users can create instances of the custom resource using Kubectl or the Kubernetes API. Custom controllers can also be written to watch for changes to custom resources and perform actions based on those changes.

What is a Custom Controller

A custom controller in Kubernetes is a software component that watches for changes to one or more Kubernetes resources and takes actions in response to those changes. Custom controllers are typically written in Go using the Kubernetes client-go library, although they can be written in other languages as well.

Custom controllers are often used in conjunction with custom resources to implement custom behavior in Kubernetes. For example, a user might define a custom resource called "MyApp" that represents an application in the cluster. A custom controller could then watch for changes to instances of the "MyApp" resource and take actions based on those changes, such as provisioning new resources or updating existing ones.

To implement a custom controller, the developer typically writes a Go program that uses the Kubernetes client-go library to interact with the Kubernetes API. The program registers itself as a watcher for one or more Kubernetes resources and defines a set of callback functions that are executed when changes occur to those resources.

For example, the program might define a callback function that is executed when a new instance of the "MyApp" resource is created. This function might then take actions such as provisioning new resources or updating configuration files.

Custom controllers can be deployed in Kubernetes as pods or as standalone processes outside of the cluster. They can also be managed using Kubernetes deployment and service objects, making them easy to scale and manage.

Overall, custom controllers provide a powerful way to extend Kubernetes and automate complex workflows in the cluster. By combining custom resources with custom controllers, users can implement custom behavior that is tailored to their specific needs.

How to write a custom controller in Kubernetes

Writing a custom controller in Kubernetes involves several steps:

1. Define the Custom Resource Definition (CRD): First, define the structure of the custom resource using a YAML or JSON file. This file should specify the name, version, and structure of the custom resource, as well as any validation rules that should be applied.

2. Generate code using code generators: Next, use code generators to generate the Go code needed to interact with the Kubernetes API. The code generators will create the client, informer, and lister objects that the custom controller will use to interact with the Kubernetes API.

3. Write the custom controller: Using the generated code, write the custom controller in Go. The controller should use the client, informer, and lister objects to watch for changes to the custom resource and take actions based on those changes. The actions might include creating or updating other Kubernetes resources, or performing other actions within the cluster.

4. Build and deploy the custom controller: Finally, build the custom controller into a container image and deploy it in Kubernetes. This can be done using Kubernetes deployment and service objects, or by running the custom controller as a standalone process outside of the cluster.

Here's an overview of the steps involved in more detail:

Step 1: Define the Custom Resource Definition (CRD)

yamlCopy codeapiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: mycustomresource.example.com
spec:
  group: example.com
  version: v1alpha1
  scope: Namespaced
  names:
    plural: mycustomresources
    singular: mycustomresource
    kind: MyCustomResource
  validation:
    openAPIV3Schema:
      type: object
      properties:
        name:
          type: string
        size:
          type: integer

Step 2: Generate code using code generators

Use the kubebuilder tool to generate the Go code needed to interact with the Kubernetes API:

bashCopy code# Install the kubebuilder tool
go install sigs.k8s.io/kubebuilder/v3/cmd/kubebuilder

# Create a new project
kubebuilder init --domain example.com

# Create a new API
kubebuilder create api --group example --version v1alpha1 --kind MyCustomResource

This will generate the Go code needed to interact with the Kubernetes API, including client, informer, and lister objects.

Step 3: Write the custom controller

Using the generated code, write the custom controller in Go. The controller should watch for changes to instances of the custom resource and take actions based on those changes. Here's an example of a custom controller that watches for changes to instances of the MyCustomResource resource:

goCopy codepackage controllers

import (
    "context"

    "github.com/go-logr/logr"
    myv1alpha1 "github.com/example/mypackage/api/v1alpha1"
    "k8s.io/apimachinery/pkg/runtime"
    ctrl "sigs.k8s.io/controller-runtime"
    "sigs.k8s.io/controller-runtime/pkg/client"
    "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
    "sigs.k8s.io/controller-runtime/pkg/handler"
    "sigs.k8s.io/controller-runtime/pkg/source"
)

// MyCustomResourceReconciler reconciles a MyCustomResource object
type MyCustomResourceReconciler struct {
    client.Client
    Log    logr.Logger
    Scheme *runtime.Scheme
}

//+kubebuilder:rbac:...
//+kubebuilder:rbac:...

func (r *MyCustomResourceReconciler) Reconcile(ctx context.Context, req ctrl.Request

Custom resources comparison with the native Kubernetes Resources

custom resources in Kubernetes are similar to native Kubernetes resources in many ways, but there are some key differences to keep in mind:

1. Native resources are built-in to Kubernetes, while custom resources are defined by users: Native resources like Pods, Services, and Deployments are part of the core Kubernetes API and are built-in to the system. Custom resources, on the other hand, are defined by users and extend the Kubernetes API with new types of resources that are specific to their needs.

2. Native resources have well-defined behavior, while custom resources can have arbitrary behavior: Because native resources are part of the core Kubernetes API, they have well-defined behavior and are subject to certain constraints and limitations. Custom resources, on the other hand, can have arbitrary behavior defined by the user.

3. Native resources have built-in controllers, while custom resources require custom controllers: Native resources have built-in controllers that manage their lifecycle and ensure that they are running correctly. Custom resources, on the other hand, require custom controllers to be defined by the user to manage their lifecycle and ensure that they are running correctly.

4. Native resources have built-in support in Kubernetes tools and APIs, while custom resources require additional tooling and APIs: Because native resources are part of the core Kubernetes API, they have built-in support in Kubernetes tools and APIs like kubectl the Kubernetes dashboard. Custom resources, on the other hand, require additional tooling and APIs to be developed by the user or third-party developers.

In summary, while custom resources in Kubernetes provide a powerful way to extend the Kubernetes API and add new types of resources that are specific to user needs, they also require additional development effort to define custom controllers and tooling and can have arbitrary behavior that may not be subject to the same constraints as native resources.