kubectl
Kubernetes cluster and kubectl configured to use it
- If you don’t have a Kubernetes cluster, you can use kind to create a local one
cert-manager installed in the cluster

Install etcd-operator:

kubectl apply -f https://github.com/aenix-io/etcd-operator/releases/latest/download/etcd-operator.yaml

Check the operator is running:

kubectl get pods -n etcd-operator-system -l control-plane=controller-manager

Create a simple etcd cluster:
```
kubectl apply -f https://github.com/aenix-io/etcd-operator/raw/main/examples/manifests/etcdcluster-simple.yaml
```
Caution: by default emptyDir storage is used. It means such cluster configuration is not intended for long-term storage.

Check the etcd cluster is running:

kubectl get pods -l app.kubernetes.io/managed-by=etcd-operator

1.3 - Examples

See etcd-operator in action!

You can find examples for deploying etcd clusters in various configuration in the GitHub repository under examples/manifests directory.

This directory provides various manifests that can help you understand how to set up etcd clusters using the etcd-operator.

1.4 - Reference

Low level reference docs for your project.

1.4.1 - API Reference

Autogenerated API Reference for the CRD

Packages

etcd.aenix.io/v1alpha1

etcd.aenix.io/v1alpha1

Package v1alpha1 contains API Schema definitions for the etcd.aenix.io v1alpha1 API group

BackupDestination

BackupDestination defines the target location for the backup. Exactly one must be specified.

Appears in:

EtcdBackupScheduleSpec
EtcdBackupSpec
RestoreSpec

Field	Description	Default	Validation
`s3` S3BackupDestination	S3 defines S3-compatible storage as the backup destination.		Optional: {}
`pvc` PVCBackupDestination	PVC defines a PersistentVolumeClaim as the backup destination.		Optional: {}

BootstrapSpec

BootstrapSpec defines how to initialize a new EtcdCluster from an existing data source.

Appears in:

EtcdClusterSpec

Field	Description	Default	Validation
`restore` RestoreSpec	Restore configures cluster initialization from an etcd snapshot.		Optional: {}

EmbeddedMetadataResource

Appears in:

EtcdClusterSpec

Field	Description	Default	Validation
`metadata` EmbeddedObjectMetadata	Refer to Kubernetes API documentation for fields of `metadata`.		Optional: {}

EmbeddedObjectMetadata

EmbeddedObjectMetadata contains a subset of the fields included in k8s.io/apimachinery/pkg/apis/meta/v1.ObjectMeta Only fields which are relevant to embedded resources are included.

Appears in:

EmbeddedMetadataResource
EmbeddedPersistentVolumeClaim
EmbeddedPodDisruptionBudget
EmbeddedService
PodTemplate

Field	Description	Validation
`name` string	Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names#names	Optional: {}
`labels` object (keys:string, values:string)	Labels Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels	Optional: {}
`annotations` object (keys:string, values:string)	Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations	Optional: {}

EmbeddedPersistentVolumeClaim

EmbeddedPersistentVolumeClaim is an embedded version of k8s.io/api/core/v1.PersistentVolumeClaim. It contains TypeMeta and a reduced ObjectMeta.

Appears in:

StorageSpec

Field	Description	Default	Validation
`metadata` EmbeddedObjectMetadata	Refer to Kubernetes API documentation for fields of `metadata`.		Optional: {}
`spec` PersistentVolumeClaimSpec	Spec defines the desired characteristics of a volume requested by a pod author. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims		Optional: {}

EmbeddedPodDisruptionBudget

EmbeddedPodDisruptionBudget describes PDB resource for etcd cluster members

Appears in:

EtcdClusterSpec

Field	Description	Default	Validation
`metadata` EmbeddedObjectMetadata	Refer to Kubernetes API documentation for fields of `metadata`.		Optional: {}
`spec` PodDisruptionBudgetSpec	Spec defines the desired characteristics of a PDB. More info: https://kubernetes.io/docs/concepts/workloads/pods/disruptions/#pod-disruption-budgets		Optional: {}

EmbeddedService

Appears in:

EtcdClusterSpec

Field	Description	Default	Validation
`metadata` EmbeddedObjectMetadata	Refer to Kubernetes API documentation for fields of `metadata`.		Optional: {}
`spec` ServiceSpec	Spec defines the behavior of the service.		Optional: {}

EtcdBackup

EtcdBackup is the Schema for the etcdbackups API

Appears in:

EtcdBackupList

Field	Description	Default	Validation
`apiVersion` string	`etcd.aenix.io/v1alpha1`
`kind` string	`EtcdBackup`
`metadata` ObjectMeta	Refer to Kubernetes API documentation for fields of `metadata`.
`spec` EtcdBackupSpec

EtcdBackupList

EtcdBackupList contains a list of EtcdBackup

Field	Description	Default	Validation
`apiVersion` string	`etcd.aenix.io/v1alpha1`
`kind` string	`EtcdBackupList`
`metadata` ListMeta	Refer to Kubernetes API documentation for fields of `metadata`.
`items` EtcdBackup array

EtcdBackupSchedule

EtcdBackupSchedule is the Schema for the etcdbackupschedules API

Appears in:

EtcdBackupScheduleList

Field	Description	Default	Validation
`apiVersion` string	`etcd.aenix.io/v1alpha1`
`kind` string	`EtcdBackupSchedule`
`metadata` ObjectMeta	Refer to Kubernetes API documentation for fields of `metadata`.
`spec` EtcdBackupScheduleSpec

EtcdBackupScheduleList

EtcdBackupScheduleList contains a list of EtcdBackupSchedule

Field	Description	Default	Validation
`apiVersion` string	`etcd.aenix.io/v1alpha1`
`kind` string	`EtcdBackupScheduleList`
`metadata` ListMeta	Refer to Kubernetes API documentation for fields of `metadata`.
`items` EtcdBackupSchedule array

EtcdBackupScheduleSpec

EtcdBackupScheduleSpec defines the desired state of EtcdBackupSchedule

Appears in:

EtcdBackupSchedule

Field	Description	Validation
`clusterRef` LocalObjectReference	ClusterRef references the EtcdCluster to back up.
`schedule` string	Schedule is a cron expression defining when backups should be taken.
`destination` BackupDestination	Destination defines where the backup will be stored.
`successfulJobsHistoryLimit` integer	SuccessfulJobsHistoryLimit is the number of successful finished CronJob children to retain.	Minimum: 0 Optional: {}
`failedJobsHistoryLimit` integer	FailedJobsHistoryLimit is the number of failed finished CronJob children to retain.	Minimum: 0 Optional: {}

EtcdBackupSpec

EtcdBackupSpec defines the desired state of EtcdBackup

Appears in:

EtcdBackup

Field	Description	Default	Validation
`clusterRef` LocalObjectReference	ClusterRef references the EtcdCluster to back up.
`destination` BackupDestination	Destination defines where the backup will be stored.

EtcdBackupStatusPhase

Underlying type: string

Appears in:

EtcdBackupStatus

EtcdCluster

EtcdCluster is the Schema for the etcdclusters API

Field	Description	Default	Validation
`apiVersion` string	`etcd.aenix.io/v1alpha1`
`kind` string	`EtcdCluster`
`metadata` ObjectMeta	Refer to Kubernetes API documentation for fields of `metadata`.
`spec` EtcdClusterSpec

EtcdClusterSpec

EtcdClusterSpec defines the desired state of EtcdCluster

Appears in:

EtcdCluster

Field	Description	Default	Validation
`replicas` integer	Replicas is the count of etcd instances in cluster.	3	Minimum: 0 Optional: {}
`options` object (keys:string, values:string)	Options are the extra arguments to pass to the etcd container.		Optional: {}
`podTemplate` PodTemplate	PodTemplate defines the desired state of PodSpec for etcd members. If not specified, default values will be used.
`serviceTemplate` EmbeddedService	Service defines the desired state of Service for etcd members. If not specified, default values will be used.		Optional: {}
`headlessServiceTemplate` EmbeddedMetadataResource	HeadlessService defines the desired state of HeadlessService for etcd members. If not specified, default values will be used.		Optional: {}
`podDisruptionBudgetTemplate` EmbeddedPodDisruptionBudget	PodDisruptionBudgetTemplate describes PDB resource to create for etcd cluster members. Nil to disable.		Optional: {}
`storage` StorageSpec
`security` SecuritySpec	Security describes security settings of etcd (authentication, certificates, rbac)		Optional: {}
`bootstrap` BootstrapSpec	Bootstrap defines initialization from an existing data source (e.g., snapshot restore). This is only used during initial cluster creation and is ignored afterward.		Optional: {}

PVCBackupDestination

PVCBackupDestination defines a PersistentVolumeClaim as the backup target.

Appears in:

BackupDestination

Field	Description	Default	Validation
`claimName` string	ClaimName is the name of the PersistentVolumeClaim to use.
`subPath` string	SubPath is an optional sub-directory within the PVC volume. The operator appends the backup filename automatically.		Optional: {}

PodDisruptionBudgetSpec

Appears in:

EmbeddedPodDisruptionBudget

Field	Description	Default	Validation
`minAvailable` IntOrString	MinAvailable describes minimum ready replicas. If both are empty, controller will implicitly calculate MaxUnavailable based on number of replicas Mutually exclusive with MaxUnavailable.		Optional: {}
`maxUnavailable` IntOrString	MinAvailable describes maximum not ready replicas. If both are empty, controller will implicitly calculate MaxUnavailable based on number of replicas Mutually exclusive with MinAvailable		Optional: {}

PodTemplate

PodTemplate allows overrides, such as sidecars, init containers, changes to the security context, etc to the pod template generated by the operator.

Appears in:

EtcdClusterSpec

Field	Description	Default	Validation
`metadata` EmbeddedObjectMetadata	Refer to Kubernetes API documentation for fields of `metadata`.		Optional: {}
`spec` PodSpec	Spec follows the structure of a regular Pod spec. Overrides defined here will be strategically merged with the default pod spec, generated by the operator.		Optional: {}

RestoreSpec

RestoreSpec defines how to restore a cluster from a snapshot.

Appears in:

BootstrapSpec

Field	Description	Default	Validation
`source` BackupDestination	Source defines where to get the snapshot for restoration.

S3BackupDestination

S3BackupDestination defines S3-compatible storage parameters.

Appears in:

BackupDestination

Field	Description	Validation
`endpoint` string	Endpoint is the S3-compatible endpoint URL (e.g., “https://s3.amazonaws.com”).
`bucket` string	Bucket is the name of the S3 bucket.
`key` string	Key is the key prefix (directory path) within the bucket. The operator appends the backup filename automatically.	Optional: {}
`credentialsSecretRef` LocalObjectReference	CredentialsSecretRef references a Secret containing AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY keys.
`region` string	Region is the AWS region for the S3 bucket.	Optional: {}
`forcePathStyle` boolean	ForcePathStyle forces path-style S3 URLs (e.g., endpoint/bucket/key) instead of virtual-hosted-style (e.g., bucket.endpoint/key). Most S3-compatible providers (MinIO, Ceph) require path style.	Optional: {}

SecuritySpec

SecuritySpec defines security settings for etcd.

Appears in:

EtcdClusterSpec

Field	Description	Default	Validation
`tls` TLSSpec	Section for user-managed tls certificates		Optional: {}
`enableAuth` boolean	Section to enable etcd auth

StorageSpec

StorageSpec defines the configured storage for a etcd members. If neither emptyDir nor volumeClaimTemplate is specified, then by default an EmptyDir will be used.

Appears in:

EtcdClusterSpec

Field	Description	Default	Validation
`emptyDir` EmptyDirVolumeSource	EmptyDirVolumeSource to be used by the StatefulSets. If specified, used in place of any volumeClaimTemplate. More info: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir		Optional: {}
`volumeClaimTemplate` EmbeddedPersistentVolumeClaim	A PVC spec to be used by the StatefulSets.		Optional: {}

TLSSpec

TLSSpec defines user-managed certificates names.

Appears in:

SecuritySpec

Field	Description	Validation
`peerTrustedCASecret` string	Trusted CA certificate secret to secure peer-to-peer communication between etcd nodes. It is expected to have ca.crt field in the secret. This secret must be created in the namespace with etcdCluster CR.	Optional: {}
`peerSecret` string	Certificate secret to secure peer-to-peer communication between etcd nodes. It is expected to have tls.crt and tls.key fields in the secret. This secret must be created in the namespace with etcdCluster CR.	Optional: {}
`serverTrustedCASecret` string	Trusted CA for etcd server certificates for client-server communication. Is necessary to set trust between operator and etcd. It is expected to have ca.crt field in the secret. If it is not specified, then insecure communication will be used. This secret must be created in the namespace with etcdCluster CR.	Optional: {}
`serverSecret` string	Server certificate secret to secure client-server communication. Is provided to the client who connects to etcd by client port (2379 by default). It is expected to have tls.crt and tls.key fields in the secret. This secret must be created in the namespace with etcdCluster CR.	Optional: {}
`clientTrustedCASecret` string	Trusted CA for client certificates that are provided by client to etcd. It is expected to have ca.crt field in the secret. This secret must be created in the namespace with etcdCluster CR.	Optional: {}
`clientSecret` string	Client certificate for etcd-operator to do maintenance. It is expected to have tls.crt and tls.key fields in the secret. This secret must be created in the namespace with etcdCluster CR.	Optional: {}

1.5 - Contribution Guidelines

How to contribute to the project

etcd operator guide for developers.

How to start contributing

First of all, etcd operator uses kubebuilder. See official docs to learn more.

Easy way

Using this way, you don’t be able to debug the controller locally. After every change you will have to redeploy changes.

Pre-requisites

Any docker-like container tool, “docker” by default. For more information search for: CONTAINER_TOOL in Makefile.
kubectl

Steps

Create and prepare kind cluster:
```
make kind-prepare
```
Build image and load it into kind cluster:
```
make kind-load
```
Deploy CRDs, etcd-operator, RBAC, webhook certs into kind cluster:
```
make deploy
```
To deploy your code changes, load a new image and redeploy etcd-operator:
```
make kind-load && make redeploy
```
To clean up after all, delete kind cluster:
```
make kind-delete
```

Advanced way

Using VSCode you can connect your IDE to a pod in kubernetes cluster and use go run cmd/main.go from the pod. It will allow you to debug easily and faster integrate code changes to you develop environment.

General steps

Install VSCode Kubernetes extension.
Install VSCode Dev Containers extension.
Create pvc to store operator code, go modules and your files.
Build Dockerfile to create image for your develop environment (git, golang , etc…). Take base image that you love most.
Patch operator deployment to
- Attach PVC from step 3.
- Change operator image to your image.
- Change command and args to endless sleep loop.
- Change rolling update strategy to recreate if you use RWO storage class.
- Change security context RunAsNonRoot to false.
- Increase requests and limits.
- Remove readiness and liveness probes.
Attach VSCode to a running container through Kubernetes extension.
Install necessary extensions to the container. They will be preserved after container restart if you attached pvc to home directory.
Run go run cmd/main.go.

Release procedure

Every PR should have a label to show what kind of changes does it bring. For example, PRs with docs changes should have documentation label. This labels will be used by the release-drafter workflow to prepare the release draft.

Cutting off a release

When all tasks for the release are merged to main branch, maintainer should

create new minor or major version in site if it’s not a patch release and merge this PR to main branch
check that tag has not been already created. If it has been, delete it
publish the release using draft, created by release-drafter
you are amazing :)