Skip to content
227 changes: 149 additions & 78 deletions docs/docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,83 +7,82 @@ sidebar_position: 3
## Create an instance with pgBackRest

The `examples` directory contains several pre-configured manifests
designed to work with [`kind`](https://kind.sigs.k8s.io/)
(Eg: the pull policy is set to `Never`).
These files may require modifications to run on other types of
Kubernetes clusters.
designed to work with [`kind`](https://kind.sigs.k8s.io/) (Eg: the pull
policy is set to `Never`). These files may require modifications to run
on other types of Kubernetes clusters.

To use this plugin with a `Cluster`, CloudNativePG users must :

1. Create a secret named `pgbackrest-s3-secret` in the namespace of the
PostgreSQL `Cluster`. This secret must contain the `key` and
`secret-key` of the `s3` bucket.

Example:

```yaml title="secret.yaml"
---
apiVersion: v1
kind: Secret
metadata:
name: pgbackrest-s3-secret
type: Opaque
stringData:
ACCESS_KEY_ID: <key_to_replace>
ACCESS_SECRET_KEY: <secret_to_replace>
```

2. Create a pgBackRest `stanza` :

Example:

```yaml title="stanza.yaml"
---
apiVersion: pgbackrest.dalibo.com/v1
kind: Stanza
metadata:
name: stanza-sample
spec:
stanzaConfiguration:
name: main
s3Repositories:
- bucket: demo
endpoint: s3.minio.svc.cluster.local
region: us-east-1
repoPath: /cluster-demo
uriStyle: path
verifyTLS: false
retentionPolicy:
full: 7
fullType: count
diff: 14
archive: 2
archiveType: full
history: 30
secretRef:
accessKeyId:
name: pgbackrest-s3-secret
key: ACCESS_KEY_ID
secretAccessKey:
name: pgbackrest-s3-secret
key: ACCESS_SECRET_KEY
```

:::note
The `s3Repositories` variable is a list. You can
configure multiple repositories. You can then select the repository to
which your backup will be performed. By default :
* the first repository is selected for backup ;
* WAL archiving always occurs on all repositories.
:::

3. Create the PostgreSQL `Cluster` and adapt the manifest by :
* adding the plugin definition `pgbackrest.dalibo.com` under the
`plugins` entry;
* referencing the pgBackRest `stanza` resource with `stanzaRef`.
1. Create a `Secret` named `pgbackrest-s3-secret` in the namespace of
the PostgreSQL `Cluster`. This secret must contain the `key` and
`secret-key` of the `s3` bucket.

Example:

```yaml title="cluster.yaml"
``` yaml
---
apiVersion: v1
kind: Secret
metadata:
name: pgbackrest-s3-secret
type: Opaque
stringData:
ACCESS_KEY_ID: <key_to_replace>
ACCESS_SECRET_KEY: <secret_to_replace>
```

2. Create a pgBackRest `stanza` :

Example:

``` yaml
---
apiVersion: pgbackrest.dalibo.com/v1
kind: Stanza
metadata:
name: stanza-sample
spec:
stanzaConfiguration:
name: main
s3Repositories:
- bucket: demo
endpoint: s3.minio.svc.cluster.local
region: us-east-1
repoPath: /cluster-demo
uriStyle: path
verifyTLS: false
retentionPolicy:
full: 7
fullType: count
diff: 14
archive: 2
archiveType: full
history: 30
secretRef:
accessKeyId:
name: pgbackrest-s3-secret
key: ACCESS_KEY_ID
secretAccessKey:
name: pgbackrest-s3-secret
key: ACCESS_SECRET_KEY
```

:::note The `s3Repositories` variable is a list. You can configure
multiple repositories. You can then select the repository to which your
backup will be performed. By default :

- the first repository is selected for backup ;
- WAL archiving always occurs on all repositories. :::

3. Create the PostgreSQL `Cluster` and adapt the manifest by :

- adding the plugin definition `pgbackrest.dalibo.com` under the
`plugins` entry;
- referencing the pgBackRest `stanza` resource with `stanzaRef`.

Example:

``` yaml
---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
Expand All @@ -100,24 +99,96 @@ spec:
size: 1Gi
```

If it runs without errors, the `Pod` dedicated to the PostgreSQL `Cluster`
should have now two containers. One for the `postgres` service (which is
the default setup), an other one for the pgbackrest plugin, named
`pgbackrest-plugin`. The injected container now holds the responsibility
for archiving the WALs and triggering backups when a backup request is
made.
If it runs without errors, the `Pod` dedicated to the PostgreSQL
`Cluster` should have now two containers. One for the `postgres` service
(which is the default setup), an other one for the pgbackrest plugin,
named `pgbackrest-plugin`. The injected container now holds the
responsibility for archiving the WALs and triggering backups when a
backup request is made.

## Stanza Initialization

Stanzas are initialized when archiving the first WAL. Since the stanza
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Stanzas are initialized when archiving the first WAL. Since the stanza
`Stanza` and repositories are initialized when archiving the first WAL. Since the `Stanza`

initialization state is tracked internally, restarting the sidecar
container will require running the `pgbackrest create-stanza` command
container will cause the `pgbackrest create-stanza` command to run
again.

Adding a new repository to a stanza can require running the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Adding a new repository to a stanza can require running the
Adding a new repository to a `Stanza` can require running the

`create-stanza` command again. Currently, this is not done
automatically. Restarting the `pgbackrest-plugin` container will launch
the create-stanza command.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
the create-stanza command.
the `create-stanza` command.


## WAL Archiving

WAL archiving can be customized through the `pgbackrest` CRD. It is
possible to define the WAL archiving strategy (e.g. [using the
`asynchronous`
mode](https://pgbackrest.org/configuration.html#section-archive/option-archive-async))
as well as configure the `pgbackrest` queue size.

## Restoring a Cluster

To restore a `Cluster` from a backup, create a new `Cluster` that
references the `Stanza` containing the backup. Below is an example:

``` yaml
---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: cluster-restored
spec:
instances: 1
plugins:
- name: pgbackrest.dalibo.com
parameters:
stanzaRef: stanza-sample
storage:
size: 1Gi
bootstrap:
recovery:
source: origin
externalClusters:
- name: origin
plugin:
name: pgbackrest.dalibo.com
parameters:
stanzaRef: stanza-sample
```

When using the recovery options, the `recoveryTarget` can be specified
to perform point-in-time recovery using a specific strategy (based on
time, LSN, etc.). If it is not specified, the recovery will continue up
to the latest available WAL.

``` yaml
---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: cluster-restored
spec:
instances: 1
plugins:
- name: pgbackrest.dalibo.com
parameters:
stanzaRef: stanza-sample
storage:
size: 1Gi
bootstrap:
recovery:
source: origin
recoveryTarget:
backupID: 20260210-101333F
externalClusters:
- name: origin
plugin:
name: pgbackrest.dalibo.com
parameters:
stanzaRef: stanza-sample
```

If no specific backup (BackupID) is specified, the plugin lets
pgBackRest automatically choose the optimal backup using its standard
algorithm. For more details, see the [pgBackRest restore
documentation](https://pgbackrest.org/command.html#command-restore).
36 changes: 27 additions & 9 deletions internal/pgbackrest/pgbackrest.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,21 @@ import (
"sync"

pgbackrestapi "github.com/dalibo/cnpg-i-pgbackrest/internal/pgbackrest/api"
"github.com/dalibo/cnpg-i-pgbackrest/internal/utils"
"sigs.k8s.io/controller-runtime/pkg/log"
)

type RestoreOptions struct {
Target string `json:"target,omitempty" env:"TARGET"`
TargetTimeline string `json:"targetTimeline,omitempty" env:"TARGET_TIMELINE"`
Type string `json:"type,omitempty" env:"TYPE"`
Set string `json:"set,omitempty" env:"SET"`
}

func (r RestoreOptions) ToEnv() ([]string, error) {
return utils.StructToEnvVars(r, "PGBACKREST_")
}

type BackupData struct {
Backup []pgbackrestapi.BackupInfo `json:"backup"`
}
Expand Down Expand Up @@ -148,7 +160,7 @@ func (p *PgBackrest) runBackgroundTask(
return result
}

func (p *PgBackrest) StanzaExists() (bool, error) {
func (p *PgBackrest) RepositoriesConfigured() (bool, error) {
cmd := p.run([]string{"info", "--output=json"}, nil)
stdout, err := cmd.CombinedOutput()
if err != nil {
Expand All @@ -158,26 +170,32 @@ func (p *PgBackrest) StanzaExists() (bool, error) {
if err := json.Unmarshal(stdout, &info); err != nil {
return false, fmt.Errorf("can't parse pgbackrest JSON: %w", err)
}
return parseDataForStatusCode(info), nil
return allReposHaveZeroStatusCode(info), nil
}

func parseDataForStatusCode(pgbackrestInfo []PgBackRestInfo) bool {
func allReposHaveZeroStatusCode(pgbackrestInfo []PgBackRestInfo) bool {
if len(pgbackrestInfo) == 0 {
return false
}
for _, entry := range pgbackrestInfo {
for _, repo := range entry.Repo {
if repo.Status.Code != nil && *(repo.Status.Code) == 0 {
return true
if len(entry.Repo) == 0 {
return false
}
for _, r := range entry.Repo {
if r.Status.Code == nil || *(r.Status.Code) != 0 {
return false
}
}
}
return false
return true
}

func (p *PgBackrest) EnsureStanzaExists(stanza string) (bool, error) {
stanzaExist, err := p.StanzaExists()
repoConfigured, err := p.RepositoriesConfigured()
if err != nil {
return false, fmt.Errorf("can't determine if stanza exists, error %w", err)
}
if stanzaExist {
if repoConfigured {
return false, nil
}
cmd := p.run([]string{"stanza-create", "--stanza=" + stanza}, nil)
Expand Down
Loading
Loading