|
| 1 | +# SourceFlow.Cloud.GCP |
| 2 | + |
| 3 | +**Google Cloud integration for distributed command and event processing** |
| 4 | + |
| 5 | +[](https://www.nuget.org/packages/SourceFlow.Cloud.GCP/) |
| 6 | +[](LICENSE) |
| 7 | + |
| 8 | +## Overview |
| 9 | + |
| 10 | +SourceFlow.Cloud.GCP extends the SourceFlow.Net framework with Google Cloud integration, enabling distributed command and event processing using Google Cloud Pub/Sub and Cloud KMS. The fluent bus API is identical to the AWS and Azure providers — only the backing services change. |
| 11 | + |
| 12 | +Google Cloud Pub/Sub has only **topics** and **subscriptions** (no separate queues). A command "queue" is modelled as a topic plus a pull subscription; an event "topic" is a topic plus a pull subscription per subscriber. |
| 13 | + |
| 14 | +**Key Features:** |
| 15 | +- 🚀 Pub/Sub command dispatching (publish to topics, pull from subscriptions) |
| 16 | +- 📢 Pub/Sub event publishing with per-subscriber pull subscriptions |
| 17 | +- 🔐 Cloud KMS envelope encryption for sensitive data |
| 18 | +- ⚙️ Fluent bus configuration API |
| 19 | +- 🔄 Automatic resource provisioning (topics + subscriptions) |
| 20 | +- 📊 Built-in health checks and OpenTelemetry metrics |
| 21 | +- 🧪 Pub/Sub emulator support for local development |
| 22 | + |
| 23 | +--- |
| 24 | + |
| 25 | +## Installation |
| 26 | + |
| 27 | +```bash |
| 28 | +dotnet add package SourceFlow.Cloud.GCP |
| 29 | +``` |
| 30 | + |
| 31 | +**Prerequisites:** SourceFlow ≥ 2.0.0, Google Cloud SDK / Application Default Credentials, .NET 8.0 / 9.0 / 10.0. |
| 32 | + |
| 33 | +--- |
| 34 | + |
| 35 | +## Quick Start |
| 36 | + |
| 37 | +```csharp |
| 38 | +using SourceFlow.Cloud.GCP; |
| 39 | + |
| 40 | +// Register SourceFlow core |
| 41 | +services.UseSourceFlow(typeof(Program).Assembly); |
| 42 | + |
| 43 | +// Configure Google Cloud Pub/Sub messaging |
| 44 | +services.UseSourceFlowGcp( |
| 45 | + options => { options.ProjectId = "my-project"; }, |
| 46 | + bus => bus |
| 47 | + .Send |
| 48 | + .Command<CreateOrderCommand>(q => q.Queue("orders")) |
| 49 | + .Command<ProcessPaymentCommand>(q => q.Queue("payments")) |
| 50 | + .Raise |
| 51 | + .Event<OrderCreatedEvent>(t => t.Topic("order-events")) |
| 52 | + .Event<PaymentProcessedEvent>(t => t.Topic("payment-events")) |
| 53 | + .Listen.To |
| 54 | + .CommandQueue("orders") |
| 55 | + .CommandQueue("payments") |
| 56 | + .Subscribe.To |
| 57 | + .Topic("order-events") |
| 58 | + .Topic("payment-events")); |
| 59 | +``` |
| 60 | + |
| 61 | +This registers GCP dispatchers, configures routing, starts the Pub/Sub pull listeners, and automatically provisions topics and subscriptions at startup. |
| 62 | + |
| 63 | +--- |
| 64 | + |
| 65 | +## Configuration Options |
| 66 | + |
| 67 | +| Option | Type | Default | Description | |
| 68 | +| --- | --- | --- | --- | |
| 69 | +| `ProjectId` | string | (required) | Google Cloud project that owns the topics/subscriptions | |
| 70 | +| `EnableCommandRouting` | bool | true | Enable command dispatching to topics | |
| 71 | +| `EnableEventRouting` | bool | true | Enable event publishing to topics | |
| 72 | +| `EnableCommandListener` | bool | true | Enable the command pull listener | |
| 73 | +| `EnableEventListener` | bool | true | Enable the event pull listener | |
| 74 | +| `MaxMessagesPerPull` | int | 10 | Messages requested per pull | |
| 75 | +| `AckDeadlineSeconds` | int | 60 | Ack deadline applied to subscriptions at bootstrap | |
| 76 | +| `SubscriptionSuffix` | string | `-sub` | Suffix used to derive a subscription id from a name | |
| 77 | + |
| 78 | +--- |
| 79 | + |
| 80 | +## Resource Provisioning |
| 81 | + |
| 82 | +The `GcpBusBootstrapper` runs as an `IHostedService` at startup and idempotently creates: |
| 83 | + |
| 84 | +- **Topics** — one per command queue name and per event topic name. |
| 85 | +- **Pull subscriptions** — `{name}-sub` for each listening command queue and each subscribed event topic. |
| 86 | + |
| 87 | +All operations tolerate `AlreadyExists`, so it is safe to run on every startup. |
| 88 | + |
| 89 | +--- |
| 90 | + |
| 91 | +## Message Encryption (Cloud KMS) |
| 92 | + |
| 93 | +```csharp |
| 94 | +services.AddSingleton(new GcpKmsOptions |
| 95 | +{ |
| 96 | + KeyName = "projects/my-project/locations/global/keyRings/my-ring/cryptoKeys/my-key" |
| 97 | +}); |
| 98 | +services.AddSingleton<IMessageEncryption, GcpKmsMessageEncryption>(); |
| 99 | +``` |
| 100 | + |
| 101 | +Envelope encryption: a random 256-bit data key encrypts the payload with AES-256-GCM, and Cloud KMS wraps (encrypts) the data key. Cloud KMS has no `GenerateDataKey` operation, so the data key is generated locally and wrapped via the KMS `Encrypt` call. |
| 102 | + |
| 103 | +--- |
| 104 | + |
| 105 | +## Local Development (Pub/Sub emulator) |
| 106 | + |
| 107 | +```bash |
| 108 | +gcloud beta emulators pubsub start --host-port=localhost:8085 |
| 109 | +export PUBSUB_EMULATOR_HOST=localhost:8085 |
| 110 | +``` |
| 111 | + |
| 112 | +The client libraries auto-detect `PUBSUB_EMULATOR_HOST` (via `EmulatorDetection.EmulatorOrProduction`). The bootstrapper creates topics/subscriptions in the emulator at startup — no manual setup required. |
| 113 | + |
| 114 | +--- |
| 115 | + |
| 116 | +## Idempotency |
| 117 | + |
| 118 | +- **In-memory (single instance)** — registered by default as a singleton with a background cleanup service. |
| 119 | +- **SQL-based (multi-instance / production)** — install `SourceFlow.Stores.EntityFramework` and call `services.AddSourceFlowIdempotency(connectionString)` before `UseSourceFlowGcp(...)`. |
| 120 | + |
| 121 | +--- |
| 122 | + |
| 123 | +## Monitoring |
| 124 | + |
| 125 | +- **Activity/Meter source:** `SourceFlow.Cloud.GCP` (`gcp.pubsub.commands.dispatched`, `gcp.pubsub.events.published`). |
| 126 | +- **Health check:** registered automatically; verifies Pub/Sub connectivity by listing topics in the project. |
| 127 | + |
| 128 | +--- |
| 129 | + |
| 130 | +## License |
| 131 | + |
| 132 | +MIT — see [LICENSE](LICENSE). |
0 commit comments