feat: db migration

.frontmatter/database/mediaDb.json

@@ -1 +1 @@
-{}
+{ "src": { "utils": {} } }

README.md

@@ -4,10 +4,6 @@ This is the official website for Datum Inc., built with Astro.
 
 ## Getting Started
 
-### Prerequisites
-
-- Node.js (version specified in package.json)
-
 ### Installation
 
 1. Install dependencies:
@@ -37,10 +33,6 @@ npm run dev
 
 For detailed information about the project structure, see [PROJECT_STRUCTURE.md](./PROJECT_STRUCTURE.md).
 
-### Handbook
-
-Main file: src/content/handbook/index.md
-
 ### Image
 
 Import component
@@ -150,7 +142,7 @@ src/content/blog/from-cage-nuts-to-kubernetes.mdx
 The setup uses a multi-stage Dockerfile:
 
 1. Base stage (`node:22-alpine`)
-   - Minimal Alpine Linux with Node.js 22
+   - Minimum requirement Alpine Linux with Node.js 22
    - Common workspace setup
 
 2. Development stage
@@ -174,44 +166,27 @@ The setup uses a multi-stage Dockerfile:
   - `.:/src`: Source code
   - `/app/node_modules`: Dependencies
 
-### Troubleshooting
-
-1. If the development server isn't accessible:
-
-   ```bash
-   # Rebuild the development image
-   docker compose up dev --build
-   ```
-
-2. To view logs:
+## Development with local kubernetes
 
-   ```bash
-   # Development logs
-   docker compose logs dev
-
-   # Production logs
-   docker compose logs prod
-   ```
-
-3. To clean up:
+1. Install [Minikube](https://minikube.sigs.k8s.io/docs/) and start minikube
 
-   ```bash
-   # Stop and remove containers
-   docker compose down
+```
+minikube start
+```
 
-   # Remove volumes too
-   docker compose down -v
-   ```
+2. Install postgresql helm. example from bitnami source
 
-### Development with local kubernetes
+```bash
+helm install postgresql -f config/dev/postgres-values.yaml -n datum-net oci://registry-1.docker.io/bitnamicharts/postgresql
+```
 
-1. Install [Minikube](https://minikube.sigs.k8s.io/docs/) and start minikube
+3. Create namespace
 
 ```
-minikube start
+kubectl create namespace datum-net
 ```
 
-2. Create secret.yaml file separated with this source. Value:
+4. Create secret.yaml file separated with this source. Value:
 
 ```yaml
 apiVersion: v1
@@ -228,36 +203,28 @@ data:
   APP_PRIVATE_KEY:
 ```
 
-3. Then apply with command:
-
-Create namespace
-
-```
-kubectl create namespace datum-net
-```
-
-Apply secret
+Then apply with command:
 
 ```bash
-kubectl apply -f config/dev/secret.yaml
+kubectl apply -f ../secret.yaml
 ```
 
-Apply the kustomize config file
+5. Apply the kustomize config file
 
 ```bash
-kubectl apply -k config/base
+kubectl apply -k config/base -n datum-net
 ```
 
-Apply the kustomize postgres config file
+Postgres config file
 
 ```bash
 kubectl apply -k config/dev/postgres-config.yaml
 ```
 
-4. Install postgresql helm. example from bitnami source
+### Create port forwarder
 
 ```bash
-helm install postgresql -f config/dev/postgres-values.yaml -n datum-net oci://registry-1.docker.io/bitnamicharts/postgresql
+kubectl port-forward pod_name -n datum-net 4321:4321
 ```
 
 ## Contributing
@@ -340,23 +307,6 @@ The linter will only check files in the `/src/content/` directory, including:
 - Changelog entries
 - Static pages
 
-### VS Code Setup
-
-For the best development experience, install the following VS Code extensions:
-
-1. [ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint)
-2. [Prettier](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode)
-3. [Markdownlint](https://marketplace.visualstudio.com/items?itemName=DavidAnson.vscode-markdownlint)
-4. [Front Matter CMS](https://marketplace.visualstudio.com/items?itemName=eliostruyf.vscode-front-matter) - For content management
-
-The project includes VS Code settings (`.vscode/settings.json`) that enable:
-
-- Live linting and error detection
-- Format on save
-- Automatic ESLint fixes on save
-- Proper formatting for Astro, TypeScript, JavaScript, and Markdown files
-- TypeScript SDK integration
-
 ## Content Management with Front Matter CMS
 
 This project uses [Front Matter CMS](https://frontmatter.codes/) - a powerful headless CMS that runs directly in VS Code, providing a GUI for managing your Astro content collections.
@@ -500,6 +450,66 @@ The tests are configured to run in CI environments:
 - Takes screenshots on failures
 - Supports parallel test execution
 
+### VS Code Setup
+
+For the best development experience, install the following VS Code extensions:
+
+1. [ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint)
+2. [Prettier](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode)
+3. [Markdownlint](https://marketplace.visualstudio.com/items?itemName=DavidAnson.vscode-markdownlint)
+4. [Front Matter CMS](https://marketplace.visualstudio.com/items?itemName=eliostruyf.vscode-front-matter) - For content management
+
+The project includes VS Code settings (`.vscode/settings.json`) that enable:
+
+- Live linting and error detection
+- Format on save
+- Automatic ESLint fixes on save
+- Proper formatting for Astro, TypeScript, JavaScript, and Markdown files
+- TypeScript SDK integration
+
+### Troubleshooting
+
+1. If the development server isn't accessible:
+
+   ```bash
+   # Rebuild the development image
+   docker compose up dev --build
+   ```
+
+2. To view logs:
+
+   ```bash
+   # Development logs
+   docker compose logs dev
+
+   # Production logs
+   docker compose logs prod
+   ```
+
+3. To clean up:
+
+   ```bash
+   # Stop and remove containers
+   docker compose down
+
+   # Remove volumes too
+   docker compose down -v
+   ```
+
+Error: Failed to pull image "ghcr.io/datum-cloud/datum-net:latest": no matching manifest for linux/arm64/v8 in the manifest list entries
+
+Build Local ARM64 Image (For Local Development)
+If you're running Kubernetes locally (Docker Desktop, Minikube, etc.), build the image locally:
+
+```
+# Build for ARM64
+docker build -t ghcr.io/datum-cloud/datum-net:latest .
+
+# If using Docker Desktop with Kubernetes, the image is already available
+# If using Minikube, load the image:
+minikube image load ghcr.io/datum-cloud/datum-net:latest
+```
+
 ### Resources
 
 - [Front Matter CMS Documentation](https://frontmatter.codes/docs)

astro.config.mjs

@@ -12,6 +12,7 @@ import compressor from 'astro-compressor';
 
 import glossary from './src/plugins/glossary.js';
 import sitemap from './src/plugins/sitemap.js';
+import databaseMigration from './src/plugins/databaseMigration.ts';
 import announcement from './src/plugins/announcement.ts';
 import { remarkModifiedTime } from './src/plugins/remarkModifiedTime.mjs';
 
@@ -192,6 +193,7 @@ export default defineConfig({
       gzip: true,
       brotli: true,
     }),
+    databaseMigration(),
   ],
   vite: {
     plugins: [tailwindcss()],

config/base/deployment.yaml

@@ -30,7 +30,6 @@ spec:
       containers:
         - name: datum-net
           image: ghcr.io/datum-cloud/datum-net:latest
-
           imagePullPolicy: Always
           ports:
             - containerPort: 4321

config/dev/postgres-values.yaml

@@ -4,6 +4,9 @@ global:
     configurationCM: 'postgresql-config'
     auth:
       database: 'datum_net'
+      username: 'postgres'
+      password: 'datum'
+      postgresPassword: 'datum' # Admin password
     resources:
       requests:
         cpu: '1'

src/pages/dev/info.astro

@@ -15,9 +15,11 @@ if (mode == 'production') {
 let psqlInfo;
 try {
   const sql = await dbConnect();
-  psqlInfo = await sql`SELECT version()`;
-} catch {
-  psqlInfo = null;
+  const listTables = await sql`SELECT tablename FROM pg_tables WHERE schemaname='public'`;
+  const version = await sql`SELECT version()`;
+  psqlInfo = { version: version[0].version, tables: listTables.map((t) => t.tablename) };
+} catch (error) {
+  psqlInfo = { error };
 }
 
 type EnvValue = string | number | boolean | undefined;

src/plugins/databaseMigration.ts

@@ -0,0 +1,60 @@
+import { dbConnect } from '../libs/postgres';
+import type { AstroIntegration } from 'astro';
+
+/**
+ * Database migration integration for Astro
+ */
+async function migrateDatabase() {
+  try {
+    const POSTGRES_USER = process.env.POSTGRES_USER;
+    const sql = await dbConnect();
+    // check if table exists
+    const table = await sql`SELECT to_regclass('public.issues') as table_exists;`;
+
+    if (table[0].table_exists) {
+      console.log('DB Init: Database already initialized');
+    } else {
+      if (POSTGRES_USER) {
+        // Initialize tables
+        await sql`
+            CREATE TABLE IF NOT EXISTS issues (
+              id VARCHAR(32) PRIMARY KEY,
+              title VARCHAR(255) NOT NULL,
+              body TEXT NOT NULL,
+              url VARCHAR(255) NOT NULL,
+              updated_at TIMESTAMP DEFAULT NOW()
+            );
+
+            CREATE TABLE IF NOT EXISTS votes (
+              id VARCHAR(32) UNIQUE NOT NULL,
+              vote INTEGER DEFAULT 0,
+              updated_at TIMESTAMP DEFAULT NOW()
+            );
+
+            CREATE TABLE IF NOT EXISTS user_votes (
+              user_id VARCHAR(64) NOT NULL,
+              issue_id VARCHAR(32) NOT NULL,
+              CONSTRAINT user_issue UNIQUE (user_id,issue_id)
+            );
+
+            ALTER TABLE issues OWNER TO ${POSTGRES_USER};
+            ALTER TABLE votes OWNER TO ${POSTGRES_USER};
+            ALTER TABLE user_votes OWNER TO ${POSTGRES_USER};`;
+      }
+    }
+    console.log('DB Init: Database initialization completed');
+  } catch {
+    console.log('DB Init: Database initialization failed');
+  }
+}
+
+export default function databaseMigration(): AstroIntegration {
+  return {
+    name: 'database-migration',
+    hooks: {
+      'astro:build:done': async () => {
+        await migrateDatabase();
+      },
+    },
+  };
+}