Skip to content

Phase 7: File Management & Upload System - PCB Viewer Frontend #447

Description

@nam20485

Phase 7: File Management & Upload System

Parent Issue: #440
Duration: 2-3 days
Goal: Implement comprehensive file management system with drag-and-drop upload, progress tracking, and local file handling.

Overview

This phase focuses on creating a robust file management system that allows users to upload ODB++ design files, track upload progress, manage local files, and handle various file formats. The system will provide an intuitive interface for file operations and ensure reliable file handling.

Detailed Development Steps

7.1 File Upload Infrastructure

  • Implement Drag & Drop Upload Component

    // src/components/FileManagement/FileUploader.tsx
    import React, { useCallback, useState } from 'react';
    import { useDropzone } from 'react-dropzone';
    import { Upload, FileText, AlertCircle, CheckCircle2, X } from 'lucide-react';
    import { Button } from '@/components/ui/button';
    import { Progress } from '@/components/ui/progress';
    import { useFileUpload } from '@/hooks/useFileUpload';
    
    interface FileUploaderProps {
      onUploadComplete: (designId: string) => void;
      onUploadError: (error: string) => void;
      maxFileSize?: number; // in MB
      acceptedFormats?: string[];
    }
    
    export const FileUploader: React.FC<FileUploaderProps> = ({
      onUploadComplete,
      onUploadError,
      maxFileSize = 500, // 500MB default
      acceptedFormats = ['.tgz', '.tar.gz', '.zip', '.odb++']
    }) => {
      const [uploadQueue, setUploadQueue] = useState<UploadTask[]>([]);
      const { uploadFile, cancelUpload } = useFileUpload();
    
      const onDrop = useCallback(async (acceptedFiles: File[]) => {
        for (const file of acceptedFiles) {
          // Validate file
          const validation = validateFile(file, maxFileSize, acceptedFormats);
          if (!validation.isValid) {
            onUploadError(validation.error!);
            continue;
          }
    
          // Create upload task
          const uploadTask: UploadTask = {
            id: generateId(),
            file,
            status: 'pending',
            progress: 0,
            startTime: Date.now()
          };
    
          setUploadQueue(prev => [...prev, uploadTask]);
    
          try {
            // Start upload with progress tracking
            await uploadFile(file, {
              onProgress: (progress) => {
                setUploadQueue(prev => prev.map(task => 
                  task.id === uploadTask.id 
                    ? { ...task, progress, status: 'uploading' }
                    : task
                ));
              },
              onComplete: (designId) => {
                setUploadQueue(prev => prev.map(task => 
                  task.id === uploadTask.id 
                    ? { ...task, status: 'completed', designId }
                    : task
                ));
                onUploadComplete(designId);
              },
              onError: (error) => {
                setUploadQueue(prev => prev.map(task => 
                  task.id === uploadTask.id 
                    ? { ...task, status: 'error', error: error.message }
                    : task
                ));
                onUploadError(error.message);
              }
            });
          } catch (error) {
            console.error('Upload failed:', error);
          }
        }
      }, [uploadFile, onUploadComplete, onUploadError, maxFileSize, acceptedFormats]);
    
      const { getRootProps, getInputProps, isDragActive } = useDropzone({
        onDrop,
        accept: {
          'application/x-compressed': acceptedFormats,
          'application/zip': ['.zip'],
          'application/x-tar': ['.tar.gz', '.tgz']
        },
        maxSize: maxFileSize * 1024 * 1024,
        multiple: true
      });
    
      const removeCompletedTask = (taskId: string) => {
        setUploadQueue(prev => prev.filter(task => task.id !== taskId));
      };
    
      const cancelUploadTask = (taskId: string) => {
        const task = uploadQueue.find(t => t.id === taskId);
        if (task && task.status === 'uploading') {
          cancelUpload(taskId);
          setUploadQueue(prev => prev.map(t => 
            t.id === taskId ? { ...t, status: 'cancelled' } : t
          ));
        }
      };
    
      return (
        <div className="space-y-4">
          {/* Drop Zone */}
          <div
            {...getRootProps()}
            className={`
              border-2 border-dashed rounded-lg p-8 text-center cursor-pointer
              transition-colors duration-200
              ${isDragActive 
                ? 'border-blue-400 bg-blue-50' 
                : 'border-gray-300 hover:border-gray-400'
              }
            `}
          >
            <input {...getInputProps()} />
            <Upload className="w-12 h-12 mx-auto mb-4 text-gray-400" />
            <p className="text-lg font-medium text-gray-900 mb-2">
              {isDragActive 
                ? 'Drop your ODB++ files here' 
                : 'Drag & drop ODB++ files here'
              }
            </p>
            <p className="text-sm text-gray-500 mb-4">
              Supports: {acceptedFormats.join(', ')} up to {maxFileSize}MB
            </p>
            <Button variant="outline">
              Or click to browse files
            </Button>
          </div>
    
          {/* Upload Queue */}
          {uploadQueue.length > 0 && (
            <div className="space-y-2">
              <h3 className="font-medium text-gray-900">Upload Progress</h3>
              {uploadQueue.map(task => (
                <UploadTaskItem
                  key={task.id}
                  task={task}
                  onRemove={removeCompletedTask}
                  onCancel={cancelUploadTask}
                />
              ))}
            </div>
          )}
        </div>
      );
    };
    
    const UploadTaskItem: React.FC<{
      task: UploadTask;
      onRemove: (id: string) => void;
      onCancel: (id: string) => void;
    }> = ({ task, onRemove, onCancel }) => {
      const getStatusIcon = () => {
        switch (task.status) {
          case 'completed':
            return <CheckCircle2 className="w-5 h-5 text-green-500" />;
          case 'error':
          case 'cancelled':
            return <AlertCircle className="w-5 h-5 text-red-500" />;
          default:
            return <FileText className="w-5 h-5 text-blue-500" />;
        }
      };
    
      const getStatusText = () => {
        switch (task.status) {
          case 'pending':
            return 'Waiting...';
          case 'uploading':
            return `Uploading... ${Math.round(task.progress)}%`;
          case 'completed':
            return 'Upload complete';
          case 'error':
            return `Error: ${task.error}`;
          case 'cancelled':
            return 'Cancelled';
        }
      };
    
      return (
        <div className="flex items-center space-x-3 p-3 bg-gray-50 rounded-lg">
          {getStatusIcon()}
          <div className="flex-1 min-w-0">
            <p className="text-sm font-medium text-gray-900 truncate">
              {task.file.name}
            </p>
            <p className="text-xs text-gray-500">
              {formatFileSize(task.file.size)}  {getStatusText()}
            </p>
            {task.status === 'uploading' && (
              <Progress value={task.progress} className="mt-1" />
            )}
          </div>
          <div className="flex space-x-1">
            {task.status === 'uploading' && (
              <Button
                size="sm"
                variant="ghost"
                onClick={() => onCancel(task.id)}
              >
                Cancel
              </Button>
            )}
            {(task.status === 'completed' || task.status === 'error' || task.status === 'cancelled') && (
              <Button
                size="sm"
                variant="ghost"
                onClick={() => onRemove(task.id)}
              >
                <X className="w-4 h-4" />
              </Button>
            )}
          </div>
        </div>
      );
    };
  • Create Upload Hook with Progress Tracking

    // src/hooks/useFileUpload.ts
    import { useState, useCallback } from 'react';
    import { useOdbDesignApi } from './useOdbDesignApi';
    import { uploadStore } from '@/stores/uploadStore';
    
    interface UploadOptions {
      onProgress?: (progress: number) => void;
      onComplete?: (designId: string) => void;
      onError?: (error: Error) => void;
      chunkSize?: number;
    }
    
    export const useFileUpload = () => {
      const { apiClient } = useOdbDesignApi();
      const [activeUploads, setActiveUploads] = useState<Set<string>>(new Set());
    
      const uploadFile = useCallback(async (
        file: File,
        options: UploadOptions = {}
      ): Promise<string> => {
        const { onProgress, onComplete, onError, chunkSize = 1024 * 1024 } = options; // 1MB chunks
        const uploadId = generateUploadId();
    
        setActiveUploads(prev => new Set(prev).add(uploadId));
    
        try {
          // For large files, use chunked upload
          if (file.size > 10 * 1024 * 1024) { // 10MB threshold
            return await uploadFileChunked(file, uploadId, { onProgress, chunkSize });
          } else {
            return await uploadFileStandard(file, uploadId, { onProgress });
          }
        } catch (error) {
          onError?.(error as Error);
          throw error;
        } finally {
          setActiveUploads(prev => {
            const next = new Set(prev);
            next.delete(uploadId);
            return next;
          });
        }
      }, [apiClient]);
    
      const uploadFileStandard = async (
        file: File,
        uploadId: string,
        options: { onProgress?: (progress: number) => void }
      ): Promise<string> => {
        const formData = new FormData();
        formData.append('file', file);
        formData.append('uploadId', uploadId);
    
        const response = await fetch('/api/designs/upload', {
          method: 'POST',
          body: formData,
          headers: {
            'X-Upload-ID': uploadId
          }
        });
    
        if (!response.ok) {
          throw new Error(`Upload failed: ${response.statusText}`);
        }
    
        const result = await response.json();
        options.onProgress?.(100);
    
        return result.designId;
      };
    
      const uploadFileChunked = async (
        file: File,
        uploadId: string,
        options: { onProgress?: (progress: number) => void; chunkSize: number }
      ): Promise<string> => {
        const { chunkSize, onProgress } = options;
        const totalChunks = Math.ceil(file.size / chunkSize);
        let uploadedBytes = 0;
    
        // Initialize chunked upload
        await apiClient.post('/designs/upload/init', {
          uploadId,
          fileName: file.name,
          fileSize: file.size,
          totalChunks
        });
    
        // Upload chunks
        for (let chunkIndex = 0; chunkIndex < totalChunks; chunkIndex++) {
          const start = chunkIndex * chunkSize;
          const end = Math.min(start + chunkSize, file.size);
          const chunk = file.slice(start, end);
    
          const chunkFormData = new FormData();
          chunkFormData.append('chunk', chunk);
          chunkFormData.append('uploadId', uploadId);
          chunkFormData.append('chunkIndex', chunkIndex.toString());
          chunkFormData.append('totalChunks', totalChunks.toString());
    
          const response = await fetch('/api/designs/upload/chunk', {
            method: 'POST',
            body: chunkFormData,
            headers: {
              'X-Upload-ID': uploadId,
              'X-Chunk-Index': chunkIndex.toString()
            }
          });
    
          if (!response.ok) {
            throw new Error(`Chunk upload failed: ${response.statusText}`);
          }
    
          uploadedBytes += chunk.size;
          const progress = (uploadedBytes / file.size) * 100;
          onProgress?.(progress);
    
          // Check if upload was cancelled
          if (!activeUploads.has(uploadId)) {
            await apiClient.delete(`/designs/upload/${uploadId}`);
            throw new Error('Upload cancelled');
          }
        }
    
        // Finalize upload
        const finalizeResponse = await apiClient.post(`/designs/upload/${uploadId}/finalize`);
        return finalizeResponse.data.designId;
      };
    
      const cancelUpload = useCallback(async (uploadId: string) => {
        if (activeUploads.has(uploadId)) {
          try {
            await apiClient.delete(`/designs/upload/${uploadId}`);
          } catch (error) {
            console.error('Failed to cancel upload:', error);
          }
          setActiveUploads(prev => {
            const next = new Set(prev);
            next.delete(uploadId);
            return next;
          });
        }
      }, [apiClient, activeUploads]);
    
      const getUploadProgress = useCallback((uploadId: string) => {
        return uploadStore.getUploadProgress(uploadId);
      }, []);
    
      return {
        uploadFile,
        cancelUpload,
        getUploadProgress,
        activeUploads: Array.from(activeUploads)
      };
    };

7.2 File Management Interface

  • Create File Manager Component
    // src/components/FileManagement/FileManager.tsx
    import React, { useState, useEffect } from 'react';
    import { 
      Folder, 
      File, 
      Download, 
      Trash2, 
      Eye, 
      MoreHorizontal,
      Search,
      SortAsc,
      SortDesc,
      Grid,
      List
    } from 'lucide-react';
    import { Button } from '@/components/ui/button';
    import { Input } from '@/components/ui/input';
    import { 
      DropdownMenu,
      DropdownMenuContent,
      DropdownMenuItem,
      DropdownMenuTrigger
    } from '@/components/ui/dropdown-menu';
    import { useDesignFiles } from '@/hooks/useDesignFiles';
    import { FileUploader } from './FileUploader';
    
    interface FileManagerProps {
      onFileSelect: (designId: string) => void;
      currentDesignId?: string;
    }
    
    export const FileManager: React.FC<FileManagerProps> = ({
      onFileSelect,
      currentDesignId
    }) => {
      const [viewMode, setViewMode] = useState<'grid' | 'list'>('list');
      const [sortBy, setSortBy] = useState<'name' | 'date' | 'size'>('date');
      const [sortOrder, setSortOrder] = useState<'asc' | 'desc'>('desc');
      const [searchQuery, setSearchQuery] = useState('');
      const [showUploader, setShowUploader] = useState(false);
    
      const {
        files,
        loading,
        error,
        deleteFile,
        downloadFile,
        refreshFiles
      } = useDesignFiles();
    
      useEffect(() => {
        refreshFiles();
      }, [refreshFiles]);
    
      const filteredFiles = files
        .filter(file => 
          file.name.toLowerCase().includes(searchQuery.toLowerCase()) ||
          file.description?.toLowerCase().includes(searchQuery.toLowerCase())
        )
        .sort((a, b) => {
          let comparison = 0;
          
          switch (sortBy) {
            case 'name':
              comparison = a.name.localeCompare(b.name);
              break;
            case 'date':
              comparison = new Date(a.uploadedAt).getTime() - new Date(b.uploadedAt).getTime();
              break;
            case 'size':
              comparison = a.size - b.size;
              break;
          }
          
          return sortOrder === 'asc' ? comparison : -comparison;
        });
    
      const handleFileUploadComplete = (designId: string) => {
        refreshFiles();
        onFileSelect(designId);
        setShowUploader(false);
      };
    
      const handleDeleteFile = async (fileId: string) => {
        if (window.confirm('Are you sure you want to delete this file?')) {
          await deleteFile(fileId);
          refreshFiles();
        }
      };
    
      if (loading) {
        return (
          <div className="flex items-center justify-center h-64">
            <div className="animate-spin rounded-full h-8 w-8 border-b-2 border-blue-600"></div>
          </div>
        );
      }
    
      if (error) {
        return (
          <div className="text-center text-red-600 p-4">
            <p>Error loading files: {error}</p>
            <Button onClick={refreshFiles} className="mt-2">
              Try Again
            </Button>
          </div>
        );
      }
    
      return (
        <div className="h-full flex flex-col">
          {/* Header */}
          <div className="flex items-center justify-between p-4 border-b">
            <h2 className="text-lg font-semibold">Design Files</h2>
            <Button onClick={() => setShowUploader(true)}>
              Upload New File
            </Button>
          </div>
    
          {/* Toolbar */}
          <div className="flex items-center justify-between p-4 bg-gray-50 border-b">
            <div className="flex items-center space-x-2">
              <div className="relative">
                <Search className="absolute left-3 top-1/2 transform -translate-y-1/2 text-gray-400 w-4 h-4" />
                <Input
                  placeholder="Search files..."
                  value={searchQuery}
                  onChange={(e) => setSearchQuery(e.target.value)}
                  className="pl-10 w-64"
                />
              </div>
            </div>
            
            <div className="flex items-center space-x-2">
              {/* Sort Controls */}
              <DropdownMenu>
                <DropdownMenuTrigger asChild>
                  <Button variant="outline" size="sm">
                    Sort by {sortBy}
                    {sortOrder === 'asc' ? <SortAsc className="w-4 h-4 ml-1" /> : <SortDesc className="w-4 h-4 ml-1" />}
                  </Button>
                </DropdownMenuTrigger>
                <DropdownMenuContent>
                  <DropdownMenuItem onClick={() => setSortBy('name')}>
                    Name
                  </DropdownMenuItem>
                  <DropdownMenuItem onClick={() => setSortBy('date')}>
                    Date
                  </DropdownMenuItem>
                  <DropdownMenuItem onClick={() => setSortBy('size')}>
                    Size
                  </DropdownMenuItem>
                  <DropdownMenuItem onClick={() => setSortOrder(sortOrder === 'asc' ? 'desc' : 'asc')}>
                    {sortOrder === 'asc' ? 'Descending' : 'Ascending'}
                  </DropdownMenuItem>
                </DropdownMenuContent>
              </DropdownMenu>
    
              {/* View Mode Toggle */}
              <div className="flex border rounded-lg">
                <Button
                  variant={viewMode === 'list' ? 'default' : 'ghost'}
                  size="sm"
                  onClick={() => setViewMode('list')}
                  className="rounded-r-none"
                >
                  <List className="w-4 h-4" />
                </Button>
                <Button
                  variant={viewMode === 'grid' ? 'default' : 'ghost'}
                  size="sm"
                  onClick={() => setViewMode('grid')}
                  className="rounded-l-none"
                >
                  <Grid className="w-4 h-4" />
                </Button>
              </div>
            </div>
          </div>
    
          {/* File List */}
          <div className="flex-1 overflow-auto p-4">
            {filteredFiles.length === 0 ? (
              <div className="text-center text-gray-500 mt-8">
                <Folder className="w-12 h-12 mx-auto mb-4 text-gray-300" />
                <p>No design files found</p>
                <p className="text-sm">Upload your first ODB++ file to get started</p>
              </div>
            ) : (
              <div className={viewMode === 'grid' ? 'grid grid-cols-3 gap-4' : 'space-y-2'}>
                {filteredFiles.map(file => (
                  <FileItem
                    key={file.id}
                    file={file}
                    viewMode={viewMode}
                    isSelected={file.id === currentDesignId}
                    onSelect={() => onFileSelect(file.id)}
                    onDelete={() => handleDeleteFile(file.id)}
                    onDownload={() => downloadFile(file.id)}
                  />
                ))}
              </div>
            )}
          </div>
    
          {/* Upload Modal */}
          {showUploader && (
            <div className="fixed inset-0 bg-black bg-opacity-50 flex items-center justify-center z-50">
              <div className="bg-white rounded-lg p-6 w-full max-w-2xl mx-4">
                <div className="flex justify-between items-center mb-4">
                  <h3 className="text-lg font-semibold">Upload Design File</h3>
                  <Button
                    variant="ghost"
                    onClick={() => setShowUploader(false)}
                  >
                    ×
                  </Button>
                </div>
                <FileUploader
                  onUploadComplete={handleFileUploadComplete}
                  onUploadError={(error) => console.error('Upload error:', error)}
                />
              </div>
            </div>
          )}
        </div>
      );
    };
    
    const FileItem: React.FC<{
      file: DesignFile;
      viewMode: 'grid' | 'list';
      isSelected: boolean;
      onSelect: () => void;
      onDelete: () => void;
      onDownload: () => void;
    }> = ({ file, viewMode, isSelected, onSelect, onDelete, onDownload }) => {
      if (viewMode === 'grid') {
        return (
          <div
            className={`
              p-4 border rounded-lg cursor-pointer transition-all
              ${isSelected ? 'border-blue-500 bg-blue-50' : 'border-gray-200 hover:border-gray-300'}
            `}
            onClick={onSelect}
          >
            <div className="flex flex-col items-center">
              <File className="w-12 h-12 text-blue-500 mb-2" />
              <p className="font-medium text-center truncate w-full" title={file.name}>
                {file.name}
              </p>
              <p className="text-xs text-gray-500">
                {formatFileSize(file.size)}
              </p>
              <p className="text-xs text-gray-400">
                {formatDate(file.uploadedAt)}
              </p>
            </div>
            <div className="flex justify-center mt-2 space-x-1">
              <Button size="sm" variant="ghost" onClick={(e) => { e.stopPropagation(); onDownload(); }}>
                <Download className="w-3 h-3" />
              </Button>
              <Button size="sm" variant="ghost" onClick={(e) => { e.stopPropagation(); onDelete(); }}>
                <Trash2 className="w-3 h-3" />
              </Button>
            </div>
          </div>
        );
      }
    
      return (
        <div
          className={`
            flex items-center p-3 border rounded-lg cursor-pointer transition-all
            ${isSelected ? 'border-blue-500 bg-blue-50' : 'border-gray-200 hover:border-gray-300'}
          `}
          onClick={onSelect}
        >
          <File className="w-8 h-8 text-blue-500 mr-3" />
          <div className="flex-1 min-w-0">
            <p className="font-medium truncate" title={file.name}>
              {file.name}
            </p>
            <p className="text-sm text-gray-500">
              {formatFileSize(file.size)}  {formatDate(file.uploadedAt)}
            </p>
          </div>
          <div className="flex items-center space-x-2">
            <Button size="sm" variant="ghost" onClick={(e) => { e.stopPropagation(); onDownload(); }}>
              <Download className="w-4 h-4" />
            </Button>
            <DropdownMenu>
              <DropdownMenuTrigger asChild>
                <Button size="sm" variant="ghost" onClick={(e) => e.stopPropagation()}>
                  <MoreHorizontal className="w-4 h-4" />
                </Button>
              </DropdownMenuTrigger>
              <DropdownMenuContent>
                <DropdownMenuItem onClick={onDownload}>
                  <Download className="w-4 h-4 mr-2" />
                  Download
                </DropdownMenuItem>
                <DropdownMenuItem onClick={onDelete} className="text-red-600">
                  <Trash2 className="w-4 h-4 mr-2" />
                  Delete
                </DropdownMenuItem>
              </DropdownMenuContent>
            </DropdownMenu>
          </div>
        </div>
      );
    };

7.3 File Validation & Processing

  • Implement File Validation System
    // src/services/fileValidation.ts
    export class FileValidator {
      private static readonly SUPPORTED_FORMATS = [
        'application/x-compressed',
        'application/zip',
        'application/x-tar',
        'application/gzip'
      ];
    
      private static readonly MAX_FILE_SIZE = 1024 * 1024 * 1024; // 1GB
      private static readonly MIN_FILE_SIZE = 1024; // 1KB
    
      static validateFile(file: File, options: ValidationOptions = {}): ValidationResult {
        const {
          maxSize = this.MAX_FILE_SIZE,
          minSize = this.MIN_FILE_SIZE,
          allowedFormats = this.SUPPORTED_FORMATS
        } = options;
    
        // Check file size
        if (file.size > maxSize) {
          return {
            isValid: false,
            error: `File size (${formatFileSize(file.size)}) exceeds maximum allowed size (${formatFileSize(maxSize)})`
          };
        }
    
        if (file.size < minSize) {
          return {
            isValid: false,
            error: `File size (${formatFileSize(file.size)}) is below minimum required size (${formatFileSize(minSize)})`
          };
        }
    
        // Check file format by extension
        const fileExtension = file.name.toLowerCase().split('.').pop();
        const isValidExtension = this.isValidFileExtension(fileExtension);
        
        if (!isValidExtension) {
          return {
            isValid: false,
            error: `Unsupported file format. Supported formats: .tgz, .tar.gz, .zip, .odb++`
          };
        }
    
        // Check MIME type if available
        if (file.type && !allowedFormats.some(format => file.type.includes(format))) {
          return {
            isValid: false,
            error: `Unsupported MIME type: ${file.type}`
          };
        }
    
        return { isValid: true };
      }
    
      private static isValidFileExtension(extension: string | undefined): boolean {
        if (!extension) return false;
        
        const validExtensions = ['tgz', 'gz', 'zip', 'odb++', 'tar'];
        return validExtensions.includes(extension);
      }
    
      static async validateFileStructure(file: File): Promise<StructureValidationResult> {
        try {
          // For zip files, we can check the structure
          if (file.name.toLowerCase().endsWith('.zip')) {
            return await this.validateZipStructure(file);
          }
          
          // For tar.gz files, we need to extract and check
          if (file.name.toLowerCase().endsWith('.tgz') || file.name.toLowerCase().endsWith('.tar.gz')) {
            return await this.validateTarStructure(file);
          }
    
          // For other formats, assume valid structure
          return {
            isValid: true,
            hasValidStructure: true,
            detectedFormat: 'unknown'
          };
        } catch (error) {
          return {
            isValid: false,
            hasValidStructure: false,
            error: `Failed to validate file structure: ${error.message}`
          };
        }
      }
    
      private static async validateZipStructure(file: File): Promise<StructureValidationResult> {
        return new Promise((resolve) => {
          const reader = new FileReader();
          reader.onload = (e) => {
            try {
              const arrayBuffer = e.target?.result as ArrayBuffer;
              const uint8Array = new Uint8Array(arrayBuffer);
              
              // Check ZIP file signature
              if (uint8Array[0] !== 0x50 || uint8Array[1] !== 0x4B) {
                resolve({
                  isValid: false,
                  hasValidStructure: false,
                  error: 'Invalid ZIP file signature'
                });
                return;
              }
    
              // TODO: Implement proper ZIP structure validation
              // For now, just check if it looks like a ZIP file
              resolve({
                isValid: true,
                hasValidStructure: true,
                detectedFormat: 'zip',
                estimatedUncompressedSize: file.size * 3 // Rough estimate
              });
            } catch (error) {
              resolve({
                isValid: false,
                hasValidStructure: false,
                error: error.message
              });
            }
          };
          
          reader.onerror = () => {
            resolve({
              isValid: false,
              hasValidStructure: false,
              error: 'Failed to read file'
            });
          };
          
          // Read first 1KB to check file signature
          reader.readAsArrayBuffer(file.slice(0, 1024));
        });
      }
    
      private static async validateTarStructure(file: File): Promise<StructureValidationResult> {
        // Similar implementation for tar.gz files
        return new Promise((resolve) => {
          const reader = new FileReader();
          reader.onload = (e) => {
            try {
              const arrayBuffer = e.target?.result as ArrayBuffer;
              const uint8Array = new Uint8Array(arrayBuffer);
              
              // Check GZIP file signature
              if (uint8Array[0] !== 0x1F || uint8Array[1] !== 0x8B) {
                resolve({
                  isValid: false,
                  hasValidStructure: false,
                  error: 'Invalid GZIP file signature'
                });
                return;
              }
    
              resolve({
                isValid: true,
                hasValidStructure: true,
                detectedFormat: 'tar.gz',
                estimatedUncompressedSize: file.size * 4 // Rough estimate
              });
            } catch (error) {
              resolve({
                isValid: false,
                hasValidStructure: false,
                error: error.message
              });
            }
          };
          
          reader.onerror = () => {
            resolve({
              isValid: false,
              hasValidStructure: false,
              error: 'Failed to read file'
            });
          };
          
          // Read first 1KB to check file signature
          reader.readAsArrayBuffer(file.slice(0, 1024));
        });
      }
    
      static async extractFileMetadata(file: File): Promise<FileMetadata> {
        const structureValidation = await this.validateFileStructure(file);
        
        return {
          name: file.name,
          size: file.size,
          type: file.type,
          lastModified: new Date(file.lastModified),
          format: structureValidation.detectedFormat || 'unknown',
          estimatedUncompressedSize: structureValidation.estimatedUncompressedSize,
          checksum: await this.calculateChecksum(file)
        };
      }
    
      private static async calculateChecksum(file: File): Promise<string> {
        const buffer = await file.arrayBuffer();
        const hashBuffer = await crypto.subtle.digest('SHA-256', buffer);
        const hashArray = Array.from(new Uint8Array(hashBuffer));
        return hashArray.map(b => b.toString(16).padStart(2, '0')).join('');
      }
    }

7.4 Local File Storage & Caching

  • Implement IndexedDB Storage for Local Files
    // src/services/storage/localFileStorage.ts
    export class LocalFileStorage {
      private dbName = 'pcb-viewer-files';
      private dbVersion = 1;
      private db: IDBDatabase | null = null;
    
      async initialize(): Promise<void> {
        return new Promise((resolve, reject) => {
          const request = indexedDB.open(this.dbName, this.dbVersion);
          
          request.onerror = () => reject(request.error);
          request.onsuccess = () => {
            this.db = request.result;
            resolve();
          };
          
          request.onupgradeneeded = (event) => {
            const db = (event.target as IDBOpenDBRequest).result;
            
            // Create object stores
            if (!db.objectStoreNames.contains('files')) {
              const fileStore = db.createObjectStore('files', { keyPath: 'id' });
              fileStore.createIndex('name', 'name', { unique: false });
              fileStore.createIndex('uploadedAt', 'uploadedAt', { unique: false });
            }
            
            if (!db.objectStoreNames.contains('metadata')) {
              db.createObjectStore('metadata', { keyPath: 'fileId' });
            }
            
            if (!db.objectStoreNames.contains('cache')) {
              const cacheStore = db.createObjectStore('cache', { keyPath: 'key' });
              cacheStore.createIndex('fileId', 'fileId', { unique: false });
              cacheStore.createIndex('timestamp', 'timestamp', { unique: false });
            }
          };
        });
      }
    
      async storeFile(file: StoredFile): Promise<void> {
        if (!this.db) throw new Error('Database not initialized');
        
        return new Promise((resolve, reject) => {
          const transaction = this.db!.transaction(['files'], 'readwrite');
          const store = transaction.objectStore('files');
          const request = store.put(file);
          
          request.onerror = () => reject(request.error);
          request.onsuccess = () => resolve();
        });
      }
    
      async getFile(fileId: string): Promise<StoredFile | null> {
        if (!this.db) throw new Error('Database not initialized');
        
        return new Promise((resolve, reject) => {
          const transaction = this.db!.transaction(['files'], 'readonly');
          const store = transaction.objectStore('files');
          const request = store.get(fileId);
          
          request.onerror = () => reject(request.error);
          request.onsuccess = () => resolve(request.result || null);
        });
      }
    
      async getAllFiles(): Promise<StoredFile[]> {
        if (!this.db) throw new Error('Database not initialized');
        
        return new Promise((resolve, reject) => {
          const transaction = this.db!.transaction(['files'], 'readonly');
          const store = transaction.objectStore('files');
          const request = store.getAll();
          
          request.onerror = () => reject(request.error);
          request.onsuccess = () => resolve(request.result);
        });
      }
    
      async deleteFile(fileId: string): Promise<void> {
        if (!this.db) throw new Error('Database not initialized');
        
        return new Promise((resolve, reject) => {
          const transaction = this.db!.transaction(['files', 'metadata', 'cache'], 'readwrite');
          
          // Delete from all stores
          transaction.objectStore('files').delete(fileId);
          transaction.objectStore('metadata').delete(fileId);
          
          // Delete related cache entries
          const cacheStore = transaction.objectStore('cache');
          const index = cacheStore.index('fileId');
          const cacheRequest = index.openCursor(IDBKeyRange.only(fileId));
          
          cacheRequest.onsuccess = (event) => {
            const cursor = (event.target as IDBRequest).result;
            if (cursor) {
              cursor.delete();
              cursor.continue();
            }
          };
          
          transaction.onerror = () => reject(transaction.error);
          transaction.oncomplete = () => resolve();
        });
      }
    
      async storeMetadata(fileId: string, metadata: FileMetadata): Promise<void> {
        if (!this.db) throw new Error('Database not initialized');
        
        return new Promise((resolve, reject) => {
          const transaction = this.db!.transaction(['metadata'], 'readwrite');
          const store = transaction.objectStore('metadata');
          const request = store.put({ fileId, ...metadata });
          
          request.onerror = () => reject(request.error);
          request.onsuccess = () => resolve();
        });
      }
    
      async getMetadata(fileId: string): Promise<FileMetadata | null> {
        if (!this.db) throw new Error('Database not initialized');
        
        return new Promise((resolve, reject) => {
          const transaction = this.db!.transaction(['metadata'], 'readonly');
          const store = transaction.objectStore('metadata');
          const request = store.get(fileId);
          
          request.onerror = () => reject(request.error);
          request.onsuccess = () => {
            const result = request.result;
            if (result) {
              // Remove fileId from the result
              const { fileId, ...metadata } = result;
              resolve(metadata);
            } else {
              resolve(null);
            }
          };
        });
      }
    
      async cacheData(key: string, data: any, fileId?: string): Promise<void> {
        if (!this.db) throw new Error('Database not initialized');
        
        return new Promise((resolve, reject) => {
          const transaction = this.db!.transaction(['cache'], 'readwrite');
          const store = transaction.objectStore('cache');
          const request = store.put({
            key,
            data,
            fileId,
            timestamp: Date.now()
          });
          
          request.onerror = () => reject(request.error);
          request.onsuccess = () => resolve();
        });
      }
    
      async getCachedData(key: string): Promise<any | null> {
        if (!this.db) throw new Error('Database not initialized');
        
        return new Promise((resolve, reject) => {
          const transaction = this.db!.transaction(['cache'], 'readonly');
          const store = transaction.objectStore('cache');
          const request = store.get(key);
          
          request.onerror = () => reject(request.error);
          request.onsuccess = () => {
            const result = request.result;
            resolve(result ? result.data : null);
          };
        });
      }
    
      async clearOldCache(maxAge: number = 24 * 60 * 60 * 1000): Promise<void> {
        if (!this.db) throw new Error('Database not initialized');
        
        const cutoffTime = Date.now() - maxAge;
        
        return new Promise((resolve, reject) => {
          const transaction = this.db!.transaction(['cache'], 'readwrite');
          const store = transaction.objectStore('cache');
          const index = store.index('timestamp');
          const request = index.openCursor(IDBKeyRange.upperBound(cutoffTime));
          
          request.onerror = () => reject(request.error);
          request.onsuccess = (event) => {
            const cursor = (event.target as IDBRequest).result;
            if (cursor) {
              cursor.delete();
              cursor.continue();
            } else {
              resolve();
            }
          };
        });
      }
    
      async getStorageUsage(): Promise<StorageUsage> {
        if (!this.db) throw new Error('Database not initialized');
        
        const [files, metadata, cache] = await Promise.all([
          this.getAllFiles(),
          this.getAllMetadata(),
          this.getAllCacheEntries()
        ]);
        
        const filesSize = files.reduce((total, file) => total + file.size, 0);
        const metadataSize = JSON.stringify(metadata).length;
        const cacheSize = cache.reduce((total, entry) => total + JSON.stringify(entry.data).length, 0);
        
        return {
          totalSize: filesSize + metadataSize + cacheSize,
          filesSize,
          metadataSize,
          cacheSize,
          fileCount: files.length,
          cacheEntries: cache.length
        };
      }
    
      private async getAllMetadata(): Promise<any[]> {
        return new Promise((resolve, reject) => {
          const transaction = this.db!.transaction(['metadata'], 'readonly');
          const store = transaction.objectStore('metadata');
          const request = store.getAll();
          
          request.onerror = () => reject(request.error);
          request.onsuccess = () => resolve(request.result);
        });
      }
    
      private async getAllCacheEntries(): Promise<any[]> {
        return new Promise((resolve, reject) => {
          const transaction = this.db!.transaction(['cache'], 'readonly');
          const store = transaction.objectStore('cache');
          const request = store.getAll();
          
          request.onerror = () => reject(request.error);
          request.onsuccess = () => resolve(request.result);
        });
      }
    }

7.5 File Download & Export

  • Implement File Download Manager
    // src/services/fileDownload.ts
    export class FileDownloadManager {
      private downloadQueue: Map<string, DownloadTask> = new Map();
      private maxConcurrentDownloads = 3;
      private activeDownloads = 0;
    
      async downloadFile(
        fileId: string,
        fileName: string,
        options: DownloadOptions = {}
      ): Promise<void> {
        const { onProgress, onComplete, onError } = options;
    
        const downloadTask: DownloadTask = {
          id: generateId(),
          fileId,
          fileName,
          status: 'pending',
          progress: 0,
          startTime: Date.now(),
          onProgress,
          onComplete,
          onError
        };
    
        this.downloadQueue.set(downloadTask.id, downloadTask);
        this.processDownloadQueue();
      }
    
      private async processDownloadQueue(): Promise<void> {
        if (this.activeDownloads >= this.maxConcurrentDownloads) {
          return;
        }
    
        const pendingTask = Array.from(this.downloadQueue.values())
          .find(task => task.status === 'pending');
    
        if (!pendingTask) {
          return;
        }
    
        this.activeDownloads++;
        pendingTask.status = 'downloading';
    
        try {
          await this.executeDownload(pendingTask);
          pendingTask.status = 'completed';
          pendingTask.onComplete?.();
        } catch (error) {
          pendingTask.status = 'error';
          pendingTask.error = error.message;
          pendingTask.onError?.(error as Error);
        } finally {
          this.activeDownloads--;
          this.downloadQueue.delete(pendingTask.id);
          
          // Process next item in queue
          setTimeout(() => this.processDownloadQueue(), 100);
        }
      }
    
      private async executeDownload(task: DownloadTask): Promise<void> {
        const response = await fetch(`/api/designs/${task.fileId}/download`, {
          method: 'GET',
          headers: {
            'Accept': 'application/octet-stream'
          }
        });
    
        if (!response.ok) {
          throw new Error(`Download failed: ${response.statusText}`);
        }
    
        const contentLength = response.headers.get('Content-Length');
        const totalSize = contentLength ? parseInt(contentLength, 10) : 0;
    
        const reader = response.body?.getReader();
        if (!reader) {
          throw new Error('Response body is not readable');
        }
    
        const chunks: Uint8Array[] = [];
        let downloadedSize = 0;
    
        while (true) {
          const { done, value } = await reader.read();
          
          if (done) break;
          
          chunks.push(value);
          downloadedSize += value.length;
          
          if (totalSize > 0) {
            const progress = (downloadedSize / totalSize) * 100;
            task.progress = progress;
            task.onProgress?.(progress);
          }
        }
    
        // Create blob and trigger download
        const blob = new Blob(chunks);
        const url = URL.createObjectURL(blob);
        
        const link = document.createElement('a');
        link.href = url;
        link.download = task.fileName;
        document.body.appendChild(link);
        link.click();
        document.body.removeChild(link);
        
        URL.revokeObjectURL(url);
      }
    
      cancelDownload(downloadId: string): void {
        const task = this.downloadQueue.get(downloadId);
        if (task && task.status === 'downloading') {
          task.status = 'cancelled';
          this.downloadQueue.delete(downloadId);
          this.activeDownloads--;
        }
      }
    
      getDownloadStatus(downloadId: string): DownloadTask | null {
        return this.downloadQueue.get(downloadId) || null;
      }
    
      getAllDownloads(): DownloadTask[] {
        return Array.from(this.downloadQueue.values());
      }
    }

Acceptance Criteria

  • Drag & Drop Upload: Users can drag ODB++ files directly into the interface
  • Progress Tracking: Real-time upload progress with ability to cancel
  • File Validation: Comprehensive validation before and during upload
  • File Management: Browse, search, sort, and manage uploaded files
  • Local Storage: Efficient local caching of file metadata and previews
  • Download Support: Download original files or processed data

Testing Checklist

  • Test drag & drop with various file formats and sizes
  • Verify progress tracking accuracy during large file uploads
  • Test file validation with invalid files
  • Verify chunked upload for files >10MB
  • Test file management operations (sort, search, delete)
  • Verify local storage persistence across browser sessions
  • Test download functionality with various file sizes

Output/Deliverables

  1. Drag & drop file upload component with progress tracking
  2. Comprehensive file validation system
  3. File management interface with search and sorting
  4. Local storage system using IndexedDB
  5. File download manager with queue support
  6. Upload hooks and state management

Dependencies

  • Phase 6 must be completed - requires performance infrastructure for upload optimization
  • Backend API endpoints - for file upload/download operations

Next Phase Dependencies

Phase 8 (User Settings) depends on this phase providing:

  • File storage capabilities for user preferences
  • Local storage infrastructure for settings persistence

Estimated Time: 2-3 days
Priority: High (Core functionality)
Complexity: Medium

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions