Skip to content

Commit 60d0bcd

Browse files
Pfannkuchensackclaudelsteindunkeroni
authored
Feature(UI): Canvas Workflow Integration - Run Workflow on Raster Layer (#8665)
* feat: Add canvas-workflow integration feature This commit implements a new feature that allows users to run workflows directly from the unified canvas. Users can now: - Access a "Run Workflow" option from the canvas layer context menu - Select a workflow with image parameters from a modal dialog - Customize workflow parameters (non-image fields) - Execute the workflow with the current canvas layer as input - Have the result automatically added back to the canvas Key changes: - Added canvasWorkflowIntegrationSlice for state management - Created CanvasWorkflowIntegrationModal and related UI components - Added context menu item to raster layers - Integrated workflow execution with canvas image extraction - Added modal to global modal isolator This integration enhances the canvas by allowing users to leverage custom workflows for advanced image processing directly within the canvas workspace. Implements feature request for deeper workflow-canvas integration. * refactor(ui): simplify canvas workflow integration field rendering - Extract WorkflowFieldRenderer component for individual field rendering - Add WorkflowFormPreview component to handle workflow parameter display - Remove workflow compatibility filtering - allow all workflows - Simplify workflow selector to use flattened workflow list - Add comprehensive field type support (String, Integer, Float, Boolean, Enum, Scheduler, Board, Model, Image, Color) - Implement image field selection UI with radio * feat(ui): add canvas-workflow-integration logging namespace * feat(ui): add workflow filtering for canvas-workflow integration - Add useFilteredWorkflows hook to filter workflows with ImageField inputs - Add workflowHasImageField utility to check for ImageField in Form Builder - Only show workflows that have Form Builder with at least one ImageField - Add loading state while filtering workflows - Improve error messages to clarify Form Builder requirement - Update modal description to mention Form Builder and parameter adjustment - Add fallback error message for workflows without Form Builder * feat(ui): add persistence and migration for canvas workflow integration state - Add _version field (v1) to canvasWorkflowIntegrationState for future migrations - Add persistConfig with migration function to handle version upgrades - Add persistDenylist to exclude transient state (isOpen, isProcessing, sourceEntityIdentifier) - Use es-toolkit isPlainObject and tsafe assert for type-safe migration - Persist selectedWorkflowId and fieldValues across sessions * pnpm fix imports * fix(ui): handle workflow errors in canvas staging area and improve form UX - Clear processing state when workflow execution fails at enqueue time or during invocation, so the modal doesn't get stuck - Optimistically update listAllQueueItems cache on queue item status changes so the staging area immediately exits on failure - Clear processing state on invocation_error for canvas workflow origin - Auto-select the only unfilled ImageField in workflow form - Fix image field overflow and thumbnail sizing in workflow form * feat(ui): add canvas_output node and entry-based staging area Add a dedicated `canvas_output` backend invocation node that explicitly marks which images go to the canvas staging area, replacing the fragile board-based heuristic. Each `canvas_output` node produces a separate navigable entry in the staging area, allowing workflows with multiple outputs to be individually previewed and accepted. Key changes: - New `CanvasOutputInvocation` backend node (canvas.py) - Entry-based staging area model where each output image is a separate navigable entry with flat next/prev cycling across all items - Frontend execute hook uses `canvas_output` type detection instead of board field heuristic, with proper board field value translation - Workflow filtering requires both Form Builder and canvas_output node - Updated QueueItemPreviewMini and StagingAreaItemsList for entries - Tests for entry-based navigation, multi-output, and race conditions * Chore pnp run fix * Chore eslint fix * Remove unused useOutputImageDTO export to fix knip lint * Update invokeai/frontend/web/src/features/controlLayers/components/CanvasWorkflowIntegration/useCanvasWorkflowIntegrationExecute.tsx Co-authored-by: dunkeroni <dunkeroni@gmail.com> * move UI text to en.json * fix conflicts merge with main * generate schema * Chore typegen --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com> Co-authored-by: dunkeroni <dunkeroni@gmail.com>
1 parent 80be1b7 commit 60d0bcd

28 files changed

Lines changed: 2510 additions & 162 deletions

‎invokeai/app/invocations/canvas.py‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
2+
from invokeai.app.invocations.fields import FieldDescriptions, ImageField, InputField
3+
from invokeai.app.invocations.primitives import ImageOutput
4+
from invokeai.app.services.shared.invocation_context import InvocationContext
5+
6+
7+
@invocation(
8+
"canvas_output",
9+
title="Canvas Output",
10+
tags=["canvas", "output", "image"],
11+
category="canvas",
12+
version="1.0.0",
13+
use_cache=False,
14+
)
15+
class CanvasOutputInvocation(BaseInvocation):
16+
"""Outputs an image to the canvas staging area.
17+
18+
Use this node in workflows intended for canvas workflow integration.
19+
Connect the final image of your workflow to this node to send it
20+
to the canvas staging area when run via 'Run Workflow on Canvas'."""
21+
22+
image: ImageField = InputField(description=FieldDescriptions.image)
23+
24+
def invoke(self, context: InvocationContext) -> ImageOutput:
25+
image = context.images.get_pil(self.image.image_name)
26+
image_dto = context.images.save(image=image)
27+
return ImageOutput.build(image_dto)

‎invokeai/frontend/web/public/locales/en.json‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2377,6 +2377,27 @@
23772377
"pullBboxIntoReferenceImageError": "Problem Pulling BBox Into ReferenceImage",
23782378
"addAdjustments": "Add Adjustments",
23792379
"removeAdjustments": "Remove Adjustments",
2380+
"workflowIntegration": {
2381+
"title": "Run Workflow on Canvas",
2382+
"description": "Select a workflow with a Canvas Output node and an image parameter to run on the current canvas layer. You can adjust parameters before executing. The result will be added back to the canvas.",
2383+
"execute": "Execute Workflow",
2384+
"executing": "Executing...",
2385+
"runWorkflow": "Run Workflow",
2386+
"filteringWorkflows": "Filtering workflows...",
2387+
"loadingWorkflows": "Loading workflows...",
2388+
"noWorkflowsFound": "No workflows found.",
2389+
"noWorkflowsWithImageField": "No compatible workflows found. A workflow needs a Form Builder with an image input field and a Canvas Output node.",
2390+
"selectWorkflow": "Select Workflow",
2391+
"selectPlaceholder": "Choose a workflow...",
2392+
"unnamedWorkflow": "Unnamed Workflow",
2393+
"loadingParameters": "Loading workflow parameters...",
2394+
"noFormBuilderError": "This workflow has no form builder and cannot be used. Please select a different workflow.",
2395+
"imageFieldSelected": "This field will receive the canvas image",
2396+
"imageFieldNotSelected": "Click to use this field for canvas image",
2397+
"executionStarted": "Workflow execution started",
2398+
"executionStartedDescription": "The result will appear in the staging area when complete.",
2399+
"executionFailed": "Failed to execute workflow"
2400+
},
23802401
"compositeOperation": {
23812402
"label": "Blend Mode",
23822403
"add": "Add Blend Mode",

‎invokeai/frontend/web/src/app/components/GlobalModalIsolator.tsx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import { GlobalImageHotkeys } from 'app/components/GlobalImageHotkeys';
22
import ChangeBoardModal from 'features/changeBoardModal/components/ChangeBoardModal';
33
import { CanvasPasteModal } from 'features/controlLayers/components/CanvasPasteModal';
4+
import { CanvasWorkflowIntegrationModal } from 'features/controlLayers/components/CanvasWorkflowIntegration/CanvasWorkflowIntegrationModal';
45
import { CanvasManagerProviderGate } from 'features/controlLayers/contexts/CanvasManagerProviderGate';
56
import { CropImageModal } from 'features/cropper/components/CropImageModal';
67
import { DeleteImageModal } from 'features/deleteImageModal/components/DeleteImageModal';
@@ -51,6 +52,7 @@ export const GlobalModalIsolator = memo(() => {
5152
<SaveWorkflowAsDialog />
5253
<CanvasManagerProviderGate>
5354
<CanvasPasteModal />
55+
<CanvasWorkflowIntegrationModal />
5456
</CanvasManagerProviderGate>
5557
<LoadWorkflowFromGraphModal />
5658
<CropImageModal />

‎invokeai/frontend/web/src/app/logging/logger.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ const $logger = atom<Logger>(Roarr.child(BASE_CONTEXT));
1616

1717
export const zLogNamespace = z.enum([
1818
'canvas',
19+
'canvas-workflow-integration',
1920
'config',
2021
'dnd',
2122
'events',

‎invokeai/frontend/web/src/app/store/store.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ import { canvasSettingsSliceConfig } from 'features/controlLayers/store/canvasSe
2525
import { canvasSliceConfig } from 'features/controlLayers/store/canvasSlice';
2626
import { canvasSessionSliceConfig } from 'features/controlLayers/store/canvasStagingAreaSlice';
2727
import { canvasTextSliceConfig } from 'features/controlLayers/store/canvasTextSlice';
28+
import { canvasWorkflowIntegrationSliceConfig } from 'features/controlLayers/store/canvasWorkflowIntegrationSlice';
2829
import { lorasSliceConfig } from 'features/controlLayers/store/lorasSlice';
2930
import { paramsSliceConfig } from 'features/controlLayers/store/paramsSlice';
3031
import { refImagesSliceConfig } from 'features/controlLayers/store/refImagesSlice';
@@ -67,6 +68,7 @@ const SLICE_CONFIGS = {
6768
[canvasSettingsSliceConfig.slice.reducerPath]: canvasSettingsSliceConfig,
6869
[canvasTextSliceConfig.slice.reducerPath]: canvasTextSliceConfig,
6970
[canvasSliceConfig.slice.reducerPath]: canvasSliceConfig,
71+
[canvasWorkflowIntegrationSliceConfig.slice.reducerPath]: canvasWorkflowIntegrationSliceConfig,
7072
[changeBoardModalSliceConfig.slice.reducerPath]: changeBoardModalSliceConfig,
7173
[dynamicPromptsSliceConfig.slice.reducerPath]: dynamicPromptsSliceConfig,
7274
[gallerySliceConfig.slice.reducerPath]: gallerySliceConfig,
@@ -98,6 +100,7 @@ const ALL_REDUCERS = {
98100
canvasSliceConfig.slice.reducer,
99101
canvasSliceConfig.undoableConfig?.reduxUndoOptions
100102
),
103+
[canvasWorkflowIntegrationSliceConfig.slice.reducerPath]: canvasWorkflowIntegrationSliceConfig.slice.reducer,
101104
[changeBoardModalSliceConfig.slice.reducerPath]: changeBoardModalSliceConfig.slice.reducer,
102105
[dynamicPromptsSliceConfig.slice.reducerPath]: dynamicPromptsSliceConfig.slice.reducer,
103106
[gallerySliceConfig.slice.reducerPath]: gallerySliceConfig.slice.reducer,

‎invokeai/frontend/web/src/features/controlLayers/components/CanvasWorkflowIntegration/CanvasWorkflowIntegrationModal.tsx‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
import {
2+
Button,
3+
ButtonGroup,
4+
Flex,
5+
Heading,
6+
Modal,
7+
ModalBody,
8+
ModalCloseButton,
9+
ModalContent,
10+
ModalFooter,
11+
ModalHeader,
12+
ModalOverlay,
13+
Spacer,
14+
Spinner,
15+
Text,
16+
} from '@invoke-ai/ui-library';
17+
import { useAppDispatch, useAppSelector } from 'app/store/storeHooks';
18+
import {
19+
canvasWorkflowIntegrationClosed,
20+
selectCanvasWorkflowIntegrationIsOpen,
21+
selectCanvasWorkflowIntegrationIsProcessing,
22+
selectCanvasWorkflowIntegrationSelectedWorkflowId,
23+
} from 'features/controlLayers/store/canvasWorkflowIntegrationSlice';
24+
import { memo, useCallback } from 'react';
25+
import { useTranslation } from 'react-i18next';
26+
27+
import { CanvasWorkflowIntegrationParameterPanel } from './CanvasWorkflowIntegrationParameterPanel';
28+
import { CanvasWorkflowIntegrationWorkflowSelector } from './CanvasWorkflowIntegrationWorkflowSelector';
29+
import { useCanvasWorkflowIntegrationExecute } from './useCanvasWorkflowIntegrationExecute';
30+
31+
export const CanvasWorkflowIntegrationModal = memo(() => {
32+
const { t } = useTranslation();
33+
const dispatch = useAppDispatch();
34+
35+
const isOpen = useAppSelector(selectCanvasWorkflowIntegrationIsOpen);
36+
const isProcessing = useAppSelector(selectCanvasWorkflowIntegrationIsProcessing);
37+
const selectedWorkflowId = useAppSelector(selectCanvasWorkflowIntegrationSelectedWorkflowId);
38+
39+
const { execute, canExecute } = useCanvasWorkflowIntegrationExecute();
40+
41+
const onClose = useCallback(() => {
42+
if (!isProcessing) {
43+
dispatch(canvasWorkflowIntegrationClosed());
44+
}
45+
}, [dispatch, isProcessing]);
46+
47+
const onExecute = useCallback(() => {
48+
execute();
49+
}, [execute]);
50+
51+
return (
52+
<Modal isOpen={isOpen} onClose={onClose} size="xl" isCentered>
53+
<ModalOverlay />
54+
<ModalContent>
55+
<ModalHeader>
56+
<Heading size="md">{t('controlLayers.workflowIntegration.title')}</Heading>
57+
</ModalHeader>
58+
<ModalCloseButton isDisabled={isProcessing} />
59+
60+
<ModalBody>
61+
<Flex direction="column" gap={4}>
62+
<Text fontSize="sm" color="base.400">
63+
{t('controlLayers.workflowIntegration.description')}
64+
</Text>
65+
66+
<CanvasWorkflowIntegrationWorkflowSelector />
67+
68+
{selectedWorkflowId && <CanvasWorkflowIntegrationParameterPanel />}
69+
</Flex>
70+
</ModalBody>
71+
72+
<ModalFooter>
73+
<ButtonGroup>
74+
<Button variant="ghost" onClick={onClose} isDisabled={isProcessing}>
75+
{t('common.cancel')}
76+
</Button>
77+
<Spacer />
78+
<Button
79+
onClick={onExecute}
80+
isDisabled={!canExecute || isProcessing}
81+
loadingText={t('controlLayers.workflowIntegration.executing')}
82+
>
83+
{isProcessing && <Spinner size="sm" mr={2} />}
84+
{t('controlLayers.workflowIntegration.execute')}
85+
</Button>
86+
</ButtonGroup>
87+
</ModalFooter>
88+
</ModalContent>
89+
</Modal>
90+
);
91+
});
92+
93+
CanvasWorkflowIntegrationModal.displayName = 'CanvasWorkflowIntegrationModal';

‎invokeai/frontend/web/src/features/controlLayers/components/CanvasWorkflowIntegration/CanvasWorkflowIntegrationParameterPanel.tsx‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
import { Box } from '@invoke-ai/ui-library';
2+
import { WorkflowFormPreview } from 'features/controlLayers/components/CanvasWorkflowIntegration/WorkflowFormPreview';
3+
import { memo } from 'react';
4+
5+
export const CanvasWorkflowIntegrationParameterPanel = memo(() => {
6+
return (
7+
<Box w="full">
8+
<WorkflowFormPreview />
9+
</Box>
10+
);
11+
});
12+
13+
CanvasWorkflowIntegrationParameterPanel.displayName = 'CanvasWorkflowIntegrationParameterPanel';

‎invokeai/frontend/web/src/features/controlLayers/components/CanvasWorkflowIntegration/CanvasWorkflowIntegrationWorkflowSelector.tsx‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
import { Flex, FormControl, FormLabel, Select, Spinner, Text } from '@invoke-ai/ui-library';
2+
import { useAppDispatch, useAppSelector } from 'app/store/storeHooks';
3+
import {
4+
canvasWorkflowIntegrationWorkflowSelected,
5+
selectCanvasWorkflowIntegrationSelectedWorkflowId,
6+
} from 'features/controlLayers/store/canvasWorkflowIntegrationSlice';
7+
import type { ChangeEvent } from 'react';
8+
import { memo, useCallback, useMemo } from 'react';
9+
import { useTranslation } from 'react-i18next';
10+
import { useListWorkflowsInfiniteInfiniteQuery } from 'services/api/endpoints/workflows';
11+
12+
import { useFilteredWorkflows } from './useFilteredWorkflows';
13+
14+
export const CanvasWorkflowIntegrationWorkflowSelector = memo(() => {
15+
const { t } = useTranslation();
16+
const dispatch = useAppDispatch();
17+
18+
const selectedWorkflowId = useAppSelector(selectCanvasWorkflowIntegrationSelectedWorkflowId);
19+
const { data: workflowsData, isLoading } = useListWorkflowsInfiniteInfiniteQuery(
20+
{
21+
per_page: 100, // Get a reasonable number of workflows
22+
page: 0,
23+
},
24+
{
25+
selectFromResult: ({ data, isLoading }) => ({
26+
data,
27+
isLoading,
28+
}),
29+
}
30+
);
31+
32+
const workflows = useMemo(() => {
33+
if (!workflowsData) {
34+
return [];
35+
}
36+
// Flatten all pages into a single list
37+
return workflowsData.pages.flatMap((page) => page.items);
38+
}, [workflowsData]);
39+
40+
// Filter workflows to only show those with ImageFields
41+
const { filteredWorkflows, isFiltering } = useFilteredWorkflows(workflows);
42+
43+
const onChange = useCallback(
44+
(e: ChangeEvent<HTMLSelectElement>) => {
45+
const workflowId = e.target.value || null;
46+
dispatch(canvasWorkflowIntegrationWorkflowSelected({ workflowId }));
47+
},
48+
[dispatch]
49+
);
50+
51+
if (isLoading || isFiltering) {
52+
return (
53+
<Flex alignItems="center" gap={2}>
54+
<Spinner size="sm" />
55+
<Text>
56+
{isFiltering
57+
? t('controlLayers.workflowIntegration.filteringWorkflows')
58+
: t('controlLayers.workflowIntegration.loadingWorkflows')}
59+
</Text>
60+
</Flex>
61+
);
62+
}
63+
64+
if (filteredWorkflows.length === 0) {
65+
return (
66+
<Text color="warning.400" fontSize="sm">
67+
{workflows.length === 0
68+
? t('controlLayers.workflowIntegration.noWorkflowsFound')
69+
: t('controlLayers.workflowIntegration.noWorkflowsWithImageField')}
70+
</Text>
71+
);
72+
}
73+
74+
return (
75+
<FormControl>
76+
<FormLabel>{t('controlLayers.workflowIntegration.selectWorkflow')}</FormLabel>
77+
<Select
78+
placeholder={t('controlLayers.workflowIntegration.selectPlaceholder')}
79+
value={selectedWorkflowId || ''}
80+
onChange={onChange}
81+
>
82+
{filteredWorkflows.map((workflow) => (
83+
<option key={workflow.workflow_id} value={workflow.workflow_id}>
84+
{workflow.name || t('controlLayers.workflowIntegration.unnamedWorkflow')}
85+
</option>
86+
))}
87+
</Select>
88+
</FormControl>
89+
);
90+
});
91+
92+
CanvasWorkflowIntegrationWorkflowSelector.displayName = 'CanvasWorkflowIntegrationWorkflowSelector';

0 commit comments

Comments
 (0)