122 lines
5.2 KiB
Markdown
122 lines
5.2 KiB
Markdown
# ComfyUI API SDK (Dart)
|
||
|
||
Light‑weight Dart client for interacting with a ComfyUI instance over HTTP + WebSocket.
|
||
|
||
## Supported Endpoints
|
||
|
||
(Original quick notes retained)
|
||
|
||
- Queue: `GET {host}/queue`
|
||
- History: `GET {host}/api/history?max_items=64`
|
||
- Submit prompt: `POST {host}/api/prompt`
|
||
- Models (aggregate): `GET {host}/api/experiment/models`
|
||
- Checkpoints:
|
||
- List: `GET {host}/api/experiment/models/checkpoints`
|
||
- Metadata: `GET {host}/api/view_metadata/checkpoints?filename={pathAndFileName}`
|
||
- LoRAs:
|
||
- List: `GET {host}/api/experiment/models/loras`
|
||
- Metadata: `GET {host}/api/view_metadata/loras?filename={pathAndFileName}`
|
||
- VAE:
|
||
- List: `GET {host}/api/experiment/models/vae`
|
||
- Metadata: `GET {host}/api/view_metadata/vae?filename={pathAndFileName}`
|
||
- Upscale Models:
|
||
- List: `GET {host}/api/experiment/models/upscale_models`
|
||
- Metadata: `GET {host}/api/view_metadata/upscale_models?filename={pathAndFileName}`
|
||
- Embeddings:
|
||
- List: `GET {host}/api/experiment/models/embeddings`
|
||
- Metadata: `GET {host}/api/view_metadata/embeddings?filename={pathAndFileName}`
|
||
- Object / Node Info: `GET {host}/api/object_info`
|
||
- Image fetch: `GET {host}/api/view?filename=ComfyUI_00006_.png`
|
||
- WebSocket events: `ws://{host}/ws?clientId={clientId}`
|
||
|
||
## WebSocket Event Model
|
||
|
||
Events are parsed into strongly typed objects:
|
||
|
||
- `WebSocketEvent` (generic envelope)
|
||
- `ProgressEvent` (value / max / promptId / node)
|
||
- `ExecutionEvent` (execution lifecycle + optional output image descriptors)
|
||
|
||
Callback registration (all delegated through `ComfyUiApi` → `WebSocketManager`):
|
||
|
||
```dart
|
||
api.onEventType(WebSocketEventType.progress, (e) { ... });
|
||
api.onProgressChanged((progress) { ... });
|
||
api.onPromptFinished((promptId) { ... });
|
||
api.onExecutionInterrupted(() { ... });
|
||
```
|
||
|
||
(See [`comfyui_api.dart`](comfyui_api_sdk/lib/src/comfyui_api.dart:45) and [`websocket_manager.dart`](comfyui_api_sdk/lib/src/websocket_manager.dart:1))
|
||
|
||
## Binary / Mixed Frame Handling (New)
|
||
|
||
Some ComfyUI deployments or reverse proxies may (now or in the future) emit non‑text WebSocket frames (e.g. experimental live previews). Previously this SDK assumed every frame was UTF‑8 JSON which caused a runtime type error:
|
||
|
||
```
|
||
type 'Uint8List' is not a subtype of type 'String'
|
||
```
|
||
|
||
### Patch Summary (2025‑08‑11)
|
||
|
||
Implemented in [`websocket_manager.dart`](comfyui_api_sdk/lib/src/websocket_manager.dart:1):
|
||
|
||
1. Frame Type Discrimination
|
||
- Accepts `String` frames directly.
|
||
- If `List<int>`: attempts UTF‑8 decode (malformed tolerant).
|
||
- Heuristic: Only treats decoded text as JSON if it begins with `{` or `[`.
|
||
- Binary frames with JPEG (`FF D8`) or PNG (`89 50 4E 47 0D 0A 1A 0A`) signatures are classified as potential preview frames (currently ignored but counted).
|
||
- Other binary frames are ignored safely (no exception thrown).
|
||
|
||
2. Defensive Parsing
|
||
- JSON decode wrapped in try/catch.
|
||
- Event construction wrapped in try/catch.
|
||
- Individual callback invocations isolated with try/catch so one faulty consumer does not break stream processing.
|
||
|
||
3. Statistics & Future Extensibility
|
||
- Exposed `stats` getter with counters: `ignoredBinaryFrames`, `malformedFrames`, `previewFrames`, `textFrames`, `retryAttempts`.
|
||
- Added (future) `previewFrames` stream (currently not emitted—line left commented to avoid premature API surface commitment).
|
||
|
||
4. Relative Imports
|
||
- Replaced `package:comfyui_api_sdk/...` self‑imports with relative imports to prevent duplicate type identities when the package is consumed via path or symlink (fixes “type X is not a subtype of type X” issues).
|
||
|
||
### No Consumer Changes Required
|
||
|
||
`PromptExecutionService` (in the parent app) already listens only for structured events; ignoring binary frames requires no modification.
|
||
|
||
## Manual Test / Reproduction Steps
|
||
|
||
1. Start app against a ComfyUI server that (optionally) emits progress.
|
||
2. (Optional) Inject a synthetic binary frame using a WebSocket proxy or small script to send raw JPEG bytes; verify no crash and counters increment:
|
||
```dart
|
||
print(api.webSocketManager.stats);
|
||
```
|
||
3. Normal JSON events continue flowing; progress & execution callbacks fire unchanged.
|
||
|
||
## Potential Future Preview Streaming
|
||
|
||
To enable real-time preview frames:
|
||
- Uncomment the line pushing binary frames to `_previewFrameController`.
|
||
- Provide a small header (e.g., magic + length + node id) if server supplies metadata.
|
||
- Add consumer in application layer to correlate preview with executing node id.
|
||
|
||
## Changelog
|
||
|
||
### 0.0.0-dev (Unreleased Internal)
|
||
- Added robust mixed text/binary WebSocket frame handling.
|
||
- Added frame classification & statistics.
|
||
- Added defensive callback isolation.
|
||
- Reworked imports to relative to avoid duplicate symbol identity issues when using path dependencies.
|
||
- Introduced (inactive) preview frame stream infrastructure.
|
||
|
||
## FAQ
|
||
|
||
**Q: How is `clientId` determined?**
|
||
A: The higher-level app assigns a UUID (or reuse a deterministic per-session id) and passes it when constructing `ComfyUiApi`. The server side merely uses it to correlate WebSocket with submitted prompts.
|
||
|
||
**Q: Why ignore binary preview frames instead of emitting them now?**
|
||
A: Keeps API stable until preview protocol (metadata framing) is formalized.
|
||
|
||
## License
|
||
|
||
Internal / TBD.
|