fixes to websocket type casting
Signed-off-by: Menno van Leeuwen <menno@vleeuwen.me>
This commit is contained in:
143
README.md
143
README.md
@@ -1,46 +1,121 @@
|
||||
// Found requests:
|
||||
// To get the current queue
|
||||
// GET ${host}/queue
|
||||
# ComfyUI API SDK (Dart)
|
||||
|
||||
// To view a image
|
||||
// GET ${host}/api/view?filename=ComfyUI_00006_.png
|
||||
Light‑weight Dart client for interacting with a ComfyUI instance over HTTP + WebSocket.
|
||||
|
||||
// To get the history of the queue
|
||||
// GET ${host}/api/history?max_items=64
|
||||
## Supported Endpoints
|
||||
|
||||
// To post a new image generation request to the queue
|
||||
// POST ${host}/api/prompt
|
||||
// Content-Type: application/json
|
||||
// { ... }
|
||||
(Original quick notes retained)
|
||||
|
||||
// To request a list of all the available models
|
||||
// GET ${host}/api/experiment/models
|
||||
- Queue: `GET {host}/queue`
|
||||
- History: `GET {host}/api/history?max_items=64`
|
||||
- Submit prompt: `POST {host}/api/prompt`
|
||||
- Models (aggregate): `GET {host}/api/experiment/models`
|
||||
- Checkpoints:
|
||||
- List: `GET {host}/api/experiment/models/checkpoints`
|
||||
- Metadata: `GET {host}/api/view_metadata/checkpoints?filename={pathAndFileName}`
|
||||
- LoRAs:
|
||||
- List: `GET {host}/api/experiment/models/loras`
|
||||
- Metadata: `GET {host}/api/view_metadata/loras?filename={pathAndFileName}`
|
||||
- VAE:
|
||||
- List: `GET {host}/api/experiment/models/vae`
|
||||
- Metadata: `GET {host}/api/view_metadata/vae?filename={pathAndFileName}`
|
||||
- Upscale Models:
|
||||
- List: `GET {host}/api/experiment/models/upscale_models`
|
||||
- Metadata: `GET {host}/api/view_metadata/upscale_models?filename={pathAndFileName}`
|
||||
- Embeddings:
|
||||
- List: `GET {host}/api/experiment/models/embeddings`
|
||||
- Metadata: `GET {host}/api/view_metadata/embeddings?filename={pathAndFileName}`
|
||||
- Object / Node Info: `GET {host}/api/object_info`
|
||||
- Image fetch: `GET {host}/api/view?filename=ComfyUI_00006_.png`
|
||||
- WebSocket events: `ws://{host}/ws?clientId={clientId}`
|
||||
|
||||
// To request a list of checkpoints (Or details for a specific checkpoint)
|
||||
// GET ${host}/api/experiment/models/checkpoints
|
||||
// GET ${host}/api/view_metadata/checkpoints?filename=${pathAndFileName}
|
||||
## WebSocket Event Model
|
||||
|
||||
// To request a list of loras (Or details for a specific lora)
|
||||
// GET ${host}/api/experiment/models/loras
|
||||
// GET ${host}/api/view_metadata/loras?filename=${pathAndFileName}
|
||||
Events are parsed into strongly typed objects:
|
||||
|
||||
// To request a list of VAEs (Or details for a specific VAE)
|
||||
// GET ${host}/api/experiment/models/vae
|
||||
// GET ${host}/api/view_metadata/vae?filename=${pathAndFileName}
|
||||
- `WebSocketEvent` (generic envelope)
|
||||
- `ProgressEvent` (value / max / promptId / node)
|
||||
- `ExecutionEvent` (execution lifecycle + optional output image descriptors)
|
||||
|
||||
// To request a list of upscale models (Or details for a specific upscale model)
|
||||
// GET ${host}/api/experiment/models/upscale_models
|
||||
// GET ${host}/api/view_metadata/upscale_models?filename=${pathAndFileName}
|
||||
Callback registration (all delegated through `ComfyUiApi` → `WebSocketManager`):
|
||||
|
||||
// To request a list of embeddings (Or details for a specific embedding)
|
||||
// GET ${host}/api/experiment/models/embeddings
|
||||
// GET ${host}/api/view_metadata/embeddings?filename=${pathAndFileName}
|
||||
```dart
|
||||
api.onEventType(WebSocketEventType.progress, (e) { ... });
|
||||
api.onProgressChanged((progress) { ... });
|
||||
api.onPromptFinished((promptId) { ... });
|
||||
api.onExecutionInterrupted(() { ... });
|
||||
```
|
||||
|
||||
// To get object info (Checkpoints, models, loras etc)
|
||||
// GET ${host}/api/object_info
|
||||
(See [`comfyui_api.dart`](comfyui_api_sdk/lib/src/comfyui_api.dart:45) and [`websocket_manager.dart`](comfyui_api_sdk/lib/src/websocket_manager.dart:1))
|
||||
|
||||
// WebSocket for progress updates
|
||||
// ws://${host}/ws?clientId=${clientId}
|
||||
## Binary / Mixed Frame Handling (New)
|
||||
|
||||
// Final question
|
||||
// How do we figure out the clientId
|
||||
Some ComfyUI deployments or reverse proxies may (now or in the future) emit non‑text WebSocket frames (e.g. experimental live previews). Previously this SDK assumed every frame was UTF‑8 JSON which caused a runtime type error:
|
||||
|
||||
```
|
||||
type 'Uint8List' is not a subtype of type 'String'
|
||||
```
|
||||
|
||||
### Patch Summary (2025‑08‑11)
|
||||
|
||||
Implemented in [`websocket_manager.dart`](comfyui_api_sdk/lib/src/websocket_manager.dart:1):
|
||||
|
||||
1. Frame Type Discrimination
|
||||
- Accepts `String` frames directly.
|
||||
- If `List<int>`: attempts UTF‑8 decode (malformed tolerant).
|
||||
- Heuristic: Only treats decoded text as JSON if it begins with `{` or `[`.
|
||||
- Binary frames with JPEG (`FF D8`) or PNG (`89 50 4E 47 0D 0A 1A 0A`) signatures are classified as potential preview frames (currently ignored but counted).
|
||||
- Other binary frames are ignored safely (no exception thrown).
|
||||
|
||||
2. Defensive Parsing
|
||||
- JSON decode wrapped in try/catch.
|
||||
- Event construction wrapped in try/catch.
|
||||
- Individual callback invocations isolated with try/catch so one faulty consumer does not break stream processing.
|
||||
|
||||
3. Statistics & Future Extensibility
|
||||
- Exposed `stats` getter with counters: `ignoredBinaryFrames`, `malformedFrames`, `previewFrames`, `textFrames`, `retryAttempts`.
|
||||
- Added (future) `previewFrames` stream (currently not emitted—line left commented to avoid premature API surface commitment).
|
||||
|
||||
4. Relative Imports
|
||||
- Replaced `package:comfyui_api_sdk/...` self‑imports with relative imports to prevent duplicate type identities when the package is consumed via path or symlink (fixes “type X is not a subtype of type X” issues).
|
||||
|
||||
### No Consumer Changes Required
|
||||
|
||||
`PromptExecutionService` (in the parent app) already listens only for structured events; ignoring binary frames requires no modification.
|
||||
|
||||
## Manual Test / Reproduction Steps
|
||||
|
||||
1. Start app against a ComfyUI server that (optionally) emits progress.
|
||||
2. (Optional) Inject a synthetic binary frame using a WebSocket proxy or small script to send raw JPEG bytes; verify no crash and counters increment:
|
||||
```dart
|
||||
print(api.webSocketManager.stats);
|
||||
```
|
||||
3. Normal JSON events continue flowing; progress & execution callbacks fire unchanged.
|
||||
|
||||
## Potential Future Preview Streaming
|
||||
|
||||
To enable real-time preview frames:
|
||||
- Uncomment the line pushing binary frames to `_previewFrameController`.
|
||||
- Provide a small header (e.g., magic + length + node id) if server supplies metadata.
|
||||
- Add consumer in application layer to correlate preview with executing node id.
|
||||
|
||||
## Changelog
|
||||
|
||||
### 0.0.0-dev (Unreleased Internal)
|
||||
- Added robust mixed text/binary WebSocket frame handling.
|
||||
- Added frame classification & statistics.
|
||||
- Added defensive callback isolation.
|
||||
- Reworked imports to relative to avoid duplicate symbol identity issues when using path dependencies.
|
||||
- Introduced (inactive) preview frame stream infrastructure.
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: How is `clientId` determined?**
|
||||
A: The higher-level app assigns a UUID (or reuse a deterministic per-session id) and passes it when constructing `ComfyUiApi`. The server side merely uses it to correlate WebSocket with submitted prompts.
|
||||
|
||||
**Q: Why ignore binary preview frames instead of emitting them now?**
|
||||
A: Keeps API stable until preview protocol (metadata framing) is formalized.
|
||||
|
||||
## License
|
||||
|
||||
Internal / TBD.
|
||||
|
||||
Reference in New Issue
Block a user