Midio docs
API Docs
  • Getting Started
    • Welcome
    • Quick Start Guide
  • Your First Midio App
  • Guides
    • HTTP
      • Making HTTP Requests
      • Responding to HTTP Requests
      • How to reach your own endpoints
      • CORS
      • Server-Sent Events (SSE)
    • Branching
    • Loops
    • Map, filter, reduce
    • Working with Data
    • Expressions
    • Building Agents
      • Streaming Agent API (experimental)
    • Debugging
    • Secrets and Environment variables
    • Convert JSON to data nodes
    • Writing tests
    • Cleaning up your flows
  • Package Manager
  • Troubleshooting
  • Tutorials
    • A Fast Path to Functional RAG Agents
    • How to build a streaming agent using Server-Sent Events (SSE)
  • Reference
    • The Midio Editor
      • The Node Editor
      • User Traces
      • Traces (execution)
      • Processes
      • Log
      • Services
      • Problems
      • Function Signature
      • Data
      • Settings
    • The Midio Language
      • Nodes and execution
      • Functions and Events
        • Anonymous functions
      • Modules
      • Contexts
      • Data type
      • Local variables
      • Portals
      • Waypoint node
      • Partial function application
  • The Midio Engine
  • Built in Nodes
    • Core (std)
    • HTTP
    • LLM
Powered by GitBook
On this page

Was this helpful?

Edit on GitHub
  1. Guides
  2. Building Agents

Streaming Agent API (experimental)

PreviousBuilding AgentsNextDebugging

Last updated 4 days ago

Was this helpful?

Some of the LLM packages, like open-ai, has a streaming version of the Chat Complete function.

To use the streamed API, first call Chat Complete Streamed, and then call Next Event in a loop, to handle all streamed events.

The streamed API supports tool calls and structured output as well. However, unlike Chat Complete, it won't automatically call the LLM again after a set of tool calls is complete. You can do this yourself by appending the tool call results to the context, and then looping back to the Chat Complete Streamed, like so:

In the example above, we explicitly create a message list, instead of using the system message and user message inputs. This is just to make it easier to manage the context when we loop back after tool calls. Another approach would be to use two separate calls to chat complete, one that handles tool calls, and then one that summarizes the results.