Last quarter, I was contracted to build a custom AI assistant for a client’s internal project management dashboard. The client already had a robust session system tied to their user authentication, and they required that all conversation data between their team members and the assistant never left their cloud environment. Early tests with SaaS-hosted agent platforms revealed a critical mismatch: their built-in conversation state storage was tied to their own session IDs, which didn’t sync with the client’s existing session tokens. Worse, moving conversation history back and forth between the client’s UI and the platform risked exposing sensitive project data through third-party servers. This left us needing a solution that let us control every layer of conversation state, from storage to access.
After ruling out SaaS options due to the data residency and session alignment issues, I evaluated several open-source projects that could be deployed on the client’s private cloud. One tool stood out for its flexible state management options: I followed the FastGPT self-hosting guide to set up a local instance. Unlike SaaS platforms, this tool lets you define exactly how conversation state is stored and accessed, so we could align it directly with the client’s existing session infrastructure.
We integrated the client’s existing session middleware so that every request to the assistant includes the client’s session ID as a custom header, which the instance uses to map conversation threads directly to the user’s active dashboard session. We didn’t have to rebuild the entire conversation history tracking from scratch, either: the tool’s API lets you retrieve, update, and clear conversation state programmatically. When a user opened a specific project ticket in the dashboard, we passed the ticket’s ID as a context variable to the assistant, which then pulled relevant project details from the client’s database without storing that data in the assistant’s own backend. This kept all sensitive project data confined to the client’s infrastructure, matching their requirements perfectly.
There are a few key caveats to this approach, however. Self-hosting requires more upfront operational work than using a SaaS platform, particularly if your team is unfamiliar with managing containerized deployments on private cloud infrastructure. We also had to configure additional firewall rules to restrict access to the instance exclusively to the client’s internal network, which added a few extra hours to the initial setup. Additionally, since all conversation state is stored locally, you’ll need to build your own tooling for auditing or deleting user data if required by compliance regulations, though this is often easier than navigating third-party data retention policies. Even with these extra steps, the end result was a solution that fully aligned with the client’s security and session requirements, without sacrificing the core functionality of a custom AI assistant.