The tl;dr of it is there are ways to engineer this so the LLM doesnt get to “make up” data, the LLMs job is to just compose the query, and then it gets run against the DB and that returns to the user directly, preventing the LLM from just making shit up.
MCP Tools are powerful as hell for this, and its actually very viable to do.
See my post for how I’d solve this problem above here: https://lemmy.world/post/46926396/23775592
The tl;dr of it is there are ways to engineer this so the LLM doesnt get to “make up” data, the LLMs job is to just compose the query, and then it gets run against the DB and that returns to the user directly, preventing the LLM from just making shit up.
MCP Tools are powerful as hell for this, and its actually very viable to do.