App Developers Rein In LLM Backends With 4 Context Levels and JSON Function Calls
Updated
Updated · InfoWorld · May 26
App Developers Rein In LLM Backends With 4 Context Levels and JSON Function Calls
2 articles · Updated · InfoWorld · May 26
Response schemas and function calling are emerging as the core pattern for turning messy user prompts into structured, executable actions without letting LLMs exceed what an app can actually do.
JSON enforcement—via response MIME types, schemas and validators such as Zod—lets models reliably return fields like SKU, quantity or location, while function calling has the model select tools and arguments and leaves execution to deterministic code.
Prompt routing can skip the model entirely on known paths, cutting both latency and token spend; the report argues developers should avoid vector databases until simpler approaches fail.
4 context tiers are outlined, from short state-driven strings and pinned rules to file-based retrieval and only then vector RAG, with MCP framed as better for broad cross-service agents than tightly controlled single-purpose apps.
Security remains the hard boundary: client-side function calls can speed prototypes and local state updates, but sensitive actions such as funds transfers or admin changes should execute only in secure server-side environments.
By forcing AI into rigid structures, are we sacrificing its creative potential just to make it fit into our existing apps?
As AI creators refuse to patch security flaws, are developers now the last line of defense against the next major cyberattack?
LLMs create a 'competence illusion' in users. How do we fix our evaluation systems before this leads to a real-world skills crisis?