1
We built a system where LLMs could schedule meetings and book rooms through a popular enterprise platform. On paper, we had our own version of Microsoft's Task Matrix, but in practice? The failure rate was brutal.
Here's the thing: making APIs "callable" by LLMs is just the easy first step. The real challenge is making them "friendly" to how models actually think. Being callable sets the floor; being friendly determines the ceiling (physical-world APIs are incredibly difficult for models to handle correctly).
Tools like Gitmcp, and mcpify.ai promise "quick" and "easy" MCP integration of existing systems. Sounds great until you realize they offer zero guarantees about actually working well.
So how do we bridge this gap between real-world APIs and model reasoning? Some technical approaches:
1. *API Composition*: Bundle low-level APIs into semantic units that match model thinking - Example: Scheduling a meeting is one concept to a model, not a series of ID lookups and calendar operations - These are fixed workflows that don't need agent-style reasoning or complex composition
2. *Input Processing*: Simplify parameter structures - Flatten nested structures into single-level representations - Add intermediate calculations for parameters (e.g., converting date strings to timestamps) to scaffold the model's reasoning
3. *Output Processing*: Refine API responses for semantic clarity - Specify clear next steps after each API call - Provide only essential information rather than raw API responses that force the model to filter through noise