Gemini function calling in Java: how much you end up doing yourself

What Gemini function calling actually does#

Given the function declarations attached to a request, Gemini chooses what to call, fills in the arguments, and returns a functionCall in one response part. It does not execute the function or fetch the result. The application owns the next four steps.

1. Pick the parts containing a functionCall out of the stream.
2. Look up the implementation by function name and execute it.
3. Wrap the result in a FunctionResponse and, if the model returned a call ID, attach that same value.
4. Put all those responses in a Content object with role="user" and send it back into the same conversation.

If the model sees those results and asks for another tool, the entire sequence runs again. Function calling is therefore less an API feature than a loop the application has to write, including its stopping condition. At the time, there was no Java example to copy. When I checked in August 2026, the official function calling docs had example tabs for Python, JavaScript, and REST, but not Java. The pattern below was therefore derived from the public API surface of com.google.genai 1.44.0 rather than translated from another language's sample. By September 2026, the default docs had added Java through the newer Interactions API; you still have to write this legacy Chat loop yourself.

A manual function-calling loop in Java#

// Illustrative — not the real code. Class and method names follow the public com.google.genai API.
public String converse(String question, String promptId) {
    Tool tools = Tool.builder().functionDeclarations(registry.visibleTo(promptId)).build();
    Chat chat = client.chats.create(model,
            GenerateContentConfig.builder().tools(List.of(tools)).build());
    Content next = Content.builder().role("user")
            .parts(List.of(Part.builder().text(question).build())).build();
    StringBuilder answer = new StringBuilder();

    while (true) {
        List<FunctionCall> pending = new ArrayList<>();
        try (ResponseStream<GenerateContentResponse> stream = chat.sendMessageStream(next)) {
            for (GenerateContentResponse chunk : stream) {
                for (Part part : partsOf(chunk)) {
                    if (part.text().isEmpty() && part.functionCall().isPresent()) {
                        pending.add(part.functionCall().get());  // Rule 1: collect until the turn is complete
                    } else {
                        part.text().ifPresent(answer::append);
                    }
                }
            }
        }
        if (pending.isEmpty()) return answer.toString();  // No tool request means this text is final

        List<Part> responses = new ArrayList<>();
        Set<String> seen = new HashSet<>();
        for (FunctionCall fc : pending) {
            String callId = fc.id().orElse(null);
            if (callId != null && !seen.add(callId)) continue;  // Rule 2: deduplicate only repeated model IDs
            var response = FunctionResponse.builder()
                    .name(fc.name().orElse("")).response(runAsResult(fc));  // Rule 3: failures are results
            fc.id().ifPresent(response::id);              // Reuse the model ID; never fabricate one
            responses.add(Part.builder().functionResponse(response.build()).build());
        }
        next = Content.builder().role("user").parts(responses).build();  // Rule 4: one turn for all responses
    }
}

/** Rule 3: convert a failure into a result the model can read. */
private Map<String, Object> runAsResult(FunctionCall fc) {
    try {
        return Map.of("status", "success",
                "data", registry.require(fc.name().orElse("")).call(fc.args().orElse(Map.of())));
    } catch (Exception e) {
        return Map.of("status", "error",
                "message", "This tool is unavailable right now. Do not call the same tool again.");
    }
}

Rule 1 — Collect. Do not execute a functionCall as soon as it appears. The model can emit several calls in separate chunks within one turn. Wait until the part traversal is complete, then process the collected calls together so one round closes in a single pass. Tools inside a round may run in parallel; only the boundary between rounds is sequential.

Rule 2 — Call IDs. The model calls the same function several times with different arguments. An aggregation tool may be called repeatedly with a different grouping dimension each time. Deduplicate by function name and you misclassify valid parallel calls, discard results, and leave the response count out of sync with the call count. The API then rejects the request with a 400. If the model supplies an ID, use that value for duplicate detection and carry the same value into the FunctionResponse. If id() is empty, do not invent a UUID: pairing is defined only by the ID the model produced. Execute each call as it came and omit the response ID instead. For reference, the legacy generateContent docs (checked September 2026) state that Gemini 3 always returns a unique id with every functionCall, so an empty id() belongs to the model and SDK combination this chapter covers.

Rule 3 — Represent failure. Let a tool failure throw and the conversation ends. A failure is also a result. Return something the model can read, such as {"status":"error", "message":"This tool is unavailable right now. Do not call the same tool again."}. The last sentence matters. Report only the failure and the model keeps calling the same tool in a loop.

Rule 4 — Keep responses in one turn. A function-call turn must be followed by a function-response turn. If several functions were called, their responses must arrive as one unit, carrying the matching model-provided ID whenever one exists. Send them one by one and the conversation history becomes invalid, so the request is rejected. Keeping the same shape even for a one-tool round removes the single-versus-batch branch entirely.

Production topology — browser, server, and model#

The loop above closes inside one server method. The production architecture did not. An HTTP boundary split the collector from the executor, and another split result reinjection from the next model round trip.

There are many steps, but one line is the hinge: the server closes the stream.

// Illustrative — not the real code. This is the end of the server's first stream.
// After collecting calls as above, text chunks continue downstream and the calls leave as one event.
if (!pending.isEmpty()) {
    emitter.send(batchToolRequest(pending));   // Function name, arguments, call ID, execution scope
    emitter.complete();                        // ← Loop control moves to the browser here
    return;                                    // The server has no variable that counts the rounds
}

When a resume request arrives, the server retrieves the Chat instance from the session, completes execution and reinjection, and asks the model again. That block is one iteration of the loop; the loop itself lives in the browser.

Why go through the browser? Some tools cannot run on the server. "Take me to that screen" is routing, and routing exists only in the browser. The clipboard is the same. I therefore added an execution-scope field to each tool definition, and the server copied it into every tool request. The browser split the batch by the scope supplied by the server before consulting its registry. I judged that putting browser-only operations in the same tool list meant some point in the loop had to pass through the browser. That decision had three costs.

Every round opens another stream. The connection closes at the end of each tool round, so N rounds open and close N+1 streams. Spring MVC's SseEmitter releases the Servlet request thread through asynchronous processing, but each round still creates another HTTP request and individual response writes remain blocking. Authentication and signature headers are sent again as well.

The loop state is absent from the server. The browser knows which round it is on. The server simply emits a new call and closes the stream, receiving each request without knowing which iteration of which conversation it represents. Another path in the same codebase — a one-shot prompt execution rather than chat — already had an internal server loop. The server-side technique was known; only the chat path remained browser-driven.

The scope field does not describe the actual executor. An action visible in the browser could take three paths.

The second row says server, but the browser performs the action. That mismatch leaves no signal and becomes visible only when someone reads the class body. The third row bypasses the tool mechanism entirely, so it appears in neither execution state nor history. Answering "Where does this tool run?" required checking all three places.

How I would build it now#

Reading the records again, the architecture needed only one SSE connection kept open toward the user. The initial request could still open an emitter on the server. The fork should have come after that: there was no need to stream every model call.

• Call genai with the non-streaming method. A turn that needs a tool returns function calls rather than user-facing text, so there is nothing useful to watch token by token.
• When function calls arrive, execute them on the server and continue the loop there. Send only progress events — "thinking," "called this tool" — through the open emitter.
• Once the loop reaches the final-answer turn, either stream that answer or send it as one response.

The genai SDK allows this combination. The same Chat exposes both non-streaming sendMessage(...) and streaming sendMessageStream(...); both continue the same history, although a stream is appended only after it has been fully consumed. A non-streaming response exposes the calls directly through functionCalls() (verified against the javadoc in August 2026; it returns null rather than an Optional when there are no parts). This design would have kept the stream count at one regardless of tool rounds, kept loop state on the server, and left only genuinely browser-only tools such as routing and clipboard access as exceptions. I started from the assumption that "model responses are streamed" and never saw the branch where they did not have to be.

Consistency traps across declaration, implementation, and exposure#

One tool was split across several places. Its declaration lived in JSON, its implementation in a Java class, its exposure to a particular prompt in a database row, and its labels and progress copy in seven frontend and resource locations. Those responsibilities are different, so splitting the files is normal. The trap came next: the only links among the pieces were a string function name and one database row, and nothing reported a mismatch. Adding one tool touched these eleven places.

The seven frontend locations are grouped because itemizing them is not the point of this chapter. What matters more than the count is where each one lives: the declaration (1) is a JSON resource and the exposure decision (4) is a database row, so searching the source alone cannot detect either mismatch.

Check for signals, not severity#

Miss any one of the eleven locations and the build still passes. If every omission survives the build, ranking them by severity does not tell you what to inspect first. Severity is assigned after discovery. The operational question is: when do you learn that something is missing? I regrouped the locations by the signal each omission leaves behind.

If your system has the same shape, these four cards are also the inspection order. A stack trace makes the exception easy to search for, and users report a broken label. The silent card is the expensive one. With the exposure mapping absent, the code builds, startup logs stay clean, and only the model cannot see the tool. The cause is not in the source, so "the model refuses to call my tool" looks like a prompt problem.

The database mapping itself was not the mistake. The second generation, built on ADK, still keeps per-agent tool mappings in the database. The key changed from prompt to agent, but the design survived. The problem was not that the row existed. It was that its absence left no signal. Declaration and implementation were cross-checked at startup, while no equivalent check covered the exposure row. This is the first answer to the question chapter 1 left open: where did mapping tools to prompts stop holding? It started with that silence. The larger answer comes later — the model had nowhere to represent delegation.

Three layers the compiler cannot catch, and a startup cross-check#

1. The browser-side tool name is a plain string, not a literal union, and the display-name map is Record<string, string>. TypeScript has no way to tell whether a new name was registered everywhere, even with strict mode enabled.
2. The backend declaration is a JSON resource and the implementation is a Java class. A string function name connects two entirely different compilation units.
3. The final decision to expose a tool to the model is not code at all. It is a database row.

The second layer caused a real misdiagnosis. The model saw a declaration and called it, the server could not find the matching implementation and reported an unknown function, and the browser showed only that the tool had failed. A startup-time cross-check can demote this failure. Collect the implementations from the Spring context into a name-keyed map, read the declarations separately, and compare both sets. Exclude declarations without implementations from the model-visible list and log a warning; log a warning for implementations without declarations as well. Stable warning messages make the mismatch searchable at startup. I also added a debugging rule: if a name appears in tool-call history but not in the registration list, suspect implementation registration before model hallucination.

This is detection, not prevention. The declaration and implementation remain separate files, so the mismatch can still happen; its consequence merely drops from runtime failure to startup warning. The scope is narrow too. The cross-check cleans only the model-visible declaration list. It does not change the internal map used to execute tools by name. An implementation with no declaration can therefore still be reached through another path. Nothing was blocked; it was only hidden from the model.

What to hand off to a framework#

The first features were one-shot calls. Translate a request or summarize a long document, send once and receive once. The layer from chapter 1 was built around that shape. Function calling changed the premise because the model's answer was no longer limited to text already present in the prompt. It could fetch business data at the moment of the question. The goal shifted toward a general-purpose assistant that could freely query and use the system's knowledge, so I built chat on top of the prompt, parameter, and tool data the first generation already held.

Getting from there to a chat agent still meant building automatic round trips, conversation compaction, delegation to subagents, and MCP integration for external tools. Delegation was blocked before implementation could begin. A data model that attaches a tool list to one prompt has nowhere to say "delegate this to another agent." That was the limit chapter 1 left behind. The question therefore changed from how to build every missing feature to what to keep and what to hand off to a framework. I used one criterion: is this the tool's job, or code that merely keeps the tool running? The tool's job remained reading data and returning a structured result. That is not something a framework needs to replace, or ought to. Every growing cost sat outside the tool.

• Who owns the loop? If the browser drives it, the loop state lives there too.
• Where do declarations live? A JSON file in the source tree cannot change without a deploy and compiles separately from the implementation.
• What happens when registration fails? A warning allows startup to succeed while silently dropping one tool; an exception stops startup.
• How does session context propagate? One tool called the model again through a different prompt. Session values did not follow automatically, so they were copied into three maps and recovered through a three-step fallback. That was convention, not a rule, and it broke the way conventions break: one map used a different key, and the consumer worked only because its fallback knew the difference.

All four belong to the execution environment, not to tool logic. The eleven change points told the same story. Driving every round trip myself meant a person had to preserve every link, and the cost bottomed out in the one location that left no signal. I therefore evaluated frameworks by promises rather than feature lists: a registration mismatch had to fail instead of warn, and session values had to live in one execution state rather than three maps.

The stack was Spring and the language was Java. I reviewed the options available under those constraints, including LangChain. The final direction was Google ADK Java with AG-UI — keep to the standards as far as possible, then open only the points that required custom behavior.

The deciding factor was the round trip. In the first generation, after the initial request, the application parsed a function call, executed it, assembled a function response, and sent a second request. If the model needed another tool, the application wrote that sequence again. With a framework, application code starts the interaction once, and the framework owns every subsequent tool round. It keeps calling tools until it reaches the final answer. If something must happen in the middle — recording each event, for example — inheritance opens that point for customization. The round trip moves from my code into the framework's default behavior. In April 2026, I chose to rebuild from scratch rather than layer it onto the first generation. The build itself continues in the following chapters. The migration was not an atomic cutover: first-generation tools kept growing after the second generation began, and the original SDK was not removed.

The question left open#

Even with the loop handed off to the server, opening a screen still happens in the browser. So who actually opens the screen the model picked? Connecting the model's tool call to an actual screen navigation is the application's job. Chapter 3 follows one request — "open the request list screen" — across that boundary.

References#

• Google GenAI Java SDK — com.google.genai:google-genai (opens in a new tab)
• Google ADK Java — com.google.adk:google-adk (opens in a new tab) (chosen in this chapter; the build begins in later chapters)
• AG-UI — open protocol (opens in a new tab)
• Gemini API function calling — official docs (opens in a new tab). When checked in August 2026, the examples covered Python, JavaScript, and REST; by September 2026, the default page had added Java through the newer Interactions API. That is not the legacy Chat loop documented here.
• Legacy generateContent function calling — legacy docs (opens in a new tab). This is the API the Chat loop in this chapter relies on; when checked in September 2026 its examples still covered only Python, JavaScript, and REST.