How one translation request became an internal LLM platform

The situation#

In early 2025, as a line-of-business developer on an enterprise SaaS product, I got a requirement that reached me in conversation before it reached a document. It was one button — and I already knew that in enterprise systems a request like this never stays one.

Put an English-translation button on the request screen. Press it, the request text gets translated, the English field fills in. Use Google GenAI.

The formal ticket did not exist yet. Nothing about it is technically hard. Add the GenAI SDK, build a system-instruction string, append the user's text, call the model, return the result.

// Illustrative — not the real code
String instruction = "Translate the following text into English. Preserve the original formatting.";
var response = genaiClient.models.generateContent(model, instruction + userText, config);
return response.text();

I stopped here.

The problem — when the second feature arrives#

What is wrong with this code is not the code. It is that this code becomes the precedent.

AI features do not arrive alone. Translation is followed by summarization, then classification, then query expansion. Build each one this way and you get:

• System instructions scattered across service classes as string constants.
• Changing one line of an instruction means edit → build → deploy.
• No single place shows which feature calls which model with which settings.
• When a prompt changes, nobody knows what the previous version said.
• When a call fails or the output is poor, you cannot reconstruct what was sent.

In enterprise systems this is a familiar failure shape. It is exactly what happens when hard-coded business rules spread through a codebase — except that what is spreading here is natural-language instruction rather than SQL or a conditional.

And natural-language instructions change far more often than code. They change when the model changes, when the output is not quite right, and when someone on the business side asks you to reword one sentence. You cannot ship a deploy every time.

The attempt — the foundation instead of the feature#

Before building the translation feature, I decided to build the layer it would sit on. I started in March 2025, before the formal requirement existed.

Here is the structure I was aiming at. Instead of instructions and model names baked into each feature, data registered from a screen drives one shared call path.

What I built was a prompt administration screen. A CMS pulls copy out of the code so it can be handled on a screen; this pulls instructions out of the code so they can be handled on a screen. Here is what the first commit and the rounds of extension that followed added — not all of it carried the same weight:

The last two are the point. Without observability you cannot operate an LLM feature. If you cannot reconstruct what was sent when something failed, you cannot fix it either.

The tables underneath look like this (shared classification and audit columns omitted). Versions accumulate as new rows instead of in a separate history table, and every parameter and tool mapping hangs off the version key, so editing a prompt leaves the configuration that past calls referenced untouched. This is where it parts from the usual approach of keeping a separate history table: with current rows and archived rows in different tables, every mapping has to decide which of the two it points at, whereas putting the version in the primary key leaves past and present the same shape and gives a mapping only one place to point. Each call record is stamped with the version that served it.

Columns go in a table. Bold marks the primary key.

The primary keys on the two mapping tables are where the design shows itself. Pointing at a prompt takes more than its identifier — the version number has to be there too before a single row is determined. That is why new versions can pile up without disturbing the parameters and tools that earlier calls were built on.

The organizational obstacle#

Building this needed tables. But I am a line-of-business developer.

Development organizations at this size divide the work. Designing database tables belongs to the data architects, and a developer creating tables on their own initiative is, as a rule, not something you do. I agree with the rule. Schemas that grow without control become nobody's to maintain.

What the organization actually asked for was something else. Mine was the first genai call in the system, and management asked me, since I was building it first, to make it easy for everyone else to call later. Meaning: factor out a good shared utility on the backend.

The ask was not wrong. Its scope was different. A shared utility addresses what the next developer hits when wiring up a call, and a utility genuinely does solve that. What I was looking at was what the business user hits when they want one sentence reworded — and no amount of shared utility removes that without a database. A structure where changing one line of instruction requires a build and a deploy is not fixed by a helper function. What separated the two asks was not technical: it was whose convenience the design was for. So the plan was set from the beginning: build the tables first, carry it through to the administration features, and show — as a working thing — a structure where AI features can be added without any of this.

I designed and built the LLM tables myself and got the screens running. Then I showed them.

The architecture group responded to the structure. A session was arranged for me to walk people through it.

What I took from this is not technical. The fastest way past "this is how we do it here" was not persuasion — it was the working thing. One screen that ran beat ten meetings.

I would not generalize the method, though. It is a card you can only play inside the range you can undo. A few tables can be dropped; once operational data has accumulated, the same move is not available to you.

The day the formal requirement arrived#

In April 2025, the request I had heard in conversation was filed as a formal requirement.

Two things happened. I registered the translation prompt on the administration screen, and I put an English-translation button on the request screen that called the shared endpoint with that prompt ID. The translation logic written that day was zero lines. Every later revision to the wording happened on the screen, and none of them involved a deploy.

From that day, a translation call flows like this.

The request did not produce the platform; the platform had been waiting for a request. That keeping prompts as data rather than as code was the right direction is something this layer proved on its own, the first day a formal requirement showed up.

Why Java#

This is the same arithmetic as building the layer first: not the convenience of one feature, but what else grows when one more language is added.

The system stack was Spring Boot on the back and React on the front. Building only the AI part in Python asks for this:

• One more language → the pool of people who can maintain it splits
• One more deployment target → its own pipeline, monitoring, certificates, firewall policy
• One more network hop → more failure points and more latency
• Authentication, sessions, and transactions implemented twice

That is a lot to pay for one translation button. So I went with the Java library from the start.

After the move to Google ADK Java, the server still ran on Spring Boot and the frontend on React. Chapter 3 follows a chat request from a server-registered tool call to a browser action.

What came of it, and what did not#

This prompt management layer became the first generation. It absorbed the AI features and tools that followed as screen registrations, and in August 2025 a chatbot was built on top of it.

While it was in operation the benefit was plain. Editing an instruction was separated from deploying. Wording requests from the business side ended on a screen. The calling code stayed unchanged across several model generations — keeping the model name in common-code data instead of in the source was the highest-return decision in the whole layer.

Building the chatbot is where the limits started to show. A model that maps tools per prompt has no concept of an agent or of delegation, so the application had to drive the tool-calling loop itself. Adding a single tool meant editing several places at once.

What that actually felt like, and how it ended in a from-scratch rebuild on Google ADK Java in April 2026, continues in chapter 2.

References#

• Google GenAI Java SDK — com.google.genai:google-genai (opens in a new tab)
• Google ADK Java — com.google.adk:google-adk (opens in a new tab) (from chapter 2 on)
• Period this article covers: early 2025 to April 2025 (starting the first generation)