Agentic AI on Google Cloud, Part 1: Which Service for Which Job, and How to Connect Them
Google Cloud has hundreds of products. An agent needs about a dozen. This post explains what each one is for, why we pick it, how identities and permissions connect them, and then deploys a first working agent to Cloud Run.
The series. Part 1 (this post): the GCP map, identity wiring, first deployed agent. Part 2: RAG on GCP, including the event-driven ingestion pipeline. Part 3: verification and citations. Part 4: testing, CI/CD and monitoring.
1. The scenario and the shape of the system
We are building an assistant for a health insurer. A signed-in customer asks, "Why was my claim CLM-1042 rejected?" The assistant must look up the real claim, find the right policy clause, and answer with evidence. The language model is only one part of that. The rest is ordinary cloud architecture: something that receives requests, something that stores documents, something that searches them, something that runs background jobs, and something that keeps secrets and watches everything.
Before we meet the cloud services, here is the app in the customer's own terms. This is the figure to keep in mind through the whole series: who asks, what the assistant does with the question, which systems it consults, and where the knowledge comes from.
Read it left to right. The user asks about a claim. Inside the assistant, five things happen. It identifies the user from the sign-in token, so it knows whose claims it may see. It finds the claim by asking the company's claims system for the real status, instead of guessing. It searches the policy in the knowledge base to find the clause that explains the rule. It explains, with Gemini turning the claim record and the policy passages into plain language. And it checks that every statement is backed by evidence and adds sources (a layer we build in Part 3). The answer, with its sources, returns to the user.
Notice the two lanes. The top lane runs every time a customer asks and must be fast. The bottom lane runs before any question: policy documents are extracted, cleaned, chunked and turned into vectors, then stored in the knowledge base that step 3 searches. Keeping those lanes separate is the most important structural idea in the project, and it is why Part 2 is about preparing knowledge while Part 1 is about the assistant itself.
2. Which GCP service for which job
The easiest way to learn GCP is to stop thinking about products and start thinking about questions. Imagine you are the engineer on day one, and ask them in the order they come up.
Where does my code run? Our assistant is a Python web application that must answer HTTPS requests, sometimes a handful a minute and sometimes hundreds. We do not want to manage servers, so we package the app as a container and hand it to Cloud Run, which gives us a URL, scales the container up and down, and checks permissions on every call.
Who does the thinking? The language model. On GCP that is Gemini, served by Vertex AI. Our code calls it over an API, and because it is called with a service account rather than an API key, there is no secret to leak. The agent framework (ADK) is not a service at all. It is a Python library that lives inside our container and runs the loop of "ask the model, run the tool it requests, ask again".
Where do the policy documents live? Someone publishes a PDF. It has to land somewhere durable and cheap, and that place should be able to tell us when a new file arrives. That is Cloud Storage. Next, we must read the PDF. If it is a clean digital file we can parse it ourselves, but scans and tables are hard, so Document AI does the extraction. The extracted text is then cut into chunks by our own code, and each chunk is turned into a list of numbers that captures its meaning by Vertex AI embeddings.
Where do those chunks go so we can search them? They are stored in BigQuery, which can search by meaning and also filter with ordinary SQL, for example "only the policy version in force on the expense date". That filter is the reason we choose it for an insurer.
How does new knowledge arrive without blocking a customer? We do not want a person waiting while a 200-page document is processed. So the arrival of a file produces a message on Pub/Sub, which delivers it to a Cloud Run Service that does the work in the background. When we need to process the whole archive, or rebuild every night, we run the same code as a Cloud Run Job started by Cloud Scheduler. Firestore keeps a small record of what was already processed, so duplicates do nothing.
Who is allowed to do what, and where are the passwords? Each service runs under its own IAM service account with only the roles it needs, and credentials for upstream systems sit in Secret Manager. Finally, we need to see what happens (Cloud Logging, Monitoring and Trace) and to rebuild environments reliably (Artifact Registry, Cloud Build and Terraform).
That is the whole cast. Notice that most of them each do exactly one job, and that the model is only one of about a dozen pieces. The table condenses the story above into a cheat sheet for the rest of the series; the "why" column is the part to remember.
| Job | Service | Why this one |
|---|---|---|
| Run the API and the agent | Cloud Run | Container in, HTTPS endpoint out; scales to zero or to load; built-in IAM-based auth; same as your other backends |
| The language model | Vertex AI (Gemini) | Called with your service account, data stays in your project's governance, no API keys to leak |
| Agent framework | Google ADK (a Python library, not a service) | Tool calling, sessions, dev UI; runs inside your Cloud Run container |
| Store source documents | Cloud Storage | Cheap, durable, emits events when files change |
| Read PDFs and scans | Document AI | OCR, layout and tables when plain text extraction is not enough |
| Turn text into vectors | Vertex AI embeddings | Managed embedding models, same auth as Gemini |
| Store and search chunks | BigQuery (custom) or Vertex AI RAG Engine / Search (managed) | BigQuery gives SQL filters on version and date; managed options trade control for speed |
| React to new documents | Eventarc + Pub/Sub | Decouples publishing from indexing; retries and dead-letter queues |
| Process documents as they arrive | Cloud Run Service behind a Pub/Sub push subscription | Stays ready, scales with the event rate, retries and dead-letters failures |
| Bulk or nightly rebuilds | Cloud Run Job + Cloud Scheduler | Runs a script to completion, split into parallel tasks; no HTTP request has to wait |
| Remember what was processed | Firestore | Tiny, serverless record per document, used for idempotency |
| Long approval workflows | Cloud Workflows | Durable steps that can wait days for a human (later) |
| Hold credentials | Secret Manager | Versioned, access-controlled, audited |
| Store container images | Artifact Registry | Cloud Run pulls from here |
| Identity between services | IAM service accounts | Each service gets its own minimal identity |
| Watch it run | Cloud Logging, Monitoring, Trace | Native to Cloud Run; OpenTelemetry plugs in |
| Repeatable infrastructure | Terraform, Cloud Build | Environments you can rebuild and review |
Decisions you will face, and how to choose
Knowing the cast is only half of it. GCP often offers two or three products for the same job, and the documentation will happily describe all of them without telling you which one fits. In this project you will hit four forks in the road. None of them has a universally right answer, so the useful skill is knowing what to ask at each one.
Where should the agent run? You can run it in your own container on Cloud Run, which is the same way your other backends are deployed, or hand it to Vertex AI Agent Engine, a managed runtime built for agents. The question to ask is how much control you need. If you want your own endpoints, authentication and logging next to your existing services, take Cloud Run. If your framework is supported and you would rather not operate the runtime, Agent Engine is worth a trial.
How should retrieval work? Google offers managed search products that index documents for you, and you can also build the index yourself in BigQuery. Ask whether your retrieval has special rules. A customer insured in March must be answered from the policy version in force in March, and that kind of rule is easy to write in SQL and hard to bend a managed product around. Start managed if your needs are ordinary and move to custom when your measurements show you need control.
Where does structured data live? Analytical data, large and read in bulk, belongs in BigQuery. Small, frequently updated application state, such as chat sessions or "was this file processed", belongs in a transactional store like Firestore or Cloud SQL. A useful test: if you would run a report over it, BigQuery; if the app reads and writes it on every request, a transactional database.
How should background work be coordinated? Pub/Sub is for events you fire and forget, such as "a file arrived", where many consumers may react. Workflows is for ordered steps that may wait, such as a claim submission that pauses for days until a human approves. If there is a sequence and a waiting period, you want Workflows; if there is just a signal, Pub/Sub.
The table gives the same four decisions as a quick reference.
| Question | Choose A when | Choose B when |
|---|---|---|
| Where does the agent run? | Cloud Run: you want full control, one platform with your other services | Vertex AI Agent Engine: you want a managed agent runtime and your framework fits it |
| How do we do retrieval? | Managed (RAG Engine or Vertex AI Search): fast prototype, standard needs | Custom on BigQuery: strict version/date filters, custom ranking, auditability |
| Where does structured data live? | BigQuery: analytics, large reads | Cloud SQL / Firestore: transactional app state, sessions |
| Pub/Sub or Workflows? | Pub/Sub: fire-and-forget events, fan-out | Workflows: ordered steps with waits and approvals |
Where does your code actually run? Cloud Run Service or Job
This is the most common confusion for newcomers. There is no "chunking service" or "ingestion service" in Google Cloud. Chunking, parsing and indexing are ordinary Python code running inside your Cloud Run container. When a step needs a model (for example semantic chunking), your code calls Vertex AI over an API. Cloud Run executes; Vertex AI supplies intelligence. What you choose is how the container is run:
| Workload | Deployment | Example |
|---|---|---|
| Chunk a document uploaded through an API | Cloud Run Service | An endpoint receives a document and starts processing |
| Index thousands of existing documents | Cloud Run Job | Initial batch ingestion |
| Process documents as they arrive in Cloud Storage | Service triggered by an event | A new PDF triggers ingestion |
| Reindex every night | Job + Cloud Scheduler | Rebuild embeddings after updates |
| Continuously consume messages | Service or Worker Pool | Background pipeline on Pub/Sub |
A Service listens for HTTP requests and stays ready. A Job runs to completion and exits, optionally as parallel tasks. Rule of thumb: Services for request-driven or event-driven work, Jobs for large or scheduled batches. Never make an API request wait while hundreds of documents are chunked and embedded; enqueue the work and process it asynchronously. Part 2 builds both.
europe-west1 Belgium or europe-west9 Paris) and keep everything there for latency and data-residency reasons. Model availability differs by region, so check your chosen Gemini model is offered in yours before you commit.3. How the pieces connect: identity first
Two terms carry most of this section, and they are worth defining properly before we use them: IAM and service account.
What is IAM?
IAM stands for Identity and Access Management. It is the system inside Google Cloud that answers one question every time anything happens: who is asking, what do they want to do, and are they allowed to? Think of an office building where every door checks your badge. IAM is the badge system. By default every door is locked, and nothing is allowed until someone grants it.
IAM has a small vocabulary:
| Term | Meaning | Example |
|---|---|---|
| Principal | The who: a person, a group or a piece of software | api-sa@my-project.iam.gserviceaccount.com |
| Permission | One specific action | bigquery.tables.getData |
| Role | A named bundle of permissions | roles/bigquery.dataViewer |
| Resource | The what: a project, a bucket, a secret, a Cloud Run service | the project insurance-agent-dev |
| Binding | "Give this principal this role on this resource" | the add-iam-policy-binding commands in Step 2 |
You never grant single permissions one by one. You grant a role, which is a bundle. There are three kinds. Basic roles (Owner, Editor, Viewer) are very broad and should be avoided for anything real. Predefined roles are curated by Google for one product, such as roles/aiplatform.user, and are what we use. Custom roles are bundles you design yourself when no predefined role fits. The guiding rule is least privilege: give each identity the narrowest role that lets it do its job, and nothing more.
What is a service account?
A service account is an identity for software rather than for a person. It looks like an email address (api-sa@PROJECT.iam.gserviceaccount.com), but nobody types a password for it and nobody logs into a browser with it. It exists so that a program can say "I am the agent API" and have IAM decide what that program may touch.
You attach a service account to a workload. When we deploy to Cloud Run with --service-account, we are saying "run this container as api-sa". Google's client libraries then fetch short-lived credentials from the environment automatically, which is why our code never contains a key or a password. Three habits keep this safe: create one service account per workload (so the customer-facing API and the ingestion worker never share power), grant it only the roles it needs, and never download a key file for it unless you have no alternative.
Putting the two ideas together, here is how to read the command from Step 2:
gcloud projects add-iam-policy-binding $PROJECT \
--member=serviceAccount:$API \ # the principal: the api-sa robot identity
--role=roles/aiplatform.user # the role: allowed to call Vertex AI models
# (on the resource: this whole project)In plain English: "in this project, let the agent API call Vertex AI." Nothing more, and nothing for anyone else.
roles/iam.serviceAccountUser role on that account). If a deploy fails with a permission error naming the service account, this is usually why.New cloud developers often think about connections as URLs and keys. On GCP the real connection is identity plus permission. Every Cloud Run service runs as a service account. When it calls Vertex AI or BigQuery, Google checks whether that service account holds a role that allows the call. No password is stored anywhere in your code.
There are also two kinds of caller to keep apart. Users authenticate with a token (a JWT) from your identity provider, and your API verifies it. Services authenticate with service-account identities. Cloud Run enforces the second automatically (roles/run.invoker); the first you implement in the API.
4. Hands-on: set up the foundation
Until now we have talked about the map. In this section we walk it, in the order that the dependencies force on us. You cannot deploy a service before you have a project, you cannot give a service permissions before it has an identity, and you cannot test an agent before it has a tool. So the nine steps fall into three phases:
- The cloud foundation (Steps 1–3): a project with the right APIs switched on, two identities with narrow permissions, and a safe place for one credential.
- The application (Steps 4–6): a code skeleton, a first tool and agent, and a web API that exposes them.
- Run and ship (Steps 7–9): run it on your laptop, deploy it to Cloud Run, and watch it in the logs.
Each step starts by saying what we are about to do and why it comes now, and ends with a check so you know it worked before moving on. The commands assume a Unix-style shell with the gcloud CLI installed and a billing account you can attach to a project.
First, the architecture of the app we are building
Before typing a command, it helps to see the shape of the code, because the steps below are simply this picture filled in, layer by layer. The application is a stack of four layers plus a shared configuration. Each layer talks only to the one beneath it, and each has a single responsibility.
Two rules make this architecture work. First, dependencies point downward: the API layer knows about the agent, the agent knows about its tools, but a tool never knows about HTTP requests. Second, the model sees only layer 3. It can ask for a tool by name, but it cannot reach layer 4 directly, which is where permissions and validation live.
| Layer | Folder | Built in |
|---|---|---|
| Shared settings and schemas | src/common | Step 4 |
| Tools and the agent | src/agent | Step 5 |
| API layer | src/api | Step 6 |
| Clients and the knowledge base | src/clients, src/rag, src/ingestion | Part 2 |
Now follow one request through those layers. This is the workflow of the app at run time, and it is exactly what you will see in the agent's trace in Step 7. A customer asks about a claim; read the numbers from top to bottom.
Notice the rhythm. The model is called twice: once to decide which tool to use, and once to write the answer from the tool's result. Everything in between (checking the token, running the tool, calling the claims API) is ordinary code. Next, here is where each of these pieces lives on disk, and which step creates it.
src/clients, as the figure shows.Step 1. Project and APIs
The next job: create the container that everything else lives in. In GCP every resource belongs to a project, which is also the unit of billing, quotas and permissions. We start here because nothing else can exist without it. We also have to enable APIs: GCP services are switched off by default, and enabling an API is how you tell Google which products this project may use. Doing it now means later commands will not fail halfway with a confusing "API not enabled" error.
Use one project per environment. This gives separate IAM, quotas and billing, so a mistake in dev cannot touch production.
export PROJECT=insurance-agent-dev REGION=europe-west1
gcloud auth login && gcloud auth application-default login
gcloud config set project $PROJECT
gcloud services enable aiplatform.googleapis.com run.googleapis.com \
artifactregistry.googleapis.com secretmanager.googleapis.com \
cloudbuild.googleapis.com bigquery.googleapis.com storage.googleapis.comCheck: gcloud services list --enabled | grep -c googleapis shows the services. If a command fails with a billing error, link a billing account to the project.
Step 2. Create the two service accounts and grant roles
The next job: decide who is allowed to act inside the project. Our code will call Vertex AI, read secrets and query BigQuery. We could let it borrow your personal login, but then it would have all your power and its actions could not be told apart from yours. Instead we create service accounts, robot identities for the software. We make two on purpose: one for the service that talks to customers and one for the ingestion worker, because they should never share the same power. We create both now and give the first its roles; the second gets its roles in Part 2, when it first needs them.
gcloud iam service-accounts create api-sa --display-name="Agent API"
gcloud iam service-accounts create ingest-sa --display-name="Ingestion worker"
API=api-sa@$PROJECT.iam.gserviceaccount.com
for ROLE in roles/aiplatform.user roles/secretmanager.secretAccessor \
roles/bigquery.dataViewer roles/bigquery.jobUser; do
gcloud projects add-iam-policy-binding $PROJECT --member=serviceAccount:$API --role=$ROLE
doneEach role is a deliberate statement. aiplatform.user lets the service call Gemini and embeddings. secretAccessor lets it read secrets. dataViewer + jobUser let it run read-only queries. Notice it cannot write to BigQuery. We add the ingestion roles in Part 2.
Step 3. Store the claims API credential
The problem in plain words. Our assistant will call the company's claims system, and that system will not answer just anyone. Our app has to prove who it is by sending a token, which works like a password for software. The question is where to keep that token. The tempting answers are all bad: typed into the code, baked into the container image, or saved in a file in the repository. Each of those eventually leaks, because code and images get copied, shared and logged.
The idea. Google Cloud has a dedicated service for this called Secret Manager. Picture a safe-deposit box at a bank. You put the token in the box once. The box has a label (its name), and only the people or robots whose names are on the access list can open it. Our app never carries the token around; it asks the box for it when it starts.
Three words are enough to understand it:
| Word | Meaning | In our project |
|---|---|---|
| Secret | The box and its label. It holds no value by itself | claims-token |
| Version | One stored value inside the box. Changing the token adds a new version; old ones are kept | version 1, and the alias latest always means the newest |
| Access | Reading a version. Needs the role secretmanager.secretAccessor | we gave it to api-sa in Step 2 |
The commands, one piece at a time.
printf 'REPLACE_WITH_TOKEN' | gcloud secrets create claims-token --data-file=-This is one command in two halves joined by a pipe (|). The left half, printf '...', prints the token text. The right half, gcloud secrets create claims-token, creates a new secret called claims-token. The option --data-file=- means "read the value from what is being piped in" (the dash stands for standard input). The result: a secret named claims-token with one version holding that text. We pipe it in, instead of typing the token as an argument, so it does not stay in your shell history as part of a command.
REPLACE_WITH_TOKEN is a placeholder. The real token comes from whoever owns the claims API. For now the placeholder is fine, because the fake claims API we use in Step 7 does not check it. Before a real deployment, add the real value as a new version:
printf 'THE_REAL_TOKEN' | gcloud secrets versions add claims-token --data-file=-How the token travels to your code. Nothing is copied by hand. Follow it once:
You store it → Secret Manager (secret: claims-token)
You deploy (Step 8) → --set-secrets AGENT_CLAIMS_API_TOKEN=claims-token:latest
Cloud Run starts → reads the secret AS api-sa, sets an environment variable
Your settings class → settings.claims_api_token (Step 4, prefix AGENT_)
Your tool (Step 5) → sends it in the Authorization header to the claims APIThis is the identity idea from the previous section at work. Cloud Run can read the secret only because the service runs as api-sa, and api-sa holds the accessor role. Without that role, the deploy in Step 8 fails with a permission error naming the secret.
Check it worked:
gcloud secrets describe claims-token # shows the name and creation time, not the value
gcloud secrets versions list claims-token # shows version 1 as enabledBoth should succeed. To see the stored value (do this only on your own screen), run gcloud secrets versions access latest --secret=claims-token; it should print the placeholder.
api-sa read every secret in it. The tighter approach is to grant it on this one secret only: gcloud secrets add-iam-policy-binding claims-token --member=serviceAccount:$API --role=roles/secretmanager.secretAccessor. Do that once you have more than one secret. If you rotate the token later, add a new version; new Cloud Run instances pick up latest when they start.Step 4. Create the app skeleton
The next job: move from cloud setup to code. The cloud side is ready, so we create the project folders and one settings class. Why settings first? The same code will run on your laptop and on Cloud Run, and the only things that should differ are values like the model name and the claims URL. Reading them from environment variables in one place keeps everything else free of hard-coded constants, and means the move to the cloud later involves no code changes.
mkdir insurance-agent && cd insurance-agent
uv init && uv add fastapi "uvicorn[standard]" pydantic-settings httpx google-adk
mkdir -p src/{api,agent/tools,clients,common} tests# src/common/config.py
from pydantic_settings import BaseSettings
class Settings(BaseSettings):
claims_api_url: str = "http://localhost:9000"
claims_api_token: str = "" # injected from Secret Manager on Cloud Run
model_name: str # a Gemini model offered in your region
model_config = {"env_prefix": "AGENT_"}
settings = Settings()Step 5. A first tool and agent
The next job: give the agent something to do. On its own, a model can only talk. A tool is a plain Python function that the model can ask the framework to run, and an agent is a model plus an instruction plus a list of tools. We write the tool first and the agent second. Pay attention to one design choice in the tool: it takes the claim ID from the model but reads the customer identity from trusted session state, so a user cannot trick the model into looking at someone else's claim.
The tool takes only what the model should control (the claim ID). The customer identity is read from trusted session state that our API sets after verifying the user's token.
# src/agent/tools/claims.py
import httpx
from google.adk.tools import ToolContext
from src.common.config import settings
def get_claim_status(claim_id: str, tool_context: ToolContext) -> dict:
"""Look up the current status of one claim belonging to the signed-in
customer. Use for any question about what happened to a claim."""
cust = tool_context.state["authenticated_customer_id"]
try:
r = httpx.get(f"{settings.claims_api_url}/claims/{claim_id}", timeout=8,
headers={"Authorization": f"Bearer {settings.claims_api_token}"})
except httpx.TransportError:
return {"success": False, "error_code": "CLAIMS_UNAVAILABLE", "retryable": True}
if r.status_code != 200 or r.json().get("customer_id") != cust:
return {"success": True, "status": "not_found", "claim_id": claim_id}
d = r.json(); d.pop("customer_id")
return {"success": True, "source": "claims_api", **d}# src/agent/agent.py
from google.adk.agents import Agent
from src.agent.tools.claims import get_claim_status
from src.common.config import settings
root_agent = Agent(
name="insurance_assistant", model=settings.model_name,
instruction=("You are a support assistant for a health insurer. For any claim "
"question call get_claim_status; never guess. If a tool fails or "
"returns not_found, say so. Never promise a reimbursement."),
tools=[get_claim_status])Step 6. Expose it through FastAPI
The next job: make the agent reachable. At this point the agent is only a Python object; nothing outside the process can talk to it. We wrap it in a small web API, which is also the place where, in production, the user's identity is verified and placed into the session that the tool reads. This is the front door of the system, and it is why the browser never calls the model directly.
The ADK Runner executes the loop: send the conversation to Gemini, run any requested tools, feed results back, return the final text. We create a session whose state carries the verified identity.
# src/api/main.py
import uuid
from fastapi import FastAPI, Header, HTTPException
from pydantic import BaseModel, Field
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.genai import types
from src.agent.agent import root_agent
app = FastAPI(title="Insurance Agent API")
sessions = InMemorySessionService() # dev only: use a shared store when you scale
runner = Runner(agent=root_agent, app_name="insurance", session_service=sessions)
class ChatIn(BaseModel):
message: str = Field(min_length=1, max_length=8000)
def verified_customer(authorization: str | None) -> str:
# Part 4 replaces this with real JWT validation against your identity provider.
if not authorization: raise HTTPException(401)
return authorization.removeprefix("Bearer ").strip() # DEV ONLY: token == customer id
@app.get("/api/v1/health")
def health(): return {"ok": True}
@app.post("/api/v1/chat")
async def chat(body: ChatIn, authorization: str | None = Header(None)):
cust = verified_customer(authorization)
sid = str(uuid.uuid4())
await sessions.create_session(app_name="insurance", user_id=cust, session_id=sid,
state={"authenticated_customer_id": cust})
answer = ""
async for ev in runner.run_async(user_id=cust, session_id=sid,
new_message=types.Content(role="user", parts=[types.Part(text=body.message)])):
if ev.is_final_response() and ev.content:
answer = ev.content.parts[0].text
return {"answer": answer, "status": "answered"}InMemorySessionService loses state when Cloud Run starts a new instance. For multi-turn chat you will move sessions to a shared store.Step 7. Run it locally against Vertex AI
The next job: prove it works before involving Cloud Run. A cloud deployment takes minutes and hides errors behind container logs, while a local run takes seconds and shows them directly. Any problem we find now is a code problem, not a cloud-configuration problem. This step also demonstrates an important idea: locally, your own credentials (Application Default Credentials) call Vertex AI, and in the cloud the service account will take over, with no code change.
export GOOGLE_GENAI_USE_VERTEXAI=TRUE GOOGLE_CLOUD_PROJECT=$PROJECT GOOGLE_CLOUD_LOCATION=$REGION
export AGENT_MODEL_NAME=<your-gemini-model>
uvicorn tests.fake_claims:app --port 9000 & # a 10-line fake returning the claim JSON
uvicorn src.api.main:app --port 8080 &
curl -s localhost:8080/api/v1/chat -H "Authorization: Bearer cust-7" \
-H "Content-Type: application/json" -d '{"message":"Why was CLM-1042 rejected?"}'Check: the answer mentions the rejection reason from the fake API. Locally, your own credentials call Vertex AI via Application Default Credentials. In the cloud, the service account takes over with no code change.
Step 8. Containerize and deploy to Cloud Run
The next job: move it into the cloud. With the code working, we do three things: describe the app as a container image, store the image in Artifact Registry, and tell Cloud Run to run it with the identity and secret we prepared. This is where the earlier steps pay off. The service account from Step 2 becomes the runtime identity, the secret from Step 3 gets injected, and the unauthenticated-access flag keeps the service closed to the open internet.
# Dockerfile
FROM python:3.12-slim
WORKDIR /app
COPY pyproject.toml uv.lock ./
RUN pip install uv && uv sync --frozen --no-dev
COPY src ./src
CMD ["uv","run","uvicorn","src.api.main:app","--host","0.0.0.0","--port","8080"]gcloud artifacts repositories create agent --repository-format=docker --location=$REGION
IMAGE=$REGION-docker.pkg.dev/$PROJECT/agent/api:v1
gcloud builds submit --tag $IMAGE .
gcloud run deploy agent-api --image $IMAGE --region $REGION \
--service-account $API --no-allow-unauthenticated \
--set-secrets AGENT_CLAIMS_API_TOKEN=claims-token:latest \
--set-env-vars GOOGLE_GENAI_USE_VERTEXAI=TRUE,GOOGLE_CLOUD_PROJECT=$PROJECT,GOOGLE_CLOUD_LOCATION=$REGION,AGENT_MODEL_NAME=<your-gemini-model>Every flag maps to a concept above: --service-account is the identity that calls Gemini, --set-secrets pulls from Secret Manager, --no-allow-unauthenticated means callers must present a Google identity token with roles/run.invoker.
URL=$(gcloud run services describe agent-api --region $REGION --format='value(status.url)')
curl -s $URL/api/v1/health -H "Authorization: Bearer $(gcloud auth print-identity-token)"Check: {"ok": true}. Without the identity token you get 403, which proves the service is not open to the internet. Note that your real client will call through an API gateway or load balancer that handles user authentication, a topic for Part 4.
Step 9. See it in the logs
The next job: look at what we have deployed. A service that runs is not the same as a service you understand. Before adding any complexity, learn where its output goes, because that is where you will look first when something breaks in Part 2 and beyond. Cloud Run needs no setup for this.
gcloud run services logs read agent-api --region $REGION --limit 20Cloud Run sends stdout to Cloud Logging automatically. In Cloud Console open Cloud Run, then the service, then Logs and Metrics, to see request counts, latency and errors with no setup. This is your first monitoring surface.
5. What you have and what comes next
You now understand the map and have a deployed agent: a service with its own identity, secrets injected safely, Gemini called without any API key, and logs flowing. The knowledge side is still empty, so the agent can report a claim's status but cannot explain the policy behind it.
Part 2 adds that and keeps it fresh: Cloud Storage for documents, Document AI, Vertex AI embeddings and BigQuery vector search with version-aware filtering, the policy tool wired into the agent, and the automation (Pub/Sub, a Cloud Run Service for live uploads, a Cloud Run Job for bulk and nightly rebuilds). Not covered here: real JWT validation, a shared session store and Terraform, which arrive in Part 4. I have not run this code, so verify the ADK imports and model name against the versions in your project.
Comments
Post a Comment