Google Cloud Solution RAG Enterprise Search Gke Sqldb

RAG for enterprise search using GKE and AlloyDB

This skill provides a workflow to design and implement a secure, low-latency,

and high-accuracy RAG-enabled conversational search solution for private

enterprise content by using an AlloyDB database, Cloud Storage, and a Google

Kubernetes Engine (GKE) cluster to host all the application components,

including an open model and an open-source inference framework.

Overview of the workflow

The workflow consists of the following phases:

• Phase 1: Requirements discovery. Gather detailed requirements related to

the cloud workload or use case that the user needs assistance for.

• Phase 2: Solution architecture. Use the requirements that were gathered

in Phase 1 to generate a detailed solution architecture for the cloud

workload or use case.

• Phase 3: Solution validation. Create a plan to validate the generated

solution, generate validation instructions and scripts, and run the

validation.

• Phase 4: Solution packing and presentation. Consolidate the generated

content and present the solution.

Important notes about the workflow:

• Strict phase separation: During Phase 1 (Requirements discovery), when

you ask the user clarifying questions, DON'T recommend, propose, or outline

any architectural designs, technical decompositions, cloud services, or

component mappings.

• When you can skip certain phases: If the user's prompt indicates that a

specific phase or task in this workflow is already completed or approved

(e.g., "requirements discovery stage is completed", "product selection is

approved", or "architecture is confirmed"), DON'T repeat that phase or task.

Instead, skip directly to the requested task (such as generating the

technical decomposition, recommending products, or compiling the solution

guide).

Phase 1: Requirements discovery

In this phase, you must gather detailed requirements related to the RAG workload

that the user wants to design and deploy in Google Cloud.

Complete the following steps strictly in the specified order:

• Ask the user to describe the functional requirements of the workload,

including data types (structured, unstructured), ingestion frequency, and

conversational features (e.g., multi-turn chat, citation requirements).

• Ask the user to describe the following non-functional requirements:
• Security, privacy, and compliance: E.g., network isolation, private

endpoints, data residency, and requirements for compliance.

• Reliability: E.g., scaling, high availability, resilience against

zone or regional outages, disaster recovery goals for RTO and RPO.

• Cost: E.g., cost of compute, storage, and database resources.
• Operational excellence: E.g., monitoring, alerts, and logging.
• Performance: E.g., data upload speed, performance expectations for

generating embedding vectors, and latency requirements for model

responses and data retrieval queries (including vector and hybrid

search).

• Sustainability: E.g., carbon footprint, low-carbon regions.
• Ask the user whether the workload currently runs on other cloud providers or

on-premises.

• If the user's answer is "yes", then ask the user to describe the

architecture of the current deployment.

• If the user's answer is "no", then proceed to the next step.
• Ask the user to describe dependencies, if any, on other workloads, products,

or tools (e.g., identity providers, external sources, CRM/ERP database

integrations).

• Review the input that the user has provided so far, and check whether there

are any ambiguities or contradictions.

If you identify any ambiguities or contradictions in the requirements that

the user has provided, then

do the following for each ambiguity or contradiction that you identify:

• Describe the ambiguity or contradiction.
• Ask the user how they wish to resolve the ambiguity or contradiction.
• If the user delegates the choice to you (e.g., the user replies with

"do what you think is best" or "you decide"), then provide a clear

suggestion to resolve the ambiguity or contradiction, explain your

reasoning, and ask the user to approve your suggestion.

Critical: Until all the ambiguities and contradictions that you identify

are resolved according to the preceding guidance, you must NOT recommend or

generate any architecture design, technical decomposition, or Google Cloud

product recommendations.

• Important: DON'T start this step if there are unresolved contradictions

or ambiguities from Step 5.

Generate a technical decomposition of the components of the workload. The

technical decomposition must break down the solution into logical

components, as follows:

• Data ingestion: Blob storage for raw corporate documents.
• Data processing and chunking: Containerized pipeline to extract

data, clean it, and chunk it.

• Embedding vectors generation: Containerized service to convert data

chunks to embedding vectors.

• Storing and indexing the embedding vectors: Vector-enabled SQL

database for storing embedding vectors.

• Handling non-vector data: Preparing non-vector data, like tables,

views, and aggregations for data retrieval. Analyzing whether any

indexing, partitioning, or other performance techniques can be applied

on the original data schema.

• Query and retrieval: Accepting client queries, identifying intent,

and routing to a retrieval workflow, which might include conversion of

the request to an embedding for semantic search, extracting and applying

filters for filtered search or supplying all to the hybrid search.

• Prompt augmentation: Augmenting the prompts with the retrieved

context.

• Response generation: Requesting and generating responses from the

model.

• Response sanity checks: Evaluating responses using an AI model and

performing procedural checks according to defined criteria.

• Ask the user to approve the generated technical decomposition.

Critical: You MUST stop execution immediately, call no more tools (such

as file editors, searches, or code tools), and wait for the user to respond

with their feedback or approval in the chat. Do NOT compile the

architecture, recommend products, construct maps, or write any files/drafts

for Phase 2 until the user's explicit approval is received.

• If the user requests changes, then generate an updated technical

decomposition.

• Repeat steps 5 through 8 until the user approves the generated technical

decomposition.

10. Only after the user has explicitly approved the technical decomposition,

proceed to Phase 2.

Important: You are strictly prohibited from recommending product

choices, generating the architecture diagram, or drafting design

recommendations until the technical decomposition is approved.

Phase 2: Solution architecture

Ground all generated content

For each task in this phase, to ensure that the generated content aligns with

the latest and official Google Cloud guidance, you must ground the generated

content by using the following resources:

• Google Developer Knowledge MCP server:

https://developers.google.com/knowledge/mcp.md.txt

• Server: https://developerknowledge.googleapis.com/mcp
• Tools:
• developerknowledge:search_documents
• developerknowledge:get_documents
• developerknowledge:answer_query
• Relevant skills from https://github.com/google/skills
• Official Google Cloud documentation, including the following:
• Primary architecture reference:

https://docs.cloud.google.com/architecture/rag-capable-gen-ai-app-using-gke.md.txt

• Architecture guidance and decision-making guides:
• references/product-selection-recommendations.md
• references/design-recommendations.md
• references/related-documentation.md

For each item in the generated guidance, you must include citations to the

relevant official Google Cloud documentation pages.

Task 2.1: Identify Google Cloud products and features required for the workload.

• Recommend the products and features that are appropriate for each component

of the user's workload.

Important: The Google Cloud products and features that you recommend

MUST be consistent with the guidance in

references/product-selection-recommendations.md.

• Present the generated product recommendations and ask the user to approve

the recommendations.

• If the user requests changes, then make the required changes.
• Repeat steps 2 and 3 until the user approves the product recommendations.
• After the user approves the product recommendations, proceed to Task 2.2.

Task 2.2: Generate an architecture diagram.

• Generate an architecture diagram in the Mermaid format:

https://github.com/mermaid-js/mermaid.

The diagram must show the data flows and request flows across the components

of the architecture, based on the technical composition that you generated.

The following is an example of the data flows and request flows that the

architecture diagram should show:

• Embedding pipeline (batch/streaming): Data source -> Cloud Storage

-> Cloud Storage FUSE -> GKE Ray Worker (Chunking) --> Embedding

generation using GemmaEmbedding -> AlloyDB.

• Serving pipeline (real-time): User client -> GKE Frontend (LangChain

Orchestration) -> Database Query (semantic or hybrid search on the

vector store) -> Retrieve matching data -> Augment prompt -> Gemma vLLM

endpoint API -> Output (Responsible AI filtering) -> User client.

• Present the generated diagram to the user and ask the user to approve the

architecture diagram.

• If the user requests changes, then make the required changes.
• Repeat steps 2 and 3 until the user approves the architecture diagram.
• After the user approves the architecture diagram, proceed to Task 2.3.

Task 2.3: Generate an architecture description.

• Generate a description that explains the purpose of each component, the

relationships between the components, and the task flow or data flow.

• Present the generated architecture description to the user and ask the user

to approve the description.

• If the user requests any changes, then make the required changes.
• Repeat steps 2 and 3 until the user approves the architecture description.
• After the user approves the architecture description, proceed to Task 2.4.

Task 2.4: Generate design recommendations.

• Generate design recommendations and best practices to optimally configure

each component in the architecture based on the workload's requirements.

Important: The design recommendations and best practices that you

generate MUST be consistent with the guidance in the resources that are

listed in the following files:

• references/related-documentation.md
• references/design-recommendations.md
• Present the generated recommendations to the user and ask whether the user

needs any changes.

• If the user needs changes, then make the required changes.
• Repeat steps 2 and 3 until the user confirms that the generated design

recommendations meet their requirements.

• Proceed to Task 2.5.

Task 2.5: Generate deployment guidance.

• Generate guidance to deploy the solution, including the following:
• Terraform code to create the required infrastructure resources.
• Steps or scripts to deploy workloads, such as Ray-on-GKE (KubeRay

coordinator and worker nodes) and the LangChain frontend deployment.

Important: The deployment guidance that you generate MUST be consistent

with the guidance in the resources that are listed in the following

resources:

• references/related-documentation.md
• references/design-recommendations.md
• Relevant skills in

https://github.com/google/skills/tree/main/skills/cloud

• Present the generated deployment guidance to the user and ask whether the

user needs any changes.

• If the user requests changes, then make the required changes.
• Repeat steps 2 and 3 until the user confirms that the generated deployment

guidance meets their requirements.

• Proceed to Phase 3.

Phase 3: Solution validation

• Create a plan to validate the generated solution. The plan must outline the

steps that are necessary to verify that the generated solution meets the

workload's requirements. The following are examples of validation steps:

• Deployment dry-run: Run commands like terraform plan to preview

the infrastructure resources that will be provisioned.

• Connectivity and routing: Verify network paths, load balancer

routing, and service endpoints.

• Vector index: Verify that vector indexes are correctly created and

populated in the AlloyDB database. This can involve querying the

database to check index status and content.

• Embedding pipeline: Test the end-to-end embedding pipeline from data

ingestion to vector storage, ensuring documents are chunked, embedded,

and stored correctly.

• Retrieval latency: Measure the latency of vector and hybrid search

queries against the AlloyDB vector store to ensure performance meets

requirements.

• Retrieval accuracy: Perform sample queries and evaluate the

relevance of retrieved documents or chunks.

• Security policies: Verify restricted access, firewall rules, and IAM

enforcement.

• Present the validation plan to the user and request feedback or approval.
• If the user requests changes, update the plan as required.
• Repeat steps 2 and 3 until the user approves the validation plan.
• Generate scripts or commands using tools like curl or gcloud to perform

the steps in the approved validation plan.

• Request permission from the user to perform the validation checks.
• If the user gives permission, run the validation checks and troubleshoot any

deployment issues.

• When all the validation checks pass, proceed to Phase 4.

Phase 4: Solution packaging and presentation

• Consolidate the final text artifacts that were generated in Phase 2 into a

single Markdown file named solution-architecture-guide.md, based on the

template in assets/output-template.md.

• Request the user's permission to write the code files in the user's

workspace.

• After the user gives permission, write the final code files in the user's

workspace.

Supporting references

• references/product-selection-recommendations.md
• references/design-recommendations.md
• references/related-documentation.md