Adding a Private Knowledge Base to My VCF PAIS Llama Agent

This follows adding my read-only MCP status tool. That tool reads current Kubernetes workload status. A knowledge base serves a different purpose: it retrieves relevant passages from documents. I can attach both capabilities to the same agent when document retrieval is ready.

1. Separate document storage from backups

I reuse the S3 storage appliance built earlier in my lab, but I keep my document library separate from DSM backups and system logs. My PAIS source points to the dedicated pais-shared-library bucket. I do not point the agent at database backup objects or a log repository.

My observed data source is named jtec shared library, uses Amazon S3 Compatible Store, and connects to https://s3.jtec.local/. No prefix is configured, so its intended scope is the bucket rather than a selected folder. In the example commands I use lab.example.com hostnames and substitute my actual names.

I use a dedicated S3 identity restricted to listing the approved bucket and reading its documents. I review the effective storage policy before indexing. A saved connector alone does not prove that every object is readable or that indexing will succeed.

2. Prepare a small, controlled document set

I start with a few documents that contain no passwords, private keys, tokens, or unrelated personal information. The PAIS interface lists PDF, DOCX, PPTX, TXT, HTML, Markdown, CSV, and XLSX as supported file types. A filename extension alone does not guarantee that the contents can be extracted; scanned PDFs may need OCR before ingestion.

For a repeatable first test I prepare a short text file such as rag-validation.txt with a distinctive, non-sensitive fact:

Title: Private AI retrieval validation
The lab retrieval validation code is CEDAR-472.
This code is synthetic test data for checking document retrieval.

I upload this sample through my S3 client to the document bucket. It is proposed test content, not a claim that I have already uploaded it. I record the object name so I can identify the expected source in retrieval results.

3. Verify the S3 data source in PAIS

In my PAIS interface I open Data Indexing and Retrieval → Data Sources. My existing source is already linked to one knowledge base, so I reuse it rather than creating a duplicate.

For a new installation, I select Add Data Source, enter a name and description, and click Next. In Data source details I select Amazon S3 Compatible Store and complete these fields:

I review the details before completing the wizard. I verify DNS, HTTPS trust, and S3 access from the PAIS environment. A successful connection from my workstation does not establish that PAIS has the same network access. I also avoid assuming that the Force Path Style checkbox used in DSM exists here: this PAIS form exposes URL, credentials, bucket, and prefix.

4. Create or reuse the knowledge base

I open Data Indexing and Retrieval → Knowledge Bases. My existing JTEC Shared Knowledge Base in my Automation-managed namespace has type Data Source and a Daily refresh policy. Its linked source is jtec shared library. I verify that relationship in the knowledge base details.

For a new setup I select Create Knowledge Base, create a Data Source knowledge base, and link the approved S3 source. I keep refresh frequency appropriate for how often my documents change. Scheduling a refresh does not replace the first successful indexing run.

5. Deploy my CPU embedding endpoint

My completion model generates answers; my embedding model converts document passages and questions into vectors for retrieval. My deployed embedding endpoint is granite-embedding-30m-offline, using IBM Granite 30M English, the Infinity engine, and CPU inference on the best-effort-medium VM class. It does not consume another passthrough GPU. My deployment record confirms a Ready endpoint and a functional response with 384 dimensions.

My PAIS instance now runs in my VCF Automation namespace pais-automation-f23bb. This is different from the earlier manually deployed wkld1-private-ai instance in this series. I target my current namespace when applying the endpoint and use its PAIS interface for indexing.

I use the following deployed configuration, replacing the example registry hostname, namespace, storage class, pull Secret, and model digest as appropriate for my environment. The digest identifies my corrected model package; another installation must use its own verified artifact.

apiVersion: pais.vmware.com/v1alpha1
kind: ModelEndpoint
metadata:
  name: granite-embedding-30m-offline
  namespace: pais-automation-f23bb
spec:
  type: Embeddings
  engine: Infinity
  replicas: 1
  routingName: granite-embedding-30m-offline
  virtualMachineClassName: best-effort-medium
  storageClassName: wkld1-cl01-optimal-datastore-default-policy-autoraid
  model:
    ociRef: harbor.lab.example.com/ai-models/jtec/granite-embedding-30m-english@sha256:be78e49e7ef7469503d2ea901751fd1669f9293b94a53775d6519fe3248f7e05
    pullSecrets:
      - name: harbor-ai-models
  inferenceServerCustomization:
    cliArgs:
      - --device
      - cpu
      - --engine
      - torch
      - --dtype
      - float32
    envVars:
      - name: HF_HUB_OFFLINE
        value: "1"
      - name: TRANSFORMERS_OFFLINE
        value: "1"

kubectl apply -f modelendpoint-embedding.yaml
kubectl get modelendpoint granite-embedding-30m-offline -n pais-automation-f23bb
kubectl describe modelendpoint granite-embedding-30m-offline -n pais-automation-f23bb

My original offline model package needed a packaging correction: nested model paths were normalized to POSIX separators, and the empty SentenceTransformers normalization-module directory was preserved with a marker file. I published a separate corrected revision and retained the original. The model weight blob digests were unchanged. This was a correction to my packaged artifact, not a model retraining step or a requirement to modify every Granite download.

The deployment uses --device cpu --engine torch --dtype float32 and sets HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1. I verify that all required model files are present before enabling offline mode. My staged model configuration has a 512-token sequence limit, so I keep document chunks below that limit and allow room for metadata.

6. Configure and run indexing

After my embedding endpoint is available, I open my knowledge base and select Index knowledge base. The dialog requires an embedding model endpoint, a Sentence text splitter, a chunk size, and a chunk overlap. I select granite-embedding-30m-offline.

My configured settings are Sentence, 256-token chunks, and 32-token overlap. These are the settings used in my completed run; they remain starting values for later retrieval tuning. I check the selected model’s token limit and allow room for metadata. The PAIS index API requires at least 100 tokens per chunk.

I select the endpoint, enter the settings, and choose Start index. I wait for completion and inspect document counts and any failures. A successful run with zero usable documents does not meet my acceptance criteria. In my completed run, Ubuntu TXT files appeared with Indexed status. The synthetic test document remains a separate proposed test; I do not claim it was part of this run.

The index defines the embedding space. Broadcom documents that its core configuration cannot be modified after creation. If I change embedding models or chunking later, I plan a new index or the supported recreation workflow rather than mixing incompatible vectors.

7. Inspect document failures before using the index

My knowledge base shows Available, while the source reports Indexed with errors. The run completed at 1:26:56 PM on October 21, 2026. In See errors, the Indexing and Data sources categories each showed zero errors; Documents showed one.

The rejected object was manuals/vmware-cloud-foundation-9-1.pdf. PAIS reported DOCUMENT_INDEXING_EXCEEDS_SIZE_LIMIT: the document exceeds 100 MB and cannot be indexed. I cannot treat this manual as searchable simply because other documents indexed successfully.

For an oversized source I prepare smaller, logically divided documents or an approved text extraction, retain source titles and links, and run indexing again. I review extraction quality and document counts after that change. The successful TXT documents make the completed index useful, but the rejected PDF remains outside its coverage.

8. Attach retrieval to my existing agent

My knowledge base currently reports no attached agents. The following is the next configuration stage, not a completed test. When the indexed coverage is suitable, I open Agent Builder → Agents → my Llama agent → Edit → Tools → Add Tools. The tool selector supports knowledge bases as well as approved MCP tools. I select the retrieval tool associated with my indexed knowledge base and save the agent.

I retain my existing workload-status MCP tool. I also retain the agent ID used by my browser chat frontend. That avoids changing the application’s API target merely to add a capability. I review my agent instructions so it uses retrieval for document questions, identifies the supporting source when available, and says when the source does not answer the question.

9. Test retrieval, grounding, and source attribution

I first ask in the PAIS Playground:

Search my knowledge base for the lab retrieval validation code.
Return the code and identify the source document.
If retrieval does not find it, say so instead of guessing.

I compare the answer with the actual retrieved passage and source metadata. The expected synthetic answer is CEDAR-472 from rag-validation.txt. Repeating the fact from conversation history is not sufficient evidence; I use a fresh conversation and verify retrieval activity.

I then ask about a fact deliberately absent from my document set. My agent should acknowledge the missing evidence. I check source names and passages rather than accepting a plausible-looking citation. I repeat both tests in my browser chat and verify how its current interface displays attribution; I do not assume it renders structured citations automatically.

10. Verify refresh behavior and access boundaries

I update the synthetic document with a new code, trigger or wait for a refresh, and test again in a fresh conversation. I record how long the update takes to become searchable. I also test removal behavior before relying on it for sensitive documents: deleting an S3 object does not by itself prove that every indexed copy, cached answer, or browser conversation has been removed.

I treat the linked document set as an access boundary. S3 read permissions and agent permissions need to match the intended audience; I do not assume that individual document permissions are automatically enforced for each chat user. I add a broader document collection only after reviewing that design.

11. Record the verified boundary

I verified a working CPU embedding endpoint, the linked S3 data source, and a completed index containing successfully processed documents. I also identified the oversized PDF that was not indexed. This establishes document ingestion and indexing; it does not yet establish agent grounding, source attribution in browser chat, or refresh and removal behavior.

My next acceptance checks are attaching the knowledge-base retrieval tool, verifying an answer against an actual retrieved passage, testing a question with no supporting document, and confirming a controlled document refresh. I keep those checks separate from the completed indexing result so I can identify which layer needs attention if an answer is wrong.

References

• Broadcom PAIS API workflow and retrieval integration
• Broadcom index creation and configuration requirements
• IBM Granite English embedding model card