Creating a Llama Agent with VCF Private AI Services

This follows my PAIS Supervisor deployment and my Llama GPU endpoint. I use generic names and pais.lab.example.com in the examples. Substitute the values from your deployment.

1. Understand what my agent adds

My model endpoint supplies inference. My agent adds a named configuration around that model: instructions, conversation history settings, and any explicitly attached tools. I can change the assistant’s behavior without uploading a different set of model weights.

The agent in this procedure starts without tools or a knowledge base. It can answer questions using its model and conversation context, but it cannot inspect vCenter, search private documents, or change infrastructure. Enabling a tool-call parser on the endpoint does not grant any of those capabilities.

2. Confirm PAIS and Llama are ready

Before opening Agent Builder, I confirm my PAIS instance and model endpoint are healthy:

vcf context use lab-supervisor
kubectl get paisconfiguration -n wkld1-private-ai
kubectl get modelendpoints -n wkld1-private-ai
kubectl describe modelendpoint llama-31-8b-main -n wkld1-private-ai

My active routing name is llama-31-8b-main. I already validated a direct chat completion and a synthetic tool-call/result exchange against that endpoint. If inference is failing, I resolve the endpoint problem before testing agent behavior.

My previous Granite endpoint remained at zero replicas for rollback. I select the active Llama endpoint rather than a similarly named, stopped, or failed resource.

3. Sign in and open Agent Builder

1. Open https://pais.lab.example.com/.
2. Choose the configured VCF Automation authentication provider.
3. Sign in using an account in the organization allowed by my PAIS configuration.
4. Confirm the browser returns to PAIS over the expected HTTPS hostname.
5. Open Agent Builder and start creating an agent.

My infrastructure administrator login and my PAIS tenant login serve different purposes. Successful kubectl access to Supervisor does not automatically prove the browser user has access to my PAIS organization.

4. Name the agent and select the model

I use a descriptive name and identify the underlying model explicitly. The following values mirror the intent of my working configuration while using generic names:

The model value is my served routing name, not the Harbor repository path. The agent’s own name is a label; I record its generated agent ID after saving because integrations use that ID.

5. Add clear instructions

I keep the initial instructions focused on useful answers and honest capability boundaries. This example is a starting prompt rather than a claim that every model response will follow it perfectly:

You are my private lab assistant for VMware Cloud Foundation, Kubernetes, networking, and Private AI.

Give clear, practical answers. State assumptions and distinguish verified facts from suggestions. Ask for missing details when they materially affect the answer.

When explaining a procedure, provide prerequisites, ordered actions, and a way to verify the result. Explain the impact of commands that change resources.

You currently have no live tools and no connection to my infrastructure or private document repositories. Do not claim that you inspected a system, executed a command, changed a resource, or verified a live condition.

If a question needs current product documentation or live environment data that is unavailable, say what information is needed. Do not invent versions, command output, or successful actions.

Do not request passwords, private keys, or access tokens in the conversation.

I select system-message instruction mode for this configuration. Instructions guide the model; they are not a substitute for authentication, tool permissions, network controls, or application authorization.

6. Configure conversation history

My recorded agent settings used a maximum history length of 8,000 characters, FIFO truncation, system-message instructions, and no explicit session expiration. I use those settings as my initial lab baseline.

The agent history limit is not the same as my model endpoint’s 4,096-token context limit. Characters and tokens are different units, and the model also needs room for instructions, the current question, tool results if present, and the generated answer.

FIFO removes older content when the history budget is reached. It does not create permanent memory or guarantee that an early fact will remain available. Choose an explicit expiration and retention policy appropriate to the intended use before extending my lab configuration to other users.

7. Leave tools disconnected for the first test

My PAIS organization had no approved tools in its tool catalog at this stage, so I saved the agent without attaching any. I do not select an unrelated tool just to populate the field.

A tool-free baseline helps isolate three questions: does the model answer, do the instructions influence the answer, and does the configured conversation history behave as expected? MCP integration and knowledge-base retrieval need their own configuration and validation.

8. Save and check agent availability

1. Review the selected Llama routing name.
2. Check the instructions and conversation settings.
3. Confirm no unintended tools are attached.
4. Save the agent.
5. Wait for its status to show Available, and inspect any reported errors.
6. Record the generated agent ID and its Agent Builder URL.

In my lab, the Llama agent later served the browser chat integration. That integration required the agent ID, not just its display name.

Broadcom’s PAIS API guide documents agent instructions, model selection, history configuration, availability status, and agent completion requests. I use the installed PAIS version’s UI and schema when field labels differ.

9. Test a new Playground conversation

Open the saved agent in the Playground and enable a conversation session when testing history. I start with a new conversation to avoid carrying over context from another model or earlier instructions.

1. Ask: Explain the difference between a PAIS model endpoint and an agent in three sentences.
2. Check that the answer addresses the question and does not claim access to my environment.
3. Ask: My test project is named Cedar. Repeat the project name and remember it for this conversation.
4. Follow with: What is the test project called?
5. Start another fresh conversation and confirm I am not accidentally reusing the earlier session.

The follow-up checks short-term conversation behavior. It does not prove indefinite retention, cross-session memory, or correctness on product-specific questions.

10. Check the capability boundaries

I also test questions that should expose the limits of the initial agent:

If the model invents access or results, I revise the instructions and repeat the test. For a real integration, I enforce permissions in the application and tools rather than trusting the prompt alone.

11. Troubleshoot the first agent

• Model missing from selection: verify the endpoint is Ready, the routing name is correct, and the signed-in user has access to the intended PAIS organization.
• Agent unavailable: inspect its reported errors and the selected model’s state before changing unrelated settings.
• Instructions appear ignored: confirm the saved instruction mode and start a new conversation before retesting.
• Follow-up loses context: check that the same session is being used and review history truncation and the endpoint context budget.
• Slow or failed responses: inspect model worker health, GPU memory, concurrency, and inference logs.
• No tools available: treat an empty catalog as an integration task, not a model failure.

What I have at this point

I now have a named PAIS agent using my private Llama endpoint, with explicit instructions and a tested conversation configuration. In my completed lab build, streaming answers and contextual follow-ups also passed through the separate browser chat application.

The next post will document that browser interface: an NGINX frontend and backend running on Kubernetes, authentication through VCF Automation, HTTPS and DNS, and the connection to my PAIS agent. MCP tools remain a later implementation and validation task.