Deploying a Read-Only MCP Service for My VCF PAIS Llama Agent

This follows deploying my browser chat interface. My custom MCP server runs in my application VKS cluster, not on Supervisor or in the PAIS model runtime cluster. It needs no GPU. My Llama inference stays on my existing PAIS GPU endpoint.

1. Confirm the starting point

I start with a working PAIS instance, Llama agent, application VKS access, private Harbor project, and the previous chat deployment. My existing HTTPS certificate covers the chat hostname, and PAIS trusts its CA. All public commands below use example hostnames; I substitute my actual names before applying them.

kubectl config current-context
kubectl get pods,svc -n pais-chat
kubectl get deployment pais-chat -n pais-chat

I verify that this context points to my application VKS cluster. A successful Supervisor login alone does not select that cluster. I save the current Deployment before changing its NGINX configuration.

kubectl get deployment pais-chat -n pais-chat -o json > chat-deployment-before.json

2. Download and inspect the example

Download my sanitized MCP example bundle. It contains the complete Node.js server, locked dependencies, Dockerfile, namespace-scoped RBAC, Deployment, ClusterIP Service, NGINX configuration, patch, and README. It contains no keys, credentials, kubeconfigs, or CA files.

SHA256: 00bc45443fa8d3ab3039b84574b7af2eb104a8b7ee0edf8346483c58241e790e

I used MCP TypeScript SDK 1.32.1 and Zod 4.3.6 on a digest-pinned Node 24 Alpine base. I review the source before building. The server exposes one tool, get_chat_workload_status, and fixes its Kubernetes namespace to pais-chat. The required scope argument only accepts chat; it cannot select another namespace.

3. Build and push to Harbor

From the extracted source directory I authenticate using my existing Harbor uploader account. I keep this application image in my private application project and reuse the existing read-only image pull Secret.

docker login harbor.lab.example.com
docker build -t harbor.lab.example.com/ai-apps/lab-status-mcp:1.0.1 .
docker push harbor.lab.example.com/ai-apps/lab-status-mcp:1.0.1

I record the digest printed by the push and replace the image in deployment.yaml with my registry path and that digest. I also replace the chat hostname in its PUBLIC_ORIGIN value. A digest from another registry or build is not a substitute for my pushed image.

4. Create the service credential

I generate a random 32-byte key in PowerShell and create a Kubernetes Secret without putting its value in the command history. I store a protected copy so I can enter it as a PAIS credential header later. This shared key is service authentication for my lab; it is not an OAuth implementation.

$key = [Convert]::ToHexString([Security.Cryptography.RandomNumberGenerator]::GetBytes(32))
ConvertTo-SecureString $key -AsPlainText -Force | Export-Clixml "$HOME/mcp-key.clixml"
@{apiVersion='v1';kind='Secret';metadata=@{name='lab-status-mcp-key';namespace='pais-chat'};stringData=@{token=$key}} | ConvertTo-Json -Depth 6 | kubectl apply -f -
$key = $null

The encrypted file uses Windows credential protection for my current account. I never upload this file or its plaintext value with the article.

5. Deploy the MCP server and limited RBAC

I apply the example manifest after editing the registry and origin. It creates a dedicated ServiceAccount, Role, RoleBinding, one-replica Deployment, and ClusterIP Service. The container runs as a non-root user with a read-only root filesystem and dropped capabilities.

kubectl apply -f deployment.yaml
kubectl rollout status deployment/lab-status-mcp -n pais-chat
kubectl get pods,svc -n pais-chat

My Role allows only get/list on pods and services in the core API group, plus deployments in the apps API group. It grants no secrets, logs, exec, or write access. I test the effective permissions:

kubectl auth can-i list pods -n pais-chat --as system:serviceaccount:pais-chat:lab-status-mcp
kubectl auth can-i get secrets -n pais-chat --as system:serviceaccount:pais-chat:lab-status-mcp
kubectl auth can-i delete pods -n pais-chat --as system:serviceaccount:pais-chat:lab-status-mcp

My results were yes, no, and no respectively. The implementation returns selected status fields rather than complete Pod specifications.

6. Add the HTTPS MCP route

I reuse my existing chat hostname and TLS certificate. In the current NGINX configuration I merge this exact location block before applying the updated configuration. I preserve my existing static files, authentication, chat API, TLS settings, and health route.

location = /mcp {
 proxy_pass http://lab-status-mcp.pais-chat.svc.cluster.local:8080;
 proxy_http_version 1.1;
 proxy_set_header Host $host;
 proxy_set_header Connection "";
 proxy_buffering off;
 proxy_request_buffering off;
 proxy_read_timeout 60s;
}

kubectl create configmap pais-chat-nginx-mcp -n pais-chat --from-file=nginx.conf --dry-run=client -o yaml | kubectl apply -f -
kubectl patch deployment pais-chat -n pais-chat --type strategic --patch-file chat-patch.json
kubectl rollout status deployment/pais-chat -n pais-chat

The patch mounts my updated NGINX file from the ConfigMap. My backend restarts during this rollout, so browser chat may require sign-in again. HTTPS protects the PAIS-to-NGINX connection; NGINX-to-MCP traffic uses HTTP within the application cluster. I keep the service internal and limit network reachability for my environment.

7. Register the server in PAIS

In my PAIS interface I select MCP → MCP Servers → Add Server and enter:

• Name: lab-status
• URL: https://chat.lab.example.com/mcp
• Transport: Streamable HTTP
• TLS verification: Full Verification
• Add Credential Header: key X-MCP-Key, value my generated service key

I save the registration and wait for Available. I verify that PAIS discovers get_chat_workload_status. If TLS verification fails, I correct DNS, certificate names, routing, or PAIS CA trust instead of disabling verification.

8. Approve the tool and attach it to my agent

I approve only get_chat_workload_status. Then I open Agent Builder → Agents → my existing Llama agent → Edit → Tools → Add Tools. I select that tool, click Add, and save the agent. I keep the same agent ID, model endpoint, and existing instructions, so my browser chat continues using the existing agent API.

9. Validate a real tool call

I enter this prompt in the PAIS Playground:

Call get_chat_workload_status with scope="chat" now. Give the observedAt timestamp and ready replicas for the pais-chat deployment.

I compare the reply with the MCP log and the live Deployment:

kubectl logs deployment/lab-status-mcp -n pais-chat --tail=20
kubectl get deployment pais-chat -n pais-chat

My verified PAIS response reported observedAt: 2026-10-21T16:06:06.006Z and ready replicas: 1. The timestamp matched a tool_called event from my MCP pod. That correlation confirms a real tool execution rather than a plausible answer generated from memory.

I also verify that a request without X-MCP-Key returns HTTP 401 and that my existing chat health endpoint still returns HTTP 200. I repeated the query through my existing browser chat. Its reply reported 2026-10-21T16:08:24.519Z and one ready replica, matching another MCP pod log event. A health response alone checks the process; it does not prove the Kubernetes permission path or LLM tool loop works.

10. Handle the empty-argument compatibility issue

My first tool used an empty argument schema. Llama called it, and the MCP response succeeded, but the next PAIS inference request failed because the streamed function call lacked its arguments field. I confirmed this in the PAIS API log: vLLM returned HTTP 400 for missing function.arguments. I changed the tool to require the fixed scope value chat, rebuilt the container, and let PAIS refresh its tool schema. The same agent then completed the real tool loop successfully. This is a tested workaround for my observed stack, not a claim that every PAIS or vLLM version needs it.

11. Operate and roll back the integration

To disable this capability I first remove the tool from my agent and unapprove it in PAIS. I can restore my saved chat Deployment and prior NGINX configuration before removing the separate MCP workload. I do not delete the chat namespace, shared TLS Secret, or Harbor pull Secret.

For key rotation I update both the Kubernetes Secret and the PAIS credential header, then restart the MCP Deployment because this implementation reads its service key at startup. Any later edit to a ConfigMap mounted through subPath also requires a chat pod rollout. I keep future tools narrowly scoped and add write operations only as a separate design decision.

References

• Broadcom PAIS MCP server APIs
• Official MCP TypeScript SDK server documentation
• Kubernetes RBAC documentation