AI

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section is a practical, vendor-neutral guide to using and building with large language models: from prompting and AI-assisted coding, through agents, the Model Context Protocol and command-line agent tooling, to retrieval-augmented generation, local and on-device models, conversational and voice channels, and finally the operations and security concerns of running all of this in production. It is written for developers of Python, Java/Spring, Node, mobile and desktop applications who want working examples against current, official documentation rather than a marketing overview of any one vendor’s product.

Each sub-section states and dates its own version baseline in its own landing page — the model IDs, SDK versions and framework releases it was written and verified against — because these move fast and a single site-wide baseline would go stale unevenly across 15 sub-sections. Check the sub-section’s own index before relying on a version number quoted deeper in one of its pages.

flowchart TD subgraph Basics A[Foundations] --> B[AI-Assisted Development] end subgraph Customizing C[Workflows] --> D[Agents] --> E[MCP] --> F[CLIs] end subgraph Building G[LangChain / Spring AI] --> H[RAG Systems] end subgraph Deploying I[Local LLMs / Hugging Face] --> J[Channels] --> K[Voice] end subgraph Operating L[LLMOps] --> M[Security] end Basics --> Customizing --> Building --> Deploying --> Operating

Sub-sections

  • LLM Foundations & Prompting — how LLMs work under the hood, and how to prompt and manage context for them reliably.

  • AI-Assisted Development — using AI coding assistants and Claude Code day to day: autocomplete, chat, agentic coding and code review.

  • Customizing AI Workflows — shaping Claude Code and GitHub Copilot to a team’s workflow with instruction and memory files, prompt files and slash commands, skills, subagents and custom agents, hooks, permissions and settings, MCP configuration, output styles and the status line, and plugins and marketplaces, using the ai-catalog repository as the worked example.

  • AI Agents — a framework-neutral guide to designing and building LLM agents: the agent loop and workflow patterns, tool design, context engineering, memory, planning, multi-agent systems and A2A, human-in-the-loop, evaluation, guardrails, UX, observability and the 2026 frameworks landscape, with examples in LangGraph, LangChain4j and Spring AI.

  • Model Context Protocol (MCP) — the 2026-07-28 revision of the protocol (stateless core, tools, resources, prompts, elicitation, Streamable HTTP, authorization) and the same docs-assistant MCP server built three times — Python (official SDK, FastMCP, mounted in FastAPI), Spring Boot (Spring AI MCP starters) and Node.js (TypeScript SDK) — plus clients and hosts (Claude Code, VS Code and GitHub Copilot), testing with the Inspector, deployment, tool design, exposing a RAG system through MCP and MCP security.

  • Building CLIs for AI Agents — designing command-line tools that humans, scripts and AI agents can all drive reliably and that can be wrapped as MCP servers or agent tools: design fundamentals (clig.dev, POSIX, GNU), streams and exit codes, TTY and colour rules, configuration and auth, machine-readable output, agent ergonomics, dry-run and confirmation safety, self-describing help, CLI versus MCP, frameworks for Python, Java, Node.js, Go, Rust, Swift and .NET, TUIs, wrapping a CLI as an MCP server or an agent tool, argument injection defence, packaging, testing, documentation and an agent-readiness checklist, all on the docsctl running example.

  • Running LLMs Locally — running open-weight models on a laptop, workstation or GPU server: hardware sizing and quantization, Ollama, llama.cpp, vLLM and SGLang, desktop runners and NVIDIA NIM, open-weight model licences, LLM gateways and inference optimisation, plugging local models into GitHub Copilot, Claude Code and the open coding assistants, and running it all in production.

  • Hugging Face & On-Device Models — finding, evaluating and downloading Hub models and checking their licences; model formats (safetensors, GGUF, ONNX, Core ML / Core AI, LiteRT, ExecuTorch, MLX); transformers v5 pipelines; curated speech, vision, small-LLM and embedding models; conversion and quantization; fine-tuning with PEFT and TRL; and integrating Hub models into Python and Spring Boot backends and into Apple, Android, Windows / .NET, web and cross-platform apps, with Inference Providers and Spaces as the cloud fallback.

  • LangChain — LangChain 1.x and LangGraph 1.x in Python: packages, chat models, prompts, LCEL, structured output, tools, create_agent and middleware, graphs and workflows, checkpointers and stores, interrupts, streaming, multi-agent and Deep Agents, MCP adapters, multimodality and voice, guardrails, LangSmith observability, evaluation and deployment, local models, LangChain.js, and LangChain4j for Java.

  • Spring AI — Spring AI 2.0 on Spring Boot 4: setup and starters, ChatModel and ChatClient, prompts, options, advisors and structured output, tool calling and tool search, chat memory and its repositories, streaming, an MCP client and server, multimodality, transcription, speech and images, observability, evaluation, testing, security and guardrails, agentic patterns, local models, production and deployment, the 1.x to 2.x migration, and a bridge to the vector-store and RAG reference.

  • RAG Systems in Production — the day-2 engineering of retrieval-augmented generation on top of the Vector Databases & RAG foundations: parsing real documents and multimodal content, ingestion at scale and freshness, query routing and text-to-SQL, advanced indexing, conversational memory persistence, citations, hallucination control, guardrails, access control, serving RAG as a REST/SSE/WebSocket API and behind MCP, caching and cost, GraphRAG and text2cypher with Neo4j, evaluation at scale, observability and deployment — each build page in LangChain 1.x with FastAPI, Spring AI 2.0 and LangChain4j 1.20, on the DocsAssistant production service.

  • Conversational Channels — exposing the DocsAssistant RAG assistant through chat: a channel-agnostic core with adapters for streaming SSE and WebSocket APIs, chat UI protocols (AI SDK UI Message Stream, AG-UI), ready-made UIs, an embeddable widget, WebRTC data channels and XMPP, and for Microsoft Teams, WhatsApp Business, Apple Messages for Business, Slack, Telegram, Google Chat and RCS, with LangChain and Spring AI back-ends and operations guidance.

  • Voice Agents — giving the DocsAssistant RAG assistant a voice: hosted platforms (ElevenLabs Agents), realtime speech-to-speech APIs (OpenAI GPT-Live and Realtime, Gemini Live, Azure Voice Live), open-source frameworks (Pipecat, LiveKit Agents) and hand-rolled pipelines, speech-to-text and text-to-speech models, turn-taking and barge-in, delivery in the browser over WebRTC or WebSocket and over the phone with Twilio, Asterisk and SIP, with LangChain and Spring AI back-ends and operations guidance.

  • LLMOps & Evaluation — running LLM applications in production: offline and online evaluation, LLM-as-a-judge, eval sets and CI regression gates, serving at scale with vLLM and SGLang, deployment and release patterns behind a gateway, observability with the OpenTelemetry GenAI conventions, drift monitoring, feedback loops, cost management and governance, with the DocsAssistant as running example.

  • AI Security & Responsible AI — threat modelling and trust boundaries for LLM applications, the OWASP Top 10 for LLM Applications (2026) and for Agentic Applications, concrete defences against prompt injection, data leakage, excessive agency, unsafe output, RAG and supply-chain attacks and denial of wallet, guardrails, AI red teaming with garak, PyRIT and promptfoo, LLMSecOps, securing the developer toolchain, the NIST AI RMF and AI 600-1 and the EU AI Act, with the DocsAssistant as the system under attack and defence.

Books used in this section

The following requester-provided books are consulted references for this section’s concepts across its 15 sub-sections — never the primary source of any code example. Every example is verified against the current official documentation of the tool or framework it uses; on any discrepancy, that documentation is authoritative.

oreilly.com answers automated fetches with an HTTP 403 response, but every publisher page above is a live, reachable page when opened in a browser. These books are consulted references, not the primary source of any example anywhere in this section; on any discrepancy the official documentation of the tool or framework in question is authoritative.

Where AI material lives elsewhere on this site

Some AI-adjacent material already has a home elsewhere on this site and is linked from here rather than duplicated:

Page What it covers

Vector Databases & RAG

Embeddings, similarity search, ANN indexing, RAG fundamentals, agentic RAG and the LangChain / Spring AI vector-store integrations.

Neo4j — Vector Search and GenAI

Vector indexes and GenAI integrations inside the Neo4j graph database.

Qdrant — RAG Integration Patterns

RAG pipelines built specifically on the Qdrant vector database.

Redis — Spring Boot Redis as a Vector Database

Using Redis as a vector store from a Spring Boot application.

Apple Intelligence and Machine Learning

Apple’s on-device Foundation Models framework: guided generation, tool calling and on-device inference.

Azure — AI, Data and IoT Services Overview

Microsoft Azure’s hosted AI, data and IoT service catalog.

AI Catalog

A catalog of Claude Code agents and skills for automating the issue-to-PR workflow.