Bloom-aligned objectives

• Understand: portability challenges and cloud-specific differences
• Analyze: where lock-in happens (model endpoints, vector stores, orchestration, monitoring)
• Design: a portable reference architecture with clear provider boundaries

Topics

• Cross-cloud drivers: data residency, multi-region resiliency, cost optimization, vendor risk mitigation
• Architecture decomposition:
  • application layer (portable)
  • LLM provider layer (adapter-based)
  • retrieval layer (portable contracts, cloud-specific implementations)
  • orchestration/tool layer (portable schemas, cloud-specific bindings)
  • ops layer (unified telemetry and evaluation)
• Pattern catalog:
  • provider adapter pattern
  • feature flag + canary per provider
  • fallback and failover strategies

Labs

• Lab 1.1: Portability blueprint — Define a reference architecture and identify portability boundaries (what moves, what stays).
• Lab 1.2: Provider capability matrix — Create a comparison matrix for models, embeddings, context limits, pricing drivers, and latency constraints (conceptual + workload-based).

Bloom-aligned objectives

• Apply: Databricks for data preparation and retrieval-ready datasets
• Create: a standardized document processing and chunking pipeline
• Analyze: data quality issues affecting retrieval and generation

Topics

• Document and text processing pipelines on Databricks:
  • structured extraction, normalization, deduplication
  • chunking standards (size/overlap/metadata)
• Delta Lake patterns for GenAI:
  • bronze/silver/gold approach for documents and chunks
  • versioning and incremental refresh
• Embedding generation patterns:
  • batching and throughput design
  • consistent metadata and provenance fields

Labs

• Lab 2.1: Delta-based corpus pipeline — Build a corpus pipeline producing “chunks” table with metadata and lineage fields.
• Lab 2.2: Embedding job — Implement an embedding generation job with batching, retries, and persistence into a vector-ready store.
• Lab 2.3: Data quality checks — Implement checks for chunk quality, missing metadata, and stale content.

Bloom-aligned objectives

• Design: a retrieval interface that is portable across clouds
• Implement: RAG retrieval and grounding consistently across platforms
• Evaluate: retrieval quality and groundedness across different stacks

Topics

• Portable retrieval contract:
  • query input schema
  • output schema (top-k chunks + scores + metadata + snippet spans)
  • citation contract and “insufficient evidence” behavior
• Vector store implementation patterns:
  • Databricks-native options (vector search patterns in the Databricks ecosystem)
  • AWS patterns (managed vector DB options, OpenSearch-based approaches, or other approved enterprise options)
  • GCP patterns (Vertex AI vector search options or approved equivalents)
  • Hybrid retrieval strategies (keyword + vector) and reranking options across clouds
• Grounding strategy:
  • evidence formatting
  • citation assembly and validation
  • refusal routing when evidence is weak

Labs

• Lab 3.1: Retrieval adapter interface — Implement a retrieval client interface and 2 adapters (Databricks + one public cloud).
• Lab 3.2: Portable RAG service — Implement a RAG API that can switch retrieval adapters via config without code changes.
• Lab 3.3: Retrieval benchmark — Run the same evaluation dataset across adapters and compare retrieval hit rate and groundedness outcomes.

Bloom-aligned objectives

• Implement: provider abstraction for chat and embeddings
• Design: model routing and fallback policies
• Analyze: quality/cost/latency tradeoffs across providers

Topics

• Provider adapter design:
  • consistent request/response schema
  • error normalization and retry semantics
  • rate limits and concurrency
• Model routing policies:
  • route by task type (summarization vs extraction vs reasoning)
  • route by data sensitivity or region
  • route by latency/cost SLOs
• Prompt portability:
  • prompt templates with provider-specific constraints
  • structured outputs and schema validation

Labs

• Lab 4.1: Multi-provider LLM client — Build a client that supports at least two providers and exposes a unified interface.
• Lab 4.2: Routing policy — Implement rule-based routing (task + cost + latency thresholds) and validate behavior with a test set.
• Lab 4.3: Structured output validator — Enforce JSON/schema outputs across providers and handle failures with safe retries.

Bloom-aligned objectives

• Design: portable tooling interfaces for agentic systems
• Implement: tool gateway patterns with safe execution
• Evaluate: workflow correctness and side-effect safety across clouds

Topics

• Tool gateway architecture:
  • typed tool schemas
  • allowlists and permission scopes
  • idempotency keys and side-effect safety
• Portable agent workflow patterns:
  • single agent with tools
  • supervisor/specialist pattern
  • bounded autonomy (max steps/tool calls/cost ceilings)
• Environment binding:
  • same workflow definition, different tool endpoints by environment/cloud

Labs

• Lab 5.1: Tool gateway build — Implement a tool gateway with 5 tools (search, create ticket, update ticket, fetch profile, write note) and strict validation.
• Lab 5.2: Cross-cloud workflow run — Execute the same tool-using workflow against two environments (Databricks-centric vs cloud-centric) and compare outcomes.
• Lab 5.3: Failure injection — Simulate tool failures (timeouts, partial data) and validate safe fallback and no duplicate side effects.

Bloom-aligned objectives

• Create: a single GenAIOps pipeline that works across clouds
• Implement: evaluation gates and promotion criteria
• Analyze: cross-cloud drift and regression causes

Topics

• Repository standardization:
  • prompts, flows, tools, evaluators, datasets, infra configs
• CI evaluation gates:
  • run offline evaluation on PRs and merges
  • compare baseline vs candidate across providers
• Promotion model:
  • dev → test → prod across clouds
  • canary and A/B strategies per provider
• Drift control:
  • retrieval drift, model version changes, tool schema changes
  • regression set updates from production failures

Labs

• Lab 6.1: CI evaluation pipeline — Implement a pipeline that runs evaluation against two providers and blocks regressions.
• Lab 6.2: Promotion report — Produce a standardized promotion report comparing quality, safety, latency, and cost signals.
• Lab 6.3: Drift drill — Simulate a provider change or retrieval change and detect regression; implement rollback.

Bloom-aligned objectives

• Design: baseline security controls and cloud-specific mappings
• Apply: least privilege and data boundary enforcement
• Evaluate: audit readiness across environments

Topics

• Cross-cloud identity and access:
  • • IAM mapping and least privilege patterns
    • service-to-service authentication strategies (conceptual)
  • Data security:
    • encryption at rest/in transit
  • data residency constraints and movement controls
  • logging and retention alignment
• Governance:
  • model approval process
  • change control for prompts/tools/models
  • audit evidence pack templates

Labs

• Lab 7.1: Control mapping — Map baseline controls to AWS and GCP equivalents; identify gaps and remediation steps.
• Lab 7.2: Audit evidence checklist — Create a cross-cloud audit checklist for a GenAI release.

Bloom-aligned objectives

• Analyze: cost/latency drivers across providers
• Implement: practical optimizations
• Create: SLOs and runbooks for multi-cloud operations

Topics

• Optimization levers:
  • context trimming and retrieval narrowing
  • caching strategies
  • concurrency controls and batching
  • model routing for cheaper/faster paths
• Multi-cloud runbooks:
  • provider outage handling
  • fallback switching
  • incident response and rollback triggers

Labs

• Lab 8.1: Cost guardrails — Implement per-provider budgets and alerts; validate routing falls back when budgets are exceeded.
• Lab 8.2: Failover drill — Simulate a provider failure and validate automatic fallback and safe failure modes.

Deliverable

A working GenAI application that:

• Uses provider adapters for LLM and retrieval layers
• Implements portable RAG with citation contracts and safe-fail behavior
• Supports a tool gateway for agentic workflows with safety constraints
• Includes an evaluation suite and CI gate comparing at least two providers
• Includes a baseline security and governance pack (controls, evidence checklist, runbooks)

Tools and platforms used

• Databricks (data engineering pipelines, Delta-based corpus, GenAI integration patterns)
• AWS (managed compute/storage + approved model endpoint strategy + vector store strategy)
• Vertex AI (managed model endpoints + vector search strategy + evaluation/monitoring options)
• Cross-cloud portability layer (provider adapters, tool gateway, structured schemas)
• CI/CD evaluation pipeline (offline evaluation gates + promotion reports)
• Telemetry and operational dashboards for cross-cloud observability