multimodal-agents-course

by the-ai-mergeVerified

An MCP Multimodal AI Agent with eyes and ears!

576
Stars
148
Forks
Python
Language
8/23/2026
Added
View on GitHubDownload ZIP

⚠️ Third-Party Software Notice

This skill is third-party open-source software developed and hosted independently on GitHub. SkillTip is an informational directory and does not control or maintain the underlying repository. Any security checks displayed are automated and limited in scope. Review the source code before installing.

Read the Terms of Service

Installation

Add to your Claude Code skills directory:

# Add to your Claude Code skills
git clone https://github.com/the-ai-merge/multimodal-agents-course

Getting Started

Guides for using skills like multimodal-agents-course.

Security Report

Verified

Last scanned: —

{
  "status": "PASSED",
  "issues": []
}

README.md

Kubrick Course

logo

Hi Dave...


Learn to build AI Agents that can understand images, text, audio and videos.

A free, Open-source course by The Neural Maze and Neural Bits in collaboration with Pixeltable and Opik


logo

📖 About This Course

Tired of tutorials that just walk you through connecting an existing MCP server to Claude Desktop?

Yeah, us too.

That's why we built Kubrick AI, an MCP Multimodal Agent for video processing tasks. Yes! You read that right.

💡 Agents + Video Processing ... and MCP!

This course is a collaboration between The Neural Maze and Neural Bits (from now on, "The Neural Bros"), and it's built for developers who want to go beyond the basics and build serious, production-ready AI Systems. In particular, you'll:

  • Learn how to build an MCP server for video processing using Pixeltable and FastMCP

  • Design a custom, Groq-powered agent, connected to your MCP server with its own MCP client

  • Integrate your agentic system with Opik for full observability and prompt versioning

🖊️ What you'll learn

  • Learn how to use Pixeltable for multimodal data processing and stateful agents

  • Create complex MCP servers using FastMCP: expose resources, prompts, and tools

  • Apply prompt versioning to your MCP server (instead of defining the prompts in the Agent API)

  • Learn how to implement custom MCP clients for your agents

  • Implement an MCP Tool Agent from scratch, using Llama 4 Scout and Maverick as the LLMs

  • Use Opik for MCP prompt versioning

  • Learn how to implement custom tracing and monitoring with Opik

🚀 No shortcuts. No fluff. Let's learn by doing.


💻 What You'll Do:

Completing this course, you'll learn how to design and enable Agents to understand multimodal data, across images, video, audio, and text inputs, all within a single system.

Specifically, you'll get to:

  • Build a complex Multimodal Processing Pipeline
  • Build a Video Search Engine and expose its functionality to an Agent via MCP (Model Context Protocol)
  • Build a production-ready API to power the Agent.
  • Integrate LLMOps principles and best software engineering practices.
  • Learn about video, embeddings, streaming APIs, Vision Language Models (VLMs), and more.

After completing this course, you'll have built your own Kubrick Agent with a HAL-themed spin-off, to play the role of a new set of eyes and ears:


Getting Started

Kubrick is not a simple tutorial. So, to get this system up and running, there are a few things you need to do first.

We have detailed the steps to get you started in this GETTING_STARTED.md file.

💡 Having Kubrick running is just the first step! Now that you have it up and running, it's time to actually understand how it works (see Course Syllabus).


Watch the Full Video Course

Kubrick Multimodal Agent


🧑‍🎓 Who is this course for?

You'll get the most out of this course by building it yourself, from the ground up. The course components are structured to cover key concepts and demonstrate how to build upon them, ultimately leading to AI Systems.

Target AudienceSkills you'll get
ML/AI EngineersBuild complex MCP Servers, learn to apply AI Models to Video, Images, and Speech.
Software EngineersLearn to connect AI Components with APIs, building end-to-end agentic applications.
Data Engineers/ScientistsLearn to design an AI System, managing Video/Audio/Image data processing and structure.

Regardless of your experience or title, this course aims to unpack complex topics in practical terms and concepts you could understand, learn, and apply - helping you to build a complete AI system.

🎓 Prerequisites

In this section, we outlined a few requirements and nice-to-haves to improve your learning experience while taking this course.

CategoryLabelDescription
Programming Skills (Beginner)RequirementUnderstanding of Programming in general, and the Python language syntax.
AI/ML Concepts (Beginner)Nice to HaveUnderstanding the basic concepts behind AI, AI Models, and AI Systems.
LLMs, MCP, AgentsNice to HavePerfect if you know about them, not a problem if you don't. We'll teach and explain it step by step.
Laptop/PC with any OSRequirementAI Models inference requires compute. To overcome that, we'll mainly use API based models.

The overall level of this course is Beginner/Intermediate, but don't worry. We'll aim to explain every component step by step designed for a larger audience.

💰 What's the cost?

This course and its materials are open-source and completely free, thanks to our sponsors, Pixeltable and Opik!

You'll be able to run Kubrick examples while staying at a minimum cost. That's because we'll use OpenAI and Groq for our LLM and VLM calls, which offer freemium plans as such:

ProviderFree Credits
OpenAI$5 on first sign-up
Groq500,000 tokens/day

[!NOTE]
In this setup, for running the Kubrick Agent example, the freemium plans are enough.

📚 Course Syllabus

The Kubrick Agent open-source course consists of five comprehensive modules, covering concepts, system design, tooling, and hands-on implementation.

To get the most out of this course, we recommend:

  • Clone this repository.
  • Follow the Get Started Sections of each module.
  • Read the module articles to understand each component.
  • Follow the Video Lessons for hands-on implementation.
  • Set up the code and run the project.
Module No.In-depth Lesson (Link)Lesson Summary (3min)DescriptionCode
0Get a SummaryCourse introduction and overview. Outlining the basic componentsN/A
1Get a SummaryDiving into the core components of the multimodal processing pipeline, covering video, images, text, and audio data.kubrick-mcp
2Get a SummaryBuilding the Kubrick MCP Server with FastMCP. Covering MCP protocol (transport, communication), capabilities, tools, resources, and prompts. Adding Opik Integration and MCP Inspector for efficient debugging and local testing.kubrick-mcp
3Get a SummaryBuilding the Kubrick MCP Agent from scratch. Memory layer implementation with Pixeltable and MCP Client with FastMCP. Understanding how to translate MCP Tools into specific provider tools (Groq)kubrick-api
4Get a SummaryBuilding the Groq Agent, covering the React UI - FastAPI API communication and adding LLMOps observability and conversation tracing with Opik (by Comet)kubrick-api
5Get a SummaryComplete Round-up of the free, open-source Kubrick Course.N/A

Sponsors

Pixeltable Opik

Pixeltable
is the only Python framework that provides incremental storage, transformation, indexing, and orchestration of your multimodal data.

Opik
LLM Evaluation platform helps you build, evaluate, and optimize LLM systems that run better, faster, and cheaper

Authors

Miguel Otero Pedrido
Miguel Otero Pedrido

AI / ML Engineer


LinkedIn

Alex Razvant
Alex Razvant

AI / ML Engineer


LinkedIn

Find us on

PublicationDescriptionSubscribe
Neural Bits Logo

Neural Bits

Learn about industry standards and production-ready AI Engineering.Subscribe Now
The Neural Maze Logo

The Neural Maze

Learn to build AI Systems that actually work, from principles to production. Delivered every Wednesday.Subscribe Now

Frequently Asked Questions

What is multimodal-agents-course?

multimodal-agents-course is an open-source ai agents skill for AI coding assistants such as Claude Code, Codex CLI, and ChatGPT, built by the-ai-merge. An MCP Multimodal AI Agent with eyes and ears!. It has 576 GitHub stars.

Is multimodal-agents-course safe to use?

Yes. multimodal-agents-course passed SkillsLLM's automated security scan — a dependency vulnerability audit plus prompt-injection heuristics — with no high-severity issues. You can read the full report in the Security Report section on this page.

How do I install multimodal-agents-course?

Clone the repository with "git clone https://github.com/the-ai-merge/multimodal-agents-course" and add it to your Claude Code skills directory (see the Installation section above).

What programming language is multimodal-agents-course written in?

multimodal-agents-course is primarily written in Python. It is open-source under the-ai-merge on GitHub, so you can review or fork the full source.

Are there alternatives to multimodal-agents-course?

Yes. SkillsLLM lists many other AI Agents skills you can browse and compare side by side. Open the AI Agents category from the badge at the top of this page, or use the Related Skills and comparison links further down to weigh multimodal-agents-course against similar tools.

Comments (0)

No comments yet. Be the first to share your thoughts!

ECC

by affaan-m

10

The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.

242,21936,702JavaScript
AI Agentsai-agentsanthropicclaude-code
View details
15

An agentic skills framework & software development methodology that works.

234,96620,863Shell
AI Agentsai-agentsbrainstorming
View details

The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.

185,94028,768JavaScript
AI Agentsai-agentsanthropicclaude-code
View details

cc-switch

by farion1231

3

A cross-platform desktop All-in-One assistant for Claude Code, Codex, OpenCode, OpenClaw, Grok Build & Hermes Agent. Only official website: ccswitch.io

128,8688,826Rust
AI Agentsclaude-codeai-tools
View details

claude-code

by anthropics

Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing routine tasks, explaining complex code, and handling git workflows - all through natural language commands.

120,03119,897Shell
AI Agents
View details

Developers Also Liked

Based on votes and bookmarks from developers who liked this skill

ECC

by affaan-m

10

The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.

242,21936,702JavaScript
AI Agentsai-agentsanthropicclaude-code
View details
15

An agentic skills framework & software development methodology that works.

234,96620,863Shell
AI Agentsai-agentsbrainstorming
View details

n8n

by n8n-io

12

Fair-code workflow automation platform with native AI capabilities. Combine visual building with custom code, self-host or cloud, 400+ integrations.

201,88160,308TypeScript
MCP Serversapisai-tools
View details

The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.

185,94028,768JavaScript
AI Agentsai-agentsanthropicclaude-code
View details

cc-switch

by farion1231

3

A cross-platform desktop All-in-One assistant for Claude Code, Codex, OpenCode, OpenClaw, Grok Build & Hermes Agent. Only official website: ccswitch.io

128,8688,826Rust
AI Agentsclaude-codeai-tools
View details