🎞️ feat: OpenRouter Audio/Video File Upload Support (#11070)

* Added video upload support for OpenRouter

- Added VIDEO_URL content type to support video_url message format
- Implemented OpenRouter video encoding using base64 data URLs
- Extended encodeAndFormatVideos() to handle OpenRouter provider
- Updated UI to accept video uploads for OpenRouter (mp4, webm, mpeg, mov)
- Fixed case-sensitivity in provider detection for agents
- Made isDocumentSupportedProvider() and isOpenAILikeProvider() case-insensitive

Videos are now converted to data:video/mp4;base64,... format compatible
with OpenRouter's API requirements per their documentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: change multimodal and google_multimodal to more transparent variable names of image_document and image_document_video_audio

(also google_multimodal doesn't apply as much since we are adding support for video and audio uploads for open router)

* fix: revert .toLowerCase change to isOpenAILikeProvider and isDocumentSupportedProvider which broke upload to provider detection for openAI endpoints

* wip: add audio support to openrouter

* fix: filetypes now properly parsed and sent rather than destructured mimetypes for openrouter

* refactor: Omit to Exclude for ESLint

* feat: update DragDropModal for new openrouter support

* fix: special case openrouter for lower case provider

(currently getting issues with the provider coming in as 'OpenRouter' and our enum being 'openrouter') This will probably require a larger refactor later to handle case insensitivity for all providers, but that will have to be thoroughly tested in its own isolated PR

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com>

packages/data-provider/src/types/agents.ts

@@ -33,11 +33,26 @@ export namespace Agents {
     image_url: string | { url: string; detail?: ImageDetail };
   };
 
+  export type MessageContentVideoUrl = {
+    type: ContentTypes.VIDEO_URL;
+    video_url: { url: string };
+  };
+
+  export type MessageContentInputAudio = {
+    type: ContentTypes.INPUT_AUDIO;
+    input_audio: {
+      data: string;
+      format: string;
+    };
+  };
+
   export type MessageContentComplex =
     | ReasoningContentText
     | AgentUpdate
     | MessageContentText
     | MessageContentImageUrl
+    | MessageContentVideoUrl
+    | MessageContentInputAudio
     // eslint-disable-next-line @typescript-eslint/no-explicit-any
     | (Record<string, any> & { type?: ContentTypes | string })
     // eslint-disable-next-line @typescript-eslint/no-explicit-any
@@ -295,6 +310,8 @@ export namespace Agents {
     | ContentTypes.THINK
     | ContentTypes.TEXT
     | ContentTypes.IMAGE_URL
+    | ContentTypes.VIDEO_URL
+    | ContentTypes.INPUT_AUDIO
     | string;
 }
 

packages/data-provider/src/types/assistants.ts

@@ -515,7 +515,9 @@ export type TMessageContentParts =
     } & ContentMetadata)
   | ({ type: ContentTypes.IMAGE_FILE; image_file: ImageFile & PartMetadata } & ContentMetadata)
   | (Agents.AgentUpdate & ContentMetadata)
-  | (Agents.MessageContentImageUrl & ContentMetadata);
+  | (Agents.MessageContentImageUrl & ContentMetadata)
+  | (Agents.MessageContentVideoUrl & ContentMetadata)
+  | (Agents.MessageContentInputAudio & ContentMetadata);
 
 export type StreamContentData = TMessageContentParts & {
   /** The index of the current content part */

packages/data-provider/src/types/runs.ts

@@ -5,6 +5,8 @@ export enum ContentTypes {
   TOOL_CALL = 'tool_call',
   IMAGE_FILE = 'image_file',
   IMAGE_URL = 'image_url',
+  VIDEO_URL = 'video_url',
+  INPUT_AUDIO = 'input_audio',
   AGENT_UPDATE = 'agent_update',
   ERROR = 'error',
 }