ByteDance-Seed, has unveiled BAGEL, a unified multimodal model designed to process and generate both text and visual content. BAGEL stands out for its open-source nature, support for downstream fine-tuning, and versatility across a wide range of AI tasks, from image generation and editing to multimodal reasoning and video-based understanding.

Unlike traditional models limited to specific modalities, BAGEL is built to natively handle interleaved image, video, and text data through a single architecture, offering developers and researchers flexibility and transparency not typically available in proprietary systems.

What BAGEL Can Do

1. Unified Text-Image Understanding and Generation

BAGEL is capable of taking mixed input formats text, images, or both and providing structured, conversational, or generative outputs. This includes tasks like:

1. Describing visual scenes with contextual accuracy
2. Answering questions about images
3. Generating photorealistic visual content based on descriptive prompts

2. Image Editing with Semantic Awareness

Pretraining on interleaved video data helps BAGEL retain visual identities and detail when editing. The model can:

1. Make localized or semantic edits
2. Apply visual transformations based on natural language instructions
3. Transfer image styles (e.g., from photo to animation)

3. Learning from Motion and Perspective

BAGEL’s exposure to video data enhances its ability to infer spatial dynamics and simulate perspective changes. This supports:

1. Basic understanding of motion in videos
2. Generating sequential frames or simulating viewpoint shifts

4. Composition and Multi-Turn Interactions

With strong multimodal reasoning capabilities, BAGEL can engage in step-by-step interactions. It understands how to refine prompts and maintain consistency in visual generation, enabling:

1. Multi-turn dialogue involving visual and textual inputs
2. Thoughtful prompt expansion before image generation

Technical Architecture

BAGEL employs a Mixture-of-Transformer-Experts (MoT) architecture. Key features include:

Two distinct image encoders: one for pixel-level features, another for semantic-level understanding

1. A decoder-only model trained on a wide range of token types (language, visual, interleaved)
2. A unified prediction task: learning to predict the next group of language or visual tokens

This design supports efficient scaling and multimodal compression without relying on separate pipelines for different data types.

Training and Performance

1. Training Scale: BAGEL was trained on trillions of interleaved multimodal tokens from web-scale data including images, text, and video.
2. Model Size: The current release includes a 14B parameter model, with 7B active parameters used via routing through experts.
3. Licensing: Released under the Apache 2.0 license, BAGEL is available for commercial and research use.

BAGEL demonstrates competitive results on standard multimodal benchmarks such as MMBench, MMVet, and MMMU. It performs strongly in tasks that require vision-language reasoning, object recognition, attribute understanding, and complex editing.

BAGEL is a thoughtfully designed open-source model that brings high-performance multimodal capabilities into the public domain. By combining large-scale pretraining, compositional reasoning, and support for visual generation and editing, it offers a transparent and versatile tool for the AI community.

Researchers, developers, and educators now have access to a model that supports both experimentation and application without the barriers of closed-source systems.

You can explore the model and try demos at bagel-ai.org, or review the source code and benchmarks via ByteDance-Seed on GitHub.