Skip to main content
Made with ❤️ by Pixit
Made with ❤️ by Pixit

AI News #78

AI News #78

Pix
Jul 8, 2024 11:10:40 AM

LMSYS Launches Multimodal Chatbot Arena for Comparing Vision-Language Models

lmsys

Story: LMSYS (Large Model Systems Organization) has expanded its popular Chatbot Arena to include image support, allowing users to chat with and compare vision-language models from major providers like OpenAI, Anthropic, and Google. In just two weeks since launch, the Multimodal Arena has collected over 17,000 user preference votes across more than 60 languages.

Key Findings:

  • Leaderboard Results: GPT-4o and Claude 3.5 Sonnet top the initial leaderboard, outperforming models like Gemini 1.5 Pro and GPT-4 Turbo. The open-source Llava 1.6 34B model also shows strong performance.

  • Alignment with Language Arena: The multimodal leaderboard rankings generally align with the language-only arena, but with some notable differences in relative model performance on vision tasks.

  • Diverse Use Cases: The Multimodal Arena has seen a wide range of applications, including general captioning, math questions, document understanding, meme explanation, and story writing.

  • Upcoming Features: LMSYS plans to add support for multiple images, PDFs, video, and audio to further expand the Multimodal Arena's capabilities.

Pixit‘s Two Cents: Providing a platform for users to interact with and vote on the performance of vision models from different providers, the arena offers valuable insights into the relative strengths and weaknesses of these systems. Since multimodal models become more and more prominent and important we are happy to see another tool helping us to navigate this crowded space. Interesting to see how the capabilities of certain models diverge strongly!

Apple Releases Public Demo of 4M: A Really Strong Multimodal AI Model

4m

Story: Apple, in collaboration with the Swiss Federal Institute of Technology Lausanne (EPFL), has made its cutting-edge 4M AI model publicly accessible through a demo on the Hugging Face Spaces platform. This release, seven months after the model's initial open-source debut, marks a significant shift in Apple's traditionally secretive approach to AI research and development.

Key Findings:

  • Versatile Multimodal Capabilities: The 4M (Massively Multimodal Masked Modeling) demo showcases a highly versatile AI model that can process and generate content across multiple modalities, including text, images, geometric data, semantics, and neural network features.

  • Accessible to a Wider Audience: By making the 4M demo publicly available on a popular open-source AI platform, Apple is expanding access to sophisticated AI technology and allowing a broader range of users to interact with and evaluate the model's capabilities firsthand.

  • Fostering an AI Ecosystem: This release demonstrates Apple's commitment to courting developer interest and fostering an ecosystem around its AI technology, marking a departure from the company's typically secretive approach to research and development.

  • Potential for Coherent AI Applications: The 4M model's unified architecture for diverse modalities could lead to more coherent and versatile AI applications across Apple's ecosystem, enhancing user experiences and enabling new possibilities for interaction.

  • Alignment with Apple's AI Strategy: The timing of the 4M demo release coincides with Apple's recent market success, AI partnerships, and the unveiling of Apple Intelligence at WWDC, positioning the company as a major player in the AI industry.

Pixit‘s Two Cents: Apple's decision to make the 4M AI model publicly accessible through a demo on Hugging Face Spaces represents a pivotal moment in the company's AI journey. By showcasing the model's impressive multimodal capabilities and inviting developers to engage with the technology, Apple is showing its commitment to leading the AI race while maintaining its focus on user privacy and seamless experiences.

Meta Introduces 3D Gen: A Breakthrough in Text-to-3D Asset Generation

3dgen

Story: Meta has unveiled 3D Gen (3DGen), a cutting-edge pipeline for generating high-quality 3D assets from textual descriptions in under a minute. This groundbreaking technology combines Meta's 3D AssetGen and 3D TextureGen models to create 3D objects with realistic shapes, textures, and physically-based rendering (PBR) materials.

Key Findings:

  • Fast and High-Quality Generation: 3DGen can generate 3D assets with high prompt fidelity and high-quality shapes and textures in less than a minute, significantly faster than industry baselines.

  • Physically-Based Rendering: The generated assets support PBR, enabling realistic relighting in real-world applications such as gaming, animation, and AR/VR.

  • Generative Retexturing: 3DGen allows users to retexture previously generated or artist-created 3D shapes using additional text inputs, providing flexibility and customization options.

  • Multimodal Representation: By integrating 3D AssetGen and 3D TextureGen, 3DGen represents 3D objects simultaneously in view space, volumetric space, and UV (texture) space, enhancing the quality and consistency of the generated assets.

  • Superior Performance: In user studies, 3DGen's two-stage generation process achieved a win rate of 68% in texture quality compared to single-stage models, outperforming numerous industry baselines in terms of prompt fidelity and visual quality for complex textual prompts.

Pixit‘s Two Cents: By combining state-of-the-art AI models and innovative techniques, 3DGen streamlines the process of creating high-quality 3D assets, reducing the time and effort required for manual modeling and texturing. The ability to generate PBR materials and retexture existing assets using text prompts opens up new possibilities for rapid prototyping, iterative design, and customization. As the technology continues to evolve, we can expect to see more advanced features and applications, such as the generation of complex scenes, animations, and interactive experiences. Are we heading to a future where complex 3D worlds are generated by the push of a button? Exciting times!

Small Bites, Big Stories:
Tags:
Pix
Post by Pix
Jul 8, 2024 11:10:40 AM