Image by Google Deepmind
Story: In collaboration with Liverpool FC, the team of Zhe Wang and Petar Veličković from Google Deepmind introduce TacticAI, an artificial intelligence system that provides tactical insights, specifically on corner kicks, through predictive and generative AI. This system builds on their first paper called “Game Plane” and uses a geometric deep learning approach to create more generalizable models, despite the limited availability of gold-standard data on corner kicks.
Key Findings:
Full AI System for Tactical Analysis: TacticAI is a comprehensive AI system that combines predictive and generative models to analyze previous plays and make adjustments to increase the likelihood of a particular outcome.
Predictive and Generative Models: TacticAI uses geometric deep learning to predict corner kick outcomes and generate alternative player setups for each routine of interest, allowing direct evaluation of possible outcomes.
Effective Tactical Assistant: TacticAI can assist coaches by finding similar corner kicks and testing different tactics. It computes numerical representations of players, allowing experts to efficiently look up relevant past routines.
Strategic Refinements: TacticAI provides tactical recommendations that adjust the positions of all players on a particular team, helping coaches identify key patterns and players more quickly.
Pixit‘s Two Cents: DeepMind's TacticAI is an interesting take on the intersection of AI and sports. Its ability to provide tactical insights and make strategic refinements based on predictive and generative models could greatly enhance the way football teams strategize. The potential for this technology to be applied in other team sports or even eSports is truly fascinating. We are eager to see where it might be applied next! League of Legends, maybe?
Image by Google Research
Story: Developed by Enric Corona and his team from Google Research, VLOGGER is a method for generating talking human videos driven by text and audio from a single input image. This innovative system leverages the power of generative diffusion models to produce high-quality, variable-length videos that are controllable through high-level representations of human faces and bodies.
Key Findings:
Stochastic Human-to-3D-Motion Diffusion Model: VLOGGER employs a stochastic human-to-3d-motion diffusion model. This model allows for the generation of high-quality videos that can be controlled through high-level representations of human faces and bodies.
Temporal and Spatial Controls: The application features a novel diffusion-based architecture that incorporates both temporal and spatial controls. This allows for the generation of videos of variable length, offering a broad spectrum of scenarios for synthesizing humans who communicate.
Superior Performance: VLOGGER has been evaluated on three different benchmarks, demonstrating superior performance in image quality, identity preservation, and temporal consistency compared to other state-of-the-art methods.
Diversity in Video Generation: VLOGGER can generate a diverse distribution of videos of the original subject, with significant motion and realism. This diversity in video generation is crucial for creating realistic videos.
Video Editing Applications: One of the main applications of VLOGGER is video editing. It can change the expression of the subject in a video, such as closing the mouth or eyes, making the video edits consistent with the original unchanged pixels.
Pixit‘s Two Cents: VLOGGER's development is an interesting advancement in the field of video generation. Its ability to generate high-quality, variable-length videos from a single input image using text and audio is very promising. This technology could lead to a wide range of applications, from video editing to video translation, making it a valuable tool in the realm of digital content creation. Further down the line it could potentially even support news stations to let them generate localized news readers. Exciting times ahead!
Image by StabilityAI
Story: Stability AI present Stable Video 3D (SV3D), an innovative generative model based on Stable Video Diffusion. This new technology enhances the field of 3D technology, delivering superior quality and view-consistency. SV3D introduces two variants, SV3D_u and SV3D_p, each with unique capabilities for generating 3D videos from single image inputs.
Key Findings:
Two Variants of SV3D: SV3D_u generates orbital videos from single image inputs without camera conditioning. On the other hand, SV3D_p extends these capabilities to include both single images and orbital views, allowing for the creation of 3D videos along specific camera paths.
Available for Commercial and Non-commercial Use: Stable Video 3D is now available for commercial use with a Stability AI Membership. For those interested in non-commercial use, the model weights can be downloaded on Hugging Face, and the research paper is readily accessible.
Advantages of Video Diffusion: By adapting the Stable Video Diffusion image-to-video diffusion model with camera path conditioning, SV3D is capable of generating multi-view videos of an object. This approach offers major advantages in the generalization and view-consistency of generated outputs.
Novel-View Generation: Stable Video 3D brings significant advancements in 3D generation, particularly in novel view synthesis (NVS). It is capable of delivering coherent views from any given angle with proficient generalization, enhancing pose-controllability and ensuring consistent object appearance across multiple views.
Enhanced 3D Generation: SV3D uses its multi-view consistency to optimize 3D Neural Radiance Fields (NeRF) and mesh representations, improving the quality of 3D meshes generated directly from novel views. It employs a disentangled illumination model that is jointly optimized along with 3D shape and texture to address the issue of baked-in lighting.
Pixit‘s Two Cents: The introduction of Stable Video 3D is just another great advancement in the field of 3D technology. Its ability to generate multi-view videos of an object from a single image input gives novel possibilities in 3D generation. If it becomes a little better it will be a great help for (3D) content creators from games, to media and TV production. This technology will soon be able to disrupt the way 3D content is generated!
Leadership Change at Stability AI: Emad Mostaque has resigned as CEO of Stability AI to focus on decentralized AI, with Shan Shan Wong and Christian Laforte appointed as interim co-CEOs. The company is actively searching for a permanent CEO to lead its next phase of growth.
SceneScript for Enhanced Scene Layout Generation: Meta AI has launched SceneScript, a new method for generating scene layouts and representing scenes using language, improving AR & AI devices' understanding of physical spaces. The model uses next token prediction, similar to an LLM, but predicts architectural tokens like 'wall' or 'door.’
Sakana AI Unveils Evolutionary Model Merge: Sakana AI has introduced a new method using evolutionary techniques to automate the creation of new foundation models without the need for extensive additional training data or compute. They have applied this method to develop three foundation models for Japan: Large Language Model (EvoLLM-JP), Vision-Language Model (EvoVLM-JP), and Image Generation Model (EvoSDXL-JP).
NVIDIA Unveils Earth Climate Digital Twin: NVIDIA has launched its Earth-2 climate digital twin cloud platform for simulating and visualizing weather and climate at an unprecedented scale. The platform's new cloud APIs, part of the NVIDIA CUDA-X™ microservices, allow users to create AI-powered emulations for interactive, high-resolution simulations of global climate phenomena.
Stability AI’s MindEye2 Improves Visual Perception Reconstructions: By pretraining a model across multiple subjects and then fine-tuning with minimal data from a new subject, MindEye2 enables high-quality reconstructions of visual perception using just one hour of fMRI training data. This approach improves generalization with limited training data and achieves state-of-the-art image retrieval and reconstruction metrics.
AnimateDiff becomes really fast: AnimateDiff-Lightning, a new model for rapid video generation, has been introduced. The model uses progressive adversarial diffusion distillation for state-of-the-art few-step video generation. It generates Video in a single distilled motion module with broader style compatibility.
Nvidia Unveils Project GR00T for Future Humanoids: GR00T is a multimodal AI designed to power future humanoid robots with advanced foundation AI. The project leverages a general-purpose foundation model that enables humanoid robots to process various forms of input and perform specific actions, enhancing their capabilities and simplifying their development and deployment.
Open Interpreter Debuts 01 Light: Open Interpreter has launched 01 Light, a portable voice interface that can control home computers, view screens, use apps, and learn new skills. The first batch sold out within 2.5 hours.
Tencent Launches AI Video-Generation Tool: They introduced an image-to-video AI model named Follow-Your-Click. The tool allows users to animate still images into short videos using simple text prompts, improving generation performance and offering precise user control.
AI Threatens Software Engineering Jobs: A startup named Cognition Labs has developed "the first AI software engineer," named Devin, which can build and deploy apps, find and fix bugs, and has passed practical engineering interviews from leading AI companies.
Microsoft Modifies AI Tool After Concerns: Microsoft has updated its Copilot AI tool to block certain prompts that previously resulted in the creation of violent or sexual images. The changes came after a staff AI engineer raised concerns about the tool's image-generation capabilities.