AI Technology & Industry Review
In a recent technical report, OpenAI introduces Sora, a groundbreaking text-to-video model. Sora stands out for its ability to generate videos and images spanning a wide range of durations, aspect ratios, and resolutions, producing up to a minute of high-definition video content.
A research team introduce Instruct-MusicGen, an innovative method that fine-tunes a pretrained MusicGen model to efficiently follow editing instructions, delivering superior performance across various tasks compared to existing benchmarks.
In a new paper Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities, a Google DeepMind research team introduces Zipper, a multi-tower decoder architecture. This architecture can flexibly combine multimodal generative models from independently pre-trained unimodal decoders and can be reused and repurposed in new multimodal combinations.
In a new paper NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models, an NVIDIA research team introduces NV-Embed. This generalist embedding model significantly boosts the performance of decoder-only LLMs in embedding and retrieval tasks while maintaining simplicity and reproducibility.
In a new paper Your Transformer is Secretly Linear, a research team uncovers a near-perfect linear relationship in transformations between sequential layers and introduces a novel distillation technique that approximates certain layers linearly while preserving model performance.
In a new paper A Generalist Learner for Multifaceted Medical Image Interpretation, a research team proposes MedVersa, a generalist AI model designed to enable flexible learning and tasking for medical image interpretation.
In a new paper SpeechVerse: A Large-scale Generalizable Audio Language Model, a research team from Amazon AWS AI Labs introduces SpeechVerse, a robust multi-task framework that leverages supervised instruction fine-tuning to achieve strong performance across various speech tasks.
In a new paper Imagine Flash: Accelerating Emu Diffusion Models with Backward Distillation, a Meta GenAI research team introduces an innovative distillation framework aimed at enabling high-fidelity, diverse sample generation within just one to three steps. This framework surpasses existing competitors in both quantitative metrics and human evaluations.
An IBM research team introduces the Granite Code model family. Specifically optimized for enterprise software development workflows, these models excel across a spectrum of coding tasks, rendering them versatile and well-suited for diverse coding challenges.
a research team from Google and Verily introduce Med-Gemini, a family of highly proficient multimodal models is tailored for medical tasks, boasting the capacity to seamlessly integrate web search functionality and adapt efficiently to new modalities through customized encoders.
In a new paper KAN: Kolmogorov-Arnold Networks, a research team introduces Kolmogorov-Arnold Networks (KANs) as promising alternatives to MLPs. These networks showcase superior performance in both accuracy and interpretability domains.
In a new paper NeMo-Aligner: Scalable Toolkit for Efficient Model Alignment, a team of researchers from Nvidia introduces NeMo-Aligner, a toolkit designed for large-scale LLM model alignment that can efficiently harness the power of hundreds of GPUs for training.
A pioneering research group introduces MovieChat, a novel framework tailored to accommodate extensive video durations exceeding 10,000 frames. This innovative system achieves unprecedented performance in deciphering prolonged video content.
In a new paper TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding, a research team from CMU and Meta introduces TriForce—a hierarchical speculative decoding system tailored for scalable long sequence generation, reaching up to 2.31× on an A100 GPU.
In a new paper NExT: Teaching Large Language Models to Reason about Code Execution, a Google DeepMind research team proposes Naturalized Execution Tuning (NExT), a method aims to equip LLMs with the ability to scrutinize program execution traces and deduce runtime behaviors through chain-of-thought (CoT) rationales.
An NVIDIA research team presents ScaleFold, a novel and scalable training methodology tailored for the AlphaFold model, which accomplishes the OpenFold partial training task in a mere 7.51 minutes—over six times faster than the benchmark baseline—ultimately slashing the AlphaFold’s initial training time to a remarkable 10 hours.
A Google DeepMind research team introduce RecurrentGemma, an open language model built on Google’s innovative Griffin architecture, which reduces memory usage and facilitates efficient inference on lengthy sequences, thereby unlocking new possibilities for highly efficient small language models in environments where resources are limited.
A Google research team unveils the latest iteration of MobileNets: MobileNetV4 (MNv4). This cutting-edge model boasts an impressive 87% ImageNet-1K accuracy, coupled with an astonishingly low Pixel 8 EdgeTPU runtime of merely 3.8ms.
In a new paper LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Models, a research team from Meta, University College London and Universitat Politècnica de Catalunya introduces the LM Transparency Tool (LM-TT), an open-source interactive toolkit designed for dissecting Transformer-based language models.
In a new paper Streaming Dense Video Captioning, a Google research team proposes a streaming dense video captioning model, which revolutionizes dense video captioning by enabling the processing of videos of any length and making predictions before the entire video is fully analyzed, thus marking a significant advancement in the field.
A collaborative effort involving researchers from 33 institutions presents AURORA-M, the inaugural open-source model not only excels in multilingual understanding and coding tasks but also underscores the collaborative ethos of the open-source community, promoting transparency and accessibility in AI development.
A research team from Huawei and Peking University introduces DiJiang, a groundbreaking Frequency Domain Kernelization approach, which facilitates the transition to a linear complexity model with minimal training overhead, achieving performance akin to LLaMA2-7B across various benchmarks, but at just 1/50th of the training cost.
A King’s College London research team delves into a theoretical exploration of the transformer architecture, employing the lens of topos theory. This innovative approach conjectures that the factorization through “choose” and “eval” morphisms can yield effective neural network architecture designs.
A research team from University of California, Berkeley presents a causal transformer model trained via autoregressive prediction of sensorimotor trajectories, culminating in the remarkable feat of enabling a full-sized humanoid to navigate the streets of San Francisco in a zero-shot manner.
In a new paper Stealing Part of a Production Language Model, a research team introduces the first model-stealing attack that unveils precise, nontrivial information from black-box production language models such as OpenAI’s ChatGPT or Google’s PaLM-2.
A research team from Google DeepMind and University of British Columbia presents Genie, the first generative interactive environment capable of seamlessly generating a diverse array of controllable virtual worlds based on textual prompts, synthetic images, photographs, and even sketches.
A ByteDance research team presents AnimateDiff-Lightning, a novel approach that utilizes progressive adversarial diffusion distillation, catapulting video generation into a realm of lightning-fast performance while simultaneously achieving unprecedented results in few-step video generation.
In a new paper VideoAgent: Long-form Video Understanding with Large Language Model as Agent, a Stanford University research team introduces VideoAgent, an innovative approach simulates human comprehension of long-form videos through an agent-based system, showcasing superior effectiveness and efficiency compared to current state-of-the-art methods.
Google introduces Gemma, a suite of lightweight, cutting-edge open models derived from the same research and technology underpinning the powerful Gemini models, which mark a significant leap forward in performance relative to existing open models across academic benchmarks for language comprehension, reasoning, and safety.
In a new paper Behavior Generation with Latent Actions, a research team introduces the Vector-Quantized Behavior Transformer (VQ-BeT), an innovative model offers a solution for behavior generation, addressing multimodal action prediction, conditional generation, and partial observations.
In a new paper BasedAI: A decentralized P2P network for Zero Knowledge Large Language Models (ZK-LLMs), Based Labs proposes BasedAI, which offers a decentralized approach that seamlessly integrates FHE with LLMs to uphold data confidentiality without sacrificing performance.
In a new paper Beyond Language Models: Byte Models are Digital World Simulators, a research team introduces bGPT, a pioneering model engineered explicitly for processing binary data and simulating the digital world through next-byte prediction.
In a new paper The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits, a research team introduces a new variant of 1-bit LLMs called BitNet b1.58, which preserves the advantages of the original 1-bit BitNet while ushering in a novel computational paradigm that significantly enhances cost-effectiveness in terms of latency, memory usage, throughput, and energy consumption.
In a new paper Nemotron-4 15B Technical Report , an NVIDIA research team introduces Nemotron-4 15B. Nemotron-4 15B is comprising 15 billion parameters, is trained on an extensive corpus of 8 trillion text tokens, showcasing unparalleled multilingual capabilities among models of comparable size.
In a new paper LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens, a Microsoft research team introduces LongRoPE, a pioneering method that extends the context window of pre-trained LLMs to an impressive 2048k tokens while preserving performance at the original short context window.
In a new paper Learning by Reconstruction Produces Uninformative Features For Perception, researchers Randall Balestriero and Yann LeCun shed light on why reconstruction-based learning yields compelling reconstructed samples but falters in delivering competitive latent representations for perception.
An Apple research team introduces Keyframer, a groundbreaking animation prototyping tool fueled by LLM technology. Keyframer facilitates the generation of animations from static images (SVGs), empowering users to explore design alternatives, facilitate comparisons, and foster ideation.
In a recent technical report, OpenAI introduces Sora, a groundbreaking text-to-video model. Sora stands out for its ability to generate videos and images spanning a wide range of durations, aspect ratios, and resolutions, producing up to a minute of high-definition video content.
A research team from Google DeepMind and Stanford University introduces a groundbreaking algorithm known as universal neural functionals (UNFs), which autonomously constructs permutation-equivariant models for any weight space, offering a versatile solution to the architectural constraints encountered in prior works.
An NVIDIA research team introduces Audio Flamingo, a groundbreaking audio language model that incorporates in-context learning (ICL), retrieval augmented generation (RAG), and multi-turn dialogue capabilities, achieving SOTA performance across various audio understanding tasks.
In a new paper Nomic Embed: Training a Reproducible Long Context Text Embedder, a Nomic AI research team introduces nomic-embed-text-v1, which marks the inception of the first fully reproducible, open-source, open-weights, open-data text embedding model, capable of handling an extensive context length of 8192 in English.