THE INTERNET IS THINKING
Earlier sources: Multimodal AI systems
Multimodal AI systems process or generate more than one kind of information, such as text, images, audio, and video. Follow model architectures, cross-modal reasoning, interfaces, and evaluation.
SYMTRI organizes observations from public sources into a living map. Connections reflect shared subject matter; they do not establish causation or endorse a source.
Open in the map ↗38 classified signals in the fourteen-day window. These counts include signals awaiting readable evidence.
Earlier sources
Up to ten sources with readable evidence, ordered by their feed timestamp. These archive sources can be older than the live fourteen-day window. This list is a sample, not a measure of total activity. Dates distinguish paper publication, repository creation, model/dataset activity, Hacker News posts, and KEV catalog additions. “First archived” records when SYMTRI first stored the signal.
- arXiv · Published · First archived
Rephrase Before You Act: Characterizing and Mitigating Language Sensitivity in Vision-Language-Action Models ↗
Vision-language-action models (VLAs) are strikingly sensitive to instruction phrasing and do not inherit the language robustness of the vision-language models they are built on. A one-word edit can move success by tens of points: $π_{0.5}$ turns on a LIBERO stove 100% of the time for "switch on the stove" and 2% for "switch on the hot plate", and a $π_0$ checkpoint finetuned with rephrase augmentation still shows swings of up to 61 points. We characterize this sensitivity with statistically tested single-edit swings and an oracle phrase search, which shows that phrasing alone nearly closes the…
- arXiv · Published · First archived
MemoCare: An Interactive Multimodal Mobile System for Automated Cognitive Screening ↗
MemoCare is an interactive mobile system for automated multimodal cognitive screening. A React Native application combines spoken responses, temporal and spatial orientation, touchscreen actions, and visuoconstruction in complete English and Vietnamese workflows. Speech is transcribed by Google Speech-to-Text and scored locally with deterministic task-specific natural language processing rules; GPS coordinates are resolved by the MemoCare spatial module before answer matching; touch tasks are scored from interaction events; and the drawing task uses a three-model convolutional neural network c…
- arXiv · Published · First archived
Juno: Taming Predictive Latents for Vision-Language-Action Models ↗
Joint-embedding predictive architectures (JEPAs) predict masked or future observations in representation space, offering a natural source of predictive latents for vision-language-action (VLA) models. Yet making these latents useful across pretraining, policy learning, and deployment requires addressing three failures: mismatch with embodiment-specific control, interference with action learning, and teacher miscalibration under distribution shifts. We introduce Juno, a unified framework built around one action-conditioned JEPA that serves as a control-aligned representation backbone, a predict…
- arXiv · Published · First archived
MUNITE: Unified Multimodal Latent Inference for Any-to-Any Multimodal Generation ↗
We introduce MUNITE, a latent-variable framework for flexible any-to-any multimodal generation that treats encoding and latent generation as the same inference problem under different amounts of observed evidence. Given any subset of modalities, MUNITE models the conditional distribution over the latent representation associated with the complete observation. Full observation recovers deterministic encoding, no observation recovers the latent marginal, and intermediate subsets define conditional latent inference, all within a single conditional flow model. A shared latent sample captures varia…
- OpenAlex · Published · First archived
How Agent-Driven Creative Trace Mining Reshapes Design Behavior in a Deployment Study of Intent-Aligned Multimodal Design ↗
Designers continuously capture inspirations across media, yet these creative traces remain underutilized in computational design support. We present DesignJournal, an agent-driven system that mines multimodal creative traces and transforms them into intent-aligned image generation. The system consolidates artifacts into a unified workspace and employs an agentic pipeline for structured grouping, belief-guided prompt refinement, and iterative intent updates. We developed a framework that classifies sequential design actions, deriving a hierarchical action taxonomy from interaction logs. We used…
- OpenAlex · Published · First archived
Multimodal deep learning architecture for gym exercise recognition and real-time posture error detection using LSTM and RAG-based expert chatbot ↗
The growing popularity of home fitness has led to an increase in home-based fitness training. Still, the lack of supervision by a trained professional has resulted in improper form, thereby increasing the risk of injury during exercise. Automated computer vision systems have also been developed to monitor exercise, but these systems rely on rigid joint-angle thresholds or require large annotated datasets of improper exercise. This paper discusses an artificial intelligence (AI) fitness coaching system that uses real-time pose-based analysis of exercise and a chatbot with expert knowledge for i…
- arXiv · Published · First archived
VLA-ACL: Action-Consistent Visual Token Pruning for Efficient Vision-Language-Action Models ↗
Vision-Language-Action (VLA) models achieve strong robotic manipulation performance but incur high computational costs from processing long token sequences at every control step, limiting real-time deployment. Visual token pruning offers a direct solution, as visual patches dominate the input sequence and contain considerable redundancy. Existing approaches, however, either rely on indirect training-free heuristics, such as attention scores and motion thresholds, or require costly fine-tuning of the base VLA model. We introduce VLA-ACL (Action Consistency Learning), which learns a lightweight…
- arXiv · Published · First archived
DSV-Mem: Evaluating Multimodal Memory in Professional Workflows for MLLM Agents ↗
Conversational MLLM agents are increasingly expected to assist in professional workflows, from AI research and engineering design to product management and business operations. Yet this capability remains underexplored: existing benchmarks largely focus on informal, everyday interactions and personal-life scenarios featuring photographic natural images, isolated static artifacts, and recall-oriented questions. In contrast, professional scenarios often involve structured, information-heavy artifacts that undergo frequent revisions and authority updates, and compositional queries requiring recon…
- OpenAlex · Published · First archived
An open vision-language model for diverse medical applications ↗
Abstract Artificial intelligence has high potential for impact in healthcare applications, but its training and deployment are challenging due to diverse data, a complex spectrum of possible tasks and important privacy needs. High-performing foundation models that enable data-efficient fine-tuning for diverse downstream tasks can meaningfully accelerate development in this domain. We introduce MedGemma, a collection of medical vision-language foundation models based on Gemma 3. MedGemma demonstrates advanced medical understanding and reasoning across images and text and multiple medical imagin…
- GitHub · Repository created · First archived
Technicalflight/aglab ↗
Windows 桌面端 AI 编程助手——接入你自己的模型服务商，在工作目录里安全地读代码、跑命令、改文件；支持生图/视频/音乐与多模态识别。Tauri 2 + React 19 + Rust
Live window recorded . This is the ingestion observation time, not the publication or discovery date of each source.
How to assess evidence about Multimodal AI systems
Check each original source for its methods, release status, limitations, and supporting data. A repository announcement, a preprint, and a published study provide different kinds of evidence. SYMTRI’s subject classification helps discovery; it does not independently validate the source’s claims.
See the collection and counting methodology ↗ · Read the weekly synthesis ↗
Related threads
Explore related regions
These navigation suggestions are part of the map’s editorial structure, rather than measured evidence of a connection.