THE INTERNET IS THINKING
Earlier sources: Multimodal AI systems
Multimodal AI systems process or generate more than one kind of information, such as text, images, audio, and video. Follow model architectures, cross-modal reasoning, interfaces, and evaluation.
SYMTRI organizes observations from public sources into a living map. Connections reflect shared subject matter; they do not establish causation or endorse a source.
Open in the map ↗39 classified signals in the fourteen-day window. These counts include signals awaiting readable evidence.
Earlier sources
Up to ten sources with readable evidence, ordered by their feed timestamp. These archive sources can be older than the live fourteen-day window. This list is a sample, not a measure of total activity. Dates distinguish paper publication, repository creation, model/dataset activity, Hacker News posts, and KEV catalog additions. “First archived” records when SYMTRI first stored the signal.
- arXiv · Published · First archived
Exploiting Vulnerabilities: Universal Adversarial Attacks on Vision-Language-Action Models in Robotics ↗
Recently, Vision-Language-Action (VLA) models have revolutionized robotic manipulation by seamlessly integrating visual perception, language understanding, and action generation in an end-to-end learning framework. However, since these models are designed to interact directly with the physical world and humans, their security is critical, and even small vulnerabilities can lead to catastrophic failures. In this work, we propose the Universal Adversarial Object, a sphere with optimized surface texture that significantly degrades task success rates when placed within the robot's field of view. S…
- arXiv · Published · First archived
Reinforcing Multimodal Reasoning via Token-Level Perception-Grounded Advantage Estimation ↗
Reinforcement Learning with Verifiable Rewards (RLVR) has improved the reasoning capabilities of Multimodal Large Language Models (MLLMs), yet existing frameworks rely on coarse, sequence-level reward signals that lack the fine-grained supervision over the visually-grounded steps within a multimodal reasoning chain. We investigate this gap through the lens of two token-level metrics: visual dependency (i.e. how much a token's prediction relies on the input image features) and predictive entropy. Our empirical analysis reveals two key findings: (1) correct reasoning chains exhibit a markedly sh…
- arXiv · Published · First archived
SceneJail: Exploiting Video Scenario Context to Jailbreak Multimodal LLMs ↗
Video Multimodal Large Language Models (Video-MLLMs) support reasoning over video inputs, yet remain vulnerable to jailbreak attacks that elicit policy-violating responses. Existing video jailbreaks primarily manipulate how harmful queries are visually presented, thereby treating video merely as a carrier. Consequently, the surrounding video scenario remains unexplored as a contextual attack surface. In this paper, we show that the same harmful query can elicit different safety responses when placed in different video scenarios. To systematically exploit this vulnerability, we propose SceneJai…
- OpenAlex · Published · First archived
A Hybrid BDI + RAG Multi-Agent Architecture: Plan-Based Reasoning over Multimodal Retrieval ↗
Retrieval-Augmented Generation (RAG) gives language models access to external text and image collections, but it does not make the decisions taken on that content reproducible. Classical Belief–Desire–Intention (BDI) agents provide explicit plans and traceable decisions, although they normally expect beliefs in symbolic form. We connect these capabilities in a hybrid architecture in which RAG updates agent beliefs while BDI plans remain responsible for reasoning and coordination. The plans are implemented in AgentSpeak on JASON/JADE, agents communicate through FIPA-ACL, and probabilistic model…
- OpenAlex · Published · First archived
UniOVA: Universal On-demand Video Analytics with Edge-Cloud Collaborative Multimodal LLM ↗
Video analytics is ubiquitous in modern society, and the emergence of Multimodal Large Language Models (MLLMs) has made its application even more extensive. A common way to support MLLM-based video analytics is to continuously stream video to the cloud and then extracts visual features and samples frames for analysis. However, this cloud-centric architecture can introduce high transmission overhead. Simultaneously, the fixed-rate or uniform sampling approaches employed in these systems will miss critical visual evidence relevant to the query, resulting in reduced accuracy.; AB@In this paper, w…
- OpenAlex · Published · First archived
Multimodal Hyperspectral Imaging Unveils Pigment Rearrangement in the Diatom Cyclotella meneghiniana under Changing Physicochemical Conditions ↗
Abstract Revealing the spatial organization and stress-induced rearrangements of the photosynthetic machinery in thylakoids of diatoms is crucial for understanding their metabolic responses and/or adaptation. Using a multimodal framework that combines coherent anti-Stokes Raman scattering (CARS), two-photon excited fluorescence, and confocal fluorescence microscopy with machine learning, we investigated the spatial pigment distribution in thylakoids of Cyclotella meneghiniana. Under physiological conditions, we observed dense central packing of fucoxanthins (Fx’s). In contrast, the peripheral…
- arXiv · Published · First archived
NeuronEye: Query-Guided Visual Concept Activation for Vision-Language Reasoning ↗
Current vision-language models (VLMs) encode visual information in dense hidden states where object identity, spatial layout, and local attributes are implicitly entangled rather than explicitly disentangled, limiting their ability to isolate and modulate the specific visual evidence required by a given language query. Inspired by sparse population coding and top-down modulation in biological vision, we introduce NeuronEye, a plug-in framework that constructs a sparse, concept-level neuron vocabulary from intermediate VLM representations and selectively activates query-relevant visual concepts…
- GitHub · Repository created · First archived
Alpha-Park/genpark-multimodal-voice-prosody-sentiment-analyzer-skill ↗
Multi-modal speech prosody and vocal sentiment analyzer detecting speech rate, pitch variance, and user cognitive fatigue to dynamically calibrate agent response pacing.
- GitHub · Repository created · First archived
alphaparkinc/genpark-multimodal-voice-prosody-sentiment-analyzer-skill ↗
Multi-modal speech prosody and vocal sentiment analyzer detecting speech rate, pitch variance, and user cognitive fatigue to dynamically calibrate agent response pacing.
- arXiv · Published · First archived
Beyond Token Importance: Preserving Spatial Scaffolds for Efficient Vision-Language-Action Inference ↗
Existing VLA pruning strategies primarily select individual visual tokens according to task-level semantic relevance, while overlooking the spatial information required for robotic manipulation. To examine this limitation, we construct a simple Stride baseline that uniformly samples tokens along the flattened one-dimensional visual sequence, representing a purely geometric pruning strategy. Surprisingly, Stride outperforms semantic pruning and random pruning at certain pruning ratios, but collapses when the token budget is only slightly reduced. We characterize this phenomenon through the spat…
Live window recorded . This is the ingestion observation time, not the publication or discovery date of each source.
How to assess evidence about Multimodal AI systems
Check each original source for its methods, release status, limitations, and supporting data. A repository announcement, a preprint, and a published study provide different kinds of evidence. SYMTRI’s subject classification helps discovery; it does not independently validate the source’s claims.
See the collection and counting methodology ↗ · Read the weekly synthesis ↗
Related threads
Explore related regions
These navigation suggestions are part of the map’s editorial structure, rather than measured evidence of a connection.