THE INTERNET IS THINKING
Multimodal AI systems
Multimodal AI systems process or generate more than one kind of information, such as text, images, audio, and video. Follow model architectures, cross-modal reasoning, interfaces, and evaluation.
SYMTRI organizes observations from public sources into a living map. Connections reflect shared subject matter; they do not establish causation or endorse a source.
Open in the map ↗35 classified signals in the fourteen-day window. These counts include signals awaiting readable evidence.
Recent sources
Up to ten sources with readable evidence, ordered by their feed timestamp. These archive sources can be older than the live fourteen-day window. This list is a sample, not a measure of total activity. Dates distinguish paper publication, repository creation, model/dataset activity, Hacker News posts, and KEV catalog additions. “First archived” records when SYMTRI first stored the signal.
- Hacker News · Posted on Hacker News · First archived
SGLang – High-performance serving framework for LLM's and multimodal models ↗
Hacker News discussion.
- arXiv · Published · First archived
Reliability-Aware Multimodal Sentiment Prediction Under Local Modality Missingness ↗
Multimodal sentiment models are usually evaluated on complete utterances, yet streams often lose a contiguous span of text, audio, or video. We study local missingness on aligned CMU-MOSEI features. Our reliability-aware network (RM-Net) keeps the original temporal positions, represents unobserved positions with a learned token, outputs bounded per-modality intensity contributions gated by observation quality, and is trained with dual-view masking of contiguous segments. We fuse five RM-Net seeds with a LoRA-adapted BERT text expert, using a fusion weight that depends only on the visible fract…
- arXiv · Published · First archived
PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis ↗
Multimodal code intelligence connects the symbolic world of programs with the visual world their execution creates. Interest in this capability is burgeoning, as models are increasingly asked to produce charts, webpages, animations, and semi-structured documents. Recently, proprietary models have pushed far ahead, producing interactive artifacts even from a text description or a screenshot alone. Yet their data and training remain closed, and open progress is held back by the quantity and quality of available data and by the pipelines and methods for producing it. Quality is the harder of the…
- OpenAlex · Published · First archived
Toward explainable traffic crash liability analysis via neural-symbolic multimodal reasoning: Integrating visual perception with legal knowledge graphs ↗
Determining traffic crash liability relies on rigorous on-site evidence and legal provisions and involves substantial manual intervention and potential human error. However, intelligent traffic monitoring systems focus on detection and tracking and cannot provide structured, rule-grounded decision support for formal crash-liability analysis. Moreover, emerging multimodal large language models (MLLMs) suffer from object hallucination that distorts or omits physical evidence, and their opaque reasoning cannot produce explicit evidence-to-rule chains required for traceable and reviewable liabilit…
- OpenAlex · Published · First archived
LLM-Orchestrated Evasion of Tabular NIDS and GAN Augmentation for Multimodal Detection ↗
Network Intrusion Detection Systems (NIDSs) based on machine learning have demonstrated strong performance against known attack patterns. However, their robustness against adaptive evasion strategies orchestrated by modern Large Language Models (LLMs) remains largely unexplored. This study presents a reproducible framework for evaluating traffic-space evasion of a tabular network intrusion detection baseline by an instruction-constrained open-weight LLM and separately examines the effect of GAN-based augmentation on a multimodal detector’s clean-traffic performance. The proposed methodology em…
- arXiv · Published · First archived
Visual Tokens on Demand: Visual Budgeting for Scalable and Efficient Multi-Image Reasoning in Multimodal LLMs ↗
Multimodal large language models have advanced rapidly, but most remain optimized for single-image inputs. Multi-image reasoning is costly because each image adds many visual tokens, increasing encoding and attention demands. Existing techniques that address the above issue, including visual token pruning, token merging, visual token compression, learned resampling, hierarchical compression, retrieval-augmented visual reasoning, and memory-based architectures, have largely been designed for single-image scenarios, where redundancy is primarily spatial and local. Multi-image reasoning introduce…
- arXiv · Published · First archived
LIBERO-RIG: A Hardware-Setup Robustness Benchmark for Vision-Language-Action Models ↗
Robot policies are expected to generalize across deployment environments, including variations in the robot itself: cameras are bumped and remounted, calibrations drift, controllers run at different rates, and end-effector pads wear down. We present LIBERO-RIG, a benchmark that perturbs only the robot's hardware and setup while holding every LIBERO task fixed. The benchmark spans eleven perturbation axes, including camera mounting and optics, camera-to-arm calibration, actuation gain, control rate, proprioceptive noise, base pose, table height, and gripper pads. Each axis has five severity lev…
- arXiv · Published · First archived
DPOison-VL: Clean Preference Poisoning in Vision-Language Direct Preference Optimization ↗
Direct Preference Optimization (DPO) has become a widely adopted approach for aligning vision-language models (VLMs), yet the security of multimodal preference data remains largely unexplored. Existing preference poisoning attacks either corrupt explicit supervision signals or require visible triggers, making them detectable under human inspection. We introduce DPOison-VL, the first clean preference poisoning attack against VLM-DPO, which manipulates model alignment solely through imperceptible perturbations on a small fraction of training images, while leaving all prompts, responses, and pref…
- GitHub · Repository created · First archived
Alpha-Park/genpark-agentic-multimodal-wardrobe-outfit-curator-skill ↗
AI Multimodal Wardrobe & Capsule Outfit Curator. Aligns digital wardrobe inventories with real-time weather forecasts, occasion formality constraints, and color harmony models (monochromatic, complementary, neutral anchors) to curate optima | GenPark: https://www.genpark.ai | LinkedIn: https://www.llinkedin.com/company/92955550
- GitHub · Repository created · First archived
alphaparkinc/genpark-agentic-multimodal-wardrobe-outfit-curator-skill ↗
AI Multimodal Wardrobe & Capsule Outfit Curator. Aligns digital wardrobe inventories with real-time weather forecasts, occasion formality constraints, and color harmony models (monochromatic, complementary, neutral anchors) to curate optima | GenPark: https://www.genpark.ai | LinkedIn: https://www.llinkedin.com/company/92955550
Live window recorded . This is the ingestion observation time, not the publication or discovery date of each source.
How to assess evidence about Multimodal AI systems
Check each original source for its methods, release status, limitations, and supporting data. A repository announcement, a preprint, and a published study provide different kinds of evidence. SYMTRI’s subject classification helps discovery; it does not independently validate the source’s claims.
See the collection and counting methodology ↗ · Read the weekly synthesis ↗
Related threads
Explore related regions
These navigation suggestions are part of the map’s editorial structure, rather than measured evidence of a connection.