THE INTERNET IS THINKING
Earlier sources: Multimodal AI systems
Multimodal AI systems process or generate more than one kind of information, such as text, images, audio, and video. Follow model architectures, cross-modal reasoning, interfaces, and evaluation.
SYMTRI organizes observations from public sources into a living map. Connections reflect shared subject matter; they do not establish causation or endorse a source.
Open in the map ↗38 classified signals in the fourteen-day window. These counts include signals awaiting readable evidence.
Earlier sources
Up to ten sources with readable evidence, ordered by their feed timestamp. These archive sources can be older than the live fourteen-day window. This list is a sample, not a measure of total activity. Dates distinguish paper publication, repository creation, model/dataset activity, Hacker News posts, and KEV catalog additions. “First archived” records when SYMTRI first stored the signal.
- arXiv · Published · First archived
Anatomy-aware Fine-grained Multimodal Fusion for Laryngopharyngeal Cancer T-Staging Prediction Using CT and Radiology Report ↗
Accurate T-staging is crucial for guiding personalized treatment strategies for laryngopharyngeal cancer. However, current clinical practice relies on invasive biopsy procedures, whereas CT-based staging remains challenging due to the complex patterns of tumor invasion. Recent computer-aided approaches face two key challenges: 1) Structural relationship modeling: existing methods underrepresent anatomically structured patterns of tumor invasion, as they either process whole CT volumes without tumor-specific anatomical constraints or rely on labor-intensive tumor segmentation. 2) Fine-grained c…
- arXiv · Published · First archived
MemPilot: Orchestrating On-Demand Multimodal Memory Curation for LLM Agents ↗
Memory has become integral to the LLM agent ecosystem, supporting information retention and reuse across interactions. However, most existing agent memory systems construct memory in a query-agnostic manner, which can incur unnecessary preprocessing cost and discard details that later prove essential. Recent studies have begun shifting memory processing toward runtime adaptation, but typically specialize in particular operations or fixed processing schemes, leaving flexible control over performance, cost, and latency largely underexplored. To address this challenge, we present \textbf{MemPilot…
- arXiv · Published · First archived
SAFE-MR: Evidence Sufficiency Learning for Selective Multimodal Rumor Detection ↗
Multimodal rumor detectors increasingly rely on retrieved evidence, yet relevant evidence is not necessarily sufficient for verification. Missing provenance, duplicated reports, and unresolved contradictions can produce confident predictions without adequate support. We introduce SAFE-MR, a framework that separates claim veracity from evidence sufficiency. The method decomposes image-text posts into verifiable claims, constructs a relation-aware claim-evidence graph, and aggregates evidence using provenance and contextual compatibility. Separate veracity and sufficiency heads support selective…
- OpenAlex · Published · First archived
Kecerdasan Buatan Multimodal dan Robotika: Keterbatasan Model Visi-Bahasa (VLM) dalam Persepsi Visual serta Integrasi Sensorik dan Pembelajaran Mendalam untuk Sistem Robot Cerdas ↗
Integrasi kecerdasan buatan multimodal dalam robotika telah membuka peluang baru bagi sistem robot cerdas yang mampu memahami lingkungan dan berinteraksi secara alami dengan manusia. Namun, keterbatasan mendasar pada Model Visi-Bahasa (Vision-Language Models/VLM) dalam persepsi visual menjadi hambatan signifikan dalam mewujudkan sistem robot yang benar-benar otonom dan adaptif. Penelitian ini mengkaji keterbatasan VLM dalam penalaran spasial dan persepsi visual, serta mengeksplorasi strategi integrasi sensorik dan pembelajaran mendalam untuk mengatasi keterbatasan tersebut. Melalui tinjauan li…
- OpenAlex · Published · First archived
Vulnerabilities and Defenses in Audio-Visual Attacks: A Survey From Audio to Multimodal Models ↗
With the rapid development of multimodal large language models (MLLMs) and the cost of large amounts of data and resources, researchers are more inclined to use public datasets and fine-tune open-source MLLMs to achieve excellent performance in audiovisual related tasks. While this trend accelerates progress, it also introduces security risks. Empirical studies demonstrate that the latest MLLMs can be manipulated to generate harmful content through crafted inputs, including adversarial perturbations and malicious queries, bypassing internal safeguards. To gain a deeper comprehension of the inh…
- OpenAlex · Published · First archived
Integrated Robotic System for Autonomous Inspection of Power Transmission Lines Using Multimodal Perception and Reconfigurable Topologies ↗
This paper presents an integrated robotic system for autonomous power transmission line inspection, evaluating three mechanical topologies–Line Walker, ModuClimber, and FlexRover–and a multimodal perception suite combining depth sensors. The comparative analysis reveals a distinct trade-off between mechanical simplicity and obstacle negotiation capabilities, with complex articulated designs proving essential for traversing obstacles. Crucially, the study uncovers significant challenges in algorithm portability; machine learning models trained on a specific robot topology suffered severe perfor…
- OpenAlex · Published · First archived
Multimodal LLM Pipeline for Automated Analysis of Technical Graphical Documents ↗
Automating the evaluation of technical documents, such as laboratory reports, requires balancing accurate multimodal information extraction with efficient downstream processing. This paper proposes a vision–language model (VLM) processing pipeline that extracts textual and graphical content into structured intermediate representations (JSON), enabling subsequent document assessment without redundant image retransmission. We evaluate the approach across a corpus of laboratory reports, investigating the impact of prompting strategies on text extraction and document analysis. The evaluation revea…
- OpenAlex · Published · First archived
AI ‐Mediated Multimodal Reading Education for EFL Learners: Effects on Students' Enjoyment and Engagement ↗
ABSTRACT Artificial intelligence (AI) has increasingly reshaped educational practices by expanding opportunities for adaptive, multimodal and learner‐centered instruction. While growing attention has been paid to AI‐supported language learning, less is known about how multimodal AI technologies may support the emotional and behavioural dimensions of education among at‐risk students, particularly in second/foreign language reading contexts. Addressing this gap, the present study examined whether AI‐mediated multimodal reading education could enhance at‐risk English as a foreign language (EFL) l…
- OpenAlex · Published · First archived
Artificial intelligence driven multimodal medical image segmentation algorithm: applied research and clinical verification ↗
Abstract Multimodal medical imaging—comprising CT, MRI, and PET—is fundamental to modern diagnosis, lesion localization, and treatment planning. The accuracy and speed of segmenting these modalities directly affect clinical decisions. To overcome limitations in cross-modal integration, sensitivity to small lesions, generalization, and workflow compatibility, we propose AMIS-Net, an AI-driven segmentation network. Built on an encoder-decoder backbone, AMIS-Net incorporates a Dual Attention Module (DAM) for adaptive feature recalibration and a Small Object Capture (SOC) module for multi-scale fe…
- OpenAlex · Published · First archived
Interpretable multimodal retrieval augmented diagnosis for breast ultrasound with multinational clinical validation and reader study ↗
Breast ultrasound is central to breast cancer diagnosis but suffers from substantial inter-observer variability in Breast Imaging Reporting and Data System (BI-RADS) assessment, and existing deep learning models remain opaque. Multimodal large language models promise interpretable report-based reasoning, yet scarce paired image-report data yields weak visual-semantic alignment. We propose B-RAD, a BI-RADS-aware retrieval-augmented diagnosis framework that mirrors the radiologist workflow of retrieving exemplars, localizing lesions, and determining the BI-RADS category. Its retrieval module lea…
Live window recorded . This is the ingestion observation time, not the publication or discovery date of each source.
How to assess evidence about Multimodal AI systems
Check each original source for its methods, release status, limitations, and supporting data. A repository announcement, a preprint, and a published study provide different kinds of evidence. SYMTRI’s subject classification helps discovery; it does not independently validate the source’s claims.
See the collection and counting methodology ↗ · Read the weekly synthesis ↗
Related threads
Explore related regions
These navigation suggestions are part of the map’s editorial structure, rather than measured evidence of a connection.