THE INTERNET IS THINKING
AI infrastructure
AI infrastructure comprises the compute, storage, serving systems, and operational tools used to train and deploy models. Follow inference engines, distributed training, observability, and the costs and constraints of running AI workloads.
SYMTRI organizes observations from public sources into a living map. Connections reflect shared subject matter; they do not establish causation or endorse a source.
Open in the map ↗34 classified signals in the fourteen-day window. These counts include signals awaiting readable evidence.
Recent sources
Up to ten sources with readable evidence, ordered by their feed timestamp. These archive sources can be older than the live fourteen-day window. This list is a sample, not a measure of total activity. Dates distinguish paper publication, repository creation, model/dataset activity, Hacker News posts, and KEV catalog additions. “First archived” records when SYMTRI first stored the signal.
- Hacker News · Posted on Hacker News · First archived
Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio ↗
Hacker News discussion.
- Hacker News · Posted on Hacker News · First archived
TezGrid: AI inference on idle computers ↗
Hacker News discussion.
- Hacker News · Posted on Hacker News · First archived
Show HN: Golang DL/ML primitives with inference, diffusion, vision, systemone ↗
I created a DL inference engine that slowly grew to have more and functionality. - Inference for a bunch of open weights architectures - Diffusion image generation fir Z ai turbo - Vision input into LLM for inference - Vulkan/fast CPU, optimized kernels - Playground for testing - Last but not least, system one fast inference mode with same API semantics as Jev It's MIT, so go nuts
- Hacker News · Posted on Hacker News · First archived
Tessaract: lightweight layer for unified LLM inference ↗
Hacker News discussion.
- Hacker News · Posted on Hacker News · First archived
MDMA: An open source federated LLM inference engine ↗
Hacker News discussion.
- GitHub · Repository created · First archived
hongfaqiu/strata-manager ↗
Strata Manager makes Niko1221/Strata, the local LLM inference engine, easier to set up and use. Manage local models and services, monitor hardware and API usage, chat and tune inference. Open-source desktop app for Windows / Linux x64.
- Hacker News · Posted on Hacker News · First archived
Local LLM Inference at Scale with vLLM ↗
Hacker News discussion.
- arXiv · Published · First archived
Inference-Time Steering Across Scaffolds and Epitopes for Sample-Efficient De Novo Nanobody Design ↗
Generative molecular design typically relies on over-generation and post-hoc filtering. Inference-time steering methods like Feynman-Kac (FK) steering improve efficiency by biasing trajectories toward high-reward candidates, yet they optimize within a single fixed configuration. In de novo nanobody design, upfront commitment to conditioning choices like scaffold identity or target epitope creates a bottleneck. Since yield varies across these conditions, steering unpromising setups wastes compute. We recast generative condition selection as an inference-time resource-allocation problem. We prop…
- Hacker News · Posted on Hacker News · First archived
Show HN: Liquid Inference – auto-routing to competitive LLM marketplace ↗
Our background is trading systems and exchanges. Liquid Inference applies that to LLM inference. On Liquid Inference, model providers compete with each other on an open marketplace to offer the best price for any given model. We paired that with an intelligent auto-router so you always pay the lowest cost on the spot for a preset quality level. Currently we're cheaper on Kimi/GPT/GLM but working hard to be the best price for all models. You can a…
- arXiv · Published · First archived
Poster: A Preliminary Study of LLM Distillation Inference ↗
Unauthorized model distillation, in which a model is trained on the outputs of a proprietary large language model (LLM), is a growing threat to model providers. We study distillation inference: determining whether a suspect model was distilled from another model or trained independently. We formulate this problem as a hypothesis test and estimate the behavior expected under each hypothesis by training shadow models: distilled shadow models learn from the teacher's reasoning traces, whereas independent shadow models learn only from reference answers. The auditor measures how closely each model…
Live window recorded . This is the ingestion observation time, not the publication or discovery date of each source.
How to assess evidence about AI infrastructure
Check each original source for its methods, release status, limitations, and supporting data. A repository announcement, a preprint, and a published study provide different kinds of evidence. SYMTRI’s subject classification helps discovery; it does not independently validate the source’s claims.
See the collection and counting methodology ↗ · Read the weekly synthesis ↗
Related threads
Explore related regions
These navigation suggestions are part of the map’s editorial structure, rather than measured evidence of a connection.