Flama is presented, an open-source Python framework for developing and deploying production-ready web APIs, machine learning services, and large-language-model (LLM) applications that unifies REST API development, predictive model serving, and generative AI inference in one architecture.
Abstract
We present Flama, an open-source Python framework for developing and deploying production-ready web APIs, machine learning services, and large-language-model (LLM) applications. Built on the Asynchronous Server Gateway Interface (ASGI), Flama offers a type-driven, async-first programming model that unifies REST API development, predictive model serving, and generative AI inference in one architecture. It is organised around seven subsystems: a component-based dependency injection system resolving handler parameters from type annotations at startup; a pluggable schema layer supporting Pydantic, Marshmallow and Typesystem behind a single adapter; an automatic CRUD generator turning a SQLAlchemy table and a schema class into REST endpoints backed by the Repository and Unit of Work patterns; a portable binary format (.flm) packaging models from scikit-learn, TensorFlow, PyTorch and Hugging Face Transformers with their metadata for zero-code deployment; a multi-backend LLM server running vLLM (Linux/CUDA) or MLX (Apple Silicon) and exposing four wire protocols (OpenAI, Anthropic, Ollama, and a native streaming dialect) through a shared codec; a Rust-accelerated core compiled via Maturin for routing, JSON encoding, compression and parsing; and a Model Context Protocol module turning any application into an MCP server over JSON-RPC 2.0. Built-in capabilities include JWT authentication, two pagination strategies, background tasks in threads or processes, WebSocket endpoints, Server-Sent Event and NDJSON streaming, OpenAPI 3.2.0 generation from handler signatures, and a command-line interface for running applications and for serving, packaging and inspecting models. We describe the architecture, present the programming model through worked examples, and compare Flama with existing frameworks, model serving platforms and LLM inference engines.
This work presents LACE, an AutoML framework that instead searches over complete executable pipeline programs: an evolutionary loop maintains a population of scikit-learn-compatible Python classes, and a large language model acts as the variation operator.
Sofoklis Kitharidis, C. Veenman, J. V. Rijn et al.· 0 citations
Vision–Language Models (VLMs) have emerged as powerful tools for image understanding tasks, yet their practical deployment remains hindered by significant architectural heterogeneity across model families. This paper introduces UVLM (Unified Vision–Language Model), a pip-installable Python (v3.9+) package that provides a unified interface for loading, configuring, and running multiple VLM architectures on custom image analysis tasks. UVLM currently supports two major model families which differ fundamentally in their vision encoding, tokenization, and decoding strategies: LLaVA-NeXT and Qwen2.5-VL. The package abstracts these differences behind a single inference function and eliminates all architecture-specific code from the user’s workflow. UVLM is organized as eight modular Python components (model loading, dual-backend inference, response parsing, consensus validation, batch processing, prompt assembly, model registry, and utilities) and can be deployed in three modes: Google Colab for zero-install cloud access, local Jupyter notebooks for on-premises GPU use, and as a programmatic API for integration into automated pipelines. Key features include a multi-task prompt builder supporting four response types (numeric, category, boolean, text), a consensus validation mechanism based on majority voting, a flexible token budget (up to 1500 tokens) for custom reasoning strategies, and built-in truncation detection. The package is designed for extensibility: adding a new VLM family requires implementing one backend-specific inference section and adding entries to the model registry, without modifying any other module. An illustrative example on 120 street-view images across 16 model configurations is provided to demonstrate the software’s evaluation workflow.
OpenLanguageModel (OLM) is an open-source PyTorch library for building and pretraining small language models while keeping their machinery visible. In OLM, model code reads like the architecture: components are ordinary modules, while Block, Residual, Repeat, and Parallel describe how they are wired. The resulting model can move unchanged from a teaching notebook to a complete pretraining run or a research ablation. OLM connects this readable model layer to tokenizers, local and streaming datasets, optimization, mixed precision, callbacks, checkpoints, and hardware-aware CPU, single-GPU, and single-node multi-GPU execution. We demonstrate the full path by tracing GPT-2 from diagram to code, launching a FineWeb-Edu training script, replacing one attention component, and letting AutoTrainer configure the available machine. The package includes 27 presets across nine familiar model families and documentation that progresses from LM fundamentals to architecture research. Validation shows close agreement with independent reference implementations, 90.6% four-GPU weak-scaling efficiency for a 348M-parameter workload, compact architecture edits, and positive early usability results. OLM is MIT-licensed and available through PyPI, GitHub, and its documentation site.
Tavish Mankash, Vardhaman Kalloli, Keshava Prasad et al.· 0 citations
An MLIR (Multi-Level Intermediate Representation) based compilation method for large language models, illustrated using two dialects of operators, TopOp and TpuOp, supporting a variety of generative models including the Qwen, Llama, InternVL, and MiniCPM-V series.
Pengchao Hu, Zhibin Xin, Yifan Chen et al.· 0 citations
This work shows that a malicious developer can pair a benign-looking wrapper with crafted metadata to deterministically alter post-generation behavior without modifying model weights, training data, or inference backend, and introduces TIF-BAH, a lightweight middleware defense that verifies wrapper integrity and records behavioral attestations during inference.
Nokimul Hasan Arif, Qian Lou, Meng Zheng· 0 citations
The entire ecosystem of open-source language models effectively relies on a single platform. What if this platform was forced to shut down tomorrow? Implementing and maintaining efficient model definitions and translating them between different training and inference regimes is a resource-heavy task that severely limits model efficiency and portability, hindering both scaling and deployment. Here, we present Axon, a strongly typed domain-specific language with Haskell-like syntax, that enables a write-once, run everywhere paradigm for LLM architectures. By basing collaboration on a language specification rather than a specific framework's vision, Axon fosters open cooperation and empowers researchers to implement highly specialized architectures without giving up optimization infrastructure or accepting deployment lock-in. Axon allows for concise, auditable specifications that can be automatically compiled to standalone implementations for leading frameworks: PyTorch, PyTorch with Triton, JAX, MLX and vLLM. In 467 inference benchmarking experiments on models ranging from 135M to 32B parameters, we demonstrate median speedups of 7% on PyTorch, 12% on PyTorch with Triton, 91% on JAX, and 107% on MLX, compared to the reference implementations from Transformers. When deployed as native vLLM architectures with PagedAttention and KV-cache, Axon models achieve a 58% median speedup over Transformers implementations.
Jacob Nielsen, Danial Namazifard, Lukas Galke Poech et al.· 0 citations
