Overview

In the past few years, autoregressive transformers have been the dominant method in Natural Language Generation (NLG). While scaling these models has brought astounding results, this approach relies on a single core method (neural networks), a single generative paradigm (sequential next-token prediction), and a single architecture (the transformer decoder). This layout comes with significant limitations: quadratic computational cost, hallucinations, and a lack of learning during inference time, just to name a few. Furthermore, relying on scaling alone becomes more difficult high-quality training data is becoming scarce and compute more expensive.

This workshop creates a dedicated forum — the Alternative Architectures Project (AAP) — to expand and diversify core NLG architecture research. Our goal is to encourage the community to focus on non-traditional machine learning structures that learn, process, and generate language differently.

Submission Site

Papers should be submitted through OpenReview:

Submit on OpenReview — Now open for submission!

Core Themes

We invite submissions exploring any alternative core models and non-traditional generation paradigms. Topics of interest include, but are not limited to:

• Non-autoregressive generative paradigms — Models that generate text by parallel refinement across a window rather than token-by-token, enabling native text rewriting and infilling.
• Test-Time Training (TTT) & deep neural memory — Architectures where model weights dynamically update during inference, replacing static KV caches to handle long contexts efficiently.
• State Space Models (SSMs) & linear hybrids — Layouts like Mamba that process sequences with linear complexity, enabling massive context processing at a fraction of the computing cost.
• Joint Embedding Predictive Architectures (JEPA) — Non-generative core structures that predict abstract concepts in a latent space instead of predicting individual surface tokens.
• Explainable & transparent generation models — Architectures with traceable reasoning paths instead of opaque calculations, allowing humans to see exactly why a model chooses its words.
• Any other creative approaches to more efficient, adaptive, or interpretable language generation.

Target Communities

This workshop bridges deep learning theoreticians, engineers, and NLG researchers. We particularly target:

• Machine learning engineers creating new sequence layouts (SSMs, diffusion, TTT) who need robust linguistic evaluation settings.
• NLG researchers focusing on long-form text, dialogue, low-resource languages, and real-time interaction who face computational, data, and explainability constraints from standard Transformer implementations.

Submission Guidelines

Paper Types

• One extra page will be granted for camera-ready revisions.
• Supplementary material (code, data, appendices) is optional but encouraged and must be anonymised.

Formatting

Submissions must follow the ACL Author Guidelines and use ACL style files. LaTeX and Microsoft Word templates are available at https://acl-org.github.io/ACLPUB/formatting.html.

Review

Review is double-blind. Please ensure your submission is fully anonymised.

Multiple-Submission Policy

Non-archival versions may be under review elsewhere. Please indicate any parallel submissions at submission time.

Important Dates

All deadlines are 23:59 Anywhere on Earth (AoE).

Contact: r.j.a.kuiper@umcutrecht.nl · a.bagheri@uu.nl