1 Introduction↩︎

Modelling phonological and lexical structure remains central to understanding how language systems can be formally represented, learned, and generated [1], [2]. A practically important instance of this problem is typologically grounded lexicon generation: constructing artificial lexicons that map pronounceable, phonotactically well-formed word forms to meanings while remaining aligned with cross-linguistic phonological patterns. Existing approaches are either rule-based tools that offer designer control but lack formal phonotactic guarantees and typological grounding [3], [4], or LLM-based pipelines that produce plausible output but surrender interpretability and reproducibility entirely [5], [6]. Critically, no existing system enables a researcher to hold the phoneme inventory and constraint set fixed while varying the phonological grammar — the minimum requirement for isolating the contribution of any single formalisation to lexical structure.

We introduce a modular, fully parameterized lexicon generation framework that resolves this gap by decomposing the pipeline into three independently controllable components. First, a phonological inventory sampler draws phoneme inventories from PHOIBLE frequency distributions [7], enforcing empirically validated implicational universals through a statistically grounded repair stage. Second, a word generation module evaluates candidate forms against a shared constraint set under four interchangeable grammatical formalisms — deterministic, Optimality Theory (OT), Harmonic Grammar (HG), and Maximum Entropy (MaxEnt) — that differ only in how constraint violations are resolved, ensuring that every generated word form is pronounceable by construction. Third, a semantic assignment module maps generated forms to meanings drawn from a merged Leipzig–Jakarta and Swadesh ontology, optimising a Spearman rank correlation between phonological and semantic distance to produce a lexicon in which form–meaning alignment is an explicit objective rather than an incidental property.

We evaluate the framework quantitatively along two complementary axes across lexicon sizes from 100 to 5,000 word forms. Results show that probabilistic grammars — OT and MaxEnt — consistently outperform the deterministic and random baselines on both axes, and that the deterministic grammar pays a measurable typological cost through categorical constraint enforcement that systematically skews the phoneme distribution.This work is relevant to the language modelling community as both a controlled testbed for probing phonotactic knowledge in LLMs and a pipeline for generating pronounceable, typologically grounded synthetic corpora for low-resource speech and language modelling.

1.0.0.1 Contributions.

Our main contributions are as follows:

• Modular, parameterized lexicon generation. We propose a unified framework in which deterministic, OT, HG, and MaxEnt phonological grammars operate over a shared inventory and shared constraint set. The novelty lies not in the grammars themselves but in their integration with typologically grounded inventory sampling, enabling confound-free empirical comparison across formalisms that existing systems do not support.

• Typologically grounded inventory sampling. We integrate PHOIBLE frequency distributions and statistically validated implicational universals as structured generative priors for phoneme inventory construction, grounding generated lexicons in cross-linguistic typological reality.

• Pronounceability as a formal guarantee. By enforcing phonotactic constraints — including the Sonority Sequencing Principle, onset/coda structure, and nasal–stop homorganicity — at generation time, the framework guarantees that every output word form is well-formed, making the lexicon a principled foundation for spoken constructed languages.

• Form–meaning alignment via semantic optimisation. We introduce a hill-climbing semantic assignment procedure that maximises Spearman rank correlation between phonological and ontological semantic distance, operationalising lexical iconicity as an explicit generation objective.

• Quantitative evaluation framework. We introduce a joint evaluation protocol combining character-level language model perplexity and average log-likelihood with KL divergence against a typological reference distribution, providing a quantitative benchmark for comparing phonotactic well-formedness and typological realism of generated lexicons.

• Cross-grammar compatibility analysis. We empirically demonstrate that OT and MaxEnt induce distributionally equivalent phonotactic spaces, and that deterministic grammars generate a strict subset of probabilistic phonotactic patterns.

3pt

2 Background↩︎

2.0.0.1 Formal phonological grammars.

Optimality Theory formalises phonotactic well-formedness through a ranked constraint hierarchy under strict domination [1], while Harmonic Grammar relaxes this into a numeric weighting scheme [11], and Maximum Entropy grammar extends it further into a fully probabilistic model over weighted constraint violations [2], [8]. These frameworks have each been studied and applied to individual natural languages in isolation. However, a unified architecture in which deterministic, OT, HG, and MaxEnt models operate over a shared inventory and shared constraint set — enabling direct, confound-free empirical comparison — has not been previously proposed.

2.0.0.2 Typological databases and phoneme inventory modelling.

Cross-linguistic resources such as PHOIBLE [7] aggregate phoneme inventories from thousands of languages, encoding implicational universals and quantitative markedness distributions that demonstrate phoneme inventories are structured objects shaped by universal typological pressures. Despite the richness of this data, the integration of typological databases as structured generative priors for phoneme inventory sampling within a language generation pipeline has not been addressed.

2.0.0.3 Rule-based and procedural conlang generation.

[3] and [4] demonstrate that automated conlang construction is feasible through user-defined phoneme templates and parameterisable syllable pipelines. However, neither system grounds its inventory in cross-linguistic data, employs a formal phonological grammar, or provides any quantitative measure of the phonotactic well-formedness or typological plausibility of the generated output — a gap that prevents rigorous evaluation or reproducible comparison across parameter settings.

2.0.0.4 Neural and probabilistic phonotactic models.

[9] introduce a Bayesian generative model that captures sub-lexical phonotactic structure across typologically diverse languages, and [10] establish a neural bits-per-phoneme measure of phonotactic complexity across 106 languages using LSTM language models. While these contributions demonstrate that probabilistic and neural methods can characterise the phonotactic structure of natural lexicons with considerable precision, the application of such models to the generation of new lexicons under controlled typological conditions and with structured semantic assignment has not been addressed.

2.0.0.5 Large language model pipelines for conlang generation.

[5] introduce ConlangCrafter, a multi-hop LLM pipeline evaluated through macro-level WALS-feature diversity metrics, and [6] present IASC, an agentic system generating phonotactic grammars as executable code. Both delegate phonotactic decisions to an LLM, producing output that is opaque, non-reproducible, and without formal well-formedness guarantees. Neither evaluates phonotactic quality at the word level nor supports controlled ablation over grammatical formalisms.

3 Methodology↩︎

3.1 Overview↩︎

Our lexicon generation framework comprises two principal components: (1) a statistically grounded phonological inventory sampler, and (2) a modular word generation system that operates over the sampled inventory. The pipeline first constructs a typologically plausible phoneme inventory by drawing on cross-linguistic frequency distributions from PHOIBLE [7]. This inventory then serves as the fixed, shared foundation for four interchangeable phonotactic grammars—deterministic, Optimality Theory (OT), Harmonic Grammar (HG), and Maximum Entropy (MaxEnt)—which share the same constraint representations but differ fundamentally in how violations are evaluated and resolved. This design deliberately separates typological realism, modelled at the inventory level, from phonotactic variation, modelled at the grammatical level, enabling controlled comparison across categorical and probabilistic phonological formalisms within a single unified architecture.

3.2 Phonological Inventory Sampling↩︎

The phonological sampler models sound systems as samples drawn from an empirically structured typological distribution. Each phoneme is assigned a base sampling probability proportional to its cross-linguistic frequency across PHOIBLE inventories, thereby operationalising markedness in probabilistic terms: typologically common segments are preferentially selected over rare ones, without categorical exclusion of any attested segment type.

Inventory sizes are constrained by cross-linguistic distributions. Consonant and vowel counts are sampled such that the vowel-to-consonant ratio falls within the range \([0.15,\, 0.40]\), reflecting attested typological ranges [7]. Optional archetype modes restrict sampling to particular structural subspaces—for instance, small CV-dominant systems or consonant-rich systems—enabling controlled typological variation while preserving empirical plausibility.

To capture implicational universals and co-occurrence tendencies, we conduct feature-level statistical analysis over PHOIBLE inventories. To validate the statistical basis for these typological constraints, we computed Pearson correlation coefficients and chi-square statistics over all PHOIBLE inventories (Table 2). Correlation analysis reveals that inventory size is positively associated with the presence of marked segment classes: consonant count correlates significantly with ejective presence (\(r = 0.324\), \(p < 0.001\)) and click presence (\(r = 0.188\), \(p < 0.001\)), while vowel count correlates with vowel length contrast (\(r = 0.412\), \(p < 0.001\)), justifying the conditioning of marked segment sampling probabilities on inventory size. For several feature pairs, chi-square tests yielded \(\chi^2 = 0\) with \(p = 1.0\), a result that initially suggests independence but upon inspection reflects near-deterministic implicational structure with empty or near-empty contingency cells. Because chi-square assumes symmetric variation, it is an inappropriate test for directional universals; we therefore turn to conditional probabilities \(P(Y \mid X)\) as a direct measure of implicational strength (Table 3). This analysis reveals that four of the five incorporated implicational universals are fully categorical (\(P = 1.0\)), while the remaining one is a near-universal (\(P = 0.997\), failure rate \(= 0.003\)). Collectively, these results confirm that the constraints encoded in the sampler reflect statistically validated typological regularities rather than heuristic assumptions, providing an empirical grounding for both the deterministic enforcement of true universals and the probabilistic enforcement of near-universals.

3.3 Word Generation Framework↩︎

All word generators operate over the same sampled phoneme inventory and share a unified constraint set encoding widely attested phonotactic generalisations: Onset, NoCoda, *Complex, the Sonority Sequencing Principle (SSP), and nasal-stop homorganicity. Candidate word forms are produced by sampling syllable structures from a parameterised template and filling syllable positions with segments drawn from the inventory.

3.3.1 Deterministic Generator↩︎

The deterministic phonotactic generator implements a rule-based grammar in which constraints function as hard filters. Candidate forms that violate structural limits (e.g., maximum onset complexity), the SSP as formalised under [12] sonority hierarchy, or the homorganicity requirement for nasal–stop sequences are rejected outright. This corresponds to the classical generative view in which phonotactic well-formedness is binary and constraint violations are categorically illicit [13].

3.3.2 Optimality Theory Generator↩︎

The Optimality-Theoretic (OT) generator replaces categorical rejection with evaluation over a fully generated candidate set. Three variants are implemented within a shared constraint architecture: Strict OT, in which constraints are hierarchically ranked under strict domination such that any higher-ranked violation is fatal [1]; Stochastic OT, which preserves this ranking but adds Gaussian noise at evaluation time to yield probabilistic variation [14]; and Harmonic Grammar (HG), which replaces strict ranking with numeric weights, computing harmony as a weighted sum of violations and permitting multiple lower-weighted violations to collectively outweigh a single higher-weighted one [11], [15]. Although all three variants share the same constraint definitions, they differ fundamentally in how constraint interaction is formalised.

3.3.3 Maximum Entropy Generator↩︎

The Maximum Entropy (MaxEnt) grammar extends Harmonic Grammar into a fully probabilistic model. Harmony is computed as a weighted sum of violations, and candidate probabilities are derived via the softmax transformation \(P(w) \propto e^{-H(w)}\) [2], [8], yielding a proper probability distribution over surface forms. Unlike winner-take-all OT, MaxEnt assigns gradient acceptability to candidates and supports intrinsic variation, making it particularly well-suited for modelling probabilistic phonotactics and lexical frequency distributions.

3.4 Semantic Ontology and Meaning Space↩︎

We construct a hierarchical ontology graph in which each node carries a canonical identifier and a human-readable gloss, with parent–child edges encoding hyponymic relations. The meaning space is derived by merging the Leipzig–Jakarta and Swadesh lists, producing a typologically motivated core vocabulary. Lexicalization operates exclusively at leaf nodes, preventing semantic collapsing. Leaf concepts are sampled to match the target lexicon size, weighted by pre-leaf parent category to ensure breadth across the ontology.

3.5 Semantic Assignment↩︎

Given \(N\) selected meanings \(\mathcal{M} = \{m_1, \ldots, m_N\}\) and \(N\) generated word forms \(\mathcal{W} = \{w_1, \ldots, w_N\}\), the module seeks a bijection \(\sigma : \mathcal{M} \rightarrow \mathcal{W}\) such that phonological similarity tracks semantic similarity — a form–meaning alignment objective grounded in lexical iconicity [16].

3.5.0.1 Semantic distance.

Distance between meanings is computed as tree path distance over the ontology: \[d_{\mathrm{sem}}(m_i, m_j) = \mathrm{depth}(m_i) + \mathrm{depth}(m_j) - 2 \cdot \mathrm{depth}\!\left(\mathrm{lca}(m_i, m_j)\right), \label{eq:sem95dist}\tag{1}\] where \(\mathrm{lca}(m_i, m_j)\) denotes their lowest common ancestor.

3.5.0.2 Form distance.

Phonological distance is measured using Levenshtein edit distance \(d_{\mathrm{form}}(w_i, w_j)\), a language-agnostic and computationally tractable proxy for phonological similarity.

3.5.0.3 Alignment objective and optimisation.

Mapping quality is assessed by the Spearman rank correlation between semantic and form distance vectors over all meaning pairs: \[\mathcal{S}(\sigma) = \rho_s\!\left( \bigl[d_{\mathrm{sem}}(m_i, m_j)\bigr]_{i < j},\; \bigl[d_{\mathrm{form}}(\sigma(m_i), \sigma(m_j))\bigr]_{i < j} \right). \label{eq:spearman}\tag{2}\] Since exhaustive search over all \(N!\) bijections is intractable, we employ hill-climbing with random restarts: each restart proposes random pairwise swaps \(\sigma(m_i) \leftrightarrow \sigma(m_j)\), retaining only improvements to \(\mathcal{S}\). The dominant cost is \(O(N^2)\) per iteration, tractable for the lexicon sizes considered. The final lexicon is stored as a JSON mapping from canonical semantic identifiers to surface word forms.

4 Evaluation↩︎

4.1 Phonotactic Well-Formedness↩︎

Phonotactic well-formedness is evaluated using a smoothed character-level \(n\)-gram language model trained on each generated lexicon. For a lexicon of size \(N\), a model is trained on the word forms produced by the corresponding generator and evaluated on a held-out sample. We report two metrics: perplexity (lower is better) and average log-likelihood (higher, i.e., less negative, is better). Both metrics are computed as a function of lexicon size, ranging from \(100\) to \(5000\) word forms, to assess how structural coherence scales with vocabulary size. In addition, we report an improvement ratio for each constrained model relative to the random baseline, providing a normalised measure of the benefit conferred by grammatical constraint enforcement.

4.2 Typological Realism↩︎

Typological realism is evaluated by measuring the divergence between the phoneme frequency distribution of a generated lexicon and the global phoneme frequency distribution derived from PHOIBLE [7]. Concretely, each generated lexicon is tokenised into its constituent phonemes, and the resulting unigram distribution \(P\) is compared against the PHOIBLE reference distribution \(Q\) using Kullback–Leibler (KL) divergence \(D_{\mathrm{KL}}(P \,\|\, Q)\) (lower is better). KL divergence is computed across lexicon sizes from \(100\) to \(5000\), providing a trajectory of typological alignment as vocabulary size grows.

4.3 Cross-Grammar Compatibility↩︎

To assess the distributional relationships between phonological formalisms, we construct a cross-grammar evaluation matrix. For each grammar \(g \in \{\text{deterministic, OT, MaxEnt, random}\}\), a character-level neural language model is trained on a lexicon generated by \(g\) and evaluated on lexicons generated by every other grammar. This yields a \(4 \times 4\) matrix of average log-likelihood and perplexity values, where entry \((g_{\text{train}}, g_{\text{test}})\) reflects how well the phonotactic distribution learned from \(g_{\text{train}}\) explains word forms generated by \(g_{\text{test}}\). Diagonal entries capture within-grammar fit; off-diagonal entries reveal cross-grammar transferability. All models are trained and evaluated at a fixed lexicon size of \(N = 5000\).

5 Results↩︎

5.1 Phonotactic Well-Formedness↩︎

Figure 2 reveals that probabilistic constraint evaluation produces measurably more learnable phonotactic structure than categorical filtering. OT and MaxEnt converge to substantially lower perplexity (\({\approx}5.2\) and \(5.4\) respectively at \(N=5000\)) than the deterministic generator (\({\approx}7.4\)), which in turn outperforms the random baseline (\({\approx}8.6\)). The consistent ranking across both metrics — and its stability over the full \(100\)–\(5000\) size range — indicates that this is a property of the grammars themselves rather than a sample-size artifact. The close alignment of OT and MaxEnt suggests that probabilistic constraint weighting, whether winner-take-all or distributional, produces similarly coherent phonotactic regularities; the deterministic generator’s gap reflects the cost of hard-filtering, which eliminates consistent structural patterns rather than penalising them probabilistically.

5.2 Typological Realism↩︎

Figure 3 shows an unexpected dissociation from the phonotactic results: the deterministic generator, which performed second-best on well-formedness, exhibits the highest KL divergence at convergence (\({\approx}1.915\)), while MaxEnt converges alongside the random baseline (\({\approx}1.852\)–\(1.855\)). This dissociation arises from a fundamental difference between the two evaluation axes. Phonotactic well-formedness rewards internal consistency — the degree to which word forms obey the same structural patterns — whereas typological realism rewards distributional breadth. Hard-filtering in the deterministic generator enforces consistency by suppressing entire syllable structures, but this categorical exclusion systematically under-samples certain phoneme types, skewing the generated distribution away from the cross-linguistic reference. MaxEnt, by contrast, samples probabilistically over the full candidate space, allowing rare phoneme contexts to surface occasionally and incidentally recovering a distribution close to PHOIBLE’s. That the random baseline matches MaxEnt here is not a failure of MaxEnt but a ceiling effect: once constraint enforcement is sufficiently gradient, typological alignment is largely determined by the inventory composition rather than the grammar.

5.3 Cross-Grammar Compatibility↩︎

5.3.0.1 OT and MaxEnt are distributionally equivalent.

The OT–MaxEnt and MaxEnt–OT off-diagonal entries (\(-1.75\) and \(-1.79\) log-likelihood; \(5.78\) and \(5.96\) perplexity) are nearly indistinguishable from the respective diagonal entries (\(-1.74\) and \(-1.74\); \(5.69\) and \(5.70\)). A model trained on OT lexicons explains MaxEnt lexicons almost as well as it explains its own — and vice versa. This near-symmetry indicates that OT and MaxEnt induce effectively the same phonotactic distribution, consistent with the theoretical relationship between the two: MaxEnt is a probabilistic relaxation of the same weighted constraint evaluation that underlies OT.

5.3.0.2 Deterministic is a subset of the probabilistic space.

The deterministic row shows moderate cross-grammar log-likelihoods on OT and MaxEnt test sets (\(-1.98\) and \(-1.91\)), indicating that a model trained on deterministic lexicons partially transfers to probabilistic ones. The converse is markedly asymmetric: OT and MaxEnt models assign substantially lower likelihood to deterministic test lexicons (\(-2.31\) and \(-2.23\)). This directional asymmetry implies that the deterministic grammar generates a restricted subspace of the phonotactic patterns covered by probabilistic grammars — its word forms are partially explained by probabilistic models, but probabilistic word forms fall outside what the deterministic model has learned.

5.3.0.3 Random lexicons are the universal outlier.

All trained models assign their worst likelihoods to random test lexicons, and the random-trained model transfers poorly to all grammars. Crucially, the random model’s diagonal entry (\(-2.05\) perplexity \(7.74\)) is substantially worse than even the deterministic diagonal, confirming that constraint enforcement — even categorical — produces genuinely learnable structure absent from unconstrained generation.

Acknowledgments↩︎

The authors acknowledge the use of AI tools such as ChatGPT, Claude, and Gemini for improving the presentation and grammar of this paper. All the results, analysis, and proposed techniques remain a concrete representation of the author’s contributions. The authors take full responsibility for the contents in this paper.

6 Typological Universals Evaluated in the Phoneme Sampler↩︎

The phoneme inventory sampler incorporates three categories of typological tendencies, each validated over PHOIBLE inventories [7] prior to integration. Correlational tendencies inform the probability of marked segment inclusion as a function of inventory size; co-occurrence tendencies constrain which feature combinations are permitted; and implicational universals enforce directional dependencies between features, either categorically (\(P = 1.0\)) or probabilistically where near-universal violations are attested.

6.0.0.1 Correlational tendencies (Pearson \(r\)).

Symmetric associations between inventory size and marked segment presence, measured using Pearson correlation.

• Inventory size \(\leftrightarrow\) presence of clicks

• Consonant count \(\leftrightarrow\) presence of ejectives

• Consonant count \(\leftrightarrow\) presence of clicks

• Vowel count \(\leftrightarrow\) presence of vowel length contrast

6.0.0.2 Co-occurrence tendencies (chi-square \(\chi^2\)).

Symmetric associations between pairs of binary phonological features, measured using chi-square tests of independence. Values of \(\chi^2 = 0\) reflect near-deterministic contingency structure rather than independence.

• Ejective consonants \(\leftrightarrow\) uvular consonants

• Pharyngeal consonants \(\leftrightarrow\) uvular consonants

• Nasal vowels \(\leftrightarrow\) oral vowels

• Voiced obstruents \(\leftrightarrow\) voiceless obstruents

• Fricatives \(\leftrightarrow\) stops

• Front rounded vowels \(\leftrightarrow\) front unrounded vowels

6.0.0.3 Implicational universals (conditional probability \(P(Y \mid X)\)).

Directional dependencies of the form \(X \Rightarrow Y\), measured as conditional probabilities over inventories containing \(X\). Universals with \(P = 1.0\) are enforced deterministically in the sampler; the one near-universal (\(P = 0.997\)) is enforced probabilistically.

• Pharyngeal consonants \(\Rightarrow\) uvular consonants

• Nasal vowels \(\Rightarrow\) oral vowels

• Voiced obstruents \(\Rightarrow\) voiceless obstruents (\(P = 0.997\), near-universal)

• Fricatives \(\Rightarrow\) stops

• Front rounded vowels \(\Rightarrow\) front unrounded vowels

7 Code and Reproducibility↩︎

All code, evaluation scripts, generated lexicons, and experimental results reported in this paper are publicly available at:

The repository includes the phoneme inventory sampler, all four phonological grammar implementations (deterministic, OT, HG, and MaxEnt), the semantic assignment module, and the phonotactic and typological evaluation pipelines described in Sections 3 and 4.

References↩︎