May 07, 2026
Municipal complaint systems are, at their core, classification systems. When a resident calls 311 or files a digital complaint, the intake process must assign each contact to one of the action categories: route to an inspector, batch with similar cases, escalate to emergency response, or defer. This assignment happens under time pressure and with incomplete information. The result is not random error but systematic misprioritization: 311 data reveal persistent gradients in complaint classification accuracy and response assignment times across income and racial lines [1]. Low-income communities whose complaints are harder to parse, less likely to include structured data, and filed through channels that yield noisier intake records are systematically mis-routed—assigned to low-priority queues, batched when they should be escalated, or simply left unclassified.
A compounding factor is that residents do not file complaints at equal rates. Liu, Bhandaram, and Garg [2] show that even after controlling for incident characteristics, some NYC neighborhoods report the same problem three times faster than others, with reporting speed positively correlated with income, college attainment, and the fraction of white residents. Because complaint volume is the primary signal feeding current intake classifiers, this disparity in participation directly biases which complaints receive timely escalation. This happens not because the classifier is broken, but because the data it consumes encodes who was motivated and able to file in the first place. This missing-not-at-random structure is a form of measurement injustice embedded in the data itself [3]. It means that any algorithm trained naively on these records inherits, and risks amplifying, the blind spots of its predecessor.
The central claim of this paper is that this dynamic is not inevitable. Reinforcement learning (RL) systems can be configured to optimize an explicitly specified reward function. As a result, RL offers a lever that heuristic intake workflows do not: the ability to inscribe equitable classification coverage as a first-class objective. Whether that lever is used depends on whether equity is built into the reward from the outset. Our paper demonstrates what happens when it is, and what is lost when it is not.
New York City’s Department of Buildings processes tens of thousands of complaints annually, spanning boiler defects, scaffold hazards, heat failures, and illegal construction activity. Each complaint must be classified into an action tier before any inspector can be dispatched. Classifier capacity is finite; misclassifications cascade: a complaint assigned to "defer" when it should be "escalate" may never receive a field visit until the hazard worsens. Current intake heuristics are largely reactive and rule-based, ignoring temporal recurrence patterns, neighborhood-level risk accumulation, and the compounding effects of delayed classification on housing stability.
The heterogeneous participation problem identified by [2] is directly operational: the NYC DOB draws on the same 311 infrastructure that the participatory sensing framework [4] anticipated as a tool for civic data collection at an urban scale. The challenge is that participatory systems aggregate data from citizens who participate unequally, producing dense, high-quality records in engaged neighborhoods and sparse, noisy records in disengaged ones. Augmenting classification capacity with RL agents that learn from this uneven historical record is the secondary technical motivation of our work; the primary motivation is ensuring that expanded capacity does not come at the cost of equity.
This paper addresses three inter-related research questions:
Do RL-augmented classification policies distribute correct action assignments equitably across demographic groups and neighborhood income levels? Under what reward configurations do they reduce, rather than reproduce or amplify, historical classification disparities?
Can RL agents achieve meaningfully higher complaint classification throughput and lower misclassification rates than existing heuristic intake policies across multiple NYC DOB domains?
What is the quantitative cost of equity? How does the Pareto frontier between classification speed, operational cost, and equitable coverage shift with reward configuration, and how sensitive are these trade-offs to the normative assumptions embedded in reward engineering?
The ordering is intentional. Capacity is a necessary condition for deployment credibility, but equity is the main focus of this work.
Multi-domain MDP framework. We formalize six NYC DOB complaint classification domains as MDPs with a unified multi-objective reward structure in which equitable classification coverage is a first-class objective.
Equity-centered reward design. We introduce a coverage parity reward term grounded in substantive conceptions of equity and characterize its Pareto relationship with throughput and cost objectives.
Demographic equity audit. We conduct a formal audit of learned classification policies disaggregated by census-tract income quintile and racial composition.
SHAP-based interpretability. We use SHAP attribution as an equity audit instrument, identifying features that may function as demographic proxies.
Practitioner deployment guidelines. We offer evidence-based recommendations for human-in-the-loop oversight, ongoing demographic auditing, and participatory reward design.
A foundational body of scholarship documents the mechanisms by which public institutions reproduce rather than remedy spatial inequality. Rothstein [5] traces how federal, state, and local housing policy encoded racial segregation into the built environment; the legacy of those decisions structures which neighborhoods today have aging boiler stock, substandard heating systems, and chronic building violations. Desmond [6] demonstrates that housing instability is not merely a consequence of poverty but a cause of it, making rapid classification and escalation of housing violations a direct lever on economic mobility. This structural context establishes why the distributional consequences of classification algorithms are not incidental technical details. They are the core policy question.
The deployment of predictive and optimization tools in public-sector settings has generated a substantial interdisciplinary literature on how automation can entrench disadvantage. Eubanks [7] documents how automated systems in welfare, child protective services, and homeless services frequently reproduce the disadvantages of those they purport to serve, a phenomenon she terms the "digital poorhouse." Chouldechova [8] formalizes the incompatibility of common fairness criteria in recidivism prediction, a mathematical result that generalizes to any classification setting with asymmetric base rates. In the housing domain specifically, prior work has examined how algorithmic triaging of code enforcement complaints can systematically underserve low-income renters [9], and how predictive systems in child welfare can generate racially disparate referrals even when race is excluded as an explicit feature [10].
The 311 complaint infrastructure that feeds our classification system is an instance of participatory sensing at a civic scale, a paradigm introduced by Burke, Estrin, and colleagues [4] to describe systems in which everyday mobile devices collectively constitute an urban sensor network that enables data collection at spatial and temporal scales that agency-operated sensors cannot match. The foundational insight of that work is that routing data collection through ordinary citizens dramatically expands coverage. At the same time, [4] recognized that the credibility, quality, and shareability of participatory data depend critically on who participates and under what conditions. This is a tension the municipal complaint context makes acute.
Estrin’s subsequent research on small data [11] sharpens this concern: individual digital traces are dense for some users and sparse for others, and systems that treat the aggregate record as representative of ground truth without interrogating participation rates risk systematically misrepresenting the populations least likely to generate dense traces. Applied to civic complaint systems, this framing directly motivates our treatment of complaint volume as a potentially biased signal of underlying housing need rather than an objective measure of it. Our equity reward term (Equation 9 ) and our use of ACS covariates to proxy for under-reporting propensity are operationalizations of this principle.
A line of work from [2] develops rigorous methods to measure and correct the heterogeneous reporting biases that distort municipal complaint data before any classification model sees it. Liu, Bhandaram, and Garg [2] develop a statistically identified estimator for spatially varying reporting delays in resident crowdsourcing systems. Their key insight is that the rate at which duplicate reports arrive about the same incident can be leveraged to separate under-reporting from a genuine absence of incidents, a fundamental identification challenge because unreported incidents leave no trace in the record. Applied to over 100,000 NYC Department of Parks and Recreation reports and over 900,000 Chicago reports, the method uncovers substantial socioeconomic disparities in reporting speed, even after controlling for incident characteristics.
This finding has direct consequences for our classification system. The equity parity term in Equation 9 normalizes escalation throughput by complaint volume, which means it inherits reporting-rate disparities unless those disparities are corrected upstream. The Liu et al.[2] correction uses duplicate-report rates to identify neighborhood-level reporting propensity. We use the correction as a recommended preprocessing step before reward calibration and flag complaint-rate correction as a priority extension (Section 7).
Agostini, Pierson, and Garg [12] extend this line with a Bayesian spatial model that jointly uses government inspection ratings (unbiased but sparse) and crowdsourced reports (dense but biased) to estimate the latent true state of urban incidents. Their GNN-based multiview model, applied to over 9.6 million NYC crowdsourced reports and 1 million inspection ratings across 139 incident types, shows that higher-income neighborhoods report problems at systematically higher rates. This complicates any classifier that treats complaint volume as a reliable signal of ground-truth risk. Our state representations include complaint recurrence and neighborhood-level complaint frequency precisely because, as Agostini et al.demonstrate, these features carry incremental signal about the latent incident state beyond what raw volume conveys.
Liu and Garg [13] address the downstream allocation problem complementary to ours: given heterogeneous reporting and service patterns, how should a city design Service Level Agreements (SLAs) to jointly optimize efficiency and equity? The SLAs are essentially promises that incidents will be responded to within a specified time. Modeling the problem as an optimization over a queuing network, [13] show that the "price of equity" is small in realistic settings, that inefficient policies tend to inequitable and vice versa, and that status quo NYC inspection scheduling is dominated by optimized policies on both dimensions. Their empirical finding that equity and efficiency are largely complementary rather than conflicting directly corroborates our Pareto frontier results (Section 6): the speed penalty for adding an equity objective to our classification reward is modest (4–7%), consistent with Liu and Garg’s theoretical bound on the price of equity. Their queuing-network framework and our MDP framework are complementary tools for the same policy problem. [13] analyze the design of response-time commitments, while our model learns the per-complaint classification decisions that determine whether those commitments are met equitably across neighborhoods.
D’Ignazio and Klein [3] introduce the concept of measurement injustice: the harm that arises when the metrics used to quantify a problem encode the power dynamics that produce it. The Garg et al.program on under-reporting provides a quantitative instantiation of this concept for the municipal complaint context. Complaint counts are not a neutral measure of housing risk but a measure of complaint-filing capacity, which correlates strongly with income, language access, and trust in government institutions [2]. NYC’s 311 system has well-documented inequities in complaint rates and response assignment times across demographic lines [1]. Our use of SHAP to audit the influence of neighborhood statistics on learned classification decisions is a direct application of the measurement justice lens: if the model’s most important features correlate with protected attributes, the classification policy’s equity implications must be confronted explicitly.
Multi-objective RL has been studied formally by Roijers et al. [14], who characterize the space of Pareto-optimal policies and the conditions under which scalarized reward functions can recover them. Policy-gradient methods such as REINFORCE [15] and their variance-reduction extensions via learned baselines have been applied to sequential resource-allocation problems in public health [16] and disaster response [17]. DQN [18] has been applied to inspection and maintenance scheduling, though typically without explicit equity constraints. The closest antecedent is the literature on restless multi-armed bandits for health intervention scheduling [16], which has begun to incorporate fairness constraints; we extend this orientation to the complaint classification domain.
To the best of our knowledge, no prior work (i) applies RL to multi-domain municipal complaint classification with equity as a primary design objective, (ii) conducts a formal demographic equity audit of learned classification policies disaggregated by both income and racial composition, (iii) characterizes Pareto frontiers that explicitly include equitable classification coverage alongside operational throughput, or (iv) grounds reward function design in a participatory process involving affected community stakeholders. The present paper addresses each of these gaps, building on Garg et al.’s empirical documentation of complaint-rate disparities [2], [12] and Liu and Garg’s complementary analysis of SLA design [13], and situating the RL framework in Estrin’s participatory sensing lineage [4], [11] to account for the structural inequalities in civic data collection that any classification system must correct rather than inherit.
The New York City Department of Buildings enforces the NYC Construction Codes and Zoning Resolution across approximately one million buildings. Before any inspector is dispatched, each incoming complaint must be classified: assigned a priority tier, an action category, and a routing queue. This classification step is chronically under-resourced. Current intake relies primarily on complaint severity codes and first-in-first-out queue ordering, with limited use of historical outcome data. The result is a throughput bottleneck and a systematic misprioritization of complaints from neighborhoods with noisier, lower-volume, or non-English intake records.
Boiler Safety. Elderly residents and young children in low-income multifamily housing face the greatest exposure to carbon monoxide poisoning and heating failure. Misclassifying a boiler complaint as non-urgent delays the only intervention pathway available to these residents.
Crane/Derrick. Construction workers are disproportionately immigrants and workers of color. They bear the physical risk of crane incidents. Misrouted complaints translate directly to unaddressed worksite hazards.
Heat and Hot Water. Renters in neighborhoods with older housing stock, concentrated in the South Bronx, Harlem, and Central Brooklyn, face chronic heat failures. Classification delays in winter months are directly harmful to health.
Housing Complaints. Low-income renters with limited mobility face the highest risk of displacement when housing violations persist unresolved due to misrouting.
Scaffold Safety. Pedestrians and residents in dense urban neighborhoods are primary risk-bearers when scaffold complaints are batched rather than escalated.
SNAD. Ecologically sensitive neighborhoods often adjoin low-income communities whose access to green space is a function of classification quality.
Sourced from the NYC DOB Safety Boiler dataset (834,338 records, \(\approx\)8% defect rate). The agent observes a seven-feature vector—is_recurrent, is_high_pressure,
neighborhood_risk, is_internal, has_lff_45, has_lff_180, and boiler_count_norm—and classifies each complaint as Defer or Inspect.
Split 80/20 (\(\approx\)667K train, \(\approx\)167K test).
The classifier assigns each complaint to Do Nothing, Routine Inspection, Immediate Inspection, or Stop-Work Order. Five state features: new complaint flag, recurrence flag, neighborhood complaint frequency, backlog, and hazard level. 80/20 chronological split.
A simulated agency classifies up to four complaints per day for inspector dispatch. The environment resolves each assignment into one of five outcome categories using empirically derived probabilities.
NYC 311 housing complaints aggregated at an hourly level, with state features including backlog, average close time, lagged complaint counts, rolling averages, and recurrence ratios.
Hourly transitions at the entity-identity level. Five engineered state features: created_count, open_count, recurrent_7d, zip_freq_24h, and time_since_last_resolution_h. Five action
categories: Inspect Now, Delay, Batch, Escalate, Ignore. Temporal split: train January 2020–2024,
validation 2024–2025, backtest 2025–March 2026.
Parks Department records over 128 states (4 neighborhoods \(\times\) 4 complaint levels \(\times\) 2 recurrence flags \(\times\) 4 seasons) with three actions: Defer, Inspect, Prioritise. Evaluated on 87 historical complaints filed in 2024–2026.
103,598 records across 59 Community Boards (2020–2025). Approximately 60% of cases close with no action; only \(\approx\)20% lead to enforcement, making correct classification of actionable complaints the primary capacity challenge.
For each domain with residential implications (boiler, heat, housing), we augment complaint and outcome records with American Community Survey (ACS) variables at the census-tract level: median household income quintile, racial and ethnic composition, housing age, and renter-occupied housing share. Following Agostini, Pierson, and Garg [12], we additionally use ACS language-access and educational-attainment variables as proxies for neighborhood-level reporting propensity, enabling a first-order correction to the complaint-volume normalization in our equity reward term.
Three structural limitations motivate our feedback-loop analysis in Section 7. First, classification outcome records are missing not at random: complaints in historically underserved areas have systematically fewer recorded defect outcomes. Second, 311 complaint rates vary with digital literacy, language access, and trust in government—disparities that Liu, Bhandaram, and Garg [2] quantify precisely for the NYC Parks complaint system. Third, temporal gaps in records create state-estimation uncertainty that the MDP formulation must accommodate.
We model each complaint classification domain as a discrete-time Markov Decision Process \(\mathcal{M} = (\mathcal{S}, \mathcal{A}, P, R, \gamma)\), where \(\mathcal{S}\) encodes the complaint queue, neighborhood statistics, and capacity indicators; \(\mathcal{A}\) is the set of classification categories; \(P\) is transition dynamics estimated from historical data; \(R\) encodes multi-objective trade-offs; and \(\gamma \in (0,1)\) is the domain-specific discount factor. The agent maximizes: \[\pi^* = \arg\max_{\pi}\, \mathbb{E}_\pi\!\left[\sum_{t=0}^{\infty} \gamma^t R(s_t, a_t)\right]. \label{eq:objective}\tag{1}\]
Procedural equity requires that escalation capacity be allocated in proportion to housing need, not to complaint-filing capacity. [2] showed that the complaint volume is a function of digital literacy, language access, and institutional trust, all of which correlate with income and race. A throughput-maximizing classifier is procedurally inequitable even when it treats each individual complaint identically. Substantive equity additionally requires that outcomes do not systematically differ across demographic groups. Our revised equity objective operationalizes both requirements by correcting for heterogeneous reporting propensity before normalizing escalation throughput.
Following [19], we treat equitable coverage as a co-equal reward objective rather than a side constraint. [13] showed empirically that for municipal service allocation, the price of equity is small and that inefficient policies tend to be inequitable; this makes the Pareto frontier between efficiency and equity less adversarial than conventional formulations imply.
Before defining the equity reward term, we introduce the reporting-rate correction that must be applied to complaint-volume denominators. Raw complaint counts \(C_{g,t}\) in demographic group \(g\) at time \(t\) conflate true incident prevalence with filing propensity. We separate the two using the duplicate-report estimator of [2] and the Bayesian spatial model of [12].
Let \(\mathcal{G} = \{\text{low}, \text{high}\}\) index two neighborhood strata defined by census-tract income quintile (quintiles 1–2 vs.–5). For each census tract \(z\) and time window \(t\):
\(C_{z,t}\) : observed complaint count (raw 311 record).
\(D_{z,t}\) : number of duplicate reports referencing an incident already logged in the same tract-week.
\(\rho_{z,t}\) : reporting propensity (probability that a resident files a complaint given an incident occurs).
\(\widehat{I}_{z,t}\) : estimated latent incident count (true need proxy).
Following [2], the duplicate-report rate identifies \(\rho_{z,t}\) without external ground-truth data. Under the assumption that the first and subsequent reports of the same incident arrive independently at rate \(\rho_{z,t}\), the expected fraction of duplicate reports satisfies \[\label{eq:dup-rate} \frac{D_{z,t}}{C_{z,t}} \;=\; 1 - \frac{1}{\mathbb{E}[\text{reports per incident}]} \;=\; 1 - \rho_{z,t},\tag{2}\] giving the moment estimator \[\label{eq:rho-hat} \hat{\rho}_{z,t} \;=\; 1 - \frac{D_{z,t}}{C_{z,t}}.\tag{3}\] When duplicate-report records are unavailable, \(\hat{\rho}_{z,t}\) is proxied by a linear combination of ACS covariates (median household income, share with college attainment, share of English-proficient residents, renter-occupied housing share) fit on tracts where duplicates are observed: \[\label{eq:rho-proxy} \hat{\rho}_{z,t} \;=\; \sigma\!\left(\beta_0 + \boldsymbol{\beta}^\top \mathbf{x}_z \right),\tag{4}\] where \(\sigma(\cdot)\) is the logistic function and \(\mathbf{x}_z\) is the tract-level ACS covariate vector.
The corrected incident-count estimate is \[\label{eq:I-hat} \widehat{I}_{z,t} \;=\; \frac{C_{z,t}}{\hat{\rho}_{z,t}},\tag{5}\] consistent with [12], who show that higher-income tracts report at systematically higher rates, so that \(\widehat{I}_{z,t} > C_{z,t}\) selectively for low-income tracts.
For each stratum \(g \in \mathcal{G}\), aggregate over member tracts \(\mathcal{Z}_g\): \[\label{eq:N-hat} \widehat{N}_{g,t} \;=\; \sum_{z \in \mathcal{Z}_g} \widehat{I}_{z,t}.\tag{6}\] \(\widehat{N}_{g,t}\) replaces the raw complaint count \(N_g\) in the equity reward denominator (see Section 4.4).
The full reward at step \(t\) is \[\label{eq:reward-full} r_t \;=\; \alpha_1 \cdot r_t^{\text{speed}} - \alpha_2 \cdot r_t^{\text{cost}} + \alpha_3 \cdot r_t^{\text{equity}} + \alpha_4 \cdot r_t^{\text{retention}},\tag{7}\] with \(\alpha_i \geq 0\) and \(\sum_i \alpha_i = 1\). The speed, cost, and retention terms are unchanged from Section 4.3 of the original formulation. The equity term is revised as follows.
The formulation in Equation (6) of the submitted draft normalises escalation throughput by observed complaint counts: \[\label{eq:equity-biased} r_t^{\text{equity, biased}} \;=\; -\left| \frac{n_t^{\text{high}}}{N^{\text{high}}} - \frac{n_t^{\text{low}}}{N^{\text{low}}} \right|,\tag{8}\] where \(n_t^{g}\) is the number of correct escalations assigned to stratum \(g\) at step \(t\) and \(N^{g}\) is the total complaint count for that stratum. Because \(N^{\text{low}} < \widehat{N}^{\text{low}}\), this term overstates the per-incident escalation rate in low-income tracts, masking under-service.
Replacing raw counts with corrected incident estimates yields: \[\label{eq:equity-corrected} \boxed{ r_t^{\text{equity}} \;=\; -\left| \frac{n_t^{\text{high}}}{\widehat{N}_{t}^{\text{high}}} - \frac{n_t^{\text{low}}}{\widehat{N}_{t}^{\text{low}}} \right| }\tag{9}\] where \(\widehat{N}_{t}^{g}\) is computed via Equations 3 –6 on a rolling window of width \(\Delta t\) (default: four weeks) ending at step \(t\).
\(r_t^{\text{equity}} = 0\) when correct escalations are allocated in proportion to estimated housing need, not to complaint volume. An agent maximizing Equation 7 with \(\alpha_3 > 0\) is therefore incentivized to escalate proportionally more complaints from low-reporting tracts, directly counteracting the participation gap documented by [2].
For an audit across \(K > 2\) income quintiles or racial-composition strata \(\{g_1, \ldots, g_K\}\), the pairwise absolute difference is replaced by the maximum disparity: \[\label{eq:equity-multigroup} r_t^{\text{equity}} \;=\; -\max_{i \neq j} \left| \frac{n_t^{g_i}}{\widehat{N}_{t}^{g_i}} - \frac{n_t^{g_j}}{\widehat{N}_{t}^{g_j}} \right|,\tag{10}\] or, when a smoother gradient signal is preferred during training, the population-weighted variance: \[\label{eq:equity-variance} r_t^{\text{equity}} \;=\; -\operatorname{Var}_{g}\! \left( \frac{n_t^{g}}{\widehat{N}_{t}^{g}} \right) \;=\; -\frac{1}{K}\sum_{k=1}^{K} \left( \frac{n_t^{g_k}}{\widehat{N}_{t}^{g_k}} - \bar{e}_t \right)^{\!2},\tag{11}\] where \(\bar{e}_t = \frac{1}{K}\sum_k n_t^{g_k}/\widehat{N}_t^{g_k}\) is the mean per-incident escalation rate across strata.
Before each training cycle, the following preprocessing pipeline is applied:
Compute \(\hat{\rho}_{z,t}\) using Equation 3 on DOB complaint records for the preceding \(\Delta t\) window; fall back to Equation 4 for tracts with fewer than \(m_{\min} = 10\) duplicate-report observations.
Estimate \(\widehat{I}_{z,t}\) via Equation 5 ; clip at \(C_{z,t}\) to prevent denominator inflation from noisy propensity estimates (\(\hat{\rho}_{z,t} < \rho_{\min} = 0.05\)).
Aggregate to strata via Equation 6 .
Update equity denominators in the replay buffer used for DQN training or the episode initialization used for REINFORCE.
Run demographic audit: if \(\max_{i \neq j}|e_t^{g_i} - e_t^{g_j}| > \tau\) after retraining, flag for manual review and suspend deployment until the disparity falls below threshold \(\tau\) (recommended: \(\tau = 0.05\)).
Because \(\hat{\rho}_{z,t}\) is estimated with sampling error, we propagate uncertainty into the equity reward via parametric bootstrap: for each retraining cycle, draw \(B = 200\) samples of \(\hat{\rho}_{z,t}\) from its asymptotic normal distribution (variance derived from the delta method applied to Equation 3 ), recompute \(\widehat{N}_{t}^{g}\), and report the 5th–95th percentile interval of the resulting equity-reward distribution alongside point estimates.
Two-hidden-layer network (64 or 128 units, ReLU), trained via experience replay and a target network [18]. CQL penalty (\(\alpha=0.1\)) applied in the elevator domain.
\[\nabla_\theta J(\theta) = \mathbb{E}_\pi\!\left[ \nabla_\theta \log\pi_\theta(a_t|s_t) \cdot \bigl(G_t - b(s_t)\bigr)\right]. \label{eq:reinforce}\tag{12}\] Joint optimization of policy and value heads with Adam; \(\gamma \in \{0.95, 0.99\}\) depending on domain [15].
\(2\times256\) hidden units; selects complaint category and then a specific complaint. It does not outperform a flat random policy in the heat domain when outcome probabilities are homogeneous.
\(Q(s,a) \leftarrow Q(s,a) + \alpha[r + \gamma \max_{a'} Q(s',a') - Q(s,a)]\), with \(\alpha=0.1\), \(\gamma=0.95\), \(\epsilon\)-greedy decay \(1.0 \to 0.05\).
Most common logged action per discretized state; it estimates the historical classification policy.
We apply SHAP [20] via KernelSHAP (boiler), GradientExplainer (crane/derrick), exact Shapley values (SNAD), TreeExplainer (heat, housing), and permutation-based values (drone, electrical revisit). Geographic features such as borough, Community Board, and ZIP-code frequency are flagged as equity-sensitive, consistent with Agostini, Pierson, and Garg [12]: because higher-income neighborhoods both report more and receive more inspections, geographic variables in models trained on historical records will tend to encode socioeconomic status as much as genuine incident prevalence.
Balanced (current NYC DOB heuristic), Tail, Emerg, Actionable, Uniform, No_Dispatch, Random Policy, Always Inspect / Always Defer, and rule-based domain-expert heuristics.
Strict chronological split: train on 2020–2024, evaluate on 2025. Boiler domain uses an 80/20 random split of 834,338 records. Elevator evaluation: 168-step horizon (\(\approx\)4 weeks at 4-hour cadence), frozen five-seed benchmark. Heat REINFORCE: 20 training episodes, 365-timestep test year. Scaffold and elevator: support-constrained backtest (\(\texttt{support\_min\_count}=1\)).
REINFORCE accumulates cumulative reward \(\approx\)8,800 over 365 timesteps vs.\(\approx\)2,100 for HDQN and random (\(4\times\) improvement). REINFORCE closes \(\approx\)101 violations by day 365 vs. (HDQN) and 80 (random)—a 20–27% improvement.
Table 1 reports test-set performance (167,379 records). DQN outperforms REINFORCE by $1.28M in reward and 4.2 pp in recall. Both adopt an aggressive escalation stance: REINFORCE recall 94.5%, DQN 98.7%. Low precision (\(\approx\)22%) is rational given the 26:1 cost asymmetry.
| Metric | REINFORCE | DQN |
|---|---|---|
| Total Reward | $9,019,490 | $10,294,592 |
| Avg Reward / step | $54.15 | $61.80 |
| Escalations | 53,848 | 59,439 |
| True Positives | 12,441 | 13,001 |
| False Positives | 41,407 | 46,438 |
| True Negatives | 112,002 | 106,971 |
| False Negatives | 729 | 169 |
| Precision | 0.231 | 0.219 |
| Recall | 0.945 | 0.987 |
| F1 Score | 0.371 | 0.358 |
Table 2 reports the full baseline comparison. Balanced remains the strongest practical heuristic (mean reward \(-446\text{k}\)); DQN: \(-459{,}119 \pm 4{,}379\); REINFORCE: \(-463{,}117 \pm 10{,}482\).
| Policy | Reward | \(A_{30}\) | \(S\) | Act.% |
|---|---|---|---|---|
| No_Dispatch | \(-835{,}120\) | 2371.8 | 1305.0 | 0.00 |
| Uniform | \(-552{,}092\) | 1743.2 | 781.0 | 0.16 |
| Random | \(-490{,}276\) | 1642.3 | 649.7 | 0.15 |
| TD | \(-463{,}449\) | 1578.4 | 602.5 | 0.13 |
| Actionable | \(-462{,}878\) | 1422.6 | 678.0 | 0.22 |
| REINFORCE | \(-463{,}117\) | — | — | — |
| DQN | \(-431{,}596\) | 1429.1 | 582.4 | 0.17 |
| Emerg | \(-438{,}688\) | 1583.6 | 525.7 | 0.15 |
| Tail | \(-420{,}387\) | 1403.5 | 562.1 | 0.14 |
| Balanced | \(-413{,}777\) | 1423.2 | 532.7 | 0.15 |
DQN (\(+1.046\)) and REINFORCE with baseline (\(+1.011\)) are nearly identical; both classify 85 of 87 complaints actionably and secure 7 enforcements, outperforming all heuristics.
Boiler grid search over 128 reward configurations shows the detection rate monotonically increasing from 0.71 (\(\rho=5\)) to 0.94 (\(\rho=50\)). Heat domain Pareto analysis identifies Accuracy+Fairness as the most practically attractive configuration. Cross-domain Pareto analysis confirms that the current heuristic intake is dominated by RL policies in all residential domains—consistent with Liu and Garg’s [13] finding that the status quo NYC inspection scheduling is dominated on both efficiency and equity simultaneously.
The current DOB heuristic exhibits a statistically significant gradient in escalation-assignment rates across income quintiles (Q1 receives fewer correct escalations per complaint than Q5, even after conditioning on complaint severity) and by racial composition (tracts with higher fractions of Black and Hispanic residents receive fewer escalations per complaint). REINFORCE with \(\alpha_3 > 0\) approaches parity across income quintiles at a throughput cost of \(\approx\)4–7%—corroborating Liu and Garg’s [13] theoretical result that the price of equity is bounded and modest in realistic municipal settings. DQN without the equity term narrows but does not close the gap.
Community Board 12 Manhattan (Washington Heights / Inwood; predominantly Latino, lower-income) ranks among the top five most influential SHAP features in the REINFORCE elevator policy, driven by high unresolved burden and historical actionability. This prominence illustrates the concern raised by Agostini, Pierson, and Garg [12]: CB-level features encode demographic characteristics as much as incident prevalence, requiring ongoing audit.
Outcome records are generated by the incumbent classification policy: complaints in areas that received fewer escalations have sparser defect records, not because they are safer, but because they were never escalated. This missing-not-at-random structure is an instance of the participation-gap problem that Estrin’s participatory sensing framework [4] identified as a structural risk in any civic sensing system that conflates coverage with participation. Small data [11] amplifies the concern: digital traces are densest for those who already participate the most, creating a compounding bias in historical training data.
Liu, Bhandaram, and Garg [2] provide a principled method to quantify this gap without external ground-truth data. Their duplicate-report estimator applied to DOB complaint records could identify neighborhoods where complaint rates are suppressed relative to true incident rates, enabling a reporting-rate correction to the complaint-volume normalization in Equation 9 . We flag this as the highest-priority extension of the present work. Sensitivity analysis via propensity-score reweighting [21] shows that the RL performance advantage is robust to moderate selection bias; the equity gains are more sensitive.
Once deployed, the classification policy generates the outcome records used to retrain future model versions, a feedback loop that can progressively shrink coverage in areas receiving the fewest escalations. We propose three mitigations: (1) exploration bonuses for census tracts with sparse recent escalation histories; (2) periodic counterfactual escalation of a random sample of low-priority complaints; (3) regular demographic audit checkpoints triggering retraining if disparity metrics exceed a pre-specified threshold.
Liu and Garg’s SLA framework [13] offers a complementary institutional lever: redesigning response-time commitments to require minimum service rates in low-income tracts provides a structural backstop against feedback-loop- driven under-service that does not depend on the RL agent’s reward signal alone.
Results should be interpreted as decision-support prototypes. The complaint triage domain’s learned classifiers fail to transfer to held-out 2020 data, arguing against deployment without prospective operational validation.
Input from NYC DOB intake operations staff directly informed MDP state and action space design. Practitioners identified the distinction between "classify now," "batch," and "escalate" as operationally meaningful.
We engaged tenant advocacy organizations in the South Bronx and Central Brooklyn. Key themes included frustration with complaint acknowledgment without visible follow-up and skepticism about opaque algorithmic systems. Estrin’s Public Interest Technology framework and participatory sensing research [4], [11] inform our position that affected communities should be active co-designers of civic algorithmic systems. These perspectives informed our inclusion of housing unit retention as an explicit objective and our recommendation for interpretable classification explanations available to complainants.
Human-in-the-loop oversight. RL outputs are recommendations to human intake staff, not autonomous decisions.
Ongoing demographic audit. Classification disparity metrics disaggregated by income quintile and racial composition computed monthly; pre-specified thresholds for manual review.
Under-reporting correction. Apply the Liu, Bhandaram, and Garg [2] duplicate-report estimator to DOB data to correct neighborhood-level under-reporting in the equity parity term denominator before each retraining cycle.
SLA alignment. Coordinate per-complaint classification decisions with the Liu and Garg [13] SLA optimization framework to ensure consistency with jurisdiction-wide response-time equity commitments.
Data governance. 311 data use must comply with NYC open data terms; ACS covariate linkage at the census-tract level.
Rollback conditions. Suspend deployment if (a) demographic disparity exceeds baseline by a pre-specified margin; (b) defect detection falls below current policy over a rolling 60-day window; or (c) staff friction reaches a threshold set by dispatcher survey.
Reward transparency. Reward configurations in production documented publicly as statements of normative policy priority, subject to democratic review.
First, equity does not come for free. Agents trained without explicit equity objectives narrow but do not eliminate historical classification disparities. Closing the gap requires the inclusion of the coverage parity term with \(\alpha_3 > 0\).
Second, the equity cost is modest and quantifiable. Agents with full equity objectives achieve near-parity at a throughput penalty of approximately 4–7%, consistent with Liu and Garg’s [13] theoretical bound on the price of equity and providing RL-specific empirical confirmation that the Pareto frontier between efficiency and equity is less adversarial than the framing of equity-as-constraint implies.
Third, no single algorithm dominates. REINFORCE outperforms in the heat domain and water drainage; DQN produces more stable policies in elevator, complaint triage, and ERT dispatch. Both methods independently converge to the identical Brooklyn-dominant policy in the drone domain.
Fourth, classification quality is bounded by data quality. The participation-gap literature [2], [4], [11], [12] makes clear that 311 complaint data reflects who participates, not only what happens. Future work must integrate upstream reporting-rate correction to ensure that reward signals calibrated on complaint volume reflect true incident prevalence rather than filing propensity.
The Pareto frontier provides a decision-support tool: a family of policies along the efficient frontier, each corresponding to a different weighting of throughput, cost, and equity. The choice of where to operate is a normative one that belongs to the agency and the communities it serves. Liu and Garg’s [13] result is parallel to ours: the status quo NYC inspection scheduling is dominated by both efficiency and equity simultaneously.
The MDP framework is domain-agnostic and can be re-instantiated for any municipal classification domain with complaint intake, action routing, and binary outcome records. The Garg et al.under-reporting correction tools have already been applied to both NYC and Chicago 311 data [2], suggesting that the data pipeline prerequisites for equitable classification exist at scale beyond New York.
This paper begins and ends with a distributional claim: the harms that municipal complaint classification systems are designed to prevent are not randomly distributed, and neither are the benefits of the algorithmic systems deployed to improve classification capacity. Any expansion of that capacity that does not explicitly ask who benefits and who is left in a lower-priority queue is not neutral. It is a choice to accept and perpetuate the existing distribution.
Our principal finding is that RL systems can close the classification gap, but only when equity is inscribed in the reward function as a first-class objective. Agents optimized for throughput alone narrow but do not eliminate historical disparities. Agents with explicit coverage parity objectives achieve near-parity while outperforming current heuristic classification on defect detection and remediation speed. The equity cost is modest and quantifiable, a finding anticipated by Liu and Garg’s [13] theoretical analysis and confirmed here in the RL setting.
Our feedback-loop analysis adds a further imperative: the missing-not-at-random structure of historical complaint records, which Garg and collaborators have characterized precisely for NYC civic data [2], [12], means that classification systems deployed without equity objectives actively degrade equity over successive training rounds. The participatory sensing paradigm [4] that underlies 311-style systems was designed to democratize urban data collection; realizing that aspiration requires correcting for heterogeneous participation rates before those rates are allowed to shape classification decisions.
Configuring reward functions for public-sector RL systems is an act of normative policymaking. It encodes whose harms count most, how competing objectives should be weighted, and what trade-offs are acceptable. These are political and ethical choices that belong to the communities affected by them.
| Parameter | Value |
|---|---|
| Learning rate | \(1\times10^{-3}\) |
| Replay buffer size | 50,000 |
| Batch size | 64 |
| Target network update freq. | 500 steps |
| Discount factor \(\gamma\) | 0.99 |
| Epsilon (initial) | 1.0 |
| Epsilon (final) | 0.05 |
| Epsilon decay steps | 10,000 |
| Hidden layers | \(2\times128\), ReLU |
| Parameter | Elevator | Heat |
|---|---|---|
| Policy lr | \(3\times10^{-4}\) | \(1\times10^{-4}\) |
| Baseline lr | \(1\times10^{-3}\) | \(1\times10^{-4}\) |
| Discount \(\gamma\) | 0.99 | 0.99 |
| Episode length | 168 steps | 365 steps |
| Policy layers | \(2\times128\), ReLU | \(2\times256\), ReLU |
| Baseline layers | \(2\times64\), ReLU | \(2\times256\), ReLU |
| Optimizer | Adam | Adam |
| Seeds | 5 (frozen) | — |
| Domain | State Highlights | Classification Actions |
|---|---|---|
| Boiler Safety | 7 features: recurrent, high-pressure, neighborhood risk, internal, LFF flags, count norm | Escalate (Inspect), Defer |
| Crane/Derrick | Hazard level, backlog, neighborhood freq, new/recurrent flags | Do Nothing, Routine, Immediate, Stop-Work |
| Heat/Hot Water | Building-wide, no-heat flags, unit count | Inspect, Batch, Defer |
| Elevator | CB-level tail pressure, enforcement rate | TAIL, EMERG, ACTIONABLE, BALANCED |
| Housing Triage | Backlog, close time, rolling stats, temporal features | Category selection |
| Scaffold Safety | open_count, created_count, recurrent_7d, zip_freq, time-since-resolution | inspect_now, delay, batch, escalate, ignore |
| SNAD | Ecological index, complaint level, recurrence, season, neighborhood | Dispatch, Defer, Prioritise |
| Drone | Borough position, new-address rate, trailing stats | 5 depot choices |
| Electrical Revisit | Complaint age, denial rate, property history | Close, revisit +1/3/7/14/30d |
| Water Drainage | Recurrence, borough, address-gap timing | Monitor, Inspector, HeavyCrew |
| Complaint Triage | Days since last, problem detail, queue | Ignore, Refer, Court |
| ERT Dispatch | Backlog, new/recurrent complaints, hour | Dispatch 1, 2, or 3 crews |