Machine learning (ML) is rapidly transforming neuroengineering by enabling adaptive encoding and decoding of neural activity in systems that restore or augment human function. In visual prostheses (Fernandez 2018, Ayton et al 2020), deep neural networks have been proposed to translate camera images into electrical stimulation patterns delivered to the retina or cortex (de Ruyter van Steveninck et al 2022, Granley et al 2022a, 2023, Moure et al 2025). These networks implement the inverse of a forward model (Chen et al 2009, Beyeler et al 2019, Granley and Beyeler 2021, Granley et al 2022b, van der Grinten et al 2024), which maps electrical stimulation to predicted neural or perceptual responses. Inverting this mapping yields a stimulus encoder that transforms a desired percept (or its visual proxy) into the per-electrode stimulation patterns expected to elicit it. Learned stimulus encoders are now being explored in early-stage clinical evaluations (Moure et al 2025) and are central to designs for next-generation prosthetic vision systems (Beyeler and Sanchez-Garcia 2022, Grani et al 2022, 2025). Because they would prescribe electrical stimuli in real time, their outputs must adhere to established limits on charge density, instantaneous current, and active electrode count (Park and Han 2018). Ensuring adherence to these constraints is therefore a prerequisite for clinical translation.

However, the safety of stimulus encoder systems remains critically understudied. Typically, firmware or hardware safeguards built into the clinical system are relied upon to clip, rescale, or drop unsafe stimuli (Second Sight 2013). While this prevents immediate harm to the user, it obscures whether the underlying encoder performed safely, and prevents iterative refinement to improve model safety while maintaining performance. The limited existing work in this area has aimed to reduce violations by penalizing unsafe stimuli during training (Küçükoğlu et al 2025), but there remains a lack of tools that clinicians and researchers can use to systematically validate that model-generated stimuli adhere to safety constraints.

Here we introduce an automated stress-testing framework for evaluating the safety of ML-driven neurostimulation. The approach adapts coverage-guided fuzzing (CGF) (Chen et al 2018) to probe encoders for unsafe output regimes. In this setting, fuzzing perturbs input images while monitoring whether the resulting stimulation exceeds predefined limits on charge density, instantaneous current, or the number of active electrodes. Exploration is directed by coverage signals that quantify how broadly the perturbations probe the encoder’s output space and its proximity to safety boundaries, enabling the systematic discovery of rare but clinically meaningful failure modes. CGF, in brief, tracks which perturbed inputs cause new behavior and emphasizes those for further mutation, pushing exploration of new behaviors of the model. Figure 1 shows an overview of our CGF framework applied to an ML-driven neurostimulation encoder.

Figure 1. Overview of our framework for discovering safety violations in ML-driven neurostimulation. Top: Visual prostheses use deep nets to convert camera input into electrical stimuli applied to the brain. Outputs must satisfy neurobiological constraints; violations may occur even under normal input. Bottom: Our coverage-guided fuzzer mutates inputs to explore model behavior, using coverage and violation checks to uncover diverse unsafe outputs. The resulting violations enable quantitative safety evaluation and model comparison.

Download figure:

Standard image High-resolution image

To guide this process, we introduce two complementary output-space coverage metrics. The first, Violation-Output K-Multisection Violation Proportion (VO-KMVP), prioritizes tests that push stimulation parameters toward their physiological limits, revealing the inputs that provoke the most severe violations. The second, Violation-Output K-Multisection Output Coverage (VO-KMOC), measures how broadly the test exercises the range of possible stimulation patterns across electrodes, emphasizing the diversity of violation types and spatial distributions. Together, these metrics characterize both the frequency and the breadth of unsafe behaviors.

We demonstrate this framework on state-of-the-art stimulus encoders for retinal and cortical prostheses (Granley et al 2023, van der Grinten et al 2024), which were trained to optimize perceptual fidelity but not explicitly constrained for safety. The stress test uncovers over-limit stimulation patterns that conventional testing does not effectively discover, offering quantitative insights that can guide model selection, retraining, and firmware policy. We then demonstrate how CGF can be used in conjunction with performance metrics to evaluate the safety improvements from different regularization strategies (Küçükoğlu et al 2025), showing its value as a tool to inform model selection and refinement.

Although our experiments focus on artificial vision, the same principles apply to any neural interface where an ML model prescribes electrical stimulation under biophysical constraints, including next-generation deep brain, spinal, and vagus nerve stimulators (Shenoy and Carmena 2014, Okorokova et al 2018, Rao 2019, Drakopoulos and Verhulst 2023). By framing safety evaluation as output-level verification and validation, coverage-guided stress testing offers a generalizable foundation for developing safer and more trustworthy ML-based neurotechnologies.

Our goal is to systematically test whether a trained stimulus encoder ever produces stimulation parameters that exceed established biophysical limits. To do so, we adapt a software testing strategy called CGF (Chen et al 2018) to the domain of neurostimulation. In conventional software testing, fuzzing automatically perturbs program inputs to uncover rare failure modes; here, it perturbs sensory inputs (e.g., images) to expose conditions under which an ML encoder produces unsafe stimulation. This allows the model to be evaluated in a black-box fashion (i.e. no internal weights or gradients are needed) and complements the usual forward simulations or loss-based analyses used in model development.

We focus on regression models that map sensory input (where d indicates the number of dimensions in the input) to a vector of stimulation parameters . In a neurostimulation system with electrodes, the model output can be expressed as , where fi denotes pulse frequency, pi the pulse duration, and ai the amplitude of a biphasic square-wave pulse train delivered by electrode i. Safety is characterized by a set of inequality constraints , each corresponding to a physiological limit (e.g., maximum charge density, instantaneous current, or co-activation area). An input x constitutes a violation input if its output violates at least one constraint.

Formally, we aim to discover a large and diverse set of inputs

subject to the domain-specific constraints . Each Vk may apply globally (e.g. total current across all electrodes) or locally (e.g. per-electrode charge density), as detailed in section 2.1.

Because a single violation type can dominate the search, we guide exploration using coverage metrics that favor both the discovery of new violations and the diversification of test cases (section 2.3). This balance ensures that the framework not only maximizes the number of unsafe cases found but also explores the search space, providing actionable insight for model redesign or retraining.

2.1. Safety constraints for electrode-based neurostimulation

Electrical stimulation delivered through implanted electrodes must obey strict biophysical limits to prevent tissue damage and patient discomfort. Typical devices control three parameters per electrode (i.e. the pulse frequency fi, duration pi, and amplitude ai of charge-balanced biphasic pulse trains), and safe operation requires that each combination remain within established physiological and device-specific bounds. Our framework treats these limits as formal constraints on the outputs of a model and identifies any violation of them as a potential safety risk.

We categorize violations into two broad types. Aggregate violations occur when a property of the stimulation pattern as a whole exceeds a system-wide limit, such as total instantaneous current across all electrodes. Electrode-wise violations occur when a single channel violates a local constraint, such as charge density or pulse timing. Formally, we express these as inequalities over the model’s output vector y: a configuration is safe when all constraints are satisfied and unsafe when at least one .

Within this schema, we define four clinically motivated safety constraints representative of real retinal and cortical prostheses:

Physically impossible stimulus: Each biphasic pulse must fit within its temporal period defined by its frequency fi (Hz). When the pulse duration pi (ms) becomes too long to complete a full cycle, the pulse is physically infeasible: Charge density limit: To avoid electrochemical damage at the electrode-tissue interface, the delivered charge per electrode must remain below a device-specific limit. For epiretinal implants such as the Argus II, this limit is specified in the surgical manual (Second Sight 2013) as a per-electrode maximum charge (derived from the FDA charge-density threshold and the device’s electrode geometry). Accordingly, we treat the product of pulse duration pi and amplitude ai as a per-electrode charge quantity that must not exceed the published limit ε1: where a positive value indicates a violation.Instantaneous current limit: The total instantaneous current across all electrodes must stay below a device-level ceiling ε2 (µA), ensuring hardware stability and avoiding unintended current spread: Active electrode limit: The number of simultaneously active electrodes must remain below ε3 to minimize crosstalk and power consumption (here denotes the Iverson bracket, equal to 1 if the condition inside is true and 0 otherwise):

In all cases here, a positive value (V > 0) denotes a violation. The specific values used for ε1, ε2, and ε3 were derived separately for retinal and cortical prostheses using published literature and FDA specifications in conjunction with consultation with clinical experts (Second Sight 2013, Fernández and Normann 2016, Fernandez 2018, Chen et al 2020, Fernández et al 2021, U.S. Food and Drug Administration n.d.).

Although these expressions are taken from visual prosthesis designs, the same formulation applies to any electrode-based neurotechnology (e.g., cochlear, spinal, deep brain, or vagus nerve stimulators) where continuous control of amplitude, frequency, and pulse width must remain within safe biophysical limits (McCreery et al 1990, Shannon 1992, Grill and Mortimer 1995, Cameron 2004). Defining safety directly in terms of model outputs allows our framework to evaluate encoder models in a black-box manner, independent of input type or behavioral context.

2.2. Coverage-Guided Fuzzing (CGF)

In traditional software testing, fuzzing repeatedly perturbs program inputs to uncover rare failures such as crashes. Here, CGF serves as an automated stress test: the algorithm perturbs sensory inputs (e.g. camera images), observes the resulting stimulation patterns, and records any cases that violate the safety constraints defined in section 2.1. This process requires no access to model internals, making it well-suited for validation of proprietary or closed-source encoders.

A coverage function quantifies how much of the model’s behavioral space has been explored by a set of test inputs T. Coverage can be based on different signals (e.g. internal activations, output statistics, violation distributions; detailed in section 2.3), but the goal is the same: higher coverage means a broader sampling of possible model behavior. The fuzzer begins with a seed set S of initial test images which are iteratively mutated to produce new tests. A new test input is added to S only if it increases coverage, that is, when . In this way, the algorithm automatically steers exploration toward novel and potentially unsafe regions of model behavior.

2.2.1. Fuzzing strategy

The high-level procedure is summarized in algorithm 1. Before fuzzing, an optional preprocessing step estimates the expected range of input or output values, if required by the coverage metric. The algorithm then enters an iterative loop: it selects seed images, applies random perturbations (‘mutations’), evaluates the resulting model outputs, and updates both the coverage and the list of discovered violations as necessary.

Algorithm 1. Fuzz(S, P)Performs fuzzing on the model to detect safety violations.Calls function PreProcess() which computes expected ranges for nodes or output values if required by the coverage metric, function Cov() which returns the model coverage as a value between 0 and 1, and function TestMutants() which is described in Algorithm 2.Input: S: seed set and P: optional set of input data for pre-processing.Output: : set of violating inputs.1: Computed values are stored globally2: 3: 4: 5: while do6:   Generates and tests m mutants, Algorithm 27:  8: end while9: return 2.2.2. Mutation strategy

Each new test input is generated by applying a random image-level transformation to a seed example, following image transformations from and procedures similar to DeepHunter (Xie et al 2019). Transformations include translation, rotation, scaling, shearing, brightness or contrast adjustment, blurring, additive noise, and pixel-level perturbation. At each iteration, the algorithm:

1.  

selects a seed for mutation, weighted by how often it has previously led to new violations (equation (6)),

2.  

applies a random transformation to create and obtain ,

3.  

checks whether violates any safety constraint and, if so, records in ,

4.  

evaluates whether increases coverage; if yes, it is added to the seed set S for further exploration.

This procedure, summarized in algorithm 2, repeats for a fixed number of mutations per seed ( in our experiments), progressively building a diverse collection of unsafe examples while exploring the available search space. We choose m as 10 to allow for reasonable exploration of each chosen seed without overwhelming the seed set with mutations of one seed early in testing. The following equation controls how seeds are weighted for selection at each iteration of the algorithm:

where P(s) is the probability of selecting seed s, g(s) counts its prior selections, γ scales sampling frequency, and prevents any seed from being permanently ignored. This equation is adapted from DeepHunter (Xie et al 2019).

Algorithm 2. TestMutants(S, , C, m)Generates m mutant images from seed s, checks violations, and adds each one to the seed set if coverage is increased.Calls function Choose() which chooses a seed as described in equation (6), function Mutate() which chooses a mutation at random, applies it, and returns the new image, function Cov() which returns the model coverage as a proportion between 0 and 1, and function Violates() which returns a boolean indicating whether or not a test produces a violation.Input: S: seed set and : set of violating inputs found. C: current proportion of coverage using existing tests in S. m: number of mutants to generate.Output: S: new seed set (may be unchanged), : new list of violations (may be unchanged), and C: current proportion of coverage using existing tests in S.  1: s is chosen from S  2: for 1 to m do  3:    4:  if then  5:     6:     7:  end if  8:  if then  9:   10:  end if11: end for12: return 2.3. Coverage metrics

Effective CGF requires a feedback signal that reflects how much of a model’s behavior has been explored. This feedback is called coverage. In conventional software testing, coverage often counts which lines of code were executed by a test set. For neural networks, prior work has used neuron activations as a stand-in for lines of code (Pei et al 2017, Ma et al 2018), but such internal signals often fail to correlate with meaningful conclusions about model outputs (Li et al 2019, Dong et al 2020, Yang et al 2022, Huang et al 2024).

In the context of neural stimulation, a good coverage metric should encourage the fuzzer to generate new tests that reveal distinct and physiologically relevant stimulation patterns—those that either approach the boundaries of safe operation or differ meaningfully in their output configuration. Without such a signal, the fuzzer would produce redundant test cases or fail to uncover rare unsafe conditions.

To systematically investigate which coverage strategies best uncover safety violations, we evaluate eleven metrics grouped into three conceptual families:

Basic strategies: simple heuristics that use no coverage signal. They serve as baselines, measuring the effect of naive approaches to utilizing mutations.Neuron coverage metrics: white-box approaches that track how many internal neurons are activated by a test. These methods, adapted from software fuzzing for image classifiers, provide a historical reference but are not aligned with safety outcomes.Violation-focused metrics: new black-box metrics we introduce that operate directly on model inputs and outputs, guiding the search toward diverse and physiologically meaningful safety violations.

Table 1 summarizes all eleven metrics, which are described in detail below. The upper categories list the basic and neuron-based metrics used for comparison, while the lower category presents our six proposed violation-focused metrics.

Table 1. Fuzzing coverage metrics used in this paper.

Basic strategies:B-NMutates user-provided seeds but does not add new tests to the seed set.B-AMutates and adds all new tests to the seed set.B-FRUses fully random images without mutation or a seed set.B-localPerturbs the seed with the most violations locally to generate many similar tests.Neuron coverage metrics (white-box):N-NCActivates neurons exceeding a fixed threshold (Pei et al 2017).N-KMNCPartitions each neuron’s output into K bins and tracks which are activated (Ma et al 2018).N-NBCTracks activations above or below neuron-specific bounds from training data (Ma et al 2018).N-SNACTracks activations that exceed the maximum seen in training data (Ma et al 2018).N-TKNCTracks top-K most activated neurons per layer (Ma et al 2018).Novel violation-focused coverage metrics (black-box):VO-KMVPBins the proportion of violation severity (including no violation) for each constraint.VO-KMOCBins each output dimension across the test set.VO-KMVP-VLike VO-KMVP, but only considers proportions which indicate a violation.VO-VCCTracks which constraints have been violated at least once.I-KMICBins each pixel’s value range across the input.I-Div-approxBins feature space from an autoencoder to approximate test diversity.2.3.1. Design rationale

Our goal is not to invent arbitrary metrics, but to span the most plausible design space for coverage in this domain. We systematically explored metrics that operate in three spaces relevant to an encoder model:

Input space: encouraging diverse sensory inputs,Feature space: encouraging diversity in latent features,Output and violation space: encouraging exploration of stimulation patterns and safety limits.

This principled organization ensures that our proposed metrics cover every meaningful axis along which coverage-guided exploration might improve safety testing.

2.3.2. Violation-focused metrics (our approach)

Among the new metrics, two perform consistently best and form the core of our framework: VO-KMVP and VO-KMOC. Both metrics quantify how much of the model’s output space has been explored in ways that are relevant to safety—either by testing the severity of constraint violations (VO-KMVP) or the diversity of stimulation outputs (VO-KMOC).

VO-KMVP quantifies how thoroughly the tests explore the range of each safety constraint. For a given output vector y, each safety constraint Vk can be expressed as an inequality , where is a biophysical quantity of interest (for example, charge density or total current) and c is the physiological limit of that quantity. The ratio is therefore a dimensionless violation proportion: values below 1 correspond to safe stimulation, while values at or above 1 indicate a violation.

Two types of constraints are considered (see section 2.1): aggregate constraints that depend on all electrodes jointly and electrode-wise constraints that apply separately to each electrode . For each type, the violation proportions are divided into K equal-width bins over a range , and a bin is considered ‘covered’ once at least one test has produced a value in that bin’s range. The coverage of a test set S is then defined as

where

Here denotes the Iverson bracket (equal to 1 if the condition inside is true and 0 otherwise), and defines the lower edge of bin k. Values below the minimum or above the maximum are assigned to the outermost bins. Intuitively, this metric rewards new tests that drive stimulation parameters closer to the safety boundary, helping the fuzzer find the most severe violations.

VO-KMOC complements VO-KMVP by measuring how broadly the tests explore the model’s output space, irrespective of whether they cause violations. Let denote the set of output dimensions (e.g. all electrode amplitudes, frequencies, and pulse widths). For each output dimension