**4.2 Review criteria**

Each paper was review against the following set of criteria:


Additionally, notes on each paper were taken for improving the discussion.

## **5. Synthesis of research outcomes**

### **5.1 Frequency across review criteria**

**Figure 3** overviews the frequency of the results matching the review criteria. For ease of understanding, review features were cataloged according to the ASM studied, modifications for MBR and AS process improvements were grouped, whether a benchmark framework or a membrane bioreactor was applied, the type of sensibility analysis, namely local or global approaches, together with the focus of the sensitivity analysis as a method introduction or an application of it. Finally, the scope of the article and the use of the sensibility analysis.

The ASM1 is the most common model whose parameters have been assessed via SA, followed by the ASM2d, ASM3, and the ASM3 BioP. However, up to 44% of the articles introduce modifications to ASMs, mainly to include soluble microbial products, extracellular polymeric substances, or two-step nitrification processes. Only 32% outputs were related to the benchmark simulation models, as well as for the

### **Figure 3.**

*Features of the included research outputs. (A) Activated sludge model, (B) model framework, (C) analysis type, (D) paper focus, and (E) scope of the article.*


**Table 2.**

*Summary of study characteristics.*





**Table 3.**

 *Synthesis of the results for each of the included scientific articles.*

membrane bioreactor scheme. Moreover, local, and global approaches seem to be balanced with a slighter preference for and global sensitivity analysis, even of the disadvantages of local approaches. The good news is that 81% of the papers highlighted the applicability of the sensibility analysis for activated sludge modeling. Finally, the general article scopes are also balanced in terms of model validation and calibration, as well as studying operation strategies, while control strategies and design scopes were less studied. Mind that model validation refers to the introduction of a model or method and usually includes calibration of it.

### **5.2 Study characteristics and individual results**

**Table 2** reports in more detail the occurrence of the activated sludge models within the sensitivity analysis field. It is evident that modelers aim for nitrogen removal either with the ASM1 or ASM3. As these models are simpler in all criteria (see **Table 1**), and the fact that most treatment facilities' objective is to remove carbon and nitrogen might be the reason for the imbalance between phosphorus removal models.

An interesting result was that 14 outcomes deal with ASM refinements, hence, demonstrating the versatility of the models to materialize modeler and stakeholders' needs. Also, it is important to notice that most MBR applications were subjected to ASM modifications, as only 2 articles applied MBRs without modeling SMP/EPSs. Moreover, 14 articles used local approaches besides their disadvantages.

For all the outcomes considered, a detailed summary of the review criteria is presented in **Table 3**. Here the sensitivity analysis method, the input factors, the model outputs, and the scenarios evaluated in each study are presented. The most common method for the LSA and the GSAs were the normalized sensitivity index (NSI, 5) and the standardized regression coefficients (SRC, 12). As for the input factors, the biokinetic (Biokyn) and stoichiometric (Stoyk) parameters were assessed together in 27 papers. Moreover, the influent fractions were studied by 12 researchers even of normally occurring fluctuations that increase uncertainty for modeling of the activated sludge process.

All the outputs were related to plant performance, either for discharged effluent quality, operational costs, or process evaluation (e.g., MLSS in the reactor, membrane and settler parameters, nutrient uptake, etc.). It is important to highlight the vast scenarios studied, remarking the applicability of the SAs for activated sludge modeling. Mind that because of the vast number of model parameters under study, discussion about the sensitivity indices results is beyond the scope of this review.

### **6. Discussion**

### **6.1 Activated sludge modeling refinements (No MBR)**

To sum up the modifications for modeling organic colloids and biopolymers, Alikhani et al. [20] improved the ASM1 for assessing the anoxic growth and decay of methylotrophs together with the growth of heterotrophs on methanol (MeOH). Gao et al. [57] introduced an ASM1-SMP-EPS for assessing the interaction between biomass and SMP and EPS kinetics. While Gao et al. [21] assay an ASM3-SMP-EPS. The main findings were the following. The methylotrophic parameters were significantly sensitive (according to the scaled sums of sensitivities), mostly those related to the growth and decay of the latter. For the ASM1-SMP-EPS, the original ASM parameters were more sensitive than the newly introduced ones. While from ASM3-SMP-EPS, 27 model parameters were proven sensitive with positive or

negative effects around the outputs. Nevertheless, all the latter used local approaches (except for one whose method remains unclear) for model validation as well as calibration, Thus, suggesting only a first approach for practitioners about the sensitive parameters, however, these must be ascertained via global analysis.

As for the two-step nitrification (SND) models, Yan et al. [22] introduced an ASM1-SND, while Zhu et al. [37] and Fortela et al. [59] employed the ASM3 model developed by Iacopozzi et al. [19]. The first used NSI for determining that the newly introduced saturation coefficients (introduced) did were proved sensitive for the model outputs. However, only the effect (positive or negative) was reported. Special attention was taken for the rest of the refinements. For example, Zhu et al. [37], were the first (up to authors' knowledge) to use the DGSM method for assessing an IWA model. The coupled the DGSM a pseudo-global covariance matrix for assessing parameter correlation. Their method was effective to prove parameter sensitivity. Also, they demonstrate a significant difference between sensitive parameters for a CSTR and an SBR. Nevertheless, comparison versus other global methods could improve the capabilities of the method. While Fortela et al. [59] developed a method that couples a Morris screening and a principal component analysis for transforming the original data into low-dimensional variables. The method allowed to provide a ranking matrix of the sensitive parameters for all the ASM state variables. Yet, it is important to mind the drawbacks of the Morris method.

### **6.2 Membrane bioreactor**

Mannina et al. [8] introduced the ASM1-SMP coupled with a physical model (cake deposition, deep-bed filtration, membrane resistances) to assess the effect of SMPs in the reduction of membrane permeability together with their foulant influencing features (e.g., hydrophobicity, floc morphology). They used the SRC method as a calibration protocol and found that 25 from the 45 model parameters were sensitive (SRC > 0.2) on the evaluated model outputs, thus, fixing 20 parameters to their original value. These results were useful for assessing start-up strategies of an MBR and found that fouling rate was lower when adding AS inoculum serves as a prefilter for dissolved and colloidal components (SMPs), thus reducing the membrane foulants [48].

All the publications that modified the ASM2d improved the model for including soluble microbial products in an MBR scheme. Like Mannina et al. [8], Cosenza et al. [29, 30] introduced an ASM2d-SMP coupled to a physical model and conducted an SRC for ease of the calibration, resulting in 24 sensitive parameters (SRC > 0.2), hence fixing 55 model parameters. Later, Cosenza et al. [30] compared the modified model between the SRC, Morris screening, and E-FAST methods. The results showed that the SRC method is not recommended as its application is considered out of range (SRC < 0.7). The sensitivity outcomes from Morris screening and E-FAST were similar for COD, SNH4, and SNO3 but for SPO4 and MLSS differ substantially in most sections of the WWTP layout. Additionally, influencing factors were very similar for the SRC and E-FAST. Nevertheless, due to convergence issues, the results between Morris screening and E-FAST protocols presented low similarity either for influencing or non-influencing factors. Finally, it was stated that SRCs are useful as a rapid method for factor prioritization due to the lower computational capacity, while the E-FAST comes with high-quality results as the method measures factor interaction (FI). Then, Cosenza et al. [51] used the E-FAST for the evaluation of sensitive parameters using real wastewater in a pilot plant and divided the influential factors into groups (e.g., COD, SNH4, etc.). Finally, Mannina et al. [55] proposed a phase protocol for assessing the uncertainty of the model for ease of the practitioners.

As for MBR biokinetics with the ASM3, Chen et al. [28] introduced the ASM3- SMP to investigate the model application in the operation of aerobic MBR. The use of the E-FAST allowed determining that 10 parameters were more sensitive within the three SRT scenarios. Mind that only the uncertain parameters were evaluated (32). Also, results indicate a strong FI in all the output, but significantly bigger for COD and MLSS. Suh et al. [50] studied a BSM1-MBR framework for rating membrane fouling using control strategies using the CES-ASM3, a model that couples EPS and SMPs. The use of an LSA highlighted that 7 membrane parameters (from 13 assessed) were more sensitive to affect the overall energy requirements or the system.

Is it important to notice that most practitioners conducted global sensitivity analysis, thus, certified as valid according to Saltelli et al. [9] perception. Even three of them did assess first-order and total sensitivity indices using the E-FAST method, thus, accounting for factor interaction. Mind that most MBR applications are concerned with ASMs refinement. Hence, only two outputs did not modify models. Besides the application of an LSA, special attention must be accounted for the "good MBR operational strategies decision tree" from Dalmau et al. [49].

### **6.3 Benchmark simulation models**

According to Gernaey et al. [6], the purpose of benchmark simulation models (BSM) is to provide a measure of reference relative to the activated sludge performance. These aim to achieve minimum costs, optimum effluent quality, and minimal sludge production by the combined assessment of control and monitoring strategies for identifying faults and optimization opportunities.

To sum up, the most relevant studies using BSMs, Flores-Alsina et al. [43] studied the sensitivity of the most uncertain ASM1 parameters within the BSM1 under different controller scenarios (combinations of oxygen, nitrate, ammonium). Sin et al. [15] use the BSM1 and the SRC for studying the design of WWTP under uncertainty scenarios concerning influent fractionation, model parameters (biokinetics, stoichiometry), plant hydraulics, and mass transfers phenomena, together with combinations of the latter. Benedetti et al. [46] evaluated the sensitivity (SRC) of the BSM2 model parameters, diving then into three categories, operation and design, water line, and sludge-line parameters. While Al et al. [14], up to the authors' knowledge were the first to use the Sobol sensitivity indices, for assessing the BSM2 framework over the most relevant plant-wide performance indicators. Notice that these articles present valid sensitivity indices as the approach was global.

### **6.4 General findings**

After conducting the model simulations and the sensitivity analysis, the previously mentioned authors (as well as the non-mentioned) found clear differences between output variance and that factor prioritization can significantly vary from one alternative to another. This is a compelling fact that practitioners must account for factor sensitivities in the function of the modeling goals to comply with [15].

Once the thorough assessment of the research outcomes concluded, the following aspects capture the authors' attention. (I) NSI, SI, SC, and RSF, seem to follow the same fundamental principles. Thus, the authors recommend using the NSI term for ease of method comparison. (II) Up to the authors' consideration, some researchers conflate local and global sensitivity analysis. Usually, because they provide a sum of the LSAs sensitivity indices, lacks an appropriate justification to be classified as global according to Saltelli et al. [9], or because the SA method is not

clear enough, as in some cases the type of analysis or the method used was unclear. (III) Most of the articles studied the sensitivity of biokinetic and stoichiometric parameters. However, sometimes calibration values can fall under non-plausible ranges. Therefore is it important to conduct appropriate assays like the ones conducted by De Arana-Sarabia et al. [58], who used respirometric systems to assess the oxygen, ammonia, and nitrate uptake rate of the activated sludge of an operating WWTP for model calibration. (IV) Only 38% of the include influent fractions as uncertain parameters. Nevertheless, these as well as temperature, are some of the most significant parameters affecting uncertainty in WWTP [15]. Especially in developing countries where WW composition is usually stronger and exhibits abrupt temporal and spatial fluctuations. Finally, (V) there are barely four articles studying total sensitivity indices (STi) for factor interaction, and in one specific results of the GSA were absent. Consequently, there still knowledge gaps concerning factor interaction.

### **6.5 Local versus global**

Despite the capabilities of the sensibility analysis, modelers tend to rely on local approaches. In accordance with Saltelli et al. [9], local sensitivity analysis have two major drawbacks. First, these are not efficient when dealing with non-linear models, given factor interaction is not accounted for. Second OAT methods leave most of the input space unexplored.

Whenever more and more factor combinations are to be studied, the dimensionality of the input space increases becoming a hypercube. Thereby, LSAs only comprise a small fraction of the input space leaving out important phenomena that could improve systems understanding [9].

It is just enough to look at the biological processes of the ASMs or the behavior of its state variable to prove models' non-linearity. Therefore, LSAs can be classified as a non-valid sensitivity measure unless model linearity is justified. Bear in mind that it is not the authors' intention to disparage previous works concerning LSAs, rather it is to encourage practitioners to consider global approaches due to their advantages over the latter. Consequently, to overcome the latter, global sensitivity analysis are preferred.

### **6.6 Future trends**

Fortunately, most papers focus on the applicability of the sensibility analysis, to investigate the uncertainty of their parameters despite the scenario studied. Consequently, application-focused methods are considered to have a broader impact on modelers and project stakeholders for decision-making [9]. Still, there is room for improvement as most WWTP have their plant configurations, objectives, as well as issues to attend to. It has been proven that sensibility analysis could be used for a wide range of purposes, mostly for calibration, model validation (of ASM refinements), control and operation strategies. Moreover, SAs can be used for project design and reengineering purposes.

An example of it is that future plant development will likely emphasize meeting stringent water quality regulations and resource recovery (water, nutrients, organics, energy) [4]. According to Regmi et al. [4], water resource recovery facilities (WRRF) will need models capable of accounting for stringent waterproduct quality, process performance stability, and operating costs, either for design, operation, or control. Hence, resource recovery will incur on integrating broader frameworks such as watershed models or similar, improved settler models, more phosphorus removal applications, as well as ASM improvements like the

above mentioned. For example, Saagi et al. [56] coupled an urban water system to a BSM1 framework for assessing the influence WWTP and sewer control handles on river quality under different rainfall scenarios. Using a Morris screening (due to the low computational capacity) they found that sewer control handles were more influential for TVOF and OQI. For EQI and SNH3 and DO exceedance both controllers seem to be sensitive, and for OCI the WWTP ones were more influential.

Another significant model improvement was introduced by Ramin et al. [53, 54]. They compared the performance of the Takács 1D settler model and second-order model that include the effect of hydrodynamic features like convection-dispersion phenomena. After conducting the GSA, it was demonstrated that settling parameters are as influential as biokinetics on the assessed outputs. However, the secondorder model seemed to provide more realistic measures, thus, suggesting the model will lead to significantly less variance in model outputs.

Regardless of the modeling application or the use of the GSA, data quality and its abundance result in better experimental designs as it provides information to support decision making [4]. Consequently, due to data abundance, the current and forthcoming increase of computer capacity, and the advancements in data-driven models, more complex models including a large number of uncertain input factors will surge [9]. Therefore, if used effectively and responsibly the sensitivity analysis could improve complex phenomena understanding concerning the activated sludge process, together with decision making.
