CHEM_SOLVER_NOTES September 30, 2004 This release contains several revisions, additions, and deletions to the array of gas-phase chemistry solvers available in the CMAQ Chemical Transport Model (CCTM). The purposes of these notes are to describe these changes and to provide some supplemental information on the relative computational performance and numerical accuracy of these solvers. 1. Correction to ebi_cb4 solver. A correction to the production of HO2 from the reaction of OH with OPEN was made to the Euler backwards iterative solver (EBI) described by Hertel et al., (1993) for the CB4 family of mechanisms. The change should have little if any effect on model predictions or computational efficiency in most situations. 2. Addition of the ROS3 solver. A new Rosenbrock solver has been added to the array of gas-phase chemistry solvers available in the CCTM. The CMAQ ROS3 solver is based on the ROS3 solver described in Sandu et al. (1997), and is a completely generalized solver (i.e., it is not mechanism specific). In-house tests indicate that ROS3 is faster than the CMAQ QSSA and SMVGEAR solvers when comparable effective tolerances are used, and it is also significantly more accurate than the CMAQ QSSA solver. However, it is not as fast as the EBI solver originally developed by Hertel et al. (1993) or the modified Euler backward iterative (MEBI) version described by Huang and Chang (2001). The ROS3 solver is accessed in a manner completely analogous to the other CMAQ solvers, and can be used with any of the CMAQ chemical mechanisms. See also the notes on blocking size, convergence tolerances and integrated reaction rate analysis below. 3. Discontinuation of QSSA, mebi_cb4 and mebi_saprc99. Since the ROS3 solver is both more accurate and faster than the CMAQ QSSA solver, the latter is not included in the latest release. Similarly, the two MEBI solvers have been dropped since the corresponding ebi_cb4 and ebi_saprc99 solvers produce essentially the same results and are also significantly faster. Note that the MEBI solver for the RADM2 mechanisms has not been discontinued since EBI solvers are currently unavailable for those mechanisms. 4. Blocking factor considerations. Both the SMVGEAR and the ROS3 solvers are implemented using cell blocking to increase vector length ( e.g., see Jacobson and Turco, 1994). In previous releases, the blocking factor was set to 500 to optimize performance on vector-based machines such as the Cray C90. When Linux- based clusters are used, considerable speed-ups can be obtained by reducing the blocking factor. In-house tests have shown that a blocking factor of ~10 is better for machines using Intel Pentium III CPUs, and a blocking factor of ~50 more efficient for machines with Intel Xeon CPUs. With this release, the blocking factor has been changed to 50. Users can experiment to determine optimal settings on their machines by varying the parameter BLKSIZE that is set in the subroutine HGRD_DEFN.F. Please note that a model recompilation is required to invoke a change. Also, the blocking factor does NOT affect computational performance of CMAQ when the EBI or MEBI solver is used since neither one employs blocking. 5. Convergence tolerances. Both the accuracy and speed of the CCTM solvers are affected by the choice of convergence tolerance parameters. In CMAQ, default tolerances have been preset in each solver in attempt to balance accuracy requirements with computational performance. Nevertheless, they can be changed by the user to increase either accuracy or speed. In SMVGEAR and ROS3, convergence tolerances are controlled by a relative tolerance and an absolute tolerance. In SMVGEAR these are set to 1.0E-03 and 1.0E-09 ppm and in ROS3 they are set to 1.0E-03 and 1.0E-07 ppm, respectively. For both solvers, the default values can be overridden by setting the following environment variables in the CMAQ run script: GEAR_RTOL and GEAR_ATOL for SMVGEAR and RB_RTOL and RB_ATOL for ROS3. In the EBI and MEBI solvers, the tolerances are preset species-by- species in the MEBI and EBI HRINIT subroutines. Hence, changes to EBI/MEBI tolerances must be made in the code directly. While the ability to change convergence tolerances is available to the user, it is strongly recommended that caution be used in relaxing the tolerances since inaccurate solutions may result. Similarly, tightening the tolerances unnecessarily can reduce computational efficiency without substantially altering model predictions. 6. Relative computational efficiency. The relative computational efficiency of the different solvers is affected by a number of factors, including the convergence tolerances described previously, blocking factor used (for some solvers), the size of the chemical mechanism, and the computer architecture being used. Hence, it is difficult to characterize the relative accuracy and speed of each solver for all situations. Nevertheless, our experiences have been that the solvers as configured in CMAQ can be ranked in terms of relative efficiency as follows (from slowest to fastest): SMVGEAR, ROS3, MEBI, and EBI, with EBI being significantly faster than the others. In terms of accuracy, MEBI and EBI are somewhat less accurate than the other two, but the largest inaccuracies tend to occur at low concentrations. For most model applications, the EBI solvers should give the best performance. The ROS3 and SMVGEAR solvers can be used to obtain more accurate solutions when computational efficiency is less of an issue, and for applications involving integrated reaction rate analysis (see below). 7. Integrated Reaction Rate (IRR) Analysis. CMAQ includes integrated reaction rate analyses that can be performed to assist the analyst in understanding the underlying reasons for model predictions. In IRR, numerical solutions of the gas-phase chemistry solvers are used to calculate integrated rates of reactions during the model simulations. Special accuracy requirements for these calculations preclude the use of some solvers and may necessitate changes to the convergence tolerances in other cases. Tests have shown that the EBI and MEBI solvers as configured in CMAQ are insufficiently accurate for use in IRR analyses (the inaccuracy here applies to the IRR part, not to the gas-chemistry solution part). Although accuracy could be improved somewhat by adjusting convergence tolerances and reducing the chemistry integration time step, the accompanying loss in computational efficiency would negate any advantage of this solver. Hence, IRR analysis is not implemented in the EBI and MEBI solvers. With respect to ROS3 and SMVGEAR, tests have shown both to be sufficiently accurate for IRR analyses in most cases. However, if IRR results are being computed for fast reacting radicals such as OH and HO2 with the ROS3 solver, then it is recommended that the absolute tolerance be decreased from the default value of 1.0E-07 ppm to 1.0E-09 ppm by setting the environment variable RB_ATOL as described above. The default SMVGEAR convergence tolerances should be sufficient for most applications. References: Hertel O., R. Berkowicz, J. Christensen, and O. Hov (1993) Test of two numerical schemes for use in atmospheric transport-chemistry models, Atmospheric Environment, 27A, 2591- 2611. Huang, H.-C. and J. S. Chang, (2001) On the performance of numerical solvers for a chemistry submodel in three-dimensional air quality models, Part1 Box-model simulations, Journal of Geophysical Research, Vol. 106, No. D17, 20175-20188. Jacobson M. and Turco R. P. (1994) SMVGEAR: A sparse-matrix, vectorized Gear code for atmospheric models. Atmospheric Environment, 28, 273-284. Sandu, A., J. G. Verwer, J. G., Blom, E. J. Spee, G. R. Carmichael, and F. A. Potra (1997), Benchmarking stiff ODE solvers for atmospheric chemistry problems II: Rosenbrock solvers, Atmospheric Environment, 31, 3459-3472.