EMF deployments by reverse chronological date
Deployment 06Mar2023 (v4.2):
Deployment 27Jun2022 (v4.1):
- Include input and parameter names as comments when building case run scripts
- Add option to copy inputs and parameters when copying a job within a case
- Hide "Compare Dataset Fields" from list of QA programs when creating a new QA step
- Add new QA program "Compare Multiple Datasets"
- Use Apache Ivy for dependency management, update to newer library versions
Deployment 28Feb2022 (v4.0):
- Strengthen password requirements including increasing the minimum length to 12 characters and requiring the password to include numbers or symbols
- Automatically disable accounts after 5 failed login attempts
- Add options for administrators to disable accounts, see failed login attempts, and see when passwords were last updated
Deployment 04Nov2021 (v3.10.1):
- Completed updates to core EMF for better accessibility
- Added matching by region code and facility ID to temporal allocation functionality
- Updated control strategy dataset types to handle longer data fields
- Added equation type 21 for use by non-EGU SO2 control measures
Deployment 29Oct2021 (v3.10):
- Fix bug that prevents editing a case job
Deployment 07June2021 (v3.9.1):
- Enhancements to keyboard navigation and screen reader compatibility
- Improve user interface focus after refreshes are performed
- Add feedback when an action can't be performed due to no item being selected
- Fixes to incorrect tooltips and added button shortcuts
- Added ability to type in more letters to the Show Datasets of Type menu in the Dataset Manager
Deployment 30May2021 (v3.9):
- Temporarily remove accessibility updates to the data viewer and editor tables that cause issues with the column headers scrolling in Java 8
Deployment 08March2021 (v3.8):
- Update design capacity checks in cost equations to check that values are greater than 0.1 MW/hr rather than just greater than 0
- Ensure all design capacity checks in cost equations use consistent units
- Add design capacity check to equation type 2a to be consistent with other equation type 2 variants
- Fix errors running control strategies by truncating SCC descriptions when creating result datasets (update deployed 5/4/21)
- Fix issues with window focus when closing windows
- Enhancements to keyboard navigation and screen reader compatibility
- Change default sizes of windows to better fit contents
- Enable the User's Guide option in the Help menu
- Add shortcut Ctrl+Shift-S to open the Status window
- Automatically open the Dataset Manager window when starting the EMF Client
- Check for spaces in filenames when trying to upload or import files
Deployment 17Aug2020 (v3.7):
- Log inventory sources with >= 100% control and non-zero emissions in the Strategy Messages dataset
- Update calculation of projected inventory percent reduction for add-on controls
- Update check for existing control at >= 100% when considering replacements in PFYI strategy
- Add equation type 2c based on existing type 2 with design capacity in HP
- Enhancements to keyboard-only navigation
- Label error messages that were previously only indicated by color
Deployment 15May2020 (v3.6):
- Add search options when selecting control programs for Project Future Year Inventory control strategies, and in the Control Program Manager
- Add .csv suffix when exporting various control strategy result datasets
- Fix issue saving edited control measure equation changes
- Fix end condition for least cost narrowing algorithm
- Add option to include dataset_id column when exporting flexible format datasets from control strategies
- Update naming for combined strategy detailed results file to use control strategy name
- Update case job cancelling for jobs that are started locally (not via SSH)
Deployment 23Oct2019 (v3.5):
- Fix a memory issue when creating controlled inventories from Least Cost control strategies
- Clarify the description of replacement control measure handling on the Control Strategy Constraints tab
- Update control strategies to not apply controls to inventory sources with existing control efficiencies over 100%
- Update cost equation type 2b to only apply to sources whose design capacity is less than 1500 HP
- Fix description for cost equation type 1
- Add option to export multiple Strategy Detailed Results in a single file
- Add quick search to the Control Strategy Manager window
- Add quick search to the Dataset Type Manager window
Deployment 23Oct2019 (v3.4):
- Update reference year cost-per-ton storage to use double precision
- Update implicit price deflators, used for cost year conversions, using values from August 2019
- Simplify cost equation type 8 (used for non-EGU PM2.5 controls) and add a new equation type 20 (used for ICI boilers)
- Revisions to the control measure database
- Add additional columns to the Control Strategy Detailed Result Extended dataset type, to contain inventory data used in cost equation calculations
- Fix errors running control strategies on inventories with no sector set
Deployment 25Mar2019 (v3.3):
- New option to export a specific version of a module type
- Updates to support dataset matching when importing modules via the case manager
- Support for exporting and importing composite modules
- New search option in module manager
- Allow modules with optional, unset input datasets to be finalized
Deployment 16Aug2018 (v3.2):
- Now requires Java 8 or higher
- Updated to work with Java 9 and higher, which supports HiDPI displays on Windows
- New version of the EMF Client Installer available to work with Java 9 and higher
- Removed out-of-date control strategy types (Annotate Inventory and Apply Measures In Series)
- Removed fixed 3% interest rate calculations from control strategies
- Relabeled columns in control strategy summaries and worksheets to match FF10 inventory terminology
- Allow control strategies to be run even if the input inventories don't have sectors set
- Module system
- Reduced the maximum length allowed for module dataset and parameter names to avoid truncation when running the module
- Allow required module datasets to be unset
- Added option for admin users to delete module history records
- Reworked how the module history is fetched to speed up viewing and editing modules
- Added option to use module parameters in dataset placeholder names
- Updates to work with Postgres 10
Deployment 17May2018 (v3.1):
- Added ability for admin users to edit dataset type column defintions for Flexible File Format-based dataset types
- Improved error messages displayed when importing datasets
- Added new option to export and import non-composite modules
- When exporting modules, added option to export all the input datasets associated with the selected modules
- Removed dataset types that can't be imported from the Import Window dataset type pull-down menu
- Added a message explaining why a new dataset version can't be created if there are no existing final versions of the dataset
- Created a new Control Strategy Detailed Result dataset type that uses column names consistent with the Flat File 2010 inventory formats
- Added the term "effective" to emission reduction and annual cost per ton column names in the new strategy detailed result dataset type
- Updated all control strategy types to output the new detailed result dataset type
- Created QA step templates for the new detailed result dataset type based on the old dataset type
- Added check to prevent users from trying to create controlled/projected inventories with old strategy detailed results datasets
Deployment 14Feb2018 (v3.0):
- Added a new dataset type named "Facility Closure Extended" for use with Project Future Year Inventory control strategies. This dataset type works like the existing "Plant Closure (CSV)" type but uses current inventory terminology.
- Added support for dataset type keywords when exporting and importing module types and associated dataset types
- Fixed an error when trying to upload a file whose name contains a percent sign
- Fixed the Module Manager window refresh so that module information is reloaded correctly
- Fixed errors when module types are imported, caused by blank dataset type creation and last modified dates
- Fixed errors when deleting modules
- Updated the implicit price deflator values used in cost year conversions for 2014 through 2017
- Added 2017 as an available cost year
- Relabeled the combustion efficiency column in the control measure SCC window for clarity
- Added formatting to numbers in the CMDB export files
- Added formatting to the cost equation parameters in the At-A-Glance report and the control measure view/edit window
- Updated cost equation descriptions
Deployment 24Aug2017 (v3.0b):
- Updated EMF server to require Apache Tomcat 8 or later
- Fixed labeling for columns FUG_WIDTH_XDIM and FUG_LENGTH_YDIM in the Flat File 2010 Point dataset type
- Fixed the Dataset Type Manager window to correctly disable non-admin buttons after a refresh
- Module Tool updates:
- Prevent deletion of datasets that are in use by modules
- Prevent non-admins from removing entire module types or finalized module type versions
- Added option to delete outputs when deleting a module
- Fixed bug running modules when username contains capital letters
- Removed unneeded Module Type Manager window refresh after export
- Fixed blanking of module description text box after save or refresh
- EMF v3.0 will be the first public release of the Module Tool system (will be disabled by default)
- Removed unused source groups
- Purged obsolete point sector control measures from the CMDB
- Removed all existing incremental cost-per-ton values from the CMDB; removed unused control-measure-specific capital recovery factors and discount rates
- Relabeled control strategy "discount rate" to "interest rate"
- Removed unused fields from the Control Measure At-A-Glance report
- Updated control strategy cost columns to use double precision values
- Added formatting to control efficiency values shown in view/edit windows and At-A-Glance reports
- Added columns to the control strategy detailed results dataset to report raw emissions reduction (based on uncontrolled inventory emissions) and associated annual cost-per-ton
- Updated labels on control strategy constraints tab to reflect that values are for effective reductions and cost-per-ton, not raw
- 3/6/2018 Deleted old control measure property records from CMDB
- 3/6/2018 Updated cost equation descriptions
Deployment 24Aug2017 (v2.15):
- Added support for new Module Tool
Deployment 27Jun2017 (v2.14):
- Fixed bug trying to finalize control strategies that use control measures to be purged
- Updated Control Measure Manager cost year menu to cover 2000 to 2016
- Fixed control measure reference search to be case-insensitive
- Fixed At-A-Glance report to handle various numbers of control efficiency records correctly
- Control Measure Database updates:
- Purged obsolete onroad and nonroad control measures, and old temporary measures
- Converted control measures using pollutants PM2_5 and PM10 to PM25-PRI and PM10-PRI
- Removed control efficiency records with old effective dates
- Removed special characters from control measure source group names and property values
- Updated control efficiencies and equipment life values for some NOx measures
- 8/28/2017 Updated control measures using equation types 1, 12, and 13
- 9/7/2017 For several PM measures, removed control efficiency records with effective dates of 1/1/2020 and updated control efficiency values
Deployment 20Mar2017 (v2.13):
- Updated the lists of pollutants in the Control Measure Manager window to only show pollutants actually controlled by measures in the system, instead of all defined pollutants
- Added checks for null cost equation parameters for Equation Type 2 and its variants
- Fixed a bug in the calculations for Equation Type 13 caused by an order of operations error
- Removed special characters (curly quotes, long dashes, and symbols) from control measure names, descriptions, and references
- Added logging for commands run from the RemoteCommand execute method
- Stopped renaming of database tables when a dataset's name is changed
- Fixed a bug when creating a new flexible format dataset type with spaces in the column names; spaces are automatically converted to underscores in the database table definition
Deployment 22Sep2016 (v2.12):
- Improved the display of file format information when viewing or editing a dataset type
- Enhanced the case comparison report to include dataset version differences
- Added the ability to upload local files to the server via the EMF client
- Automatic clean-up of uploaded files after 12 hours
- Changed the control strategy cost year to use a pull-down menu of available years instead of a text field
- Updated cost year conversion factors (implicit price deflators) for 2013 through 2016
- Updated NOx non-EGU control measures based on SRA recommendations
- Miscellaneous CMDB clean-up and fixes
Deployment 15Jun2016 (v2.11.1):
- Bug fixes
- Updated the uniqueness checks when importing control measure efficiency records to take into account the minimum and maximum capacity columns
- Updated the labels used for the maximum cost per ton and maximum annualized cost control strategy constraints to reflect that the constraints are specified in dollars matching the cost year of the control strategy, not a fixed reference year
- Consolidated database connections via the Tomcat-managed connection pool
- Expanded control measure cost equation Type 2 to support new NOx non-EGU control measures
- Updated NOx non-EGU control measures with new cost data, SCC assignments, and cost equations
- Changed the final cost-per-ton calculations in CoST to use the net emissions reduction instead of gross; this change impacts costs when replacement controls are applied
Deployment 06Jun2016 (v2.11):
- Bug fixes
- Fix checks for design capacity columns in inventories when applying new generating capacity thresholds; allows control strategies using nonpoint inventories to run
Deployment 28Jan2016 (v2.10):
- Bug fixes
- Fixed a bug when editing cases where changes after an initial save weren't saved to the database
- Fixed a bug that caused multi-pollutant control strategies to fail in some circumstances
- Fixed issues in the control measure import interface where the import button wasn't enabled after errors
- The control measure database has been updated based on IPM v5.15 for the ptipm sector. Updates include revised control efficiency and cost equation parameters, removal of obsolete control measures, addition of new measures, and fixes to cost equation calculations. These updates make use of several new features included in EMF v2.11:
- Include the control measure's equation type in the Control Measure Manager details display
- Allow control measures to be targeted to sources based on generating capacity
- Use SCC-specific combustion efficiency percentages when calculating generating capacity from design capacity
Deployment 09Sept2015 (v2.9.1):
- Bug fixes
- Cost-per-ton values displayed in the Control Measure Manager window when Show Details is checked are now correctly adjusted for the selected Cost Year.
- Fixed an intermittent problem when running a Least Cost Curve control strategy that caused calculations at different increments to apply maximum controls instead of the specified percent reduction.
- Allow cancelled control strategies to be re-run.
- Conversions of dollar amounts between different years are now done using annual GDP implicit price deflator values. Previously, the EMF used chained GDP values for converting between cost years.
- Added support for cost years 2007 through 2014.
- Updated the default cost year to 2013; previously the default year was 2006.
Deployment 20Jul2015 (v2.9):
- Prevent datasets that are being used as a temporal profile or cross-reference input in a temporal allocation run from being deleted.
- Allow the project associated with a dataset, control strategy, or temporal allocation to be set back to none.
- Fixed a bug when matching inventory sources that could cause newer sources to be excluded from least cost analyses.
- Create the server export folder if needed when exporting control measures for automatic download.
- When deleting control strategies and associated datasets (control strategy results, generated inventories), fully delete the selected datasets. Previously, the datasets would be marked as deleted but not purged.
- Fixed the calculation of per-source percent reductions when creating projected inventories. This bug affected sources that had additive controls applied in the Project Future Year Inventory strategy.
Deployment 24Jun2015 (v2.8.2):
- This release adds a new feature that will allow a user to Archive or Restore a dataset or QA Step result. The archive process will still keep around the meta data on the dataset or QA Step, but the underlying data will be archived and removed from the database. The Restore process will allow the user to bring back the underlying data for the dataset or QA Step.
- The Control Measure Manager search box will now match control measures by name or abbreviation.
- When editing control programs, the list of available dataset types is now limited to those types used by control programs: Allowable Packet, Allowable Packet Extended, Control Packet, Control Packet Extended, Plant Closure (CSV), Projection Packet, and Projection Packet Extended.
Deployment 07May2015 (v2.8.1):
- Fixed a bug where case jobs that had been previously run were automatically run again when the EMF server was restarted
- Fixed an issue when importing datasets with long, nearly identical names that would cause the database table names to not be unique
- Fixed the source matching hierarchy used by the Project Future Year Inventory strategy. This change affects records where a state code is formatted as 5 digits (i.e. 36000 rather than 36). Matching records with an exact county specified are now given priority over the state-level record.
- Added a new control strategy option to only apply control measures whose major pollutant matches the strategy's target pollutant. This option is on by default for newly created strategies.
- Fixed a spurious error message when checking if an inventory has sources matching the strategy's target pollutant
- Updated post-run calculations of the maximum emissions reduction and total cost to take target pollutant matching into account
Deployment 20February2015 (server only):
- Added sector column to FF10 merged inventory format
- Make sure sector column in strategy detailed result reports original inventory sector when processing multiple FF10 inventories
- Added class and major pollutant columns to control measure selector for control strategy editing
- Temporal Allocation
- Fixed bug where run status could get overwritten if temporal allocation was saved while it was running
- Added default indices to temporal cross-reference and profiles datasets
- Run database analysis on intermediate result tables during temporal allocation run
Deployment 19February2015 (v2.8):
- Fixed error exporting datasets from Dataset Manager when not downloading files to local machine
- Fixed error exporting FF10 datasets from cases
- Check syntax of user-entered SQL Filter and UPDATE SET SQL text before running Advanced Find and Replace
- Updated Least Cost algorithm to select control measures with the greatest emissions reduction in a given cost range rather than measures with the lowest marginal cost; allows Least Cost runs to achieve maximum emissions reductions
- Added option to automatically download exported control measure files to local machine
- Added options for which datasets should be deleted when removing control strategies; datasets that can be deleted are controlled inventories and/or strategy results, messages, and summaries
- Added ability to automatically export and download control strategy results from the Control Strategy Manager window
- Added user preference to select which columns are included when exporting FF10 inventories from the CoST interface; users can also specify custom column names and enter a prefix to use when naming the columns
- Temporal Allocation
- Added ability to cancel in-progress temporal allocation runs
- Added messages dataset that logs any sources that didn't match a temporal cross-reference entry or were missing a temporal profile
- Include summary of temporal allocation run in metadata for results datasets
- Fixed issue detecting monthly values in FF10 inventories where some versions of the dataset have monthly values and other versions don't
- Fixed issue calculating episodic totals/averages when processing multiple inventories
- When running Least Cost control strategies with multiple FF10 inventories, the inventories will now be merged into a unified FF10-based dataset. This avoids problems using inventory filters due to column name differences between FF10 and ORL inventories.
- Bug fixes
- Fixed broken data value filter search option in the Advanced Dataset Search.
- Fixed bug regarding hidden columns for different dataset types. Previously, if more than half the columns for a dataset type were hidden, this bug prevented the data from being displayed.
- Refresh local project cache when a new project is created by editing a dataset, control strategy, or temporal allocation.
- Fixed issue in temporal allocation module date handling if server and client are in different time zones.
- Increased field size for design capacity units in ORL Merged dataset type. Fixes error processing FF10 inventories in a Least Cost Analysis.
- User interface enhancements for the temporal allocation module: view (read-only) mode, prompt user before deleting existing outputs, confirm runs with no inventory filter.
Client Update 29September2014:
- Fixed contextual menu pop-up in the File Downloads window for Mac OS X users.
- When exporting datasets or QA steps, fixed an issue with the user-entered prefix/filename that would cause the downloaded file to be empty. Spaces in the prefix or filename are converted to underscores.
- Simplified target pollutant selection when setting up control strategies. The target pollutant is now matched to the inventory sources using a wildcard search of pollutant names.
- Added Summarize function to the Control Strategy Manager window which creates a locally-saved spreadsheet summarizing the selected control strategies, including name, strategy type, constraints, emission reductions, and strategy costs.
- Added option to enter notes about projects when editing control strategies.
- Added interface for creating control strategy groups. This allows grouping of related control strategies with a custom name and notes about each group.
- Automatically suppress header comments when exporting input or controlled inventories directly from the CoST interface.
- Added option to download files to local computer when exporting inventories or results from the CoST interface.
- Added option to use a custom prefix when naming exported inventory or result files from the CoST interface.
- Temporal allocation module
- Fixed checks for duplicate temporal allocation names.
- Added support for daily FF10 inventories. The temporal allocation module can sum daily emissions to produce monthly, episodic, or weekday/weekend output (totals or average day).
- Added support for applying month-to-day temporal profiles.
Client Update 08September2014:
- Fixed display of dataset properties Sources tab for external datasets
- Removed recent user interface optimization updates made to data editing and viewing. This change fixes the following known issues:
- Unable to delete selected records when editing a dataset
- Applying a row filter causes the editing window to think there are unsaved changes and locks the Find and Replace Column Values option
Deployment 03September2014 (EMF/CoST):
- Fixed sorting and filtering in the Data Viewer
- Fixed tab display after saving case
Deployment 13August2014 (EMF/CoST):
- Bug fixes:
- Fixed issue where dataset and QA step table names could be longer than 63 characters.
- Fixed bug where Project Future Year Inventory Strategy was not passing through the design capacity for Flat File 2010 Point inventories when creating a controlled inventory.
- Fixed issue in File Download window where the progress bar could be hidden behind a long filename.
- Removed requirement that sector be entered for control strategies that don't use this information.
- Reduced excessive server logging about cases.
- Added control measure class to summary display of control measures in the Control Measure Manager.
- Updated SCC reference tables with newest SCC codes.
- Added user interface optimization (threading and asynchronous processing) to Dataset Export window, Dataset Manager Advanced Search, Status window, File Downloads window, and data editing and viewing.
Deployment 17June2014 (EMF/CoST):
- Added ability to use multiple QA Step argument editing windows at the same time
- Modified Flat File Inventory formats to handle EIS Dataset Shortname
- Leave export button available when exporting datasets using row filtering options
- Continuing work on user interface responsiveness: updates have been made to the Case Manager window, viewing and editing cases, and viewing and editing dataset properties
- Added new options for specifying if replacement controls should be applied to sources with incomplete existing device information
- Beta version of new temporal allocation module
- New module to apply temporal profiles to annual or monthly inventories
- Module can calculate monthly, daily, or episodic total emissions or average-day values
- Options for weekday and weekend average-day values
- Supports ORL and FF10 inventories
- Uses updated formats for cross-reference and profile datasets
- Preliminary guide to new module available at http://www.cmascenter.org/emf/internal/temporal_allocation.html
Deployment 08May2014 (EMF/CoST):
- Bug fixes:
- When exporting a dataset from the Dataset Manager, certain types of row filters would cause records from older versions of the dataset to be included in the exported file. This behavior is now fixed and the exported file will only contain records for the correct dataset version.
- When creating a controlled inventory after running a control strategy, the resulting datasets wouldn't use a user-specified prefix correctly. The controlled inventories will now be named with the custom prefix prepended to the original dataset name. Also, the custom prefix can start with a digit without causing any issues.
- When exporting control measures, measures marked as Obsolete or Temporary will now be excluded by default. A new checkbox in the export dialog will allow you to include obsolete and temporary measures if desired.
- Ongoing work is being done to keep the interface more responsive when the client application needs to communicate with the server or handle longer tasks.
Deployment 25October2013 (EMF):
- Fixed bug where Least Cost Strategy Merge Process was expecting Point IDs to be 15 characters instead of the new 20 character length. Also needed to up the SCC code to be 12 characters not 10.
- Fixed bug copying QA Step templates to the same dataset type
- Added new SO2 and PM2.5 equation types for estimating costs for ICI boiler control measures
- Updated CoST to use stack flow values from the inventories when available instead of calculating from stack velocity and diameter
- Updated CoST control strategy processing to look for PM25-PRI instead of PM2_5
- Updated CSV Dataset Type importer to not translate PM25-PRI to PM2_5
- Added lightweight dataset types, client-side caching, and threading for user interface updates to improve UI responsiveness
- Code repository was moved from CVS (SourceForge) to Git (GitHub)
Deployment 12September2013 (EMF):
- Fixed bug where the PM pollutant �primary� suffix (i.e., PM10-PRI was changed to PM10) is stripped off the pollutant code when importing Control, Projection, and Allowable datasets. These pollutant codes are now allowed in inventories, and should be allowed to be controlled when running the Project Future Year Control Strategy Process.
- Fixed build issue where some EMF Client installations (using Java 6) were having issues due to software being built in Java 7.
Deployment 12September2013 (CoST):
- Fixed bug with new "Download Manager" capability where user specific server download directory was not created correctly.
- Added validation code to new Shapefile exporter to not allow results to have a geometry (i.e., state or county) with more than one row defined in the result.
Deployment 18July2013 (EMF):
- Added control information to the Strategy Detailed Results reports (control_ids and ann_pct_red) which is used with main control measure costing algorithms.
- Added Discount Rate 3% cost information to the Strategy Detailed Results reports. Added fields that will display the results recalculated at a Discount Rate of 3%.
- Fixed bug when saving (Type 13 cost equation type) Low NOx Burner/Selective (Non-) Catalytic Reduction ICI Boiler control measures, was overriding key input variable to engineering cost equations.
Deployment 21May2013 (CoST):
- Fixed bug with the way the queue Id is read while running a case job (3710)
- Added new CSV based and Shapefile QA Step Exporter. The new Shapefile generator can transform data from rows to columns (i.e., pollutants can be pivoted from being row based to column based).
- Added new "Download Manager" capability. Exported files (datasets and QA Step results) can now be downloaded to local machine instead of having to manually move files from the EMF server.
Deployment 02May2013 (CoST):
- Update to Control Strategy Tool (CoST) to fix bug when merging Flat File and ORL inventories into a Merged ORL Inventory
Deployment 29April2013 (CoST):
- Updates to Control Strategy Tool (CoST) Least Cost algorithms to support Flat File 2010 point and nonpoint inventories
Deployment 29April2013 (EMF):
- Updates to Control Strategy Tool (CoST) algorithms to support Flat File 2010 point and nonpoint inventories
Deployment 14March2013 (EMF):
- Minor look and feel changes to Case Output Comparison report (Bug 3697)
Deployment 07March2013 (EMF):
- Fixed bug with Cases Output Report (Bug 3697)
Deployment 05February2013 (EMF):
- Added some additional logging to further research why the queue Id (from remote job server) is not read while running a case job (Bug 3710)
- Enhanced Cases Output Report to handle cases based on differing regions, also added functionality to report various issues/warnings when running a case comparison analysis (Bug 3697)
- Prepared PM Inventory (for FF10 Point and Nonpoint) Speciation QA Step SQL Templates to help with inventory speciation efforts
Deployment 29January2013 (EMF):
- Fixed bug with the way the queue Id is read while running a case job (3710)
- Fixed bug with new beta feature that allows users to compare case outputs by reporting at various aggregate levels (sector, pollutant, state, etc.), was targeting wrong output dataset type (3697)
Deployment 04January2013 (EMF):
- Added new beta feature that will allow users to Compare Case outputs by reporting at various aggregate levels (sector, pollutant, county, state, etc.) (3697)
- Fixed bug when sorting in QA result viewer (3685)
- Fixed problem when limiting QA Step results to preview (3678)
- Fixed bug when opening "Country, state, and county (COSTCY)" dataset
- Fixed issue where exported QA step result file is not set to group writable for everyone to view (3690)
- Fixed bug where Analysis Engine can't display datetime data types (3691)
- Fixed bug where Case Comparison Report was creating duplicate rows (3696)
- Created new Toxicity Weighted QA Step template a "National" based aggregate query that is used with the Flat File 2010 dataset types (3699)
Deployment 27September2012 (EMF):
- Upgrade database application server from 9.0 to 9.2
- Updated dataset types using profile codes to handle 10 characters profile code instead of 5 characters
Deployment 06September2012 (EMF):
- Fixed slowness issue with preview QA Step result (3678)
- Fixed bug with refreshing and deleting control strategy programs when Editing or Viewing Control Strategy Windows (3686)
- Fixed bug with search and replace functionality where dataset editor filter was not being applied correctly during update process (3679)
Deployment 21June2012 (EMF):
- Moved versions table into "emf" database schema, useful when backing/restoring database to new server (3503)
- Updated Project Future Year Inventory Control Strategy to handle monthly specific projection factors (3652)
- Fixed bug where COSTCY dataset type is giving wrong record count
- Fixed Delete Dataset Type functionality, needed to purge from table_consolidations table for datasets that are consolidated into one table.
- Changed Compare QA Programs to be able to specify a base and compare suffix for fields being compared (don't use ann_emis_b and ann_emis_c instead for example use ann_emis_2005 and ann_emis_2008)
- Added ability to preview QA Step result, now can limit to certain number of rows that can specified in user's local preference file (3678)
- Fixed bug with Projection pre-run validation process when only one control program is used
- Fix slowness issue with projection process
- Fixed problem sorting when viewing QA Step Results (3685)
- Fixed bug with projection process when start/end date are not set in the baseline inventory
- Fixed problem with warning messages in Strategy Messages output when applying closure and projection control programs (3687)
- Added ability to perform "Search and replace" functionality on float date type columns/fields (aka Advanced Search/Replace). User can now specify an advanced SQL filter and update multiple database columns in one pass, old approach could only change one column at a time (3679).
Deployment 21June2012 (CoST):
- Updated Project Future Year Inventory Control Strategy to handle monthly specific packet factors. This change will allow the Projection, Control and Allowable packets to specify monthly or annual source specific factors (3652).
- Fixed bug when copying QA Step to multiple datasets, was only copying to one of the datasets not all of them (3670)
- Enhanced Compare Dataset QA Program (in addition to other QA Programs) to be more intuitive with respect to the default dataset type that is shown via the Dataset Chooser (3518) when defining the QA Program.
- Updated reference pollutant code tables (invtable and pollutant_codes ) to have the latest known list of pollutants
- Update reference county table with most recent county data
- Added lat-long to states reference table
- Adjusted Flat File 2010 dataset types State Query QA templates to use state references table instead of county reference table (more efficient approach)
- Enhanced Compare Dataset Fields QA Program to include a match column, which shows when the data being compared has the same value or not (3626).
- Added ability to hide certain EMF Features (e.g., Control Strategies and/or Control Measures) (3662)
- Fixed bug when resetting password would reset user specific settings (e.g., features hidden for that user or dataset types that are hidden for that user) (3663)
- Added ability for the user to specify the order of columns when viewing a dataset's data in addition to being able to hide certain columns (3635). This feature will allow the user to define the column layout per dataset type when viewing the dataset's data.
- Added country and state name information to the Least Cost Worksheet and Strategy Detailed Result output datasets
- Added control technology and source group to the Least Cost Worksheet
Result output dataset
- Added scc descriptive information (level1, level2, level3, and level4) to the Least Cost Worksheet and Strategy Detailed Result output datasets
- Updated CSV Dataset Importer to support datasets with no column labels
and inline comments
- Finalize adding settings to make sure FF10 dataset types (Activity and
Nonpoint Daily) are auto indexed when imported into the EMF
- Indexed preexisting FF10 Activity and Nonpoint Daily datasets
- Fixed bug with Flat File Exporter when dataset has no string data types.
- Add ability to hide certain Dataset Types for users (3663)
- Updated Project Future Year Inventory Control Strategy to handle FF10 datasets (3651)
- Added settings to make sure FF10 dataset types are auto indexed when imported into the EMF
- Indexed preexisting FF10 datasets
- Fixed bug when running a case where newly created directories are not created as group writable (3659)
- Fixed bug when exporting a QA step result
Deployment 06December2011 (EMF):
- Fixed bug when running multiple QA steps with similar names (3637)
- Added ability to specify a prefix for the file name of dataset exports (3648)
- Fixed aliasing bug when running Compare Datasets QA Program that includes group by expressions with similar names (3592)
- Added new "Compare Dataset Fields" QA Program that is useful for performing value comparisons between various datasets (3626)
- Added new functionality to Compare various control strategies, a report is generated that allows the user to identify differences in strategy configurations (3627)
- Added QA step that performs toxicity weighting on the Flat File 2010 Nonpoint dataset type (3216)
- Added QA step, "Summarize by State with Toxicity Weighting", that performs toxicity weighting while aggregating at the State Level on the Flat File 2010 Point and Nonpoint dataset types (3216)
- Added ability to change dataset version of a QA step (3650)
Deployment 06December2011 (CoST):
- Fixed bug with Dataset Data Search & Replace where null and blank values were not always being replaced with new value (3646)
- Limited Dataset Data Search & Replace to certain data types - strings (varchar, char, text) and integers (smallint, integer, bigint), and not double, floats, timestamps (3647)
- Updated Dataset Advanced Search to allow freeform SQL filter to specified. Any datasets that match the filter will be retrieved. (3598)
- Updated Dataset Advanced Search to be able to specify a Case in the search process. Any datasets that are attached to the specified Case will be retrieved. (3598)
- Added some new SCC codes to underlying reference SCC database tables (3633).
- Added new dataset type, "Toxicity Weighting". (3216)
- Added prototype QA Step that will generate a toxicity weighted Flat File 2010 Point inventory.
- Added ability to specify name of QA Step Export (3655)
Deployment 01November2011 (EMF):
- Removed database dependency of using the PL/TCL language
Deployment 11October2011 (EMF):
- Add status message during a dataset import when the dataset import queue is full. The status message will indicate the dataset will be imported when the queue has available resources (3634).
- Fixed bug with the QA Step engine when running complex SQL queries (i.e., using derived subqueries or using UNIONs) (3618).
- Added menu items to the "Window" menu to "Hide All" (minimize) windows and another menu item to "Close All" windows (3629)
- Changed look and feel of the Dataset Manager Window, clarified what different Edit buttons accomplish (i.e., changed button to say "Edit Version" or "Edit Data") (3631).
- Added functionality to Copy and Paste rows in the Dataset Editor (3632).
- Fixed bug with viewing QA Step with no results (3639).
Deployment 11October2011 (CoST):
- Fixed bug with Country, County, State Information dataset type exporter (3638).
- Implemented approach to allow main strategy algorithms (Max Emission Reduction and Least Cost Types) to apply VOC and PM2.5 measures to Organic HAPs and PM Metals.
- Added capability to post scan control strategy detailed result for -Type X equations, which is used to populate Strategy Message output with sources that used default costing approach instead of the engineering costing approach (also indicate what might be the culprit for not using engineering cost equation).
- Fixed bug when setting county dataset version via Strategy Edit Window, was saving to database but loading of window would reset to dataset to default version.
- Fixed bug to auto populate sector (to "All Sectors") for "Least Cost" ORL Merged created datasets.
- Fixed bug when rerunning "Least Cost" strategies would producing erratic Detailed Results.
- Changed post-run validation routine for the "Project Future Year Inventory" Strategies to report issues with control program matching criteria. Also, integrated new hierarchy matching approach into core algorithm.
- Installed QA Step Templates to help identify strange characters in plant name for ORL Point and Flat File Point dataset types (3588)
- Externalized server debug level so that redeploy is not needed to change debug levels in the supporting EMF log files (3156).
- Setup procedure to be able to delete QA Steps (3615).
- Added ability to report key features on a case (3621).
- Fixed bug with Country, County, State Information dataset type exporter (3636).
- Added ability to dump memory settings to the Apache Tomcat web server log file during an "Out of Memory" error.
- Added client-side error handling/logging to help explain an infrequent issue found while exporting a QA report (3595)
- Added new Dataset Keywords to Projected ORL Inventories and Control Strategy Detailed Result Datasets that are created form Project Future Year Control Strategy runs (3602). These dataset keywords can be used to figure out what strategy created the output dataset and more relevant information on the Projection strategy (i.e., Target Year of Projection). Also, filled in the start/end date for resulting Control Strategy Detailed Result datasets.
- Fixed bug where the Dataset Access Log time required was too large during Case runs (3589). The Case dataset export process was using the incorrect start time as a basis to figure out how long a dataset took to export. Changed the access log time required to zero when no export occurs and changed the process to use the correct start time when an export does occur.
- Added new QA Program, "Merge In Supporting Data". This QA Program will create a report that expand the Flat File 2010 Point Dataset to include new fields (i.e., facility company name and unit description) from some various supporting datasets. (Bug 3601)
- Added new QA Step Templates for the Flat File 2010 Point and Nonpoint Dataset Type that will convert the dataset to an equivalent ORL format (3603).
- Fixed the Dataset Viewer and Editor paging capabilities. The viewer was having issues showing the correct number of records when positioned at increments of 10,000 (i.e., 10,000, 20,000, etc...) (3605).
- Fixed bug where CSV and SMOKE Dataset Exporter Types would insert column header multiple times in the exported data (3606)
- Fixed bug where CSV and SMOKE Dataset Exporter Types would export the data in a random fashion (3606). This issue was only prevalent for very large datasets (200,000+ records).
- Fixed bug when renaming a QA Step would create a copy of the with original name (3608).
- Fixed bug when exporting QA Step that contains column names with capital letters (3610).
Deployment 22March2011(Sector Scenario):
- Added capability to include WHERE clause (filter records) during the "Compare Datasets" QA Program.
- Updated Case Comparison report -- removed some unnecessary records (sector list and case description).
- Updated Project Future Year Control Strategy to not include "other" packet record results when the source has been closed via a closure packet.
- Updated Project Future Year Control Strategy to rework the number of errors/warning that are seen in the Strategy Message Output.
- Optimized CoST strategy algorithms -- Maximum Emission Reduction, Multi-Pollutant Maximum Emission Reduction, Least Cost, and Least Cost Curve.
- Updated sector mapping file format.
- Updated mapping rules to have a straight mapping between sector and other key columns (such as mact,naics,scc) in the following types of inventories: ORL Point NATA, ORL Point NATA Sector Annotated, NOF Point, NOF Nonpoint.
- Added a functionality to automatically transfer the sector scenario analysis result datasets to a separate database (sandbox) for interoperable calls.
- Enabled 'Remove' button on sector scenarios manager window to remove sector scenarios.
- Documented the Compare Datasets QA program.
- Right justified the numerical columns in the dataset editor/viewer windows.
- Verified the correctness of the number of records for dataset versions.
- Auto refreshed the job's status on the EMF client side once users cancel a running case job.
- Extended the capability of the 'Find and Replace' function on the dataset editor window so that users can find any value in a column and replace with a particular value through the SQL wild card character '%'.
- Fixed a bug where the KMZ generator fails without any error message for some files that have mal-formatted lines.
- Made the notification message better to reveal the error nature if registering output data for cases fails from the EMF command line client.
- Finished the implementation of filtering from another dataset on export. When exporting, this feauture provides an option to export only those rows that match values from a specified column in another dataset. For example, one would do an inner join on the specified column of the selected dataset and dump only the rows that match the inner join.
- Reconfigured Tomcat JVM parameters so to ease the memory contraint on the server.
- Added a feauter that users can choose from multiple join types running the Compare Datasets QA program. This QA program allows users to compare two point datasets or different versions of the same dataset at the plant or state level. Adding join types gives the flexibility of controling the output values to only the matches (INNER JOIN) for example.
- Modified the naming schema for viewing QA step results so that each result file has its own unique name in the specified temporary directory.
- Modified the error messages during export so that users can easily identify the nature of the export failures.
- Updated the rendeing of status messages so that users can now copy and paste the status messages from the Status window at the EMF client.
- Updated the dataset name rendering to use uneditable text field so that users can copy and paste dataset names on the dataset properties' viewer.
- Fixed a memory issue with search and replace on large datasets on dataset editor.
- Updated the advanced dataset search document.
- Updated the document for the new QA program for comparing datasets/versions of datasets.
- Fixed a bug where leading/trailing spaces in a dataset name prevented the QA programs to run successfully.
- Added a new feature at the dataset export window so to export dataset by filters from another dataset.
- Added new columns to SCC table.
- Improved resize behavior of arguments editor on various QA programs.
- Grayed out the 'Copy' button for external dataset on data versions editor window.
- Trimmed dataset names before creating new datasets or after editing datasets.
- Fixed a bug where dangling dataset IDs found in the emissions table.
- Added a new field into the advanced search window so to find datasets with QA steps that use a dataset.
- Fixed a bug where reviewing a QA step result throws a null pointer exception.
- Fixed a bug in a QA program where the query syntax was not correct.
- Enabled viewing/editing timestamp data in Data Viewer/Editor window.
- Fixed an issue where re-import Control Packet datasets truncates comments column data.
- Enabled viewing/editing timestamp data in flexible file format datasets.
- Removed redundancy in the delete versions column in emissions tables.
- Added capacity for searching and replacing for blank values to data editor.
- Added creator and QA step name to the advanced dataset search dialog.
- Added QA program to allow comparisons of different versions of the same dataset and different versions of different datasets.
- Fixed a bug where the default version of dataset reverts to 0 when new final version is not selected.
- Fixed a bug where external and line-based dataset types cannot be deleted.
- Fixed a bug where versions table information isn't updated correctly.
- Added additional refinements to the messages when deleting and/or purging datasets.
- Fixed a bug where KMZ generator legend is wrong.
- Updated EMF installer to handle deleted files.
- Fixed an issue where pollutant descriptions are truncated in some queries.
- Added a QA step template to detect missing xloc, yloc or lat-lons.
- Added a refresh button to the remote file browser.
- Transfered EMF to new server and new version of Postgres database.
- Reworked the dataset removal/purge process so that it checks Cases, Strategies, Programs, FaST, and Sector scenarios.
- Fixed a bug when creating a new dataset type.
- Fixed a bug with the internal_sources table, which was not able to store the import file size for very large files.
- Added new functionality for exporting a portion of a dataset using an SQL filter.
- Added a version selection drop down when exporting a dataset.
- Added version info and user notes to header of exported datasets.
- Added a display of the dataset ID in the dataset properties viewer/editor.
- Fixed a bug in editing an Input within the Case editor: when try to select a new dataset if the user clicked cancel, dataset area should not be cleared.
- In the data editor window, fixed the total number of records after deleting or adding rows.
- Developed beta version of new Control Strategy Type, Multi-Pollutant Max Emission Reduction.
- Added reference cost year to Control Measure Summary file export and import process.
- Added mobile sources to Estimate EC Impacts QA Program.
- Improved usability of CoST Estimate EC Impacts QA program by usng the keywords (STRATEGY_INVENTORY_NAME and STRATEGY_INVENTORY_VERSION) defined in the Strategy Detailed Result Dataset to auto populate program setup.
- Added Fixed and Variable O&M Costs to CoST Equation Types 3, 4, 5, and 6.
- Fixed some typos in the CoST Equation Type 3 and 9 definition.
- Implemented incremental costs like in AirControl.Net in CoST.
- Implemented approach to display CoST run warning messages to the client via the EMF Status Window.
- Prevented the sub dir column from being set in the Input when using an external dataset.
- Added a reminder for the user to save changes before performing a find and replace when editing a dataset.
- Fixed the Find and Replace function for datasets that reside in a consolidated table.
- Refined the Case Manager columns to make them more concise and complete.
- Added a pound sign to the automatic comment added to the header when copying a dataset.
- Fixed how the KMZ generator adjusts the decimal places for all columns.
- Updated the sensitivity wizard to prevent it from accidentally updating the sensitivity template Job's region.
- Added percent reduction to QA step Summaries for CoST Detailed Results.
- Added 2 letter State abbreviation to QA step Summaries for CoST Detailed Results.
- Added county name and lat/lon to FIPS report for CoST Detailed Results.
- Limited number of decimal places and include commas for real numbers in the Dataset Data Viewer.
- Fixed export directory to be consistent for QA Steps and CoST Strategy Results.
- Fixed bug in the GUI when displaying CoST strategy result costs and reductions.
- Fixed bug when entering a 'date reviewed' for a Control Measure, validate that it has the format mm/dd/yyyy.
- Enhanced the CoST Detailed Strategy Result to include three new columns: strategy_name, control_technology, and source_group.
- Provided a new keyword, INDICES, for creating an index on one or more columns for dataset types derived from the flexible file format.
- Prevented Case import from adding a new empty sector.
- Added delete to the Dataset Type Manager.
- Added support for column labels in flexible file format typed datasets. If the keyword, EXPORT_COLUMN_LABEL equals false, the dataset will not expect column labels during import and will not export column labels. If the keyword isn't set or is set to true, import will expect column labels and export will add the column labels to the output file.
- Improved error checking on import of flexible file format typed datasets.
- Provided support for EMF specific keywords on import of flexible file format typed datasets. For example, EMF_SECTOR, EMF_COUNTRY, EMF_START_DATE, etc.
- Changed the Job uniqueness criteria so that there could be Jobs with the same name within a Case. New uniqueness criteria is the combination of Job name, sector, and region. Modified helper script, emf_cleanup.csh, to support this change.
- Added floating point precision control to the KMZ generator.
- Added support for a difference plot in the KMZ generator.
- Added new color palettes to the KMZ generator.
- Added controls for the location of the legend in the KMZ generator.
- Added equal distance binning algorithm to the KMZ generator.
- Added controls for the number of bins in the KMZ generator.
- Added functionality of defining new dataset types derived from a
flexible file format. An administrator can specify a dataset type
that has custom cols for direct database import. This is
particularly useful for new inventory formats.
- Modified dataset type editor to show dataset creator, creation
date, last modified date, and flexible format if it is specified.
- Added flexible file format database importer.
- Added flexible file format database exporter.
- Enforce consistent regions in a Case's summary, jobs, inputs, and
- Fixed bug cannot save new control program
- Added custom format (no decimal places) for the "Total Cost",
"Reduction (tons)", and "Average Cost Per Ton" columns in the
Control Strategy Manager.
- When "viewing" a Control Measure made sure pull-downs reverted to
- Added show last modified date column for a Control Program to the
Control Program Manager.
- Fixed bug where CoST was not filling in the number of records for
datasets it creates.
- Enhanced EC Control Scenario (now named Estimate EC Impacts) QA
Program to work with nonpoint inventories.
- Improved usabilitiy of Estimate EC Impacts QA program by creating
a QA template to drive the QA program.
- Enhanced CoST Projections Algorithm to include NAICS in Control
Program Packets -- Allowable, Control, or Projection.
- Fix Minimum Percent Reduction Difference for Replacement Control
Constraint -- was including sources with no controls and add-on
controls using this constraint.
- Fixed the dataset last access date so that the date is consistent with the dataset history.
- Improved the error message when importing CSV files with misformatted data. New error includes line number of problematic data.
- Fixed bug in the export of temporal xref file (A/M/PTREF).
- Fixed a typo in the ptday format. The corrected column is now called 'STATE'.
- Updated the dataset viewer/editor to show the full name of the dataset creator
- Added a "beta" control measure PDF generator.
- Upgraded Control Strategy Manager to use comma separators when displaying numbers (i.e., 11500 is 11,500).
- Added feature - To pass through control program info into controlled inventories created from a CoST Project Future Year Strategy run.
- Added cost equation inventory fields and equation definition to the CoST Control Measure View and Edit Windows.
- Changed control measure importer to default equipment life to null when its zero in the import file.
- Added last log in date and whether or not user want to receive emails concerning EMF upgrades.
- Added capability to store extensions and longer phone numbers in the user profile.
- Added the ability to store date of last user password reset and added a requirement to reset the password after a certain period of time (90 days).
- Added the ability to store the last export folder and the last import folder used in the dataset manager.
- Made import/export windows for case/dataset manager, and folder selector wider. In addition, all text fields are adjusted when the window size is changed.
- Added feature - during import/export of a Case, newline characters in various fields (e.g. parameter purpose) are preserved.
- Added feature - during import/export of a Case, double quotes (e.g. in Job arguments and Parameter values)are preserved.
- Added feature - during import/export of a Case, the Base Year from the summary tab is stored.
- Fixed bug where CoST was not also adding # to metadata header when creating controlled ORL inventories.
- Fixed bug where CoST was not updating the ORL header YEAR when projecting ORL inventories.
- Added control measure efficiency record count to the Control Strategy Detailed Result datasst type "Summarize all Control Measures" QA step template.
- Defined more appropriate names for control measure properties.
- Removed identified control measures that have no supporting efficiency records.
- Update control measure database with properties from Pechan measure review.
- Fixed bug with control measure importer on not dealing with ensuring uniquess of efficiency records.
- Identified and fixed control measures that have duplicate efficiency records.
- Fixed the dataset last access date so that the date is consistent with the dataset history.
- Improved the error message when importing CSV files with misformatted data. New error includes line number of problematic data.
- Fixed bug in the export of temporal xref file (A/M/PTREF).
- Fixed a typo in the ptday format. The corrected column is now called 'STATE'.
- Minor updates to show the full name of the dataset creator in the dataset viewer/editor.
- Implemented add on control measure functionality for the Maximum Emission Reduction and Least Cost strategy types.
- Fixed bug in control measure importer and exporter with sector and nei device codes.
- Added name contains filter to jobs/inputs/paras copy dialog on case viewer.
- Restrict the creation of new projects on import of datasets to administrative users.
- Added version information about server, client, and database schemas.
- When importing ORL files, added a query that replaces -9 with nulls in key fields.
- Changed the dataset creator to show the user's full name on the Dataset Manager and Dataset Properties editor and viewer instead of the login ID.
- When importing ORL files, added a query to trim spaces from key fields.
- Removed the FILE EXISTS checkbox from the Export dialog
- Added code to replace newline characters with a special character on export of a case (e.g. in parameter notes or purpose). Use that special character to reformat to a newline on import pf the case.
- When picking input datasets, added code to remember the last name contains filter if the dataset type is the same.
- Improved uniqueness check to include georegions when importing case parameters and inputs.
- The dataset manager advanced search can now also filter on project.
- Fixed bug in import of cases with georegions.
- Added a name contains filter on the case manager window.
- Added a name contains filter when copying jobs, inputs and parameters.
- Implemented functionality to import control measure reference file.
- Implemented functionality to export control measure reference and property files.
- Added more sophisticated "Add Reference: dialog", with ability to
search for and add an existing reference.
- The dataset manager advanced search more cleanly deals with searches that return a large number of matches.
- File browser has an extended timeout.
- Refined the version information (description, etc) on data tab
- Editing of version information is restricted to creator of the version or a user with administrative privileges.
- Fixed bug in export of a Case's Jobs.
- Keywords can have values greater than 256 characters
- Improved error message when importing an ORL dataset which is missing the header "#ORL"