Difference between revisions of "Releases/Roadmap"

From AtoM wiki
(Authority records: Add link to authority CSV templates)
(Remove 2.6.2 from roadmap (release now out))
(29 intermediate revisions by the same user not shown)
Line 11: Line 11:
 
__TOC__
 
__TOC__
  
==Release 2.6==
+
==Release 2.6.3==
  
===Accession records===
+
Release 2.6.3 will include bug fixes, security patches, and a replacement for AtoM's previous Flash-based multi-uploader included in the [[Releases/Release_announcements/Release_2.6|2.6.0 release]].
  
We've added the ability to include alternative identifiers on accession records. Following the guidelines of the new Canadian Archival Accession Information Standard ([http://archivescanada.ca/CWG_AccessionStandard CAAIS], the first national accessions standard, modeled off the ICA standards in structure), these identifiers can each be given a type (reusable terms managed in a new taxonomy) and also have corresponding note fields. These new fields are also supported in the CSV import templates, and have been added to the search index. We've also added the ability to associate physical storage containers with accession records, and turned the Extent field into a multi-line input.
+
===Replacement for Flash-based multi-uploader===
  
''Sponsored by the Provincial Archives of Saskatchewan''
+
In [[Releases/Release_announcements/Release_2.6|Release 2.6]] and earlier, AtoM's "Upload multiple digital objects" workflow (documentation [https://www.accesstomemory.org/docs/2.6/user-manual/import-export/upload-digital-object/#upload-multiple-digital-objects-to-archival-descriptions here]) relied on a Flash-based uploader utility. As of December 2020, Flash has been deprecated and is now formally no longer supported in all major browsers.
  
* #13198 - Allow multi-line input on accession "Received Extent Units" field in edit page
+
The 2.6.3 release will include a new multi-uploader, using [https://uppy.io/ Uppy] as the base library.
  
''Sponsored by the Government of the Northwest Territories''
+
===General bug fixes and enhancements===
  
* #13254 - Add the ability to add multiple identifiers to accessions, with type and note values
+
* #13341 - Fix formatting issue when displaying linked physical storage locations
* #13262 - CSV import for accession alternative identifiers
+
* #13348 - Fix PHP warnings and notices
* #13265 - Add search support for accession alternative identifiers
+
* #13366 - A single value autocomplete field creates a new iframe whenever it loses focus
* #13269 - Add the ability to associate accessions with physical storage containers
+
* #13382 - Clipboard save message strings are currently not tranlsatable
 +
* #13393 - Editing your own user accounts set the account to inactive on save
 +
* #13398 - reset-password CLI task --activate option not working as expected
 +
* #13407 - Archival description "Calculate dates" button causes page to load very slowly
 +
* #13412 - Updating repository upload limit triggers unnecessary search index update
 +
* #13418 - Archival description identifier counter doesn't increment when the generate identifier link is used
 +
* #13421 - Authority record relationships with descriptions break EAC export
 +
* #13430 - CSV export sometimes includes duplicate columns
 +
* #13439 - CSV import update fallback matching not working in 2.6.x
 +
* #13443 - Error message "You must specify a valid path" in export tasks is unclear
 +
* #13447 - Update available notification displays repeatedly when navigating in the full-width treeview
 +
* #13448 - Running "tools:upgrade-sql" deletes all user passwords if the Argon2i hashing function is not installed
  
'''Related documentation'''
+
''Community code contribution from Jason Hildebrand and PeaceWorks Technology Solutions''
  
* Alternative identifiers: https://www.accesstomemory.org/docs/2.6/user-manual/add-edit-content/accessions/#add-accession-alt-ids
+
* #13441 - Improve Dockerfile layers caching
* Accession CSV import: https://www.accesstomemory.org/docs/2.6/user-manual/import-export/csv-import/#prepare-accession-records-for-csv-import
 
* Link physical storage: https://www.accesstomemory.org/docs/2.6/user-manual/add-edit-content/physical-storage/#link-physical-storage
 
* Updated accessions CSV template: https://wiki.accesstomemory.org/Resources/CSV_templates#Accessions
 
  
===Analytics===
+
''Sponsored by Dalhousie University Archives''
  
We've extended the way that Google Analytics can be customized in AtoM, by adding basic [https://marketingplatform.google.com/about/tag-manager/ Google Tag Manager] support. With proper configuration, this would allow site administrators to get insight into how often a download is triggered, a finding aid accessed, and more.
+
* #13431 - Syntactic error in authority record relationship area labels
  
''Sponsored by the United Nations Archives and Records Management Section''
+
''Sponsored by the Oregon Historical Society''
  
* #13284 - Add Google Tag Manager support
+
* #13402 - Carousel breaks if user has no permissions to access thumbnails
  
'''Related documentation'''
+
==Release 2.7==
  
* https://www.accesstomemory.org/docs/2.6/admin-manual/maintenance/web-analytics/#google-tag-manager
+
Our next major release. Features described below are currently scheduled for inclusion; more will be added over time.
  
===Authority records===
+
===Accessions===
  
In addition to overhauling the authority record import templates, we've also added the ability to see related authority records in term search and browse results (such as subjects and places), as well as adding new advanced search options to filter authority records by the relationship type to other authorities.
+
''Sponsored by the Government of the Northwest Territories''
 
 
''General bug fixes and enhancements''
 
  
* #13157 - Add paging to authority record description relations to avoid editing timeouts
+
* #13315 - Add ability to associate events with accession records
  
''Sponsored by an undisclosed organization''
+
===Authentication===
  
* #13275 - Display related authority records on Term view and Taxonomy browse pages
+
''Sponsored by Simon Fraser University Archives''
* #13276 - Filter authority record searches by relationship type
 
  
'''Related documentation'''
+
* #13383 - Support CAS single sign-on
  
Term browse:
+
===Archivematica integration===
  
* https://www.accesstomemory.org/docs/2.6/user-manual/access-content/browse/#subjecticon-browse-subjects-and-places-placeicon
+
''Sponsored by Simon Fraser University Archives''
* https://www.accesstomemory.org/docs/2.6/user-manual/access-content/search-atom/#dedicated-search-terms
 
* https://www.accesstomemory.org/docs/2.6/user-manual/access-content/context-menu/#subject-and-place-browse-pages
 
  
Authority record relation search:
+
* #13351 - Support AIP download from AtoM UI for authorized users
  
* https://www.accesstomemory.org/docs/2.6/user-manual/access-content/search-atom/#search-authority-record-relationships
 
  
Updated 2.6 Authority record CSV import templates:
+
===Authority records===
  
* https://wiki.accesstomemory.org/Resources/CSV_templates#Authority_records
+
''Sponsored by an undisclosed institution''
  
===Clipboard===
+
* #13373 - Display warning for duplicate authority record identifiers
  
The Clipboard module now includes a configurable section that can be used to allow clipboard results to be sent to a third-party site as a JSON array, using HTTP POST or GET methods. This can be used to integrate with a third party reference management application, for example.
 
 
''Sponsored by the Bernard Becker Medical Library Archives at Washington University in St. Louis''
 
 
* #12800 - Add JSON export of AtoM clipboard items for third-party integrations
 
  
 
===Command-line tasks===
 
===Command-line tasks===
 
In addition to the enhancements described in the Digital Obects section of these release notes, we've also added a few new enhancements to help with maintenance. These include a new task to output the index for a specific description, a task to move authority record description relations from one actor to another, and more.
 
 
* #13189 - Add task to move authority record description relations from one actor to another
 
* #13257 - Add ES version to search:status task output
 
* #13291 - Add task to output search index data for a target description
 
* #13358 - Add option to the build nested set task to exclude types
 
 
 
See also the various CLI task enhancements in the Digital objects section.
 
 
===Digital objects===
 
 
We have replaced the Flash-based flowplayer used for streaming media with a modern HTML5 based player ([https://www.mediaelementjs.com/ mediaElementJs]). MediaElement has Flash fallback for legacy browsers, and supports aria tags, captions, plugins for additional functionality and more. We've also updated two digital object command-line tasks and added new options to them, including adding a method to delete all digital objects associated with a particular repository, and an option in the digital object load task to replace existing attached digital objects.
 
  
 
''General bug fixes and enhancements''
 
''General bug fixes and enhancements''
  
* #13196 - CLI regenerate derivatives task produces error when --slug used with --no-overwrite
+
* #13440 - Add ability for tools:expire-data to delete temp files
* #13243 - Regression: identifier and title context lost on link digital object page
+
* #13368 - Add CLI option to migration task to run individual migrations
 
 
''Sponsored by the Oregon Historical Society''
 
  
* #12902 - Replace flowplayer with an HTML5 media player and update default derivative generation
 
* #12903 - Add new option to Digital object regen-derivatives task to allow regenerating only video file derivatives
 
  
''Sponsored by the Provincial Archives of Saskatchewan''
+
''Sponsored by Mills Archive''
  
* #13103 - Add --attach-only option to digital object load task
+
* #13361 - Add option to cache XML task to generate only EAD or DC XML
* #13126 - Add --skip-nested-set-build option to digital object load task
 
  
 
''Sponsored by the University of Saskatchewan Libraries''
 
''Sponsored by the University of Saskatchewan Libraries''
  
* #13152 - Add --replace option to digital object load task to allow uploads to replace existing digital objects
+
* #13380 - Add CLI task to identify and delete unused access point terms
  
''Sponsored by the Archives Society of Alberta''
+
===Digital objects===
  
* #13288 - Overhaul digitalobject:delete task and add new options to delete all digital objects associated with a repository, and limit to a specific media type
+
''Sponsored by an undisclosed institution''
  
===Finding aids===
+
* #13351 - Download AIP and AIP files from AtoM UI
 +
* #13395 - Add option to include digital objects in clipboard exports
  
In collaboration with Dalhousie University Library, we've implemented some minor but much needed usability and formatting improvements to the finding aid layouts.
+
''Sponsored by Simon Fraser University Library''
  
''General bug fixes and enhancements''
+
* #13388 - Improve digital object metadata display
 +
* #13417 - Digital object preservation copy displays UCT time stamps
  
* #12707 - Finding aids will display custom levels as "otherlevel" instead of using otherlevel value
+
''Sponsored by the Mills Archive''
  
''Collaboratively developed / sponsored by Dalhousie University Library''
+
* #13436 - Add API endpoint to download master digital object
  
* #10591 - Update finding aid XSLTs with formatting improvements
 
  
===Import/Export===
+
===Scalability and performance===
 
 
The authority record CSV import templates have received a major overhaul - now there is no separate Alias CSV (it has been merged into the main authority record CSV), and the relationships CSV can be imported via the user interface, as well as used to add relationships to existing authorities (previously it could only be imported via the command-line at the same time as an accompanying authority record CSV that contained all referenced records). The accessions CSV template got three previously unsupported fields added, a new bulk authority CSV export task was added to the command-line,  and a brand new physical storage CSV import has been added to the command-line as well, which will support update imports. A command-line "roundtrip" option has been added to the descriptions CSV import that vastly improves matching when exporting records, updating the CSV, and then reimporting them into the same system. Finally, a few fixes and new setting options for the DC XML export should improve its utility for those using the OAI Repository module to expose records for harvesting.
 
  
 
''General bug fixes and enhancements''
 
''General bug fixes and enhancements''
  
* #6317 - EAD <repository> will not import properly if it is not nested in a <corpname> element
+
* #13435 - Permission denied creating a new child description when Grant "create" on parent
* #13130 - DC XML should include inherited creator(s) at lower levels
+
* #13437 - Digital object authorization code is too complicated
* #12281 - Add roundtrip option to CLI CSV import for better matching
 
* #13204 - An apostrophe in the repository code will cause EAD XML imports to fail
 
* #13246 - Add --limit option to task for caching XML representations
 
  
''Sponsored by the WUSTL Bernard Becker Medical Library Archives''
+
''Sponsored by an undisclosed institution''
  
* #12799 - Add donorNotes column to CSV import template for Accessions
+
* #11556 - Make AtoM's clipboard work with the Varnish Cache web application accelerator
 +
* #13389 - Support ability for reverse proxies like Varnish to cache content that's not in the default culture
 +
* #13415 - An archival description with many descendants loads slowly when the DO carousel is enabled
  
''Sponsored by the University of Saskatchewan Libraries''
+
===Settings===
 
 
* #13148 - Add support for command-line bulk authority record CSV export task
 
 
 
''Sponsored by the Government of the Northwest Territories''
 
 
 
* #13262 - Add CSV import support for accession alternative identifiers
 
* #13310 - Add donorFax and donorContactPerson columns to Accession CSV import
 
 
 
''Sponsored by the City of Vancouver Archives''
 
 
 
* #13183 - Add CSV import for physical locations
 
* #13185 - Add ability to update physical storage data via CSV import
 
 
 
''Sponsored by the University of Strathclyde Archives and Special Collections''
 
 
 
* #13244 - Add setting to determine if identifier or reference code is used in DC XML
 
 
 
''Sponsored by an undisclosed organization''
 
 
 
* #13249 - Add support for digital object CSV import/export for authority records
 
* #13277: Alias and relations added to CSV GUI import for Authority records
 
* #13279: Add relationship type to authority record relations CSV import and export
 
* #13280: Create relationships between existing authority records via CSV import
 
* #13282: Add Alias fields (parallelNames, standardizedNames, and otherFormNames) to Authority Record CSV export
 
 
 
===OAI Repository===
 
 
 
In addition to some of the DC XML export enhancements outlined in the Import/Export section, we've also addressed some outstanding bugs in the OAI Repository module.
 
 
 
''General bug fixes and enhancements''
 
 
 
* #12387 - OAI-PMH GetRecord verb should filter Draft descriptions consistent with other verb responses
 
* #13163 - OAI identify command base URL does not respect HTTPS; appends index.php unnecessarily
 
 
 
''Contributed via community pull request from [https://github.com/rferris Rohan Ferris]''
 
 
 
* #13025 - Convert UTC time to localtime in OAI requests
 
 
 
===Performance, Stability, and Internals===
 
  
We've implemented a number of internal enhancements to improve AtoM's overall performance and stability - these changes are predominantly focused on deletions, upgrades, indexing, and CSV imports. One of the biggest changes was upgrading AtoM’s MySQL version to 8.0, and using the Percona Monitoring & Management ([https://www.percona.com/software/database-tools/percona-monitoring-and-management PMM]) tool to identify key areas for query optimization. Most significantly, in several areas we have replaced the nested set queries (used to manage hierarchical relations in a flat relational database) with MySQL8’s implementation of Common Table Expressions ([https://dev.mysql.com/doc/refman/8.0/en/with.html CTE]; sometimes referred to as WITH queries). Overall this has led to significant performance gains - details of which have been captured on issue ticket #13224. We've also optimized the search indexing and the CSV import task - and some new options in some of the command line task (captured in other sections of this roadmap) also helped us optimize some of our processes that use these tasks.
+
''Sponsored by an undisclosed institution''
  
* #10306 - Improve foreign key declarations in database schema
+
* #13411 - Add setting to enable or disable repository "upload limit" widget
* #13220 - Upgrade to MySQL 8
 
* #13221 - Avoid schema differences between a default install and an upgraded database
 
* #13224 - Improve hierarchy management queries
 
* #13237 - Reconsider nested set implementation in some models
 
* #13239 - Improve deletion of nested set hierarchies (especially terms)
 
* #13350 - Reduce likelihood of issues when upgrading from 1.x to 2.6.x
 
* #13352 - Reduce resource save/indexing count on description CSV import
 
* #13354 - Avoid nested set update on descriptions CSV import (reduce DB deadlocks)
 
* #13355 - Reduce memory usage on descriptions CSV import
 
* #13356 - Remove lft field from description ES index
 
* #13261 - Improve Elasticsearch indexing process
 
* #13362 - Move analyze table calls to the end of the upgrade process
 
  
===Physical storage===
 
  
In addition to the new command-line physical storage CSV import (described in the Import/Export section), we've also added a brand new CSV report to the physical storage module. This report focuses on container relations with accessions and descriptions, and can be customized on export.
+
===Treeview===
  
 
''General bug fixes and enhancements''
 
''General bug fixes and enhancements''
  
* #13201 - Browse physical storage page does not show values that have not been translated
+
* #13423 - Full-width treeview fails to load for descriptions with 900K+ children
 
+
* #13446 - Clicking on a description in the full-width treeview doesn't update the page URL
''Sponsored by the Government of the Northwest Territories''
 
 
 
* #13269 - Add new CSV report for physical storage location reporting
 
 
 
===Search/Browse===
 
 
 
We've added some minor enhancements and bug fixes to the search and browse options in AtoM that should improve the overall experience.
 
 
 
''General bug fixes and enhancements''
 
 
 
* #5173 - Improve default alphabetic sort to better reflect natural sort expectations
 
* #13186 - Add user-configurable setting for treeview paging limit
 
 
 
''Sponsored by the Government of the Northwest Territories''
 
 
 
* #12471 - Hovering over a truncated facet filter doesn't show the full name in simple label tooltip
 
 
 
''Sponsored by the Bernard Becker Medical Library Archives at Washington University in St. Louis''
 
 
 
* #12797 - Make "Browse digital objects" link on description view page context sensitive
 
 
 
''Sponsored by the Provincial Archives of Sasksatchewan''
 
 
 
* #13175 - Improve information object search indexing speed
 
 
 
''Sponsored by the Mills Archive''
 
 
 
* #13230 - Date range sorting is broken following Elasticsearch upgrade in 2.5.x
 
 
 
''Collaboratively prepared by community user Monica Wood (University of Tasmania) and Artefactual Systems''
 
 
 
* #13299 - If unique records count matches count of only other language, language facet should not display
 
 
 
===Settings===
 
 
 
The Visible elements module got a few new DACS-related options, and the full-width treeview has some additional settings to increase the paging limit (up to 10,000 records per load), and to make the tree collapsible on first load (much like the advanced search panel). We've also added a simple notification that appears on Save in the settings page, to better confirm to end users that the changes have been applied.
 
 
 
* #13105 - Add warning message to security settings page
 
 
 
''Sponsored by the WUSTL Bernard Becker Medical Library Archives''
 
 
 
* #12798 - Add Visible elements module for DACS area headings
 
* #13049 - Add DACS Physical access field to Visible elements
 
* #13135 - Add new setting to make full-width treeview collapsible
 
 
 
''Sponsored by Strathclyde University Archives''
 
 
 
* #13320 - Increase treeview pager limit setting from 1,000 to a max value of 10,000
 
 
 
===Bug fixes and other tickets===
 
 
 
* #13173 - If identifiers contain syntax that conforms to markdown syntax, the identifier is rendered in markdown
 
* #13174 - Jobs can hang if console output is greater than 64KiB
 
* #13196 - CLI regenerate derivatives task produces error when --slug used with --no-overwrite
 
 
 
 
 
<admonition type="seealso">
 
For a full list of issues related to the 2.6 release, see the following links to our issue tracker:
 
* [https://projects.artefactual.com/versions/127 Release 2.6 Roadmap / Overview]
 
* [http://bit.ly/AtoM-26 2.6 issues organized by category]
 
</admonition>
 
  
 
==Wishlist==
 
==Wishlist==

Revision as of 18:56, 5 February 2021

Main Page > Releases > Releases/Roadmap

On this page, you'll find summaries about upcoming releases, and issue tickets related to them. As the releases are finalized, elements on this page will be used to draft the formal release announcements found here.

Tip

You can also check out the issues associated with upcoming releases by looking at the Roadmap page in our issue tracker.

Release 2.6.3

Release 2.6.3 will include bug fixes, security patches, and a replacement for AtoM's previous Flash-based multi-uploader included in the 2.6.0 release.

Replacement for Flash-based multi-uploader

In Release 2.6 and earlier, AtoM's "Upload multiple digital objects" workflow (documentation here) relied on a Flash-based uploader utility. As of December 2020, Flash has been deprecated and is now formally no longer supported in all major browsers.

The 2.6.3 release will include a new multi-uploader, using Uppy as the base library.

General bug fixes and enhancements

  • #13341 - Fix formatting issue when displaying linked physical storage locations
  • #13348 - Fix PHP warnings and notices
  • #13366 - A single value autocomplete field creates a new iframe whenever it loses focus
  • #13382 - Clipboard save message strings are currently not tranlsatable
  • #13393 - Editing your own user accounts set the account to inactive on save
  • #13398 - reset-password CLI task --activate option not working as expected
  • #13407 - Archival description "Calculate dates" button causes page to load very slowly
  • #13412 - Updating repository upload limit triggers unnecessary search index update
  • #13418 - Archival description identifier counter doesn't increment when the generate identifier link is used
  • #13421 - Authority record relationships with descriptions break EAC export
  • #13430 - CSV export sometimes includes duplicate columns
  • #13439 - CSV import update fallback matching not working in 2.6.x
  • #13443 - Error message "You must specify a valid path" in export tasks is unclear
  • #13447 - Update available notification displays repeatedly when navigating in the full-width treeview
  • #13448 - Running "tools:upgrade-sql" deletes all user passwords if the Argon2i hashing function is not installed

Community code contribution from Jason Hildebrand and PeaceWorks Technology Solutions

  • #13441 - Improve Dockerfile layers caching

Sponsored by Dalhousie University Archives

  • #13431 - Syntactic error in authority record relationship area labels

Sponsored by the Oregon Historical Society

  • #13402 - Carousel breaks if user has no permissions to access thumbnails

Release 2.7

Our next major release. Features described below are currently scheduled for inclusion; more will be added over time.

Accessions

Sponsored by the Government of the Northwest Territories

  • #13315 - Add ability to associate events with accession records

Authentication

Sponsored by Simon Fraser University Archives

  • #13383 - Support CAS single sign-on

Archivematica integration

Sponsored by Simon Fraser University Archives

  • #13351 - Support AIP download from AtoM UI for authorized users

Authority records

Sponsored by an undisclosed institution

  • #13373 - Display warning for duplicate authority record identifiers

Command-line tasks

General bug fixes and enhancements

  • #13440 - Add ability for tools:expire-data to delete temp files
  • #13368 - Add CLI option to migration task to run individual migrations

Sponsored by Mills Archive

  • #13361 - Add option to cache XML task to generate only EAD or DC XML

Sponsored by the University of Saskatchewan Libraries

  • #13380 - Add CLI task to identify and delete unused access point terms

Digital objects

Sponsored by an undisclosed institution

  • #13351 - Download AIP and AIP files from AtoM UI
  • #13395 - Add option to include digital objects in clipboard exports

Sponsored by Simon Fraser University Library

  • #13388 - Improve digital object metadata display
  • #13417 - Digital object preservation copy displays UCT time stamps

Sponsored by the Mills Archive

  • #13436 - Add API endpoint to download master digital object

Scalability and performance

General bug fixes and enhancements

  • #13435 - Permission denied creating a new child description when Grant "create" on parent
  • #13437 - Digital object authorization code is too complicated

Sponsored by an undisclosed institution

  • #11556 - Make AtoM's clipboard work with the Varnish Cache web application accelerator
  • #13389 - Support ability for reverse proxies like Varnish to cache content that's not in the default culture
  • #13415 - An archival description with many descendants loads slowly when the DO carousel is enabled

Settings

Sponsored by an undisclosed institution

  • #13411 - Add setting to enable or disable repository "upload limit" widget

Treeview

General bug fixes and enhancements

  • #13423 - Full-width treeview fails to load for descriptions with 900K+ children
  • #13446 - Clicking on a description in the full-width treeview doesn't update the page URL

Wishlist

This project in our issue tracker is where new unsponsored feature requests for Access to Memory (AtoM) can be added. This allows us to log feature requests, and begin discussion about requirements and design without adding confusion to the main AtoM project's issue tracker as to which features will be included in an upcoming release.

See the AtoM Wishlist project here:

Tip

Once you've followed the link above, you can click on the "Issues" link at the top of the page to browse all issue tickets in the AtoM Wishlist. You can sort the list of issues by clicking on a column header, and use the Filter drop-down menu to the right to customize the query.

How the Wishlist works

When features are sponsored by an institution, or developed and submitted via community pull request, they will be copied to the main AtoM project in our issue tracker (here), and marked verified in the Wishlist project. Developer discussion, quality assurance testing, and other related updates will be posted to the related development ticket in the main AtoM project.

The tested version tag can be used in the Wishlist's issue description to keep track of when the feature was requested (e.g. which version of AtoM was most recent when the feature request came in)

The target version will only be applied before a Wishlist ticket is verified (closed) and copied to the main AtoM project.

Seealso

  • We've got information on our main AtoM issue tracker, and how you can use it to submit a bug report here: Resources/Issue tracker.
  • Got questions? Not sure if something is a bug or not, if a wishlist feature might be of interest to the rest of the community, or if something is already on our upcoming development roadmap? Why not make a post in our AtoM User Forum?