This page lists the major new features and improvements added to each release of the Data Access Portal.
The current production release of the Data Access Portal is 2.23.2034 (25 October 2018)
v2.23.2034 (25 October 2018)
New User Interface
- The new DAP user interface that meets the current CSIRO visual identity has been released.
- Front page allows free text search, location search and browse.
- Has domain search for ATNF Pulsar, AAHL Virus, CASDA Observation and CASDA skymap.
- All collections available in the current DAP are findable and viewable in the new UI.
- The Gallery tab for collections containing images has been significantly improved over the previous UI.
- Added support for ATNF, CASDA and AAHL domain searches.
- New service to retrieve ATNF thumbnails.
- New service to return paginated list of collection image files for Gallery view.
- New service to report notifications (ie. banner text) to new UI.
- New service to download standard collection files in ZIP and TAR archives.
- New service to download files from an AAHL search result.
- New service to export collection metadata in Excel format.
- Added support for ATNF source typeahead lookup.
- Updated mount endpoint to allow a location parameter.
- Added delayed zip file creation feature to web mount requests.
- ATOM format enabled for collections endpoint.
- Mount service enhanced so that authenticated users can now request any mount type.
- New service to allow searching for files within a collection.
- New service to support a keyword cloud.
- New service that returns current featured collections.
- AAHL domain search has been migrated from Solr to Elastic Search.
- Interface for ATNF team member management.
- FTP and SFTP links are now allowed in related links and service URLs.
- New small collections are now mounted immediately after publication.
- AAF login updated to new embedded discovery service.
v2.22.1847 (28 June 2018)
- Improved performance when viewing files for a collection with a large amount of files.
- Mounted collections no longer missing from admin page.
- Collections that are less than 1gb in size will be automatically mounted.
- Added 'maintenance mode' for DAP system administrators.
- Fixed issue where CASDA Skymap was not loading.
- MNF Publishing can now be restarted on failure.
- Pulsar Observation search now supports UWL.
- Security Enhancements.
- new endpoint to retrieve FOR codes and counts
- increased performance of the feature collections endpoint.
- new endpoint to retrieve commonly used keywords.
- add additional details to related links endpoint.
- new endpoint to list the files in collection from for a specific folder.
- Web mounts via the UI now work with OAUTH.
- new endpoint to allow better searching on files within a collection.
- new endpoint that lists available download options.
- new endpoints to download files from a collection as a zip file.
- new endpoint to retrieve 'service endpoints' for Service.
- Mount endpoints now allow mounting via File Share, SFTP, Unix Machine and WEB DAV.
- public / external users can now mount collections via the web services.
- new endpoint to search for AAHL collection files.
Bug Fixes and Enhancements
- Service endpoints no longer being lost after saving as draft.
- Failure emails will no longer be sent when the collection is successfully published.
- EXIF Metadata no longer extracted for files within an MNF collection.
- When copying a collection the 'supporting files' are now also copied.
- Fix mounting issues for small collections.
v.2.21.1707 (8 March 2018)
- Added file level metadata searching
- Added popup notification to file download when no files are selected
- Changed formatting in approver email notification
- Facet name changes in file search web service
- Added recommended collections to collection landing page populated by the recommender service
- Added analytics around SFTP downloads
v.2.20.1647 (24 Jan 2018)
- Better handling of repeated mount requests
- Added Activities list to web service API
- Added more detail to Collection search web service
- Web service now allows encoded characters as input
- Related Materials have been renamed Related Links
- Added function to clear file level metadata during collection deposit
- Contributors included in web service search results
- DOI links are now HTTPS
- Added OAuth2 authentication to web service api
- Added field that indicates if data is restricted in web service search
- Collection search web service can now be filtered by date
- Added facet filtering to collection web service search
- Featured collection functionality enhancement
- REST user password is now visible in on the user's "My Defaults" page
- Update the "Access" statement on the landing page
- Commas can now be used as delimiters for the collection keywords
v.2.19.1489 (5 Oct 2017)
- Introduction of rest services
- Introduction of pre-existing license type
- Changes to tab visibility based on content
- Changes to attribution statement due to introduction of service endpoints
- Metadata popups when viewing files via landing page
- Correction of fonts on services tab of landing page
- Ability to add metadata to any file type
- Improve user experience for small collection types
- Ability to approve all future versions
- Ability to bypass approval for csiro only collections
- Activity no longer mandatory
- Psrda data migrated to postgres database
- Introduction of hyperlinks for the following metadata on the landing page for easy searching
- Project title
- Field of research
- Lead researcher
v.2.18.1306 (29 Jun 2017)
- Added live links in the DAP notification banner
- Added direct web download link to email that is sent out when mounting a collection
- Added a conditional drop down list for download of files
- Status of a collection is now visible when viewing a DAP draft collection
- Statistical reporting of downloads by IP address
- Report on download statistics when downloads are via WebDAV
- Web services
- Return a list of files of a mounted 'large', 'category D', 'mnf_eov' collections
- Modify Virtual Observatory Resource schema for a collection
- Modify access permissions for data and metadata of a draft collection (including embargo period)
- Search: collections by WBS, Lead Researcher or Contributor, Field of Research, Business Unit, Program, Group or Team
- Semantic Enablement
- Mint DOI for data-restricted collection with public metadata
- Add schema.org tags to DAP collection pages
- Various other enhancements and bug fixes
v.2.17.1151 (15 Mar 2017)
- Added widget to Landing page files tab for easier file navigation.
- Added Web Service's Swagger.
- Added the ability to 'Download All' data for a collection.
- Added support to extract partial EXIF metadata from image files.
- Metadata and licence information is now included as part of a download of DAP assets.
- User will now be warned if attempting to publish a collection with no data.
- Data collection FOR codes now retrieved from ANDS RVA-registered 'Fields of Research' vocabulary
- Increased throughput of depositor uploads
- Added sitemap to allow the site to be easily crawled by search engines.
- Fixed issue where publishing of a collection fails while minting DOI
- Fixed issue where previous version data files would be assigned to the latest version.
- Fixed issue where files would occasionally fail to upload via the UI.
- Fixed issue where duplicate entries would be displayed for ATNF puslar searched for collections that have been updated.
v.2.16.1063 (15 Dec 2016)
- Return the user to the same page after successfully logs into DAP.
- Enable file search and paginated file loading on Provide Your Data page.
- Provide map view on Describe Your Data page.
- Validate new Non-CSIRO authors to reduce duplication on Create Your Citation page.
- MongoDB document structure enhancement.
- Additional RESTful Web Services for creating and updating Collection.
- Elasticsearch search engine for DAP Search Data, Browse Data , Search By Location and search Sensor Network Data.
v.2.15.981 (6 Oct 2016)
- Web Services :
- collection metadata can now be edited using web service calls. For more information contact email@example.com
- Web Services endpoint - activity service to search for activity IDs based on wbs
- Web services endpoint - Share with other systems services
- Web services endpoint - User service, for looking up user IDs that can be added to several fields
- Web service endpoint - Cost centre service, for looking up cost centre codes
- Web service endpoint - National Facility and National collection lookup service
- File level metadata: MongoDB is now the database for collection file level metadata. The existing Oracle ADOs records have been migrated into MongoDB. DAP now write to and retrieve file level metadata from MongoDB
- Project Service : Switched DAP to connect to new Project service version 2.
- Large collection UI view: Limit visualisation of files in a container to the first 1,000 files.
- Security: WebDAV authentication more secure - SSL
- A number of enchantments and bug fixes
v2.14.859 (30 Jun 2016)
- Web Services : New collections can now be created and published using web service calls. For more information contact firstname.lastname@example.org
- Licence Management : Added Creative Commons Attribution-No Derivatives 4.0 International Licence to list of available licences.
- Organisation Service : Switched DAP to connect to new Org Service available at https://orgunitserv.csiro.au/ which provides authenticated access.
- Scheduled Collections
- Support now available to schedule data ingests for scheduled collections on a hourly/daily schedule frequency.
- Scheduled collections can also now be created using the XMLUpload interface.
- Web Services Help API can now also be accessed directly from the DAP Home Page.
- Several improvements applied relating to rendering of the data file tree on the Files tab.
v2.13.754 (29 Apr 2016)
- Collection Type - add new software type
- Web Services - add fields to output
- RIF-CS - upgrade to v1.6
- Port RIF-CS generation from DMC to DAP
- Login attempt cap - implement like RPR solution
- Meta-visualisation of files on the Data tab
v2.12.642 (11 Feb 2016)
- Allows Depositor to upload supporting documentation in "Submit for Approval" function.
- Related materials, Collaborating Organisations and Contributors can be re-orderd using drag and drop.
- Relaxed approval process for 'metadata-only' updates.
- Validate geospatial coordinates before passing them to ANDS.
- Data admin page to create links to external repositories.
- Extended the DAP web service for use with non-public collections.
- Improved Field of Research (FOR) selection.
- Included accessRights type vocabulary in RIF-CS feed to RDA.
v2.11.538 (3 Dec 2015)
- Displays DOI as a full hyperlink.
- Includes all contributors in Attribution Statement.
- Ability to change Approver for a submitted Collection.
- Configurable Collection mount time based on its type.
- RIF-CS improvements, including licence link, and change of citation name format.
v2.10.534 (2 Nov 2015)
- Enables Groups to be included as Contact Names.
- Corporate Names can be included by Depositors in the Authors list.
- Related Materials section can now include a relevant Attribution Statement.
- New section added for tracking Funding Sources.
- Reminder emails for Approvers will be sent.
- Some RIF-CS improvements were included that followed on from the previous release.
v2.9.477 (August, 2015)
- DAP User Interface
- Modified help text and added visual cues to assist the user in understanding when files are available for download and what they need to do if certain files are not downloadable to make them available for access.
- Publication notification email to Data Depositor will contain metadata on the collection published which can be used to publicise the collection.
- Reassign collection notification email to Data Admins will now contain the Data Collection Identifier.
- Label indicating the permanent link of the collection landing page has been made clearer. The link has also been made available on the Data tab.
- RIF-CS improvements : The RIF-CS feed that harvests DAP collections to ANDS now contains a breakdown of the Data citation elements in its constituents.
- Licence Management :
- The Creative Commons Licence served by DAP has been updated to version 4 while the previous version has been deprecated.
- The Licence Management interface has been updated to serve information depending on whether it is a deposit or a download scenario. This will assist data depositors and consumers with their licence selections.
v2.9.404 (April, 2015)
- Link metadata to pre-loaded data : Marine End of Voyage (EoV) data will be loaded into pre-defined storage following a voyage end. The data depositor is then able to create a metadata record in DAP and link it to the pre-loaded end of voyage data. The XML file name containing EOV is used to link the DAP record to the pre-loaded data.
- Tagging records : The previous release allowed new records to be tagged for harvesting by external organisations. A bulk process has now tagged pre-existing records that contain public metadata for Research Data Australia and TERN Soils
- Tag added for MARLIN so that Marine Community Profile (MCP) records can be harvested.
- Marine Community Profile (MCP) : Data depositor is able to use the self-service deposit to create a Collection which uses the Marine Community Profile schema.
- Depositor is able to select Marine Community Profile of ISO19115 (MCP) as an option in the “More about this Collection” section of the deposit screen enabling them to construct a valid MCP metadata record.
- MCP metadata available via web services for individual collections.
- Upload XML Metadata : Extension of current ANZLIC XML upload functionality to allow a depositor to upload MCP compliant XML files to create a new draft collection. Successfully uploading a MCP XML file creates a new draft record with relevant fields populated from the XML file, including MCP specific fields in the "More about this Collection" section.
- My defaults : Users are able to use the National Facility default value as defined in their My Defaults section while uploading XML metadata to pre-populate the Collection fields.
- Collection file number limitations : Support is now available for Collections with up to 100,000 files. This is an increase from 30,000 files.
v2.8.241 (Dec, 2014)
- Reassign Data Collections : Collections in Draft, Embargoed and Published state can have their depositors changed either by original depositor or by the Data Admin.
- Tag Collections: Collections can be tagged using an External System Tag. All Public metadata collections are tagged with RDA (Research Data Australia).
v2.7.154 (Nov, 2014)
- Software Refresh: Refreshed the core libraries of the Data Access Portal to their latest versions.
v2.7.90 (Sept, 2014)
- IRP Reform : Applied IRP Reform changes and commissioned the Org Structure Client.
v2.6.5 (July, 2014)
- Extended Storage Locations: Data Collections can now be stored and accessed from 4 data centres. These Data Centres are Black Mountain, Clayton, Brisbane and Perth.
- Enhanced Logical Collection Manager (LCM): The LCM has been enhanced to make data access and storage requests more efficient. This will enable a quicker turnaround time when managing large data sets.
- Activated use of tape and DMF for DAP collections.
v2.6.1 (April, 2014)
- Growth Stage Enhancements: Migrated application, storage and data nodes from DMSPILOTx to DAPPRDx-CDC servers.
- IPV6 connectivity : Enabled IPV6 connectivity for DAP.
v2.5 (June, 2013)
- Collection Sizes: Support provided for up to 750GB and a total of 30,000 files per Collection. Support for collection sizes over 1 Terabyte is planned for a future release.
- ATNF Self Service Deposit - Replaces the "ATNF Automated Deposit" allowing Pulsar Observations to be stored inside the DAP. Pulsar Observations to be progressively migrated to the DAP. Please check the ATNF Pulsar Observation Data Availability page for updates.
- Delete Draft and Rejection Collections - Data Depositors are able to delete Collections with a status of 'Draft', 'Draft - Requires Revision' or 'Rejected for Publication'.
- Large Collection Access - Support added to allow users to download up to 2GB of files using a Web Browser. ATNF Collections have the same download methods available to Large Collections. OPAL users are now able to download Large Collections.
- Collection Permalinks - End users are able to retrieve a URL to the latest version of any published collection.
- Supporting Attachments - Data Depositors are able to upload supporting information (e.g. a "readme.txt" file) for the data files uploaded on the Data Tab. This will not generate a new DOI for a new version of an existing published collection
v2.4 (February, 2013)
- Search By Location - End users have the ability to search for Collections that contain location coordinates by drawing an Area or selecting a Point on a World Map. Search Results may be further filtered with additional search text.
- Improved Navigation between Search Results - When a Collection is selected from search results obtained from the Search Data, Browse Data or Search By Location options, it is now possible to navigate to the previous and next search results without returning to the list of search results.
- Digital Object Identifiers (DOI) - DOI's are now easily identifiable when viewing the Description Tab on the Landing Page for Public Collections.
- Open Source Release - Source Code from Version 2.3 of the DAP released to SourceForge.
v2.3 (October, 2012)
- Enhanced Large Collection Access - Extends the ability to download Data from Large Collections (1GB or greater size) to include non-CSIRO users and restricted access Collections (where the user has access to the Collection). Public, Australian Access Federation and CSIRO Partners are able to use SFTP and WebDAV to download Large Collection Data.
- Web Service Interface - A RESTful Web Service Interface provides the ability to search for Public Collections and download their Data files using a remote application.
- RSS Feed - The Search Results page allows you to subscribe to an RSS feed showing the most recent Public Collections matching the search criteria entered.
- RIF-CS Support - Updated mapping to support RIF-CS v1.3.0 for harvesting of Collection Metadata by the Australian National Data Service (ANDS).
- Deposit Data Improvement - Collection Titles now allow commas and semicolons.
v2.2 (July, 2012)
- Large Collection Access - CSIRO users have the ability to download Data from publicly available Large Collections (1GB or greater in size) using File Share or SFTP. Data files may additionally be viewed locally on a UNIX or Windows machine.
- Digital Object Identifiers (DOI) - Public Collections automatically have a Digital Object Identifier (DOI) minted by the Australian National Data Service (ANDS). DOI's are a persistent and globally unique identifier that can be used to reference a Collection in publications and be used to cite data.
- Data Citation - Depositors may nominate non-CSIRO people as Lead Researchers and Contributors for Collections. The Depositor has the ability to add non-CSIRO people and their contact details directly when publishing Collections. An Attribution Statement is generated from the Title, Authors, Publication Year, version and unique identifier (Digital Object Identifier or Persistent Identifier) of the Collection.
- Data Administration - include functionality to allow managing DOI generation in updated published public Collections.
- Scheduled Automated Deposits - Depositors are able to schedule automated deposits for approval. The Depositor may specify how often Data is deposited for a single Collection and how often a new Collection is created in the Data Access Portal.
- Upload XML Metadata - Depositors have the ability to upload XML files containing ANZLIC metadata to automate the creation of Draft Collections.
- Software Collections - Depositors may now deposit Software Collections and associated documentation.
- Sensor Network Data Support - Depositors may now specify that a Collection contains Sensor Network Data using specific Sensor Network metadata fields.
- Sensor Network Data Search - This specific search enables end users to search Sensor Network Data.
- Image Galleries - Any Collections containing images are viewable as an Image Gallery with the choice of small, medium or large thumbnails.
- Approval Process - The Approval Process allows the Approver to nominate a third person, an Advisor, to provide advice on whether a Collection should be approved for publishing. The Depositor may request who the Advisor is when the Collection is submitted. The Advisor can be a CSIRO staff member, a CSIRO partner or an AAF collaborator.
v2.1 (December 2011)
- Self Service Deposit - A generic interface is provided which allows Depositors to publish new Collections. As well as being able to add Metadata and Data files, the Depositor can select a Data Licence and configure Access Control for the Metadata and Data. Data files are able to be uploaded using SFTP and HTTP (Web Browser). Collection's may be saved as Drafts and then completed and Published later.
- Approval Process - Collections published external to CSIRO undergo an Approval process before they can be accessed. This may be bypassed for CSIRO only published Collections.
- Embargo Controls - The Depositor may restrict access to the Collection until a specified embargo end date.
- Collection Management - Allows Depositors to edit Draft Collections and Update Published Collections. The Data Access Portal stores a complete version history for updated Collections.
- Generic Search - A new generic search function enables the end user to search through all Collections by entering search criteria (keywords). Search results may be refined using Metadata filters.
- Browse Collections - The end user is able to browse published collections in the Data Access Portal by the Field of Research.
- Core Scientific Metadata Model (CSMD) - Allows multiple metadata schemas to be supported by the Data Access Portal. Metadata schemas (e.g. ANZLIC, Darwin Core, Dublin Core and VO Resource) are now able to be mapped to a common metadata model for storage.
- Metadata Support - The Data Access Portal now supports the ANZLIC (Core fields), Darwin Core, Dublin Core and VO Resource metadata standards.
- AAHL Microscopy Data - Support is added to deposit and search for Microscopy Data from the Australian Animal Health Laboratory (AAHL).
v2.0 (Feb 2011)
Australia Telescope National Facility (ATNF) - Pulsar Observational Data recorded at the Parkes Observatory is added to the Data Access Portal. Including:
- Conversion of data into a standard PSRFITS format
- Transfer of Pulsar Data into the Data Access Portal.
- Semi-automated deposit of Pulsar Data.
- Automated generation of Pulsar Metadata.
- Automated creation of Collections and assignment of Australian National Data Service (ANDS) Persistent Identifiers (PIDs).
- Automated feed of Metadata into the Australian National Data Service (ANDS) Research Data Australia (RDA) System and Virtual Observatory.
- Automated de-embargo of Pulsar Data
- Search facility to enable end users to search Pulsar Data
v1.0 (Jan 2011)
Water Resource Data Management (WRDM) - The first release of the Data Access Portal provides Water Resource Data from the CSIRO Murray-Darling Basin Sustainable Yields (MDBSY) project. Including:
- Tools for converting Raw Data to NetCDF format.
- Transfer of Water Resource Data into the Data Access Portal.
- Semi-automated deposit of Water Research Data and ANZLIC Metadata.
- Automated creation of Collections and support for Data Licensing.
- Support for Australian National Data Service (ANDS) Persistent Identifiers (PIDs).
- Automated feed of Metadata into the Australian National Data Service (ANDS) Research Data Australia (RDA) System and GeoNetworks server.