Document toolboxDocument toolbox

Release Notes for Data Sprint 20.03

Owned by Hamelmal Kiros
Last updated: Apr 09, 2020 by Andrew Somers
5 min read

Introduction

Dated: 08 April, 2020

In this Release Notes document, the following main features, tasks and defects have been resolved by the Hub Data team and deployed to the FDE/FDW production server - Version 20.03.6.

Features

Tasks

Bugs/Defects

Features/Tasks

Customizable domains for semi-structured data

  • This feature is about customizing the domains to manage the semi-structured data types, adding more flexibility to the platform. For example, we can now create new Data Series subtypes by defining the requirements in metadata.

Add Survey Type as part of the unique identifier for Survey Population Indicator, so that imports do not overwrite

  • This feature is about adding the survey type as a part of unique identifier which allows to create Data Source Documents with the same name but different Document Types which in turn allows us to avoid the problem where loading crop production data overwrites existing crop production data of different survey type.

Automated Ingestion of Market Price Data from FAO using their API

  • This feature is a prototype for automating API ingestion, and specifically using the API for automating the ingestion of FAO market price data.

Fill geographic filter property even when item does not have parent id

  • This feature is about displaying the geographic filter list of the IPC FIC data domain with the correct level of hierarchy.

Enhance Data Inventory Dashboard to include Data Points by Domain and Usage Policy

  • This feature is about visualizing the data points broken down by data domains and data usage policies.

Sub National Crop Production Data Visualization

  • This feature is about visualizing crop production data at the corresponding admin levels, which in addition shows the spatial representation of the data according to the crop production value.

Bulk load of Crop Production Country Archives

  • This task was about assisting in the uploading a large set of crop production data from multiple countries, and the data was successfully loaded.

Develop list of Metadata Types and recommend maintainers

  • This task was about creating all the list of metadata types and a recommendation as to who should administer them. This has been delivered directly to USAID.

Develop FDW/FDE Infrastructure Design Artifacts

  • This was about developing an artifact that contains: a Hardware list, Configuration/connection of hardware components (i.e. infrastructure design diagram), and Ports and Protocols used. This has been delivered directly to USAID.

Develop FDW/FDE Software Design Artifacts

  • This task was about developing an artifacts document which describes the Software Design of FDE/FDW to include: a list of software, system data types, a data flow diagram, and API designs. This has been delivered directly to USAID.

Develop FDW/FDE Configuration and Misc Artifacts

  • This task was about documenting the current configuration within FDW/FDE for: Users, Roles and Privileges, and System Descriptions. This has been delivered directly to USAID.

Resolved Bugs/Defects

Increase ECS memory allocation for Mayan celery_slow container ​ 

  • There was an issue upon loading a document using the bulk upload process and it was due to low memory allocation of the Mayan celery_slow container. This was resolved by increasing the memory of Mayan celery_slow container.

FDW services sending double emails ​

  • The FDW services were sending duplicate emails while resetting password function and this was resolved.

Warehouse.tasks.cache_datasourcedocumentviewset_list is extremely slow

  • The warehouse.tasks.cache_datasourcedocumentviewset_list was very slow and by introducing the performance improvements and the feature that adds the survey type as part of unique identifier has improved the performance of the endpoint.

DRC market exchange rates upload file not recognized

  • There was an issue in recognizing the monthly exchange rate values of the country DRC . This was caused due to the meta data change of the file and resolved.

Missing metadata from single price data series extract of the FDW ​

  • Some metadata information was not available on the extracts of the Price data series and this was resolved by displaying those metadata under the /admin/price/marketproduct/.

IPC FIC drop down menu on the FDE

  • The scale filter values of the IPC FIC data domain were incorrectly shown as IPC20 and IPC30. This was resolved and currently the values are shown as IPC v2.0 and IPC v3.0.

Markets bulk upload results in errors ​

Â