Document toolboxDocument toolbox

Release Notes for Data Sprint 20.06

Owned by Hamelmal Kiros
Last updated: Jul 21, 2020 by Andrew Somers
6 min read

Introduction

Date: 7/20/2020

The following list of new features, tasks and resolved bugs are included in the Sprint 20.06 release:

Features

Tasks

Bugs

Features

Ingestion Pipeline for FAO Market Price data https://fewsnet.atlassian.net/browse/DATA-246

  • This enhancement is about implementing better approaches for matching data between the FAO data and the FDW in order to import a large number of data series of the FAO data via their API.

Add a DataSourceDocument filter to GeographicUnit, etc. https://fewsnet.atlassian.net/browse/DATA-311

  • This feature is about adding the DataSourceDocument filter  to GeographicUnitSet, GeographicUnitSetVersion, Layer, GeographicUnit and Feature end points as well as enabling the user to filter the browsable APIs’ by DataSourceDocument filter.

Improved management of Units of Measures https://fewsnet.atlassian.net/browse/DATA-342

  • This task is about enhancing the Unit of Measure and Unit of Measure Conversion metadata tables by adding an inline form to the Unit Of Measure Admin widget that allows the user to create the “From Conversion” at the same time that the Unit of Measure is added, and also incorporating validation to Unit Of Measure Conversion to ensure that the Unit Type of the From Unit and the To Unit Matches.

Add “Time Series” download option to the Export tab of the FDE https://fewsnet.atlassian.net/browse/DATA-347

  • This feature is about adding the Time Series option to Export tab both in the CSV download dropdown and refreshable link dropdown of the Crop data domain of the FDE.

Add f-like string on SemiStructuredDataSeries to permit friendly stringifiction of metadata on name https://fewsnet.atlassian.net/browse/DATA-349

  • This feature is about adding f-string formatting to all the data series names which helps to easily differentiate each of the semi-structured data.

Create FDE component for semistructured data https://fewsnet.atlassian.net/browse/DATA-353

  • This feature is about adding entries for the semi-structured data under the Other domain of the FDE.

Weekly average datasets assistance https://fewsnet.atlassian.net/browse/DATA-355

  • This feature is about implementing aggregation of data that were collected DAILY to now be on a WEEKLY schedule.

Admin view for FDE user permissions handling https://fewsnet.atlassian.net/browse/DATA-361

Show the Data Explorer and Data Management tabs in Mayan for users who are a member of the groups Field office, Home Office Markets & Trade, etc https://fewsnet.atlassian.net/browse/DATA-362

  • This task is about showing the Data Explorer and Data Management tabs in the Mayan system for the users who are a member of the Early Warning Team.

Tasks

Check inconsistencies in Crop Production Data for Afghanistan https://fewsnet.atlassian.net/browse/DATA-240

  • This task was about resolving the inconsistency of the Afghanistan’s crop production data which had duplicates of Admin regions, indicators, seasons & products.

Switch zero values in Crop Production Morocco to NULL https://fewsnet.atlassian.net/browse/DATA-242

  • This task was about switching the “zero” values of the crop production data of the country Morocco to “Null” values, which improves data consistency.

Upload error of Ethiopian IPC data https://fewsnet.atlassian.net/browse/DATA-319

  • This task was about assisting the Early Warning Team in uploading the shapefile of the country Ethiopia in order to successfully load the IPC data of the country, and the data was successfully loaded.

Uploading error of the Nigeria IPC data https://fewsnet.atlassian.net/browse/DATA-320

  • This task was about giving support in uploading the shapefile of the country Nigeria in order to upload the IPC data and the data was successfully loaded.

Move Celery Results to System Administration menu https://fewsnet.atlassian.net/browse/DATA-338

  • This task was about moving the menu option of the Celery Results | Task Results from Metadata Management to Systems Administration menu.

Better Shape File Upload Handling for Erroneous Shape Files https://fewsnet.atlassian.net/browse/DATA-352

  • This task was about adding validation error message for an Erroneous Shape Files uploaded into the FDW, which allows for more clear error reporting.

Import Landscan Demography Data into Production https://fewsnet.atlassian.net/browse/DATA-357

Bugs

Impossible to fetch data error message pops up when one selects data from the Geographic filter drop down box of the PRICE and IPC FIC data domains https://fewsnet.atlassian.net/browse/DATA-307

  • The “Impossible to fetch data error” message was shown when one selected data from the Geographic filter drop down box of the PRICE and IPC FIC data domains and this was resolved.

Data Inventory Dashboard doesn't show countries on map for IPC Phase https://fewsnet.atlassian.net/browse/DATA-340

  • There was an issue in showing countries on the map of the Data Inventory Dashboard for the IPC Phase data domain and this was resolved.

Error when filtering Crop domain data by Geo aggregation filter https://fewsnet.atlassian.net/browse/DATA-346

  • Trying to filter the crop data by Geo aggregation filter was giving an error and this was resolved.

The column information of is_allowing_for_assistance is shown empty on the tabular tab of the IPC FIC data domain of the FDE https://fewsnet.atlassian.net/browse/DATA-359

Investigate caching on the Data Inventory dashboard https://fewsnet.atlassian.net/browse/DATA-360

  • There was a long delay in loading the maps of the Data Inventory dashboard (https://fdw.fews.net/dashboard/inventory/) when one tries to access it for the first time & this was resolved and the map loads in a reasonable time.

Investigate Value Error for a call /api/ipcphasemap/?format=geojson&scenario=CS&country_code=RW https://fewsnet.atlassian.net/browse/DATA-374

  • Trying to access the URL string: /api/ipcphasemap/?format=geojson&scenario=CS&country_code=RW was giving a value error and this was resolved & the API is now accessible.

Incorrect Forex calculation of GTQ to USD https://fewsnet.atlassian.net/browse/DATA-376

  • There was an incorrect calculated field of the inverse_value which was shown in the data extract and this was resolved.

Secondary exchange rates for sudan and south sudan not available via FDE "Other" https://fewsnet.atlassian.net/browse/DATA-395

Price and Trade Charts are not being displayed https://fewsnet.atlassian.net/browse/DATA-414

  • There was an issue when one tried to render chart of the Price and Trade data domains of the FDE and this was resolved.