Introduction
Date: 7/20/2020
The following list of new features, tasks and resolved bugs are included in the Sprint 20.06 release:
Features
Ingestion Pipeline for FAO Market Price data https://fewsnet.atlassian.net/browse/DATA-246
Add a DataSourceDocument filter to GeographicUnit, etc. https://fewsnet.atlassian.net/browse/DATA-311
Improved management of Units of Measures https://fewsnet.atlassian.net/browse/DATA-342
Add “Timeseries” download option to the Export tab of the FDE https://fewsnet.atlassian.net/browse/DATA-347
Add f-like string on SemiStructuredDataSeries to permit friendly stringifiction of metadata on name https://fewsnet.atlassian.net/browse/DATA-349
Create FDE component for semistructured data https://fewsnet.atlassian.net/browse/DATA-353
Weekly average datasets assistance https://fewsnet.atlassian.net/browse/DATA-355
Admin view for FDE user permissions handling https://fewsnet.atlassian.net/browse/DATA-361
Show the Data Explorer and Data Management tabs in Mayan for users who are a member of the groups Field office, Home Office Markets & Trade, etc https://fewsnet.atlassian.net/browse/DATA-362
Tasks
Check inconsistencies in Crop Production Data for Afghanistan https://fewsnet.atlassian.net/browse/DATA-240
Switch zero values in Crop Production Morocco to NULL https://fewsnet.atlassian.net/browse/DATA-242
Upload error of Ethiopian IPC data https://fewsnet.atlassian.net/browse/DATA-319
Uploading error of the Nigeria IPC data https://fewsnet.atlassian.net/browse/DATA-320
Move Celery Results to System Administration menu https://fewsnet.atlassian.net/browse/DATA-338
Better Shape File Upload Handling for Erroneous Shape Files https://fewsnet.atlassian.net/browse/DATA-352
Import Landscan Demography Data into Production https://fewsnet.atlassian.net/browse/DATA-357
Bugs
Impossible to fetch data error message pops up when one selects data from the Geographic filter drop down box of the PRICE and IPC FIC data domains https://fewsnet.atlassian.net/browse/DATA-307
Data Inventory Dashboard doesn't show countries on map for IPC Phase https://fewsnet.atlassian.net/browse/DATA-340
Error when filtering Crop domain data by Geo aggregation filter https://fewsnet.atlassian.net/browse/DATA-346
The column information of is_allowing_for_assistance is shown empty on the tabular tab of the IPC FIC data domain of the FDE https://fewsnet.atlassian.net/browse/DATA-359
Investigate caching on the Data Inventory dashboard https://fewsnet.atlassian.net/browse/DATA-360
Investigate Value Error for a call /api/ipcphasemap/?format=geojson&scenario=CS&country_code=RW https://fewsnet.atlassian.net/browse/DATA-374
Incorrect Forex calculation of GTQ to USD https://fewsnet.atlassian.net/browse/DATA-376
Secondary exchange rates for Sudan and South Sudan not available via FDE "Other" https://fewsnet.atlassian.net/browse/DATA-395
Price and Trade Charts are not being displayed https://fewsnet.atlassian.net/browse/DATA-414
Features
Ingestion Pipeline for FAO Market Price data https://fewsnet.atlassian.net/browse/DATA-246
This enhancement is about implementing better approaches for matching data between the FAO data and the FDW in order to import a large number of data series of the FAO data via their API.
Add a DataSourceDocument filter to GeographicUnit, etc. https://fewsnet.atlassian.net/browse/DATA-311
This feature is about adding the
DataSourceDocument
filter toGeographicUnitSet
,GeographicUnitSetVersion
,Layer
,GeographicUnit
andFeature
end points as well as enabling the user to filter the browsable APIs’ byDataSourceDocument
filter.
Improved management of Units of Measures https://fewsnet.atlassian.net/browse/DATA-342
This task is about enhancing the
Unit of Measure
andUnit of Measure Conversion
metadata tables by adding an inline form to theUnit Of Measure
Admin widget that allows the user to create the “From Conversion” at the same time that the Unit of Measure is added, and also incorporating validation to Unit Of Measure Conversion to ensure that the Unit Type of theFrom Unit
and theTo Unit
Matches.
...
Add “Time Series” download option to the Export tab of the FDE https://fewsnet.atlassian.net/browse/DATA-347
This feature is about adding the Time Series option to Export tab both in the CSV download dropdown and refreshable link dropdown of the Crop data domain of the FDE.
...
Add f-like string on SemiStructuredDataSeries to permit friendly stringifiction of metadata on name https://fewsnet.atlassian.net/browse/DATA-349
This feature is about adding f-string formatting to all the data series names which helps to easily differentiate each of the semi-structured data.
Create FDE component for semistructured data https://fewsnet.atlassian.net/browse/DATA-353
This feature is about adding entries for the semi-structured data under the Other domain of the FDE.
...
Weekly average datasets assistance https://fewsnet.atlassian.net/browse/DATA-355
This feature is about implementing aggregation of data that were collected DAILY to now be on a WEEKLY schedule.
Admin view for FDE user permissions handling https://fewsnet.atlassian.net/browse/DATA-361
This feature is about developing a user interface for the admin view permission screen (https://fdwdev.fews.net/en/admin/auth/permission/) which allows the site Admins to see and manage the users permission.
Show the Data Explorer and Data Management tabs in Mayan for users who are a member of the groups Field office, Home Office Markets & Trade, etc https://fewsnet.atlassian.net/browse/DATA-362
This task is about showing the Data Explorer and Data Management tabs in the Mayan system for the users who are a member of the Early Warning Team.
Tasks
Check inconsistencies in Crop Production Data for Afghanistan https://fewsnet.atlassian.net/browse/DATA-240
This task was about resolving the inconsistency of the Afghanistan’s crop production data which had duplicates of Admin regions, indicators, seasons & products.
Switch zero values in Crop Production Morocco to NULL https://fewsnet.atlassian.net/browse/DATA-242
This task was about switching the “zero” values of the crop production data of the country Morocco to “Null” values, which improves data consistency.
Upload error of Ethiopian IPC data https://fewsnet.atlassian.net/browse/DATA-319
This task was about assisting the Early Warning Team in uploading the shapefile of the country Ethiopia in order to successfully load the IPC data of the country, and the data was successfully loaded.
Uploading error of the Nigeria IPC data https://fewsnet.atlassian.net/browse/DATA-320
This task was about giving support in uploading the shapefile of the country Nigeria in order to upload the IPC data and the data was successfully loaded.
Move Celery Results to System Administration menu https://fewsnet.atlassian.net/browse/DATA-338
This task was about moving the menu option of the Celery Results | Task Results from Metadata Management to Systems Administration menu.
Better Shape File Upload Handling for Erroneous Shape Files https://fewsnet.atlassian.net/browse/DATA-352
This task was about adding validation error message for an Erroneous Shape Files uploaded into the FDW, which allows for more clear error reporting.
...
Import Landscan Demography Data into Production https://fewsnet.atlassian.net/browse/DATA-357
This task was about loading the Landscan Demography data into the FDW Production environment (https://fdw.fews.net/en/admin/semistructured/semistructureddataseries/) and the data was loaded successfully.
...
Bugs
Impossible to fetch data error message pops up when one selects data from the Geographic filter drop down box of the PRICE and IPC FIC data domains https://fewsnet.atlassian.net/browse/DATA-307
The “Impossible to fetch data error” message was shown when one selected data from the Geographic filter drop down box of the PRICE and IPC FIC data domains and this was resolved.
Data Inventory Dashboard doesn't show countries on map for IPC Phase https://fewsnet.atlassian.net/browse/DATA-340
There was an issue in showing countries on the map of the Data Inventory Dashboard for the IPC Phase data domain and this was resolved.
Error when filtering Crop domain data by Geo aggregation filter https://fewsnet.atlassian.net/browse/DATA-346
Trying to filter the crop data by Geo aggregation filter was giving an error and this was resolved.
The column information of is_allowing_for_assistance is shown empty on the tabular tab of the IPC FIC data domain of the FDE https://fewsnet.atlassian.net/browse/DATA-359
The column information of
is_allowing_for_assistance
was shown empty on the tabular tab of the IPC FIC data domain of the FDE (https://fdw.fews.net/data-explorer/#/ipcfic) and this was resolved.
...
Investigate caching on the Data Inventory dashboard https://fewsnet.atlassian.net/browse/DATA-360
There was a long delay in loading the maps of the Data Inventory dashboard (https://fdw.fews.net/dashboard/inventory/) when one tries to access it for the first time & this was resolved and the map loads in a reasonable time.
Investigate Value Error for a call /api/ipcphasemap/?format=geojson&scenario=CS&country_code=RW https://fewsnet.atlassian.net/browse/DATA-374
Trying to access the URL string: /api/ipcphasemap/?format=geojson&scenario=CS&country_code=RW was giving a value error and this was resolved & the API is now accessible.
Incorrect Forex calculation of GTQ to USD https://fewsnet.atlassian.net/browse/DATA-376
There was an incorrect calculated field of the inverse_value which was shown in the data extract and this was resolved.
Secondary exchange rates for sudan and south sudan not available via FDE "Other" https://fewsnet.atlassian.net/browse/DATA-395
The secondary exchange rates of the countries Sudan and South Sudan were not showing via the Other data domain of the FDE (i.e https://fdw.fews.net/data-explorer/#/other) and this was resolved.
...
Price and Trade Charts are not being displayed https://fewsnet.atlassian.net/browse/DATA-414
There was an issue when one tried to render chart of the Price and Trade data domains of the FDE and this was resolved.