Release Notes for Data Sprint 21.02

Introduction

The following list of new features, tasks, and resolved bugs are included in the latest version 21.02.4.

Features

• Pull in Price data from Farmers in Sudan API https://fewsnet.atlassian.net/browse/DATA-262

• Common Currency Price should use aggregates for Exchange Rates https://fewsnet.atlassian.net/browse/DATA-563

• Allow demographic data extracts for all FDW Geographic Units by combining LandScan population estimates with raster demographic data https://fewsnet.atlassian.net/browse/DATA-721

• Re-enable the sending of notification from Data Publication workflow for data collections https://fewsnet.atlassian.net/browse/DATA-737

• Allow IPC Classification data upload from the data collection screen https://fewsnet.atlassian.net/browse/DATA-743

• API End Point to report metadata for the Spatial domain https://fewsnet.atlassian.net/browse/DATA-756

• Better population and demography data in spatial extracts https://fewsnet.atlassian.net/browse/DATA-757

• Create a prototype of class-based visualizations for Price https://fewsnet.atlassian.net/browse/DATA-759

• Update Data Inventory Dashboard to use `DataUsagePolicy`.openness https://fewsnet.atlassian.net/browse/DATA-768

• Expand Data Source Document Import to cover Data Usage Policy https://fewsnet.atlassian.net/browse/DATA-783

• Regional IPC Shapefiles and Maps Feedback: Shapefiles changes https://fewsnet.atlassian.net/browse/DATA-807

• Regional IPC Shapefiles and Maps Feedback: PNG Maps Changes https://fewsnet.atlassian.net/browse/DATA-808

• Regional IPC Shapefiles and Maps Feedback: ReadMe.txt File Changes https://fewsnet.atlassian.net/browse/DATA-809

• Support Country Groups in Data Inventory Dashboard https://fewsnet.atlassian.net/browse/DATA-815

Tasks

• Move LandScan rasters to S3 https://fewsnet.atlassian.net/browse/DATA-719

• Creating LHZ Geographic unit set for FEWS NET countries and reorganizing the existing uploaded files https://fewsnet.atlassian.net/browse/DATA-765

Bugs

• Error when correcting ML2 values https://fewsnet.atlassian.net/browse/DATA-761

• Error uploading Uganda admin data to FDW https://fewsnet.atlassian.net/browse/DATA-785

• Error uploading intersect shapefiles of Uganda and Sudan to FDW https://fewsnet.atlassian.net/browse/DATA-788

• Select all button is only working for the Demography subtype of the other data domain https://fewsnet.atlassian.net/browse/DATA-794

• Adjusting Mayan logout link https://fewsnet.atlassian.net/browse/DATA-802

Features

Pull in Price data from Farmers in Sudan API https://fewsnet.atlassian.net/browse/DATA-262

• Price data and matching metadata are periodically imported from the Sudan farmers API into FDW automatically as a reoccurring task.

Common Currency Price should use aggregates for Exchange Rates https://fewsnet.atlassian.net/browse/DATA-563

• Common Currency Price has been updated to calculate aggregates for exchange rates if the Exchange Rate is collected at a more frequent schedule than the Market Price.   

Allow demographic data extracts for all FDW Geographic Units by combining LandScan population estimates with raster demographic data https://fewsnet.atlassian.net/browse/DATA-721

• LandScan population estimates, and raster demographic data have been combined to allow demographic data extracts for all FDW geographic units. https://fdw.fews.net/api/geographicunit/?fields=with_demography&format=json&search=hadejia%20valley&demography_year=2000

Re-enable the sending of notification from Data Publication workflow for data collections https://fewsnet.atlassian.net/browse/DATA-737

• The data publication workflow has been updated to send notification emails for data collections that are Published or Under Review.

Allow IPC Classification data upload from the data collection screen https://fewsnet.atlassian.net/browse/DATA-743

• The Add Data Collections page has been updated to allow the upload of IPC Classification data. Navigation: FDW Home > Warehouse > Data Collections > Add data collection. https://fdw.fews.net/en/admin/warehouse/datacollection/add/)

API End Point to report metadata for the Spatial domain https://fewsnet.atlassian.net/browse/DATA-756

• The API endpoint (/api/geographicunit/) has been enhanced to display the source of population and demographic data for the Spatial domain in the sample format below:

  “Geographic information from Landscan, FEWS NET Population distributed by Landscan YYYY. Demographic data by WorldPop YYYY.”

  https://fdw.fews.net/api/geographicunit/metadata/?fields=with_demography&format=json&search=hadejia valley&demography_year=2000

Better population and demography data in spatial extracts https://fewsnet.atlassian.net/browse/DATA-757

• The /api/geographicunit and /api/feature API endpoints have been enhanced to include better population and demography data in spatial extracts.

  o   https://fdw.fews.net/api/geographicunit/?format=json&search=hadejia%20valley

  o   https://fdw.fews.net/api/geographicunit/?format=json&search=hadejia%20valley&fields=with_population ,

  o   https://fdw.fews.net/api/geographicunit/?fields=with_demography&format=json&search=hadejia%20valley&demography_year=2000 and

  o   https://fdw.fews.net/api/feature/?search=Western%20Kalahari%20Sandveld%20Communal&format=geojson&fields=with_demography

Create a prototype of class-based visualizations for Price https://fewsnet.atlassian.net/browse/DATA-759

• Documentation of a prototype to show class-based visualizations for Price charts and maps is currently in progress.

Update Data Inventory Dashboard to use `DataUsagePolicy`.openness https://fewsnet.atlassian.net/browse/DATA-768

• The Data Inventory Dashboard visualization has been updated to use the openness attribute from the Data Usage Policy.

Open

Expand Data Source Document Import to cover Data Usage Policy https://fewsnet.atlassian.net/browse/DATA-783

• The data source document import (https://fdw.fews.net/en/admin/warehouse/datasourcedocument/import/) has been enhanced to cover the Data Usage Policy and display the following headers in the data source document extract (https://fdw.fews.net/api/datasourcedocument/?format=xlsx): data_usage_policy, datausage_notes, data_usage_policy_reason and data_usage_policy_justification

Open

Regional IPC Shapefiles and Maps Feedback: Shapefiles changes https://fewsnet.atlassian.net/browse/DATA-807

• Regional IPC shapefiles have been updated with multiple enhancements including outlines, color schemes and shapefile package naming conventions.

Regional IPC Shapefiles and Maps Feedback: PNG Maps Changes https://fewsnet.atlassian.net/browse/DATA-808

• PNG maps have been updated to enhance visuals and text.

Regional IPC Shapefiles and Maps Feedback: ReadMe.txt File Changes https://fewsnet.atlassian.net/browse/DATA-809

• Text in ReadMe.text files for regional and country shapefiles have been updated.

Support Country Groups in Data Inventory Dashboard https://fewsnet.atlassian.net/browse/DATA-815

• The Data Inventory Dashboard has been updated to include the Country Groups filter. Navigation: FDW Home > Data Analysis > Data Inventory (https://fdw.fews.net/dashboard/inventory/).

Tasks

Move LandScan rasters to S3 https://fewsnet.atlassian.net/browse/DATA-719

• Population and other raster data have been moved to S3 in order to support other sources of population data.

Creating LHZ Geographic unit set for FEWS NET countries and reorganizing the existing uploaded files https://fewsnet.atlassian.net/browse/DATA-765

• New livelihood zone geographic unit sets were created for FEWS NET countries and existing uploaded unit sets were reorganized. (https://fdw.fews.net/en/admin/spatial/geographicunitsetversion/).

Bugs

Error when correcting ML2 values https://fewsnet.atlassian.net/browse/DATA-761

• An error populated upon trying to edit the phase value of the IPC classification data was resolved.

Error uploading Uganda admin data to FDW https://fewsnet.atlassian.net/browse/DATA-785

• Issues uploading admin shapefiles for Uganda were resolved.

Error uploading intersect shapefiles of Uganda and Sudan to FDW

• Issues uploading shapefiles for Uganda and Sudan were resolved.

Select all button is only working for the Demography subtype of the other data domain https://fewsnet.atlassian.net/browse/DATA-794

• The select all button of the Other data domain in FDE (https://fdw.fews.net/data-explorer/#/other) was only working for the demography sub type. This was resolved and functionality is restored for all data domain sub types.

Adjusting Mayan logout link https://fewsnet.atlassian.net/browse/DATA-802

• Issues in the Mayan logout link were resolved and now redirect to the correct logout link. (https://fdw.fews.net/accounts/logout/).