Release Notes for Data Sprint 21.10

Introduction

The following list of new features, tasks and resolved bugs are included in the latest Sprint 21.10.2 release to FDW/FDE:

Features

• Price availability validation https://fewsnet.atlassian.net/browse/DATA-972

• Replace the Southern Africa Cross-Border Trade on Data Centre with a direct download from FDW https://fewsnet.atlassian.net/browse/DATA-1064  

• Operationalize FAOB Ingestion https://fewsnet.atlassian.net/browse/DATA-1110

• Enhancements to Market Price Time Series extracts https://fewsnet.atlassian.net/browse/DATA-1129

• Create specific viewsets for calculated data series https://fewsnet.atlassian.net/browse/DATA-1135

• Add "Fare" to market products price_type https://fewsnet.atlassian.net/browse/DATA-1178

• Use Direct ExchangeRate api end point when filtering/searching exchange rates other domain by country https://fewsnet.atlassian.net/browse/DATA-1180

• Remove Copyright footer and add Rights Disclaimer page https://fewsnet.atlassian.net/browse/DATA-1181  

• Allow `NOT_COLLECTED` data points to be reported and imported in a generic way https://fewsnet.atlassian.net/browse/DATA-1201  

• Review Other domain end points to ensure they point to their specific end points for specific metadata https://fewsnet.atlassian.net/browse/DATA-1206

Tasks

• Commodity Balances historic data upload https://fewsnet.atlassian.net/browse/DATA-820

• Change drupal node title from Central America and Caribbean to Guatemala and Haiti https://fewsnet.atlassian.net/browse/DATA-1200

Bugs

• The buttons “Download “ , “Browse” , “Chart” and the “Excel Web Query URL” are missing from the price data set page of the FDW https://fewsnet.atlassian.net/browse/DATA-1189

• Upon creating calculated indices (https://fdw.fews.net/en/admin/price/calculatedindex/add/), the Owner dropdown field doesn’t populate the username of the user that logged in https://fewsnet.atlassian.net/browse/DATA-1190

• New price data information for Zimbabwe is not recognized by the FDW https://fewsnet.atlassian.net/browse/DATA-1207

Features

Price availability validation https://fewsnet.atlassian.net/browse/DATA-972

• This enhancement is about adding a validation error message , i.e ( Availability should be set to 0 (Zero) or left empty for unavailable prices ) , for adding an availability value other than zero for the Not Available data point values.

Replace the Southern Africa Cross-Border Trade on Data Centre with a direct download from FDW https://fewsnet.atlassian.net/browse/DATA-1064  

• This implementation allows the direct download of the Souter Africa Cross Border Trade data file from the data center on the FEWS NET website (https://fdw.fews.net/api/tradeflowquantityvalue/?dataset=SA XBT trade dataset&format=xlsx&fields=website) and the downloaded file contains the selected fields fit for the website extracts.

Operationalize FAOB Ingestion https://fewsnet.atlassian.net/browse/DATA-1110

• This feature is about introducing a pipeline that imports FAOB data into FDW.

Enhancements to Market Price Time Series extracts https://fewsnet.atlassian.net/browse/DATA-1129

• This feature is about enhancing the time series extract of the market price data (for instance, https://fdw.fews.net/api/marketpricefacts/?format=html&timeseries=true&dataseries=6453165) which includes ;

-Showing Country or Geographic Region for the national-level Exchange rates columns

- Adding new metadata heading column after the last exchange rate and before the first secondary

price index with the same values except using "base value" instead of "unit" and "name" instead of

indicator_type

-Updating the existing second metadata heading column that appears before the first exchange

rate so that is uses "-" instead of "base period"; "rate type" instead of "indicator type"“.

-Removing prefix from the base value and base period metadata values.

-To use Collected" instead of "Stored" as the "field_source” for the Exchange Rate and Secondary

Price Index data series that are retrieved from the system.

-To use the actual indicator name of the exchange rates , such as e.g. "Domestic Currency per USD"

instead of the generic "Exchange Rate".

-Rounding Exchange Rate Values to 8 decimal places.

-Populating  the Latitude and Longitude metadata for the Market-level exchange rates.

Create specific viewsets for calculated data series https://fewsnet.atlassian.net/browse/DATA-1135

• This feature is about implementing the calculatedpriceindex, calculatedpriceindexvalue, calculatedpriceindexvaluefacts , priceratio , priceratiovalue and priceratiovaluefacts APIs (i.e https://fdw.fews.net/api/calculatedpriceindex/ ,

  https://fdw.fews.net/api/calculatedpriceindexvalue/ , https://fdw.fews.net/api/calculatedpriceindexvaluefacts/ ,

  https://fdw.fews.net/api/priceratio/ ,

  https://fdw.fews.net/api/priceratiovalue/ and https://fdw.fews.net/api/priceratiovaluefacts/) which return data series with the appropriate domain-specific metadata, for instance ,

  (https://fdw.fews.net/api/calculatedpriceindex/?format=xml&fields=simple  and

  https://fdw.fews.net/api/priceratio/?format=xml&fields=simple).

Add "Fare" to market products price_type https://fewsnet.atlassian.net/browse/DATA-1178

• This enhancement is about incorporating the Fare price type to the price data  domain. It is selectable under the price type dropdown list. i.e (https://fdw.fews.net/en/admin/price/marketproduct/add/).

Use Direct ExchangeRate api end point when filtering/searching exchange rates other domain by country https://fewsnet.atlassian.net/browse/DATA-1180

• This feature enhanced the search and filter capability of the Exchange Rate data series endpoint (/api/exchangerate) so that it used all country currency assignments instead of just the country assigned to the Data Series.

Remove Copyright footer and add Rights Disclaimer page https://fewsnet.atlassian.net/browse/DATA-1181  

• This enhancement is about removing the copyright footer and adding the right disclaimer page.  

Allow `NOT_COLLECTED` data points to be reported and imported in a generic way https://fewsnet.atlassian.net/browse/DATA-1201  

• This feature is about allowing the generic import of data points with Not Collected status which later can be replaced with actual status and data values.

Review Other domain end points to ensure they point to their specific end points for specific metadata https://fewsnet.atlassian.net/browse/DATA-1206

• This feature is about enhancing the other data domain end point (i.e http://fdw.fews.net/api/priceindex/ and http://fdw.fews.net/api/priceindexvaluefacts/) of the Secondary Price Index which return data with the appropriate domain-specific metadata.

Create an export breakdown of Active Users and their current associated data policy, broken out so each data policy is identified by columns https://fewsnet.atlassian.net/browse/DATA-1233

Tasks

Commodity Balances historic data upload https://fewsnet.atlassian.net/browse/DATA-820

• This task was about uploading the commodity balances historic data and it was successfully loaded i.e. (https://fdw.fews.net/en/admin/balance/commoditybalance/).

Change drupal node title from Central America and Caribbean to Guatemala and Haiti https://fewsnet.atlassian.net/browse/DATA-1200

• This task was about making Drupal node title change from Central America and Caribbean to Guatemala and Haiti.

Bugs

The buttons “Download “ , “Browse” , “Chart” and the “Excel Web Query URL” are missing from the price data set page of the FDW https://fewsnet.atlassian.net/browse/DATA-1189

• In the price data set page of the FDW i.e ( https://fdw.fews.net/en/admin/price/pricedataset/), the buttons Download , Browse , Chart and the Excel Web Query URL were missing and this was resolved.

Upon creating calculated indices (https://fdw.fews.net/en/admin/price/calculatedindex/add/), the Owner dropdown field doesn’t populate the username of the user that logged in https://fewsnet.atlassian.net/browse/DATA-1190

• Upon creating calculated indices (https://fdw.fews.net/en/admin/price/calculatedindex/add/), the Owner dropdown field didn’t populate the username of the user that logged in and this was resolved.

New price data information for Zimbabwe is not recognized by the FDW https://fewsnet.atlassian.net/browse/DATA-1207

• There was a problem in recognizing the price data for the country Zimbabwe and this was resolved and the metadata successfully loaded.