Release Notes for Data Sprint 21.12

Introduction

The following list of new features, tasks and resolved bugs are included in the latest Sprint 21.12 release to FDW/FDE:

Features

Calculated Price Index & Calculated Price Ratio end points of the FDE should point to their specific end points https://fewsnet.atlassian.net/browse/DATA-1247
Feature request for adding export button to the exchange rate data series of the FDW https://fewsnet.atlassian.net/browse/DATA-1248
Adding data series ID to available data series "info" field https://fewsnet.atlassian.net/browse/DATA-1263
Prototype of PIN Retrieval Tool https://fewsnet.atlassian.net/browse/DATA-1264
Updating the validation error message for checking the availability data of the base period of each component selected for a Calculated Price Index https://fewsnet.atlassian.net/browse/DATA-1267
Include schedule field to the data source document extract https://fewsnet.atlassian.net/browse/DATA-1306
Include data_usage_policy field in the simple fields extracts of the exchange rate and secondary price index of the FDE https://fewsnet.atlassian.net/browse/DATA-1309
Add filters.ModelMultipleChoiceFilter on DataUsagePolicy, Country and CountryGroup to DataSourceDocumentFilter https://fewsnet.atlassian.net/browse/DATA-1312
Add geographic_unit field in the simple fields extract of the calculated price index of the FDE https://fewsnet.atlassian.net/browse/DATA-1318

Tasks

Create geographic unit tree for exchange rate domain https://fewsnet.atlassian.net/browse/DATA-1303

Bugs

Issue of loading the Geographic filter on Crop and Nutrition domains of the data explorer pages https://fewsnet.atlassian.net/browse/DATA-955
Exchange rate data series naming errors in the user interface of the FDE https://fewsnet.atlassian.net/browse/DATA-1262
Error Importing Price Data from CSV https://fewsnet.atlassian.net/browse/DATA-1274
Data Inventory Dashboard - The tables for Cumulative Number of Data Points and Number of data points by month show incorrect time series on x-axis https://fewsnet.atlassian.net/browse/DATA-1299
Geographic filter is disabled for all the subtypes of the OTHER data domain https://fewsnet.atlassian.net/browse/DATA-1304
Searching by the parameters of the search box of the OTHER data domain results in data of all the data domains https://fewsnet.atlassian.net/browse/DATA-1305
Restore days work available and remittances from semi-structured domain https://fewsnet.atlassian.net/browse/DATA-1310
The period dates are disabled for all the subtypes of the OTHER data domain https://fewsnet.atlassian.net/browse/DATA-1315
Sub type column is shown blank in the UI of the Other data domain https://fewsnet.atlassian.net/browse/DATA-1316

Features

Calculated Price Index & Calculated Price Ratio end points of the FDE should point to their specific end points https://fewsnet.atlassian.net/browse/DATA-1247

This enhancement is about making changes to the FDE API end points of calculated price index and calculated price ratio .
For the calculated price index :
- The data series is pointing to https://fdw.fews.net/api/calculatedpriceindex/
- The data points/tabular view is pointing to https://fdw.fews.net/api/calculatedpriceindexvaluefacts/
- The Export link is pointing to https://fdw.fews.net/api/calculatedpriceindexvaluefacts/
And for the calculated price ratio :
- The data series is pointing to https://fdw.fews.net/api/priceratio/
- The data points/tabular view is pointing to https://fdw.fews.net/api/priceratiovaluefacts/
- The Export link is pointing to https://fdw.fews.net/api/priceratiovaluefacts/

Feature request for adding export button to the exchange rate data series of the FDW https://fewsnet.atlassian.net/browse/DATA-1248

This feature is about adding the Export button to the exchange rate data series of the FDW (i.e. https://fdw.fews.net/en/admin/price/exchangerate/). The export file contains all of the metadata for each exchange rate data series, including the associated data usage policy (e.g. Public, FEWS NET, restricted).

Adding data series ID to available data series "info" field https://fewsnet.atlassian.net/browse/DATA-1263

This enhancement is about adding the data series ID to the available data series Info icon of the FDE.

Prototype of PIN Retrieval Tool https://fewsnet.atlassian.net/browse/DATA-1264

This feature is a prototype for implementing a tool that allows users self-service retrieval of the Population in Need data using the superset dashboard.

Updating the validation error message for checking the availability data of the base period of each component selected for a Calculated Price Index https://fewsnet.atlassian.net/browse/DATA-1267

This enhancement is about changing an existing validation error message which notifies the user that the index component does not have data for the given base period from The index component must have data points for the base period to Base period data missing. Please ensure index component has data points for the base period .

Include schedule field to the data source document extract https://fewsnet.atlassian.net/browse/DATA-1306

This feature is about adding the schedule field to the data source document extract (i.e. https://fdw.fews.net/en/admin/warehouse/datasourcedocument/export/?). The schedule field indicates the frequency of data collections (e.g. monthly or weekly).

Include data_usage_policy field in the simple fields extracts of the exchange rate and secondary price index of the FDE https://fewsnet.atlassian.net/browse/DATA-1309

This enhancement is about adding the data_usage_policy field to the simple fields extracts of the exchange rate (http://fdw.fews.net/api/exchangeratevaluefacts/?format=csv&subtype=ExchangeRateValue&fields=simple) and secondary price index (http://fdw.fews.net/api/priceindexvaluefacts/?format=csv&subtype=PriceIndexValue&fields=simple) of the FDE.

Add filters.ModelMultipleChoiceFilter on DataUsagePolicy, Country and CountryGroup to DataSourceDocumentFilter https://fewsnet.atlassian.net/browse/DATA-1312

This feature is about enhancing the data source document API which allow users to extract list of documents based on:
- One or more specific data usage policies (e.g. https://fdw.fews.net/api/datasourcedocument/?format=html&datausagepolicy=Public)
- One or more countries (e.g. https://fdw.fews.net/api/datasourcedocument/?format=html&country=Kenya&country=Uganda or https://fdw.fews.net/api/datasourcedocument/?format=html&country_code=KE&country_code=UG
- One or more country groups (e.g. https://fdw.fews.net/api/datasourcedocument/?format=html&geographic_group=911&geographic_group=912)

In addition to this, the datapoint_count has also been added to https://fdw.fews.net/api/datasourcedocument/?format=html.

Add geographic_unit field in the simple fields extract of the calculated price index of the FDE https://fewsnet.atlassian.net/browse/DATA-1318

This enhancement is about adding the geographic_unit field to the simple fields extract of the calculated price index of the FDE (i.e. https://fdw.fews.net/api/calculatedpriceindexvaluefacts/?format=csv&subtype=CalculatedIndexValue&fields=simple).

Tasks

Create geographic unit tree for exchange rate domain https://fewsnet.atlassian.net/browse/DATA-1303

This backend task was about implementing the geographic unit tree structure (Regions>>Countries>>Markets) for the exchange rate data domain (i.e. https://fdw.fews.net/api/geographicunittree/?used_for=exchangerate&format=json).

Bugs

Issue of loading the Geographic filter on Crop and Nutrition domains of the data explorer pages https://fewsnet.atlassian.net/browse/DATA-955

There was an issue in loading the data values of the geographic filter for the Crop and Nutrition data domains of the FDE and this was resolved.

Exchange rate data series naming errors in the user interface of the FDE https://fewsnet.atlassian.net/browse/DATA-1262

In the Other data domain of the FDE, there was a naming error in the UI of the exchange rate data series with OANDA data source document and this was resolved.

Error Importing Price Data from CSV https://fewsnet.atlassian.net/browse/DATA-1274

There was an error in importing price data with a csv format. This was resolved and the functionality is working as expected.

Data Inventory Dashboard - The tables for Cumulative Number of Data Points and Number of data points by month show incorrect time series on x-axis https://fewsnet.atlassian.net/browse/DATA-1299

There were incorrect x-axis values which were shown in the tables of the cumulative number of data points by date and number of data points by month of the data inventory dashboard. This was resolved and the correct values are now shown in the tables.

Geographic filter is disabled for all the subtypes of the OTHER data domain https://fewsnet.atlassian.net/browse/DATA-1304

The geographic filter drop down list was disabled for some of the sub types of the Other data domain (Days work available per week , Demography , Remittances, Calculated Price Index, Secondary Price Index and Calculated Price Ratio) of the FDE and this was resolved.

Searching by the parameters of the search box of the OTHER data domain results in data of all the data domains https://fewsnet.atlassian.net/browse/DATA-1305

The search parameter of the Other data domain had given incorrect results when no Sub type filter was selected. This was resolved and the functionality is working now as expected.

Restore days work available and remittances from semi-structured domain https://fewsnet.atlassian.net/browse/DATA-1310

The data series of the semi-structured data domain (i.e. Days work available & Remittances) were shown as zero data point. This was resolved and the values are now shown correctly (i.e. https://fdw.fews.net/en/admin/semistructured/semistructureddataseries/?document_type=days_work_available and https://fdw.fews.net/en/admin/semistructured/semistructureddataseries/?document_type=remittances).

The period dates are disabled for all the subtypes of the OTHER data domain https://fewsnet.atlassian.net/browse/DATA-1315

The period dates (i.e. Start date and End date) were disabled for all the subtypes of the Other data domain of the FDE (https://fdw.fews.net/data-explorer/#/other) and this was resolved.

Sub type column is shown blank in the UI of the Other data domain https://fewsnet.atlassian.net/browse/DATA-1316

The UI Screen of the Other data domain of the FDE (i.e. https://fdw.fews.net/data-explorer/#/other) did not show the data for the Sub type column information and this was resolved.