Page Comparison

Dated: 12 July 2021

Introduction

The following list of new features, tasks, and resolved bugs are included in the latest Sprint 21.06.1 release to FDW/FDE:

Features

• Investigate dissolve performance in /api/ipcpasemap - https://fewsnet.atlassian.net/browse/DATA-621

• Replace Staple Food Price Data Set on Data center with direct download from FDW - https://fewsnet.atlassian.net/browse/DATA-906

• Review the pre-processing and matching of DataSourceDocuments in uploads - https://fewsnet.atlassian.net/browse/DATA-935

• Add collection_start_date and collection_end_date filters to DataPointFilter - https://fewsnet.atlassian.net/browse/DATA-968

• Support fields parameter for geojson extracts of DataPointViewSet - https://fewsnet.atlassian.net/browse/DATA-969

• Support As-Is imports on the three usual import screens https://fewsnet.atlassian.net/browse/DATA-971

• As a Food Security Analyst, I want to be able to view the admin 1 & 2 names when I extract market price data https://fewsnet.atlassian.net/browse/DATA-974

•  As a Food Security Analyst, I want to be able to view the CPI subnational data associated with an admin 1 in my extract https://fewsnet.atlassian.net/browse/DATA-975

• Standard Price Charts in Superset - https://fewsnet.atlassian.net/browse/DATA-987

Tasks

• Trade Flow resource generates lots of duplicate queries https://fewsnet.atlassian.net/browse/DATA-950

• Enable extraction of reusable visualizations JSON format - https://fewsnet.atlassian.net/browse/DATA-993

...

Investigate dissolve performance in /api/ipcpasemap - https://fewsnet.atlassian.net/browse/DATA-621

• This feature improved the performance of GeoJSON downloads for map visualizations by dissolving internal boundaries prior to extraction so that the download doesn't need to contain any of the

...

• internal

...

• boundaries under each phase; for instance

...

• :

  • 1) https://fdw.fews.net/api/ipcphasemap/?format=geojson&country_code=MZ&collection_date=2020-10-01&scenario=ML1&_refresh=true

...

• • 2) https://fdw.fews.net/api/ipcphasemap/?format=geojson&country_code=NG&collection_date=2020-10-01&scenario=ML1&_refresh=true

...

• • 3) https://fdw.fews.net/api/ipcphasemap/?format=geojson&country_code=MG&collection_date=2020-10-01&scenario=ML1&_refresh=true

...

• • .

Replace Staple Food Price Data Set on Data center with direct download from FDW - https://fewsnet.atlassian.net/browse/DATA-906

• This implementation allows the direct download of the price data file from the data center on the FEWS NET website (https://fdw.fews.net/api/marketprice.xlsx/?dataset=FEWS_NET_Price_Data)

  and the downloaded file should also contain contains the selected fields fit for the website extracts and a read me tab as well.

Review the pre-

...

processing and matching of DataSourceDocuments in uploads - https://fewsnet.atlassian.net/browse/DATA-935

• This feature is about making updating the data_source_document field with a trailing space to be recognized in an uploading process.

Add collection_start_date and collection_end_date flters to DataPointFilter - https://fewsnet.atlassian.net/browse/DATA-968

• This feature enables users to easily update local data visualizations each month via the API by using the enhancement is about incorporating the collection_start_date and collection_end_date filters in the Data Point Filter; for instance ,(to the DataPointFilter , DataCollectionPeriodFilter and DataCollectionFilter which enables users to easily update local data visualizations each month via the API, as seen:

  • https://fdw.fews.net/api/ipcphase/?format=geojson&collection_date_start=2021-04-01&country_code=KE , https://fdw.fews.net/api/datacollectionperiod/?format=json&collection_start_date=2021-04-01&data_domain=IPCCLASSIFICATION

  and

  • https://fdw.fews.net/api/datacollection/?format=json&collection_start_date=2021-04-01&data_domain=IPCCLASSIFICATION ).

Support fields parameter for geojson extracts of DataPointViewSet - https://fewsnet.atlassian.net/browse/DATA-969

• This enhancement is about incorporating the fields parameter for GeoJSON extracts of the DataPointViewSet API endpoint, for instance; (example: https://fdw.fews.net/api/ipcphase/?format=geojson&collection_date=2020-10-01&country_code=KE&fields=id,scenario,start_date,end_date,collection_date,value).

Support As-Is imports on the three usual import screens https://fewsnet.atlassian.net/browse/DATA-971

• This feature is about implementing the As-Is imports in the three import screens ;data import screens of the FDW. Users can now upload as-is data (i.e. data that has not been cleaned to match the metadata standards) into the FDW in the same way that data is currently uploaded, for example:

1. Offline Data Uploads screen (https://fdw.fews.net/en/admin/warehouse/dataupload/add/)

...

2. Add Data Collection screen  (https://fdw.fews.net/en/admin/warehouse/datacollection/add/)

...

3. Semi-Structured Data Point Admin (via import button on list view)  Screen (https://fdw.fews.net/en/admin/semistructured/asisdatapoint/import/)

...

As a Food Security Analyst, I want to be able to view the admin 1 & 2 names when I extract market price data https://fewsnet.atlassian.net/browse/DATA-974

• This feature is about allowing the inclusions of the names of the Admin1 and Admin2 names in the market price data extract extracts.

As a Food Security Analyst, I want to be able to view the CPI subnational data associated with an admin 1 in my extract https://fewsnet.atlassian.net/browse/DATA-975

• This enhancement is about showing the data of the CPI subnational data which are linked with Admin1 in the data that will be extracted.

...

• units, (for instance, https://

...

• fdw.

...

• fews.net/api/

...

• This feature developed standard price charts in superset and applied customization where possibledatapointfacts/?format=html&dataseries=26133&fields=simple).

Tasks

Trade Flow resource generates lots of duplicate queries https://fewsnet.atlassian.net/browse/DATA-950

• Since the trade flow resource generates multiple queries which will make the data upload process slow so , this task was about generating fewer queries which makes the data upload process faster as a result .

Enable extraction of reusable visualizations in JSON format - https://fewsnet.atlassian.net/browse/DATA-993

• This task was about allowing the extraction of a reusable visualization in a JSON format, for instance , i.e:

  • https://fdw.fews.net/viz/viz_visualizations_price_augmentedpricechart.json?dataseries=314489

  • https://fdw.fews.net/viz/viz_visualizations_price_currentprevioushistoricpricechart.json?dataseries__in=314459&historic_range=2y

  • https://fdw.fews.net/viz/viz_visualizations_price_averagepricechangemap.json?dataseries__marketproduct__product=R01112AA&historic_range=1m&dataseries__geographic_unit__country=SD&period_date=2021-02-28

Bugs

Zimbabwe common currency prices not being calculated in data extracts - https://fewsnet.atlassian.net/browse/DATA-533

• The Zimbabwe common currency prices were not being calculated or presented in data extracts . This issue and this was resolved.

Fix price charts located under the “Old Charts” tab - https://fewsnet.atlassian.net/browse/DATA-560

• There was an issue in the augmented price chart and this was resolved. The where the x-axis label has been reviewed, three and tab names have also been changedwere incorrect, and this has been updated and resolved.

...

Error uploading ET 2018 and 2009 LHZ shapefiles - https://fewsnet.atlassian.net/browse/DATA-784

...

• There was a problem in selecting data values for some pages of the FDW, for instance , (:

  • https://fdw.fews.net/en/admin/warehouse/datacollection/add/

  ,

  • https://fdw.fews.net/en/admin/price/marketproduct/add/

  ,

  • https://fdw.fews.net/en/admin/price/exchangerate/add/

  ,

  • https://fdw.fews.net/en/admin/price/priceindex/add/

  and

  • https://fdw.fews.net/en/admin/warehouse/datasourcedocument/add/

  )

  • 
    This was due to the implementation of the new autocomplete functionality and this was resolved and all the pages are working as expected.

MarketProduct search doesn’t recognize partial Price Types - https://fewsnet.atlassian.net/browse/DATA-961

• There was a problem in  in the /api/marketproduct/ end point  point in making search with a partial terms of the price_type , for instance with whol instead of the full one wholesale or ret instead of the full retail and this . This was resolved and can be searching searched using these partial terms, (such as https://fdw.fews.net/api/marketproduct/?format=json&search=eth mai ret or
  https://fdw.fews.net/api/marketproduct/?format=json&search=nair mai whol ) .

Error extracting Haiti and Madagascar availability data - https://fewsnet.atlassian.net/browse/DATA-1000

• There was a problem in extracting the availability data of the countries Haiti and Madagascar  and Madagascar and this was resolved.

Collection date field is not automatically populated on the data collection page - https://fewsnet.atlassian.net/browse/DATA-1002

• The collection_date field was not populated automatically, which it should have been, while uploading data on the data collection entry page (https://fdw.fews.net/en/admin/warehouse/datacollection/add/). This was resolved and the functionality is now working as  expectedas expected.