The following list of updates and bug fixes are included in the latest Sprint 24.10 release to the FEWS NET Data Platform.
New features
IOM data ingestion
Work to ingest internal displacement data from IOM was completed under the following tickets:
A new pipeline ingests data on from the IOM DTM API to the FDW (DATA-4113).
As IOM reports its data uses P-codes (place codes) that needed to be matched to FNIDs, a
pcodefield was added to the existing GeographicUnit and Feature models in FDW and populated with the relevant codes (DATA-4149).
Create ingestion pipeline for the SMPG reports (DATA-4121)
A pipeline can pull the data used for SMPG reports from USGS, create the data collections needed, and then calculate the data points.
Updates
...
FDW now supports administrative levels for Crop Production Units through the addition of new Unit Types.
Resolve spatial representation of IPC data for remote monitoring countries (DATA-819)
...
A Luigi Task periodically checks across CKAN to determine whether any resource URLs linking to FDW are broken.
...
More useable implementation of publication_sources field in FDE metadata (DATA-
...
3732)
Added Improved the submittedpublication_bysources field to in the HTML endpoint of the API as it was only appearing in the JSON format. The field was also added to the Data Point extracts as an additional field for Crop Production Indicator and facts Metadata tab of FDE. This included repositioning it after the documents section and renaming the DATASOURCEDOCUMENT NAME column to SOURCE DOCUMENT NAME.
Follow up on Market Functions (DATA-4030)
Updated the structure of trade routes so that Source Market and Destination Market are now tied to the Trade Route Geographic Unit. The Trade Route Geographic Unit Type will be road.
Filter data series in FDE domains by active or inactive (DATA-4056)
Data series can now be filtered by active or inactive status in FDE using a new option in the search and filter header of the domain pages.
Add icon for submitted_by as a field for Crop in the metadata endpoint (DATA-4090)
Added an icon to allow the new submitted_by field to render correctly in FDE.
Allowing saving of country local names for country classified products and returning them on the queries (DATA-4120)
Local names for classified products stored on CountryClassifiedProductAliases can be returned by the API with the new used_in_country and used_in_country_group filters that were added to ClassifiedProductViewSet.
...
Trigger Reset to default button when Use market-level Exchange Rates is selected (DATA-
...
4126)
Data series can now be filtered by active or inactive status in FDE using a new option in the search and filter header of the domain pages.
More useable implementation of publication_sources field in FDE metadata (DATA-3732)
Improved the publication_sources field in the Metadata tab of FDE. This included repositioning it after the documents section and renaming the DATASOURCEDOCUMENT NAME column to SOURCE DOCUMENT NAME.
...
The Reset to defaults button will now appear in the Customize sidebar of the FDE Prices domain when the Use market-level Exchange Rates option is selected.
Support administrative hierarchy for crop regions (DATA-4148)
FDW now supports administrative levels for Crop Production Units through the addition of new Unit Types.
Bugs
No error for taken username (DATA-4047)
The error messages during account creation in FDE have been updated to clearly indicate when a username is already taken.
FAO price ingestion failure (DATA-4105)
The FAO price ingestion pipeline failure recurring due to metadata mismatches was repaired. The pipeline now runs successfully.
Fix API for extracting Shapefiles for global LHZ boundaries (DATA-3874)
The global LHZ spatial boundaries could be extracted in KML and GeoJSON, but failed for Shapefiles due to a character restriction for the column names. The API can now generate a global LHZ Shapefile with column names that match the columns in the global Shapefile on the FEWS NET website.
Issue of selecting and clearing the Admin 2 units of the HFA Domain (DATA-4138)
...
The Clear sections button was not clearing the selected values of the Data Usage Policy filter of the Acute Food Insecurity Classifications Domain in FDE. The selected values can now be cleared from the filter box.
...