The following list of new features, updates and bug fixes are included in the latest Sprint 24.04 release to the FEWS NET Data Platform.
New Features
Create alerting system for failing Luigi tasks (DATA-3577)
Email alerts that report on failing tasks can now be added to FDW pipelines on an ad hoc basis.
Updates
Cached Response Updates
Work to detect and improve cached responses was completed through the following tickets:
A new task calculates a list of responses that we expect to be cached (staple foods market prices, various IPC extracts, etc.) and then attempts to download them in order to report on the cache hit, confirming whether the correct data is being returned (DATA-3152).
Failing caches can be detected and removed to prevent the extra load being applied to celery medium (DATA-3166).
The content size of the cached responses stored in FDW are now included in the cached response model (DATA-3529).
The production database instance size was upgraded in order to increase the number of cache responses that can be processed in a given time frame (DATA-3692).
The cache request for an FIC map data visualization will expire when changes are made to the corresponding FIC data in FDW, preventing outdated data from appearing when the map is accessed (DATA-3699).
Key Drivers Map Updates (DATA-3662)
Improvements were made to the Key Drivers Map data visualizations, including:
The base layers and time series overlays are both selected from a drop down above the time slider
It is now clearer when data is not available for specific countries
Monthly data is shown by default
Crop Assessment Dashboard Updates (DATA-3570)
In addition to adding two new forms to the dashboard in FDW, a new Select Survey Year dropdown was created. The Summary Crop Details table has also been reordered to place indicators without data for the user’s selection at the end of the table.
Implement regular comparison between FDW FIC data collections and CKAN datasets (DATA-3398)
A new celery task compares FDW and CKAN published datasets, including countries and country groups, on a daily basis.
Add FEWS NET branding to PDF formats of metadata endpoint outputs (DATA-3547)
The PDF format extracts from the FDW API now have FEWS NET branding and improved formatting.
Return a better error when endpoint is filtered by `datasourcedocument` that has no data series (DATA-3543)
The FDW API now returns a more descriptive error when any of domain data series endpoints is filtered by a datasourcedocument that does not have any related data series.
Check auto complete views apply user permissions to the return values (DATA-3418)
Dataset autocomplete views will now only show data the user has permissions to view, using a parent dataset autocomplete class in the warehouse app that takes user permissions into account.