The following list of new features, updates and bug fixes are included in the latest Sprint 23.06 release to the FEWS NET Data Platform.
New Features
Data Freshness Dashboard Prototype (DATA-2335)
A dashboard prototype is designed to allow users to view the current status of the Data Collections for the Data Source Documents they have access permissions for. Data collections that are scheduled but not yet loaded will reflect a Missing status.
Data usage policy notes in FDE metadata (DATA-2415)
The data usage policy notes for each source document are displayed in the Metadata tab in a new Usage Requirements column, as well as in the Customize sidebar through a new data Usage Requirements drawer.
Updates
Pipeline for ACLED data (DATA-2807)
A new pipeline can pull data from ACLED into FDW. Data is currently ingested for FEWS NET presence and remotely monitored countries.
Support events data in semistructured domain (DATA-2813)
A new contains_events field in FDW for document_type allows data with overlapping period dates, to support events data which may include multiple events in one day.
Create migration flow geographic units (DATA-2814)
A new geographic feature type in FDW for migration_flow captures migration data involving the population origin and current location. The origin_pcode and affected_pcode are stored in the feature.attributes field.
Load IOM data into semistructured domain (DATA-2815)
IOM migration data can be stored in the semistructured domain in FDW. Data will be added manually as it is produced on an ad hoc basis.
FDE Updates
Improvements to the FDE were completed through the following tickets:
Added a “reset to defaults” button to the Customize sidebar (DATA-2442)
Improved performance of the FDE by cacheing frequent API calls (DATA-2628)
Aligned the time series charts in FDE with the 16 color selection order from the FEWS NET Chart Style Guidance and adjusted the preview map scales to follow the new color pattern outlined in the FEWS NET Map Style Guidance (DATA-2761)
Ensured all data entry fields in each FDE domain search header have accurate field labels that are visually hidden using a WCAG-compliant method, and revised text colors to create accessible contrast(DATA-2774)
A search bar was added to the Crop domain (DATA-2528)
Add outlook related table for Assumptions database (DATA-2799)
Created the concept of outlook periods in FDW as a database table that all reports for a single outlook can be linked to. This will allow the Assumption reports and report items to be queried for those belonging to a single outlook.
Create API Endpoint to render PNG format of shapefiles (DATA-2447)
A new API endpoint will return a PNG image for a given layer using the BaseRenderer class PNGRenderer that has been added to FDW.
FAOB Ingestion
A single file for Highest FIC, FIPE, internal, and public data can be uploaded and automatically divided by FDW into separate data source documents (DATA-1110)
Monthly FAOB data can be uploaded to FDW through the Add data collection screen (DATA-2216)
Separate Data Warehouse source document to have different visibility levels (DATA-2575)
Split the Data Warehouse source document into three Calculated Dataseries Data warehouse documents with public, FESNET Only and Restricted usage policies, with an additional source document for the Geographic unit versions. Existing data series have been migrated to the correct source document.
XBT Ingestion pipeline from Kobo toolbox (DATA-1669)
A data ingestion pipeline pulls data saved through the East Africa XBT Form to FDW.
Create Market Price API endpoint for ODK related exports (DATA-2289)
Market price data-series from FDW can be accessed in ODK through an API extracting the data in CSV format, using the query parameter value odk.
Fix Geographic Unit metadata endpoint to return a dictionary (DATA-2428)
Updated the metadata end point for geographic unit to return a formatted dictionary like the other metadata end points, rather than a string.
Raise an error when season metadata is changed when there is existing data for that season (DATA-2653)
A new validation check will raise a warning when a user attempts to change season metadata that has data associated with the season, in order to prevent unnecessary changes.
Bug Fixes
FDE Bug Fixes
User issues with saving datasets in FDE have been resolved (DATA-2580).
Existing users who attempt to create a new account will receive an error message confirming the existing account (DATA-2741).
Livelihood Zone and FSC Livelihood Zone filters in the FDW (DATA-2742)
Corrected the behavior of the unit type filters Livelihood Zone and FSC Livelihood Zone in FDW due to an issue with livelihood zone shapefiles being returned by the FSC Livelihood Zone unit type filter, rather than the Livelihood Zone filter. The filters now return the expected results.