Introduction
In this sprint release document, the following main features , tasks and defects are resolved and deployed to the production server.
Features
Customizable domains for semi-structured data https://fewsnet.atlassian.net/browse/DATA-97
Add Survey Type as part of the unique identifier for Survey Population Indicator, so that imports do not overwrite https://fewsnet.atlassian.net/browse/DATA-133
Automated Ingestion of Market Price Data from FAO using their API https://fewsnet.atlassian.net/browse/DATA-162
Fill geographic filter property even when item does not have parent id https://fewsnet.atlassian.net/browse/DATA-184
Enhance Data Inventory Dashboard to include Data Points by Domain and Usage Policy https://fewsnet.atlassian.net/browse/DATA-213
Sub National Crop Production Data Visualization https://fewsnet.atlassian.net/browse/DATA-224
Tasks
Bulk load of Crop Production Country Archives https://fewsnet.atlassian.net/browse/DATA-144
Develop list of Metadata Types and recommend maintainers https://fewsnet.atlassian.net/browse/DATA-210
Develop FDW/FDE Infrastructure Design Artifacts https://fewsnet.atlassian.net/browse/DATA-217
Develop FDW/FDE Software Design Artifacts https://fewsnet.atlassian.net/browse/DATA-218
Develop FDW/FDE Configuration and Misc Artifacts https://fewsnet.atlassian.net/browse/DATA-219
Bugs/Defects
Increase ECS memory allocation for Mayan celery_slow container https://fewsnet.atlassian.net/browse/DATA-147
FDW services sending double emails https://fewsnet.atlassian.net/browse/DATA-177
warehouse.tasks.cache_datasourcedocumentviewset_list is extremely slow https://fewsnet.atlassian.net/browse/DATA-204
DRC market exchange rates upload file not recognized https://fewsnet.atlassian.net/browse/DATA-221
Missing metadata from single price data series extract of the FDW https://fewsnet.atlassian.net/browse/DATA-227
IPC FIC drop down menu on the FDE https://fewsnet.atlassian.net/browse/DATA-228
Markets bulk upload results in errors https://fewsnet.atlassian.net/browse/DATA-237
Features/Tasks
Customizable domains for semi-structured data https://fewsnet.atlassian.net/browse/DATA-97
This feature is about customizing the domains to manage the semi-structured data types.
Add Survey Type as part of the unique identifier for Survey Population Indicator, so that imports do not overwrite https://fewsnet.atlassian.net/browse/DATA-133
This feature is about adding the survey type as a part of unique identifier which allows to create Data Source Documents with the same name but different Document Types which in turn allows to avoid the problem where loading crop production data overwrites existing crop production data of different survey type.
Automated Ingestion of Market Price Data from FAO using their API https://fewsnet.atlassian.net/browse/DATA-162
This feature is about using the APIs' for automating the ingestion of the market price data of the FAO.
Fill geographic filter property even when item does not have parent id https://fewsnet.atlassian.net/browse/DATA-184
This is about displaying the geographic filter list of the IPC FIC data domain with the correct level of hierarchy.
Enhance Data Inventory Dashboard to include Data Points by Domain and Usage Policy https://fewsnet.atlassian.net/browse/DATA-213
This feature is about visualizing the data points break down by data domains and data usage policies.
Sub National Crop Production Data Visualization https://fewsnet.atlassian.net/browse/DATA-224
This is about visualizing crop production data at the corresponding admin levels which in addition shows the spatial representation of the data according to the crop production value.
Bulk load of Crop Production Country Archives https://fewsnet.atlassian.net/browse/DATA-144
This task was about uploading the archive data of the crop production and all the data loaded into the system.
Develop list of Metadata Types and recommend maintainers https://fewsnet.atlassian.net/browse/DATA-210
This task was about creating all the list of metadata types and a recommendation as to who should administer them & it’s available at https://drive.google.com/drive/u/0/folders/1e_wsA5QpQTBkMlhLiy-DyyZ68GgqnAA8
Develop FDW/FDE Infrastructure Design Artifacts https://fewsnet.atlassian.net/browse/DATA-217
This task was about developing an artifact that contains: a Hardware list, Configuration/connection of hardware components (i.e. infrastructure design diagram), and Ports and Protocols Used and it’s available at https://docs.google.com/spreadsheets/d/16p4cUwyFUNn22gUdr-TPPeKFPWzwOPyOcHNeay0eZYY/edit#gid=0
Develop FDW/FDE Software Design Artifacts https://fewsnet.atlassian.net/browse/DATA-218
This task was developing an artifacts around the Software Design of FDE/FDW to include: a List of Software, System Data Types, a Data Flow Diagram, and API Designs & it is available at https://drive.google.com/drive/u/0/folders/1VKoFSeFfTx07hbX_6ibI4g5EuUe0fF-f
Develop FDW/FDE Configuration and Misc Artifacts https://fewsnet.atlassian.net/browse/DATA-219
This task was about documenting the current configuration within FDW/FDE for: Users, Roles and Privileges, and System Descriptions and it’s available at https://docs.google.com/spreadsheets/d/1u61h3p8E_HNAdlQHoJIBwCPhpUdbVT7zIUpya1pwjWs/edit#gid=0
Resolved Bugs/Defects
Increase ECS memory allocation for Mayan celery_slow container https://fewsnet.atlassian.net/browse/DATA-147
There was an issue upon loading a document using the bulk upload process and it was due to low memory allocation of the Mayan celery_slow container. This was resolved by increasing the memory of Mayan celery_slow container.
FDW services sending double emails https://fewsnet.atlassian.net/browse/DATA-177
The FDW services were sending duplicate emails while reseting password function and this was resolved.
Warehouse.tasks.cache_datasourcedocumentviewset_list is extremely slow https://fewsnet.atlassian.net/browse/DATA-204
The warehouse.tasks.cache_datasourcedocumentviewset_list was very slow and by introducing the performance improvements and the feature adding the survey type as part of unique identifier of the survey population indicator has improved the performance of this endpoint.
DRC market exchange rates upload file not recognized https://fewsnet.atlassian.net/browse/DATA-221
There was an issue in recognizing the monthly exchange rate values of the country DRC , this was caused due to meta data change in the file and this was resolved.
Missing metadata from single price data series extract of the FDW https://fewsnet.atlassian.net/browse/DATA-227
Some meta data information were not available on the extracts of the price data series and this was resolved by displaying those meta data under the /admin/price/marketproduct/
IPC FIC drop down menu on the FDE https://fewsnet.atlassian.net/browse/DATA-228
The scale filter values of the IPC FIC Data domain were shown as IPC20 and IPC30 and this was resolved . The values are now shown correctly as IPC v2.0 and IPC v3.0.
Markets bulk upload results in errors https://fewsnet.atlassian.net/browse/DATA-237
There was an issue upon trying to import markets as bulk from the import screen (https://fdw.fews.net/en/admin/spatial/market/import/) and this was resolved.
