Following Data entry, Data processing is the second step in the data ingestion process. The Data processing module allows users to review and approve submitted data and source documents in preparation for publishing to the FEWS NET Data Warehouse. Once published, the data becomes available for users to view and export in the Data Explorer system.
From the Data processing module, the following actions can be performed:
Display or edit an existing data collection
Display or edit the data for an existing collection period
Change a survey collection status
Display or edit an existing data collection
From the system wide menu bar, select Data processing > Display or edit an existing Data Collection (including Submit, Review, and Publish).
This will bring up a data collections page that lists the currently available collections in a grid format with the following columns:
Source document: The name of the source document as defined in the Data entry process.
Document type: The domain or data type qualification that the data is being submitted under.
Collection date: The collection date of the submitted data collection.
File: Link to the submitted file. Because data can be entered manually, not all collections have a file available.
Status: Status types include, Draft, Submitted, Under Review, Published.
Publication name: The name of the publication that the data source document came from.
Status changed: Indicator of when the status was last updated.
Data point count: Indicator of file size as represented by the number of data points submitted in the data collection.
Navigating the data collections table
In addition to a free text criteria search bar, filters can also be applied. To set a filter criteria, locate the Filter menu. Once a filter is selected from the list, the system will automatically apply the filter and display the results.
Editing the status information of a data collection
To make changes to the status information of a data collection:
Select the data collection to be updated
Modify the data in the Change data collection page as needed
Select one of the Save options
Deleting a data collection
Select the data collection to be deleted.
Locate and select the Delete button to delete the data collection.
Deleting one or more data collections
Select the data collections to be deleted.
Locate and open the Action drop-down list and select Delete selected unpublished data collections.
Once the action is selected, click Go.
Note: Deleted records cannot be recovered and this functionality should be used with caution. Only users with special permission can delete a data collection.
Viewing change history of a data collection
When various events are performed against data within a data collection, including the addition of new data, updates, and deletion of data in the system, they are tracked for auditing purposes.
To view the change history of a data collection:
Select the data collections for which the change history is needed.
Locate and select the History button.
Submitting and Publishing Country-Level Data
The order of operations for the data review and approval process are outlined below.
Entering Data: New data are entered or uploaded by the Deputy Country Representative (DCR) using Data entry > Add a new data collection.
Submitting Data: The new data is submitted by the Country Representative (CR):
The Country Representative (CR) selects the Data Collection to be submitted by going to Data Processing > Display or edit an existing Data Collection (including Submit, Review and Publish) and reviews the uploaded data. If there are anomalies, the CR will inform the DCR to check those anomalies.
The DCR addresses issues raised by the CR by completing the following steps:
Click on the Edit icon for the entries which are still in Draft status.
Apply the necessary changes and then select Save or Save and continue editing.
If any data needs additional explanation, write a comment in the Comment box and select Post Comment. (e.g., Leave a comment if a price has changed significantly from the previous month and the reason is unknown or if data has been corrected after being placed Under Review.)
Notify the CR that the data are ready for review.
Once the data are ready to be submitted, the CR opens the Data Collection page and presses the Submit button.
Note: Data will not appear in reports and extracts until they have been submitted.The CR notifies the Regional Food Security Specialist, M&T (RFSS-MT) in the Regional Office (RO) that the data is ready for review.
Reviewing Data: The Regional Food Security Specialist, M&T (RFSS-MT) in the Regional Office (RO) reviews the data collection using the Data processing module:
If the data are determined to be complete and accurate, the RFSS-MT in the RO puts a comment in the Comment section which notifies the Home Office (HO) M&T Analyst assigned to the region that the data are ready to publish.
If the data are not complete or accurate, the RFSS-MT in the RO communicates with the CR about the anomalies.
The CR completes the following steps:
Select the Review button to change the status of the Data Collection to Under Review.
Explain why the data are Under Review using the Comment box and select Post Comment.
Data that are placed under review will need to be corrected and resubmitted by following the process in Step 2 (Submitting Data).
Once the data are corrected, the CR notifies the RFSS-MT.
Once the data is determined to be complete and accurate, the RFSS-MT in the RO puts a comment in the Comment section which notifies the Home Office (HO) M&T Analyst assigned to the region that the data are ready to publish.
Publishing Data: Once the data is determined to be complete and accurate by the RFSS-MT, HO will perform another review and complete the following steps:
If the data is ready to be published, open the Data Collection and click the Publish button.
If the data needs additional review, the Markets and Trade team completes the same steps as the Regional Market Specialist in step 3 to place the data Under Review.
Editing Published Data: To prevent accidental edits, a Published Data Collection cannot be altered. In the rare case where a correction is required to Published data, the RO updates the status to Under Review and adds comments to the Comment box. The DCR will then initiate a new review and approval process.
Note: If at any stage the data collection is Published, the Regional Office will be notified via an email alert. If at any stage the data collection is placed Under Review, the Deputy Country Representative will be notified via an email alert.
