System.OutOfMemoryException: OutOfMemoryException - Failed to process document due to memory issue. – ZappySys Help Center

Problem

If you are using JSON Source or XML Source and suddenly you start getting below error.

System.OutOfMemoryException: 
OutOfMemoryException - Failed to process document due to memory issue.

Cause

OutofMemory error indicates that entire XML files are loaded in memory and the Parser reached its limit to handle nodes it can parse in memory.

Possible Solution

To solve that change the following settings to use streaming and avoid full document load in memory. There are a few ways you can avoid this.

First, to generate metadata at design time we suggest you to use a smaller file rather than a really large file. Once Metadata is generated you can try to change the file path to a large file and then tweak settings like below it helps to avoid Runtime Error.

STREAM MODE: Append --FAST option in your filter expression. ( e.g. $.data.rows[*]--FAST ). This enables data streaming rather than loading full files in-memory
STREAM MODE: If you don't care parent attributes then uncheck [Include parent] option on Filter Options Tab.
STREAM MODE + URL ACCESS MODE: Make sure you did not set [Data Format] option. It should be [Notset]. And Un-check the [Indent].
FOR XML PARSING: Check [Enable Performance Mode] option to enable stream mode for large files. Make sure array node name is entered under array handling tab (e.g. if your filter is $.DATA.ROW[*] then ROW must be entered as array name). If you have multiple arrays under same root then you can also enter node names you like to exclude under Excluded Properties under [Advanced Filter Options] tab to avoid OutOfMemory error.
DESIGN TIME: If possible use smaller file during design time so you can browse structure using full scan (For Select Filter). Once metadata is set you can change path to real file via properties grid or using Runtime variable/expression.
FOR XML PARSING: If you select [Output as RAW document] option then check [Convert to JSON] option for faster parsing downstream (Useful for extracting multiple arrays from same documents).
FOR URL MODE: Make sure you did not set pagination-related options else stream mode is disabled.