Network.getResponseBody hangs when data is 150MB+

[priutex]

Hi,

Background: I have the internal buffers set to 2GB
driver.execute_cdp_cmd('Network.enable', {'maxResourceBufferSize':2048000000, 'maxPostDataSize':2048000000, 'maxTotalBufferSize':2048000000})

Then once my page is loaded with data, I'm using the following code to extract the response body of my request:
driver.execute_cdp_cmd('Network.getResponseBody', {'requestId': req_id})

It works well running in headless mode for response bodies under 150MB, but for those response body data that is above 150MB or so, the above request just hangs. The driver doesn't get any data back.

The website i'm trying to parse successfully loads a maximum of 1GB of data to their tables that are paginated, but if you look at chrome network inspector, the data response has been evicted from cache.

In the headless mode, where is this response data stored? This Network.getResponseBody approach works well, but why does it hang when the data is large? Is there some other max limit set somewhere?

Thank you.