Can emscripten fetch work in parallel when called from multiple threads?

[SeanBottomley]

Hi,

I am trying to port a c++ application which can stream data which is then displayed via GL. The code uses multiple threads grab chunks of very large files. I have tried both synchronous and asynchronous fetches from multiple threads but it feels like something is blocking which results in consecutive fetches rather than something more concurrent

[sbc100]

Can you explain more about what you mean "tried both synchronous and asynchronous fetches". What APIs are you referring to exactly?

[SeanBottomley]

The Fetch API

[sbc100]

I believe the emscripten/fetch.h API should work completely separately on each thread, in that there is no proxying or synchronizing with the main thread that happens with that API specifically.

Are you doing anything with the fetched data such as writing it to the filesystem? (Which does require synchronization). How about the EMSCRIPTEN_FETCH_PERSIST_FILE option are you using that?

[SeanBottomley]

no, it isnt being written to the filesystem and no i'm not by default using EMSCRIPTEN_FETCH_PERSIST_FILE but am sure i did try it. ATM i am trying to via Javascript write directly to XMLHttpRequest, i'm not 100% there yet but initial signs show that this approach may work - i've definitely had 2 requests work as i'd expect.