Hacker News Reader

Comment by maxfurman | Hacker News Reader

What's the right way to do throttled async in modern C#? For some context, we have a process that needs to make an API call for each row in a file - maybe hundreds or thousands. What's the best way beyond Wait()'ing for each one to get decent performance without DOS'ing the server?

View on HN

Use System.Threading.Channels.

BoundedChannelFullMode.DropNewest, DropOldest, DropWrite, Wait specifies the behavior to use when writing to a bounded channel that is already full

Thank you! I didn't know that existed. I gotta test the performance of that compared to something like a list or array with an explicit lock, which is otherwise my go to solution precisely for performance reasons.

I personally used a semaphore for that. You create a semaphore with an initial count of MAX_REQS_PER_SECOND, create WORKER_COUNT of looping "worker" tasks that each call WaitAsync() on that semaphore before doing request (and don't call Release() after request is done), plus a separate task that does either

    Sleep(100);
    Release(MAX_REQS_PER_SECOND / 10);

   Sleep(1000 * WORKER_COUNT / MAX_REQS_PER_SECOND);
   Release(WORKER_COUNT);

in a loop, depending on what numbers make more sense.

Fire off one task per row, but within each of those tasks use a SemaphoreSlim to rate limit your requests to the API.

Can you use MaxDegreeOfParallelism or var throttler = new SemaphoreSlim(initialCount: MAX_CALLS)?

Maybe Bulkhead policy in Polly is a good match here?

Reply on news.ycombinator.com