Stack OverflowGuilherme AppelProblem & solution
Your .NET parallel loop can become serial in a one-CPU container
Summary
In a .NET container with a one-CPU limit, Parallel.ForEachAsync defaults its MaxDegreeOfParallelism to Environment.ProcessorCount, which returns 1, causing I/O-bound calls to run serially. The author’s lab processed 200 records, each requiring three enrichment calls and one write, taking 83.6 seconds with one CPU versus 42.1 seconds with two. The solution uses a DependencyBudgets class with separate SemaphoreSlim instances per dependency to cap concurrency independently. Strategy C4, with 24 enrichment permits and 6 write permits, avoided all HTTP 503 rejections while accepting 35.08 records per second in a slow write scenario. The author recommends configuring per-dependency concurrency limits, validating them at startup, and using separate timeouts for service calls and permit waits.
Context
In .NET, Parallel.ForEachAsync without explicit MaxDegreeOfParallelism defaults to Environment.ProcessorCount, which is determined at runtime from the CPU limit, container CPU affinity, and logical processors. On a one-CPU container, this value is 1, causing I/O-bound work to be processed sequentially even though the work is waiting on external services. This can significantly increase completion time without obvious errors, as all records still get written. The post presents a lab experiment with simulated dependencies to compare four concurrency strategies.
Approach / What changed
The approach introduces a DependencyBudgets class that holds separate SemaphoreSlim instances for each external dependency. The outer loop uses Parallel.ForEachAsync with a MaxDegreeOfParallelism set high enough to keep both dependencies busy. Before each call to a dependency, the corresponding semaphore is acquired, ensuring that no dependency receives more concurrent requests than its configured limit. This is demonstrated in strategy C4 with 24 enrichment permits and 6 write permits. Limits should be determined from service capacities and validated at startup, with separate timeouts for the service call (started after permit acquisition) and for waiting on the permit.
Takeaways
Parallel.ForEachAsync without explicit MaxDegreeOfParallelism uses Environment.ProcessorCount as the default, which can drop to 1 in a one-CPU container and serialize I/O-bound work, doubling completion time in the author’s lab.
A shared concurrency limit across all dependencies can cause a slow dependency to consume permits, leading to rejections or reduced throughput; per-dependency budgets prevent this.
Separate timeouts for the service call (started after permit acquisition) and for waiting on the permit are recommended to avoid local queue time consuming the service timeout.
FAQ
Why does Parallel.ForEachAsync become serial in a single-CPU container?
Because Parallel.ForEachAsync defaults MaxDegreeOfParallelism to Environment.ProcessorCount, which returns 1 when the container's CPU limit is 1. This causes I/O-bound calls to be processed sequentially, increasing completion time.
How can I limit concurrency per dependency instead of globally?
Use separate SemaphoreSlim instances per dependency, as shown in the DependencyBudgets class. This allows each dependency to have its own concurrency limit, preventing a slow dependency from consuming all permits.
What is the trade-off between more concurrency and protecting downstream services?
More concurrency can increase throughput but may overwhelm services with limited capacity. The lab shows per-dependency budgets (C4) avoid rejections even if slightly slower, while global limits (C2, C3) cause 503 errors when the system of record is overloaded.
Source and freshness
- Source
- stackoverflow.blog
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.