Introduction
Recently I read a great blog post by Jeremy Lindsay about using async/await and Task.WhenAll to improve performance of C# code. It is something that every .Net developer should know and add to their tool belt when it comes to executing multiple tasks in parallel. It can dramatically boost application performance and reduce app run time. The cool thing is the async/await/Task related threading apis are easy to understand, use and implement.
In this blog post I want to build on top of what Jeremy shared, and talk about how we can use a combination of async/await, Task.WhenAny, Task.WhenAll to not only improve performance, but also throttle it based on max number of tasks/threads you want to have running in parallel.
If you haven’t read Jeremy’s blog post yet, and are not very familiar with the async/await or Task.WhenAll apis, please take a few moments to read it now, as it will help you better grasp the concepts covered here.
First, some context
There are times in application development when it’s highly desirable to execute tasks/do things in parallel bc the overall time taken to execute all of them can be dramatically reduced compared to executing them sequentially. For eg, if you want to call an external api for some data for each user in your application, so that you can populate some UI, you can and should make many calls to this api in parallel for each user. This will dramatically cut down the overall time taken to fetch all the data for all the users, which will result in faster load time for the UI.
However, there are times when you cannot make these parallel calls bc the calls have to be made in sequential order, for whatever reason. For eg, if you need to populate a UI that shows a salesperson’s info along with all his/her clients, you might need to get the salesperson’s data first including a list of clientIds, and then for a clientId, you can get the info for that client, and so on. In this example, you are out of luck. You cannot parallelize this process due to the limitation of the api or database design.
But, if you keep your eyes open and are always thinking in terms of parallelization and optimization, you will be surprised to see that ther are more opportunities to parallelize than you realize.
Running MANY tasks in parallel
In Jeremy’s example, we see that he uses Task.WhenAll to execute the 2 tasks in parallel, and is able to dramatically reduce the overall run time. Obviously sometimes you may have more than 2 tasks that can all be run in parallel, and your overall run time would be approx. equal to the overall time of the task that ran longest. Another common scenario that involves many parallel tasks is similar to the one I mentioned above: fetching user data from an api for each user in your database.
For eg, we can call an api to get the customer’s shopping points that he/she has earned. Since each customer’s shopping points don’t depend on another customer’s, we can fetch these data in parallel. We can iterate through the list of customers and trigger the api call for each. When all of the customers’ shopping points have been fetched, we can return control to the UI and the UI can show them all.
private static async Task<int> GetCustomerShoppingPointsAsync()
{
int sum = 0;
foreach (var counter in Enumerable.Range(0, 25))
{
sum += counter;
await Task.Delay(100);
}
return sum; // made up customer shopping points
}
private static int GetCustomerShoppingPoints()
{
int sum = 0;
foreach (var counter in Enumerable.Range(0, 25))
{
sum += counter;
Task.Delay(100).Wait();
}
return sum; // made up customer shopping points
}
private static async Task RunManyTasksAsync(int numberOfTasks)
{
var tasks = new List<Task>();
var stopwatch = new Stopwatch();
stopwatch.Start();
for (int i = 0; i < numberOfTasks; i++)
{
tasks.Add(GetCustomerShoppingPointsAsync());
}
await Task.WhenAll(tasks);
Console.WriteLine("Time elapsed when all tasks have finished..." + stopwatch.Elapsed);
Console.ReadLine();
}
Running MANY tasks in parallel, but throttled
It’s awesome that we can fetch all of our customers’ shopping points in parallel, but what if we can’t do it for all of our customers at the same time for some reason. For eg, what if we don’t want to use up too many threads in the thread pool, or what if we don’t want to hit the customer api too hard, like 200 calls per second, etc. There are various legitimate reasons for why we want to throttle the number of tasks running in parallel. So, how do we do that? Well, glad you asked. The Task.WhenAny method comes to the rescue.
The Task.WhenAny method allows us to wait for any task to complete before we trigger another task to run, thus letting us throttle how many tasks are currently running.
private static async Task RunManyTasksAsyncWithThrottle(int numberOfTasks, int throttle)
{
var tasks = new List<Task>();
var stopwatch = new Stopwatch();
stopwatch.Start();
for (int i = 0; i < numberOfTasks; i++)
{
tasks.Add(GetCustomerShoppingPointsAsync());
// when tasks count reaches throttle limit, we wait till one task finishes before we trigger another task to run
if (tasks.Count == throttle)
{
var finishedTask = await Task.WhenAny(tasks);
// the elapsed time when the last task finished
Console.WriteLine("Time elapsed when the last task finished..." + stopwatch.Elapsed);
tasks.Remove(finishedTask);
}
}
await Task.WhenAll(tasks);
Console.WriteLine("Time elapsed when all tasks have finished..." + stopwatch.Elapsed);
Console.ReadLine();
}
Running MANY synchronous tasks in parallel, but throttled, using the thread pool
At this point, you might ask, “Sure, it’s easy to create a task from an async method, but what if my method is synchronous?” Very fair question. I’ve got you covered in that case too: we can use Task.Run to create a task for your synchronous method in the ThreadPool.
private static async Task RunManyTasksAsyncWithThrottleUsingThreadPool(int numberOfTasks, int throttle)
{
var tasks = new List<Task>();
var stopwatch = new Stopwatch();
stopwatch.Start();
for (int i = 0; i < numberOfTasks; i++)
{
// use Task.Run to spin up a thread in the ThreadPool to run your synchronous code and return a task
tasks.Add(Task.Run(GetCustomerShoppingPoints));
// when tasks count reaches thread limit, we wait till one task finishes before we trigger another task to run
if (tasks.Count == throttle)
{
var finishedTask = await Task.WhenAny(tasks);
// the elapsed time when the last task finished
Console.WriteLine("Time elapsed when the last task in the thread pool finished..." + stopwatch.Elapsed);
tasks.Remove(finishedTask);
}
}
await Task.WhenAll(tasks);
Console.WriteLine("Time elapsed when all tasks in the thread pool have finished..." + stopwatch.Elapsed);
Console.ReadLine();
}
Full .Net core console app code
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading.Tasks;
namespace MultiThreadingUsingAsyncAwait
{
class Program
{
private static async Task Main(string[] args)
{
while (true)
{
Console.WriteLine("Enter the number of tasks (100 to quit program): ");
var numberOfTasks = int.Parse(Console.ReadLine());
if (numberOfTasks == 100)
{
break;
}
Console.WriteLine("Enter the task throttle, aka, max number of tasks allowed to run in parallel (100 to quit program): ");
var throttle = int.Parse(Console.ReadLine());
if (throttle == 100)
{
break;
}
await RunManyTasksAsync(numberOfTasks);
await RunManyTasksAsyncWithThrottle(numberOfTasks, throttle);
await RunManyTasksAsyncWithThrottleUsingThreadPool(numberOfTasks, throttle);
}
}
private static async Task<int> GetCustomerShoppingPointsAsync()
{
int sum = 0;
foreach (var counter in Enumerable.Range(0, 25))
{
sum += counter;
await Task.Delay(100);
}
return sum; // made up customer shopping points
}
private static int GetCustomerShoppingPoints()
{
int sum = 0;
foreach (var counter in Enumerable.Range(0, 25))
{
sum += counter;
Task.Delay(100).Wait();
}
return sum; // made up customer shopping points
}
private static async Task RunManyTasksAsync(int numberOfTasks)
{
var tasks = new List<Task>();
var stopwatch = new Stopwatch();
stopwatch.Start();
for (int i = 0; i < numberOfTasks; i++)
{
tasks.Add(GetCustomerShoppingPointsAsync());
}
await Task.WhenAll(tasks);
Console.WriteLine("Time elapsed when all tasks have finished..." + stopwatch.Elapsed);
Console.ReadLine();
}
private static async Task RunManyTasksAsyncWithThrottle(int numberOfTasks, int throttle)
{
var tasks = new List<Task>();
var stopwatch = new Stopwatch();
stopwatch.Start();
for (int i = 0; i < numberOfTasks; i++)
{
tasks.Add(GetCustomerShoppingPointsAsync());
// when tasks count reaches throttle limit, we wait till one task finishes before we trigger another task to run
if (tasks.Count == throttle)
{
var finishedTask = await Task.WhenAny(tasks);
// the elapsed time when the last task finished
Console.WriteLine("Time elapsed when the last task finished..." + stopwatch.Elapsed);
tasks.Remove(finishedTask);
}
}
await Task.WhenAll(tasks);
Console.WriteLine("Time elapsed when all tasks have finished..." + stopwatch.Elapsed);
Console.ReadLine();
}
private static async Task RunManyTasksAsyncWithThrottleUsingThreadPool(int numberOfTasks, int throttle)
{
var tasks = new List<Task>();
var stopwatch = new Stopwatch();
stopwatch.Start();
for (int i = 0; i < numberOfTasks; i++)
{
// use Task.Run to spin up a thread in the ThreadPool to run your synchronous code and return a task
tasks.Add(Task.Run(GetCustomerShoppingPoints));
// when tasks count reaches thread limit, we wait till one task finishes before we trigger another task to run
if (tasks.Count == throttle)
{
var finishedTask = await Task.WhenAny(tasks);
// the elapsed time when the last task finished
Console.WriteLine("Time elapsed when the last task in the thread pool finished..." + stopwatch.Elapsed);
tasks.Remove(finishedTask);
}
}
await Task.WhenAll(tasks);
Console.WriteLine("Time elapsed when all tasks in the thread pool have finished..." + stopwatch.Elapsed);
Console.ReadLine();
}
}
}
Console app output
When the number of tasks == task throttle
when the number of tasks > task throttle
Wrap up
In this blog post, I am using Jeremy Lindsay’s great blog post as a starting point, and build upon it to demostrate how we can use C# async/await/Task.WhenAny/Task.WhenAll to execute tasks in parallel to dramatically improve performance. If appropriate, we can also throttle the nubmer of tasks running in parallel. This methodology works for both asynchronous and synchronous code.
I hope you find this blog post helpful to you as a .Net developer, and a great addition to your tool belt when it comes to performance improvement.