In this blog post, we will delve into the comprehensive process of constructing a "Fire and Forget" style background task in .NET web applications. The term "Fire and Forget" refers to an action initiated by a user, such as executing a report, that may consume several minutes or even longer to complete and needs to run in the background. By employing this approach, the user interface can continuously request updates from the application and obtain the final result upon completion.
Raytha implemented its own background task architecture due to the unavailability of a suitable .NET background task library that fulfilled all the requirements. While there are popular and well-established frameworks like Hangfire.io and Quartz.NET, they could not satisfy our specific prerequisites. These prerequisites include being MIT open source and supporting the utilization of SQL Server for managing the queue.
Consequently, we embarked on the journey of constructing our own background task system. Raytha, being MIT open source, grants you the freedom to adopt and utilize any of the code we have developed in your own projects.
Below is an example of running an export to CSV in the Raytha platform. Example of running fire and forget background task in a web app The Raytha platform and its background task infrastructure are all part of the open source CMS project available on github here: https://github.com/raythahq/raytha.
Overview
The background task infrastructure we implemented with Raytha uses SQL Server as the storage mechanism for which tasks are enqueued, in processing, and completed. However, you may want to use a different platform such as Redis, RabbitMQ, or something else.
However, the Microsoft guide is rudimentary and does not go far enough to provide what is needed for a Fire and Forget style implementation, nor any of the intricacies and “gotchas” that come with building this, such as dealing with concurrency.
The example we will follow in this post is the “Export to CSV” feature (demonstrated in the animation above) where we want to build functionality to support exporting all the data associated with an admin view to a csv file. Depending on how much data there is, it could take a while to pull all the data.
Enqueue a background task on to the queue
Raytha follows a CQRS pattern, but this will work in any scenario. You want to inject the IBackgroundTaskQueue class via dependency injection as shown below.
We pass in `request` which is of class type Command received from the parameters of the Handle() method.
The EnqueueAsync<T> is using generics with the type BackgroundTask.
We receive back a background job ID that we can use to poll for status and result.
The Command class is an artifact of CQRS if you are familiar with that pattern. It is nothing more than a data class that you can pass in as arguments to your background task.
Your data object can be any json serializable object because the IBackgroundTaskQueue interface is defined as such:
The EnqueueAsync<T> function uses generics, and in our situation, we declare a type called BackgroundTask. The BackgroundTask class can be called anything of your choosing because it is where you will write your business logic for your background task. A requirement, however, is that it must implement the IExecuteBackgroundTask interface.
You see that you must implement the Execute() method which is defined in the IExecuteBackgroundTask interface like so:
In the above section, we already used the EnqueueAsync<T> function. But this interface also has a DequeueAsync function. We are going to implement the interface as shown below.
Enqueuing a new background task is as simple as just adding a new row into the database table called BackgroundTask (ignore the similar naming from the above section). This function also just uses out of the box EntityFramework. The BackgroundTask entity is defined as the following:
We will come back to this class to discuss the Dequeue method and IBackgroundTaskDb interface shortly, but this generally concludes putting a background task on to the queue for processing.
Setup the QueuedHostedService
In order for tasks to run in the background, there has to be a class that inherits the BackgroundService class that is provided by .NET. We will do this with our custom QueuedHostedService class. The idea is that we will have a QueuedHostedService running indefinitely in the background in a `while loop` that constantly checks if there are any tasks enqueued for processing. If so, it will grab that task off the queue and execute it.
Take notice that we create an IExecuteBackgroundTask object that is based on the type from the generic <T> that we specified when enqueuing the task in the first place. At the same time, we are passing in a JsonSerializer.Deserialize<JsonElement> the arguments that were serialized when we enqueued the task in the first place.
However, we promised that we would revisit the DequeueAsync method from IBackgroundTaskQueue briefly mentioned earlier.
We have to use the lock feature of .NET because we will have multiple threads running the `while loop` trying to dequeue items. This can result in a race condition. By adding a lock(), we are saying that only one thread can attempt to run this code snippet at a time.
Unfortunately, this is not enough to prevent a race condition. Why? This is assuming we only have one instance of the Raytha platform running at a time. If we are running multiple instances of Raytha connected to the same database in a scale out scenario, there could be a race condition at the database level when pulling the next available record from the database.
This is why we injected the IBackgroundTaskDb interface into this class and call the DequeueBackgroundTask() method.
The following code implements the IBackgroundTaskDb interface.
ENTREPRENEUR & SOFTWARE ENGINEER, AUSTIN, TX I enjoy tackling a wide array of business challenges ranging from front line product support and operations to sales and marketing efforts. My core expertise is in software development building enterprise level web applications.
