Use Stepping To Perform Atomic Multi-Step Operations
Stepping is a distributed BASE jobs implementation. You can use it as a workflow engine, event outbox/inbox, email/SMS sender, remote invoker, and more.
Step in Stepping?
Job is a distributed transaction unit, and
Step is a specific task inside a job.
A job contains one or many steps, and the transaction manager will execute them in order. If step 1 fails, it will be retried until success, and then step 2 starts to execute.
Scenarios for Using Stepping
Want To Execute Steps and Ensure Atomicity
When you start a job, Stepping will eventually complete the steps you require. If the app crashes during the executions, the transaction manager will continue to execute the rest steps after it recovers.
Stepping will complete your steps one by one. If a step fails, it will be tried later until success, which makes the job atomic. Please ensure all your steps can eventually succeed after retrying unless it is a Saga step.
Stepping may already complete the current step when your app crashes during the execution. When your app recovers, Stepping will execute it redundantly. Therefore, all your steps should be idempotent.
Want To Ensure Executing Steps After a DB Transaction Commits
When you start a job with a DB transaction, Stepping will eventually complete the steps you require after the DB transaction commits.
You don't need to worry about the non-atomicity caused by the app crashes after the transaction commits but before the steps' execution. We have handled this case by using the DTM's 2-phase messaging pattern.
Stepping also supports the "multi-tenant with multi-DB" scenario, meaning it works no matter how many different databases there are in your app.
The transaction manager will eventually complete the added steps:
var job = await distributedJobFactory.CreateJobAsync(); job.AddStep(new RequestBank1TransferOutStep(args)); // step with args job.AddStep<RequestBank2TransferInStep>(); // step without args await job.StartAsync();
The Steps document shows how to define a step.
If you want to execute the steps after a DB transaction commits and ensure they will eventually be done:
var db = serviceProvider.GetRequiredService<MyDbContext>(); // example for EF Core await db.Database.BeginTransactionAsync(); var order = new Order(args); db.Orders.Add(order); await db.SaveChangesAsync(); var job = await distributedJobFactory.CreateJobAsync(new EfCoreSteppingDbContext(db)); job.AddStep(new SendOrderCreatedEmailStep(order)); job.AddStep(new SendOrderCreatedSmsStep(order)); await job.StartAsync(); // it will commit the DB transaction
ADO.NET(coming soon), and
For details, please see the Usage document.
See the Installation document.
Supported Transaction Managers
Stepping requires transaction managers. You can choose an implementation you like.
DTM is a mature transaction manager you can use as the TM provider for Stepping. DTM allows you to use many other distributed transaction patterns like Saga, TCC, and XA.
See the DTM document.
Stepping provides a simple built-in TM implementation. The local-TM runs with your app as a local transaction manager. Which app starts a job should be the TM of this job.
See the Local-TM document.