Decrease batch job success retention time. Redis Storage Blew Up

question
1

We just setup batch jobs to to run nightly. There are 150K jobs in each batch.

5 days later, our redis storage blew up. We just scaled up our storage but its a temporary fix, in a few days we’d have to scale up again.

How do we configure how long batch data is retained, it looks like the default is 7 days.

2

I had the same problem and i did the following attribute:

public class ProlongExpirationTimeAttribute : JobFilterAttribute, IApplyStateFilter
{
    public void OnStateApplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
    {
        if (context.NewState is SucceededState)
            context.JobExpirationTimeout = TimeSpan.FromDays(1);
        else
            context.JobExpirationTimeout = TimeSpan.FromDays(7);
    }

    public void OnStateUnapplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
    {
        if (context.NewState is SucceededState)
            context.JobExpirationTimeout = TimeSpan.FromDays(1);
        else
            context.JobExpirationTimeout = TimeSpan.FromDays(7);
    }
}
1 Like