We are using AWS ECS Fargate to host our web, api, and hangfire processes. They run under docker. Hangfire handles both ad-hoc and scheduled jobs. For the most part it works fine day to day. But one scheduled job at night destroys the memory usage (> 13GB on a 16GB instance) and bombs the Hangfire container.
Crazy thing is that when I run this setup locally, the memory usage (running the same schedule job on the same files) never gets above 2GB.
It feels like an environment difference to me (how linux and windows differ in memory usage or garbage collection in the runtime.)