Simulator Memory Issue when Acting at Ten-Minute Frequency

[amol-HRT]

Hi! The simulator runs into memory issues. If you try to act at a higher frequency than a day, the simulator quickly runs out of memory. An easy way to reproduce this error is to change the getTomorrow function in the bot repo's date utils to

export function getTomorrow(date: Date): Date {
  return new Date(date.getTime() + 60 * 10 * 1000);
}

This modification allows you to act every 10 minutes. I'm not sure why this error occurs, as it is possible to push more database events than custom actions to the simulator. I've tried debugging this by taking snapshots more frequently, but that didn't help.

Here is the error message:

<--- Last few GCs --->

[97445:0x138008000]   237794 ms: Scavenge 4034.0 (4126.1) -> 4020.1 (4126.1) MB, 15.3 / 0.0 ms  (average mu = 0.164, current mu = 0.023) allocation failure 
[97445:0x138008000]   237912 ms: Scavenge 4034.3 (4126.4) -> 4021.5 (4128.4) MB, 36.7 / 0.0 ms  (average mu = 0.164, current mu = 0.023) allocation failure 
[97445:0x138008000]   240544 ms: Mark-sweep 4036.7 (4129.3) -> 4020.0 (4128.9) MB, 2601.1 / 0.0 ms  (average mu = 0.140, current mu = 0.112) allocation failure scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x1010daaf8 node::Abort() [/opt/homebrew/Cellar/node/17.3.0/bin/node]
 2: 0x1010dbaf8 node::OnFatalError(char const*, char const*) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
 3: 0x1012332a4 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
 4: 0x101233238 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
 5: 0x101383a00 v8::internal::Heap::EnsureFillerObjectAtTop() [/opt/homebrew/Cellar/node/17.3.0/bin/node]
 6: 0x101385bd0 v8::internal::Heap::ComputeMutatorUtilization(char const*, double, double) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
 7: 0x101384234 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
 8: 0x1013824a4 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
 9: 0x10138f3ec v8::internal::Heap::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
10: 0x10138f46c v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
11: 0x101366ed8 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
12: 0x10161c2f8 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
13: 0x100f503ec Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/opt/homebrew/Cellar/node/17.3.0/bin/node]
14: 0x105e887b0 
15: 0x105e8a158 
16: 0x1059d0378 
17: 0x105dda694 
18: 0x100f2c324 Builtins_MapPrototypeForEach [/opt/homebrew/Cellar/node/17.3.0/bin/node]
19: 0x105ea931c 
20: 0x1059d1a10 
21: 0x105ddd390 
22: 0x100f2c324 Builtins_MapPrototypeForEach [/opt/homebrew/Cellar/node/17.3.0/bin/node]
23: 0x105c306a8 
24: 0x1059d199c 
25: 0x105dda93c 
26: 0x100f2c324 Builtins_MapPrototypeForEach [/opt/homebrew/Cellar/node/17.3.0/bin/node]
27: 0x105ea931c 
28: 0x1059d1a10 
29: 0x105e9c72c 
30: 0x105e9aea4 
31: 0x105ef86cc 
32: 0x105ef3e9c 
33: 0x100f13674 Builtins_AsyncFunctionAwaitResolveClosure [/opt/homebrew/Cellar/node/17.3.0/bin/node]
34: 0x100f984f8 Builtins_PromiseFulfillReactionJob [/opt/homebrew/Cellar/node/17.3.0/bin/node]
35: 0x100f05a54 Builtins_RunMicrotasks [/opt/homebrew/Cellar/node/17.3.0/bin/node]
36: 0x100ee2c84 Builtins_JSRunMicrotasksEntry [/opt/homebrew/Cellar/node/17.3.0/bin/node]
37: 0x10131fc38 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
38: 0x10131ffa4 v8::internal::(anonymous namespace)::InvokeWithTryCatch(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
39: 0x1013412dc v8::internal::MicrotaskQueue::RunMicrotasks(v8::internal::Isolate*) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
40: 0x101341108 v8::internal::MicrotaskQueue::PerformCheckpointInternal(v8::Isolate*) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
41: 0x101014984 node::InternalCallbackScope::Close() [/opt/homebrew/Cellar/node/17.3.0/bin/node]
42: 0x10101485c node::CallbackScope::~CallbackScope() [/opt/homebrew/Cellar/node/17.3.0/bin/node]
43: 0x1010aae08 (anonymous namespace)::uvimpl::Work::AfterThreadPoolWork(int) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
44: 0x1031fef20 uv__work_done [/opt/homebrew/opt/libuv/lib/libuv.1.dylib]
45: 0x103202260 uv__async_io [/opt/homebrew/opt/libuv/lib/libuv.1.dylib]
46: 0x103211ac4 uv__io_poll [/opt/homebrew/opt/libuv/lib/libuv.1.dylib]
47: 0x103202680 uv_run [/opt/homebrew/opt/libuv/lib/libuv.1.dylib]
48: 0x101015b1c node::SpinEventLoop(node::Environment*) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
49: 0x101118444 node::NodeMainInstance::Run(int*, node::Environment*) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
50: 0x101118098 node::NodeMainInstance::Run(node::EnvSerializeInfo const*) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
51: 0x1010a879c node::Start(int, char**) [/opt/homebrew/Cellar/node/17.3.0/bin/node]
52: 0x182989430 start [/usr/lib/system/libdyld.dylib]
error Command failed with signal "SIGABRT".