Enhanced Compiler Sandboxing for Improved Security

[coretravis]

Hi everyone,

I'd like to share a proposal to significantly enhance the sandboxing of our compiler. As our project continues to evolve, ensuring the secure execution of untrusted code becomes increasingly critical. The plan below outlines a comprehensive, multi‑layered defense‑in‑depth strategy from pre‑execution static analysis all the way through to robust monitoring and cleanup. This approach is designed not only to catch potentially dangerous code early but also to provide secure, isolated execution environments that protect the system and provide responsive feedback to users.

---

1. Pre‑Execution Static Analysis

• Static Code Analysis:
  • Pass the submitted code through a dedicated static analysis service (e.g., a StaticAnalysisService).
  • Analyze the syntax tree for dangerous constructs such as:
    • Unsafe blocks or methods
    • P/Invoke (DllImport usage)
    • Reflection abuse or dynamic type instantiation
    • Unauthorized file I/O, network calls, or use of restricted APIs (some of which we have already have a foundation: UnsupportedApiRewriter)
  • Outcome:
    • If dangerous patterns are detected, return an error response with warnings.
    • If no issues are found (or if the warnings are acceptable), proceed.

---

2. Preparation for Compilation

• Sandbox Environment Setup:

  • Create a temporary working directory within an allowed base directory (e.g., a dedicated “sandbox” folder defined by an environment variable or some configuration such as ALLOWED_SANDBOX_BASE_DIR).
  • Ensure that all file I/O operations (reading/writing temporary files) will be confined to this directory.

• Configuration of Compiler Settings:

  • Determine the correct compiler version and any necessary compilation options.
  • Optionally integrate further static checks or API whitelisting during the compilation phase.

---

3. Compilation

• Compilation via Roslyn (Implemented):

  • Use an ICompilationService that leverages Roslyn to compile the submitted code.
  • Write the code to a temporary file within the sandbox.
  • Attempt to compile the code into an executable (or DLL) and capture any compilation errors.

• Compilation Outcome:

  • If there are compilation errors, return an error response with details.
  • If compilation succeeds, generate a clean executable ready for execution.

---

4. Process Execution and Runtime Sandboxing

• Spawning a Dedicated Process:

  • Use the StreamingCodeExecutionService to launch a new process that runs the compiled executable.
  • The process should run under a restricted, low‑privilege account if possible, and be attached to additional OS-level isolation mechanisms (e.g., Windows Job Object) to enforce limits.

• Enforce Resource Limits:

  • Set execution timeouts using cancellation tokens so that runaway processes are terminated.
  • Lower the process’s priority to mitigate CPU hogging.
  • Optionally, apply further limits (memory, CPU) at the container or OS level.

• Restrict Outbound Network Access:

  • Ensure that the sandbox prevents arbitrary outbound calls from the executed code.
  • Whitelist only the necessary endpoints

---

5. Process Monitoring, Cleanup, and Response

• Monitor Execution:

  • Continuously monitor the running process for timeouts or resource limit breaches.
  • If the process exceeds allowed limits, terminate it and send an appropriate error message to the client.

• Cleanup:

  • Once execution completes (or is terminated), clean up all temporary resources (files, directories, process handles, Job Object references).
  • Log all operatins and any security-related events for auditing.

• Return Final Response:

  • Stream the final output, exit code, and any error messages back to the client as part of the response.
  • Optionally, update the client via a final SignalR message indicating completion.

---

Summary

This pipeline implements a multi‑layered, defense‑in‑depth strategy. To further improve sand-boxing, we should containerize the application which further isolates execution. Ideally, the Code Execution should be offloaded to an Exection Service outside our main server and this will be part of our roadmap

I’d love to hear your thoughts on this approach. Do you have any suggestions or concerns about the measures outlined? Are there additional safeguards we should consider? Your feedback is invaluable as we refine and implement these improvements, and I look forward to collaborating with you to ensure our system remains secure and resilient.

[coretravis]

The discussed features have been implemented and merged #11