Feature: Disable updating environment for certain files/modes

I have a workflow where i switch to me org file for my projects quite often, and direnv updates the environment accordingly, but it's kind of slow, so it's pretty annoying. I would like if there was some kind of setting where we can put a list of modes or directories, so that direnv doesn't update the path.