Skip to content

Conversation

dworthen
Copy link
Contributor

@dworthen dworthen commented Sep 10, 2024

  • Fixes load_config eagering resolving directories. Directories are only resolved when the output directories are local.
  • Add support for --output and --reporting flags for index CLI. To achieve previous output structure index --output run1/artifacts --reports run1/reports.
  • Use static output directories when initializing a new project.
  • Maintains backward compatibility for those using timestamp outputs locally.
  • Addresses Support loading from storage blob in query library #799

Checklist

  • I have tested these changes locally.
  • I have reviewed the code changes.
  • I have updated the documentation (if necessary).
  • I have added appropriate unit tests (if applicable).

- Fixes load_config eagering resolving directories.
    Directories are only resolved when the output
    directories are local.
- Add support for `--output` and `--reporting` flags
    for index CLI. To achieve previous output structure
    `index --output run1/artifacts --reports run1/reports`.
- Use static output directories when initializing
    a new project.
- Maintains backward compatibility for those using
    timestamp outputs locally.
@dworthen dworthen requested review from a team as code owners September 10, 2024 13:55
@nociza
Copy link

nociza commented Sep 10, 2024

Does this support using timestamp directories remotely?

@dworthen
Copy link
Contributor Author

Does this support using timestamp directories remotely?

HI @nociza, we are migrating away from timestamp paths both remotely and locally. If you look at the v1-breaking-changes.md file in this PR it describes the change and migration. This PR still maintains support for timestamp paths locally for backwards compatibility but timestamp paths are not supported for remote storage. In issue #799 if you update your config to use static directories for storage then it should work though you may need to re-index or at least move your current index outputs to the new static location.

AlonsoGuevara
AlonsoGuevara previously approved these changes Sep 11, 2024
@AlonsoGuevara AlonsoGuevara self-requested a review September 18, 2024 23:29
@AlonsoGuevara AlonsoGuevara merged commit 3b09df6 into main Sep 18, 2024
16 checks passed
@AlonsoGuevara AlonsoGuevara deleted the static-output branch September 18, 2024 23:36
opensourcemukul pushed a commit to opensourcemukul/graphrag that referenced this pull request Sep 13, 2025
* Migrate towards using static output directories

- Fixes load_config eagering resolving directories.
    Directories are only resolved when the output
    directories are local.
- Add support for `--output` and `--reporting` flags
    for index CLI. To achieve previous output structure
    `index --output run1/artifacts --reports run1/reports`.
- Use static output directories when initializing
    a new project.
- Maintains backward compatibility for those using
    timestamp outputs locally.

* fix smoke tests

* update query cli to work with static directories

* remove eager path resolution from load_config. Support CLI overrides that can be resolved.

* add docs and output logs/artifacts to same directory

* use match statement

* switch back to if statement

---------

Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants