Skip to content
This repository was archived by the owner on Aug 22, 2023. It is now read-only.
This repository was archived by the owner on Aug 22, 2023. It is now read-only.

Better ortholog cache #8

@gcornut

Description

@gcornut

For now the cache is in memory and grows indefinitely
TODO

  • Add cache persistance (to disk)
  • Flatten cache for easier management
    {:org {:prot :ortho-group}} => {[:org :prot] :ortho-group}
  • Set up caching strategy
  • Link memory cache, disk cache and inparanoid webservices all together

Pseudo-code:

(if in-memory-cache?
   (get-from-memory-cache)
   (save-in-memory-cache
     (if in-disk-cache?
        (get-from-disk-cache)
        (save-in-disk-cache
          (get-from-inparanoid-ws)))))

Proposed caching strategy:

  • LRU with size threshold (with clojure core.cache)
  • Write to memory and disk on new entry (or periodically write to disk?)

Persistence:

  • Use nippy for writting/reading clojure data structures in binary
  • Split the cache in organism pairs (like in PMB)

TODO cache loader:

  • Download OrthologXML by organism pairs
  • Parse & load to cache
  • Add approx loaded percent
  • Add progress monitoring interface

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions