Explore pinned dependencies along with caching to speed up build #128
Description
Currently, builds for pypy take a very long time. we should explore https://github.com/actions/setup-python#caching-packages-dependencies and try to fasten it.
It can be seen that adding pandas to the dependencies is the primary reason for very slow builds. Need to explore more on this. For pypy, currently, it seems we are building pandas from source, which is the issue. Need to use prebuilt binaries/wheels.