Add info to test switch mode #468
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* enable cpu_features code for M1 (cherry picked from commit 3a73990) * cpu feature tag update (cherry picked from commit 3533000) * fix include (cherry picked from commit 39e7a36) * whitespace alignment (cherry picked from commit 65eff3b) * includes cleanup (cherry picked from commit 8d87b46) --------- Co-authored-by: GuyAv46 <guy.avimor@gmail.com>
* fix flow (cherry picked from commit 494408a) * add config file (cherry picked from commit b5c618f) * fixes (cherry picked from commit 3eaca32) * another attempt for v4 (cherry picked from commit 0fbd812) * TEMP run coverage fast (cherry picked from commit f7b6c00) * fix path (cherry picked from commit fe23a8f) * Revert "TEMP run coverage fast" This reverts commit f7b6c00. (cherry picked from commit bd74e3d) --------- Co-authored-by: GuyAv46 <guy.avimor@gmail.com> Co-authored-by: GuyAv46 <47632673+GuyAv46@users.noreply.github.com>
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 40 to 41. - [Release notes](https://github.com/tj-actions/changed-files/releases) - [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md) - [Commits](tj-actions/changed-files@v40...v41) --- updated-dependencies: - dependency-name: tj-actions/changed-files dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> (cherry picked from commit c115204) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* upgrade codcov instance to c7i (cherry picked from commit b5cc93b) * [MOD-6763] bfloat16 - distance calculation functions (#445) (cherry picked from commit 850d6a8) * update arch_opt macros in bf16 BM (cherry picked from commit 253dcd3) * fix macro check (cherry picked from commit 26b047d) * Remove arch_opt enum and use cpu features flags instead (#459) (cherry picked from commit f98793a) * FP16 distance functions (#453) (cherry picked from commit 9420e77) * [MOD-6764] Introduce bfloat16 datatype (#454) * Introduce bfloat16 datatype add Datatype to `VecSimIndexAbstract` template args. add template to SetDistFunc and have specialized implemntation for each <dataType, distType> combintation now the dist function pointer argument is not neseccesary, we can simple return the function. move using dist_func_t to spaces_aux.h (should be in a file included by both L2_spaces.h/IP.spaces.h and spaces.h) * move getdistance imp to spaces.cpp * move normalize function from vec_utils to spaces: spaces.h: get normalize function. spaces.cpp: specialized templates according to the datatype for the normalization functions spaces/normalize/normalize_naive.h: templated naive implementations of normalization functions VecSimIndexAbstract must have both DataType and DistType add VecSimType_BFLOAT16 update vec_utile accordingly bfloat16.h: add bfloat16_to_float32 for little and big TieredFactory::NewIndex calls TieredHNSWFactory::TieredFactory::NewIndex instead of implmentation duplication add bfloat to index factory for NewIndex and EstimateInitialSize * add no_opt functions tests for bfloat16 also add normalization function test for bf16 * fixed nohrmalized value * add bf16 benchmarks: add serialized hnsw bf16 files hnswfactory: add ctor from file for bf16 add fitMemory and run it at the begining of each benchmark to tighten memory. * update bm_files script * add flow tests to bf16 bruteforce: L2 batch iterator range query multivalue hnsw: serialization L2 batch iterator range multi tiered: create search and insert * add bfloat16 module to toml * upgrade scipy nupy * also use python 3.10 * upgrade PR python to 3.10 * update serializer * bm: fix casts unit tests: add test_bf16 * more unit tests * tiered tests * add index_type assertion ti hnsw serialization test add desgnated classes for bf and flat in bf16 unit test * unit test final * repalce cast to dynamic casts * add get neighors test to tiered add normalize test for bfloat16 in testcommon * use bit_cast * upgrade codecov machine * Revert "upgrade codecov machine" This reverts commit 7619b05. * Revert "use bit_cast" This reverts commit 59b8fab. * fix alignment intialization by changing the order * make bfloat16_to_float32 templated accroding to system's endianess * eliminate default in GetDistFunc add test for invalid metric * add test fot fit memory * split fit memory tests (cherry picked from commit e018d91) * run merge queue flow (cherry picked from commit 4670c59) * add override to fit memory (cherry picked from commit f994c79) * upload valgrind artifacts on failure (cherry picked from commit 7ece68d) * _mm256_loadu_epi16 is supported as of gcc1, use _mm256_lddqu_si256 instead (cherry picked from commit c521d82) * fix fit memory test (#461) * fix fit memory test * remove xenial checks * remove irrelevant comment * free index in seralization test (cherry picked from commit a0b70c8) * Remove avx512_BW_VL (#460) * remove bw_vl arch (not needed, avx512 is enough) * remove xenial from merge queue flow * fix fit memory test (#461) * fix fit memory test * remove xenial checks * remove irrelevant comment * free index in seralization test * install gcc10 bionic * install newer vg ubuntu18 * exit valgrind directory (cherry picked from commit d2e1d4f) * [MOD-6775] introduce fp16 type! (#462) * remove bw_vl arch (not needed, avx512 is enough) * remove xenial from merge queue flow * introduce fp16 type! change bfloat and fp16 to struct typedef to avoid ambiguty between them * added flow tests * fp16 benchmarks * fix hnsw flow test * add float616 to serializing script add fp16 files to bm files decrease testSizeEstimation dim size to make it run fasetr in debug * set log call back to null in benchmarks * remove struct typedef add default ctor * add comma seriaizer file * disable temp flow * disabe for real * remove constexpr from bfloat default ctor declaring a default ctor as constexpr is allowed as of c++20, and this feature in particular is supported as of gcc10 I don't see any benefit using this feature... * test focal * disable temp flow (cherry picked from commit 704282b) --------- Co-authored-by: meiravgri <meirav.grimberg@redis.com> Co-authored-by: meiravgri <109056284+meiravgri@users.noreply.github.com> Co-authored-by: alonre24 <alonreshef24@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe the changes in the pull request
Print HNSW graph info upon failure in
switchWriteModeflaky test. We expect to validate that the reason for the flaky test is the fact that the graph is not connected.Mark if applicable