Skip to content

Commit 3f60d5d

Browse files
committed
Test properties not affected by floating point error
This build found a particular failing example of `TestTournament.test_seeding_equality` https://github.com/Axelrod-Python/Axelrod/pull/1368/checks?check_run_id=975415322 Upon closer investigation it looks like that was not due to seeding but due to the floating point error of some calculations made by the result set. I investigated using: ``` import axelrod as axl import numpy as np seed = 2 repetitions = 10 rng = axl.RandomGenerator(seed=seed) players = [axl.Random(rng.random()) for _ in range(8)] tournament1 = axl.Tournament( players=players, turns=10, repetitions=repetitions, seed=seed ) tournament2 = axl.Tournament( players=players, turns=10, repetitions=repetitions, seed=seed ) for _ in range(4): results1 = tournament1.play(processes=2, progress_bar=False) results2 = tournament2.play(processes=2, progress_bar=False) assert results1.wins == results2.wins assert results1.match_lengths == results2.match_lengths assert results1.scores == results2.scores assert np.allclose(results1.normalised_scores, results2.normalised_scores) assert np.allclose(results1.ranking, results2.ranking) assert results1.ranked_names == results2.ranked_names assert results1.payoffs == results2.payoffs assert results1.payoff_matrix == results2.payoff_matrix assert np.allclose(results1.payoff_stddevs, results2.payoff_stddevs) assert results1.score_diffs == results2.score_diffs assert results1.payoff_diffs_means == results2.payoff_diffs_means assert results1.cooperation == results2.cooperation assert results1.normalised_cooperation == results2.normalised_cooperation assert results1.vengeful_cooperation == results2.vengeful_cooperation assert results1.cooperating_rating == results2.cooperating_rating assert results1.good_partner_matrix == results2.good_partner_matrix assert results1.good_partner_rating == results2.good_partner_rating assert np.allclose(results1.eigenmoses_rating, results2.eigenmoses_rating) assert np.allclose(results1.eigenjesus_rating, results2.eigenjesus_rating) ``` Note I'm using `np.isclose` for some properties. In this commit: - I add the specific seed for which the error was found as a hypothesis example (`seed=2`). - Replace the `results1 == results2` check with just a check of some properties (from which the others are essentially calculated). - Added `progress_bar=False`
1 parent 917eead commit 3f60d5d

File tree

1 file changed

+13
-4
lines changed

1 file changed

+13
-4
lines changed

axelrod/tests/unit/test_tournament.py

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -732,13 +732,19 @@ def test_write_to_csv_without_results(self):
732732
self.assertTrue(df.equals(expected_df))
733733

734734
@given(seed=integers(min_value=1, max_value=4294967295))
735+
@example(seed=2)
735736
@settings(max_examples=5, deadline=None)
736737
def test_seeding_equality(self, seed):
737738
"""Tests that a tournament with a given seed will return the
738739
same results each time. This specifically checks when running using
739740
multiple cores so as to confirm that
740741
https://github.com/Axelrod-Python/Axelrod/issues/1277
741-
is fixed."""
742+
is fixed.
743+
744+
Note that the final asserts test only specific properties of the results
745+
sets and not the entire result sets as some floating point errors can
746+
emerge.
747+
"""
742748
rng = axl.RandomGenerator(seed=seed)
743749
players = [axl.Random(rng.random()) for _ in range(8)]
744750
tournament1 = axl.Tournament(
@@ -758,9 +764,12 @@ def test_seeding_equality(self, seed):
758764
seed=seed
759765
)
760766
for _ in range(4):
761-
results1 = tournament1.play(processes=2)
762-
results2 = tournament2.play(processes=2)
763-
self.assertEqual(results1.ranked_names, results2.ranked_names)
767+
results1 = tournament1.play(processes=2, progress_bar=False)
768+
results2 = tournament2.play(processes=2, progress_bar=False)
769+
self.assertEqual(results1.wins, results2.wins)
770+
self.assertEqual(results1.match_lengths, results2.match_lengths)
771+
self.assertEqual(results1.scores, results2.scores)
772+
self.assertEqual(results1.cooperation, results2.cooperation)
764773

765774
def test_seeding_inequality(self):
766775
players = [axl.Random(0.4), axl.Random(0.6)]

0 commit comments

Comments
 (0)