Update some READMEs and tests.

Author: lschoe

demos/OneWayHashChainsExplained.ipynb

@@ -8,7 +8,7 @@
     "\n",
     "[Hash chains](https://en.wikipedia.org/wiki/Hash_chain) are well-known as a simple and efficient solution for asymmetric identification and authentication. They appear in many guises, often in contexts with severe performance constraints. For example, the [TESLA](https://www.rfc-editor.org/info/rfc4082) scheme and friends use hash chains at their core and provide authentication in wireless sensor networks and satellite communication.\n",
     "\n",
-    "In this notebook we construct an MPyC program for handling hash chains in a multiparty setting. There will be no single point of failure, as no single party will ever know the secret information from which the hash chains are built. This way of protecting secret keys is similar to what companies like [Sepior](https://sepior.com/) and [Unbound](https://www.unboundsecurity.com/) do using threshold cryptography and MPC-based hardware security modules (HSMs).\n",
+    "In this notebook we construct an MPyC program for handling hash chains in a multiparty setting. There will be no single point of failure, as no single party will ever know the secret information from which the hash chains are built. This way of protecting secret keys is similar to what companies like [Sepior](https://sepior.com/) and [Unbound](https://github.com/unboundsecurity) do using threshold cryptography and MPC-based hardware security modules (HSMs).\n",
     "\n",
     "## Hash Chains\n",
     "\n",
@@ -184,21 +184,21 @@
      "output_type": "stream",
      "text": [
       "Order-0 hash chain of length 1 (1 rounds):\n",
-      " 1 x0 = dce02350924adb59103b4000066ba9b2\n",
+      " 1 x0 = 604fc4433bbbba2e5c4796510010ed48\n",
       "\n",
       "Order-1 hash chain of length 2 (3 rounds):\n",
       " 1 -\n",
-      " 2 x1 = 1f25ad5f0aec7b7da294b0eaf8a197ad\n",
-      " 3 x0 = dce02350924adb59103b4000066ba9b2\n",
+      " 2 x1 = fe242be2c20da3cd87370d1614277d0d\n",
+      " 3 x0 = 604fc4433bbbba2e5c4796510010ed48\n",
       "\n",
       "Order-2 hash chain of length 4 (7 rounds):\n",
       " 1 -\n",
       " 2 -\n",
       " 3 -\n",
-      " 4 x3 = 86310b3f27481602941f8092771b6b45\n",
-      " 5 x2 = 0f3d5981719b4410636824cb98433953\n",
-      " 6 x1 = 1f25ad5f0aec7b7da294b0eaf8a197ad\n",
-      " 7 x0 = dce02350924adb59103b4000066ba9b2\n",
+      " 4 x3 = fb38d81680a23a43d8063ee81de10c90\n",
+      " 5 x2 = 8b613cbcfaa17ff36b1823d599f535e2\n",
+      " 6 x1 = fe242be2c20da3cd87370d1614277d0d\n",
+      " 7 x0 = 604fc4433bbbba2e5c4796510010ed48\n",
       "\n",
       "Order-3 hash chain of length 8 (15 rounds):\n",
       " 1 -\n",
@@ -208,14 +208,14 @@
       " 5 -\n",
       " 6 -\n",
       " 7 -\n",
-      " 8 x7 = d06f7022bce379c300c5522b5859be10\n",
-      " 9 x6 = 6bd7f5507fe750c620e562b5740d1618\n",
-      "10 x5 = 21f54ab687cd1e4d8fbfa1b25401887a\n",
-      "11 x4 = 7cc31a0733a1c1f9dc4723d56976dc37\n",
-      "12 x3 = 86310b3f27481602941f8092771b6b45\n",
-      "13 x2 = 0f3d5981719b4410636824cb98433953\n",
-      "14 x1 = 1f25ad5f0aec7b7da294b0eaf8a197ad\n",
-      "15 x0 = dce02350924adb59103b4000066ba9b2\n",
+      " 8 x7 = 104a0c631c5a3fdac68ddc47dc140bce\n",
+      " 9 x6 = a5a807b0270387b6a599545bf23a455c\n",
+      "10 x5 = 74ba842fc3c53005b3ba4a3d56ae3228\n",
+      "11 x4 = 5f6d0953546e5fbdc4a4c1f3cd1dd380\n",
+      "12 x3 = fb38d81680a23a43d8063ee81de10c90\n",
+      "13 x2 = 8b613cbcfaa17ff36b1823d599f535e2\n",
+      "14 x1 = fe242be2c20da3cd87370d1614277d0d\n",
+      "15 x0 = 604fc4433bbbba2e5c4796510010ed48\n",
       "\n"
      ]
     }
@@ -238,7 +238,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The Python program [onewayhashchains.py](onewayhashchains.py) follows the same approach as presented in this notebook. In addition to the recursive pebbler shown above, however, the optimal binary pebbler is also implemented as an iterative algorithm."
+    "The Python program [onewayhashchains.py](onewayhashchains.py) follows the same approach as presented in this notebook. In addition to the recursive pebbler shown above, however, the optimal binary pebbler is also implemented as an iterative algorithm. Moreover, [np_onewayhashchains.py](np_onewayhashchains.py) demos the use of the faster Numpy-based [np_aes.py](np_aes.py) reimplementation of the AES demo as well as the use of the Numpy-based [sha3.py](sha3.py) threshold SHA-3 hash functions."
    ]
   }
  ],

demos/README.md

@@ -7,14 +7,15 @@ Use `-H`, `--HELP` option with any demo to see the MPyC help message.
 `python secretsanta.py -H`
 
 ```
-usage: secretsanta.py [-H] [-h] [-C ini] [-P addr] [-M m] [-I i] [-T t] [-B b]
-                      [--ssl] [-L l] [-K k] [--no-log] [--no-async]
-                      [--no-barrier] [--no-gmpy2] [--no-prss] [--mix32-64bit]
+usage: secretsanta.py [-H] [-h] [-C ini] [-P addr] [-M m] [-I i] [-T t]
+                      [-B b] [--ssl] [-L l] [-K k] [--log-level ll]
+                      [--no-log] [--no-async] [--no-barrier] [--no-gmpy2]
+                      [--no-numpy] [--no-prss] [--mix32-64bit]
                       [--output-windows] [--output-file] [-f F]
 
-optional arguments:
+MPyC help:
   -H, --HELP            show this help message for MPyC and exit
-  -h, --help            show secretsanta.py help message (if any)
+  -h, --help            show help message for this MPyC program (if any)
 
 MPyC configuration:
   -C ini, --config ini  use ini file, defining all m parties
@@ -28,16 +29,18 @@ MPyC configuration:
 MPyC parameters:
   -L l, --bit-length l  default bit length l for secure numbers
   -K k, --sec-param k   security parameter k, leakage probability 2**-k
+  --log-level ll        logging level ll=debug/info(default)/warning/error
   --no-log              disable logging messages
   --no-async            disable asynchronous evaluation
   --no-barrier          disable barriers
   --no-gmpy2            disable use of gmpy2 package
+  --no-numpy            disable use of numpy package
   --no-prss             disable use of PRSS (pseudorandom secret sharing)
   --mix32-64bit         enable mix of 32-bit and 64-bit platforms
 
 MPyC misc:
-  --output-windows      screen output for parties i>0 (only on Windows)
-  --output-file         append output for parties i>0 to party{m}_{i}.log
+  --output-windows      screen output for parties 0<i<m (one window each)
+  --output-file         append output of parties 0<i<m to party{m}_{i}.log
   -f F                  consume IPython's -f argument F
 ```
 
@@ -89,7 +92,7 @@ optional arguments:
 
 `python cnnmnist.py 1 0`
 
-`python cnnmnist.py 3`
+`python np_cnnmnist.py 3.5`
 
 `python cnnmnist.py -M1 1.5 0`
 
@@ -121,6 +124,6 @@ optional arguments:
 
 [KaplanMeierSurvivalExplained](KaplanMeierSurvivalExplained.ipynb) presents privacy-preserving Kaplan-Meier survival analysis, featuring aggregate Kaplan-Meier curves and secure logrank tests.
 
-[4demos](4demos.ipynb) gives quick access to demos secretsanta.py, id3gini.py, lpsolver.py, cnnmnist.py.
+[4demos](4demos.ipynb) gives quick access to demos secretsanta.py, id3gini.py, lpsolver.py, np_cnnmnist.py.
 
 [OneWayHashChainsExplained](OneWayHashChainsExplained.ipynb) shows a more advanced MPyC program.

demos/np_onewayhashchains.py

@@ -3,13 +3,13 @@
 This demo is an extended reimplementation of the onewayhashchain.py demo for
 generating and reversing one-way hash chains in a multiparty setting.
 
-In addition to the Matyas-Meyer-Oseas one-way function based on AES, the SHAKE128
-oneway function from the SHA3 faimlty is also provided as an option.
+Next to the Matyas-Meyer-Oseas one-way function based on AES, the SHAKE128
+one-way function from the SHA3 family is also provided as an option.
 
 Note that in the output stage the hashes pertaining to different pebbles are
 evaluated in parallel, without increasing the overall round complexity. Multiple
 hashes pertaining to the same pebble, however, are necessarily evaluated in series,
-increasing the overall round complxity accordingly. 
+increasing the overall round complexity accordingly.
 
 See demo onewayhashchain.py for more information.
 """

docs/mpyc.numpy.html

@@ -0,0 +1,35 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
+<html><head><title>Python: module mpyc.numpy</title>
+<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+</head><body bgcolor="#f0f0f8">
+
+<table width="100%" cellspacing=0 cellpadding=2 border=0 summary="heading">
+<tr bgcolor="#7799ee">
+<td valign=bottom>&nbsp;<br>
+<font color="#ffffff" face="helvetica, arial">&nbsp;<br><big><big><strong><a href="mpyc.html"><font color="#ffffff">mpyc</font></a>.numpy</strong></big></big></font></td
+><td align=right valign=bottom
+><font color="#ffffff" face="helvetica, arial"><a href=".">index</a><br><a href="https://github.com/lschoe/mpyc/blob/master/mpyc/numpy.py">github.com/lschoe/mpyc/blob/master/mpyc/numpy.py</a></font></td></tr></table>
+    <p><tt>This&nbsp;module&nbsp;acts&nbsp;as&nbsp;a&nbsp;stub&nbsp;to&nbsp;avoid&nbsp;a&nbsp;dependency&nbsp;for&nbsp;the&nbsp;numpy&nbsp;package.<br>
+&nbsp;<br>
+If&nbsp;the&nbsp;numpy&nbsp;package&nbsp;is&nbsp;not&nbsp;available,&nbsp;MPyC&nbsp;still&nbsp;runs&nbsp;but&nbsp;with&nbsp;less&nbsp;functionality.<br>
+Use&nbsp;of&nbsp;NumPy&nbsp;can&nbsp;be&nbsp;disabled&nbsp;to&nbsp;avoid&nbsp;loading&nbsp;the&nbsp;numpy&nbsp;package.<br>
+&nbsp;<br>
+If&nbsp;NumPy&nbsp;is&nbsp;enabled&nbsp;(available&nbsp;and&nbsp;not&nbsp;disabled),&nbsp;the&nbsp;MPyC&nbsp;runtime&nbsp;supports&nbsp;array<br>
+types---along&nbsp;with&nbsp;vectorized&nbsp;implementations---for&nbsp;secure&nbsp;numbers&nbsp;and&nbsp;the&nbsp;underlying<br>
+finite&nbsp;field&nbsp;types.&nbsp;The&nbsp;array&nbsp;types&nbsp;are&nbsp;accessible&nbsp;through&nbsp;the&nbsp;'array'&nbsp;attribute,<br>
+e.g.,&nbsp;for&nbsp;secint48=mpc.SecInt(48),&nbsp;the&nbsp;array&nbsp;type&nbsp;is&nbsp;secint48.array&nbsp;and&nbsp;the&nbsp;array&nbsp;type<br>
+for&nbsp;the&nbsp;underlying&nbsp;prime&nbsp;field&nbsp;is&nbsp;secint48.field.array.<br>
+&nbsp;<br>
+...&nbsp;work&nbsp;in&nbsp;progress&nbsp;for&nbsp;MPyC&nbsp;version&nbsp;0.9</tt></p>
+<p>
+<table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section">
+<tr bgcolor="#aa55cc">
+<td colspan=3 valign=bottom>&nbsp;<br>
+<font color="#ffffff" face="helvetica, arial"><big><strong>Modules</strong></big></font></td></tr>
+    
+<tr><td bgcolor="#aa55cc"><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</tt></td><td>&nbsp;</td>
+<td width="100%"><table width="100%" summary="list"><tr><td width="25%" valign=top><a href="logging.html">logging</a><br>
+</td><td width="25%" valign=top><a href="numpy.html">numpy</a><br>
+</td><td width="25%" valign=top><a href="os.html">os</a><br>
+</td><td width="25%" valign=top></td></tr></table></td></tr></table>
+</body></html>

mpyc/README.md

@@ -1,24 +1,25 @@
 ## Synopsis
 
-[MPyC](https://lschoe.github.io/mpyc) currently consists of 13 modules (all in pure Python):
+[MPyC](https://lschoe.github.io/mpyc) currently consists of 14 modules (all in pure Python):
 
 1. [gmpy](https://lschoe.github.io/mpyc/mpyc.gmpy.html): some basic number theoretic algorithms (using GMP via Python package gmpy2, if installed)
-2. [gfpx](https://lschoe.github.io/mpyc/mpyc.gfpx.html): polynomial arithmetic over arbitrary prime fields
-3. [finfields](https://lschoe.github.io/mpyc/mpyc.finfields.html): arbitrary finite fields, including binary fields and prime fields
-4. [fingroups](https://lschoe.github.io/mpyc/mpyc.fingroups.html): finite groups, in particular for use in cryptography (elliptic curves, Schnorr groups, etc.)
-5. [thresha](https://lschoe.github.io/mpyc/mpyc.thresha.html): threshold Shamir (and also pseudorandom) secret sharing
-6. [sectypes](https://lschoe.github.io/mpyc/mpyc.sectypes.html): SecInt/Fld/Fxp/Flt types for secure (secret-shared) integer/finite-field/fixed-/floating-point values
-7. [asyncoro](https://lschoe.github.io/mpyc/mpyc.asyncoro.html): asynchronous communication and computation of secret-shared values
-8. [runtime](https://lschoe.github.io/mpyc/mpyc.runtime.html): core MPC protocols (many hidden by Python's operator overloading)
-9. [mpctools](https://lschoe.github.io/mpyc/mpyc.mpctools.html): reduce and accumulate with log round complexity
-10. [seclists](https://lschoe.github.io/mpyc/mpyc.seclists.html): secure lists with oblivious access and updates
-11. [secgroups](https://lschoe.github.io/mpyc/mpyc.secgroups.html): SecGrp types for secure (secret-shared) finite group elements
-12. [random](https://lschoe.github.io/mpyc/mpyc.random.html): securely mimicking Python’s [random](https://docs.python.org/3/library/random.html) module
-13. [statistics](https://lschoe.github.io/mpyc/mpyc.statistics.html): securely mimicking Python’s [statistics](https://docs.python.org/3/library/statistics.html) module
+2. [numpy](https://lschoe.github.io/mpyc/mpyc.numpy.html): stub to avoid dependency on NumPy package (also handling version issues, etc.)
+3. [gfpx](https://lschoe.github.io/mpyc/mpyc.gfpx.html): polynomial arithmetic over arbitrary prime fields
+4. [finfields](https://lschoe.github.io/mpyc/mpyc.finfields.html): arbitrary finite fields, including binary fields and prime fields
+5. [fingroups](https://lschoe.github.io/mpyc/mpyc.fingroups.html): finite groups, in particular for use in cryptography (elliptic curves, Schnorr groups, etc.)
+6. [thresha](https://lschoe.github.io/mpyc/mpyc.thresha.html): threshold Shamir (and also pseudorandom) secret sharing
+7. [sectypes](https://lschoe.github.io/mpyc/mpyc.sectypes.html): SecInt/Fld/Fxp/Flt types for secure (secret-shared) integer/finite-field/fixed-/floating-point values
+8. [asyncoro](https://lschoe.github.io/mpyc/mpyc.asyncoro.html): asynchronous communication and computation of secret-shared values
+9. [runtime](https://lschoe.github.io/mpyc/mpyc.runtime.html): core MPC protocols (many hidden by Python's operator overloading)
+10. [mpctools](https://lschoe.github.io/mpyc/mpyc.mpctools.html): reduce and accumulate with log round complexity
+11. [seclists](https://lschoe.github.io/mpyc/mpyc.seclists.html): secure lists with oblivious access and updates
+12. [secgroups](https://lschoe.github.io/mpyc/mpyc.secgroups.html): SecGrp types for secure (secret-shared) finite group elements
+13. [random](https://lschoe.github.io/mpyc/mpyc.random.html): securely mimicking Python’s [random](https://docs.python.org/3/library/random.html) module
+14. [statistics](https://lschoe.github.io/mpyc/mpyc.statistics.html): securely mimicking Python’s [statistics](https://docs.python.org/3/library/statistics.html) module
 
 The modules are listed in topological order w.r.t. internal dependencies:
 
-- Modules 1-4 are basic modules which can also be used outside an MPC context
-- Modules 5-8 form the core of MPyC
-- Modules 9-11 form the extended core of MPyC
-- Modules 12-13 are small libraries on top of the (extended) core
+- Modules 1-5 are basic modules which can also be used outside an MPC context
+- Modules 6-9 form the core of MPyC
+- Modules 10-12 form the extended core of MPyC
+- Modules 13-14 are small libraries on top of the (extended) core

mpyc/runtime.py

@@ -2373,7 +2373,7 @@ def block_shape(a):
             return tuple(_block_shape(a, block_ndim(a))[0])  # TODO: move this to mpyc.numpy module
 
         await self.returnType((sectype, block_shape(arrays)))
-        arrays = await self.gather(arrays)
+        arrays = await self.gather(arrays)  # TODO: handle secfxp
         return np.block(arrays)
 
     @mpc_coro_no_pc
@@ -2520,14 +2520,18 @@ async def np_roll(self, a, shift, axis=None):
         return np.roll(a, shift, axis)
 
     @mpc_coro_no_pc
-    async def np_neg(self, a):
+    async def np_negative(self, a):
         if not a.frac_length:
             await self.returnType((type(a), a.shape))
         else:
             await self.returnType((type(a), a.integral, a.shape))
         a = await self.gather(a)
         return -a
 
+    def np_absolute(self, a, l=None):
+        """Secure absolute value of a."""
+        return (-2*self.np_sgn(a, l=l, LT=True) + 1) * a
+
     def np_less(self, a, b):
         """Secure comparison a < b."""
         return self.np_sgn(a - b, LT=True)

mpyc/sectypes.py

@@ -1090,7 +1090,11 @@ def _coerce2(self, other):
 
     def __neg__(self):
         """Matrix negation."""
-        return runtime.np_neg(self)
+        return runtime.np_negative(self)
+
+    def __abs__(self):
+        """Matrix absolute value."""
+        return runtime.np_absolute(self)
 
     def __add__(self, other):
         """Matrix addition."""
@@ -1381,8 +1385,6 @@ def _coerce(self, other):
 
     def _coerce2(self, other):
         if isinstance(other, float):
-            if other.is_integer():
-                other = round(other)
             return other  # TODO: consider returning np.array(other) here
 
         return super()._coerce2(other)

tests/test_finfields.py

@@ -354,9 +354,13 @@ def test_array_ufunc(self):
         np.negative.at(a, (1, 1))  # NB: in-place
         np.assertEqual(np.add.reduce(a, 1), [16, 0])
         np.assertEqual(np.add.reduce(np.add.reduce(a, 1)), 16)
+        a != a
         a += 2
         a -= 2
         a *= 3
+        a >>= 2
+        a <<= 1
+        a = np.right_shift(np.left_shift(a, 2), 1)
         np.add(np.array([1], dtype=np.int32), a)
         np.add(a, np.array([1], dtype=np.int64))
         self.assertRaises(TypeError, np.add, np.array([1], dtype=np.float64), a)
@@ -382,8 +386,13 @@ def test_array_ufunc(self):
         np.assertEqual(np.reciprocal(F_b) * F_b, np.ones(b.shape, dtype='O'))
         np.assertEqual(np.sqrt(F_a**2)**2, F_a**2)
 
-        F81_b = self.f81.array(b)
-        np.assertEqual(np.sqrt(F81_b**2)**2, F81_b**2)
+        F27_b = self.f27.array(b)
+        F27_b = 1 / (1 / F27_b)
+        np.assertEqual(np.sqrt(F27_b**2)**2, F27_b**2)
+
+        F81_b2 = self.f81.array(b)**2
+        self.assertTrue((F81_b2).is_sqr().all())
+        np.assertEqual(F81_b2.sqrt(INV=True)**2, 1/F81_b2)
 
     @unittest.skipIf(not np, 'NumPy not available or inside MPyC disabled')
     def test_ndarray(self):

tests/test_gmpy.py

@@ -43,7 +43,7 @@ def te_s_t(a, b):
         self.assertTrue(te_s_t(1234, -2*1234))
         self.assertTrue(te_s_t(-2*12364, 12364))
 
-        # self.assertEqual(gmpy.invert(3, -1), 0)  # pending gmpy2 issue if modulus is 1 or -1
+        self.assertEqual(gmpy.invert(3, -1), 0)
         self.assertEqual(gmpy.invert(3, 257), 86)
         self.assertRaises(ZeroDivisionError, gmpy.invert, 2, 0)
         self.assertRaises(ZeroDivisionError, gmpy.invert, 2, 4)