Reject invalid keys, with even moduli, in rsa_compute_root_tr.

ChangeLog

+2016-08-04  Niels Möller  <nisse@lysator.liu.se>
+
+	* rsa-sign-tr.c (rsa_compute_root_tr): Return failure if any of p,
+	q or n is even, to avoid crashing inside mpz_powm_sec. Invalid
+	keys with even modulo are rejected by rsa_public_key_prepare and
+	rsa_private_key_prepare, but some applications, notably gnutls,
+	don't use them.
+
 2016-07-31  Niels Möller  <nisse@lysator.liu.se>
 
 	* rsa.c (_rsa_check_size): Check that n is odd. Otherwise, using

rsa-sign-tr.c

@@ -88,6 +88,14 @@ rsa_compute_root_tr(const struct rsa_public_key *pub,
   int res;
   mpz_t t, mb, xb, ri;
 
+  /* mpz_powm_sec handles only odd moduli. If p, q or n is even, the
+     key is invalid and rejected by rsa_private_key_prepare. However,
+     some applications, notably gnutls, don't use this function, and
+     we don't want an invalid key to lead to a crash down inside
+     mpz_powm_sec. So do an additional check here. */
+  if (mpz_even_p (pub->n) || mpz_even_p (key->p) || mpz_even_p (key->q))
+    return 0;
+
   mpz_init (mb);
   mpz_init (xb);
   mpz_init (ri);