Commit f5c01aa1 authored by Niels Möller's avatar Niels Möller
Browse files

Added this rfc as test input for yarrow.

Rev: src/nettle/testsuite/rfc1750.txt:1.1
parent 92fa8bbe
Network Working Group D. Eastlake, 3rd
Request for Comments: 1750 DEC
Category: Informational S. Crocker
Cybercash
J. Schiller
MIT
December 1994
Randomness Recommendations for Security
Status of this Memo
This memo provides information for the Internet community. This memo
does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.
Abstract
Security systems today are built on increasingly strong cryptographic
algorithms that foil pattern analysis attempts. However, the security
of these systems is dependent on generating secret quantities for
passwords, cryptographic keys, and similar quantities. The use of
pseudo-random processes to generate secret quantities can result in
pseudo-security. The sophisticated attacker of these security
systems may find it easier to reproduce the environment that produced
the secret quantities, searching the resulting small set of
possibilities, than to locate the quantities in the whole of the
number space.
Choosing random quantities to foil a resourceful and motivated
adversary is surprisingly difficult. This paper points out many
pitfalls in using traditional pseudo-random number generation
techniques for choosing such quantities. It recommends the use of
truly random hardware techniques and shows that the existing hardware
on many systems can be used for this purpose. It provides
suggestions to ameliorate the problem when a hardware solution is not
available. And it gives examples of how large such quantities need
to be for some particular applications.
Eastlake, Crocker & Schiller [Page 1]
RFC 1750 Randomness Recommendations for Security December 1994
Acknowledgements
Comments on this document that have been incorporated were received
from (in alphabetic order) the following:
David M. Balenson (TIS)
Don Coppersmith (IBM)
Don T. Davis (consultant)
Carl Ellison (Stratus)
Marc Horowitz (MIT)
Christian Huitema (INRIA)
Charlie Kaufman (IRIS)
Steve Kent (BBN)
Hal Murray (DEC)
Neil Haller (Bellcore)
Richard Pitkin (DEC)
Tim Redmond (TIS)
Doug Tygar (CMU)
Table of Contents
1. Introduction........................................... 3
2. Requirements........................................... 4
3. Traditional Pseudo-Random Sequences.................... 5
4. Unpredictability....................................... 7
4.1 Problems with Clocks and Serial Numbers............... 7
4.2 Timing and Content of External Events................ 8
4.3 The Fallacy of Complex Manipulation.................. 8
4.4 The Fallacy of Selection from a Large Database....... 9
5. Hardware for Randomness............................... 10
5.1 Volume Required...................................... 10
5.2 Sensitivity to Skew.................................. 10
5.2.1 Using Stream Parity to De-Skew..................... 11
5.2.2 Using Transition Mappings to De-Skew............... 12
5.2.3 Using FFT to De-Skew............................... 13
5.2.4 Using Compression to De-Skew....................... 13
5.3 Existing Hardware Can Be Used For Randomness......... 14
5.3.1 Using Existing Sound/Video Input................... 14
5.3.2 Using Existing Disk Drives......................... 14
6. Recommended Non-Hardware Strategy..................... 14
6.1 Mixing Functions..................................... 15
6.1.1 A Trivial Mixing Function.......................... 15
6.1.2 Stronger Mixing Functions.......................... 16
6.1.3 Diff-Hellman as a Mixing Function.................. 17
6.1.4 Using a Mixing Function to Stretch Random Bits..... 17
6.1.5 Other Factors in Choosing a Mixing Function........ 18
6.2 Non-Hardware Sources of Randomness................... 19
6.3 Cryptographically Strong Sequences................... 19
Eastlake, Crocker & Schiller [Page 2]
RFC 1750 Randomness Recommendations for Security December 1994
6.3.1 Traditional Strong Sequences....................... 20
6.3.2 The Blum Blum Shub Sequence Generator.............. 21
7. Key Generation Standards.............................. 22
7.1 US DoD Recommendations for Password Generation....... 23
7.2 X9.17 Key Generation................................. 23
8. Examples of Randomness Required....................... 24
8.1 Password Generation................................. 24
8.2 A Very High Security Cryptographic Key............... 25
8.2.1 Effort per Key Trial............................... 25
8.2.2 Meet in the Middle Attacks......................... 26
8.2.3 Other Considerations............................... 26
9. Conclusion............................................ 27
10. Security Considerations.............................. 27
References............................................... 28
Authors' Addresses....................................... 30
1. Introduction
Software cryptography is coming into wider use. Systems like
Kerberos, PEM, PGP, etc. are maturing and becoming a part of the
network landscape [PEM]. These systems provide substantial
protection against snooping and spoofing. However, there is a
potential flaw. At the heart of all cryptographic systems is the
generation of secret, unguessable (i.e., random) numbers.
For the present, the lack of generally available facilities for
generating such unpredictable numbers is an open wound in the design
of cryptographic software. For the software developer who wants to
build a key or password generation procedure that runs on a wide
range of hardware, the only safe strategy so far has been to force
the local installation to supply a suitable routine to generate
random numbers. To say the least, this is an awkward, error-prone
and unpalatable solution.
It is important to keep in mind that the requirement is for data that
an adversary has a very low probability of guessing or determining.
This will fail if pseudo-random data is used which only meets
traditional statistical tests for randomness or which is based on
limited range sources, such as clocks. Frequently such random
quantities are determinable by an adversary searching through an
embarrassingly small space of possibilities.
This informational document suggests techniques for producing random
quantities that will be resistant to such attack. It recommends that
future systems include hardware random number generation or provide
access to existing hardware that can be used for this purpose. It
suggests methods for use if such hardware is not available. And it
gives some estimates of the number of random bits required for sample
Eastlake, Crocker & Schiller [Page 3]
RFC 1750 Randomness Recommendations for Security December 1994
applications.
2. Requirements
Probably the most commonly encountered randomness requirement today
is the user password. This is usually a simple character string.
Obviously, if a password can be guessed, it does not provide
security. (For re-usable passwords, it is desirable that users be
able to remember the password. This may make it advisable to use
pronounceable character strings or phrases composed on ordinary
words. But this only affects the format of the password information,
not the requirement that the password be very hard to guess.)
Many other requirements come from the cryptographic arena.
Cryptographic techniques can be used to provide a variety of services
including confidentiality and authentication. Such services are
based on quantities, traditionally called "keys", that are unknown to
and unguessable by an adversary.
In some cases, such as the use of symmetric encryption with the one
time pads [CRYPTO*] or the US Data Encryption Standard [DES], the
parties who wish to communicate confidentially and/or with
authentication must all know the same secret key. In other cases,
using what are called asymmetric or "public key" cryptographic
techniques, keys come in pairs. One key of the pair is private and
must be kept secret by one party, the other is public and can be
published to the world. It is computationally infeasible to
determine the private key from the public key [ASYMMETRIC, CRYPTO*].
The frequency and volume of the requirement for random quantities
differs greatly for different cryptographic systems. Using pure RSA
[CRYPTO*], random quantities are required when the key pair is
generated, but thereafter any number of messages can be signed
without any further need for randomness. The public key Digital
Signature Algorithm that has been proposed by the US National
Institute of Standards and Technology (NIST) requires good random
numbers for each signature. And encrypting with a one time pad, in
principle the strongest possible encryption technique, requires a
volume of randomness equal to all the messages to be processed.
In most of these cases, an adversary can try to determine the
"secret" key by trial and error. (This is possible as long as the
key is enough smaller than the message that the correct key can be
uniquely identified.) The probability of an adversary succeeding at
this must be made acceptably low, depending on the particular
application. The size of the space the adversary must search is
related to the amount of key "information" present in the information
theoretic sense [SHANNON]. This depends on the number of different
Eastlake, Crocker & Schiller [Page 4]
RFC 1750 Randomness Recommendations for Security December 1994
secret values possible and the probability of each value as follows:
-----
\
Bits-of-info = \ - p * log ( p )
/ i 2 i
/
-----
where i varies from 1 to the number of possible secret values and p
sub i is the probability of the value numbered i. (Since p sub i is
less than one, the log will be negative so each term in the sum will
be non-negative.)
If there are 2^n different values of equal probability, then n bits
of information are present and an adversary would, on the average,
have to try half of the values, or 2^(n-1) , before guessing the
secret quantity. If the probability of different values is unequal,
then there is less information present and fewer guesses will, on
average, be required by an adversary. In particular, any values that
the adversary can know are impossible, or are of low probability, can
be initially ignored by an adversary, who will search through the
more probable values first.
For example, consider a cryptographic system that uses 56 bit keys.
If these 56 bit keys are derived by using a fixed pseudo-random
number generator that is seeded with an 8 bit seed, then an adversary
needs to search through only 256 keys (by running the pseudo-random
number generator with every possible seed), not the 2^56 keys that
may at first appear to be the case. Only 8 bits of "information" are
in these 56 bit keys.
3. Traditional Pseudo-Random Sequences
Most traditional sources of random numbers use deterministic sources
of "pseudo-random" numbers. These typically start with a "seed"
quantity and use numeric or logical operations to produce a sequence
of values.
[KNUTH] has a classic exposition on pseudo-random numbers.
Applications he mentions are simulation of natural phenomena,
sampling, numerical analysis, testing computer programs, decision
making, and games. None of these have the same characteristics as
the sort of security uses we are talking about. Only in the last two
could there be an adversary trying to find the random quantity.
However, in these cases, the adversary normally has only a single
chance to use a guessed value. In guessing passwords or attempting
to break an encryption scheme, the adversary normally has many,
Eastlake, Crocker & Schiller [Page 5]
RFC 1750 Randomness Recommendations for Security December 1994
perhaps unlimited, chances at guessing the correct value and should
be assumed to be aided by a computer.
For testing the "randomness" of numbers, Knuth suggests a variety of
measures including statistical and spectral. These tests check
things like autocorrelation between different parts of a "random"
sequence or distribution of its values. They could be met by a
constant stored random sequence, such as the "random" sequence
printed in the CRC Standard Mathematical Tables [CRC].
A typical pseudo-random number generation technique, known as a
linear congruence pseudo-random number generator, is modular
arithmetic where the N+1th value is calculated from the Nth value by
V = ( V * a + b )(Mod c)
N+1 N
The above technique has a strong relationship to linear shift
register pseudo-random number generators, which are well understood
cryptographically [SHIFT*]. In such generators bits are introduced
at one end of a shift register as the Exclusive Or (binary sum
without carry) of bits from selected fixed taps into the register.
For example:
+----+ +----+ +----+ +----+
| B | <-- | B | <-- | B | <-- . . . . . . <-- | B | <-+
| 0 | | 1 | | 2 | | n | |
+----+ +----+ +----+ +----+ |
| | | |
| | V +-----+
| V +----------------> | |
V +-----------------------------> | XOR |
+---------------------------------------------------> | |
+-----+
V = ( ( V * 2 ) + B .xor. B ... )(Mod 2^n)
N+1 N 0 2
The goodness of traditional pseudo-random number generator algorithms
is measured by statistical tests on such sequences. Carefully chosen
values of the initial V and a, b, and c or the placement of shift
register tap in the above simple processes can produce excellent
statistics.
Eastlake, Crocker & Schiller [Page 6]
RFC 1750 Randomness Recommendations for Security December 1994
These sequences may be adequate in simulations (Monte Carlo
experiments) as long as the sequence is orthogonal to the structure
of the space being explored. Even there, subtle patterns may cause
problems. However, such sequences are clearly bad for use in
security applications. They are fully predictable if the initial
state is known. Depending on the form of the pseudo-random number
generator, the sequence may be determinable from observation of a
short portion of the sequence [CRYPTO*, STERN]. For example, with
the generators above, one can determine V(n+1) given knowledge of
V(n). In fact, it has been shown that with these techniques, even if
only one bit of the pseudo-random values is released, the seed can be
determined from short sequences.
Not only have linear congruent generators been broken, but techniques
are now known for breaking all polynomial congruent generators
[KRAWCZYK].
4. Unpredictability
Randomness in the traditional sense described in section 3 is NOT the
same as the unpredictability required for security use.
For example, use of a widely available constant sequence, such as
that from the CRC tables, is very weak against an adversary. Once
they learn of or guess it, they can easily break all security, future
and past, based on the sequence [CRC]. Yet the statistical
properties of these tables are good.
The following sections describe the limitations of some randomness
generation techniques and sources.
4.1 Problems with Clocks and Serial Numbers
Computer clocks, or similar operating system or hardware values,
provide significantly fewer real bits of unpredictability than might
appear from their specifications.
Tests have been done on clocks on numerous systems and it was found
that their behavior can vary widely and in unexpected ways. One
version of an operating system running on one set of hardware may
actually provide, say, microsecond resolution in a clock while a
different configuration of the "same" system may always provide the
same lower bits and only count in the upper bits at much lower
resolution. This means that successive reads on the clock may
produce identical values even if enough time has passed that the
value "should" change based on the nominal clock resolution. There
are also cases where frequently reading a clock can produce
artificial sequential values because of extra code that checks for
Eastlake, Crocker & Schiller [Page 7]
RFC 1750 Randomness Recommendations for Security December 1994
the clock being unchanged between two reads and increases it by one!
Designing portable application code to generate unpredictable numbers
based on such system clocks is particularly challenging because the
system designer does not always know the properties of the system
clocks that the code will execute on.
Use of a hardware serial number such as an Ethernet address may also
provide fewer bits of uniqueness than one would guess. Such
quantities are usually heavily structured and subfields may have only
a limited range of possible values or values easily guessable based
on approximate date of manufacture or other data. For example, it is
likely that most of the Ethernet cards installed on Digital Equipment
Corporation (DEC) hardware within DEC were manufactured by DEC
itself, which significantly limits the range of built in addresses.
Problems such as those described above related to clocks and serial
numbers make code to produce unpredictable quantities difficult if
the code is to be ported across a variety of computer platforms and
systems.
4.2 Timing and Content of External Events
It is possible to measure the timing and content of mouse movement,
key strokes, and similar user events. This is a reasonable source of
unguessable data with some qualifications. On some machines, inputs
such as key strokes are buffered. Even though the user's inter-
keystroke timing may have sufficient variation and unpredictability,
there might not be an easy way to access that variation. Another
problem is that no standard method exists to sample timing details.
This makes it hard to build standard software intended for
distribution to a large range of machines based on this technique.
The amount of mouse movement or the keys actually hit are usually
easier to access than timings but may yield less unpredictability as
the user may provide highly repetitive input.
Other external events, such as network packet arrival times, can also
be used with care. In particular, the possibility of manipulation of
such times by an adversary must be considered.
4.3 The Fallacy of Complex Manipulation
One strategy which may give a misleading appearance of
unpredictability is to take a very complex algorithm (or an excellent
traditional pseudo-random number generator with good statistical
properties) and calculate a cryptographic key by starting with the
current value of a computer system clock as the seed. An adversary
who knew roughly when the generator was started would have a
Eastlake, Crocker & Schiller [Page 8]
RFC 1750 Randomness Recommendations for Security December 1994
relatively small number of seed values to test as they would know
likely values of the system clock. Large numbers of pseudo-random
bits could be generated but the search space an adversary would need
to check could be quite small.
Thus very strong and/or complex manipulation of data will not help if
the adversary can learn what the manipulation is and there is not
enough unpredictability in the starting seed value. Even if they can
not learn what the manipulation is, they may be able to use the
limited number of results stemming from a limited number of seed
values to defeat security.
Another serious strategy error is to assume that a very complex
pseudo-random number generation algorithm will produce strong random
numbers when there has been no theory behind or analysis of the
algorithm. There is a excellent example of this fallacy right near
the beginning of chapter 3 in [KNUTH] where the author describes a
complex algorithm. It was intended that the machine language program
corresponding to the algorithm would be so complicated that a person
trying to read the code without comments wouldn't know what the
program was doing. Unfortunately, actual use of this algorithm
showed that it almost immediately converged to a single repeated
value in one case and a small cycle of values in another case.
Not only does complex manipulation not help you if you have a limited
range of seeds but blindly chosen complex manipulation can destroy
the randomness in a good seed!
4.4 The Fallacy of Selection from a Large Database
Another strategy that can give a misleading appearance of
unpredictability is selection of a quantity randomly from a database
and assume that its strength is related to the total number of bits
in the database. For example, typical USENET servers as of this date
process over 35 megabytes of information per day. Assume a random
quantity was selected by fetching 32 bytes of data from a random
starting point in this data. This does not yield 32*8 = 256 bits
worth of unguessability. Even after allowing that much of the data
is human language and probably has more like 2 or 3 bits of
information per byte, it doesn't yield 32*2.5 = 80 bits of
unguessability. For an adversary with access to the same 35
megabytes the unguessability rests only on the starting point of the
selection. That is, at best, about 25 bits of unguessability in this
case.
The same argument applies to selecting sequences from the data on a
CD ROM or Audio CD recording or any other large public database. If
the adversary has access to the same database, this "selection from a
Eastlake, Crocker & Schiller [Page 9]
RFC 1750 Randomness Recommendations for Security December 1994
large volume of data" step buys very little. However, if a selection
can be made from data to which the adversary has no access, such as
system buffers on an active multi-user system, it may be of some
help.
5. Hardware for Randomness
Is there any hope for strong portable randomness in the future?
There might be. All that's needed is a physical source of
unpredictable numbers.
A thermal noise or radioactive decay source and a fast, free-running
oscillator would do the trick directly [GIFFORD]. This is a trivial
amount of hardware, and could easily be included as a standard part
of a computer system's architecture. Furthermore, any system with a
spinning disk or the like has an adequate source of randomness
[DAVIS]. All that's needed is the common perception among computer
vendors that this small additional hardware and the software to
access it is necessary and useful.
5.1 Volume Required
How much unpredictability is needed? Is it possible to quantify the
requirement in, say, number of random bits per second?
The answer is not very much is needed. For DES, the key is 56 bits
and, as we show in an example in Section 8, even the highest security
system is unlikely to require a keying material of over 200 bits. If
a series of keys are needed, it can be generated from a strong random
seed using a cryptographically strong sequence as explained in
Section 6.3. A few hundred random bits generated once a day would be
enough using such techniques. Even if the random bits are generated
as slowly as one per second and it is not possible to overlap the
generation process, it should be tolerable in high security
applications to wait 200 seconds occasionally.
These numbers are trivial to achieve. It could be done by a person
repeatedly tossing a coin. Almost any hardware process is likely to
be much faster.
5.2 Sensitivity to Skew
Is there any specific requirement on the shape of the distribution of
the random numbers? The good news is the distribution need not be
uniform. All that is needed is a conservative estimate of how non-
uniform it is to bound performance. Two simple techniques to de-skew
the bit stream are given below and stronger techniques are mentioned
in Section 6.1.2 below.
Eastlake, Crocker & Schiller [Page 10]
RFC 1750 Randomness Recommendations for Security December 1994
5.2.1 Using Stream Parity to De-Skew
Consider taking a sufficiently long string of bits and map the string
to "zero" or "one". The mapping will not yield a perfectly uniform
distribution, but it can be as close as desired. One mapping that
serves the purpose is to take the parity of the string. This has the
advantages that it is robust across all degrees of skew up to the
estimated maximum skew and is absolutely trivial to implement in
hardware.
The following analysis gives the number of bits that must be sampled:
Suppose the ratio of ones to zeros is 0.5 + e : 0.5 - e, where e is
between 0 and 0.5 and is a measure of the "eccentricity" of the
distribution. Consider the distribution of the parity function of N
bit samples. The probabilities that the parity will be one or zero
will be the sum of the odd or even terms in the binomial expansion of
(p + q)^N, where p = 0.5 + e, the probability of a one, and q = 0.5 -
e, the probability of a zero.
These sums can be computed easily as
N N
1/2 * ( ( p + q ) + ( p - q ) )
and
N N
1/2 * ( ( p + q ) - ( p - q ) ).
(Which one corresponds to the probability the parity will be 1
depends on whether N is odd or even.)
Since p + q = 1 and p - q = 2e, these expressions reduce to
N
1/2 * [1 + (2e) ]
and
N
1/2 * [1 - (2e) ].
Neither of these will ever be exactly 0.5 unless e is zero, but we
can bring them arbitrarily close to 0.5. If we want the
probabilities to be within some delta d of 0.5, i.e. then
N
( 0.5 + ( 0.5 * (2e) ) ) < 0.5 + d.
Eastlake, Crocker & Schiller [Page 11]
RFC 1750 Randomness Recommendations for Security December 1994
Solving for N yields N > log(2d)/log(2e). (Note that 2e is less than
1, so its log is negative. Division by a negative number reverses
the sense of an inequality.)
The following table gives the length of the string which must be
sampled for various degrees of skew in order to come within 0.001 of
a 50/50 distribution.
+---------+--------+-------+
| Prob(1) | e | N |
+---------+--------+-------+
| 0.5 | 0.00 | 1 |
| 0.6 | 0.10 | 4 |
| 0.7 | 0.20 | 7 |
| 0.8 | 0.30 | 13 |
| 0.9 | 0.40 | 28 |
| 0.95 | 0.45 | 59 |
| 0.99 | 0.49 | 308 |
+---------+--------+-------+
The last entry shows that even if the distribution is skewed 99% in
favor of ones, the parity of a string of 308 samples will be within
0.001 of a 50/50 distribution.
5.2.2 Using Transition Mappings to De-Skew
Another technique, originally due to von Neumann [VON NEUMANN], is to
examine a bit stream as a sequence of non-overlapping pairs. You
could then discard any 00 or 11 pairs found, interpret 01 as a 0 and
10 as a 1. Assume the probability of a 1 is 0.5+e and the
probability of a 0 is 0.5-e where e is the eccentricity of the source
and described in the previous section. Then the probability of each
pair is as follows:
+------+-----------------------------------------+
| pair | probability |
+------+-----------------------------------------+