/*
=APPENDIX  RVG  A Short Introduction to Random Variate Generation

=UP TOP [a03]

=DESCRIPTION

   Random variate generation is the small field of research that deals
   with algorithms to generate random variates from various
   distributions. It is common to assume that a uniform random number
   generator is available. This is a program that produces a sequence
   of independent and identically  distributed continuous
   @unurmath{U(0,1)} random variates (i.e. uniform random variates  
   on the interval @unurmath{(0,1)}). Of course real world computers can
   never generate ideal random numbers and they cannot produce numbers
   of arbitrary precision but state-of-the-art uniform random number
   generators come close to this aim. Thus random variate generation
   deals with the problem of transforming such a sequence of
   @unurmath{U(0,1)} random numbers into non-uniform random variates.

   Here we shortly explain the basic ideas of the @emph{inversion},
   @emph{rejection}, and the @emph{ratio of uniforms} method. How
   these ideas can be used to design a particular automatic random
   variate generation algorithms that can be applied to large classes
   of distributions is shortly explained in the description of the
   different methods included in this manual. 
   
   For a deeper treatment of the ideas presented here, for other
   basic methods and for automatic generators we refer the interested
   reader to our book [HLD04].

=EON
*/
/* ------------------------------------------------------------------------- */
/*
=NODE  Inversion  The Inversion Method

=UP RVG [10]

=DESCRIPTION

   When the inverse @unurmath{F^{-1}} of the cumulative distribution
   function is known, then random variate generation is easy.
   We just generate a uniformly @unurmath{U(0,1)} distributed random
   number @unurmath{U} and return 

   @unurmathdisplay{X=F^{-1}(U).} 

   The following figure shows how the inversion method works for
   the exponential distribution.

   @unurimage{figures/inversion}

   This algorithm is so simple that inversion is certainly the method
   of choice if the inverse CDF is available in closed form. This is
   the case e.g. for the exponential and the Cauchy distribution. 

   The inversion method also has other special advantages that make it
   even more attractive for simulation purposes.  
   It preserves the structural properties of the underlying uniform
   pseudo-random number generator.
   Consequently it can be used, e.g., for variance reduction techniques, 
   it is easy to sample from truncated distributions, from marginal
   distributions, and from order statistics.
   Moreover, the quality of the generated random variables depends only
   on the underlying uniform (pseudo-) random number generator. 
   Another important advantage of the inversion method is that we can
   easily characterize its performance. To generate one random variate
   we always need exactly one uniform variate and one evaluation of
   the inverse CDF. So its speed mainly depends on the costs for
   evaluating the inverse CDF. Hence inversion is often considered as
   the method of choice in the simulation literature.

   Unfortunately computing the inverse CDF is, for many important
   standard distributions (e.g. for normal, student, gamma, and
   beta-distributions), comparatively difficult and slow. Often no
   such routines are available in standard programming libraries. 
   Then numerical methods for inverting the CDF are necessary, e.g.
   Newton's method or (polynomial or rational) approximations of the
   inverse CDF. Such procedures, however, have the disadvantage that
   they are not exact and/or slow.
   UNU.RAN implements several methods:
   NINV (@pxref{NINV}), HINV (@pxref{HINV}) and PINV (@pxref{PINV}).
   
   For such approximate inversion methods the approximation error is
   important for the quality of the generated point set.
   Let @unurmath{X=G^{-1}(U)} denote the approximate inverse CDF,
   and let @unurmath{F} and @unurmath{F^{-1}} be the exact CDF and
   inverse CDF of the distribution, resp.
   There are three measures for the approximation error:

   @table @emph   
   @item u-error
   is given by
   @unurmathdisplay{uerror = |U-F(G^{-1}(U))|}
   Goodness-of-fit tests like the Kolmogorov-Smirnov test or the
   chi-squared test look at this type of error.
   We are also convinced that it is the most suitable error measure for
   Monte Carlo simulations as pseudo-random numbers and points of low
   discrepancy sets are located on a grid of restricted resolution.

   @item x-error
   is given by 
   @unurmathdisplay{absolute xerror = |F^{-1}(U)-G^{-1}(U)|}
   @unurmathdisplay{relative xerror = |F^{-1}(U)-G^{-1}(U)|\cdot |F^{-1}(U)|}
   The x-error measure the deviation of @unurmath{G^{-1}(U)}
   from the exact result. This measure is suitable when the inverse
   CDF is used as a quantile function in some computations.
   The main problem with the x-error is that we have to use the
   @emph{absolute x-error} for @unurmath{X=F^{-1}(U)} close to zero
   and the @emph{relative x-error} in the tails.

   @end table
   
   We use the terms @emph{u-resolution} and @emph{x-resolution} as the
   maximal tolerated u-error and x-error, resp.

   UNU.RAN allows to set u-resolution and x-resolution
   independently. Both requirements must be fulfilled.
   We use the following strategy for checking whether the
   precision goal is reached:

   @table @emph   
   @item checking u-error:
   The u-error must be slightly smaller than the given u-resolution:
   @unurmathdisplay{|U-F(G^{-1}(U))| < 0.9\cdot uresolution.}
   There is no necessity to consinder the relative u-error as we have
   @unurmath{0 < U < 1.}

   @item checking x-error:
   We combine absoute and relative x-error and use the criterion
   @unurmathdisplay{|F^{-1}(U)-G^{-1}(U)| < 
   xresolution \cdot (|G^{-1}(U)| + xresolution).}

   @end table

   @strong{Remark:}
   It should be noted here that the criterion based on the u-error is
   too stringent whereever the CDF is extremely steep (and thus the
   PDF has a pole or a high and narrow peak). This is in particular a
   problem for distributions with a pole (e.g., the gamma distribution
   with shape parameter less than 0.5).
   On the other hand using a criterion based on the x-error causes
   problems whereever the CDF is extremly flat. This is in particular
   the case in the (far) tails of heavy-tailed distributions (e.g.,
   for the Cauchy distribution).

=EON
*/
/* ------------------------------------------------------------------------- */

/*
=NODE  Rejection  The Rejection Method

=UP RVG [20]

=DESCRIPTION

   The rejection method, often called @emph{acceptance-rejection
   method}, has been suggested by John von Neumann in 1951.
   Since then it has proven to be the most flexible and most efficient 
   method to generate variates from continuous distributions. 

   We explain the rejection principle first for the density
   @unurmath{f(x) = sin(x)/2} on the interval @unurmath{(0,\pi).}
   To generate random variates from this distribution we also can
   sample random points that are uniformly distributed in the region
   between the graph of @unurmath{f(x)} and the @i{x}-axis,
   i.e., the shaded region in the below figure.

   @unurimage{figures/rejection_from_constant_hat}

   In general this is not a trivial task but in this example we can 
   easily use the rejection trick: 
   Sample a random point @unurmath{(X,Y)} uniformly in the
   bounding rectangle @unurmath{(0,\pi)\times(0,0.5).} This is easy since
   each coordinate can be sampled independently from the respective
   uniform distributions @unurmath{U(0,\pi)} and @unurmath{U(0,0.5).}
   Whenever the point falls into the shaded region below the graph 
   (indicated by dots in the figure), i.e., when @unurmath{Y <
   sin(X)/2,} we accept it and return @unurmath{X} as a random variate
   from the distribution with density @unurmath{f(x).} 
   Otherwise we have to reject the point (indicated by small circles
   in the figure), and try again.

   It is quite clear that this idea works for every distribution with
   a bounded density on a bounded domain. Moreover, we can use this
   procedure with any multiple of the density, i.e., with any positive
   bounded function with bounded integral and it is not necessary to
   know the integral of this function. So we use the term density in
   the sequel for any positive function with bounded integral.

   From the figure we can conclude that the performance of a rejection
   algorithm depends heavily on the area of the enveloping
   rectangle. Moreover, the method does not work if the target
   distribution has infinite tails (or is unbounded). Hence  
   non-rectangular shaped regions for the envelopes are important and
   we have to solve the problem of sampling points uniformly from such
   domains. Looking again at the example above we notice that the
   @i{x}-coordinate of the random point @unurmath{(X,Y)} was
   sampled by inversion from the uniform distribution on the domain of
   the given density. This motivates us to replace the density of the
   uniform distribution by the (multiple of a) density @unurmath{h(x)}
   of some other appropriate distribution. We only have to take care
   that it is chosen such that it is always an upper bound, i.e.,
   @unurmath{h(x) >= f(x)} for all @unurmath{x} in the domain of the
   distribution. To generate the pair @unurmath{(X,Y)} we generate
   @unurmath{X} from the distribution with density proportional to
   @unurmath{h(x)} and @unurmath{Y} uniformly between @unurmath{0} and
   @unurmath{h(X).} The first step (generate @unurmath{X}) is usually
   done by inversion (@pxref{Inversion}).

   Thus the general rejection algorithm for a hat @unurmath{h(x)} with 
   inverse CDF @unurmath{H^{-1}}consists of the following steps:
   @enumerate
   @item
   Generate a @unurmath{U(0,1)} random number @unurmath{U.}
   @item
   Set @unurmath{X} to @unurmath{H^{-1}(U).}
   @item
   Generate a @unurmath{U(0,1)} random number @unurmath{V.}
   @item
   Set @unurmath{Y} to @unurmath{V h(X).}
   @item
   If @unurmath{Y <= f(X)} accept @unurmath{X} as the random variate.
   @item
   Else try again.
   @end enumerate

   If the evaluation of the density @unurmath{f(x)} is expensive
   (i.e., time consuming) it is possible to use a simple lower bound
   of the density as so called @emph{squeeze function}
   @unurmath{s(x)} (the triangular shaped function in the above
   figure is an example for such a squeeze). We can then accept
   @unurmath{X} when @unurmath{Y <= s(X)} and can thus often save the
   evaluation of the density. 

   We have seen so far that the rejection principle leads to
   short and simple generation algorithms. The main practical problem
   to apply the rejection algorithm is the search for a good fitting
   hat function and for squeezes. We do not discuss these topics here
   as they are the heart of the different automatic algorithms
   implemented in UNU.RAN. Information about the construction of hat
   and squeeze can therefore be found in the descriptions of the
   methods. 
   
   The performance characteristics of rejection algorithms mainly
   depend on the fit of the hat and the squeeze. It is not difficult
   to prove that:
   @itemize @bullet
   @item
   The expected number of trials to generate one variate is the ratio
   between the area below the hat and the area below the density. 
   @item
   The expected number of evaluations of the density necessary to
   generate one variate is equal to the ratio between the area below
   the hat and the area below the density, when no squeeze is used.
   Otherwise, when a squeeze is given it is equal to the ratio
   between the area between hat and squeeze and the area below the hat.
   @item
   The @code{sqhratio} (i.e., the ratio between the area below the
   squeeze and the area below the hat) used in some of the UNU.RAN
   methods is easy to compute. It is useful as its reciprocal is an
   upper bound for the expected number of trials of the rejection
   algoritm. The expected number of evaluations of the density is
   bounded by @unurmath{(1/sqhratio)-1.}
   @end itemize

=EON
*/
/* ------------------------------------------------------------------------- */
/*
=NODE  Composition  The Composition Method

=UP RVG [30]

=DESCRIPTION

   The composition method is an important principle to facilitate and
   speed up random variate generation. The basic idea is simple. 
   To generate random variates with a given density we first split the
   domain of the density into subintervals. Then we select one of
   these randomly with probabilities given by the area below the
   density in the respective subintervals. Finally we generate a
   random variate from the density of the selected part by inversion
   and return it as random variate of the full distribution.

   Composition can be combined with rejection. Thus it is possible to
   decompose the domain of the distribution into subintervals and to
   construct hat and squeeze functions seperatly in every
   subinterval. The area below the hat must be determined in every
   subinterval. Then the Composition rejection algorithm contains the
   following steps:

   @enumerate
   @item
   Generate the index @unurmath{J} of the subinterval as the
   realisation of a discrete random variate with probabilities
   proportional to the area below the hat.
   @item
   Generate a random variate @unurmath{X} proportional to the hat in
   interval @unurmath{J.}
   @item
   Generate the @unurmath{U(0,f(X))} random number @unurmath{Y.}
   @item
   If @unurmath{Y <= f(X)} accept @unurmath{X} as random variate.
   @item
   Else start again with generating the index @unurmath{J.}
   @end enumerate
   
   The first step can be done in constant time (i.e., independent of
   the number of chosen subintervals) by means of the indexed search
   method (@pxref{IndexedSearch}).
   
   It is possible to reduce the number of uniform random numbers
   required in the above algorithm by recycling the random numbers
   used in Step 1 and additionally by applying the principle of
   @emph{immediate acceptance}. 
   For details see @unurbibref{HLD04: Sect. 3.1}. 

=EON
*/
/* ------------------------------------------------------------------------- */
/*
=NODE  Ratio-of-Uniforms The Ratio-of-Uniforms Method

=UP RVG [40]

=DESCRIPTION

   The construction of an appropriate hat function for the given
   density is the crucial step for constructing rejection algorithms.
   Equivalently we can try to find an appropriate envelope for the region
   between the graph of the density and the @i{x}-axis, such
   that we can easily sample uniformly distributed random points.
   This task could become easier if we can find transformations that map
   the region between the density and the axis into a region of more
   suitable shape (for example into a bounded region).

   As a first example we consider the following simple algorithm for
   the Cauchy distribution.
   @enumerate
   @item
   Generate a @unurmath{U(-1,1)} random number @unurmath{U} and a
   @unurmath{U(0,1)} random number @unurmath{V.}
   @item
   If @unurmath{U^2 + V^2 <= 1} accept @unurmath{X=U/V} as a Cauchy
   random variate. 
   @item
   Else try again.
   @end enumerate

   It is possible to prove that the above algorithm indeed  generates
   Cauchy random variates. The fundamental principle behind this
   algorithm is the fact that the region below the density is mapped
   by the transformation 

   @unurmathdisplay{(X,Y)\mapsto(U,V)=(2\,X\sqrt{Y},2\,\sqrt{Y})}

   into a half-disc in such a way that the ratio between the area of 
   the image to the area of the preimage is constant. This is due to
   the fact that that the Jacobian of this transformation is constant.

   @unurimage{figures/rou-cauchy}

   The above example is a special case of a more general principle,
   called the @emph{Ratio-of-uniforms (RoU) method}. It is based on
   the fact that for a random variable @unurmath{X} with density
   @unurmath{f(x)} and some constant @unurmath{\mu} we can generate
   @unurmath{X} from the desired density by calculating
   @unurmath{X=U/V+\mu} for a pair @unurmath{(U,V)} uniformly
   distributed in the set

   @unurmathdisplay{A_f= \{(u,v)\colon 0 < v \leq \sqrt{f(u/v+\mu)}\}.}

   For most distributions it is best to set the constant
   @unurmath{\mu} equal to the mode of the distribution. For sampling
   random points uniformly distributed in @unurmath{A_f} rejection
   from a convenient enveloping region is used, usually the minimal
   bounding rectangle, i.e., the smallest possible rectangle that
   contains @unurmath{A_f} (see the above figure).
   It is given by @unurmath{(u^-,u^+)\times (0,v^+)} where
   
   @unurmathdisplay{
   v^+ = \sup\limits_{b_l<x<b_r}         \sqrt{f(x)}, \\
   u^- = \inf\limits_{b_l<x<b_r} (x-\mu) \sqrt{f(x)}, \\
   u^+ = \sup\limits_{b_l<x<b_r} (x-\mu) \sqrt{f(x)}.}

   Then the ratio-of-uniforms method consists of the following simple
   steps: 
   @enumerate
   @item
   Generate a @unurmath{U(u^-,u^+)} random number @unurmath{U} and a
   @unurmath{U(0,v^+)} random number @unurmath{V.}
   @item
   Set @unurmath{X} to @unurmath{U/V+\mu.}
   @item
   If @unurmath{V^2 \leq f(X)} accept @unurmath{X} as the random
   variate.
   @item
   Else try again.
   @end enumerate

   To apply the ratio-of-uniforms algorithm to a certain density 
   we have to solve the simple optimization problems in the
   definitions above to obtain the design constants @unurmath{u^-,}
   @unurmath{u^+,} and @unurmath{v^+.}  
   This simple algorithm works for all distributions
   with bounded densities that have subquadratic tails (i.e.,
   tails like @unurmath{1/x^2} or lower). For most standard
   distributions it has quite good rejection constants. 
   (E.g. 1.3688 for the normal and 1.4715 for the exponential
   distribution.)

   Nevertheless, we use more sophisticated method that construct
   better fitting envelopes, like method AROU (@pxref{AROU}), or even
   avoid the computation of these design constants and thus have
   almost no setup, like method SROU (@pxref{SROU}).

   @subheading The Generalized Ratio-of-Uniforms Method

   The Ratio-of-Uniforms method can be generalized: If a point
   @unurmath{(U,V)} is uniformly distributed in the set

   @unurmathdisplay{A_f= \{(u,v)\colon 0 < v \leq (f(u/v^r+\mu))^{1/(r+1)}\}}

   then @unurmath{X=U/V^r+\mu} has the denity @unurmath{f(x).}
   The minimal bounding rectangle of this region is given by 
   @unurmath{(u^-,u^+)\times (0,v^+)} where
   
   @unurmathdisplay{
   v^+ = \sup\limits_{b_l<x<b_r}         (f(x))^{1/(r+1)}, \\
   u^- = \inf\limits_{b_l<x<b_r} (x-\mu) (f(x))^{r/(r+1)}, \\
   u^+ = \sup\limits_{b_l<x<b_r} (x-\mu) (f(x))^{r/(r+1)}. }

   The above algorithm has then to be adjusted accordingly.
   Notice that the original Ratio-of-Uniforms method is the special
   case with @unurmath{r=1.}

=EON
*/
/* ------------------------------------------------------------------------- */
/*
=NODE  DiscreteInversion  Inversion for Discrete Distributions

=UP RVG [50]

=DESCRIPTION

   We have already presented the idea of the inversion method to
   generate from continuous random variables (@pxref{Inversion}). For
   a discrete random variable @unurmath{X} we can write it
   mathematically in the same way:

   @unurmathdisplay{X=F^{-1}(U),}

   where @unurmath{F} is the CDF of the desired distribution and 
   @unurmath{U} is a uniform @unurmath{U(0,1)} random number. The
   difference compared to the continuous case is that @unurmath{F} is
   now a step-function. The following figure illustrates the idea of
   discrete inversion for a simple distribution. 

   @unurimage{figures/discrete_inversion}

   To realize this idea on a computer we have to use a search
   algorithm. For the simplest version called @emph{Sequential Search}
   the CDF is computed on-the-fly as sum of the probabilities
   @unurmath{p(k),} since this is usually much cheaper than computing
   the CDF directly. It is obvious that the basic form of the search
   algorithm only works for discrete random variables with probability
   mass functions @unurmath{p(k)} for nonnegative @unurmath{k.} The
   sequential search algorithm consists of the following basic steps:

   @enumerate
   @item
   Generate a @unurmath{U(0,1)} random number @unurmath{U.}
   @item
   Set @unurmath{X} to @unurmath{0} and @unurmath{P} to
   @unurmath{p(0).}
   @item
   Do while @unurmath{U > P} 
   @item
   @w{ } @w{ } @w{ } Set @unurmath{X} to @unurmath{X+1} and 
   @unurmath{P} to @unurmath{P+p(X).}
   @item
   Return @unurmath{X.}
   @end enumerate

   With the exception of some very simple discrete distributions,
   sequential search algorithms become very slow as the while-loop has
   to be repeated very often. The expected number of iterations,
   i.e., the number of comparisons in the while condition, is equal to
   the expectation of the distribution plus @unurmath{1.}
   It can therefore become arbitrary large or even infinity if the tail
   of the distribution is very heavy. Another serious problem can be
   critical round-off errors due to summing up many probabilities
   @unurmath{p(k).} To speed up the search procedure it is best to use
   indexed search.

=EON
*/
/* ------------------------------------------------------------------------- */
/*
=NODE  IndexedSearch  Indexed Search (Guide Table Method)

=UP RVG [60]

=DESCRIPTION

   The idea to speed up the sequential search algorithm is easy to
   understand. Instead of starting always at @unurmath{0} we store a
   table of size @unurmath{C} with starting points for our search. For
   this table we compute @unurmath{F^{-1}(U)} for @unurmath{C}
   equidistributed values of @unurmath{U,} i.e., 
   for @unurmath{u_i = i/C,} @unurmath{i=0,...,C-1.} Such a table is
   called @emph{guide table} or @emph{hash table}.
   Then it is easy to prove that for every @unurmath{U} in
   @unurmath{(0,1)} the guide table entry for @unurmath{k=floor(UC)}
   is bounded by @unurmath{F^{-1}(U).} This shows that we can really
   start our sequential search procedure from the table entry for
   @unurmath{k} and the index @unurmath{k} of the correct table entry
   can be found rapidly by means of the truncation operation.

   The two main differences between @emph{indexed search} and 
   @emph{sequential search} are that we start searching at the number
   determined by the guide table, and that we have to compute and
   store the cumulative probabilities in the setup as we have to know
   the cumulative probability for the starting point of the search
   algorithm. The rounding problems that can occur in the sequential
   search algorithm can occur here as well. 
   Compared to sequential search we have now the obvious drawback
   of a slow setup. The computation of the cumulative probabilities
   grows linear with the size of the domain of the distribution
   @unurmath{L.} What we gain is really high speed as the marginal
   execution time of the sampling algorithm becomes very small. The
   expected number of comparisons is bounded by @unurmath{1+L/C.}
   This shows that there is a trade-off between speed and
   the size of the guide table. Cache-effects in modern computers will
   however slow down the speed-up for really large table sizes. 
   Thus we recommend to use a guide table that is about two times
   larger than the probability vector to obtain optimal speed. 

=EON
*/
/* ------------------------------------------------------------------------- */

