ON THE ESTIMATION OF MULTIPLE RANDOM INTEGRALS AND U-STATISTICS

Péter Major Alfréd Rényi Mathematical Institute of the Hungarian Academy of Sciences

1. Introduction.

First I briefly describe the main subject of this work. Fix a positive integer n, consider n independent and identically distributed random variables ξ_{1}, …, ξ_{n} on a measurable space (X, X) with some distribution μ and take their empirical distribution μ_{n} together with its normalization \sqrt{n}(μ_{n} − μ). Beside this, take a function f(x_{1}, …, x_{k}) of k variables on the k-fold product (X^{k}, X^{k}) of the space (X, X), introduce the k-th power of the normalized empirical measure \sqrt{n}(μ_{n} −μ) on (X^{k}, X^{k}), and define the integral of the function f with respect to this signed product measure. This integral is a random variable, and we want to give a good estimate on its tail distribution. More precisely, we take the integrals not on the whole space, the diagonals x_{s} = x_{s′}, 1 ≤ s, s′ ≤ k, s ≠ s′, of the space X^{k} are omitted from the domain of integration. Such a modification of the integral seems to be natural.

We shall also be interested in the following generalized version of the above problem. Let us have a nice class of functions F of k variables on the product space (X^{k}, X^{k}), and consider the integrals of all functions in this class with respect to the k-fold direct product of our normalized empirical measure. Give a good estimate on the tail distribution of the supremum of these integrals.

It may be asked why the above problems deserve a closer study. I found them important, because they may help in solving some essential problems in probability theory and mathematical statistics. I met such problems when I tried to adapt the method of proof about the Gaussian limit behaviour of the maximum likelihood estimate to some similar but more difficult questions. In the original problem the asymptotic behaviour of the solution of the so-called maximum likelihood equation has to be investigated. The study of this problem is hard in its original form. But by applying an appropriate Taylor expansion of the function that appears in this equation and throwing away its higher order terms we get an approximation whose behaviour can be simply understood. So to describe the limit behaviour of the maximum likelihood estimate it suffices to show that this approximation causes only a negligible error.

One would try to apply a similar procedure in more difficult situations. I met some non-parametric maximum likelihood problems, for instance the description of the limit behaviour of the so-called Kaplan–Meyer product limit estimate when such an approach could be applied. But in these problems it was harder to justify that the simplifying approximation causes only a negligible error. To show this, the solution of the above mentioned problems was needed. In the non-parametric maximum likelihood estimate problems I met, the estimation of multiple (random) integrals played a role similar to the estimation of the coefficients in the Taylor expansion in the study of maximum likelihood estimates. Although I could apply this approach only in some special cases, I believe that it works in very general situations. But it demands some further work to show this.

The above formulated problems about random integrals are interesting and nontrivial even in the special case k = 1. Their solution leads to some interesting and non-trivial generalization of the fundamental theorem of the mathematical statistics about the difference of the empirical and real distribution of a large sample.

These problems have a natural counterpart about the behaviour of so-called Ustatistics, a fairly popular subject in probability theory. The investigation of multiple random integrals and U-statistics are closely related, and it turned out that it is useful to consider them simultaneously.

Let us try to get some feeling about what kind of results can be expected in these problems. For a large sample size n the normalized empirical measure \sqrt{n}(μ_{n} − μ) behaves similarly to a Gaussian random measure. This suggests that in the problems we are interested in similar results should hold as in the problems about multiple Gaussian integrals, called Wiener–Itô integrals in the literature. We may expect that the tail behaviour of the distribution of a k-fold random integral with respect to a normalized empirical measure is similar to that of the k-th power of a Gaussian random variable with expectation zero and an appropriate variance. Beside this, a similar estimate should hold for the supremum of multiple random integrals of a class of functions with respect to a normalized empirical measure or with respect to a Gaussian random measure under not too restrictive conditions. We may also hope that the methods of the theory of multiple Gaussian integrals can be adapted to the investigation of our problems.

The above consideration supplies a fairly good description of the situation, but it does not take into account a very essential difference between the behaviour of multiple Gaussian integrals and multiple integrals with respect to a normalized empirical measure. If the variance of a multiple integral with respect to a normalized empirical measure is very small, what turns out to be equivalent to a very small L_{2}-norm of the function we are integrating, then the behaviour of this integral is different from that of a multiple Gaussian integral with the same kernel function. In this case the effect of some irregularities of the normalized empirical distribution turns out to be non-negligible, and no good Gaussian approximation holds any longer. This case must be better understood, and some new methods have to be worked out to handle it.

The precise formulation of the results will be given in the main part of the work. Beside their proof I also tried to explain the main ideas behind them and the notions introduced in their investigation. This work contains some new results, and also the proof of some already rather classical theorems is presented. The results about Gaussian random variables and their non-linear functionals, in particular multiple integrals with respect to a Gaussian field, have a most important role in the study of the present work. Hence they will be discussed in detail together with some counterparts about multiple random integrals with respect to a normalized empirical measure and some results about U-statistics.

The proofs apply results from different parts of the probability theory. Papers investigating similar results refer to works dealing with quite different subjects, and this makes their reading rather hard. To overcome this difficulty I tried to work out the details and to present a self-contained discussion even at the price of a longer text. Thus I wrote down (in the main text or in the Appendix) the proof of many interesting and basic results, like results about Vapnik–Červonenkis classes, about U-statistics and their decomposition to sums of so-called degenerate U-statistics, about so-called decoupled U-statistics and their relation to ordinary U-statistics, the diagram formula about the product of Wiener–Itô integrals, their counterpart about the product of degenerate Ustatistics, etc. I tried to give such an exposition where different parts of the problem are explained independently of each other, and they can be understood in themselves.

An earlier version of this work was explained at the probability seminar of the University Debrecen (Hungary).

2. Motivation of the investigation. Discussion of some problems.

Here I try to show by means of some examples why the solution of the problems mentioned in the introduction may be useful in the study of some important problems of the probability theory. I try to give a good picture about the main ideas, but I do not work out all details. Actually, the elaboration of some details omitted would demand hard work. But as the discussion of this section is quite independent of the rest of the paper, these omissions cause no problem in understanding the subsequent part.

I start with a short discussion of the maximum likelihood estimate in the simplest case. The following problem is considered. Let us have a class of density functions f(x, ϑ) on the real line depending on a parameter ϑ ∈ R^{1}, and observe a sequence of independent random variables ξ_{1}(ω), …, ξ_{n}(ω) with a density function f(x, ϑ_{0}), where ϑ_{0} is an unknown parameter we want to estimate with the help of the above sequence of random variables.

The maximum likelihood method suggests the following approach. Choose that value ϑ̂_{n} = ϑ̂_{n}(ξ_{1}, …, ξ_{n}) as the estimate of the parameter ϑ_{0} where the density function of the random vector (ξ_{1}, …, ξ_{n}), i.e. the product

∏_{k=1}^{n} f(ξ_{k}, ϑ) = exp {∑_{k=1}^{n} log f(ξ_{k}, ϑ)}

takes its maximum. This point can be found as the solution of the so-called maximum likelihood equation

∑_{k=1}^{n} \frac{∂}{∂ϑ} log f(ξ_{k}, ϑ) = 0.  (2.1)

We are interested in the asymptotic behaviour of the random variable ϑ̂_{n} − ϑ_{0}, where ϑ̂_{n} is the (appropriate) solution of the equation (2.1).

The direct study of this equation is rather hard, but a Taylor expansion of the expression at the left-hand side of (2.1) around the (unknown) point ϑ_{0} yields a good and simple approximation of ϑ̂_{n}, and it enables us to describe the asymptotic behaviour of ϑ̂_{n} − ϑ_{0}.

This Taylor expansion yields that

∑_{k=1}^{n} \frac{∂}{∂ϑ} log f(ξ_{k}, ϑ̂_{n}) = ∑_{k=1}^{n} \frac{\frac{∂}{∂ϑ}f(ξ_{k}, ϑ_{0})}{f(ξ_{k}, ϑ_{0})}

+ (ϑ̂_{n} − ϑ_{0}) (∑_{k=1}^{n} ( \frac{\frac{∂^{2}}{∂ϑ^{2}} f(ξ_{k}, ϑ_{0})}{f(ξ_{k}, ϑ_{0})} − \frac{(\frac{∂}{∂ϑ}f(ξ_{k}, ϑ_{0}))^{2}}{f^{2}(ξ_{k}, ϑ̄_{0})} )) + O (n(ϑ̂_{n} − ϑ_{0})^{2})

=  ∑_{k=1}^{n} (η_{k} + ζ_{k}(ϑ̂_{n} − ϑ_{0})) + O (n(ϑ̂_{n} − ϑ_{0})^{2}) ,  (2.2)

where

η_{k} = \frac{\frac{∂}{∂ϑ}f(ξ_{k}, ϑ_{0})}{f(ξ_{k}, ϑ_{0})} and ζ_{k} = \frac{\frac{∂^{2}}{∂ϑ^{2}} f(ξ_{k}, ϑ_{0})}{f(ξ_{k}, ϑ_{0})} − \frac{(\frac{∂}{∂ϑ}f(ξ_{k}, ϑ_{0}))^{2}}{f^{2}(ξ_{k}, ϑ̄_{0})}

for k = 1, …, n. We want to understand the asymptotic behaviour of the (random) expression on the right-hand side of (2.2). The relation

Eη_{k} = ∫ \frac{\frac{∂}{∂ϑ}f(x, ϑ_{0})}{f(x, ϑ_{0})}f(x, ϑ_{0}) dx = \frac{∂}{∂ϑ} ∫ f(x, ϑ_{0}) dx = 0

holds, since ∫ f(x, ϑ) dx = 1 for all ϑ, and a differentiation of this relation gives the last

identity. Similarly, Eη_{k}^{2} = −Eζ_{k} = ∫ \frac{(\frac{∂}{∂ϑ} f(x,ϑ_{0}))^{2}}{f(x,ϑ_{0})} dx > 0, k = 1, …, n. Hence by the

central limit theorem χ_{n} = \frac{1}{\sqrt{n}} ∑_{k=1}^{n} η_{k} is asymptotically normal with expectation zero

and variance I^{2} = ∫ \frac{(\frac{∂}{∂ϑ} f(x,ϑ_{0}))^{2}}{f(x,ϑ_{0})} dx > 0. In the statistics literature this number I is

called the Fisher information. By the laws of large numbers \frac{1}{n} ∑_{k=1}^{n} ζ_{k} ∼ −I^{2}.

Thus relation (2.2) suggests the approximation ϑ̃_{n} = −\frac{∑_{k=1}^{n} η_{k}}{∑_{k=1}^{n} ζ_{k}} of the maximum-

likelihood estimate ϑ̂_{n}, and \sqrt{n}(ϑ̃_{n} − ϑ_{0}) is asymptotically normal with expectation zero and variance \frac{1}{I^{2}} . The random variable ϑ̃_{n} is not a solution of the equation (2.1), the value of the expression at the left-hand side is of order O(n(ϑ̃_{n} − ϑ_{0})^{2}) = O(1) in this point. On the other hand, the derivative of the function at the left-hand side is large in this point, it is greater than const. n with some const. > 0. This implies that the maximum-likelihood equation has a solution ϑ̂_{n} such that ϑ̂_{n} − ϑ̃_{n} = O (\frac{1}{n}). Hence \sqrt{n}(ϑ̂_{n} − ϑ_{0}) and \sqrt{n}(ϑ̃_{n} − ϑ_{0}) have the same asymptotic limit behaviour.

The previous method can be summarized in the following way: Take a simpler linearized version of the expression we want to estimate by means of an appropriate Taylor expansion, describe the limit distribution of this linearized version and show that the linearization causes only a negligible error.

We want to show that such a method also works in more difficult situations. But in some cases it is harder to show that the error committed by a replacement of the original expression by a simpler linearized version is negligible, and to show this the solution of the problems mentioned in the introduction is needed. The discussion of the following problem, called the Kaplan–Meyer method for the estimation of the empirical distribution function with the help of censored data shows such an example.

The following problem is considered. Let (X_{i}, Z_{i}), i = 1, …, n, be a sequence of independent, identically distributed random vectors such that the components X_{i} and Z_{i} are also independent with some unknown distribution functions F (x) and G(x). We want to estimate the distribution function F of the random variables X_{i}, but we cannot observe the variables X_{i}, only the random variables Y_{i} = min(X_{i}, Z_{i}) and δ_{i} = I(X_{i} ≤ Z_{i}). In other words, we want to solve the following problem. There are certain objects whose lifetime X_{i} are independent and F distributed. But we cannot observe this lifetime X_{i}, because after a time Z_{i} the observation must be stopped. We also know whether the real lifetime X_{i} or the censoring variable Z_{i} was observed. We make n independent experiments and want to estimate with their help the distribution function F.

Kaplan and Meyer, on the basis of some maximum-likelihood estimation type considerations, proposed the following so-called product limit estimator S_{n}(u) to estimate the unknown survival function S(u) = 1 − F (u):

1 − F_{n}(u) = S_{n}(u) = \{ \begin{matrix}∏_{i=1}^{n} (\frac{N(Y_{i})}{N(Y_{i}) + 1})^{I(Y_{i}≤u,δ_{i}=1)} & if u ≤ max(Y_{1}, …, Y_{n}) \\ 0 & if u ≥ max(Y_{1}, …, Y_{n}), δ_{n} = 1, \\ undefined & if u ≥ max(Y_{1}, …, Y_{n}), δ_{n} = 0, \end{matrix}

(2.3)

where

N(t) = #{Y_{i}, Y_{i} > t, 1 ≤ i ≤ n} = ∑_{i=1}^{n} I(Y_{i} > t).

We want to show that the above estimate (2.3) is really good. For this goal we shall approximate the random variables S_{n}(u) by some appropriate random variables. To do this first we introduce some notations.

Put

\begin{aligned}H(u) & = P (Y_{i} ≤ u) = 1 − H̄(u), \\ H̃(u) & = P (Y_{i} ≤ u, δ_{i} = 1), ˜ H̃(u) = P (Y_{i} ≤ u, δ_{i} = 0) \end{aligned}  (2.4)

and

\begin{aligned}H_{n}(u) & = \frac{1}{n} ∑_{i=1}^{n} I(Y_{i} ≤ u) \\ H̃_{n}(u) & = \frac{1}{n} ∑_{i=1}^{n} I(Y_{i} ≤ u, δ_{i} = 1), ˜ H̃_{n}(u) = \frac{1}{n} ∑_{i=1}^{n} I(Y_{i} ≤ u, δ_{i} = 0). \end{aligned}  (2.5)

Clearly H(u) = H̃(u) + ˜ H̃(u) and H_{n}(u) = H̃_{n}(u) + ˜ H̃_{n}(u). We shall estimate F_{n}(u) − F (u) for u ∈ (−∞, T] if

1 − H(T) > δ with some fixed δ > 0.  (2.6)

Condition (2.6) implies that there are more than \frac{δ}{2}n sample points Y_{j} larger than T with probability almost 1. The complementary event has only an exponentially small probability. This observation helps to show in the subsequent calculations that some events have negligibly small probability.

We introduce the so-called cumulative hazard function and its empirical version

Λ(u) = − log(1 − F (u)), Λ_{n}(u) = − log(1 − F_{n}(u)).  (2.7)

Since F_{n}(u) − F (u) = exp(−Λ(u)) (1 − exp(Λ(u) − Λ_{n}(u))) a simple Taylor expansion yields

F_{n}(u) − F (u) = (1 − F (u)) (Λ_{n}(u) − Λ(u)) + R_{1}(u),  (2.8)

and it is easy to see that R_{1}(u) = O (Λ(u) − Λ_{n}(u))^{2}). It follows from the subsequent estimations that Λ(u) − Λ_{n}(u) = O(n^{−1/2}), thus nR_{1}(u) = O(1). Hence it is enough to investigate the term Λ_{n}(u). We shall show that Λ_{n}(u) has an expansion with Λ(u) as the main term plus n^{−1/2} times a term which is a linear functional of an appropriate normalized empirical distribution function plus an error term of order O(n^{−1}).

From (2.3) it is obvious that

Λ_{n}(u) = − ∑_{i=1}^{n} I(Y_{i} ≤ u, δ_{i} = 1) log (1 − \frac{1}{1 + N(Y_{i})}) .

It is not difficult to get rid of the unpleasant logarithmic function in this formula by means of the relation − log(1 − x) = x + O(x^{2}) for small x. It yields that

Λ_{n}(u) = ∑_{i=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)}{N(Y_{i})} + R_{2}(u) = Λ̃_{n}(u) + R_{2}(u),  (2.9)

and the error term nR_{2}(u) is exponentially small.

The expression Λ̃_{n}(u) is still inappropriate for our purposes. Since the denominators

N(Y_{i}) = ∑_{j=1}^{n} I(Y_{j} > Y_{i}) are dependent for different indices i we cannot see directly the

limit behaviour of Λ̃_{n}(u).

We try to approximate Λ̃_{n}(u) by a simpler expression. A natural approach would be to approximate the terms N(Y_{i}) in it by their conditional expectation (n−1)H̄(Y_{i}) = (n − 1)(1 − H(Y_{i})) = E(N(Y_{i})|Y_{i}). This is a too rough ‘first order’ approximation, but the following ‘second order approximation’ will be sufficient for our goals. Put

N(Y_{i}) = ∑_{j=1}^{n} I(Y_{j} > Y_{i}) = nH̄(Y_{i}) (1 + \frac{∑_{j=1}^{n} I(Y_{j} > Y_{i}) − nH̄(Y_{i})}{nH̄(Y_{i})})

and express the terms \frac{1}{N(Y_{i})} in the sum defining Λ̃_{n} by means of the relation \frac{1}{1+z} =

∑_{k=0}^{∞}(−1)^{k}z^{k} = 1 − z + ε(z) with the choice z = ^{\frac{∑_{j=1}^{n} I(Y_{j}>Y_{i})−nH̄(Y_{i})}{nH̄(Y_{i})}}. As |ε(z)| < 2z^{2} for

|z| < \frac{1}{2} we get that

Λ̃_{n}(u) = ∑_{i=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)}{nH̄(Y_{i})} (1 + ∑_{k=1}^{∞} (−\frac{∑_{j=1}^{n} I(Y_{j} > Y_{i}) − nH̄(Y_{i})}{nH̄(Y_{i})})^{k})

= ∑_{i=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)}{nH̄(Y_{i})} (1 − \frac{∑_{j=1}^{n} I(Y_{j} > Y_{i}) − nH̄(Y_{i})}{nH̄(Y_{i})}) + R_{3}(u) (2.10)

= 2A(u) − B(u) + R_{3}(u),

where

A(u) = A(n, u) = ∑_{i=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)}{nH̄(Y_{i})}

and

B(u) = B(n, u) = ∑_{i=1}^{n} ∑_{j=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)I(Y_{j} > Y_{i})}{n^{2}H̄^{2}(Y_{i})}.

It can be proved by means of standard methods that nR_{3}(u) is exponentially small. Thus relations (2.9) and (2.10) yield that

Λ_{n}(u) = 2A(u) − B(u) + negligible error.  (2.11)

This means that to solve our problem the asymptotic behaviour of the random variables A(u) and B(u) has to be given. We can get a better insight to this problem by rewriting the sum A(u) as an integral and the double sum B(u) as a two-fold integral with respect to empirical measures. Then these integrals can be rewritten as sums of random integrals with respect to normalized empirical measures and deterministic measures. Such an approach yields a representation of Λ_{n}(u) in the form of a sum whose terms can be well understood.

Let us write

A(u) = ∫_{−∞}^{+∞} \frac{I(y ≤ u)}{1 − H(y)} dH̃_{n}(y),
B(u) = ∫_{−∞}^{+∞} ∫_{−∞}^{+∞} \frac{I(y ≤ u)I(x > y)}{(1 − H(y))^{2}} dH_{n}(x)dH̃_{n}(y).

To rewrite the term B(u) in a form better for our purposes observe that

H_{n}(x)H̃_{n}(y) = H(x)H̃(y) + H(x)(H̃_{n}(y) − H̃(y)) + (H_{n}(x) − H(x))H̃(y)
+ (H_{n}(x) − H(x))(H̃_{n}(y) − H̃(y)).

Hence it can be written in the form B(u) = B_{1}(u) + B_{2}(u) + B_{3}(u) + B_{4}(u), where

B_{1}(u) = ∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} dH(x) dH̃(y) ,

B_{2}(u) = \frac{1}{\sqrt{n}} ∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} dH(x) d (\sqrt{n}(H̃_{n}(y) − H̃(y))) ,

B_{3}(u) = \frac{1}{\sqrt{n}} ∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} d (\sqrt{n} (H_{n}(x) − H(x))) dH̃(y) ,

B_{4}(u) = \frac{1}{n} ∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} d (\sqrt{n} (H_{n}(x) − H(x))) d (\sqrt{n}(H̃_{n}(y) − H̃(y))) .

In the above decomposition of B(u) the term B_{1} is a deterministic function, B_{2}, B_{3} are linear functionals of normalized empirical processes and B_{4} is a nonlinear functional of normalized empirical processes. The deterministic term B_{1}(u) can be calculated explicitly. Indeed,

B_{1}(u) = ∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} dH(x)dH̃(y) = ∫_{−∞}^{u} \frac{dH̃(y)}{1 − H(y)}.

Then the relations H̃(u) = ∫_{−∞}^{u} (1 − G(t)) dF (t) and 1 − H = (1 − F)(1 − G) imply that

B_{1}(u) = ∫_{−∞}^{u} \frac{dF (y)}{1 − F (y)} = − log(1 − F (u)) = Λ(u).  (2.12)

Observe that

\begin{aligned}A(u) & = ∫_{−∞}^{u} \frac{d H̃_{n}(y)}{1 − H(y)} \\ & = ∫_{−∞}^{u} \frac{dH̃(y)}{1 − H(y)} + \frac{1}{\sqrt{n}} ∫_{−∞}^{u} \frac{d (\sqrt{n}(H̃_{n}(y) − H̃(y)))}{1 − H(y)} \\ & = B_{1}(u) + B_{2}(u). \end{aligned}  (2.13)

From relations (2.11), (2.12) and (2.13) it follows that

Λ_{n}(u) − Λ(u) = B_{2}(u) − B_{3}(u) − B_{4}(u) + negligible error.  (2.14)

Integration of B_{2} and B_{3} with respect to the variable x and then integration by parts in the expression B_{2} yields that

B_{2}(u) = \frac{1}{\sqrt{n}} ∫_{−∞}^{u} \frac{d (\sqrt{n}(H̃_{n}(y) − H̃(y)))}{1 − H(y)}

= \frac{\sqrt{n} (H̃_{n}(u) − H̃(u))}{\sqrt{n}(1 − H(u))} − \frac{1}{\sqrt{n}} ∫_{−∞}^{u} \frac{\sqrt{n}(H̃_{n}(y) − H̃(y))}{(1 − H(y))^{2}} dH(y)

B_{3}(u) = \frac{1}{\sqrt{n}} ∫_{−∞}^{u} \begin{matrix}\sqrt{n} & (H(y) − H_{n}(y)) \\ (1 − H(y)) & 2 \end{matrix} dH̃(y).

With the help of the above expressions for B_{2} and B_{3}, (2.14) can be rewritten as

\sqrt{n} (Λ_{n}(u) − Λ(u)) = \frac{\sqrt{n} (H̃_{n}(u) − H̃(u))}{1 − H(u)} − ∫_{−∞}^{u} \frac{\sqrt{n}(H̃_{n}(y) − H̃(y))}{(1 − H(y))^{2}} dH(y)

+ ∫_{−∞}^{u} \begin{matrix}\sqrt{n} & (H_{n}(y) − H(y)) \\ (1 − H(y)) & 2 \end{matrix} dH̃(y)

− \sqrt{n}B_{4}(u) + negligible error.

(2.15)

Formula (2.15) almost agrees with the statement we wanted to prove. Here the normalized error \sqrt{n} (Λ_{n}(u) − Λ(u)) is expressed as a sum of linear functionals of normalized empirical measures plus some negligible error terms plus the error term \sqrt{n}B_{4}(u). So to get a complete proof it is enough to show that \sqrt{n}B_{4}(u) also yields a negligible error. But B_{4}(u) is a double integral of a bounded function (here we apply again formula (2.6)) with respect to a normalized empirical measure. Hence to bound this term we need a good estimate of multiple stochastic integrals (with multiplicity 2), and this is just the problem formulated in the introduction. The estimate we need here follows from Theorem 8.1 of the present work. Let us remark that the problem discussed here corresponds to the estimation of the coefficient of the second term in the Taylor expansion considered in the study of the maximum likelihood estimation. One may worry a little bit how to bound B_{4}(u) with the help of estimations of double stochastic integrals, since in the definition of B_{4}(u) integration is taken with respect to different normalized empirical processes in the two coordinates. But this is a not too difficult technical problem. It can be simply overcome for instance by rewriting the integral as a double integral with respect to the empirical process (\sqrt{n} (H_{n}(x) − H(x)) , \sqrt{n} (H̃_{n}(y) − H̃(y))) in the space

R^{2}.

By working out the details of the above calculation we get that the linear functional B_{2}(u)−B_{3}(u) of normalized empirical processes yields a good estimate on the expression \sqrt{n}(Λ_{n}(u) − Λ(u)) for a fixed parameter u. But we want to prove somewhat more, we want to get an estimate uniform in the parameter u, i.e. to show that even the random variable sup_{u≤T} |\sqrt{n}(Λ_{n}(u) − Λ(u)) − B_{2}(u) + B_{3}(u)| is small. This can be done by making estimates uniform in the parameter u in all steps of the above calculation. There appears only one difficulty when trying to carry out this program. Namely, we need an estimate on sup_{u} |B_{4}(u)|, i.e. we have to bound the supremum of multiple random integrals with

respect to a normalized random measure for a nice class of kernel functions. This can be done, but at this point the second problem mentioned in the introduction appears. This difficulty can be overcome by means of Theorem 8.2 of this work.

Thus the limit behaviour of the Kaplan–Meyer estimate can be described by means of an appropriate expansion. The steps of the calculation leading to such an expansion are fairly standard, the only hard part is the solution of the problems mentioned in the introduction. It can be expected that such a method also works in a much more general situation.

I finish this section with a remark of Richard Gill he made in a personal conversation after my talk on this subject at a conference. He told that this approach had given a complete proof about the limit behaviour of this estimate, but it had exploited the explicit formula given in the Kaplan–Meyer estimate. He missed the application of an argument based on the non-parametric maximum likelihood character of this estimate. This was a completely justified remark, since if we do not restrict our attention to this problem, but try to generalize it to general non-parametric maximum likelihood estimates, then we have to understand how the maximum likelihood character can be exploited. I believe that this can be done, but it demands further studies.

3. Some estimates about sums of independent random variables.

We need some results about the distribution of sums of independent random variables bounded by a constant with probability one. Later only the results about sums of independent and identically distributed variables will be interesting for us. But since they can be generalized without any effort to sums of not necessarily identically distributed random variables the condition about identical distribution of the summands will be dropped. We are interested in the question when these estimates give such a good bound as the central limit theorem suggests, and what can be told otherwise.

More explicitly, the following problem will be considered: Let X_{1}, …, X_{n} be independent random variables, EX_{j} = 0, Var X_{j} = σ_{j}^{2}, 1 ≤ j ≤ n, and take the random

sum S_{n} = ∑_{j=1}^{n} X_{j} and its variance Var S_{n} = V_{n}^{2} = ∑_{j=1}^{n} σ_{j}^{2}. We want to get a good bound

on the probability P (S_{n} > uV_{n}). The central limit theorem suggests that under general conditions an upper bound of the order 1 − Φ(u) should hold for this probability, where Φ(u) denotes the standard normal distribution function. Since the standard normal dis-

tribution function satisfies the inequality (\frac{1}{u} − \frac{1}{u^{3}} ) \frac{e^{−u^{2}/2}}{\sqrt{2π}} < 1 − Φ(u) < \frac{1}{u} \frac{e^{−u^{2}/2}}{\sqrt{2π}} for all u > 0 it is natural to ask when the probability P (S_{n} > uV_{n}) is comparable with the value e^{−u^{2}/2}. More generally, we shall call an upper bound of the form P (S_{n} > uV_{n}) ≤ e^{−Cu^{2}} with some constant C > 0 a Gaussian type estimate.

First I formulate Bernstein’s inequality which tells for which values u the probability P (S_{n} > uV_{n}) has a Gaussian type estimate. It supplies such an estimate if u ≤ const. V_{n}.

On the other hand, for u ≥ const. V_{n} it yields a much weaker estimate. I also present an example which shows that in this case only a very weak improvement of Bernstein’s inequality is possible. I also discuss another result, called Bennett’s inequality, which shows that such an improvement is possible. The main difficulties we meet in this work are closely related to the weakness of the estimates we have for the probability of the event P (S_{n} > uV_{n}) if u ≫ const. V_{n}.

In the usual formulation of Bernstein’s inequality a real number M is introduced, and it is assumed that the terms in the sum we investigate are bounded by this number. But since the problem can be simply reduced to the special case M = 1 I shall consider only this special case.

Theorem 3.1. (Bernstein’s inequality). Let X_{1}, …, X_{n} be independent random variables, P (|X_{j}| ≤ 1) = 1, EX_{j} = 0, 1 ≤ j ≤ n. Put σ_{j}^{2} = EX_{j}^{2}, 1 ≤ j ≤ n,

S_{n} = ∑_{j=1}^{n} X_{j} and V_{n}^{2} = Var S_{n} = ∑_{j=1}^{n} σ_{j}^{2}. Then

P (S_{n} > uV_{n}) ≤ exp \{−\frac{u^{2}}{2 (1 + \frac{1}{3} \frac{u}{V_{n}} )}\} for all u > 0.  (3.1)

Proof of Theorem 3.1. Let us give a good bound on the exponential moments Ee^{tS_{n}} for appropriate parameters t > 0. Since EX_{j} = 0 and E|X_{j}^{k+2}| ≤ σ^{2} for k ≥ 0 we can

write Ee^{tX_{j}} = ∑_{k=0}^{∞} \frac{t^{k}}{k!} EX_{j}^{k} ≤ 1 + \frac{t^{2}σ_{j}^{2}}{2} (1 + ∑_{k=1}^{∞} \frac{2t^{k}}{(k+2)!}) ≤ 1 + \frac{t^{2}σ_{j}^{2}}{2} (1 + ∑_{k=1}^{∞} 3^{−k}t^{k}) =

1+ \frac{t^{2}σ_{j}^{2}}{2} \frac{1}{1− \frac{t}{3}} ≤ exp \{\frac{t^{2}σ_{j}^{2}}{2} \frac{1}{1− \frac{t}{3}} \} if 0 ≤ t < 3. Hence Ee^{tS_{n}} = ∏_{j=1}^{n} Ee^{tX_{j}} ≤ exp \{\frac{t^{2}V_{n}^{2}}{2} \frac{1}{1− \frac{t}{3}} \}

for 0 ≤ t < 3.

The above relation implies that

P (S_{n} > uV_{n}) = P (e^{tS_{n}} > e^{tuV_{n}}) ≤ Ee^{tS_{n}}e^{−tuV_{n}} ≤ exp \{\frac{t^{2}V_{n}^{2}}{2} \frac{1}{1 − \frac{t}{3}} − tuV_{n}\}

if 0 ≤ t < 3. Choose the number t in this inequality as the solution of the equation t^{2}V_{n}^{2} \frac{1}{1− \frac{t}{3}} = tuV_{n}, i.e. put t = \frac{u}{V_{n}+ \frac{u}{3}} . Then 0 ≤ t < 3, and we get that P (S_{n} > uV_{n}) ≤

e^{−tuV_{n}/2} = exp \{−\frac{u^{2}}{2(1+ \frac{1}{3} \frac{u}{Vn})}\}.

If the random variables X_{1}, …, X_{n} satisfy the conditions of Bernstein’s inequality, then also the random variables −X_{1}, …, −X_{n} satisfy them. By applying the above

result in both cases we get that P (|S_{n}| > uV_{n}) ≤ 2 exp \{−\frac{u^{2}}{2(1+ \frac{1}{3} \frac{u}{Vn})}\} under the condi-

tions of Bernstein’s inequality.

By Bernstein’s inequality for all ε > 0 there is some number α(ε) > 0 such that in the case \frac{u}{V_{n}} < α(ε) P (S_{n} > uV_{n}) ≤ e^{−(1−ε)u^{2}/2}. Beside this, for all fixed numbers A > 0 there is some constant C = C(A) > 0 such that in the case \frac{u}{V_{n}} < A the inequality P (S_{n} > uV_{n}) ≤ e^{−Cu^{2}} holds. This can be interpreted as a Gaussian type estimate for the probability P (S_{n} > uV_{n}) if u ≤ const. V_{n}.

On the other hand, if \frac{u}{V_{n}} is very large, then Bernstein’s inequality yields a much worse estimate. The question arises whether in this case Berstein’s inequality can be replaced by a better, more useful result. Next we present Theorem 3.2, the so-called Bennett’s inequality which provides a slight improvement of Bernstein’s inequality. But if \frac{u}{V_{n}} is very large, then also Bennett’s inequality provides a much weaker estimate on the probability P (S_{n} > uV_{n}) than the bound suggested by a Gaussian comparison. On the other hand, we shall give an example that shows that (without imposing some additional conditions) no real improvement of this estimate is possible.

Theorem 3.2. (Bennett’s inequality). Let X_{1}, …, X_{n} be independent random variables, P (|X_{j}| ≤ 1) = 1, EX_{j} = 0, 1 ≤ j ≤ n. Put σ_{j}^{2} = EX_{j}^{2}, 1 ≤ j ≤ n,

S_{n} = ∑_{j=1}^{n} X_{j} and V_{n}^{2} = Var S_{n} = ∑_{j=1}^{n} σ_{j}^{2}. Then

P (S_{n} > u) ≤ exp \{−V_{n}^{2} [(1 + \frac{u}{V_{n}^{2}} ) log (1 + \frac{u}{V_{n}^{2}} ) − \frac{u}{V_{n}^{2}} ]\} for all u > 0. (3.2)

As a consequence, for all ε > 0 there exists some B = B(ε) > 0 such that

P (S_{n} > u) ≤ exp \{−(1 − ε)u log \frac{u}{V_{n}^{2}} \} if u > BV_{n}^{2},  (3.3)

and there exists some positive constant K > 0 such that

P (S_{n} > u) ≤ exp \{−Ku log \frac{u}{V_{n}^{2}} \} if u > 2V_{n}^{2}.  (3.4)

Proof of Theorem 3.2. We have

Ee^{tX_{j}} = ∑_{k=0}^{∞} \frac{t^{k}}{k!} EX_{j}^{k} ≤ 1 + σ_{j}^{2} ∑_{k=2}^{∞} \frac{t^{k}}{k!} = 1 + σ_{j}^{2} (e^{t} − 1 − t) ≤ e^{σ_{j}^{2}(e^{t}−1−t)}, 1 ≤ j ≤ n,

and Ee^{tS_{n}} ≤ e^{V_{n}^{2}(e^{t}−1−t)} for all t ≥ 0. Hence P (S_{n} > u) ≤ e^{−tu}Ee^{tS_{n}} ≤ e^{−tu+V_{n}^{2}(e^{t}−1−t)} for all t ≥ 0. We get relation (3.2) from this inequality with the choice t = log (1 + \frac{u}{V_{n}^{2}} ).

(This is the place of minimum of the function −tu + V_{n}^{2}(e^{t} − 1 − t) for fixed u in the parameter t.)

Relation (3.2) and the observation lim_{v→∞} \frac{(v+1) log(v+1)−v}{v log v} = 1 with the choice v = \frac{u}{V_{n}^{2}} imply formula (3.3). Because of relation (3.3) to prove formula (3.4) it is enough to check it for 2 ≤ \frac{u}{V_{n}^{2}} ≤ B with some sufficiently large constant B > 0. In this case relation (3.4) follows directly from formula (3.2). This can be seen for instance by observing

that the expression ^{\frac{V_{n}^{2}[(1+_{\frac{u}{V ^{2}_{n}}} ) log(1+_{\frac{u}{V ^{2}_{n}}} )−_{\frac{u}{V ^{2}_{n}}} ]}{u log _{\frac{u}{V ^{2}_{n}}}}} is a continuous and positive function

of the variable \frac{u}{V_{n}^{2}} in the interval 2 ≤ \frac{u}{V_{n}^{2}} ≤ B, hence its minimum in this interval is strictly positive.

Let us make a short comparison between Bernstein’s and Bennett’s inequality. Both results yield an estimate on the probability P (S_{n} > u), and their proofs are very similar. They are based on an estimate of the moment generating functions R_{j}(t) = Ee^{tX_{j}} of the summands X_{j}, but Bennett’s inequality yields a better estimate. It may be worth mentioning that the estimate given for R_{j}(t) = Ee^{tX_{j}} in the proof of Bennett’s inequality agrees with the moment generating function Ee^{t(Y_{j}−EY_{j})} of the normalization Y_{j} −EY_{j} of a Poissonian random variable Y_{j} with parameter Var X_{j}. As a consequence, we get, by using the standard method of estimating tail-distributions by means of the moment generating functions such an estimate for the probability P (S_{n} > u) which is comparable with the probability P (T_{n} − ET_{n} > u), where T_{n} is a Poissonian random variable with parameter V_{n} = Var S_{n}. It can be told that Bernstein’s inequality yields a Gaussian and Bennett’s inequality a Poissonian type estimate for the sums of independent, bounded random variables.

Next we present Example 3.3 which shows that Bennett’s inequality yields a sharp estimate also in the case u ≫ V_{n}^{2} when Bernstein’s inequality yields a weak bound. But Bennett’s inequality provides only a small improvement which has only a limited importance. This may be the reason why Bernstein’s inequality which yields a more transparent estimate is more popular.

Example 3.3. (Sums of independent random variables with bad tail distribution for large values). Let us fix some positive integer n, real numbers u and σ^{2} such that 0 < σ^{2} ≤ \frac{1}{8}, n > 4u ≥ 6 and u > 4nσ^{2}. Let σ̄^{2} be that solution of the equation x^{2} − x + σ^{2} = 0 which is smaller than \frac{1}{2}. Take a sequence of independent and identically distributed random variables X̄_{1}, …, X̄_{n} such that P (X̄_{j} = 1) = σ̄^{2}, P (X̄_{j} = 0) = 1 − σ̄^{2} for all 1 ≤ j ≤ n. Put X_{j} = X̄_{j} − EX̄_{j} = X_{j} − σ̄^{2}, 1 ≤ j ≤ n,

S_{n} = ∑_{j=1}^{n} X_{j} and V_{n}^{2} = nσ^{2}. Then P (|X_{1}| ≤ 1) = 1, EX_{1} = 0, Var X_{1} = σ^{2}, hence

ES_{n} = 0, and Var S_{n} = V_{n}^{2}. Beside this

P (S_{n} ≥ u) > exp \{−Bu log \frac{u}{V_{n}^{2}} \}

with some appropriate constant B > 0 not depending on n, σ and u.

Proof of Example 3.3. Simple calculation shows that EX_{j} = 0, Var X_{j} = σ̄^{2} − σ̄^{4} = σ^{2}, P (|X_{j}| ≤ 1) = 0, and also the inequality σ^{2} ≤ σ̄^{2} ≤ \frac{3}{2}σ^{2} holds. To see the upper bound in the last inequality observe that σ̄^{2} ≤ \frac{1}{3}, i.e. 1−σ̄^{2} ≥ \frac{2}{3}, hence σ^{2} = σ̄^{2}(1−σ̄^{2}) ≥ \frac{2}{3}σ̄^{2}. In the proof of the inequality of Example 3.3 we can restrict our attention to the case when u is an integer, because in the general case we can apply the inequality with ū = [u] + 1 instead of u, where [u] denotes the integer part of u, and since u ≤ ū ≤ 2u, the application of the result in this case supplies the desired inequality with a possibly worse constant B > 0.

Put S̄_{n} = ∑_{j=1}^{n} X̄_{j}. We can write P (S_{n} ≥ u) = P (S̄_{n} ≥ u + nσ̄^{2}) ≥ P (S̄_{n} ≥ 2u) ≥

P (S̄_{n} = 2u) = \binom{n}{2u}σ̄^{4u}(1 − σ̄^{2})^{(n−2u)} ≥ (\frac{nσ̄^{2}}{2u})^{2u}(1 − σ̄^{2})^{(n−2u)}, since u ≥ nσ̄^{2}, and n ≥ 2u. On the other hand (1 − σ̄^{2})^{(n−2u)} ≥ e^{−2σ̄^{2}(n−2u)} ≥ e^{−2nσ̄^{2}} ≥ e^{−u}, hence

P (S_{n} ≥ u) ≥ exp \{−2u log (\frac{u}{nσ̄^{2}} ) − 2u log 2 − u\}

= exp \{−2u log (\frac{u}{nσ^{2}} ) − 2u log \frac{σ̄^{2}}{σ^{2}} − 2u log 2 − u\}

≥ exp \{−100u log (\frac{u}{V_{n}^{2}} )\} .

Example 3.3 is proved.

In the case u > 4V_{n}^{2} Bernstein’s inequality yields the estimate P (S_{n} > u) ≤ e^{−αu} with some universal constant α > 0, and the above example shows that at most an additional logarithmic factor K log \frac{u}{V_{n}^{2}} can be expected in the exponent of the upper bound in an improvement of this estimate. Bennett’s inequality shows that such an improvement is really possible.

I finish this section with another estimate due to Hoeffding which will be later useful in some symmetrization arguments.

Theorem 3.4. (Hoeffding’s inequality). Let ε_{1}, …, ε_{n} be independent random variables, P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, and let a_{1}, …, a_{n} be arbitrary real

numbers. Put V = ∑_{j=1}^{n} a_{j}ε_{j}. Then

P (V > u) ≤ exp \{−\frac{u^{2}}{2 ∑_{j=1}^{n} a_{j}^{2}} \} for all u > 0.  (3.5)

Remark 1: Clearly EV = 0 and Var V = ∑_{j=1}^{n} a_{j}^{2}, hence Hoeffding’s inequality yields

such an estimate for P (V > u) which the central limit theorem suggests. This estimate holds for all real numbers a_{1}, …, a_{n} and u > 0.

Remark 2: The Rademacher functions r_{k}(x), k = 1, 2, …, defined by the formulas r_{k}(x) = 1 if (2j − 1)2^{−k} ≤ x < 2j2^{−k} and r_{k}(x) = −1 if 2(j − 1)2^{−k} ≤ x < (2j − 1)2^{−k}, 1 ≤ j ≤ 2^{k−1}, for all k = 1, 2, …, can be considered as random variables on the probability space Ω = [0, 1] with the Borel σ-algebra and the Lebesgue measure as probability measure on the interval [0, 1]. They are independent random variables with the same distribution as the random variables ε_{1}, …, ε_{n} considered in Theorem 3.4. Therefore results about such sequences of random variables whose distributions agree with those in Theorem 3.4 are also called sometimes results about Rademacher functions in the literature. At some points we will also apply this terminology.

Proof of Theorem 3.4. Let us give a good bound on the exponential moment Ee^{tV} for

all t > 0. The identity Ee^{tV} = ∏_{j=1}^{n} Ee^{ta_{j}ε_{j}} = ∏_{j=1}^{n} \frac{(e^{a_{j} t}+e^{−a_{j} t})}{2} holds, and \frac{(e^{a_{j} t}+e^{−a_{j} t})}{2} =

∑_{k=0}^{∞} \frac{a_{j}^{2k}}{(2k)!}t^{2k} ≤ ∑_{k=0}^{∞} \frac{(a_{j}t)^{2k}}{2^{k}k!} = e^{a_{j}^{2}t^{2}/2}, since (2k)! ≥ 2^{k}k! for all k ≥ 0. This implies

that Ee^{tV} ≤ exp \{\frac{t^{2}}{2} ∑_{j=1}^{n} a_{j}^{2}\}. Hence P (V > u) ≤ exp \{−tu + \frac{t^{2}}{2} ∑_{j=1}^{n} a_{j}^{2}\}, and we get

relation (3.5) with the choice t = u ( ∑_{j=1}^{n} a_{j}^{2})^{−1}.

4. On the supremum of a nice class of partial sums.

This section contains an estimate about the supremum of a nice class of normalized sums of independent and identically distributed random variables together with an analogous result about the supremum of an appropriate class of random one-fold integrals with respect to a normalized empirical measure. The second result deals with a one-variate version of the problem about the estimation of multiple integrals with respect to a normalized empirical measure. Thie problem was mentioned in the introduction. Some natural questions related to these results will be also discussed. It will be examined how restrictive their conditions are. In particular, we are interested in the question how the condition about the countable cardinality of the class of random variables can be weakened. A natural Gaussian counterpart of the supremum problems about random one-fold integrals will be also considered. Most proofs will be postponed to later sections.

To formulate these results first a notion will be introduced that plays a most important role in the sequel.

Definition of L_{p}-dense classes of functions. Let a measurable space (Y, Y) be given together with a set G of Y measurable real valued functions on this space. The class of functions G is called an L_{p}-dense class of functions, 1 ≤ p < ∞, with parameter D and exponent L if for all numbers 0 < ε ≤ 1 and probability measures ν on the space (Y, Y) there exists a finite ε-dense subset G_{ε,ν} = {g_{1}, …, g_{m}} ⊂ G in the space L_{p}(Y, Y, ν) with m ≤ Dε^{−L} elements, i.e. there exists such a set G_{ε,ν} ⊂ G with m ≤ Dε^{−L} elements for which inf_{g_{j}∈G_{ε,ν}} ∫ |g − g_{j}|^{p} dν < ε^{p} for all functions g ∈ G. (Here the set G_{ε,ν} may depend

on the measure ν, but its cardinality is bounded by a number depending only on ε.)

In most results of this work the above defined L_{p}-dense classes will be considered only for the parameter p = 2. But at some points it will be useful to work also with L_{p}- dense classes with a different parameter p. Hence to avoid some repetitions I introduced the above definition for a general parameter p.

The following estimate will be proved.

Theorem 4.1. (Estimate on the supremum of a class of partial sums). Let us consider a sequence of independent and identically distributed random variables ξ_{1}, …, ξ_{n}, n ≥ 2, with values in a measurable space (X, X) and with some distribution μ. Beside this, let a countable and L_{2}-dense class of functions F with some parameter D ≥ 1 and exponent L ≥ 1 be given on the space (X, X) which satisfies the conditions

∥f∥_{∞} = sup_{x∈X} |f(x)| ≤ 1,  for all f ∈ F  (4.1)
∥f∥_{2}^{2} = ∫ f^{2}(x)μ( dx) ≤ σ^{2}  for all f ∈ F  (4.2)

with some constant 0 < σ ≤ 1, and

∫ f(x)μ( dx) = 0 for all f ∈ F.  (4.3)

Define the normalized partial sums S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{k=1}^{n} f(ξ_{k}) for all f ∈ F.

There exist some universal constants C > 0, α > 0 and M > 0 such that the supremum of the normalized random sums S_{n}(f), f ∈ F, satisfies the inequality

P (sup_{f∈F} |S_{n}(f)| ≥ u) ≤ C exp \{−α (\frac{u}{σ} )^{2}\} for those numbers u
for which \sqrt{n}σ^{2} ≥ u ≥ Mσ(L^{3/4} log^{1/2} \frac{2}{σ} + (log D)^{3/4}),

(4.4)

where the numbers D and L in formula (4.4) agree with the parameter and exponent of the L_{2}-dense class F.

The condition \sqrt{n}σ^{2} ≥ u ≥ Mσ(L^{3/4} log^{1/2} \frac{2}{σ} + D^{3/4}) about the number u in formula (4.4) is natural. I discuss this after the formulation of Theorem 4.2 which can be considered as the Gaussian counterpart of Theorem 4.1. I also formulate a result in Example 4.3 which can be considered as part of this discussion.

The condition about the countable cardinality of F can be weakened with the help of the notion of countable approximability introduced below. For the sake of later applications I define it in a more general form than needed in this section.

Definition of countably approximable classes of random variables. Let us have a class of random variables U(f), f ∈ F, indexed by a class of functions f ∈ F on a measurable space (Y, Y). This class of random variables is called countably approximable if there is a countable subset F′ ⊂ F such that for all numbers u > 0 the sets A(u) = {ω: sup_{f∈F} |U(f)(ω)| ≥ u} and B(u) = {ω: sup_{f∈F′} |U(f)(ω)| ≥ u} satisfy the

identity P (A(u) \ B(u)) = 0.

Clearly, B(u) ⊂ A(u). In the above definition it was demanded that for all u > 0 the set B(u) should be almost as large as A(u). The following corollary of Theorem 4.1 holds.

Corollary of Theorem 4.1. Let a class of functions F satisfy the conditions of Theorem 4.1 with the only exception that instead of the condition about the countable cardinality of F it is assumed that the class of random variables S_{n}(f), f ∈ F, is countably approximable. Then the random variables S_{n}(f), f ∈ F, satisfy relation (4.4).

This corollary can be simply proved, only Theorem 4.1 has to be applied for the class F′. To do this it has to be checked that if F is an L_{2}-dense class with some parameter D and exponent L, and F′ ⊂ F, then F′ is also an L_{2}-dense class with the same exponent L, only with a possibly different parameter D′.

To prove this statement let us choose for all numbers 0 < ε ≤ 1 and probability measures ν on (Y, Y) some functions f_{1}, …, f_{m} ∈ F with m ≤ D (\frac{ε}{2})^{−L} elements, such

that the sets D_{j} = \{f: ∫ |f − f_{j}|^{2} dν ≤ (\frac{ε}{2})^{2}\} satisfy the relation ⋃_{j=1}^{m} D_{j} = Y. For all

sets D_{j} for which D_{j} ∩ F′ is non-empty choose a function f′_{j} ∈ D_{j} ∩ F′. In such a way we get a collection of functions f′_{j} from the class F′ containing at most 2^{L}Dε^{−L} elements which satisfies the condition imposed for L_{2}-dense classes with exponent L and parameter 2^{L}D for this number ε and measure ν.

Next I formulate in Theorem 4.1′ a result about the supremum of the integral of a class of functions with respect to a normalized empirical distribution. It can be considered as a simple version of Theorem 4.1. I formulated this result, because Theorems 4.1 and 4.1′ are special cases of their multivariate counterparts about the supremum of so-called U-statistics and multiple integrals with respect to a normalized empirical distribution function discussed in Section 8. These results are also closely related, but the explanation of their relation demands some work.

Given a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{n} taking values in (X, X) let us introduce their empirical distribution on (X, X) as

μ_{n}(A)(ω) = \frac{1}{n}# \{j: 1 ≤ j ≤ n, ξ_{j}(ω) ∈ A\} , A ∈ X ,  (4.5)

and define for all measurable and μ integrable functions f the (random) integral

J_{n}(f) = J_{n,1}(f) = \sqrt{n} ∫ f(x)(μ_{n}( dx) − μ( dx)).  (4.6)

Clearly J_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n}(f(ξ_{j}) − Ef(ξ_{j})) = S_{n}(f̄) with f̄(x) = f(x) − ∫ f(x)μ( dx).

It is not difficult to see that sup_{x∈X} |f̄(x)| ≤ 2 if sup_{x∈X} |f(x)| ≤ 1, ∫ f̄(x)μ( dx) = 0,

∫ f̄^{2}(x)μ( dx) ≤ ∫ f^{2}(x)μ( dx), and if F is an L_{2}-dense class of functions with parameter D and exponent L, then the class of functions F̄ consisting of the functions f̄(x) = f(x) − ∫ f(x)μ( dx), f ∈ F, is an L_{2}-dense class of functions with parameter 2^{L}D and exponent L, since ∫ (f̄ − ḡ)^{2} dμ ≤ ε if f, g ∈ F, and ∫ (f − g)^{2} dμ ≤ (\frac{ε}{2})^{2}. Hence Theorem 4.1 implies the following result.

Theorem 4.1′. (Estimate on the supremum of random integrals with respect to a normalized empirical measure). Let us have a sequence of independent and identically distributed random variables ξ_{1}, …, ξ_{n}, n ≥ 2, with distribution μ on a measurable space (X, X) together with some class of functions F on this space which satisfies the conditions of Theorem 4.1 with the possible exception of condition (4.3). The estimate (4.4) remains valid if the random sums S_{n}(f) are replaced in it by the random integrals J_{n}(f) defined in (4.6). Moreover, similarly to the corollary of Theorem 4.1, the condition about the countable cardinality of the set F can be replaced by the condition that the class of random variables J_{n}(f), f ∈ F, is countably approximable.

All finite dimensional distributions of the set of random variables S_{n}(f), f ∈ F, considered in Theorem 4.1 converge to those of a Gaussian random field Z(f), f ∈ F, with expectation EZ(f) = 0 and correlation EZ(f)Z(g) = ∫ f(x)g(x)μ( dx), f, g ∈ F as n → ∞. Here, and in the subsequent part of the paper a collection of random variables indexed by some set of parameters will be called a Gaussian random field if for all finite subsets of these parameters the random variables indexed by this finite set are jointly Gaussian. We shall also define so-called linear Gaussian random fields. They consist of jointly Gaussian random variables Z(f), f ∈ G, indexed by a linear space f ∈ G which satisfy the relation Z(af + bg) = aZ(f) + bZ(g) for all real numbers a and b and f, g ∈ G.

Let us consider a linear Gaussian random field Z(f), f ∈ G, where the set of indices G = G_{μ} consists of the functions f square integrable with respect to a σ-finite measure μ, and take an appropriate restriction of this field to some parameter set F ⊂ G. In the next Theorem 4.2 we shall present a natural Gaussian counterpart of Theorem 4.1 by means of an appropriate choice of F. Let me also remark that in Section 10 multiple Wiener–Itô integrals of functions of k variables with respect to a white noise will be defined for all k ≥ 1. In the special case k = 1 the Wiener–Itô integrals for an appropriate class of functions f ∈ F yield a model for which Theorem 4.2 is applicable. Before formulating this result let us introduce the following definition which is a version of the definition of L_{p}-dense functions.

Definition of L_{p}-dense classes of functions with respect to a measure μ. Let a measurable space (X, X) be given together with a measure μ on the σ-algebra X and a set F of X measurable real valued functions on this space. The set of functions F is called an L_{p}-dense class of functions, 1 ≤ p < ∞, with respect to the measure μ with parameter D and exponent L if for all numbers 0 < ε ≤ 1 there exists a finite ε-dense subset F_{ε} = {f_{1}, …, f_{m}} ⊂ F in the space L_{p}(X, X , μ) with m ≤ Dε^{−L} elements, i.e. such a set F_{ε} ⊂ F with m ≤ Dε^{−L} elements for which inf_{f_{j}∈F_{ε}} ∫ |f − f_{j}|^{p} dμ < ε^{p} for all

functions f ∈ F.

Theorem 4.2. (Estimate on the supremum of a class of Gaussian random variables). Let a probability measure μ be given on a measurable space (X, X) together with a linear Gaussian random field Z(f), f ∈ G, such that EZ(f) = 0, EZ(f)Z(g) = ∫ f(x)g(x)μ( dx), f, g ∈ G, where G is the space of square integrable functions with respect to this measure μ. Let F ⊂ G be a countable and L_{2}-dense class of functions with respect to the measure μ with some exponent L ≥ 1 and parameter D ≥ 1 which also satisfies condition (4.2) with some 0 < σ ≤ 1.

Then there exist some universal constants C > 0 and M > 0 (for instance C = 4 and M = 16 is a good choice) such that the inequality

P (sup_{f∈F} |Z(f)| ≥ u) ≤ C(D + 1) exp \{−\frac{1}{256} (\frac{u}{σ} )^{2}\} if u ≥ ML^{1/2}σ log^{1/2} \frac{2}{σ} (4.7)

holds with the parameter D and exponent L introduced in this theorem.

The exponent at the right-hand side of inequality (4.7) does not contain the best possible universal constant. One could choose the coefficient \frac{1−ε}{2} with arbitrary small ε > 0 instead of the coefficient \frac{1}{256} in the exponent at the right-hand side of (4.7) if the universal constants C > 0 and M > 0 are chosen sufficiently large in this inequality. Actually, later in Theorem 8.6 such an estimate will be proved which can be considered as the multivariate generalization of Theorem 4.2 with the exponent −\frac{(1−ε)u^{2}}{2σ^{2}} .

The condition about the countable cardinality of the set F in Theorem 4.2 could be weakened similarly to Theorem 4.1. But I omit the discussion of this question, since Theorem 4.2 was only introduced for the sake of a comparison between the Gaussian and non-Gaussian case. An essential difference between Theorems 4.1 and 4.2 is that the class of functions F considered in Theorem 4.1 had to be L_{2}-dense, while in Theorem 4.2 a weaker version of this property was needed. In that result it was demanded that there exists a subset of F of relatively small cardinality which is dense in the L_{2}(μ) norm. In the L_{2}-density property imposed in Theorem 4.1 a similar property was demanded for all probability measures ν. The appearance of such a property may be unexpected. But as we shall see, the proof of Theorem 4.1 contains a conditioning argument where a lot of new conditional measures appear, and the L_{2}-density property is needed to work with all of them. One would also like to know some results that enable us to check when this condition holds. In the next section a notion popular in probability theory, the Vapnik–Červonenkis classes will be introduced, and it will be shown that a Vapnik–Červonenkis class of functions bounded by 1 is L_{2}-dense.

Another difference between Theorems 4.1 and 4.2 is that the conditions of formula (4.4) contain the upper bound \sqrt{n}σ^{2} > u, and no such condition was imposed in formula (4.7). The appearance of this condition in Theorem 4.1 can be explained by comparing this result with those of Section 3. As we have seen, we do not loose much information if we restrict our attention to the case u ≤ const. V_{n}^{2} = const. nσ^{2} in Bernstein’s inequality (if sums of independent and identically distributed random variables are considered). Theorem 4.1 gives an almost as good estimate for the supremum of normalized partial sums under appropriate conditions for the class F of functions we consider in this theorem, as Bernstein’s inequality yields for the normalized partial sums of independent and identically distributed random variables with variance bounded by σ^{2}. But we could prove the estimate of Theorem 4.1 only under the condition \sqrt{n}σ^{2} > u. We shall show in Example 4.3 discussed below that in the case \sqrt{n}σ^{2} > u only a weaker estimate holds. It has also a natural reason why condition (4.1) about the supremum of the functions f ∈ F appeared in Theorems 4.1 and 4.1′, and no such condition was needed in Theorem 4.2.

The lower bounds for the level u were imposed in formulas (4.4) and (4.7) because of a similar reason. To understand why such a condition is needed in formula (4.7) let us consider the following example. Take a Wiener process W (t), 0 ≤ t ≤ 1, define for all 0 ≤ s < t ≤ 1 the functions f_{s,t}(·) on the interval [0, 1] as f_{s,t}(u) = 1 if s ≤ u ≤ t, f_{s,t}(u) = 0 if 0 ≤ u < s or t < u ≤ 1, and introduce for all σ > 0 the following class of functions F_{σ}. F_{σ} = {f_{s,t}: 0 ≤ s < t ≤ 1, t − s ≤ σ^{2}, s and t are rational numbers.}. The integral Z(f) = ∫_{0}^{1} f(x)W ( dx) can be defined for all square integrable functions f on the interval [0, 1], and this yields a linear Gaussian random field on the space of square integrable functions. In the special case f = f_{s,t} we have Z(f_{s,t}) = ∫ f_{s,t}(u)W ( du) = W (t) − W (s). It is not difficult to see that the Gaussian random field Z(f), f ∈ F_{σ}, satisfies the conditions of Theorem 4.2 with the number σ in formula (4.2). It

is natural to expect that P ( sup_{f∈F_{σ}} Z(f) > u) ≤ e^{−const. (u/σ)^{2}}. However, this relation

does not hold if u = u(σ) < (1 − ε)\sqrt{2}σ log^{1/2} \frac{1}{σ} with some ε > 0. In such cases

P ( sup_{f∈F_{σ}} Z(f) > u) → 1, as σ → 0. This can be proved relatively simply with the help

of the estimate P (Z(f_{s,t}) > u(σ)) ≥ const. σ^{1−ε} if |t − s| = σ^{2} and the independence of the random integrals Z(f_{s,t}) if the functions f_{s,t} are indexed by such pairs (s, t) for which the intervals (s, t) are disjoint. This means that in this example formula (4.7) holds only under the condition u ≥ Mσ log^{1/2} \frac{1}{σ} with M = \sqrt{2}.

There is a classical result about the modulus of continuity of Wiener processes, and actually this result helped us to find the previous example. It is also worth mentioning that there are some concentration inequalities, see Ledoux [28] and Talagrand [51], which state that under very general conditions the distribution of the supremum of a class of partial sums of independent random variables or of the elements of a Gaussian random field is strongly concentrated around the expected value of this supremum. (Talagrand’s result in this direction is also formulated in Theorem 18.1 of this lecture note.) These results imply that the problems discussed in Theorems 4.1 and 4.2 can be reduced to a good estimate of the expected value E sup_{f∈F} |S_{n}(f)| and E sup_{f∈F} |Z(f)| of the

supremum considered in these results. However, the estimation of the expected value of these suprema is not much simpler than the original problem.

Theorem 4.2 implies that under its conditions E sup_{f∈F} |Z(f)| ≤ const. σ log^{1/2} \frac{2}{σ} with

an appropriate multiplying constant depending on the parameter D and exponent L of the class of functions F. In the case of Theorem 4.1 a similar estimate holds, but under more restrictive conditions. We also have to impose that \sqrt{n}σ^{2} ≥ const. σ log^{1/2} \frac{2}{σ} with a sufficiently large constant. This condition is needed to guarantee that the set of numbers u satisfying condition (4.4) is not empty. If this condition is violated, then Theorem 4.1 supplies a weaker estimate which we get by replacing σ by an appropriate σ̄ > σ, and by applying Theorem 4.1 with this number σ̄.

One may ask whether the above estimate about the expected value of supremum of normalized partial sums may hold without the condition \sqrt{n}σ^{2} ≥ const. σ log^{1/2} \frac{2}{σ}. We show an example which gives a negative answer to this question. Since here we discuss a rather particular problem which is outside of our main interest in this work I give a rather sketchy explanation of this example. I present this example together with a Poissonian counterpart of it which may help explain why such a result holds.

Example 4.3. (Supremum of partial sums with bad tail behaviour). Let ξ_{1}, …, ξ_{n} be a sequence of independent random variables with uniform distribution in the interval [0, 1]. Choose a sequence of real numbers, ε_{n}, n = 3, 4, …, such that ε_{n} → 0 as

n → ∞, and \frac{1}{2} ≥ ε_{n} ≥ n^{−δ} with a sufficiently small number δ > 0. Put σ_{n} = ε_{n}\sqrt{\frac{log n}{n}}, and define the set of functions f̄_{j,n}(·) and f_{j,n}(·) on the interval [0, 1] by the formulas f̄_{j,n}(x) = 1 if (j − 1)σ_{n}^{2} ≤ x < jσ_{n}^{2}, f̄_{j,n}(x) = 0 otherwise, and f_{j,n}(x) = f̄_{j,n}(x) − σ_{n}^{2}, n = 3, 4, …, 1 ≤ j ≤ \frac{1}{σ_{n}^{2}} . Put F_{n} = \{f_{j,n}(·): 1 ≤ j ≤ \frac{1}{σ_{n}^{2}} \}, and u_{n} = _{\frac{A}{log \frac{1}{εn}}} \frac{log n}{\sqrt{n}} with a sufficiently small A > 0. Then

lim_{n→∞} P ( sup_{f∈F_{n}} S_{n}(f) > u_{n}) = 1.

This example has the following Poissonian counterpart.

Example 4.3′. (A Poissonian counterpart of Example 4.3). Let P̄_{n}(x) be a Poisson process on the interval [0, 1] with parameter n and P_{n}(x) = \frac{1}{\sqrt{n}}[P̄_{n}(x) − nx], 0 ≤ x ≤ 1. Consider the same sequences of numbers ε_{n}, σ_{n} and u_{n} as in Example 4.3, and define the random variables Z_{n,j} = P_{n}(jσ_{n}^{2}) − P_{n}((j − 1)σ_{n}^{2}) for all n = 3, 4, … and 1 ≤ j ≤ \frac{1}{σ_{n}^{2}} . Then

lim_{n→∞} P ( sup_{1≤j≤ \frac{1}{σn}} (Z_{n,j} − Z_{n,j−1}) > u_{n}) = 1.

The classes of functions F_{n} in Example 4.3 are L_{2}-dense classes of functions with some exponent L and parameter D not depending on the parameter n and the choice of the numbers σ_{n}. It can be seen that even the class of function F = {f: f(x) = 1, if s ≤ x < t, f(x) = 0 otherwise.} consisting of functions defined on the interval [0, 1] is an L_{2}-dense class with some exponent L and parameter D. This follows from the results discussed in the later part of this work (mainly Theorem 5.2), but it can be proved directly that this statement holds e.g. with L = 1 and D = 8. The classes of functions F_{n} also satisfy conditions (4.1), (4.2) and (4.3) of Theorem 4.1 with σ^{2} = σ̄_{n}^{2} = σ_{n}^{2} − σ_{n}^{4}, lim_{n→∞} \frac{σ̄_{n}}{σ_{n}} = 1, and the number u_{n} satisfies the second condi-

tion u_{n} ≥ Mσ̄_{n}(L^{3/4}σ̄_{n}(log^{1/2} \frac{2}{σ̄_{n}} + (log D)^{3/4}) in (4.4) for sufficiently large n. But it does not satisfy the first condition \sqrt{n}σ̄_{n}^{2} ≥ u_{n} of (4.4), and as a consequence Theorem 4.1 cannot be applied in this case. On the other hand, some calculation shows that

u_{n} ≥ (\frac{2}{1+4δ})^{1/2}_{\frac{A}{ε_{n} log \frac{1}{εn}}} σ_{n} log^{1/2} \frac{2}{σ}_{n}. Hence lim inf_{n→∞} ε_{n} log \frac{1}{ε_{n}} · _{\frac{1}{σ̄_{n} log^{1/2} \frac{2}{σ̄n}}} E sup_{f∈F_{n}} S_{n}(f) >

0 in this case. As ε_{n} log \frac{1}{ε_{n}} → 0 as n → ∞, this means that the expected value of the supremum of the random sums considered in Example 4.3 does not satisfy the estimate lim sup_{n→∞} _{\frac{1}{σ̄_{n} log^{1/2} \frac{2}{σ̄n}}} E sup_{f∈F_{n}} S_{n}(f) < ∞ suggested by Theorem 4.1. Ob-

serve that \sqrt{n}σ̄_{n}^{2} ∼ const. ε_{n}σ̄_{n} log^{1/2} \frac{2}{σ̄_{n}} in this case, since \sqrt{n}σ̄_{n}^{2} ∼ ε_{n}^{2} \frac{log n}{\sqrt{n}}, and

σ̄_{n} log^{1/2} \frac{2}{σ̄_{n}} ∼ const. ε_{n} \frac{log n}{\sqrt{n}}.

The proof of Examples 4.3 and 4.3′. First we prove the statement of Example 4.3′. As the random variables Z_{n,j} are independent for a fixed number n, and their number is less than n^{1+3δ} ≤ n^{2−δ} if δ ≤ \frac{1}{4}, it is enough to show that P (Z_{n,j} > u_{n}) ≥ \frac{1}{n^{2}} if n ≥ n_{0} with some threshold index n_{0} = n_{0}(A, δ). Put ū_{n} = [\sqrt{n}u_{n} + nσ_{n}^{2}] + 1, where [·] denotes integer part. Then P (Z_{n,j} > u_{n}) ≥ P (P̄_{n}(σ_{n}^{2}) ≥ ū_{n}) ≥ P (P̄_{n}(σ_{n}^{2}) = ū_{n}). On the other hand, ū_{n} ∼ (_{\frac{A}{log \frac{1}{εn}}} + ε_{n}^{2}) log n < _{\frac{2A}{log \frac{1}{εn}}} log n, hence P (P̄_{n}(σ_{n}^{2}) = ū_{n}) = \frac{(nσ_{n}^{2})ūn}{ū_{n}!} e^{−nσ_{n}^{2}} >

(\frac{nσ_{n}^{2}·e}{ū_{n}})ū_{n}e^{−nσ_{n}^{2}} ≥ exp \{−2ū_{n} log \frac{ū_{n}}{nσ_{n}^{2}} \} ≥ e^{−3ū_{n} log \frac{1}{εn}} ≥ n^{−2}.

The statement of Example 4.3 can be deduced from Example 4.3′ by applying Poissonian approximation. Let us apply the result of Example 4.3′ for a Poisson process P̄_{n/2} with parameter \frac{n}{2} and with such a number ε̄_{n/2} with which the value of σ_{n/2} equals the previously defined σ_{n}. Then ε̄_{n/2} ∼ \frac{ε_{n}}{\sqrt{2}}, and the number of sample points of P̄_{n/2} is less than n with probability almost 1. Attaching additional sample points to get exactly n sample points we can get the result of Example 4.3. I omit the details.

In formulas (4.4) and (4.7) we formulated such a condition for the validity of Theorem 4.1 and Theorem 4.2 which contains a large multiplying constant of σ log^{1/2} \frac{2}{σ} in the lower bound for the number u if we deal with such an L_{2} class of functions F which has a large exponent L. At a heuristic level it is clear that in such a case a large multiplying constant appears. On the other hand, I did not try to find the best possible coefficients in the lower bound in relations (4.4) and (4.7).

In Theorem 4.1 (and in its version 4.1′) it was demanded that the class of functions F should be countable. Later this condition was replaced by a weaker one about countable approximability. By restricting our attention to countable or countably approximable classes we could avoid some unpleasant measure theoretical problems which would have arisen if we had worked with the supremum of non-countable number of random variables which may be non-measurable. There are some papers where possibly non-measurable models are also considered with the help of some rather deep results of the analysis and measure theory. Actually, the problem we met here is the natural analog of an important problem in the theory of the stochastic processes about the smoothness property of the trajectories of an appropriate version of a stochastic process which we can get by exploiting our freedom to change all random variables on a set of probability zero.

The study of the problem in this work is simpler in one respect. Here the set of random variables S_{n}(f)(ω) or J_{n}(f)(ω), f ∈ F, are constructed directly with the help of the underlying random variables ξ_{1}(ω), …, ξ_{n}(ω) for all ω ∈ Ω separately. We are interested in when the sets of random variables constructed in this way are countably approximable, i.e. we are not looking for a possibly different, better version of them with the same finite dimensional distributions. The next simple Lemma 4.4 yields a sufficient condition for countable approximability. Its condition can be interpreted as a smoothness type condition for the trajectories of a stochastic process indexed by the functions f ∈ F.

Lemma 4.4. Let a class of random variables U(f), f ∈ F, indexed by some set F of functions be given on a space (Y, Y). If there exists a countable subset F′ ⊂ F of the set F such that the sets A(u) = {ω: sup_{f∈F} |U(f)(ω)| ≥ u} and B(u) = {ω: sup_{f∈F′} |U(f)(ω)| ≥

u} introduced for all u > 0 in the definition of countable approximability satisfy the relation A(u) ⊂ B(u − ε) for all u > ε > 0, then the class of random variables U(f), f ∈ F, is countably approximable.

The above property holds if for all f ∈ F, ε > 0 and ω ∈ Ω there exists a function f̄ = f̄(f, ε, ω) ∈ F′ such that |U(f̄)(ω)| ≥ |U(f)(ω)| − ε.

Proof of Lemma 4.4. If A(u) ⊂ B(u − ε) for all ε > 0, then P^{∗}(A(U) \ B(u)) ≤ lim_{ε→0} P (B(u−ε)\B(u)) = 0, where P^{∗}(X) denotes the outer measure of a not necessarily

measurable set X ⊂ Ω, since ⋂_{ε→0} B(u − ε) = B(u), and this is what we had to prove. If

ω ∈ A(u), then for all ε > 0 there exists some f = f(ω) ∈ F such that |U(f)(ω)| > u− \frac{ε}{2}. If there exists some f̄ = f̄(f, \frac{ε}{2}, ω), f ∈ F′ such that |U(f̄)(ω)| ≥ |Uf(ω)| − \frac{ε}{2}, then |U(f̄)(ω)| > u − ε, and ω ∈ B(u − ε). This means that A(u) ⊂ B(u − ε).

The question about countable approximability also appears in the case of multiple random integrals with respect to a normalized empirical measure. To avoid some repetition we prove a result which also covers such cases. For this goal first we introduce the notion of multiple integrals with respect to a normalized empirical measure.

Given a measurable function f(x_{1}, …, x_{k}) on the k-fold product space (X^{k}, X^{k}) and a sequence of independent random variables ξ_{1}, …, ξ_{n} with some distribution μ on the space (X, X) we define the integral J_{n,k}(f) of the function f with respect to the k-fold product of the normalized version of the empirical measure μ_{n} introduced in (4.5) by the formula

J_{n,k}(f) = \frac{n^{k/2}}{k!} ∫ ′ f(x_{1}, …, x_{k})(μ_{n}( dx_{1}) − μ( dx_{1})) … (μ_{n}( dx_{k}) − μ( dx_{k})),

where the prime in ∫′ means that the diagonals x_{j} = x_{l}, 1 ≤ j < l ≤ k,

are omitted from the domain of integration.  (4.8)

In the case k ≥ 2 it will be assumed that the probability measure μ has no atoms.

Lemma 4.4 enables us to prove that certain classes of random integrals J_{n,k}(f), f ∈ F, defined with the help of some set of functions f ∈ F of k variables are countably approximable. I present an example of a class of such random integrals which is important in certain applications.

Let us consider the case when X = R^{s}, the s-dimensional Euclidean space with some s ≥ 1. For two vectors u = (u^{(1)}, …, u^{(s)}) ∈ R^{s}, v = (v^{(1)}, …, v^{(s)}) ∈ R^{s} such that u < v, i.e. u^{(j)} < v^{(j)} for all 1 ≤ j ≤ s let B(u, v) denote the s-dimensional rectangle B(u, v) = {z: u < z < v}. Let us fix some function f(x_{1}, …, x_{k}) of k variables such that sup |f(x_{1}, …, x_{k})| ≤ 1, on the space (X^{k}, X^{k}) = (R^{ks}, B^{ks}), where B^{t} denotes the Borel σ-algebra on the Euclidean space R^{t}, together with some probability measure μ on (R^{s}, B^{s}). For all pairs of vectors (u_{1}, …, u_{k}), (v_{1}, …, v_{k}) such that u_{j}, v_{j} ∈ R^{s} and u_{j} ≤ v_{j}, 1 ≤ j ≤ k, let us define the function f_{u_{1},...,u_{k},v_{1},...,v_{k}} which equals the function f on the rectangle (u_{1}, v_{1}) × ⋯ × (u_{k}, v_{k}), and it is zero outside of this rectangle. Let us call a class of functions F consisting of functions of the form f_{u_{1},...,u_{k},v_{1},...,v_{k}} closed if it has the following property. If f_{u_{1},...,u_{k},v_{1},...,v_{k}} ∈ F for some vectors (u_{1}, …, u_{k}) and (v_{1}, …, v_{k}), and u_{j} ≤ ū_{j} < v̄_{j} ≤ v_{j}, 1 ≤ j ≤ k, then f_{ū_{1},...,ū_{k},v̄_{1},...,v̄_{k}} ∈ F. In Lemma 4.5 it will be proved that the random integrals introduced in formula (4.8) of functions from a closed class F constitute a countably approximable class.

Lemma 4.5. Let us have a function f on the Euclidean space R^{ks} such that the |f| ≤ 1 in all points, and consider a closed class F of functions of the form f_{u_{1},...,u_{k},v_{1},...,v_{k}} ∈ (R^{sk}, B^{sk}), u_{j}, v_{j} ∈ R^{s}, u_{j} ≤ v_{j}, 1 ≤ j ≤ k, introduced in the previous paragraph with the help of this function f. Let us take n independent and identically distributed random variables ξ_{1}, …, ξ_{n} with some distribution μ and values in the space (R^{s}, B^{s}). Let μ_{n} denote the empirical distribution of this sequence. Then the class of random integrals J_{n,k}(f_{u_{1},...,u_{k},v_{1},...,v_{k}}) defined in formula (4.8) with functions f_{u_{1},...,u_{k},v_{1},...,v_{k}} ∈ F is countably approximable.

Proof of Lemma 4.5. We shall prove that the definition of countable approximability is satisfied in this model if the class of functions F′ consists of those functions f_{u_{1},...,u_{k},v_{1},...,v_{k}}, u_{j} ≤ v_{j}, 1 ≤ j ≤ k, for which all coordinates of the vectors u_{j} and v_{j} are rational numbers.

Given some function f_{u_{1},...,u_{k},v_{1},...,v_{k}}, a real number 0 < ε < 1 and ω ∈ Ω let us choose a function f_{ū_{1},...,ū_{k},v̄_{1},...,v̄_{k}} ∈ F′ determined with some vectors ū_{j} = ū_{j}(ε, ω), v̄_{j} = v̄_{j}(ε, ω) 1 ≤ j ≤ k, with rational coordinates u_{j} ≤ ū_{j} < v̄_{j} ≤ v_{j} such that the sets K_{j} = B(u_{j}, v_{j}) \ B(ū_{j}, v̄_{j}) satisfy the relations μ(K_{j}) ≤ ε2^{−2k+1}n^{−k/2}, and ξ_{l}(ω) ∉ K_{j} for all j = 1, …, k and l = 1, …, n. Let us show that

|J_{n,k}(f_{ū_{1},...,ū_{k},v̄_{1},...,v̄_{k}})(ω) − J_{n,k}(f_{u_{1},...,u_{k},v_{1},...,v_{k}})(ω)| ≤ ε.  (4.9)

Then lemma 4.4 (with the choice U(f) = J_{n,k}(f)) and relation (4.9) imply Lemma 4.5.

Relation (4.9) holds, since the difference of integrals at its left-hand side can be written as the sum of the 2^{k} − 1 integrals of the function f with respect to the k-fold product of the measure \sqrt{n}(μ_{n} − μ) on the domains D_{1} × ⋯ × D_{k} with the omission of the diagonals x_{j} = xj̄, 1 ≤ j, j̄ ≤ k, j ≠ j̄, where D_{j} is either the set K_{j} or B(u_{j}, v_{j}) and D_{j} = K_{j} for at least one index j. It is enough to show that the absolute value of all these integrals is less than ε2^{−k}. This follows from the observations that |f(x_{1}, …, x_{k})| ≤ 1, \sqrt{n}(μ_{n} − μ)(K_{j}) = −\sqrt{n}μ(K_{j}), μ(K_{j}) ≤ ε2^{−2k+1}n^{−k/2}, and the total variation of the signed measure \sqrt{n}(μ_{n} − μ) (restricted to the set B(u_{j}, v_{j})) is less than 2\sqrt{n}.

In Lemma 4.5 we have shown with the help of Lemma 4.4 about an important class of functions that it is countably approximable. There are other interesting classes of functions about which it can be proved with the help of Lemma 4.4 that they are countably approximable, but here we shall not discuss this problem.

Let us discuss the relation of the results in this section to an important result, the so-called fundamental theorem of the mathematical statistics. In that problem a sequence of independent random variables ξ_{1}(ω), …, ξ_{n}(ω) is considered with distribution function F (x), the empirical distribution function F_{n}(x) = F_{n}(x, ω) = \frac{1}{n}#\{j: 1 ≤ j ≤ n, ξ_{j}(ω) < x} is introduced, and the difference F_{n}(x) − F (x) is considered. This result states that sup_{x} |F_{n}(x) − F (x)| tends to zero with probability one.

Observe that sup_{x} |F_{n}(x) − F (x)| = n^{−1/2} sup_{f∈F} |J_{n}(f)|, where F consists of the func-

tions f_{x}(·), x ∈ R^{1}, defined by the relation f_{x}(u) = 1 if u < x, and f_{x}(u) = 0 if

u ≥ x. Theorem 4.1′ yields an estimate for the probabilities P (sup_{f∈F} |J_{n}(f)| > u).

We have seen that the above class of functions F is countably approximable. The results of the next section imply that this class of functions is also L_{2}-dense. Otherwise it is not difficult to check this property directly. Hence we can apply Theorem 4.1′ to the above defined class of functions with σ = 1, and it yields that

P (n^{−1/2} sup_{f∈F} |J_{n}(f)| > u) ≤ e^{−Cnu^{2}} if 1 ≥ u ≥ C̄n^{−1/2} with some universal constants

C > 0 and C̄ > 0. (The condition 1 ≥ u can actually be dropped.) The application of this estimate for the numbers ε > 0 together with the Borel–Cantelli lemma imply the fundamental theorem of the mathematical statistics.

In short, the results of this section yield more information about the closeness the empirical distribution function F_{n} and distribution function F than the fundamental theorem of the mathematical statistics. Moreover, since these results can also be applied for other classes of functions, they yield useful information about the closeness of the probability measure μ and the empirical measure μ_{n}.

5. Vapnik–Červonenkis classes and L_{2}-dense classes of functions.

In this section the most important notions and results will be presented about Vapnik– Červonenkis classes, and it will be explained how they help to show in some important cases that certain classes of functions are L_{2}-dense. The classes of L_{2}-dense classes played an important role in the study of the previous section. The results of this section may help to find interesting classes of functions with this property. Some of the results formulated in this section will be proved in Appendix A.

First I recall the following notions.

Definition of Vapnik-Červonenkis classes of sets and functions. Let a set X be given, and let us select a class D of subsets of this set X. We call D a Vapnik– Červonenkis class if there exist two real numbers B and K such that for all positive integers n and subsets S(n) = {x_{1}, …, x_{n}} ⊂ X of cardinality n of the set X the collection of sets of the form S(n) ∩ D, D ∈ D, contains no more than Bn^{K} subsets of S(n). We shall call B the parameter and K the exponent of this Vapnik–Červonenkis class.

A class of real valued functions F on a space (Y, Y) is called a Vapnik–Červonenkis class if the collection of graphs of these functions is a Vapnik–Červonenkis class, i.e. if the sets A(f) = {(y, t): y ∈ Y, min(0, f(y)) ≤ t ≤ max(0, f(y))}, f ∈ F, constitute a Vapnik–Červonenkis class of subsets of the product space X = Y × R^{1}.

The following result which was first proved by Sauer is of fundamental importance in the theory of Vapnik–Červonenkis classes. This result provides a relatively simple condition for a class D of subsets of a set X to be a Vapnik–Červonenkis class. Its proof is given in Appendix A. Before its formulation I introduce some terminology which seems to be wide spread and generally accepted in the literature.

Definition of shattering of a set. Let a set S and a class E of subsets of S be given. A finite set F ⊂ S is called shattered by the class E if all its subsets H ⊂ F can be written in the form H = E ∩ F with some element E ∈ E of the class of sets of E.

Theorem 5.1. (Sauer’s lemma). Let a finite set S = S(n) consisting of n elements be given together with a class E of subsets of S. If E shatters no subset of S of cardinality k, then E contains at most \binom{n}{0} + \binom{n}{1} + ⋯ + \binom{n}{k−1} subsets of S.

The estimate of Sauer’s lemma is sharp. Indeed, if E contains all subsets of S of cardinality less than or equal to k −1, then it shatters no subset of a set F of cardinality k (a set F of cardinality k cannot be written in the form E ∩ F, E ∈ E), and E contains \binom{n}{0} + \binom{n}{1} + ⋯ + \binom{n}{k−1} subsets of S. Sauer’s lemma states, that this is an extreme case. Any class of subsets E of S with cardinality greater than \binom{n}{0} + \binom{n}{1} + ⋯ + \binom{n}{k−1} shatters at least one subset of S with cardinality k.

Let us have a set X and a class of subsets D of it. One may be interested in when D is a Vapnik–Červonenkis class. Sauer’s lemma gives a useful condition for it. Namely, it implies that if there exists a positive integer k such that the class D shatters no subset of X of cardinality k, then D is a Vapnik–Červonenkis class. Indeed, let us take some number n ≥ k, fix an arbitrary set S(n) = {x_{1}, …, x_{n}} ⊂ X of cardinality n, and introduce the class of subsets E = E(S(n)) = {S(n) ∩ D: D ⊂ D}. If D shatters no subset of X of cardinality k, then E shatters no subset of S(n) of cardinality k. Hence by Sauer’s lemma the class E contains at most \binom{n}{0} + \binom{n}{1} + ⋯ + \binom{n}{k−1} elements. Let me

remark that it is also proved that \binom{n}{0} + \binom{n}{1} + ⋯ + \binom{n}{k−1} ≤ 1.5\frac{n^{k−1}}{(k−1)!} if n ≥ k + 1. This estimate gives a bound on the parameter and exponent of a Vapnik–Červonenkis class which satisfies the above condition.

Moreover, Theorem 5.1 also has the following consequence. Take an (infinite) set X and a class of its subsets D. There are two possibilities. Either there is some set S(n) ⊂ X of cardinality n for all integers n such that E(S(n)) contains all subsets of S(n), i.e. D shatters this set, or sup_{S: S⊂X, |S|=n} |E(S)| tends to infinity at most in a

polynomial order as n → ∞, where |S| and |E(S)| denote the cardinality of S and E(S).

The following Theorem 5.2, an important result of Richard Dudley, states that a Vapnik–Červonenkis class of functions bounded by 1 is an L_{1}-dense class of functions.

Theorem 5.2. (A relation between the L_{1}-dense class and Vapnik–Červonenkis class property). Let f(y), f ∈ F, be a Vapnik–Červonenkis class of real valued functions on some measurable space (Y, Y) such that sup_{y∈Y} |f(y)| ≤ 1 for all f ∈ F.

Then F is an L_{1}-dense class of functions on (Y, Y). More explicitly, if F is a Vapnik– Červonenkis class with parameter B ≥ 1 and exponent K > 0, then it is an L_{1}-dense class with exponent L = 2K and parameter D = CB^{2}(4K)^{2K} with some universal constant C > 0.

Proof of Theorem 5.2. Let us fix some probability measure ν on (Y, Y) and a real number 0 < ε ≤ 1. We are going to show that any finite set D(ε, ν) = {f_{1}, …, f_{M}} ⊂ F such that ∫ |f_{j} − f_{k}| dν ≥ ε if j ≠ k, f_{j}, f_{k} ∈ D(ε, ν) has cardinality M ≤ Dε^{−L} with some D > 0 and L > 0. This implies that F is an L_{1}-dense class with parameter D and exponent L. Indeed, let us take a maximal subset D̄(ε, ν) = {f_{1}, …, f_{M}} ⊂ F such that the L_{1}(ν) distance of any two functions in this subset is at least ε. Maximality means in this context that no function f_{M+1} ∈ F can be attached to D̄(ε, ν) without violating this condition. Thus the inequality M ≤ Dε^{−L} means that D̄(ε, ν) is an ε-dense subset of F in the space L_{1}(Y, Y, ν) with no more than Dε^{−L} elements.

In the estimation of the cardinality M of a (finite) set D(ε, ν) = {f_{1}, …, f_{M}} with the property ∫ |f_{j} − f_{k}| dν ≥ ε if j ≠ k the Vapnik–Červonenkis class property of F is exploited in the following way. Let us choose relatively few p points (y_{l}, t_{l}), y_{l} ∈ Y, −1 ≤ t_{l} ≤ 1, 1 ≤ l ≤ p, in the space (Y × [−1, 1]) in such a way that the set S_{0}(p) = {(y_{l}, t_{l}), 1 ≤ l ≤ p} and graphs A(f_{j}) = {(y, t): y ∈ Y, min(0, f_{j}(y)) ≤ t ≤ max(0, f_{j}(y))}, f_{j} ∈ D(ε, ν) ⊂ F have the property that all sets A(f_{j}) ∩ S_{0}(p), 1 ≤ j ≤ M, are different. Then the Vapnik–Červonenkis class property of F implies that M ≤ Bp^{K}. Hence if there exists a set S_{0}(p) with the above property and with a relatively small number p, then this yields a useful estimate on M. Such a set S_{0}(p) will be given by means of the following random construction.

Let us choose the p points (y_{l}, t_{l}), 1 ≤ l ≤ p, of the (random) set S_{0}(p) independently of each other in such a way that the coordinate y_{l} is chosen with distribution ν on (Y, Y) and the coordinate t_{l} with uniform distribution on the interval [−1, 1] independently of y_{l}. (The number p will be chosen later.) Let us fix some indices 1 ≤ j, k ≤ M, and estimate the probability that the sets A(f_{j}) ∩ S_{0}(p) and A(f_{k}) ∩ S_{0}(p) agree, where A(f) denotes the graph of the function f. Consider the symmetric difference A(f_{j})ΔA(f_{k}) of the sets A(f_{j}) and A(f_{k}). The sets A(f_{j}) ∩ S_{0}(p) and A(f_{k}) ∩ S_{0}(p) agree if and only if (y_{l}, t_{l}) ∉ A(f_{j})ΔA(f_{k}) for all (y_{l}, t_{l}) ∈ S_{0}(p). Let us observe that for a fixed l the estimate P ((y_{l}, t_{l}) ∈ A(f_{j})ΔA(f_{k})) = \frac{1}{2}(ν × λ)(A(f_{j})ΔA(f_{k})) = \frac{1}{2} ∫ |f_{j} − f_{k}| dν ≥ \frac{ε}{2} hold, where λ denotes the Lebesgue measure. This implies that the probability that the (random) sets A(f_{j}) ∩ S_{0}(p) and A(f_{k}) ∩ S_{0}(p) agree can be bounded from above by (1 − \frac{ε}{2})^{p} ≤ e^{−pε/2}. Hence the probability that all sets A(f_{j}) ∩ S_{0}(p) are different is greater than 1 − \binom{M}{2}e^{−pε/2} ≥ 1 − \frac{M^{2}}{2} e^{−pε/2}. Choose p such that \frac{7}{4}e^{pε/2} > e^{(p+1)ε/2} > M^{2} ≥ e^{pε/2}. Then the above probability is greater than \frac{1}{8}, and there exists some set S_{0}(p) with the desired property.

The inequalities M ≤ Bp^{K} and M^{2} ≥ e^{pε/2} imply that M ≥ e^{εM^{1/K}/4B^{1/K}}, i.e. \frac{log M^{1/K}}{M^{1/K}} ≥ \frac{ε}{4KB^{1/K}}. As \frac{log M^{1/K}}{M^{1/K}} ≤ CM^{−1/2K} for M ≥ 1 with some universal constant C > 0, this estimate implies that Theorem 5.2 holds with the exponent L and parameter D given in its formulation.

Let us observe that if F is an L_{1}-dense class of functions on a measure space (Y, Y) with some exponent L and parameter D, and also the inequality sup_{y∈Y} |f(y)| ≤ 1 holds

for all f ∈ F, then F is an L_{2}-dense class of functions with exponent 2L and parameter D2^{L}. Indeed, if we fix some probability measure ν on (Y, Y) together with a number 0 < ε ≤ 1, and D(ε, ν) = \{f_{1}, …, f_{M}\} is an \frac{ε^{2}}{2} -dense set of F in the space L_{1}(Y, Y, ν), M ≤ 2^{L}Dε^{−2L}, then for all function f ∈ F some function f_{j} ∈ D(ε, ν) can be chosen in such a way that ∫ (f − f_{j})^{2} dν ≤ 2 ∫ |f − f_{j}| dν ≤ ε^{2}. This implies that F is an L_{2}-dense class with the given exponent and parameter.

It is not easy to check whether a collection of subsets D of a set X is a Vapnik– Červonenkis class even with the help of Theorem 5.1. Therefore the following Theorem 5.3 which enables us to construct many non-trivial Vapnik–Červonenkis classes is of special interest. Its proof is given in Appendix A.

Theorem 5.3. (A way to construct Vapnik–Červonenkis classes). Let us consider a k-dimensional subspace G_{k} of the linear space of real valued functions defined on a set X, and define the level-set A(g) = {x: x ∈ X, g(x) ≥ 0} for all functions g ∈ G_{k}. Take the class of subsets D = {A(g): g ∈ G_{k}} of the set X consisting of the above introduced level sets. No subset S = S(k + 1) ⊂ X of cardinality k + 1 is shattered by D. Hence by Theorem 5.1 D is a Vapnik–Červonenkis class of subsets of X.

Theorem 5.3 enables us to construct many interesting Vapnik–Červonenkis classes. Thus for instance the class of all half-spaces in a Euclidean space, the class of all ellipses in the plane, or more generally the level sets of k-order algebraic functions with a fixed number k constitute a Vapnik–Červonenkis class. It can be proved that if C and D are Vapnik–Červonenkis classes of subsets of a set S, then also their intersection C ∩ D = {C ∩ D: C ∈ C, D ∈ D}, their union C ∪ D = {C ∪ D: C ∈ C, D ∈ D} and complementary sets C^{c} = {S \C: C ∈ C} are Vapnik–Červonenkis classes. These results are less important for us, and their proofs will be omitted. We are interested in Vapnik– Červonenkis classes not for their own sake. We are going to study L_{2}-dense classes of functions, and Vapnik–Červonenkis classes make possible to find some examples. Indeed, Theorem 5.2 implies that if D is a Vapnik–Červonenkis class of subsets of a set S, then their indicator functions constitute an L_{1}-dense, hence also an L_{2}-dense class of functions. Then the results of Lemma 5.4 formulated below enable us to construct new L_{2}-dense class of functions.

Lemma 5.4. (Some useful properties of L_{2}-dense classes). Let G be an L_{2}-dense class of functions on some space (Y, Y) whose absolute values are bounded by one, and let f be a function on (Y, Y) also with absolute value bounded by one. Then f ·G = {f ·g: g ∈ G} is also an L_{2}-dense class of functions. Let G_{1} and G_{2} be two L_{2}-dense classes of functions on some space (Y, Y) whose absolute values are bounded by one. Then the classes of functions G_{1} +G_{2} = {g_{1} +g_{2}: g_{1} ∈ G_{1}, g_{2} ∈ G_{2}}, G_{1} ·G_{2} = {g_{1}g_{2}: g_{1} ∈ G_{1}, g_{2} ∈ G_{2}}, min(G_{1}, G_{2}) = {min(g_{1}, g_{2}): g_{1} ∈ G_{1}, g_{2} ∈ G_{2}}, max(G_{1}, G_{2}) = {max(g_{1}, g_{2}): g_{1} ∈ G_{1}, g_{2} ∈ G_{2}} are also L_{2}-dense. If G is an L_{2}-dense class of functions, and G′ ⊂ G, then G′ is also an L_{2}-dense class.

The proof of Lemma 5.4 is rather straightforward. One has to observe for instance that if g_{1}, ḡ_{1} ∈ G_{1}, g_{2}, ḡ_{2} ∈ G_{2} then | min(g_{1}, g_{2}) − min(ḡ_{1}, ḡ_{2})| ≤ |g_{1} − ḡ_{1})| + |g_{2} − ḡ_{2}|, hence if g_{1,1}, …, g_{1,M_{1}} is an \frac{ε}{2}-dense subset of G_{1} and g_{2,1}, …, g_{2,M_{2}} is an \frac{ε}{2}-dense subset of G_{2} in the space L_{2}(Y, Y, ν) with some probability measure ν, then the functions min(g_{1,j}, g_{2,k}), 1 ≤ j ≤ M_{1}, 1 ≤ k ≤ M_{2} constitute an ε-dense subset of min(G_{1}, G_{2}) in L_{2}(Y, Y, ν). The last statement of Lemma 5.4 was proved after the Corollary of Theorem 4.1. The details are left to the reader.

The above result enable us to construct some L_{2} dense class of functions. We give an example for it in the following Example 5.5 which is a consequence of Theorem 5.2 and Lemma 5.4.

Example 5.5. Take m measurable functions f_{j}(x), 1 ≤ j ≤ m, functions on a measurable space (X, X) which have the property sup_{x∈X} |f_{j}(x)| ≤ 1 for all 1 ≤ j ≤ m. Let D be a

Vapnik-Červonenkis class consisting of measurable subsets of the set X. Define for all pairs f_{j}, 1 ≤ j ≤ m, and D ∈ D the function f_{j,D}(·) as f_{j,D}(x) = f_{j}(x) if x ∈ D, and f_{j,D}(x) = 0 if x ∉ D, i.e. f_{j,D}(·) is the restriction of the function f_{j}(·) to the set D. The set of functions f_{j,D}, 1 ≤ j ≤ m, D ∈ D, is an L_{2}-dense class of functions.

Beside this, Theorem 5.3 helps us to construct Vapnik-Červonenkis classes of sets. Let me also remark that it follows from the result of this section that the random variables considered in Lemma 4.5 are not only countably approximable, but the class of functions f_{u_{1},...,u_{k},v_{1},...,v_{k}} appearing in their definition is L_{2}-dense.

6. The proof of Theorems 4.1 and 4.2 on the supremum of random sums.

In this section we prove Theorem 4.2, an estimate about the tail distribution of the supremum of an appropriate class of Gaussian random variables with the help of a method, called the chaining argument. We also investigate the proof of Theorem 4.1 which can be considered as a version of Theorem 4.2 about the supremum of partial sums of independent and identically distributed random variables. The chaining argument is not a strong enough method to prove Theorem 4.1, but it enables us to prove a weakened form of it formulated in Proposition 6.1. This result turned out to be useful in the proof of Theorem 4.1. It enables us to reduce the proof of Theorem 4.1 to a simpler statement formulated in Proposition 6.2. In this section we prove Proposition 6.1, formulate Proposition 6.2, and reduce the proof of Theorem 4.1 with the help of Proposition 6.1 to this result. The proof of Proposition 6.2 which demands different arguments is postponed to the next section. Before presenting the proofs of this section I briefly describe the chaining argument.

Let us consider a countable class of functions F on a probabality space (X, X , μ) which is L_{2}-dense with respect to the probability measure μ. Let us have either a class of Gaussian random variables Z(f) with zero expectation such that EZ(f)Z(g) =

∫ f(x)g(x)μ( dx), f, g ∈ F, or a set of normalized partial sums S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}),

f ∈ F, where ξ_{1}, …, ξ_{n} is a sequence of independent μ distributed random variables with values in the space (X, X), and assume that Ef(ξ_{j}) = 0 for all f ∈ F. We want

to get a good estimate on the probability P (sup_{f∈F} Z(f) > u) or P (sup_{f∈F} S_{n}(f) > u) if

the class of functions F has some nice properties. The chaining argument suggests to prove such an estimate in the following way.

Let us try to find an appropriate sequence of subset F_{1} ⊂ F_{2} ⊂ ⋯ ⊂ F such

that ⋃_{N=1}^{∞} F_{N} = F, F_{N} is such a set of functions from F with relatively few elements for

which inf_{f∈F_{N}} ∫ (f −f̄)^{2} dμ ≤ δ_{N} with an appropriately chosen number δ_{N} for all functions

f̄ ∈ F, and let us give a good estimate on the probability P ( sup_{f∈F_{N}} Z(f) > u_{N}) or

P ( sup_{f∈F_{N}} S_{n}(f) > u_{N}) for all N = 1, 2, … with an appropriately chosen monotone

increasing sequence u_{N} such that lim_{N→∞} u_{N} = u.

We can get a relatively good estimate under appropriate conditions for the class of functions F by choosing the classes of functions F_{N} and numbers δ_{N} and u_{N} in an appropriate way. We try to bound the difference of the probabilities

P ( sup_{f∈F_{N+1}} Z(f) > u_{N+1}) − P ( sup_{f∈F_{N}} Z(f) > u_{N})

or of the analogous difference if Z(f) is replaced by S_{n}(f). For the sake of completeness define this difference also in the case N = 1 with the choice F_{0} = ∅, when the second probability in this difference equals zero.

This probability can be estimated in a natural way by taking for all functions f_{j_{N+1}} ∈ F_{N+1} a function f_{j_{N}} ∈ F_{N} which is close to it, more explicitly ∫ (f_{j_{N+1}} − f_{j_{N}})^{2} dμ ≤ δ_{N}^{2}, and calculating the probability that the difference of the random variables corresponding to these two functions is greater than u_{N+1} − u_{N}. We can estimate these probabilities with the help of some results which give a relatively good bound on the tail distribution of Z(g) or S_{n}(g) if ∫ g^{2} dμ is small. The sum of all such probabilities gives an upper bound for the above considered difference of probabilities. Then we get

an estimate for the probability P ( sup_{f∈F_{N}} Z(f) > u_{N}) for all N = 1, 2, …, by summing

up the above estimate, and we get a bound on the probability we are interested in by taking the limit N → ∞. This method is called the chaining argument. It got this name, because we estimate the contribution of a random variable corresponding to a function f_{j_{N+1}} ∈ F_{N+1} to the bound of the probability we investigate by taking the random variable corresponding to a function f_{j_{N}} ∈ F_{N} close to it, then we choose another random variable corresponding to a function f_{j_{N−1}} ∈ F_{N−1} close to this function, and so on we take a chain of subsequent functions and the random variables corresponding to them.

First we show how this method supplies the proof of Theorem 4.2. Then we turn to the investigation of Theorem 4.1. In the study of this problem the above method does not work well, because if two functions are very close to each other in the L_{2}(μ)- norm, then the Bernstein inequality (or an improvement of it) supplies a much weaker estimate for the difference of the partial sums corresponding to these two functions than the bound suggested by the central limit theorem. On the other hand, we shall prove a weaker version of Theorem 4.1 in Proposition 6.1 with the help of the chaining argument. This result will be also useful for us.

Proof of Theorem 4.2. Let us list the elements of F as {f_{0}, f_{1}, … } = F, and choose for all p = 0, 1, 2, … a set of functions F_{p} = {f_{a(1,p)}, …, f_{a(m_{p},p)}} ⊂ F with m_{p} ≤ (D + 1) 2^{2pL}σ^{−L} elements in such a way that inf_{1≤j≤m_{p}} ∫ (f − f_{a(j,p)})^{2} dμ ≤ 2^{−4p}σ^{2} for all

f ∈ F, and let f_{p} ∈ F_{p}. For all indices a(j, p) of the functions in F_{p}, p = 1, 2, …, define a predecessor a(j′, p − 1) from the indices of the set of functions F_{p−1} in such a way that the functions f_{a(j,p)} and f_{a(j′,p−1))} satisfy the relation ∫ (f_{(j,p)} − f_{(j′,p−1)})^{2} dμ ≤ 2^{−4(p−1)}σ^{2}. With the help of the behaviour of the standard normal distribution function we can write the estimates

P (A(j, p)) = P (|Z(f_{a(j,p)}) − Z(f_{a(j′,p−1)})| ≥ 2^{−(1+p)}u) ≤ 2 exp \{−\frac{2^{−2(p+1)}u^{2}}{2 · 2^{−4(p−1)}σ^{2}} \}

= 2 exp \{−\frac{2^{2p}u^{2}}{128σ^{2}} \} 1 ≤ j ≤ m_{p}, p = 1, 2, …,

and

P (B(j)) = P (|Z(f_{a(j,0)})| ≥ \frac{u}{2}) ≤ exp \{−\frac{u^{2}}{8σ^{2}} \} , 1 ≤ j ≤ m_{0}.

The above estimates together with the relation ⋃_{p=0}^{∞} F_{p} = F which implies that

{|Z(f)| ≥ u} ⊂ ⋃_{p=1}^{∞} ⋃_{j=1}^{m_{p}} A(j, p) ∪ ⋃_{s=1}^{m_{0}} B(s) for all f ∈ F yield that

P (sup_{f∈F} |Z(f)| ≥ u) ≤ P ( ⋃_{p=1}^{∞} ⋃_{j=1}^{m_{p}} A(j, p) ∪ ⋃_{s=1}^{m_{0}} B(s))

≤ ∑_{p=1}^{∞} ∑_{j=1}^{m_{p}} P (A(j, p)) + ∑_{s=1}^{m_{0}} P (B(s))

≤ ∑_{p=1}^{∞} 2(D + 1)2^{2pL}σ^{−L} exp \{−\frac{2^{2p}u^{2}}{128σ^{2}} \} + 2(D + 1)σ^{−L} exp \{−\frac{u^{2}}{8σ^{2}} \} .

If u ≥ ML^{1/2}σ log^{1/2} \frac{2}{σ} with M ≥ 16 (and L ≥ 1 and 0 < σ ≤ 1), then

2^{2pL}σ^{−L} exp \{−\frac{2^{2p}u^{2}}{256σ^{2}} \} ≤ 2^{2pL}σ^{−L} (\frac{σ}{2} )^{2^{2p}M^{2}L/256} ≤ 2^{−pL} ≤ 2^{−p}

for all p = 0, 1 …, hence the previous inequality implies that

P (sup_{f∈F} |Z(f)| ≥ u) ≤ 2(D + 1) ∑_{p=0}^{∞} 2^{−p} exp \{−\frac{2^{2p}u^{2}}{256σ^{2}} \} = 4(D + 1) exp \{−\frac{u^{2}}{256σ^{2}} \} .

Theorem 4.2 is proved.

With an appropriate choice of the bound of the integrals in the definition of the sets F_{p} in the proof of Theorem 4.2 and some additional calculation it can be proved that the coefficient \frac{1}{256} in the exponent of the right-hand side (4.7) can be replaced by \frac{1−ε}{2} with arbitrary small ε > 0 if the remaining (universal) constants in this estimate are chosen sufficiently large.

The proof of Theorem 4.2 was based on a sufficiently good estimate on the probabilities P (|Z(f) − Z(g)| > u) for pairs of functions f, g ∈ F and numbers u > 0. In the case of Theorem 4.1 only a weaker bound can be given for the corresponding probabilities. There is no good estimate on the tail distribution of the difference S_{n}(f) − S_{n}(g) if its variance is small. As a consequence, the chaining argument supplies only a weaker result in this case. This result, where the tail distribution of the supremum of the normalized random sums S_{n}(f) is estimated on a relatively dense subset of the class of functions f ∈ F in the L_{2}(μ) norm will be given in Proposition 6.1. Another result will be formulated in Proposition 6.2 whose proof is postponed to the next section. It will be shown that Theorem 4.1 follows from Propositions 6.1 and 6.2.

Before the formulation of Proposition 6.1 I recall an estimate which is a simple

consequence of Bernstein’s inequality. If S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}) is the normalized sum of independent, identically random variables, P (|f(ξ_{1})| ≤ 1) = 1, Ef(ξ_{1}) = 0, Ef(ξ_{1})^{2} ≤ σ^{2}, then there exists some constant α > 0 such that

P (|S_{n}(f)| > u) ≤ 2e^{−αu^{2}/σ^{2}} if 0 < u < \sqrt{n}σ^{2}.  (6.1)

Proposition 6.1. Let us have a countable L_{2}-dense class of functions F with parameter D ≥ 1 and exponent L ≥ 1 on a measurable space (X, X) whose elements satisfy relations (4.1), (4.2) and (4.3) with some probability measure μ on (X, X) and real number 0 < σ ≤ 1. Take a sequence of independent, μ-distributed random variables ξ_{1}, …, ξ_{n},

n ≥ 2, and define the normalized random sums S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{l=1}^{n} f(ξ_{l}), for all f ∈ F. Let

us fix some number Ā ≥ 1. There exists some number M = M(Ā) such that with these parameters Ā and M = M(Ā) ≥ 1 the following relations hold.

For all numbers u > 0 such that nσ^{2} ≥ (\frac{u}{σ})^{2} ≥ M(L log \frac{2}{σ} + log D) a number σ̄ = σ̄(u), 0 ≤ σ̄ ≤ σ ≤ 1, and a collection of functions Fσ̄ = {f_{1}, …, f_{m}} ⊂ F with m ≤ Dσ̄^{−L} elements can be chosen in such a way that the sets D_{j} = {f: f ∈

F, ∫ |f − f_{j}|^{2} dμ ≤ σ̄^{2}}, 1 ≤ j ≤ m, satisfy the relation ⋃_{j=1}^{m} D_{j} = F, and the normalized

random sums S_{n}(f), f ∈ Fσ̄, n ≥ 2, satisfy the inequality

P ( sup_{f∈Fσ̄} |S_{n}(f)| ≥ \frac{u}{Ā}) ≤ 4 exp \{−α (\frac{u}{10Āσ} )^{2}\} if nσ^{2} ≥ (\frac{u}{σ})^{2} ≥ M(L log \frac{2}{σ} + log D)

(6.2)

with the constants α in formula (6.1) and the exponent L and parameter D of the L_{2}-dense class F, and also the inequality \frac{1}{16}(\frac{u}{Āσ̄})^{2} ≥ nσ̄^{2} ≥ \frac{1}{64} (\frac{u}{Āσ} )^{2} holds with the number σ̄ = σ̄(u). If the number u satisfies also the inequality

nσ^{2} ≥ (\frac{u}{σ} )^{2} ≥ M (L^{3/2} log \frac{2}{σ} + (log D)^{3/2})  (6.3)

with a sufficiently large number M = M(Ā), then the relation nσ̄^{2} ≥ L log n + log D holds, too.

Proposition 6.1 helps to reduce the proof of Theorem 4.1 to the case when the L_{2}- norm of the functions in the class F is bounded by a relatively small number σ̄. In more detail, the proof of Theorem 4.1 can be reduced to a good estimate on the distribution of the supremum of random variables sup_{f∈D_{j}} |S_{n}(f − f_{j})| for all classes D_{j}, 1 ≤ j ≤ m,

by means of Proposition 6.1. We also have to know that the number m of the classes D_{j} is not too large. Beside this, we need some estimates on the number σ̄ which is the upper bound of the L_{2}-norm of the functions f − f_{j}, f ∈ D_{j}. To get such bounds for σ̄ that we need in the applications of Proposition 6.1 we introduced a large parameter Ā in the formulation of Proposition 6.1 and imposed a condition with a sufficiently large number M = M(Ā) in formula (6.3). This condition reappears in Theorem 4.1 in the conditions of the estimate (4.4).

Let me remark that one of the inequalities the number σ̄ introduced in Proposition 6.1 satisfies has the consequence u > const. \sqrt{n}σ̄^{2} with an appropriate constant,

and we want to estimate the probability P (sup_{f∈F} S_{n}(f)| > u) with this number u and

a class of functions F whose L_{2} norm is bounded by σ̄. Formula (6.1), that will be applied in the proof of Proposition 6.1 holds under the condition u < \sqrt{n}σ^{2}, which is an inequality in the opposite direction. Hence to complete the proof of Theorem 4.1 with the help of Proposition 6.1 we need a result whose proof demands an essentially different method. Proposition 6.2 formulated below is such a result. I shall show that Theorem 4.1 is a consequence of Propositions 6.1 and 6.2. Proposition 6.1 is proved at the end of this section, while the proof of Proposition 6.2 is postponed to the next section.

Proposition 6.2. Let us have a probability measure μ on a measurable space (X, X) together with a sequence of independent and μ distributed random variables ξ_{1}, …, ξ_{n}, n ≥ 2, and a countable, L_{2}-dense class of functions f = f(x) on (X, X) with some parameter D ≥ 1 and exponent L ≥ 1 which satisfies conditions (4.1), (4.2) and (4.3) with some 0 < σ ≤ 1 such that the inequality nσ^{2} > L log n + log D holds. Then there exists a threshold index A_{0} ≥ 5 such that the normalized random sums S_{n}(f), f ∈ F, introduced in Theorem 4.1 satisfy the inequality

P (sup_{f∈F} |S_{n}(f)| ≥ An^{1/2}σ^{2}) ≤ e^{−A^{1/2}nσ^{2}/2} if A ≥ A_{0}.  (6.4)

I did not try to find optimal parameters in formula (6.4). Even the coefficient −A^{1/2} in the exponent at its right-hand side could be improved. The result of Proposition 6.2 is similar to that of Theorem 4.1. Both of them give an estimate on a probability of

the form P (sup_{f∈F} |S_{n}(f)| ≥ u) with some class of functions F. The essential difference

between them is that in Theorem 4.1 this probability is considered for u ≤ n^{1/2}σ^{2} while in Proposition 6.2 the case u = An^{1/2}σ^{2} with A ≥ A_{0} is taken, where A_{0} is a sufficiently large positive number. Let us observe that in this case no good Gaussian type estimate can be given for the probabilities P (S_{n}(f) ≥ u), f ∈ F. In this case Bernstein’s

inequality yields the bound P (S_{n}(f) > An^{1/2}σ^{2}) = P (∑_{l=1}^{n} f(ξ_{l}) > uV_{n}) < e^{−const. Anσ^{2}}

with u = A\sqrt{n}σ and V_{n} = \sqrt{n}σ for each single function f ∈ F which takes part in the supremum of formula (6.4). The estimate (6.4) yields a slightly weaker estimate for the supremum of such random variables, since it contains the coefficient A^{1/2} instead of A in the exponent of the estimate at the right-hand side. But also such a bound will be sufficient for us.

In Proposition 6.2 such a situation is considered when the irregularities of the summands provide a non-negligible contribution to the probabilities P (|S_{n}(f)| ≥ u), and the chaining argument applied in the proof of Theorem 4.2 does not give a good estimate on the probability at the left-hand side of (6.4). This is the reason why we separated the proof of Theorem 4.1 to two different statements given in Proposition 6.1 and 6.2.

In the proof of Theorem 4.1 Proposition 6.1 will be applied with a sufficiently large number Ā ≥ 1 and an appropriate number M = M(Ā) appearing in the formulation of this result. Proposition 6.2 will be applied for the set of functions F = F_{j} = \{\frac{g−f_{j}}{2} : g ∈ D_{j}\} and number σ = σ̄, with the number σ̄, functions f_{j} and sets of functions D_{j} introduced in Proposition 6.1 and with the parameter A_{0} appearing in the formulation of Proposition 6.2. We can write

P (sup_{f∈F} |S_{n}(f)| ≥ u) ≤ P ( sup_{f∈Fσ̄} |S_{n}(f)| ≥ \frac{u}{Ā})  (6.5)

+ ∑_{j=1}^{m} P ( sup_{g∈D_{j}} |S_{n} (\frac{f_{j} − g}{2} )| ≥ (\frac{1}{2} − \frac{1}{2Ā}) u) ,

where m is the cardinality of the set of functions Fσ̄ appearing in Proposition 6.1, which is bounded by m ≤ Dσ̄^{−L}. We want to choose the number Ā in such a way that the inequality (\frac{1}{2} − \frac{1}{2Ā})u ≥ A_{0}\sqrt{n}σ̄^{2} holds, since this enables us to estimate the second term in (6.5) by Proposition 6.2 with the choice A = A_{0}. This inequality is equivalent to nσ̄^{2} ≤ (\frac{1}{2A_{0}} − \frac{1}{2A_{0}Ā})^{2}(\frac{u}{σ̄})^{2}. On the other hand, (\frac{u}{4Āσ̄})^{2} ≥ nσ̄^{2} by Proposition 6.1, hence the desired inequality holds if \frac{1}{2A_{0}} − \frac{1}{2A_{0}Ā} ≥ \frac{1}{4Ā}. Hence with the choice Ā = max(1, \frac{A_{0}+2}{2}) and a sufficiently large M = M(Ā) we can bound both terms at the right-hand side of (6.5) with Propositions (6.1) and (6.2) respectively.

With such a choice of Ā we can write by Proposition 6.2

P ( sup_{g∈D_{j}} |S_{n} (\frac{f_{j} − g}{2} )| ≥ (\frac{1}{2} − \frac{1}{2Ā}) u) ≤ P ( sup_{g∈D_{j}} |S_{n} (\frac{f_{j} − g}{2} )| ≥ A_{0}\sqrt{n}σ̄^{2})
≤ e^{−A_{0}^{1/2}nσ̄^{2}/2} for all 1 ≤ j ≤ m.

(Observe that the set of functions \frac{f_{j}−g}{2} , g ∈ D_{j}, is an L_{2}-dense class with parameter D and exponent L.) Hence Proposition 6.1 together with the bound m ≤ Dσ̄^{−L} and formula 6.5 imply that

P (sup_{f∈F} |S_{n}(f)| ≥ u) ≤ 4 exp \{−α (\frac{u}{10Āσ} )^{2}\} + Dσ̄^{−L}e^{−A_{0}^{1/2}nσ̄^{2}/2}.  (6.6)

To get the estimate in Theorem 4.1 from inequality (6.6) we show that the inequality nσ̄^{2} ≥ L log n + log D (with L ≥ 1, D ≥ 1 and n ≥ 2) which is valid under the conditions of Proposition 6.1 implies that Dσ̄^{−L} ≤ e^{nσ̄^{2}}. Indeed, we have to show that log D + L log \frac{1}{σ̄} ≤ nσ̄^{2}. But we have nσ̄^{2} ≥ L log n ≥ log n, hence \frac{1}{σ̄} ≤ \sqrt{\frac{n}{log n}} ≤ n, thus

log \frac{1}{σ̄} ≤ log n, and log D + L log \frac{1}{σ̄} log D + L log n ≤ nσ̄^{2}, as we claimed.

This inequality together with the inequality nσ̄^{2} ≥ \frac{1}{64}(\frac{u}{Āσ})^{2}, proved in Proposition 6.1 imply that

Dσ̄^{−L}e^{−A_{0}^{1/2}nσ̄^{2}/2} ≤ exp \{− (\frac{A_{0}^{1/2}}{2} − 1) nσ̄^{2}\} ≤ exp \{−\frac{(A_{0}^{1/2} − 2)}{128Ā^{2}} (\frac{u}{σ} )^{2}\} .

Hence relation (6.6) yields that

P (sup_{f∈F} |S_{n}(f)| ≥ u) ≤ 4 exp \{−\frac{α}{100Ā^{2}} (\frac{u}{σ} )^{2}\} + exp \{−\frac{(A_{0}^{1/2} − 2)}{128Ā^{2}} (\frac{u}{σ} )^{2}\} ,

and because of the relation A_{0} ≥ 5 this estimate implies Theorem 4.1. Let me remark that the condition \sqrt{n}σ^{2} ≥ u ≥ Mσ(L^{3/4} log^{1/2} \frac{2}{σ} + (log D)^{3/4}) appears in formula (4.4) because of condition (6.3) imposed in Proposition 6.1. (The parameter M in formula (4.4) can be chosen as the double of the parameter M in (6.3).)

I finish this section with the proof of Proposition 6.1.

Proof of Proposition 6.1. Let us list the members of F, as f_{1}, f_{2}, …, and choose for all p = 0, 1, 2, … a set F_{p} = {f_{a(1,p)}, …, f_{a(m_{p},p)}} ⊂ F with m_{p} ≤ D 2^{2pL}σ^{−L} elements in such a way that inf_{1≤j≤m_{p}} ∫ (f − f_{a(j,p)})^{2} dμ ≤ 2^{−4p}σ^{2} for all f ∈ F. For all indices a(j, p),

p = 1, 2, …, 1 ≤ j ≤ m_{p}, choose a predecessor a(j′, p − 1), j′ = j′(j, p), 1 ≤ j′ ≤ m_{p−1}, in such a way that the functions f_{a(j,p)} and f_{a(j′,p−1)} satisfy the relation ∫ |f_{a(j,p)} −

f_{a(j′,p−1)}|^{2} dμ ≤ σ^{2}2^{−4(p−1)}. Then we have ∫ (\frac{f_{a(j,p)}−f_{a(j′,p−1)}}{2} )^{2} dμ ≤ 4σ^{2}2^{−4p} and

sup_{x_{j}∈X, 1≤j≤k} |\frac{f_{a(j,p)}(x_{1},...,x_{k})−f_{a(j′,p−1)}(x_{1},...,x_{k})}{2} | ≤ 1. Relation (6.1) yields that

P (A(j, p)) = P (\frac{1}{2}|S_{n}(f_{a(j,p)} − f_{a(j′,p−1)})| ≥ \frac{2^{−(1+p)}u}{2Ā}) ≤ 2 exp \{−α (\frac{2^{p}u}{8Āσ} )^{2}\}

if nσ^{2} ≥ 2^{6p} (\frac{u}{16Āσ} )^{2} , 1 ≤ j ≤ m_{p}, p = 1, 2, …,  (6.7)

and

\begin{aligned}P (B(s)) & = P (|S_{n}(f_{s,0})| ≥ \frac{u}{2Ā}) ≤ 2 exp \{−α (\frac{u}{2Āσ} )^{2}\} , 1 ≤ s ≤ m_{0}, \\ & if nσ^{2} ≥ (\frac{u}{2Āσ} )^{2} . \end{aligned}  (6.8)

Choose an integer number R = R(u), R ≥ 1, by the inequality 2^{6(R+1)} (\frac{u}{16Āσ} )^{2} > nσ^{2} ≥ 2^{6R} (\frac{u}{16Āσ} )^{2}, define σ̄^{2} = 2^{−4R}σ^{2} and Fσ̄ = F_{R}. (As nσ^{2} ≥ (\frac{u}{σ})^{2} and Ā ≥ 1 by our conditions, there exists such a number R ≥ 1. The number R was chosen as the largest number p for which the second relation of formula (6.7) holds.) Then the cardinality m of the set Fσ̄ equals m_{R} ≤ D2^{2RL}σ^{−L} = Dσ̄^{−L}, and the sets D_{j} are

D_{j} = {f: f ∈ F, ∫ (f_{a(j,R)} − f)^{2} dμ ≤ 2^{−4R}σ^{2}}, 1 ≤ j ≤ m_{R}, hence ⋃_{j=1}^{m} D_{j} = F.

Beside this, with our choice of the number R inequalities (6.7) and (6.8) can be applied for 1 ≤ p ≤ R. Hence the definition of the predecessor of an index (j, p) implies that

\{ω: sup_{f∈Fσ̄} |S_{n}(f)(ω)| ≥ \frac{u}{Ā}\} ⊂ ⋃_{p=1}^{R} ⋃_{j=1}^{m_{p}} A(j, p) ∪ ⋃_{s=1}^{m_{0}} B(s), and

P ( sup_{f∈Fσ̄} |S_{n}(f)| ≥ \frac{u}{Ā}) ≤ P ( ⋃_{p=1}^{R} ⋃_{j=1}^{m_{p}} A(j, p) ∪ ⋃_{s=1}^{m_{0}} B(s))

≤ ∑_{p=1}^{R} ∑_{j=1}^{m_{p}} P (A(j, p)) + ∑_{s=1}^{m_{0}} P (B(s)) ≤ ∑_{p=1}^{∞} 2D 2^{2pL}σ^{−L} exp \{−α (\frac{2^{p}u}{8Āσ} )^{2}\}

+ 2Dσ^{−L} exp \{−α (\frac{u}{2Āσ} )^{2}\} .

If the relation (\frac{u}{σ})^{2} ≥ M(L log \frac{2}{σ} + log D) holds with a sufficiently large constant M (depending on Ā), and σ ≤ 1, then the inequalities

D2^{2pL}σ^{−L} exp \{−α (\frac{2^{p}u}{8Āσ} )^{2}\} ≤ 2^{−p} exp \{−α (\frac{2^{p}u}{10Āσ} )^{2}\}

hold for all p = 1, 2, …, and

Dσ^{−L} exp \{−α (\frac{u}{2Āσ} )^{2}\} ≤ exp \{−α (\frac{u}{10Āσ} )^{2}\} .

Hence the previous estimate implies that

P ( sup_{f∈Fσ̄} |S_{n}(f)| ≥ \frac{u}{Ā}) ≤ ∑_{p=1}^{∞} 2 · 2^{−p} exp \{−α (\frac{2^{p}u}{10Āσ} )^{2}\}

+ 2 exp \{−α (\frac{u}{10Āσ} )^{2}\} ≤ 4 exp \{−α (\frac{u}{10Āσ} )^{2}\} ,

and relation (6.2) holds.

As σ^{2} = 2^{4R}σ̄^{2} the inequality

2^{−4R} · \frac{2^{6R}}{256} (\frac{u}{Āσ} )^{2} ≤ nσ̄^{2} = 2^{−4R}nσ^{2} ≤ 2^{−4R} · \frac{2^{6(R+1)}}{256} (\frac{u}{Āσ} )^{2} = \frac{1}{4} · 2^{−2R} (\frac{u}{Āσ̄} )^{2}

holds, and this implies (together with the relation R ≥ 1) that

\frac{1}{64} (\frac{u}{Āσ} )^{2} ≤ nσ̄^{2} ≤ \frac{1}{16} (\frac{u}{Āσ̄} )^{2} ,

as we have claimed. It remained to show that under the condition (6.3) nσ̄^{2} ≥ L log n + log D.

This inequality clearly holds under the conditions of Proposition 6.1 if σ ≤ n^{−1/3}, since in this case log \frac{2}{σ} ≥ \frac{log n}{3}, and nσ̄^{2} ≥ \frac{1}{64}(\frac{u}{Āσ})^{2} ≥ \frac{1}{64Ā^{2}} M(L^{3/2} log \frac{2}{σ} + (log D)^{3/2}) ≥ \frac{1}{192Ā^{2}} M(L^{3/2} log n + (log D)^{3/2}) ≥ L log n + log D if M ≥ M_{0}(Ā) with a sufficiently large number M_{0}(Ā).

If σ ≥ n^{−1/3}, we can exploit that the inequality 2^{6R} (\frac{u}{Āσ} )^{2} ≤ 256nσ^{2} holds because

of the definition of the number R. It can be rewritten as 2^{−4R} ≥ 2^{−16/3} [\frac{(\frac{u}{Āσ} )2}{nσ^{2}} ]^{2/3}.

Hence nσ̄^{2} = 2^{−4R}nσ^{2} ≥ \frac{2^{−16/3}}{Ā^{4/3}} (nσ^{2})^{1/3} (\frac{u}{σ})^{4/3}. As log \frac{2}{σ} ≥ log 2 > \frac{1}{2} the inequalities nσ^{2} ≥ n^{1/3} and (\frac{u}{σ})^{2} ≥ M(L^{3/2} log \frac{2}{σ} + (log D)^{3/2}) ≥ \frac{M}{2} (L^{3/2} + (log D)^{3/2}) hold. They yield that

nσ̄^{2} ≥ \frac{Ā^{−4/3}}{50} (nσ^{2})^{1/3} (\frac{u}{σ} )^{4/3} ≥ \frac{Ā^{−4/3}}{50} n^{1/9} (\frac{M}{2} )^{2/3} (L^{3/2} + (log D)^{3/2})^{2/3}

≥ \frac{M^{2/3}n^{1/9}(L + log D)}{100Ā^{4/3}} ≥ L log n + log D

if M = M(Ā) is chosen sufficiently large.

7. The completion of the proof of Theorem 4.1.

This section contains the proof of Proposition 6.2 with the help of a symmetrization argument which completes the proof of Theorem 4.1. By symmetrization argument I mean the reduction of the investigation of sums of the form ∑ f(ξ_{j}) to sums of the form ∑ ε_{j}f(x_{j}), where ε_{j} are independent random variables, independent also of the random variables ξ_{j}, and P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}. First a symmetrization lemma is proved, and then with the help of this result and a conditioning argument the proof of Proposition 6.2 is reduced to the estimation of a probability which can be bounded by means of the Hoeffding inequality formulated in Theorem 3.4. Such an approach makes possible to prove Proposition 6.2.

First I formulate the symmetrization lemma we shall apply.

Lemma 7.1. (Symmetrization Lemma). Let Z_{n} and Z̄_{n}, n = 1, 2, …, be two sequences of random variables independent of each other, and let the random variables Z̄_{n}, n = 1, 2, …, satisfy the inequality

P (|Z̄_{n}| ≤ α) ≥ β for all n = 1, 2, …  (7.1)

with some numbers α ≥ 0 and β ≥ 0. Then

P ( sup_{1≤n<∞} |Z_{n}| > u + α) ≤ \frac{1}{β} P ( sup_{1≤n<∞} |Z_{n} − Z̄_{n}| > u) for all u > 0.

Proof of Lemma 7.1. Put τ = min{n: |Z_{n}| > u + α} if there exists such an index n, and τ = 0 otherwise. Then the event {τ = n} is independent of the sequence of random variables Z̄_{1}, Z̄_{2}, … for all n = 1, 2, …, and because of this independence

P (\{τ = n\}) ≤ \frac{1}{β} P (\{τ = n\} ∩ \{|Z̄_{n}| ≤ α\}) ≤ \frac{1}{β} P (\{τ = n\} ∩ \{|Z_{n} − Z̄_{n}| > u\})

for all n = 1, 2, . . .. Hence

P ( sup_{1≤n<∞} |Z_{n}| > u + α) = ∑_{l=1}^{∞} P (τ = l) ≤ \frac{1}{β} ∑_{l=1}^{∞} P (\{τ = l\} ∩ \{|Z_{l} − Z̄_{l}| > u\})

≤ \frac{1}{β} ∑_{l=1}^{∞} P (\{τ = l\} ∩ sup_{1≤n<∞} |Z_{n} − Z̄_{n}| > u\}) ≤ \frac{1}{β} P ( sup_{1≤n<∞} |Z_{n} − Z̄_{n}| > u) .

Lemma 7.1 is proved.

We shall apply the following consequence Lemma 7.2 of the symmetrization lemma.

Lemma 7.2. Let us fix a countable class of functions F on a measurable space (X, X) together with a real number 0 < σ < 1. Consider a sequence of independent and identically distributed random variables ξ_{1}, …, ξ_{n} with values in the space (X, X) such that Ef(ξ_{1}) = 0, Ef^{2}(ξ_{1}) ≤ σ^{2} for all f ∈ F together with another sequence ε_{1}, …, ε_{n} of independent random variables with distribution P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, independent also of the random sequence ξ_{1}, …, ξ_{n}. Then

P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} f(ξ_{j})| ≥ An^{1/2}σ^{2})

≤ 4P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2}) if A ≥ \frac{3\sqrt{2}}{\sqrt{n}σ} .

(7.2)

Proof of Lemma 7.2. Let us construct an independent copy ξ̄_{1}, …, ξ̄_{n} of the sequence ξ_{1}, …, ξ_{n} in such a way that all three sequences ξ_{1}, …, ξ_{n}, ξ̄_{1}, …, ξ̄_{n} and ε_{1}, …, ε_{n}

are independent. Define the random variables S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}) and S̄_{n}(f) =

\frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ̄_{j}) for all f ∈ F. The inequality

P (sup_{f∈F} |S_{n}(f)| > A\sqrt{n}σ^{2}) ≤ 2P (sup_{f∈F} |S_{n}(f) − S̄_{n}(f)| > \frac{2}{3}A\sqrt{n}σ^{2}) .  (7.3)

follows from Lemma 7.1 if it is applied for the countable set of random variables Z_{n}(f) = S_{n}(f) and Z̄_{n}(f) = S̄_{n}(f), f ∈ F, and the numbers u = \frac{2}{3}A\sqrt{n}σ^{2} and α = \frac{1}{3}A\sqrt{n}σ^{2}, since the random fields S_{n}(f) and S̄_{n}(f) are independent, and P (|S̄_{n}(f)| ≤ α) > \frac{1}{2} for all f ∈ F. Indeed, α = \frac{1}{3}A\sqrt{n}σ^{2} ≥ \sqrt{2}σ, ES̄_{n}(f)^{2} ≤ σ^{2}, thus Chebishev’s inequality implies that P (|S̄_{n}(f)| ≤ α) ≥ P (|S̄_{n}(f)| ≤ \sqrt{2}σ) ≥ \frac{1}{2} for all f ∈ F.

Let us observe that the random field

S_{n}(f) − S̄_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} (f(ξ_{j}) − f(ξ̄_{j})) , f ∈ F,  (7.4)

and its randomization

\frac{1}{\sqrt{n}} ∑_{j=1}^{n} ε_{j} (f(ξ_{j}) − f(ξ̄_{j})) , f ∈ F,  (7.4′)

have the same distribution. Indeed, even the conditional distribution of (7.4′) under the condition that the values of the ε_{j}-s are prescribed agrees with the distribution of (7.4) for all possible values of the ε_{j}-s. This follows from the observation that the distribution of the random field (7.4) does not change if we exchange the random variables ξ_{j} and ξ̄_{j} for those indices j for which ε_{j} = −1 and do not change them for those indices j for which ε_{j} = 1. On the other hand, the distribution of the random field obtained in such a way agrees with the conditional distribution of the random field defined in (7.4′) under the condition that the values of the random variables ε_{j} are prescribed.

The above relation together with formula (7.3) imply that

P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} f(ξ_{j})| ≥ An^{1/2}σ^{2})

≤ 2P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j} [f(ξ_{j}) − f̄(ξ_{j})]| ≥ \frac{2}{3}An^{1/2}σ^{2})

≤ 2P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2})

+ 2P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ̄_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2})

= 4P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2}) .

Lemma 7.2 is proved.

Let me briefly explain the method of proof of Proposition 6.2. A probability of the

form P (n^{−1/2} sup_{f∈F} | ∑_{j=1}^{n} f(ξ_{j})| > u) has to be estimated. Lemma 7.2 enables us to re-

place this problem by the estimation of the probability P (n^{−1/2} sup_{f∈F} | ∑_{j=1}^{n} ε_{j}f(ξ_{j})| > \frac{u}{3})

with some independent random variables ε_{j}, P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, j = 1, …, n, which are also independent of the random variables ξ_{j}. We shall bound the conditional probability of the event appearing in this modified problem under the condition that the values of the random variables ξ_{j} have prescribed values. This can be done with the help of Hoeffding’s inequality formulated in Theorem 3.4 and the L_{2}-density property of the class of functions F we consider. We hope to get a sharp estimate in such a way which is similar to the result we got in the study of the Gaussian counterpart of this problem, because Hoeffding’s inequality yields always a Gaussian type upper bound for the tail distribution of the random sum we are studying.

Nevertheless, there appears a problem when we try to apply such an approach. To get a good estimate on the conditional tail distribution of the supremum of the random sums we are studying with the help of Hoeffding’s inequality we need a good estimate on the conditional variances of the random sums we are studying, i.e. on the

tail distribution of sup_{f∈F} \frac{1}{n} ∑_{j=1}^{n} f^{2}(ξ_{j}). This problem is similar to the original one, and it

is not simpler.

But a more detailed study shows that our approach to get a good estimate with the help of Hoeffding’s inequality works. In comparing our original problem with the new, complementary problem we have to understand at which level we need a good estimate on the tail distribution of the supremum in the complementary problem to get a good tail distribution estimate at level u in the original problem. A detailed study shows that to bound the probability in the original problem with parameter u we have to estimate

the probability P (n^{−1/2} sup_{f∈F′} | ∑_{j=1}^{n} f(ξ_{j})| > u^{1+α}) with some new nice, appropriately

defined L_{2}-dense class of bounded functions F′ and some number α > 0. We shall exploit that the number u is replaced by a larger number u^{1+α} in the new problem. Let us also observe that if the sum of bounded random variables is considered, then for very large values u the probability we investigate equals zero. On the basis of these observations an appropriate backward induction procedure can be worked out. In its

n-th step we give a good upper bound on the probability P (n^{−1/2} sup_{f∈F} | ∑_{j=1}^{n} f(ξ_{j})| > u)

if u ≥ T_{n} with an appropriately chosen number T_{n}, and try to diminish the number T_{n} in each step of this induction procedure. We can prove Proposition 6.2 as a consequence of the result we get by means of this backward induction procedure. To work out the details we introduce the following notion.

Definition of good tail behaviour for a class of normalized random sums. Let us have some measurable space (X, X) and a probability measure μ on it together with some integer n ≥ 2 and real number σ > 0. Consider some class F of functions f(x) on the space (X, X), and take a sequence of independent and μ distributed random variables ξ_{1}, …, ξ_{n} with values in the space (X, X). Define the normalized random sums

S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}), f ∈ F. Given some real number T > 0 we say that the set of

normalized random sums S_{n}(f), f ∈ F, has a good tail behaviour at level T (with parameters n and σ^{2} which will be fixed in the sequel) if the inequality

P (sup_{f∈F} |S_{n}(f)| ≥ A\sqrt{n}σ^{2}) ≤ exp \{−A^{1/2}nσ^{2}\}  (7.5)

holds for all numbers A > T.

Now I formulate Proposition 7.3 and show that Proposition 6.2 follows from it.

Proposition 7.3. Let us fix a positive integer n ≥ 2, a real number 0 < σ ≤ 1 and a probability measure μ on a measurable space (X, X) together with some numbers L ≥ 1 and D ≥ 1 such that nσ^{2} ≥ L log n + log D. Let us consider those countable L_{2}-dense classes F of functions f = f(x) on the space (X, X) with exponent L and parameter D for which all functions f ∈ F satisfy the conditions sup_{x∈X} |f(x)| ≤ \frac{1}{4}, ∫ f(x)μ( dx) = 0

and ∫ f^{2}(x)μ( dx) ≤ σ^{2}.

Let a number T > 1 be such that for all classes of functions F which satisfy the above

conditions the set of normalized random sums S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}), f ∈ F, defined with

the help of a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{n} have a good tail behaviour at level T^{4/3}. There is a universal constant Ā_{0} such that if T ≥ Ā_{0}, then the set of the above defined normalized sums, S_{n}(f), f ∈ F, have a good tail behaviour for all such classes of functions F not only at level T^{4/3} but also at level T.

Proposition 6.2 simply follows from Proposition 7.3. To show this let us first observe that a class of normalized random sums S_{n}(f), f ∈ F, has a good tail behaviour at level T_{0} = \frac{1}{4σ^{2}} if this class of functions F satisfies the conditions of Proposition 7.3. Indeed,

in this case P (sup_{f∈F} |S_{n}(f)| ≥ A\sqrt{n}σ^{2}) ≤ P (sup_{f∈F} |S_{n}(f)| > \frac{\sqrt{n}}{4}) = 0 for all A > T_{0}.

Then the repetitive application of Proposition 7.3 yields that a class of random sums S_{n}(f), f ∈ F, has a good tail behaviour at all levels T ≥ T_{0}^{(3/4)^{j}} with an index j such that T_{0}^{(3/4)^{j}} ≥ Ā_{0} if the class of functions F satisfies the conditions of Proposition 7.3. Hence it has a good tail behaviour for T = Ā_{0}^{4/3}. If a class of functions f ∈ F satisfies the conditions of Proposition 6.2, then the class of functions F̄ = \{f̄ = \frac{f}{4} : f ∈ F\} satisfies the conditions of Proposition 7.3, with the same parameters σ, L and D. (Actually some of the inequalities that must hold for the elements of a class of functions F satisfying the conditions of Proposition 7.3 are valid with smaller parameters. But we did not change these parameters to satisfy also the condition nσ^{2} ≥ L log n + log D.) Hence the class of functions S_{n}(f̄), f̄ ∈ F̄, has a good tail behaviour at level T = Ā_{0}^{4/3}. This implies that the original class of functions F satisfies formula (6.4) in Proposition 6.2, and this is what we had to show.

Proof of Proposition 7.3. Fix a class of functions F which satisfies the conditions of Proposition 7.3 together with two independent sequences ξ_{1}, …, ξ_{n} and ε_{1}, …, ε_{n} of independent random variables, where ξ_{j} is μ-distributed, P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, and investigate the conditional probability

P (f, A|ξ_{1}, …, ξ_{n}) = P ( \frac{1}{\sqrt{n}} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{6}\sqrt{n}σ^{2}| ξ_{1}, …, ξ_{n})

for all functions f ∈ F, A > T and values (ξ_{1}, …, ξ_{n}) in the condition. By the Hoeffding inequality formulated in Theorem 3.4

P (f, A|ξ_{1}, …, ξ_{n}) ≤ 2 exp \{−\frac{\frac{1}{36}A^{2}nσ^{4}}{2S̄^{2}(f, ξ_{1}, …, ξ_{n})}\}  (7.6)

with

S̄^{2}(f, x_{1}, …, x_{n}) = \frac{1}{n} ∑_{j=1}^{n} f^{2}(x_{j}), f ∈ F.

Let us introduce the set

H = H(A) = {(x_{1}, …, x_{n}): sup_{f∈F} S̄^{2}(f, x_{1}, …, x_{n}) ≥ (1 + A^{4/3}) σ^{2}} .  (7.7)

I claim that

P ((ξ_{1}, …, ξ_{n}) ∈ H) ≤ e^{−A^{2/3}nσ^{2}} if A > T.  (7.8)

(The set H plays the role of the small exceptional set, where we cannot provide a good estimate for P (f, A|ξ_{1}, …, ξ_{n}) for some f ∈ F.)

To prove relation (7.8) let us consider the functions f̄ = f̄(f), f̄(x) = f^{2}(x) − ∫ f^{2}(x)μ( dx), and introduce the class of functions F̄ = {f̄(f): f ∈ F}. Let us show that the class of functions F̄ satisfies the conditions of Proposition 7.3, hence the estimate (7.5) holds for the class of functions F̄ if A > T^{4/3}.

The relation ∫ f̄(x)μ( dx) = 0 clearly holds. The condition sup |f̄(x)| ≤ \frac{1}{8} < \frac{1}{4} also holds if sup |f(x)| ≤ \frac{1}{4}, and ∫ f̄^{2}(x)μ( dx) ≤ ∫ f^{4}(x)μ( dx) ≤ \frac{1}{16} ∫ f^{2}(x) μ( dx) ≤ \frac{σ^{2}}{16} < σ^{2} if f ∈ F. It remained to show that F̄ is an L_{2}-dense class with exponent L and parameter D. For this goal we need a good estimate on ∫ (f̄(x) − ḡ(x))^{2}ρ( dx), where f̄, ḡ ∈ F̄, and ρ is an arbitrary probability measure.

Observe that ∫ (f̄(x) − ḡ(x))^{2}ρ( dx) ≤ 2 ∫ (f^{2}(x) − g^{2}(x))^{2}ρ( dx) + 2 ∫ (f^{2}(x) − g^{2}(x))^{2}μ( dx) ≤ 2(sup(|f(x)| + |g(x)|)^{2} (∫ (f(x) − g(x))^{2}(ρ( dx) + μ( dx)) ≤ ∫ (f(x) − g(x))^{2}ρ̄( dx) for all f, g ∈ F, f̄ = f̄(f), ḡ = ḡ(g) and probability measure ρ, where ρ̄ = \frac{ρ+μ}{2}. This means that if \{f_{1}, …, f_{m}\} is an ε-dense subset of F in the space L_{2}(X, X , ρ̄), then {f̄_{1}, …, f̄_{m}} is an ε-dense subset of F̄ in the space L_{2}(X, X , ρ), and not only F, but also F̄ is an L_{2}-dense class with exponent L and parameter D.

An application of the conditions of Proposition 7.3 for the number A^{4/3} > T^{4/3} and the class of functions F̄ yields that

P ((ξ_{1}, …, ξ_{n}) ∈ H) = P (sup_{f∈F} (\frac{1}{n} ∑_{j=1}^{n} f̄(f)(ξ_{j}) + \frac{1}{n} ∑_{j=1}^{n} Ef^{2}(ξ_{j})) ≥ (1 + A^{4/3}) σ^{2})

≤ P (sup_{f̄∈F̄} \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f̄(ξ_{j}) ≥ A^{4/3}n^{1/2}σ^{2}) ≤ e^{−A^{2/3}nσ^{2}},

i.e. relation (7.8) holds.

By formula (7.6) and the definition of the set H given in (7.7) the estimate

P (f, A|ξ_{1}, …, ξ_{n}) ≤ 2e^{−A^{2/3}nσ^{2}/144} if (ξ_{1}, …, ξ_{n}) ∉ H  (7.9)

holds for all f ∈ F and A > T ≥ 1. (Here we used the estimate 1 + A^{4/3} ≤ 2A^{4/3}.) Let us introduce the conditional probability

P (F, A|ξ_{1}, …, ξ_{n}) = P ( sup_{f∈F} \frac{1}{\sqrt{n}} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}\sqrt{n}σ^{2}| ξ_{1}, …, ξ_{n})

for all (ξ_{1}, …, ξ_{n}) and A > T. We shall estimate this conditional probability with the help of relation (7.9) if (ξ_{1}, …, ξ_{n}) ∉ H.

Given a vector x^{(n)} = (x_{1}, …, x_{n}) ∈ X^{n}, let us introduce the measure ν = ν(x_{1}, …, x_{n}) = ν(x^{(n)}) on (X, X) which is concentrated in the coordinates of the vector x^{(n)} = (x_{1}, …, x_{n}), and ν(\{x_{j}\}) = \frac{1}{n} for all points x_{j}, j = 1, …, n. If

∫ f^{2}(u)ν( du) ≤ δ^{2} for a function f, then |\frac{1}{\sqrt{n}} ∑_{j=1}^{n} ε_{j}f(x_{j})| ≤ n^{1/2} ∫ |f(u)|ν( du) ≤ n^{1/2}δ.

As a consequence, we can write that

|\frac{1}{\sqrt{n}} ∑_{j=1}^{n} ε_{j}f(x_{j}) − \frac{1}{\sqrt{n}} ∑_{j=1}^{n} ε_{j}g(x_{j})| ≤ \frac{A}{6}\sqrt{n}σ^{2} if ∫ (f(u) − g(u))^{2} dν(u) ≤ (\frac{Aσ^{2}}{6} )^{2} .

(7.10)

Let us list the elements of the (countable) set F as F = {f_{1}, f_{2}, … }, fix a number δ = \frac{Aσ^{2}}{6} , and choose for all vectors x^{(n)} = (x_{1}, …, x_{n}) ∈ X^{n} a sequence of indices p_{1}(x^{(n)}), …, p_{m}(x^{(n)}) taking positive integer values with m = max(1, Dδ^{−L}) = max(1, D(\frac{6}{Aσ^{2}} )^{L}) elements in such a way that inf_{1≤l≤m} ∫ (f(u)−f_{p_{l}(x^{(n)})}(u))^{2} dν(x^{(n)})(u) ≤

δ^{2} for all f ∈ F and x^{(n)} ∈ X^{n} with the above defined measure ν(x^{(n)}) on the space (X, X). This is possible because of the L_{2}-dense property of the class of functions F. (This is the point where the L_{2}-dense property of the class of functions F is exploited in its full strength.) In a complete proof of Theorem 7.3 we still have to show that we can choose the indices p_{j}(x^{(n)}), 1 ≤ j ≤ m, as measurable functions of their argument x^{(n)} on the space (X^{n}, X^{n}). We shall show this in Lemma 7.4 at the end of the proof.

Put ξ^{(n)}(ω) = (ξ_{1}(ω), …, ξ_{n}(ω)). Because of relation (7.10), the choice of the number δ and the property of the functions f_{p_{l}(x^{(n)})}(·) we have

\{ω: sup_{f∈F} \frac{1}{\sqrt{n}} |∑_{j=1}^{n} ε_{j}(ω)f(ξ_{j}(ω))| ≥ \frac{A}{3}\sqrt{n}σ^{2}\}

⊂ ⋃_{l=1}^{m} \{ω: \frac{1}{\sqrt{n}} |∑_{j=1}^{n} ε_{j}(ω)f_{p_{l}(ξ^{(n)}(ω))}(ξ_{j}(ω))| ≥ \frac{A}{6}\sqrt{n}σ^{2}\} .

This relation together with inequality (7.9) yield that

P (F, A|ξ_{1}, …, ξ_{n}) ≤ ∑_{l=1}^{m} P (f_{p_{l}(ξ^{(n)})}, A|ξ_{1}, …, ξ_{n})

≤ 2 max (1, D (\frac{6}{Aσ^{2}} )^{L}) e^{−A^{2/3}nσ^{2}/144}

if (ξ_{1}, …, ξ_{n}) ∉ H and A > T.

If A ≥ Ā_{0} with a sufficiently large constant Ā_{0}, then this inequality together with Lemma 7.2 and the estimate (7.8) imply that

P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} f(ξ_{j})| ≥ An^{1/2}σ^{2}) ≤ 4P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2})

≤ max (4, 8D (\frac{6}{Aσ^{2}} )^{L}) e^{−A^{2/3}nσ^{2}/144} + 4e^{−A^{2/3}nσ^{2}} if A > T.

(7.11)

By the conditions of Proposition 7.3 the inequalities nσ^{2} ≥ L log n + log D hold with some L ≥ 1, D ≥ 1 and n ≥ 2. This implies that nσ^{2} ≥ L log 2 ≥ \frac{1}{2}, (\frac{6}{Aσ^{2}} )^{L} ≤ (\frac{n}{2nσ^{2}} )^{L} ≤ n^{L} = e^{L log n} ≤ e^{nσ^{2}} if A ≥ Ā_{0} with some sufficiently large constant Ā_{0} > 0, and 2D = e^{log 2+log D} ≤ e^{3nσ^{2}}. Hence the first term at the right-hand side of (7.11) can be bounded by

max (4, 8D (\frac{6}{Aσ^{2}} )^{L}) e^{−A^{2/3}nσ^{2}/144} ≤ e^{−A^{2/3}nσ^{2}/144} · 4e^{4nσ^{2}} ≤ \frac{1}{2}e^{−A^{1/2}nσ^{2}}

if A ≥ Ā_{0} with a sufficiently large Ā_{0}. The second term at the right-hand side of (7.11) can also be bounded as 4e^{−A^{2/3}nσ^{2}} ≤ \frac{1}{2}e^{−A^{1/2}nσ^{2}} with an appropriate choice of the number Ā_{0}.

By the above calculation formula (7.11) yields the inequality

P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} f(ξ_{j})| ≥ An^{1/2}σ^{2}) ≤ e^{−A^{1/2}nσ^{2}}

if A > T, and the constant Ā_{0} is chosen sufficiently large.

To complete the proof of Proposition 7.3 we still prove the following Lemma 7.4 together with some of its generalizations needed in the proof of Propositions 15.3 and 15.4. The latter results are those multivariate versions of Proposition 7.3 that we need in the proof of the multivariate version of Proposition 6.2. We formulated them not in their most general possible form, but in the way as we need them in this work.

Lemma 7.4. Let F = {f_{1}, f_{2}, … } be a countable and L_{2}-dense class of functions with some exponent L > 0 and parameter D ≥ 1 on a measurable space (X, X). Fix some positive integer n, and define for all x^{(n)} = (x_{1}, …, x_{n}) ∈ X^{n} the probability measure ν(x^{(n)}) = ν(x_{1}, …, x_{n}) on the space (X, X) by the formula ν(x^{(n)})(x_{j}) = \frac{1}{n}, 1 ≤ j ≤ n. For a number 0 ≤ ε ≤ 1 put m = m(ε) = [Dε^{−L}], where [·] denotes integer part. For all 0 ≤ ε ≤ 1 there exists m = m(ε) measurable functions p_{l}(x^{(n)}), 1 ≤ l ≤ m, on the measurable space (X^{n}, X^{n}) with positive integer values in such a way

that inf_{1≤l≤m} ∫ (f(u) − f_{p_{l}(x^{(n)})}(u))^{2}ν(x^{(n)})( du) ≤ ε^{2} for all x^{(n)} ∈ X^{n} and f ∈ F.

In the proof of Proposition 15.3 we need the following result. Let F = {f_{1}, f_{2}, … } be a countable and L_{2}-dense class of functions with some exponent L > 0 and parameter D ≥ 1 on the k-fold product (X^{k}, X^{k}) of a measurable space (X, X) with some k ≥ 1. Fix some positive integer n, and define for all vectors x^{(n)} = (x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) ∈ X^{kn}, where x_{l}^{(j)} ∈ X for all j and l the probability measure ρ(x^{(n)}) on the space (X^{k}, X^{k}) by the formula ρ(x^{(n)})(x_{l_{j}}^{(j)}, 1 ≤ j ≤ k, 1 ≤ l_{j} ≤ n) = \frac{1}{n^{k}} for all sequences

(x_{l_{1}}^{(1)}, …, x_{l_{k}}^{(k)}) , 1 ≤ j ≤ k, 1 ≤ l_{j} ≤ n, with coordinates of the elements of the vector x^{(n)} = (x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k). For all 0 ≤ ε ≤ 1 there exist m = m(ε) = [Dε^{−L}] measurable functions p_{r}(x^{(n)}), 1 ≤ r ≤ m, on the measurable space (X^{kn}, X^{kn}) with positive integer values in such a way that inf_{1≤r≤m} ∫ (f(u) − f_{p_{r}(x^{(n)})}(u))^{2}ρ(x^{(n)})( du) ≤ ε^{2}

for all x^{(n)} ∈ X^{kn} and f ∈ F.

In the proof of Proposition 15.4 we need the following result. Let F = {f_{1}, f_{2}, … } be a countable and L_{2}-dense class of functions with some exponent L > 0 and parameter D ≥ 1 on the product space (X^{k} × Y, X^{k} × Y) with some measurable spaces (X, X) and (Y, Y) and integer k ≥ 1. Fix some positive integer n, and define for all vectors x^{(n)} = (x_{l}^{(j,1)}, x_{l}^{(j,−1)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) ∈ X^{2kn}, where x_{l}^{(j,±1)} ∈ X for all j and l a probability measure α(x^{(n)}) in the space (X^{k} ×Y, X^{k} ×Y) in the following way. Fix some probability measure ρ on the space (Y, Y) and two ±1 sequences ε_{1}^{(k)} = (ε_{1,1}, …, ε_{k,1}) and ε_{2}^{(k)} = (ε_{1,2}, …, ε_{k,2}) of length k. Define with their help first the following probability measures α_{1}(x^{(n)}) = α_{1}(x^{(n)}, ε_{1}^{(k)}, ε_{2}^{(k)}, ρ) and α_{2}(x^{(n)}) = α_{2}(x^{(n)}, ε_{1}^{(k)}, ε_{2}^{(k)}, ρ) on (X^{k} × Y, X^{k} × Y) for all x^{(n)} ∈ X^{2kn}. Let α_{1}(x^{(n)})({x_{l_{1}}^{(1,ε_{1,1})}} × ⋯ × {x_{l_{k}}^{(k,ε_{k,1})}} × B) =

\frac{ρ(B)}{n^{k}} and α_{2}(x^{(n)})(\{x_{l_{1}}^{(1,ε_{1,2})}\} × ⋯ × \{x_{l_{k}}^{(k,ε_{k,2})}\} × B) = \frac{ρ(B)}{n^{k}} with 1 ≤ l_{j} ≤ n for

all 1 ≤ j ≤ k and B ∈ Y if x_{l_{j}}^{(j,ε_{j,1})} and x_{l_{j}}^{(j,ε_{j,2})} are the appropriate coordinates

of the vector x^{(n)} ∈ X^{2kn}. Put α(x^{(n)}) = \frac{α_{1}(x^{(n)})+α_{2}(x^{(n)})}{2}. For all 0 ≤ ε ≤ 1 there exist m = m(ε) = [Dε^{−L}] measurable functions p_{r}(x^{(n)}), 1 ≤ r ≤ m, on the measurable space (X^{2kn}, X^{2kn}) with positive integer values in such a way that

inf_{1≤r≤m} ∫ (f(u) − f_{p_{r}(x^{(n)})}(u))^{2}α(x^{(n)})( du) ≤ ε^{2} for all x^{(n)} ∈ X^{2kn} and f ∈ F.

Proof of Lemma 7.4. Fix some 0 < ε ≤ 1, put the number m = m(ε) introduced in the lemma, and let us list the set of all vectors (j_{1}, …, j_{m}) of length m with positive integer coordinates in some way. Define for all of these vectors (j_{1}, …, j_{m}) the set B(j_{1}, …, j_{m}) ⊂ X^{n} in the following way. We have x^{(n)} = (x_{1}, …, x_{n}) ∈ B(j_{1}, …, j_{m}) if and only if inf_{1≤r≤m} ∫ (f(u) − f_{j_{r}}(u))^{2} dν(x^{(n)})(u) ≤ ε^{2} for all f ∈ F. Then all sets

B(j_{1}, …, j_{m}) are measurable, and ⋃_{(j_{1},...,j_{m})} B(j_{1}, …, j_{m}) = X^{n} because F is an L_{2}-

dense class of functions with exponent L and parameter D. Given a point x^{(n)} = (x_{1}, …, x_{n}) let us choose the first vector (j_{1}, …, j_{m}) = (j_{1}(x^{(n)}), …, j_{m}(x^{(n)})) in our list of vectors for which x^{(n)} ∈ B(j_{1}, …, j_{m}), and define p_{l}(x^{(n)}) = j_{l}(x^{(n)}) for all 1 ≤ l ≤ m with this vector (j_{1}, …, j_{m}). Then the functions p_{l}(x^{(n)}) are measurable, and the functions f_{p_{l}(x^{(n)})}, 1 ≤ l ≤ m, defined with their help together with the probability measures ν(x^{(n)}) satisfy the inequality demanded in the first statement of Lemma 7.4.

The proof of the remaining two statements is almost the same. We only have to modify the definition of the sets B(j_{1}, …, j_{m}) in a natural way. The space of arguments x^{(n)} are the spaces X^{kn} and X^{2kn} in these two cases, and we have to integrate with respect to the measures ρ(x^{(n)}) in the space X^{k} and with respect to the measures α(x^{(n)}) in the space X^{k} × Y respectively. The sets B(j_{1}, …, j_{m}) are measurable also in these cases, and the rest of the proof can be applied without any change.

8. Formulation of the main results of this work.

Former sections of this work contain estimates about the tail distribution of normalized sums of independent, identically distributed random variables and about the tail distribution of the supremum of appropriate classes of such random sums. We have also formulated some estimates about the tail distribution of the integral of a (deterministic) function or of the supremum of an appropriate class of functions with respect to a normalized empirical distribution. These two kinds of problems are closely related. To understand them better we have also considered their natural Gaussian counterpart.

In this section we formulate the natural multivariate versions of these results. They will be proved in the subsequent part of this work. To formulate them we have to introduce some new notions. In the subsequent sections I discuss some new problems which help to work out some methods that enable us to prove the results of this section. I finish this section with a short overview about the content of the remaining part of this work. I shall also briefly indicate why it helps us to prove the results of this section.

I start this section with the formulation of two results, Theorems 8.1 and 8.2 together with some of their simple consequences which yield a sharp estimate about the tail distribution of a multiple random integral with respect to a normalized empirical distribution and about the analogous problem when the tail distribution of the supremum of such integrals is considered. These results are the natural versions of the corresponding one-variate results about the tail behaviour of an integral or of the supremum of a class of integrals with respect to a normalized empirical distribution. They can be formulated with the help of the notions introduced before, in particular with the help of the notion of multiple random integrals with respect to a normalized empirical distribution function introduced in formula (4.8).

To formulate the following two results, Theorems 8.3 and 8.4 and their consequences, which are the natural multivariate versions of the results about the tail distribution of partial sums of independent random variables, and of the supremum of such sums we have to make some preparation. First we introduce the so-called U-statistics which can be considered as the natural multivariate generalizations of the sum of independent and identically distributed random variables. Moreover, we had a good estimation about the tail distribution of sums of independent random variables only if the summands had expectation zero, and we have to find the natural multivariate version of this notion. Hence we define the so-called degenerate U-statistics which can be considered as the natural multivariate counterpart of sums of independent and identically distributed random variables with zero expectation. Theorems 8.3 and 8.4 contain estimates about the tail-distribution of degenerate U-statistics and of the supremum of such expressions.

In Theorems 8.5 and 8.6 we formulate the Gaussian counterparts of the above results. They deal with multiple Wiener-Itô integrals with respect to a so-called white noise. The notion of multiple Wiener–Itô integrals and their properties needed to have a good understanding of these results will be explained in a later section. Still two results are discussed in this section. They are Examples 8.7 and 8.8, which state that the estimates of Theorems 8.5 and 8.3 are in a certain sense sharp.

To formulate the first two results of this section let us consider a sequence of independent and identically distributed random variables ξ_{1}, …, ξ_{n} with values in a measurable space (X, X). Let μ denote the distribution of the random variables ξ_{j}, and introduce the empirical distribution of the sequence ξ_{1}, …, ξ_{n} defined in (4.5). Given a measurable function f(x_{1}, …, x_{k}) on the k-fold product space (X^{k}, X^{k}) consider its integral J_{n,k}(f) with respect to the k-fold product of the normalized empirical measure \sqrt{n}(μ_{n} − μ) defined in formula (4.8). In the definition of this integral the diagonals x_{j} = x_{l}, 1 ≤ j < l ≤ k, were omitted from the domain of integration. The following Theorem 8.1 can be considered as the multiple integral version of Bernstein’s inequality formulated in Theorem 3.1.

Theorem 8.1. (Estimate on the tail distribution of a multiple random integral with respect to a normalized empirical distribution). Let us take a measurable function f(x_{1}, …, x_{k}) on the k-fold product (X^{k}, X^{k}) of a measurable space (X, X) with some k ≥ 1 together with a non-atomic probability measure μ on (X, X) and a sequence of independent and identically distributed random variables ξ_{1}, …, ξ_{n} with distribution μ on (X, X). Let the function f satisfy the conditions

∥f∥_{∞} = sup_{x_{j}∈X, 1≤j≤k} |f(x_{1}, …, x_{k})| ≤ 1,  (8.1)

and

∥f∥_{2}^{2} = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2}  (8.2)

with some constant 0 < σ ≤ 1. There exist some constants C = C_{k} > 0 and α = α_{k} > 0, such that the random integral J_{n,k}(f) defined by formulas (4.5) and (4.8) satisfies the inequality

P (|J_{n,k}(f)| > u) ≤ C max (e^{−α(u/σ)^{2/k}}, e^{−α(nu^{2})^{1/(k+1)}})  (8.3)

for all u > 0. The constants C = C_{k} > 0 and α = α_{k} > 0 in formula (8.3) depend only on the parameter k.

Theorem 8.1 can be reformulated in the following equivalent form.

Theorem 8.1′. Under the conditions of Theorem 8.1

P (|J_{n,k}(f)| > u) ≤ Ce^{−α(u/σ)^{2/k}} for all 0 < u ≤ n^{k/2}σ^{k+1}  (8.3′)

with a number σ, 0 ≤ σ ≤ 1, satisfying relation in (8.2) and some universal constants C = C_{k} > 0, α = α_{k} > 0, depending only on the multiplicity k of the integral J_{n,k}(f).

Theorem 8.1 clearly implies Theorem 8.1′, since in the case u ≤ n^{k/2}σ^{k+1} the first term is larger than the second one in the maximum at the right-hand side of formula (8.3). On the other hand, Theorem 8.1′ implies Theorem 8.1 also if u > n^{k/2}σ^{k+1}. Indeed, in this case Theorem 8.1′ can be applied with σ̄ = (un^{−k/2})^{1/(k+1)} ≥ σ if u ≤ n^{k/2}, hence also condition 0 < σ̄ ≤ 1 is satisfied. This yields that P (|J_{n,k}(f)| > u) ≤ C exp \{−α (\frac{u}{σ̄})^{2/k}\} = C exp \{−α(nu^{2})^{1/(k+1)}\} if n^{k/2} ≥ u > n^{k/2}σ^{k+1}, and rela-

tion (8.3) holds in this case. If u > n^{k/2}, then P (|J_{n,k}(f)| > u) = 0, and relation (8.3) holds again.

Theorem 8.1 or Theorem 8.1′ state that the tail distribution P (|J_{n,k}(f)| > u) of the k-fold random integral J_{n,k}(f) can be bounded similarly to the probability P (|const. ση^{k}| > u), where η is a random variable with standard normal distribution and the number 0 ≤ σ ≤ 1 satisfies relation (8.2), provided that the level u we consider is less than n^{k/2}σ^{k+1}. As we shall see later (see Corollary 1 of Theorem 9.4), the value of the number σ^{2} in formula (8.2) is closely related to the variance of J_{n,k}(f). At the end of this section an example is given which shows that the condition u ≤ n^{k/2}σ^{k+1} is really needed in Theorem 8.1′.

The next result, Theorem 8.2, is the generalization of Theorem 4.1′ for multiple random integrals with respect to a normalized empirical measure. In its formulation the notions of L_{2}-dense classes and countably approximability introduced in Section 4 are applied.

Theorem 8.2. (Estimate on the supremum of multiple random integrals with respect to an empirical distribution). Let us have a non-atomic probability measure μ on a measurable space (X, X) together with a countable and L_{2}-dense class F of functions f = f(x_{1}, …, x_{k}) of k variables with some parameter D ≥ 1 and exponent L ≥ 1 on the product space (X^{k}, X^{k}) which satisfies the conditions

∥f∥_{∞} = sup_{x_{j}∈X, 1≤j≤k} |f(x_{1}, …, x_{k})| ≤ 1,  for all f ∈ F  (8.4)

and

∥f∥_{2}^{2} = Ef^{2}(ξ_{1}, …, ξ_{k}) = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2} for all f ∈ F

(8.5)

with some constant 0 < σ ≤ 1. There exist some constants C = C(k) > 0, α = α(k) > 0 and M = M(k) > 0 depending only on the parameter k such that the supremum of the random integrals J_{n,k}(f), f ∈ F, defined by formula (4.8) satisfies the inequality

P (sup_{f∈F} |J_{n,k}(f)| ≥ u) ≤ C exp \{−α (\frac{u}{σ} )^{2/k}\} for those numbers u

for which nσ^{2} ≥ (\frac{u}{σ} )^{2/k} ≥ M(L^{3/2} log \frac{2}{σ} + (log D)^{3/2}),

(8.6) where the numbers D and L agree with the parameter and exponent of the L_{2}-dense class F.

The condition about the countable cardinality of the class F can be replaced by the weaker condition that the class of random variables J_{n,k}(f), f ∈ F, is countably approximable.

The condition given for the number u in formula (8.6) appears in Theorem 8.2 for a similar reason as the analogous condition formulated in (4.4) in its one-variate counterpart, Theorem 4.1. The lower bound is needed, since we have a good estimate in formula (8.6) only for u ≥ E sup_{f∈F} |J_{n,k}(f)|. The upper bound appears, since we have

a good estimate in Theorem 8.1′ only for 0 < u < n^{k/2}σ^{k+1}. If a pair of numbers (u, σ) does not satisfy condition (8.6), than we may try to get an estimate by increasing the number σ or decreasing the number u.

To formulate such a version of Theorems 8.1 and 8.2 which corresponds to the results about sums of independent random variables in the case k = 1 the following notions will be introduced.

Definition of U-statistics. Let us consider a function f = f(x_{1}, …, x_{k}) on the k-th power (X^{k}, X^{k}) of a space (X, X) together with a sequence of independent and identically distributed random variables ξ_{1}, …, ξ_{n}, n ≥ k, which take their values in this space (X, X). The expression

I_{n,k}(f) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f (ξ_{l_{1}}, …, ξ_{l_{k}})  (8.7)

is called a U-statistic of order k with the sequence ξ_{1}, …, ξ_{n}, and kernel function f.

Remark. In later calculations sometimes we shall work with U-statistics with kernel functions of the form f(x_{u_{1}}, …, x_{u_{k}}) instead of f(x_{1}, …, x_{k}), where {u_{1}, …, u_{k}} is an arbitrary set with different elements. The U-statistic with such a kernel function will also be defined, and it equals the U-statistic with the original kernel function f defined in (8.7), i.e.

I_{n,k}(f(x_{u_{1}}, …, x_{u_{k}})) = I_{n,k}(f(x_{1}, …, x_{k})).  (8.7′)

(Observe that if we define the function f_{π}(x_{1}, …, x_{k}) = f(x_{π(1)}, …, x_{π(k)}) for all permutations π of the set {1, …, k}, then I_{n,k}(f_{π}) = I_{n,k}(f), hence the above definition is legitimate.) Such a definition is natural, and it simplifies the notation in some calculations. A similar convention will be introduced about Wiener–Itô integrals in Section 10.

Some special U-statistics, called degenerate U-statistics, will be also introduced. They can be considered as the natural multivariate version of sums of identically distributed random variables with expectation zero. Degenerate U-statistics will be defined together with canonical kernel functions, because these notions are closely related.

Definition of degenerate U-statistics. A U-statistic I_{n,k}(f) of order k with a sequence of independent and identically distributed random variables ξ_{1}, …, ξ_{n} is called degenerate if its kernel function f(x_{1}, …, x_{k}) satisfies the relation

E(f(ξ_{1}, …, ξ_{k})|ξ_{1} = x_{1}, …, ξ_{j−1} = x_{j−1}, ξ_{j+1} = x_{j+1}, …, ξ_{k} = x_{k}) = 0
for all 1 ≤ j ≤ k and x_{s} ∈ X, s ≠ j.

Definition of a canonical kernel function. A function f(x_{1}, …, x_{k}) taking values in the k-fold product of a measurable space (X, X) is called a canonical function with respect to a probability measure μ on (X, X) if

∫ f(x_{1}, …, x_{j−1}, u, x_{j+1}, …, x_{k})μ( du) = 0 for all 1 ≤ j ≤ k and x_{s} ∈ X, s ≠ j.

(8.8)

For the sake of more convenient notations in the future we shall speak also of Ustatistics of order zero. We shall write I_{n,0}(c) = c for any constant c, and I_{n,0}(c) will be called a degenerate U-statistic of order zero. A constant will be considered as a canonical function with zero arguments.

It is clear that a U-statistic I_{n,k}(f) with kernel function f and independent μdistributed random variables ξ_{1}, …, ξ_{n} is degenerate if and only if its kernel function is canonical with respect to the probability measure μ. Let us also observe that

I_{n,k}(f) = I_{n,k}(Sym f)  (8.9)

for all functions of k variables.

The next two results, Theorems 8.3 and 8.4, deal with degenerate U-statistics. Theorem 8.3 is the U-statistic version of Theorem 8.1 and Theorem 8.4 is the U-statistic version of Theorem 8.2. Actually Theorem 8.3 yields a sharper estimate than Theorems 8.1, because it contains more explicit and better universal constants. I shall return to this point later.

Theorem 8.3. (Estimate on the tail distribution of a degenerate U-statistic). Let us have a measurable function f(x_{1}, …, x_{k}) on the k-fold product (X^{k}, X^{k}), k ≥ 1, of a measurable space (X, X) together with a probability measure μ on (X, X) and a sequence of independent and identically distributed random variables ξ_{1}, …, ξ_{n}, n ≥ k, with distribution μ on (X, X). Let us consider the U-statistic I_{n,k}(f) of order k with this sequence of random variables ξ_{1}, …, ξ_{n}. Assume that this U-statistic is degenerate, i.e. its kernel function f(x_{1}, …, x_{k}) is canonical with respect to the measure μ. Let us also assume that the function f satisfies conditions (8.1) and (8.2) with some number 0 < σ ≤ 1. Then there exist some constants A = A(k) > 0 and B = B(k) > 0 depending only on the order k of the U-statistic I_{n,k}(f) such that

P (k!n^{−k/2}|I_{n,k}(f)| > u) ≤ A exp \{−\frac{u^{2/k}}{2σ^{2/k} (1 + B (un^{−k/2}σ^{−(k+1)})^{1/k})}\} (8.10)

for all 0 ≤ u ≤ n^{k/2}σ^{k+1}.

Let us also formulate the following simple corollary of Theorem 8.3.

Corollary of Theorem 8.3 Under the conditions of Theorem 8.3 there exist some universal constants C = C(k) > 0 and α = α(k) > 0 that

P (k!n^{−k/2}|I_{n,k}(f)| > u) ≤ C exp \{−α (\frac{u}{σ} )^{2/k}\} for all 0 ≤ u ≤ n^{k/2}σ^{k+1}. (8.10′)

The following estimate holds about the supremum of degenerate U-statistics.

Theorem 8.4. (Estimate on the supremum of degenerate U-statistics). Let us have a probability measure μ on a measurable space (X, X) together with a countable and L_{2}-dense class F of functions f = f(x_{1}, …, x_{k}) of k variables with some parameter D ≥ 1 and exponent L ≥ 1 on the product space (X^{k}, X^{k}) which satisfies conditions (8.4) and (8.5) with some constant 0 < σ ≤ 1. Let us take a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{n}, n ≥ k, and consider the U statistics I_{n,k}(f) with these random variables and kernel functions f ∈ F. Let us assume that all these U-statistics I_{n,k}(f), f ∈ F, are degenerate, or in an equivalent form, all functions f ∈ F are canonical with respect to the measure μ. Then there exist some constants C = C(k) > 0, α = α(k) > 0 and M = M(k) > 0 depending only on the parameter k such that the inequality

P (sup_{f∈F} n^{−k/2}|I_{n,k}(f)| ≥ u) ≤ C exp \{−α (\frac{u}{σ} )^{2/k}\} holds for those numbers u

for which nσ^{2} ≥ (\frac{u}{σ} )^{2/k} ≥ M(L^{3/2} log \frac{2}{σ} + (log D)^{3/2}),

(8.11)

where the numbers D and L agree with the parameter and exponent of the L_{2}-dense class F.

The condition about the countable cardinality of the class F can be replaced by the weaker condition that the class of random variables n^{−k/2}I_{n,k}(f), f ∈ F, is countably approximable.

Next I formulate a Gaussian counterpart of the above results. To do this I need some notions that will be introduced in Section 10. In that section the white noise with a reference measure μ will be defined. It is an appropriate set of jointly Gaussian random variables indexed by those measurable sets A ∈ X of a measure space (X, X , μ) with a σ-finite measure μ for which μ(A) < ∞. Its distribution depends on the measure μ which will be called the reference measure of the white noise.

In Section 10 it will be also shown that given a white noise μ_{W} with a non-atomic σ-additive reference measure μ on a measurable space (X, X) and a measurable function f(x_{1}, …, x_{k}) of k variables on the product space (X^{k}, X^{k}) such that

∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2} < ∞  (8.12)

a k-fold Wiener-Itô integral of the function f with respect to the white noise μ_{W}

Z_{μ,k}(f) = \frac{1}{k!} ∫ f(x_{1}, …, x_{k})μ_{W} ( dx_{1}) … μ_{W} ( dx_{k})  (8.13)

can be defined, and the main properties of this integral will be proved there. It will be seen that Wiener-Itô integrals have a similar relation to degenerate U-statistics and multiple integrals with respect to normalized empirical measures as normally distributed random variables have to partial sums of independent random variables. Hence it is useful to find the analogs of the previous estimates of this section about the tail distribution of Wiener-Itô integrals. The subsequent Theorems 8.5 and 8.6 contain such results.

Theorem 8.5. (Estimate on the tail distribution of a multiple Wiener–Itô integral). Let us fix a measurable space (X, X) together with a σ-finite non-atomic measure μ on it, and let μ_{W} be a white noise with reference measure μ on (X, X). If f(x_{1}, …, x_{k}) is a measurable function on (X^{k}, X^{k}) which satisfies relation (8.12) with some 0 < σ < ∞, then

P (k!|Z_{μ,k}(f)| > u) ≤ C exp \{−\frac{1}{2} (\frac{u}{σ} )^{2/k}\}  (8.14)

for all u > 0 with some constants C = C(k) depending only on k.

Theorem 8.6. (Estimate on the supremum of Wiener–Itô integrals). Let F be a countable class of functions of k variables defined on the k-fold product (X^{k}, X^{k}) of a measurable space (X, X) such that

∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2} with some 0 < σ ≤ 1 for all f ∈ F

with some non-atomic σ-additive measure μ on (X, X). Let us also assume that F is an L_{2}-dense class of functions in the space (X^{k}, X^{k}) with respect to the measure μ^{k} with some exponent L ≥ 1 and parameter D ≥ 1, where μ^{k} is the k-fold product of the measure μ. (The classes of L_{2}-dense classes with respect to a measure were defined in Section 4.)

Take a white noise μ_{W} on (X, X) with reference measure μ, and define the Wiener– Itô integrals Z_{μ,k}(f) for all f ∈ F. Fix some 0 < ε ≤ 1. The inequality

P (sup_{f∈F} k!|Z_{μ,k}(f)| > u) ≤ CD exp \{−\frac{1}{2} (\frac{(1 − ε)u}{σ})^{2/k}\}  (8.15)

holds with some universal constants C = C(k) > 0, M = M(k) > 0 for those numbers u for which u ≥ ML^{k/2} \frac{1}{ε} log^{k/2} \frac{2}{ε} · σ log^{k/2} \frac{2}{σ}.

Formula (8.15) yields an almost as good estimate for the supremum of Wiener–Itô integrals with the choice of a small ε > 0 as formula (8.14) for a single Wiener–Itô integral. But the lower bound imposed on the number u in the estimate (8.15) depends on ε, and for a small number ε > 0 it is large.

The subsequent result presented in Example 8.7 may help to understand why Theorems 8.3 and 8.5 are sharp. Its proof and the discussion of the question about the sharpness of Theorems 8.3 and 8.5 will be postponed to Section 13.

Example 8.7. (A converse estimate to Theorem 8.5). Let us have a σ-finite measure μ on some measure space (X, X) together with a white noise μ_{W} on (X, X) with counting measure μ. Let f_{0}(x) be a real valued function on (X, X) such that ∫ f_{0}(x)^{2}μ( dx) = 1, and take the function f(x_{1}, …, x_{k}) = σf_{0}(x_{1}) ⋯ f_{0}(x_{k}) with some number σ > 0 together with the Wiener–Itô integral Z_{μ,k}(f) introduced in formula (8.13).

Then the relation ∫ f(x_{1}, …, x_{k})^{2} μ( dx_{1}) … μ( dx_{k}) = σ^{2} holds, and the Wiener– Itô integral Z_{μ,k}(f) satisfies the inequality

P (k!|Z_{μ,k}(f)| > u) ≥ \frac{C̄}{(\frac{u}{σ})^{1/k} + 1} exp \{−\frac{1}{2} (\frac{u}{σ} )^{2/k}\} for all u > 0  (8.16)

with some constant C̄ > 0.

The above results show that multiple integrals with respect to a normalized empirical measure or degenerate U-statistics satisfy some estimates similar to those about multiple Wiener–Itô integrals, but they hold under more restrictive conditions. The difference between the estimates in these problems is similar to the difference between the corresponding results in Section 4 whose reason was explained there. Hence they will be only briefly discussed here. The estimates of Theorem 8.1 and 8.3 are similar to that of Theorem 8.5. Moreover, for 0 ≤ u ≤ εn^{k/2}σ^{k+1} with a small number ε > 0 Theorem 8.3 yields an almost as good estimate about degenerate U-statistics as Theorem 8.5 yields for a Wiener–Itô integral with the same kernel function f and underlying measure μ. Example 8.7 shows that the constant in the exponent of formula (8.14) cannot be improved, at least there is no possibility of an improvement if only the L_{2}-norm of the kernel function f is known. Some results discussed later indicate that neither the estimate of Theorem 8.3 can be improved.

The main difference between Theorem 8.5 and the results of Theorem 8.1 or 8.3 is that in the latter case the kernel function f must satisfy not only an L_{2} but also an L_{∞} norm type condition, and the estimates of these results are formulated under the additional condition u ≤ n^{k/2}σ^{k+1}. It can be shown that the condition about the L_{∞} norm of the kernel function cannot be dropped from the conditions of these theorems, and a version of Example 3.3 will be presented in Example 8.8 which shows that in the case u ≫ n^{k/2}σ^{k+1} the left-hand side of (8.10) may satisfy only a much weaker estimate. This estimate will be given only for k = 2, but with some work it can be generalized for general indices k.

Theorems 8.2, 8.4 and 8.6 show that for the tail distribution of the supremum of a not too large class of degenerate U-statistics or multiple integrals a similar upper bound can be given as for the tail distribution of a single degenerate U-statistic or multiple integral, only the universal constants may be worse in the new estimates. However, they hold only under the additional condition that the level at which the tail distribution of the supremum is estimated is not too low. A similar phenomenon appeared already in the results of Section 4. Moreover, such a restriction had to be imposed in the formulation of the results here and in Section 4 for the same reason.

In Theorem 8.2 and 8.4 an L_{2}-dense class of kernel functions was considered, and this meant that the class of random integrals or U-statistics we consider in this result is not too large. In Theorem 8.6 a similar, but weaker condition was imposed on the class of kernel functions. They had to satisfy a similar condition, but only for the reference measure μ of the white noise appearing in the Wiener–Itô integral. A similar difference appears in the comparison of Theorems 4.1 or 4.1′ with Theorem 4.2, and this difference has the same reason in the two cases.

I still present the proof of the following Example 8.8 which is a multivariate version of Example 3.3. For the sake of simplicity I restrict my attention to the case k = 2.

Example 8.8. (A converse estimate to Theorem 8.3). Let us take a sequence of independent and identically distributed random variables ξ_{1}, …, ξ_{n} with values in the plane X = R^{2} such that ξ_{j} = (η_{j,1}, η_{j,2}), η_{j,1} and η_{j,2} are independent random variables with the following distributions. The distribution of η_{j,1} is defined with the help of a parameter σ^{2}, 0 < σ^{2} ≤ \frac{1}{8}, in the same way as the distribution of the random variables X_{j} in Example 3.3, i.e. η_{j,1} = η̄_{j,1} − Eη̄_{j,1} with P (η̄_{j,1} = 1) = σ̄^{2}, P (η̄_{j,1} = 0) = 1 − σ̄^{2}, where σ̄^{2} is that solution of the equation x^{2} − x + σ^{2} = 0, which is smaller than \frac{1}{2}. The distribution of the random variables is given by the formula P (η_{j,2} = 1) = P (η_{j,2} = −1) = \frac{1}{2} for all 1 ≤ j ≤ n. Introduce the function f(x, y) = f((x_{1}, x_{2}), (y_{1}, y_{2})) = x_{1}y_{2} + x_{2}y_{1}, x = (x_{1}, x_{2}) ∈ R^{2}, y = (y_{1}, y_{2}) ∈ R^{2} if (x, y) is in the support of the distribution of (ξ_{1}, ξ_{2}, i.e. if x_{1} and y_{1} take the values 1 − σ̄^{2} or −σ̄^{2} and x_{2} and y_{2} take the values ±1. Put f(x, y) = 0 otherwise. Define the U-statistic

I_{n,2}(f) = \frac{1}{2} ∑_{1≤j,k≤n, j≠k} f(ξ_{j}, ξ_{k}) = \frac{1}{2} ∑_{1≤j,k≤n, j≠k}(η_{j,1}η_{k,2} + η_{k,1}η_{j,2})

of order 2 with the above kernel function f and sequence of independent random variables ξ_{1}, …, ξ_{n}. Then I_{n,2}(f) is a degenerate U-statistic, | sup f(x, y)| ≤ 1 and Ef^{2}(ξ_{j}, ξ_{j}) = σ^{2}.

If u ≥ B_{1}nσ^{3} with some appropriate constant B_{1} > 2, B̄_{2}^{−1}n ≥ u ≥ B̄_{2}n^{−1/2} with a sufficiently large fixed number B̄_{2} > 0 and \frac{1}{4} ≥ σ^{2} ≥ \frac{1}{n^{2}} , and n is a sufficiently large number, then the estimate

P (n^{−1}I_{n,2}(f) > u) ≥ exp \{−Bn^{1/3}u^{2/3} log (\frac{u}{nσ^{3}} )\}  (8.17)

holds with some B > 0.

Remark: In Theorem 8.3 we got the estimate P (n^{−1}I_{n,2}(f) > u) ≤ e^{−αu/σ} for the above defined degenerate U-statistic I_{n,2}(f) if 0 ≤ u ≤ nσ^{3}. In the particular case u = nσ^{3} we have the estimate P (n^{−1}I_{n,2}(f) > nσ^{3}) ≤ e^{−αnσ^{2}}. On the other hand, the above example shows that in the case u ≫ nσ^{3} we can get only a weaker estimate. It is worth looking at the estimate (8.17) with fixed parameters n and u and to observe the dependence of the upper bound on the variance σ^{2} of I_{n,2}(f). In the case σ^{2} = u^{2/3}n^{−2/3} we have the upper bound e^{−αn^{1/3}u^{2/3}}. Example 8.8 shows that in the case σ^{2} ≪ u^{2/3}n^{−2/3} we can get only a relatively small improvement of this estimate. A similar picture appears as in Example 3.3 in the case k = 1.

It is simple to check that the U-statistic introduced in the above example is degenerate because of the independence of the random variables η_{j,1} and η_{j,2} and the identity Eη_{j,1} = Eη_{j,2} = 0. Beside this, Ef(ξ_{j}, ξ_{j})^{2} = σ^{2}. In the proof of the estimate (8.17) the results of Section 3, in particular Example 3.3 can be applied for the sequence η_{j,1}, j = 1, 2, …, n. Beside this, the following result known from the theory of large deviations will be applied. If X_{1}, …, X_{n} are independent and identically distributed random variables, P (X_{1} = 1) = P (X_{1} = −1) = \frac{1}{2}, then for any number 0 ≤ α < 1 there exists some numbers C_{1} = C_{1}(α) > 0 and C_{2} = C_{2}(α) > 0 such that

P ( ∑_{j=1}^{n} X_{j} > u) ≥ C_{1}e^{−C_{2}u^{2}/n} for all 0 ≤ u ≤ αn.

Proof of Example 8.8. The inequality

P (n^{−1}I_{n,2}(f) > u) ≥ P ((∑_{j=1}^{n} η_{j,1}) (∑_{j=1}^{n} η_{j,2}) > 4nu) − P (∑_{j=1}^{n} η_{j,1}η_{j,2} > 2nu)

(8.18)

holds. Because of the independence of the random variables η_{j,1} and η_{j,2} the first probability at the right-hand side of (8.18) can be bounded from below by bounding the multiplicative terms in it with v_{1} = 4n^{1/3}u^{2/3} and v_{2} = n^{2/3}u^{1/3}. The first term will be estimated by means of Example 3.3. This estimate can be applied with the choice y = v_{1}, since the relation v_{1} ≥ 4nσ^{2} holds if u ≥ B_{1}nσ^{3} with B_{1} > 1, and the remaining conditions 0 ≤ σ^{2} ≤ \frac{1}{8} and n ≥ 4v_{1} ≥ 6 also hold under the conditions of Example 8.8. The second term can be bounded with the help of the large-deviation result mentioned after the remark, since v_{2} ≤ \frac{1}{2}n if u ≤ B̄_{2}^{−1}n with a sufficiently large B̄_{2} > 0. In such a way we get the estimate

P ((∑_{j=1}^{n} η_{j,1}) (∑_{j=1}^{n} η_{j,2}) > 4nu) ≥ P (∑_{j=1}^{n} η_{j,1} > v_{1}) P (∑_{j=1}^{n} η_{j,2} > v_{2})

≥ C exp \{−B_{1}v_{1} log (\frac{v_{1}}{nσ^{2}} ) − B_{2} \frac{v_{2}^{2}}{n} \} ≥ C exp \{−B_{3}n^{1/3}u^{2/3} log (\frac{u}{nσ^{3}} )\}

with appropriate constants B_{1} > 1, B_{2} > 0 and B_{3} > 0. On the other hand, by applying Bennett’s inequality, more precisely its consequence given in formula (3.4) for the sum of the random variables X_{j} = η_{j,1}η_{j,2} at level nu instead of level u we get the following upper bound for the second term at the right-hand side of (8.18).

P (∑_{j=1}^{n} η_{j,1}η_{j,2} > 2nu) ≤ exp \{−Knu log \frac{u}{σ^{2}} \}

≤ exp \{−2B_{4}n^{1/3}u^{2/3} log (\frac{u}{nσ^{3}} )\} ,

since Eη_{j,1}η_{j,2} = 0, Eη_{j,1}^{2}η_{j,2}^{2} = σ^{2}, nu ≥ B_{1}n^{2}σ^{3} ≥ 2nσ^{2} because of the conditions B_{1} > 2 and nσ ≥ 1. Hence the estimate (3.4) (with parameter nu) can be applied in this case. Beside this, the constant B_{4} can be chosen sufficiently large in the last inequality if the number n or the bound B̄_{2} in Example 8.8 us chosen sufficiently large. This means that this term is negligible small. The above estimates imply the statement of Example 8.8.

Let me remark that under some mild additional restrictions the estimate (8.17) can be slightly improved, the term log can be replaced by log^{2/3} in the exponent of the right-hand side of (8.17). To get such an improvement some additional calculation is needed where the numbers v_{1} and v_{2} are replaced by v̄_{1} = 4n^{1/3}u^{2/3} log^{−1/3} (\frac{u}{nσ^{3}} ) and v̄_{2} = n^{2/3}u^{1/3} log^{1/3} (\frac{u}{nσ^{3}} ).

At the end of this section I present a short overview about the content of the remaining part of this work.

In our proofs we needed some results about U-statistics, and this is the main topic of Section 9. One of the results discussed here is the so-called Hoeffding decomposition of U-statistics to the linear combination of degenerate U-statistics of different order. We also needed some additional results which explain how some properties (e.g. a bound on the L_{2} and L_{∞} norm of a kernel function, the L_{2}-density property of a class F of kernel function) is inherited if we turn from the original U-statistics to the degenerate U-statistics appearing in their Hoeffding decomposition. Section 9 contains some results in this direction. Another important result in it is Theorem 9.4 which yields a decomposition of multiple integrals with respect to a normalized empirical distribution to the linear combination of degenerate U-statistics. This result is very similar to the Hoeffding decomposition of U-statistics. The main difference between them is that in the decomposition of multiple integrals much smaller coefficients appear. Theorem 9.4 makes possible to reduce the proof of Theorems 8.1 and 8.2 to the corresponding results in Theorems 8.3 and 8.4 about degenerate U-statistics.

The definition and the main properties of Wiener–Itô integrals needed in the proof of Theorems 8.5 and 8.6 are presented in Section 10. It also contains a result, called the diagram formula for Wiener–Itô integrals which plays an important role in our considerations. Beside this we proved a limit theorem, where we expressed the limit of normalized degenerate U-statistics with the help of multiple Wiener–Itô integrals. This result may explain why it is natural to consider Theorem 8.5 as the natural Gaussian counterpart of Theorem 8.5, and Theorem 8.6 as the natural Gaussian counterpart of Theorem 8.6.

We could prove Bernstein’s and Bennett inequality by means of a good estimation of the exponential moments of the partial sums we were investigating. In the proof of their multivariate versions, in Theorems 8.3 and 8.5 this method does not work, because the exponential moments we have to bound in these cases may be infinite. On the other hand, we could prove these results by means of a good estimate on the high moments of the random variables whose tail distribution we wanted to estimate. In the proof of Theorem 8.5 when the moments of multiple Wiener–Itô integrals have to be bounded this can be done with the help of the diagram formula for Wiener–Itô integrals. In Sections 11 and 12 we proved that there is a version of the diagram formula for degenerate U-statistics, and this enables us to estimate the moments needed in the proof of Theorem 8.3. In Section 13 we proved Theorems 8.3, 8.5 and a multivariate version of the Hoeffding inequality. At the end of this section we still discussed some results which state that in certain cases when we have, beside the upper bound of their L_{2} and L_{∞} norm some additional information about the behaviour of the kernel function f in Theorems 8.3 or 8.5, then the estimates of these results can be improved.

Section 14 contains the natural multivariate versions of the results in Section 6. In Section 6 Theorem 4.2 is proved about the supremum of Gaussian random variables with the help of the chaining argument. In Section 14 its multivariate version, Theorem 8.6 is proved with the help of the chaining argument. The chaining argument is not strong enough to prove Theorem 4.1. But as it is shown in Section 6, it enables us to prove a result formulated in Proposition 6.1, and to reduce the proof of Theorem 4.1 with its help to a simpler result formulated in Proposition 6.2. One of the results of Section 14, Proposition 14.1 is a multivariate version of Proposition 6.1. We showed that the proof of Theorem 8.4 can be reduced with its help to the proof of a result formulated in Proposition 14.2, which can be considered a multivariate version of Proposition 6.2. Section 14 contains still another result. It turned out that it is simpler to work with so-called decoupled U-statistics introduced in this section than with usual U-statistics, because they have more independence properties. In Proposition 14.2′ a version of Proposition 14.2 is formulated about degenerate U-statistics, and it is shown with the help of a result of de la Peña and Montgomery–Smith that the proof of Proposition 14.2, and thus of Theorem 8.4 can be reduced to the proof of Proposition 14.2′.

Proposition 14.2′ is proved similarly to its one-variate version, Proposition 6.2. The strategy of the proof is explained in Section 15. The main difference between the proof of the two propositions is that since the independence properties exploited in the proof of Proposition 6.2 hold only in a weaker form in the present case, we have to apply a more refined and more difficult argument. In particular, we have to apply instead of the symmetrization lemma, Lemma 7.1, a more general version of it, Lemma 15.2. It is hard to check its conditions when we try to apply it in the problems arising in the proof of Proposition 14.2′. This is the reason why we had to prove this result with the help of two inductive propositions, formulated in Proposition 15.3 and 15.4, while in the proof of Proposition 6.2 it was enough to prove one such result, presented in Proposition 7.3. We discuss the details of the problems and the strategy of the proof in Section 15. The proof of Propositions 15.3 and 15.4 is given in Sections 16 and 17. Section 16 contains the symmetrization arguments needed for us, and the proof is completed with its help in Section 17.

Finally in Section 18 we give an overview of this work, and explain its relation to some similar researches. The proof of some results is given in the Appendix.

9. Some results about U-statistics.

This section contains the proof of the Hoeffding decomposition theorem, an important result about U-statistics. It states that all U-statistics can be represented as a sum of degenerate U-statistics of different order. This representation can be considered as the natural multivariate version of the decomposition of a random variable as the sum of a random variable with expectation zero plus a constant (which can be interpreted as a random variable of zero variable). Some important properties of the Hoeffding decomposition will also be proved. The properties of the kernel function of a U-statistic will be compared to those of the kernel functions of the U-statistics in its Hoeffding decomposition.

If the Hoeffding decomposition of a U-statistic is taken, then the L_{2} and L_{∞}-norms of the kernel functions appearing in the U-statistics of the Hoeffding decomposition will be bounded by means of the corresponding norm of the kernel function of the original U-statistic. It will be also shown that if we take a class of U-statistics with an L_{2}- dense class of kernel functions (and the same sequence of independent and identically distributed random variables in the definition of each U-statistic), and we make the Hoeffding decomposition of all U-statistics in this class, then the kernel functions of the degenerate U-statistics appearing in these Hoeffding decompositions also constitute an L_{2}-dense class. Another important result of this section is Theorem 9.4. It yields a decomposition of a k-fold random integral with respect to a normalized empirical measure to the linear combination of degenerate U-statistics. This result makes possible to derive Theorem 8.1 from Theorem 8.3 and Theorem 8.2 from Theorem 8.4, and it is also useful in the proof of Theorems 8.3 and 8.4.

Let us first consider the Hoeffding’s decomposition. In the special case k = 1 it

states that the sum S_{n} = ∑_{j=1}^{n} ξ_{j} of independent and identically distributed random

variables can be rewritten as S_{n} = ∑_{j=1}^{n}(ξ_{j} − Eξ_{j}) + ( ∑_{j=1}^{n} Eξ_{j}), i.e. as the sum of

independent random variables with zero expectation plus a constant. We introduced the convention that a constant is the kernel function of a degenerate U-statistic of order zero, and I_{n,0}(c) = c for a U-statistic of order zero. I wrote down the above trivial formula, because Hoeffding’s decomposition is actually its adaptation to a more general situation. To understand this let us first see how to adapt the above construction to the case k = 2.

In this case a sum of the form 2I_{n,2}(f) = ∑_{1≤j,k≤n,j≠k} f(ξ_{j}, ξ_{k}) has to be consid-

ered. Write f(ξ_{j}, ξ_{k}) = [f(ξ_{j}, ξ_{k})−E(f(ξ_{j}, ξ_{k})|ξ_{k})]+E(f(ξ_{j}, ξ_{k})|ξ_{k}) = f_{1}(ξ_{j}, ξ_{k})+f̄_{1}(ξ_{k}) with f_{1}(ξ_{j}, ξ_{k}) = f(ξ_{j}, ξ_{k}) − E(f(ξ_{j}, ξ_{k})|ξ_{k}), and f̄_{1}(ξ_{k}) = E(f(ξ_{j}, ξ_{k})|ξ_{k}) to make the conditional expectation of f_{1}(ξ_{j}, ξ_{k}) with respect to ξ_{k} equal zero. Repeating this procedure for the first coordinate we define f_{2}(ξ_{j}, ξ_{k}) = f_{1}(ξ_{j}, ξ_{k}) − E(f_{1}(ξ_{j}, ξ_{k})|ξ_{j}) and f̄_{2}(ξ_{j}) = E(f_{1}(ξ_{j}, ξ_{k})|ξ_{j}). Let us also write f̄_{1}(ξ_{k}) = [f̄_{1}(ξ_{k}) − Ef̄_{1}(ξ_{k})] + Ef̄_{1}(ξ_{k}) and f̄_{2}(ξ_{j}) = [f̄_{2}(ξ_{j}) − Ef̄_{2}(ξ_{j})] + Ef̄_{2}(ξ_{j}). Simple calculation shows that 2I_{n,2}(f_{2}) is a degenerate U-statistics of order 2, and the identity 2I_{n,2}(f) = 2I_{n,2}(f_{2})+I_{n,1}((n−1)(f̄_{1} − Ef̄_{1}))+I_{n,1}((n−1)((f̄_{2} −Ef̄_{2}))+n(n−1)E(f̄_{1} +f̄_{2}) yields the decomposition of I_{n,2}(f) into a sum of degenerate U-statistics of different orders.

Hoeffding’s decomposition can be obtained by working out the details of the above argument in the general case. But it is simpler to calculate the appropriate conditional expectations with the help of the kernel functions of the U-statistics. To carry out such a program in the study of U-statistics of order k the following notations will be introduced.

Let us consider the k-fold product (X^{k}, X^{k}, μ^{k}) of a measure space (X, X , μ) with some probability measure μ, and define for all integrable functions f(x_{1}, …, x_{k}) and indices 1 ≤ j ≤ k the projection P_{j}f of the function f to its j-th coordinate as

(P_{j}f)(x_{1}, …, x_{j−1}, x_{j+1}, …, x_{k}) = ∫ f(x_{1}, …, x_{k})μ( dx_{j}), 1 ≤ j ≤ k.  (9.1)

Let us also define the operators Q_{j} = I − P_{j} i.e. Q_{j}f = f − P_{j}f for all integrable functions on f on the space (X^{k}, X^{k}, μ^{k}), 1 ≤ j ≤ k. In the definition (9.1) P_{j}f is a function not depending on the coordinate x_{j}, but in the definition of Q_{j} we introduce the fictive coordinate x_{j} to make the expression Q_{j}f = f − P_{j}f meaningful. The following result holds.

Theorem 9.1. (The Hoeffding decomposition). Let f(x_{1}, …, x_{k}) be an integrable function on the k-fold product space (X^{k}, X^{k}, μ^{k}) of a space (X, X , μ) with a probability measure μ. It has such a decomposition

f = ∑_{V ⊂{1,...,k}} f_{V} , with f_{V} (x_{j}, j ∈ V) = ( ∏_{j∈{1,...,k}\V} P_{j} ∏_{j∈V} Q_{j}) f(x_{1}, …, x_{k})

(9.2)

for which all functions f_{V}, V ⊂ {1, …, k}, in (9.2) are canonical with respect to the probability measure μ, and the function f_{V} depends on the arguments x_{j}, j ∈ V.

Let ξ_{1}, …, ξ_{n} be a sequence of independent μ distributed random variables, and consider the U-statistics I_{n,k}(f) and I_{n,|V |}(f_{V}) corresponding to the kernel functions f, f_{V} defined in (9.2) and random variables ξ_{1}, …, ξ_{n}. Then

k!I_{n,k}(f) = ∑_{V ⊂{1,...,k}}(n − |V |)(n − |V | − 1) ⋯ (n − k + 1)|V |!I_{n,|V |}(f_{V})  (9.3)

is a representation of I_{n,k}(f) as a sum of degenerate U-statistics, where |V | denotes the cardinality of the set V. (The product (n − |V |)(n − |V | − 1) ⋯ (n − k + 1) is defined as 1 for V = {1, …, k}, i.e. if |V | = k.) This representation is called the Hoeffding decomposition of I_{n,k}(f).

Proof of Theorem 9.1. Write f = ∏_{j=1}^{k}(P_{j} + Q_{j})f. By carrying out the multiplications in

this identity and applying the commutativity of the operators P_{j} and Q_{j} for different indices j we get formula (9.2). To show that the functions f_{V} in formula (9.2) are canonical let us observe that this property can be rewritten in the form P_{j}f_{V} ≡ 0 (in all coordinates x_{s}, s ∈ V \ {j} if j ∈ V). Since P_{j} = P_{j}^{2}, and the identity P_{j}Q_{j} = P_{j} − P_{j}^{2} = 0 holds for all j ∈ {1, …, k} this relation follows from the above mentioned commutativ-

ity of the operators P_{j} and Q_{j}, as P_{j}f_{V} = ( ∏_{s∈{1,...,k}\V} P_{s} ∏_{s∈V \{j}} Q_{s}) P_{j}Q_{j}f = 0. By

applying identity (9.2) for all terms f(ξ_{j_{1}}, …, ξ_{j_{k}}) in the sum defining the U-statistic I_{n,k}(f) and then summing them up we get relation (9.3).

In the Hoeffding decomposition we rewrote a general U-statistic in the form of a linear combination of degenerate U-statistics. In many applications of this result we still we have to know how the properties of the kernel function f of the original U-statistic are reflected in the properties of the kernel functions f_{V} of the degenerate U-statistics taking part in the Hoeffding composition. In particular, we need a good estimate on the L_{2} and L_{∞} norm of the functions f_{V} by means of the corresponding norm of the function f. Moreover, if we want to prove estimates on the tail distribution of the supremum of U-statistics I_{n,k}(f) for a nice class of kernel functions f ∈ F which is an L_{2}-dense class of functions with some exponent L and parameter D, then we may need a similar estimate on the class of kernel functions f_{V}, f ∈ F, with some V ∈ {1, …, k} appearing in the Hoeffding decomposition of these functions. We have to show that this class of functions is also L_{2}-dense, and we also need a good bound on the exponent and parameter of this L_{2}-dense class. The next result formulates such a statement.

Theorem 9.2. (Some properties of the Hoeffding decomposition). Let us consider a square integrable function f(x_{1}, …, x_{k}) on the k-fold product space (X^{k}, X^{k}, μ^{k}) and take its decomposition defined in formula (9.2). The inequalities

∫ f_{V}^{2} (x_{j}, j ∈ V) ∏_{j∈V} μ( dx_{j}) ≤ ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k})  (9.4)

and

sup_{x_{j}, j∈V} |f_{V} (x_{j}, j ∈ V)| ≤ 2^{|V |} sup_{x_{j}, 1≤j≤k} |f(x_{1}, …, x_{k})|  (9.4′)

hold for all V ⊂ {1, …, k}. (In particular, f_{∅}^{2} ≤ ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) for V = ∅.)

Let us consider an L_{2}-dense class F of functions with some parameter D ≥ 1 and exponent L ≥ 0 on the space (X^{k}, X^{k}), take the decomposition (9.2) of all functions f ∈ F, and define the classes of functions F_{V} = {2^{−|V |}f_{V}: f ∈ F} for all V ⊂ {1, …, k} with the functions f_{V} taking part in this decomposition. These classes of functions F_{V} are also L_{2}-dense with the same parameter D and exponent L for all V ⊂ {1, …, k}.

Theorem 9.2 will be proved as a consequence of Proposition 9.3 presented below. To formulate it first some notations will be introduced:

Let us consider the product (Y × Z, Y × Z) of two measurable spaces (Y, Y) and (Z, Z) together with a probability measure μ on (Z, Z) and the operator

P f(y) = P_{μ}f(y) = ∫ f(y, z)μ( dz), y ∈ Y, z ∈ Z  (9.5)

defined for those y ∈ Y for which the above integral is finite. Let I denote the identity operator on the space of functions on Y × Z, i.e. let If(y, z) = f(y, z), and introduce the operator Q = Q_{μ} = I − P = I − P_{μ}

Q_{μ}f(y, z) = (I − P_{μ})f(y, z) = f(y, z) − P_{μ}f(y, z) = f(y, z) − ∫ f(y, z)μ( dz), (9.6)

defined for those points (y, z) ∈ Y × Z whose first coordinate y is such that the expression P_{μ}f(y) is meaningful. (Here, and in the sequel a function g(y) defined on the space (Y, Y) will be sometimes identified with the function ḡ(y, z) = g(y) on the space (Y × Z, Y × Z) which actually does not depend on the coordinate z.) The following result holds:

Proposition 9.3. Let us consider the direct product (Y × Z, Y × Z) of two measure spaces (Y, Y) and (Z, Z) together with a probability measure μ on the space (Z, Z). Take the transformations P_{μ} and Q_{μ} defined in formulas (9.5) and (9.6). Given any probability measure ρ on the space (Y, Y) consider the product measure ρ × μ on (Y × Z, Y ×Z). Then the transformations P_{μ} and Q_{μ}, as maps from the space L_{2}(Y ×Z, Y × Z, μ × ρ) to L_{2}(Y, Y, ρ) and L_{2}(Y × Z, Y × Z, ρ × μ) respectively, have a norm less than or equal to 1, i.e.

∫ P_{μ}f(y)^{2}ρ( dy) ≤ ∫ f(y, z)^{2}ρ( dy)μ( dz),  (9.7)

and

∫ Q_{μ}f(y, z)^{2}ρ( dy)μ( dz) ≤ ∫ f(y, z)^{2}ρ( dy)μ( dz)  (9.8)

for all functions f ∈ L_{2}(Y × Z, Y × Z, ρ × μ).

If F is an L_{2}-dense class of functions f(y, z) in the product space (Y × Z, Y × Z), with some parameter D ≥ 1 and exponent L ≥ 0, then also the classes F_{μ} = {P_{μ}f, f ∈ F\} and G_{μ} = \{\frac{1}{2}Q_{μ}f = \frac{1}{2}(f −P_{μ}f), f ∈ F\} are L_{2}-dense classes with the same exponent L and parameter D in the spaces (Y, Y) and (Y × Z, Y × Z) respectively.

The following corollary of Proposition 9.3 is formally more general, but it is a simple consequence of this result. Actually we shall need this corollary.

Corollary of Proposition 9.3. Let us consider the product (Y_{1} × Z × Y_{2}, Y_{1} × Z × Y_{2}) of three measurable spaces (Y_{1}, Y_{1}), (Z, Z) and (Y_{2}, Y_{2}) with a probability measure μ on the space (Z, Z) and a probability measure ρ on Y_{1} × Y_{2}, Y_{1} × Y_{2}), and define the transformations

P_{μ}f(y_{1}, y_{2}) = ∫ f(y_{1}, z, y_{2})μ( dz), y_{1} ∈ Y_{1}, z ∈ Z, y_{2} ∈ Y_{2}  (9.5′)

and

\begin{aligned}Q_{μ}f(y_{1}, z, y_{2}) & = (I − P_{μ})f(y_{1}, z, y_{2}) = f(y_{1}, z, y_{2}) − P_{μ}f(y_{1}, z, y_{2}) \\ & = f(y_{1}, z, y_{2}) − ∫ f(y_{1}, z, y_{2})μ( dz), y_{1} ∈ Y_{1}, z ∈ Z, y_{2} ∈ Y_{2} \end{aligned} (9.6′)

for the measurable functions f on the space Y_{1} × Z × Y_{2} integrable with respect the measure μ × ρ. Then

∫ P_{μ}f(y_{1}, y_{2})^{2}ρ( dy_{1}, dy_{2}) ≤ ∫ f(y, z)^{2}(ρ × μ)( dy_{1}, dz, dy_{2}),  (9.7′)

for all probability measures ρ on (Y_{1} × Y_{2}, Y_{1} × Y_{2}), where ρ × μ is the product of the probability measure ρ on (Y_{1} × Y_{2}, Y_{1} × Y_{2}) and μ on (Z, Z), Also the inequality

∫ Q_{μ}f(y_{1}, z, y_{2})^{2}ρ( dy_{1}, dy_{2})μ( dz) ≤ ∫ f(y_{1}, z, y_{2})^{2}ρ( dy_{1}, dy_{2})μ( dz)  (9.8′)

holds for all functions f ∈ L_{2}(Y × Z, Y × Z, ρ × μ).

If F is an L_{2}-dense class of functions f(y_{1}, z, y_{2}) in the product space (Y_{1} × Z × Y_{2}, Y_{1} × Z × Y_{2}), with some parameter D ≥ 1 and exponent L ≥ 0, then also the classes F_{μ} = \{P_{μ}f, f ∈ F\} and G_{μ} = \{\frac{1}{2}Q_{μ}f = \frac{1}{2}(f − P_{μ}f), f ∈ F\} are L_{2}-dense classes with exponent L and parameter D in the spaces (Y_{1}×Y_{2}, Y_{1}×Y_{2}) and (Y_{1}×Z×Y_{2}, Y_{1}×Z×Y_{2}) respectively.

This corollary is a simple consequence of Proposition 9.3 if we apply it with (Y, Y) = (Y_{1}×Y_{2}, Y_{1}×Y_{2}) and take the natural mapping f((y_{1}, y_{2}), z) → f(y_{1}, z, y_{2}) of a function from the space (Y × Z, Y × Z) to a function on (Y_{1} × Z × Y_{2}, Y_{1} × Z × Y_{2}). Beside this, we apply that measure on (Y_{1} × Z × Y_{2}, Y_{1} × Z × Y_{2}) which is the image of the product measure ρ × μ with respect to the map induced by the above transformation on the space of measures.

Proposition 9.3, more precisely its corollary implies Theorem 9.2, since it implies that the operators P_{s}, Q_{s}, 1 ≤ s ≤ k, applied in Theorem 9.2 do not increase the L_{2}(μ) norm of a function f, and it is also clear that the norm of P_{s} is bounded by 1 the norm of Q_{s} = I − P_{s} is bounded by 2 as an operator from L_{∞} spaces to L_{∞} spaces. The corollary of Proposition 9.3 also implies that if F is an L_{2}-dense class of functions with parameter D and exponent L, then the same property holds for the classes of functions F_{P_{s}} = \{P_{s}f: f ∈ F\} and F_{Q_{s}} = \{\frac{1}{2}Q_{s}f: f ∈ F\}, 1 ≤ s ≤ k. These relations together

with the identity f_{V} = ( ∏_{s∈{1,...,k}\V} P_{s} ∏_{s∈V} Q_{s}) f imply Theorem 9.2.

Proof of Proposition 9.3. The Schwarz inequality yields that P_{μ}(f)^{2} ≤ ∫ f(y, z)^{2}μ( dz), and integrating this inequality with respect to the probability measure ρ( dy) we get inequality (9.7). Also the inequality

∫ Q_{μ}f(y, z)^{2}ρ( dy)μ( dz) = ∫ [f(y, z) − P_{μ}f(y, z)]^{2}ρ( dy)μ( dz)
≤ ∫ f(y, z)^{2}ρ( dy)μ( dz)

holds, and this is relation (9.8). This follows for instance from the observation that the functions f(y, z) − P_{μ}f(y, z) and P_{μ}f(y, z) are orthogonal in the space L_{2}(Y × Z, Y × Z, ρ × μ).

Let us consider an arbitrary probability measure ρ on the space (Y, Y). To prove that F_{μ} is an L_{2}-dense class with parameter D and exponent L if the same relation holds for F we have to find for all 0 < ε ≤ 1 a set {f_{1}, …, f_{m}} ⊂ F_{μ}, 1 ≤ j ≤ m with m ≤ Dε^{−L} elements, such that inf_{1≤j≤m} ∫ (f_{j} − f)^{2} dρ ≤ ε^{2} for all f ∈ F_{μ}. But a

similar property holds for F in the space Y × Z with the probability measure ρ × μ. This property together with the L_{2} contraction property of P_{μ} formulated in (9.7) imply that F_{μ} is an L_{2}-dense class.

To prove that G_{μ} is also L_{2}-dense with parameter D and exponent L under the same condition we have to find for all numbers 0 < ε ≤ 1 and probability measures ρ on Y ×Z

a subset {g_{1}, …, g_{m}} ⊂ G_{μ} with m ≤ Dε^{−L} elements such that inf_{1≤j≤m} ∫ (g_{j} −g)^{2} dρ ≤ ε^{2}

for all g ∈ G_{μ}.

To show this let us consider the probability measure ρ̃ = \frac{1}{2}(ρ + ρ̄ × μ) on (Y × Z, Y × Z), where ρ̄ is the projection of the measure ρ to (Y, Y), i.e. ρ̄(A) = ρ(A × Z) for all A ∈ Y, take a class of function F_{0}(ε, ρ̃) = {f_{1}, …, f_{m}} ⊂ F with m ≤ Dε^{−L} elements such that inf_{1≤j≤m} ∫ (f_{j} − f)^{2} dρ̃ ≤ ε^{2} for all f ∈ F, and put {g_{1}, …, g_{m}} =

\{\frac{1}{2}Q_{μ}f_{1}, …, \frac{1}{2}Q_{μ}f_{m}\}. All functions g ∈ G_{μ} can be written in the form g = \frac{1}{2}Q_{μ}f with some f ∈ F, and there exists some function f_{j} ∈ F_{0}(ε, ρ̃) such that ∫ (f − f_{m})^{2} dρ̃ ≤ ε^{2}. Hence to complete the proof of Proposition 9.3 it is enough to show that ∫ \frac{1}{4}(Q_{μ}f − Q_{μ}f̄)^{2} dρ ≤ ∫ (f − f̄)^{2} dρ̃ for all pairs f, f̄ ∈ F. This inequality holds, since ∫ \frac{1}{4}(Q_{μ}f − Q_{μ}f̄)^{2} dρ ≤ ∫ \frac{1}{2}(f − f̄)^{2} dρ + ∫ \frac{1}{2}(P_{μ}f − P_{μ}f̄)^{2} dρ, and ∫ (P_{μ}f − P_{μ}f̄)^{2} dρ = ∫ (P_{μ}f − P_{μ}f̄)^{2} dρ̄ ≤ ∫ (f −f̄)^{2} d(ρ̄×μ) by formula 9.7. The above relations imply that ∫ \frac{1}{4}(Q_{μ}f − Qμf̄)^{2} dρ ≤ ∫ (f − f̄)^{2} \frac{1}{2}d (ρ + ρ̄ × μ) = ∫ (f − f̄)^{2}d ρ̃ as we have claimed.

Now we shall discuss the relation between Theorem 8.1′ and Theorem 8.3 and between Theorem 8.2 and Theorem 8.4. First we show that Theorem 8.1 (or Theorem 8.1′) is equivalent to the estimate (8.10′) in the corollary of Theorem 8.3 which is slightly weaker than the estimate (8.10) of Theorem 8.3. We also claim that Theorems 8.2 and 8.4 are equivalent. Both in Theorem 8.2 and in Theorem 8.4 we can restrict our attention to the case when the class of functions F is countable, since the case of countably approximable classes can be simply reduced to this situation. Let us remark that integration with respect to the measure μ_{n} −μ in the definition (4.8) of the integral J_{n,k}(f) yields some kind of normalization which is missing in the definition of the U-statistics I_{n,k}(f). This is the cause why degenerate U-statistics had to be considered in Theorems 8.3 and 8.4. The deduction of the corollary of Theorem 8.3 from Theorems 8.1′ or of Theorem 8.4 from Theorem 8.2 is fairly simple if the underlying probability measure μ is non-atomic, since in this case the identity I_{n,k}(f) = J_{n,k}(f) holds for a canonical function with respect to the measure μ. Let us remark that the non-atomic property of the measure μ is needed in this argument not only because of the conditions of Theorems 8.1′ and 8.2, but since in the proof of the above identity we need the identity ∫ f(x_{1}, …, x_{k})μ( dx_{j}) ≡ 0 in the case when the domain of integration is not the whole space X but the set X \ {x_{1}, …, x_{j−1}, x_{j+1}, …, x_{k}}.

The case of possibly atomic measures μ can be simply reduced to the case of nonatomic measures by means of the following enlargement of the space (X, X , μ). Let us introduce the product space (X̄, X̄, μ̄) = (X, X , μ)×([0, 1], B, λ), where B is the σ-algebra and λ is the Lebesgue measure on [0, 1]. Define the function f̄((x_{1}, u_{1}), …, (x_{k}, u_{k})) = f(x_{1}, …, x_{k}) in this enlarged space. Then I_{n,k}(f) = I_{n,k}(f̄), the measure μ̄ = μ × λ is non-atomic, and f̄ is canonical with respect to μ̄ if f is canonical with respect to μ. Hence the corollary of Theorem 8.3 and Theorem 8.4 can be derived from Theorems 8.1′ and 8.2 respectively by proving them first for their counterpart in the above constructed enlarged space with the above defined functions.

Also Theorems 8.1′ and 8.2 can be derived from Theorems 8.3 and 8.4 respectively, but this is a much harder problem. To do this let us observe that a random integral J_{n,k}(f) can be written as a sum of U-statistics of different order, and it can also be expressed as a sum of degenerate U-statistics if Hoeffding’s decomposition is applied for each U-statistic in this sum. We show that the coefficients of the degenerate U-statistics in the above representation have relatively small coefficients. This result is formulated in the following Theorem 9.4. To make its content more understandable I shall describe it in the special case of two-fold random integrals in a more explicit form in Corollary 2 of Theorem 9.4 at the end of this section.

Theorem 9.4. (Decomposition of a multiple random integral with respect to a normalized empirical measure to a linear combination of degenerate U-statistics). Let a non-atomic measure μ be given on a measurable space (X, X) together with a sequence of independent, μ-distributed random variables ξ_{1}, …, ξ_{n}. Take a function f(x_{1}, …, x_{k}) of k variables on the space (X^{k}, X^{k}) such that

∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) < ∞

and consider the empirical distribution function μ_{n} of the sequence ξ_{1}, …, ξ_{n} introduced in (4.5) together with the k-fold random integral J_{n,k}(f) of the function f defined in (4.8). The identity

J_{n,k}(f) = ∑_{V ⊂{1,...,k}} C(n, k, V)n^{−|V |/2}I_{n,|V |}(f_{V})  (9.9)

holds with the set of (canonical) functions f_{V} (x_{j}, j ∈ V) (with respect to the measure μ) defined in formula (9.2) together with some real numbers C(n, k, V), V ⊂ {1, …, k}, where I_{n,|V |}(f_{V}) denotes the (degenerate) U-statistic of order |V | with the random variables ξ_{1}, …, ξ_{n} and kernel function f_{V}. The constants C(n, k, V) in formula (9.9) satisfy the inequality |C(n, k, V)| ≤ C(k) with some constant C(k) depending only on the order k of the integral J_{n,k}(f). The relations lim_{n→∞} C(n, k, V) = C(k, V) hold with some appropriate constant such that 0 ≤ |C(k, V)| < ∞, and C(n, k, {1, …, k}) = 1 for V = {1, …, k}.

Remark: Some considerations show that the coefficients C(n, k, V) in formula (9.9) depend only on the cardinality |V | of the set V, i.e. C(n, k, V) = C(n, k, |V |) can be written. This fact will be not applied in this work.

Theorems 8.1′ and 8.2 can be simply derived from Theorems 8.3 and 8.4 respectively with the help of Theorem 9.4. Indeed, to get Theorem 8.1′ observe that formula (9.9) implies the inequality

P (|J_{n,k}(f)| > u) ≤ ∑_{V ⊂\{1,...,k\}} P (n^{−|V |/2}|I_{n,|V |}(f_{V})| > \frac{u}{2^{k}C(k)})  (9.10)

with a constant C(k) satisfying the inequality C(n, k, |V |) ≤ C(k) for all coefficients C(n, k, |V |) in (9.9). Hence Theorem 8.1′ follows from Theorem 8.3 and relations (9.4) and (9.4′) in Theorem 9.2 by which the L_{2}-norm of the functions f_{V} is bounded by the L_{2}-norm of the function f and the L_{∞}-norm of f_{V} is bounded by the 2^{|V |}-times the L_{∞}-norm or f. It is enough to estimate each term at the right-hand side of (9.10) by means of Theorem 8.3. It can be assumed that 2^{k}C(k) > 1. Let us first assume that also the inequality \frac{u}{2^{k}C(k)σ} ≥ 1 holds. In this case formula (8.3′) in Theorem 8.1′ can be obtained by means of the estimation of each term at the right-hand side of

(9.10). Observe that exp \{−α (\frac{u}{2^{k}C(k)σ} )^{2/s}\} ≤ exp \{−α (\frac{u}{2^{k}C(k)σ} )^{2/k}\} for all s ≤ k

if \frac{u}{2^{k}C(k)σ} ≥ 1. In the other case, if \frac{u}{2^{k}C(k)σ} ≤ 1, formula (8.3′) holds again with a sufficiently large C > 0, because in this case its right-hand side is greater than 1.

Theorem 8.2 can be similarly derived from Theorem 8.4 by observing that relation (9.10) remains valid if |J_{n,k}(f)| is replaced by sup_{f∈F} |J_{n,k}(f)| and |I_{n,|V |}(f_{V})| by

sup_{f_{V} ∈F_{V}} |I_{n,|V |}(f_{V})| in it, and we have the right to choose the constant M in formula (8.6)

of Theorem 8.2 sufficiently large. The only difference in the argument is that beside formulas (9.4) and (9.4′) the last statement of Theorem 9.2 also has to be applied in this case. It tells that if F is an L_{2}-dense class of functions on a space (X^{k}, X^{k}), then the classes of functions F_{V} = {2^{−|V |}f_{V}: f ∈ F} are also L_{2}-dense classes of functions for all V ⊂ {1, …, k} with the same exponent and parameter.

Next I make some comments about the content of Theorem 9.4. The expression J_{n,k}(f) was defined as a k-fold random integral, where the diagonals were omitted from the domain of integration. We have integrated with respect to the signed measure μ_{n} − μ, which means some kind of normalization. Thus it is not surprising that the tail distribution behaviour of J_{n,k}(f) is similar to that of degenerate U-statistics. Theorem 9.4 has such a consequence. Formula (9.9) expresses the random integral J_{n,k}(f) as a linear combination of degenerate U-statistics of different order. This is similar to the Hoeffding decomposition of the U-statistic I_{n,k}(f) where the same kernel functions f_{V} appear. But the coefficients C(n, k, |V |)n^{−|V |/2} of the terms I_{n,|V |}(f_{V}) in the expansion (9.9) are small. On the other hand, they do not have to disappear. In particular, the expansion (9.9) may contain a non-zero constant term. In such a case the expected value EJ_{n,k}(f) is not equal to zero. But even in this case it can be bounded by a finite number not depending on the sample size n. Next I show an example for such a random integral J_{n,2}(f) where EJ_{n,2}(f) ≠ 0.

Let us choose a sequence of independent random variables ξ_{1}, …, ξ_{n} with uniform distribution on the unit interval, let μ_{n} denote its empirical distribution, let f = f(x, y) denote the indicator function of the unit square, i.e. let f(x, y) = 1 if 0 ≤ x, y ≤ 1, and f(x, y) = 0 otherwise. Let us consider the random integral 2J_{n,2}(f) = n ∫_{x≠y} f(x, y)(μ_{n}( dx) − dx)(μ_{n}( dy) − dy), and calculate its expected value E2J_{n,2}(f). By adjusting the diagonal x = y to the domain of integration and taking out the contribution obtained in this way we get that E2J_{n,2}(f) = nE(∫_{0}^{1} (μ_{n}( dx) − μ( dx))^{2} − n^{2} · \frac{1}{n^{2}} = −1. (The last term is the integral of the function f(x, y) on the diagonal x = y with respect to the product measure μ_{n} × μ_{n} which equals (μ_{n} − μ) × (μ_{n} − μ) on the diagonal.)

Now I turn to the proof of Theorem 9.4.

Proof of Theorem 9.4. Let us first introduce the (random) probability measures μ^{(l)}, 1 ≤ l ≤ n, concentrated in the (single) sample points ξ_{l}, i.e. let μ^{(l)}(A) = 1 if ξ_{l} ∈ A,

and μ^{(l)}(A) = 0 if ξ_{l} ∉ A, A ∈ A, 1 ≤ l ≤ n. Then μ_{n} − μ = \frac{1}{n} ∑_{l=1}^{n} (μ^{(l)} − μ), and

formula (4.8) can be rewritten as

J_{n,k}(f) = \frac{1}{n^{k/2}k!} ∑_{(l_{1},...,l_{k}): 1≤l_{j}≤n, 1≤j≤k,} ∫ ′ f(x_{1}, …, x_{k})  (9.11)

(μ^{(l_{1})}( dx_{1}) − μ( dx_{1})) … (μ^{(l_{k})}( dx_{k}) − μ( dx_{k})) .

To rearrange the above sum in a way more appropriate for us let us introduce the class of all partitions P = P_{k} of the set {1, 2, …, k}. For a partition P = {R_{1}, …, R_{u}}

⋃_{j=1}^{u} R_{j} = {1, …, k}, R_{j} ∩ R_{l} = ∅, 1 ≤ j < l ≤ u, the sets R_{j}, 1 ≤ j ≤ u, will be called

the components of the partition P. Given a sequence (l_{1}, …, l_{k}), 1 ≤ l_{j} ≤ n, 1 ≤ j ≤ k, of length k let P_{H}(l_{1}, …, l_{k}) denote that partition of the set {1, …, k} in which two points s and t, 1 ≤ s, t ≤ k, belong to the same component of this partition if and only if l_{s} = l_{t}. For a partition P ∈ P_{k} let us define the set H(P) = H_{n}(P) consisting of sequences (l_{1}, …, l_{k}) with 1 ≤ l_{j} ≤ n for all 1 ≤ j ≤ k as H(P) = {(l_{1}, …, l_{k}): 1 ≤ l_{j} ≤ n, 1 ≤ j ≤ k, P_{H}(l_{1}, …, l_{k}) = P }.

Let us rewrite formula (9.11) in the form

J_{n,k}(f) = \frac{1}{n^{k/2}k!} ∑_{P ∈P} ∑_{(l_{1},...,l_{k}): (l_{1},...,l_{k})∈H(P)} ∫ ′ f(x_{1}, …, x_{k})  (9.12)

(μ^{(l_{1})}( dx_{1}) − μ( dx_{1})) … (μ^{(l_{k})}( dx_{k}) − μ( dx_{k})) .

Let us remember that the diagonals x_{s} = x_{t}, s ≠ t, were omitted from the domain of integration in the formula defining J_{n,k}(f). This implies that if l_{s} = l_{t} for some s ≠ t, then the measure μ^{(l_{s})}( dx_{s})μ^{(l_{t})}( dx_{t}) has zero measure in the domain of integration. We have to understand the cancellation effects caused because of this fact. It will be shown that because of these cancellations the expression in formula (9.12) can be rewritten as a linear combination of degenerate U-statistics with not too large coefficients. Beside this, it will be seen from the calculations that the same degenerate U-statistics I_{n,|V |}(f_{V}) appear in the representation of J_{n,k}(f) as in formula (9.2). This is a natural approach, but the detailed proof demands some rather unpleasant calculation.

Let us fix some partition P ∈ P and investigate the integrals in the internal sum at the right-hand side of (9.12) corresponding to the sequences (l_{1}, …, l_{k}) ∈ H(P). For the sake of a better understanding let us first consider such a partition P ∈ P which has a component of the form {1, …, s} with some s ≥ 2. The products of measures by which we have to integrate in this case contain a part of length s of the form (μ^{(l)}(dx_{1}) − μ(dx_{1})) … (μ^{(l)}(dx_{s}) − μ(dx_{s})) This part of the product measure can be rewritten in the domain of integration as

∑_{j=1}^{s}(−1)^{s−1}μ( dx_{1}) … μ( dx_{j−1})μ^{(l)}( dx_{j})μ( dx_{j+1}) … μ( dx_{s}) + (−1)^{s}μ(dx_{1}) … μ(dx_{s})

= ∑_{j=1}^{s}(−1)^{s−1}μ( dx_{1}) … μ( dx_{j−1})(μ^{(l)}( dx_{j}) − μ( dx_{l}))μ( dx_{j+1}) … μ( dx_{s})

+ (−1)^{s−1}(s − 1)μ(dx_{1}) … μ(dx_{s}).  (9.13)

Here we have exploited that all other terms of this product disappear in the domain of integration which does not contain the diagonals. Let us also observe that the term (−1)^{s−1}(s − 1)μ(dx_{1}) … μ(dx_{j}) appears n-times if we sum up for all 1 ≤ l ≤ n. We have assumed that s ≥ 2, since the case s = 1 is slightly different. In this case only the term μ^{(l)}(dx_{1}) − μ( dx_{1}) appears, i.e. have to put no additional term consisting only of (deterministic) measures μ.

More generally, let us fix some partition P = {R_{1}, …, R_{u}}, consider the integral corresponding to a sequence (l_{1}, …, l_{k}) ∈ H(P) in the internal sum of (9.12), and let us rewrite it as the sum of integrals with respect to product measures with components of the form μ^{(l_{s})}( dx_{s})−μ( dx_{s}) or μ( dx_{s}), where all measures μ^{(l_{s})} appearing in a product measure are different. Such a representation can be given, similarly to the calculation leading to relation (9.13), only the notations will be more complicated. To write down what we get first we define a set T (P) whose elements are certain subsets of {1, …, k} depending on the partition P = {R_{1}, …, R_{u}} together with a subset T̄(P) of it. The elements of the set T (P) are those sets {j_{1}, …, j_{u′}} ⊂ {1, …, k}, u′ ≤ u, which contain zero or one element from each component R_{j}, 1 ≤ j ≤ u, of the partition P. Let T̄(P) ⊂ T (P) consist of those sets V = {j_{1}, …, j_{u′}} ∈ T (P) which satisfy the following additional condition: If some component R_{t} = {b_{t}}, 1 ≤ t ≤ u, of the partition P consists of only one point, then all sets V ∈ T̄(P) ⊂ T (P) contain this point b_{t}. With the help of the above quantities we can write in the case (l_{1}, …, l_{k}) ∈ H(P), similarly to the calculation in (9.13), that

∫ ′ f(x_{1}, …, x_{k}) (μ^{(l_{1})}( dx_{1}) − μ( dx_{1})) … (μ^{(l_{k})}( dx_{k}) − μ( dx_{k}))  (9.14)

= ∑_{V ∈T̄ (P)} α(V, P) ∫ f(x_{1}, …, x_{k}) ∏_{j∈V} (μ^{(l_{j})}( dx_{j}) − μ( dx_{j})) ∏_{j′∈{1,...,k}\V} μ( dx_{j′})

with some appropriate finite constants α(V, P). These constants could be calculated explicitly, but it is enough for us to know that they depend only on the partition P and the set V ∈ T̄(P). On the other hand, it is important to observe that a term with non-zero coefficient α(V, P) appears at the right-hand side of (9.14) only for V ∈ T̄(P). This was the reason for the introduction of the sets T̄(P). To understand relation (9.14) observe that if R_{t} = {b_{t}} is a one-point component of the the partition P, then only the term μ^{(l_{b_{t}})}( dx_{b_{t}}) − μ( dx_{b_{t}}) appears as a component depending on b_{t} in the product of measures at the right-hand side of (9.14), and a component of the form μ( dx_{b_{t}}) is missing. Hence a product ∏_{j′∈{1,...,k}\V} μ( dx_{j′}) cannot appear at the right-hand side of

(9.14) if V ∉ T̄(P).

Let me remark that at the right-hand side of (9.14) ∫ was written and not ∫′, i.e. the diagonal was not omitted from the domain of integration. This is allowed, since the measure μ is non-atomic, and this also has the consequence that the sample points ξ_{1}, …, ξ_{n} are different with probability 1.

Formula (9.14) can be rewritten, by expressing its right-hand side with the help of the random variables ξ_{l} instead of the measures μ^{(l)} as

∫ ′ f(x_{1}, …, x_{k}) (μ^{(l_{1})}( dx_{1}) − μ( dx_{1})) … (μ^{(l_{k})}( dx_{k}) − μ( dx_{k})) (9.15)

= ∑_{V ∈T̄ (P)} α(V, P) ( ∏_{j′∈{1,...,k}\V} P_{μ,j′} ∏_{j∈V} Q_{μ,j}) f(ξ_{l_{j}}, j ∈ V)

if (l_{1}, …, l_{k}) ∈ H(P). Here Q_{μ,j} = I − P_{μ,j} is the operator Q_{μ} defined in (9.6′), with the choice Y_{1} which is the product of the first j − 1 components of X^{k}, Z is the j-th component and Y_{2} is the product of the last k − j components of the product space X^{k}. The operator P_{μ,j′} is the operator P_{μ} defined in (9.5′) with the choice of Y_{1} as the product of the first j′ − 1, Z the j-th component and Y_{2} as the product of the last k − j′ components of the space X^{k}. To see why formula (9.15) holds we have to understand that integration with respect to (μ^{(l_{j})}( dx_{j}) − μ( dx_{j})) means the application of the operator Q_{μ,j} and then putting the value ξ_{l_{j}} in the argument x_{j}, while integration with respect to μ( dx_{j′}) means the application of the operator P_{μ,j′}. Beside this, the operators Q_{μ,j} and P_{μ,j′} are exchangeable.

Fix some partition P ∈ P_{k}, a set V ∈ T̄(P) and sum up the expressions at the right-hand side of (9.15) with this set V for all sequences (l_{1}, …, l_{k}) ∈ H(P). We get that

α(V, P) ∑_{(l_{1},...,l_{k})∈H(P)} ( ∏_{j′∈{1,...,k}\V}P_{μ,j′} ∏_{j∈V} Q_{μ,j}) f(ξ_{l_{j}}, j ∈ V) = ᾱ(V, P, k, n)I_{n,|V |}(f_{V})

(9.16) where I_{n,|V |}(f_{v}) is the U-statistic of order |V | with the kernel function

f_{V} (x_{j}, j ∈ V) = ( ∏_{j′∈{1,...,k}\V} P_{μ,j′} ∏_{j∈V} Q_{μ,j}) f  (9.17)

with that function on f(x_{1}, …, x_{k}) which is considered in Theorem 9.4 and some appropriate coefficients ᾱ(V, P, k, n) at the right-hand side of (9.16). These coefficients could be explicitly calculated. We do not need an explicit formula for ᾱ(V, P, k, n), but we shall need the inequality |ᾱ(V, P, k, n)| ≤ D(k)n^{β(P,V)}, where β(P, V) = u−|V | is the number of those components R_{j}, 1 ≤ j ≤ u, of the partition P for which R_{j} ∩ V = ∅, (here u denotes the number of components in the partition P), and the constant D(k) < ∞ depends only on the multiplicity k of the integral J_{n,k}(f).

To show that |ᾱ(V, P, k, n)| ≤ D(k)n^{β(P,V)} let us observe that if we fix the coordinates l_{j}, j ∈ V, in an arbitrary way and sum up the expression at left-hand side of (9.16) for the remaining indices l_{j′}, j′ ∉ V, then we get the term depending on the variables ξ_{l_{j}}, j ∈ V, in the sum defining the U-statistic I_{n,|V |}(f_{V}) multiplied by ᾱ(V, P, k, n). Hence to get a good estimate on ᾱ(V, P, k, n) the number of the vectors (l_{j}, j ∉ V) taking part in the summation at the left-hand side of (9.16) has to be well bounded. For this aim let us consider the class of vectors (l_{1}, …, l_{k}) ∈ H(P). Two coordinates l_{j′} and l_{j′′} must agree if their indices j′ and j′′ belong to the same component of the partition P. Beside this, if the number j is contained in such a component R_{t} of the partition P for which R_{t} ∩ V ≠ ∅, then the coordinate l_{j} of these vectors is fixed. Hence the value l_{j′} of those non-fixed coordinates whose indices j′ belong to the same component R_{t} of the partition P agree and only such components R_{t} have to be considered for which R_{t} ∩ V = ∅. This yields the upper bound n^{β(P,V)} for the number of possible choices of the indices l_{j′}, j′ ∉ V. A more careful consideration shows that the finite limit

C(k, V, P) = lim_{n→∞} n^{−β(P,V)}ᾱ(V, P, k, n),  |C(k, V, P)| < ∞,  (9.18)

also exists.

We get by applying relations (9.12) and (9.15) and summing up relation (9.16) first for all V ∈ T̄(P) for a partition P ∈ P_{k} and then for all P ∈ P that the identity

J_{n,k}(f) = ∑_{V ⊂\{1,2,...,k\}} n^{−|V |/2}C(n, k, V)\frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, \\ l_{j}≠l_{j′} if j≠j′ for j∈V \end{matrix}} f_{V} (ξ_{l_{j}}, j ∈ V) (9.19)

holds with the functions f_{V} (x_{j}, j ∈ V) defined in (9.17) for all V ⊂ {1, …, k} and some appropriate coefficients C(n, k, V). We shall show that these coefficients satisfy the inequality |C(n, k, V)| ≤ C(k) with some constant C(k) > 0. Beside this, it is not difficult to see that the identity C(n, k, {1, …, k}) = 1 holds. To see the estimate |C(n, k, V)| ≤ C(k) observe that n^{−|V |/2}C(n, k, |V |) can be written as a sum of finitely many terms, (their number is less than a number depending only on k) such that all of them can be bounded by a number of the form \frac{|ᾱ(V,P,k,n)|}{n^{k/2}k!} ≤ D(k)n^{−k/2+β(P,V)} with some partition P and the number β(P, V) introduced after formula (9.16) with some P ∈ P_{k} and V ∈ T̄(P). Hence it is enough to show that −\frac{k}{2} + β(P, V) ≤ −\frac{|V |}{2}, i.e. β(P, V) ≤ \frac{k−|V |}{2} if V ∈ T̄(P). This relation clearly holds, since β(P, V) is the number of components of such a partition of a set with less than or equal to k − |V | elements whose components have at least 2 elements.

Relation (9.19) can be rewritten as J_{n,k}(f) = ∑_{V ⊂{1,2,...,k}}C(n, k, V)n^{−|V |/2}I_{n,|V |}(f_{V}),

where I_{n,|V |}(f_{V}) is the U-statistic with the random variables ξ_{1}, …, ξ_{n} and the kernel function f_{V} defined in (9.17). This kernel function agrees with the function f_{V} defined in (9.2). We have also seen that the coefficients C(n, k, V) satisfy the inequality stated in Theorem 9.4. Relation (9.18) together with the bound on the terms β(P, V) also imply that the finite limits lim_{n→∞} C(n, k, V) = C(k, V) also exist. Theorem 9.4 is proved.

Two corollaries of Theorem 9.4 will be formulated. The first one explains the content of conditions (8.2) and (8.5) in Theorems 8.1—8.4.

Corollary 1 of Theorem 9.4. If I_{n,k}(f) is a degenerate U-statistic of order k with some kernel function f, then

E (n^{−k/2}I_{n,k}(f))^{2} = \frac{n(n − 1) ⋯ (n − k + 1)}{k!n^{k}} ∫ Sym f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k})
≤ \frac{1}{k!} ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}),  (9.20)

where μ is the distribution of the random variables taking part in the definition of the Ustatistic I_{n,k}(f), and Sym f is the symmetrization of the function f. The k-fold multiple random integral J_{k,n}(f) with an arbitrary square integrable kernel function f satisfies the inequality

EJ_{n,k}(f)^{2} ≤ C̄(k) ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k})

with some constant C̄(k) depending only on the order k of the integral J_{n,k}(f).

Proof of Corollary 1 of Theorem 9.4. The identity

E(n^{−k/2}I_{n,k}(f))^{2} = \frac{1}{(k!)^{2}n^{k}} ∑′Ef(ξ_{l_{1}}, …, ξ_{l_{k}})f(ξ_{l′_{1}}, …, ξ_{l′_{k}})  (9.21)

holds, where the prime in ∑′ means that summation is taken for such pairs of k-tuples (l_{1}, …, l_{k}), (l′_{1}, …, l′_{k}), 1 ≤ l_{j}, l′_{j} ≤ n, for which l_{j} ≠ l_{j′} and l′_{j} ≠ l′_{j′} if j ≠ j′. The degeneracy of the U-statistic I_{n,k}(f) implies that Ef(ξ_{l_{1}}, …, ξ_{l_{k}})f(ξ_{l′_{1}}, …, ξ_{l′_{k}}) = 0 if the two sets {l_{1}, …, l_{k}} and {l′_{1}, …, l′_{k}} differ. This can be seen by taking such an index l_{j} from the first k-tuple which does not appear in the second one, and by observing that the conditional expectation of the product we consider equals zero by the degeneracy condition of the U-statistic under the condition that the value of all random variables except that of ξ_{l_{j}} is fixed in this product. On the other hand,

Ef(ξ_{l_{1}}, …, ξ_{l_{k}})f(ξ_{l′_{1}}, …, ξ_{l′_{k}}) = ∫ f(x_{1}, …, x_{k})f(x_{π(1)}, …, x_{π(k)})μ( dx_{1}) … μ( dx_{k})

if (l′_{1}, …, l′_{k}) = (π(l_{1}), …, π(l_{k})) with some (π(1), …, π(k)) ∈ Π_{k}, where Π_{k} denotes the set of all permutations of the set {1, …, k}. By summing up the above identities for all pairs (l_{1}, …, l_{k}) and (l′_{1}, …, l′_{k}) and by applying formula (9.21) we get the identity at the left-hand side of formula (9.20). The second relation in (9.20) is obvious.

The bound for J_{n,k}(f) follows from Theorem 9.4, formula (9.4) in Theorem 9.2 by which the L_{2}-norm of the functions f_{V} is not greater than the L_{2}-norm of the function f and the bound that formula (9.20) yields for the second moment of the degenerate Ustatistics n^{−|V |/2}I_{n,|V |}(f_{V}) appearing in the expansion (9.9).

In Corollary 2 the decomposition (9.9) of a random integral J_{n,2}(f) of order 2 is described in an explicit form.

Corollary 2 of Theorem 9.4. Let the random integral J_{n,2}(f) satisfy the conditions of Theorem 9.4. In this case formula (9.9) can be written in the following explicit form:

2J_{n,2}(f) = \frac{2}{n}I_{n,2}(f_{\{1,2\}}) − \frac{1}{n}I_{n,1}(f_{\{1\}}) − \frac{1}{n}I_{n,1}(f_{\{2\}}) − f_{∅}  (9.9′)

with the functions

f_{{1,2}}(x, y) = f(x, y) − ∫ f(x, y)μ( dx) − ∫ f(x, y)μ( dy) + ∫ f(x, y)μ( dx)μ( dy),
f_{{1}}(x) = ∫ f(x, y)μ( dy) − ∫ f(x, y)μ( dx)μ( dy),
f_{{2}}(y) = ∫ f(x, y)μ( dx) − ∫ f(x, y)μ( dx)μ( dy), and
f_{∅} = ∫ f(x, y)μ( dx)μ( dy).

10. Multiple Wiener–Itô integrals and their properties.

In this section I present the definition of multiple Wiener–Itô integrals and some of their most important properties needed in the proof of the results formulated in Section 8. First the notion of the white noise with some reference measure will be introduced, then multiple Wiener–Itô integrals with respect to a white noise with some non-atomic reference measure will be defined. A most important result in the theory of multiple Wiener–Itô integrals is the so-called diagram formula presented in Theorem 10.2A. It enables us to write the product of two Wiener–Itô integrals in the form of a sum of Wiener–Itô integrals. The proof of the diagram formula is given in Appendix B.

Another interesting result about Wiener-Itô integrals, formulated at the end of this section in Theorem 10.5 states that the class of random variables which can be written in the form of a sum of Wiener–Itô integrals of different order is sufficiently rich. All random variables with finite second moment which are measurable with respect to the σ-algebra generated by the (Gaussian) random variables appearing in the underlying white noise in the construction of multiple Wiener–Itô integrals can be written in such a form. This result explains why it is natural to expect a result like the diagram formula. The product of two Wiener–Itô integrals is also measurable with respect to the σ-algebra generated by the random variables in the underlying white noise, hence if we know that such a product is square integrable, then Theorem 10.5 implies that it can be written as the sum of multiple Wiener–Itô integrals of different order. The diagram formula is especially useful for us, because it yields an explicit representation of a product of Wiener–Itô integrals in the form of a sum of Wiener–Itô integrals. I shall also give a heuristic explanation of the diagram formula which may indicate why it has the form appearing in Theorem 10.2A. It also helps to find the analogue of the diagram formula for (random) integrals with respect to the product of normalized empirical measures. Such a result will be useful later.

Once the diagram formula is proved, it is not difficult to generalize it to the product of finitely many Wiener–Itô integrals. This generalization, formulated in Theorem 10.2, will also be called the diagram formula. It has an important corollary about the calculation of the moments of Wiener–Itô integrals. Theorem 8.5 can be proved relatively simply by means of this corollary.

I shall give the proof of two other results about Wiener–Itô integrals in Appendix C. The first one, Theorem 10.3, is called Itô’s formula for Wiener–Itô integrals, and it explains the relation between multiple Wiener-Itô integrals and Hermite polynomials of Gaussian random variables. This result is a relatively simple consequence of the diagram formula and some basic recursive relations about Hermite polynomials.

The other result proved in Appendix C, Theorem 10.4, is a limit theorem about a sequences of appropriately normalized degenerate U-statistics. Here the limit is presented in the form of a multiple Wiener–Itô integral. This result is interesting for us, because it helps to compare Theorems 8.3 and 8.1 with their one-variate counterpart, Bernstein’s inequality. In the one-variate case Bernstein’s inequality provides a comparison of the distribution of sums of independent random variables and normal distribution functions, i.e. the limit distribution in the central limit theorem. Theorem 8.3 yields a similar result about degenerate U-statistics. Its comparison with Theorem 8.5 and the limit theorem proved in Appendix C about the limit distribution of degenerate Ustatistics show that degenerate U-statistics satisfy an estimate similar to Bernstein’s inequality. The upper bound in it is similar to the estimate on the tail-distribution of the limit distribution of normalized degenerate U-statistics, which equals the distribution of an appropriate multiple Wiener–Itô integral. Theorem 8.1 which is an estimate of multiple integrals with respect to a normalized empirical distribution also has such an interpretation.

My Lecture Note [29] contains a rather detailed description of Wiener–Itô integrals. But in that work the emphasis was put on the study of a slightly different version of it. The original version of this integral introduced in [24] was also only briefly discussed there, not all details were worked out. In particular, the diagram formula needed in this work was formulated and proved only for modified Wiener–Itô integrals. I shall discuss the difference between these random integrals together with the question why a modified version of Wiener–Itô integrals was discussed in [29] at the end of the section.

To define multiple Wiener–Itô integrals first the notion of a white noise has to be introduced. This is done in the following definition.

Definition of a white noise with some reference measure. Let us have a σ-finite measure μ on a measurable space (X, X). A white noise with reference measure μ is a Gaussian random field μ_{W} = {μ_{W} (A): A ∈ X , μ(A) < ∞}, i.e. a set of jointly Gaussian random variables indexed by the above sets A, which satisfies the relations Eμ_{W} (A) = 0 and Eμ_{W} (A)μ_{W} (B) = μ(A ∩ B).

It is worth making some comments about this definition.

Remark: In the definition of a white noise sometimes also the property μ_{W} (A ∪ B) = μ_{W} (A) + μ_{W} (B) with probability 1 if A ∩ B = ∅, and μ(A) < ∞, μ(B) < ∞ is mentioned. But this condition can be omitted, because it follows from the remaining properties of the white noise. Indeed, simple calculation shows that E(μ_{W} (A ∪ B) − μ_{W} (A) − μ_{W} (B))^{2} = 0 if A ∩ B = ∅, hence μ_{W} (A ∪ B) − μ_{W} (A) − μ_{W} (B) = 0 with probability 1 in this case. It also can be observed that if some sets A_{1}, …, A_{k} ∈ X, μ(A_{j}) < ∞, 1 ≤ j ≤ k, are disjoint, then the random variables μ_{W} (A_{j}), 1 ≤ j ≤ k, are independent because of the uncorrelatedness of these jointly Gaussian random variables.

It is not difficult to see that for an arbitrary reference measure μ on a space (X, X) a white noise μ_{W} with this reference measure really exists. This follows simply from Kolmogorov’s fundamental theorem, by which if the finite dimensional distributions of a random field are prescribed in a consistent way, then there exists a random field with these finite dimensional distributions.

Now I turn to the definition of multiple Wiener–Itô integrals with respect to a white noise with some reference measure. First I introduce the class of functions whose Wiener–Itô integrals with respect to a white noise μ_{W} with a non-atomic reference measure μ will be defined.

Let us consider a measurable space (X, X), a non-atomic σ-finite measure μ on it and a white noise μ_{W} on (X, X) with reference measure μ. Let us define the classes of functions H_{μ,k}, k = 1, 2, …, consisting of functions of k variables on (X, X) by the formula

H_{μ,k} = {f(x_{1}, …, x_{k}): f(x_{1}, …, x_{k}) is an X^{k} measurable, real valued
function on X^{k}, and ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) …, μ( dx_{k}) < ∞}.

(10.1)

We shall call a σ-finite measure μ on a measurable space (X, X) non-atomic if for all sets

A ∈ X such that μ(A) < ∞ and numbers ε > 0 there is a finite partition A = ⋃_{s=1}^{N} B_{s} of

the set A with the property μ(B_{s}) < ε for all 1 ≤ s ≤ N. There is a formally weaker definition of a non-atomic measures by which a σ-finite measure μ is non-atomic if for all measurable sets A such that 0 < μ(A) < ∞ there exists a B ⊂ A with the property 0 < μ(B) < μ(A). But these two definitions of non-atomic measures are actually equivalent, although this equivalence is far from trivial. I do not discuss this problem here, since it is a little bit outside from the direction of the present work. In our further considerations we shall work with the first definition of non-atomic measures.

The k-fold Wiener-Itô integrals of the functions f ∈ H_{μ,k} with respect to the white noise μ_{W} will be defined in a rather standard way. First they will be defined for some simple functions, called elementary functions, then it will be shown that the integral for this elementary functions have an L_{2} contraction property which makes possible to extend it to the class of functions in H_{μ,k}.

Let us first introduce the following class of elementary functions H̄_{μ,k} of k variables. A function f(x_{1}, …, x_{k}) on (X^{k}, X^{k}) belongs to H̄_{μ,k} if there exist finitely many disjoint measurable subsets A_{1}, …, A_{M}, 1 ≤ M < ∞, of the set X (i.e. A_{j} ∩ A_{j′} = ∅ if j ≠ j′) such that μ(A_{j}) < ∞ for all 1 ≤ j ≤ M, and the function f has the form

f(x_{1}, …, x_{k}) = \{ \begin{matrix}c(j_{1}, …, j_{k}) if (x_{1}, …, x_{k}) ∈ A_{j_{1}} × ⋯ × A_{j_{k}} \\ with some indices (j_{1}, …, j_{k}), 1 ≤ j_{s} ≤ M, 1 ≤ s ≤ k, \\ such that all numbers j_{1}, …, j_{k} are different \\ 0 if (x_{1}, …, x_{k}) ∉ ⋃_{(j_{1},...,j_{k}): 1≤j_{s}≤M, 1≤s≤k, and all j_{1},...,j_{k} are different.} A_{j_{1}} × ⋯ × A_{j_{k}} \end{matrix}

(10.2)

with some real numbers c(j_{1}, …, j_{k}), 1 ≤ j_{s} ≤ M, 1 ≤ s ≤ k, if all j_{1}, …, j_{k} are different numbers. This means that the function f is constant on all k-dimensional rectangles A_{j_{1}} × ⋯ × A_{j_{k}} with different, non-intersecting edges, and it equals zero on the complementary set of the union of these rectangles. The property that the support of the function f is on the union of rectangles with non-intersecting edges is sometimes interpreted so that the diagonals are omitted from the domain of integration of Wiener– Itô integrals.

The Wiener-Itô integral of an elementary function f(x_{1}, …, x_{k}) of the form (10.2) with respect to a white noise μ_{W} with the (non-atomic) reference measure μ is defined by the formula

∫ f(x_{1}, …, x_{k})μ_{W} ( dx_{1}) … μ_{W} ( dx_{k})

= ∑_{1≤j_{s}≤M, 1≤s≤k all j_{1},...,j_{k} are different} c(j_{1}, …, j_{k})μ_{W} (A_{j_{1}}) ⋯ μ_{W} (A_{j_{k}}). (10.3)

(The representation of the function f in (10.2) is not unique, the sets A_{j} can be divided to smaller disjoint sets, but its Wiener–Itô integral defined in (10.3) does not depend on its representation.) The notation

Z_{μ,k}(f) = \frac{1}{k!} ∫ f(x_{1}, …, x_{k})μ_{W} ( dx_{1}) … μ_{W} ( dx_{k}),  (10.4)

will be used in the sequel, and the expression Z_{μ,k}(f) will be called the normalized Wiener–Itô integral of the function f. Such a terminology will be applied also for the Wiener–Itô integrals of all functions f ∈ H_{μ,k} to be defined later.

If f is an elementary function in H̄_{μ,k} defined in (10.2), then its normalized Wiener– Itô integral defined in (10.3) and (10.4) satisfies the relations

Ek!Z_{μ,k}(f) = 0,
E(k!Z_{μ,k}(f))^{2} = ∑_{(j_{1},...,j_{k}): 1≤j_{s}≤M, 1≤s≤k, and all j_{1},...,j_{k} are different.} ∑_{π∈Π_{k}} c(j_{1}, …, j_{k})c(j_{π(1)}, …, j_{π(k)})

Eμ_{W} (A_{j_{1}}) ⋯ μ_{W} (A_{j_{k}})μ_{W} (A_{j_{π(1)}}) ⋯ μ_{W} (A_{j_{π(k)}})
= k! ∫ Sym f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k})
≤ k! ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}),

(10.5)

with Sym f(x_{1}, …, x_{k}) = \frac{1}{k!} ∑_{π∈Π_{k}} f(x_{π(1)}, …, x_{π(k)}), where Π_{k} denotes the set of all

permutations π = {π(1), …, π(k)} of the set {1, …, k}.

The identities written down in (10.5) can be simply checked. The first relation follows from the identity Eμ_{W} (A_{j_{1}}) ⋯ μ_{W} (A_{j_{k}}) = 0 for disjoint sets A_{j_{1}}, …, A_{j_{k}}, which holds, since the expectation of the product of independent random variables with zero expectation is taken. The second identity follows similarly from the identity

Eμ_{W} (A_{j_{1}}) ⋯ μ_{W} (A_{j_{k}})μ_{W} (A_{j′_{1}}) ⋯ μ_{W} (A_{j′_{k}}) = 0

if the sets of indices {j_{1}, …, j_{k}} and {j′_{1}, …, j′_{k}} are different,

Eμ_{W} (A_{j_{1}}) ⋯ μ_{W} (A_{j_{k}})μ_{W} (A_{j′_{1}}) ⋯ μ_{W} (A_{j′_{k}}) = μ(A_{j_{1}}) ⋯ μ(A_{j_{k}})

if {j_{1}, …, j_{k}} = {j′_{1}, …, j′_{k}} i.e. if j′_{1} = j_{π(1)}, …, j′_{k} = j_{π(k)} with some permutation π ∈ Π_{k}, which holds because of the facts that the μ_{W} measure of disjoint sets are independent with expectation zero, and Eμ_{W} (A)^{2} = μ(A). The remaining relations in (10.5) can be simply checked.

It is not difficult to check that

EZ_{μ,k}(f)Z_{μ,k′}(g) = 0  (10.6)

for all functions f ∈ H̄_{μ,k} and g ∈ H̄_{μ,k′} if k ≠ k′, and

Z_{μ,k}(f) = Z_{μ,k}(Sym f)  (10.7)

for all functions f ∈ H̄_{μ,k}.

The definition of Wiener–Itô integrals can be extended to general functions f ∈ H_{μ,k} with the help of the estimate (10.5). But to carry out this extension we still have to know that the class of functions H̄_{μ,k} is a dense subset of the class H_{μ,k} in the Hilbert space L_{2}(X^{k}, X^{k}, μ^{k}), where μ^{k} is the k-th power of the reference measure μ of the white noise μ_{W}. I briefly explain how this property of H̄_{μ,k} can be proved. The non-atomic property of the measure μ is exploited at this point.

To prove this statement it is enough to show that the indicator function of any product set A_{1} × ⋯ × A_{k} such that μ(A_{j}) < ∞, 1 ≤ j ≤ k, but the sets A_{1}, …, A_{k} may be non-disjoint is in the L_{2}(μ^{k}) closure of H̄_{μ,k}. In the proof of this statement it will be exploited that since μ is a non-atomic measure, the sets A_{j} can be represented for all ε > 0 and 1 ≤ j ≤ k as a finite union A_{j} = ⋃_{s} B_{j,s} of disjoint sets B_{j,s} with the

property μ(B_{j,s}) < ε. By means of these relations the product A_{1} × ⋯ × A_{k} can be written in the form

A_{1} × ⋯ × A_{k} = ⋃_{s_{1},...,s_{k}} B_{1,s_{1}} × ⋯ × B_{k,s_{k}}  (10.8)

with some sets B_{j,s_{j}} such that μ(B_{j,s_{j}}) < ε for all sets in this union. Moreover, we may assume, by refining the partitions of the sets A_{j} if this is necessary that any two sets B_{j,s_{j}} and B_{j′,s′_{j′}} in this representation are either disjoint, or they agree. Take such a representation of A_{1} × ⋯ × A_{k}, and consider the set we obtain by omitting those products B_{1,s_{1}} × ⋯ × B_{k,s_{k}} from the union at the right-hand side of (10.8) for which B_{i,s_{i}} = B_{j,s_{j}} for some 1 ≤ i < j ≤ k. The indicator function of the remaining set is in the class H̄_{μ,k}. Hence it is enough to show that the distance between this indicator function and the indicator function of the set A_{1} × ⋯ × A_{k} is less than const. ε in the L_{2}(μ^{k}) norm with some const. which may depend on the sets A_{1}, …, A_{k}, but not on ε. Indeed, by letting ε tend to zero we get from this relation that the indicator function of the set A_{1} × A_{2} × ⋯ × A_{k} is in the closure of H̄_{μ,k} in the L_{2}(μ^{k}) norm.

Hence to prove the desired property of H̄_{μ,k} it is enough to prove the following statement. Take the representation (10.8) of A_{1} × ⋯ × A_{k} (which depends on ε) and an arbitrary pair of integers i and j such that 1 ≤ i < j ≤ k. Then the sum of the measures μ^{k}(B_{1,s_{1}} × ⋯ × B_{k,s_{k}}) of those sets B_{1,s_{1}} × ⋯ × B_{k,s_{k}} at the right-hand side of (10.8) for which B_{i,s_{i}} = B_{j,s_{j}} is less than const. ε. To prove such an estimate observe that the μ^{k} measure of such a set can be bounded by the μ^{k−1} measure of the set we obtain by omitting the i-th term from the product defining it in the following way:

μ^{k}(B_{1,s_{1}} × ⋯ × B_{k,s_{k}}) ≤ εμ^{k−1}(B_{1,s_{1}} × ⋯ × B_{i−1,s_{i−1}} × B_{i+1,s_{i+1}} × ⋯ × B_{k,s_{k}}).

Let us sum up this inequality for all such sets B_{1,s_{1}} × ⋯ × B_{k,s_{k}} at the right-hand side of (10.8) for which B_{i,s_{i}} = B_{j,s_{j}}. The left-hand side of the inequality we get in such a way equals the quantity we want to estimate. The expression at its right-hand side is less than ε ∏_{1≤s≤k, s≠i} μ(A_{s}), since ε-times the μ^{k−1} measure of such disjoint sets are

summed up in it which are contained in the set A_{1} × ⋯ × A_{i−1} × A_{i+1} × ⋯ × A_{k}. In such a way we get the estimate we wanted to prove.

Knowing that H̄_{μ,k} is a dense subset of H_{μ,k} in L_{2}(μ^{k}) norm we can finish the definition of k-fold Wiener–itô integrals in the standard way. Given any function f ∈ H_{μ,k}, a sequence of functions f_{n} ∈ H̄_{μ,k}, n = 1, 2, …, can be defined in such a way that ∫ |f(x_{1}, …, x_{k}) − f_{n}(x_{1}, …, x_{k})|^{2}μ( dx_{1}) … μ( dx_{k}) → 0 as n → ∞. By relation (10.5) the normalizations Z_{μ,k}(f_{n}) of the already defined Wiener–Itô integrals of the functions f_{n}, n = 1, 2, …, constitute a Cauchy sequence in the space of square integrable random variables on the probability space, where the white noise is given. (Observe that the difference of two functions from the class H̄_{μ,k} also belongs to this class.) Hence the limit lim_{n→∞} Z_{μ,k}(f_{n}) exists in L_{2} norm, and this limit can be defined as the normalized Wiener–Itô integral Z_{μ,k}(f) of the function f. The definition of this limit does not depend on the choice of the approximating functions f_{n}, hence it is meaningful. It can be seen that relations (10.5) and (10.6) remain valid for all functions f ∈ H_{μ,k}. The following Theorem 10.1 describes the properties of multiple Wiener–Itô integrals. It contains already proved results. The only still non-discussed part of this Theorem is Property f) of Wiener–Itô integrals. But it is easy to check this property by observing that one-fold Wiener–Itô integrals are (jointly) Gaussian, they are measurable with respect to the σ-algebra generated by the white noise μ_{W}. Beside this, the random variable μ_{W} (A) for a set A ∈ X, μ(A) < ∞, equals the (one-fold) Wiener–Itô integral of the indicator function of the set A.

Theorem 10.1. (Some properties of multiple Wiener–Itô integrals). Let a white noise μ_{W} be given with some non-atomic σ-additive reference measure on a measurable space (X, X). Then the k-fold Wiener–Itô integral of all functions in the class H_{μ,k} introduced in formula (10.1) can be defined, and its normalized version Z_{μ,k}(f) = \frac{1}{k!} ∫ f(x_{1}, …, x_{k})μ_{W} ( dx_{1}) … μ_{W} (dx_{k}) satisfies the following relations:

a) Z_{μ,k}(αf +βg) = αZ_{μ,k}(f)+βZ_{μ,k}(g) for all f, g ∈ H_{μ,k} and real numbers α and β.

b) If A_{1}, …, A_{k} are disjoint sets, μ(A_{j}) < ∞, then the function f_{A_{1},...,A_{k}} defined by the relation f_{A_{1},...,A_{k}}(x_{1}, …, x_{k}) = 1 if x_{1} ∈ A_{1}, …, x_{k} ∈ A_{k}, f_{A_{1},...,A_{k}}(x_{1}, …, x_{k}) = 0 otherwise, satisfies the identity

Z_{μ,k}(f_{A_{1},...,A_{k}}(x_{1}, …, x_{k})) = \frac{1}{k!}μ_{W} (A_{1}) ⋯ μ_{W} (A_{k}).

EZ_{μ,k}(f) = 0, and EZ_{μ,k}^{2}(f) = \frac{1}{k!}∥Sym f∥_{2}^{2} ≤ \frac{1}{k!}∥f∥_{2}^{2}

for all f ∈ H_{μ,k}, where ∥f∥_{2}^{2} = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) is the square of the L_{2} norm of a function f ∈ H_{μ,k}.

d) Relation (10.6) holds for all functions f ∈ H_{μ,k} and g ∈ H_{μ,k′} if k ≠ k′.

e) Relation (10.7) holds for all functions f ∈ H_{μ,k}.

f) The Wiener–Itô integrals Z_{μ,1}(f) of order k = 1 are jointly Gaussian. The smallest σ-algebra with respect to which they are all measurable agrees with the σ-algebra generated by the random variables μ_{W} (A), A ∈ X, μ(A) < ∞, of the white noise.

We have defined Wiener–Itô integrals of order k for all k = 1, 2, . . .. For the sake of completeness let us introduce the class H_{μ,0} for k = 0 which consists of the real constants (functions of zero variables), and put Z_{μ,0}(c) = c. Because of relation (10.7) we could have restricted our attention to Wiener–Itô integrals with symmetric kernel functions. But it turned out more convenient to work also with Wiener–Itô integrals of not necessarily symmetric functions.

Now I formulate the diagram formula for the product of two Wiener–Itô integrals. For this goal some notations have to be introduced. To present the product of the multiple Wiener–Itô integrals of two functions f(x_{1}, …, x_{k}) ∈ H_{μ,k} and g(x_{1}, …, x_{l}) ∈ H_{μ,l} in the form of sums of Wiener–Itô integrals a class of diagrams Γ = Γ(k, l) will be defined. The diagrams γ ∈ Γ(k, l) have vertices (1, 1), …, (1, k) and (2, 1), …, (2, l), and edges ((1, j_{1}), (2, j′_{1})), …, ((1, j_{s}), (2, j′_{s})) with some 1 ≤ s ≤ min(k, l). The indices j_{1}, …, j_{s} in the definition of the edges are all different, and the same relation holds for the indices j′_{1}, …, j′_{s}. All such diagrams γ belongs to Γ(k, l). The set of vertices of the form (1, j), 1 ≤ j ≤ k, will be called the first row, and the set of vertices of the form (2, j′), 1 ≤ j′ ≤ l, the second row of a diagram. We demanded that edges of a diagram can connect only vertices of different rows, and at most one edge may start from each vertex of a diagram.

Given a diagram γ ∈ Γ(k, l) with the set of edges

E(γ) = {(1, j_{1}), (2, j′_{1})), …, ((1, j_{s}), (2, j′_{s})}

let V_{1}(γ) = {(1, 1), …, (1, k)} \ {(1, j_{1}), …, (1, j_{s})} and V_{2}(γ) = {(2, 1), …, (2, l)} \ {(2, j′_{1}), …, (2, j′_{s})} denote the set of vertices in the first and in the second row of the diagram γ respectively from which no edge starts. Put α_{γ}(1, j) = (2, j′) if ((1, j), (2, j′)) ∈ E(γ) and α_{γ}(1, j) = (1, j) if the diagram γ contains no edge of the form ((1, j), (2, j′)) ∈ E(γ). In words, the function α_{γ}(·) is defined on the vertices of the first row of the diagram γ. It replaces a vertex to the vertex it is connected to by an edge of the diagram if there is such a vertex, and it does not change those vertices from which no edge starts. Put |γ| = k+l−2s, i.e. |γ| equals the number of vertices in γ from which no edge starts.

Given two functions f(x_{1}, …, x_{k}) ∈ H_{μ,k} and g(x_{1}, …, x_{l}) ∈ H_{μ,l} let us introduce their product

\begin{matrix}F (x_{(1,1)}, …, x_{(1,k)}, x_{(2,1)}, …, x_{(2,l)}) \\ = F_{f,g}(x_{(1,1)}, …, x_{(1,k)}, x_{(2,1)}, …, x_{(2,l)}) \\ = f(x_{(1,1)}, …, x_{(1,k)})g(x_{(2,1)}, …, x_{(2,l)}) \end{matrix}  (10.9)

together with its modification

\begin{matrix}F̄_{γ}(x_{(1,j)},: (1, j) ∈ V_{1}(γ), x_{(2,1)} …, x_{(2,l)}) \\ = f(x_{α_{γ}(1,1)}, …, x_{α_{γ}(1,k)})g(x_{(2,1)}, …, x_{(2,l)}). \end{matrix}  (10.9a)

(Here the function f(x_{1}, …, x_{k}) is replaced by f(x_{(1,1)}, …, x_{(1,k)}) and the function g(x_{1}, …, x_{l}) by g(x_{(2,1)}, …, x_{(2,l)}).) With the help of the above introduced sets V_{1}(γ), V_{2}(γ) and function α_{γ}(·) let us introduce the functions F_{γ} = F_{γ}(f, g) as

\begin{matrix}F_{γ}(x_{(1,j)}, x_{(2,j′)}: (1, j) ∈ V_{1}(γ), (2, j′) ∈ V_{2}(γ)) \\ = ∫ F̄_{γ}(x_{α_{γ}(1,j)}: (1, j) ∈ V_{1}(γ), x_{(2,1)}, …, x_{(2,l)}) \\ ∏_{(2,j)∈\{(2,1),...,(2,l)\}\V_{2}(γ)} μ( dx_{(2,j)}) \end{matrix}  (10.10)

for all diagrams γ ∈ Γ(k, l). In words: We take the product defined in (10.9), then if the index (1, j) of a variable x_{(1,j)} is connected with the index (2, j′) of some variable x_{(2,j′)} by an edge of the diagram γ, then we replace the variable x_{(1,j)} by x_{(2,j′)} in this product. Finally we integrate the function obtained in such a way with respect to the arguments with indices (2, j′_{1}), …, (2, j′_{s}), i.e. with those vertices of the second row of the diagram γ from which en edge starts. It is clear that F_{γ} is a function of |γ| variables. It depends on those coordinates whose indices are such vertices of γ from which no edge starts.

For the sake of simpler notations we shall also consider Wiener–Itô integrals with such kernel functions whose variables are more generally indexed. If the k-fold Wiener– Itô integral with a kernel function f(x_{1}, …, x_{k}) is well-defined, then we shall say that the Wiener–Itô integral with kernel function f(x_{u_{1}}, …, x_{u_{k}}), where {u_{1}, …, u_{k}} is an arbitrary set with k different elements, is also well defined, and it equals the Wiener–Itô integral with the original kernel function f(x_{1}, …, x_{k}). (We have right to make such a convention since the value of a Wiener–Itô integral does not change if we permute the indices of the variables of the kernel function in an arbitrary way.) In particular, we shall speak about the Wiener–Itô integral of the function F_{γ} defined in (10.10) without reindexing its variables x_{(1,j)} and x_{(2,j′)} ‘in the right way’. Now we can formulate the diagram formula for the product of two Wiener–Itô integrals.

Theorem 10.2A. (The diagram formula for the product of two Wiener– Itô integrals). Let a non-atomic σ-finite measure μ be given on a measurable space (X, X) together with a white noise μ_{W} with reference measure μ, and take two functions f(x_{1}, …, x_{k}) ∈ H_{μ,k} and g(x_{1}, …, x_{l}) ∈ H_{μ,l}. (The classes of functions H_{μ,k} and H_{μ,l} were introduced in (10.1).) Let us consider the class of diagrams Γ(k, l) introduced above together with the functions F_{γ}, γ ∈ Γ(k, l), defined by formulas (10.9), (10.9a) and (10.10) with its help. They satisfy the inequality

∥F_{γ}∥_{2} ≤ ∥f∥_{2}∥g∥_{2} for all γ ∈ Γ(k, l),  (10.11)

where the L_{2} norm of a (generally indexed) function h(x_{u_{1}}, …, x_{u_{s}}) is defined as

∥h∥_{2}^{2} = ∫ h^{2}(x_{u_{1}}, …, x_{u_{s}})μ( dx_{u_{1}}) … μ dx_{u_{s}}).

Beside this, the product Z_{μ,k}(f)Z_{μ,l}(g) of the normalized Wiener–Itô integrals of the functions f and g (the notation Z_{μ,k} was introduced in (10.4)) satisfies the identity

(k!Z_{μ,k}(f))(l!Z_{μ,l}(g)) = ∑_{γ∈Γ(k,l)} |γ|!Z_{μ,|γ|}(F_{γ}) = ∑_{γ∈Γ(k,l)} |γ|!Z_{μ,|γ|}(Sym F_{γ}). (10.12)

Theorem 10.2A will be proved in Appendix B. The following consideration yields a heuristic explanation for it. Actually, it can also be considered as a sketch of proof.

In the theory of general Itô integrals when stochastic processes are integrated with respect to a Wiener processes, one of the most basic results is Itô’s formula about differentiation of functions of Itô integrals. It has a heuristic interpretation by means of the informal ‘identity’ dW^{2} = dt. In the case of general white noises this ‘identity’ can be generalized as μ_{W} ( dx)^{2} = μ( dx). We present a rather informal ‘proof’ of the diagram formula on the basis of this ‘identity’ and the fact that the diagonals are omitted from the domain of integration in the definition of Wiener–Itô integrals.

In this ‘proof’ we fix two numbers k ≥ 1 and l ≥ 1, and consider the product of the Wiener–Itô integrals of the functions f and g of order k and l. This product is a bilinear form of the functions f and g. Hence it is enough to check formula (10.12) for a sufficiently rich class of functions. It is enough to consider functions of the form f(x_{1}, …, x_{k}) = I_{A_{1}}(x_{1}) ⋯ I_{A_{k}}(x_{k}) and g(x_{1}, …, x_{l}) = I_{B_{1}}(x_{1}) ⋯ I_{B_{l}}(x_{l}) with disjoint sets A_{1}, …, A_{k} and disjoint sets B_{1}, …, B_{l}, where I_{A}(x) is the indicator function of a set A. (Here we have exploited that the functions f and g disappear in the diagonals.) Let us divide the sets A_{j} into the union of small disjoint sets D_{j}^{(m)}, 1 ≤ j ≤ k with some fixed number 1 ≤ m ≤ M in such a way that μ(D_{j}^{(m)}) ≤ ε with some fixed ε > 0, and the sets B_{j} into the union of small disjoint sets F_{j}^{(m)}, 1 ≤ j ≤ l, with some fixed number 1 ≤ m ≤ M, in such a way that μ(F_{j}^{(m)}) ≤ ε with some fixed ε > 0. Beside

this, we also require that two sets D_{j}^{(m)} and F_{j′}^{(m′)} should be either disjoint or they

should agree. (The sets D_{j}^{(m)} are disjoint for different indices, and the same relation

holds for the sets F_{j′}^{(m′)}.)

Then the identity

k!Z_{μ,k}(f) = ∏_{j=1}^{k} ( ∑_{m=1}^{M} μ_{W} (D_{j}^{(m)})) and l!Z_{μ,l}(g) = ∏_{j′=1}^{l} ( ∑_{m′=1}^{M} μ_{W} (F_{j′}^{(m′)})) ,

holds, and the product of these two Wiener–Itô integrals can be written in the form of a sum by means of a term by term multiplication. Let us divide the terms of the sum we get in such a way into classes indexed by the diagrams γ ∈ Γ(k, l) in the following

way: Each term in this sum is a product of the form ∏_{j=1}^{k} μ_{W} (D_{j}^{(m_{j})}) ∏_{j′=1}^{l} μ_{W} (F_{j′}^{(m′_{j})}).

Let it belong to the class indexed by the diagram γ with edges ((1, j_{1}), (2, j′_{1})), …, and ((1, j_{s}), (2, j′_{s})) if the elements in the pairs (D_{j_{1}}^{m_{j_{1}}} , F_{j′_{1}}^{m_{j′_{1}}} ), …, (D_{j_{s}}^{m_{js}} , F_{j′_{s}}^{m_{j′_{s}}}) agree, and otherwise all terms are different. Then letting ε → 0 (and taking partitions of the sets D_{j} and F_{j′} corresponding to the parameter ε) the sums of the terms in each class turn to integrals, and our calculation suggests the identity

(k!Z_{μ,k}(f))(l!Z_{μ,l}(g)) = ∑_{γ∈Γ(k,l)} Z̄_{γ}  (10.13)

with

\begin{aligned}Z̄_{γ} & = ∫ f(x_{α_{γ}(1,1)}, …, x_{α_{γ}(1,k)})g(x_{(2,1)}, …, x_{(2,l)}) \\ & μ_{W} ( dx_{α_{γ}(1,1)}) … μ_{W} ( dx_{α_{γ}(1,k)})μ_{W} ( dx_{(2,1)}) … μ_{W} ( dx_{(2,l)}) \end{aligned}  (10.13a)

with the function α_{γ}(·) introduced before formula (10.9). The indices α(1, j) of the arguments in (10.13a) mean that in the case α_{γ}(1, j) = (2, j′) the argument x_{(1,j)} has to be replaced by x_{(2,j′)}. In particular, μ_{W} ( dx_{α(1,j)})μ_{W} ( dx_{(2,j′)}) = μ_{W} ( dx_{(2,j′)})^{2} = μ( dx_{(2,j′)}) in this case because of the ‘identity’ μ_{W} ( dx)^{2} = μ( dx). Hence the above informal calculation yields the identity Z̄_{γ} = |γ|!Z_{μ,|γ|}(F_{γ}). Hence relations (10.13) and (10.13a) imply formula (10.12).

A similar heuristic argument can be applied to get formulas for the product of integrals of normalized empirical distributions or (normalized) Poisson fields, only the starting formula μ_{W} ( dx)^{2} = μ( dx) changes in these cases, some additional terms appear which modify the final result. I return to this question in the next section.

It is not difficult to generalize Theorem 10.2A with the help of some additional notations to a diagram formula about the product of finitely many Wiener–Itô integrals. Let us consider m ≥ 2 Wiener–Itô integrals k_{p}!Z_{μ,k_{p}}(f_{p}), of functions f_{p}(x_{1}, …, x_{k_{p}}) ∈ H_{μ,k_{p}}, of order k_{p} ≥ 1, 1 ≤ p ≤ m, and define a class of diagrams Γ = Γ(k_{1}, …, k_{m}) in the following way.

The diagrams γ ∈ Γ = Γ(k_{1}, …, k_{m}) have vertices of the form (p, r), 1 ≤ p ≤ m, 1 ≤ r ≤ k_{p}. The set of vertices {(p, r): 1 ≤ r ≤ k_{p}} with a fixed number p will be called the p-th row of the diagram γ. A diagram γ ∈ Γ = Γ(k_{1}, …, k_{m}) may have some edges. All edges of a diagram connect vertices from different rows, and from each vertex there starts at most one edge. All diagrams satisfying these properties belong to Γ(k_{1}, …, k_{m}). If a diagram γ contains an edge of the form ((p_{1}, r_{1}), (p_{2}, r_{2})) with p_{1} < p_{2}, then (p_{1}, r_{1}) will be called the upper and (p_{2}, r_{2}) the lower end point of this edge. Let E(γ) = {((p_{1}^{(u)}, r_{1}^{(u)}), (p_{2}^{(u)}, r_{2}^{(u)})), p_{1}^{(u)} < p_{2}^{(u)}, 1 ≤ u ≤ s} denote the set of all edges of a diagram γ (the number of edges in γ was denoted by s = |E(γ)|), and let us also introduce the sets V^{u}(γ) = {((p_{1}^{(u)}, r_{1}^{(u)}), 1 ≤ u ≤ s}, the set of all upper end points and V^{b}(γ) = {((p_{2}^{(u)}, r_{2}^{(u)}), 1 ≤ u ≤ s}, the set of all lower end points of edges in a diagram γ. Let V = V (γ) = {(p, r): 1 ≤ p ≤ m, 1 ≤ r ≤ k_{p}} denote the set of all vertices of γ, and let |γ| = k_{1} + ⋯ + k_{m} − 2|E(γ)| be equal to the number of vertices in γ from which no edge starts. Vertices from which no edge starts will be called free vertices in the sequel. Let us also define the function α_{γ}(p, r) for a vertex (p, r) of the diagram γ in the following way: α_{γ}(p, r) = (p̄, r̄), if there is some pair of integers (p̄, r̄) such that ((p, r), (p̄, r̄)) ∈ E(γ) and p < p̄, i.e. (p, r) ∈ V^{u}(γ) and ((p, r), (p̄, r̄)) ∈ E(γ), and put α_{γ}(p, r) = (p, r) for (p, r) ∈ V (γ) \ V^{u}(γ). In words, the function α_{γ}(·) was defined on the set of vertices V (γ) in such a way that it replaces an upper end point of an edge with the lower end point of this edge, and it does not change the remaining vertices of the diagram.

With the help of the above quantities the appropriate multivariate version of the functions given in (10.9), (10.9a) and (10.10) can be defined. Put

\begin{matrix}F (x_{(p,r)},: 1 ≤ p ≤ m, 1 ≤ r ≤ k_{p}) = F_{f_{1},...,f_{m}}(x_{(p,r)},: 1 ≤ p ≤ m, 1 ≤ r ≤ k_{p}) \\ = ∏_{p=1}^{m} f_{p}(x_{(p,1)}, …, x_{(p,k_{p})}), \end{matrix} (10.14)

F̄_{γ}(x_{(p,r)},: (p, r) ∈ V (γ) \ V^{u}(γ)) = ∏_{p=1}^{m} f_{p}(x_{α_{γ}(p,1)}, …, x_{α_{γ}(p,k_{p})}),  (10.14a)

and

F_{γ}(x_{(p,r)}: (p, r) ∈ V (γ) \ (V^{b}(γ) ∪ V^{u}(γ))
= ∫ F̄_{γ}(x_{(p,r)},: (p, r) ∈ V (γ) \ V^{b}(γ)) ∏_{(p,r)∈V^{b}(γ)} μ( dx_{(p,r)}). (10.15)

With the help of the above notations the diagram formula for the product of finitely many Wiener–Itô integrals can be formulated.

Theorem 10.2. (The diagram formula for the product of finitely many Wiener–Itô integrals). Let a non-atomic σ-finite measure μ be given on a measurable space (X, X) together with a white noise μ_{W} with reference measure μ. Take m ≥ 2 functions f_{p}(x_{1}, …, x_{k_{p}}) ∈ H_{μ,k_{p}} with some order k_{p} ≥ 1, 1 ≤ p ≤ m. Let us consider the class of diagrams Γ(k_{1}, …, k_{m}) introduced above together with the functions F_{γ}, γ ∈ Γ(k_{1}, …, k_{m}), defined by formulas (10.14), (10.14a) and (10.15) with its help. The L_{2}-norm of these functions satisfies the inequality

∥F_{γ}∥_{2} ≤ ∏_{p=1}^{m} ∥f_{p}∥_{2} for all γ ∈ Γ(k_{1}, …, k_{m}).  (10.16)

Beside this, the product ∏_{p=1}^{m} Z_{μ,k_{p}}(f_{p}) of the normalized Wiener–Itô integrals of the

functions f_{p}, 1 ≤ p ≤ m, satisfies the identity

∏_{p=1}^{m} k_{p}!Z_{μ,k_{p}}(f_{p}) = ∑_{γ∈Γ(k_{1},...,k_{m})} |γ|!Z_{μ,|γ|}(F_{γ}) = ∑_{γ∈Γ(k_{1},...,k_{m})} |γ|!Z_{μ,|γ|}(Sym F_{γ}).

(10.17)

Theorem 10.2 can be relatively simply derived from Theorem 10.2A by means of induction with respect the number of terms whose product we consider. We still have to check that with the introduction of an appropriate notation Theorem 10.2A remains valid also in the case when the function f is a constant.

Let us also consider the case when f = c and g ∈ H_{μ,l}. In this case we apply the convention Z_{μ,0}(c) = c, define the class of diagrams Γ(0, l) that consists only of one diagram γ whose first row is empty, its second row contains the vertices (2, 1), …, (2, l), and it has no edges. Beside this, we define F_{γ}(x_{(2,1)}, …, x_{(2,l)}) = cg(x_{(2,1)}, …, x_{(2,l)}) in this case. With such a convention Theorem 10.2A can be extended to the case of the product of two Wiener–Itô integrals of order k ≥ 0 and l ≥ 1. Theorem 10.2 can be derived from this slightly generalized result by induction.

By statement c) of Theorem 10.1 all Wiener–Itô integrals of order k ≥ 1 have expectation zero. This fact together with Theorem 10.2 enable us to compute the expectation of a product of Wiener–Itô integrals. Theorem 10.2 makes possible to rewrite a product of Wiener–Itô integrals as a sum of Wiener–Itô integrals. Then its expectation can be calculated by taking the expected value of each term and summing them up. Only constant terms yield a non-zero contribution to this expectation. These constant terms agree with the functions F_{γ} corresponding to diagrams with no free vertices. The next corollary writes down the result we get in such a way.

Corollary of Theorem 10.2 about the expectation of a product of Wiener– Itô integrals. Let a non-atomic σ-finite measure μ be given on a measurable space (X, X) together with a white noise μ_{W} with reference measure μ. Take m ≥ 2 functions f_{p}(x_{1}, …, x_{k_{p}}) ∈ H_{μ,k_{p}}, and consider their Wiener–Itô integrals Z_{μ,k_{p}}(f_{p}), 1 ≤ p ≤ m. The expectation of the product of these random variables satisfies the identity

E (∏_{p=1}^{m} k_{p}!Z_{μ,k_{p}}(f_{p})) = ∑_{γ∈Γ̄(k_{1},...,k_{m})} F_{γ},  (10.18)

where Γ̄(k_{1}, …, k_{m}) denotes the set of all such diagrams γ ∈ Γ(k_{1}, …, k_{m}) which have no free vertices, i.e. |γ| = 0. Such diagrams will be called closed in the sequel. (If Γ̄(k_{1}, …, k_{m}) is empty, then the sum at the right-hand side of (10.17) equals zero.) The functions F_{γ} for γ ∈ Γ̄(k_{1}, …, k_{m}) are constants, and they satisfy the inequality

|F_{γ}| ≤ ∏_{p=1}^{m} ∥f_{p}∥_{2} for all γ ∈ Γ̄(k_{1}, …, k_{m}).  (10.19)

Proof of the Corollary. Relation (10.18) is a straight consequence of formula (10.17), part c) of Theorem 10.1 and the identity Z_{μ,0}(F_{γ}) = F_{γ}, if |γ| = 0. Relation (10.19) follows from (10.16).

The next result I formulate, Itô’s formula for multiple Wiener–Itô integrals, can also be considered as a consequence of the diagram formula. It will be proved in Appendix C.

Theorem 10.3. (Itô’s formula for multiple Wiener–Itô integrals). Let a nonatomic σ-finite measure μ be given on a measurable space (X, X) together with a white noise μ_{W} with reference measure μ. Let us take some real valued, orthonormal functions φ_{1}(x), …, φ_{m}(x) on the measure space (X, X , μ). Let H_{k}(u) denote the k-th Hermite polynomial with leading coefficient 1. Take the one-fold Wiener–Itô integrals η_{p} = Z_{μ,1}(φ_{p}), 1 ≤ p ≤ m, and introduce the random variables H_{k_{p}}(η_{p}), 1 ≤ p ≤ m,

with some integers k_{p} ≥ 1, 1 ≤ p ≤ m. Put K_{p} = ∑_{j=1}^{p} k_{r}, 1 ≤ p ≤ m, K_{0} = 0. Then

η_{1}, …, η_{m} are independent, standard normal random variables, and the identity

\begin{aligned}∏_{p=1}^{m} H_{k_{p}}(η_{p}) & = K_{m}!Z_{μ,K_{m}} (∏_{p=1}^{m} ( ∏_{j=K_{p−1}+1}^{K_{p}} φ_{p}(x_{j}))) \\ & = K_{m}!Z_{μ,K_{m}} (Sym (∏_{p=1}^{m} ( ∏_{j=K_{p−1}+1}^{K_{p}} φ_{p}(x_{j})))) \end{aligned}  (10.20)

holds. In particular, for a single real valued function φ(x) such that ∫ φ^{2}(x)μ( dx) = 1

H_{k} (∫ φ(x)μ_{W} ( dx)) = ∫ φ(x_{1}) ⋯ φ(x_{k})μ_{W} ( dx_{1}) … μ_{W} ( dx_{k}).  (10.21)

I also formulate a limit theorem about the distribution of normalized degenerate U-statistics. The limit distribution in this result can be described by means of multiple Wiener–Itô integrals. It will be proved in Appendix C.

Theorem 10.4. (Limit theorem about normalized degenerate U-statistics). Let us consider a sequence of degenerate U-statistics I_{n,k}(f) of order k, n = k, k+1, …, defined in (8.7) with the help of a sequence of independent and identically distributed random variables ξ_{1}, ξ_{2}, … taking values in a measurable space (X, X) with a nonatomic distribution μ and a kernel function f(x_{1}, …, x_{k}), canonical with respect to the measure μ, defined on the k-fold product (X^{k}, X^{k}) of the space (X, X) for which ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) < ∞. Then the sequence of normalized U-statistics n^{−k/2}I_{n,k}(f) converges in distribution, as n → ∞, to the k-fold Wiener–Itô integral

Z_{μ,k}(f) = \frac{1}{k!} ∫ f(x_{1}, …, x_{k})μ_{W} (dx_{1}) … μ_{W} (dx_{k})

with kernel function f(x_{1}, …, x_{k}) and a white noise μ_{W} with reference measure μ.

Remark. The limit behaviour of degenerate U-statistics I_{n,k}(f) with an atomic measure μ which satisfy the remaining conditions of Theorem 10.4 can be described in the following way. Take the probability space (U, U, λ), where U = [0, 1], U is the Borel σ-algebra and λ is the Lebesgue measure on it. Introduce a sequence of independent random variables η_{1}, η_{2}, … with uniform distribution on the interval [0, 1], which is independent also of the sequence ξ_{1}, ξ_{2}, . . .. Define the product space (X̃, X̃, μ̃) = (X × U, X × U, μ × λ) together with the function f̃(x̃_{1}, …, x̃_{k}) = f̃((x_{1}, u_{1}), …, (x_{k}, u_{k})) = f(x_{1}, …, x_{k}) with the notation x̃ = (x, u) ∈ X × U, and ξ̃_{j} = (ξ_{j}, η_{j}), j = 1, 2, . . .. Then I_{n,k}(f) = I_{n,k}(f̃) (with the above defined function f̃ and μ̃ distributed random variables ξ̃_{j}). Beside this, Theorem 10.4 can be applied for the degenerate U-statistics I_{n,k}(f̃), n = 1, 2, . . ..

In the next result I give an interesting representation of the Hilbert space consisting of the square integrable functions measurable with respect to a white noise μ_{W}. An isomorphism will be given with the help of Wiener–Itô integrals between this Hilbert space and the so-called Fock space to be defined below. To formulate this result first some notations will be introduced.

Let H_{μ,k}^{0} ⊂ H_{μ,k} denote the class of symmetric functions in the space H_{μ,k}, k = 0, 1, 2, …, i.e. f ∈ H_{μ,k} is in its subspace H_{μ,k}^{0} if and only if f(x_{1}, …, x_{k}) = Sym f(x_{1}, …, x_{k}). Let us introduce for all k = 0, 1, 2, … the Hilbert space G_{k} consisting of those random variables η (on the probability space where the white noise μ_{W} is defined) which can be written in the form

η = Z_{μ,k}(f) = \frac{1}{k!} ∫ f(x_{1}, …, x_{k})μ_{W} ( dx_{1}) … μ_{W} ( dx_{k}) with some f ∈ H_{k,μ}^{0}.

It follows from part a) and c) of Theorem 10.1 that the map f → Z_{μ,k}(f) is a linear transformation of H_{μ,k}^{0} to G_{k}, and \frac{1}{k!}∥f∥_{2}^{2} = EZ_{μ,k}^{2}(f) for all f ∈ H_{μ,k}^{0}, where ∥f∥_{2} denotes the usual L_{2}-norm of the function f with respect to the k-fold power of the measure μ. By the definition of Wiener–Itô integrals the set G_{1} consists of jointly Gaussian random variables with expectation zero. The spaces H_{μ,0} and G_{0} consist of the real constants. Let us define the space Exp (H_{μ}) of infinite sequences f = (f_{0}, f_{1}, …),

f_{k} ∈ H_{μ,k}^{0}, k = 0, 1, 2, …, such that ∥f∥_{2}^{2} = ∑_{k=0}^{∞} \frac{1}{k!}∥f_{k}∥_{2}^{2} < ∞. The space Exp (H_{μ})

with the natural addition and multiplication by a constant and the above introduced norm ∥f∥_{2} for f ∈ Exp (H_{μ}) is a Hilbert space which is called the Fock space in the literature.

Let G denote the class of random variables of the form

Z(f) = ∑_{k=0}^{∞} Z_{μ,k}(f_{k}), f = (f_{0}, f_{1}, f_{2}, …) ∈ Exp (H_{μ}).

The next result describes the structure of the space of random variables G. It is useful for a better understanding of Wiener–Itô integrals, but it will be not used in the sequel. In its proof I shall refer to some basic measure theoretical results.

Theorem 10.5. (Isomorphism of the space of square integrable random variables measurable with respect to a white noise with a Fock space). Let a non-atomic σ-finite measure μ be given on a measurable space (X, X) together with a white noise μ_{W} with reference measure μ. Let us consider the class of functions H_{μ,k}^{0}, k = 0, 1, 2, …, and Exp (H_{μ}) together with the spaces of random variables G_{k},

k = 0, 1, 2, …, and G defined above. The transformation Z: Z(f) = ∑_{k=0}^{∞} Z_{μ,k}(f_{k}),

f = (f_{0}, f_{1}, f_{2}, …) ∈ Exp (H_{μ}), is a unitary transformation from the Hilbert spaces Exp (H_{μ}) to G. The Hilbert space G consists of all random variables with finite second moment, measurable with respect to the σ-algebra generated by the random variables μ_{W} (A), A ∈ X, μ(A) < ∞. This σ-algebra agrees with the σ-algebra generated by the random variables Z_{μ,1}(f_{1}), f_{1} ∈ H_{μ,1}^{0}.

Proof of Theorem 10.5. Properties a) and c) in Theorem 10.1 imply that the transformation f_{k} → Z_{μ,k}(f_{k}) is a linear transformation of H_{μ,k}^{0} to G_{k}, and \frac{1}{k!}∥f_{k}∥_{2}^{2} = EZ_{μ,k}(f)^{2}. Beside this, EZ_{μ,k}(f)Z_{μ,k′}(f′_{k′}) = 0 if f_{k} ∈ H_{μ,k}^{0}, and f′_{k′} ∈ H_{μ,k′}^{0} with k ≠ k′ by properties d) and c). (The latter property is needed to guarantee this relation also holds if

k = 0 or k′ = 0.) From these relations follows that the map Z: Z(f) = ∑_{k=0}^{∞} Z_{μ,k}(f_{k}),

f = (f_{0}, f_{1}, f_{2}, …) ∈ Exp (H_{μ}) is an isomorphism between the Hilbert spaces Exp (H_{μ}) and G.

It remained to show that G contains all random variables with finite second moment, measurable with respect to the corresponding σ-algebra. Let g_{j}(u), j = 1, 2, …, be an orthonormal basis in H_{μ,1}^{0} = H_{μ,1}, and introduce the random variables η_{j} = Z_{μ,1}(g_{j}), j = 1, 2, . . .. By Itô’s formula for Wiener–Itô integrals (Theorem 10.3) these random variables are independent with standard normal distribution, and all expressions of the form H_{r_{1}}(η_{j_{1}}) … H_{r_{p}}(η_{j_{p}}) with r_{1} + ⋯ + r_{p} = k are in the space G_{k}, where H_{r}(·) denotes the Hermite polynomial of order r with leading coefficient 1. To prove the desired statement by means of these relations we still need the following results from the classical analysis:

a) Hermite polynomials constitute a complete orthonormal system in the L_{2}-space on the real line with respect to the standard normal distribution. (This result will be proved in Section C in Proposition C2.)

b) If a random variable ζ is measurable with respect to the σ-algebra generated by some random variables η_{1}, η_{2}, …, then there exists a Borel measurable function f(x_{1}, x_{2}, …) on the infinite product of the real line (R^{∞}, B^{∞}) in such a way that ζ = f(η_{1}, η_{2}, …).

This means in our case that any random variable ζ measurable with respect to the σ-algebra generated by the random variables η_{j} = Z_{μ,1}(g_{j}), j = 1, 2, …, can be written in the form ζ = f(η_{1}, η_{2}, …) with the above introduced independent, standard normal random variables η_{1}, η_{2}, . . .. If ζ has finite second moment, then the function f appearing in its representation is a function of finite L_{2}-norm in the infinite product of the real line with the infinite product of the standard normal distribution on it. Hence some classical results in analysis enable us to expand the function f with respect to products of Hermite polynomials, and this also yields the identity

ζ = ∑ c(j_{1}, r_{1}, …, j_{s}, r_{s})H_{r_{1}}(η_{j_{1}}) ⋯ H_{r_{s}}(η_{j_{s}})

with some coefficients c(j_{1}, r_{1}, …, j_{s}, r_{s}) such that

∑ c^{2}(j_{1}, r_{1}, …, j_{s}, r_{s})∥H_{r_{1}}(u)∥^{2} ⋯ ∥H_{r_{s}}(u)∥^{2} < ∞.

(Actually it is known that ∥H_{k}(u)∥^{2} = k!, but here we do not need this knowledge.)

The above relations yield the desired representation of a random variable ζ with finite second moment, if it is measurable with respect to the σ-algebra generated by the

random variables in G_{1}. Indeed, the identity ζ = ∑_{k=0}^{∞} ζ_{k} holds with

ζ_{k} = ∑_{r_{1}+···+r_{s}=k} c(j_{1}, r_{1}, …, j_{s}, r_{s})H_{r_{1}}(η_{j_{1}}) ⋯ H_{r_{s}}(η_{j_{s}}),

and ζ_{k} ∈ G_{k} by Itô’s formula.

To complete the proof it is enough to remark that the σ-algebra generated by the random variables η_{1}, η_{2}, … and μ_{W} (A), A ∈ X, μ(A) < ∞ agree, as it was stated in part f) of Theorem 10.1.

The results about Wiener–Itô integrals discussed in this Section are useful in the study of non-linear functionals of a set of jointly Gaussian random variables defined by means of a white noise. In my Lecture Note [29] similar problems were discussed, but in that work a slightly different version of Wiener–Itô integrals was introduced. The reason for it was that the solution of the problems studied in [29] demanded different methods.

In work [29] stationary Gaussian random fields were considered, and the main problem studied there was the description of the limit distribution of certain sequences of non-linear functionals of such Gaussian random fields. In a stationary Gaussian random field a shift operator can be introduced. The shift of all random variables measurable with respect to the underlying stationary Gaussian random field can be defined. In [29] we needed a technique which helps in working with the shift operator. Fourier analysis is a useful tool in the study of the shift operator. In the work [29] we tried to unify the tools of multiple Wiener–Itô integrals and Fourier analysis. This led to the definition of a slightly different version of Wiener–Itô integrals.

The idea behind this definition was the observation that not only the correlation function of a stationary Gaussian field can be expressed by means of the Fourier transform of its spectral measure, but also a random spectral measure can be constructed whose Fourier transform expresses the stationary Gaussian process itself. After the introduction of this random spectral measure a version of the multiple Wiener–Itô integral can be defined with respect to it, and all square integrable random variables, measurable with respect to the σ-algebra generated by the underlying Gaussian stationary random field can be expressed with its help. Moreover, it enables us to apply the methods of multiple Wiener–Itô integrals and Fourier analysis simultaneously. In [29] such a method was worked out. The modified Wiener–Itô integral introduced there shows a behaviour similar to that of the original Wiener–Itô integral, only it has to be taken into account that the random spectral measure behaves not like a white noise, but as its ‘Fourier transform’. I omit the details. They can be found in [29].

The spaces G_{k} consisting of all k-fold Wiener–Itô integrals were introduced also in [29], and this was done for a special reason. In that work the Hilbert space of square integrable functions, measurable with respect to an underlying stationary Gaussian field was studied together with the shift operator acting on this Gaussian field, which could be extended to a unitary operators on this Hilbert space. It was useful to decompose the Hilbert space we were working with to the direct sum of orthogonal subspaces, invariant with respect to the shift operator. The spaces G_{k} were elements of such a decomposition.

In the present work no shift operator was defined, and no limit theorem was studied for non-linear functionals of a Gaussian field. Here the introduction of the spaces G_{k} was useful because of a different reason. In the study of our problems we shall need good estimates on the 2p-th moment of random variables, measurable with respect to the underlying white noise for large numbers p. As it will be shown, the high moments of the random variables in the spaces G_{k} with different indices k show an essentially different behaviour. For a large number p the p-th moment of a random variable in G_{k} behaves similarly to that of the k-th power ξ^{k} of a Gaussian random variable ξ with zero expectation. An estimate of this type will be formulated in Proposition 13.1 or in its consequence, in formula (13.2) and in a partial converse of this result, in Theorem 13.6.

11. The diagram formula for products of degenerate U-statistics.

There is a natural analog of the diagram formula for the products of Wiener–Itô integrals both for the products of multiple integrals with respect to normalized empirical measures and for the products of degenerate U-statistics. These two results are closely related. They express the products of multiple random integrals or of degenerate U-statistics as a sum of multiple random integrals or degenerate U-statistics respectively. In this work the diagram formula for multiple integrals with respect to a normalized empirical measure will be discussed only at an informal level, while a complete proof of the analogous result about degenerate U-statistics will be given. The reason for such an approach is that the diagram formula for the product of degenerate U-statistics is more useful in the study of the problems discussed in this work.

We want to get good estimates about the high moments both of multiple random integrals and of degenerate U-statistics. In the case of degenerate U-statistics the diagram formula yields an explicit formula for these moments. It expresses the product whose expected value has to be calculated as a sum of degenerate U-statistics of different order. Beside this the expected value of all degenerated U-statistics of order k ≥ 1 equals zero. Hence the expected value we are interested in equals the sum of the zero order terms appearing in the diagram formula.

The analogous problem about the moments of multiple integrals with respect to a normalized empirical measure is more difficult. The diagram formula enables us to express these moments as the sum of the expectation of multiple random integrals of different order also in this case. But the expected value of random integrals of order k ≥ 1 with respect to a normalized empirical distribution may be non-zero. It was shown in an example presented before the proof of Theorem 9.4 that this is really possible.

First I give an informal description of the diagram formula for the product of two random integrals with respect to a normalized empirical measure. Its analog, the diagram formula for the product of two Wiener–Itô integrals can be described in an informal way by means of formulas (10.13) and (10.13a) together with the ‘identity’ μ_{W} ( dx)^{2} = μ( dx) in their interpretation. The diagram formula for the product of two multiple integrals with respect to a normalized empirical measure has a similar representation. (Observe that in the definition of the random integral J_{n,k}(·) given in formula (4.8) the diagonals are omitted from the domain of integration, similarly to the case of Wiener–Itô integrals.) In this case such a version of formulas (10.13) and (10.13a) can be applied, where the random integrals Z_{μ,k} are replaced by J_{n,k}, and the white noise measures μ_{W} are replaced by the normalized empirical measures ν_{n} = \sqrt{n}(μ_{n}−μ). But the analog of the ‘identity’ μ_{W} ( dx)^{2} = μ( dx) needed in the interpretation of these formulas has a different form. Namely, it states that ν_{n}( dx)^{2} = μ( dx) + \frac{1}{\sqrt{n}}ν_{n}( dx). Let us ‘prove’ this new ‘identity’.

Take a small set Δ, i.e. a set Δ such that μ(Δ) is very small, write down the identity ν_{n}(Δ)^{2} = nμ_{n}(Δ)^{2} + nμ(Δ)^{2} − 2nμ_{n}(Δ)μ(Δ), and observe that only a second order error is committed if the terms nμ(Δ)^{2} and 2nμ_{n}(Δ)μ(Δ) are omitted at the righthand side of this identity. Moreover, also a second order error is committed if nμ_{n}(Δ)^{2} is replaced by μ_{n}(Δ), because it has second order small probability that there are at least two sample points in the small set Δ. On the other hand, nμ_{n}(Δ)^{2} = μ_{n}(Δ) if Δ contains only zero or one sample point. The above considerations suggest that ν_{n}( dx)^{2} = μ_{n}( dx) = μ( dx) + \frac{1}{\sqrt{n}}[\sqrt{n}(μ_{n}( dx) − μ( dx))] = μ( dx) + \frac{1}{\sqrt{n}}ν_{n}( dx). (This means that in the ‘identity’ expressing the square ν_{n}( dx)^{2} of a normalized empirical measure a correcting term \frac{1}{\sqrt{n}}ν_{n}( dx) appears. If the sample size n → ∞, then the normalized empirical measure tends to a white noise with counting measure μ, and this correcting term disappears.)

In paper [32] the diagram formula for the product of two multiple integrals with respect to a normalized empirical measure is proved with a different notation. It says that the identity suggested by the above heuristic argument really holds. This result may also help in the proof of the diagram formula for degenerate U-statistics. But a direct proof of this result seems to be simpler.

In the proof of the diagram formula for the product of two degenerate U-statistics first we write this product as the sum of (not necessarily degenerate) U-statistics. Then by applying Hoeffding’s decomposition for each term in this sum the product of two degenerate U-statistics can also be written as a sum of degenerate U-statistics. Actually we apply a slightly refined version of the Hoeffding decomposition where we exploit that we took the product of two degenerated U-statistics. Such a calculation yields the diagram formula for the product of two degenerate U-statistics. With the help of a good notation and some additional work also the product of several degenerate U-statistics can be written as the sum of appropriate degenerate U-statistics. In such a way we get the general form of the diagram formula for the product of degenerate U-statistics.

In this section I formulate the diagram formula for the product of two and finitely many degenerate U-statistics together with an estimate about the L_{2}-norm of the kernel functions of the degenerate U-statistics appearing in the diagram formula, and a formula about the expectation of products of degenerate U-statistics. To formulate these results some new notations have to be introduced. The proofs of the results in this section are postponed to the next section.

In the formulation of the diagram formula for the product of degenerate U-statistics a more general class of diagrams have to be considered than in the case of multiple Wiener–Itô integrals. We shall define these new diagrams under the name coloured diagrams. Tthe kernel functions of the U-statistics appearing in the diagram formula will be introduced with their help.

A class of coloured diagrams Γ(k_{1}, …, k_{m}) will be defined whose vertices will be the pairs (p, r), 1 ≤ p ≤ m, 1 ≤ r ≤ k_{p}, and the set of vertices (p, r), 1 ≤ r ≤ k_{p}, with a fixed number p will be called the p-th row of the diagram. To define the coloured diagrams of the class Γ(k_{1}, …, k_{m}) first the notions of chain and coloured chain will be introduced. A sequence β = {(p_{1}, r_{1}), …, (p_{s}, r_{s})} with 1 ≤ p_{1} < p_{2} < ⋯ < p_{s} ≤ m and 1 ≤ r_{u} ≤ k_{p_{u}} for all 1 ≤ u ≤ s will be called a chain. The number s of the pairs (p_{u}, r_{u}) in this sequence, denoted by ℓ(β), will be called the length of the chain β. Chains of length ℓ(β) = 1, i.e. chains consisting only of one element (p_{1}, r_{1}) are also allowed. We shall define a function c(β) = ±1 which will be called the colour of the chain β, and the pair (β, c(β)) will be called a coloured chain. We shall allow arbitrary colouring c(β) = ±1 of a chain with the only restriction that a chain of length 1 can only get the colour −1, i.e. c(β) = −1 if ℓ(β) = 1.

A coloured diagram γ ∈ Γ(k_{1}, …, k_{m}), γ = {β(l_{1}), …, β(l_{s})} is a partition of the set {(p, r): 1 ≤ p ≤ m, 1 ≤ r ≤ k_{p}} to the union of some coloured chains β(l_{1}), …, β(l_{s}), i.e. each vertex (p, r) is the element of exactly one chain β(l_{j}) ∈ γ. Beside this, each chain β(l_{j}) of a diagram γ has a colour c_{γ}(β(l_{j})) = ±1. The set Γ(k_{1}, …, k_{m}) consists of all partitions of the set of vertices {(p, r), 1 ≤ p ≤ m, 1 ≤ r ≤ k_{p}} to coloured chains, where an arbitrary colouring of the chains with the numbers ±1 is allowed with the only restriction that for a chain β ∈ γ of length ℓ(β) = 1 of a diagram γ ∈ Γ(k_{1}, …, k_{m}) c_{γ}(β) = −1. In our notation we have introduced an indexation β(l_{s}) = β(l_{s}, γ), 1 ≤ l_{1} < l_{2} < ⋯ < l_{s}, of the chains of a coloured diagram γ ∈ Γ(k_{1}, …, k_{m}). Both the number s and the indices l_{1}, …, l_{s} may depend on γ. Such a notation will be useful in our later considerations. It also turned out useful to allow more general indexation of these chains with numbers l_{1}, …, l_{s} and not only with the numbers 1, …, s.

We shall also introduce an enumeration of the vertices of a coloured diagram γ ∈ Γ(k_{1}, …, k_{m}) with the help of the enumeration of its chains. Given a coloured diagram γ = (β(l_{1}), …, β(l_{s})) ∈ Γ(k_{1}, …, k_{m}) we define the indices α_{γ}(p, r) of a vertex (p, r) of this diagram by the formula α_{γ}(p, r) = l_{j} if (p, r) ∈ β(l_{j}). We shall divide the set of indices {l_{1}, …, l_{s}} of the chains contained in a coloured diagram γ into two disjoint sets O(γ) = {l_{j}: 1 ≤ j ≤ s, c_{γ}(β(l_{j})) = −1}, called the set of open indices of the diagram γ and C(γ) = {l_{j}: 1 ≤ j ≤ s, c_{γ}(β(l_{j})) = 1}, called the set of closed indices of the diagram γ. We shall also list the elements of O(γ) in an increasing order, i.e. write O(γ) = {l̄_{1}, …, l̄_{|O(γ)|}}, l̄_{1} < l̄_{2} < ⋯ < l̄_{|O(γ)|}. (We shall denote the cardinality of a finite set A by |A| in the sequel.) We defined the coloured diagrams and introduced their open and closed indices, because, as we shall see, in the diagram formula such degenerate U-statistics appear whose kernel functions are defined with the help of these coloured diagrams, and the indices of the arguments of the kernel function corresponding to the coloured diagram γ are closely related to the chains of γ with colour −1, hence to the open indices of γ.

In the diagram formula we express the product ∏_{p=1}^{m} I_{n,k_{p}}(f_{p}) of degenerate U-

statistics with canonical kernel functions f_{p} of k_{p} variables as the sum of appropriate degenerate U-statistics. The kernel functions of the degenerate U-statistics appearing in this representation of the product of degenerate U-statistics will depend on the above defined coloured diagrams γ, and they will be denoted by F_{γ}, γ ∈ Γ(k_{1}, …, k_{m}). In the definition of these functions F_{γ} we shall apply the operators introduced below.

Given a function h(x_{u_{1}}, …, x_{u_{r}}) with coordinates in the space (X, X) (the indices u_{1}, …, u_{r} are all different, otherwise they can be chosen in an arbitrary way) and a probability measure μ on the space (X, X) let us introduce its transforms P_{u_{j}}h and Q_{u_{j}}h, 1 ≤ j ≤ r, by the formulas

(P_{u_{j}}h)(x_{u_{l}}: u_{l} ∈ {u_{1}, …, u_{r}} \ {u_{j}}) = ∫ h(x_{u_{1}}, …, x_{u_{r}})μ( dx_{u_{j}}), 1 ≤ j ≤ r, (11.1)

and

(Q_{u_{j}}h)(x_{u_{1}}, …, x_{u_{r}}) = h(x_{u_{1}}, …, x_{u_{r}}) − ∫ h(x_{u_{1}}, …, x_{u_{r}})μ( dx_{u_{j}}), 1 ≤ j ≤ r.

(11.2)

(These formulas are very similar to the definition of the operators P_{j} and Q_{j} introduced in formula (9.1) before the proof of the Hoeffding decomposition.)

First we consider the product of two degenerate U-statistics, i.e. the case m = 2. Let us have a measurable space (X, X) with a probability measure μ on it together with two measurable functions f_{1}(x_{1}, …, x_{k_{1}}) and f_{2}(x_{1}, …, x_{k_{2}}) of k_{1} and k_{2} variables on this space which are canonical with respect to the measure μ. Let ξ_{1}, ξ_{2}, … be a sequence of (X, X) valued, independent and identically distributed random variables with distribution μ. We want to express the product I_{n,k_{1}}(f_{1})I_{n,k_{2}}(f_{2}) of degenerate U-statistics defined with the help of the above random variables and kernel functions f_{1} and f_{2} as a sum of degenerate U-statistics. For this goal we introduce some notations.

Given two functions f_{1}(x_{1}, …, x_{k_{1}}) and f_{2}(x_{1}, …, x_{k_{2}}) and a coloured diagrams γ ∈ Γ(k_{1}, k_{2}) consisting of s coloured chains β(l_{1}), …, β(l_{s}) we define the function

\overline{(f_{1} ∘ f_{2})}_{γ}(x_{l_{1}}, …, x_{l_{s}}) = f_{1}(x_{α_{γ}(1,1)}, …, x_{α_{γ}(1,k_{1})})f_{2}(x_{α_{γ}(2,1)}, …, x_{α_{γ}(2,k_{2})}), (11.3)

where α_{γ}(p, r) denotes the index of the vertex (p, r) of the diagram γ in their above defined enumeration α_{γ}. (In formula (11.3) all arguments of the functions f_{1} and f_{2} have different indices. But the indices α_{γ}(1, j) and α_{γ}(2, j′) may agree for some pairs (j, j′). This happens if the vertices (1, j) and (2, j′) belong to the same chain β ∈ γ of length 2.) Let us also define the function

(f_{1} ∘ f_{2})_{γ}(x_{l_{p}}, l_{p} ∈ O(γ)) = ( ∏_{p∈C(γ)} P_{p} ∏_{p∈O_{2}(γ)} Q_{p}) \overline{(f_{1} ∘ f_{2})}_{γ}(x_{l_{1}}, …, x_{l_{s}}), (11.4)

with the operators P_{p} and Q_{p} defined (with a different indexation) in formulas (11.1) and (11.2), where C(γ) is the set of indices of the closed diagrams of γ, and O_{2}(γ) ⊂ O(γ), defined as O_{2}(γ) = {l: c_{γ}(β_{l}) = −1, and ℓ(β(l)) = 2}, is the set of indices of the chains of γ with colour −1 and length 2. are the above defined sets of open and closed indices of the diagram γ. The arguments of the function (f_{1} ∘ f_{2})_{γ} are the indices of the open vertices of the diagram γ. Let us also remark that the operators P_{p} and Q_{p} in formula (11.4) are exchangeable, hence it is not important in what order we apply them.

The function F_{γ}(f_{1}, f_{2}) we apply in the formulation of the diagram formula in the special case when the product of two degenerate U-statistics is considered is similar to the function (f_{1} ∘ f_{2})_{γ} introduced in (11.4). We need a small technical step for its definition. We want to work with such a function whose variables are indexed with the numbers 1, 2, …, |O(γ)| while the indices of the function f_{1} ∘ f_{2})_{γ} are the elements of the set O(γ) = {l̄_{1}, …, l̄_{|O(γ)|}}. Hence we define the function t = t_{γ} on the set O(γ) by the formula t(l̄_{j}) = j, 1 ≤ j ≤ |O(γ)|, and introduce the function

F_{γ}(f_{1}, f_{2})(x_{1}, x_{2}, …, x_{|O(γ)|}) = (f_{1} ∘ f_{2})_{γ}(x_{t(l_{p})}, l_{p} ∈ O(γ)).  (11.5)

Let me remark that for different enumerations β(l_{1}), …, β(l_{s}) of the chains of a coloured diagram γ the function F_{γ}(f_{1}, f_{2}) we defined by formulas (11.1)–(11.5) may be slightly different. One of them can be obtained by reindexing the variables x_{1}, …, x_{|O(γ)|} in these functions. But the value of the U-statistic I_{n,|O(n)|}(F_{γ}(f_{1}, f_{2})) does not depend on the indexation of the variables in its kernel function, hence on the enumeration of the chains of γ. For a similar reason the value of I_{n,|O(n)|}(F_{γ}(f_{1}, f_{2})) depends only on the cardinality of |O(γ)|, |O_{2}(γ)| and |C(γ)| of the coloured diagram γ, and also a reindexation of the arguments of f_{1} or f_{2} does not change the value of the U-statistic I_{n,|O(n)|}(F_{γ}(f_{1}, f_{2})).

Next I formulate the diagram formula for the product of two degenerate U-statistics with the help of the above defined quantities.

Theorem 11.1. (The diagram formula for the product of two degenerate Ustatistics). Let a sequence of independent and identically distributed random variables ξ_{1}, ξ_{2}, … be given with some distribution μ on a measurable space (X, X) together with two bounded canonical functions f_{1}(x_{1}, …, x_{k_{1}}) and f_{2}(x_{1}, …, x_{k_{2}}) with respect to the probability measure μ on the product spaces (X^{k_{1}}, X^{k_{1}}) and (X^{k_{2}}, X^{k_{2}}) respectively.

Let us take the class of coloured diagrams Γ(k_{1}, k_{2}) introduced above together with the functions F_{γ}(f_{1}, f_{2}) defined in formulas (11.1)—(11.5).

For all γ ∈ Γ F_{γ}(f_{1}, f_{2})_{γ} is a canonical function with respect to the measure μ with |O(γ)| arguments, where O(γ) and C(γ) denote the set of open and closed indices of the diagram γ. The product of the degenerate U-statistics I_{n,k_{1}}(f_{1}) and I_{n,k_{2}}(f_{2}), n ≥ max(k_{1}, k_{2}), defined in (8.7) can be expressed as

\begin{matrix}(n^{−k_{1}/2}k_{1}!I_{n,k_{1}}(f_{1}))(n^{−k_{2}/2}k_{2}!I_{n,k_{2}}(f_{2})) \\ = \begin{matrix}∑ & ′(n) \\ γ∈Γ(k_{1},k_{2}) \end{matrix} ∏_{j=1}^{|C(γ)|} (\frac{n − s(γ) + j}{n} ) n^{−W (γ)/2} · n^{−|O(γ)|/2}|O(γ)|!I_{n,|O(γ)|}(F_{γ}(f_{1}, f_{2})) \end{matrix}

(11.6)

with W (γ) = k_{1} + k_{2} − |O(γ)| − 2|C(γ)| and s(γ) = |O(γ)| + |C(γ)| (which equals the number of coloured diagrams in γ), where ∑^{′(n)} means that summation is taken only for such coloured diagrams γ ∈ Γ(k_{1}, k_{2}) which satisfy the inequality s(γ) ≤ n, and

∏_{j=1}^{|C(γ)|} equals 1 in the case |C(γ)| = 0. The term I_{n,|O(γ)|}(F_{γ}(f_{1}, f_{2})) can be replaced by

I_{n,|O(γ)|}(SymF_{γ}(f_{1}, f_{2})) in formula (11.6).

Consider the L_{2}-norm of the functions F_{γ}(f_{1}, f_{2}) defined by the formula

∥F_{γ}(f_{1}, f_{2})∥_{2}^{2} = ∥(f_{1} ∘ f_{2})_{γ}∥_{2}^{2} = ∫ (f_{1} ∘ f_{2})_{γ}^{2}(x_{l_{p}}, l_{p} ∈ O(γ)) ∏_{l_{p}∈O(γ)} μ( dx_{l_{p}}).

The inequality

∥F_{γ}(f_{1}, f_{2})∥_{2} = ∥(f_{1} ∘ f_{2})_{γ}∥_{2} ≤ ∥f_{1}∥_{2}∥f_{2}∥_{2} if W (γ) = 0  (11.7)

holds for this norm. The condition W (γ) = 0 in formula (11.7) means that the diagram γ ∈ Γ(k_{1}, k_{2}) has no chains β of length ℓ(β) = 2 with colour c_{γ}(β) = −1. In the case of a general diagram γ ∈ Γ(k_{1}, k_{2}) the inequality

∥F_{γ}(f_{1}, f_{2})∥_{2} = ∥(f_{1} ∘ f_{2})_{γ}∥_{2} ≤ 2^{W (γ)} min(∥f_{1}∥_{2}, ∥f_{2}∥_{2})  (11.8)

holds if the L_{∞}-norm of the functions f_{1} and f_{2} satisfies the inequalities ∥f_{1}∥_{∞} ≤ 1 and ∥f_{2}∥_{∞} ≤ 1. Relations (11.7) and (11.8) also hold for non-canonical functions f_{1} and f_{2}.

Inequality (11.7) is actually a repetition of estimate (10.11) about the diagrams appearing in the case of Wiener–Itô integrals. Inequality (11.8) yields a weaker bound about the L_{2}-norm ∥F_{γ}(f_{1}, f_{2})∥_{2} = ∥(f_{1} ∘ f_{2})_{γ}∥_{2} for a general diagram γ. In particular, it depends not only on the L_{2}-norm, but also on the L_{∞}-norm of the functions f_{1} and f_{2}. This is closely related to the fact that in the estimates on the distribution of U-statistics, — unlike the case of Wiener–Itô integrals, — a condition is imposed not only on the L_{2}-norm of the kernel function f, but also on its L_{∞}-norm. I return to this question later.

Remark 1. The expression W (γ) = k_{1} + k_{2} − |O(γ)| − 2|C(γ)| appearing in formulas (11.6), (11.7) and (11.8) has the following content. It equals the number of those diagrams β(l_{j}) ∈ γ for which ℓ(β(l_{j})) = 2, and c_{γ}(β(l_{j})) = −1. Indeed, if W (γ) denotes the number of such chains, and W̄ (γ) equals the number of chains β(l_{j}) ∈ γ for which ℓ(β(l_{j})) = 1 (and as a consequence c_{γ}(β(l_{j})) = −1), then W (γ) + W̄ (γ) = |O(γ)|, and 2W (γ)+W̄ (γ)+2|C(γ)| = k_{1} +k_{2}. These identities imply the statement of this remark.

Remark 2. The term I_{n,|O(γ)|}(F_{γ}(f_{1}, f_{2})) appeared in the sum at the right-hand side of (11.6) only if the condition s(γ) ≤ n was satisfied. This restriction in the summation had a technical character, which has no great importance in our investigations. It is related to the fact that a U-statistic I_{n,k}(f) exists only if n ≥ k. As a consequence, some U-statistics disappear at the right-hand side of (11.6) if the sample size n of the Ustatistics is relatively small. The term I_{n,|O(γ)|}(F_{γ}(f_{1}, f_{2})) appeared in (11.6) through the Hoeffding decomposition of a U-statistic with kernel function \overline{(f_{1} ∘ f_{2})}_{γ} defined in (11.3). This function has s(γ) arguments, and the U-statistic corresponding to it appears in our calculations only if the sample size n is not smaller than this number.

Let us recall the convention introduced after the definition of canonical degenerate U-statistics by which I_{n,0}(c) is a degenerate U-statistic of order zero, and I_{n,0}(c) = c for a constant c. By applying this convention we write F_{γ}((f_{1}, f_{2}) = f_{1}∘f_{2} in relation (11.6) for those diagrams γ for which |O(γ)| = 0, i.e. c_{γ}(β) = 1 for all chains β ∈ γ. We shall introduce another convention which implies that Theorem 11.1 is valid also in the degenerate case when the function f_{k_{1}} = c with a constant c, and k_{1} = 0. In this case Γ(k_{1}, k_{2}) consists of only one diagram γ containing the chains β_{j} = {j} of length one and colour c_{γ}({j}) = −1, 1 ≤ j ≤ k_{2}. We define I(F_{γ}(f_{1}, f_{2})) = cf_{2} in this case. Beside this, we have W (γ) = k_{1} + k_{2} − |O(γ)| − 2|C(γ)| = 0, |O(γ)| = k_{2}, and |C(γ)| = 0. Hence formula (11.6) remains valid also in the case k_{1} = 0. We have introduced this convention because the following inductive argument leading to the proof of the diagram formula for the product of degenerate U-statistics in the general case is valid under such a convention.

Let us turn to the formulation of the general form of the diagram formula for the product of degenerate U-statistics. First I define a function F_{γ} = F_{γ}(f_{1}, …, f_{m}) for each coloured diagram γ ∈ Γ(k_{1}, …, k_{m}) and collection of canonical functions (with respect to a probability measure μ on a measurable space (X, X)) f_{1}, …, f_{m} with k_{1}, …, and k_{m} variables. These functions F_{γ} will be the kernel functions of the degenerate Ustatistics at the right-hand side of the diagram formula.

These functions F_{γ} will be defined by induction with respect to the number m of the components in the product. For m = 2 we have already defined the function F_{γ}(f_{1}, f_{2}). Let the functions F_{γ}(f_{1}, …, f_{m−1}) be defined for each coloured diagram γ ∈ Γ(k_{1}, …, k_{m−1}). To define F_{γ}(f_{1}, …, f_{m}) for a coloured diagram γ ∈ Γ(k_{1}, …, k_{m}) first we define the predecessor γ_{pr} = γ_{pr}(γ) ∈ Γ(k_{1}, …, k_{m−1}) of γ. We shall define the coloured diagram γ_{pr} together with an appropriate indexation of its element with the help of the enumeration of the elements of γ. Roughly speaking, the elements of γ_{pr} are the restrictions of the chains contained in γ to the first m − 1 rows of the diagram, i.e. to the set {(p, r): 1 ≤ p ≤ m − 1, 1 ≤ r ≤ k_{p}}. But we must define also the colour of these resticted chains.

To define precisely the predecessor γ_{pr} of γ let us divide first the chains of the coloured diagram γ = {β(l_{1}), …, β(l_{s})} ∈ Γ(k_{1}, …, k_{m}) into two disjoint subsets γ = γ_{1} ∪ γ_{2}, defined as γ_{1} = {β(l_{j}): β(l_{j}) ∈ γ, β(l_{j}) ∩ {(m, 1), …, (m, k_{m})} ≠ ∅} and γ_{2} = {β(l_{j}): β(l_{j}) ∈ γ, β(l_{j}) ∩ {(m, 1), …, (m, k_{m})} = ∅}, i.e. a coloured chain β ∈ γ belongs to γ_{1} if it contains a vertex from the last row {(m, 1), …, (m, k_{m})} of the diagram, and it belongs to γ_{2} if it does not contain such a vertex. We define with the help of the chains β(l_{j}) ∈ γ_{1} the chains β_{pr}(l_{j}) = β(l_{j}) \ {(m, 1), …, (m, k_{m})} and with the help of the chains β(l_{j}) ∈ γ_{2} the chains β_{pr}(l_{j}) = β(l_{j}). (For those chains β(l_{j}) ∈ γ_{1} which consist only of one vertex of the form (m, r), 1 ≤ r ≤ k_{m}, the corresponding chain β_{pr}(l_{j}) would be the empty set. These empty sets are omitted from the set of chains β_{pr}(l_{j}) ∈ γ_{pr}.) The set of all above defined chains β_{pr}(l_{j}) provides a partition of the set of vertices {(p, r): 1 ≤ p ≤ m − 1, 1 ≤ r ≤ k_{p}}. The diagram γ_{pr} will consist of these chains β_{pr}(l_{j}). To complete the definition of the coloured diagram γ_{pr} we still have to define the colour c_{γ_{pr}}(β_{pr}(l_{j})) of these chains.

We define the colour of these chains by the formulas c_{γ_{pr}}(β_{pr}(l_{j})) = −1 if β(l_{j}) ∈ γ_{1}, and c_{γ_{pr}}(β_{pr}(l_{j})) = c_{γ}(β(l_{j})) if β(l_{j}) ∈ γ_{2}. In such a way we defined the predecessor γ_{pr} ∈ Γ(k_{1}, …, k_{m−1}) of the diagram γ ∈ Γ(k_{1}, …, k_{m}). Moreover we gave an indexation of the chains of γ_{pr} with the help of the indexation of the chains of γ.

With the help of the coloured diagram γ_{pr} ∈ Γ(k_{1}, …, k_{m−1}) we can define the function F_{γ_{pr}} = F_{γ_{pr}}(f_{1}, …, f_{m−1}) which is a function of |O(γ_{pr})| variables x_{1}, …, x_{|O(γ_{pr})|}. We shall define the function F_{γ} = F_{γ}(f_{1}, …, f_{m}) similarly to the definition of F_{γ}(f_{1}, f_{2}) given by formulas (11.3), (11.4) and (11.5) in the case m = 2. In this case F_{γ_{pr}} plays the role of the function f_{1} and f_{m} the role of the function f_{2}. To define the function F_{γ}(f_{1}, …, f_{m}) we still have to define a coloured diagram γ_{cl} = γ_{cl}(γ) ∈ Γ(|O(γ_{pr})|, k_{m}) that we shall call the closing diagram of γ. The heuristic content of the diagram γ_{cl} is that it contains the additional information we need to reconstruct the diagram γ ∈ Γ(k_{1}, …, k_{m}) if we know its predecessor γ_{pr}. We shall define it together with an enumeration of its chains that depends on the enumeration of the chains of the diagram γ.

To define the diagram γ_{cl} let us first consider the listing O(γ_{pr}) = {l̄_{1}, …, l̄_{|O(γ_{pr})|}}, 1 ≤ l̄_{1} < l̄_{2} < ⋯ < l_{|O(γ_{pr}|}, of the indices of the open indices of the diagram γ_{pr} in increasing order. Let us fix a vertex (1, j), 1 ≤ j ≤ |O(γ_{pr})| in the first row of γ_{cl}. We shall denote the chain of γ_{cl} containing this vertex by β_{cl}(l̄_{j}), i.e. this chain get the index l̄_{j}, and define it together with its colour in the following way. Let us consider the (open) chain β_{pr}(l̄_{j}) together with its ‘continuation’ β(l̄_{j}). Clearly, β_{pr}(l̄_{j}) ⊂ β(l̄_{j}). If β(l̄_{j}) ∈ γ_{1}, then β(l̄_{j}) = β_{pr}(l̄_{j}) ∪ {(m, r_{j})} with some integer 1 ≤ r_{j} ≤ k_{m}. In this case we define the chain containing the vertex (1, j) as the diagram β_{cl}(l̄_{j}) = {(1, j), (2, r_{j})} with this number r_{j}, and it gets the colour c_{γ_{cl}}(β_{cl}(l̄_{j})) = c_{γ}(β(l̄_{j})). If β(l̄_{j}) ∈ γ_{2}, then β_{pr}(l̄_{j}) = β(l̄_{j}), and we define the chain containing the vertex (1, j) as the chain β_{cl}(l̄_{j}) = {(1, j)} of length 1 and with colour c_{γ_{cl}}(β_{cl}(l̄_{j})) = −1.

We still have to consider those vertices (2, r) of Γ(|O(γ_{pr})|, k_{m}), 1 ≤ r ≤ k_{m}, for which there exists a chain β(l_{j(r)}) ∈ γ such that β(l_{j(r)}) = {m, r)}, because these are the vertices of the set of vertices {(1, j): 1 ≤ j ≤ |O(γ_{pr})| ∪ {(2, r): 1 ≤ r ≤ k_{m}} which are not contained in the previously defined chains β_{cl}(l̄_{j}). To cover these vertices with an (appropriately indexed) chain of γ_{cl} let us define the chains β_{cl}(l_{j(r)}) = {(2, r)} with the colour c_{γ_{cl}}(β_{cl}(l_{j(r)})) = −1 for such vertices (2, r). The above defined coloured chains provide a partition of the set {(1, j): 1 ≤ j ≤ |O(γ_{pr})| ∪ {(2, r): 1 ≤ r ≤ k_{m}}, and they are the elements of the coloured diagram γ_{cl}.

We shall define the function F_{γ}(f_{1}, …, f_{m}) with the help of the above introduced diagrams γ_{pr} and γ_{cl} in the following way. Put, similarly to formula (11.3),

\begin{matrix}\overline{(F_{γ_{pr}}(f_{1}, …, f_{m−1}) ∘ f_{m})}_{γ}(x_{l_{1}}, …, x_{l_{s}}) \\ = F_{γ_{cl}}(x_{α_{γ_{cl}} (1,1)}, …, x_{α_{γ_{cl}} (1,|O(γ_{pr})|)})f_{m}(x_{α_{γ_{cl}} (2,1)}, …, x_{α_{γpr} (2,k_{m})}), \end{matrix} (11.9)

where s = s(γ_{cl}) is the number of the chains contained in γ_{cl}. The indices l_{1}, l_{2} …, and l_{s} of the variables at the left-hand side of (11.9) agree with the indices of the chains of the diagram γ_{cl}, and α_{γ_{cl}}(p, r) denotes the index of the vertex (p, r) of the diagram γ_{cl} which is induced by the enumeration of the indices of the chains in γ_{cl}. Next we define with the help of formula (11.9), similarly to the relation (11.4), the function

\begin{matrix}(F_{γ_{pr}}(f_{1}, …, f_{m−1}) ∘ f_{m})_{γ}(x_{p}, p ∈ O(γ_{cl})) \\ = ( ∏_{p∈C(γ_{cl})} P_{p} ∏_{p∈O_{2}(γ_{cl})} Q_{p}) \overline{(F_{γ_{pr}}(f_{1}, …, f_{m−1} ∘ f_{m})}_{γ}(x_{p}, p ∈ O(γ_{cl}) ∪ C(γ_{cl})) \end{matrix}

(11.10)

with the operators P_{p} and Q_{p} defined (with a different indexation) in formulas (11.1) and (11.2), where the sets O(γ_{cl}) and C(γ_{cl}) are the sets of open and closed indices of the diagram γ_{cl}, and the set O_{2}(γ_{cl}) (for a general diagram with two rows) was defined after formula (11.4). The function (F_{γ_{pr}}(f_{1}, …, f_{m−1})∘f_{m})_{γ} depends only on the arguments indexed by the open indices of the diagram γ_{cl}.

The function F_{γ}(f_{1}, …, f_{m}) will be defined by means of a reindexation of the arguments of the function (F_{γ_{pr}}(f_{1}, …, f_{m−1}) ∘ f_{m})_{γ}(x_{l_{p}}, l_{p} ∈ O(γ_{cl})) which will be made to get a function with arguments x_{1}, x_{2}, …, x_{|O(γ_{cl})|}. It is defined, similarly to formula (11.5), as

F_{γ}(f_{1}, …, f_{m})(x_{1}, x_{2}, …, x_{|O(γ_{cl})|}) = (F_{γ_{pr}}(f_{1}, …, f_{m−1}) ∘ f_{m})_{γ}(x_{t(l_{p})}, l_{p} ∈ O(γ_{cl})),

(11.11)

where the indices t(l_{p}) are defined in the following way. We list the open indices of the diagram γ_{cl} in an increasing order as O(γ_{cl}) = {l̄_{1}, …, l̄_{|O(γ_{cl})|}}, l̄_{1} < l̄_{2} < ⋯ < l̄_{|O(γ_{cl})|}, and define the function t(·) on the set O(γ_{cl}) as t(l̄_{p}) = p for 1 ≤ p ≤ |O(γ_{cl})|.

To complete the definition of the function F_{γ}(f_{1}, …, f_{m}) observe that |O(γ_{cl})| = |O(γ)|. (Even the sets O(γ_{cl}) and O(γ) agree with the enumeration of the chains of these two diagrams we have chosen.) Hence we can write

F_{γ}(f_{1}, …, f_{m})(x_{1}, x_{2}, …, x_{|O(γ_{cl})|}) = F_{γ}(f_{1}, …, f_{m})(x_{1}, x_{2}, …, x_{|O(γ)|}). (11.12)

Let me remark, that, just as in the case m = 2, also in the case m ≥ 2 the value of the U-statistic I_{n,|O(γ)|}(F_{γ}(f_{1}, …, f_{m})) does not depend on the enumeration of the chains of the coloured diagram γ.

To formulate the general form of the diagram formula for the product of degenerate U-statistics we introduce some quantities which will be the version of the quantities appearing in the coefficients of the right-hand side of (11.6) in Theorem 11.1. Put

W (γ) = ∑_{l_{p}∈O(γ)}(ℓ(β(l_{p})) − 1) + ∑_{l_{p}∈C(γ)}(ℓ(β(l_{p})) − 2), γ ∈ Γ(k_{1}, …, k_{m}), (11.13)

where ℓ(β) denotes the length of the chain β.

To define the next quantity we need let us first introduce the following notation. Given a chain β = {(p_{1}, r_{1}), …, (p_{l}, r_{l})}, 1 ≤ p_{1} < p_{2} < ⋯ < p_{l} ≤ m, in the set {(p, r): 1 ≤ p ≤ m, 1 ≤ r ≤ k_{p}} let us define its upper level u(β) = p_{1}, and its deepest level d(β) = l_{p}. Let us define with their help for all diagrams γ ∈ Γ(k_{1}, …, k_{m}) and integers p, 1 ≤ p ≤ m, the sets B_{1}(γ, p) = {β: β ∈ γ, c_{γ}(β) = 1, d(β) = p}, and B_{2}(γ, p) = {β: β ∈ γ, c_{γ}(β) = −1, d(β) ≤ p} ∪ {β: β ∈ γ, u(β) ≤ p, d(β) > p}, i.e. B_{1}(γ, p) consists of those chains β ∈ Γ which have colour 1, all their vertices are in the first p rows of the diagram, and contain a vertex in the p-th row, while B_{2}(γ, p) consists of those chains β ∈ γ which have either colour −1, and all their vertices are in the first p rows of the diagram, or they have (with an arbitrary colour) a vertex both in the first p rows both in the remaining rows of the diagram. Put B_{1}(γ, p) = |B_{1}(γ, p)| and B_{2}(γ, p) = |B_{2}(γ, p)|. With the help of these numbers we define

J_{n}(γ, p) = \{ \begin{matrix}∏_{j=1}^{B_{1}(γ,p)} & (\frac{n − B_{1}(γ, p) − B_{2}(γ, p) + j}{n} ) if B_{1}(γ, p) ≥ 1 \\ 1 if B_{1}(γ, p) = 0 \end{matrix}  (11.14)

for all 2 ≤ p ≤ m and diagrams γ ∈ Γ(k_{1}, …, k_{m}).

Theorem 11.2 will be formulated with the help of the above notations.

Theorem 11.2. (The diagram formula for the product of several degenerate U-statistics). Let a sequence of independent and identically distributed random variables ξ_{1}, ξ_{2}, … be given with some distribution μ on a measurable space (X, X) together with m ≥ 2 bounded functions f_{p}(x_{1}, …, x_{k_{p}}) on the spaces (X^{k_{p}}, X^{k_{p}}), 1 ≤ p ≤ m, canonical with respect to the probability measure μ. Let us consider the class of coloured diagrams Γ(k_{1}, …, k_{m}) together with the functions F_{γ} = F_{γ}(f_{1}, …, f_{m}), γ ∈ Γ(k_{1}, …, k_{m}), defined in formulas (11.9)—(11.12) and the constants W (γ) and J_{n}(γ, p), 1 ≤ p ≤ m, given in formulas (11.13) and (11.14).

The functions F_{γ}(f_{1}, …, f_{m}) are canonical with respect to the measure μ with |O(γ)| variables, and the product of the degenerate U-statistics I_{n,k_{p}}(f_{p}), 1 ≤ p ≤ m, n ≥ max_{1≤p≤m} k_{p}, defined in (8.7) can be expressed as

\begin{aligned}∏_{p=1}^{m} n^{−k_{p}/2}k_{p}!I_{n,k_{p}}(f_{k_{p}}) & = \begin{matrix}∑ & ′(n, m ) \\ γ∈Γ(k_{1},...,k_{m}) \end{matrix} (∏_{p=2}^{m} J_{n}(γ, p)) n^{−W (γ)/2} \\ & n^{−|O(γ)|/2}|O(γ)|!I_{n,|O(γ)|}(F_{γ}(f_{1}, …, f_{m})), \end{aligned} (11.15)

where ∑^{′(n, m)} means that summation is taken for those γ ∈ Γ(k_{1}, …, k_{m}) which satisfy the relation B_{1}(γ, p) + B_{2}(γ, p) ≤ n for all 2 ≤ p ≤ m with the quantities B_{1}(γ, p) and B_{2}(γ, p) introduced before the definition of J_{n}(γ, p) in (11.14), and the expression W (γ) was defined in (11.13). The terms I_{n,|O(γ)|}(F_{γ}(f_{1}, …, f_{m})) at the right-hand side of formula (11.15) can be replaced by I_{n,|O(γ)|}(Sym F_{γ}(f_{1}, …, f_{m})).

In Theorem 11.2 the product of such degenerate U-statistics were considered, whose kernel functions were bounded. This also implies that all functions F_{γ} appearing at the right-hand side of (11.15) are well-defined (i.e. the integrals appearing in their definition are convergent) and bounded. In the applications of Theorem 11.2 it is useful to have more information about the behaviour of the functions F_{γ}. We shall need some good bound on their L_{2}-norm. Such a result is formulated in the following

Lemma 11.3. (Estimate about the L_{2}-norm of the kernel functions of the U-statistics appearing in the diagram formula). Let m functions f_{p}(x_{1}, …, x_{k_{p}}) be given on the products (X^{k_{p}}, X^{k_{p}}) of some measurable space (X, X), 1 ≤ p ≤ m, with a probability measure μ on it, which satisfy inequalities (8.1) and (8.2) (if the index k is replaced by the index k_{p} in them), but these functions need not be canonical. Let us take a coloured diagram γ ∈ Γ(k_{1}, …, k_{m}), and consider the function F_{γ}(f_{1}, …, f_{m}) defined by formulas (11.9)—(11.12). The L_{2}-norm of the function F_{γ}(f_{1}, …, f_{m}) (with respect to the power of the measure μ to the space where F_{γ}(f_{1}, …, f_{m}) is defined) satisfies the inequality

∥F_{γ}(f_{1}, …, f_{m})∥_{2} ≤ 2^{W (γ)} ∏_{p∈U(γ)} ∥f_{p}∥_{2},

where W (γ) is given in (11.13), and the set U(γ) ⊂ {1, …, m} is defined in the following way. Let us define for a coloured chain β = {(l_{1}, r_{1}), (l_{2}, r_{2}), …, (l_{s}, r_{s})} ∈ γ with 1 ≤ l_{1} < ⋯ < l_{s} ≤ m the set of its interior levels as and Int (β) = {l_{2}, …, l_{s−1}, l_{s}} if c_{γ}(β) = −1 and Int (β) = {l_{2}, …, l_{s−1}} if c_{γ}(β) = 1. Then we define U(γ) =

{1, …, m} \ ( ⋃_{β∈γ} Int (β)).

The last result of this section is a corollary of Theorem 11.2. In this corollary we give an estimate on the expected value of product of degenerate U-statistics. To formulate this result we introduce the following terminology. Let us call a (coloured) diagram γ ∈ Γ(k_{1}, …, k_{m}) closed if c_{γ}(β) = 1 for all chains β ∈ γ. Let us denote the set of all closed diagrams by Γ̄(k_{1}, …, k_{m}). Observe that F_{γ}(f_{1}, …, f_{m}) is constant (a function of zero variable) for all closed diagram γ ∈ Γ̄(k_{1}, …, k_{m}), and I_{n,|O(γ)|}(F_{γ}(f_{1}, …, f_{m})) = I_{n,0}(F_{γ}(f_{1}, …, f_{m})) = F_{γ}(f_{1}, …, f_{m}) in this case. Now we formulate the following result.

Corollary of Theorem 11.2 about the expectation of a product of degenerate U-statistics. Let a finite sequence of functions f_{p}(x_{1}, …, x_{k_{p}}), 1 ≤ p ≤ m, be given on the products (X^{k_{p}}, X^{k_{p}}) of some measurable space (X, X) together with a sequence of independent and identically distributed random variables with value in the space (X, X) which satisfy the conditions of Theorem 11.2.

Let us apply the notation of Theorem 11.2 together with the notion of the above introduced class of closed diagrams Γ̄(k_{1}, …, k_{m}). The identity

E (∏_{p=1}^{m} k_{p}!n^{−k_{p}/2}I_{n,k_{p}}(f_{k_{p}})) = \begin{matrix}∑ & ′(n, m ) \\ γ∈Γ̄(k_{1},...,k_{m}) \end{matrix} (∏_{p=1}^{m} J_{n}(γ, p)) n^{−W (γ)/2} · F_{γ}(f_{1}, …, f_{m})

(11.16)

holds. This identity has the consequence

|E (∏_{p=1}^{m} k_{p}!n^{−k_{p}/2}I_{n,k_{p}}(f_{k_{p}}))| ≤ ∑_{γ∈Γ̄(k_{1},...,k_{m})} n^{−W (γ)/2}|F_{γ}(f_{1}, …, f_{m})|. (11.17)

Beside this, if ∥f_{p}∥_{2} ≤ σ for all 1 ≤ p ≤ m, then the numbers F_{γ}(f_{1}, …, f_{m}) at the right-hand side of (11.17) satisfy the inequality

|F_{γ}(f_{1}, …, f_{m})| ≤ 2^{W (γ)}σ^{|U(γ)|} for all γ ∈ Γ̄(k_{1}, …, k_{m}).  (11.18)

In formula (11.18) the same number W (γ) and set U(γ) appear as in Lemma 11.3. The only difference is that in the present case c_{γ}(β) = 1 for all chains β ∈ γ which appear in the definition of U(γ).

Remark: We have applied a different terminology for diagrams in this section and in Section 10, where the theory of Wiener–Itô integrals was discussed. But there is a simple relation between the terminology of these sections. If we take only those diagrams from the diagrams considered in this section which contain only chains of length 1 or 2, and beside this the chains of length 1 have colour −1, and the chains of length 2 have colour 1, then we get the diagrams considered in the previous section. Moreover, the functions F_{γ} = F_{γ}(f_{1}, …, f_{m}) are the same in the two cases. Hence formula (10.18) in the Corollary of Theorem 10.2 and formula (11.17) in the Corollary of Theorem 11.2 make possible to compare the moments of Wiener–Itô integrals and degenerate U-statistics.

The main difference between these estimates is that formula (11.17) contains some additional terms. They are the contributions of those diagrams γ ∈ Γ̄(k_{1}, …, k_{m}) which contain chains β ∈ γ with length ℓ(β) > 2. These are those diagrams γ ∈ Γ̄(k_{1}, …, k_{m}) for which W (γ) > 1. The estimate (11.18) given for the terms F_{γ} corresponding to such diagrams is weaker, than the estimate given for the terms F_{γ} with W (γ) = 0, since |U(γ)| < m if W (γ) ≥ 1, while |U(γ)| = m, if W (γ) = 0. On the other hand, such terms have a coefficient n^{−W (γ)/2} at the right-hand side of formula (11.17). A closer study of these formulas may explain the relation between the estimates given for the tail distribution of Wiener–Itô integrals and degenerate U-statistics.

12. The proof of the diagram formula for U-statistics.

In this section the results of the previous section will be proved. First I prove its main result, the diagram formula for the product of two degenerate U-statistics.

Proof of Theorem 11.1. In the first step of the proof the product k_{1}!I_{n,k_{1}}(f_{1})k_{2}!I_{n,k_{2}}(f_{2}) of two degenerate U-statistics will be rewritten as a sum of not necessarily degenerate U-statistics. In this step a term by term multiplication is carried out for the product k_{1}!I_{n,k_{1}}(f_{1})k_{2}!I_{n,k_{2}}(f_{2}), and the terms of the sum obtained in such a way are put in different classes indexed by the (non-coloured) diagrams with two rows of length k_{1} and k_{2}. This step is very similar to the heuristic argument leading to formulas (10.13) and (10.13a) in our explanation about the diagram formula for Wiener-Itô integrals.

To carry out this step of the proof consider all sets of pairs

{(u_{1}, u′_{1}), …, (u_{r}, u′_{r})}, 1 ≤ r ≤ min(k_{1}, k_{2}),

with the following properties: 1 ≤ u_{1} < u_{2} < ⋯ < u_{r} ≤ k_{1}, the numbers u′_{1}, …, u′_{r} are different, and 1 ≤ u′_{s} ≤ k_{2}, for all 1 ≤ s ≤ r. To a set of pairs {(u_{1}, u′_{1}), …, (u_{r}, u′_{r})} with these properties let us correspond the following diagram γ̄((u_{1}, u′_{1}), …, (u_{r}, u′_{r})) ∈ Γ̄(k_{1}, k_{2}), where Γ̄(k_{1}, k_{2}) denotes the set of (non-coloured) diagrams with two rows of length k_{1} and k_{2}. The diagram γ̄((u_{1}, u′_{1}), …, (u_{r}, u′_{r})) has two rows, {1, …, k_{1}}, and {2, …, k_{2}}, its chains of length 2 are the sets {(1, u_{s}), (2, u′_{s})}, 1 ≤ s ≤ r, and beside this it contains the chains {(1, r)}, r ∈ {1, …, k_{1}} \ {u_{1}, …, u_{r}}, and {(2, r)}, r ∈ {1, …, k_{2}} \ {u′_{1}, …, u′_{r}} of length 1. All (non-coloured) diagrams γ̄ ∈ Γ̄(k_{1}, k_{2}) can be represented in the form γ̄ = γ̄((u_{1}, u′_{1}), …, (u_{r}, u′_{r})) with the help of a set of pairs {(u_{1}, u′_{1}), …, (u_{r}, u′_{r})}, 1 ≤ r ≤ min(k_{1}, k_{2}), with the above properties in a unique way.

To make the notation in the subsequent discussion simpler we fix, similarly to the case of coloured diagrams, an indexation of the chains of a diagram γ̄ ∈ Γ̄(k_{1}, k_{2}), and we define with its help an indexation of the vertices of this diagram γ̄, too. Let us take the following natural indexation. Consider the diagram γ̄ = γ̄((u_{1}, u′_{1}), …, (u_{r}, u′_{r})) ∈ Γ̄(k_{1}, k_{2}) which has s(γ̄) = k_{1} + k_{2} − r chains. The chain β ∈ γ̄ containing the vertex (1, j) gets the index j, i.e. (1, j) ∈ β(j) for 1 ≤ j ≤ k_{1}. To define the index of the remaining chains of γ̄ which are chains of length 1 of the form (2, j) with j ∈ {1, …, k_{2}} \ {u′_{1}, …, u′_{r}} let us take the list {l̄_{1}, …, l̄_{k_{2}−r}}, 1 ≤ l̄_{1} < ⋯ < l̄_{k_{2}−r}, of the elements of the set {1, …, k_{2}} \ {u′_{1}, …, u′_{r}} in an increasing order. Then we define the indices of the remaining chains by the formula β(k_{2} +j) = {(2, l̄_{j})}), 1 ≤ j ≤ k_{2} −r. After this we define the indexation of the vertices of the diagram γ by the formula αγ̄(p, r) = l with that index l for which (p, r) ∈ β(l). Let us also define the sets V_{1} = V_{1}(γ̄) = {1, …, k_{1} + k_{2} − r} \ {u_{1}, …, u_{r}} and V_{2} = V_{2}(γ̄) = {u_{1}, …, u_{r}}, i.e. V_{1} is the set of indices of the chains of γ̄ of length 1, and V_{2} is the set of indices of the chains of γ̄ of length 2.

Let us consider the product k_{1}!I_{n,k_{1}}(f_{1})k_{2}!I_{n,k_{2}}(f_{2}), and rewrite it in the form of the sum we get by carrying out a term by term multiplication in this expression. We put the terms obtained in such a way into disjoint classes indexed by the diagrams γ̄ ∈ Γ̄(k_{1}, k_{2}) in the following way: A product f_{1}(ξ_{j_{1}}, …, ξ_{j_{k_{1}}} )f_{2}(ξ_{j′_{1}}, …, ξ_{j′_{k_{2}}} ) belongs to the class indexed by the diagram γ̄((u_{1}, u′_{1}), …, (u_{r}, u′_{r})) with the parameters (u_{1}, u′_{1}), …, (u_{r}, u′_{r}), 1 ≤ r ≤ min(k_{1}, k_{2}), where 1 ≤ u_{1} < u_{2} < ⋯ < u_{r} ≤ k_{1}, the numbers u′_{1}, …, u′_{r} are different, and 1 ≤ u′_{s} ≤ k_{2}, for all 1 ≤ s ≤ r if the indices j_{1}, …, j_{k_{1}}, j′_{1}, …, j′_{k_{2}} in the arguments of the variables in f_{1}(·) and f_{2}(·) satisfy the relation j_{u_{s}} = j′_{u′_{s}}, 1 ≤ s ≤ r, and there is no more coincidence between the indices j_{1}, …, j_{k_{1}}, j′_{1}, …, j′_{k_{2}}.

It is not difficult to see by applying the above partition of the terms in the product k_{1}!I_{n,k_{1}}(f_{1})k_{2}!I_{n,k_{2}}(f_{2}), and exploiting that each diagram of Γ̄(k_{1}, k_{2}) can be written in the form γ̄((u_{1}, u′_{1}), …, (u_{r}, u′_{r})) in a unique way that the identity

n^{−k_{1}/2}k_{1}!I_{n,k_{1}}(f_{1})k_{2}!n^{−k_{2}/2}I_{n,k_{2}}(f_{2}) = \begin{matrix}∑ & ′(n) \\ γ̄∈Γ̄(k_{1},k_{2}) \end{matrix} n^{−(k_{1}+k_{2})/2}s(γ̄)!I_{n,s(γ̄)}((\overline{f_{1} ∘ f_{2}})γ̄)

(12.1)

holds, where the functions (\overline{f_{1} ∘ f_{2}})γ̄ are defined in formula (11.3), s(γ̄) = k_{1}+k_{2}−|V_{2}(γ̄)| denotes the number of chains in γ̄, (both chains of length 1 and 2) and the notation ∑^{′(n)} means that summation is taken only for such diagrams γ̄ ∈ Γ̄(k_{1}, k_{2}) for which n ≥ s(γ̄). (Let me remark that although formula (11.3) was defined for coloured diagrams, the colours of the chains played no role in it.)

Relation (12.1) is not appropriate for our purposes, since the functions (\overline{f_{1} ∘ f_{2}})γ̄ in it may be non-canonical. To get the desired formula, Hoeffding’s decomposition will be applied for the U-statistics I_{n,s(γ̄)}((\overline{f_{1} ∘ f_{2}})γ̄) appearing at the right-hand side of formula (12.1). This decomposition becomes slightly simpler because of some special properties of the function (\overline{f_{1} ∘ f_{2}})γ̄ related to the canonical property of the initial functions f_{1} and f_{2}.

To carry out this procedure let us observe that a function f(x_{u_{1}}, …, x_{u_{k}}) is canonical if and only if P_{u_{s}}f(x_{u_{1}}, …, x_{u_{k}}) = 0 with the operator P_{u_{s}} defined in (11.1) for all indices u_{s}, 1 ≤ s ≤ k. Beside this, the condition that the functions f_{1} and f_{2} are canonical implies the relation P_{v}(\overline{f_{1} ∘ f_{2}})γ̄ = 0 for v ∈ V_{1}(γ̄) for all diagrams γ̄ ∈ Γ̄(k_{1}, k_{2}), and this relation remains valid if the function (\overline{f_{1} ∘ f_{2}})γ̄ is replaced by such functions which we get by applying the product of some transforms P_{v′} and Q_{v′}, v′ ∈ P_{2} for the function (\overline{f_{1} ∘ f_{2}})γ̄ with the transforms P and Q defined in formulas (11.1) and (11.2).

Beside this, the transforms P_{v} or Q_{v} are exchangeable with the operators P_{v′} or Q_{v′} if v ≠ v′, P_{v} + Q_{v}) = I, where I denotes the identity operator, and P_{v}Q_{v} = 0, since P_{v}Q_{v} = P_{v} − P_{v}^{2} = 0. The above relations make possible the following decomposition of the function (\overline{f_{1} ∘ f_{2}})γ̄ for all γ̄ ∈ Γ̄(k_{1}, k_{2}) to the sum of canonical functions (just as it was done in the Hoeffding decomposition):

(\overline{f_{1} ∘ f_{2}})γ̄ = ∏_{v∈V_{2}}(P_{v} + Q_{v})(\overline{f_{1} ∘ f_{2}})γ̄

= ∑_{A⊂V_{2}} ( ∏_{v∈A} P_{v} ∏_{v∈V_{2}\A} Q_{v}) (\overline{f_{1} ∘ f_{2}})γ̄ = ∑_{γ∈Γ(γ̄)}(f_{1} ∘ f_{2})_{γ}, (12.2)

where the function (f_{1} ∘ f_{2})_{γ} is defined in formula (11.4), and Γ(γ̄) denotes the set of those coloured diagrams γ ∈ Γ(k_{1}, k_{2}) which consist of those chains (with a colour ±1) as the non-coloured diagram γ̄. (Clearly, s(γ) = s(γ̄) for the number of chains of γ and γ̄ if γ ∈ Γ(γ̄).) Indeed, given a set A ⊂ V_{2}, we have ( ∏_{v∈A} P_{v} ∏_{v∈V_{2}\A} Q_{v})(\overline{f_{1} ∘ f_{2}})γ̄ =

(f_{1} ∘f_{2})_{γ} with that coloured diagram γ ∈ Γ(γ̄) whose chains with colour 1 are the chains β(l) ∈ γ̄ with l ∈ A, and which contains the remaining chains β(l) ∈ γ̄ with colour −1. Then we get relation (12.2) by summing up this identity for all A ⊂ V_{2}. The function (f_{1} ∘f_{2})_{γ} corresponding to the coloured diagram obtained with the help of the set A has |O(γ)| = k_{1} + k_{2} − |V_{2}(γ̄)| − |A| variables, where |O(γ)| is the number of open indices in γ.

Let us consider the functions F_{γ}(f_{1}, f_{2}), γ ∈ Γ(k_{1}, k_{2}), defined in (11.5) which means a reindexation of the functions (f_{1} ∘ f_{2})_{γ} to get functions with variables x_{1}, …, x_{|O(γ)|}. We claim that

\begin{matrix}n^{−(k_{1}+k_{2})/2}|O(γ̄)|!I_{n,s̄(γ̄)} ((\overline{f_{1} ∘ f_{2}})γ̄) \\ = ∑_{γ∈Γ(γ̄)} n^{−(k_{1}+k_{2})/2}n^{|C(γ)|}J_{n}(γ)|O(γ)|!I_{n,|O(γ)|} (F_{γ}(f_{1}, f_{2})) \end{matrix}  (12.3)

with J_{n}(γ) = 1 if |C(γ)| = 0, and

J_{n}(γ) = ∏_{j=1}^{|C(γ)|} (\frac{n − s(γ) + j}{n} ) if |C(γ)| > 0.  (12.4)

for all γ̄ ∈ Γ̄(k_{1}, k_{2}).

Since I_{n,|O(γ)|} (F_{γ}(f_{1}, f_{2})) = I_{n,|O(γ)|} (f_{1} ∘ f_{2})_{γ}) relation (12.3) follows from relation (12.2) just as formula (9.3) follows from formula (9.2) in the proof of the Hoeffding decomposition. Let us understand why the coefficient n^{|C(γ)|}J_{n}(γ) appears at the righthand side of (12.3).

This coefficient can be calculated in the following way. Take a general term (f_{1} ∘ f_{2})_{γ}(ξ_{j_{lu}} , l_{u} ∈ O(γ)) in the U-statistic |O(γ)|!I_{n,|O(γ)|}((f_{1} ∘ f_{2})_{γ}), and calculate the number of terms (\overline{f_{1} ∘ f_{2}})γ̄(ξ_{j′_{1}}, ξ_{j′_{2}} …, ξ_{j′_{s(γ̄)}}) in the U-statistic |O(γ̄)|!I_{n,s̄(γ̄)}((\overline{f_{1} ∘ f_{2}})γ̄) for which the sequence of indices (j′_{1}, …, j′_{s(γ̄)}) satisfies the relation j′_{l_{u}} = j_{l_{u}} for all l_{u} ∈ O(γ). I claim that it equals n^{|C(γ)|}J_{n}(γ). It can be seen that this number n^{|C(γ)|}J_{n}(γ) appears as the coefficient at right-hand side of (12.3).

Indeed, we have to calculate the number of such sequences j′_{1}, j′_{2}, …, j′_{s(γ̄)} for which the value j′_{l_{u}} = j_{l_{u}} is prescribed for the indices l_{u} ∈ O(γ), and the other elements of the sequence can take arbitrary integer value between 1 and n with the only restriction that all elements of the sequence j′_{1}, j′_{2}, …, j′_{s(γ̄)} must be different. The number of such sequences equals (n−|O(γ)|)(n−|O(γ)|−1) ⋯ (n−|C(γ)|−|O(γ)|+1) = J_{n}(γ)n^{|C(γ)|}. (In this calculation we exploited the fact that |O(γ)| + |C(γ)| = s(γ).)

Let us observe that k_{1} + k_{2} − 2|C(γ)| = |O(γ)| + W (γ) with the number W (γ) introduced in the formulation of Theorem 11.1. Hence

n^{−(k_{1}+k_{2})/2}n^{|C(γ)|} = n^{−W (γ)/2}n^{−|O(γ)|/2}.

Let us replace the left-hand side of the last identity by its right-hand side in (12.3), and let us sum up the identity we get in such a way for all γ̄ ∈ Γ̄(k_{1}, k_{2}) such that s(γ̄) ≤ n. The identity we get in such a way together with formulas (12.1) and (12.4) imply the identity (11.6). Clearly, I_{n,|O(γ)|}(F_{γ}(f_{1}, f_{2})) = I_{n,|O(γ)|}(SymF_{γ}(f_{1}, f_{2})), hence the term I_{n,|O(γ)|}(F_{γ}(f_{1}, f_{2})) can be replaced by I_{n,|O(γ)|}(SymF_{γ}(f_{1}, f_{2})) in formula (11.6). We still have to prove inequalities (11.7) and (11.8).

Inequality (11.7), the estimate of the L_{2}-norm of the function (f_{1} ∘ f_{2})_{γ} follows from the Schwarz inequality, and actually it agrees with inequality (10.11), proved at the start of Appendix B. Hence its proof is omitted here.

To prove inequality (11.8) let us introduce, similarly to formula (11.2), the operators

Q̃_{u_{j}}h(x_{u_{1}}, …, x_{u_{r}}) = h(x_{u_{1}}, …, x_{u_{r}}) + ∫ h(x_{u_{1}}, …, x_{u_{r}})μ( dx_{u_{j}}), 1 ≤ j ≤ r, (12.5)

in the space of functions h(x_{u_{1}}, …, x_{u_{r}}) with coordinates in the space (X, X). (The indices u_{1}, …, u_{r} are all different.) Observe that both the operators Q̃_{u_{j}} and the operators P_{u_{j}} defined in (11.1) are positive, i.e. these operators map a non-negative function to a non-negative function. Beside this, Q_{u_{j}} ≤ Q̃_{u_{j}}, and the norms of the operators

\frac{Q̃_{u_{j}}}{2} and P_{u_{j}} are bounded by 1 both in the L_{1}(μ), the L_{2}(μ) and the supremum norm. Let us define the function

(f̃_{1} ∘ f_{2})_{γ}(x_{j}, j ∈ O(γ)) = ( ∏_{j∈C(γ)} P_{j} ∏_{j∈O_{2}(γ)} Q̃_{j}) \overline{(f_{1} ∘ f_{2})}_{γ}(x_{j}, j ∈ C(γ) ∪ O(γ))

(12.6)

with the notation of Section 11. The function (f̃_{1} ∘ f_{2})_{γ} was defined with the help of \overline{(f_{1} ∘ f_{2})}_{γ} similarly to (f_{1} ∘ f_{2})_{γ} defined in (11.4), only the operators Q_{j} were replaced by Q̃_{j} in its definition.

In the proof of (11.8) it may be assumed that ∥f_{1}∥_{2} ≤ ∥f_{2}∥_{2}. The properties of the operators P_{u_{j}} and Q̃_{u_{j}} listed above together with the condition sup |f_{2}(x_{1}, …, x_{k})| ≤ 1 imply that

|(f_{1} ∘ f_{2})_{γ}| ≤ ( ˜ |f_{1}| ∘ |f_{2}|)_{γ} ≤ ( ˜ |f_{1}| ∘ 1)_{γ},  (12.7)

where ‘≤’ means that the function at the right-hand side is greater than or equal to the function at the left-hand side in all points, and the term 1 in (12.7) denotes the function which equals identically 1. Because of the identity ∥F_{γ}(f_{1}, f_{2})∥_{2} = ∥(f_{1} ∘ f_{2})_{γ}∥_{2} and relation (12.7) it is enough to show that

\begin{aligned}∥( ˜ |f_{1}| ∘ 1)_{γ}∥_{2} & = ∥( ∏_{j∈C(γ)} P_{j} ∏_{j∈O_{2}(γ)} Q̃_{j}) |f_{1}(x_{α_{γ}(1,1)}, …, x_{α_{γ}(1,k_{1})})|∥_{2} \\ & ≤ 2^{W (γ)}∥f_{1}∥_{2} \end{aligned}  (12.8)

to prove relation (11.8). But this inequality trivially holds, since the norm of all operators P_{j} in formula (12.8) is bounded by 1, the norm of all operators Q̃_{j} is bounded by 2 in the L_{2}(μ) norm, and |O_{2}(γ)| = W (γ).

Proof of Theorem 11.2. Theorem 11.2 will be proved with the help of Theorem 11.1 by induction with respect to the number of degenerate U-statistics k_{p}!I_{n,k_{p}}(f_{p}), 1 ≤ p ≤ m. Formula (11.15) holds for m = 2 by Theorem 11.1. To prove it for a general parameter m let us first fix a coloured diagram γ̄ ∈ Γ(k_{1}, …, k_{m−1}) and consider the set of diagrams of m rows which are its ‘continuation’, i.e. let

Γ(γ̄) = {γ: γ ∈ Γ(k_{1}, …, k_{m}), γ_{pr} = γ̄}.

(Here we work with the diagrams γ_{pr} and γ_{cl} introduced for a diagram γ ∈ Γ(k_{1}, …, k_{m}) in the previous section.) I claim that

\begin{matrix}n^{−|O(γ̄)|/2}|O(γ̄)|!I_{n,|O(γ̄)|}(Fγ̄(f_{1}, …, f_{m−1})) · n^{−k_{m}/2}k_{m}!I_{n,k_{m}}(f_{m}) \\ = \begin{matrix}∑ & ′(n) \\ γ∈Γ(γ̄) \end{matrix} ∏_{j=1}^{|C(γ_{cl})|} (\frac{n − s(γ_{cl}) + j}{n} ) n^{−W (γ_{cl})/2} \\ n^{−|O(γ)|/2}|O(γ))|!I_{n,|O(γ)|}(F_{γ}(f_{1}, …, f_{m})), \end{matrix}  (12.9)

where ^{∑′(n)} means that summation is taken for such γ ∈ Γ(γ̄) for which s(γ_{cl}) ≤ n,

and ∏_{j=1}^{|C(γ_{cl})|} equals 1, if |C(γ_{cl})| = 0.

Relation (12.9) can be checked by applying Theorem 11.1 for the pair of functions Fγ̄(f_{1}, …, f_{m−1}) and f_{m}. To get it first we show that there is a mutual correspondence between the coloured diagrams γ ∈ Γ(|O(γ̄)|, k_{m}) and the class of diagrams {γ_{cl}: γ ∈ Γ(γ̄)} in such a way that if γ ∈ Γ(γ̄) and γ′ ∈ Γ(|O(γ)|, k_{m}) correspond to each other, then γ′ = γ_{cl}. We shall fix an enumeration of the chains of the diagram γ̄, and we shall make such an enumeration of the chains of a diagram γ ∈ Γ(γ̄) for which the enumeration of the chains of γ̄ and γ_{pr} agree. The correspondence between the above mentioned two classes of diagrams depends on the enumeration of the chains of γ̄, but this will cause no problem. To get it observe that for each γ ∈ Γ(γ̄) there is a diagram γ′ = γ_{cl} ∈ Γ(|O(γ̄)|, k_{m}). On the other hand, I claim that for all diagrams γ′ ∈ Γ(|O(γ̄)|, k_{m}) such a diagram γ(γ′) ∈ Γ(γ̄) can be found for which γ(γ′)_{cl} = γ′.

This diagram γ(γ′) ∈ Γ(γ̄) will be defined in the following way. Let l̄_{1}, l̄_{2}, …, l̄_{|O(γ̄)|} be the indices of the chains of the diagram γ̄ with colour −1. The diagram γ(γ′) will be defined so that the chains of colour 1 of γ̄ will be chains of colour 1 of γ(γ′), too. If the vertex (1, j) of the diagram γ′ is contained in a chain of length 1, then the diagram γ(γ′) contains the chain β(l̄_{j}) with colour −1. If this vertex is contained in a chain {(1, j), (2, r_{j})} ∈ γ′ of length 2, then γ(γ′) contains the diagram β(l̄_{j}) ∪ {(m, r_{j})} with the same colour as the chain {(1, j), (2, r_{j})} has in γ′. Finally, if the vertex (2, r) is contained in the chain {(2, r)} of length 1 in γ′, then {(m, r)} will be a chain of length 1 of γ(γ′) with colour −1. In such a way we get such a diagram γ(γ′) ∈ Γ(γ̄) for which γ(γ′)_{cl} = γ′.

We get relation (12.9) by applying Theorem 11.1 for the product

n^{−|O(γ̄)|/2}|O(γ̄)|!I_{n,|O(γ̄)|}(Fγ̄(f_{1}, …, f_{m−1})) · n^{−k_{m}/2}k_{m}!I_{n,k_{m}}(f_{m})

and writing all diagrams γ′ ∈ Γ(|O(γ)|, k_{m}) in the form γ_{cl}, where γ_{cl} is the closing diagram of the diagram γ(γ′) ∈ Γ(γ̄) defined in the previous paragraph.

Relation (11.15) for the parameter m can be proved with the help of relation (12.9) and the inductive assumption by which it holds for m − 1. Indeed, let us multiply

formula (12.9) by ∏_{p=2}^{m−1} J_{n}(γ̄, p)n^{−W (γ̄)/2}, and sum up this identity for all such diagrams

γ̄ ∈ Γ(k_{1}, …, k_{m−1}) for which B_{1}(γ, p) + B_{2}(γ̄, p) ≤ n for all 2 ≤ p ≤ m − 1. Then the sum of the terms at the left-hand side equals the left-hand side of formula (11.15) for parameter m.

I claim that the sum of the terms at the right-hand side equals the right-hand side of formula (11.15) for parameter m. To see this it is enough to check that for all γ ∈ Γ(γ̄) we

have W (γ̄) + W (γ_{cl}) = W (γ_{pr}) + W (γ_{cl}) = W (γ), ∏_{p=2}^{m−1} J_{n}(γ_{pr}, p) ∏_{j=1}^{|C(γ_{cl})|} (\frac{n−s(γ_{cl})+j}{n} ) =

∏_{p=2}^{m} J_{n}(γ, p), where ∏_{j=1}^{|C(γ_{cl})|} = 1 if |C(γ_{cl})| = 0, and the relation B_{1}(γ, p) + B_{2}(γ, p) ≤ n

holds for all 2 ≤ p ≤ m if and only if B_{1}(γ_{pr}, p) + B_{2}(γ_{pr}, p) ≤ n for all 2 ≤ p ≤ m − 1, and s(γ_{cl}) ≤ n. But these relations can be simply checked. The identity about the function W (·) can be checked by taking into account the definition of the diagrams γ_{pr} and γ_{cl}, in particular the colouring of the chains in these diagrams. The remaining relations can be proved with the help of the observation that for a diagram γ ∈ Γ(k_{1}, …, k_{m}) B_{1}(γ_{pr}, p) = B_{1}(γ, p) and B_{2}(γ_{pr}, p) = B_{2}(γ, p) for all 2 ≤ p ≤ m − 1. Beside this |C(γ_{cl})| = B_{1}(γ, m) and |O(γ_{cl})| = B_{2}(γ, m). Theorem 11.2 is proved.

Proof of Lemma 11.3. The proof is similar to that of formula (11.8) at the end of Theorem 11.1. Let us define the functions F̃_{γ}(f_{1}, …, f_{p}), γ ∈ Γ(k_{1}, …, k_{p}), recursively for all 2 ≤ p ≤ m similarly to the definition of the functions F_{γ}(f_{1}, …, f_{p}) with the difference that the operator Q_{u_{j}} = I − P_{u_{j}} is replaced by Q̃_{u_{j}} = I + P_{u_{j}} in the new definition. Then we have |F_{γ}(f_{1}, …, f_{m})| ≤ F̃_{γ}(|f_{1}|, …, |f_{m}|) in all points. Hence ∥F_{γ}(f_{1}, …, f_{m})∥_{2} ≤ ∥F̃_{γ}(f_{1}, …, f_{m})∥_{2}, and to prove Lemma 11.3 it is enough to show that

∥F̃_{γ}(|f_{1}|, …, |f_{m}|)∥_{2} ≤ 2^{W (γ)} ∏_{p∈U(γ)} ∥f_{p}∥_{2} if γ ∈ Γ(k_{1}, …, k_{m})  (12.10)

with the same number W (γ) and set U(γ) which were considered in Lemma 11.3. Relation (12.10) will be proved by induction with respect to m.

Relation (12.10) holds for m = 2. Indeed, if W (γ) = 0, then U(γ) = {1, 2}, we have F̃_{γ} = F_{γ}, and formula (11.7) supplies the estimate. If W (γ) ≥ 1, then U(γ) = {1}, and actually in the proof of relation (11.8) we proved this relation.

In the case m > 2 this inequality will be proved by induction with the help of the identity (with the notation of formula (11.3)

\begin{matrix}∥F̃_{γ}(|f_{1}|, …, |f_{m}|)∥_{2} = ∥( ∏_{p∈C(γ_{cl})} P_{p} ∏_{p∈O_{2}(γ_{cl})} Q̃_{p}) \\ \overline{(F̃_{γ_{pr}}(|f_{1}|, …, |f_{m−1}|) ∘ |f_{m}|)}_{γ_{cl}}(x_{p}, p ∈ O(γ_{cl}) ∪ C(γ_{cl}))∥_{2}. \end{matrix} (12.11)

In the case W (γ_{cl}) = 0, i.e. if γ_{cl} contains no open chain of length 2 we have U(γ) = U(γ_{pr}) ∪ {m}, W (γ) = W (γ_{pr}), and formula (2.11) contains no operator Q̃_{p}. In this case inequality (12.10) follows from the representation of ∥F̃_{γ}(|f_{1}|, …, |f_{m}|)∥_{2} given in (12.11), relation (11.7) and from the inductive hypothesis by which inequality (12.10) holds for ∥(F̃_{γ_{pr}}(|f_{1}|, …, |f_{m−1}|)∥_{2}.

In the case W (γ_{cl}) > 0 we have U(γ) = U(γ_{pr}), W (γ) = W (γ_{pr}) + W (γ_{cl}), and inequality (12.10) can be proved similarly to the case W (γ_{cl}) = 0 with the only difference that in this case instead of (11.7) we have to apply that strengthened version of (11.8) which is contained in formula (12.10) in the special case m = 2. Lemma 11.3 is proved.

The corollary of Theorem 11.2 is a simple consequence of Theorem 11.2 and Lemma 11.3.

Proof of the corollary of Theorem 11.2. Observe that F_{γ} is a function of |O(γ)| arguments. Hence a coloured diagram γ ∈ Γ(k_{1}, …, k_{m}) is in the class of closed diagrams, i.e. γ ∈ Γ̄(k_{1}, …, k_{m}) if and only if F_{γ}(f_{1}, …, f_{m}) is a constant. Thus formula (11.16) is a simple consequence of relation (11.15) and the observation that EI_{n,|O(γ)|}(F_{γ}(f_{1}, …, f_{m})) = 0 if |O(γ)| ≥ 1, i.e. if γ ∉ Γ̄(k_{1}, …, k_{m}), and

I_{n,|O(γ)|}(F_{γ}(f_{1}, …, f_{m})) = I_{n,0}(F_{γ}(f_{1}, …, f_{m})) = F_{γ}(f_{1}, …, f_{m})
if γ ∈ Γ̄(k_{1}, …, k_{m}).

Relations (11.17) and (11.18) follow from relation (11.16) and Lemma 11.3.

13. The proof of Theorems 8.3, 8.5 and Example 8.7.

This section contains the proof of the estimates on the distribution of a multiple Wiener– Itô integral or degenerate U-statistic formulated in Theorems 8.5 and 8.3 together with the proof of Example 8.7. Beside this, also a multivariate version of Hoeffding’s inequality (Theorem 3.4) will be proved here. The latter result is useful in the estimation of the supremum of degenerate U-statistics. The estimate on the distribution of a multiple random integral with respect to a normalized empirical distribution given in Theorem 8.1 is omitted, because, as it was shown in Section 9, this result follows from the estimate of Theorem 8.3 on degenerate U-statistics. This section will be finished with a separate part Section 13 B, where the results proved in this section are discussed together with the method of their proofs and some recent results.

The proof of Theorems 8.5 and 8.3 is based on a good estimate on high moments of Wiener–Itô integrals and degenerate U-statistics. These estimates follow from the corollaries of Theorems 10.2 and 11.2. Such an approach slightly differs from the classical proof in the one-variate case. The natural one-variate version of the problems discussed here is an estimate about the tail distribution of a sum of independent random variables. This estimate is generally proved with the help of a good bound on the moment generating function of the sum. Such a method may not work in the multivariate case, because, as later calculations will show, there is no good estimate on the moment-generating function estimate of U-statistics or multiple Wiener–Itô integrals of order k ≥ 3. Actually, the moment-generating function of a Wiener–Itô integral of order k ≥ 3 is always divergent, because the tail behaviour of such a random integral is similar to that of the k-th power of a Gaussian random variable. On the other hand, good bounds on the moments EZ^{2M} of a random variable Z for all positive integers M (or at least for a sufficiently rich class of parameters M) together with the application of the Markov inequality for Z^{2M} and an appropriate choice of the parameter M yield a good estimate on the distribution of Z.

Propositions 13.1 and 13.2 give estimates on the moments of Wiener–Itô integrals and degenerate U-statistics.

Proposition 13.1. (Estimate of the moments of Wiener–Itô integrals). Let f(x_{1}, …, x_{k}) be a function of k variables on some measurable space (X, X) that satisfies formula (8.12) with some σ-finite measure μ. Take the k-fold Wiener–Itô integral Z_{μ,k}(f) of this function with respect to a white noise μ_{W} with reference measure μ. The inequality

E (k!|Z_{μ,k}(f)|)^{2M} ≤ 1 · 3 · 5 ⋯ (2kM − 1)σ^{2M} for all M = 1, 2, …  (13.1)

holds.

By Stirling’s formula Proposition 13.1 implies that

E(k!|Z_{μ,k}(f)|)^{2M} ≤ \frac{(2kM)!}{2^{kM}(kM)!}σ^{2M} ≤ A (\frac{2}{e})^{kM} (kM)^{kM}σ^{2M}  (13.2)

for any A > \sqrt{2} if M ≥ M_{0} = M_{0}(A). Formula (13.2) can be considered as a simpler, better applicable version of Proposition 13.1. It can be better compared with the moment estimate on degenerate U-statistics given in (13.3).

Proposition 13.2 provides a similar, but weaker inequality for the moments of normalized degenerate U-statistics.

Proposition 13.2. (Estimate on the moments of degenerate U-statistics). Let us consider a degenerate U-statistic I_{n,k}(f) of order k with sample size n and with a kernel function f satisfying relations (8.1) and (8.2) with some 0 < σ^{2} ≤ 1. Fix a positive number η > 0. There exist some universal constants A = A(k) > \sqrt{2}, C = C(k) > 0 and M_{0} = M_{0}(k) ≥ 1 depending only on the order of the U-statistic I_{n,k}(f) such that

E (n^{−k/2}k!I_{n,k}(f))^{2M} ≤ A (1 + C\sqrt{η})^{2kM} (\frac{2}{e})^{kM} (kM)^{kM} σ^{2M}
for all integers M such that kM_{0} ≤ kM ≤ ηnσ^{2}.

(13.3)

In formula (13.3) such a constant C = C(k) can be chosen which does not depend on the order k of the U-statistic I_{n,k}(f). For instance C = 4 is an appropriate choice.

Theorem 13.2 yields a good estimate on E (n^{−k/2}k!I_{n,k}(f))^{2M} with a fixed exponent 2M with the choice η = \frac{kM}{nσ^{2}} . With such a choice of the number η formula (13.3)

yields an estimate on the moments E (n^{−k/2}k!I_{n,k}(f))^{2M} comparable with the estimate on the corresponding Wiener–Itô integral if M ≤ nσ^{2}, while it yields a much weaker estimate if M ≫ nσ^{2}.

Now I turn to the proof of these propositions.

Proof of Proposition 13.1. Proposition 13.1 can be simply proved by means of the Corollary of Theorem 10.2 with the choice m = 2M, and f_{p} = f for all 1 ≤ p ≤ 2M. Formulas (10.18) and (10.19) yield that

E (k!Z_{μ,k}(f)^{2M}) ≤ (∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ(dx_{k}))^{M} |Γ_{2M}(k)| ≤ |Γ_{2M}(k)|σ^{2M},

where |Γ_{2M}(k)| denotes the number of closed diagrams γ in the class Γ̄(\underbrace{k, …, k}_{2M times}) intro-

duced in the corollary of Theorem 10.2. Thus to complete the proof of Proposition 13.1 it is enough to show that |Γ_{2M}(k)| ≤ 1 · 3 · 5 ⋯ (2kM − 1). But this can easily be seen with the help of the following observation. Let Γ̄_{2M}(k) denote the class of all graphs with vertices (l, j), 1 ≤ l ≤ 2M, 1 ≤ j ≤ k, such that from all vertices (l, j) exactly one edge starts, all edges connect different vertices, but edges connecting vertices (l, j) and (l, j′) with the same first coordinate l are also allowed. Let |Γ̄_{2M}(k)| denote the number of graphs in Γ̄_{2M}(k). Then clearly |Γ_{2M}(k)| ≤ |Γ̄_{2M}(k)|. On the other hand, |Γ̄_{2M}(k)| = 1 · 3 · 5 ⋯ (2kM − 1). Indeed, let us list the vertices of the graphs from Γ̄_{2M}(k) in an arbitrary way. Then the first vertex can be paired with another vertex in 2kM − 1 way, after this the first vertex from which no edge starts can be paired with 2kM − 3 vertices from which no edge starts. By following this procedure the next edge can be chosen 2kM − 5 ways, and by continuing this calculation we get the desired formula.

Proof of Proposition 13.2. Relation (13.3) will be proved by means of relations (11.17) and (11.18) in the Corollary of Theorem 11.2 with the choice m = 2M and f_{p} = f for all 1 ≤ p ≤ 2M. Let us take the class of closed coloured diagrams Γ(k, M) = Γ̄(\underbrace{k, …, k}_{2Mtimes}).

This will be partitioned into subclasses Γ(k, M, r), 1 ≤ r ≤ kM, where Γ(M, k, r) contains those closed diagrams γ ∈ Γ(k, M) for which W (γ) = 2r. Let us recall that W (γ) was defined in (11.13), and in the case of closed diagrams W (γ) = ∑_{β∈γ}(ℓ(β) − 2).

For a diagram γ ∈ Γ(k, M), W (γ) is an even number, since W (γ)+2s(γ) = 2kM, where s(γ) denotes the number of chains in γ.

First we prove an estimate about the cardinality of Γ(M, k, r). We claim that there exist some constant A = A(k) > 0 and threshold index M_{0} = M_{0}(k) depending only the order k of the U-statistic In, k(f) for which

|Γ(k, M, r)| ≤ A\binom{2kM}{2r} (\frac{2}{e})^{kM} (kM)^{kM+r}2^{2r} for all 0 ≤ r ≤ kM  (13.4)

if A ≥ A_{0}(k) and M ≥ M_{0}(k).

To prove formula (13.4) we define a map T: γ → T (γ) from the set of diagrams γ ∈ Γ(k, M, r) to the set of paired diagrams in such a way that T (γ) ≠ T (γ′) if γ ≠ γ′, and give a good bound on the number of paired diagrams T (γ), γ ∈ Γ(k, M, r), obtained in such a way. (We shall call a diagram γ a paired diagram, if all of its chains have length 2, i.e. the have the form β = {(p, r), (p′, r′)} ∈ γ, with p ≠ p′.) To define the paired diagrams we shall work with first we introduce the

set W(γ) = ⋃_{β∈γ}{(p_{2}(β), q_{2}(β)), …, (p_{s−1}(β), q_{s−1}(β))}, for all γ ∈ Γ(k, M, r), where

β = {(p_{1}(β), q_{1}(β)), …, (p_{s}(β), q_{s}(β))} with 1 ≤ p_{1}(β) < p_{2}(β) < ⋯ < p_{s}(β) ≤ 2M for all β ∈ γ, i.e. W(γ) is the set of vertices we get by omitting the first and last vertices of all chains β ∈ γ, and then taking the union of the vertices of these diminished chains. Observe that |W(γ)| = W (γ) for a closed diagram.

We take a copy (p, q, C) of all elements (p, q) ∈ W(γ) of a diagram γ ∈ Γ(k, M, r). First we define the set of vertices V (T (γ)) of the paired diagram T (γ). It is a set of vertices consisting of 2M rows, and its p-th row is {(p, 1), …, (p, k_{p})} ∪ {(p, q, C): (p, q) ∈ W(γ)} for all 1 ≤ p ≤ 2M. We have |V (T (γ)| = 2kM + |W(γ)| = 2kM + 2r. We define the paired diagram T (γ) on the set V (T (γ)) in the following way. Given a chain β = {(p_{1}(β), q_{1}(β)), …, (p_{s}(β), q_{s}(β))} ∈ γ, with 1 ≤ p_{1}(β) < p_{2}(β) < ⋯ < p_{s}(β) ≤ 2M, we correspond to it the following sets of pairs (chains of length 2) in V (T (γ)):

{((p_{1}(β), q_{1}(β)), ((p_{2}(β), q_{2}(β), C)}, {((p_{2}(β), q_{2}(β)), ((p_{3}(β), q_{3}(β), C)}, …, {((p_{s−2}(β), q_{s−2}(β)), ((p_{s−1}(β), q_{s−1}(β), C)}, {((p_{s−1}(β), q_{s−1}(β)), ((p_{s}(β), q_{s}(β)}.

(In the case ℓ(β) = 2, we map β to itself.) Defining these pairs of vertices for all β ∈ γ we get the paired diagram T (γ) with the desired properties.

The number of the above defined sets V (T (γ)), γ ∈ Γ(k, M, r), is less than or equal to \binom{2kM}{2r}, and each of these sets V (T (γ)) has 2kM + 2r vertices. Hence the number of paired diagrams with vertices in a fixed set V (T (γ)) is bounded by 1·3··5·(2kM −2r−1). The above considerations provide the bound

|Γ(k, M, r)| ≤ \binom{2kM}{2r}1 · 3 · 5 ⋯ (2kM + 2r − 1) = \binom{2kM}{2r}\frac{(2kM + 2r)!}{2^{kM+r}(kM + r)!}. (13.5)

Stirling’s formula yields that \frac{(2kM+2r)!}{2^{kM+r}(kM+r)!} ≤ A (\frac{2}{e})^{kM+r} (kM + r)^{kM+r} with some constant A > \sqrt{2} if M ≥ M_{0} with some M_{0} = M_{0}(A). Since r ≤ kM we can write (kM + r)^{kM+r} ≤ (kM)^{kM} (1 + \frac{r}{kM} )^{kM} (2kM)^{r} ≤ (kM)^{kM+r}e^{r}2^{r}. The above calculation together with (13.5) imply inequality (13.4).

For a diagram γ ∈ Γ(k, M, r) we have W (γ) = 2r, and beside this the cardinality of the set U(γ) defined in the formulation of Lemma 11.3 satisfies the inequality |U(γ)| ≥ 2M − W (γ) = 2M − 2r. Hence by relation (11.18) n^{−W (γ)/2}|F_{γ}| ≤ 2^{2r}n^{−r}σ^{|U(γ)|} ≤ 2^{2r} (nσ^{2})^{−r} σ^{2M} ≤ η^{r}2^{2r}(kM)^{−r}σ^{2M} for γ ∈ Γ(k, M, r) if kM ≤ ηnσ^{2} and σ^{2} ≤ 1.

This estimate together with relation (11.17) imply that for kM ≤ ηnσ^{2}

E (n^{−k/2}k!I_{n,k}(f_{k}))^{2M} ≤ ∑_{γ∈Γ(k,M)} n^{−W (γ)/2} · |F_{γ}| ≤ ∑_{r=0}^{kM} |Γ(k, M, r)|η^{r}2^{2r}(kM)^{−r}σ^{2M}.

Hence by formula (13.4)

E (n^{−k/2}k!I_{n,k}(f_{k}))^{2M} ≤ A (\frac{2}{e})^{kM} (kM)^{kM}σ^{2M} ∑_{r=0}^{kM} \binom{2kM}{2r} (4\sqrt{η})^{2r}

≤ A (\frac{2}{e})^{kM} (kM)^{kM}σ^{2M} (1 + 4\sqrt{η})^{2kM}

if kM_{0} ≤ kM ≤ ηnσ^{2}. Thus we have proved Proposition 13.2 with C = 4.

It is not difficult to prove Theorem 8.5 with the help of Proposition 13.1.

Proof of Theorem 8.5. By formula (13.2) which is a consequence of Proposition 13.1 and the Markov inequality

P (|k!Z_{μ,k}(f)| > u) ≤ \frac{E (k!Z_{μ,k}(f))^{2M}}{u^{2M}} ≤ A (\frac{2kMσ^{2/k}}{eu^{2/k}} )^{kM}  (13.6)

with some constant A > \sqrt{2} if M ≥ M_{0} with some constant M_{0} = M_{0}(A), and M is an integer.

Put M̄ = M̄(u) = \frac{1}{2k} (\frac{u}{σ})^{2/k}, and M = M(u) = [M̄], where [x] denotes the integer

part of a real number x. Choose some number u_{0} such that \frac{1}{2k} (\frac{u_{0}}{σ} )^{2/k} ≥ M_{0} + 1. Then relation (13.6) can be applied with M = M(u) for u ≥ u_{0}, and it yields that

\begin{aligned}P (|k!Z_{μ,k}(f)| > u) & ≤ A (\frac{2kMσ^{2/k}}{eu^{2/k}} )^{kM} ≤ e^{−kM} ≤ Ae^{k}e^{−kM̄} \\ & = Ae^{k} exp \{−\frac{1}{2} (\frac{u}{σ} )^{2/k}\} if u ≥ u_{0}. \end{aligned}  (13.7)

Relation (13.7) means that relation (8.14) holds for u ≥ u_{0} with the pre-exponential coefficient Ae^{k}. By enlarging this coefficient if it is needed it can be guaranteed that relation (8.14) holds for all u > 0. Theorem 8.5 is proved.

Theorem 8.3 can be proved similarly by means of Proposition 13.2. Nevertheless, the proof is technically more complicated, since in this case the optimal choice of the parameter in the Markov inequality cannot be given in such a direct form as in the proof of Theorem 8.5. In this case the Markov inequality is applied with an only almost optimal choice of the parameter M.

Proof of Theorem 8.3. The Markov inequality and relation (13.3) with η = \frac{kM}{nσ^{2}} imply that

\begin{aligned}P (k!n^{−k/2}|I_{n,k}(f)| > u) & ≤ \frac{E (k!n^{−k/2}I_{n,k}(f))^{2M}}{u^{2M}} \\ & ≤ A (\frac{1}{e} · 2kM (1 + C \frac{\sqrt{kM}}{\sqrt{n}σ} )^{2} (\frac{σ}{u} )^{2/k})^{kM} \end{aligned}  (13.8)

for all integers M ≥ M_{0} with some M_{0} = M_{0}(A).

Relation (8.10) will be proved with the help of estimate (13.8) first in the case D ≤ \frac{u}{σ} ≤ n^{k/2}σ^{k} with a sufficiently large constant D = D(k, C) > 0 depending on k and the constant C in (13.8). To this end let us introduce the number M̄ by means of the formula

kM̄ = \frac{1}{2} (\frac{u}{σ} )^{2/k} \frac{1}{1 + B \frac{(\frac{u}{σ})^{1/k}}{\sqrt{n}σ}} = \frac{1}{2} (\frac{u}{σ} )^{2/k} \frac{1}{1 + B (un^{−k/2}σ^{−(k+1)})^{1/k}}

with a sufficiently large number B = B(C) > 0 and M = [M̄], where [x] means the integer part of the number x.

Observe that \sqrt{kM̄} ≤ (\frac{u}{σ})^{1/k}, \frac{\sqrt{kM̄}}{\sqrt{n}σ} ≤ (un^{−k/2}σ^{−(k+1)})^{1/k} ≤ 1, and

(1 + C \frac{\sqrt{kM̄}}{\sqrt{n}σ} )^{2} ≤ 1 + B \frac{\sqrt{kM̄}}{\sqrt{n}σ} ≤ 1 + B (un^{−k/2}σ^{−(k+1)})^{1/k}

with a sufficiently large B = B(C) > 0 if \frac{u}{σ} ≤ n^{k/2}σ^{k}. Hence

\begin{aligned}\frac{1}{e} · 2kM (1 + C \frac{\sqrt{kM}}{\sqrt{n}σ} )^{2} (\frac{σ}{u} )^{2/k} & ≤ \frac{1}{e} · 2kM̄ (1 + C \frac{\sqrt{kM̄}}{\sqrt{n}σ} )^{2} (\frac{σ}{u} )^{2/k} \\ & = \frac{1}{e} · \frac{(1 + C \frac{\sqrt{kM̄}}{\sqrt{n}σ} )^{2}}{1 + B (un^{−k/2}σ^{−(k+1)})^{1/k}} ≤ \frac{1}{e} \end{aligned}  (13.9)

if \frac{u}{σ} ≤ n^{k/2}σ^{k}. If the inequality D ≤ \frac{u}{σ} also holds with a sufficiently large D = D(B, k) > 0, then M ≥ M_{0}, and the conditions of inequality (13.8) hold. This inequality together with inequality (13.9) yield that

P (k!n^{−k/2}|I_{n,k}(f)| > u) ≤ Ae^{−kM} ≤ Ae^{k}e^{−kM̄}

if D ≤ \frac{u}{σ} ≤ n^{k/2}σ^{k}, i.e. inequality (8.10) holds in this case with a pre-exponential constant Ae^{k}. In the case \frac{u}{σ} ≤ D the right-hand side of (8.10) is larger than 1 if we choose the pre-exponential term A sufficiently large. Hence inequality (8.10) holds for all 0 ≤ \frac{u}{σ} ≤ n^{k/2}σ^{k} with a sufficiently large pre-exponential term A. Theorem 8.3 is proved.

Example 8.7 is a relatively simple consequence of Itô’s formula for multiple Wiener– Itô integrals.

Proof of Example 8.7. We may restrict our attention to the case k ≥ 2. Itô’s formula for multiple Wiener-Itô integrals, more explicitly relation (10.21), implies that the random variable k!Z_{μ,k}(f) can be expressed as k!Z_{μ,k}(f) = σH_{k} (∫ f_{0}(x)μ_{W} ( dx)) = σH_{k}(η), where H_{k}(x) is the k-th Hermite polynomial with leading coefficient 1, and η = ∫ f_{0}(x)μ_{W} ( dx) is a standard normal random variable. Hence we get by exploiting that the coefficient of x^{k−1} in the polynomial H_{k}(x) is zero that P (k!|Z_{μ,k}(f)| > u) = P (|H_{k}(η)| ≥ \frac{u}{σ}) ≥ P (|η^{k}| − D|η^{k−2}| > \frac{u}{σ}) with a sufficiently large constant D > 0 if \frac{u}{σ} > 1. There exist such positive constants A and B that

P (|η^{k}| − D|η^{k−2}| > \frac{u}{σ} ) ≥ P (|η^{k}| > \frac{u}{σ} + A (\frac{u}{σ} )^{(k−2)/k}) if \frac{u}{σ} > B.

Hence

P (k!|Z_{μ,k}(f)| > u) ≥ P (|η| > (\frac{u}{σ} )^{1/k} (1 + A (\frac{u}{σ} )^{−2/k})) ≥ \frac{C̄ exp \{−\frac{1}{2} (\frac{u}{σ})^{2/k}\}}{(\frac{u}{σ})^{1/k} + 1}

with an appropriate C̄ > 0 if \frac{u}{σ} > B. Since P (k!|Z_{μ,k}(f)| > 0) > 0, the above inequality also holds for 0 ≤ \frac{u}{σ} ≤ B if the constant C̄ > 0 is chosen sufficiently small. This means that relation (8.16) holds.

Next we prove a multivariate version of Hoeffding’s inequality. Before its formulation some notations will be introduced.

Let us fix two positive integers k and n and some real numbers a(j_{1}, …, j_{k}) for all sequences of arguments {j_{1}, …, j_{k}} such that 1 ≤ j_{l} ≤ n, 1 ≤ l ≤ k, and j_{l} ≠ j_{l′} if l ≠ l′.

With the help of the above real numbers a(·) and a sequence of independent random variables ε_{1}, …, ε_{n}, P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, the random variable

V = ∑_{(j_{1},...,j_{k}): 1≤j_{l}≤n for all 1≤l≤k, j_{l}≠j_{l′} if l≠l′} a(j_{1}, …, j_{k})ε_{j_{1}} ⋯ ε_{j_{k}}  (13.10)

and number

S^{2} = ∑_{(j_{1},...,j_{k}): 1≤j_{l}≤n for all 1≤l≤k, j_{l}≠j_{l′} if l≠l′} a^{2}(j_{1}, …, j_{k}).  (13.11)

will be introduced.

With the help of the above notations the following result can be formulated.

Theorem 13.3. (The multivariate version of Hoeffding’s inequality). The random variable V defined in formula (13.10) satisfies the inequality

P (|V | > u) ≤ C exp \{−\frac{1}{2} (\frac{u}{S} )^{2/k}\} for all u ≥ 0  (13.12)

with the constant S defined in (13.11) and some constants C > 0 depending only on the parameter k in the expression V.

Theorem 13.3 will be proved by means of two simple lemmas. Before their formulation the random variable

Z = ∑_{(j_{1},...,j_{k}): 1≤j_{l}≤n for all 1≤l≤k, j_{l}≠j_{l′} if l≠l′} |a(j_{1}, …, j_{k})|η_{j_{1}} ⋯ η_{j_{k}}  (13.13)

will be introduced, where η_{1}, …, η_{n} are independent random variables with standard normal distribution, and the numbers a(j_{1}, …, j_{k}) agree with those in formula (13.10). The following lemmas will be proved.

Lemma 13.4. The random variables V and Z introduced in (13.10) and (13.13) satisfy the inequality

EV^{2M} ≤ EZ^{2M} for all M = 1, 2, ….  (13.14)

Lemma 13.5. The random variable Z defined in formula (13.13) satisfies the inequality

EZ^{2M} ≤ 1 · 3 · 5 ⋯ (2kM − 1)S^{2M} for all M = 1, 2, …  (13.15)

with the constant S defined in formula (13.11).

Proof of Lemma 13.4. We can write, by carrying out the multiplications in the expressions EV^{2M} and EZ^{2M}, by exploiting the additive and multiplicative properties of the expectation for sums and products of independent random variables together with the identities Eε_{j}^{2k+1} = 0 and Eη_{j}^{2k+1} = 0 for all k = 0, 1, … that

EV^{2M} = ∑_{\begin{matrix}(j_{1},...,j_{l}, m_{1},...,m_{l}): \\ 1≤j_{s}≤n, m_{s}≥1, 1≤s≤l, m_{1}+···+m_{l}=kM \end{matrix}} A(j_{1}, …, j_{l}, m_{1}, …, m_{l})Eε_{j_{1}}^{2m_{1}} ⋯ Eε_{j_{l}}^{2m_{l}} (13.16)

and

EZ^{2M} = ∑_{\begin{matrix}(j_{1},...,j_{l}, m_{1},...,m_{l}): \\ 1≤j_{s}≤n, m_{s}≥1, 1≤s≤l, m_{1}+···+m_{l}=kM \end{matrix}} B(j_{1}, …, j_{l}, m_{1}, …, m_{l})Eη_{j_{1}}^{2m_{1}} ⋯ Eη_{j_{l}}^{2m_{l}} (13.17)

with some coefficients A(j_{1}, …, j_{l}, m_{1}, …, m_{l}) and B(j_{1}, …, j_{l}, m_{1}, …, m_{l}) such that

|A(j_{1}, …, j_{l}, m_{1}, …, m_{l})| ≤ B(j_{1}, …, j_{l}, m_{1}, …, m_{l}).  (13.18)

The coefficients A(·, ·, ·) and B(·, ·, ·) could be expressed explicitly, but we do not need such a formula. What is important for us is that A(·, ·, ·) can be expressed as the sum of certain terms, and B(·, ·, ·) as the sum of the absolute value of the same terms. Hence relation (13.18) holds. Since Eε_{j}^{2m} ≤ Eη_{j}^{2m} for all parameters j and m formulas (13.16), (13.17) and (13.18) imply Lemma 13.4.

Proof of Lemma 13.5. Let us consider a white noise W (·) on the unit interval [0, 1] with the Lebesgue measure λ on [0, 1] as its reference measure, i.e. let us take a set of Gaussian random variables W (A) indexed by the measurable sets A ⊂ [0, 1] such that EW (A) = 0, EW (A)W (B) = λ(A∩B) with the Lebesgue measure λ for all measurable subsets of the interval [0, 1]. Let us introduce n orthonormal functions φ_{1}(x), …, φ_{n}(x) with respect to the Lebesgue measure on the interval [0, 1], and define the random variables η_{j} = ∫ φ_{j}(x)W ( dx), 0 ≤ j ≤ n. Then η_{1}, …, η_{n} are independent random variables with standard normal distribution, hence we may assume that they appear in the definition of the random variable Z in formula (13.13). Beside this, the identity η_{j_{1}} ⋯ η_{j_{k}} = ∫ φ_{j_{1}}(x_{1}) ⋯ φ_{j_{k}}(x_{k})W ( dx_{1}) … W ( dx_{k}) holds for all k-tuples (j_{1}, …, j_{k}), such that 1 ≤ j_{s} ≤ n for all 1 ≤ s ≤ k, and the indices j_{1}, …, j_{s} are different. This identity follows from Itô’s formula for multiple Wiener–Itô integrals formulated in formula (10.20) of Theorem 10.3.

Hence the random variable Z defined in (13.13) can be written in the form

Z = ∫ f(x_{1}, …, x_{k})W ( dx_{1}) … W ( dx_{k})

with the function

f(x_{1}, …, x_{k}) = ∑_{(j_{1},...,j_{k}): 1≤j_{l}≤n for all 1≤l≤k, j_{l}≠j_{l′} if l≠l′} |a(j_{1}, …, j_{k})|φ_{j_{1}}(x_{1}) ⋯ φ_{j_{k}}(x_{k}).

Because of the orthogonality of the functions φ_{j}(x)

S^{2} = ∫_{[0,1]^{k}} f^{2}(x_{1}, …, x_{k}) dx_{1} … dx_{k}.

Lemma 13.5 is a straightforward consequence of the above relations and formula (13.1) in Proposition 13.1.

Proof of Theorem 13.3. The proof of Theorem 13.3 with the help of Lemmas 13.4 and 13.5 is an almost word for word repetition of the proof of Theorem 8.5. By Lemma 13.4 inequality (13.15) remains valid if the random variable Z is replaced by the random variable V at its left-hand side. Hence the Stirling formula yields that

EV^{2M} ≤ EZ^{2M} ≤ \frac{(2kM)!}{2^{kM}(kM)!}S^{2M} ≤ C (\frac{2}{e})^{kM} (kM)^{kM}S^{2M}

for any C ≥ \sqrt{2} if M ≥ M_{0}(A). As a consequence, by the Markov inequality the estimate

P (|V | > u) ≤ \frac{EV^{2M}}{u^{2M}} ≤ C (\frac{2kM}{e} (\frac{S}{u} )^{2/k})^{kM}  (13.19)

holds for all C > \sqrt{2} if M ≥ M_{0}(C). Put kM̄ = kM̄(u) = \frac{1}{2} (\frac{u}{S} )^{2/k} and M = M(u) = [M̄], where [x] denotes the integer part of the number x. Let us choose a threshold number u_{0} by the identity \frac{1}{2k} (\frac{u_{0}}{S} )^{2/k} = M_{0}(C) + 1. Formula (13.19) can be applied with M = M(u) for u ≥ u_{0}, and it yields that

P (|V | > u) ≤ Ce^{−kM} ≤ Ce^{k}e^{−kM̄} = Ce^{k} exp \{−\frac{1}{2} (\frac{u}{S} )^{2/k}\} if u ≥ u_{0}.

The last inequality means that relation (13.12) holds for u ≥ u_{0} if the constant C is replaced by Ce^{k} in it. With the choice of a sufficiently large constant C relation (13.12) holds for all u ≥ 0. Theorem 13.3 is proved.

13. B) A short discussion about the methods and results.

A comparison of Theorem 8.5 and Example 8.7 shows that the estimate (8.15) is sharp. At least no essential improvement of this estimate is possible which holds for all Wiener– Itô integrals with a kernel function f satisfying the conditions of Theorem 8.5. This fact also indicates that the bounds (13.1) and (13.2) on high moments of Wiener–Itô integrals are sharp. It is worth while comparing formula (13.2) with the estimate of Proposition 13.2 on moments of degenerate U-statistics.

Let us consider a normalized k-fold degenerate U-statistic n^{−k/2}k!I_{n,k}(f) with some kernel function f and a μ-distributed sample of size n. Let us compare its moments with those of a k-fold Wiener–Itô integral k!Z_{μ,k}(f) with the same kernel function f with respect to a white noise μ_{W} with reference measure μ. Let σ denote the L_{2}-norm of the kernel function f. If M ≤ εnσ^{2} with a small number ε > 0, then Proposition 13.2 (with an appropriate choice of the parameter η which is small in this case) provides an almost as good bound on the 2M-th moment of the normalized U-statistic as Proposition 13.1 provides on the 2M-th moment of the corresponding Wiener–Itô integral. In the case M ≤ Cnσ^{2} with some fixed (not necessarily small) number C > 0 the 2M-th moment of the normalized U-statistic can be bounded by C(k)^{M} times the natural estimate on the 2M-th moment of the Wiener–Itô integral with some constant C(k) > 0 depending only on the number C. This can be so interpreted that in this case the estimate on the moments of the normalized U-statistic is weaker than the estimate on the moments of the Wiener–Itô integral, but they are still comparable. Finally, in the case M ≫ nσ^{2} the estimate on the 2M-th moment of the normalized U-statistic is much worse than the estimate on the 2M-th moment of the Wiener–Itô integral.

A similar picture arises if the distribution of the normalized degenerate U-statistic

F_{n}(u) = P (n^{−k/2}k!|I_{n,k}(f)| > u)

is compared to the distribution of the Wiener–Itô integral

G(u) = P (k!|Z_{μ,k}(f)| > u).

A comparison of Theorems 8.3 and 8.5 shows that for 0 ≤ u ≤ εn^{k/2}σ^{k+1} with a small ε > 0 an almost as good estimate holds F_{n}(u) as for G(u). In the case 0 ≤ u ≤ n^{k/2}σ^{k+1} the behaviour of F_{n}(u) and G(u) is similar, only in the exponent of the estimate on F_{n}(u) in formula (8.10) a worse constant appears. Finally, if u ≫ n^{k/2}σ^{k+1}, then — as Example 8.8 shows, at least in the case k = 2, — the (tail) distribution function F_{n}(u) satisfies a much worse estimate than the function G(u). Thus a similar picture arises as in the case when the estimate on the tail-distribution of normalized sums of independent random variables, discussed in Section 3, is compared to the behaviour of the standard normal distribution in the neighbourhood of infinity. To understand this similarity better it is useful to recall Theorem 10.4, the limit theorem about normalized degenerate U-statistics. Theorems 8.3 and 8.5 enable us to compare the tail behaviour of normalized degenerate U-statistics with their limit presented in the form of multiple Wiener–Itô integrals, while the one-variate versions of these results compare the distribution of sums of independent random variables with their Gaussian limit.

The above results show that good bounds on the moments of degenerate U-statistics and multiple Wiener–Itô also provide a good estimate on their distribution. To understand the behaviour of high moments of degenerate U-statistics it is useful to have a closer look at the simplest case k = 1, when the moments of sums of independent random variables with expectation zero are considered.

Let us consider a sequence of independent and identically distributed random vari-

ables ξ_{1}, …, ξ_{n} with expectation zero, take their sum S_{n} = ∑_{j=1}^{n} ξ_{j}, and let us try to

give a good estimate on the moments ES_{n}^{2M} for all M = 1, 2, . . .. Because of the independence of the random variables ξ_{j} and the condition Eξ_{j} = 0 the identity

ES_{n}^{2M} = ∑_{(j_{1},...,j_{s},l_{1},...,l_{s}) j_{1}+···+j_{s}=2M, j_{u}≥2, for all 1≤u≤s l_{u}≠l_{u′} if u≠u′} Eξ_{l_{1}}^{j_{1}} ⋯ Eξ_{l_{s}}^{j_{s}}  (13.20)

holds. Simple combinatorial considerations show that a dominating number of terms at the right-hand side of (13.20) are indexed by a vector (j_{1}, …, j_{M}; l_{1}, …, l_{M}) such that j_{u} = 2 for all 1 ≤ u ≤ M, and the number of such vectors is equal to \binom{n}{M} \frac{(2M)!}{2^{M}} ∼ n^{M} \frac{(2M)!}{2^{M} M!}. The last asymptotic relation holds if the number n of terms in the random sum S_{n} is sufficiently large. The above considerations suggest that under not too restrictive conditions ES_{n}^{2M} ∼ (nσ^{2})^{M} \frac{(2M)!}{2^{M} M!} = Eη_{nσ^{2}}^{2M}, where σ^{2} = Eξ^{2} is the variance of the terms in the sum S_{n}, and η_{u} denotes a random variable with normal distribution with expectation zero and variance u. The question arises when the above heuristic argument gives a right estimate.

For the sake of simplicity let us restrict our attention to the case when the absolute value of the random variables ξ_{j} is bounded by 1. Let us observe that even in this case the above heuristic argument holds only under the condition that the variance σ^{2} of the random variables ξ_{j} is not too small. Indeed, let us consider such random variables ξ_{j}, for which P (ξ_{j} = 1) = P (ξ_{j} = −1) = \frac{σ^{2}}{2} , P (ξ_{j} = 0) = 1 − σ^{2}. Then these random variables ξ_{j} have variance σ^{2}, and the contribution of the terms Eξ_{j}^{2M}, 1 ≤ j ≤ n, to the sum in

(13.20) equals nσ^{2}. If σ^{2} is very small, then it may happen that nσ^{2} ≫ (nσ^{2})^{M} \frac{(2M)!}{2^{M} M!}, and the approximation given for ES_{n}^{2M} in the previous paragraph does not hold any longer. Hence the asymptotic relation for a very high moment ES_{n}^{2M} suggested by the above heuristic argument may only hold if the variance σ^{2} of the summands satisfies an appropriate lower bound.

In the proof of Proposition 13.2 a similar picture appears in a hidden way. In the calculation of the moments of a degenerate U-statistic the contribution of certain (closed) diagrams, more precisely of some integrals defined with their help, has to be estimated. Some of these diagrams (those in which all chains have length 2) appear also in the calculation of the moments of multiple Wiener–Itô integrals. In the calculation of the moments of sums of independent random variables the terms consisting of products of second moments play such a role in the sum in formula (13.20) as the ‘nice’ diagrams consisting of chains of length 2 play in the calculation of the moments of degenerate U-statistics in formula (11.17). In nice cases the remaining diagrams do not give a much greater contribution than these ‘nice’ diagrams, and we get an almost as good bound for the moments of a normalized degenerate U-statistic as for the moments of the corresponding multiple Wiener–Itô integral. The proof of Proposition 13.2 shows that such a situation appears under very general conditions.

Let me also remark that there is an essential difference between the tail behaviour of Wiener–Itô integrals and normalized degenerate U-statistics. A good estimate can be given on the tail distribution of Wiener–Itô integrals which depends only on the L_{2}-norm of the kernel function, while in the case of normalized degenerate U-statistics the corresponding estimate depends not only on the L_{2}-norm but also on the L_{∞} norm of the kernel function. In Theorem 8.3 such an estimate is proved.

For k ≥ 2 the distribution of k-fold Wiener-Itô integrals are not determined by the L_{2}-norm of their kernel functions. This is an essential difference between Wiener–Itô integrals of order k ≥ 2 and k = 1. In the case k = 1 a Wiener–Itô integral is a Gaussian random variable with expectation zero, and its variance equals the square of the L_{2}-norm of its kernel function. Hence its distribution is completely determined by the L_{2}-norm of its kernel function. On the other hand, the distribution of a Wiener–Itô integral of order k ≥ 2 is not determined by its variance. Theorem 8.5 yields a ‘worst case’ estimate on the distribution of Wiener–Itô integrals if we have a bound on their variance. In the statistical problems which provided the main motivation for this work such estimates are needed, but it may be interesting to know what kind of estimates are known about the distribution of a multiple Wiener–Itô integral or degenerate Ustatistic if we have some additional information about its kernel function. Some results will be mentioned in this direction, but most technical details will be omitted from their discussion.

H. P. Mc. Kean proved the following lower bound on the distribution of multiple Wiener–Itô integrals. (See [29] or [42].)

Theorem 13.6. (Lower bound on the distribution of Wiener–Itô integrals). All k-fold Wiener–Itô integrals Z_{μ,k}(f) satisfy the inequality

P (|Z_{μ,k}(f)| > u) > Ke^{−Au^{2/k}}  (13.21)

with some numbers K = K(f, μ) > 0 and A = A(f, μ) > 0.

The constant A in the exponent Au^{2/k} of formula (13.21) is always finite, but Mc. Kean’s proof yields no explicit upper bound on it. The following example shows that in certain cases if we fix the constant K in relation (13.21), then this inequality holds only with a very large constant A > 0 even if the variance of the Wiener–Itô integral equals 1.

Take a probability measure μ and a white noise μ_{W} with reference measure μ on a measurable space (X, X), and let φ_{1}, φ_{2}, … be a sequence of orthonormal functions on (X, X) with respect to this measure μ. Define for all L = 1, 2, …, the function

f(x_{1}, …, x_{k}) = f_{L}(x_{1}, …, x_{k}) = (k!)^{1/2}L^{−1/2} ∑_{j=1}^{L} φ_{j}(x_{1}) ⋯ φ_{j}(x_{k})  (13.22)

and the Wiener–Itô integral

Z_{μ,k}(f) = Z_{μ,k}(f_{L}) = \frac{1}{k!} ∫ f_{L}(x_{1}, …, x_{k})μ_{W} ( dx_{1}) … μ_{W} ( dx_{k}).

Then EZ_{μ,k}^{2}(f) = 1, and the high moments of Z_{μ,k}(f) can be well estimated. For a large parameter L these moments are much smaller, than the quantities suggested by Proposition 13.1. (The calculation leading to the estimation of the moments of Z_{μ,k}(f) will be omitted.) These moment estimates also imply that if the parameter L is large, then for not too large numbers u the probability P (|Z_{μ,k}(f)| > u) has a much better estimate than that given in Theorem 8.5. As a consequence, for a large number L and fixed number K relation (13.21) may hold only with a very big number A > 0.

We can expect that if we take a Gaussian random polynomial P (ξ_{1}, …, ξ_{n}) whose arguments are Gaussian random variables ξ_{1}, …, ξ_{n}, and which is the sum of many small almost independent terms, then a similar picture arises as in the case of a Wiener– Itô integral with kernel function (13.22) with a large parameter L. Such a random polynomial has an almost Gaussian distribution by the central limit theorem, and we can also expect that its not too high moments behave so as the corresponding moments of a Gaussian random variable with expectation zero and the same variance as the Gaussian random polynomial we consider. Such a bound on the moments has the consequence that the estimate on the probability (P (ξ_{1}, …, ξ_{n}) > u) given in Theorem 8.5 can be improved if the number u is not too large. A similar picture arises if we consider Wiener– Itô integrals whose kernel function satisfies some ‘almost independence’ properties. The problem is to find the right properties under which we can get a good estimate that exploits the almost independence property of a Gaussian random polynomial or of a Wiener–Itô integral. The main result of R. Latała’s paper [26] can be considered as a response to this question. I describe this result below.

To formulate Latała’s result some new notions have to be introduced. Given a finite set A let P(A) denote the set of all its partitions. If a partition P = {B_{1}, …, B_{s}} ∈ P(A) consists of s elements then we say that this partition has order s, and write |P | = s. In the special case A = {1, …, k} the notation P(A) = P_{k} will be used. Given a measurable space (X, X) with a probability measure μ on it together with a finite set B = {b_{1}, …, b_{j}} let us introduce the following notations. Take j different copies (X_{b_{r}}, X_{b_{r}}) and μ_{b_{r}}, 1 ≤ r ≤ j, of this measurable space and probability measure indexed by the elements of the set B, and define their product (X^{(B)}, X^{(B)}, μ^{(B)}) =

( ∏_{r=1}^{j} X_{b_{r}}, ∏_{r=1}^{j} X_{b_{r}}, ∏_{r=1}^{j} μ_{b_{r}}). The points (x_{b_{1}}, …, x_{b_{j}}) ∈ X^{(B)} will be denoted by x^{(B)} ∈

X^{(B)} in the sequel. With the help of the above notations I introduce the quantities needed in the formulation of the following Theorem 13.7.

Let a function f = f(x_{1}, …, x_{k}) be given on the k-fold product (X^{k}, X^{k}, μ^{k}) of a measurable space (X, X) with a probability measure μ. For all partitions P = {B_{1}, …, B_{s}} ∈ P_{k} of the set {1, …, k} consider the functions g_{r} (x^{(B_{r})}) on the space X^{(B_{r})}, 1 ≤ r ≤ s, and define with their help the quantities

α(P) = α(P, f, μ) = sup_{g_{1},...,g_{s}} ∫ f(x_{1}, …, x_{k})g_{1} (x^{(B_{1})}) ⋯ g_{s} (x^{(B_{s})}) μ(dx_{1}) … μ(dx_{k});
where supremum is taken for such functions g_{1}, …, g_{s}, g_{r}: X^{B_{r}} → R^{1}
for which ∫ g_{r}^{2} (x^{(B_{r})}) μ^{(B_{r})} ( dx^{(B_{r})}) ≤ 1 for all 1 ≤ r ≤ s,

(13.23) and put

α_{s} = max_{P ∈P_{k}, |P |=s} α(P), 1 ≤ s ≤ k.  (13.24)

In Latała’s estimation of Wiener–Itô integrals of order k the quantities α_{s}, 1 ≤ s ≤ k, play a similar role as the number σ^{2} in Theorem 8.5. Observe that in the case |P | = 1, i.e. if P = {1, …, k} the identity α^{2}(P) = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) holds, which means that α_{1} = σ. The following estimate is valid for Wiener–Itô integrals of general order.

Theorem 13.7. (Latała’s estimate about the tail-distribution of Wiener–Itô integrals). Let a k-fold Wiener–Itô integral Z_{μ,k}(f), k ≥ 1, be defined with the help of a white noise μ_{W} with a non-atomic reference measure μ and a kernel function f of k-variable such that ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) < ∞. There is some universal constant C(k) < ∞ depending only of the order k of the random integral such that the inequalities

E(Z_{μ,k}(f))^{2M} ≤ (C(k) max_{1≤s≤k}(M^{s/2}α_{s}))^{2M} ,  (13.25)

and

P (|Z_{μ,k}(f)| > u) ≤ C(k) exp \{−\frac{1}{C(k)} min_{1≤s≤k} (\frac{u}{α_{s}} )^{2/s}\}  (13.26)

hold for all M = 1, 2, … and u > 0 with the quantities α_{s}, defined in formulas (13.23) and (13.24).

Inequality (13.26) is a simple consequence of (13.25). In the special case when α_{s} ≤ M^{−(s−1)/2} for all 1 ≤ s ≤ k, then inequality (13.25) says that the moment EZ_{μ,k}(f)^{2M} has the same magnitude as the 2M-th moment of a standard Gaussian random variable multiplied by a constant, and it implies a good estimate on P (|Z_{μ,k}(f)| > u) given in (13.26). Actually the result of Theorem 13.7 can be reduced to the special case when α_{s} ≤ M^{−(s−1)/2} for all 1 ≤ s ≤ k. Thus it can be interpreted so that if the quantities α_{s} of a k-fold Wiener–Itô integral are sufficiently small, then these ‘almost independence’ conditions imply that the 2M-th moment of this integrals behaves like a one-fold Wiener–Itô integral with the same variance.

Actually Latała formulated his result in a different form, and he proved a slightly weaker result. He considered Gaussian polynomials of the following form:

P (ξ_{j}^{(s)}, 1 ≤ j ≤ n, 1 ≤ s ≤ k) = \frac{1}{k!} ∑_{(j_{1},...,j_{k}): 1≤j_{s}≤n, 1≤s≤k} a(j_{1}, …, j_{k})ξ_{j_{1}}^{(1)} ⋯ ξ_{j_{k}}^{(k)},

(13.27)

where ξ_{j}^{(s)}, 1 ≤ j ≤ n and 1 ≤ s ≤ k, are independent standard normal random variables. Latała gave an estimate about the moments and tail-distribution of such random polynomials.

The problem about the behaviour of such random polynomials can be reformulated as a problem about the behaviour of Wiener–Itô integrals in the following way: Take a measurable space (X, X) with a non-atomic measure μ on it. Let Z_{μ} be a white noise with reference measure μ, let us choose a set of orthogonal functions h_{j}^{(s)}(x), 1 ≤ j ≤ n, 1 ≤ s ≤ k, on the space (X, X) with respect to the measure μ, and define the function

f(x_{1}, …, x_{k}) = \frac{1}{k!} ∑_{(j_{1},...,j_{k}): 1≤j_{s}≤n, 1≤s≤k} a(j_{1}, …, j_{k})h_{j_{1}}^{(1)}(x_{1}) ⋯ h_{j_{k}}^{(k)}(x_{k}) (13.28)

together with the Wiener–Itô integral Z_{μ,k}(f). Since the random integrals ξ̄_{j}^{(s)} = ∫ h_{j}^{(s)}(x)Z_{μ}( dx), 1 ≤ j ≤ n, 1 ≤ s ≤ k, are independent, standard Gaussian random variables, it is not difficult to see with the help of Itô’s formula (Theorem 10.3 in this work) that the distributions of the random polynomial P (ξ_{j}^{(s)}, 1 ≤ j ≤ n, 1 ≤ s ≤ k) and Z_{μ,k}(f) agree. Here we reformulated Latała’s estimates about random polynomials of the form (13.27) to estimates about Wiener–Itô integrals with kernel function of the form (13.28).

These estimates are equivalent to Latała’s result if we restrict our attention to the special class of Wiener–Itô integrals with kernel functions of the form (13.28). But we have formulated our result for Wiener–Itô integrals with a general kernel function. Latała’s proof heavily exploits the special structure of the random polynomials given in (13.27), the independence of the random variables ξ_{j}^{(s)} for different parameters s in it. (It would be interesting to find a proof which does not exploit this property.) On the other hand, this result can be generalized to the case discussed in Theorem 13.7. This generalization can be proved by exploiting the theorem of de la Peña and Montgomery– Smith about the comparison of U-statistics and decoupled U-statistics (formulated in Theorem 14.3 of this work) and the properties of the Wiener–Itô integrals. I omit the details of the proof.

Latała also proved a converse estimate in [26] about random polynomials of Gaussian random polynomials which shows that the estimates of Theorem 13.7 are sharp. We formulate it in its original form, i.e. we restrict our attention to the case of Wiener–Itô integrals with kernel functions of the form (13.28).

Theorem 13.8. (A lower bound about the tail distribution of Wiener–Itô integrals). A random integral Z_{μ,k}(f) with a kernel function of the form (13.28) satisfies the inequalities

E(Z_{μ,k}(f))^{2M} ≥ (C(k) max_{1≤s≤k}(M^{s/2}α_{s}))^{2M} ,

and

P (|Z_{μ,k}(f)| > u) ≥ \frac{1}{C(k)} exp \{−C(k) min_{1≤s≤k} (\frac{u}{α_{s}} )^{2/s}\}

for all M = 1, 2, … and u > 0 with some universal constant C(k) > 0 depending only on the order k of the integral and the quantities α_{s}, defined in formula (13.23) and (13.24).

Let me finally remark that there is a counterpart of Theorem 13.7 about degenerate U-statistics. Adamczak’w paper [1] contains such a result. Here we do not discuss it, because this result is far from the main topic of this work. We only remark that some new quantities have to be introduced to formulate it. The appearance of these conditions is related to the fact that in an estimate about the tail-behaviour of a degenerate Ustatistic we need a bound not only on the L_{2}-norm but also on the supremum norm of the kernel function. In a sharp estimate the bound about the supremum of the kernel function has to be replaced by a more complex system of conditions, just as the condition about the L_{2}-norm of the kernel function was replaced by a condition about the quantities α_{s}, 1 ≤ s ≤ k, defined in formulas (13.23) and (13.24) in Theorem 13.7.

14. Reduction of the main result in this work.

The main result of this work is Theorem 8.4 or its multiple integral version Theorem 8.2. It was shown in Section 9 that Theorem 8.2 follows from Theorems 8.4. Hence it is enough to prove Theorem 8.4. It may be useful to study this problem together with its multiple Wiener–Itô integral version, Theorem 8.6.

Theorems 8.6 and 8.4 will be proved similarly to their one-variate versions, Theorems 4.2 and 4.1. Theorem 8.6 will be proved with the help of Theorem 8.5 about the estimation of the tail distribution of multiple Wiener–Itô integrals. A natural modification of the chaining argument applied in the proof of Theorem 4.2 works also in this case. No new difficulties arise. On the other hand, in the proof of Theorem 8.4 several new difficulties have to be overcome. I start with the proof of Theorem 8.6.

Proof of Theorem 8.6. Fix a number 0 < ε < 1, and let us list the elements of the countable set F as f_{1}, f_{2}, . . .. For all p = 0, 1, 2, … let us choose by exploiting the conditions of Theorem 8.6 a set F_{p} = {f_{a(1,p)}, …, f_{a(m_{p},p)}} ⊂ F of function with

m_{p} ≤ 2D 2^{(2p+4)L}ε^{−L}σ^{−L} elements in such a way that inf_{1≤j≤m_{p}} ∫ (f − f_{a(j,p)})^{2} dμ ≤

2^{−4p−8}ε^{2}σ^{2} for all f ∈ F, and beside this let f_{p} ∈ F_{p}. For all indices a(j, p), p = 1, 2, …, 1 ≤ j ≤ m_{p}, choose a predecessor a(j′, p−1), j′ = j′(j, p), 1 ≤ j′ ≤ m_{p−1}, in such a way that the functions f_{a(j,p)} and f_{a(j′,p−1)} satisfy the relation ∫ |f_{a(j,p)} − f_{a(j′,p−1)}|^{2} dμ ≤ ε^{2}σ^{2}2^{−4(p+1)}. Theorem 8.5 with the choice ū = ū(p) = 2^{−(p+1)}εu and σ̄ = σ̄(p) = 2^{−2p−2}εσ yields the estimates

\begin{aligned}P (A(j, p)) & = P (k!|Z_{μ,k}(f_{a(j,p)} − f_{a(j′,p−1)})| ≥ 2^{−(1+p)}εu) \\ & ≤ C exp \{−\frac{1}{2} (\frac{2^{p+1}u}{σ})^{2/k}\} ,  1 ≤ j ≤ m_{p}, \end{aligned}  (14.1)

for all p = 1, 2, …, and

\begin{aligned}P (B(s)) & = P (k!|Z_{μ,k}(f_{a(0,s)})| ≥ (1 − \frac{ε}{2}) u) ≤ C exp \{−\frac{1}{2} (\frac{(1 − \frac{ε}{2}) u}{σ})^{2/k}\} , \\ & 1 ≤ s ≤ m_{0}. \end{aligned}

(14.2)

Since all f ∈ F is the element of at least one set F_{p}, p = 0, 1, 2, …, (f_{p} ∈ F_{p}), the definition of the predecessor of an index a(j, p) and of the events A(j, p) and B(s) in formulas (14.1) and (14.2) together with the previous estimates imply that

P (sup_{f∈F} k!|Z_{μ,k}(f)| ≥ u) ≤ P ( ⋃_{p=1}^{∞} ⋃_{j=1}^{m_{p}} A(j, p) ∪ ⋃_{s=1}^{m_{0}} B(s))

≤ ∑_{p=1}^{∞} ∑_{j=1}^{m_{p}} P (A(j, p)) + ∑_{s=1}^{m_{0}} P (B(s))  (14.3)

≤ ∑_{p=1}^{∞} 2CD2^{(2p+4)L}ε^{−L}σ^{−L} exp \{−\frac{1}{2} (\frac{2^{p+1}u}{σ})^{2/k}\}

+ 2^{1+4L}CDε^{−L}σ^{−L} exp \{−\frac{1}{2} (\frac{(1 − \frac{ε}{2}) u}{σ})^{2/k}\} .

Standard calculation shows that if u ≥ ML^{k/2} \frac{1}{ε} log^{k/2} \frac{2}{ε}σ log^{k/2} \frac{2}{σ} with a sufficiently large constant M, then the inequalities

2^{(2p+4)L}ε^{−L}σ^{−L} exp \{−\frac{1}{2} (\frac{2^{p+1}u}{σ})^{2/k}\} ≤ 2^{−p} \{−\frac{1}{2} (\frac{(1 − ε)u}{σ})^{2/k}\}

hold for all p = 1, 2 …, and

2^{4L}ε^{−L}σ^{−L} exp \{−\frac{1}{2} (\frac{(1 − \frac{ε}{2}) u}{σ})^{2/k}\} ≤ exp \{−\frac{1}{2} (\frac{(1 − ε) u}{σ})^{2/k}\} .

These inequalities together with relation (14.3) imply relation (8.15). Theorem 8.6 is proved.

The proof of Theorem 8.4 is harder. In this case the chaining argument in itself does not supply the proof, since Theorem 8.3 gives a good estimate about the distribution of a degenerate U-statistic only if it has a not too small variance. The same difficulty appeared in the proof of Theorem 4.1, and the method applied in that case will be adapted to the present situation.

A multivariate version of Proposition 6.1 will be proved in Proposition 14.1, and another result which can be considered as a multidimensional version of Proposition 6.2 will be formulated in Proposition 14.2. It will be shown that Theorem 8.4 follows from Propositions 14.1 and 14.2. Most steps of these proofs can be considered as a simple repetition of the corresponding arguments in the proof of the results in Section 6. Nevertheless, I wrote them down for the sake of completeness.

The result formulated in Proposition 14.1 can be proved in almost the same way as its one-variate version, Proposition 6.1. The only essential difference is that now we have to apply a multivariate version of the Bernstein’s inequality. In the calculations of the proof of Proposition 14.1 the term (\frac{u}{σ})^{2/k} shows a behaviour similar to the term (\frac{u}{σ})^{2} in Proposition 6.1. Theorem 14.1 contains the information we can get by applying Theorem 8.3 together with the chaining argument. Its main content, inequality (14.4), yields a good estimate on the supremum of degenerated U-statistics if it is taken for an appropriate finite subclass Fσ̄ of the original class of kernel functions F. The class of kernel functions Fσ̄ is a relatively dense subclass of F in the L_{2} norm. Proposition 14.1 also provides some useful estimates on the value of the parameter σ̄ which describes how dense the class of functions Fσ̄ is in F.

Proposition 14.1. Let the k-fold power (X^{k}, X^{k}) of a measurable space (X, X) be given together with some probability measure μ on (X, X) and such a countable L_{2}-dense class F of functions f(x_{1}, …, x_{k}) of k variables on (X^{k}, X^{k}) with some exponent L ≥ 1 and parameter D ≥ 1 with the following properties. All functions f ∈ F are canonical with respect to the measure μ, and they satisfy conditions (8.4) and (8.5) with some real number 0 < σ ≤ 1. Take a sequence of independent μ-distributed random variables ξ_{1}, …, ξ_{n}, n ≥ max(k, 2), and consider the (degenerate) U-statistics I_{n,k}(f), f ∈ F, defined in formula (8.7), and fix some number Ā = Ā_{k} ≥ 2^{k}.

There is a number M = M(Ā, k) such that for all numbers u > 0 for which the inequality nσ^{2} ≥ (\frac{u}{σ})^{2/k} ≥ M(L log \frac{2}{σ} + log D) holds, a number σ̄ = σ̄(u), 0 ≤ σ̄ ≤ σ ≤ 1, and a collection of functions Fσ̄ = {f_{1}, …, f_{m}} ⊂ F with m ≤ Dσ̄^{−L} elements can be chosen in such a way that the sets D_{j} = {f: f ∈ F, ∫ |f − f_{j}|^{2} dμ ≤ σ̄^{2}}, 1 ≤ j ≤ m,

satisfy the relation F = ⋃_{j=1}^{m} D_{j}, and for the (degenerate) U-statistics I_{n,k}(f), f ∈ F_{σ̄(u)}

the inequality

P ( sup_{f∈F_{σ̄(u)}} n^{−k/2}|I_{n,k}(f)| ≥ \frac{u}{Ā}) ≤ 2C exp \{−α (\frac{u}{10Āσ} )^{2/k}\}  (14.4)

if nσ^{2} ≥ (\frac{u}{σ} )^{2/k} ≥ M(L log \frac{2}{σ} + log D)

holds with the constants α = α(k), C = C(k) appearing in formula (8.10′) of the Corollary of Theorem 8.3 and the exponent L and parameter D of the L_{2}-dense class F. Beside this also the inequality 4 (\frac{u}{Āσ̄} )^{2/k} ≥ nσ̄^{2} ≥ \frac{1}{64} (\frac{u}{Āσ} )^{2/k} holds for this number σ̄ = σ̄(u). If the number u satisfies also the inequality

nσ^{2} ≥ (\frac{u}{σ} )^{2/k} ≥ M(L^{3/2} log \frac{2}{σ} + (log D)^{3/2})  (14.5)

with a sufficiently large number M = M(Ā, k), then the relation nσ̄^{2} ≥ L log n + log D holds, too.

Proof of Proposition 14.1. Let us list the elements of the countable set F as f_{1}, f_{2}, . . .. For all p = 0, 1, 2, … let us choose, by exploiting the L_{2}-density property of the class F, a set F_{p} = {f_{a(1,p)}, …, f_{a(m_{p},p)}} ⊂ F with m_{p} ≤ D 2^{2pL}σ^{−L} elements in such a way that inf_{1≤j≤m_{p}} ∫ (f − f_{a(j,p)})^{2} dμ ≤ 2^{−4p}σ^{2} for all f ∈ F. For all indices

a(j, p), p = 1, 2, …, 1 ≤ j ≤ m_{p}, choose a predecessor a(j′, p − 1), j′ = j′(j, p), 1 ≤ j′ ≤ m_{p−1}, in such a way that the functions f_{a(j,p)} and f_{a(j′,p−1)} satisfy the relation

∫ |f_{a(j,p)}−f_{a(j′,p−1)}|^{2} dμ ≤ σ^{2}2^{−4(p−1)}. Then the inequalities ∫ (\frac{f_{a(j,p)}−f_{a(j′,p−1)}}{2} )^{2} dμ ≤

4σ^{2}2^{−4p} and sup_{x_{j}∈X, 1≤j≤k} |\frac{f_{a(j,p)}(x_{1},...,x_{k})−f_{a(j′,p−1)}(x_{1},...,x_{k})}{2} | ≤ 1 hold. The Corollary of

Theorem 8.3 yields that

\begin{aligned}P (A(j, p)) & = P (n^{−k/2}|I_{n,k}(f_{a(j,p)} − f_{a(j′,p−1)})| ≥ \frac{2^{−(1+p)}u}{Ā}) \\ & ≤ C exp \{−α (\frac{2^{p}u}{8Āσ} )^{2/k}\} if 4nσ^{2}2^{−4p} ≥ (\frac{2^{p}u}{8Āσ} )^{2/k} , \\ & 1 ≤ j ≤ m_{p}, p = 1, 2, …, \end{aligned}  (14.6)

and

P (B(s)) = P (n^{−k/2}|I_{n,k}(f_{0,s})| ≥ \frac{u}{2Ā}) ≤ C exp \{−α (\frac{u}{2Āσ} )^{2/k}\} , 1 ≤ s ≤ m_{0},
if nσ^{2} ≥ (\frac{u}{2Āσ} )^{2/k} .  (14.7)

Introduce an integer R = R(u), R > 0, which satisfies the relations

2^{(4+2/k)(R+1)} (\frac{u}{Āσ} )^{2/k} ≥ 2^{2+6/k}nσ^{2} ≥ 2^{(4+2/k)R} (\frac{u}{Āσ} )^{2/k} ,

and define σ̄^{2} = 2^{−4R}σ^{2} and Fσ̄ = F_{R} (this is the class of functions F_{p} introduced at the start of the proof with p = R). We defined the number R, analogously to the proof of Theorem 6.1, as the largest number p for which the condition formulated in (14.6) holds. As nσ^{2} ≥ (\frac{u}{σ})^{2/k}, and Ā ≥ 2^{k} by our conditions, there exists such a positive integer R.) The cardinality m of the set Fσ̄ is clearly not greater than Dσ̄^{−L}, and

⋃_{j=1}^{m} D_{j} = F. Beside this, the number R was chosen in such a way that the inequalities

(14.6) and (14.7) hold for 1 ≤ p ≤ R. Hence the definition of the predecessor of an index a(j, p) implies that

P ( sup_{f∈Fσ̄} n^{−k/2}|I_{n,k}(f)| ≥ \frac{u}{Ā}) ≤ P ( ⋃_{p=1}^{R} ⋃_{j=1}^{m_{p}} A(j, p) ∪ ⋃_{s=1}^{m_{0}} B(s))

≤ ∑_{p=1}^{R} ∑_{j=1}^{m_{p}} P (A(j, p)) + ∑_{s=1}^{m_{0}} P (B(s)) ≤ ∑_{p=1}^{∞} CD 2^{2pL}σ^{−L} exp \{−α (\frac{2^{p}u}{8Āσ} )^{2/k}\}

+ CDσ^{−L} exp \{−α (\frac{u}{2Āσ} )^{2/k}\} .

If the condition (\frac{u}{σ})^{2/k} ≥ M(L log \frac{2}{σ} + log D) holds with a sufficiently large constant M (depending on Ā), then the inequalities

D2^{2pL}σ^{−L} exp \{−α (\frac{2^{p}u}{8Āσ} )^{2/k}\} ≤ 2^{−p} exp \{−α (\frac{2^{p}u}{10Āσ} )^{2/k}\}

hold for all p = 1, 2, …, and

Dσ^{−L} exp \{−α (\frac{u}{2Āσ} )^{2/k}\} ≤ exp \{−α (\frac{u}{10Āσ} )^{2/k}\} .

Hence the previous estimate implies that

P ( sup_{f∈Fσ̄} n^{−k/2}|I_{n,k}(f)| ≥ \frac{u}{Ā}) ≤ ∑_{p=1}^{∞} C2^{−p} exp \{−α (\frac{2^{p}u}{10Āσ} )^{2/k}\}

+ C exp \{−α (\frac{u}{10Āσ} )^{2/k}\} ≤ 2C exp \{−α (\frac{u}{10Āσ} )^{2/k}\} ,

and relation (14.4) holds.

The estimates

\frac{1}{64} (\frac{u}{Āσ} )^{2/k} ≤ 2^{−2−6/k}2^{2k/R} (\frac{u}{Āσ} )^{2/k} = 2^{−4R} · 2^{(4+2/k)R−2−6/k} (\frac{u}{Āσ} )^{2/k}

≤ nσ̄^{2} = 2^{−4R}nσ^{2} ≤ 2^{−4R} · 2^{(4+2/k)(R+1)−2−6/k} (\frac{u}{Āσ} )^{2/k}

= 2^{2−4/k} · 2^{2R/k} (\frac{u}{Āσ} )^{2/k} = 2^{2−4/k} · 2^{−2R/k} (\frac{u}{Āσ̄} )^{2/k} ≤ 4 (\frac{u}{Āσ̄} )^{2/k}

hold because of the relation R ≥ 1. This means that nσ̄^{2} has the upper and lower bound formulated in Proposition 14.1. It remained to show that nσ̄^{2} ≥ L log n + D if relation (14.5) holds.

This inequality clearly holds under the conditions of Proposition 14.1 if σ ≤ n^{−1/3}, since in this case log \frac{2}{σ} ≥ \frac{log n}{3}, and nσ̄^{2} ≥ \frac{1}{64} (\frac{u}{Āσ} )^{2/k} ≥ \frac{1}{64} Ā^{−2/k}M(L^{3/2} log \frac{2}{σ} + (log D)^{3/2})^{3/2} ≥ \frac{1}{192} Ā^{−2/k}M(L^{3/2} log n + (log D)^{3/2}) ≥ L log n + log D if M = M(Ā, k) is sufficiently large.

If σ ≥ n^{−1/3}, then the inequality 2^{(4+2/k)R} (\frac{u}{Āσ} )^{2/k} ≤ 2^{2+6/k}nσ^{2} can be applied.

This implies that 2^{−4R} ≥ 2^{−4(2+6/k))/(4+2/k)} [\frac{(\frac{u}{Āσ} )2/k}{nσ^{2}} ]^{4/(4+2/k)}, and

nσ̄^{2} = 2^{−4R}nσ^{2} ≥ \frac{2^{−16/3}}{Ā^{4/3}} (nσ^{2})^{1−γ} [(\frac{u}{σ} )^{2/k}]^{γ} with γ = \frac{4}{4 + \frac{2}{k}} ≥ \frac{2}{3}.

The inequalities nσ^{2} ≥ n^{1/3} and nσ^{2} ≥ (\frac{u}{σ})^{2/k} ≥ M(L^{3/2} log \frac{2}{σ} +(log D)^{3/2}) ≥ \frac{M}{2} (L^{3/2}+ (log D)^{3/2}) hold, (since log \frac{2}{σ} ≥ \frac{1}{2}). They yield that for sufficiently large M = M(Ā, k)

(nσ^{2})^{1−γ} [(\frac{u}{σ})^{2/k}]^{γ} ≥ (nσ^{2})^{1−γ} [(\frac{u}{σ})^{2/k}]^{2/3} = (nσ^{2})^{1/(2k+1)} [(\frac{u}{σ})^{2/k}]^{2/3}, and

nσ̄^{2} ≥ \frac{Ā^{−4/3}}{50} (nσ^{2})^{1/(2k+1)} [(\frac{u}{σ} )^{2/k}]^{2/3}

≥ \frac{Ā^{−4/3}}{50} n^{1/3(2k+1)} (\frac{M}{2} )^{2/3} (L^{3/2} + (log D)^{3/2})^{2/3} ≥ L log n + log D.

A multivariate analog of Proposition 6.2 is formulated in Proposition 14.2, and it will be shown that Propositions 14.1 and 14.2 imply Theorem 8.4.

Proposition 14.2. Let a probability measure μ be given on a measurable space (X, X) together with a sequence of independent and μ distributed random variables ξ_{1}, …, ξ_{n} and a countable L_{2}-dense class F of canonical (with respect to the measure μ) kernel functions f = f(x_{1}, …, x_{k}) with some parameter D ≥ 1 and exponent L ≥ 1 on the product space (X^{k}, X^{k}). Let all functions f ∈ F satisfy conditions (8.1) and (8.2) with some 0 < σ ≤ 1 such that nσ^{2} > L log n + D. Let us consider the (degenerate) U-statistics I_{n,k}(f) with the random sequence ξ_{1}, …, ξ_{n}, n ≥ max(2, k), and kernel functions f ∈ F. There exists a threshold index A_{0} = A_{0}(k) > 0 and two numbers C̄ = C̄(k) > 0 and γ = γ(k) > 0 depending only on the order k of the U-statistics such that the degenerate U-statistics I_{n,k}(f), f ∈ F, satisfy the inequality

P (sup_{f∈F} |n^{−k/2}I_{n,k}(f)| ≥ An^{k/2}σ^{k+1}) ≤ C̄e^{−γA^{1/2k}nσ^{2}} if A ≥ A_{0}.  (14.8)

Proposition 14.2 yields an estimate for the tail distribution of the supremum of degenerate U-statistics at level u ≥ A_{0}n^{k/2}σ^{k+1}, i.e. in the case when Theorem 8.3 does not give a good estimate on the tail-distribution of the single degenerate U-statistics taking part in the supremum at the left-hand side of (14.8).

Formula (8.11) will be proved by means of Proposition 14.1 with an appropriate choice of the parameter Ā in it and Proposition 14.2 with the choice σ = σ̄ = σ̄(u) and the classes of functions F_{j} = \{\frac{g−f_{j}}{2} : g ∈ D_{j}\} with the number σ̄, functions f_{j} and sets of functions D_{j}, 1 ≤ j ≤ m, introduced in Proposition 14.1. Clearly,

P (sup_{f∈F} n^{−k/2}|I_{n,k}(f)| ≥ u) ≤ P ( sup_{f∈Fσ̄} n^{−k/2}|I_{n,k}(f)| ≥ \frac{u}{Ā})

+ ∑_{j=1}^{m} P ( sup_{g∈D_{j}} n^{−k/2} |I_{n,k} (\frac{f_{j} − g}{2} )| ≥ (\frac{1}{2} − \frac{1}{2Ā}) u) ,

(14.9) where m is the cardinality of the set of functions Fσ̄ appearing in Proposition 14.1. We shall estimate the two terms of the sum at the right-hand side of (14.9) by means of Propositions 14.1 and 14.2 with a good choice of the parameters Ā and the corresponding M = M(Ā) in Proposition 14.1, and the parameter Ā_{0} = A ≥ A_{0} in Proposition 14.2.

We shall choose the parameter Ā_{0} ≥ A_{0} so that it satisfies also the relation γĀ_{0}^{1/2k} ≥ 2 with the number γ appearing in Proposition 14.2, hence we put Ā_{0} = max(A_{0}, (\frac{2}{γ})^{2k}). We want to define the parameter Ā in such a way that the numbers u satisfying the conditions of Proposition 14.1 also satisfy the relation (\frac{1}{2} − \frac{1}{2Ā})u ≥ Ā_{0}n^{k/2}σ̄^{k+1} with the already fixed number Ā_{0}. This inequality can be rewritten in the form Ā_{0}^{−2/k}(\frac{1}{2} − \frac{1}{2Ā})^{2/k}(\frac{u}{σ̄})^{2/k} ≥ nσ̄^{2}. On the other hand, we have under the conditions of Proposition 14.1 4(\frac{u}{Āσ̄})^{2/k} ≥ nσ̄^{2}. Hence the desired inequality holds if Ā_{0}^{−2/k}(\frac{1}{2} − \frac{1}{2Ā})^{2/k} ≥ 4Ā^{−2/k}, thus it holds with the choice Ā = 2^{k+1}Ā_{0} + 1.

With such a choice of Ā (together with the corresponding M = M(Ā, k)) and Ā_{0} we can write

P ( sup_{g∈D_{j}} n^{−k/2} |I_{n,k} (\frac{f_{j} − g}{2} )| ≥ (\frac{1}{2} − \frac{1}{2Ā}) u)

≤ P ( sup_{g∈D_{j}} n^{−k/2} |I_{n,k} (\frac{f_{j} − g}{2} )| ≥ Ā_{0}n^{k/2}σ̄^{k+1}) ≤ C̄e^{−γĀ_{0}^{1/2k}nσ̄^{2}}

for all 1 ≤ j ≤ m. (Observe that the set of functions \frac{f_{j}−g}{2} , g ∈ D_{j}, is an L_{2}-dense class with parameter D and exponent L.) Hence Proposition 14.1 (relation (14.4) together with the inequality m ≤ Dσ̄^{−L}) and formula (14.8) with A = Ā_{0} and relation (14.9) imply that

P (sup_{f∈F} n^{−k/2}|I_{n,k}(f)| ≥ u) ≤ 2C exp \{−α (\frac{u}{10Āσ} )^{2/k}\} + C̄Dσ̄^{−L}e^{−γĀ_{0}^{1/2k}nσ̄^{2}}.

(14.10)

We show by repeating the corresponding argument in Section 6 that Dσ̄^{−L} ≤ e^{nσ̄^{2}}. Indeed, we have to show that log D + L log \frac{1}{σ̄} ≤ nσ̄^{2}. But, as we have seen, the relation nσ̄^{2} ≥ L log n + log D with L ≥ 1 and D ≥ 1 implies that nσ̄^{2} ≥ log n, hence log \frac{1}{σ} ≤ log n, and log D + L log \frac{1}{σ̄} ≤ log D + L log n ≤ nσ̄^{2}. On the other hand, γĀ_{0}^{1/2k} ≥ 2 by the definition of the number Ā_{0}, and by the estimates of Proposition 14.1 nσ̄^{2} ≥ \frac{1}{64} (\frac{u}{Āσ} )^{2/k}. The above relations imply that Dσ̄^{−L}e^{−γĀ_{0}^{1/2k}nσ̄^{2}} ≤ e^{−γĀ_{0}^{1/2k}nσ̄^{2}/2} ≤

exp \{−\frac{γ}{128} Ā_{0}^{1/2k}Ā^{−2/k} (\frac{u}{σ})^{2/k}\}. Hence relation (14.10) yields that

P (sup_{f∈F} n^{−k/2}|I_{n,k}(f)| ≥ u)

≤ 2C exp \{−\frac{α}{(10Ā)^{2}} (\frac{u}{σ} )^{2/k}\} + C̄ exp \{−\frac{γ}{128}Ā_{0}^{1/2k}Ā^{−2/k} (\frac{u}{σ} )^{2/k}\} ,

and this estimate implies Theorem 8.4.

To complete the proof of Theorem 8.4 we have to prove Proposition 14.2. It will be proved, similarly to its one-variate version Proposition 6.2, by means of a symmetrization argument. We want to find its right formulation. It would be natural to formulate it as a result about the supremum of degenerate U-statistics. However, we shall choose a slightly different approach. There is a notion, called decoupled U-statistic. Decoupled U-statistics behave similarly to U-statistics, but it is simpler to work with them, because they have more independence properties. It turned out to be useful to introduce this notion and to apply a result of de la Peña and Montgomery–Smith which enables us to reduce the estimation of U-statistics to the estimation of decoupled U-statistics, and to work out the symmetrization argument for decoupled U-statistics.

Next we introduce the notion of decoupled U-statistics together with their randomized version. We also formulate a result of de la Peña and Montgomery–Smith in Theorem 14.3 which enables us to reduce Proposition 14.2 to a version of it, presented in Proposition 14.2′. It states a result similar to Proposition 14.2 about decoupled U-statistics. The proof of Proposition 14.2′ is the hardest part of the problem. In Sections 15, 16 and 17 we deal essentially with this problem. The result of de la Peña and Montgomery–Smith will be proved in Appendix D.

Now we introduce the following notions.

The definition of decoupled and randomized decoupled U-statistics. Let us have k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence ξ_{1}, …, ξ_{n} of independent and identically distributed random variables taking their values in a measurable space (X, X) together with a measurable function f(x_{1}, …, x_{k}) on the product space (X^{k}, X^{k}) with values in a separable Banach space. The decoupled U-statistic Ī_{n,k}(f) determined by the random sequences ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, and kernel function f is defined by the formula

Ī_{n,k}(f) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f (ξ_{l_{1}}^{(1)}, …, ξ_{l_{k}}^{(k)}) .  (14.11)

Let us have beside the sequences ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, and function f(x_{1}, …, x_{k}) a sequence of independent random variables ε = (ε_{1}, …, ε_{n}), P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is independent also of the sequences of random variables ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k. The randomized decoupled U-statistic Ī_{n,k}(f, ε) (depending on the random sequences ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, the kernel function f and the randomizing sequence ε_{1}, …, ε_{n}) is defined by the formula

Ī_{n,k}^{ε}(f) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ε_{l_{1}} ⋯ ε_{l_{k}}f (ξ_{l_{1}}^{(1)}, …, ξ_{l_{k}}^{(k)}) .  (14.12)

A decoupled or randomized decoupled U-statistics (with real valued kernel function) will be called degenerate if its kernel function is canonical. This terminology is in full accordance with the definition of (usual) degenerate U-statistics.

A result of de la Peña and Montgomery–Smith will be formulated below. It gives an upper bound for the tail distribution of a U-statistic by means of the tail distribution of an appropriate decoupled U-statistic. It also has a generalization, where the supremum of U-statistics is bounded by the supremum of decoupled U-statistics. It enables us to reduce Proposition 14.2 to a version formulated Proposition 14.2′, which gives a bound on the tail distribution of the supremum of decoupled U-statistics. It is simpler to prove this result than the original one.

Before the formulation of the theorem of de la Peña and Montgomery–Smith I make some remark about it. It considers more general U-statistics with kernel functions taking values in a separable Banach space, and it compares the norm of Banach space valued Ustatistics and decoupled U-statistics. (Decoupled U-statistics were defined with general Banach space valued kernel functions, and the definition of U-statistics can also be generalized to separable Banach space valued kernel functions in a natural way.) This result was formulated in such a general form for a special reason. This helped to derive formula (14.14) of the subsequent theorem from formula (14.13). It can be exploited in the proof of formula (14.14) that the constants in the estimate (14.13) do not depend on the Banach space, where the kernel function f takes its values.

Theorem 14.3. (Theorem of de la Peña and Montgomery–Smith about the comparison of U-statistics and decoupled U-statistics). Let us consider a sequence of independent and identically distributed random variables ξ_{1}, …, ξ_{n} with values in a measurable space (X, X) together with k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of this sequence. Let us also have a function f(x_{1}, …, x_{k}) on the k-fold product space (X^{k}, X^{k}) which takes its values in a separable Banach space B. Let us take the Ustatistic and decoupled U-statistic I_{n,k}(f) and Ī_{n,k}(f) with the help of the above random sequences ξ_{1}, …, ξ_{n}, ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, and kernel function f. There exist some constants C̄ = C̄(k) > 0 and γ = γ(k) > 0 depending only on the order k of the U-statistic such that

P (∥I_{n,k}(f)∥ > u) ≤ C̄P (∥Ī_{n,k}(f)∥ > γu)  (14.13)

for all u > 0. Here ∥ · ∥ denotes the norm in the Banach space B where the function f takes its values.

More generally, if we have a countable sequence of functions f_{s}, s = 1, 2, …, taking their values in the same separable Banach-space, then

P ( sup_{1≤s<∞} ∥I_{n,k}(f_{s})∥ > u) ≤ C̄P ( sup_{1≤s<∞} ∥Ī_{n,k}(f_{s})∥ > γu) .  (14.14)

Now I formulate the following version of Proposition 4.2.

Proposition 14.2′. Let a probability measure μ be given on a measurable space (X, X) together with a sequence of independent and μ distributed random variables ξ_{1}, …, ξ_{n}, n ≥ max(k, 2), and a countable L_{2}-dense class F of canonical (with respect to the measure μ) kernel functions f = f(x_{1}, …, x_{k}) with some parameter D ≥ 1 and exponent L ≥ 1 on the product space (X^{k}, X^{k}). Let all functions f ∈ F satisfy conditions (8.1) and (8.2) with some 0 < σ ≤ 1 such that nσ^{2} > L log n + log D. Let us take k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of the random sequence ξ_{1}, …, ξ_{n}, and consider the decoupled U-statistics Ī_{n,k}(f), f ∈ F, defined with their help in formula (14.11).

There exists a threshold index A_{0} = A_{0}(k) > 0 depending only on the order k of the decoupled U-statistics I_{n,k}(f), f ∈ F, such that the (degenerate) decoupled U-statistics Ī_{n,k}(f), f ∈ F, satisfy the following version of inequality (14.8):

P (sup_{f∈F} n^{−k/2}|Ī_{n,k}(f)| ≥ An^{k/2}σ^{k+1}) ≤ e^{−2^{−(1/2+1/2k)}A^{1/2k}nσ^{2}} if A ≥ A_{0}. (14.15)

It is clear that Proposition 14.2′ and Theorem 14.3, more explicitly formula (14.14) in it, imply Proposition 14.2. Hence the proof of Theorem 8.4 was reduced to Proposition 14.2′ in this section. The proof of Proposition 14.2′ is based on a symmetrization argument. Its main ideas will be explained in the next section.

15. The strategy of the proof for the main result of this work.

In the previous section the proof of Theorem 8.4 was reduced to that of Proposition 14.2′. Proposition 14.2′ is a multivariate version of Proposition 6.2, and its proof is based on similar ideas. An important step in the proof of Theorem 6.2 was a symmetrization

argument in which we reduced the estimation of the probability P (sup_{f∈F} ∑_{j=1}^{n} f(ξ_{j}) > u)

to that of the probability P (sup_{f∈F} ∑_{j=1}^{n} ε_{j}f(ξ_{j}) > \frac{u}{3}), where ξ_{1}, …, ξ_{n} is a sequence of

independent and identically distributed random variables, and ε_{j}, 1 ≤ j ≤ n, is a sequence of independent random variables with distribution P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, independent of the sequence ξ_{j}. Let us understand how to formulate the corresponding symmetrization argument in the proof of Proposition 14.2′ and how to prove it.

The symmetrization argument applied in the proof of Proposition 6.2 was based on two observations. We took a copy ξ′_{1}, …, ξ′_{n} of the sequence ξ_{1}, …, ξ_{n}, i.e. a sequence of independent random variables which is independent also of the original sequence ξ_{1}, …, ξ_{n}, and has the same distribution. In the first step we compared the tail distri-

bution of the expression sup_{f∈F} ∑_{j=1}^{n}[f(ξ_{j})−f(ξ′_{j})] with that of sup_{f∈F} ∑_{j=1}^{n} f(ξ_{j}). This was done

with the help of Lemma 7.1. In the second step, in Lemma 7.2, we proved a ‘randomiza-

tion argument’ which stated that the distribution of the random fields ∑_{j=1}^{n}[f(ξ_{j})−f(ξ′_{j})] and ∑_{j=1}^{n} ε_{j}[f(ξ_{j}) − f(ξ′_{j})], f ∈ F, agree. The symmetrization argument was proved with

the help of these two observations.

In the proof of Proposition 14.2′ we would like to reduce the estimation of the tail distribution of the supremum of decoupled U-statistics sup_{f∈F} Ī_{n,k}(f) defined in for-

mula (14.11) to the estimation of the tail distribution of the supremum of randomized decoupled U-statistics sup_{f∈F} Ī_{n,k}^{ε}(f) defined in formula (14.12) by means of a similar ar-

gument. To do this first we have to understand what kind of random fields should be

introduced instead of ∑_{j=1}^{n}[f(ξ_{j}) − f(ξ′_{j})], f ∈ F, in the new case. In formula (15.1) we

shall define such a random field. Its definition reminds a bit to the definition of Stieltjes measures. In Lemma 15.1 we will show that a version of the ‘randomization argument’ of Lemma 7.2 can be applied when we working with this random field.

The adaptation of the first step of the symmetrization argument in the proof of Proposition 6.2 to the present case is much harder. The proof of Proposition 6.2 was based on the symmetrization lemma, Lemma 7.1, which does not work in the present case. Hence we shall prove a generalization of this result in Lemma 15.2. The proof is difficult even with the help of this result. The hardest part of the problem appears at this point. I shall write about it after the formulation of Lemma 15.2.

To formulate Lemma 15.1 needed in our proof we introduce some notations.

Let V_{k} denote the set of all sequences (v(1), …, v(k)) of length k such that v(j) = +1 or v(j) = −1 for all 1 ≤ j ≤ k. Let m(v), v = (v(1), …, v(k)) ∈ V_{k}, denote the number of digits −1 in the sequence v. Let a (real valued) function f(x_{1}, …, x_{k}) of k variables be given on a measurable space (X, X) together with a sequence of independent and identically distributed random variables ξ_{1}, …, ξ_{n} with values in the space (X, X) and 2k independent copies ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)}, …, ξ_{n}^{(j,−1)}, 1 ≤ j ≤ k, of this sequence. Let us have beside them another sequence ε = (ε_{1}, …, ε_{n}), P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, of independent random variables, also independent of all previously introduced random variables. With the help of the above quantities we introduce the random variables

Ĩ_{n,k}(f) = \frac{1}{k!} ∑_{v∈V_{k}}(−1)^{m(v)} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{r}≤n, r=1,...,k, \\ l_{r}≠l_{r′} if r≠r′ \end{matrix}} f (ξ_{l_{1}}^{(1,v(1))}, …, ξ_{l_{k}}^{(k,v(k))}) (15.1)

and

Ĩ_{n,k}^{ε}(f) = \frac{1}{k!} ∑_{v∈V_{k}}(−1)^{m(v)} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{r}≤n, r=1,...,k, \\ l_{r}≠l_{r′} if r≠r′ \end{matrix}} ε_{l_{1}} ⋯ ε_{l_{k}}f (ξ_{l_{1}}^{(1,v(1))}, …, ξ_{l_{k}}^{(k,v(k))})

(15.2)

The number m(v) in the above formulas denotes the number of the digits −1 in the ±1 sequence v of length k, hence it counts how many random variables ξ_{l_{j}}^{(j,1)}, 1 ≤ j ≤ k, were replaced by the ‘secondary copy’ ξ_{l_{j}}^{(j,−1)} for a v ∈ V_{k} in the inner sum in formulas (15.1) or (15.2).

The following result holds.

Lemma 15.1. Let us consider a (non-empty) class of functions F of k variables f(x_{1}, …, x_{k}) on the space (X^{k}, X^{k}) together with the random variables Ĩ_{n,k}(f) and Ĩ_{n,k}^{ε}(f) defined in formulas (15.1) and (15.2) for all f ∈ F. The distributions of the random fields Ĩ_{n,k}(f), f ∈ F, and Ĩ_{n,k}^{ε}(f), f ∈ F, agree.

Let me recall that we say that the distribution of two random fields X(f), f ∈ F, and Y (f), f ∈ F, agree if for any finite sets {f_{1}, …, f_{p}} ∈ F the distribution of the random vectors (X(f_{1}), …, X(f_{p})) and (Y (f_{1}), …, Y (f_{p})) agree.

Proof of Lemma 15.1. I even claim that for any fixed sequence u = (u(1), …, u(n)), u(l) = ±1, 1 ≤ l ≤ n, of length n, the conditional distribution of the field Ĩ_{n,k}^{ε}(f), f ∈ F, under the condition (ε_{1}, …, ε_{n}) = u = (u(1), …, u(n)) agrees with the distribution of the field of Ĩ_{n,k}(f), f ∈ F.

Indeed, the random variables Ĩ_{n,k}(f), f ∈ F, defined in (15.1) are functions of a random vector with coordinates (ξ_{l}^{(j)}, ξ̄_{l}^{(j)}) = (ξ_{l}^{(j,1)}, ξ_{l}^{(j,−1)}), 1 ≤ l ≤ n, 1 ≤ j ≤ k, and the distribution of this random vector does not change if the coordinates (ξ_{l}^{(j)}, ξ̄_{l}^{(j)}) = (ξ_{l}^{(j,1)}, ξ_{l}^{(j,−1)}) with such indices (l, j) for which u(l) = −1 (and the index j is arbitrary) are replaced by (ξ̄_{l}^{(j)}, ξ_{l}^{(j)}) = (ξ_{l}^{(j,−1)}, ξ_{l}^{(j,1)}), and the coordinates (ξ_{l}^{(j)}, ξ̄_{l}^{(j)}) with such indices (l, j) for which u(l) = 1 are not changed. As a consequence, the distribution of the random field Ĩ_{n,k}(f|u), f ∈ F, we get by replacing the original vector (ξ_{l}^{(j)}, ξ̄_{l}^{(j)}), 1 ≤ l ≤ n, 1 ≤ j ≤ k, in the definition of the expression Ĩ_{n,k}(f) in (15.1) for all f ∈ F by this modified vector depending on u has the same distribution as the random field Ĩ_{n,k}(f), f ∈ F. On the other hand, I claim that the distribution of the random field Ĩ_{n,k}(f|u), f ∈ F, agrees with the conditional distribution of the random field Ĩ_{n,k}^{ε}(f), f ∈ F, defined in (15.2) under the condition that (ε_{1}, …, ε_{n}) = u with u = (u(1), …, u(n)).

To prove the last statement let us observe that the conditional distribution of the random field Ĩ_{n,k}^{ε}(f), f ∈ F, under the condition (ε_{1}, …, ε_{n}) = u is the same as the distribution of the random field we obtain by putting u_{l} = ε_{l}, 1 ≤ l ≤ n, in all coordinates ε_{l} of the random variables Ĩ_{n,k}^{ε}(f). On the other hand, the random variables we get in such a way agree with the random variables appearing in the sum defining Ĩ_{n,k}(f|u), only the terms in this sum are listed in a different order. Lemma 15.1 is proved.

Next we prove the following generalization of Lemma 7.1.

Lemma 15.2. (Generalized version of the Symmetrization Lemma). Let Z_{p} and Z̄_{p}, p = 1, 2, …, be two sequences of random variables on a probability space (Ω, A, P). Let a σ-algebra B ⊂ A be given on the probability space (Ω, A, P) together with a B-measurable set B and two numbers α > 0 and β > 0 such that the random variables Z_{p}, p = 1, 2, …, are B measurable, and the inequality

P (|Z̄_{p}| ≤ α|B)(ω) ≥ β for all p = 1, 2, … if ω ∈ B  (15.3)

holds. Then

P ( sup_{1≤p<∞} |Z_{p}| > α + u) ≤ \frac{1}{β} P ( sup_{1≤p<∞} |Z_{p} − Z̄_{p}| > u) + (1 − P (B)) for all u > 0.

(15.4)

Proof of Lemma 15.2. Put τ = min{p: |Z_{p}| > α + u) if there exists such an index p ≥ 1, and put τ = 0 otherwise. Then

P (\{τ = p\} ∩ B) ≤ ∫_{\{τ=p\}∩B} \frac{1}{β} P (|Z̄_{p}| ≤ α|B) dP = \frac{1}{β} P (\{τ = p\} ∩ \{|Z̄_{p}| ≤ α\} ∩ B)

≤ \frac{1}{β} P (\{τ = p\} ∩ \{|Z_{p} − Z̄_{p}| > u\}) for all p = 1, 2, ….

Hence

P ( sup_{1≤p<∞} |Z_{p}| > α + u) − (1 − P (B)) ≤ P ({ sup_{1≤p<∞} |Z_{p}| > α + u} ∩ B)

= ∑_{p=1}^{∞} P (\{τ = p\} ∩ B) ≤ \frac{1}{β} ∑_{p=1}^{∞} P (\{τ = p\} ∩ \{|Z_{p} − Z̄_{p}| > u\})

≤ \frac{1}{β} P ( sup_{1≤p<∞} |Z_{p} − Z̄_{p}| > u) .

Lemma 15.2 is proved.

To find a symmetrization argument useful in the proof of Proposition 14.2′ we

want to bound the probability P (sup_{f∈F} |Ī_{n,k}(f)| > u) by C · P (sup_{f∈F} |Ĩ_{n,k}(f)| > cu)

plus a negligible error term with some appropriate numbers C < ∞ and 0 < c < 1. The random variables Ī_{n,k}(f) and Ĩ_{n,k}(f) appearing in these formulas were defined in (14.11) and (15.1). (Actually we work with a slightly modified version of formula (14.11) where the random variables ξ_{l}^{(j)} are replaced by the random variables ξ_{l}^{(j,1)}.) We shall prove the above mentioned estimate with the help of Lemma 15.2. To find the random variables Z_{p} and Z̄_{p} we want to work with in Lemma 15.2 let us list the elements of the class of functions F as F = {f_{1}, f_{2}, … }. We shall apply Lemma 15.2 with the choice Z_{p} = Ī_{n,k}(f_{p}) and Z̄_{p} = Ī_{n,k}(f_{p}) − Ĩ_{n,k}(f_{p}), p = 1, 2, …, together with the σ-algebra B = B(ξ_{l}^{(j,1)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k}.

Let us observe that Z_{p} is a decoupled U-statistic depending on the random variables ξ_{l}^{(j,1)}, 1 ≤ j ≤ k, 1 ≤ l ≤ n, while Z̄_{p} is a linear combination of decoupled U-statistics, whose arguments contain not only the random variables of the form ξ_{l}^{(j,−1)}, but also the random variables of the form ξ_{l}^{(j,1)}. As a consequence, the random variables Z_{p} and Z̄_{p} are not independent. This is the reason why we cannot apply Lemma 7.2 in the proof of Proposition 14.2′.

We shall show that Lemma 15.2 with the choice of the above defined random variables Z_{p} and Z̄_{p} and the σ-algebra B may help us to prove the estimates we need in our considerations. To apply this lemma we have to show that condition (15.3) holds with an appropriate pair of numbers (α, β) and a B measurable set B of probability almost 1. To check this condition is a hard but solvable problem.

In Lemma 7.2 condition (7.1) played a role similar to the condition (15.3) in Lemma 15.2. In that case we could check this condition by estimating the second moment EZ̄_{n}^{2}. In the present case we shall estimate the supremum sup_{f_{p}∈F} E(Z̄_{p}^{2}|B) of

conditional second moments. In this formula Z̄_{p} is a (complicated) random variable depending on the function f_{p} ∈ F. The estimation of the supremum of the conditional second moments we want to work with is a hard problem, and the main difficulties of our proof appear at this point.

The conditional second moments whose supremum we want to estimate can be expressed as the integral of a random function that can be written down explicitly. In such a way we get a problem similar to our original one about the estimation of sup_{f∈F} Ī_{n,k}(f). It turned out that these two problems can be handled similarly. We can

work out a symmetrization argument with the help of Lemma 15.2 in both cases, and an inductive argument similar to Proposition 7.3 can be formulated and proved which supplies the results we want to prove.

We shall prove Proposition 14.2′ as a consequence of two inductive propositions formulated in Propositions 15.3 and 15.4. Here we apply an approach similar to the proof of Proposition 6.2 with the help of Proposition 7.3. The main difference is that now we have to prove two inductive propositions simultaneously, because we have to bound the supremum of some conditional variances, which demands special attention. To formulate these propositions first we introduce the notions of good tail behaviour for a class of decoupled U-statistics and good tail behaviour for a class of integrals of decoupled U-statistics.

Definition of good tail behaviour for a class of decoupled U-statistics. Let some measurable space (X, X) be given together with a probability measure μ on it. Let us consider some countable class F of functions f(x_{1}, …, x_{k}) on the k-fold product (X^{k}, X^{k}) of the space (X, X). Fix some positive integer n ≥ k and a positive number 0 < σ ≤ 1, and take k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence of independent μ-distributed random variables ξ_{1}, …, ξ_{n}. Let us introduce with the help of these random variables the decoupled U-statistics Ī_{n,k}(f), f ∈ F, defined in formula (14.11). Given some real number T > 0 we say that the set of decoupled U-statistics determined by the class of functions F has a good tail behaviour at level T (with parameters n and σ^{2} which are fixed in the sequel) if

P (sup_{f∈F} |n^{−k/2}Ī_{n,k}(f)| ≥ An^{k/2}σ^{k+1}) ≤ exp {−A^{1/2k}nσ^{2}} for all A > T. (15.5)

Definition of good tail behaviour for a class of integrals of decoupled Ustatistics. Let us have a product space (X^{k} × Y, X^{k} × Y) with some product measure μ^{k} ×ρ, where (X^{k}, X^{k}, μ^{k}) is the k-fold product of some probability space (X, X , μ), and (Y, Y, ρ) is some other probability space. Fix some positive integer n ≥ k and a positive number 0 < σ ≤ 1, and consider some countable class F of functions f(x_{1}, …, x_{k}, y) on the product space (X^{k} ×Y, X^{k} ×Y, μ^{k} ×ρ). Take k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence of independent, μ-distributed random variables ξ_{1}, …, ξ_{n}. For all f ∈ F and y ∈ Y let us define the decoupled U-statistics Ī_{n,k}(f, y) = Ī_{n,k}(f_{y}) by means of these random variables ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, the kernel function f_{y}(x_{1}, …, x_{k}) = f(x_{1}, …, x_{k}, y) and formula (14.11). Define with the help of these U-statistics Ī_{n,k}(f, y) the random integrals

H_{n,k}(f) = ∫ Ī_{n,k}(f, y)^{2}ρ( dy), f ∈ F.  (15.6)

Choose some real number T > 0. We say that the set of random integrals H_{n,k}(f), f ∈ F, has a good tail behaviour at level T (with parameters n and σ^{2} which we fix in the sequel) if

P (sup_{f∈F} n^{−k}H_{n,k}(f) ≥ A^{2}n^{k}σ^{2k+2}) ≤ exp {−A^{1/(2k+1)}nσ^{2}} for all A > T. (15.7)

Propositions 15.3 and 15.4 will be formulated with the help of the above notions.

Proposition 15.3. Let us fix a positive integer n ≥ max(k, 2), a real number 0 < σ ≤ 2^{−(k+1)}, a probability measure μ on a measurable space (X, X) together with two real numbers L ≥ 1 and D ≥ 1 such that nσ^{2} ≥ L log n+log D. Let us consider those countable L_{2}-dense class F of canonical kernel functions f = f(x_{1}, …, x_{k}) (with respect to the measure μ) on the k-fold product space (X^{k}, X^{k}) with exponent L and parameter D for which all functions f ∈ F satisfy the inequalities sup_{x_{j}∈X,1≤j≤k} |f(x_{1}, …, x_{k})| ≤ 2^{−(k+1)}

and ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2}.

There is some real number A_{0} = A_{0}(k) > 1 such that if for all classes of functions F which satisfy the above conditions the sets of decoupled U-statistics Ī_{n,k}(f), f ∈ F, have a good tail behaviour at level T^{4/3} for some T ≥ A_{0}, then they also have a good tail behaviour at level T.

Proposition 15.4. Fix some positive integer n ≥ max(k, 2), a real number 0 < σ ≤ 2^{−(k+1)}, a product space (X^{k} × Y, X^{k} × Y) with some product measure μ^{k} × ρ, where (X^{k}, X^{k}, μ^{k}) is the k-fold product of some probability space (X, X , μ), and (Y, Y, ρ) is some other probability space together with two real numbers L ≥ 1 and D ≥ 1 such that nσ^{2} > L log n + log D hold.

Let us consider those countable L_{2}-dense classes F consisting of canonical functions f(x_{1}, …, x_{k}, y) on the product space (X^{k} × Y, X^{k} × Y) with exponent L ≥ 1 and parameter D ≥ 1 whose elements f ∈ F satisfy the inequalities

sup_{x_{j}∈X,1≤j≤k,y∈Y} |f(x_{1}, …, x_{k}, y)| ≤ 2^{−(k+1)}  (15.8)

and

∫ f^{2}(x_{1}, …, x_{k}, y)μ( dx_{1}) … μ( dx_{k})ρ( dy) ≤ σ^{2} for all f ∈ F.  (15.9)

There exists some number A_{0} = A_{0}(k) > 1 such that if for all classes of functions F which satisfy the above conditions the random integrals H_{n,k}(f), f ∈ F, defined in (15.6) have a good tail behaviour at level T^{(2k+1)/2k} with some T ≥ A_{0}, then they also have a good tail behaviour at level T.

Remark: To complete the formulation of Proposition 15.4 we still have to clarify when we call a function f(x_{1}, …, x_{k}, y) defined on the product space (X^{k} ×Y, X^{k} ×Y, μ^{k} ×ρ) canonical. Here the definition is slightly differs from that given in formula (8.8).

We say that a function f(x_{1}, …, x_{k}, y) on the product space (X^{k}×Y, X^{k}×Y, μ^{k}×ρ) is canonical if

∫ f(x_{1}, …, x_{j−1}, u, x_{j+1}, …, x_{k}, y)μ( du) = 0
for all 1 ≤ j ≤ k, x_{s} ∈ X, s ≠ j and y ∈ Y.

We do not require the analogous identity if we integrate with respect to the variable Y wit fixed arguments x_{j} ∈ X, 1 ≤ j ≤ k.

Let me also remark that the estimate (15.7) we have formulated in the definition of the property ‘good tail behaviour for a class of integrals of U-statistics’ is fairly natural. We have applied the natural normalization, and with such a normalization it is natural to expect that the tail behaviour of the distribution of sup_{f∈F} n^{−k}H_{n,k}(f) is similar to

that of const. (ση^{k})^{2}, where η is a standard normal random variable. Formula (15.7) expresses such a behaviour, only the power of the number A in the exponent at the right-hand side was chosen in a non-optimal way. Formula (15.5) in the formulation of the property ‘good tail behaviour for a class of decoupled U-statistics’ has a similar interpretation. It says that sup_{f∈F} |n^{−k/2}I_{n,k}(f)| behaves similarly to const. σ|η^{k}| with a

standard normal random variable η.

We wanted to prove the property of good tail behaviour for a class of integrals of decoupled U-statistics under appropriate, not too restrictive conditions. Let me remark that in Proposition 15.4 we have imposed beside formula (15.8) a fairly weak condition (15.9) about the L_{2}-norm of the function f. Most difficulties appear in the proof, because we did not want to impose more restrictive conditions.

It is not difficult to derive Proposition 14.2′ from Proposition 15.3. Indeed, let us observe that the set of decoupled U-statistics determined by a class of functions F satisfying the conditions of Proposition 15.3 has a good tail-behaviour at level T_{0} = σ^{−(k+1)}, since under the conditions of this Proposition the probability at the left-hand side of (15.5) equals zero for A > σ^{−(k+1)}. Then we get from Proposition 15.3 by induction with respect to the number j, that this set of decoupled U-statistics has a good tail-behaviour also for all T = T_{j} =≥ T_{0}^{(3/4)^{j}} = σ^{−(k+1)(3/4)^{j}}, j = 0, 1, 2, …, with such indices j for which T_{j} = σ^{−(k+1)(3/4)^{j}} ≥ A_{0}. This implies that if a class of functions F satisfies the conditions of Proposition 15.3, then the set of decoupled U-statistics determined by this class of functions has a good tail-behaviour at level T = A_{0}^{4/3}, i.e. at a level which depends only on the order k of the decoupled Ustatistics. This result implies Proposition 14.2′, only it has to be applied for the class of function F′ = {2^{−(k+1)}f, f ∈ F} instead of the original class of functions F which appears in Proposition 14.2′ with the same parameters σ, L and D.

Similarly to the above argument an inductive procedure yields a corollary of Proposition 15.4 formulated below. Actually, we shall need this corollary of Proposition 15.4.

Corollary of Proposition 15.4. If the class of functions F satisfies the conditions of Proposition 15.4, then there exists a constant Ā_{0} = Ā_{0}(k) > 0 depending only on k such that the class of integrals H_{n,k}(f), f ∈ F, defined in formula (15.6) have a good tail behaviour at level Ā_{0}.

The main difficulty in the proof of Proposition 15.3 arises in the application of the symmetrization procedure corresponding to Lemma 7.2 in the one-variate case. This difficulty can be overcome by means of Proposition 15.4, more precisely by means of its corollary. It helps us to estimate the conditional variances of the decoupled U-statistics we have to handle in the proof of Proposition 15.3. The proof of Propositions 15.3 and 15.4 apply similar arguments, and they will be proved simultaneously. The following inductive procedure will be applied in their proof. First Proposition 15.3 and then Proposition 15.4 is proved for k = 1. If Propositions 15.3 and 15.4 are already proved for all k′ < k for some number k, then first we prove Proposition 15.3 and then Proposition 15.4 for this number k.

The proof both of Proposition 15.3 and 15.4 applies a symmetrization argument that will be proved in Section 16. In Section 17 Propositions 15.3 and 15.4 will be proved with its help. They imply Proposition 14.2′, hence also Theorem 8.4.

16. A symmetrization argument.

The proof of Propositions 15.3 and 15.4 applies some ideas similar to the argument in the proof of Proposition 7.3. But here some additional technical difficulties have to be overcome. As a first step, two results formulated in Lemma 16.1A and 16.1B will be proved. They can be considered as a randomization argument with the help of Rademacher functions analogous to the application of Lemma 7.2 in the proof of Propositions 7.3. Lemma 16.1A will be applied in the proof of Proposition 15.3 and Lemma 16.1B in the proof of Proposition 15.4. In this section these lemmas will be proved. Their proofs will be based on some additional lemmas formulated in Lemmas 16.2A, 16.2B, 16.3A and 16.3B. By exploiting the structure of Propositions 15.3 and 15.4 we may assume when proving them for parameter k that they hold (together with their consequences) for all parameters k′ < k.

Lemma 16.1A is a natural multivariate version of Lemma 7.2. Lemma 7.2 enables us to replace the estimation of the distribution of the supremum of a class of sums of independent random variables with the estimation of the distribution of the supremum of the randomized version of these sums. Lemma 16.1A will enable us to reduce the proof of Proposition 15.3 to the estimation of the tail-distribution of the supremum of an appropriately defined class of randomized degenerate U-statistics. This supremum will be estimated by means of the multi-dimensional version of Hoeffding’s inequality given in Theorem 13.3. Lemma 16.B plays a similar role in the proof of Proposition 15.4. But its application is more difficult. In this result the probability investigated in Proposition 15.4 is bounded by the distribution of the supremum of some random variables W̄ (f), f ∈ F, which will be defined in formula (16.7). The expressions W̄ (f), f ∈ F, are rather complicated, and they are worth studying more closely. This will be done in the proof of Corollary of Lemma 16.1B which yields a more appropriate bound for the expression we want to estimate in Proposition 15.4, than Lemma 16.1B. In the proof of Proposition 15.4 the Corollary of 16.1B will be applied instead of the original lemma.

The proof of Lemmas 16.1A and 16.1B is similar to that of Lemma 7.2. First we introduce k additional independent copies ξ̄_{1}^{(j)}, …, ξ̄_{n}^{(j)} beside the k (independent and identically distributed) copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of the sequence ξ_{1}, …, ξ_{n} and construct with their help some appropriate random sums. We shall prove in Lemmas 16.2A and 16.2B that these random sums have the same distribution as their randomized versions we shall work with in the proof of Lemmas 16.1A and 16.1B. These Lemmas formulate a natural multivariate version of an importantant argument in the proof of Lemma 7.2. In the proof of this lemma we have exploited that the random sums defined in (7.4) have the same joint distribution as their randomized versions defined in (7.4′). Lemmas 16.2A and 16.2B formulate a multivariate version of this statement. They enable us (similarly to the corresponding argument in the proof of Lemma 7.2) to reduce the proof of Propositions 16.1A and 16.1B to the study of some simpler questions. This will be done with the help of Lemmas 16.3A and 16.3B. In Lemma 16.3A the supremum of some conditional variances is estimated under appropriate conditions. This lemma plays a similar role in the proof of Lemma 16.1A as condition (7.1) plays in the proof of Lemma 7.1. Its result together with the generalized form of the symmetrization Lemma, Lemma 15.2, enable us to prove Lemma 16.1A. Lemma 16.1B can be proved similarly, but here the conditional distribution of a more complicated expression has to be estimated. This can be done with the help of Lemma 16.3B. In Lemma 16.3B the supremum of the conditional expectation of some appropriate expressions is bounded.

The main results of this section are the following two lemmas.

Lemma 16.1A. (Randomization argument in the proof of Proposition 15.3). Let F be a class of functions on the space (X^{k}, X^{k}) which satisfies the conditions of Proposition 15.3 with some probability measure μ. Let us have k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{n}, and a sequence of independent random variables ε = (ε_{1}, …, ε_{n}), P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is independent also of the random sequences ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k. Consider the decoupled U-statistics Ī_{n,k}(f), f ∈ F, defined with the help of these random variables by formula (14.11) together with their randomized version Ī_{n,k}^{ε}(f) defined in formula (14.12).

There exists some constants A_{0} = A_{0}(k) > 0 and γ = γ_{k} > 0 such that the inequality

\begin{aligned}P (sup_{f∈F} n^{−k/2} |Ī_{n,k}(f)| > An^{k/2}σ^{k+1}) & < 2^{k+1}P (sup_{f∈F} |Ī_{n,k}^{ε}(f)| > 2^{−(k+1)}An^{k}σ^{k+1}) \\ & + 2^{k}n^{k−1}e^{−γ_{k}A^{1/(2k−1)}nσ^{2}/k} \end{aligned}

(16.1)

holds for all A ≥ A_{0}.

It may be worth remarking that the second term at the right-hand side of formula (16.1) yields a small contribution to the upper bound in this relation because of the condition nσ^{2} ≥ L log n + log D.

To formulate Lemma 16.1B first some new quantities have to be introduced. Some of them will be used somewhat later. The quantities Ī_{n,k}^{V}(f, y) introduced in the subsequent formula (16.2) depend on the sets V ⊂ {1, …, k}, and they are the natural modifications of the inner sum terms in formula (15.1). Such expressions are needed in the formulation of the symmetrization result applied in the proof of Proposition 15.4. Their randomized versions Ī_{n,k}^{(V,ε)}(f, y), introduced in formula (16.5), correspond to the inner sum terms in formula (15.2). The integrals of these expressions will be also introduced in formulas (16.3) and (16.6).

Let us consider a class F of functions f(x_{1}, …, x_{k}, y) ∈ F on a space (X^{k} ×Y, X^{k} × Y, μ^{k} ×ρ) which satisfies the conditions of Proposition 15.4. Let us take 2k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, ξ̄_{1}^{(j)}, …, ξ̄_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{k} together with a sequence of independent random variables (ε_{1}, …, ε_{n}), P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is also independent of the previous random sequences. Let us introduce the notation ξ_{l}^{(j,1)} = ξ_{l}^{(j)} and ξ_{l}^{(j,−1)} = ξ̄_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k. For all subsets V ⊂ {1, …, k} of the set {1, …, k} let |V | denote the cardinality of this set, and define for all functions f(x_{1}, …, x_{k}, y) ∈ F and V ⊂ {1, …, k} the decoupled U-statistics

Ī_{n,k}^{V}(f, y) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f (ξ_{l_{1}}^{(1,δ_{1}(V))}, …, ξ_{l_{k}}^{(k,δ_{k}(V))}, y) ,  (16.2)

where δ_{j}(V) = ±1, 1 ≤ j ≤ k, δ_{j}(V) = 1 if j ∈ V, and δ_{j}(V) = −1 if j ∉ V, together with the random variables

H_{n,k}^{V}(f) = ∫ Ī_{n,k}^{V}(f, y)^{2}ρ( dy), f ∈ F.  (16.3)

We shall consider Ī_{n,k}^{V}(f, y) defined in (16.2) as a random variable with values in the space L_{2}(Y, Y, ρ).

Put

Ī_{n,k}(f, y) = Ī_{n,k}^{{1,...,k}}(f, y), H_{n,k}(f) = H_{n,k}^{{1,...,k}}(f),  (16.4)

i.e. Ī_{n,k}(f, y) and H_{n,k}(f) are the random variables Ī_{n,k}^{V}(f, y) and H_{n,k}^{V}(f) with V = {1, …, k} which means that these expressions are defined with the help of the random variables ξ_{l}^{(j)} = ξ_{l}^{(j,1)}, 1 ≤ j ≤ k, 1 ≤ l ≤ n.

Let us also define the ‘randomized version’ of the random variables Ī_{n,k}^{V}(f, y) and H_{n,k}^{V}(f) as

Ī_{n,k}^{(V,ε)}(f, y) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ε_{l_{1}} ⋯ ε_{l_{k}}f (ξ_{l_{1}}^{(1,δ_{1}(V))}, …, ξ_{l_{k}}^{(k,δ_{k}(V))}, y) , f ∈ F,

(16.5)

and

H_{n,k}^{(V,ε)}(f) = ∫ Ī_{n,k}^{(V,ε)}(f, y)^{2}ρ( dy), f ∈ F,  (16.6)

where δ_{j}(V) = 1 if j ∈ V, and δ_{j}(V) = −1 if j ∈ {1, …, k} \ V. Similarly to formula (16.2), we shall consider Ī_{n,k}^{V,ε}(f, y) defined in (16.5) as a random variable with values in the space L_{2}(Y, Y, ρ).

Let us also introduce the random variables

W̄ (f) = ∫ [ ∑_{V ⊂{1,...,k}}(−1)^{(k−|V |)}Ī_{n,k}^{(V,ε)}(f, y)]^{2} ρ( dy), f ∈ F.  (16.7)

With the help of the above notations Lemma 16.1B can be formulated in the following way.

Lemma 16.1B. (Randomization argument in the proof of Proposition 15.4). Let F be a set of functions on (X^{k} × Y, X^{k} × Y) which satisfies the conditions of Proposition 15.4 with some probability measure μ^{k} × ρ. Let us have 2k independent copies ξ_{1}^{(j,±1)}, …, ξ_{n}^{(j,±1)}, 1 ≤ j ≤ k, of a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{n} together with a sequence of independent random variables ε_{1}, …, ε_{n}, P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, which is independent also of the previously considered sequences.

Then there exists some constants A_{0} = A_{0}(k) > 0 and γ = γ_{k} such that if the integrals H_{n,k}(f), f ∈ F, determined by this class of functions F have a good tail behaviour at level T^{(2k+1)/2k} for some T ≥ A_{0}, (this property was defined in Section 15 in the definition of good tail behaviour for a class of integrals of decoupled U-statistics before the formulation of Propositions 15.3 and 15.4), then the inequality

\begin{aligned}P (sup_{f∈F} |H_{n,k}(f)| > A^{2}n^{2k}σ^{2(k+1)}) & < 2P (sup_{f∈F} |W̄ (f)| > \frac{A^{2}}{2} n^{2k}σ^{2(k+1)}) \\ & + 2^{2k+1}n^{k−1}e^{−γ_{k}A^{1/2k}nσ^{2}/k} \end{aligned} (16.8)

holds with the random variables H_{n,k}(f) introduced in the second identity of relation (16.4) and with W̄ (f) defined in formula (16.7) if γ_{k} > 0 is a sufficiently small positive number for all A ≥ T .

A corollary of Lemma 16.1B will be formulated which can be better applied than the original lemma. Lemma 16.B is a little bit inconvenient, because the expression at the right-hand side of formula (16.8) contains a probability depending on sup_{f∈F} |W̄ (f)|,

and W̄ (f) is a too complicated expression. Some new formulas (16.9) and (16.10) will be introduced which enable us to rewrite W̄ (f) in a slightly simpler form. These formulas yield such a corollary of Lemma 16.B which is more appropriate for our purposes. To work out the details first some diagrams will be introduced.

Let G = G(k) denote the set of all diagrams consisting of two rows, such that both rows of these diagrams are the set {1, …, k}, and these diagrams contain some edges {(j_{1}, j′_{1}) …, (j_{s}, j′_{s})}, 0 ≤ s ≤ k, connecting a point (vertex) of the first row with a point (vertex) of the second row. The vertices j_{1}, …, j_{s} which are end points of some edge in the first row are all different, and the same relation holds also for the vertices j′_{1}, …, j′_{s} in the second row which are endpoints of some edge. Given some diagram G ∈ G let e(G) = {(j_{1}, j′_{1}) …, (j_{s}, j′_{s})} denote the set of its edges, and let v_{1}(G) = {j_{1}, …, j_{s}} be the set of those vertices in the first row and v_{2}(G) = {j′_{1}, …, j′_{s}} the set of those vertices in the second row of the diagram G from which an edge of G starts.

Given some diagram G ∈ G and two sets V_{1}, V_{2} ⊂ {1, …, k}, we define the following random variables H_{n,k}(f|G, V_{1}, V_{2}) with the help of the random variables ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)}, ξ_{1}^{(j,−1)}, …, ξ_{n}^{(j,−1)}, 1 ≤ j ≤ k, and ε = (ε_{1}, …, ε_{n}) taking part in the definition of the random variables W̄ (f):

H_{n,k}(f|G, V_{1}, V_{2}) = ∑_{(l_{1},...,l_{k}, l′_{1},...,l′_{k}): 1≤l_{j}≤n, l_{j}≠l_{j′} if j≠j′, 1≤j,j′≤k, 1≤l′_{j}≤n, l′_{j}≠l′_{j′} if j≠j′, 1≤j,j′≤k, l_{j}=l′_{j′} if (j,j′)∈e(G), l_{j}≠l′_{j′} if (j,j′)∉e(G)} ∏_{j∈{1,...,k}\v_{1}(G)}ε_{l_{j}} ∏_{j∈{1,...,k}\v_{2}(G)}ε_{l′_{j}}

\frac{1}{k!^{2}} ∫ f(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))}, …, ξ_{l_{k}}^{(k,δ_{k}(V_{1}))}, y)

f(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))}, …, ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))}, y)ρ( dy),  (16.9)

where δ_{j}(V_{1}) = 1 if j ∈ V_{1}, δ_{j}(V_{1}) = −1 if j ∉ V_{1}, and δ_{j}(V_{2}) = 1 if j ∈ V_{2}, δ_{j}(V_{2}) = −1 if j ∉ V_{2}. (Let us observe that if the graph G contains s edges, then the product of the ε-s in (16.9) contains 2(k − s) terms, and the number of terms in the sum (16.9) is less than n^{2k−s}.) As the Corollary of Lemma 16.1B will indicate, in the proof of Proposition 15.4 we shall need a good estimate on the tail distribution of the random variables H_{n,k}(f|G, V_{1}, V_{2}) for all f ∈ F and G ∈ G, V_{1}, V_{2} ⊂ {1, …, k}. Such an estimate can be obtained by means of Theorem 13.3, the multivariate version of Hoeffding’s inequality. But the estimate we get in such a way will be rewritten in a form more appropriate for our inductive procedure. This will be done in the next section.

The identity

W̄ (f) = ∑_{G∈G, V_{1},V_{2}⊂{1,...,k}}(−1)^{|V_{1}|+|V_{2}|}H_{n,k}(f|G, V_{1}, V_{2})  (16.10)

will be proved.

To prove this identity let us write first

W̄ (f) = ∑_{V_{1},V_{2}⊂{1,...,k}}(−1)^{|V_{1}|+|V_{2}|} ∫ Ī_{n,k}^{(V_{1},ε)}(f, y)Ī_{n,k}^{(V_{2},ε)}(f, y)ρ( dy).

Let us express the products Ī_{n,k}^{(V_{1},ε)}(f, y)Ī_{n,k}^{(V_{2},ε)}(f, y) by means of formula (16.5). Let

us rewrite this product as a sum of products of the form \frac{1}{k!^{2}} ∏_{j=1}^{k} ε_{l_{j}}f(⋯) ∏_{j=1}^{k} ε_{l′_{j}}f(⋯)

and let us define the following partition of the terms in this sum. The elements of this partition are indexed by the diagrams G ∈ G, and if we take a diagram G ∈ G with the set of edges e(G) = {(j_{1}, j′_{1}), …, (j_{s}, j′_{s})}, then the term of this sum determined by the indices l_{1}, …, l_{k}, l′_{1}, …, l′_{k} belongs to the element of the partition indexed by this diagram G if and only if l_{j_{u}} = l′_{j′_{u}} for all 1 ≤ u ≤ s, and no more numbers between the indices l_{1}, …, l_{k}, l′_{1} …, l′_{k} may agree. Since ε_{l_{ju}} ε_{l′_{j′_{u}}} = 1 for all 1 ≤ u ≤ s and the set of

indices of the remaining random variables ε_{l_{j}} is {l_{j}: j ∈ {1, …, k} \ v_{1}(G)}, the set of indices of the remaining random variables ε_{l′_{j}} is {l′_{j′}: j ∈ {1, …, k} \ v_{2}(G)}, we get by

integrating the product Ī_{n,k}^{(V_{1},ε)}(f, y)Ī_{n,k}^{(V_{2},ε)}(f, y) with respect to the measure ρ that

∫ Ī_{n,k}^{(V_{1},ε)}(f, y)Ī_{n,k}^{(V_{2},ε)}(f, y)ρ( dy) = ∑_{G∈G} H_{n,k}(f|G, V_{1}, V_{2})

for all V_{1}, V_{2} ∈ {1, …, k}. The last two relations imply formula (16.10).

Since the number of terms in the sum of formula (16.10) is less than 2^{4k}k!, this relation implies that Lemma 16.1B has the following corollary:

Corollary of Lemma 16.1B. (A simplified version of the randomization argument of Lemma 16.1B). Let a set of functions F satisfy the conditions of Proposition 15.4. Then there exist some constants A_{0} = A_{0}(k) > 0 and γ = γ_{k} > 0 such that if the integrals H_{n,k}(f), f ∈ F, determined by this class of functions F have a good tail behaviour at level T^{(2k+1)/2k} for some T ≥ A_{0}, then the inequality

\begin{matrix}P (sup_{f∈F} |H_{n,k}(f)| > A^{2}n^{2k}σ^{2(k+1)}) \\ ≤ 2 ∑_{G∈G, V_{1},V_{2}⊂\{1,...,k\}} P (sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})| > \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}k!} ) \\ + 2^{2k+1}n^{k−1}e^{−γ_{k}A^{1/2k}nσ^{2}/k} \end{matrix} (16.11)

holds with the random variables H_{n,k}(f) and H_{n,k}(f|G, V_{1}, V_{2}) defined in formulas (16.4) and (16.9) for all A ≥ T.

In the proof of Lemmas 16.1A and 16.1B the result of the following Lemmas 16.2A and 16.2B will be applied.

Lemma 16.2A. Let us take 2k independent copies

ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)}, …, ξ_{n}^{(j,−1)}, 1 ≤ j ≤ k,

of a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{n} together with a sequence of independent random variables (ε_{1}, …, ε_{n}), P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is also independent of the previous sequences.

Let F be a class of functions which satisfies the conditions of Proposition 15.3. Introduce with the help of the above random variables for all sets V ⊂ {1, …, k} and functions f ∈ F the decoupled U-statistic

Ī_{n,k}^{V}(f) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f (ξ_{l_{1}}^{(1,δ_{1}(V))}, …, ξ_{l_{k}}^{(k,δ_{k}(V))})  (16.12)

and its ‘randomized version’

Ī_{n,k}^{(V,ε)}(f) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ε_{l_{1}} ⋯ ε_{l_{k}}f (ξ_{l_{1}}^{(1,δ_{1}(V))}, …, ξ_{l_{k}}^{(k,δ_{k}(V))}) , f ∈ F,

(16.12′)

where δ_{j}(V) = ±1, and δ_{j}(V) = 1 if j ∈ V, and δ_{j}(V) = −1 if j ∈ {1, …, k} \ V.

Then the sets of random variables

S(f) = ∑_{V ⊂{1,...,k}}(−1)^{(k−|V |)}Ī_{n,k}^{V}(f), f ∈ F,  (16.13)

and

S̄(f) = ∑_{V ⊂{1,...,k}}(−1)^{(k−|V |)}Ī_{n,k}^{(V,ε)}(f), f ∈ F,  (16.13′)

have the same joint distribution.

Lemma 16.2B. Let us take 2k independent copies

ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)}, …, ξ_{n}^{(j,−1)}, 1 ≤ j ≤ k,

of a sequence of independent, μ distributed random variables ξ_{1}, …, ξ_{n} together with a sequence of independent random variables (ε_{1}, …, ε_{n}), P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is also independent of the previous sequences.

Let us consider a class F of functions f(x_{1}, …, x_{k}, y) ∈ F on a space (X^{k}×Y, X^{k}× Y, μ^{k}×ρ) which satisfies the conditions of Proposition 15.4. For all functions f ∈ F and V ∈ {1, …, k} consider the decoupled U-statistics Ī_{n,k}^{V}(f, y) defined by formula (16.2)

with the help of the random variables ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)}, …, ξ_{n}^{(j,−1)}, and define with their help the random variables

W (f) = ∫ [ ∑_{V ⊂{1,...,k}}(−1)^{(k−|V |)}Ī_{n,k}^{V}(f, y)]^{2} ρ( dy), f ∈ F.  (16.14)

Then the random vectors {W (f): f ∈ F} defined in (16.14) and {W̄ (f): f ∈ F} defined in (16.7) have the same distribution.

Proof of Lemmas 16.2A and 16.2B. Lemma 16.2A actually agrees with the already proved Lemma 15.1, only the notation is different. The proof of Lemma 16.2B is very similar to the proof of Lemma 15.1. It can be shown that even the following stronger statement holds. For any ±1 sequence u = (u_{1}, …, u_{n}) of length n the conditional distribution of the random field W̄ (f), f ∈ F, under the condition (ε_{1}, …, ε_{n}) = u = (u_{1}, …, u_{n}) agrees with the distribution of the random field W (f), f ∈ F.

To see this relation let us first observe that the conditional distribution of the field W̄ (f) under this condition agrees with the distribution of the random field we get by replacing the random variables ε_{l} by u_{l} for all 1 ≤ l ≤ n in formulas (16.5), (16.6) and (16.7). Beside this, define the vector (ξ(u)_{l}^{(j,1)}, ξ(u)_{l}^{(j,−1)}), 1 ≤ j ≤ k, 1 ≤ l ≤ n, by the formula (ξ(u)_{l}^{(j,1)}, ξ(u)_{l}^{(j,−1)}) = (ξ_{l}^{(j,−1)}, ξ_{l}^{(j,1)}) for those indices (j, l) for which u_{l} = −1, and (ξ(u)_{l}^{(j,1)}, ξ(u)_{l}^{(j,−1)}) = (ξ_{l}^{(j,1)}, ξ_{l}^{(j,−1)}) for which u_{l} = 1 (independently of the value of the parameter j). Then the joint distribution of the vectors (ξ(u)_{l}^{(j,1)}, ξ(u)_{l}^{(j,−1)}), 1 ≤ j ≤ k, 1 ≤ l ≤ n, and (ξ_{l}^{(j,1)}, ξ_{l}^{(j,−1)}), 1 ≤ j ≤ k, 1 ≤ l ≤ n, agree. Hence the joint distribution of the random vectors Ī_{n,k}^{V}(f, y), f ∈ F, V ⊂ {1, …, k} defined in (16.2) and of the random vectors W (f), f ∈ F, defined in (16.14) do not change if we replace in their definition the random variables ξ_{l}^{(j,1)} and ξ_{l}^{(j,−1)} by ξ(u)_{l}^{(j,1)} and ξ(u)_{l}^{(j,−1)}. But the set of random variables W (f), f ∈ F, obtained in this way agrees with the set of random variables we introduced to get a set of random variables with the same distribution as the conditional distribution of W̄ (f), f ∈ F under the condition (ε_{1}, …, ε_{n}) = u. (These random variables are defined as the square integral of the same sum, only the terms of this sum are listed in a different order in the two cases.) These facts imply Lemma 16.2B.

In the next step we prove the following Lemma 16.3A.

Lemma 16.3A. Let us consider a class of functions F satisfying the conditions of Proposition 15.3 with parameter k together with 2k independent copies ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)}, …, ξ_{n}^{(j,−1)}, 1 ≤ j ≤ k, of a sequence of independent, μ-distributed random variables ξ_{1}, …, ξ_{n}. Take the random variables Ī_{n,k}^{V}(f), f ∈ F, V ⊂ {1, …, k}, defined

with the help of these quantities in formula (16.12). Let B = B(ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)}; 1 ≤ j ≤ k) denote the σ-algebra generated by the random variables ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)} , 1 ≤ j ≤ k, i.e. by the random variables with upper indices of the form (j, 1), 1 ≤ j ≤ k. There exists a number A_{0} = A_{0}(k) > 0 such that for all V ⊂ {1, …, k}, V ≠ {1, …, k}, the inequality

P (sup_{f∈F} E (Ī_{n,k}^{V}(f)^{2}| B) > 2^{−(3k+3)}A^{2}n^{2k}σ^{2k+2}) < n^{k−1}e^{−γ_{k}A^{1/(2k−1)}nσ^{2}/k} (16.15)

holds with a sufficiently small γ_{k} > 0 if A ≥ A_{0}.

Proof of Lemma 16.3A. Let us first consider the case V = ∅. In this case the estimate E (Ī_{n,k}^{∅}(f)^{2}| B) = E (Ī_{n,k}^{∅}(f)^{2}) ≤ \frac{n^{k}}{k!} σ^{2} ≤ 2^{k}n^{2k}σ^{2k+2} holds for all f ∈ F. In the above calculation it was exploited that the functions f ∈ F are canonical, which implies certain orthogonalities, and beside this the inequality nσ^{2} ≥ \frac{1}{2} holds, because of the relation nσ^{2} ≥ L log n+log D. The above relations imply that for V = ∅ the probability at the left-hand side of (16.15) equals zero if the number A_{0} is chosen sufficiently large. Hence inequality (16.15) holds in this case.

To avoid some complications in the notation let us first restrict our attention to sets of the form V = {1, …, u} with some 1 ≤ u < k, and prove relation (16.15) for such sets. For this goal let us introduce the random variables

Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k}) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{u}): \\ 1≤l_{j}≤n, j=1,...,u, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f (ξ_{l_{1}}^{(1,1)}, …, ξ_{l_{u}}^{(u,1)}, ξ_{l_{u+1}}^{(u+1,−1)}, …, ξ_{l_{k}}^{(k,−1)})

for all f ∈ F and sequences l(u) = (l_{u+1}, …, l_{k}) with the properties 1 ≤ l_{j} ≤ n for all u+1 ≤ j ≤ k and l_{j} ≠ l_{j′} if j ≠ j′, i.e. let us fix the last k−u coordinates ξ_{l_{u+1}}^{(u+1,−1)}, …, ξ_{l_{k}}^{(k,−1)} of the random variable Ī_{n,k}^{V}(f) and sum up with respect the first u coordinates. Then we can write

E (Ī_{n,k}^{V}(f)^{2}| B) = E ((  ∑_{\begin{matrix}(l_{u+1},...,l_{k}): 1≤l_{j}≤n j=u+1,...,k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k}))^{2}| B)

= ∑_{\begin{matrix}(l_{u+1},...,l_{k}): 1≤l_{j}≤n, j=u+1,...,k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k})^{2}| B) . (16.16)

The last relation follows from the identity

E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k})Ī_{n,k}^{V}(f, l′_{u+1}, …, l′_{k})| B) = 0

if (l_{u+1}, …, l_{k}) ≠ (l′_{u+1}, …, l′_{k}), which holds, since f is a canonical function. We still exploit that the random variables ξ_{l}^{(j,1)}, 1 ≤ j ≤ u are B measurable, while the random variables ξ_{l_{j}}^{(j,−1)}, u + 1 ≤ j ≤ k, are independent of the σ-algebra B. These facts enable us to calculate the above conditional expectation in a simple way.

It follows from relation (16.16) that

{ω: sup_{f∈F} E (Ī_{n,k}^{V}(f)^{2}| B) (ω) > 2^{−(3k+3)}A^{2}n^{2k}σ^{2k+2}}

⊂ ⋃_{\begin{matrix}(l_{u+1},...,l_{k}): \\ 1≤l_{j}≤n, j=u+1,...,k. \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} \{ω: sup_{f∈F} E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k})^{2}| B) (ω) > \frac{A^{2}n^{2k}σ^{2k+2}}{2^{(3k+3)}n^{k−u}} \} .

(16.17)

The probability of the events in the union at the right-hand side of (16.17) can be estimated with the help of the Corollary of Proposition 15.4 with parameter u < k instead of k. (We may assume that Proposition 15.4 holds for u < k.) We claim that this corollary yields that

P (sup_{f∈F} E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k})^{2}| B) > \frac{A^{2}n^{k+u}σ^{2k+2}}{2^{(3k+3)}} ) ≤ e^{−γ_{k}A^{1/(2u+1)}(n+u−k)σ^{2}}

(16.18)

with an appropriate γ_{k} > 0 for all sequences (l_{u+1}, …, l_{k}), 1 ≤ l_{j} ≤ n, u + 1 ≤ j ≤ k, and such that l_{j} ≠ l_{j′} if j ≠ j′.

Let us show that if a class of functions f ∈ F satisfies the conditions of Proposition 15.3 then it also satisfies relation (16.18). For this goal introduce the space (Y, Y, ρ) = (X^{k−u}, X^{k−u}, μ^{k−u}), the k − u-fold power of the measure space (X, X , μ), and for the sake of simpler notations write y = (x_{u+1}, …, x_{k}) for a point y ∈ Y. Let us also introduce the class of those function F̄ in the space (X^{u}×Y, X^{u}×Y, μ^{u}×ρ) consisting of functions f̄ of the form f̄(x_{1}, …, x_{u}, y) = f(x_{1}, …, x_{k}) with y = (x_{u+1}, …, x_{k}) and some function f(x_{1}, …, x_{k}) ∈ F. If the class of function F satisfies the conditions of Proposition 15.3 (with parameter k), then the class of functions F̄ satisfies the conditions of Proposition 15.4 with parameter u < k. Hence the Corollary of Proposition 15.4 can be applied for the class of functions F̄ by our inductive hypothesis. We shall apply it for decoupled U-statistics with the class of kernel functions F̄ and parameters n + k − u and u (instead of n and k), and with the expressions Ī_{n+u−k,u}^{l(u)}(f̄)

and H_{n+u−k,u}^{l(u)}(f̄) defined below with the help of the independent random sequences

ξ_{l}^{(j,1)}, 1 ≤ j ≤ u, l ∈ {1, …, n} \ {l_{u+1}, …, l_{k}} of independent, μ-distributed random variables of length n+u−k, where the set of numbers {l_{u+1}, …, l_{k}} is the set of indices appearing in formula (16.18). It can be seen that with the definition of the random variables Ī_{n+u−k,u}^{l(u)}(f̄, y) and H_{n+u−k,u}^{l(u)}(f̄) we shall give below the identity

E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k})^{2}|B) = (\frac{u!}{k!})^{2} ∫ Ī_{n+u−k,u}^{l(u)}(f̄, y)^{2}ρ( dy) = (\frac{u!}{k!})^{2} H_{n+u−k,u}^{l(u)}(f̄)

(16.19)

holds. In formula (16.19) the function f̄ ∈ F̄ is defined by the formula f̄(x_{1}, …, x_{u}, y) = f(x_{1}, …, x_{k}) with y = (x_{u+1}, …, x_{k}), and the random variables Ī_{n+u−k,u}^{l(u)}(f̄, y) and

H_{n+u−k,u}^{l(u)}(f̄) are defined, similarly to (16.2)–(16.4), by the formulas

Ī_{n+u−k,u}^{l(u)}(f̄, y) = \frac{1}{u!} ∑_{\begin{matrix}(l_{1},...,l_{u}): l_{j}∈\{1,...,n\}\\{l_{u+1},...,l_{k}\}, j=1,...,u \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f̄(ξ_{l_{1}}^{(1,1)}, …, ξ_{l_{u}}^{(u,1)}, y)

and

H_{n+u−k,u}^{l(u)}(f̄) = ∫ Ī_{n+u−k,u}^{l(u)}(f̄, y)^{2}ρ( dy), f̄ ∈ F̄.

The value of H_{n+u−k,u}^{l(u)}(f̄) depends on the choice of the sequence l(u), but its distribution does not depend on it. We can give the following estimate by the corollary of Proposition (15.4) for u < k and relation (16.19). Choose a sufficiently small γ = γ_{k} > 0. We have

P (sup_{f̄∈F̄}(n + u − k)^{−u}H_{n+u−k,u}^{l(u)}(f̄) ≥ γ_{k}^{2/(2u+1)}A^{2}(n + u − k)^{u}σ^{2u+2})

= P (sup_{f̄∈F̄} E(Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k})^{2}|B) ≥ (\frac{k!}{u!})^{2} γ_{k}^{2/(2u+1)}A^{2}(n + u − k)^{2u}σ^{2u+2})

≤ e^{−γ_{k}A^{1/(2u+1)}(n+k−u)σ^{2}} for A > A_{0}(u)γ_{k}^{−2/(2u+1)}.  (16.20)

It is not difficult to derive formula (16.18) from relation (16.20). It is enough to check that the level \frac{A^{2}n^{k+u}σ^{2k+2}}{2^{(3k+3)}} in the probability at the left-hand side of (16.18) can be replaced by γ_{k}^{2/(2u+1)}A^{2} (\frac{k!}{u!})^{2} (n + u − k)^{2u}σ^{2u+2} if γ_{k} > 0 is chosen sufficiently small. This statement holds, since γ_{k}^{2/(2u+1)}A^{2} (\frac{k!}{u!})^{2} (n + u − k)^{2u}σ^{2u+2} < γ_{k}^{2/(2k+1)}A^{2} (\frac{k!}{u!})^{2} n^{2u}σ^{2u+2} ≤ \frac{A^{2}n^{k+u}σ^{2k+2}}{2^{(3k+3)}} if the constant γ_{k} > 0 is chosen sufficiently small, since nσ^{2} > L log n ≤ \frac{1}{2} by the conditions of Proposition 15.3.

Relations (16.17) and (16.18) imply that

P (sup_{f∈F} E (Ī_{n,k}^{V}(f)^{2}| B) (ω) > 2^{−(3k+3)}A^{2}n^{2k}σ^{2k+2}) ≤ n^{k−u}e^{−γ_{k}A^{1/(2u+1)}(n+u−k)σ^{2}}.

Since e^{−γ_{k}A^{1/(2u+1)}(n+u−k)σ^{2}} ≤ e^{−γ_{k}A^{1/(2k−1)}nσ^{2}/k} if u ≤ k − 1, n ≥ k and A > A_{0} with a sufficiently large number A_{0}, inequality (16.15) holds for all sets V of the form V = {1, …, u}, 1 ≤ u < k.

The case of a general set V ⊂ {1, …, k}, 1 ≤ |V | < k, can be handled similarly, only the notation becomes more complicated. Moreover, the case of general sets V can be reduced to the case of sets of form we have already considered. Indeed, given some set V ⊂ {1, …, k}, 1 ≤ |V | < k, let us define a new class of function F_{V} we get by applying a rearrangement of the indices of the arguments x_{1}, …, x_{k} of the functions f ∈ F in such a way that the arguments indexed by the set V are the first |V | arguments of the functions f_{V} ∈ F_{V}, and put V̄ = {1, …, |V |}. Then the class of functions F_{V} also satisfies the condition of Proposition 15.3, and we can get relation (16.15) with the set V by applying it for the set of function F_{V} and set V̄.

Now we prove Lemma 16.1A. It will be proved with the help of Lemma 16.2A, the generalized symmetrization lemma 15.2 and Lemma 16.3A.

Proof of Lemma 16.1A. First we show with the help of the generalized symmetrization lemma, i.e. of Lemma 15.2 and Lemma 16.3A that

\begin{aligned}P (sup_{f∈F} n^{−k/2} |Ī_{n,k}(f)| > An^{k/2}σ^{k+1}) & < 2P (sup_{f∈F} |S(f)| > \frac{A}{2}n^{k}σ^{k+1}) \\ & + 2^{k}n^{k−1}e^{−γ_{k}A^{1/(2k−1)}nσ^{2}/k} \end{aligned} (16.21)

with the function S(f) defined in (16.13). To prove relation (16.21) introduce the ran-

dom variables Z(f) = Ī_{n,k}^{{1,...,k}}(f) and Z̄(f) = − ∑_{V ⊂{1,...,k}, V ≠{1,...,k}}(−1)^{k−|V |}Ī_{n,k}^{V}(f)

for all f ∈ F, the σ-algebra B considered in Lemma 16.3A and the set

B = ⋂_{\begin{matrix}V ⊂\{1,...,k\} \\ V ≠\{1,...,k\} \end{matrix}} \{ω: sup_{f∈F} E (Ī_{n,k}^{V}(f)^{2}| B) (ω) ≤ 2^{−(3k+3)}A^{2}n^{2k}σ^{2k+2}\} .

Observe that S(f) = Z(f) − Z̄(f), f ∈ F, B ∈ B, and by Lemma 16.3A the inequality 1 − P (B) ≤ 2^{k}n^{k−1}e^{−γ_{k}A^{1/(2k−1)}nσ^{2}/k} holds. To prove relation (16.21) apply Lemma 15.2 with the above introduced random variables Z(f) and Z̄(f), f ∈ F, (both here and in the subsequent proof of Lemma 16.1B we work with random variables Z(·) and Z̄(·) indexed by the countable set of functions f ∈ F, hence the functions f ∈ F play the role of the parameters p when Lemma 15.2 is applied) random set B and α = \frac{A}{2}n^{k}σ^{k+1}, u = \frac{A}{2}n^{k}σ^{k+1}. It is enough to show that

P (|Z̄(f)| > \frac{A}{2}n^{k}σ^{k+1}|B) (ω) ≤ \frac{1}{2} for all f ∈ F if ω ∈ B.  (16.22)

But P (|Ī_{n,k}^{|V |}(f)| > 2^{−(k+1)}An^{k}σ^{k+1}|B) (ω) ≤ \frac{2^{2(k+1)}E(Ī_{n,k}^{|V |}(f)^{2}|B)(ω)}{A^{2}n^{2k}σ^{2(k+1)}} ≤ 2^{−(k+1)} for all functions f ∈ F and sets V ⊂ {1, …, k}, V ≠ {1, …, k}, if ω ∈ B by the ‘conditional Chebishev inequality’, hence relations (16.22) and (16.21) hold.

Lemma 16.1A follows from relation (16.21), Lemma 16.2A and the observation that the random variables Ī_{n,k}^{(V,ε)}(f), f ∈ F, defined in (16.12′) have the same distribution for all V ⊂ {1, …, k} as the random variables Ī_{n,k}^{ε}(f), defined in formula (14.12). Hence Lemma 16.2A, the definition (16.13′) of the random variables S̄(f), f ∈ F, implies the inequality

P (sup_{f∈F} |S(f)| > \frac{A}{2}n^{k}σ^{k+1}) = P (sup_{f∈F} |S̄(f)| > \frac{A}{2}n^{k}σ^{k+1})

≤ 2^{k}P (sup_{f∈F} |Ī_{n,k}^{ε}(f)| > 2^{−(k+1)}An^{k}σ^{k+1}) .

Lemma 16.1A is proved.

Lemma 16.1B will be proved with the help of the following Lemma 16.3B, which is a version of Lemma 16.3A.

Lemma 16.3B. Let us consider a class of functions F satisfying the conditions of Proposition 15.4 together with 2k independent copies ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)}, …, ξ_{n}^{(j,−1)}, 1 ≤ j ≤ k, of a sequence of independent, μ-distributed random variables ξ_{1}, …, ξ_{n}. Take the random variables Ī_{n,k}^{V}(f, y) and H_{n,k}^{V}(f), f ∈ F, V ⊂ {1, …, k}, defined in formulas (16.2) and (16.3) with the help of these quantities. Let B = B(ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)}; 1 ≤ j ≤ k) denote the σ-algebra generated by the random variables ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)}, 1 ≤ j ≤ k, i.e. by those random variables which appear in the definition of the random variables Ī_{n,k}^{V}(f, y) and H_{n,k}^{V}(f) introduced in formulas (16.2) and (16.3), and have second argument 1 in their upper index.

a) There exist some numbers A_{0} = A_{0}(k) > 0 and γ = γ_{k} > 0 such that for all V ⊂ {1, …, k}, V ≠ {1, …, k}, the inequality

P (sup_{f∈F} E(H_{n,k}^{V}(f)|B) > 2^{−(4k+4)}A^{(2k−1)/k}n^{2k}σ^{2k+2}) < n^{k−1}e^{−γ_{k}A^{1/2k}nσ^{2}/k}

(16.23)

holds if A ≥ A_{0}.

b) Given two subsets V_{1}, V_{2} ⊂ {1, …, k} of the set {1, …, k} define the integrals (of random kernel functions)

H_{n,k}^{(V_{1},V_{2})}(f) = ∫ |Ī_{n,k}^{V_{1}}(f, y)Ī_{n,k}^{V_{2}}(f, y)|ρ( dy), f ∈ F,  (16.24)

with the help of the functions Ī_{n,k}^{V}(f, y) defined in (16.2). There exist some number A_{0} = A_{0}(k) > 0 and γ = γ_{k} such that if the integrals H_{n,k}(f), f ∈ F, determined by this class of functions F have a good tail behaviour at level T^{(2k+1)/2k} for some T ≥ A_{0}, then the inequality

P (sup_{f∈F} E(H_{n,k}^{(V_{1},V_{2})}(f)|B) > 2^{−(2k+2)}A^{2}n^{2k}σ^{2k+2}) < 2n^{k−1}e^{−γ_{k}A^{1/2k}nσ^{2}/k}

(16.25)

holds for any pairs of subsets V_{1}, V_{2} ⊂ {1, …, k} with the property that at least one of them does not equal the set {1, …, k} if the number A satisfies the condition A > T.

Proof of Lemma 16.3B. Part a) of Lemma 16.3B can be proved in almost the same way as Lemma 16.3A. Hence I only briefly explain the main step of the proof. In the case V = ∅ the identity E(H_{n,k}^{V}(f)|B) = E(H_{n,k}^{V}(f)) holds, hence it is enough to show that

E(H_{n,k}^{V}(f)) ≤ \frac{n^{k}σ^{2}}{k!} ≤ 2^{k} \frac{n^{2k}σ^{2k+2}}{k!} for all f ∈ F under the conditions of Proposition 15.4. (This relation holds, because the functions of the class F are canonical.) The case of a general set V, V ≠ ∅ and V ≠ {1, …, k}, can be reduced to the case V = {1, …, u} with some 1 ≤ u < k.

Given a set V = {1, …, u}, 1 ≤ u < k, let us define for all f ∈ F and sequences l(u) = (l_{u+1}, …, l_{k}) with the properties 1 ≤ l_{j} ≤ n for all u + 1 ≤ j ≤ k and l_{j} ≠ l_{j′} if j ≠ j′ the random variable

Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k}, y) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{u}): \\ 1≤l_{j}≤n, j=1,...,u, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}}f (ξ_{l_{1}}^{(1,1)}, …, ξ_{l_{u}}^{(u,1)}, ξ_{l_{u+1}}^{(u+1,−1)}, …, ξ_{l_{k}}^{(k,−1)}, y) .

It can be shown, similarly to the proof of relation (16.16) in the proof of Proposition 16.3A that because of the canonical property of the functions f ∈ F

E (H̄_{n,k}^{V}(f)| B) = ∑_{\begin{matrix}(l_{u+1},...,l_{k}): \\ 1≤l_{j}≤n, j=u+1,...,k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ∫ E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k}, y)^{2}| B) ρ( dy),

and the proof of part a) of Lemma 16.3B can be reduced to the inequality

P (sup_{f∈F} E (∫ Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k}, y)^{2}ρ( dy)| B) > \frac{A^{(2k−1)/k}n^{k+u}σ^{2k+2}}{2^{(4k+4)}} )
≤ e^{−γ_{k}A^{(2k−1)/2k(2u+1)}(n+u−k)σ^{2}}

with a sufficiently small γ_{k} > 0. This inequality can be proved, similarly to relation (16.18) in the proof of Lemma 16.3A with the help of the Corollary of Proposition 15.4. Only here we have to work in the space (X^{u} × Ȳ , X^{u} × Ȳ, μ^{u} × ρ̄) where Ȳ = X^{k−u} × Y, Ȳ = X^{k−u}×Y, ρ̄ = μ^{k−u}×ρ with the class of function f̄ ∈ F̄ consisting of the functions f̄ defined by the formula f̄(x_{1}, …, x_{u}, ȳ) = f(x_{1}, …, x_{k}, y) with some f(x_{1}, …, x_{k}, y) ∈ F, where ȳ = (x_{u+1}, …, x_{k}, y). Here we apply the following version of formula (16.19).

E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k}, y)^{2}|B) = (\frac{u!}{k!})^{2} ∫ Ī_{n+u−k,u}^{l(u)}(f̄, ȳ)^{2}ρ̄( dȳ) = (\frac{u!}{k!})^{2} H_{n+u−k,u}(f̄)

with the function f̄ ∈ F̄ for which the identity f̄(x_{1}, …, x_{u}, ȳ) = f(x_{1}, …, x_{k}, y) holds with ȳ = (x_{u+1}, …, x_{k}, y) and the and the random variables Ī_{n+u−k,u}^{l(u)}(f̄, ȳ) and H_{n+u−k,u}(f̄) defined similarly as the corresponding terms after formula (16.19), only y is replaced by ȳ, the measure ρ by ρ̄, and the presently defined f̄ ∈ F̄ are considered in the present case. I omit the details.

Part b) of Lemma 16.3B will be proved with the help of Part a) and the inequality

sup_{f∈F} E(H_{n,k}^{(V_{1},V_{2})}(f)|B) ≤ (sup_{f∈F} E(H_{n,k}^{V_{1}}(f)|B))^{1/2} (sup_{f∈F} E(H_{n,k}^{V_{2}}(f)|B))^{1/2}

which follows from the Schwarz inequality applied for integrals with respect to conditional distributions. Let us assume that V_{1} ≠ {1, …, k}. The last inequality implies that

P (sup_{f∈F} E(H_{n,k}^{(V_{1},V_{2})}(f)|B) > 2^{−(2k+2)}A^{2}n^{2k}σ^{2k+2})

≤ P (sup_{f∈F} E(H_{n,k}^{V_{1}}(f)|B) > 2^{−(4k+4)}A^{(2k−1)/k}n^{2k}σ^{2k+2})

+ P (sup_{f∈F} E(H_{n,k}^{V_{2}}(f)|B) > A^{(2k+1)/k}n^{2k}σ^{2k+2})

Hence if we know that also the inequality

P (sup_{f∈F} E(H_{n,k}^{V_{2}}(f)|B) > A^{(2k+1)/k}n^{2k}σ^{2k+2}) ≤ n^{k−1}e^{−γ_{k}A^{1/2k}nσ^{2}}  (16.26)

holds, then we can deduce relation (16.25) from the estimate (16.23) and the last inequality. Relation (16.26) follows from Part a) of Lemma 16.3B if V_{2} ≠ {1, …, k} and A ≥ 1, since in this case the level A^{(2k+1)/k}n^{2k}σ^{2k+2} can be replaced by the smaller number 2^{−(4k+2)}A^{(2k−1)/k}n^{2k}σ^{2k+2} in the probability of formula (16.26). In the case V_{2} = {1, …, k} it follows from the conditions of Part b) of Lemma 16.3B if the number γ_{k} is chosen for some γ_{k} ≤ 1. Indeed, since A^{(2k+1)/2k} > T^{(2k+1)/2k}, by the conditions of Proposition 15.4 the estimate (15.7) holds if the number A is replaced in it by A^{(2k+1)/2k} (at both side of the inequality), and this relation implies inequality (16.26) in this case.

Now we turn to the proof of Lemma 16.1B.

Proof of Lemma 16.1B. By Lemma 16.2B it is enough to prove that relation (16.8) holds if the random variables W̄ (f) are replaced in it by the random variables W (f) defined in formula (16.14). We shall prove this by applying the generalized form of the symmetrization lemma, Lemma 15.2, with the choice of Z(f) = H_{n,k}^{(V̄ ,V̄)}(f), V̄ =

{1, …, k}, Z̄(f) = Z(f) − W (f), f ∈ F, B = B(ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)}; 1 ≤ j ≤ k), α = \frac{A^{2}}{2} n^{2k}σ^{2k+2}, u = \frac{A^{2}}{2} n^{2k}σ^{2k+2} and the set

B = ⋂_{\begin{matrix}(V_{1},V_{2}): V_{j}∈\{1,...,k\}, j=1,2, \\ V_{1}≠\{1,...,k\} or V_{2}≠\{1,...,k\} \end{matrix}} \{ω: sup_{f∈F} E(H_{n,k}^{(V_{1},V_{2})}(f)|B)(ω) ≤ 2^{−(2k+2)}A^{2}n^{2k}σ^{2k+2}\} .

By part b) of Lemma 16.3B the inequality 1 − P (B) ≤ 2^{2k+1}n^{k−1}e^{−γ_{k}A^{1/2k}nσ^{2}/k} holds. Observe that Z(f) = H_{n,k}^{(V̄ ,V̄)}(f) = H_{n,k}(f) for all f ∈ F. Hence to prove Lemma 16.1B with the help of Lemma 15.2 it is enough to show that

P (|Z̄(f)| > \frac{A^{2}}{2} n^{2k}σ^{2k+2}| B) (ω) ≤ \frac{1}{2} for all f ∈ F if ω ∈ B.  (16.27)

To prove this relation observe that because of the definition of the set B

E(|Z̄(f)||B)(ω) ≤ ∑_{\begin{matrix}(V_{1},V_{2}): V_{j}∈\{1,...,k\}, j=1,2, \\ V_{1}≠\{1,...,k\} or V_{2}≠\{1,...,k\} \end{matrix}} E(H_{n,k}^{(V_{1},V_{2})}(f)|B)(ω) ≤ \frac{A^{2}}{4} n^{2k}σ^{2k+2}

if ω ∈ B for all f ∈ F. Hence the ‘conditional Markov inequality’ implies that P (|Z̄(f)| > \frac{A^{2}}{2} n^{2k}σ^{2(k+1)}| B) (ω) ≤ \frac{2E(|Z̄(f)||B)(ω)}{A^{2}n^{2k}σ^{2k+2}} ≤ \frac{1}{2} if ω ∈ B, and inequality (16.27) holds. Lemma 16.1B is proved.

17. The proof of the main result.

This section contains the proof of Proposition 15.3 together with Proposition 15.4. They complete the proof of the main result of this work, of Theorem 8.4.

A.) The proof of Proposition 15.3.

The proof of Proposition 15.3 is similar to that of Proposition 7.3. It applies an induction procedure with respect to the parameter k. In the proof of Proposition 15.3 for parameter k we may assume that Propositions 15.3 and 15.4 hold for u < k. In the proof we want to give a good estimate on the expression

P (sup_{f∈F} |Ī_{n,k}^{ε}(f)| > 2^{−(k+1)}An^{k}σ^{k+1})

appearing in the estimate (16.1) of Lemma 16.1A. To estimate this probability we introduce (using the notation of Proposition 15.3) the functions

S_{n,k}^{2}(f)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): \\ 1≤l_{j}≤n, j=1,...,k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f^{2} (x_{l_{1}}^{(1)}, …, x_{l_{k}}^{(k)}) , f ∈ F,

(17.1)

with x_{l}^{(j)} ∈ X, 1 ≤ l ≤ n, 1 ≤ j ≤ k. We define with the help of this function the following set H = H(A) ⊂ X^{kn} for all A > T similarly to the set defined in formula (7.7).

\begin{aligned}H = H(A) & = \{(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k): \\ & sup_{f∈F} S_{n,k}^{2}(f)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) > 2^{k}A^{4/3}n^{k}σ^{2}\}. \end{aligned} (17.2)

We want to show that

P ({ω: (ξ_{l}^{(j)}(ω), 1 ≤ j ≤ n, 1 ≤ j ≤ k) ∈ H}) ≤ 2^{k}e^{−A^{2/3k}nσ^{2}} if A ≥ T.  (17.3)

To prove relation (17.3) we take the Hoeffding decomposition of the U-statistics with kernel functions f^{2}(x_{1}, …, x_{k}), f ∈ F, given in Theorem 9.1, i.e. we write

f^{2}(x_{1}, …, x_{k}) = ∑_{V ⊂{1,...,k}} f_{V} (x_{j}, j ∈ V), f ∈ F,  (17.4)

with f_{V} (x_{j}, j ∈ V) = ∏_{j∉V} P_{j} ∏_{j∈V} Q_{j}f^{2}(x_{1}, …, x_{k}), where P_{j} is the projection defined in

formula (9.1) and Q_{j} = I − P_{j} agrees with the operator Q_{j} defined in formula (9.2).

The functions f_{V} appearing in formula (17.4) are canonical (with respect to the measure μ), and the identity S_{n,k}^{2}(f)(ξ_{l}^{(j)} 1 ≤ l ≤ n, 1 ≤ j ≤ k) = Ī_{n,k}(f^{2}) holds for all f ∈ F with the expression Ī_{n,k}(·) defined in (14.11). By applying the Hoeffding decomposition (17.4) for each term f^{2}(ξ_{l_{1}}^{(1)} …, ξ_{l_{k}}^{(k)}) in the expression S_{n,k}^{2}(f) we get that

\begin{matrix}P (sup_{f∈F} S_{n,k}^{2}(f)(ξ_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) > 2^{k}A^{4/3}n^{k}σ^{2}) \\ ≤ ∑_{V ⊂\{1,...,k\}} P (\frac{|V |!}{k!} sup_{f∈F} n^{k−|V |}|Ī_{n,|V |}(f_{V})| > A^{4/3}n^{k}σ^{2}) \end{matrix}  (17.5)

with the functions f_{V} appearing in formula (17.4). We want to give a good estimate for each term in the sum at the right-hand side in (17.5). For this goal first we show that the classes of functions {f_{V}: f ∈ F} in the expansion (17.4) satisfy the conditions of Proposition 15.3 for all V ⊂ {1, …, k}.

The functions f_{V} are canonical for all V ⊂ {1, …, k}. It follows from the conditions of Proposition 15.3 that |f^{2}(x_{1}, …, x_{k})| ≤ 2^{−2(k+1)} and

∫ f^{4}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ 2^{−(k+1)}σ^{2}.

Hence relations (9.4) and (9.4′) of Theorem 9.2 imply that | sup_{x_{j}∈X,j∈V} f_{V} (x_{j}, j ∈ V)| ≤

2^{−(k+2)} ≤ 2^{−(k+1)} for all V ⊂ {1, …, k} and ∫ f_{V}^{2} (x_{j}, j ∈ V) ∏_{j∈V} μ( dx_{j}) ≤ 2^{−(k+1)}σ^{2} ≤

σ^{2} for all V ⊂ {1, …, k}. Finally, to check that the class of functions F_{V} = {f_{V}: f ∈ F} is L_{2}-dense with exponent L and parameter D observe that for all probability measures ρ on (X^{k}, X^{k}) and pairs of functions f, g ∈ F ∫ (f^{2} − g^{2})^{2} dρ ≤ 2^{−2k} ∫ (f − g)^{2} dρ. This implies that if {f_{1}, …, f_{m}}, m ≤ Dε^{−L}, is an ε-dense subset of F in the space L_{2}(X^{k}, X^{k}, ρ), then the set of functions {2^{k}f_{1}^{2}, …, 2^{k}f_{m}^{2}} is an ε-dense subset of the class of functions F′ = {2^{k}f^{2}: f ∈ F}, hence F′ is also an L_{2}-dense class of functions with exponent L and parameter D. Then by Theorem 9.2 the class of functions F_{V} is also L_{2}-dense with exponent L and parameter D for all sets V ⊂ {1, …, k}.

For V = ∅, the function f_{V} is constant, f_{V} = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2} holds, and Ī_{|V |}(f_{|V |})| = f_{V} ≤ σ^{2}. Therefore the term corresponding to V = ∅ in the sum of probabilities at the right-hand side of (17.5) equals zero under the conditions of Proposition 15.3 with the choice of some A_{0} ≥ 1. I claim that the remaining terms in the sum at the right-hand side of (17.5) satisfy the inequality

P (\frac{|V |!}{k!}n^{k−|V |} sup_{f∈F} |Ī_{n,|V |}(f_{V})| > A^{4/3}n^{k}σ^{2})

≤ P (sup_{f∈F} |Ī_{n,|V |}(f_{V})| > A^{4/3} \frac{k!}{|V |!}n^{|V |}σ^{|V |+1}) ≤ e^{−A^{2/3k}nσ^{2}} if 1 ≤ |V | ≤ k. (17.6)

The first inequality in (17.6) holds, since σ^{|V |+1} ≤ σ^{2} for |V | ≥ 1, and n ≥ k ≥ |V |. The second inequality follows from the inductive hypothesis if |V | < k, since in this case the middle expression in (17.6) can be bounded with the help of Proposition 15.3 by e^{−(A^{4/3}k!/|V |!)^{1/2|V |}nσ^{2}} ≤ e^{−A^{2/3k}nσ^{2}} if A_{0} = A_{0}(k) in Proposition 15.3 is chosen sufficiently large. In the case V = {1, …, k} it follows from the inequality A ≥ T and the inductive assumption by which the supremum of decoupled U-statistics determined by such a class of kernel-functions which satisfies the conditions of Proposition 15.3 has a good tail behaviour at level T^{4/3}. Relations (17.5) and (17.6) together with the estimate in the case V = ∅ imply formula (17.3).

By conditioning the probability P (|Ī_{n,k}^{ε}(f)| > 2^{−(k+2)}An^{k/2}σ^{k+1}) with respect to

the random variables ξ_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k we get with the help of the multivariate version of Hoeffding’s inequality (Theorem 13.3) that

P (|Ī_{n,k}^{ε}(f)| > 2^{−(k+2)}An^{k}σ^{k+1}| ξ_{l}^{(j)}(ω) = x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)

≤ C exp \{−\frac{1}{2} (\frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{2k+4}S_{n,k}^{2}(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)/k!})^{1/k}\}  (17.7)

≤ Ce^{−2^{−4−4/k}A^{2/3k}(k!)^{1/k}nσ^{2}} for all f ∈ F if (x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) ∉ H

with some appropriate constant C = C(k) > 0.

Define for all 1 ≤ j ≤ k and sets of points x_{l}^{(j)} ∈ X, 1 ≤ l ≤ n, the probability measures ρ_{j} = ρ_{j, (x_{l}^{(j)}, 1≤l≤n)}, 1 ≤ j ≤ k, uniformly distributed on the set of points

\{x_{l}^{(j)}, 1 ≤ l ≤ n\}, i.e. let ρ_{j}(x_{l}^{(j)}) = \frac{1}{n} for all 1 ≤ l ≤ n. Let us also define the product ρ = ρ(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) = ρ_{1} × ⋯ × ρ_{k} of these measures on the space (X^{k}, X^{k}). If f is a function on (X^{k}, X^{k}) such that ∫ f^{2} dρ ≤ δ^{2} with some δ > 0, then

sup_{ε_{1},...,ε_{n}} |Ī_{n,k}^{ε}(f)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)| ≤ \frac{n^{k}}{k!} ∫ |f(u_{1}, …, u_{k})|ρ( du_{1}, …, du_{k})

≤ \frac{n^{k}}{k!} (∫ f^{2} dρ)^{1/2} ≤ \frac{n^{k}}{k!} δ,

u_{j} ∈ R^{k}, 1 ≤ j ≤ k, and as a consequence

sup_{ε_{1},...,ε_{n}} |Ī_{n,k}^{ε}(f)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) − Ī_{n,k}^{ε}(g)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)|

≤ 2^{−(k+2)}An^{k}σ^{k+1} if ∫ (f − g)^{2} dρ ≤ (2^{−(k+2)}k!Aσ^{k+1})^{2},  (17.8)

where Ī_{n,k}^{ε}(f)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) equals the expression Ī_{n,k}^{ε}(f) defined in (14.12)

if we replace ξ_{l_{j}}^{(j)} by x_{l_{j}}^{(j)} for all 1 ≤ j ≤ k, and 1 ≤ l_{j} ≤ n in it, and ρ is the measure

ρ = ρ(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) defined above.

Let us fix the number δ = 2^{−(k+2)}k!Aσ^{k+1}, and let us list the elements of the set F as F = {f_{1}, f_{2}, … }. Put

m = m(δ) = max(1, Dδ^{−L}) = max(1, D(2^{(k+2)}(k!)^{−1}A^{−(1)}σ^{−(k+1)})^{L}),

and choose for all vectors x^{(n)} = (x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) ∈ X^{kn}, such a sequence of positive integers p_{1}(x^{(n)}), …, p_{m}(x^{(n)})) for which

inf_{1≤l≤m} ∫ (f(u) − f_{p_{l}(x^{(n)})}(u))^{2} dρ(x^{(n)}) ≤ δ^{2} for all f ∈ F.

(Here we apply the notation ρ(x^{(n)}) = ρ(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k).) This is possible, since F is an L_{2}-dense class with exponent L and parameter D, and we can choose m = Dδ^{−l}, if δ < 1, Beside this, we can choose m = 1 if δ = 1, since ∫ |f − g|^{2} dρ ≤ sup |f(x) − g(x)|^{2} ≤ 2^{−2k} ≤ 1 for all f, g ∈ F. Moreover, it follows from Lemma 7.4 that the functions p_{l}(x^{(n)}), 1 ≤ l ≤ m, can be chosen as measurable functions of the argument x^{(n)} ∈ X^{kn}.

Let us introduce the random vector ξ^{(n)}(ω) = (ξ_{l}^{(j)}(ω), 1 ≤ l ≤ n, 1 ≤ j ≤ k). By arguing similarly as we did in the proof of Proposition 7.3 we get with the help of relation (17.8) and the property of the functions f_{p_{l}(x^{(n)})}(·) constructed above that

{ω: sup_{f∈F} |Ī_{n,k}^{ε}(f)(ω)| ≥ 2^{−(k+1)}An^{k}σ^{k+1}}

⊂ ⋃_{l=1}^{m} {ω: |Ī_{n,k}^{ε}(f_{p_{l}(ξ^{(n)}(ω))})(ω)| ≥ 2^{−(k+2)}An^{k}σ^{(k+1)}} .

The above relation and formula (17.7) imply that

P (sup_{f∈F} |Ī_{n,k}^{ε}(f)(ω)| > 2^{−(k+1)}An^{k}σ^{k+1}| ξ_{l}^{(j)}(ω) = x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)

≤ ∑_{l=1}^{m} P (|Ī_{n,k}^{ε}(f_{p_{l}(ξ^{(n)}(ω))}(ω)| > \frac{An^{k}σ^{k+1}}{2^{k+2}} | ξ_{l}^{(j)}(ω) = x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)

≤ Cm(δ)e^{−2^{−4−4/k}A^{2/3k}(k!)^{1/k}nσ^{2}}

≤ C(1 + D(2^{k+2}A^{−1}(k!)^{−1}σ^{−(k+1)})^{L})e^{−2^{−4−4/k}A^{2/3k}(k!)^{1/k}nσ^{2}}  (17.9)
if {x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k} ∉ H.

Relations (17.3) and (17.9) imply that

\begin{matrix}P (sup_{f∈F} |Ī_{n,k}^{ε}(f)| > 2^{−(k+1)}An^{k}σ^{k+1}) ≤ C(1 + D(2^{k+2}A^{−1}(k!)^{−1}σ^{−(k+1)})^{L}) \\ e^{−2^{−4−4/k}A^{2/3k}(k!)^{1/k}nσ^{2}} + 2^{k}e^{−A^{2/3k}nσ^{2}} if A > T. \end{matrix} (17.10)

Proposition 15.3 follows from the estimates (16.1), (17.10) and the condition nσ^{2} ≥ L log n + log D, L, D ≥ 1, if A ≥ A_{0} with a sufficiently large number A_{0}. Indeed, in this case nσ^{2} ≥ \frac{1}{2}, (2^{k+2}A^{−1}(k!)^{−1}σ^{−(k+1)})^{L} ≤ (\frac{n^{(k+1)/2}}{(2nσ^{2})^{(k+1)/2}} )^{L} ≤ n^{L(k+1)/2} =

e^{L log n·(k+1)/2} ≤ e^{(k+1)nσ^{2}/2}, D = e^{log D} ≤ e^{nσ^{2}}, and

C(1 + D(2^{k+2}A^{−1}(k!)^{−1}σ^{−(k+1)})^{L})e^{−2^{−4−4/k}A^{2/3k}(k!)^{1/k}nσ^{2}} ≤ \frac{1}{3}e^{−A^{1/2k}nσ^{2}}.

The estimation of the remaining terms in the upper bound of the estimates (16.1) and (17.10) leading to the proof of relation (15.5) is simpler. We can exploit that e^{−A^{2/3k}nσ^{2}} ≪ e^{−A^{1/2k}nσ^{2}} and as n^{k−1} ≤ e^{(k−1)nσ^{2}}

2^{k}n^{k−1}e^{−γ_{k}A^{1/(2k−1)}nσ^{2}/k} ≤ 2^{k}e^{(k−1)nσ^{2}}e^{−γ_{k}A^{1/(2k−1)}nσ^{2}/k} ≪ e^{−A^{1/2k}nσ^{2}}

for a large number A.

Now we turn to the proof of Proposition 15.4.

B.) The proof of Proposition 15.4.

Because of formula (16.11) in the Corollary of Lemma 16.1B to prove Proposition 15.4 i.e. inequality (15.7) it is enough to choose a sufficiently large parameter A_{0} and to show that with such a choice the random variables H_{n,k}(f|G, V_{1}, V_{2}) defined in formula (16.9) satisfy the inequality

P (sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})| > \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}k!} ) ≤ 2^{k+1}e^{−A^{1/2k}nσ^{2}}

for all G ∈ G and V_{1}, V_{2} ∈ {1, …, k} if A > T ≥ A_{0}

(17.11)

under the conditions of Proposition 15.4.

Let us first prove formula (17.11) in the case when |e(G)| = k, i.e. when all vertices of the diagram G are end-points of some edge, and the expression H_{n,k}(f|G, V_{1}, V_{2}) contains no ‘symmetrizing term’ ε_{j}. In this case we apply a special argument to prove relation (17.11).

It can be seen with the help of the Schwarz inequality that for a diagram G such that |e(G)| = k

|H_{n,k}(f|G, V_{1}, V_{2})| ≤ \frac{1}{k!} (  ∑_{\begin{matrix}(l_{1},...,l_{k}): \\ 1≤l_{j}≤n, 1≤j≤k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l_{1}}^{(1),δ_{1}(V_{1}))}, …, ξ_{l_{k}}^{(k,δ_{k}(V_{1}))}, y)ρ( dy))^{1/2}

\frac{1}{k!} (  ∑_{\begin{matrix}(l_{1},...,l_{k}): \\ 1≤l_{j}≤n, 1≤j≤k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{2}))}, …, ξ_{l_{k}}^{(k,δ_{k}(V_{2}))}, y)ρ( dy))^{1/2}

(17.12)

with δ_{j}(V_{1}) = 1 if j ∈ V_{1}, δ_{j}(V_{1}) = −1 if j ∉ V_{1}, and δ_{j}(V_{2}) = 1 if j ∈ V_{2}, δ_{j}(V_{2}) = −1 if j ∉ V_{2}.

Relation (17.12) can be proved for instance by bounding first each integral in formula (16.9) by means of the Schwarz inequality, and then by bounding the sum appearing in such a way by means of the inequality ∑ |a_{j}b_{j}| ≤ (∑ a_{j}^{2})^{1/2} (∑ b_{j}^{2})^{1/2}. Observe that in the case |(e(G)| = k the summation in (16.9) is taken for such vectors (l_{1}, …, l_{k}, l′_{1}, …, l′_{k}) for which (l′_{1}, …, l′_{k}) is a permutation of the sequence (l_{1}, …, l_{k}) determined by the diagram G. Hence the sum we get after applying the Schwarz inequality for each integral in (16.9) has the form ∑ a_{j}b_{j} where the set of indices j in this sum agrees with the set of vectors (l_{1}, …, l_{k}) such that 1 ≤ l_{p} ≤ n for all 1 ≤ p ≤ k, and l_{p} ≠ l_{p′} if p ≠ p′.

By formula (17.12)

\{ω: sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})(ω)| > \frac{A^{2}n^{2k}σ^{(2(k+1)}}{2^{4k+1}k!} \}

⊂ \{ω: sup_{f∈F} ∑_{\begin{matrix}(l_{1},...,l_{k}): \\ 1≤l_{j}≤n, 1≤j≤k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))}(ω), …, ξ_{l_{k}}^{(k,δ_{k}(V_{1}))}(ω), y)ρ( dy)

> \frac{A^{2}n^{2k}σ^{2(k+1)}k!}{2^{4k+1}}\}

∪ \{ω: sup_{f∈F} ∑_{\begin{matrix}(l_{1},...,l_{k}): \\ 1≤l_{j}≤n, 1≤j≤k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{2}))}(ω), …, ξ_{l_{k}}^{(k,δ_{k}(V_{2}))}(ω), y)ρ( dy)

> \frac{A^{2}n^{2k}σ^{2(k+1)}k!}{2^{4k+1}}\},

hence

P (sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})| > \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}k!} )  (17.13)

≤ 2P (sup_{f∈F} \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, 1≤j≤k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} h_{f}(ξ_{l_{1}}^{(1,1)}, …, ξ_{l_{k}}^{(k,1)}) > \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}} )

with the functions h_{f}(x_{1}, …, x_{k}) = ∫ f^{2}(x_{1}, …, x_{k}, y)ρ( dy), f ∈ F. (In this upper bound we could get rid of the terms δ_{j}(V_{1}) and δ_{j}(V_{2}), i.e. on the dependence of the expression H_{n,k}(f|G, V_{1}, V_{2}) on the sets V_{1} and V_{2}, since the probability of the events in the previous formula do not depend on them.)

I claim that

P (sup_{f∈F} |Ī_{n,k}(h_{f})| ≥ 2^{k}An^{k}σ^{2}) ≤ 2^{k}e^{−A^{1/2k}nσ^{2}} for A ≥ A_{0}  (17.14)

if the constant A_{0} = A_{0}(k) is chosen sufficiently large in Proposition 15.4. Relation (17.14) together with the relation A^{2} \frac{n^{2k}σ^{2(k+1)}}{2^{4k+1}} ≥ 2^{k}An^{k}σ^{2} (if A > A_{0} with a sufficiently large A_{0}) imply that the probability at the right-hand side of (17.13) can be bounded by 2^{k+1}e^{−A^{1/2k}nσ^{2}}, and the estimate (17.11) holds in the case |e(G)| = k.

Relation (17.14) is similar to relation (17.3) (together with the definition of the random set H in formula (17.2)), and a modification of the proof of the latter estimate yields the proof also in this case. Indeed, it follows from the conditions of Proposition 15.4 that 0 ≤ ∫ h_{f}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2} for all f ∈ F, and it is not difficult to check that sup |h_{f}(x_{1}, …, x_{k})| ≤ 2^{−2(k+1)}, and the class of functions H = {2^{k}h_{f}, f ∈ F} is an L_{2}-dense class with exponent L and parameter D. Hence by applying the Hoeffding decomposition of the functions h_{f}, f ∈ F, similarly to formula (17.4) we get for all V ⊂ {1, …, k} such a set of functions {h_{f})_{V} , f ∈ F}, which satisfies the conditions of Proposition 15.3. Hence a natural adaptation of the estimate given for the expression at the right-hand side of (17.5) (with the help of (17.6) and the investigation of Ī_{|V |}(f_{V}) for V = ∅) yields the proof of formula (17.14). We only have to replace S_{n,k}(f) by Ī_{n,k}(h_{f}), Ī_{n,|V |}(f_{V}) by Ī_{n,|V |}((h_{f})_{V}) and the levels 2^{k}A^{4/3}n^{k}σ^{2} and A^{4/3}n^{k}σ^{2} by 2^{k}An^{k}σ^{2} and An^{k}σ^{2}. Let us observe that each term of the upper bound we get in such a way can be directly bounded, since during the proof of Proposition 15.4 for parameter k we may assume that the result of Proposition 15.3 holds also for this parameter k.

In the case e(G) < k formula (17.11) will be proved with the help of the multivariate version of Hoeffding’s inequality, Theorem 13.3. In the proof of this case an expression, analogous to S_{n,k}^{2}(f) defined in formula (17.1) will be introduced and estimated for all sets V_{1}, V_{2} ⊂ {1, …, k} and diagrams G ∈ G such that |e(G)| < k. To define it first some notations will be introduced.

Let us consider the set J_{0}(G) = J_{0}(G, k, n),

J_{0}(G) = {(l_{1}, …, l_{k}, l′_{1}, …, l′_{k}): 1 ≤ l_{j}, l′_{j} ≤ n, 1 ≤ j ≤ k, l_{j} ≠ l_{j′} if j ≠ j′,
l′_{j} ≠ l′_{j′} if j ≠ j′, l_{j} = l′_{j′} if (j, j′) ∈ e(G), l_{j} ≠ l′_{j′} if (j, j′) ∉ e(G)}.

The set J_{0}(G) contains those sequences (l_{1}, …, l_{k}, l′_{1}, …, l′_{k}) which appear as indices in the summation in formula (16.9) for a fixed diagram G. We also introduce an appropriate partition of it.

For this aim let us first define the sets M_{1}(G) = {j(1), …, j(k − |e(G)|)} = {1, …, k} \ v_{1}(G), j(1) < ⋯ < j(k − |e(G)|), and M_{2}(G) = {j̄(1), …, j̄(k − |e(G|)} = {1, …, k} \ v_{2}(G), j̄(1) < ⋯ < j̄(k − |e(G|), the sets of those vertices of the first and second row of the diagram G in increasing order from which no edge starts. Let us also introduce the set V (G) = V (G, n, k),

V (G) = {(l_{j(1)}, …, l_{j(k−|e(G)|)}, l′_{j̄(1)}, …, l′_{j̄(k−|e(G)|)}): 1 ≤ l_{j(p)}, l′_{j̄(p)} ≤ n,

1 ≤ p ≤ k − |e(G)|, l_{j(p)} ≠ l_{j(p′)}, l′_{j̄(p)} ≠ l′_{j̄(p′)} if p ≠ p′, 1 ≤ p, p′ ≤ k − |e(G)|,

l_{j(p)} ≠ l′_{j̄(p′)}, 1 ≤ p, p′ ≤ k − |e(G)|}.

The set V (G) consists of those vectors which can appear as the restriction of some vector (l_{1}, …, l_{k}, l′_{1}, …, l′_{k}) ∈ J_{0}(G) to the coordinates indexed by the elements of the set M_{1}(G) ∪ M_{2}(G). The elements of V (G) are such vectors whose coordinates are indexed by the set M_{1}(G)∪M_{2}(G), and they take different integer values between 1 and n. Given a vector v ∈ V (G) put v = (v^{(1)}, v^{(2)}) with v^{(1)} = {v(r), 1 ≤ r ≤ k − |e(G)|}, and v^{(2)} = {v̄(r), 1 ≤ r ≤ k − |e(G)|}, where v^{(1)} and v^{(2)} denote the set of coordinates of v indexed by the elements of the set M_{1}(G) and M_{2}(G) respectively. For all vectors v ∈ V (G) define the set

E_{G}(v) = {(l_{1}, …, l_{k}, l′_{1}, …, l′_{k}): 1 ≤ l_{j} ≤ n, 1 ≤ l′ ; j̄ ≤ n, for 1 ≤ j, j̄ ≤ k,

l_{j} ≠ l_{j′} if j ≠ j′, l′ ; j̄ ≠ l′_{j̄′} if j̄ ≠ j̄′,

l_{j} = l′ ; j̄ if (j, j̄) ∈ e(G) and l_{j} ≠ l′ ; j̄ if (j, j̄) ∉ e(G),

l_{j(r)} = v(r), l′_{j̄(r)} = v̄(r), 1 ≤ r ≤ k − |e(G)|}, v ∈ V (G),

where {j(1), …, j(k − |e(G)|)} = M_{1}(G), {j̄(1), …, j̄(k − |e(G)|)} = M_{2}(G), v = (v^{(1)}, v^{(2)}) with v^{(1)} = (v(1), …, v(k − |e(G)|)) and v^{(2)} = (v̄(1), …, v̄(k − |e(G)|)) in the last line of this definition. Beside this, let us define

E_{G}^{1}(v) = {(l_{1}, …, l_{k}): (l_{1}, …, l_{k}, l′_{1}, …, l′_{k}) ∈ E_{G}(v)}

and

E_{G}^{2}(v) = {(l′_{1}, …, l′_{k}): (l_{1} …, l_{k}, l′_{1}, …, l′_{k}) ∈ E_{G}(v)}.

Given a vector v ∈ V (G), v = (v^{(1)}, v^{(2)}), the set E_{G}(v) consists of those vectors ℓ = (l_{1}, …, l_{k}, l′_{1}, …, l′_{k}) ∈ J_{0}(G) whose restrictions to M_{1}(G) and M_{2}(G) equal v^{(1)} and v^{(2)} respectively. More explicitly, ℓ ∈ E_{G}(v), if for j ∈ M_{1}(G) its coordinate l_{j} agrees with the corresponding element of v^{(1)}, for j̄ ∈ M_{2}(G) its coordinate l′ ; j̄ agrees with the corresponding element of v^{(2)}, and the remaining coordinates of ℓ satisfy the following properties. The indices of the remaining coordinates of ℓ can be partitioned into pairs (j_{s}, j̄_{s′}), 1 ≤ s, s′ ≤ |e(G)| in such a way that (j_{s}, j̄_{s′}) ∈ e(G). The identity l_{j_{s}} = l′ ; j̄_{s′} holds for such pairs (j_{s}, j̄_{s′}), and if (j_{s}, j̄_{s′}) ∉ e(G), then the coordinates l_{j_{s}} and l′ ; j̄_{s′} are different. Otherwise, the coordinates l_{j_{s}} and l′ ; j̄_{s′} can be freely chosen from the set {1, …, n} \ {v^{(1)}, v^{(2)}}. The sets E_{G}^{1}(v) and E_{G}^{2}(v) consist of the vectors containing the first k and the second k coordinates of the vectors ℓ ∈ E_{G}(v).

The sets E_{G}(v), v ∈ V (G), constitute a partition of the set J_{0}(G), and the random variables H_{n,k}(f|G, V_{1}, V_{2}) defined in (16.9) can be rewritten with their help as

\begin{matrix}H_{n,k}(f|G, V_{1}, V_{2})(ω) = ∑_{v=(v^{(1)},v^{(2)})∈V (G)} ∏_{s=1}^{k−|e(G)|} ε_{l_{j(s)}}(ω) ∏_{s=1}^{k−|e(G)|} ε_{l′_{j̄(s)}}(ω) \\ ∑_{(l_{1},...,l_{k},l′_{1}...,l′_{k})∈E_{G}(v)} \frac{1}{k!^{2}} ∫ f(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))}(ω), …, ξ_{l_{k}}^{(k,δ_{k}(V_{1}))}(ω), y) \\ f(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))}(ω), …, ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))}(ω), y)ρ( dy), \end{matrix} (17.15)

where δ_{j}(V_{1}) = 1 if j ∈ V_{1}, δ_{j}(V_{1}) = −1 if j ∉ V_{1}, and δ_{j}(V_{2}) = 1 if j ∈ V_{2}, δ_{j}(V_{2}) = −1 if j ∉ V_{2}.

Let us fix some G ∈ G and V_{1}, V_{2} ⊂ {1, …, k}. The inequality

P (S^{2}(F|G, V_{1}, V_{2}) > 2^{2k}A^{8/3}n^{2k}σ^{4}) ≤ 2^{k+1}e^{−A^{2/3k}nσ^{2}} if A ≥ A_{0} and e(G) < k

(17.16)

will be proved for the random variable

S^{2}(F|G, V_{1}, V_{2}) = sup_{f∈F} \frac{1}{k!^{2}} ∑_{v∈V (G)}(  ∑_{(l_{1},...,l_{k},l′_{1},...,l′_{k})∈E_{G}(v)} ∫ f(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))}, …, ξ_{l_{k}}^{(k,δ_{k}(V_{1}))}, y)

f(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))}, …, ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))}, y)ρ( dy))^{2},  (17.17)

where δ_{j}(V_{1}) = 1 if j ∈ V_{1}, δ_{j}(V_{1}) = −1 if j ∉ V_{1}, and δ_{j}(V_{2}) = 1 if j ∈ V_{2}, δ_{j}(V_{2}) = −1 if j ∉ V_{2}. The random variable S^{2}(F|G, V_{1}, V_{2}) defined in (17.17) plays a similar role in the proof of Proposition 15.4 as the random variable sup_{f∈F} S_{n,k}^{2}(f) with S_{n,k}^{2}(f) defined

in formula (17.1) played in the proof of Proposition 15.3.

To prove formula (17.16) let us first fix some v ∈ V (G), and let us observe that the following inequality similar to relation (17.12) holds.

( ∑_{(l_{1},...,l_{k},l′_{1},...,l′_{k})∈E_{G}(v)} ∫ f(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))}, …, ξ_{l_{k}}^{(k,δ_{k}(V_{1}))}, y)

f(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))}, …, ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))}, y)ρ( dy))^{2}

≤ ( ∑_{(l_{1},...,l_{k})∈E_{G}^{1}(v)} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))}, …, ξ_{l_{k}}^{(k,δ_{k}(V_{1}))}, y)ρ( dy))

(  ∑_{(l′_{1},...,l′_{k})∈E_{G}^{2}(v)} ∫ f^{2}(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))}, …, ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))}, y)ρ( dy))  (17.18)

for all f ∈ F and v ∈ V (G). Indeed, observe that for a vector v̄ = (v̄_{1}, v̄_{2}) ∈ E_{G}(v) with v̄_{1} ∈ E_{G}^{1}(v) and v̄_{2} ∈ E_{G}^{2}(v), the coordinates of the vector v̄_{1} in the set M_{1}(G) and the coordinates of the vector v̄_{2} in the set M_{2}(G) are prescribed, while the coordinates of v̄_{1} in the set v_{1}(G) are given by a permutation of the coordinates v̄_{2} in the set v_{2}(G). (The sets v_{1}(G) and v_{2}(G) were defined before the introduction of formula (16.9) as the sets of those vertices in the first and second row of the diagram G respectively from which an edge of G starts.) This permutation is determined by the diagram G. Inequality (17.18) can be proved on the basis of the above observation similarly to formula (17.12).

We shall prove with the help of formula (17.18) the following inequality.

S^{2}(F|G, V_{1}, V_{2})

≤ sup_{f∈F} ∑_{v∈V (G)} \frac{1}{k!} ( ∑_{(l_{1},...,l_{k})∈E_{G}^{1}(v)} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))}, …, ξ_{l_{k}}^{(k,δ_{k}(V_{1}))}, y)ρ( dy))

\frac{1}{k!} (  ∑_{(l′_{1},...,l′_{k})∈E_{G}^{2}(v)} ∫ f^{2}(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))}, …, ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))}, y)ρ( dy))  (17.19)

≤ sup_{f∈F} \frac{1}{k!} ( ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, 1≤j≤k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))}, …, ξ_{l_{k}}^{(k,δ_{k}(V_{1}))}, y)ρ( dy))

sup_{f∈F} \frac{1}{k!} ( ∑_{\begin{matrix}(l′_{1},...,l′_{k}): 1≤l′_{j}≤n, 1≤j≤k, \\ l′_{j}≠l′_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))}, …, ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))}, y)ρ( dy)) .

The first inequality of (17.19) is a simple consequence of formula (17.18) and the definition of the random variable S^{2}(F|G, V_{1}, V_{2}). To check its second inequality let us observe that it can be reduced to the simpler relation, where the expression sup_{f∈F} is omitted at

each place. The simplified inequality obtained after the omission of the expressions sup can be checked by carrying out the term by term multiplication between the products of sums appearing in (17.19). At both sides of the inequality a sum consisting of terms of the form

\frac{1}{k!^{2}} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))}, …, ξ_{l_{k}}^{(k,δ_{k}(V_{1}))}, y)ρ( dy) ∫ f^{2}(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))}, …, ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))}, y)ρ( dy),

(17.20) appears. It is enough to check that if a term of this form appears in the middle term of the simplified version formula of (17.19), then it appears with coefficient 1, and it also appears at the right-hand side of this formula. To see this, observe that each term of the form (17.20) which appears in the sum we get by carrying out the multiplications in middle term of (17.19) determines uniquely the index v = (v^{(1)}, v^{(2)}) ∈ V (G) in the outer sum of the middle term in the inequality (17.19). Indeed, if the random variables defining this expression of the form (17.20) have indices ℓ = (l_{1}, …, l_{k}, l′_{1}, …, l′_{k}), then this vector ℓ uniquely determines the vector v = (v^{(1)}, v^{(2)}) ∈ V (G), since v^{(1)} must agree with the restriction of the vector l = (l_{1}, …, l_{k}) to the coordinates with indices in M_{1}(G) and v^{(2)} must agree with the restriction of the vector l′ = (l′_{1}, …, l′_{k}) to the coordinates with indices in M_{2}(G). Beside this, by carrying out the multiplication at the right-hand side of (17.19) we get such a sum which contains all such terms of the form (17.20) which appeared in the sum expressing the middle term in inequality (17.19). The above arguments imply inequality (17.19).

Relation (17.19) implies that

P (S^{2}(F|G, V_{1}, V_{2})) > 2^{2k}A^{8/3}n^{2k}σ^{4}) ≤ 2P (sup_{f∈F} Ī_{n,k}(h_{f}) > 2^{k}A^{4/3}n^{k}σ^{2})

with h_{f}(x_{1}, …, x_{k}) = ∫ f^{2}(x_{1}, …, x_{k}, y)ρ( dy). (Here we exploited that in the last formula S^{2}(F|G, V_{1}, V_{2}) is bounded by the product of two random variables whose distributions do not depend on the sets V_{1} and V_{2}.) Thus to prove inequality (17.16) it is enough to show that

2P (sup_{f∈F} Ī_{n,k}(h_{f}) > 2^{k}A^{4/3}n^{k}σ^{2}) ≤ 2^{k+1}e^{−A^{2/3k}nσ^{2}} if A ≥ A_{0}.  (17.21)

Actually formula (17.21) follows from the already proven formula (17.14), only the parameter A has to be replaced by A^{4/3} in it.

With the help of relation (17.16) the proof of Proposition 15.4 can be completed similarly to Proposition 15.3. The following version of inequality (17.7) can be proved with the help of the multivariate version of Hoeffding’s inequality, Theorem 13.3, and the representation of the random variable H_{n,k}(f|G, V_{1}, V_{2}) in the form (17.15).

\begin{matrix}P (|H_{n,k}(f|G, V_{1}, V_{2})| > \frac{A^{2}}{2^{4k+2}k!}n^{2k}σ^{2(k+1)}| ξ_{l}^{j,±1}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) (ω) \\ ≤ Ce^{−2^{−(6+2/k)}A^{2/3k}nσ^{2}} if S^{2}(F|G, V_{1}, V_{2})(ω) ≤ 2^{2k}A^{8/3}n^{2k}σ^{4} and A ≥ A_{0} \end{matrix}

(17.22)

with an appropriate constant C = C(k) > 0 for all f ∈ F and G ∈ G such that |e(G)| < k and V_{1}, V_{2} ⊂ {1, …, k}. (Observe that the conditional probability estimated in (17.22) can be represented in the following way. In a point ω ∈ Ω fix the values of ξ_{l}^{(j,±1)}(ω) for all indices 1 ≤ l ≤ n and 1 ≤ j ≤ k in the random variable H_{n,k}(f|G, V_{1}, V_{k}), and the conditional probability in this point ω equals the probability that the random variable, (depending on the random variables ε_{l}, 1 ≤ l ≤ n), obtained in such a way is greater than \frac{A^{2}}{2^{4k+2}k!}n^{2k}σ^{2(k+1)}.)

Indeed, in this case the conditional probability considered in (17.22) can be bounded because of the multivariate version of the Hoeffding inequality (Theorem 13.3) by

C exp \{−\frac{1}{2} (\frac{A^{4}n^{4k}σ^{4(k+1)}}{2^{8k+4}(k!)^{2}S^{2}(F|G,V_{1},V_{2})/(k!)^{2}} )^{1/2j}\} ≤ C exp \{−\frac{1}{2} (\frac{A^{4/3}n^{2k}σ^{4k}}{2^{10k+4}} )^{1/2j}\} with an

appropriate C = C(k) > 0, where 2j = 2k − 2|e(G)|, and 0 ≤ |e(G)| ≤ k − 1. Since j ≤ k, nσ^{2} ≥ \frac{1}{2}, and also \frac{A^{4/3}}{2^{10k+4}} ≥ 2 if A_{0} is chosen sufficiently large we can write in the above upper bound for the left-hand side of (17.22) j = k, and in such a way we get inequality (17.22).

The next inequality in which we estimate sup_{f∈F} H_{n,k}(f|G, V_{1}, V_{2}) is a natural version

of formula (17.9) in the proof of Proposition 15.3.

P ( sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})| > \frac{A^{2}}{2^{4k+1}k!}n^{2k}σ^{2(k+1)}| ξ_{l}^{(j,±1)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) (ω)

≤ C (1 + D (\frac{2^{4k+3}k!}{A^{2}σ^{2(k+1)}} )^{L}) e^{−2^{−(6+2/k)}A^{2/3k}nσ^{2}}

if S^{2}(F|G, V_{1}, V_{2}))(ω) ≤ 2^{2k}A^{8/3}n^{2k}σ^{4} and A ≥ A_{0}  (17.23)

for all G ∈ G such that |e(G)| < k and V_{1}, V_{2} ⊂ {1, …, k}.

To prove formula (17.23) let us fix two sets V_{1}, V_{2} ⊂ {1, …, k} and a diagram G such that |e(G)| < k. Let us define for all vectors x^{(n)} = (x_{l}^{(j,1)}, x_{l}^{(j,−1)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) ∈ X^{2kn} some probability measure α(x^{(n)}) on the space X^{k} ×Y (with the space Y which appears in the formulation of Proposition 15.4) with which we can work similarly as with the probability measures ν(x^{(n)} and ρ(x^{(n)} in the proof of Propositions 7.3 and 15.3.

To do this let us consider some vector x^{(n)} = (x_{l}^{(j,1)}, x_{l}^{(j,−1)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) ∈ X^{2kn}, and define first the probability measures ν_{j}^{(1)} = ν_{j}^{(1)}(x^{(n)}, V_{1}) and ν_{j}^{(2)} = ν_{j}^{(2)}(x^{(n)}, V_{2}) in the space (X, X) for all 1 ≤ j ≤ k which are uniformly distributed

in the set of points x_{l}^{(j,δ_{j}(V_{1}))}, 1 ≤ l ≤ n and x_{l}^{(j,δ_{j}(V_{2}))}, 1 ≤ l ≤ n, respectively. This means that we define for all 1 ≤ j ≤ k (and sets V_{1} and V_{2}) the probability measures ν_{j}^{(1)} (\{x_{l}^{(j,δ_{j}(V_{1}))}\}) = \frac{1}{n} and ν_{j}^{(2)} (\{x_{l}^{(j,δ_{j}(V_{2}))}\}) = \frac{1}{n}, 1 ≤ l ≤ n, where δ_{j}(V_{1}) = 1 if j ∈ V_{1}, δ_{j}(V_{1}) = −1 if j ∉ V_{1}, and similarly δ_{j}(V_{2}) = 1 if j ∈ V_{2} and δ_{j}(V_{2}) = −1 if j ∉ V_{2}. Let us consider the product measures α_{1} = α_{1}(x^{(n)}, V_{1}) = ν_{1}^{(1)}×⋯×ν_{k}^{(1)}×ρ and α_{2} = α_{2}(x^{(n)}, V_{2}) = ν_{1}^{(2)}×⋯×ν_{k}^{(2)}×ρ on the product space (X^{k}×Y, X^{k}×Y), where ρ is that probability measure on (Y, Y) which appears in Proposition 15.4. With the help of the measures α_{1} and α_{2} define the measure α = α(x^{(n)}) = α(x^{(n)}, V_{1}, V_{2}) = \frac{α_{1}+α_{2}}{2} in the space (X^{k} × Y, X^{k}, ×Y). Let us also define the measure α̃ = α̃(x^{(n)}) = α̃(x^{(n)}, V_{1}, V_{2}) = ν_{1}^{(1)} × ⋯ ν_{k}^{(1)} × ν_{1}^{(2)} × ⋯ ν_{k}^{(2)} × ρ in the space (X^{2k} × Y, X^{2k}, ×Y).

Let us define H_{n,k}(f|G, V_{1}, V_{2}) as a function in the product space (X^{2kn}, X^{2kn}) (with arguments x_{l}^{(j,1)} and x_{l}^{(j,−1)}, 1 ≤ j ≤ k, 1 ≤ l ≤ n) by means of formula (17.15) by replacing the random variables ξ_{l_{j}}^{(j,δ_{j}(V_{1}))}(ω) by x_{l_{j}}^{(j,δ_{j}(V_{1}))} and the random variables

ξ_{l′_{j}}^{(j,δ_{j}(V_{2}))}(ω) by x_{l′_{j}}^{(j,δ_{j}(V_{2}))} in it for all 1 ≤ j ≤ k and 1 ≤ l_{j}, l′_{j} ≤ n. With such a notation

we can write for any pairs f, g ∈ F and x^{(n)} = (x_{l}^{j,1)}, x_{l}^{(j,−1)}, 1 ≤ j ≤ k, 1 ≤ l ≤ n) ∈ X^{2kn}, by exploiting the properties of the above defined measure α̃ the inequality

sup_{ε_{1},...,ε_{n}} |H_{n,k}(f|G, V_{1}, V_{2})(x^{(n)}) − H_{n,k}(f|G, V_{1}, V_{2})(x^{(n)})|

≤ ∑_{v=(v^{(1)},v^{(2)})∈V (G)} ∑_{(l_{1},...,l_{k},l′_{1}...,l′_{k})∈E_{G}(v)}

\frac{1}{k!^{2}} ∫ |f(x_{l_{1}}^{(1,δ_{1}(V_{1}))}, …, x_{l_{k}}^{(k,δ_{k}(V_{1}))}, y)f(x_{l′_{1}}^{(1,δ_{1}(V_{2}))}, …, x_{l′_{k}}^{(k,δ_{k}(V_{2}))}, y)

− g(x_{l_{1}}^{(1,δ_{1}(V_{1}))}, …, x_{l_{k}}^{(k,δ_{k}(V_{1}))}, y)g(x_{l′_{1}}^{(1,δ_{1}(V_{2}))}, …, x_{l′_{k}}^{(k,δ_{k}(V_{2}))}, y)|ρ( dy)

≤ n^{2k} ∫ |f(x_{1}, …, x_{k}, y)f(x_{k+1}, …, x_{2k}, y) − g(x_{1}, …, x_{k}, y)g(x_{k+1}, …, x_{2k}, y)|

α̃( dx_{1}, …, dx_{2k}, dy).

(17.24)

Beside this, since both sup |f(x_{1}, …, x_{k}, y)| ≤ 1 and sup |g(x_{1}, …, x_{k}, y)| ≤ 1, we have

|f(x_{1}, …, x_{k}, y)f(x_{k+1}, …, x_{2k}, y) − g(x_{1}, …, x_{k}, y)g(x_{k+1}, …, x_{2k}, y)|
≤ |f(x_{1}, …, x_{k}, y)||f(x_{k+1}, …, x_{2k}, y) − g(x_{k+1}, …, x_{2k}, y)|
+ |g(x_{k+1}, …, x_{2k})||f(x_{1}, …, x_{k}, y) − g(x_{1}, …, x_{k}, y)|
≤ |f(x_{k+1}, …, x_{2k}, y) − g(x_{k+1}, …, x_{2k}, y)|
+ |f(x_{1}, …, x_{k}, y) − g(x_{1}, …, x_{k}, y)|.

It follows from this inequality, formula (17.24) and the definition of the measures α̃, α_{1}, α_{2} and α that

sup_{ε_{1},...,ε_{n}} |H_{n,k}(f|G, V_{1}, V_{2})(x^{(n)}) − H_{n,k}(f|G, V_{1}, V_{2})(x^{(n)})|

≤ n^{2k} ∫ (|f(x_{k+1}, …, x_{2k}, y) − g(x_{k+1}, …, x_{2k}, y)|
+ |f(x_{1}, …, x_{k}, y) − g(x_{1}, …, x_{k}, y)|)α̃( dx_{1}, …, dx_{2k}, dy)
= n^{2k} ∫ |f(x_{1}, …, x_{k}, y) − g(x_{1}, …, x_{k}, y)|
(α_{1}( dx_{1}, …, dx_{k}, dy) + α_{2}( dx_{1}, …, dx_{k}, dy))  (17.25)
= 2n^{2k} ∫ |f(x_{1}, …, x_{k}, y) − g(x_{1}, …, x_{k}, y)|α( dx_{1}, …, dx_{k}, dy)
≤ 2n^{2k} (∫ |f(x_{1}, …, x_{k}, y) − g(x_{1}, …, x_{k}, y)|^{2}α( dx_{1}, …, dx_{k}, dy))^{1/2}

with the previously defined probability measure α = α(x^{(n)}). Put δ = \frac{A^{2}σ^{2(k+1)}}{2^{4k+3}k!} , list the elements of F as F = {f_{1}, f_{2}, … }, and choose a set of indices p_{1}(x^{(n)}), …, p_{m}(x^{(n)}) taking positive integer values with m = max(1, Dδ^{−L}) elements such that sup_{1≤l≤m} ∫ f(u)−

f_{p_{l}(x^{(n)})}(u))^{2}α(x^{(n)})( du) ≤ δ^{2} for all f ∈ F. Such a choice of the indices p_{l}(x^{(n)}), 1 ≤ l ≤ m, is possible, since F is L_{2}-dense with exponent L and parameter D. Moreover, by Lemma 7.4 we may chose the functions p_{l}(x^{(n)}), 1 ≤ l ≤ m, as measurable functions of their argument x^{(n)} ∈ X^{2kn}.

Put ξ^{(n)}(ω) = (ξ_{l}^{(j,±1)}(ω), 1 ≤ l ≤ n, 1 ≤ j ≤ k). By arguing similarly as we did in the proof of Propositions 7.3 and (15.3) we get with the help of relation (17.25) and the property of the functions f_{p_{l}(x^{(n)})}(·) constructed above that

\{ω: sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})(ω)| ≥ \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{(4k+1)}k!} \}

⊂ ⋃_{l=1}^{m} \{ω: |H_{n,k}(f_{p_{l}(ξ^{(n)}(ω)}|G, V_{1}, V_{2})(ω)(ω)| ≥ \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{(4k+2)}k!} \} .

Hence

P ( sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})| > \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}k!} | ξ_{l}^{(j,±1)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) (ω)

≤ ∑_{l=1}^{m} P (|H_{n,k}(f_{p_{l}(ξ^{(n)}(ω))}|G, V_{1}, V_{2})| > \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}k!} |

ξ_{l}^{(j,±1)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)(ω)

for almost all ω. The last inequality together with (17.22) and the inequality m =

max(1, Dδ^{−L}) ≤ 1 + D (\frac{2^{4k+3}k!}{A^{2}σ^{2(k+1)}} )^{L} imply relation (17.23).

It follows from relations (17.16) and (17.23) that

P (sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})| > \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}k!} ) ≤ 2^{k+1}e^{−A^{2/3k}nσ^{2}}

+ C (1 + D (\frac{2^{4k+3}k!}{A^{2}σ^{2(k+1)}} )^{L}) e^{−2^{−(6+2/k)}A^{2/3k}nσ^{2}} if A ≥ A_{0}

for all V_{1}, V_{2} ⊂ {1, …, k} and diagram G ∈ G such that |e(G)| ≤ k − 1. This inequality implies that relation (17.11) holds also in the case |e(G)| ≤ k − 1 if the constants A_{0} is chosen sufficiently large in Proposition 15.4, and we this completes the proof of Proposition 15.4. To prove relation (17.11) in the case |e(G)| ≤ k − 1 we still have to show that D(\frac{2^{4k+3}k!}{A^{2}σ^{2(k+1)}} )^{L} ≤ e^{const. nσ^{2}} if A > A_{0} with a sufficiently large A_{0}, since this implies that the second term at the right-hand of our last estimation is not too large.

This follows from the inequality nσ^{2} ≥ L log n + log D which implies that

(\frac{2^{4k+3}k!}{A^{2}σ^{2(k+1)}} )^{L} ≤ (\frac{n^{(k+1)}}{(2nσ^{2})^{(k+1)}} )^{L} ≤ e^{(k+1)L log n} ≤ e^{(k+1)nσ^{2}}

if A_{0} is sufficiently large, and D = e^{log D} ≤ e^{nσ^{2}}.

18. An overview of the results in this work.

I discuss briefly the problems investigated in this work, recall some basic results related to them, and also give some references. I also write about the background of these problems which may explain the motivation for their study.

I met the main problem considered in this work when I tried to adapt the method of proof of the central limit theorem for maximum-likelihood estimates to some more difficult questions about so-called non-parametric maximum likelihood estimate problems. The Kaplan–Meyer estimate for the empirical distribution function with the help of censored data investigated in the second section is such a problem. It is not a maximum-likelihood estimate in the classical sense, but it can be considered as a non-parametric maximum likelihood estimate. In the estimation of the empirical distribution function with the help of censored data we cannot apply the classical maximum likelihood method, since in the solution of this problem we have to choose our estimate from a too large class of distribution functions. The main problem is that there is no dominating measure with respect to which all candidates which may appear as our estimate have a density function. A natural way to overcome this difficulty is to choose a smaller class of distribution functions, to compare the probability of the appearance of the sample we observed with respect to all distribution functions of this class and to choose that distribution function as our estimate for which this probability takes its maximum.

The Kaplan–Meyer estimate can be found on the basis of the above principle in the following way: Let us estimate the distribution function F (x) of the censored data simultaneously together with the distribution function G(x) of the censoring data. (We have a sample of size n and know which sample elements are censored and which are censoring data.) Let us consider the class of such pairs of estimates (F_{n}(x), G_{n}(x)) of the pair (F (x), G(x)) for which the distribution function F_{n}(x) is concentrated in the censored sample points and the distribution function G_{n}(x) is concentrated in the censoring sample points; more precisely, let us also assume that if the largest sample point is a censored point, then the distribution function G_{n}(x) of the censoring data takes still another value which is larger than any sample point, and if it is a censoring point then the distribution function F_{n}(x) of the censored data takes still another value larger than any sample point. (This modification at the end of the definition is needed, since if the largest sample points is from the class of censored data, then the distribution G(x) of the censoring data in this point must be strictly less than 1, and if it is from the class of censoring data, then the value of the distribution function F (x) of the censored data must be strictly less than 1 in this point.) Let us take this class of pairs of distribution functions (F_{n}(x), G_{n}(x)), and let us choose that pair of distribution functions of this class as the (non-parametric maximum likelihood) estimate with respect to which our observation has the greatest probability.

The above extremal problem for the pairs of distribution functions (F_{n}(x), G_{n}(x)) can be solved explicitly, (see [25]), and it yields the estimate of F_{n}(x) written down in formula (2.3). (The function G_{n}(x) satisfies a similar relation, only the random variables X_{j} and Y_{j} and the events δ_{j} = 1 and δ_{j} = 0 have to be replaced in it.) Then, as I have indicated, a natural analog of the linearization procedure in the proof of the central limit theorem for the classical maximum likelihood estimate works also in this case, and there is only one really hard part of the proof. We have to show that the linearization procedure gives a small error. The estimation of this error led to the problem about a good estimate on the tail distribution of the integral of an appropriate function of two variables with respect to the product of a normalized empirical measure with itself. Moreover, as a more detailed investigation showed, we actually need the solution of a more general problem where we have to bound the tail distribution of the supremum of a class of such integrals. The main subject of this work is to solve the above problems in a more general setting, to estimate not only two-fold, but also k-fold random integrals and the supremum of such integrals for an appropriate class of kernel functions with respect to a normalized empirical distribution for all k ≥ 1.

The proof of the limit theorem for the Kaplan–Meyer estimate explained in this work applied the explicit form of this estimate. It would be interesting to find such a modification of this proof which only exploits that the Kaplan–Meyer estimate is the solution of an appropriate extremal problem. We may expect that such a proof can be generalized to a general result about the limit behaviour for a wide class of nonparametric maximum likelihood estimates. Such a consideration is behind the remark of Richard Gill I quoted at the end of Section 2.

A detailed proof together with a sharp estimate on the speed of convergence for the limit behaviour of the Kaplan–Meyer estimate based on the ideas presented in Section 2 is given in paper [38]. Paper [39] explains more about its background, and it also discusses the solution of some other non-parametric maximum likelihood problems. The results about multiple integrals with respect to a normalized empirical distribution function needed in these works were proved in [30]. These results were satisfactory for the study in [38], but they also have some drawbacks. They do not show that if the random integrals we are considering have small variances, then they satisfy better estimates. Beside this, if we consider the supremum of random integrals of an appropriate class of functions, then these results can be applied only in very special cases. Moreover, the method of proof of [30] did not allow a real generalization of these results, hence I had to find a different approach when tried to generalize them.

I do not know of other works where the distribution of multiple random integrals with respect to a normalized empirical distribution is studied. On the other hand, there are some works where the distribution of (degenerate) U-statistics is investigated. The most important results obtained in this field are contained in the book of de la Peña and Giné Decoupling, From Dependence to Independence [7]. The problems about the behaviour of degenerate U-statistics and multiple integrals with respect to a normalized empirical distribution function are closely related, but the explanation of their relation is far from trivial. The main difference between them is that integration with respect to μ_{n} − μ instead of the empirical distribution μ_{n} means some sort of normalization, while this normalization is missing in the definition of U-statistics. I return to this question later.

The main part of this work starts at Section 3. A general overview of the results without the hard technical details can be found in [33].

First the estimation of sums of independent random variables or one-fold random integrals with respect to a normalized empirical distribution and the supremum of such expressions is investigated in Sections 3 and 4. This question has a fairly big literature. I would mention first of all the books A course on empirical processes [11], Real Analysis and Probability [12] and Uniform Central Limit Theorems [13] of R. M. Dudley. These books contain a much more detailed description of the empirical processes than the present work together with a lot of interesting results.

Section 3 deals with the tail behaviour of sums of independent and bounded random variables with expectation zero. The proof of two already classical results, Bernstein’s and Bennett’s inequalities is given there. (Their proofs can be found e.g. in Theorem 1.3.2 of [13] and [5]). We are also interested in the question when they give such an estimate which the central limit theorem suggests. Actually, as it is explained in Section 3, Bennett’s inequality gives a bound suggested by a Poissonian approximation of partial sums of independent random variables. Bernstein’s inequality provides an estimate suggested by the central limit theorem if the variance of the sum we consider is not too small. (The results in Section 3 explain this statement more explicitly.) If the variance of the sum is too small, then Bennett’s inequality provides a slight improvement of Bernstein’s inequality. Moreover, as Example 3.3 shows, Bennett’s inequality is essentially sharp in this case.

The estimate on the tail distribution of a sum of independent random variables is weak if this sum has a small variance. This means that in this case the probability that the sum is larger than a given value may be much larger than the (rather small) value suggested by the central limit theorem. Such a behaviour may occur, because the contribution of some unpleasant irregularities to this probability may be non-negligible in the case of a small variance.

In the study of the supremum of sums of independent random variables a good control is needed on the tail distribution of the (supremum of) sums of independent random variables even if they have small variance. The solution of this problem (and of its natural multivariate version) turned out to be the hardest part of this work. The results based on the similar behaviour of partial sums and their Gaussian counterpart is not sufficient in this case, some new ideas have to be applied. In the proof of sharp estimates in this case we also use some kind of symmetrization arguments. The last result of Section 3, Hoeffding’s inequality presented in Theorem 3.4 is an important ingredient of these symmetrization arguments. It is also a classical result whose proof can be found for instance in [23].

Section 4 contains the one-variate version of our main result about the supremum of the integrals of a class F of functions with respect to a normalized empirical measure together with an equivalent statement about the tail distribution of the supremum of a class of random sums defined with the help of a sequence of independent and identically distributed random variables and a class of functions F with some nice properties. These results are given in Theorems 4.1 and 4.1′. Also a Gaussian version of them is presented in Theorem 4.2 about the distribution of the supremum of a Gaussian random field with some appropriate properties. The content of these results can be so interpreted that if we take the supremum of random integrals or of random sums determined by a nice class of functions F in the way described in Section 4, then the tail distribution of this supremum satisfies an almost as good estimate as the ‘worst element’ of the random variables taking part in this supremum. I also discussed a result in Example 4.3 which shows that some rather technical conditions of Theorem 4.1 cannot be omitted.

The most important condition in Theorem 4.1 was that the class of functions F we considered in it is L_{2}-dense. This property was introduced before the formulation of this result. One may ask whether we can introduce different classes of functions in Theorem 4.1 and prove with their help better results. It is worth mentioning that Talagrand proved results similar to Theorem 4.1 for different classes of functions F in his book [52]. These classes of functions are very different of ours, and Talagrand’s results seem to be incomparable with ours. I return to this question later.

In the above mentioned results we have imposed the condition that the class of functions F or what is equivalent, the set of random variables whose supremum we estimate is countable. In the proofs this condition is really exploited. On the other hand, in some important applications we also need results about the supremum of a possibly non-countable set of random variables. To handle such cases I introduced the notion of countably approximable classes of random variables and proved that in the results of this work the condition about countability can be replaced by the weaker condition that the supremum of countably approximable classes is taken. R. M. Dudley worked out a different method to handle the supremum of possibly non-countably many random variables, and generally his method is applied in the literature. The relation between these two methods deserves some discussion.

Let us first recall that if a class of random variables S_{t}, t ∈ T, indexed by some index set T is given, and a set A is measurable with respect to the σ-algebra generated by the random variables S_{t}, t ∈ T, then there exists a countable subset T′ = T′(A) ⊂ T such that the set A is measurable also with respect to the smaller σ-algebra generated by the random variable S_{t}, t ∈ T′. Beside this, if the finite dimensional distributions of the random variables S_{t}, t ∈ T, are given, then by the results of classical measure theory the probability of the events measurable with respect to the σ-algebra generated by these random variables S_{t}, t ∈ T, is also determined. But we cannot get the probability of all events we are interested in such a way. In particular, if T is a non-countable set, then

the events {ω: sup_{t∈T} S_{t}(ω) > u} are non-measurable with respect to the above σ-algebra,

and generally we cannot speak of their probabilities. To overcome this difficulty Dudley worked out a theory which enabled him to work also with outer measures. His theory is based on some rather deep results of the analysis. It can be found for instance in his book [13].

I restricted my attention to such cases when after the completion of the probability measure P we can also speak of the real (and not only outer) probabilities

P (sup_{t∈T} S_{t} > u). I tried to find appropriate conditions under which these probabilities

really exist. More explicitly, we are interested in the case when for all u > 0 there exists some set A = A_{u} measurable with respect to the σ-algebra generated by the random variables S_{t}, t ∈ T, such that the symmetric difference of the sets A_{u} and

{ω: sup_{t∈T} S_{t}(ω) > u} is contained in a set measurable with respect to the σ-algebra gen-

erated by the random variables S_{t}, t ∈ T, which has probability zero. In such a case

the probability P (sup_{t∈T} S_{t} > u) can be defined as P (A_{u}). This approach led me to the

definition of countable approximable classes of random variables. If this property holds, then we can speak about the probability of the event that the supremum of the random variables we are interested in is larger than some fixed value. I proved a simple but useful result in Lemma 4.4 which provides a condition for the validity of this property. In Lemma 4.5 I proved with its help that an important class of functions is countably approximable. It seems that this property can be proved for many other interesting classes of functions with the help of Lemma 4.4, but I did not investigate this question in more detail.

The problem we met here is not an abstract, technical difficulty. Indeed, the distribution of such a supremum can become different if we modify each random variable on a set of probability zero, although the finite dimensional distributions of the random variables we consider remain the same after such an operation. Hence, if we are interested in the probability of the supremum of a non-countable set of random variables with described finite dimensional distributions we have to describe more explicitly which version of this set of random variables we consider. It is natural to look for such an appropriate version of the random field S_{t}, t ∈ T, whose ‘trajectories’ S_{t}(ω), t ∈ T, have nice properties for all elementary events ω ∈ Ω. Lemma 4.4 can be interpreted as a result in this spirit. The condition given for the countable approximability of a class of random variables at the end of this lemma can be considered as a smoothness condition about the ‘trajectories’ of the random field we consider. This approach shows some analogy to some important problems in the theory of stochastic processes when a regular version of a stochastic process is considered and the smoothness properties of its trajectories are investigated.

In our problems the version of the set of random variables S_{t}, t ∈ T, we shall work with appears in a simple and natural way. In these problems we have finitely many random variables ξ_{1}, …, ξ_{n} at the start, and all random variables S_{t}(ω), t ∈ T, we are considering can be defined individually for each ω as a functional of these random variables ξ_{1}(ω), …, ξ_{n}(ω). We take the version of the random field S_{t}(ω), t ∈ T, we get in such a way and want to show that it is countably approximable. In Section 4 this property is proved in an important model, probably in the most important model in possible applications we are interested in. In more complicated situations when our random variables are defined not as a functional of finitely many sample points, for instance in the case when we define our set of random variables by means of integrals with respect to a Gaussian random field it is harder to find the right regular version of our sets of random variables. In this case the integrals we consider are defined only with probability 1, and it demands some extra work to find their right version. But in the problems we study in this work our approach is satisfactory for our purposes, and it is simpler than that of Dudley; we do not have to follow his rather difficult technique. On the other hand, I must admit that I do not know the precise relation between the approach of this work and that of Dudley.

In Section 4 the notion of L_{p}-dense classes, 1 ≤ p < ∞, also has been introduced. The notion of L_{2}-dense classes appeared in the formulation Theorems 4.1 and 4.1′. It can be considered as a version of the ε-entropy, discussed at many places in the literature. On the other hand, there seems to be no standard definition of the ε-entropy. The term of L_{2}-dense classes seemed to be the appropriate object to work with in this work. To apply the results related to L_{2}-dense classes we also need some knowledge about how to check this property in concrete models. For this goal I discussed here Vapnik– Červonenkis classes, a popular and important notion of modern probability theory. Several books and papers, (see e.g. the books [13], [44], [53] and the references in them) deal with this subject. An important result in this field is Sauer’s lemma, (Lemma 5.1) which together with some other results, like Lemma 5.3 imply that several interesting classes of sets or functions are Vapnik–Červonenkis classes.

I put the proof of these results to the Appendix, partly because they can be found in the literature, partly because in this work Vapnik–Červonenkis classes play a different and less important role than at other places. Here Vapnik–Červonenkis classes are applied to show that certain classes of functions are L_{2}-dense. A result of Dudley formulated in Lemma 5.2 implies that a Vapnik–Červonenkis class of functions with absolute value bounded by a fixed constant is an L_{1}, and as a consequence, also an L_{2}-dense class of functions. The proof of this important result which seems to be less known even among experts of this subject than it would deserve is contained in the main text. Dudley’s original result was formulated in the special case when the functions we consider are indicator functions of some sets. But its proof contains all important ideas needed in the proof of Lemma 5.2.

Theorem 4.2, which is the Gaussian counterpart of Theorems 4.1 and 4.1′ is proved in Section 6 by means of a natural and important technique, called the chaining argument. This means the application of an inductive procedure, in which an appropriate sequence of finite subsets of the original set of random variables is introduced, and a good estimate is given on the supremum of the random variables in these subsets by means of an inductive procedure. The subsets became denser subsets of the original set of the random variables at each step of this procedure. This chaining argument is a popular method in certain investigation. It is hard to say with whom to attach it. Its introduction may be connected to some works of R. M. Dudley. It is worth mentioning that Talagrand [52] worked out a sharpened version of it which yields in the study of certain problems a sharper and more useful estimate. But it seems to me that in the study of the problems of this work it does not provide a real improvement.

Theorem 4.2 can be proved by means of the chaining argument, but this method is not strong enough to supply a proof of Theorem 4.1. The chaining argument provides only a weak estimate in this case, because there is no good estimate on the probability that a sum of independent random variables is greater than a prescribed value if these random variables have too small variances. As a consequence the chaining argument supplies a much weaker estimate than the result we want to prove under the conditions of Theorem 4.1. Lemma 6.1 contains the result the chaining argument yields under these conditions. In Section 6 still another result, Lemma 6.2 is formulated. It can be considered as a special case of Theorem 4.1 where only the supremum of partial sums with small variances is estimated. It is also shown that Lemmas 6.1 and 6.2 together imply Theorem 4.1. The proof is not difficult, despite of some non-attractive details. It has to be checked that the parameters in Lemmas 6.1 and 6.2 can be fitted to each other.

Lemma 6.2 is proved in Section 7. It is based on a symmetrization argument. This proof applies the ideas of a paper of Kenneth Alexander [2], and although its presentation is different from Alexander’s approach, it can be considered as a version of his proof.

A similar problem should also be mentioned at this place. M. Talagrand wrote a series of papers about concentration inequalities, (see e.g. [50] or [51]), and his research was also continued by some other authors. I would mention the works of M. Ledoux [27] and P. Massart [41]. Concentration inequalities give a bound about the difference between the supremum of a set of appropriately defined random variables and the expected value of this supremum. They express how strongly this supremum is concentrated around its expected value. Such results are closely related to Theorem 4.1, and the discussion of their relation deserves some attention. A typical concentration inequality is the following result of Talagrand [51].

Theorem 18.1. (Theorem of Talagrand). Consider n independent and identically distributed random variables ξ_{1}, …, ξ_{n} with values in some measurable space (X, X). Let F be some countable family of real-valued measurable functions of (X, X) such that

∥f∥_{∞} ≤ b < ∞ for every f ∈ F. Let Z = sup_{f∈F} ∑_{i=1}^{n} f(ξ_{i}) and v = E(sup_{f∈F} ∑_{i=1}^{n} f^{2}(ξ_{i})).

Then for every positive number x,

P (Z ≥ EZ + x) ≤ K exp \{−\frac{1}{K′} \frac{x}{b} log (1 + \frac{xb}{v})\}

and

P (Z ≥ EZ + x) ≤ K exp \{−\frac{x^{2}}{2(c_{1}v + c_{2}bx)}\} ,

where K, K′, c_{1} and c_{2} are universal positive constants. Moreover, the same inequalities hold when replacing Z by −Z.

Theorem 18.1 yields, similarly to Theorem 4.1, an estimate about the distribution of the supremum for a class of sums of independent random variables. It can be considered as a generalization of Bernstein’s and Bennett’s inequalities when the distribution of the supremum of partial sums (and not only the distribution of one partial sum) is estimated. A remarkable feature of this result is that it assumes no condition about the structure of the class of functions F (like the condition of L_{2}-dense property of the class F imposed in Theorem 4.1.) On the other hand, the estimates in Theorem 18.1

contain the quantity EZ = E (sup_{f∈F} ∑_{i=1}^{n} f(ξ_{i})). Such an expectation of some supremum

appears in all concentration inequalities. As a consequence, they are useful only if we can bound the expected value of the supremum we want to estimate. This is a hard question in the general case. There is a paper [16] which provides a useful estimate about the expected value of the supremum of random sums under the conditions of Theorem 4.1. But I preferred a direct proof of this result. Let me remark that because of the above mentioned concentration inequality the condition u ≥ const. σ log^{1/2} \frac{2}{σ} with some appropriate constant which cannot be dropped from Theorem 4.1 can be interpreted so that under the conditions of Theorem 4.1 const. σ log^{1/2} \frac{2}{σ} is an upper bound for the expected value of the supremum we are studying.

It is also worth mentioning Talagrand’s work [52] which contains several interesting results similar to Theorem 4.1. But despite their formal similarity, they are essentially different from the results of this work. This difference deserves some special discussion.

Talagrand proved in [52] by working out a more refined, better version of the chaining argument a sharp upper bound for the expected value E sup_{t∈T} ξ_{t} of the supremum of

countably many (jointly) Gaussian random variable with zero expectation. This result is sharp. Indeed, Talagrand proved also a lower bound for this expected value, and the proportion of his upper and lower bound is bounded by a universal constant. He also gave

by applying similar arguments an upper bound for E sup_{f∈F} ∑_{k=1}^{N} f(ξ_{k}) in Proposition 2.7.2

of his book, where ξ_{1}, …, ξ_{N} is a sequence of independent, identically distributed random variables with some known distribution μ, and F is a class of functions with some nice properties. Then he proved in Chapter 3 some estimates with the help of this result for certain models which solved some problems that could not be solved with the help of the original version of the chaining argument.

Talagrand investigated in his book [52] the expected value of the supremum of partial sums, while we gave an estimate on its tail distribution. But this is not a great difference. Talagrand’s results also give an estimate on the tail distribution of the supremum by means of concentration inequalities, and actually his proofs also provide a direct estimate for the tail distribution we are interested in without the application of these results. The main difference between the two works is that Talagrand’s method gives a sharp estimate for different classes of functions F.

Talagrand could prove sharp results in such cases when the class of functions F for which the supremum is taken consists of smooth functions. An example for such classes of function which he thoroughly investigated is the class of Lipschitz 1 functions. On the other hand we can give sharp results in such cases when F consists of non-smooth functions. (See Example 5.5.)

This difference in the conditions of the results in these two books is not a small technical detail. Talagrand exploited in his proof that he worked with such classes of functions F from which he could select such a subclass of functions of relatively small cardinality which is dense in F in the supremum norm. He needed this property, because this enabled him to get sharp estimates on the tail distribution of the differences of functions he had to work with by means of the Bernstein’s inequality. He needed such estimates to apply (a refined version of) the chaining argument. On the other hand, we considered such classes of functions F which may have no small subclasses which are dense in F in the supremum norm. I would characterize the difference between the results of the two works in the following way. Talagrand proved the sharpest possible estimates which can be obtained by a refinement of the chaining argument, while our main problem was to get sharp estimates also in such cases when the chaining argument does not work.

The main results of this work are presented in Section 8. A weaker version of Theorem 8.3 about an estimate of the distribution of a degenerate U-statistic was first proved in a paper of Arcones and Giné in [3]. The result of Theorem 8.3 in the present form is proved in my paper [36]. Its version about multiple integrals with respect to a normalized empirical measure formulated in Theorem 8.1 is proved in [32]. This paper contains a direct proof. On the other hand, Theorem 8.1 can be derived from Theorem 8.3 by means of Theorem 9.4 of this paper. Theorem 8.5 is the natural Gaussian counterpart of Theorem 8.3. The limit theorem about degenerate U-statistics, Theorem 10.4 (and its version about limit theorems for multiple integrals with respect to normalized empirical measures, presented in Theorem 10.4′ of Appendix C) was discussed in this work to explain better the relation between degenerate U-statistics (or multiple integrals with respect to normalized empirical measures) and multiple Wiener– Itô integrals. A proof of this result based on similar ideas as that discussed here can be found in [14]. Theorem 6.6 of my lecture note [29] contains such a weakened version of Theorem 8.5 which does not take into account the variance of the random integral.

Example 8.7 is a natural supplement of Theorem 8.5. It shows that the estimate of Theorem 8.5 is sharp if only the variance of a Wiener–Itô integral is known. At the end of Section 13 I also mentioned the results of papers [1] and [26] without proof which also have some relation to this problem. I discussed mainly the content of [26] and explained its relation to some results discussed in this work. The proof of these papers apply a method different of those of this work. It would be interesting to prove them with the methods discussed here. These papers contain such a refinement of Theorems 8.5 and 8.3 respectively whose estimates depend on some other rather complicated quantities. In some cases they supply a better estimate. On the other hand, in the problems discussed here they have a restricted importance because their conditions are hard to check.

Theorems 8.2 and 8.4 yield an estimate about the supremum of (degenerate) Ustatistics or of multiple random integrals with respect to a normalized empirical measure when the class of kernel functions in these U-statistics or random integrals satisfy some conditions. They were proved in my paper [34]. Earlier Arcones and Giné proved a weaker form of this result in paper [4], but their work did not help in the proof of the results of this note. They were based on an adaptation of Alexander’s method to the multivariate case. Theorem 8.6 contains the natural Gaussian counterpart of Theorems 8.2 and 8.4.

Example 8.8 in Section 8 shows that the condition u ≤ const. nσ^{3} imposed in Theorem 8.3 in the case k = 2 cannot be dropped. The paper of Arcones and Giné [3] contains another example explained by Talagrand to the authors of that paper which also has a similar consequence. But that example does not provide such an explicit comparison of the upper and lower bound on the probability investigated in Theorem 8.3 as Example 8.8. Similar examples could be constructed for all k ≥ 1.

Example 8.8 shows that at high levels only a very weak (and from practical point of view not really important) improvement of the estimation on the tail distribution of degenerate U-statistics is possible. But probably there exists a multivariate version of Bennett’s inequality, i.e. of Theorem 3.2 which provides such an estimate. Moreover, there is some hope to get a similar strengthened form of Theorems 8.2 and 8.4 (or of Theorem 4.2 in the one-dimensional case). This question is not investigated in the present work.

Section 9 deals with the properties of U-statistics. Its first result, Theorem 9.1, is a rather classical result. It is the so-called Hoeffding decomposition of U-statistics to the sum of degenerate statistics. Its proof first appeared in the paper [22], but it can be found at many places. The explanation of this work contains some ideas similar to [49]. I tried to explain that Hoeffding’s decomposition is the natural multivariate version of the (trivial) decomposition of sums of independent random variables to sums of independent random variables with expectation zero plus the sum of the expectations of the original random variables. Moreover, even the proof of the Hoeffding’s decomposition shows some similarity to this simple decomposition.

Theorem 9.2 and Proposition 9.3 can be considered as a continuation of the investigation of the Hoeffding’s decomposition in Theorem 9.1. They tell how the properties of the kernel function of the original U-statistic are inherited in the properties of the kernel functions of the degenerate U-statistics taking part in its Hoeffding decomposition. In several applications of Hoeffding’s decomposition we need such results.

The last result of Section 9, Theorem 9.4, enables us to reduce the estimation of multiple random integrals with respect to normalized empirical measures to the estimation of degenerate U-statistics. This result is a version of Hoeffding’s decomposition, where multiple integrals with respect to a normalized empirical distribution are decomposed to the sum of degenerate U-statistics. Multiple random integrals with respect to a normalized empirical measure can be simply written as sums of U-statistics, and by applying the Hoeffding decomposition for each term of these sums we get the desired decomposition. Theorem 9.4 yields the result we get in such a way. This formula is very similar to the original Hoeffding decomposition. The main difference between them is that the coefficients of the degenerate U-statistics in the decomposition of Theorem 9.4 are relatively small. The cancellation effect caused by integration with respect to a normalized empirical measure is reflected in the appearance of small coefficients in the decomposition given in Theorem 9.4. Theorem 9.4 was proved in [34]. The same proof is given in this note, but some calculations are worked out in more detail.

Theorem 8.1 can be derived from Theorem 8.3 and Theorem 8.2 from Theorem 8.4 by means of Theorem 9.4. The proof of the latter results is simpler. The results of Sections 10–12 contain the results needed in the proof of Theorem 8.3 and its Gaussian counterpart Theorems 8.5 and 8.7. The proof of these results is based on good estimates of high moments of degenerate U-statistics and multiple Wiener–Itô integrals. The classical proof of the one-variate counterparts of these results is based on a good estimate of the moment generating function. This method was replaced by the estimate of high moments, because the moment generating function of a k-fold Wiener–Itô integral is divergent for k ≥ 3, and this property is also reflected in the behaviour of degenerate Ustatistics. On the other hand, good estimates on high moments can replace the estimate of the moment generating function. A good estimate can be given for all moments of a Wiener–Itô integral, while we have a good estimate only on not too high moments of degenerate U-statistics. This has the consequence that we can give a good estimate on the tail distribution of degenerate U-statistic only for not too large values.

I know of two deep methods to study high moments of multiple Wiener–Itô integrals. Both of them can be adapted to the study of the moments of degenerate U-statistics. They deserve a more detailed discussion.

The first one of them is called Nelson’s inequality named after Edward Nelson who published it in his paper [43]. This inequality simply implies Theorem 8.5 about multiple Wiener–Itô integrals, although with worse constants. Later Leonhard Gross discovered a deep and useful generalization of this result which he published in the work Logarithmic Sobolev inequalities [19]. In that paper Gross compared two Markov processes with the same infinitesimal operator but with possibly different initial distribution, where the second Markov process had stationary distribution. He could give a sharp bound on the Radon–Nikodym derivative of the distribution of the first Markov process with respect to the (stationary) distribution of the second Markov process at all time T on the basis of the properties of the infinitesimal operator of the Markov processes. With the help of this result he could prove a more general form of Nelson’s inequality. In particular, his result may help to prove (a weaker version of) Theorem 8.3 (with worse universal constants). Let me also remark that Gross’ method works not only in the study of these problems, but in several hard problems of the probability theory. (See e.g [20] or [27]). Nevertheless, in the present note I applied a different method, because this seemed to be more appropriate here.

I applied a method related to the names of Kyoshi Itô and Roland L’vovich Dobrushin. This is the theory of multiple Wiener–Itô integrals with respect to a white noise. This integral was introduced in paper [24]. It is useful, because every random variable measurable with respect to the σ-algebra generated by the Gaussian random variables of the underlying white noise with finite second moment can be written as the sum of Wiener–Itô integrals of different order. Moreover, if only Wiener–Itô integrals of symmetric kernel functions are taken, then this representation is unique. An important result, the so-called diagram formula, formulated in Theorem 10.2, expresses products of Wiener–Itô integrals as a sum of such integrals. This result which shows some similarity to the Feynman diagrams applied in the statistical physics was proved in [9]. Actually this paper discussed a modified version of Wiener–Itô integrals which is more appropriate to study the action of shift operators for non-linear functionals of a stationary Gaussian field. But these modified Wiener–Itô integrals can be investigated in almost the same way as the original ones. The diagram formula has a simple consequence formulated in Corollary of Theorem 10.2 of this note. It enables us to calculate the expectation of products of Wiener–Itô integrals, in particular it yields an explicit formula about the moments of a Wiener–Itô integral. This result was applied in the proof of Theorem 8.5, i.e. in the estimation of the tail-distribution of Wiener–Itô integrals. Itô’s formula for multiple Wiener–Itô integrals (Theorem 10.3) was proved in [24].

The diagram formula has a natural and useful analog both for degenerate Ustatistics and multiple integrals with respect to a normalized empirical measure. They enable us to express the product of degenerate U-statistics and multiple integrals as the sum of such expressions. These results enable us to adapt several useful methods in the study of non-linear functionals of a Gaussian random field to the study of nonlinear functionals of normalized empirical measures. A version of the diagram formula was proved for degenerate U-statistics in [36] and for multiple random integrals with respect to a normalized empirical measures in [32]. Let me remark that in the formulation of the result in the work [36] a different notation was applied than in the present note. In that paper I wanted to formulate version of the diagram formula for U-statistics with the help of such diagrams which appear in the classical form of diagram formula presented for Wiener–Itô integrals. I could do this only in a somewhat artificial way. In this work I formulated this result by introducing first more general diagrams which may contain some chains. The formulation of the result with the help of such more general diagrams seems to be more natural. Let me also remark that the study of results similar to the diagram formula for Wiener–Itô integrals did not get such an attention in the literature as it would deserve in my opinion. I know only of one work where such questions were investigated. It is the paper of Surgailis [46], where a version of the diagram formula is proved for Poissonian integrals. The Corollary of Theorem 11.2 is of special interest for us, because it enables us to prove such moment estimates which are useful in the proof of Theorem 8.3.

It is worth mentioning that the problems about Wiener–Itô integrals are closely related to the study of Hermite polynomials or to their multivariate version, to the socalled Wick polynomials. (See e.g. [29] or [40] for the definition of Wick polynomials.) Appendix C contains the most important properties of Hermite polynomials needed in the study of Wiener–Itô integrals. In particular, it contains the proof of Proposition C2 which states that the set of all Hermite polynomials is a complete orthogonal system in the Hilbert space of the functions square integrable with respect to the Gaussian distribution. This result can be found for instance in Theorem 5.2.7 of [48]. In the present proof I wanted to show that this result is closely related to the so-called moment problem, i.e. to the question when a distribution is determined by its moments uniquely. This method, with some refinement, can be applied to prove some generalizations of Proposition C2 about the completeness of orthogonal polynomials with respect to more general weight functions.

Itô’s formula creates a relation between Wiener–Itô integrals and Hermite polynomials. The results about multiple Wiener–Itô integrals have their analogs for Wick polynomials. Thus for instance there is a diagram formula for the product of Wick polynomials which also has some interesting generalizations. Such questions are studied both in probability theory and statistical physics, see [40] and [45]. The relation between Wiener–Itô integrals and Hermite polynomials also has a natural counterpart in the study of other multiple random integrals. The so-called Appell polynomials, (see [47]), appeared in such a way.

Theorems 8.3, 8.5 and 8.7 were proved on the basis of the results in Sections 10– 12 and in Section 13. Section 13 also contains the proof of a multivariate version of Hoeffding’s inequality, formulated in Theorem 13.3. This result is needed in the symmetrization argument applied in the proof of Theorem 8.4. A weaker version of it (an estimate with a worse constant in the exponent) which would be satisfactory for our purposes would simply follow from a classical result, called Borell’s inequality. But since this result is not discussed in this note, and I was interested in a proof which yields the best estimate in the exponent of this estimate I have chosen another proof, given in [35] which is based on the results of Sections 10–12. Later I have learned that this estimate is contained in an implicit form also in the paper [6] of A. Bonami.

Sections 14–17 are devoted to the proof of Theorems 8.4 and 8.6. They are based on a similar argument as their one-variate counterparts, Theorems 4.1 and 4.2. The proof of Theorem 8.6 about the supremum of Wiener–Itô integrals is based, similarly to the proof of Theorem 4.2 on the chaining argument. In the proof of Theorem 8.4, the chaining argument yields only a weaker result formulated in Proposition 14.1 which helps to reduce Theorem 8.4 to the proof of Proposition 14.2. In the one-variate case a similar approach was applied. In that case the proof of Theorem 4.1 was reduced to that of Proposition 6.2 by means of Proposition 6.1. The next step in the proof of Theorem 8.6 has no one-variate counterpart. The notion of so-called decoupled Ustatistics was introduced, and Proposition 14.2 was reduced to a similar result about decoupled U-statistics formulated in Proposition 14.2′.

The adjective ‘decoupled’ in the expression decoupled U-statistic refers to the fact that it is such a version of a U-statistic where independent copies of a sequence of independent and identically distributed random variables are put into different coordinates of the kernel function of the U-statistic. Their study is a popular subject of some mathematicians. In particular, the main subject of the book [7] is a comparison of the properties of U-statistics and decoupled U-statistics. A result of de la Peña and Montgomery–Smith [8] formulated in Theorem 14.3 helps in reducing some problems about U-statistics to a similar problem about decoupled U-statistics. In this lecture note the proof of Theorem 14.3 is given in Appendix D. It follows the argument of the original proof, but several steps are worked out in detail where the authors gave only a very short explanation. Paper [8] also contains some kind of converse results to Theorem 14.3, but as they are not needed in the present work, I omitted their discussion.

Decoupled U-statistics behave similarly to the original U-statistics. Beside this, some symmetrization arguments becomes considerably simpler if we are working with decoupled U-statistics instead of the original U-statistics. This can be exploited in some investigations. For example the proof of Proposition 14.2′ is simpler than a direct proof of Proposition 14.2. On the other hand, Theorem 14.3 enables us to reduce the proof of Proposition 14.2 to that of Proposition 14.2′, and we have exploited this possibility.

The proof of Theorem 8.4 was reduced to that of Proposition 14.2′ in Section 14. Sections 15–17 deal with the proof of this result. It was proved in my paper [34]. The proof is similar to that of its one-variate version, Proposition 6.2, but some additional difficulties have to be overcome. The main difficulty appears when we want to find the multivariate analog of the symmetrization argument which could be carried out by means of the Symmetrization Lemma, Lemma 7.1 and Lemma 7.2 in the one-variate case.

In the multivariate case Lemma 7.1 is not sufficient for us. We work instead of it with a generalized version of this result, formulated in Lemma 15.2. The proof of Lemma 15.2 is not hard. The real difficulty arises when we want to apply it in the proof of Proposition 14.2′. We have to check formula (15.3), and this means in this case a non-trivial estimation of some complicated conditional probabilities. This is the hardest part of the proof of Proposition 14.2′.

Proposition 14.2′ was proved by means of an inductive procedure formulated in Proposition 15.3, which is the multivariate analog of Proposition 7.3. A basic ingredient of both proofs was a symmetrization argument. But while this symmetrization argument could be simply carried out in the one-variate case, its adaptation to the multivariate case in the proof of Theorem 15.3 was a most serious problem. To overcome this difficulty another result was formulated in Proposition 15.4. Propositions 15.3 and 15.4 were proved simultaneously by means of an appropriate inductive procedure. Their proofs were based on a refinement of the arguments in the proof of Proposition 7.3. We also had to apply Theorem 13.3, a multivariate version of Hoeffding’s inequality, and some properties of the Hoeffding decomposition of U-statistics proved in Section 9.

Appendix A.

The proof of some results about Vapnik–Červonenkis classes.

Proof of Theorem 5.1. (Sauer’s lemma). This result has several different proofs. Here I write down a relatively simple proof of P. Frankl and J. Pach which appeared in [15]. It is based on some linear algebraic arguments.

The following equivalent reformulation of Sauer’s lemma will be proved. Let us take a set S = S(n) consisting of n elements and a class E of subsets of S consisting of m elements E_{1}, …, E_{m} ⊂ S. Assume that m ≥ m_{0} + 1 with m_{0} = m_{0}(n, k) = \binom{n}{0} + \binom{n}{1} + ⋯ + \binom{n}{k−1}. Then there exists a set F ⊂ S of cardinality k which the class of sets E shatters. Actually, it is enough to show that there exists a set F of cardinality greater than or equal to k which the class of sets E shatters, because if a set has this property, then all of its subsets have it. This latter statement will be proved.

To prove this statement let us first list the subsets X_{0}, …, X_{m_{0}} of the set S of cardinality less than or equal to k − 1, and correspond to all sets E_{i} ∈ E the vector e_{i} = (e_{i,1}, …, e_{i,m_{0}}), 1 ≤ i ≤ m, with elements

e_{i,j} = \{ \begin{matrix}1 & if X_{j} ⊆ E_{i} \\ 0 & if X_{j} ⊈ E_{i} \end{matrix} 1 ≤ i ≤ m, and 1 ≤ j ≤ m_{0}.

Since m > m_{0}, the vectors e_{1}, …, e_{m} are linearly dependent. Because of the definition of the vectors e_{i}, 1 ≤ i ≤ m, this can be expressed in the following way: There is a non-zero vector (f(E_{1}), …, f(E_{m})) such that

∑_{E_{i}: E_{i}⊇X_{j}} f(E_{i}) = 0 for all 1 ≤ j ≤ m_{0}.  (A1)

Let F, F ⊂ S, be a minimal set with the property

∑_{E_{i}: E_{i}⊇F} f(E_{i}) = α ≠ 0.  (A2)

Such a set F really exists, since every maximal element of the family {E_{i}: 1 ≤ i ≤ m, f(E_{i}) ≠ 0} satisfies relation (A2). The requirement that F should be a minimal set means that if F is replaced by some H ⊂ F, H ≠ F, at the left-hand side of (A2), then this expression equals zero. The inequality |F | ≥ k holds because of relation (A1) and the definition of the sets X_{j}.

Introduce the quantities

Z_{F} (H) = ∑_{E_{i}: E_{i}∩F =H} f(E_{i})

for all H ⊆ F.

Then Z_{F} (F) = α, and for any set of the form H = F \ {x}, x ∈ F,

Z_{F} (H) = ∑_{E_{i}: E_{i}∩F =H} f(E_{i}) = ∑_{E_{i}: E_{i}⊇H} f(E_{i}) − ∑_{E_{i}: E_{i}⊇F} f(E_{i}) = 0 − α = −α

because of the minimality property of the set F.

Moreover, the identity

Z_{F} (H) = (−1)^{p}α for all H ⊆ F such that |H| = |F | − p, 0 ≤ p ≤ |F |.  (A3)

holds. To show relation (A3) observe that

Z_{F} (H) = ∑_{E_{i}: E_{i}∩F =H} f(E_{i}) = ∑_{j=0}^{p}(−1)^{j} ∑_{G: H⊂G⊂F, |G|=|H|+j} ∑_{E_{i}: E_{i}⊇G} f(E_{i}) (A4)

for all sets H ⊂ F with cardinality |H| = |F | − p. Identity (A4) holds, since the term

f(E_{i}) is counted at the right-hand side of (A4) ∑_{j=0}^{l}(−1)^{j}\binom{l}{j} = (1 − 1)^{l} = 0 times if

E_{i} ∩ F = G with some H ⊂ G ⊆ F with |G| = |H| + l elements, 1 ≤ l ≤ p, while in the case E_{i} ∩ F = H it is counted once. Relation (A4) together with (A2) and the minimality property of the set F imply relation (A3).

It follows from relation (A3) and the definition of the function Z_{F} (H) that for all sets H ⊆ F there exists some set E_{i} such that H = E_{i} ∩ F, i.e. F is shattered by E. Since |F | ≥ k, this implies Theorem 5.1.

Proof of Theorem 5.3. Let us fix an arbitrary set F = {x_{1}, …, x_{k+1}} of the set X, and consider the set of vectors G_{k}(F) = {(g(x_{1}), …, g(x_{k+1})): g ∈ G_{k}} of the k + 1dimensional space R^{k+1}. By the conditions of Theorem 5.3 G_{k}(F) is an at most kdimensional subspace of R^{k+1}. Hence there exists a non-zero vector a = (a_{1}, …, a_{k+1})

such that ∑_{j=1}^{k+1} a_{j}g(x_{j}) = 0 for all g ∈ G_{k}. We may assume that the set A = A(a) =

{j: a_{j} < 0, 1 ≤ j ≤ k + 1} is non-empty, by multiplying the vector a by −1 if it is necessary.

Thus the identity

∑_{j∈A} a_{j}g(x_{j}) = ∑_{j∈{1,...,k+1}\A}(−a_{j})g(x_{j}),  for all g ∈ G_{k}  (A5)

holds. Put B = {x_{j}: j ∈ A}. Then B ⊂ F, and F \ B ≠ {x: g(x) ≥ 0} ∩ F for all g ∈ G_{k}. Indeed, if there were some g ∈ G_{k} such that F \ B = {x: g(x) ≥ 0} ∩ F, then the left-hand side of the equation (A5) would be strictly positive (as a_{j} < 0, g(x_{j}) < 0 if j ∈ A, and A ≠ ∅) its right-hand side would be non-positive for this g ∈ G_{k}, and this is a contradiction.

The above proved property means that D shatters no set F ⊂ X of cardinality k+1. Hence Theorem 5.1 implies that D is a Vapnik–Červonenkis class.

Appendix B. The proof of the diagram formula for Wiener–Itô integrals.

The proof of Theorem (10.2A) (the diagram formula for the product of two Wiener– Itô integrals) will be started with the proof of inequality (10.11). To show that this relation holds let us observe that the Cauchy inequality yields the following bound on the function F_{γ} defined in (10.10) (with the notation introduced there):

\begin{matrix}F_{γ}^{2}(x_{(1,j)}, x_{(2,j′)}, (1, j) ∈ V_{1}(γ), (2, j′) ∈ V_{2}(γ)) \\ ≤ ∫ f^{2}(x_{α_{γ}(1,1)}, …, x_{α_{γ}(1,k)}) ∏_{(2,j)∈\{(2,1),...,(2,l)\}\V_{2}(γ)} μ( dx_{(2,j)}) \\ ∫ g^{2}(x_{(2,1)}, …, x_{(2,l)}) ∏_{(2,j)∈\{(2,1),...,(2,l)\}\V_{2}(γ)} μ( dx_{(2,j)}). \end{matrix}  (B1)

The expression at the right-hand side of inequality (B1) is the product of two functions with different arguments. The first function has arguments x_{(1,j)} with (1, j) ∈ V_{1}(γ) and the second one x_{(2,j′)} with (2, j′) ∈ V_{2}(γ). By integrating both sides of inequality (B1) with respect to these arguments we get inequality (10.11).

Relation (10.12) will be proved first for the product of the Wiener–Itô integrals of two elementary functions. Let us consider two (elementary) functions f(x_{1}, …, x_{k}) and g(x_{1}, …, x_{l}) given in the following form: Let some disjoint sets A_{1}, …, A_{M}, μ(A_{s}) < ∞, 1 ≤ s ≤ M, be given together with some real numbers c(s_{1}, …, s_{k}) indexed with such k-tuples (s_{1}, …, s_{k}), 1 ≤ s_{j} ≤ M, 1 ≤ j ≤ k, for which the numbers s_{1}, …, s_{k} in a ktuple are all different. Put f(x_{1}, …, x_{k}) = c(s_{1}, …, s_{k}) on the rectangles A_{s_{1}} ×⋯×A_{s_{k}} with edges A_{s}, indexed with the above k-tuples, and let f(x_{1}, …, x_{k}) = 0 outside of these rectangles. Take similarly some disjoint sets B_{1}, …, B_{M′}, μ(B_{t}) < ∞, 1 ≤ t ≤ M′, and some real numbers d(t_{1}, …, t_{l}), indexed with such l-tuples (t_{1}, …, t_{l}), 1 ≤ t_{j′} ≤ M′, 1 ≤ j′ ≤ l, for which the numbers t_{1}, …, t_{l} in an l-tuple are different. Put g(x_{1}, …, x_{l}) = d(t_{1}, …, t_{l}) on the rectangles B_{t_{1}} × ⋯ × B_{t_{l}} with edges indexed with the above introduced l-tuples, and let g(x_{1}, …, x_{l}) = 0 outside of these rectangles.

Let us take some small number ε > 0 and rewrite the above introduced functions f(x_{1}, …, x_{k}) and g(x_{1}, …, x_{l}) with the help of this number ε > 0 in the following way.

Divide the sets A_{1}, …, A_{M} to smaller sets A_{1}^{ε}, …, A_{M(ε)}^{ε}, ⋃_{s=1}^{M(ε)} A_{s}^{ε} = ⋃_{s=1}^{M} A_{s}, in such a

way that all sets A_{1}^{ε}, …, A_{M(ε)}^{ε} are disjoint, and μ(A_{s}^{ε}) ≤ ε, 1 ≤ s ≤ M(ε). Similarly,

take sets B_{1}^{ε}, …, B_{M′(ε)}^{ε}, ⋃_{t=1}^{M′(ε)} B_{t}^{ε} = ⋃_{t=1}^{M′} B_{t}, in such a way that all sets B_{1}^{ε}, …, B_{M′(ε)}^{ε} are

disjoint, and μ(B_{t}^{ε}) ≤ ε, 1 ≤ t ≤ M′(ε). Beside this, let us also demand that two sets A_{s}^{ε} and B_{t}^{ε}, 1 ≤ s ≤ M(ε), 1 ≤ t ≤ M′(ε), are either disjoint or they agree. Such a partition exists because of the non-atomic property of measure μ. The above defined functions f(x_{1}, …, x_{k}) and g(x_{1}, …, x_{l}) can be rewritten by means of these new sets A_{s}^{ε} and B_{t}^{ε}. Namely, let f(x_{1}, …, x_{k}) = c^{ε}(s_{1}, …, s_{k}) on the rectangles A_{s_{1}}^{ε} ×⋯×A_{s_{k}}^{ε} with 1 ≤ s_{j} ≤ M(ε), 1 ≤ j ≤ k, with different indices s_{1}, …, s_{k}, where c^{ε}(s_{1}, …, s_{k}) = c(p_{1}, …, p_{k}) with those indices (p_{1}, …, p_{k}) for which A_{s_{1}}^{ε} ×⋯×A_{s_{k}}^{ε} ⊂ A_{p_{1}} ×⋯×A_{p_{k}}. The function f disappears outside of these rectangles. The function g(x_{1}, …, x_{l}) can be written similarly in the form g(x_{1}, …, x_{l}) = d^{ε}(t_{1}, …, t_{l}) on the rectangles B_{t_{1}}^{ε} × ⋯ × B_{t_{l}}^{ε} with 1 ≤ t_{j′} ≤ M′(ε), 1 ≤ j′ ≤ l, and different indices, t_{1}, …, t_{l}. Beside this, the function g disappears outside of these rectangles.

The above representation of the functions f and g through a parameter ε is useful, since it enables us to give a good asymptotic formula for the product k!Z_{μ,k}(f)l!Z_{μ,l}(g) which yields the diagram formula for the product of Wiener–Itô integrals of elementary functions with the help of a limiting procedure ε → 0.

Fix a small number ε > 0, take the representation of the functions f and g with its help, and write

k!Z_{μ,k}(f)l!Z_{μ,l}(g) = ∑_{γ∈Γ(k,l)} Z_{γ}(ε)  (B2)

with

Z_{γ}(ε) = ∑^{γ}c^{ε}(s_{1}, …, s_{k})d^{ε}(t_{1}, …, t_{l})μ_{W} (A_{s_{1}}^{ε}) … μ_{W} (A_{s_{k}}^{ε})μ_{W} (B_{t_{1}}^{ε}) … μ_{W} (B_{t_{l}}^{ε}),

(B3)

where Γ(k, l) denotes the class of diagrams introduced before the formulation of Theorem 10.2A, and ∑^{γ} denotes summation for such k + l-tuples (s_{1}, …, s_{k}, t_{1}, …, t_{l}), 1 ≤ s_{j} ≤ M(ε), 1 ≤ j ≤ k, and 1 ≤ t_{j′} ≤ M′(ε), 1 ≤ j′ ≤ l, for which A_{s_{j}}^{ε} = B_{t_{j′}}^{ε} if ((1, j), (2, j′)) ∈ E(γ), i.e. if it is an edge of γ, and otherwise all sets A_{s_{j}}^{ε} and B_{t_{j′}}^{ε} are disjoint. (This sum also depends on ε.) In the case of an empty sum Z_{γ}(ε) equals zero.

For all γ ∈ Γ(k, l) the expression Z_{γ} will be written in the form

Z_{γ}(ε) = Z_{γ}^{(1)}(ε) + Z_{γ}^{(2)}(ε), γ ∈ Γ(k, l),  (B4)

with

\begin{aligned}Z_{γ}^{(1)}(ε) & = ∑^{γ}c^{ε}(s_{1}, …, s_{k})d^{ε}(t_{1}, …, t_{l}) \\ & ∏_{j: (1,j)∈V_{1}(γ)} μ_{W} (A_{s_{j}}^{ε}) ∏_{j′: (2,j′)∈V_{2}(γ)} μ_{W} (B_{t_{j′}}^{ε}) \\ & ∏_{j: (1,j)∈\{(1,1),...,(1,k)\}\V_{1}(γ)} μ(A_{s_{j}}^{ε}) \end{aligned}  (B5)

and

Z_{γ}^{(2)}(ε) = ∑^{γ}c^{ε}(s_{1}, …, s_{k})d^{ε}(t_{1}, …, t_{l})

∏_{j: (1,j)∈V_{1}(γ)} μ_{W} (A_{s_{j}}^{ε}) ∏_{j′: (2,j′)∈V_{2}(γ)} μ_{W} (B_{t_{j′}}^{ε})

[  ∏_{j: (1,j)∈{(1,1),...,(1,k)}\V_{1}(γ)} μ_{W} (A_{s_{j}}^{ε}) ∏_{j′: (2,j′)∈{(2,1),...,(2,l)}\∈V_{2}(γ)} μ_{W} (B_{t_{j′}}^{ε})

− ∏_{j: (1,j)∈{(1,1),...,(1,k)}\V_{1}(γ)} μ(A_{s_{j}}^{ε})],  (B6)

where V_{1}(γ) and V_{2}(γ) (introduced before formula (10.9) during the preparation to the formulation of Theorem 10.2A) are the sets of vertices in the first and second row of the diagram γ from which no edge starts.

I claim that there is some constant C > 0 not depending on ε such that

E (|γ|!Z_{μ,|γ|}(F_{γ}) − Z_{γ}^{(1)}(ε))^{2} ≤ Cε for all γ ∈ Γ(k, l)  (B7)

with the Wiener–Itô integral with the kernel function F_{γ} defined in (10.9), (10.9a) and (10.10), and

E (Z_{γ}^{(2)}(ε))^{2} ≤ Cε for all γ ∈ Γ(k, l).  (B8)

Relations (B7) and (B8) imply relation (10.12) if f and g are elementary functions. Indeed, they imply that

lim_{ε→0} ∥ |γ|!Z_{μ,|γ|}(F_{γ}) − Z_{γ}(ε)∥_{2} → 0 for all γ ∈ Γ(k, l),

and this relation together with (B2) yield relation (10.12) with the help of a limiting procedure ε → 0.

To prove relation (B7) let us introduce the function

F_{γ}^{ε}(x_{(1,j)}, x_{(2,j′)}, (1, j) ∈ V_{1}(γ), (2, j′) ∈ V_{2}(γ))

= F_{γ}(x_{(1,j)}, x_{(2,j′)}, (1, j) ∈ V_{1}(γ), (2, j′) ∈ V_{2}(γ))
if x_{(1,j)} ∈ A_{s_{j}}^{ε}, for all (1, j) ∈ V_{1}(γ),
x_{(2,j′)} ∈ B_{t_{j′}}^{ε} , for all (2, j′) ∈ V_{2}(γ)), and
all sets A_{s_{j}}^{ε}, (1, j) ∈ V_{1}(γ), and B_{t_{j′}}^{ε} , (2, j′) ∈ V_{2}(γ) are different.

with the function F_{γ} defined in (10.9a) and (10.10), and put

F_{γ}^{ε}(x_{(1,j)}, x_{(2,j′)}, (1, j) ∈ V_{1}(γ), (2, j′) ∈ V_{2}(γ)) = 0 otherwise.

The function F_{γ}^{ε} is elementary, and a comparison of its definition with relation (B5) and the definition of the function F_{γ} yield that

Z_{γ}^{(1)}(ε) = |γ|!Z_{μ,|γ|}(F_{γ}^{ε}).  (B9)

The function F_{γ}^{ε} slightly differs from F_{γ}, since the function F_{γ} may not disappear in such points (x_{(1,j)}, x_{(2,j′)}, (1, j) ∈ V_{1}(γ), (2, j′) ∈ V_{2}(γ)) for which there is some pair (j, j′) with the property x_{(1,j)} ∈ A_{s_{j}}^{ε} and x_{(2,j′)} ∈ B_{t_{j′}}^{ε} with some sets A_{s_{j}}^{ε} and B_{t_{j′}}^{ε} such that A_{s_{j}}^{ε} = B_{t_{j′}}^{ε}, while F_{γ}^{ε} must be zero in such points. On the other hand, in the case |γ| = max(k, l) − min(k, l), i.e. if one of the sets V_{1}(γ) or V_{2}(γ) is empty, F_{γ} = F_{γ}^{ε}, Z_{γ}^{(1)} = |γ|!Z_{μ,|γ|}(F_{γ}), and relation (B7) clearly holds for such diagrams γ.

In the case |γ| = max(k, l) − min(k, l) > 0 such an estimate will be proved for the probability of the the set where F_{γ} ≠ F_{γ}^{ε} which implies relation (B7).

Let us define the sets A = ⋃_{s=1}^{M(ε)} A_{s}^{ε} and B = ⋃_{t=1}^{M′(ε)} B_{t}^{ε}. These sets A and B do not

depend on the parameter ε. Beside this μ(A) < ∞, and μ(B) < ∞. Define for all pairs (j_{0}, j′_{0}) such that (1, j_{0}) ∈ V_{1}(γ), (2, j′_{0}) ∈ V_{2}(γ) the set

D(j_{0}, j′_{0}) = {(x_{(1,j)}, x_{(2,j′)}, (1, j) ∈ V_{1}(γ), (2, j′) ∈ V_{2}(γ)):

x_{(1,j_{0})} ∈ A_{s_{j_{0}}}^{ε} , x_{(1,j′_{0})} ∈ B_{t_{j′_{0}}}^{ε} for some s_{j_{0}} and t_{j′_{0}} such that A_{s_{j_{0}}}^{ε} = B_{t_{j′_{0}}}^{ε}

x_{(1,j)} ∈ A for all (1, j) ∈ V_{1}(γ), and x_{(2,j′)} ∈ B for all (2, j′) ∈ V_{2}(γ)}.

Introduce the notation x^{γ} = (x_{(1,j)}, x_{(2,j′)}), (1, j) ∈ V_{1}(γ), (2, j′) ∈ V_{2}(γ)) and put

D_{γ} = {x^{γ}: F_{γ}^{ε}(x^{γ}) ≠ F_{γ}(x^{γ})}. The relation D_{γ} ⊂ ⋃_{j=1}^{k} ⋃_{j′=1}^{l} D(j_{0}, j′_{0}) holds, since if

F_{γ}^{ε}(x^{γ}) ≠ F_{γ}(x^{γ}) for some vector x^{γ}, then it has some coordinates (1, j_{0}) ∈ V_{1}(γ) and (2, j′_{0}) ∈ V_{2}(γ) such that x_{(1,j_{0})} ∈ A_{s_{j_{0}}}^{ε} and x_{(1,j′_{0})} ∈ B_{t_{j′_{0}}}^{ε} with some sets A_{s_{j_{0}}}^{ε} = B_{t_{j′_{0}}}^{ε},

and the relation in the last line of the definition of D(j_{0}, j′_{0}) must also hold for this vector x^{γ}, since otherwise F_{γ}(x_{γ}) = 0 = F_{γ}^{ε}(x_{γ}). I claim that there is some constant C_{1} such that

μ^{|V_{1}(γ)|+|V_{2}(γ)|}(D(j_{0}, j′_{0})) ≤ C_{1}ε for all sets D(j_{0}, j′_{0}),

where μ^{|V_{1}(γ)|+|V_{2}(γ)|} denotes the direct product of the measure μ on some copies of the original space (X, X) indexed by (1, j) ∈ V_{1}(γ) and (2, j′) ∈ V_{2}(γ). To see this relation one has to observe that ∑_{A_{s_{j_{0}}}^{ε} =B_{t_{j′_{0}}}^{ε}} μ(A_{s_{j_{0}}}^{ε})μ(B_{t_{j′_{0}}}^{ε}) ≤ ∑ εμ(A_{s_{j_{0}}}^{ε}) = εμ(A). Thus the set

D(j_{0}, j′_{0}) can be covered by the direct product of a set whose μ measure is not greater than εμ(A) and of a rectangle whose edges are either the set A or the set B.

The above relations imply that

μ^{|V_{1}(γ)|+|V_{2}(γ)|}(D_{γ}) ≤ C_{2}ε  (B10)

with some constant C_{2} > 0.

Relation (B9), estimate (B10), the property c) formulated in Theorem 10.1 for Wiener–Itô integrals and the observation that the function F_{γ} = F_{γ}(f, g) is bounded in supremum norm if f and g are elementary functions imply the inequality

E (|γ|!Z_{μ,|γ|}(F_{γ}) − Z_{γ}^{(1)}(ε))^{2} = |γ!|^{2}E (Z_{μ,|γ|}(F_{γ} − F_{γ}^{ε}))^{2} ≤ |γ|!∥F_{γ} − F_{γ}^{ε}∥_{2}^{2}
≤ Kμ^{|V_{1}(γ)|+|V_{2}(γ)|}(D_{γ}) ≤ Cε.

This means that relation (B7) holds.

To prove relation (B8) write E (Z_{γ}^{(2)}(ε))^{2} in the following form:

\begin{aligned}E (Z_{γ}^{(2)}(ε))^{2} & = ∑^{γ}∑^{γ}c^{ε}(s_{1}, …, s_{k})d^{ε}(t_{1}, …, t_{l})c^{ε}(s̄_{1}, …, s̄_{k})d^{ε}(t̄_{1}, …, t̄_{l}) \\ & EU(s_{1}, …, s_{k}, t_{1}, …, t_{l}, s̄_{1}, …, s̄_{k}, t̄_{1}, …, t̄_{l}) \end{aligned} (B11)

with

U(s_{1}, …, s_{k}, t_{1}, …, t_{l}, s̄_{1}, …, s̄_{k}, t̄_{1}, …, t̄_{l})

= ∏_{j: (1,j)∈V_{1}(γ)} μ_{W} (A_{s_{j}}^{ε}) ∏_{j′: (2,j′)∈V_{2}(γ)} μ_{W} (B_{t_{j′}}^{ε})

∏_{j̄: (1,j̄)∈V_{1}(γ)} μ_{W} (Aε ; s̄j̄) ∏_{j̄′: (2,j̄′)∈V_{2}(γ)} μ_{W} (Bε ; t̄_{}_{j̄′})

[  ∏_{j: (1,j)∈{(1,1),...,(1,k)}\V_{1}(γ)} μ_{W} (A_{s_{j}}^{ε}) ∏_{j′: (2,j′)∈{(2,1),...,(2,l)}\∈V_{2}(γ)} μ_{W} (B_{t_{j′}}^{ε})

− ∏_{j: (1,j)∈{(1,1),...,(1,k)}\V_{1}(γ)} μ(A_{s_{j}}^{ε})]

[  ∏_{j̄: (1,j̄)∈{(1,1),...,(1,k)}\V_{1}(γ)} μ_{W} (Aε ; s̄j̄) ∏_{j̄′: (2,j̄′)∈{(2,1),...,(2,l)}\∈V_{2}(γ)} μ_{W} (Bε ; t̄_{}_{j̄′})

− ∏_{j̄: (1,j)∈{(1,1),...,(1,k)}\V_{1}(γ)} μ(Aε ; s̄j̄)].  (B12)

The double sum ∑^{γ} ∑^{γ} in (B11) has to be understood in the following way. The first summation is taken for vectors (s_{1}, …, s_{k}, t_{1}, …, t_{l}), and these vectors take such values which were defined in ∑^{γ} in formula (B3). The second summation is taken for vectors (s̄_{1}, …, s̄_{k}, t̄_{1}, …, t̄_{l}), and again with values defined in the summation ∑^{γ}.

Relation (B8) will be proved by means of some estimates about the expectation of the above defined random variable U(·) which will be presented in the following Lemma B. Before its formulation I introduce the following Properties A and B.

Property A. A sequence s_{1}, …, s_{k}, t_{1}, …, t_{l}, s̄_{1}, …, s̄_{k}, t̄_{1}, …, t̄_{l}, with elements 1 ≤ s_{j}, s̄j̄ ≤ M(ε), for 1 ≤ j, j̄ ≤ k, and 1 ≤ t_{j}, t̄_{}_{j̄′} ≤ M′(ε) for 1 ≤ j′, j̄′ ≤ l, satisfies Property A (depending on a fixed diagram γ and number ε > 0) if the sequences of sets

{A_{s_{j}}^{ε}, B_{t_{j′}}^{ε} , (1, j) ∈ V_{1}(γ), (2, j′) ∈ V_{2}(γ)} and {Aε ; s̄j̄, Bε ; t̄_{}_{j̄′} , (1, j̄) ∈ V_{1}(γ), (2, j̄′) ∈ V_{2}(γ)}

agree. (Here we say that two sequences agree if they contain the same elements in a possibly different order.)

Property B. A sequence s_{1}, …, s_{k}, t_{1}, …, t_{l}, s̄_{1}, …, s̄_{k}, t̄_{1}, …, t̄_{l}, with elements 1 ≤ s_{j}, s̄j̄ ≤ M(ε), for 1 ≤ j, j̄ ≤ k, and 1 ≤ t_{j}, t̄_{}_{j̄′} ≤ M′(ε) for 1 ≤ j′, j̄′ ≤ l, satisfies Property B (depending on a fixed diagram γ and number ε > 0) if the sequences of sets

{A_{s_{j}}^{ε}, B_{t_{j′}}^{ε} , (1, j) ∈ {(1, 1), …, (1, k)} \ V_{1}(γ), (2, j′) ∈ {(2, 1), …, (2, l)} \ V_{2}(γ)} and

{Aε ; s̄j̄, Bε ; t̄_{}_{j̄′} , (1, j̄) ∈ {(1, 1), …, (1, k)} \ V_{1}(γ), (2, j̄′) ∈ {(2, 1), …, (2, l)} \ V_{2}(γ)}

have at least one common element.

(In the above definitions two sets A_{s}^{ε} and B_{t}^{ε} are identified if A_{s}^{ε} = B_{t}^{ε}.)

Now I formulate the following

Lemma B. Let us consider the function U(·) introduced in formula (B12). Assume that its arguments s_{1}, …, s_{k}, t_{1}, …, t_{l}, s̄_{1}, …, s̄_{k}, t̄_{1}, …, t̄_{l} are chosen in such a way that the function U(·) with these arguments appears in the double sum ∑^{γ} ∑^{γ} in formula (B11), i.e. A_{s_{j}}^{ε} = B^{ε}t_{j′} if ((1, j), (2, j′)) ∈ E(γ), otherwise all sets A_{s_{j}}^{ε} and B_{t_{j′}}^{ε} are disjoint, and an analogous statement holds if the coordinates s_{1}, …, s_{k}, t_{1}, …, t_{l} are replaced by s̄_{1}, …, s̄_{k}, t̄_{1}, …, t̄_{l}. Then

EU(s_{1}, …, s_{k}, t_{1}, …, t_{l}, s̄_{1}, …, s̄_{k}, t̄_{1}, …, t̄_{l}) = 0  (B13)

if the sequence of the arguments in U(·) does not satisfies either Property A or Property B.

If the sequence of the arguments in U(·) satisfies both Property A and Property B, then

\begin{matrix}|EU(s_{1}, …, s_{k}, t_{1}, …, t_{l}, s̄_{1}, …, s̄_{k}, t̄_{1}, …, t̄_{l})| \\ ≤ Cε ∏′μ(A_{s_{j}}^{ε})μ(Aε ; s̄j̄)μ(B_{t_{j′}}^{ε})μ(Bε ; t̄_{}_{j̄′}) \end{matrix}  (B14)

with some appropriate constant C = C(k, l) > 0 depending only on the number of variables k and l of the functions f and g. The prime in the product ∏′ at the righthand side of (B14) means that in this product the measure μ of those sets A_{s_{j}}^{ε}, Aε ; s̄j̄, B_{t_{j′}}^{ε} and Bε ; t̄_{}_{j̄′} are considered, whose indices are listed among the arguments s_{j}, s̄j̄, t_{j′} or t̄_{}_{j̄′} of U(·), and the measure μ of each such set appears exactly once. (This means e.g. that if A_{s_{j}}^{ε} = B_{t_{j′}}^{ε} or A_{s_{j}}^{ε} = Bε ; t̄_{}_{j̄′} for some indices j and j′ or j̄′, then one of the terms between μ(A_{s_{j}}^{ε}) and μ(B_{t_{j′}}^{ε}) or μ(Bε ; t̄_{}_{j̄′}) is omitted from the product. For the sake of definitiveness let us preserve the set μ(A_{j_{s}}^{ε}) in such a case.)

Remark. The content of Lemma B is that most terms in the double sum in formula (B11) equal zero, and even the non-zero terms are small.

The proof of Lemma B. Let us prove first relation (B13) in the case when Property A does not hold. It will be exploited that for disjoint sets the random variables μ_{W} (A_{s}) and μ_{W} (B_{t}) are independent, and this provides a good factorization of the expectation of certain products. Let us carry out the multiplications in the definition of U(·) in formula (B12), and show that each product obtained in such a way has zero expectation. If Property A does not hold for the arguments of U(·), and beside this the arguments s_{1}, …, s_{k}, t_{1}, …, t_{l}, s̄_{1}, …, s̄_{k}, t̄_{1}, …, t̄_{l} satisfy the remaining conditions of Lemma B, then each product we consider contains a factor μ_{W} (A_{s_{j_{0}}}^{ε}), (1, j_{0}) ∈ V_{1}(γ), which is independent of all those terms in this product which are in the following list: μ_{W} (A_{s_{j}}^{ε}) with some j ≠ j_{0}, 1 ≤ j ≤ k, or μ_{W} (B_{t_{j′}}^{ε}), 1 ≤ j ≤ l, or μ_{W} (Aε ; s̄j̄) with (1, j̄) ∈ V_{1}(γ), or μ_{W} (Bε ; t̄_{}_{j̄′}) with (2, j̄′) ∈ V_{2}(γ). We will show with the help of this property that the expectation of each term has a factorization with a factor either of the form Eμ_{W} (A_{s_{j_{0}}}^{ε}) = 0 or Eμ_{W} (A_{s_{j_{0}}}^{ε})^{3} = 0, hence it equals zero. Indeed, although the above properties do not exclude the appearance of such a pair of arguments A_{tj̄′}^{ε}, (1, j̄′) ∈ {(1, 1), …, (1, k) \ V_{1}(γ) and B_{tj̄′}^{ε}, (2, j̄′) ∈ {(2, 1), …, (2, l)} \ V_{2}(γ) in the product for which A_{tj̄}^{ε} = B_{tj̄′}^{ε} = A_{s_{j_{0}}}^{ε}, and in such a case a term of the form Eμ_{W} (A_{s_{j_{0}}}^{ε}) will not appear in the product, but if this happens, then the product contains a factor of the form Eμ_{W} (A_{s_{j_{0}}}^{ε})^{3} = 0. Hence an appropriate factorization of each term of EU(·) contains either a factor of the form Eμ_{W} (A_{s_{j_{0}}}^{ε}) = 0 or Eμ_{W} (A_{s_{j_{0}}}^{ε})^{3} = 0 if U(·) does not satisfy Property A.

To finish the proof of relation (B13) it is enough consider the case when the arguments of U(·) satisfy Property A, but they do not satisfy Property B. The validity

of Property A implies that the sets {A_{s_{j}}^{ε}, j ∈ V_{1}} ∪ {B_{t_{j′}}^{ε} , j′ ∈ V_{2}} and {Aε ; s̄_{j}, j ∈

V_{1}} ∪ {Bε ; t̄_{j′} , j′ ∈ V_{2}} agree. The conditions of Lemma B also imply that the elements of these sets are such sets which are disjoint of the sets A_{s_{j}}^{ε}, B_{t_{j′}}^{ε}, Aε ; s̄j̄ and Bε ; t̄_{}_{j̄′} with indices (1, j), (1, j̄) ∈ {(1, 1), …, (1, k)} \ V_{1}(γ) and (2, j′), (2, j̄′) ∈ {(2, 1), …, (2, l)} \ V_{2}(γ). If Property B does not hold, then the latter class of sets can be divided into two subclasses in such a way that the elements in different subclasses are disjoint. The first subclass consists of the sets A_{s_{j}}^{ε} and B_{t_{j′}}^{ε}, and the second one of the sets Aε ; s̄j̄ and Bε ; t̄_{}_{j̄′} with indices such that (1, j), (1, j̄) ∈ {(1, 1), …, (1, k)} \ V_{1}(γ) and (2, j′), (2, j̄′) ∈ {(2, 1), …, (2, l)} \ V_{2}(γ). These facts imply that EU(·) has a factorization, which contains the term

E[  ∏_{j: (1,j)∈{(1,1),...,(1,k)}\V_{1}(γ)} μ_{W} (A_{s_{j}}^{ε}) ∏_{j′: (2,j′)∈{(2,1),...,(2,l)}\∈V_{2}(γ)} μ_{W} (B_{t_{j′}}^{ε})

− ∏_{j: (1,j)∈{(1,1),...,(1,k)}\V_{1}(γ)} μ(A_{s_{j}}^{ε})] = 0,

hence relation (B13) holds also in this case. The last expression has zero expectation, since if we take such pairs A_{s_{j}}^{ε}, B_{t′_{j}}^{ε} for the sets appearing in it for which that ((1, j), (2, j′)) ∈ E(γ), i.e. these vertices are connected with an edge of γ, then A_{s_{j}}^{ε} = B_{t′_{j}}^{ε} in a pair, and elements in different pairs are disjoint. This observation allows a factorization in the product whose expectation is taken, and then the identity Eμ_{W} (A_{s_{j}}^{ε})μ_{W} (B_{t_{j′}}^{ε}) = μ(A_{s_{j}}^{ε}) implies the desired identity.

To prove relation (B14) if the arguments of the function U(·) satisfy both Properties A and B consider the expression (B12) which defines U(·), carry out the term by term multiplication between the two differences at the end of this formula, take expectation for each term of the sum obtained in such a way, and factorize them. Since Eμ_{W} (A)^{2} = μ(A), Eμ_{W} (A)^{4} = 3μ(A)^{2} for all sets A ∈ X, μ(A) < ∞, some calculation shows that each term can be expressed as constant times a product whose elements are those probabilities μ(A_{s}^{ε}) and μ(B_{t}^{ε}) or their square which appear at the right-hand side of (B14). Moreover, since the arguments of U(·) satisfy Property B, there will be at least one term of the form μ(A_{s}^{ε})^{2} in this product. Since μ(A_{s}^{ε})^{2} ≤ εμ(A_{s}^{ε}), these calculations provide formula (B14). Lemma B is proved.

Relation (B11) implies that

E (Z_{γ}^{(2)}(ε))^{2} ≤ K ∑^{γ} ∑^{γ}|EU(s_{1}, …, s_{k}, t_{1}, …, t_{l}, s̄_{1}, …, s̄_{k}, t̄_{1}, …, t̄_{l})| (B15)

with some appropriate K > 0. By Lemma B it is enough to sum up only for such terms U(·) in (B15) whose arguments satisfy both Properties A and B. Moreover, each such term can be bounded by means of inequality (B14). Let us list the sets A_{s_{j}}^{ε}, Aε ; s̄j̄, B_{t_{j′}}^{ε} , Bε ; t̄_{}_{j̄′} appearing in the upper bound at the right-hand side of (B14) for all functions U(·) taking part in the sum at the right-hand side of (B15). Since all fixed sequences of the sets A_{s}^{ε} and B_{t}^{ε} appear less than C(k, l) times with an appropriate constant C(k, l)

depending only on the order k and l of the integrals we are considering, and ∑_{s=1}^{M(ε)} μ(A_{s}^{ε})+

∑_{t=1}^{M′(ε)} μ(B_{t}^{ε}) = μ(A) + μ(B) < ∞, the above relations imply that

E (Z_{γ}^{(2)}(ε))^{2} ≤ C_{1}ε ∑_{j=1}^{k+l}(μ(A) + μ(B))^{j} ≤ Cε.

Hence relation (B8) holds.

To prove Theorem 10.2A in the general case take for all pairs of functions f ∈ H_{μ,k} and g ∈ H_{μ,l} two sequences of elementary functions f_{n} ∈ H̄_{μ,k} and g_{n} ∈ H̄_{μ,l}, n = 1, 2, …, such that ∥f_{n} − f∥_{2} → 0 and ∥g_{n} − g∥_{2} → 0 as n → ∞. Let us introduce the notation F_{γ}(f, g) = F_{γ} if the function F_{γ} is defined in formulas (10.9a) and (10.10) with the help of the functions f and g. It is enough to show that

E|k!Z_{μ,k}(f)l!Z_{μ,l}(g) − k!Z_{μ,k}(f_{n})l!Z_{μ,l}(g_{n})| → 0 as n → ∞,  (B16)

and

|γ|!E |Z_{μ,|γ|}(F_{γ}(f, g)) − Z_{μ,|γ|}(F_{γ}(f_{n}, g_{n}))| → 0 as n → ∞ for all γ ∈ Γ(k, l),

(B17)

since then a simple limiting procedure n → ∞, and the already proved part of the theorem for Wiener–Itô integrals of elementary functions imply Theorem 10.2A.

To prove relation (B16) write

E|k!Z, μ, k(f)l!Z_{μ,l}(g) − k!Z_{μ,k}(f_{n})l!Z_{μ,l}(g_{n})| ≤ k!l! (E|Z_{μ,k}(f)Z_{μ,l}(g − g_{n})| + E|Z_{μ,k}(f − f_{n})Z_{μ,l}(g_{n})) |

≤ k!l! ((EZ_{μ,k}^{2}(f))^{1/2} (EZ_{μ,l}^{2}(g − g_{n}))^{1/2} + (EZ_{μ,k}^{2}(f − f_{n}))^{1/2} (EZ_{μ,l}^{2}(g_{n}))^{1/2})

≤ (k!l!)^{1/2} (∥f∥_{2}∥g − g_{n}∥_{2} + ∥f − f_{n}∥_{2}∥g_{n}∥_{2}) .

Relation (B16) follows from this inequality with a limiting procedure n → ∞. To prove relation (B17) write

|γ|!E |Z_{μ,|γ|}(F_{γ}(f, g)) − Z_{μ,|γ|}(F_{γ}(f_{n}, g_{n}))|

≤ |γ|!E |Z_{μ,|γ|}(F_{γ}(f, g − g_{n}))| + |γ|!E |Z_{μ,|γ|}(F_{γ}(f − f_{n}, g_{n}))|
≤ |γ|! (EZ_{μ,|γ|}^{2}(F_{γ}(f, g − g_{n})))^{1/2} + |γ|! (EZ_{μ,|γ|}^{2}(F_{γ}(f − f_{n}, g_{n})))^{1/2}
≤ (|γ|!)^{1/2} (∥F_{γ}(f, g − g_{n})∥_{2} + ∥F_{γ}(f − f_{n}, g_{n})∥_{2}) ,

and observe that by relation (10.11) ∥F_{γ}(f, g − g_{n})∥_{2} ≤ ∥f∥_{2}∥g − g_{n}∥_{2}, and ∥F_{γ}(f − f_{n}, g_{n})∥_{2} ≤ ∥f − f_{n}∥_{2}∥g_{n}∥_{2}. Hence

|γ|!E |Z_{μ,|γ|}(F_{γ}(f, g)) − Z_{μ,|γ|}(F_{γ}(f_{n}, g_{n}))|
≤ (|γ|!)^{1/2} (∥f∥_{2}∥g − g_{n}∥_{2} + ∥f − f_{n}∥_{2}∥g_{n}∥_{2}) .

The last inequality implies relation (B17) with a limiting procedure n → ∞. Theorem 10.2A is proved.

Appendix C. The proof of some results about Wiener–Itô integrals.

First I prove Itô’s formula about multiple Wiener–Itô integrals (Theorem 10.3). The proof is based on the diagram formula for Wiener–Itô integrals and a recursive formula about Hermite polynomials proved in Proposition C. In Proposition C2 I present the proof of another important property of Hermite polynomials. This result states that the class of all Hermite polynomials is a complete orthogonal system in an appropriate Hilbert space. It is needed in the proof of Theorem 10.5 about the isomorphism of Fock spaces to the Hilbert space generated by Wiener–Itô integrals. At the end of Appendix C the proof of Theorem 10.4, a limit theorem about degenerated U-statistics is given.

Proposition C about some properties of Hermite polynomials. The functions

H_{k}(x) = (−1)^{k}e^{x^{2}/2} \frac{d^{k}}{dx^{k}} e^{−x^{2}/2}, k = 0, 1, 2, …  (C1)

are the Hermite polynomials with leading coefficient 1, i.e. H_{k}(x) is a polynomial of order k with leading coefficient 1 such that

∫_{−∞}^{∞} H_{k}(x)H_{l}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = 0 if k ≠ l,  (C2)

and

∫_{−∞}^{∞} H_{k}^{2}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = k! for all k = 0, 1, 2 ….  (C2′)

The recursive relation

H_{k}(x) = xH_{k−1}(x) − (k − 1)H_{k−2}(x)  (C3)

holds for all k = 1, 2, . . ..

Remark. It is more convenient to consider relation (C3) valid also in the case k = 1. In this case H_{1}(x) = x, H_{0}(x) = 1, and relation holds with an arbitrary function H_{−1}(x).

Proof of Proposition C. It is clear from formula (C1) that H_{k}(x) is a polynomial of order k with leading coefficient 1. Take l ≥ k, and write by means of integration by parts

∫_{−∞}^{∞} H_{k}(x)H_{l}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = ∫_{−∞}^{∞} \frac{1}{\sqrt{2π}} H_{k}(x)(−1)^{l} \frac{d^{l}}{dx^{l}} e^{−x^{2}/2} dx

= ∫_{−∞}^{∞} \frac{1}{\sqrt{2π}} \frac{d}{dx}H_{k}(x)(−1)^{l−1} \frac{d^{l−1}}{dx^{l−1}} e^{−x^{2}/2} dx.

Successive partial integration together with the identity \frac{d^{k}}{dx^{k}} H_{k}(x) = k! yield that

∫_{−∞}^{∞} H_{k}(x)H_{l}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = k! ∫_{−∞}^{∞} \frac{1}{\sqrt{2π}} (−1)^{l−k} \frac{d^{l−k}}{dx^{l−k}} e^{−x^{2}/2} dx.

The last relation supplies formulas (C2) and (C2′).

To prove relation (C3) observe that H_{k}(x) − xH_{k−1}(x) is a polynomial of order k − 2. (The term x^{k−1} is missing from this expression. Indeed, if k is an even number, then the polynomial H_{k}(x) − xH_{k−1}(x) is an even function, and it does not contain the term x^{k−1} with an odd exponent k − 1. Similar argument holds if the number k is odd.) Beside this, it is orthogonal (with respect to the standard normal distribution) to all Hermite polynomials H_{l}(x) with 0 ≤ l ≤ k − 3. Hence H_{k}(x) − xH_{k−1}(x) = CH_{k−2}(x) with some constant C to be determined.

Multiply both sides of the last identity with H_{k−2}(x) and integrate them with respect to the standard normal distribution. Apply the orthogonality of the polynomials H_{k}(x) and H_{k−2}(x), and observe that the identity

∫ H_{k−1}(x)xH_{k−2}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = ∫ H_{k−1}^{2}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = (k − 1)!

holds. (In this calculation we have exploited that H_{k−1}(x) is orthogonal to H_{k−1}(x) − xH_{k−2}(x), because the order of the latter polynomial is less than k − 1.) In such a way we get the identity −(k − 1)! = C(k − 2)! for the constant C in the last identity, i.e. C = −(k − 1), and this implies relation (C3).

Proof of Itô’s formula for multiple Wiener–Itô integrals. Let K = ∑_{p=1}^{m} k_{p}, the sum of the

order of the Hermite polynomials, denote the order of the expression in relation (10.20).

Formula (10.20) clearly holds for expressions of order K = 1. It will be proved in the general case by means of induction with respect to the order K.

In the proof the functions f(x_{1}) = φ_{1}(x_{1}) and

g(x_{1}, …, x_{K_{m}−1}) = ∏_{j=1}^{K_{1}−1} φ_{1}(x_{j}) · ∏_{p=2}^{m} ∏_{j=K_{p−1}}^{K_{p}−1} φ_{p}(x_{j}),

will be introduced and the product Z_{μ,1}(f)(K_{m} − 1)!Z_{μ,K_{m}−1}(g) will be calculated by means of the diagram formula. (The same notation is applied as in Theorem 10.3.

In particular, K = K_{m}, and in the case K_{1} = 1 the convention ∏_{j=1}^{K_{1}−1} φ_{1}(x_{j}) = 1 is

applied.) In the application of the diagram formula diagrams with two rows appear. The first row of these diagrams contains the vertex (1, 1) and the second row contains the vertices (2, 1), …, (2, K_{m} − 1). It is useful to divide the diagrams to three disjoint classes. The first class contains only the diagram γ_{0} without any edges. The second class Γ_{1} consists of those diagrams which have an edge of the form ((1, 1), (2, j)) with some 1 ≤ j ≤ k_{1} − 1, and the third class Γ_{2} is the set of those diagrams which have an edge of the form ((1, 1), (2, j)) with some k_{1} ≤ j ≤ K_{m} − 1. Because of the orthogonality of the functions φ_{s} for different indices s F_{γ} ≡ 0 and Z_{μ,K_{m}−2}(F_{γ}) = 0 for γ ∈ Γ_{2}. The class Γ_{1} contains k_{1} − 1 diagrams. Let us consider a diagram γ from this class with an edge ((1, 1), (2, j_{0})), 1 ≤ j ≤ k_{1} − 1. We have for such a diagram F_{γ} =

∏_{j∈{1,...,K_{1}−1}\{j_{0}}} φ_{1}(x_{(2,j)}) ∏_{p=2}^{m} ∏_{j=K_{p−1}}^{K_{p}−1} φ_{p}(x_{(2,j)}), and by our inductive hypothesis (K_{m}−

2)!Z_{μ,K_{m}−2}(F_{γ}) = H_{k_{1}−2}(η_{1}) ∏_{p=2}^{m} H_{k_{p}}(η_{p}). Finally

K_{m}!Z_{μ,K_{m}}(F_{γ_{0}}) = K_{m}!Z_{μ,K_{m}} (∏_{p=1}^{m} ( ∏_{j=K_{p−1}+1}^{K_{p}} φ_{p}(x_{j})))

for the diagram γ_{0} without any edge.

Our inductive hypothesis also implies the following identity for the expression we wanted to calculate with the help of the diagram formula.

Z_{μ,1}(f)(K_{m} − 1)!Z_{μ,K_{m}−1}(g) = η_{1}H_{k_{1}−1}(η_{1}) ∏_{p=2}^{m} H_{k_{p}}(η_{p}).

The above calculations together with the observation |Γ_{1}| = k_{1}−1 yield the identity

K_{m}!Z_{μ,K_{m}} (∏_{p=1}^{m} ( ∏_{j=K_{p−1}+1}^{K_{p}} φ_{p}(x_{j}))) = K_{m}!Z_{μ,K_{m}}(F_{γ_{0}})

= Z_{μ,1}(f)(K_{m} − 1)!Z_{μ,K_{m}−1}(g) − ∑_{γ∈Γ_{1}}(K_{m} − 2)!Z_{μ,K_{m}−2}(F_{γ})

= η_{1}H_{k_{1}−1}(η_{1}) ∏_{p=2}^{m} H_{k_{p}}(η_{p}) − (k_{1} − 1)H_{k_{1}−2}(η_{1}) ∏_{p=2}^{m} H_{k_{p}}(η_{p})

= [η_{1}H_{k_{1}−1}(η_{1}) − (k_{1} − 1)H_{k_{1}−2}(η_{1})] ∏_{p=2}^{m} H_{k_{p}}(η_{p}).  (C4)

On the other hand, η_{1}H_{k_{1}−1}(η_{1})−(k_{1} −1)H_{k_{1}−2}(η_{1}) = H_{k_{1}}(η_{1}) by formula (C3). These relations imply formula (10.20), i.e. Itô’s formula.

I present the proof of another important property of the Hermite polynomials in the following Proposition C2.

Proposition C2 on the completeness of the orthogonal system of Hermite polynomials. The Hermite polynomials H_{k}(x), k = 0, 1, 2, …, defined in formula (C4) constitute a complete orthonormal system of in the L_{2}-space of the functions square integrable with respect to the Gaussian measure \frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx on the real line.

Proof of Proposition C2. Let us consider the orthogonal complement of the subspace generated by the Hermite polynomials in the space of the square integrable functions with respect to the measure \frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx. It is enough to prove that this orthogonal completion contains only the identically zero function. Since the orthogonality of a function to all polynomials of the form x^{k}, k = 0, 1, 2, … is equivalent to the orthogonality of this function to all Hermite polynomials H_{k}(x), k = 0, 1, 2, …, Proposition C2 can be reformulated in the following form:

If a function g(x) on the real line is such that

∫_{−∞}^{∞} x^{k}g(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = 0 for all k = 0, 1, 2, …  (C5)

and

∫_{−∞}^{∞} g^{2}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx < ∞,  (C6)

then g(x) = 0 for almost all x.

Given a function g(x) on the real line whose absolute value is integrable with respect to the Gaussian measure \frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx define the (finite) measure ν_{g},

ν_{g}(A) = ∫_{A} g(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx

on the measurable sets of the real line together with its Fourier transform ν̃_{g}(t) = ∫_{−∞}^{∞} e^{itx}ν_{g}( dx). (This measure ν_{g} and its Fourier transform can be defined for all functions g satisfying relation (C6), because their absolute value is integrable with respect to the Gaussian measure.) First I show that Proposition C2 can be reduced to the following statement: If a function g satisfies both (C5) and (C6) then ν̃_{g}(t) = 0 for all −∞ < t < ∞.

Indeed, if there were a function g satisfying (C5) and (C6) which is not identically zero, then the non-negative functions g^{+}(x) = max(0, g(x)) and g^{−}(x) = − min(0, g(x)) would be different. Then also their Fourier transform ν̃_{g^{+}}(t) and ν̃_{g^{−}}(t) would be different, since a finite measure is uniquely determined by its Fourier transform. (This statement is equivalent to an important result in probability theory, by which a probability measure on the real line is determined by its characteristic function.) But this would mean that ν̃_{g}(t) = ν̃_{g^{+}}(t) − ν̃_{g^{−}}(t) ≠ 0 for some t. Hence Proposition C2 can be reduced to the above statement.

Since |e^{itx} − 1 − (itx) − ⋯ − \frac{(itx)^{k}}{k!} | ≤ \frac{|tx|^{(k+1)}}{(k+1)!} for all real numbers t, x and integer k = 1, 2, … we may write because of relation (C5)

|ν̃_{g}(t)| = |∫_{−∞}^{∞} (e^{itx} − 1 − (itx) − ⋯ − \frac{(itx)^{k}}{k!} ) g(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx|

≤ ∫_{−∞}^{∞} \frac{|t|^{(k+1)}}{(k + 1)!}|x|^{k+1}|g(x)|\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx

for all k = 1, 2, … and real number t if the function g satisfies relation (C5). If it satisfies both relation (C5) and (C6), then from the last relation and the Schwarz inequality

|ν̃_{g}(t)|^{2} ≤ const. \frac{|t|^{2(k+1)}}{((k + 1)!)^{2}} ∫_{−∞}^{∞} |x|^{2(k+1)} \frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx

= const. \frac{|t|^{2(k+1)}}{((k + 1)!)^{2}} 1 · 3 · 5 ⋯ (2k + 1)

for all real number t and integer k = 1, 2, . . .. Simple calculation shows that the righthand side of the last estimate tends to zero as k → ∞. This implies that ν̃_{g}(t) = 0 for all t, and Proposition C2 holds.

I finish Appendix C with the proof of Theorem 10.4, a limit theorem about a sequence of normalized degenerate U-statistics. It is based on an appropriate representation of the U-statistics by means of multiple random integrals which makes possible to carry out an appropriate limiting procedure.

Proof of Theorem 10.4. For all n = 1, 2, …, the normalized degenerate U-statistics n^{−k/2}I_{n,k}(f) can be written in the form

\begin{aligned}n^{−k/2}k!I_{n,k}(f) & = n^{k/2} ∫ ′ f(x_{1}, …, x_{k})μ_{n}( dx_{1}) … μ_{n}( dx_{k}) \\ & = n^{k/2} ∫ ′ f(x_{1}, …, x_{k})(μ_{n}( dx_{1}) − μ( dx_{1})) … (μ_{n}( dx_{k}) − μ( dx_{1})), \end{aligned}

(C7) where μ_{n} is the empirical distribution function of the sequence ξ_{1}, …, ξ_{n} defined in (4.5), and the prime in ∫′ denotes that the diagonals, i.e. the points x = (x_{1}, …, x_{k}) such that x_{j} = x_{j′} for some pairs of indices 1 ≤ j, j′ ≤ k, j ≠ j′, are omitted from the domain of integration. The second identity in relation (C7) can be justified by means of the identity

\begin{matrix}∫ ′ f(x_{1}, …, x_{k})(μ_{n}( dx_{1}) − μ( dx_{1})) … (μ_{n}( dx_{k}) − μ( dx_{1})) − I_{n,k}(f) \\ = ∑_{V: V ∈\{1,...,k\}, |V |≥1}(−1)^{|V |} ∫ ′ f(x_{1}, …, x_{k}) ∏_{j∈V} μ( dx_{j}) ∏_{j∈\{1,...,k\}\V} μ_{n}( dx_{j})) = 0. \end{matrix}

(C8)

This identity holds for a function f canonical with respect to a non-atomic measure μ, because each term in the sum at the right-hand side of (C8) equals zero. Indeed, the integral of a canonical function f with respect to μ( dx_{j}) with some index j ∈ V equals zero for all fixed values x_{1}, …, x_{j−1}, x_{j+1}, …, x_{k}. The non-atomic property of the measure μ was needed to guarantee that this integral equals zero also in the case when the diagonals are omitted from the domain of integration.

We would like to derive Theorem 10.4 from relation (C7) by means of an appropriate limiting procedure which exploits the convergence of the random fields n^{1/2}(μ_{n}(A) − μ(A)), A ∈ X, to a Gaussian field ν(A), A ∈ X, as n → ∞. But some problems arise if we want to carry out such a program, because the fields n^{1/2}(μ_{n} − μ) converge to a non white noise type Gaussian field. The limit we get is similar to a Wiener bridge on the real line. Hence a relation between Wiener processes and Wiener bridges suggests to write the following version of formula (C7).

Let us take a standard Gaussian random variable η, independent of the random sequence ξ_{1}, ξ_{2}, . . .. For a canonical function f the following version of (C7) holds.

n^{−k/2}k!I_{n,k}(f) = J′_{n,k}(f)  (C9)

with

\begin{aligned}J′_{n,k}(f) = ∫ ′ & f(x_{1}, …, x_{k}) [\sqrt{n}(μ_{n}( dx_{1}) − μ( dx_{1})) + ημ( dx_{1})] \\ & … [\sqrt{n}(μ_{n}( dx_{k}) − μ( dx_{k})) + ημ( dx_{k})] . \end{aligned}  (C10)

This relation can be seen similarly to (C7).

The random measures n^{1/2}(μ_{n} − μ) + ημ converge to a white noise with reference measure μ. Hence Theorem 10.4 can be proved by means of formulas (C9) and (C10) with the help of an appropriate limiting procedure. More explicitly, I claim that the following slightly more general result holds. The expressions J′_{n,k}(f) introduced in (C10) converge in distribution to the Wiener–Itô integral k!Z_{μ,k}(f) as n → ∞ for all functions f square integrable with respect to the product measure μ^{k}. This result also holds for non-canonical functions f. This limit theorem together with relation (C9) imply Theorem 10.4.

The convergence of the random variables J′_{n,k}(f) defined in (C10) to the Wiener–Itô integral k!Z_{μ,k}(f) can be easily checked for elementary functions f ∈ H̄_{μ,k}. Indeed, if A_{1}, …, A_{M} are disjoint sets with μ(A_{s}) < ∞, then the multi-dimensional central limit theorem implies that the random vectors \{\sqrt{n}((μ_{n}(A_{s}) − μ(A_{s})) + ημ(A_{s}), 1 ≤ s ≤ M\} converge in distribution to the random vector {(μ_{W} (A_{s}), 1 ≤ s ≤ M}, i.e. to a set of independent normal random variables ζ_{s}, Eζ_{s} = 0, 1 ≤ s ≤ M, with variance Eζ_{s}^{2} = μ(A_{s}) as n → ∞. The definition of the elementary functions given in (10.2) shows that this central limit theorem implies the demanded convergence of the sequence J′_{n,k}(f) to k!Z_{μ,k}(f) for elementary functions.

To show the convergence of the sequence J′_{n,k}(f) to k!Z_{μ,k}(f) in the general case, take for any function f ∈ H_{μ,k} a sequence of elementary functions f_{N} ∈ H̄_{μ,k} such that ∥f − f_{N}∥_{2} → 0 as N → ∞. Then E(Z_{μ,k}(f) − Z_{μ,k}(f_{N}))^{2} = E(Z_{μ,k}(f − f_{N}))^{2} → 0 as N → ∞ by Property c) in Theorem 10.1. Hence the already proved part of the theorem implies that there exists some sequence of positive integers, N(n), n = 1, 2, …, in such a way that N(n) → ∞, and the sequence J′_{n,k}(f_{N(n)}) converges to k!Z_{μ,k}(f) in distribution as n → ∞. Thus to complete the proof of Theorem 10.4 it is enough to show that E(J′_{n,k}(f_{N(n)}) − J′_{n,k}(f))^{2} = E(J′_{n,k}(f_{N(n)} − f))^{2} → 0 as n → ∞.

It is enough to show that

E(J′_{n,k}(f))^{2} ≤ C∥f∥_{2}^{2} for all f ∈ H_{μ,k}  (C11)

with a constant C = C_{k} depending only on the order k of the function f and to apply inequality (C11) for the functions f_{N(n)} − f. Relation (C11) is a relatively simple consequence of Corollary 1 of Theorem 9.4.

Indeed,

J′_{n,k}(f) = ∑_{V ⊂{1,...,k}} η^{k−|V |}|V |!J_{n,|V |}(f_{V})

with

f_{V} (x_{j}, j ∈ V) = ∫ f(x_{1}, …, x_{k}) ∏_{j′∈{1,...,k}\V} μ(dx_{j′})

and the random integral J_{n,k}(·) defined in (4.8), hence

E(J′_{n,k}(f))^{2} ≤ 2^{k} ∑_{V ⊂{1,...,k}}(|V |!)^{2}Eη^{2(k−|V |)} · EJ_{n,|V |}^{2}(f_{V}).  (C12)

Inequality ∥f_{V} ∥_{2} ≤ ∥f∥_{2} holds for all sets V ⊂ {1, …, k}, hence an application of Corollary 1 of Theorem (9.4) to all random integrals J_{n,|V |}(f) supplies (C11).

The above proof also yields the following slight generalization of Theorem 10.4. Let us consider a finite sequence of functions f_{j} ∈ H_{μ,j}, 1 ≤ j ≤ k, canonical with respect to a non-atomic probability measure μ. The vectors {n^{−j/2}I_{n,j}(f_{j}), 1 ≤ j ≤ k}, consisting of normalized degenerate U-statistics defined with the help of a sequence of independent μ-distributed random variables converge to the random vector {Z_{μ,j}(f_{j}), 1 ≤ j ≤ k} in distribution as n → ∞. This result together with Theorem 9.4 imply the following limit theorem about multiple random integrals J_{n,k}(f).

Theorem 10.4′. (Limit theorem about multiple random integrals with respect to normalized empirical measures). Let a sequence of independent and identically distributed random variables ξ_{1}, ξ_{2}, … be given with some non-atomic distribution μ on a measurable space (X, X) together with a function f(x_{1}, …, x_{k}) on the k-fold product (X^{k}, X^{k}) of the space (X, X) such that

∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) < ∞.

Let us consider for all n = 1, 2, … the random integrals J_{n,k}(f) of order k defined in formulas (4.5) and (4.8) with the help of the empirical distribution μ_{n} of the sequence ξ_{1}, …, ξ_{n} and the function f. These random integrals J_{n,k}(f) converge in distribution, as n → ∞, to the following sum U(f) of multiple Wiener–Itô integrals:

U(f) = ∑_{V ⊂{1,...,k}} C(k, V)Z_{μ,|V |}(f_{V})

= ∑_{V ⊂\{1,...,k\}} \frac{C(k, V)}{|V |!} ∫ f_{V} (x_{j}, j ∈ V) ∏_{j∈V} μ_{W} (dx_{j}),

where the functions f_{V} (x_{j} j ∈ V), V ⊂ {1, …, k}, are those functions defined in formula (9.2) which appear in the Hoeffding decomposition of the function f(x_{1}, …, x_{k}), the constants C(k, V) are the limits appearing in the limit relation lim_{n→∞} C(n, k, V) = C(k, V) satisfied by the coefficients C(n, k, V) in formula (9.9), and μ_{W} is a white noise with reference measure μ.

An essential step of the proof of Theorem 10.4 was the reduction of the case of general kernel functions to the case of elementary kernel functions. Let me make some comments about it.

It would be simple to make such a reduction if we had a good approximation of a canonical function with such elementary functions which are also canonical. But it is very hard to find such an approximation. To overcome this difficulty we reduced the proof of Theorem 10.4 to a modified version of this result, where instead of a limit theorem for degenerate U-statistics a limit theorem for the random variables J′_{n,k}(f) introduced in formula (C10) has to be proved. In the proof of such a version we could apply the approximation of a general kernel function with not necessarily canonical elementary functions. Theorem 9.4 helped us to work with such an approximation. Another natural way to overcome the above difficulty is to apply a Poissonian approximation of the normalized empirical measure. Such an approach was applied in [14] and in [31], where some generalizations of Theorem 10.4 were proved.

Appendix D. The proof of Theorem 14.3.

A result about the comparison of U-statistics and decoupled U-statistics.

The proof of Theorem 14.3. It will be simpler to formulate and prove a generalized version of Theorem 14.3 where such generalized U-statistics are considered in which different kernel functions may appear in each term of the sum. More explicitly, let ℓ = ℓ(n, k) denote the set of all such sequences l = (l_{1}, …, l_{k}) of integers of length k for which 1 ≤ l_{j} ≤ n, 1 ≤ j ≤ k. To define generalized U-statistics let us fix a set of functions {f_{l_{1},...,l_{k}}(x_{1}, …, x_{k}), (l_{1}, …, l_{k}) ∈ ℓ} which map the space (X^{k}, X^{k}) to a separable Banach space B, and have the property f_{l_{1},...,l_{k}}(x_{1}, …, x_{k}) ≡ 0 if l_{j} = l_{j′} for some indices j ≠ j′. (The last condition corresponds to that property of U-statistics that the diagonals are omitted from the summation in their definition.) Let us denote this set of functions by f(ℓ) and define, similarly to the U-statistics and decoupled U-statistics the generalized U-statistics and generalized decoupled U-statistics by the formulas

I_{n,k}(f(ℓ)) = \frac{1}{k!} ∑_{(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k} f_{l_{1},...,l_{k}} (ξ_{l_{1}}, …, ξ_{l_{k}})  (D1)

and

Ī_{n,k}(f(ℓ)) = \frac{1}{k!} ∑_{(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k} f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(1)}, …, ξ_{l_{k}}^{(k)})  (D2)

(with the same independent random variables ξ_{l} and ξ_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k, as in the definition of the original U-statistics and decoupled U-statistics.)

The following generalization of relation (14.13) will be proved.

P (∥I_{n,k}(f(ℓ))∥ > u) ≤ A(k)P (∥Ī_{n,k}(f(ℓ))∥ > γ(k)u)  (14.13d)

with some constants A(k) > 0 and γ(k) > 0 depending only on the order k of these generalized U-statistics.

We concentrate mainly on the proof of the generalization (14.13d) of relation (14.13). Formula (14.14) is a relatively simple consequence of it. Formula (14.13d) will be proved by means of an inductive procedure which works only in this more general setting. It will be derived from the following statement.

Let us take two independent copies ξ_{1}^{(1)}, …, ξ_{n}^{(1)} and ξ_{1}^{(2)}, …, ξ_{n}^{(2)} of our original sequence of random variables ξ_{1}, …, ξ_{n}, and introduce for all sets V ⊂ {1, …, k} the function α_{V} (j), 1 ≤ j ≤ k, defined as α_{V} (j) = 1 if j ∈ V and α_{V} (j) = 2 if j ∉ V. Let us define with their help the following version of decoupled U-statistics

I_{n,k,V} (f(ℓ)) = \frac{1}{k!} ∑_{(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k}f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(α_{V} (1))}, …, ξ_{l_{k}}^{(α_{V} (k))})
for all V ⊂ {1, …, k}.  (D3)

The following inequality will be proved: There are some constants C_{k} > 0 and D_{k} > 0 depending only on the order k of the generalized U-statistic I_{n,k}(f(ℓ)) such that for all numbers u > 0

P (∥I_{n,k}(f(ℓ))∥ > u) ≤ ∑_{V ⊂{1,...,k}, 1≤|V |≤k−1} C_{k}P (D_{k}∥I_{n,k,V} (f(ℓ))∥ > u) .  (D4)

Here |V | denotes the cardinality of the set V, and the condition 1 ≤ |V | ≤ k − 1 in the summation of formula (D4) means that the sets V = ∅ and V = {1, …, k} are omitted from the summation, i.e. the terms where either α_{V} (j) = 1 or α_{V} (j) = 2 for all 1 ≤ j ≤ k are not considered. Formula (14.13d) can be derived from formula (D4) by means of an inductive argument. The hard part of the problem is to prove formula (D4). To do this first the following simple lemma will be proved.

Lemma D1. Let ξ and η be two independent and identically distributed random variables taking values in a separable Banach space B. Then

3P (|ξ + η| > \frac{2}{3}u) ≥ P (|ξ| > u) for all u > 0.

Proof of Lemma D1. Let ξ, η and ζ be three independent, identically distributed random variables taking values in B. Then

3P (|ξ + η| > \frac{2}{3}u) = P (|ξ + η| > \frac{2}{3}u) + P (|ξ + ζ| > \frac{2}{3}u) + P (| − (η + ζ)| > \frac{2}{3}u)
≥ P (|ξ + η + ξ + ζ − η − ζ| > 2u) = P (|ξ| > u).

To prove formula (D4) we introduce the random variable

T_{n,k}(f(ℓ)) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}), (s_{1},...,s_{k}): \\ 1≤l_{j}≤n, s_{j}=1 or s_{j}=2, j=1,...,k, \end{matrix}} f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(s_{1})}, …, ξ_{l_{k}}^{(s_{k})}) = ∑_{V ⊂\{1,...,k\}} I_{n,k,V} (f(ℓ)).

(D5)

Observe that the random variables I_{n,k}(f(ℓ)), I_{n,k,∅}(f(ℓ)) and I_{n,k,{1,...,k}}(f(ℓ)) are identically distributed, and the last two random variables are independent of each other. Hence Lemma D1 yields that

P (∥I_{n,k}(f(ℓ))∥ > u) ≤ 3P (∥I_{n,k,∅}(f(ℓ)) + I_{n,k,\{1,...,k\}}(f(ℓ))∥ > \frac{2}{3}u)

= 3P (∥T_{n,k}(f(ℓ)) − ∑_{V: V ⊂\{1,...,k\}, 1≤|V |≤k−1} I_{n,k,|V |}(f(ℓ))∥ > \frac{2}{3}u)

≤ 3P (3 · 2^{k−1}∥T_{n,k}(f(ℓ))∥ > u)  (D6)

+ ∑_{V: V ⊂{1,...,k}, 1≤|V |≤k−1} 3P (3 · 2^{k−1}∥I_{n,k,|V |}(f(ℓ))∥ > u).

To derive relation (D4) from relation (D6) a good estimate is needed on the probability P (3·2^{k−1}∥T_{n,k}(f(ℓ))∥ > u). To get such an estimate the tail distribution of ∥T_{n,k}(f(ℓ))∥ will be compared with that of ∥I_{n,k,V} (f(ℓ))∥ for an arbitrary set V ⊂ {1, …, k}. This will be done with the help of Lemmas D2 and D4 formulated below.

In Lemma D2 such a random variable ∥Ī_{n,k,V} (f(ℓ))∥ will be constructed whose distribution agrees with that of ∥I_{n,k,V} (f(ℓ))∥. The expression Ī_{n,k,V} (f(ℓ)), whose norm will be investigated will be defined in formulas (D7) and (D8). It is a random polynomial of some Rademacher functions ε_{1}, …, ε_{n}. The coefficients of this polynomial are random variables, independent of the Rademacher functions ε_{1}, …, ε_{n}. Beside this, the constant term of this polynomial equals T_{n,k}(f(ℓ). These properties of the polynomial Ī_{n,k,V} (f(ℓ)) together with Lemma D4 formulated below enable us prove such an estimate on the distribution of ∥T_{n,k}(f(ℓ))∥ that together with formula (D6) imply relation (D4). Let us formulate these lemmas.

Lemma D2. Let us consider a sequence of independent random variables ε_{1}, …, ε_{n}, P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is also independent of the random variables ξ_{1}^{(1)}, …, ξ_{n}^{(1)} and ξ_{1}^{(2)}, …, ξ_{n}^{(2)} appearing in the definition of the modified decoupled U-statistics I_{n,k,V} (f(ℓ)) given in formula (D3). Let us define with their help the sequences of random variables η_{1}^{(1)}, …, η_{n}^{(1)} and η_{1}^{(2)}, …, η_{n}^{(2)} whose elements (η_{l}^{(1)}, η_{l}^{(2)}) = (η_{l}^{(1)}(ε_{l}), η_{l}^{(2)}(ε_{l})), 1 ≤ l ≤ n, are defined by the formula

(η_{l}^{(1)}(ε_{l}), η_{l}^{(2)}(ε_{l})) = (\frac{1 + ε_{l}}{2} ξ_{l}^{(1)} + \frac{1 − ε_{l}}{2} ξ_{l}^{(2)}, \frac{1 − ε_{l}}{2} ξ_{l}^{(1)} + \frac{1 + ε_{l}}{2} ξ_{l}^{(2)}) ,

i.e. let (η_{l}^{(1)}(ε_{l}), η_{l}^{(2)}(ε_{l})) = (ξ_{l}^{(1)}, ξ_{l}^{(2)}) if ε_{l} = 1, and (η_{l}^{(1)}(ε_{l}), η_{l}^{(2)}(ε_{l})) = (ξ_{l}^{(2)}, ξ_{l}^{(1)}) if ε_{l} = −1, 1 ≤ l ≤ n. Then the joint distribution of the pair of sequences of random variables ξ_{1}^{(1)}, …, ξ_{n}^{(1)} and ξ_{1}^{(2)}, …, ξ_{n}^{(2)} agrees with that of the pair of sequences η_{1}^{(1)}, …, η_{n}^{(1)} and η_{1}^{(2)}, …, η_{n}^{(2)}, which is also independent of the sequence ε_{1}, …, ε_{n}.

Let us fix some V ⊂ {1, …, k}, and introduce the random variable

Ī_{n,k,V} (f(ℓ)) = \frac{1}{k!} ∑_{(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k} f_{l_{1},...,l_{k}} (η_{l_{1}}^{(α_{V} (1))}, …, η_{l_{k}}^{(α_{V} (k))}) ,  (D7)

where similarly to formula (D3) α_{V} (j) = 1 if j ∈ V, and α_{V} (j) = 2 if j ∉ V. Then the identity

2^{k}Ī_{n,k,V} (f(ℓ))  (D8)

= \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}), (s_{1},...,s_{k}): \\ 1≤l_{j}≤n, s_{j}=1 or s_{j}=2, j=1,...,k, \end{matrix}} (1 + κ_{s_{1},V}^{(1)} ε_{l_{1}}) ⋯ (1 + κ_{s_{k},V}^{(k)} ε_{l_{k}})f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(s_{1})}, …, ξ_{l_{k}}^{(s_{k})})

holds, where κ_{1,V}^{(j)} = 1 and κ_{2,V}^{(j)} = −1 if j ∈ V, and κ_{1,V}^{(j)} = −1 and κ_{2,V}^{(j)} = 1 if j ∉ V,

i.e. κ_{1,V}^{(j)} = 3 − 2α_{V} (j) and κ_{2,V}^{(j)} = −κ_{1,V}^{(j)} .

Before the formulation of Lemma D4 another Lemma D3 will be presented which will be applied in its proof.

Lemma D3. Let Z be a random variable taking values in a separable Banach space B with expectation zero, i.e. let Eκ(Z) = 0 for all κ ∈ B′, where B′ denotes the (Banach) space of all (bounded) linear transformations of B to the real line. Then

P (∥v + Z∥ ≥ ∥v∥) ≥ inf_{κ∈B′} \frac{(E|κ(Z)|)^{2}}{4Eκ(Z)^{2}} for all v ∈ B.

Lemma D4. Let us consider a positive integer n and a sequence of independent random variables ε_{1}, …, ε_{n}, P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n. Beside this, fix some positive integer k, take a separable Banach space B and choose some elements a_{s}(l_{1}, …, l_{s}) of this Banach space B, 1 ≤ s ≤ k, 1 ≤ l_{j} ≤ n, l_{j} ≠ l_{j′} if j ≠ j′, 1 ≤ j, j′ ≤ s. With the above notations the inequality

P (∥v + ∑_{s=1}^{k} ∑_{\begin{matrix}(l_{1},...,l_{s}): 1≤l_{j}≤n, j=1,...,s, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} a_{s}(l_{1}, …, l_{s})ε_{l_{1}} ⋯ ε_{l_{s}} ∥ ≥ ∥v∥) ≥ c_{k}  (D9)

holds for all v ∈ B with some constant c_{k} > 0 which depends only on the parameter k.

Proof of Lemma D2. Let us consider the conditional joint distribution of the sequences of random variables η_{1}^{(1)}, …, η_{n}^{(1)} and η_{1}^{(2)}, …, η_{n}^{(2)} under the condition that the random vector ε_{1}, …, ε_{n} takes the value of some prescribed ±1 series of length n. Observe that this conditional distribution agrees with the joint distribution of the sequences ξ_{1}^{(1)}, …, ξ_{n}^{(1)} and ξ_{1}^{(2)}, …, ξ_{n}^{(2)} for all possible conditions. This fact implies the statement about the joint distribution of the sequences η_{l}^{(1)}, η_{l}^{(2)}, 1 ≤ l ≤ n and their independence of the sequence ε_{1}, …, ε_{n}.

To prove identity (D8) let us fix a set M ⊂ {1, …, n}, and consider the case when ε_{l} = 1 if l ∈ M and ε_{l} = −1 if l ∉ M. Put β_{V,M}(j, l) = 1 if j ∈ V and l ∈ M or j ∉ V and l ∉ M, and let β_{V,M}(j, l) = 2 otherwise. Then we have for all (l_{1}, …, l_{k}), 1 ≤ l_{j} ≤ n, 1 ≤ j ≤ n, and our fixed set V

\begin{matrix}∑_{(s_{1},...,s_{k}): s_{j}=1 or s_{j}=2, j=1,...,k,}(1 + κ_{s_{1},V}^{(1)} ε_{l_{1}}) ⋯ (1 + κ_{s_{k},V}^{(k)} ε_{l_{k}})f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(s_{1})}, …, ξ_{l_{k}}^{(s_{k})}) \\ = 2^{k}f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(β_{V,M} (1,l_{1}))}, …, ξ_{l_{k}}^{(β_{V,M} (k,l_{k}))}) , \end{matrix}

(D10)

since the product (1+κ_{s_{1},V}^{(1)} ε_{l_{1}}) ⋯ (1+κ_{s_{k},V}^{(k)} ε_{l_{k}}) equals either zero or 2^{k}, and it equals 2^{k}

for that sequence (s_{1}, …, s_{k}) for which κ_{s_{j},V}^{(j)} ε_{l_{j}} = 1 for all 1 ≤ j ≤ k, and the relation

κ_{s_{j},V}^{(j)} ε_{l_{j}} = 1 is equivalent to β_{V,M}(j, l_{j}) = s_{j} for all 1 ≤ j ≤ k. (In relation (D10) it is sufficient to consider only such products for which l_{j} ≠ l_{j′} if j ≠ j′ because of the properties of the functions f_{l_{1},...,l_{k}}.)

Beside this, ξ_{l}^{β_{V,M} (l,j)} = η_{l}^{α_{V} (j)} for all 1 ≤ l ≤ n and 1 ≤ j ≤ k, and as a consequence

f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(β_{V,M} (1,l_{1}))}, …, ξ_{l_{k}}^{(β_{V,M} (k,l_{k}))}) = f_{l_{1},...,l_{k}} (η_{l_{1}}^{(α_{V} (1))}, …, η_{l_{k}}^{(α_{V} (k))}) .

Summing up the identities (D10) for all 1 ≤ l_{1}, …, l_{k} ≤ n and applying the last identity we get relation (D8), since the identity obtained in such a way holds for all M ⊂ {1, …, n}.

Proof of Lemma D3. Let us first observe that if ξ is a real valued random variable with zero expectation, then P (ξ ≥ 0) ≥ \frac{(E|ξ|)^{2}}{4Eξ^{2}} since (E|ξ|)^{2} = 4(E(ξI(\{ξ ≥ 0\}))^{2} ≤ 4P (ξ ≥ 0)Eξ^{2} by the Schwarz inequality, where I(A) denotes the indicator function of the set A. (In the above calculation and in the subsequent proofs I apply the convention \frac{0}{0} = 1. We need this convention if Eξ^{2} = 0. In this case we have, because of the condition Eξ = 0 the identity P (ξ = 0) = 1, hence the above proved identity holds in this case, too.)

Given some v ∈ B, let us choose a linear operator κ such that ∥κ∥ = 1 and κ(v) = ∥v∥. Such an operator exists by the Banach–Hahn theorem. Observe that {ω: ∥v + Z(ω)∥ ≥ ∥v∥} ⊃ {ω: κ(v + Z(ω)) ≥ κ(v)} = {ω: κ(Z(ω)) ≥ 0}. Beside this Eκ(Z) = 0. Hence we can apply the above proved inequality for ξ = κ(Z), and it yields that P (∥v + Z∥ ≥ ∥v∥) ≥ P (κ(Z) ≥ 0) ≥ \frac{(E|κ(Z)|)^{2}}{4Eκ(Z)^{2}} . Lemma D3 is proved.

Proof of Lemma D4. Take the class of random polynomials

Y = ∑_{s=1}^{k} ∑_{\begin{matrix}(l_{1},...,l_{s}): 1≤l_{j}≤n, j=1,...,s, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} b_{s}(l_{1}, …, l_{s})ε_{l_{1}} ⋯ ε_{l_{s}},

where ε_{l}, 1 ≤ l ≤ n, are independent random variables with P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, and the coefficients b_{s}(l_{1}, …, l_{s}), 1 ≤ s ≤ k, are arbitrary real numbers. The proof of Lemma D4 can be reduced to the statement that there exists a constant c_{k} > 0 depending only on the order k of these polynomials such that the inequality

(E|Y |)^{2} ≥ 4c_{k}EY^{2}.  (D11)

holds for all such polynomials Y. Indeed, consider the polynomial

Z = ∑_{s=1}^{k} ∑_{\begin{matrix}(l_{1},...,l_{s}): 1≤l_{j}≤n, j=1,...,s, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} a_{s}(l_{1}, …, l_{s})ε_{l_{1}} ⋯ ε_{l_{s}},

and observe that Eκ(Z) = 0 for all linear functionals κ on the space B. Hence Lemma D3 implies that the left-hand side expression in (D9) is bounded from below by inf_{κ∈B′} \frac{(E|κ(Z)|)^{2}}{4Eκ(Z)^{2}} . On the other hand, relation (D11) implies that inf_{κ∈G′} \frac{(E|κ(Z)|)^{2}}{4Eκ(Z)^{2}} ≥ c_{k}.

To prove relation (D11) first we compare the moments EY^{2} and EY^{4}. Let us introduce the random variables

Y_{s} = ∑_{\begin{matrix}(l_{1},...,l_{s}): 1≤l_{j}≤n, j=1,...,s, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} b_{s}(l_{1}, …, l_{s})ε_{l_{1}} ⋯ ε_{l_{s}} 1 ≤ s ≤ k.

We shall show that the estimates of Section 13 imply that

EY_{s}^{4} ≤ 2^{4s} (EY_{s}^{2})^{2}  (D12)

for these random variables Y_{s}.

Relation (D12) together with the uncorrelatedness of the random variables Y_{s}, 1 ≤ s ≤ k, imply that

EY^{4} = E (∑_{s=1}^{k} Y_{s})^{4} ≤ k^{3} ∑_{s=1}^{k} EY_{s}^{4} ≤ k^{3}2^{4k} ∑_{s=1}^{k}(EY_{s}^{2})^{2}

≤ k^{3}2^{4k} (∑_{s=1}^{k} EY_{s}^{2})^{2} = k^{3}2^{4k}(EY^{2})^{2}.

This estimate together with the Hölder inequality with p = 3 and q = \frac{3}{2} yield that EY^{2} = E|Y |^{4/3}| · |Y |^{2/3} ≤ (EY^{4})^{1/3}(E|Y |)^{2/3} ≤ k2^{4k/3}(EY^{2})^{2/3}(E|Y |)^{2/3}, i.e. EY^{2} ≤ k^{3}2^{4k}(E|Y |)^{2}, and relation (D11) holds with 4c_{k} = k^{−3}2^{−4k}. Hence to complete the proof of Lemma D4 it is enough to check relation (D12).

In the proof of relation (D12) it can be assumed that the coefficients b_{s}(l_{1}, …, l_{s}) of the random variable Y_{s} are symmetric functions of the arguments l_{1}, …, l_{s}, since a symmetrization of these coefficients does not change the value of Y. Put

B_{s}^{2} = ∑_{\begin{matrix}(l_{1},...,l_{s}): 1≤l_{j}≤n, j=1,...,s, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} b_{s}^{2}(l_{1}, …, l_{s}), 1 ≤ s ≤ k.

Then

EY_{s}^{2} = s!B_{s}^{2},

and

EY_{s}^{4} ≤ 1 · 3 · 5 ⋯ (4s − 1)B_{s}^{4} = \frac{(4s)!}{2^{2s}(2s)!}B_{s}^{4}

by Lemmas 13.4 and 13.5 with the choice M = 2 and k = s. Inequality (D12) follows from the last two relations. Indeed, to prove formula (D12) by means of these relations it is enough to check that \frac{(4s)!}{2^{2s}(2s)!(s!)^{2}} ≤ 2^{4s}. But it is easy to check this inequality with induction with respect to s. (Actually, there is a well-known inequality in the literature, known under the name Borell’s inequality, which implies inequality (D12) with a better coefficient at the right hand side of this estimate.)

Let us turn back to the estimation of the probability P (3 · 2^{k−1}∥T_{n,k}(f)∥ > u). Let us introduce the σ-algebra F = B(ξ_{l}^{(1)}, ξ_{l}^{(2)}, 1 ≤ l ≤ n) generated by the random variables ξ_{l}^{(1)}, ξ_{l}^{(2)}, 1 ≤ l ≤ n, and fix some set V ⊂ {1, …, k}. I show with the help of Lemma D4 and formula (D8) that there exists some constant c_{k} > 0 such that the random variables T_{n,k}f(ℓ)) defined in formula (D5) and Ī_{n,k,V} (f(ℓ)) defined in formula (D7) satisfy the inequality

P (∥2^{k}Ī_{n,k,V} (f(ℓ))∥ > ∥T_{n,k}(f(ℓ))∥|F) ≥ c_{k} with probability 1.  (D13)

In the proof I shall exploit that in formula (D8) 2^{k}Ī_{n,k,V} (f(ℓ)) is represented by a polynomial of the Rademacher functions ε_{1}, …, ε_{n} whose constant term is T_{n,k}(f(ℓ)). The coefficients of this polynomial are functions of the random variables ξ_{l}^{(1)} and ξ_{l}^{(2)}, 1 ≤ l ≤ n. The independence of these random variables from ε_{l}, 1 ≤ l ≤ n, and the definition of the σ-algebra F yield that

P (∥2^{k}Ī_{n,k,V} (f(ℓ))∥ > ∥T_{n,k}(f(ℓ))∥|F)

= P_{ε_{V}} (∥\frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}), (s_{1},...,s_{k}): \\ 1≤l_{j}≤n,s_{j}=1 or s_{j}=2, j=1,...,k, \end{matrix}} (1 + κ_{s_{1},V}^{(1)} ε_{l_{1}}) ⋯ (1 + κ_{s_{k},V}^{(k)} ε_{l_{k}})f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(s_{1})}, …, ξ_{l_{k}}^{(s_{k})})∥

> ∥T_{n,k}(f(ℓ))(ξ_{l}^{(j)}, 1 ≤ l ≤ n, j = 1, 2)∥),  (D14)

where P_{ε_{V}} means that the values of the random variables ξ_{l}^{(1)}, ξ_{l}^{(2)}, 1 ≤ l ≤ n, are fixed, (their value depend on the atom of the σ-algebra F we are considering) and the probability is taken with respect to the remaining random variables ε_{l}, 1 ≤ l ≤ n. At the right-hand side of (D14) the probability of such an event is considered that the norm of a polynomial of order k of the random variables ε_{1}, …, ε_{n} is larger than ∥T_{n,k}(f(ℓ))(ξ_{l}^{(j)}, 1 ≤ l ≤ n, j = 1, 2)∥. Beside this, the constant term of this polynomial equals T_{n,k}(f(ℓ))(ξ_{l}^{(j)}, 1 ≤ l ≤ n, j = 1, 2). Hence this probability can be bounded by means of Lemma D4, and this result yields relation (D13).

As the distributions of I_{n,k,V} (f(ℓ)) and Ī_{n,k,V} (f(ℓ)) agree by the first statement of Lemma D2 and a comparison of formulas (D3) and (D7), relation (D13) implies that

P (∥2^{k}I_{n,k,V} (f(ℓ))∥ ≥ \frac{1}{3} · 2^{1−k}u) = P (∥2^{k}Ī_{n,k,V} (f(ℓ))∥ ≥ \frac{1}{3} · 2^{1−k}u)

≥ P (∥2^{k}Ī_{n,k,V} (f(ℓ))∥ ≥ ∥T_{n,k}(f(ℓ))∥, ∥T_{n,k}(f(ℓ))∥ ≥ \frac{1}{3} · 2^{1−k}u)

= ∫_{\{ω: ∥T_{n,k}(f(ℓ))(ω)∥≥ \frac{1}{3} ·2^{1−k}u\}} P (∥2^{k}Ī_{n,k,V} (f(ℓ))∥ > ∥T_{n,k}(f(ℓ))∥|F) dP

≥ c_{k}P (3 · 2^{k−1}∥T_{n,k}(f(ℓ))∥ ≥ u).

The last inequality with the choice of any set V ⊂ {1, …, k}, 1 ≤ |V | ≤ k − 1, together with relation (D6) imply formula (D4).

Relation (14.13d) will be proved together with another inductive hypothesis with the help of relation (D4) by means of an induction procedure with respect to the order k of the U-statistic. To formulate the other inductive hypothesis some new quantities will be introduced. Let W = W(k) denote the set of all partitions of the set {1, …, k}. Let us fix k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of the sequence of random variables ξ_{1}, …, ξ_{n}. Given a partition W = (U_{1}, …, U_{s}) ∈ W(k) let us introduce the function s_{W} (j), 1 ≤ j ≤ k, which tells for all arguments j the index of that element of the partition W which contains the point j, i.e. the value of the function s_{W} (j), 1 ≤ j ≤ k, in a point j is defined by the relation j ∈ V_{s_{W} (j)}. Let us introduce the expression

I_{n,k,W} (f(ℓ)) = \frac{1}{k!} ∑_{(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k} f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(s_{W} (1))}, …, ξ_{l_{k}}^{(s_{W} (k))})
for all W ∈ W(k).

An expression of the form I_{n,k,W} (f(ℓ)), W ∈ W_{k}, will be called a decoupled U-statistic with generalized decoupling. Given a partition W = (U_{1}, …, U_{s}) ∈ W_{k} let us call the number s, i.e. the number of the elements of this partition the rank both of the partition W and of the decoupled U-statistic I_{n,k,W} (f(ℓ)) with generalized decoupling.

Now I formulate the following hypothesis. For all k ≥ 2 and 2 ≤ j ≤ k there exist some constants C(k, j) > 0 and δ(k, j) > 0 such that for all W ∈ W_{k} a decoupled U-statistic I_{n,k,W} (f(ℓ)) with generalized decoupling satisfies the inequality

P (∥I_{n,k,W} (f(ℓ))∥ > u) ≤ C(k, j)P (∥Ī_{n,k}(f(ℓ))∥ > δ(k, j)u)

for all 2 ≤ j ≤ k if the rank of W equals j.

(D15)

It will be proved by induction with respect to k that both relations (14.13d) and (D15) hold for U-statistics of order k. Let us observe that for k = 2 relation (14.13d) follows from (D4). Relation (D15) also holds for k = 2, since in this case we have to consider only the case j = k = 2, and relation (D15) clearly holds in this case with C(2, 2) = 1 and δ(2, 2) = 1. Hence we can start our inductive proof with k = 3. First I prove relation (D15).

In relation (D15) the tail-distribution of decoupled U-statistics with generalized decoupling is compared with that of the decoupled U-statistic Ī_{n,k}(f(ℓ)) introduced in (D2). Given the order k of these U-statistics it will be proved by means of a backward induction with respect to the rank j of the decoupled U-statistics I_{n,k,W} (f(ℓ)) with generalized decoupling.

Relation (D15) clearly holds for j = k with C(k, k) = 1 and δ(k, k) = 1. To prove it for decoupled U-statistics with generalized decoupling of rank 2 ≤ j < k first the following observation will be made. If the rank j of the partition W = (U_{1}, …, U_{j}) satisfies the relation 2 ≤ j ≤ k − 1, then it contains an element with cardinality strictly less than k and strictly greater than 1. For the sake of simpler notation let us assume that the element U_{j} of this partition is such an element, and U_{j} = {t, …, k} with some 2 ≤ t ≤ k − 1. The investigation of general U-statistics of rank j, 2 ≤ j ≤ k − 1, can be reduced to this case by a reindexation of the arguments in the U-statistics if it is necessary. Let us consider the partition W̄ = (U_{1}, …, U_{j−1}, {t}, …, {k}) and the decoupled U-statistic I_{n,k,W̄} (f(ℓ)) with generalized decoupling corresponding to this partition W̄. It will be shown that our inductive hypothesis implies the inequality

P (∥I_{n,k,W} (f(ℓ))∥ > u) ≤ Ā(k)P (∥I_{n,k,W̄} (f(ℓ))∥ > γ̄(k)u)  (D16)

with Ā(k) = sup_{2≤p≤k−1} A(p), γ̄(k) = inf_{2≤p≤k−1} γ(p) if the rank j of W is such that 2 ≤ j ≤

k − 1, where the constants A(p) and γ(p) agree with the corresponding coefficients in formula (14.13d).

To prove relation (D16) (in the case when U_{j} = {t, …, k}) let us define the σalgebra F generated by the random variables appearing in the first t − 1 coordinates of these U-statistics, i.e. by the random variables ξ_{l_{j}}^{s_{W} (j)}, 1 ≤ j ≤ t − 1, and 1 ≤ l_{j} ≤ n for all 1 ≤ j ≤ t − 1. We have 2 ≤ t ≤ k − 1. By our inductive hypothesis relation (14.13d) holds for U-statistics of order p = k −t+1, since 2 ≤ p ≤ k −1. I claim that this implies that

P (∥I_{n,k,W} (f(ℓ))∥ > u|F) ≤ A(k − t + 1)P (∥I_{n,k,W̄} (f(ℓ))∥ > γ(k − t + 1)u|F) (D17)

with probability 1. Indeed, by the independence properties of the random variables ξ_{l}^{s_{W} (j)} (and ξ_{l}^{sW̄ (j)}), 1 ≤ j ≤ k, 1 ≤ l ≤ n,

P (∥I_{n,k,W} (f(ℓ))∥ > u|F) = P_{ξ_{l}^{s_{W} (j)},1≤j≤t−1}(∥I_{n,k,W} (f(ℓ)∥ > u)

and

P (∥I_{n,k,W̄} (f(ℓ))∥ > γ(k − t + 1)u|F) = P_{ξ_{l}^{s_{W} (j)},1≤j≤t−1}(∥I_{n,k,W̄} f(ℓ)∥ > γ(k − t + 1)u),

where P_{ξ_{l}^{s_{W} (j)},1≤j≤t−1} denotes that the values of the random variables ξ_{l}^{s_{W} (j)}(ω), 1 ≤ j ≤ t − 1, 1 ≤ l ≤ n, are fixed, and we consider the probability that the appropriate functions of these fixed values and of the remaining random variables ξ^{s_{W} (j)} and ξ^{sW̄ (j)}, t ≤ j ≤ k, satisfy the desired relation. These identities and the relation between the sets W and W̄ imply that relation (D17) is equivalent to the identity (14.13d) for the generalized U-statistics of order 2 ≤ k − t + 1 ≤ k − 1 with kernel functions

f_{l_{t},...,l_{k}}(x_{t}, …, x_{k})

= ∑_{(l_{1},...,l_{t−1}): 1≤l_{j}≤n, 1≤j≤t−1} f_{l_{1},...,l_{k}}(ξ_{l_{1}}^{s_{W} (1)}(ω), …, ξ_{l_{t−1}}^{s_{W} (t−1)}(ω), x_{t}, …, x_{k}).

Relation (D16) follows from inequality (D17) if expectation is taken at both sides. As the rank of W̄ is strictly greater than the rank of W, relation (D16) together with our backward inductive assumption imply relation (D15) for all 2 ≤ j ≤ k.

Relation (D15) implies in particular (with the applications of partitions of order k and rank 2) that the terms in the sum at the right-hand side of (D4) satisfy the inequality P (D_{k}∥I_{n,k,V} (f(ℓ))∥ > u) ≤ C̄(k, j)P (∥Ī_{n,k}(f(ℓ))∥ > D̄_{k}u) with some appropriate C̄_{k} > 0 and D̄_{k} > 0 for all V ⊂ {1, …, k}, 1 ≤ |V | ≤ k − 1. This inequality together with relation (D4) imply that inequality (14.13d) also holds for the parameter k.

In such a way we get the proof of relation (14.13d) and of its special case, relation (14.13). Let us prove formula (14.14) with its help first in the simpler case when the supremum of finitely many functions is taken. If M < ∞ functions f_{1}, …, f_{M} are considered, then relation (14.14) for the supremum of the U-statistics and decoupled U-statistics with these kernel functions can be derived from formula (14.13) if it is applied for the function f = (f_{1}, …, f_{M}) with values in the separable Banach space B_{M} which consists of the vectors (v_{1}, …, v_{M}), v_{j} ∈ B, 1 ≤ j ≤ M, and the norm ∥(v_{1}, …, v_{M})∥ = sup_{1≤j≤m} ∥v_{j}∥ is introduced in it. The application of formula (14.13)

with this choice yields formula (14.14) for this supremum. Let us emphasize that the constants appearing in this estimate do not depend on the number M. (We took only M < ∞ kernel functions, because with such a choice the Banach space B_{M} defined above is also separable.) Since the distribution of the random variables sup_{1≤s≤M} ∥I_{n,k}(f_{s})∥

converge to that of sup_{1≤s<∞} ∥I_{n,k}(f_{s})∥, and the distribution of the random variables

sup_{1≤s≤M} ∥Ī_{n,k}(f_{s})∥ converge to that of sup_{1≤s<∞} ∥Ī_{n,k}(f_{s})∥ as M → ∞, relation (14.14)

in the general case follows from its already proved special case and a limiting procedure M → ∞.

Remark. The above proved formula (14.13d) can be slightly generalized. It also holds if the expressions I_{n,k}(f(ℓ)) and Ī_{n,k}(f(ℓ)) appearing in this inequality are defined in a more general way. Namely, they are the random functions introduced in formulas (D1) and (D2), but the sequences ξ_{1}, …, ξ_{n} and their independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)} in these formulas are independent random variables which may also be non-identically distributed. This generalization can be shown without any essential change in the original proof.

References:

1.) Adamczak, R. (2006) Moment inequalities for U-statistics. Annals of Probability

34, 2288–2314 2.) Alexander, K. (1987) The central limit theorem for empirical processes over Vapnik–

Červonenkis classes. Annals of Probability 15, 178–203 3.) Arcones, M. A. and Giné, E. (1993) Limit theorems for U-processes. Annals of

Probability, 21, 1494–1542 4.) Arcones, M. A. and Giné, E. (1994) U-processes indexed by Vapnik–Červonenkis

classes of functions with application to asymptotics and bootstrap of U-statistics with estimated parameters. Stoch. Proc. Appl. 52, 17–38

5.) Bennett, G. (1962) Probability inequality for the sum of independent random vari-

ables. J. Amer. Statist. Assoc. 57, 33–45 6.) Bonami, A. (1970) Étude des coefficients de Fourier des fonctions de L^{p}(G). Ann.

Inst. Fourier (Grenoble) 20 335–402 7.) de la Peña, V. H. and Giné, E. (1999) Decoupling. From dependence to inde-

pendence. Springer series in statistics. Probability and its application. Springer Verlag, New York, Berlin, Heidelberg

8.) de la Peña, V. H. and Montgomery–Smith, S. (1995) Decoupling inequalities for

the tail-probabilities of multivariate U-statistics. Ann. Probab., 23, 806–816 9.) Dobrushin, R. L. (1979) Gaussian and their subordinated fields. Annals of Proba-

bility 7, 1-28 10.) Dudley, R. M. (1978) Central limit theorems for empirical measures. Annals of

Probability 6, 899–929 11.) Dudley, R. M. (1984) A course on empirical processes. Lecture Notes in Mathemat-

ics 1097, 1–142 Springer Verlag, New York 12.) Dudley, R. M. (1989) Real Analysis and Probability. Wadsworth & Brooks, Pacific

Grove, California 13.) Dudley, R. M. (1998) Uniform Central Limit Theorems. Cambridge University

Press, Cambridge U.K. 14.) Dynkin, E. B. and Mandelbaum, A. (1983) Symmetric statistics, Poisson processes

and multiple Wiener integrals. Annals of Statistics 11, 739–745 15.) Frankl, P. and Pach J. (1983) On the number of sets in null-t-design. European J.

Combinatorics 4 21–23 16.) Giné, E. and Guillou, A. (2001) On consistency of kernel density estimators for

randomly censored data: Rates holding uniformly over adaptive intervals. Ann. Inst. Henri Poincaré PR 37 503–522

17.) Giné, E., Kwapień, S, Latała, R. and Zinn, J. (2001) The LIL for canonical U-

statistics of order 2. Annals of Probability 29 520–527 18.) Giné, E., Latała, R. and Zinn, J. (2000) Exponential and moment inequalities for

U-statistics in High dimensional probability II. Progress in Probability 47. 13–38. Birkhäuser Boston, Boston, MA.

19.) Gross, L. (1975) Logarithmic Sobolev inequalities. Amer. J. Math. 97, 1061–1083 20.) Guionnet, A. and Zegarlinski, B. (2003) Lectures on Logarithmic Sobolev inequal-

ities. Lecture Notes in Mathematics 1801 1–134 2. Springer Verlag, New York 21.) Hanson, D. L. and Wright, F. T. (1971) A bound on the tail probabilities for

quadratic forms in independent random variables. Ann. Math. Statist. 42 52–61 22.) Hoeffding, W. (1948) A class of statistics with asymptotically normal distribution.

Ann. Math. Statist. 19 293–325 23.) Hoeffding, W. (1963) Probability inequalities for sums of bounded random vari-

ables. J. Amer. Math. Society 58, 13–30 24.) Itô K. (1951) Multiple Wiener integral. J. Math. Soc. Japan 3. 157–164 25.) Kaplan, E.L. and Meier P. (1958) Nonparametric estimation from incomplete data,

Journal of American Statistical Association, 53, 457–481. 26.) Latała, R. (2006) Estimates of moments and tails of Gaussian chaoses. Annals of

Probability 34 2315–2331 27.) Ledoux, M. (1996) On Talagrand deviation inequalities for product measures.

ESAIM: Probab. Statist. 1. 63–87. Available at http://www.emath./fr/ps/. 28.) Ledoux, M. (2001) The concentration of measure phenomenon. Mathematical Sur-

veys and Monographs 89 American Mathematical Society, Providence, RI. 29.) Major, P. (1981) Multiple Wiener–Itô integrals. Lecture Notes in Mathematics

849, Springer Verlag, Berlin, Heidelberg, New York, 30.) Major, P. (1988) On the tail behaviour of the distribution function of multiple

stochastic integrals. Probability Theory and Related Fields, 78, 419–435 31.) Major, P. (1994) Asymptotic distributions for weighted U-statistics. The Annals

of Probability, 22 1514–1535 32.) Major, P. (2005) An estimate about multiple stochastic integrals with respect to

a normalized empirical measure. Studia Scientarum Mathematicarum Hungarica. 295–341

33.) Major, P. (2005) Tail behaviour of multiple random integrals and U-statistics.

Probability Reviews. 448–505 34.) Major, P. (2006) An estimate on the maximum of a nice class of stochastic integrals.

Probability Theory and Related Fields. 134, 489–537 35.) Major, P. (2006) A multivariate generalization of Hoeffding’s inequality. Electronic

Communication in Probability 2 (220–229) 36.) Major, P. (2007) On a multivariate version of Bernstein’s inequality Electronic

Journal of Probability 12 966–988 37.) Major, P. (2005) On the tail behaviour of multiple random integrals and degenerate

U-statistics. (First version of this lecture note) http://www.renyi.hu/˜ major 38.) Major, P. and Rejtő, L. (1988) Strong embedding of the distribution function under

random censorship. Annals of Statistics 16, 1113–1132 39.) Major, P. and Rejtő, L. (1998) A note on nonparametric estimations. In the con-

ference volume to the 65. birthday of Miklós Csörgő. 759–774 40.) Malyshev, V. A. and Minlos, R. A. (991) Gibbs Random Fields. Method of cluster

expansion. Kluwer, Academic Publishers, Dordrecht 41.) Massart, P. (2000) About the constants in Talagrand’s concentration inequalities

for empirical processes. Annals of Probability 28, 863–884 42.) Mc. Kean, H. P. (1973) Wiener’s theory of non-linear noise. in Stochastic Differ-

ential Equations SIAM–AMS Proc. 6 197–209 43.) Nelson, E. (1973) The free Markov field. J. Functional Analysis 12, 211–227 44.) Pollard, D. (1984) Convergence of Stochastic Processes. Springer Verlag, New York 45.) Rota, G.-C. and Wallstrom, C. (1997) Stochastic integrals: a combinatorial ap-

proach. Annals of Probability 25 (3) 1257–1283 46.) Surgailis, D. (1984) On multiple Poisson stochastic integrals and associated Markov

semigroups. Probab. Math. Statist. 3. no. 2 217-239 47.) Surgailis, D. (2000) Long-range dependence and Appell rank. Annals of Probability

28 478–497 48.) Szegő, G. (1967) Orthogonal Polynomials. American Mathematical Society Collo-

quium Publications. Vol. 23 49.) Takemura, A. (1983) Tensor Analysis of ANOVA decomposition. J. Amer. Statist.

Assoc. 78, 894–900 50.) Talagrand, M. (1994) Sharper bounds for Gaussian and empirical processes. Annals

of Probability 22, 28–76 51.) Talagrand, M. (1996) New concentration inequalities in product spaces. Invent.

Math. 126, 505–563 52.) Talagrand, M. (2005) The general chaining. Springer Monographs in Mathematics.

Springer Verlag, Berlin Heidelberg New York 53.) Vapnik, V. N. (1995) The Nature of Statistical Learning Theory. Springer Verlag,

New York

CONTENT

1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1
2. Motivation of the investigation. Discussion of some problems…  3
3. Some estimates about sums of independent random variables… 10
4. On the supremum of a nice class of partial sums. . . . . . . . . . . . . . . 15
5. Vapnik–Červonenkis classes and L_{2}-dense classes of functions. . 25

6. The proof of Theorems 4.1 and 4.2 on the supremum of random

sums. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

7. The completion of the proof of Theorem 4.1. . . . . . . . . . . . . . . . . . . . 39
8. Formulation of the main results of this work. . . . . . . . . . . . . . . . . . . 48
9. Some results about U-statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
10. Multiple Wiener–Itô integrals and their properties. . . . . . . . . . . . . 74
11. The diagram formula for products of degenerate U-statistics… 90
12. The proof of the diagram formula for U-statistics. . . . . . . . . . . . . . 102
13. The proof of Theorems 8.3, 8.5 and Example 8.7. . . . . . . . . . . . . . . 109
14. Reduction of the main result in this work. . . . . . . . . . . . . . . . . . . . . . 124
15. The strategy of the proof for the main result of this work. . . . . . 133
16. A symmetrization argument. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
17. The proof of the main result. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
18. An overview of the results in this work. . . . . . . . . . . . . . . . . . . . . . . . . 170

Appendix A. The proof of some results about Vapnik–Červonenkis

classes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

Appendix B. The proof of the diagram formula for Wiener–Itô

integrals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

Appendix C. The proof of some results about Wiener–Itô inte-

grals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

Appendix D. The proof of Theorem 14.3. ( A result about the comparison of U-statistics and decoupled U-statistics.) . . . . . . . . .

202

References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
Content. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215