Péter Major

On the estimation of multiple random integrals and U-statistics

– Lecture Note –

February 4, 2013

Springer

Contents

1 Introduction ................................................... 1 2 Motivation of the investigation. Discussion of some problems ........ 5 3 Some estimates about sums of independent random variables ....... 13 4 On the supremum of a nice class of partial sums ................... 19 5 Vapnik–Červonenkis classes and L_{2}-dense classes of functions ....... 31 6 The proof of Theorems 4.1 and 4.2 on the supremum of random sums 37 7 The completion of the proof of Theorem 4.1 ....................... 49 8 Formulation of the main results of this work ...................... 61 9 Some results about U-statistics................................... 75 10 Multiple Wiener–Itô integrals and their properties ................. 91 11 The diagram formula for products of degenerate U-statistics ........ 115 12 The proof of the diagram formula for U-statistics .................. 133 13 The proof of Theorems 8.3, 8.5 and Example 8.7 ................... 143 14 Reduction of the main result in this work ......................... 161 15 The strategy of the proof for the main result of this work ........... 171 16 A symmetrization argument ..................................... 179 17 The proof of the main result ..................................... 197

vi Contents

18 An overview of the results and a discussion of the literature ......... 213 A The proof of some results about Vapnik–Červonenkis classes ........ 233 B The proof of the diagram formula for Wiener–Itô integrals ......... 237 C The proof of some results about Wiener–Itô integrals............... 247 D The proof of Theorem 14.3 about U-statistics and decoupled

U-statistics .................................................... 255 References ..................................................... 265

Index ............................................................. 269

Preface

This lecture note has a fairly long history. Its starting point was an attempt to solve some limit problems about the behaviour of non-linear functionals of a sequence of independent random variables. These problems could not be solved by means of classical probabilistic methods. I tried to solve them with the help of some sort of Taylor expansion. The idea was to represent the functional we are investigating as a sum with a leading term whose asymptotic behaviour can be well described by means of classical results of probability theory and with some error terms whose effect is negligible. This approach worked well, but to bound the error terms I needed some non-trivial estimates. The proof of these estimates was interesting in itself, it was a problem worth of a closer study on its own right. So I tried to work out the details and to present the most important and most interesting results I met during this research. This lecture note is the result of these efforts.

To solve the problems I met I had to give a good estimate on the tail distribution of the integral of a function of several variables with respect to the appropriate power of a normalized empirical distribution. Beside this I also had to consider a generalized version of this problem when the tail distribution of the supremum of such integrals has to be bounded. The difficulties in these problems concentrate around two points.

a) We consider non-linear functionals of independent random variables, and we have to work out some techniques to deal with such problems.

b) The idea behind several arguments is the observation that independent random variables behave in many respects almost as if they were Gaussian. But we have to understand how strong this similarity is, when we can apply the techniques worked out for Gaussian random variables. Beside this we have to find methods to deal with our problems also in such cases when the techniques related to Gaussian and almost Gaussian random variables do not work.

To deal with problem a) I have discussed the theory of multiple random integrals and their most important properties together with the properties of so-called (degenerate) U-statistics. I considered the Wiener–Itô integrals which are multiple Gaussian type integrals, and provide a useful tool to handle non-linear functionals of Gaussian sequences. I also proved some results about a good representation of the product of Wiener–Itô integrals or degenerate U-statistics as a sum of Wiener–Itô integrals or degenerate U-statistics. A comparison of these results indicates some similarity between the behaviour of Wiener–Itô integrals and degenerate U-statistics. I tried to present a fairly detailed discussion of Wiener–Itô integrals and degenerate U-statistics which contains their most important properties.

Problem b) appeared in particular in the study of the supremum of a class of random integrals. It may be worth mentioning that there is a deep theory worked out mainly by Michel Talagrand which gives good estimates in such problems, at least in the case if only one-fold integrals are considered. It turned out however that the results and methods of this theory are not appropriate to prove such estimates that I needed in this work. Roughly speaking, the problems I met have a different character than those investigated in Talagrand’s theory. This point is discussed in more detail in the main text of this work, in particular in Chapter 18, which gives an overview of the problems investigated in this work together with their history. The problems get even harder if the supremum not only of one-fold but also of multiple random integrals have to be estimated. Here some new methods are needed which we can find by refining some symmetrization arguments appearing in the theory of so-called Vapnik–Červonenkis classes.

I have also considered an example in Chapter 2 which shows how to apply the estimates proved in this work in the study of some limit theorem problems in mathematical statistics. Actually this was the starting point of the research described in this work. I discussed only one example, but I consider it more than just an example. My goal was to explain a method that can help in solving some non-trivial limit problems and to show why the results of this lecture notes are useful in their investigation. I think that this approach works in a very general setting, but this is the task of future research. Let me also remark that to understand how this method works and how to apply it one does not have to learn the whole material of this lecture note. It is enough to understand the content of the results in Chapter 8 together with some results of Chapter 9 about the properties of U-statistics.

I had two kinds of readers in mind when writing this lecture note. The first kind of them would like to learn more about such problems in which relatively few independence is available, and as a consequence the methods of classical probability theory do not work in their study. They would like to acquire some results and methods useful in such cases, too. The second kind of readers would not like to go into the details of complicated, unpleasant arguments. They would restrict their attention to some useful methods which may help them in proving the limit theorem problems of probability theory they meet also in such cases when the standard methods do not work. This lecture note can be considered as an attempt to satisfy the wishes of both kinds of readers.

Budapest, January 2013 Péter Major

Chapter 1

Introduction

First I briefly describe the main subject of this work.

Fix a positive integer n, consider n independent and identically distributed random variables ξ_{1},...,ξ_{n} on a measurable space (X,X ) with some distribution μ and take their empirical distribution μ_{n} together with its normalization \sqrt{n}(μ_{n}−μ). Beside this, take a function f(x_{1},...,x_{k}) of k variables on the k-fold product (X^{k},X^{k}) of the space (X,X ), introduce the k-th power of the normalized empirical measure \sqrt{n}(μ_{n}− μ) on (X^{k},X^{k}) and define the integral of the function f with respect to this signed product measure. This integral is a random variable, and we want to give a good estimate on its tail distribution. More precisely, we take the integrals not on the whole space, the diagonals x_{s} = x_{s′}, 1 ≤ s,s′≤ k, s ≠ s′, of the space X^{k} are omitted from the domain of integration. Such a modification of the integral seems to be natural.

We shall also be interested in the following generalized version of the above problem. Let us have a nice class of functions F of k variables on the product space (X^{k},X^{k}), and consider the integrals of all functions in this class with respect to the k-fold direct product of our normalized empirical measure. Give a good estimate on the tail distribution of the supremum of these integrals.

One may ask why the above problems deserve a closer study. I found them important, because they may help in solving some essential problems in probability theory and mathematical statistics. I met such problems when I tried to adapt the method of proof about the Gaussian limit behaviour of the maximum likelihood estimate to some similar but more difficult questions. In the original problem the asymptotic behaviour of the solution of the so-called maximum likelihood equation has to be investigated. The study of this problem is hard in its original form. But by applying an appropriate Taylor expansion of the function that appears in this equation and throwing out its higher order terms we get an approximation whose behaviour can be well understood. So to describe the limit behaviour of the maximum likelihood estimate it suffices to show that this approximation causes only a negligible error.

One would try to apply a similar method in the study of more difficult questions. I met some non-parametric maximum likelihood problems, for instance the description of the limit behaviour of the so-called Kaplan–Meyer product limit estimate when such an approach could be applied. But in these problems it was harder to show that the simplifying approximation causes only a negligible error. In this case the solution of the above mentioned problems was needed. In the non-parametric maximum likelihood estimate problems I met, the estimation of multiple (random) integrals played a role similar to the estimation of the coefficients in the Taylor expansion in the study of maximum likelihood estimates. Although I could apply this approach only in some special cases, I believe that it works in very general situations. But it demands some further work to show this.

The above formulated problems about random integrals are interesting and nontrivial even in the special case k = 1. Their solution leads to some interesting and non-trivial generalization of the fundamental theorem of the mathematical statistics about the difference of the empirical and real distribution of a large sample.

These problems have a natural counterpart about the behaviour of so-called Ustatistics, which is a fairly popular subject in probability theory. The investigation of multiple random integrals and U-statistics are closely related, and it turned out to be useful to consider them simultaneously.

Let us try to get some feeling about what kind of results can be expected in these problems. For a large sample size n the normalized empirical measure \sqrt{n}(μ_{n}− μ) behaves similarly to a Gaussian random measure. This suggests that in the problems we are interested in similar results should hold as in the analogous problems about multiple Gaussian integrals. The behaviour of multiple Gaussian integrals, called Wiener–Itô integrals in the literature, is fairly well understood, and it suggests that the tail distribution of a k-fold random integral with respect to a normalized empirical measure should satisfy such estimates as the tail distribution of the k-th power of a Gaussian random variable with expectation zero and appropriate variance. Beside this, if we consider the supremum of multiple random integrals of a class of functions with respect to a normalized empirical measure or with respect to a Gaussian random measure, then we expect that under not too restrictive conditions this supremum is not much larger than the ‘worst’ random integral with the largest variance taking part in this supremum. We may also hope that the methods of the theory of multiple Gaussian integrals can be adapted to the investigation of our problems.

The above presented heuristic considerations supply a fairly good description of the situation, but they do not take into account a very essential difference between the behaviour of multiple Gaussian integrals and multiple integrals with respect to a normalized empirical measure. If the variance of a multiple integral with respect to a normalized empirical measure is very small, what turns out to be equivalent to a very small L_{2}-norm of the function we are integrating, then the behaviour of this integral is different from that of a multiple Gaussian integral with the same kernel function. In this case the effect of some irregularities of the normalized empirical distribution turns out to be non-negligible, and no good Gaussian approximation holds any longer. This case must be better understood, and some new methods have to be worked out to handle it. The hardest problems discussed in this work are related to this phenomenon.

The precise formulation of the results will be given in the main part of the work. Beside their proofs I also tried to explain the main ideas behind them and the notions introduced in their investigation. This work contains some new results, and also the proof of some already rather classical theorems is presented. The results about Gaussian random variables and their non-linear functionals, in particular multiple integrals with respect to a Gaussian field, have a most important role in the study of the present work. Hence they are discussed in detail together with some of their counterparts about multiple random integrals with respect to a normalized empirical measure and some results about U-statistics.

The proofs apply results from different parts of the probability theory. Papers investigating similar results refer to works dealing with quite different subjects, and this makes their reading rather hard. To overcome this difficulty I tried to work out the details and to present a self-contained discussion even at the price of a longer text. Thus I wrote down (in the main text or in the Appendix) the proof of many interesting and basic results, like results about Vapnik–Červonenkis classes, about U-statistics and their decomposition to sums of so-called degenerate U-statistics, about so-called decoupled U-statistics and their relation to ordinary U-statistics, the diagram formula about the product of Wiener–Itô integrals, their counterpart about the product of degenerate U-statistics, etc. I tried to give such an exposition where different parts of the problem are explained independently of each other, and they can be understood in themselves.

As all the topics treated in the individual chapters relate to each other it seemed natural to me to tell the history of how the various results were reached in one last chapter. This last chapter, Chapter 18, just before the Appendix, also contains the complete reference list. I tried to give satisfactory referencing to all essential problems discussed, concentrate on explaining the main ideas behind the proofs and indicate where they were published. I did not attempt to provide an exhaustive literature list for fear that more would be less. As a consequence the reference list reflects my subjctive preferences, my way of thinking.

Motivation of the investigation. Discussion of some problems

In this chapter I try to show by means of an example why the solution of the problems mentioned in the introduction may be useful in the study of some important problems of probability theory. I try to give a good picture about the main ideas, but I do not work out all details. Actually the elaboration of some details omitted from this discussion would demand hard work. But as the present chapter is quite independent of the rest of the work, these omissions cause no problem in understanding the subsequent part.

I start with a short discussion of the maximum likelihood estimate in the simplest case. The following problem is considered. Let us have a class of density functions f(x,ϑ) on the real line depending on a parameter ϑ ∈ R^{1}, and observe a sequence of independent random variables ξ_{1}(ω),...,ξ_{n}(ω) with a density function f(x,ϑ_{0}), where ϑ_{0} is an unknown parameter we want to estimate with the help of the above sequence of random variables.

The maximum likelihood method suggests the following approach. Choose that value ϑ̂_{n} = ϑ̂_{n}(ξ_{1},...,ξ_{n}) as the estimate of the parameter ϑ_{0} where the density function of the random vector (ξ_{1},...,ξ_{n}), i.e. the product

∏_{k=1}^{n} f(ξ_{k},ϑ) = exp{ ∑_{k=1}^{n} logf(ξ_{k},ϑ)}

takes its maximum. This point can be found as the solution of the so-called maximum likelihood equation

∑_{k=1}^{n} \frac{∂}{∂ϑ} logf(ξ_{k},ϑ) = 0.  (2.1)

We are interested in the asymptotic behaviour of the random variable ϑ̂_{n} −ϑ_{0}, where ϑ̂_{n} is the (appropriate) solution of the equation (2.1).

The direct study of this equation is rather hard, but a Taylor expansion of the expression at the left-hand side of (2.1) around the (unknown) point ϑ_{0} yields a good and simple approximation of ϑ̂_{n}, and it enables us to describe the asymptotic behaviour of ϑ̂_{n} − ϑ_{0}.

This Taylor expansion yields that

∑_{k=1}^{n} \frac{∂}{∂ϑ} logf(ξ_{k},ϑ̂_{n}) = ∑_{k=1}^{n} \frac{\frac{∂}{∂ϑ} f(ξ_{k},ϑ_{0})}{f(ξ_{k},ϑ_{0})}

+(ϑ̂_{n} − ϑ_{0})( ∑_{k=1}^{n} ( \frac{\frac{∂^{2}}{∂ϑ^{2}} f(ξ_{k},ϑ_{0})}{f(ξ_{k},ϑ_{0})} − \frac{( \frac{∂}{∂ϑ} f(ξ_{k},ϑ_{0})) ^{2}}{f^{2}(ξ_{k},ϑ̄_{0})} ) ) + O(n(ϑ̂_{n} − ϑ_{0})^{2})

= ∑_{k=1}^{n} (η_{k}+ ζ_{k}(ϑ̂_{n} − ϑ_{0})) + O(n(ϑ̂_{n} − ϑ_{0})^{2}) ,  (2.2)

where

η_{k} = \frac{\frac{∂}{∂ϑ} f(ξ_{k},ϑ_{0})}{f(ξ_{k},ϑ_{0})} and ζ_{k} = \frac{\frac{∂^{2}}{∂ϑ^{2}} f(ξ_{k},ϑ_{0})}{f(ξ_{k},ϑ_{0})} − \frac{( \frac{∂}{∂ϑ} f(ξ_{k},ϑ_{0})) ^{2}}{f^{2}(ξ_{k},ϑ̄_{0})}

for k = 1,...,n. We want to understand the asymptotic behaviour of the (random) expression on the right-hand side of (2.2). The relation

Eη_{k} = ∫ \frac{\frac{∂}{∂ϑ} f(x,ϑ_{0})}{f(x,ϑ_{0})} f(x,ϑ_{0})dx = \frac{∂}{∂ϑ} ∫ f(x,ϑ_{0})dx = 0

holds, since ∫ f(x,ϑ)dx = 1 for all ϑ, and a differentiation of this relation gives the

last identity. Similarly, Eη_{k}^{2} = −Eζ_{k}= ∫ \frac{(\frac{∂}{∂ϑ} f(x,ϑ_{0}))^{2}}{f(x,ϑ_{0})} dx > 0, k = 1,...,n. Hence by

the central limit theorem χ_{n} = \frac{1}{\sqrt{n}} ∑_{k=1}^{n} η_{k} is asymptotically normal with expectation

zero and variance I^{2} = ∫ \frac{(\frac{∂}{∂ϑ} f(x,ϑ_{0}))^{2}}{f(x,ϑ_{0})} dx > 0. In the statistics literature this number

I is called the Fisher information. By the laws of large numbers \frac{1}{n} ∑_{k=1}^{n} ζ_{k} ∼ −I^{2}.

Thus relation (2.2) suggests the approximation of the maximum-likelihood es-

timate ϑ̂_{n} by the random variable ϑ̃_{n} given by the identity ϑ̃_{n} − ϑ_{0} = −\frac{∑_{k=1}^{n} η_{k}}{∑_{k=1}^{n} ζ_{k}},

and the previous calculations imply that \sqrt{n}(ϑ̃_{n} − ϑ_{0}) is asymptotically normal with expectation zero and variance \frac{1}{I^{2}}. The random variable ϑ̃_{n} is not a solution of the equation (2.1), the value of the expression at the left-hand side is of order O(n(ϑ̃_{n} − ϑ_{0})^{2}) = O(1) in this point. On the other hand, some calculations show that the derivative of the function at the left-hand side is large in this point, it is greater than const.n with some const. > 0. This implies that the maximumlikelihood equation has a solution ϑ̂_{n} such that ϑ̂_{n} −ϑ̃_{n} = O( \frac{1}{n}). Hence \sqrt{n}(ϑ̂_{n} −ϑ_{0}) and \sqrt{n}(ϑ̃_{n} − ϑ_{0}) have the same asymptotic limit behaviour.

The previous method can be summarized in the following way: Take a simpler linearized version of the expression we want to estimate by means of an appropriate Taylor expansion, describe the limit distribution of this linearized version and show that the linearization causes only a negligible error.

We want to show that such a method also works in more difficult situations. But in some cases it is harder to show that the error committed by a replacement of the original expression by a simpler linearized version is negligible, and to show this the solution of the problems mentioned in the introduction is needed. The discussion of the following problem, called the Kaplan–Meyer method for the estimation of the empirical distribution function with the help of censored data shows such an example.

The following problem is considered. Let (X_{i},Z_{i}), i = 1,...,n, be a sequence of independent, identically distributed random vectors such that the components X_{i} and Z_{i} are also independent with some unknown, continuous distribution functions F (x) and G(x). We want to estimate the distribution function F of the random variables X_{i}, but we cannot observe the variables X_{i}, only the random variables Y_{i} = min(X_{i},Z_{i}) and δ_{i} = I(X_{i} ≤ Z_{i}). In other words, we want to solve the following problem. There are certain objects whose lifetime X_{i} are independent and F distributed. But we cannot observe this lifetime X_{i}, because after a time Z_{i} the observation must be stopped. We also know whether the real lifetime X_{i} or the censoring variable Z_{i} was observed. We make n independent experiments and want to estimate with their help the distribution function F.

Kaplan and Meyer, on the basis of some maximum-likelihood estimation type considerations, proposed the following so-called product limit estimator S_{n}(u) to estimate the unknown survival function S(u) = 1 − F (u):

1 − F_{n}(u) = S_{n}(u) = \{ \begin{matrix}∏_{i=1}^{n} ( \frac{N(Y_{i})}{N(Y_{i})+1})^{I(Y_{i}≤u,δ_{i}=1)} if u ≤ max(Y_{1},...,Y_{n}) \\ 0 if u ≥ max(Y_{1},...,Y_{n}), and δ_{n} = 1, \\ undefined ifu ≥ max(Y_{1},...,Y_{n}), and δ_{n} = 0, \end{matrix} (2.3)

where

N(t) = #{Y_{i}, Y_{i} > t, 1 ≤ i ≤ n} = ∑_{i=1}^{n} I(Y_{i} > t).

We want to show that the above estimate (2.3) is really good. For this goal we shall approximate the random variables S_{n}(u) by some appropriate random variables. To do this first we introduce some notations.

Put

H(u)= P(Y_{i} ≤ u) = 1 − H̄(u),
H̃(u) = P(Y_{i} ≤ u, δ_{i} = 1), ˜ H̃(u) = P(Y_{i} ≤ u, δ_{i} = 0)  (2.4)

and

H_{n}(u)= \frac{1}{n} ∑_{i=1}^{n} I(Y_{i} ≤ u)  (2.5)

H̃_{n}(u) = \frac{1}{n} ∑_{i=1}^{n} I(Y_{i} ≤ u, δ_{i} = 1), ˜ H̃_{n}(u) = \frac{1}{n} ∑_{i=1}^{n} I(Y_{i} ≤ u, δ_{i} = 0).

Clearly H(u) = H̃(u)+ ˜H̃(u) and H_{n}(u) = H̃_{n}(u)+ ˜H̃_{n}(u). We shall estimate F_{n}(u)− F (u) for u ∈ (−∞,T] if

1 − H(T) > δ with some fixedδ > 0.  (2.6)

Condition (2.6) implies that there are more than \frac{δ}{2}n sample points Y_{j} larger than T with probability almost 1. The complementary event has only an exponentially small probability. This observation helps to show in the subsequent calculations that some events have negligibly small probability.

We introduce the so-called cumulative hazard function and its empirical version

Λ(u) = −log(1 − F (u)), Λ_{n}(u) = −log(1 − F_{n}(u)).  (2.7)

Since F_{n}(u)−F (u) = exp(−Λ(u))(1 − exp(Λ(u) − Λ_{n}(u))) a simple Taylor expansion yields

F_{n}(u) − F (u) = (1 − F (u))(Λ_{n}(u) − Λ(u)) + R_{1}(u),  (2.8)

and it is easy to see that R_{1}(u) = O((Λ(u) − Λ_{n}(u))^{2}) . It follows from the subsequent estimations that Λ(u) − Λ_{n}(u) = O(n^{−1/2} ), thus nR_{1}(u) = O(1). Hence it is enough to investigate the term Λ_{n}(u). We shall show that Λ_{n}(u) has an expansion with Λ(u) as the main term plus n^{−1/2} times a term which is a linear functional of an appropriate normalized empirical distribution function plus an error term of order O(n^{−1} ).

From (2.3) it is obvious that

Λ_{n}(u) = −∑_{i=1}^{n} I(Y_{i} ≤ u, δ_{i} = 1)log( 1 − \frac{1}{1 + N(Y_{i})}) .

It is not difficult to get rid of the unpleasant logarithmic function in this formula by means of the relation −log(1 − x) = x + O(x^{2}) for small x. It yields that

Λ_{n}(u) = ∑_{i=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)}{N(Y_{i})} + R_{2}(u) = Λ̃_{n}(u) + R_{2}(u)  (2.9)

with an error term R_{2}(u) such that nR_{2}(u) is smaller than a constant with probability almost one. (The probability of the exceptional set is exponentially small.)

The expression Λ̃_{n}(u) is still inappropriate for our purposes. Since the denom-

inators N(Y_{i}) = ∑_{j=1}^{n} I(Y_{j} > Y_{i}) are dependent for different indices i we cannot see

directly the limit behaviour of Λ̃_{n}(u).

We try to approximate Λ̃_{n}(u) by a simpler expression. A natural approach would be to approximate the terms N(Y_{i}) in it by their conditional expectation (n−1)H̄(Y_{i}) = (n−1)(1−H(Y_{i})) = E(N(Y_{i})|Y_{i}) with respect to the σ-algebra generated by the random variable Y_{i}. This is a too rough ‘first order’ approximation, but the following ‘second order approximation’ will be sufficient for our goals. Put

N(Y_{i}) = ∑_{j=1}^{n} I(Y_{j} > Y_{i}) = nH̄(Y_{i})( 1 + \frac{∑_{j=1}^{n} I(Y_{j} > Y_{i}) − nH̄(Y_{i})}{nH̄(Y_{i})})

and express the terms \frac{1}{N(Y_{i})}in the sum defining Λ̃_{n}, (with Λ̃_{n} introduced in (2.9))

by means of the relation \frac{1}{1+z} = ∑_{k=0}^{∞}(−1)^{k}z^{k} = 1 − z + ε(z) with the choice z =

\frac{∑_{j=1}^{n} I(Y_{j}>Y_{i})−nH̄(Y_{i})}{nH̄(Y_{i})}. As |ε(z)| < 2z^{2} for |z| < \frac{1}{2} we get that

Λ̃_{n}(u) = ∑_{i=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)}{nH̄(Y_{i})} ( 1 + ∑_{k=1}^{∞} ( −\frac{∑_{j=1}^{n} I(Y_{j} > Y_{i}) − nH̄(Y_{i})}{nH̄(Y_{i})}) ^{k})

= ∑_{i=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)}{nH̄(Y_{i})} ( 1 − \frac{∑_{j=1}^{n} I(Y_{j} > Y_{i}) − nH̄(Y_{i})}{nH̄(Y_{i})}) + R_{3}(u)

= 2A(u) − B(u) + R_{3}(u),  (2.10)

where

A(u) = A(n,u) = ∑_{i=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)}{nH̄(Y_{i})}

and

B(u) = B(n,u) = ∑_{i=1}^{n} ∑_{j=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)I(Y_{j} > Y_{i})}{n^{2}H̄^{2}(Y_{i})}.

It can be proved by means of standard methods that nR_{3}(u) is exponentially small. Thus relations (2.9) and (2.10) yield that

Λ_{n}(u) = 2A(u) − B(u) + negligible error.  (2.11)

This means that to solve our problem the asymptotic behaviour of the random variables A(u) and B(u) has to be given. We can get a better insight to this problem by rewriting the sum A(u) as an integral and the double sum B(u) as a two-fold integral with respect to empirical measures. Then these integrals can be rewritten as sums of random integrals with respect to normalized empirical measures and deterministic measures. Such an approach yields a representation of Λ_{n}(u) in the form of a sum whose terms can be well understood.

Let us write

A(u)= ∫_{−∞}^{+∞} \frac{I(y ≤ u)}{1 − H(y)} dH̃_{n}(y),
B(u)= ∫_{−∞}^{+∞} ∫_{−∞}^{+∞} \frac{I(y ≤ u)I(x > y)}{(1 − H(y))^{2}} dH_{n}(x)dH̃_{n}(y).

We rewrite the terms A(u) and B(u) in a form better for our purposes. We express these terms as a sum of integrals with respect to dH(u), dH̃(u) and the normalized empirical processes d\sqrt{n}(H_{n}(x) − H(x)) and d\sqrt{n}(H̃_{n}(y) − H̃(y)). For this goal observe that

H_{n}(x)H̃_{n}(y) = H(x)H̃(y) + H(x)(H̃_{n}(y) − H̃(y)) + (H_{n}(x) − H(x))H̃(y)
+(H_{n}(x) − H(x))(H̃_{n}(y) − H̃(y)).

Hence we can write that B(u) = B_{1}(u) + B_{2}(u) + B_{3}(u) + B_{4}(u), where

B_{1}(u) = ∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} dH(x)dH̃(y) ,
B_{2}(u) = \frac{1}{\sqrt{n}}∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} dH(x)d (\sqrt{n}(H̃_{n}(y) − H̃(y))) ,
B_{3}(u) = \frac{1}{\sqrt{n}}∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} d (\sqrt{n}(H_{n}(x) − H(x))) dH̃(y) ,
B_{4}(u) = \frac{1}{n}∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} d (\sqrt{n}(H_{n}(x) − H(x)))
d (\sqrt{n}(H̃_{n}(y) − H̃(y))) .

In the above decomposition of B(u) the term B_{1} is a deterministic function, B_{2}, B_{3} are linear functionals of normalized empirical processes and B_{4} is a nonlinear functional of normalized empirical processes. The deterministic term B_{1}(u) can be calculated explicitly. Indeed,

B_{1}(u) = ∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} dH(x)dH̃(y) = ∫_{−∞}^{u} \frac{dH̃(y)}{1 − H(y)} .

Then the relations H̃(u) = ∫_{−∞}^{u} (1 − G(t)) dF (t) and 1 − H = (1 − F)(1 − G) imply that

B_{1}(u) = ∫_{−∞}^{u} \frac{dF (y)}{1 − F (y)} = −log(1 − F (u)) = Λ(u).  (2.12)

Observe that

A(u)= ∫_{−∞}^{u} \frac{d H̃_{n}(y)}{1 − H(y)}
= ∫_{−∞}^{u} \frac{dH̃(y)}{1 − H(y)} + \frac{1}{\sqrt{n}}∫_{−∞}^{u} \frac{d (\sqrt{n}(H̃_{n}(y) − H̃(y)))}{1 − H(y)}
= B_{1}(u) + B_{2}(u).  (2.13)

From relations (2.11), (2.12) and (2.13) it follows that

Λ_{n}(u) − Λ(u) = B_{2}(u) − B_{3}(u) − B_{4}(u) + negligible error.  (2.14)

Integration of B_{2} and B_{3} with respect to the variable x and then integration by parts in the expression B_{2} yields that

B_{2}(u)= \frac{1}{\sqrt{n}}∫_{−∞}^{u} \frac{d (\sqrt{n}(H̃_{n}(y) − H̃(y)))}{1 − H(y)}
= \frac{\sqrt{n}(H̃_{n}(u) − H̃(u))}{\sqrt{n}(1 − H(u))} − \frac{1}{\sqrt{n}}∫_{−∞}^{u} \frac{\sqrt{n}(H̃_{n}(y) − H̃(y))}{(1 − H(y))^{2}} dH(y),
B_{3}(u)= \frac{1}{\sqrt{n}}∫_{−∞}^{u} \begin{matrix}\sqrt{n} & (H(y) − H_{n}(y)) \\ (1 − H(y)) & 2 \end{matrix} dH̃(y).

With the help of the above expressions for B_{2} and B_{3} (2.14) can be rewritten as

\sqrt{n}(Λ_{n}(u) − Λ(u)) = ^{\frac{\sqrt{n}(H̃_{n}(u)−H̃(u))}{1−H(u)}} − ∫_{−∞}^{u} \frac{\sqrt{n}(H̃_{n}(y)−H̃(y))}{(1−H(y))^{2}} dH(y)

+∫_{−∞}^{u} \begin{matrix}\sqrt{n} & (H_{n}(y)−H(y)) \\ (1−H(y)) & ^{2} \end{matrix} dH̃(y)

−\sqrt{n}B_{4}(u) + negligible error.  (2.15)

Formula (2.15) (together with formula (2.8)) almost agrees with the statement we wanted to prove. Here the random variable \sqrt{n}(Λ_{n}(u) − Λ(u)) is expressed as a sum of linear functionals of normalized empirical distributions plus some negligible error terms plus the error term \sqrt{n}B_{4}(u). So to get a complete proof it is enough to show that \sqrt{n}B_{4}(u) also yields a negligible error. But nB_{4}(u) is a double integral of a bounded function (here we apply again formula (2.6)) with respect to a normalized empirical distribution. Hence to bound this term we need a good estimate of multiple stochastic integrals (with multiplicity 2), and this is just the problem formulated in the introduction. The estimate we need here follows from Theorem 8.1 of the present work. Let us remark that the problem discussed here corresponds to the estimation of the coefficient of the second term in the Taylor expansion considered in the study of the maximum likelihood estimation. One may worry a little bit how to bound nB_{4}(u) with the help of estimations of double stochastic integrals, since in the definition of B_{4}(u) integration is taken with respect to different normalized empirical processes in the two coordinates. But this is a not too difficult technical problem. It can be simply overcome for instance by rewriting the integral as a double integral with respect to the empirical process (\sqrt{n}(H_{n}(x) − H(x)),\sqrt{n}(H̃_{n}(y) − H̃(y))) in the space R^{2}.

By working out the details of the above calculation we get that the linear functional B_{2}(u)−B_{3}(u) of normalized empirical processes yields a good estimate on the expression \sqrt{n}(Λ_{n}(u) − Λ(u)) for a fixed parameter u. But we want to prove somewhat more, we want to get an estimate uniform in the parameter u, i.e. to show that even the random variable sup_{u≤T}|\sqrt{n}(Λ_{n}(u) − Λ(u)) − B_{2}(u) + B_{3}(u)| is small. This

can be done by making estimates uniform in the parameter u in all steps of the above calculation. There appears only one difficulty when trying to carry out this program. Namely, we need an estimate on sup_{u≤T}|nB_{4}(u)|, i.e. we have to bound the

supremum of multiple random integrals with respect to a normalized random measure for a nice class of kernel functions. This can be done, but at this point the second problem mentioned in the introduction appears. This difficulty can be overcome by means of Theorem 8.2 of this work.

Thus the limit behaviour of the Kaplan–Meyer estimate can be described by means of an appropriate expansion. The steps of the calculation leading to such an expansion are fairly standard, the only hard part is the solution of the problems mentioned in the introduction. It can be expected that such a method also works in a much more general situation.

I finish this chapter with a remark of Richard Gill he made in a personal conversation after my talk on this subject at a conference. While he accepted my proof he missed an argument in it about the maximum likelihood character of the Kaplan– Meyer estimate. This was a completely justified remark, since if we do not restrict our attention to this problem, but try to generalize it to general non-parametric maximum likelihood estimates, then we have to understand how the maximum likelihood character of the estimate can be exploited. I believe that this can be done, but only with the help of some further studies.

Some estimates about sums of independent random variables

We shall need a good bound on the tail distribution of sums of independent random variables bounded by a constant with probability one. Later only the results about sums of independent and identically distributed variables will be interesting for us. But since they can be generalized without any effort to sums of not necessarily identically distributed random variables the condition about identical distribution of the summands will be dropped. We are interested in the question when these estimates give such a good bound as the central limit theorem suggests, and what can be told otherwise.

More explicitly, the following problem will be considered: Let X_{1},...,X_{n} be independent random variables, EX_{j} = 0, VarX_{j} = σ_{j}^{2}, 1 ≤ j ≤ n, and take the random

sum S_{n} = ∑_{j=1}^{n} X_{j} and its variance VarS_{n} = V_{n}^{2} = ∑_{j=1}^{n} σ_{j}^{2}. We want to get a good bound

on the probability P(S_{n} > uV_{n}). The central limit theorem suggests that under general conditions an upper bound of the order 1−Φ(u) should hold for this probability, where Φ(u) denotes the standard normal distribution function. Since the standard

normal distribution function satisfies the inequality ( \frac{1}{u} − \frac{1}{u^{3}}) \frac{e^{−u^{2}/2}}{\sqrt{2π}} < 1 − Φ(u) <

\frac{1}{u} \frac{e^{−u^{2}/2}}{\sqrt{2π}} for all u > 0 it is natural to ask when the probability P(S_{n} > uV_{n}) is com-

parable with the value e^{−u^{2}/2} . More generally, we shall call an upper bound of the form P(S_{n} > uV_{n}) ≤ e^{−Cu^{2}} with some constant C > 0 a Gaussian type estimate.

First I formulate Bernstein’s inequality which tells for which values u the probability P(S_{n} > uV_{n}) has a Gaussian type estimate. It supplies such an estimate if u ≤ constV_{n}. On the other hand, for u ≥ const.V_{n} it yields a much weaker bound. I shall formulate another result, called Bennett’s inequality, which is a slight improvement of Bernstein’s inequality. It helps us to tell what can be expected if Bernstein’s inequality does not provide a Gaussian type estimate. I shall also present an example which shows that Bennett’s inequality is in some sense sharp. The main difficulties we meet in this work are closely related to the weakness of the estimates we have for the probability P(S_{n} > uV_{n}) if it does not satisfy a Gaussian type estimate. As we shall see this happens if u ≫ const.V_{n}.

In the usual formulation of Bernstein’s inequality a real number M is introduced, and it is assumed that the terms in the sum we investigate are bounded by this number. But since the problem can be simply reduced to the case M = 1 I shall consider only this special case.

Theorem 3.1 (Bernstein’s inequality). Let X_{1},...,X_{n} be independent random vari-

ables, P(|X_{j}| ≤ 1) = 1, EX_{j} = 0, 1 ≤ j ≤ n. Put σ_{j}^{2} = EX_{j}^{2}, 1 ≤ j ≤ n, S_{n} = ∑_{j=1}^{n} X_{j}

and V_{n}^{2} = VarS_{n} = ∑_{j=1}^{n} σ_{j}^{2}. Then

P (S_{n} > uV_{n})≤ exp\{ −\frac{u^{2}}{2( 1 + \frac{1}{3} \frac{u}{V_{n}})} \} for all u > 0.  (3.1)

Proof of Theorem 3.1. Let us give a good bound on the exponential moments Ee^{tS_{n}} for appropriate parameters t > 0. Since EX_{j} = 0 and E|X_{j}^{k+2}| ≤ σ_{j}^{2} for k ≥ 0 we can

write Ee^{tX_{j}}= ∑_{k=0}^{∞} \frac{t^{k}}{k!}EX_{j}^{k} ≤ 1 + \frac{t^{2}σ_{j}^{2}}{2} ( 1 + ∑_{k=1}^{∞} \frac{2t^{k}}{(k+2)!} ) ≤ 1 + \frac{t^{2}σ_{j}^{2}}{2} ( 1 + ∑_{k=1}^{∞} 3^{−k} t^{k}) =

1 + \frac{t^{2}σ_{j}^{2}}{2} \frac{1}{1− \frac{t}{3}} ≤ exp\{ \frac{t^{2}σ_{j}^{2}}{2} \frac{1}{1− \frac{t}{3}} \} if 0 ≤ t < 3. Hence

Ee^{tS_{n}}= ∏_{j=1}^{n} Ee^{tX_{j}}≤ exp\{ \frac{t^{2}V_{n}^{2}}{2}\frac{1}{1 − \frac{t}{3}} \} for 0 ≤ t < 3.

The above relation implies that

P (S_{n} > uV_{n})= P(e^{tS_{n}}> e^{tuV_{n}}) ≤ Ee^{tS_{n}}e^{−tuV_{n}}≤ exp\{ \frac{t^{2}V_{n}^{2}}{2}\frac{1}{1 − \frac{t}{3}} − tuV_{n}\}

if 0 ≤ t < 3. Choose the number t in this inequality as the solution of the equation t^{2}V_{n}^{2} \frac{1}{1− \frac{t}{3}} = tuV_{n}, i.e. put t = \frac{u}{V_{n}+ \frac{u}{3}} . Then 0 ≤ t < 3, and we get that P(S_{n} > uV_{n}) ≤

e^{−tuV_{n}/2}= exp\{ −\frac{u^{2}}{2(1+ \frac{1}{3} \frac{u}{Vn})} \} .

If the random variables X_{1},...,X_{n} satisfy the conditions of Bernstein’s inequality, then also the random variables −X_{1},...,−X_{n} satisfy them. By applying the above

result in both cases we get that P(|S_{n}| > uV_{n}) ≤ 2exp\{ −\frac{u^{2}}{2(1+ \frac{1}{3} \frac{u}{Vn})} \} under the con-

ditions of Bernstein’s inequality.

By Bernstein’s inequality for all ε > 0 there is some number α(ε) > 0 such that in the case \frac{u}{V_{n}} < α(ε) the inequality P(S_{n} > uV_{n}) ≤ e^{−(1−ε)u^{2}/2} holds. Beside this, for all fixed numbers A > 0 there is some constant C = C(A) > 0 such that if \frac{u}{V_{n}}< A,

then P(S_{n} > uV_{n}) ≤ e^{−Cu^{2}}. This can be interpreted as a Gaussian type estimate for the probability P(S_{n} > uV_{n}) if u ≤ const.V_{n}.

On the other hand, if \frac{u}{V_{n}} is very large, then Bernstein’s inequality yields a much worse estimate. The question arises whether in this case Bernstein’s inequality can be replaced by a better, more useful result. Next I present Theorem 3.2, the so-called Bennett’s inequality which provides a slight improvement of Bernstein’s inequality. But if \frac{u}{V_{n}} is very large, then also Bennett’s inequality provides a much weaker estimate on the probability P(S_{n} > uV_{n}) than the bound suggested by a Gaussian comparison. On the other hand, I shall present an example that shows that (without imposing some additional conditions) no real improvement of this estimate is possible.

Theorem 3.2 (Bennett’s inequality). Let X_{1},...,X_{n} be independent random vari-

ables, P(|X_{j}| ≤ 1) = 1, EX_{j} = 0, 1 ≤ j ≤ n. Put σ_{j}^{2} = EX_{j}^{2}, 1 ≤ j ≤ n, S_{n} = ∑_{j=1}^{n} X_{j}

and V_{n}^{2} = VarS_{n} = ∑_{j=1}^{n} σ_{j}^{2}. Then

P(S_{n} > u) ≤ exp\{ −V_{n}^{2}[( 1 + \frac{u}{V_{n}^{2}}) log( 1 + \frac{u}{V_{n}^{2}}) − \frac{u}{V_{n}^{2}}] \} for all u > 0. (3.2)

As a consequence, for all ε > 0 there exists some B = B(ε) > 0 such that

P (S_{n} > u) ≤ exp\{ −(1 − ε)ulog \frac{u}{V_{n}^{2}}\} ifu > BV_{n}^{2},  (3.3)

and there exists some positive constant K > 0 such that

P (S_{n} > u) ≤ exp\{ −Kulog \frac{u}{V_{n}^{2}}\} ifu > 2V_{n}^{2}.  (3.4)

Proof of Theorem 3.2. We have

Ee^{tX_{j}}= ∑_{k=0}^{∞} \frac{t^{k}}{k!}EX_{j}^{k} ≤ 1 + σ_{j}^{2} ∑_{k=2}^{∞} \frac{t^{k}}{k!} = 1 + σ_{j}^{2}(e^{t}− 1 − t) ≤ e^{σ_{j}^{2}(e^{t}−1−t)} ,
1 ≤ j ≤ n,

and Ee^{tS_{n}}≤ e^{V_{n}^{2}(e^{t}−1−t)} for all t ≥ 0. Hence P(S_{n}> u) ≤ e^{−tu}Ee^{tS_{n}}≤ e^{−tu+V_{n}^{2}(e^{t}−1−t)} for all t ≥ 0. We get relation (3.2) from this inequality with the choice t = log( 1 + \frac{u}{V_{n}^{2}}) . (This is the place of minimum of the function −tu + V_{n}^{2}(e^{t}− 1 − t) for fixed u in the parameter t.)

Relation (3.2) and the observation lim_{v→∞} \frac{(v+1)log(v+1)−v}{vlogv} = 1 with the choice v = \frac{u}{V_{n}^{2}} imply formula (3.3). Because of relation (3.3) to prove formula (3.4) it is enough to check it for 2 ≤ \frac{u}{V_{n}^{2}} ≤ B with some sufficiently large constant B > 0. In this case relation (3.4) follows directly from formula (3.2). This can be seen for instance by observing that the expression ^{\frac{V_{n}^{2}[( 1+_{\frac{u}{V^{2}_{n}}}) log( 1+_{\frac{u}{V^{2}_{n}}}) −_{\frac{u}{V^{2}_{n}}}]}{ulog _{\frac{u}{V^{2}_{n}}}}} is a continuous and positive

function of the variable \frac{u}{V_{n}^{2}} in the interval 2 ≤ \frac{u}{V_{n}^{2}} ≤ B, hence its minimum in this interval is strictly positive.

Let us make a short comparison between Bernstein’s and Bennett’s inequalities. Both results yield an estimate on the probability P(S_{n} > u), and their proofs are very similar. They are based on an estimate of the moment generating functions R_{j}(t) = Ee^{tX_{j}}of the summands X_{j}, but Bennett’s inequality yields a better estimate. It may be worth mentioning that the estimate given for R_{j}(t) = Ee^{tX_{j}} in the proof of Bennett’s inequality agrees with the moment generating function Ee^{t(Y_{j}−EY_{j})}of the normalization Y_{j} −EY_{j} of a Poissonian random variable Y_{j} with parameter VarX_{j}. As a consequence, we get, by using the standard method of estimating tail-distributions by means of the moment generating functions such an estimate for the probability P(S_{n} > u) which is comparable with the probability P(T_{n}− ET_{n} > u), where T_{n} is a Poissonian random variable with parameter V_{n}= VarS_{n}. We can say that Bernstein’s inequality yields a Gaussian and Bennett’s inequality a Poissonian type estimate for the sums of independent, bounded random variables.

Remark. Bennett’s inequality yields a sharper estimate for the probability P(S_{n} > u) than Bernstein’s inequality for all numbers u > 0. To prove this it is enough to show that for all 0 ≤ t < 3 the inequality Ee^{tS_{n}}≤ e^{V_{n}^{2}(e^{t}−1−t)} appearing in the proof of Bennett’s inequality is a sharper estimate than the corresponding inequality Ee^{tS_{n}}≤ exp\{ \frac{t^{2}V_{n}^{2}}{2}\frac{1}{1− \frac{t}{3}} \} appearing in the proof of Bernstein’s inequality. (Recall, how we estimate the probability P(S_{n} > u) in these proofs with the help of the exponential moment Ee^{tS_{n}}.) But to prove this it is enough to check that e^{t}− 1 − t ≤ \frac{t^{2}}{2} \frac{1}{1− \frac{t}{3}} for

all 0 ≤ t < 3. This inequality clearly holds, since e^{t}− 1 − t = ∑_{k=2}^{∞} \frac{t^{k}}{k!}, and \frac{t^{2}}{2} \frac{1}{1− \frac{t}{3}} =

∑_{k=2}^{∞} \frac{1}{2}(\frac{1}{3})^{k−2}t^{k}.

Next I present Example 3.3 which shows that Bennett’s inequality yields a sharp estimate also in the case u ≫ V_{n}^{2} when Bernstein’s inequality yields a weak bound. But Bennett’s inequality provides only a small improvement which has only a limited importance. This may be the reason why Bernstein’s inequality which yields a more transparent estimate is more popular.

Example 3.3 (Sums of independent random variables with bad tail distribution for large values). Let us fix some positive integer n, real numbers u and σ^{2} such that 0 < σ^{2} ≤ \frac{1}{8}, n > 4u ≥ 6 and u > 4nσ^{2}. Let σ̄^{2} be that solution of the equation x^{2}− x + σ^{2} = 0 which is smaller than \frac{1}{2}. Take a sequence of independent and identically distributed random variables X̄_{1},...,X̄_{n} such that P(X̄_{j} = 1) = σ̄^{2}, P(X̄_{j} = 0) =

1 − σ̄^{2} for all 1 ≤ j ≤ n. Put X_{j} = X̄_{j}− EX̄_{j} = X_{j} − σ̄^{2}, 1 ≤ j ≤ n, S_{n} = ∑_{j=1}^{n} X_{j}

and V_{n}^{2} = nσ^{2}. Then P(|X_{1}| ≤ 1) = 1, EX_{1} = 0, VarX_{1} = σ^{2}, hence ES_{n} = 0, and VarS_{n} = V_{n}^{2}. Beside this

P(S_{n} ≥ u) > exp\{ −Bulog \frac{u}{V_{n}^{2}}\}

with some appropriate constant B > 0 not depending on n, σ and u.

Proof of Example 3.3. Simple calculation shows that EX_{j} = 0, VarX_{j} = σ̄^{2} − σ̄^{4} = σ^{2}, P(|X_{j}| ≤ 1) = 0, and also the inequality σ^{2} ≤ σ̄^{2} ≤ \frac{3}{2}σ^{2} holds. To see the upper bound in the last inequality observe that σ̄^{2} ≤ \frac{1}{3}, i.e. 1 − σ̄^{2} ≥ \frac{2}{3}, hence σ^{2} = σ̄^{2}(1−σ̄^{2}) ≥ \frac{2}{3} σ̄^{2}. In the proof of the inequality of Example 3.3 we can restrict our attention to the case when u is an integer, because in the general case we can apply the inequality with ū= [u] + 1 instead of u, where [u] denotes the integer part of u, and since u ≤ ū≤ 2u, the application of the result in this case supplies the desired inequality with a possibly worse constant B > 0.

Put S̄_{n} = ∑_{j=1}^{n} X̄_{j}. We can write P(S_{n} ≥ u) = P(S̄_{n} ≥ u + nσ̄^{2}) ≥ P(S̄_{n} ≥ 2u) ≥

P(S̄_{n} = 2u) = \binom{n}{2u}σ̄^{4u}(1 − σ̄^{2})^{(n−2u)} ≥ (\frac{nσ̄^{2}}{2u})^{2u}(1 − σ̄^{2})^{(n−2u)}, since u ≥ nσ̄^{2}, and n ≥ 2u. On the other hand (1 − σ̄^{2})^{(n−2u)} ≥ e^{−2σ̄^{2}(n−2u)} ≥ e^{−2nσ̄^{2}} ≥ e^{−u}, hence

P(S_{n} ≥ u) ≥ exp\{−2ulog( \frac{u}{nσ̄^{2}} ) − 2ulog2 − u\}
= exp\{ −2ulog( \frac{u}{nσ^{2}} ) − 2ulog \frac{σ̄^{2}}{σ^{2}} − 2ulog2 − u\}
≥ exp\{ −100ulog( \frac{u}{V_{n}^{2}}) \} .

Example 3.3 is proved.

In the case u > 4V_{n}^{2} Bernstein’s inequality yields the estimate P(S_{n} > u) ≤ e^{−αu} with some universal constant α > 0, and the above example shows that at most an additional logarithmic factor K log \frac{u}{V_{n}^{2}} can be expected in the exponent of the upper bound in an improvement of this estimate. Bennett’s inequality shows that such an improvement is really possible.

I finish this chapter with another estimate due to Hoeffding which will be later useful in some symmetrization arguments.

Theorem 3.4 (Hoeffding’s inequality). Let ε_{1},...,ε_{n} be independent random variables, P(ε_{j} = 1) = P(ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, and let a_{1},...,a_{n} be arbitrary real

numbers. Put V = ∑_{j=1}^{n} a_{j}ε_{j}. Then

P(V > u) ≤ exp\{ −\frac{u^{2}}{2∑ _{j=1}^{n} a_{j}^{2}}\} for all u > 0.  (3.5)

Remark 1: Clearly EV = 0 and VarV = ∑_{j=1}^{n} a_{j}^{2}, hence Hoeffding’s inequality yields

such an estimate for P(V > u) which the central limit theorem suggests. This estimate holds for all real numbers a_{1},...,a_{n} and u > 0.

Remark 2: The Rademacher functions r_{k}(x), k = 1,2,..., defined by the formulas r_{k}(x) = 1 if (2j − 1)2^{−k} ≤ x < 2j2^{−k} and r_{k}(x) = −1 if 2(j − 1)2^{−k} ≤ x < (2j − 1)2^{−k}, 1 ≤ j ≤ 2^{k−1}, for all k = 1,2,..., can be considered as random variables on the probability space Ω = [0,1] with the Borel σ-algebra and the Lebesgue measure as probability measure on the interval [0,1]. They are independent random variables with the same distribution as the random variables ε_{1},...,ε_{n} considered in Theorem 3.4. Therefore results about such sequences of random variables whose distributions agree with those in Theorem 3.4 are also called sometimes results about Rademacher functions in the literature. At some points we will also apply this terminology.

Proof of Theorem 3.4. Let us give a good bound on the exponential moment

Ee^{tV} for all t > 0. The identity Ee^{tV} = ∏_{j=1}^{n} Ee^{ta_{j}ε_{j}} = ∏_{j=1}^{n} ^{\frac{(e^{a_{j} t}+e^{−a_{j} t})}{2}} holds, and

\frac{(e^{a_{j} t}+e^{−a_{j} t})}{2} = ∑_{k=0}^{∞} \frac{a_{j}^{2k}}{(2k)!}t^{2k}≤ ∑_{k=0}^{∞} \frac{(a_{j}t)^{2k}}{2^{k}k!} = e^{a_{j}^{2}t^{2}/2}, since (2k)! ≥ 2^{k}k! for all k ≥ 0. This

implies that Ee^{tV}≤ exp\{ \frac{t^{2}}{2} ∑_{j=1}^{n} a_{j}^{2}\} . Hence P(V > u) ≤ exp\{ −tu + \frac{t^{2}}{2} ∑_{j=1}^{n} a_{j}^{2}\} , and

we get relation (3.5) with the choice t = u( ∑_{j=1}^{n} a_{j}^{2}) ^{−1} .

On the supremum of a nice class of partial sums

This chapter contains an estimate about the supremum of a nice class of normalized sums of independent and identically distributed random variables together with an analogous result about the supremum of an appropriate class of one-fold random integrals with respect to a normalized empirical distribution. The second result deals with a one-variate version of the problem about the estimation of multiple integrals with respect to a normalized empirical distribution. This problem was mentioned in the introduction. Some natural questions related to these results will be also discussed. It will be examined how restrictive their conditions are. In particular, we are interested in the question how the condition about the countable cardinality of the class of random variables can be weakened. A natural Gaussian counterpart of the supremum problems about random one-fold integrals will be also considered. Most proofs will be postponed to later chapters.

To formulate these results first a notion will be introduced that plays a most important role in the sequel.

Definition of L_{p}-dense classes of functions. Let a measurable space (Y,Y ) be given together with a class G of Y measurable real valued functions on this space. The class of functions G is called an L_{p}-dense class of functions, 1 ≤ p < ∞, with parameter D and exponent L if for all numbers 0 < ε ≤ 1 and probability measures ν on the space (Y,Y ) there exists a finite ε-dense subset G_{ε,ν} = {g_{1},...,g_{m}} ⊂ G in the space L_{p}(Y,Y ,ν) with m ≤ Dε^{−L} elements, i.e. there exists such a set G_{ε,ν} ⊂ G with m ≤ Dε^{−L} elements for which inf_{g_{j}∈G_{ε,ν}}∫ |g−g_{j}|^{p}dν < ε^{p} for all functions g ∈ G.

(Here the set G_{ε,ν} may depend on the measure ν, but its cardinality is bounded by a number depending only on ε.)

In most results of this work the above defined L_{p}-dense classes will be considered only for the parameter p = 2. But at some points it will be useful to work also with L_{p}-dense classes with a different parameter p. Hence to avoid some repetitions I introduced the above definition for a general parameter p. When working with L_{p}-dense classes we shall consider only such classes of functions G whose elements are functions with bounded absolute value. Hence all integrals appearing in the definition of L_{p}-dense classes of functions are finite.

The following estimate will be proved.

Theorem 4.1 (Estimate on the supremum of a class of partial sums). Let us consider a sequence of independent and identically distributed random variables ξ_{1},...,ξ_{n}, n ≥ 2, with values in a measurable space (X,X ) and with some distribution μ. Beside this, let a countable and L_{2}-dense class of functions F with some parameter D ≥ 1 and exponent L ≥ 1 be given on the space (X,X ) which satisfies the conditions

∥f∥_{∞} = sup_{x∈X}|f(x)| ≤ 1,  for all f ∈ F  (4.1)
∥f∥_{2}^{2} = ∫ f^{2}(x)μ(dx) ≤ σ^{2}  for all f ∈ F  (4.2)

with some constant 0 < σ ≤ 1, and

∫ f(x)μ(dx) = 0 for all f ∈ F .  (4.3)

Define the normalized partial sums S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{k=1}^{n} f(ξ_{k}) for all f ∈ F.

There exist some universal constants C > 0, α > 0 and M > 0 such that the supremum of the normalized random sums S_{n}(f), f ∈ F, satisfies the inequality

P ( sup_{f∈F}|S_{n}(f)| ≥ u) ≤ C exp\{ −α ( \frac{u}{σ})^{2}\} for those numbers u

for which \sqrt{n}σ^{2} ≥ u ≥ Mσ(L^{3/4}log^{1/2} \frac{2}{σ} + (logD)^{3/4}),  (4.4)

where the numbers D and L in formula (4.4) agree with the parameter and exponent of the L_{2}-dense class F.

Remark. Here and also in the subsequent part of this work we consider random variables which take their values in a general measurable space (X,X ). The only restriction we impose on these spaces is that all sets consisting of one point are measurable, i.e. {x} ∈ X for all x ∈ X.

The condition \sqrt{n}σ^{2} ≥ u ≥ Mσ(L^{3/4}log^{1/2} \frac{2}{σ} + D^{3/4}) for the numbers u for which inequality (4.4) holds is natural. I discuss this after the formulation of Theorem 4.2 which can be considered as the Gaussian counterpart of Theorem 4.1. I also formulate a result in Example 4.3 which can be considered as part of this discussion.

The condition about the countable cardinality of F can be weakened with the help of the notion of countable approximability introduced below. For the sake of later applications I define it in a more general form than needed in this chapter. In the subsequent part of this work I shall assume that the probability measure I work with is complete, i.e. for all such pairs of sets A and B in the probability space (Ω,A ,P) for which A ∈ A, P(A) = 0 and B ⊂ A we have B ∈ A and P(B) = 0.

Definition of countably approximable classes of random variables. Let us have a class of random variables U(f), f ∈ F, indexed by a class of functions f ∈ F on a measurable space (Y,Y ). This class of random variables is called countably approximable if there is a countable subset F′⊂ F such that for all numbers u > 0 the sets A(u) = {ω: sup_{f∈F}|U(f)(ω)| ≥ u} and B(u) = {ω: sup_{f∈F′} |U(f)(ω)| ≥ u}

satisfy the identity P(A(u) \ B(u)) = 0.

Clearly, B(u) ⊂ A(u). In the above definition it was demanded that for all u > 0 the set B(u) should be almost as large as A(u). The following corollary of Theorem 4.1 holds.

Corollary of Theorem 4.1. Let a class of functions F satisfy the conditions of Theorem 4.1 with the only exception that instead of the condition about the countable cardinality of F it is assumed that the class of random variables S_{n}(f), f ∈ F, is countably approximable. Then the random variables S_{n}(f), f ∈ F, satisfy relation (4.4).

This corollary can be simply proved, only Theorem 4.1 has to be applied for the class F′. To do this it has to be checked that if F is an L_{2}-dense class with some parameter D and exponent L, and F′⊂ F, then F′is also an L_{2}-dense class with the same exponent L, only with a possibly different parameter D′.

To prove this statement let us choose for all numbers 0 < ε ≤ 1 and probability measures ν on (Y,Y ) some functions f_{1},...,f_{m} ∈ F with m ≤ D( \frac{ε}{2})^{−L} elements,

such that the sets D_{j} = \{f: ∫ |f − f_{j}|^{2}dν ≤ ( \frac{ε}{2})^{2}\} satisfy the relation ⋃_{j=1}^{m} D_{j} = Y.

For all sets D_{j} for which D_{j} ∩ F′is non-empty choose a function f′_{j}∈ D_{j} ∩ F′. In such a way we get a collection of functions f′_{j}from the class F′containing at most 2^{L}Dε^{−L} elements which satisfies the condition imposed for L_{2}-dense classes with exponent L and parameter 2^{L}D for this number ε and measure ν.

Next I formulate in Theorem 4.1′ a result about the supremum of the integral of a class of functions with respect to a normalized empirical distribution. It can be considered as a simple version of Theorem 4.1. I formulated this result, because Theorems 4.1 and 4.1′are special cases of their multivariate counterparts about the supremum of so-called U-statistics and multiple integrals with respect to a normalized empirical distribution discussed in Chapter 8. These results are also closely related, but the explanation of their relation demands some work.

Given a sequence of independent μ distributed random variables ξ_{1},...,ξ_{n} taking values in (X,X ) let us introduce their empirical distribution on (X,X ) as

μ_{n}(A)(ω) = \frac{1}{n}#\{j: 1 ≤ j ≤ n, ξ_{j}(ω) ∈ A\}  for all A ∈ X ,  (4.5)

and define for all measurable and μ integrable functions f the (random) integral

J_{n}(f) = J_{n,1}(f) = \sqrt{n}∫ f(x)(μ_{n}(dx) − μ(dx)).  (4.6)

Clearly

J_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} (f(ξ_{j}) − Ef(ξ_{j})) = S_{n}(f̂)

with f̂(x) = f(x) − ∫ f(x)μ(dx). It is not difficult to see that sup_{x∈X}|f̂(x)| ≤ 2 if

sup_{x∈X}|f(x)| ≤ 1, ∫ f̂(x)μ(dx) = 0, ∫ f̂^{2}(x)μ(dx) ≤ ∫ f^{2}(x)μ(dx), and if F is an L_{2}-

dense class of functions with parameter D and exponent L, then the class of functions F̄ consisting of the functions f̄(x) = \frac{1}{2} (f(x) − ∫ f(x)μ(dx)) , f ∈ F, is an L_{2}-dense class of functions with parameter D and exponent L. Indeed, since ∫(f̄− ḡ)^{2}dν ≤ \frac{1}{2} ∫(f − g)^{2}dν + \frac{1}{2} ∫(f − g)^{2}dμ = ∫(f − g)^{2} \frac{dμ+dν}{2} , hence \{f̄_{1},...,f̄_{m}\} is an εdense set of F̄ in the L_{2}(ν)-norm if {f_{1},...,f_{m}} is an ε-dense set of F in the L_{2}(\frac{μ+ν}{2} )-norm. Hence Theorem 4.1 implies the following result.

Theorem 4.1′(Estimate on the supremum of random integrals with respect to a normalized empirical distribution). Let us have a sequence of independent and identically distributed random variables ξ_{1},...,ξ_{n}, n ≥ 2, with distribution μ on a measurable space (X,X ) together with some class of functions F on this space which satisfies the conditions of Theorem 4.1 with the possible exception of condition (4.3). The estimate (4.4) remains valid if the random sums S_{n}(f) are replaced in it by the random integrals J_{n}(f) defined in (4.6). Moreover, similarly to the corollary of Theorem 4.1, the condition about the countable cardinality of the set F can be replaced by the condition that the class of random variables J_{n}(f), f ∈ F, is countably approximable.

All finite dimensional distributions of the set of random variables S_{n}(f), f ∈ F, considered in Theorem 4.1 converge to those of a Gaussian random field Z(f), f ∈ F, with expectation EZ(f) = 0 and correlation EZ(f)Z(g) = ∫ f(x)g(x)μ(dx), f,g ∈ F as n → ∞. Here, and in the subsequent part of the paper a collection of random variables indexed by some set of parameters will be called a Gaussian random field if for all finite subsets of these parameters the random variables indexed by this finite set are jointly Gaussian. We shall also define so-called linear Gaussian random fields. They consist of jointly Gaussian random variables Z(f), f ∈ G, indexed by the elements of a linear space f ∈ G which satisfy the relation Z(af + bg) = aZ(f) + bZ(g) with probability 1 for all real numbers a and b and f,g ∈ G. (Let us observe that a set of Gaussian random variables Z(f), indexed by the elements of a linear space f ∈ G such that EZ(f) = 0, and EZ(f)Z(g) = ∫ f(x)g(x)μ(dx) for all f,g ∈ F is a linear Gaussian random field. This can be seen by checking the identity E[Z(af +bg)−(aZ(f)+bZ(g))]^{2} = 0 for all real numbers a and b and f,g ∈ G in this case.)

Let us consider a linear Gaussian random field Z(f), f ∈ G, where the set of indices G = G_{μ} consists of the functions f square integrable with respect to a σfinite measure μ, and take an appropriate restriction of this field to some parameter set F ⊂ G. In the next Theorem 4.2 I present a natural Gaussian counterpart of Theorem 4.1 by means of an appropriate choice of F. Let me also remark that in Chapter 10 the multiple Wiener–Itô integrals of functions of k variables with respect to a white noise will be defined for all k ≥ 1. In the special case k = 1 the Wiener–Itô integrals for an appropriate class of functions f ∈ F yield a model for which Theorem 4.2 is applicable. Before formulating this result let us introduce the following definition which is a version of the definition of L_{p}-dense functions.

Definition of L_{p}-dense classes of functions with respect to a measure μ. Let a measurable space (X,X ) be given together with a measure μ on the σ-algebra X and a set F of X measurable real valued functions on this space. The set of functions F is called an L_{p}-dense class of functions, 1 ≤ p < ∞, with respect to the measure μ with parameter D and exponent L if for all numbers 0 < ε ≤ 1 there exists a finite ε-dense subset F_{ε} = {f_{1},...,f_{m}} ⊂ F in the space L_{p}(X,X ,μ) with m ≤ Dε^{−L} elements, i.e. such a set F_{ε} ⊂ F with m ≤ Dε^{−L} elements for which

inf_{f_{j}∈F_{ε}} ∫ |f − f_{j}|^{p}dμ < ε^{p} for all functions f ∈ F.

Theorem 4.2 (Estimate on the supremum of a class of Gaussian random variables). Let a probability measure μ be given on a measurable space (X,X ) together with a linear Gaussian random field Z(f), f ∈ G, such that EZ(f) = 0, EZ(f)Z(g) = ∫ f(x)g(x)μ(dx), f,g ∈ G, where G is the space of square integrable functions with respect to this measure μ. Let F ⊂ G be a countable and L_{2}-dense class of functions with respect to the measure μ with some exponent L ≥ 1 and parameter D ≥ 1 which also satisfies condition (4.2) with some 0 < σ ≤ 1.

Then there exist some universal constants C > 0 and M > 0 (for instance C = 4 and M = 16 is a good choice) such that the inequality

P ( sup_{f∈F}|Z(f)| ≥ u) ≤ C(D + 1)exp\{ −\frac{1}{256} ( \frac{u}{σ})^{2}\}
ifu ≥ ML^{1/2}σ log^{1/2} \frac{2}{σ}  (4.7)

holds with the parameter D and exponent L introduced in this theorem.

Remark. In formulas (4.4) of Theorem 4.1 and in (4.7) of Theorem 4.2 we had a slightly different lower bound on the numbers u for which these results give an estimate on the probability that the supremum of certain random variables is larger then u. Nevertheless in the most interesting cases when the exponent L and the parameter D of the L_{2}-dense class of functions we consider in these theorems are separated both from zero and infinity these bounds behave similarly. In such cases they have the magnitude const.σ log^{1/2} \frac{2}{σ}. In (4.7) the lower bound on the number u did not depend on the parameter D, since the dependence on this parameter appeared in the coefficient at the right-hand side of the inequality in this relation. The formula providing a lower bound on the number u had a coefficient L^{3/4} in (4.4) and not a coefficient L^{1/2} as in (4.7). This is a weak bound if L is very large, and it could be improved. But we did not work on this problem, because we were mainly interested in a good bound in the case when the exponent L is separated from infinity.

The exponent at the right-hand side of inequality (4.7) does not contain the best possible universal constant. One could choose the coefficient \frac{1−ε}{2} with arbitrary small ε > 0 instead of the coefficient \frac{1}{256} in the exponent at the right-hand side of (4.7) if the universal constants C > 0 and M > 0 are chosen sufficiently large in this inequality. Actually, later in Theorem 8.6 such an estimate will be proved which can be considered as the multivariate generalization of Theorem 4.2 with the expression −\frac{(1−ε)u^{2}}{2σ^{2}} in the exponent.

The condition about the countable cardinality of the set F in Theorem 4.2 could be weakened similarly to Theorem 4.1. But I omit the discussion of this question, since Theorem 4.2 was only introduced for the sake of a comparison between the Gaussian and non-Gaussian case. An essential difference between Theorems 4.1 and 4.2 is that the class of functions F considered in Theorem 4.1 had to be L_{2}- dense, while in Theorem 4.2 a weaker version of this property was needed. In Theorem 4.2 it was demanded that there exists a finite subset of F of relatively small cardinality which is dense in the L_{2}(μ) norm. In the L_{2}-density property imposed in Theorem 4.1 a similar property was demanded for all probability measures ν. The appearance of such a condition may be unexpected. It is not clear why we demand this property for such probability measures ν which have nothing to do with our problem. But as we shall see, the proof of Theorem 4.1 contains a conditioning argument where a lot of new conditional measures appear, and the L_{2}-density property is needed to work with all of them. One would also like to know some results that enable us to check when this condition holds. In the next chapter a notion popular in probability theory, the notion of Vapnik–Červonenkis classes will be introduced, and it will be shown that a Vapnik–Červonenkis class of functions bounded by 1 is L_{2}-dense.

Another difference between Theorems 4.1 and 4.2 is that the conditions of formula (4.4) contain the upper bound \sqrt{n}σ^{2} > u, and no similar condition was imposed in formula (4.7). The appearance of this condition in Theorem 4.1 can be explained by comparing this result with those of Chapter 3. As we have seen, we do not loose much information if we restrict our attention to the case u ≤ const.V_{n}^{2} = const.nσ^{2} in Bernstein’s inequality (if sums of independent and identically distributed random variables are considered). Theorem 4.1 gives an almost as good estimate for the supremum of normalized partial sums under appropriate conditions for the class F of functions we consider in this theorem as Bernstein’s inequality yields for the normalized partial sums of independent and identically distributed random variables with variance bounded by σ^{2}. But we could prove the estimate of Theorem 4.1 only under the condition \sqrt{n}σ^{2} > u. (Actually we could slightly improve this result. We could impose the condition B\sqrt{n}σ^{2} > u with an arbitrary constant B > 0 in (4.4) if the remaining constants are appropriately chosen in dependence of B in this formula.) It has also a natural reason why condition (4.1) about the supremum of the functions f ∈ F appeared in Theorems 4.1 and 4.1′, and no such condition was needed in Theorem 4.2.

The lower bounds for the level u were imposed in formulas (4.4) and (4.7) because of a similar reason. To understand why such a condition is needed in formula (4.7) let us consider the following example. Take a Wiener process W (t), 0 ≤ t ≤ 1, define for all 0 ≤ s < t ≤ 1 the functions f_{s,t}(·) on the interval [0,1] as f_{s,t}(u) = 1 if s ≤ u ≤ t, f_{s,t}(u) = 0 if 0 ≤ u < s or t < u ≤ 1, and introduce for all σ > 0 the following class of functions F_{σ}. F_{σ} = {f_{s,t}: 0 ≤ s < t ≤ 1, t − s ≤ σ^{2}, s and t are rational numbers.}. The integral Z(f) = ∫_{0}^{1}f(x)W (dx) can be defined for all square integrable functions f on the interval [0,1], and this yields a linear Gaussian random field on the space of square integrable functions. In the special case f = f_{s,t} we have Z(f_{s,t}) = ∫ f_{s,t}(u)W (du) = W (t) − W (s). It is not difficult to see that the Gaussian random field Z(f), f ∈ F_{σ}, satisfies the conditions of Theorem 4.2 with the number σ in formula (4.2). It is natural to expect that

P ( sup_{f∈F_{σ}} Z(f) > u) ≤ e^{−const.(u/σ)^{2}} . However, this relation does not hold if u =

u(σ) < 2(1−ε)σ log^{1/2} \frac{1}{σ} with some ε > 0. In such cases P ( sup_{f∈F_{σ}} Z(f) > u) → 1,

as σ → 0. This can be proved relatively simply with the help of the estimate P(Z(f_{s,t}) > u(σ)) ≥ const.σ^{2(1−ε)^{2}} if |t − s| = σ^{2} and the independence of the random integrals Z(f_{s,t}) if the functions f_{s,t} are indexed by such pairs (s,t) for which the intervals (s,t) are disjoint. This means that in this example formula (4.7) holds only under the condition u ≥ Mσ log^{1/2} \frac{1}{σ} with M = 2.

There is a classical result about the modulus of continuity of Wiener processes, and actually this result helped us to find the previous example. It is also worth mentioning that there are some concentration inequalities, see Ledoux [31] and Talagrand [56], which state that under very general conditions the distribution of the supremum of a class of partial sums of independent random variables or of the elements of a Gaussian random field is strongly concentrated around the expected value of this supremum. (Talagrand’s result in this direction is also formulated in Theorem 18.1 of this lecture note.) These results imply that the problems discussed in Theorems 4.1 and 4.2 can be reduced to a good estimate of the expected value E sup_{f∈F}|S_{n}(f)| and E sup_{f∈F}|Z(f)| of the supremum considered in these results. How-

ever, the estimation of the expected value of these suprema is not much simpler than the original problem.

Theorem 4.2 implies that under its conditions

E sup_{f∈F}|Z(f)| ≤ const.σ log^{1/2} \frac{2}{σ}

with an appropriate multiplying constant depending on the parameter D and exponent L of the class of functions F. In the case of Theorem 4.1 a similar estimate holds, but under more restrictive conditions. We also have to impose that \sqrt{n}σ^{2} ≥ const.σ log^{1/2} \frac{2}{σ} with a sufficiently large constant. This condition is needed to guarantee that the set of numbers u satisfying condition (4.4) is not empty. If this condition is violated, then Theorem 4.1 supplies a weaker estimate which we get by replacing σ by an appropriate σ̄ > σ, and by applying Theorem 4.1 with this number σ̄.

One may ask whether the above estimate on the expected value of the supremum of normalized partial sums holds without the condition \sqrt{n}σ^{2} ≥ const.σ log^{1/2} \frac{2}{σ}.

We show an example which gives a negative answer to this question. Since here we discuss a rather particular problem which is outside of our main interest in this work I give a rather sketchy explanation of this example. I present this example together with a Poissonian counterpart of it which may help to explain its background.

Example 4.3 (Supremum of partial sums with bad tail behaviour). Let ξ_{1},...,ξ_{n} be a sequence of independent random variables with uniform distribution in the interval [0,1]. Choose a sequence of real numbers, ε_{n}, n = 3,4,..., such that ε_{n} → 0 as n → ∞, and \frac{1}{2} ≥ ε_{n} ≥ n^{−δ} with a sufficiently small number δ > 0. Put σ_{n} =

ε_{n}\sqrt{\frac{logn}{n}}, and define the set of functions f̄_{j,n} (·) and f_{j,n}(·) on the interval [0,1] by the formulas f̄_{j,n} (x) = 1 if (j −1)σ_{n}^{2} ≤ x < jσ_{n}^{2}, f̄_{j,n} (x) = 0 otherwise, and f_{j,n}(x) = f̄_{j,n} (x) − σ_{n}^{2}, n = 3,4,..., 1 ≤ j ≤ \frac{1}{σ_{n}^{2}} . Put F_{n} = \{f_{j,n}(·): 1 ≤ j ≤ \frac{1}{σ_{n}^{2}} \}, S_{n}(f) =

\frac{1}{\sqrt{n}} ∑_{k=1}^{n} f(ξ_{k}) for f ∈ F_{n} and u_{n} = _{\frac{A}{log \frac{1}{εn}}} \frac{logn}{\sqrt{n}} with a sufficiently small A > 0. Then

lim_{n→∞} P ( sup_{f∈F_{n}} S_{n}(f) > u_{n}) = 1.

This example has the following Poissonian counterpart.

Example 4.3′(A Poissonian counterpart of Example 4.3). Let P̄_{n}(x) be a Poisson process on the interval [0,1] with parameter n and P_{n}(x) = \frac{1}{\sqrt{n}}[P̄_{n}(x)−nx], 0 ≤ x ≤ 1. Consider the same sequences of numbers ε_{n}, σ_{n} and u_{n} as in Example 4.3, and define the random variables Z_{n,j} = P_{n}(jσ_{n}^{2})−P_{n}((j −1)σ_{n}^{2}) for all n = 3,4,... and 1 ≤ j ≤ \frac{1}{σ_{n}^{2}} . Then

lim_{n→∞} P ( sup_{1≤j≤ \frac{1}{σn}} Z_{n,j} > u_{n}) = 1.

The classes of functions F_{n} in Example 4.3 are L_{2}-dense classes of functions with some exponent L and parameter D not depending on the parameter n and the choice of the numbers σ_{n}. It can be seen that even the class of function F = {f_{s,t}: f_{s,t}(x) = 1, if s ≤ x < t, f_{s,t}(x) = 0 otherwise.} consisting of functions defined on the interval [0,1] is an L_{2}-dense class with some exponent L and parameter D. This follows from the results discussed in the later part of this work (mainly Theorem 5.2), but it can be proved directly that this statement holds e.g. with L = 1 and D = 8. The classes of functions F_{n} also satisfy conditions (4.1), (4.2) and (4.3) of Theorem 4.1 with σ^{2} = σ̄_{n}^{2} = σ_{n}^{2} − σ_{n}^{4}, lim_{n→∞} \frac{σ̄_{n}}{σ_{n}} = 1, and the

number u_{n} satisfies the second condition u_{n} ≥ Mσ̄_{n}(L^{3/4}log^{1/2} \frac{2}{σ̄_{n}} + (logD)^{3/4}) in (4.4) for sufficiently large n. But it does not satisfy the first condition \sqrt{n}σ̄_{n}^{2} ≥ u_{n} of (4.4), and as a consequence Theorem 4.1 cannot be applied in this case. On the other hand, some calculation shows that u_{n} ≥ (\frac{2}{1+4δ} )^{1/2}_{\frac{A}{ε_{n}log \frac{1}{εn}}}σ_{n}log^{1/2} \frac{2}{σ} _{n}. Hence liminf_{n→∞}ε_{n}log \frac{1}{ε_{n}} ·_{\frac{1}{σ̄_{n} log^{1/2} \frac{2}{σ̄n}}} E sup_{f∈F_{n}} S_{n}(f) > 0 in this case. As ε_{n}log \frac{1}{ε_{n}} → 0 as n → ∞,

this means that the expected value of the supremum of the random sums considered in Example 4.3 does not satisfy the estimate limsup_{n→∞}_{\frac{1}{σ̄_{n} log^{1/2} \frac{2}{σ̄n}}} E sup_{f∈F_{n}} S_{n}(f) < ∞ sug-

gested by Theorem 4.1. Observe that \sqrt{n}σ̄_{n}^{2} ∼ const.ε_{n}σ̄_{n} log^{1/2} \frac{2}{σ̄_{n}} in this case, since \sqrt{n}σ̄_{n}^{2} ∼ ε_{n}^{2} \frac{logn}{\sqrt{n}}, and σ̄_{n} log^{1/2} \frac{2}{σ̄_{n}} ∼ const.ε_{n} \frac{logn}{\sqrt{n}}.

The proof of Examples 4.3 and 4.3′. First we prove the statement of Example 4.3′. For a fixed index n the number of random variables Z_{n,j} equals \frac{1}{σ_{n}^{2}} ≥ \frac{1}{ε_{n}^{2}} \frac{n}{logn} ≥ \frac{n}{logn},

and they are independent. Hence it is enough to show that P(Z_{n,j} > u_{n}) ≥ n^{−1/2} if first A > 0 and then δ > 0 (appearing in the condition ε_{n} > n^{−δ} ) are chosen sufficiently small, and n ≥ n_{0} with some threshold index n_{0} = n_{0}(A,δ).

Put ū_{n}= [\sqrt{n}u_{n}+ nσ_{n}^{2}] + 1, where [·] denotes integer part. Then P(Z_{n,j} > u_{n}) ≥

P(P̄_{n}(σ_{n}^{2}) ≥ ū_{n}) ≥ P(P̄_{n}(σ_{n}^{2}) = ū_{n}) = \frac{(nσ_{n}^{2})ūn}{ū_{n}!}e^{−nσ_{n}^{2}} ≥ ( \frac{nσ_{n}^{2}}{ū_{n}} ) ū_{n}e^{−nσ_{n}^{2}} . Some calcu-

lation shows that ū_{n}≤ _{\frac{Alogn}{log \frac{1}{εn}}}+ ε_{n}^{2}logn + 1 ≤ _{\frac{2Alogn}{log \frac{1}{εn}}}, \frac{nσ_{n}^{2}}{ū_{n}} ≥ \frac{ε_{n}^{2}log \frac{1}{εn}}{2A}, and log \frac{nσ_{n}^{2}}{ū_{n}} ≥

−2log \frac{1}{ε_{n}} if the constants A > 0, δ > 0 and threshold index n_{0} are appropriately

chosen. Hence P(Z_{n,j} > u_{n}) ≥ e^{−2ū_{n}log(1/ε_{n})−nσ_{n}^{2}} ≥ e^{−2Alogn−ε_{n}^{2}logn} ≥ \frac{1}{\sqrt{n}} if A_{0} > 0 is small enough.

The statement of Example 4.3 can be deduced from Example 4.3′ by applying Poissonian approximation. Let us apply the result of Example 4.3′ for a Poisson process P̄_{n/2}with parameter \frac{n}{2} and with such a number ε̄_{n/2}with which the value of σ_{n/2} equals the previously defined σ_{n}. Then ε̄_{n/2}∼ \frac{ε_{n}}{\sqrt{2}}, and the number of sample points of P̄_{n/2}is less than n with probability almost 1. Attaching additional sample points to get exactly n sample points we can get the result of Example 4.3. I omit the details.

In formulas (4.4) and (4.7) we formulated such a condition for the validity of Theorem 4.1 and Theorem 4.2 which contains a large multiplying constant ML^{3/4} and ML^{1/2} of σ log^{1/2} \frac{2}{σ} in the lower bound for the number u if we deal with such an L_{2}-dense class of functions F which has a large exponent L. At a heuristic level it is clear that in such a case a large multiplying constant appears. On the other hand, I did not try to find the best possible coefficients in the lower bound in relations (4.4) and (4.7).

In Theorem 4.1 (and in its version, Theorem 4.1′) it was demanded that the class of functions F should be countable. Later this condition was replaced by a weaker one about countable approximability. By restricting our attention to countable or countably approximable classes we could avoid some unpleasant measure theoretical problems which would have arisen if we had worked with the supremum of non-countably many random variables which may be non-measurable. There are some papers where possibly non-measurable models are also considered with the help of some rather deep results of the analysis and measure theory. Here I chose a different approach. I proved a simple result in the following Lemma 4.4 which enables us to show that in many interesting problems we can restrict our attention to countably approximable classes of random variables. In Chapter 18, in the discussion of the content of Chapter 4 I write more about the relation of this approach to the results of other works.

Lemma 4.4. Let a class of random variables U(f), f ∈ F, indexed by some set F of functions be given on a space (Y,Y ). If there exists a countable subset F′⊂ F of the set F such that the sets A(u) = {ω: sup_{f∈F}|U(f)(ω)| ≥ u} and B(u) =

{ω: sup_{f∈F′} |U(f)(ω)| ≥ u} introduced for all u > 0 in the definition of countable

approximability satisfy the relation A(u) ⊂ B(u−ε) for all u > ε > 0, then the class of random variables U(f), f ∈ F, is countably approximable.

The above property holds if for all f ∈ F, ε > 0 and ω ∈ Ω there exists a function f̄= f̄(f,ε,ω) ∈ F′such that |U(f̄)(ω)| ≥ |U(f)(ω)| − ε.

Proof of Lemma 4.4. If A(u) ⊂ B(u − ε) for all ε > 0, then P^{∗}(A(U) \ B(u)) ≤ lim_{ε→0}P(B(u − ε) \ B(u)) = 0, where P^{∗}(X) denotes the outer measure of a not nec-

essarily measurable set X ⊂ Ω, since ⋂_{ε→0} B(u − ε) = B(u), and this is what we

had to prove. If ω ∈ A(u), then for all ε > 0 there exists some f = f(ω) ∈ F such that |U(f)(ω)| > u − \frac{ε}{2}. If there exists some f̄= f̄(f, \frac{ε}{2},ω), f̄∈ F′such that |U(f̄)(ω)| ≥ |Uf(ω)| − \frac{ε}{2}, then |U(f̄)(ω)| > u − ε, and ω ∈ B(u − ε). This means that A(u) ⊂ B(u − ε).

The question about countable approximability also appears in the case of multiple random integrals with respect to a normalized empirical measure. To avoid some repetition we prove a result which also covers such cases. For this goal first we introduce the notion of multiple integrals with respect to a normalized empirical distribution.

Given a measurable function f(x_{1},...,x_{k}) on the k-fold product space (X^{k},X^{k}) and a sequence of independent random variables ξ_{1},...,ξ_{n} with some distribution μ on the space (X,X ) we define the integral J_{n,k}(f) of the function f with respect to the k-fold product of the normalized version of the empirical distribution μ_{n} introduced in (4.5) by the formula

J_{n,k}(f) = \frac{n^{k/2}}{k!}∫ ′f(x_{1},...,x_{k})(μ_{n}(dx_{1}) − μ(dx_{1}))...(μ_{n}(dx_{k}) − μ(dx_{k})),

where the prime in ∫ ′means that the diagonals x_{j} = x_{l},

1 ≤ j < l ≤ k, are omitted from the domain of integration.  (4.8)

In the case k ≥ 2 it will be assumed that the probability measure μ has no atoms.

Lemma 4.4 enables us to prove that certain classes of random integrals J_{n,k}(f), f ∈ F, defined with the help of some set of functions f ∈ F of k variables are countably approximable. I present an example for a class of such random integrals. I restrict my attention in this work to this case, because this seems to be the most important case in possible statistical applications. The result I formulate says roughly speaking that if we take the (multiple) integral of a function restricted to all possible rectangles (with respect to a normalized empirical distribution), then the class of these integrals is countably approximable. Hence the results of this lecture note is applicable for them.

Let us consider the case when X = R^{s}, the s-dimensional Euclidean space with some s ≥ 1. For two vectors u = (u^{(1)},...,u^{(s)}) ∈ R^{s}, v = (v^{(1)},...,v^{(s)}) ∈ R^{s} such that u < v, i.e. u^{(j)} < v^{(j)} for all 1 ≤ j ≤ s let B(u,v) denote the s-dimensional rectangle B(u,v) = {z: u < z < v}. Let us fix some function f(x_{1},...,x_{k}) of k variables such that sup|f(x_{1},...,x_{k})| ≤ 1, on the space (X^{k},X^{k}) = (R^{ks},B^{ks}), where B^{t} denotes the Borel σ-algebra on the Euclidean space R^{t}, together with some probability measure μ on (R^{s},B^{s}). For all pairs of vectors (u_{1},...,u_{k}), (v_{1},...,v_{k}) such that u_{j},v_{j} ∈ R^{s} and u_{j} ≤ v_{j}, 1 ≤ j ≤ k, let us define the function f_{u_{1},...,u_{k},v_{1},...,v_{k}}which equals the function f on the rectangle (u_{1},v_{1}) × ··· × (u_{k},v_{k}), and it is zero outside of this rectangle. Let us call a class of functions F consisting of functions of the form f_{u_{1},...,u_{k},v_{1},...,v_{k}}closed if it has the following property. If f_{u_{1},...,u_{k},v_{1},...,v_{k}}∈ F for some vectors (u_{1},...,u_{k}) and (v_{1},...,v_{k}), and u_{j} ≤ ū_{j} < v̄_{j} ≤ v_{j}, 1 ≤ j ≤ k, then f_{ū_{1},...,ū_{k},v̄_{1},...,v̄_{k}}∈ F. In Lemma 4.5 a closed class F of functions will be considered, and it will be proved that the random integrals of the functions from this class of functions F introduced in formula (4.8) constitute a countably approximable class.

Lemma 4.5. Let a function f on the Euclidean space R^{ks} satisfy the condition |f| ≤ 1 in all points, and let us consider a closed class F of functions of the form f_{u_{1},...,u_{k},v_{1},...,v_{k}}∈ (R^{sk},B^{sk}), u_{j},v_{j} ∈ R^{s}, u_{j} ≤ v_{j}, 1 ≤ j ≤ k, introduced in the previous paragraph with the help of this function f. Let us take n independent and identically distributed random variables ξ_{1},...,ξ_{n} with some distribution μ and values in the space (R^{s},B^{s}). Let μ_{n} denote the empirical distribution of this sequence. Then the class of random integrals J_{n,k}(f_{u_{1},...,u_{k},v_{1},...,v_{k}}) defined in formula (4.8) with functions f_{u_{1},...,u_{k},v_{1},...,v_{k}}∈ F is countably approximable.

Proof of Lemma 4.5. We shall prove that the definition of countable approximability is satisfied in this model if the class of functions F′ consists of those functions f_{u_{1},...,u_{k},v_{1},...,v_{k}}, u_{j} ≤ v_{j}, 1 ≤ j ≤ k, for which all coordinates of the vectors u_{j} and v_{j} are rational numbers.

Given some function f_{u_{1},...,u_{k},v_{1},...,v_{k}}, a real number 0 < ε < 1 and ω ∈ Ω let us choose a function f_{ū_{1},...,ū_{k},v̄_{1},...,v̄_{k}} ∈ F′determined with some vectors ū_{j} = ū_{j}(ε,ω), v̄_{j} = v̄_{j}(ε,ω) 1 ≤ j ≤ k, with rational coordinates u_{j} ≤ ū_{j} < v̄_{j} ≤ v_{j} in such a way that the sets K_{j} = B(u_{j},v_{j}) \ B(ū_{j},v̄_{j}) satisfy the relations μ(K_{j}) ≤ ε2^{−2k+1} n^{−k/2}, and ξ_{l}(ω) ∉ K_{j} for all j = 1,...,k and l = 1,...,n. Let us show that

|J_{n,k}(f_{ū_{1},...,ū_{k},v̄_{1},...,v̄_{k}})(ω) − J_{n,k}(f_{u_{1},...,u_{k},v_{1},...,v_{k}})(ω)| ≤ ε.  (4.9)

Lemma 4.4 (with the choice U(f) = J_{n,k}(f)) and relation (4.9) imply Lemma 4.5.

Relation (4.9) holds, since the difference of integrals at its left-hand side can be written as the sum of the 2^{k}− 1 integrals of the function f with respect to the k-fold product of the measure \sqrt{n}(μ_{n}− μ) on the domains D_{1} × ··· × D_{k} with the omission of the diagonals x_{j} = xj̄, 1 ≤ j,j̄≤ k, j ≠ j̄, where D_{j} is either the set K_{j} or B(u_{j},v_{j}) and D_{j} = K_{j} for at least one index j. It is enough to show that the absolute value of all these integrals is less than ε2^{−k}. This follows from the observations that |f(x_{1},...,x_{k})| ≤ 1, \sqrt{n}(μ_{n}− μ)(K_{j}) = −\sqrt{n}μ(K_{j}), μ(K_{j}) ≤ ε2^{−2k+1} n^{−k/2}, and the total variation of the signed measure \sqrt{n}(μ_{n}− μ) (restricted to the set B(u_{j},v_{j})) is less than 2\sqrt{n}.

In Lemma 4.5 we have shown with the help of Lemma 4.4 about an important class of functions that it is countably approximable. There are other interesting classes of functions whose countable approximability can be proved with the help of Lemma 4.4. But here we shall not discuss this problem.

Let us discuss the relation of the results in this chapter to an important result in probability theory, to the so-called fundamental theorem of the mathematical statistics. In that result a sequence of independent random variables ξ_{1}(ω),...,ξ_{n}(ω) is taken with some distribution function F (x), the empirical distribution function F_{n}(x) = F_{n}(x,ω) = \frac{1}{n}#\{j: 1 ≤ j ≤ n, ξ_{j}(ω) < x\} is introduced, and the difference F_{n}(x) − F (x) is considered. This result states that sup_{x}|F_{n}(x) − F (x)| tends to zero

with probability one.

Observe that sup_{x}|F_{n}(x) − F (x)| = n^{−1/2} sup_{f∈F}|J_{n}(f)|, where F consists of the

functions f_{x}(·), x ∈ R^{1}, defined by the relation f_{x}(u) = 1 if u < x, and f_{x}(u) = 0 if

u ≥ x. Theorem 4.1′yields an estimate for the probabilities P ( sup_{f∈F}|J_{n}(f)| > u) .

We have seen that the above class of functions F is countably approximable. The results of the next chapter imply that this class of functions is also L_{2}-dense. Let me remark that actually it is not difficult to check this property directly. Hence we can apply Theorem 4.1′to the above defined class of functions with σ = 1, and it yields

that P ( n^{−1/2} sup_{f∈F}|J_{n}(f)| > u) ≤ e^{−Cnu^{2}} if 1 ≥ u ≥ C̄n^{−1/2} with some universal

constants C > 0 and C̄ > 0. (The condition 1 ≥ u can actually be dropped.) The application of this estimate for the numbers ε > 0 together with the Borel–Cantelli lemma imply the fundamental theorem of the mathematical statistics.

In short, the results of this chapter yield more information about the closeness the empirical distribution function F_{n} and distribution function F than the fundamental theorem of the mathematical statistics. Moreover, since these results can also be applied for other classes of functions, they yield useful information about the closeness of the probability measure μ to the empirical distribution μ_{n}.

Vapnik–Červonenkis classes and L_{2}-dense classes of functions

In this chapter the most important notions and results will be presented about Vapnik–Červonenkis classes, and it will be explained how they help to show in some important cases that certain classes of functions are L_{2}-dense. The classes of L_{2}-dense classes played an important role in the previous chapter. The results of this chapter may help to find interesting classes of functions with this property. Some of the results of this chapter will be proved in Appendix A.

First I recall the definition of the following notion.

Definition of Vapnik-Červonenkis classes of sets and functions. Let a set X be given, and let us select a class D of subsets of this set X. We call D a Vapnik– Červonenkis class if there exist two real numbers B and K such that for all positive integers n and subsets S(n) = {x_{1},...,x_{n}} ⊂ X of cardinality n of the set X the collection of sets of the form S(n)∩D, D ∈ D, contains no more than Bn^{K} subsets of S(n). We shall call B the parameter and K the exponent of this Vapnik–Červonenkis class.

A class of real valued functions F on a space (Y,Y ) is called a Vapnik– Červonenkis class if the collection of graphs of these functions is a Vapnik–Červonenkis class, i.e. if the sets A(f) = {(y,t): y ∈ Y, min(0,f(y)) ≤ t ≤ max(0,f(y))}, f ∈ F, constitute a Vapnik–Červonenkis class of subsets of the product space X = Y × R^{1}.

The following result which was first proved by Sauer plays a fundamental role in the theory of Vapnik–Červonenkis classes. This result provides a relatively simple condition for a class D of subsets of a set X to be a Vapnik–Červonenkis class. Its proof is given in Appendix A. Before its formulation I introduce some terminology which is often applied in the literature.

Definition of shattering of a set. Let a set S and a class E of subsets of S be given. A finite set F ⊂ S is called shattered by the class E if all its subsets H ⊂ F can be written in the form H = E ∩ F with some element E ∈ E of the class of sets of E.

Theorem 5.1 (Sauer’s lemma). Let a finite set S = S(n) consisting of n elements be given together with a class E of subsets of S. If E shatters no subset of S of cardinality k, then E contains at most \binom{n}{0} + \binom{n}{1} + ··· + \binom{n}{k−1} subsets of S.

32 5 Vapnik–Červonenkis classes and L_{2}-dense classes of functions

The estimate of Sauer’s lemma is sharp. Indeed, if E contains all subsets of S of cardinality less than or equal to k − 1, then it shatters no subset of a set F of cardinality k (a set F of cardinality k cannot be written in the form E ∩ F, E ∈ E), and E contains \binom{n}{0} + \binom{n}{1} + ··· + \binom{n}{k−1} subsets of S. Sauer’s lemma states, that this is an extreme case. Any class of subsets E of S with cardinality greater than \binom{n}{0} + \binom{n}{1} + ··· + \binom{n}{k−1} shatters at least one subset of S with cardinality k.

Let us have a set X and a class of subsets D of it. One may be interested in when D is a Vapnik–Červonenkis class. Sauer’s lemma gives a useful condition for it. Namely, it implies that if there exists a positive integer k such that the class D shatters no subset of X of cardinality k, then D is a Vapnik–Červonenkis class. Indeed, let us take some number n ≥ k, fix an arbitrary set S(n) = {x_{1},...,x_{n}} ⊂ X of cardinality n, and introduce the class of subsets E = E (S(n)) = {S(n)∩D: D ⊂ D}. If D shatters no subset of X of cardinality k, then E shatters no subset of S(n) of cardinality k. Hence by Sauer’s lemma the class E contains at most \binom{n}{0}+\binom{n}{1}+··· + \binom{n}{k−1} elements. Let me remark that it is also proved that \binom{n}{0} + \binom{n}{1} + ··· + \binom{n}{k−1} ≤

1.5\frac{n^{k−1}}{(k−1)!} if n ≥ k + 1. This estimate gives a bound on the parameter and exponent of a Vapnik–Červonenkis class which satisfies the above condition.

Moreover, Theorem 5.1 also has the following consequence. Take an (infinite) set X and a class of its subsets D. There are two possibilities. Either there is some set S(n) ⊂ X of cardinality n for all integers n such that E (S(n)) contains all subsets of S(n), i.e. D shatters this set, or sup_{S: S⊂X,|S|=n} |E (S)| tends to infinity at most in a

polynomial order as n → ∞, where |S| and |E (S)| denote the cardinality of S and E (S).

To understand why Sauer’s lemma plays an important role in the theory of Vapnik–Červonenkis classes let us formulate the following consequence of the above considerations.

Corollary of Sauer’s lemma. Let a set X be given together with a class D of subsets of this set X. This class of sets D is a Vapnik–Červonenkis class if there exists a positive integer k such that D shatters no subset F ⊂ X of cardinality k. In other words if each set F = {x_{1},...,x_{k}} ⊂ X of cardinality k has a subset G ⊂ F which cannot be written in the form G = D ∩ F with some D ∈ D, then D is a Vapnik– Červonenkis class.

The following Theorem 5.2, an important result of Richard Dudley, states that a Vapnik–Červonenkis class of functions bounded by 1 is an L_{1}-dense class of functions.

Theorem 5.2 (A relation between the L_{1}-dense class and Vapnik–Červonenkis class property). Let f(y), f ∈ F, be a Vapnik–Červonenkis class of real valued functions on some measurable space (Y,Y ) such that sup_{y∈Y}|f(y)| ≤ 1 for all f ∈ F.

Then F is an L_{1}-dense class of functions on (Y,Y ). More explicitly, if F is a Vapnik–Červonenkis class with parameter B ≥ 1 and exponent K > 0, then it is an L_{1}-dense class with exponent L = 2K and parameter D = CB^{2}(4K)^{2K} with some universal constant C > 0.

Proof of Theorem 5.2. Let us fix some probability measure ν on (Y,Y ) and a real number 0 < ε ≤ 1. We are going to show that any finite set D(ε,ν) = {f_{1},...,f_{M}} ⊂ F such that ∫ |f_{j} − f_{k}|dν ≥ ε if j ≠ k, f_{j},f_{k} ∈ D(ε,ν) has cardinality M ≤ Dε^{−L} with some D > 0 and L > 0. This implies that F is an L_{1}-dense class with parameter D and exponent L. Indeed, let us take a maximal subset D̄(ε,ν) = {f_{1},...,f_{M}} ⊂ F such that the L_{1}(ν) distance of any two functions in this subset is at least ε. Maximality means in this context that no function f_{M+1} ∈ F can be attached to D̄(ε,ν) without violating this condition. Thus the inequality M ≤ Dε^{−L} means that D̄(ε,ν) is an ε-dense subset of F in the space L_{1}(Y,Y ,ν) with no more than Dε^{−L} elements.

In the estimation of the cardinality M of a set D(ε,ν) = {f_{1},...,f_{M}} ⊂ F with the property ∫ |f_{j} − f_{k}|dν ≥ ε if j ≠ k we exploit the Vapnik–Červonenkis class property of F in the following way. Let us choose relatively few p = p(M,ε) points (y_{l},t_{l}), y_{l} ∈ Y, −1 ≤ t_{l} ≤ 1, 1 ≤ l ≤ p, in the space Y ×[−1,1] in such a way that the set S_{0}(p) = {(y_{l},t_{l}), 1 ≤ l ≤ p} and graphs A(f_{j}) = {(y,t): y ∈ Y, min(0,f_{j}(y)) ≤ t ≤ max(0,f_{j}(y))}, f_{j} ∈ D(ε,ν) ⊂ F have the property that all sets A(f_{j}) ∩ S_{0}(p), 1 ≤ j ≤ M, are different. Then the Vapnik–Červonenkis class property of F implies that M ≤ Bp^{K}. Hence if there exists a set S_{0}(p) with the above property and with a relatively small number p, then this yields a useful estimate on M. Such a set S_{0}(p) will be given by means of the following random construction.

Let us choose the p points (y_{l},t_{l}), 1 ≤ l ≤ p, of the (random) set S_{0}(p) independently of each other in such a way that the coordinate y_{l} is chosen with distribution ν on (Y,Y ) and the coordinate t_{l} with uniform distribution on the interval [−1,1] independently of y_{l}. (The number p will be chosen later.) Let us fix some indices 1 ≤ j,k ≤ M, and estimate from above the probability that the sets A(f_{j}) ∩ S_{0}(p) and A(f_{k}) ∩ S_{0}(p) agree, where A(f) denotes the graph of the function f. Consider the symmetric difference A(f_{j})ΔA(f_{k}) of the sets A(f_{j}) and A(f_{k}). The sets A(f_{j}) ∩ S_{0}(p) and A(f_{k}) ∩ S_{0}(p) agree if and only if (y_{l},t_{l}) ∉ A(f_{j})ΔA(f_{k}) for all (y_{l},t_{l}) ∈ S_{0}(p). Let us observe that for a fixed l the estimate P((y_{l},t_{l}) ∈ A(f_{j})ΔA(f_{k})) = \frac{1}{2}(ν ×λ)(A(f_{j})ΔA(f_{k})) = \frac{1}{2} ∫ |f_{j} −f_{k}|dν ≥ \frac{ε}{2} holds, where λ denotes the Lebesgue measure. This implies that the probability that the (random) sets A(f_{j}) ∩ S_{0}(p) and A(f_{k}) ∩ S_{0}(p) agree can be bounded from above by (1 − \frac{ε}{2}) ^{p} ≤ e^{−pε/2} . Hence the probability that all sets A(f_{j}) ∩ S_{0}(p) are different is greater than 1 − \binom{M}{2}e^{−pε/2} ≥ 1 − \frac{M^{2}}{2}e^{−pε/2} . Choose p such that \frac{7}{4}e^{pε/2} > e^{(p+1)ε/2} > M^{2} ≥ e^{pε/2}. (We may assume that M > 1, in which case there is such a number p ≥ 1. We may really assume that M > 1, since we want to give an upper bound on M. Moreover, the estimate we shall give on it, satisfies this inequality.) Then the above probability is greater than \frac{1}{8}, and there exists some set S_{0}(p) with the desired property.

The inequalities M ≤ Bp^{K} and M^{2} ≥ e^{pε/2} imply that M ≥ M^{pε/4} ≥ e^{εM^{1/K}/4B^{1/K}}, i.e. \frac{logM^{1/K}}{M^{1/K}} ≥ \frac{ε}{4KB^{1/K}}. As \frac{logM^{1/K}}{M^{1/K}} ≤ CM^{−1/2K} for M ≥ 1 with some universal constant C > 0, this estimate implies that Theorem 5.2 holds with the exponent L and parameter D given in its formulation.

Let us observe that if F is an L_{1}-dense class of functions on a measure space (Y,Y ) with some exponent L and parameter D, and also the inequality sup_{y∈Y}|f(y)| ≤ 1

holds for all f ∈ F, then F is an L_{2}-dense class of functions with exponent 2L and parameter D2^{L}. Indeed, if we fix some probability measure ν on (Y,Y ) together with a number 0 < ε ≤ 1, and D(ε,ν) = \{f_{1},...,f_{M}\} is an \frac{ε^{2}}{2}-dense set of F in the space L_{1}(Y,Y ,ν), M ≤ 2^{L}Dε^{−2L}, then for all function f ∈ F some function f_{j} ∈ D(ε,ν) can be chosen in such a way that ∫(f − f_{j})^{2}dν ≤ 2∫ |f − f_{j}|dν ≤ ε^{2}. This implies that F is an L_{2}-dense class with the given exponent and parameter.

It is not easy to check whether a collection of subsets D of a set X is a Vapnik– Červonenkis class even with the help of Theorem 5.1. Therefore the following Theorem 5.3 which enables us to construct many non-trivial Vapnik–Červonenkis classes is of special interest. Its proof is given in Appendix A.

Theorem 5.3 (A way to construct Vapnik–Červonenkis classes). Let us consider a k-dimensional subspace G_{k} of the linear space of real valued functions defined on a set X, and define the level-set A(g) = {x: x ∈ X, g(x) ≥ 0} for all functions g ∈ G_{k}. Take the class of subsets D = {A(g): g ∈ G_{k}} of the set X consisting of the above introduced level sets. No subset S = S(k + 1) ⊂ X of cardinality k + 1 is shattered by D. Hence by Theorem 5.1 D is a Vapnik–Červonenkis class of subsets of X.

Theorem 5.3 enables us to construct interesting Vapnik–Červonenkis classes. Thus for instance the class of all half-spaces in a Euclidean space, the class of all ellipses in the plane, or more generally the level sets of k-order algebraic functions of p variables with a fixed number k constitute a Vapnik–Červonenkis class in the p-dimensional Euclidean space R^{p}. It can be proved that if C and D are Vapnik–Červonenkis classes of subsets of a set S, then also their intersection C ∩ D = {C ∩ D: C ∈ C , D ∈ D}, their union C ∪ D = {C ∪ D: C ∈ C , D ∈ D} and complementary sets C^{c} = {S \ C: C ∈ C } are Vapnik–Červonenkis classes. These results are less important for us, and their proofs will be omitted. We are interested in Vapnik–Červonenkis classes not for their own sake. We are going to find L_{2}-dense classes of functions, and Vapnik–Červonenkis classes help us in this. Indeed, Theorem 5.2 implies that if D is a Vapnik–Červonenkis class of subsets of a set S, then their indicator functions constitute a Vapnik–Červonenkis class of functions, and as a consequence an L_{1}-dense, hence also an L_{2}-dense class of functions. Then the results of Lemma 5.4 formulated below enable us to construct new L_{2}-dense classes of functions.

Lemma 5.4 (Some useful properties of L_{2}-dense classes). Let G be an L_{2}-dense class of functions on some space (Y,Y ) whose absolute values are bounded by one, and let f be a function on (Y,Y ) also with absolute value bounded by one. Then f · G = {f · g: g ∈ G } is also an L_{2}-dense class of functions. Let G_{1} and G_{2} be two L_{2}-dense classes of functions on some space (Y,Y ) whose absolute values are bounded by one. Then the classes of functions G_{1} + G_{2} = {g_{1} + g_{2}: g_{1} ∈ G_{1}, g_{2} ∈ G_{2}}, G_{1}·G_{2} = {g_{1}g_{2}: g_{1} ∈ G_{1}, g_{2} ∈ G_{2}}, min(G_{1},G_{2}) = {min(g_{1},g_{2}): g_{1} ∈ G_{1}, g_{2} ∈ G_{2}}, max(G_{1},G_{2}) = {max(g_{1},g_{2}): g_{1} ∈ G_{1}, g_{2} ∈ G_{2}} are also L_{2}-dense. If G is an L_{2}-dense class of functions, and G′⊂ G, then G′is also an L_{2}-dense class.

The proof of Lemma 5.4 is rather straightforward. One has to observe for instance that if g_{1},ḡ_{1} ∈ G_{1}, g_{2},ḡ_{2} ∈ G_{2} then |min(g_{1},g_{2}) − min(ḡ_{1},ḡ_{2})| ≤ |g_{1}− ḡ_{1})| + |g_{2}− ḡ_{2}|, hence if g_{1,1},...,g_{1,M_{1}} is an \frac{ε}{2}-dense subset of G_{1} and g_{2,1},...,g_{2,M_{2}} is an \frac{ε}{2}- dense subset of G_{2} in the space L_{2}(Y,Y ,ν) with some probability measure ν, then the functions min(g_{1,j},g_{2,k}), 1 ≤ j ≤ M_{1}, 1 ≤ k ≤ M_{2} constitute an ε-dense subset of min(G_{1},G_{2}) in L_{2}(Y,Y ,ν). The last statement of Lemma 5.4 was proved after the Corollary of Theorem 4.1. The details are left to the reader.

The above result enable us to construct some L_{2} dense class of functions. We give an example for it in the following Example 5.5 which is a consequence of Theorem 5.2 and Lemma 5.4.

Example 5.5. Take m measurable functions f_{j}(x), 1 ≤ j ≤ m, on a measurable space (X,X ) which have the property sup_{x∈X}|f_{j}(x)| ≤ 1 for all 1 ≤ j ≤ m. Let D be a

Vapnik-Červonenkis class consisting of measurable subsets of the set X. Define for all pairs (f_{j},D), f_{j}, 1 ≤ j ≤ m, and D ∈ D the function f_{j,D}(·) as f_{j,D}(x) = f_{j}(x) if x ∈ D, and f_{j,D}(x) = 0 if x ∉ D, i.e. f_{j,D}(·) is the restriction of the function f_{j}(·) to the set D. Then the set of functions F = {f_{j,D}: 1 ≤ j ≤ m, D ∈ D} is L_{2}-dense.

Beside this, Theorem 5.3 helps us to construct Vapnik-Červonenkis classes of sets. Let me also remark that it follows from the result of this chapter that the random variables considered in Lemma 4.5 are not only countably approximable, but the class of functions f_{u_{1},...,u_{k},v_{1},...,v_{k}}appearing in their definition is L_{2}-dense.

The proof of Theorems 4.1 and 4.2 on the supremum of random sums

In this chapter we prove Theorem 4.2, an estimate about the tail distribution of the supremum of an appropriate class of Gaussian random variables with the help of a method, called the chaining argument. We also investigate the proof of Theorem 4.1 which can be considered as a version of Theorem 4.2 about the supremum of partial sums of independent and identically distributed random variables. The chaining argument is not a strong enough method to prove Theorem 4.1, but it enables us to prove a weakened form of it formulated in Proposition 6.1. This result turned out to be useful in the proof of Theorem 4.1. It enables us to reduce the proof of Theorem 4.1 to a simpler statement formulated in Proposition 6.2. In this chapter we prove Proposition 6.1, formulate Proposition 6.2, and reduce the proof of Theorem 4.1 with the help of Proposition 6.1 to this result. The proof of Proposition 6.2 which demands different arguments is postponed to the next chapter. Before presenting the proofs I briefly describe the chaining argument.

Let us consider a countable class of functions F on a probability space (X,X ,μ) which is L_{2}-dense with respect to the probability measure μ. Let us have either a class of Gaussian random variables Z(f) with zero expectation such that EZ(f)Z(g) = ∫ f(x)g(x)μ(dx), f,g ∈ F, or a set of normalized partial sums

S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}), f ∈ F, where ξ_{1},...,ξ_{n} is a sequence of independent μ

distributed random variables with values in the space (X,X ), and assume that Ef(ξ_{j}) = 0 for all f ∈ F. We want to get a good estimate on the probability

P ( sup_{f∈F}Z(f) > u) or P ( sup_{f∈F}S_{n}(f) > u) if the class of functions F has some

nice properties. The chaining argument suggests to prove such an estimate in the following way.

Let us try to find an appropriate sequence of subset F_{1} ⊂ F_{2} ⊂ ···⊂ F such that

⋃_{N=1}^{∞} F_{N} = F, F_{N} is such a set of functions from F with relatively few elements for

which inf_{f∈F_{N}} ∫(f −f̄)^{2}dμ ≤ δ_{N} with an appropriately chosen number δ_{N} for all functions f̄∈ F, and let us give a good estimate on the probability P ( sup_{f∈F_{N}} Z(f) > u_{N})

or P ( sup_{f∈F_{N}} S_{n}(f) > u_{N}) for all N = 1,2,... with an appropriately chosen mono-

tone increasing sequence u_{N} such that lim_{N→∞} u_{N} = u.

We can get a relatively good estimate under appropriate conditions for the class of functions F by choosing the classes of functions F_{N} and numbers δ_{N} and u_{N} in an appropriate way. We try to bound the difference of the probabilities

P ( sup_{f∈F_{N+1}} Z(f) > u_{N+1}) − P ( sup_{f∈F_{N}} Z(f) > u_{N})

or of the analogous difference if Z(f) is replaced by S_{n}(f). For the sake of completeness define this difference also in the case N = 1 with the choice F_{0} = ∅, when the second probability in this difference equals zero.

The above mentioned difference of probabilities can be estimated in a natural way by taking for all functions f_{j_{N+1}} ∈ F_{N+1} a function f_{j_{N}} ∈ F_{N} which is close to it, more explicitly ∫(f_{j_{N+1}} − f_{j_{N}})^{2}dμ ≤ δ_{N}^{2}, and calculating the probability that the difference of the random variables corresponding to these two functions is greater than u_{N+1} − u_{N}. We can estimate these probabilities with the help of some results which give a relatively good bound on the tail distribution of Z(g) or S_{n}(g) if ∫ g^{2}dμ is small. The sum of all such probabilities gives an upper bound for the above considered difference of probabilities. Then we get an estimate for the probability

P ( sup_{f∈F_{N}} Z(f) > u_{N}) for all N = 1,2,..., by summing up the above estimate, and

we get a bound on the probability we are interested in by taking the limit N → ∞. This method is called the chaining argument. It got this name, because we estimate the contribution of a random variable corresponding to a function f_{j_{N+1}} ∈ F_{N+1} to the bound of the probability we investigate by taking the random variable corresponding to a function f_{j_{N}} ∈ F_{N} close to it, then we choose another random variable corresponding to a function f_{j_{N−1}} ∈ F_{N−1} close to this function, and by continuing this procedure we take a chain of subsequent functions and the random variables corresponding to them.

First we show how this method supplies the proof of Theorem 4.2. Then we turn to the investigation of Theorem 4.1. In the study of this problem the above method does not work well, because if two functions are very close to each other in the L_{2}(μ)-norm, then the Bernstein inequality (or an improvement of it) supplies a much weaker estimate for the difference of the partial sums corresponding to these two functions than the bound suggested by the central limit theorem. On the other hand, we shall prove a weaker version of Theorem 4.1 in Proposition 6.1 with the help of the chaining argument. This result will be also useful for us.

Proof of Theorem 4.2. Let us list the elements of F as {f_{0},f_{1},...} = F, and choose for all p = 0,1,2,... a set of functions F_{p} = {f_{a(1,p)},...,f_{a(m_{p},p)}} ⊂ F with m_{p} ≤ (D + 1)2^{2pL}σ^{−L} elements in such a way that inf_{1≤j≤m_{p}} ∫(f − f_{a(j,p)})^{2}dμ ≤ 2^{−4p} σ^{2}

for all f ∈ F, and let the set F_{p} contain also the function f_{p}. (We imposed the condition f_{p} ∈ F_{p} to guarantee that the relation f ∈ F_{p} holds with some index p for all f ∈ F. We could do this by slightly enlarging the upper bound we can give for the number m_{p} by replacing the factor D by D + 1 in it.) For all indices a(j,p) of the functions in F_{p}, p = 1,2,..., define a predecessor a(j′,p − 1) from the indices of the set of functions F_{p−1} in such a way that the functions f_{a(j,p)} and f_{a(j′,p−1))} satisfy the relation ∫(f_{(j,p)} −f_{(j′,p−1)} )^{2}dμ ≤ 2^{−4(p−1)} σ^{2}. With the help of the behaviour of the standard normal distribution function we can write the estimates

P(A(j,p)) = P ( |Z(f_{a(j,p)}) − Z(f_{a(j′,p−1)} )| ≥ 2^{−(1+p)} u)

≤ 2exp\{ −\frac{2^{−2(p+1)} u^{2}}{2 · 2^{−4(p−1)} σ^{2}}\} = 2exp\{ −\frac{2^{2p}u^{2}}{128σ^{2}} \}

1 ≤ j ≤ m_{p}, p = 1,2,...,

and

P(B(j)) = P ( |Z(f_{a(j,0)})| ≥ \frac{u}{2}) ≤ exp\{ −\frac{u^{2}}{8σ^{2}} \} , 1 ≤ j ≤ m_{0}.

The above estimates together with the relation ⋃_{p=0}^{∞} F_{p} = F which implies that

{|Z(f)| ≥ u} ⊂ ⋃_{p=1}^{∞} ⋃_{j=1}^{m_{p}} A(j,p) ∪ ⋃_{s=1}^{m_{0}} B(s) for all f ∈ F yield that

P ( sup_{f∈F}|Z(f)| ≥ u) ≤ P ( ⋃_{p=1}^{∞} ⋃_{j=1}^{m_{p}} A(j,p) ∪ ⋃_{s=1}^{m_{0}} B(s))

≤ ∑_{p=1}^{∞} ∑_{j=1}^{m_{p}} P(A(j,p)) + ∑_{s=1}^{m_{0}} P(B(s))

≤ ∑_{p=1}^{∞} 2(D + 1)2^{2pL}σ^{−L} exp\{ −\frac{2^{2p}u^{2}}{128σ^{2}} \} + 2(D + 1)σ^{−L} exp\{ −\frac{u^{2}}{8σ^{2}} \} .

If u ≥ ML^{1/2}σ log^{1/2} \frac{2}{σ} with M ≥ 16 (and L ≥ 1 and 0 < σ ≤ 1), then

2^{2pL}σ^{−L} exp\{ −\frac{2^{2p}u^{2}}{256σ^{2}} \} ≤ 2^{2pL}σ^{−L} ( \frac{σ}{2} )^{2^{2p}M^{2}L/256}≤ 2^{−pL} ≤ 2^{−p}

for all p = 0,1..., hence the previous inequality implies that

P ( sup_{f∈F}|Z(f)| ≥ u) ≤ 2(D + 1) ∑_{p=0}^{∞} 2^{−p} exp\{ −\frac{2^{2p}u^{2}}{256σ^{2}} \}

= 4(D + 1)exp\{ −\frac{u^{2}}{256σ^{2}} \} .

Theorem 4.2 is proved.

With an appropriate choice of the bound of the integrals in the definition of the sets F_{p} in the proof of Theorem 4.2 and some additional calculation it can be proved that the coefficient \frac{1}{256} in the exponent of the right-hand side (4.7) can be replaced by \frac{1−ε}{2} with arbitrary small ε > 0 if the remaining (universal) constants in this estimate are chosen sufficiently large.

The proof of Theorem 4.2 was based on a sufficiently good estimate on the probabilities P(|Z(f) − Z(g)| > u) for pairs of functions f,g ∈ F and numbers u > 0. In the case of Theorem 4.1 only a weaker bound can be given for the corresponding probabilities. There is no good estimate on the tail distribution of the difference S_{n}(f)−S_{n}(g) if its variance is small. As a consequence, the chaining argument supplies only a weaker result in this case. This result, where the tail distribution of the supremum of the normalized random sums S_{n}(f) is estimated on a relatively dense subset of the class of functions f ∈ F in the L_{2}(μ) norm will be given in Proposition 6.1. Another result will be formulated in Proposition 6.2 whose proof is postponed to the next chapter. It will be shown that Theorem 4.1 follows from Propositions 6.1 and 6.2.

Before the formulation of Proposition 6.1 I recall an estimate which is a simple

consequence of Bernstein’s inequality. If S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}) is the normalized

sum of independent, identically random variables, P(|f(ξ_{1})| ≤ 1) = 1, Ef(ξ_{1}) = 0, Ef(ξ_{1})^{2} ≤ σ^{2}, then there exists some constant α > 0 such that

P(|S_{n}(f)| > u) ≤ 2e^{−αu^{2}/σ^{2}} if 0 < u < \sqrt{n}σ^{2}.  (6.1)

In Proposition 6.1 we shall give a good (Gaussian type) estimate on the probabil-

ity P ( sup_{f∈Fσ̄} |S_{n}(f)| > \frac{u}{Ā}) with some parameter Ā> 1, where Fσ̄ is an appropriate

finite subset of a set of functions F satisfying the conditions of Theorem 4.1. (We introduced the number Ābecause of some technical reasons. We can formulate with its help such a result which simplifies the reduction of the proof of Theorem 4.1 to the proof of another result formulated in Proposition 6.2.) We cannot give a good estimate for the above probability for all u > 0, we can do this only for such numbers u which are in an appropriate interval depending on the parameter σ appearing in condition (4.2) of Theorem 4.1 and the parameter Āwe chose in Proposition 6.1. This fact may explain why we could prove the estimate of Theorem 4.1 only for such numbers u which satisfy the condition imposed in formula (4.4). The choice of the set of functions Fσ̄ ⊂ F depends of the number u appearing in the probability we want to estimate. It is such a subset of relatively small cardinality of F whose L_{2}(μ)-norm distance from all elements of F is less than σ̄ = σ̄(u) with an appropriately defined number σ̄(u). With the help of Proposition 6.1 we want to reduce the proof of Theorem 4.1 to a result formulated in the subsequent Proposition 6.2. To do this we still need an upper bound on the cardinality of Fσ̄ and some upper and lower bounds on the value of σ̄(u). In Proposition 6.1 we shall formulate such results, too.

Proposition 6.1. Let us have a countable, L_{2}-dense class of functions F with parameter D ≥ 1 and exponent L ≥ 1 with respect to some probability measure μ on a measurable space (X,X ) whose elements satisfy relations (4.1), (4.2) and (4.3) with this probability measure μ on (X,X ) and some real number 0 < σ ≤ 1. Take a sequence of independent, μ-distributed random variables ξ_{1},...,ξ_{n}, n ≥ 2, and

define the normalized random sums S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{l=1}^{n} f(ξ_{l}), for all f ∈ F. Let us fix

some number Ā≥ 1. There exists some number M = M(Ā) such that with these parameters Āand M = M(Ā) ≥ 1 the following relations hold.

For all numbers u > 0 such that nσ^{2} ≥ ( \frac{u}{σ})^{2} ≥ M(Llog \frac{2}{σ} + logD) a number σ̄ = σ̄(u), 0 ≤ σ̄ ≤ σ ≤ 1, and a collection of functions Fσ̄ = {f_{1},...,f_{m}} ⊂ F with m ≤ Dσ̄^{−L} elements can be chosen in such a way that the union of the sets D_{j} = {f: f ∈ F ,∫ |f − f_{j}|^{2}dμ ≤ σ̄^{2}}, 1 ≤ j ≤ m, cover the set of functions F,

i.e. ⋃_{j=1}^{m} D_{j} = F, and the normalized random sums S_{n}(f), f ∈ Fσ̄, n ≥ 2, satisfy the

inequality

P ( sup_{f∈Fσ̄} |S_{n}(f)| ≥ \frac{u}{Ā}) ≤ 4exp\{ −α ( \frac{u}{10Āσ} )^{2}\}

under the condition nσ^{2} ≥ (\frac{u}{σ})^{2} ≥ M(Llog \frac{2}{σ} + logD)  (6.2)

with the constants α in formula (6.1) and the exponent L and parameter D of the

L_{2}-dense class F. The inequality \frac{1}{16}(\frac{u}{Ā ; σ̄})^{2} ≥ nσ̄^{2} ≥ \frac{1}{64} ( \frac{u}{Āσ} )^{2} also holds with the number σ̄ = σ̄(u). If the number u satisfies also the inequality

nσ^{2} ≥ ( \frac{u}{σ})^{2} ≥ M ( L^{3/2}log \frac{2}{σ} + (logD)^{3/2})  (6.3)

with a sufficiently large number M = M(Ā), then the relation nσ̄^{2} ≥ Llogn + logD holds, too.

Remark. Under the conditions L ≥ 1 and D ≥ 1 of Proposition 6.1 the condition formulated in relation (6.3) (with a sufficiently large number M = M(Ā)) is stronger than the condition (\frac{u}{σ})^{2} ≥ M(Llog \frac{2}{σ} + logD) imposed in formula (6.2). To see this observe that although (logD)^{3/2} ≤ logD if logD ≤ 1, but this effect can be compensated by choosing a sufficiently large parameter M in formula (6.3) and exploiting that Llog \frac{2}{σ} ≥ log2.

Proposition 6.1 helps to reduce the proof of Theorem 4.1 to the case when such classes of functions F are considered whose elements are such functions whose L_{2}-norm is bounded by a relatively small number σ̄. In more detail, the proof of Theorem 4.1 can be reduced to a good estimate on the distribution of the supremum of random variables sup_{f∈D_{j}} |S_{n}(f − f_{j})| for all classes D_{j}, 1 ≤ j ≤ m, by

means of Proposition 6.1. To carry out such a reduction we also need the inequality nσ̄^{2} ≥ Llogn + logD (or a slightly weaker version of it). This is the reason why we have finished Proposition 6.1 with the statement that this inequality holds under the condition (6.3). We also have to know that the number m of the classes D_{j} is not too large. Beside this, we need some estimates on the number σ̄ = σ̄(u) which is an upper bound for the L_{2}-norm of the functions f − f_{j}, f ∈ D_{j}. To get such bounds for σ̄ that we need in the applications of Proposition 6.1 we introduced a large parameter Ā in the formulation of Proposition 6.1 and imposed a condition with a sufficiently large number M = M(Ā) in formula (6.3). This condition reappears in Theorem 4.1 in the conditions of the estimate (4.4).

Let me remark that one of the inequalities the number σ̄ introduced in Proposition 6.1 satisfies has the consequence u > const.\sqrt{n}σ̄^{2} with an appropriate constant. Hence to complete the proof of Theorem 4.1 we have to estimate the probability

P ( sup_{f∈F}S_{n}(f)| > u) also in such cases when the L_{2} norm of the functions in F is

bounded with such a number σ̄ for which u > const.\sqrt{n}σ̄^{2}. On the other hand, we got an estimate in Proposition 6.1 if u < \sqrt{n}σ^{2}, (see formula (6.2), and this is an inequality in the opposite direction. Hence to complete the proof of Theorem 4.1 with the help of Proposition 6.1 we need a result whose proof demands an essentially different method. Proposition 6.2 formulated below is such a result. I shall show that Theorem 4.1 is a consequence of Propositions 6.1 and 6.2. Proposition 6.1 is proved at the end of this chapter, while the proof of Proposition 6.2 is postponed to the next chapter.

Proposition 6.2. Let us have a probability measure μ on a measurable space (X,X ) together with a sequence of independent and μ distributed random variables ξ_{1},...,ξ_{n}, n ≥ 2, and a countable, L_{2}-dense class of functions f = f(x) on (X,X ) with some parameter D ≥ 1 and exponent L ≥ 1 which satisfies conditions (4.1), (4.2) and (4.3) with some 0 < σ ≤ 1 such that the inequality nσ^{2} > Llogn + logD holds. Then there exists a threshold index A_{0} ≥ 5 such that the normalized random sums S_{n}(f), f ∈ F, introduced in Theorem 4.1 satisfy the inequality

P ( sup_{f∈F}|S_{n}(f)| ≥ An^{1/2}σ^{2}) ≤ e^{−A^{1/2}nσ^{2}/2} ifA ≥ A_{0}.  (6.4)

I did not try to find optimal parameters in formula (6.4). Even the coefficient −A^{1/2} in the exponent at its right-hand side could be improved. The result of Proposition 6.2 is similar to that of Theorem 4.1. Both of them give an estimate on a probability of the form P ( sup_{f∈F}|S_{n}(f)| ≥ u) with some class of functions F.

The essential difference between them is that in Theorem 4.1 this probability is considered for u ≤ n^{1/2}σ^{2} while in Proposition 6.2 the case u = An^{1/2}σ^{2} with A ≥ A_{0} is taken, where A_{0} is a sufficiently large positive number. Let us observe that in this case no good Gaussian type estimate can be given for the probabilities P(S_{n}(f) ≥ u), f ∈ F. In this case Bernstein’s inequality yields the bound

P(S_{n}(f) > An^{1/2}σ^{2}) = P ( ∑_{l=1}^{n} f(ξ_{l}) > uV_{n}) < e^{−const.Anσ^{2}} with u = A\sqrt{n}σ and

V_{n} = \sqrt{n}σ for each single function f ∈ F which takes part in the supremum of formula (6.4). The estimate (6.4) yields a slightly weaker estimate for the supremum of such random variables, since it contains the coefficient A^{1/2} instead of A in the exponent of the estimate at the right-hand side. But also such a bound will be sufficient for us.

In Proposition 6.2 such a situation is considered when the irregularities of the summands provide a non-negligible contribution to the probabilities P(|S_{n}(f)| ≥ u), and the chaining argument applied in the proof of Theorem 4.2 does not give a good estimate on the probability at the left-hand side of (6.4). This is the reason why we separated the proof of Theorem 4.1 to two different statements given in Proposition 6.1 and 6.2.

In the proof of Theorem 4.1 Proposition 6.1 will be applied with a sufficiently large number Ā≥ 1 and an appropriate number M = M(Ā) appearing in the formulation of this result. Proposition 6.2 will be applied for the sets of functions F = F_{j} = \{ \frac{g−f_{j}}{2} : g ∈ D_{j}\} and number σ = σ̄, with the number σ̄, functions f_{j} and sets of functions D_{j} introduced in Proposition 6.1 and with the parameter A_{0} appearing in the formulation of Proposition 6.2. We can write

P ( sup_{f∈F}|S_{n}(f)| ≥ u) ≤ P ( sup_{f∈Fσ̄} |S_{n}(f)| ≥ \frac{u}{Ā})  (6.5)

+ ∑_{j=1}^{m} P ( sup_{g∈D_{j}} |S_{n}( \frac{f_{j} − g}{2} ) | ≥ ( \frac{1}{2} − \frac{1}{2Ā}) u) ,

where m is the cardinality of the set of functions Fσ̄ appearing in Proposition 6.1, which is bounded by m ≤ Dσ̄^{−L} . We want to choose the number Āin such a way that the inequality (\frac{1}{2} − \frac{1}{2Ā})u ≥ A_{0}\sqrt{n}σ̄^{2} holds, since in this case Proposition 6.2 with the choice A = A_{0} yields a good estimate on the second term in (6.5). This inequality is equivalent to nσ̄^{2} ≤ (\frac{1}{2A_{0}} − \frac{1}{2A_{0}Ā})^{2}(\frac{u}{σ̄})^{2}. On the other hand, (\frac{u}{4Āσ̄})^{2} ≥ nσ̄^{2} by

Proposition 6.1, hence the desired inequality holds if \frac{1}{2A_{0}} − \frac{1}{2A_{0}Ā}≥ \frac{1}{4Ā}. Hence with

the choice Ā= max(1, \frac{A_{0}+2}{2} ) and a sufficiently large M = M(Ā) we can bound both terms at the right-hand side of (6.5) with the help of Propositions 6.1 and 6.2.

With such a choice of Āwe can write by Proposition 6.2

P ( sup_{g∈D_{j}} |S_{n}( \frac{f_{j} − g}{2} ) | ≥ ( \frac{1}{2} − \frac{1}{2Ā}) u) ≤ P ( sup_{g∈D_{j}} |S_{n}( \frac{f_{j} − g}{2} ) | ≥ A_{0}\sqrt{n}σ̄^{2})
≤ e^{−A_{0}^{1/2}nσ̄^{2}/2} for all 1 ≤ j ≤ m.

(Observe that the set of functions \frac{f_{j}−g}{2} , g ∈ D_{j}, is an L_{2}-dense class with parameter D and exponent L.) Hence Proposition 6.1 together with the bound m ≤ Dσ̄^{−L} and formula (6.5) imply that

P ( sup_{f∈F}|S_{n}(f)| ≥ u) ≤ 4exp\{ −α ( \frac{u}{10Āσ} )^{2}\} + Dσ̄^{−L} e^{−A_{0}^{1/2}nσ̄^{2}/2}.  (6.6)

To get the estimate in Theorem 4.1 from inequality (6.6) we show that the inequality nσ̄^{2} ≥ Llogn + logD (with L ≥ 1, D ≥ 1 and n ≥ 2) which is valid under the conditions of Proposition 6.1 implies that Dσ̄^{−L} ≤ e^{nσ̄^{2}}. Indeed, we have to show that logD + Llog \frac{1}{σ̄} ≤ nσ̄^{2}. But we have nσ̄^{2} ≥ Llogn ≥ logn, hence \frac{1}{σ̄} ≤ \sqrt{\frac{n}{logn}} ≤ n, thus log \frac{1}{σ̄} ≤ logn, and logD + Llog \frac{1}{σ̄} ≤ logD + Llogn ≤ nσ̄^{2}, as we have claimed.

This inequality together with the inequality nσ̄^{2} ≥ \frac{1}{64}(\frac{u}{Āσ} )^{2}, proved in Proposition 6.1 imply that

Dσ̄^{−L} e^{−A_{0}^{1/2}nσ̄^{2}/2} ≤ exp\{ −( \frac{A_{0}^{1/2}}{2}− 1) nσ̄^{2}\} ≤ exp\{ −\frac{(A_{0}^{1/2}− 2)}{128Ā^{2}} ( \frac{u}{σ})^{2}\} .

Hence relation (6.6) yields that

P ( sup_{f∈F}|S_{n}(f)| ≥ u) ≤ 4exp\{ −\frac{α}{100Ā^{2}}( \frac{u}{σ})^{2}\} + exp\{ −\frac{(A_{0}^{1/2}− 2)}{128Ā^{2}} ( \frac{u}{σ})^{2}\} ,

and because of the relation A_{0} ≥ 5 this estimate implies Theorem 4.1. Let me remark that the condition \sqrt{n}σ^{2} ≥ u ≥ Mσ(L^{3/4}log^{1/2} \frac{2}{σ} +(logD)^{3/4}) appears in formula (4.4) because of condition (6.3) imposed in Proposition 6.1. (The parameter M in formula (4.4) can be chosen as twice the parameter M in (6.3).)

I finish this chapter with the proof of Proposition 6.1.

Proof of Proposition 6.1. Let us list the members of F, as f_{1},f_{2},..., and choose for all p = 0,1,2,... a set F_{p} = {f_{a(1,p)},...,f_{a(m_{p},p)}} ⊂ F with m_{p} ≤ D2^{2pL}σ^{−L} elements in such a way that inf_{1≤j≤m_{p}} ∫(f − f_{a(j,p)})^{2}dμ ≤ 2^{−4p} σ^{2} for all f ∈ F.

For all indices a(j,p), p = 1,2,..., 1 ≤ j ≤ m_{p}, choose a predecessor a(j′,p − 1), j′ = j′(j,p), 1 ≤ j′ ≤ m_{p−1} , in such a way that the functions f_{a(j,p)} and f_{a(j′,p−1)} satisfy the relation ∫ |f_{a(j,p)} −f_{a(j′,p−1)} |^{2}dμ ≤ σ^{2}2^{−4(p−1)} . Then we have

∫ ( \frac{f_{a(j,p)}−f_{a(j′,p−1)}}{2} )^{2} dμ ≤ 4σ^{2}2^{−4p} and

sup_{x_{j}∈X,1≤j≤k} | \frac{f_{a(j,p)}(x_{1},...,x_{k}) − f_{a(j′,p−1)} (x_{1},...,x_{k})}{2} | ≤ 1.

Relation (6.1) yields that

P(A(j,p)) = P ( \frac{1}{2}|S_{n}(f_{a(j,p)} − f_{a(j′,p−1)} )| ≥ \frac{2^{−(1+p)} u}{2Ā})

≤ 2exp\{ −α ( \frac{2^{p}u}{8Āσ} ) ^{2}\} if nσ^{2} ≥ 2^{6p}( \frac{u}{16Āσ} )^{2},

1 ≤ j ≤ m_{p}, p = 1,2,...,  (6.7)

and

P(B(s)) = P ( |S_{n}(f_{s,0})| ≥ \frac{u}{2Ā}) ≤ 2exp\{ −α ( \frac{u}{2Āσ} )^{2}\} , 1 ≤ s ≤ m_{0},
if nσ^{2} ≥ ( \frac{u}{2Āσ} )^{2}.  (6.8)

Choose an integer R = R(u), R ≥ 1, by the inequality

2^{6(R+1)} ( \frac{u}{16Āσ} )^{2} > nσ^{2} ≥ 2^{6R}( \frac{u}{16Āσ} )^{2},

define σ̄^{2} = 2^{−4R}σ^{2} and Fσ̄ = F_{R}. (As nσ^{2} ≥ ( \frac{u}{σ})^{2} and Ā≥ 1 by our conditions, there exists such a number R ≥ 1. The number R was chosen as the largest number p for which the second relation of formula (6.7) holds.) Then the cardinality m of the set Fσ̄ equals m_{R} ≤ D2^{2RL}σ^{−L} = Dσ̄^{−L} , and the sets D_{j} are

D_{j} = {f: f ∈ F ,∫(f_{a(j,R)} − f)^{2}dμ ≤ 2^{−4R}σ^{2}}, 1 ≤ j ≤ m_{R}, hence ⋃_{j=1}^{m} D_{j} = F.

Beside this, with our choice of the number R inequalities (6.7) and (6.8) can be applied for 1 ≤ p ≤ R. Hence the definition of the predecessor of an index (j,p)

implies that \{ ω: sup_{f∈Fσ̄} |S_{n}(f)(ω)| ≥ \frac{u}{Ā}\} ⊂ ⋃_{p=1}^{R} ⋃_{j=1}^{m_{p}} A(j,p) ∪ ⋃_{s=1}^{m_{0}} B(s), and

P ( sup_{f∈Fσ̄} |S_{n}(f)| ≥ \frac{u}{Ā}) ≤ P ( ⋃_{p=1}^{R} ⋃_{j=1}^{m_{p}} A(j,p) ∪ ⋃_{s=1}^{m_{0}} B(s))

≤ ∑_{p=1}^{R} ∑_{j=1}^{m_{p}} P(A(j,p)) + ∑_{s=1}^{m_{0}} P(B(s))

≤ ∑_{p=1}^{∞} 2D2^{2pL}σ^{−L} exp\{ −α ( \frac{2^{p}u}{8Āσ} ) ^{2}\} + 2Dσ^{−L} exp\{ −α ( \frac{u}{2Āσ} )^{2}\} .

If the relation (\frac{u}{σ})^{2} ≥ M(Llog \frac{2}{σ} + logD) holds with a sufficiently large constant M (depending on Ā), and σ ≤ 1, then the inequalities

D2^{2pL}σ^{−L} exp\{ −α ( \frac{2^{p}u}{8Āσ} ) ^{2}\} ≤ 2^{−p} exp\{ −α ( \frac{2^{p}u}{10Āσ} ) ^{2}\}

hold for all p = 1,2,..., and

Dσ^{−L} exp\{ −α ( \frac{u}{2Āσ} )^{2}\} ≤ exp\{ −α ( \frac{u}{10Āσ} )^{2}\} .

Hence the previous estimate implies that

P ( sup_{f∈Fσ̄} |S_{n}(f)| ≥ \frac{u}{Ā}) ≤ ∑_{p=1}^{∞} 2 · 2^{−p} exp\{ −α ( \frac{2^{p}u}{10Āσ} ) ^{2}\}

+2exp\{ −α ( \frac{u}{10Āσ} )^{2}\} ≤ 4exp\{ −α ( \frac{u}{10Āσ} )^{2}\} ,

and relation (6.2) holds.

As σ^{2} = 2^{4R}σ̄^{2} the inequality

2^{−4R} · \frac{2^{6R}}{256} ( \frac{u}{Āσ} )^{2}≤ nσ̄^{2} = 2^{−4R}nσ^{2}
≤ 2^{−4R} · \frac{2^{6(R+1)}}{256} ( \frac{u}{Āσ} )^{2} = \frac{1}{4} · 2^{−2R} ( \frac{u}{Ā ; σ̄})^{2}

holds, and this implies (together with the relation R ≥ 1) that

\frac{1}{64} ( \frac{u}{Āσ} )^{2} ≤ nσ̄^{2} ≤ \frac{1}{16} ( \frac{u}{Ā ; σ̄})^{2},

as we have claimed. It remained to show that under the condition (6.3) nσ̄^{2} ≥ Llogn + logD.

This inequality clearly holds under the conditions of Proposition 6.1 if σ ≤ n^{−1/3} , since in this case log \frac{2}{σ} ≥ \frac{logn}{3}, and nσ̄^{2} ≥ \frac{1}{64}(\frac{u}{Āσ} )^{2} ≥ \frac{1}{64Ā^{2}}M(Llog \frac{2}{σ} + logD) ≥ \frac{1}{192Ā^{2}}M(Llogn + logD)) ≥ Llogn + logD if M ≥ M_{0}(Ā) with a sufficiently large number M_{0}(Ā).

If σ ≥ n^{−1/3} , we can exploit that the inequality 2^{6R}( \frac{u}{Āσ} )^{2} ≤ 256nσ^{2} holds because of the definition of the number R. It can be rewritten as

2^{−4R} ≥ 2^{−16/3} [ \frac{( \frac{u}{Āσ} ) 2}{nσ^{2}} ]^{2/3}.

6 The proof of Theorems 4.1 and 4.2 on the supremum of random sums 47

Hence nσ̄^{2} = 2^{−4R}nσ^{2} ≥ \frac{2^{−16/3}}{Ā^{4/3}} (nσ^{2})^{1/3}( \frac{u}{σ})^{4/3}. As log \frac{2}{σ} ≥ log2 > \frac{1}{2} the inequalities nσ^{2} ≥ n^{1/3} and (\frac{u}{σ})^{2} ≥ M(L^{3/2}log \frac{2}{σ} +(logD)^{3/2}) ≥ \frac{M}{2}(L^{3/2}+(logD)^{3/2}) hold. They yield that

nσ̄^{2} ≥ \frac{Ā^{−4/3}}{50} (nσ^{2})^{1/3}( \frac{u}{σ})^{4/3} ≥ \frac{Ā^{−4/3}}{50} n^{1/9}( \frac{M}{2}) ^{2/3}(L^{3/2}+ (logD)^{3/2})^{2/3}
≥ \frac{M^{2/3}n^{1/9}(L + logD)}{100Ā^{4/3}} ≥ Llogn + logD

if M = M(Ā) is chosen sufficiently large.

The completion of the proof of Theorem 4.1

This chapter contains the proof of Proposition 6.2 with the help of a symmetrization argument, and this completes the proof of Theorem 4.1. By symmetrization argument I mean the reduction of the investigation of sums of the form ∑_{j} f(ξ_{j}) to sums of the form ∑_{j} ε_{j}f(ξ_{j}), where ε_{j} are independent random variables, independent also of the random variables ξ_{j}, and P(ε_{j} = 1) = P(ε_{j} = −1) = \frac{1}{2}. First a symmetrization lemma is proved, and then such an inductive statement is formulated in Proposition 7.3 which implies Proposition 6.2. Proposition 7.3 will be proved with the help of the symmetrization lemma and a conditioning argument. To carry out such a program we shall need some estimates which follow from Hoeffding’s inequality formulated in Theorem 3.4.

First I formulate the symmetrization lemma we shall apply.

Lemma 7.1 (Symmetrization Lemma). Let Z_{n} and Z̄_{n}, n = 1,2,..., be two sequences of random variables independent of each other, and let the random variables Z̄_{n}, n = 1,2,..., satisfy the inequality

P(|Z̄_{n}| ≤ α) ≥ β for all n = 1,2,...  (7.1)

with some numbers α > 0 and β > 0. Then

P ( sup_{1≤n<∞} |Z_{n}| > u + α) ≤ \frac{1}{β} P ( sup_{1≤n<∞} |Z_{n}− Z̄_{n}| > u) for all u > 0.

Proof of Lemma 7.1. Put τ = min{n: |Z_{n}| > u + α} if there exists such an index n, and τ = 0 otherwise. Then the event {τ = n} is independent of the sequence of random variables Z̄_{1},Z̄_{2},... for all n = 1,2,..., and because of this independence

P(\{τ = n\}) ≤ \frac{1}{β} P(\{τ = n\} ∩ \{|Z̄_{n}| ≤ α\}) ≤ \frac{1}{β} P(\{τ = n\} ∩ \{|Z_{n}− Z̄_{n}| > u\})

for all n = 1,2,.... Hence

P ( sup_{1≤n<∞} |Z_{n}| > u + α) = ∑_{l=1}^{∞} P(τ = l)

≤ \frac{1}{β} ∑_{l=1}^{∞} P(\{τ = l\} ∩ \{|Z_{l} − Z̄_{l}| > u\})

≤ \frac{1}{β} ∑_{l=1}^{∞} P(\{τ = l\} ∩ sup_{1≤n<∞} |Z_{n}− Z̄_{n}| > u\})

≤ \frac{1}{β} P ( sup_{1≤n<∞} |Z_{n}− Z̄_{n}| > u) .

Lemma 7.1 is proved.

We shall apply the following Lemma 7.2 which is a consequence of the Symmetrization Lemma 7.1.

Lemma 7.2. Let us fix a countable class of functions F on a measurable space (X,X ) together with a real number 0 < σ < 1. Consider a sequence of independent and identically distributed random variables ξ_{1},...,ξ_{n} with values in the space (X,X ) such that Ef(ξ_{1}) = 0, Ef^{2}(ξ_{1}) ≤ σ^{2} for all f ∈ F together with another sequence ε_{1},...,ε_{n} of independent random variables with distribution P(ε_{j} = 1) = P(ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, independent also of the random sequence ξ_{1},...,ξ_{n}. Then

P ( \frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} f(ξ_{j})| ≥ An^{1/2}σ^{2})

≤ 4P ( \frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2}) ifA ≥ \frac{3\sqrt{2}}{\sqrt{n}σ} . (7.2)

Proof of Lemma 7.2. Let us construct an independent copy ξ̄_{1},...,ξ̄_{n} of the sequence ξ_{1},...,ξ_{n} in such a way that all three sequences ξ_{1},...,ξ_{n}, ξ̄_{1},...,ξ̄_{n} and ε_{1},...,ε_{n} are independent. Define the random variables

S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}) and S̄_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ̄_{j})

for all f ∈ F. The inequality

P ( sup_{f∈F}|S_{n}(f)| > A\sqrt{n}σ^{2}) ≤ 2P ( sup_{f∈F}|S_{n}(f) − S̄_{n}(f)| > \frac{2}{3}A\sqrt{n}σ^{2}) . (7.3)

follows from Lemma 7.1 if it is applied for the countable set of random variables Z_{n}(f) = S_{n}(f) and Z̄_{n}(f) = S̄_{n}(f), f ∈ F, and the numbers u = \frac{2}{3}A\sqrt{n}σ^{2} and α = \frac{1}{3}A\sqrt{n}σ^{2}, since the random fields S_{n}(f) and S̄_{n}(f) are independent, and P(|S̄_{n}(f)| ≤ α) > \frac{1}{2} for all f ∈ F. Indeed, α = \frac{1}{3}A\sqrt{n}σ^{2} ≥ \sqrt{2}σ, ES̄_{n}(f)^{2} ≤ σ^{2}, thus Chebishev’s inequality implies that P(|S̄_{n}(f)| ≤ α) ≥ P(|S̄_{n}(f)| ≤ \sqrt{2}σ) ≥ \frac{1}{2} for all f ∈ F.

Let us observe that the random field

S_{n}(f) − S̄_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} (f(ξ_{j}) − f(ξ̄_{j})) , f ∈ F ,  (7.4)

and its randomized version

\frac{1}{\sqrt{n}} ∑_{j=1}^{n} ε_{j} (f(ξ_{j}) − f(ξ̄_{j})) , f ∈ F ,  (7.5)

have the same distribution. Indeed, even the conditional distribution of (7.5) under the condition that the values of the ε_{j}-s are prescribed agrees with the distribution of (7.4) for all possible values of the ε_{j}-s. This follows from the observation that the distribution of the random field (7.4) does not change if we exchange the random variables ξ_{j} and ξ̄_{j} for those indices j for which ε_{j} = −1 and do not change them for those indices j for which ε_{j} = 1. On the other hand, the distribution of the random field obtained with such an exchange of its variables agrees with the conditional distribution of the random field defined in (7.5) under the condition that the random variables ε_{j} take these prescribed values.

The above relation together with formula (7.3) imply that

P ( \frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} f(ξ_{j})| ≥ An^{1/2}σ^{2})

≤ 2P ( \frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j} [f(ξ_{j}) − f̄(ξ_{j})] | ≥ \frac{2}{3}An^{1/2}σ^{2})

≤ 2P ( \frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2})

+2P ( \frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ̄_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2})

= 4P ( \frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2}) .

Lemma 7.2 is proved.

First I try to explain briefly the method of proof of Proposition 6.2. A probability

of the form P ( n^{−1/2} sup_{f∈F} | ∑_{j=1}^{n} f(ξ_{j})| > u) has to be estimated. Lemma 7.2 enables

us to replace this problem by the estimation of the probability

P ( n^{−1/2} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| > \frac{u}{3})

with some independent random variables ε_{j}, P(ε_{j} = 1) = P(ε_{j} = −1) = \frac{1}{2}, j = 1,...,n, which are also independent of the random variables ξ_{j}. We shall bound the conditional probability of the event appearing in this modified problem under the condition that each random variable ξ_{j} takes a prescribed value. This can be done with the help of Hoeffding’s inequality formulated in Theorem 3.4 and the L_{2}-density property of the class of functions F we consider. We hope to get a sharp estimate in such a way which is similar to the result we got in the study of the Gaussian counterpart of this problem, because Hoeffding’s inequality yields always a Gaussian type upper bound for the tail distribution of the random sum we are studying.

Nevertheless, there appears a problem when we try to apply such an approach. To get a good estimate on the conditional tail distribution of the supremum of the random sums we are studying with the help of Hoeffding’s inequality we need a good estimate on the supremum of the conditional variances of the random sums we

are studying, i.e. on the tail distribution of sup_{f∈F} \frac{1}{n} ∑_{j=1}^{n} f^{2}(ξ_{j}). This problem is similar

to the original one, and it is not simpler.

But a more detailed study shows that our approach to get a good estimate with the help of Hoeffding’s inequality works. In comparing our original problem with the new, complementary problem we have to understand at which level we need a good estimate on the tail distribution of the supremum in the complementary problem to get a good tail distribution estimate at level u in the original problem. A detailed study shows that to bound the probability in the original problem with pa-

rameter u we have to estimate the probability P ( n^{−1/2} sup_{f∈F′} | ∑_{j=1}^{n} f(ξ_{j})| > u^{1+α} )

with some new nice, appropriately defined L_{2}-dense class of bounded functions F′ and some number α > 0. We shall exploit that the number u is replaced by a larger number u^{1+α} in the new problem. Let us also observe that if the sum of bounded random variables is considered, then for very large numbers u the probability we investigate equals zero. On the basis of these observations an appropriate backward induction procedure can be worked out. In its n-th step we give a good upper bound

on the probability P ( n^{−1/2} sup_{f∈F} | ∑_{j=1}^{n} f(ξ_{j})| > u) if u ≥ T_{n} with an appropriately

chosen number T_{n}, and try to diminish the number T_{n} in each step of this induction procedure. We can prove Proposition 6.2 as a consequence of the result we get by means of this backward induction procedure. To work out the details we introduce the following notion.

Definition of good tail behaviour for a class of normalized random sums. Let us have some measurable space (X,X ) and a probability measure μ on it together with some integer n ≥ 2 and real number σ > 0. Consider some class F of functions f(x) on the space (X,X ), and take a sequence of independent and μ distributed random variables ξ_{1},...,ξ_{n} with values in the space (X,X ). Define the normalized

random sums S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}), f ∈ F. Given some real number T > 0 we say

that the set of normalized random sums S_{n}(f), f ∈ F, has a good tail behaviour at level T (with parameters n and σ^{2} which will be fixed in the sequel) if the inequality

P ( sup_{f∈F}|S_{n}(f)| ≥ A\sqrt{n}σ^{2}) ≤ exp\{−A^{1/2}nσ^{2}\}  (7.6)

holds for all numbers A > T.

Now I formulate Proposition 7.3 and show that Proposition 6.2 follows from it.

Proposition 7.3. Let us fix a positive integer n ≥ 2, a real number 0 < σ ≤ 1 and a probability measure μ on a measurable space (X,X ) together with some numbers L ≥ 1 and D ≥ 1 such that nσ^{2} ≥ Llogn + logD. Let us consider those countable L_{2}-dense classes F of functions f = f(x) on the space (X,X ) with exponent L and parameter D for which all functions f ∈ F satisfy the conditions sup_{x∈X}|f(x)| ≤ \frac{1}{4},

∫ f(x)μ(dx) = 0 and ∫ f^{2}(x)μ(dx) ≤ σ^{2}.

Let a number T > 1 be such that for all classes of functions F which satisfy the

above conditions the set of normalized random sums S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}), f ∈ F,

defined with the help of a sequence of independent μ distributed random variables ξ_{1},...,ξ_{n} have a good tail behaviour at level T^{4/3}. There is a universal constant Ā_{0} such that if T ≥ Ā_{0}, then the set of the above defined normalized sums, S_{n}(f), f ∈ F, have a good tail behaviour for all such classes of functions F not only at level T^{4/3} but also at level T.

Proposition 6.2 simply follows from Proposition 7.3. To show this let us first observe that a class of normalized random sums S_{n}(f), f ∈ F, has a good tail behaviour at level T_{0} = \frac{1}{4σ^{2}} if this class of functions F satisfies the conditions of Proposition 7.3. Indeed, in this case

P ( sup_{f∈F}|S_{n}(f)| ≥ A\sqrt{n}σ^{2}) ≤ P ( sup_{f∈F}|S_{n}(f)| > \frac{\sqrt{n}}{4}) = 0

for all A > T_{0}. Then the repetitive application of Proposition 7.3 yields that a class of random sums S_{n}(f), f ∈ F, has a good tail behaviour at all levels T ≥ T_{0}^{(3/4)^{j}}

with an index j such that T_{0}^{(3/4)^{j}} ≥ Ā_{0}if the class of functions F satisfies the conditions of Proposition 7.3. Hence it has a good tail behaviour for T = Ā_{0}^{4/3}with the number Ā_{0}appearing in Proposition 7.3. If a class of functions f ∈ F satisfies the conditions of Proposition 6.2, then the class of functions F̄ = \{ f̄= \frac{f}{4} : f ∈ F \} satisfies the conditions of Proposition 7.3, with the same parameters σ, L and D. (Actually some of the inequalities that must hold for the elements of a class of functions F satisfying the conditions of Proposition 7.3 are valid with smaller parameters. But we did not change these parameters to satisfy also the condition nσ^{2} ≥ Llogn + logD.) Hence the class of functions S_{n}(f̄), f̄∈ F̄, has a good tail behaviour at level T = Ā_{0}^{4/3}. This implies that the original class of functions F satisfies formula (6.4) in Proposition 6.2, and this is what we had to show.

Proof of Proposition 7.3. Fix a class of functions F which satisfies the conditions of Proposition 7.3 together with two independent sequences ξ_{1},...,ξ_{n} and ε_{1},...,ε_{n} of independent random variables, where ξ_{j} is μ-distributed, P(ε_{j} = 1) = P(ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, and investigate the conditional probability

P(f,A|ξ_{1},...,ξ_{n}) = P ( \frac{1}{\sqrt{n}} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{6}\sqrt{n}σ^{2}| ξ_{1},...,ξ_{n})

for all functions f ∈ F, A > T and values (ξ_{1},...,ξ_{n}) in the condition. By the Hoeffding inequality formulated in Theorem 3.4

P(f,A|ξ_{1},...,ξ_{n}) ≤ 2exp\{ −\frac{\frac{1}{36}A^{2}nσ^{4}}{2S̄^{2}(f,ξ_{1},...,ξ_{n})} \}  (7.7)

with

S̄^{2}(f,x_{1},...,x_{n}) = \frac{1}{n} ∑_{j=1}^{n} f^{2}(x_{j}), f ∈ F .

Let us introduce the set

H = H(A) = { (x_{1},...,x_{n}): sup_{f∈F} S̄^{2}(f,x_{1},...,x_{n}) ≥ ( 1 + A^{4/3}) σ^{2}} .  (7.8)

I claim that

P((ξ_{1},...,ξ_{n}) ∈ H) ≤ e^{−A^{2/3}nσ^{2}} if A > T.  (7.9)

(The set H is the small exceptional set of those points (x_{1},...,x_{n}) for which we cannot give a good estimate for P(f,A|ξ_{1}(ω),...,ξ_{n}(ω)) with ξ_{1}(ω) = x_{1},…, ξ_{n}(ω) = x_{n} for some f ∈ F.)

To prove relation (7.9) let us consider the functions f̄= f̄(f), f̄(x) = f^{2}(x) − ∫ f^{2}(x)μ(dx), and introduce the class of functions F̄ = {f̄(f): f ∈ F }. Let us show that the class of functions F̄ satisfies the conditions of Proposition 7.3, hence the estimate (7.6) holds for the class of functions F̄ if A > T^{4/3}.

The relation ∫ f̄(x)μ(dx) = 0 clearly holds. The condition sup|f̄(x)| ≤ \frac{1}{8} < \frac{1}{4} also holds if sup|f(x)| ≤ \frac{1}{4}, and ∫ f̄^{2}(x)μ(dx) ≤ ∫ f^{4}(x)μ(dx) ≤ \frac{1}{16} ∫ f^{2}(x)μ(dx) ≤ \frac{σ^{2}}{16} < σ^{2} if f ∈ F. It remained to show that F̄ is an L_{2}-dense class with exponent L and parameter D. For this goal we need a good estimate on ∫(f̄(x) − ḡ(x))^{2}ρ(dx), where f̄, ḡ∈ F̄, and ρ is an arbitrary probability measure.

Observe that

∫ (f̄(x) − ḡ(x))^{2}ρ(dx)
≤ 2∫ (f^{2}(x) − g^{2}(x))^{2}ρ(dx) + 2∫ (f^{2}(x) − g^{2}(x))^{2}μ(dx)
≤ 2(sup(|f(x)| + |g(x)|)^{2}( ∫ (f(x) − g(x))^{2}(ρ(dx) + μ(dx))
≤ ∫ (f(x) − g(x))^{2}ρ̄(dx)

for all f,g ∈ F, f̄= f̄(f), ḡ= ḡ(g) and probability measure ρ, where ρ̄ = \frac{ρ+μ}{2} . This means that if {f_{1},...,f_{m}} is an ε-dense subset of F in the space L_{2}(X,X ,ρ̄), then {f̄_{1},...,f̄_{m}} is an ε-dense subset of F̄ in the space L_{2}(X,X ,ρ), and not only F, but also F̄ is an L_{2}-dense class with exponent L and parameter D.

Because of the conditions of Proposition 7.3 we can write for the number A^{4/3} > T^{4/3} and the class of functions F̄ that

P((ξ_{1},...,ξ_{n}) ∈ H)

= P ( sup_{f∈F}( \frac{1}{n} ∑_{j=1}^{n} f̄(f)(ξ_{j}) + \frac{1}{n} ∑_{j=1}^{n} Ef^{2}(ξ_{j})) ≥ ( 1 + A^{4/3}) σ^{2})

≤ P ( sup_{f̄∈F̄} \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f̄(ξ_{j}) ≥ A^{4/3}n^{1/2}σ^{2}) ≤ e^{−A^{2/3}nσ^{2}} ,

i.e. relation (7.9) holds.

By formula (7.7) and the definition of the set H given in (7.8) the estimate

P(f,A|ξ_{1},...,ξ_{n}) ≤ 2e^{−A^{2/3}nσ^{2}/144} if (ξ_{1},...,ξ_{n}) ∉ H  (7.10)

holds for all f ∈ F and A > T ≥ 1. (Here we used the estimate 1 + A^{4/3} ≤ 2A^{4/3}.) Let us introduce the conditional probability

P(F ,A|ξ_{1},...,ξ_{n}) = P ( sup_{f∈F} \frac{1}{\sqrt{n}} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}\sqrt{n}σ^{2}| ξ_{1},...,ξ_{n})

for all (ξ_{1},...,ξ_{n}) and A > T. We shall estimate this conditional probability with the help of relation (7.10) if (ξ_{1},...,ξ_{n}) ∉ H.

Given a vector x^{(n)} = (x_{1},...,x_{n}) ∈ X^{n}, let us introduce the probability measure

ν = ν(x_{1},...,x_{n}) = ν(x^{(n)}) on (X,X )

which is concentrated in the coordinates of the vector x^{(n)} = (x_{1},...,x_{n}), and ν(\{x_{j}\}) = \frac{1}{n} for all points x_{j}, j = 1,...,n. If ∫ f^{2}(u)ν(du) ≤ δ^{2} for a function

f, then | \frac{1}{\sqrt{n}} ∑_{j=1}^{n} ε_{j}f(x_{j})| ≤ n^{1/2}∫ |f(u)|ν(du) ≤ n^{1/2}δ. As a consequence, we can

write that

| \frac{1}{\sqrt{n}} ∑_{j=1}^{n} ε_{j}f(x_{j}) − \frac{1}{\sqrt{n}} ∑_{j=1}^{n} ε_{j}g(x_{j})| ≤ \frac{A}{6}\sqrt{n}σ^{2}

if ∫ (f(u) − g(u))^{2}dν(u) ≤ ( \frac{Aσ^{2}}{6}) ^{2}.  (7.11)

Remark. We may assume in our proof that the distribution of the random variables ξ_{j}, 1 ≤ j ≤ n, are non-atomic, and as a consequence we can restrict our attention to such measures ν(x^{(n)}) for which all coordinates of the vector x^{(n)} are different. Otherwise we can define independent and uniformly distributed random variables on the interval [0,1], η_{1},...,η_{n}, which are also independent of the random variables ξ_{j}, 1 ≤ j ≤ n. With the help of these random variables η_{j} we can introduce the random variables ξ̃_{j} = (ξ_{j},η_{j}), 1 ≤ j ≤ n, and the class of functions F̃ on the space X × [0,1] consisting of functions f̃(x,y) = f(x), f ∈ F, with x ∈ X and 0 ≤ y ≤ 1. It is not difficult to see that the random variables ξ̃_{j} and the class of functions F̃ satisfy the conditions of Proposition 7.3, and the distribution of the random variables ξ̃_{j} is non-atomic. Hence we can apply Proposition 7.3 with such a choice, and this provides the statement of Proposition 7.3 in the original case, too.

Let us list the elements of the (countable) set F as F = {f_{1},f_{2},...}, fix the number δ = \frac{Aσ^{2}}{6}, and choose for all vectors x^{(n)} = (x_{1},...,x_{n}) ∈ X^{n} a sequence of indices p_{1}(x^{(n)}),...,p_{m}(x^{(n)}) taking positive integer values with m = max(1,Dδ^{−L}) = max(1,D(\frac{6}{Aσ^{2}})^{L}) elements in such a way that inf_{1≤l≤m} ∫(f(u) −

f_{p_{l}(x^{(n)})}(u))^{2}dν(x^{(n)})(u) ≤ δ^{2} for all f ∈ F and x^{(n)} ∈ X^{n} with the above defined measure ν(x^{(n)}) on the space (X,X ). This is possible because of the L_{2}-dense property of the class of functions F. (This is the point where the L_{2}-dense property of the class of functions F is exploited in its full strength.) In a complete proof of Proposition 7.3 we still have to show that we can choose the indices p_{j}(x^{(n)}), 1 ≤ j ≤ m, as measurable functions of their argument x^{(n)} on the space (X^{n},X^{n}). We shall show this in Lemma 7.4 at the end of the proof.

Put ξ^{(n)}(ω) = (ξ_{1}(ω),...,ξ_{n}(ω)). Because of relation (7.11), the choice of the number δ and the property of the functions f_{p_{l}(x^{(n)})}(·) we have

\{ ω: sup_{f∈F} \frac{1}{\sqrt{n}} |∑_{j=1}^{n} ε_{j}(ω)f(ξ_{j}(ω))| ≥ \frac{A}{3}\sqrt{n}σ^{2}\}  (7.12)

⊂ ⋃_{l=1}^{m} \{ ω: \frac{1}{\sqrt{n}} |∑_{j=1}^{n} ε_{j}(ω)f_{p_{l}(ξ^{(n)}(ω))}(ξ_{j}(ω))| ≥ \frac{A}{6}\sqrt{n}σ^{2}\} .

We can estimate the conditional probability at the right-hand side of (7.12) under the condition that the vector (ξ_{1}(ω),...,ξ_{n}(ω)) takes such a prescribed value for which (ξ_{1}(ω),...,ξ_{n}(ω)) ∈ H. We get with the help of (7.12), inequality (7.10) and the definition of the quantity P(f,A|ξ_{1},...,ξ_{n}) before formula (7.7) that

P(F ,A|ξ_{1},...,ξ_{n}) ≤ ∑_{l=1}^{m} P(f_{p_{l}(ξ^{(n)})},A|ξ_{1},...,ξ_{n})

≤ 2max( 1,D( \frac{6}{Aσ^{2}}) ^{L}) e^{−A^{2/3}nσ^{2}/144}

if (ξ_{1},...,ξ_{n}) ∉ H and A > T.  (7.13)

If A ≥ Ā_{0} with a sufficiently large constant Ā_{0}, then this inequality together with Lemma 7.2 and the estimate (7.9) imply that

P ( \frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} f(ξ_{j})| ≥ An^{1/2}σ^{2})

≤ 4P ( \frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2})  (7.14)

≤ max( 4,8D( \frac{6}{Aσ^{2}}) ^{L}) e^{−A^{2/3}nσ^{2}/144} + 4e^{−A^{2/3}nσ^{2}} if A > T.

(We may apply Lemma 7.2 if A ≥ A_{0} with a sufficiently large A_{0}, since nσ^{2} ≥ Llogn+logD ≥ log2, hence \sqrt{n}σ ≥ \sqrt{log2}, and the condition A ≥ \frac{3\sqrt{2}}{\sqrt{n}σ} demanded in relation (7.2) is satisfied.)

By the conditions of Proposition 7.3 the inequalities nσ^{2} ≥ Llogn + logD hold with some L ≥ 1, D ≥ 1 and n ≥ 2. This implies that nσ^{2} ≥ Llog2 ≥ \frac{1}{2}, (\frac{6}{Aσ^{2}})^{L} ≤

(\frac{n}{2nσ^{2}} )^{L}≤ n^{L}= e^{Llogn}≤ e^{nσ^{2}} if A ≥ Ā_{0}with some sufficiently large constant Ā_{0}> 0,

and 2D = e^{log2+logD} ≤ e^{3nσ^{2}}. Hence the first term at the right-hand side of (7.14) can be bounded by

max( 4,8D( \frac{6}{Aσ^{2}}) ^{L}) e^{−A^{2/3}nσ^{2}/144} ≤ e^{−A^{2/3}nσ^{2}/144} · 4e^{4nσ^{2}} ≤ \frac{1}{2}e^{−A^{1/2}nσ^{2}}

if A ≥ Ā_{0} with a sufficiently large Ā_{0}. The second term at the right-hand side of (7.14) can also be bounded as 4e^{−A^{2/3}nσ^{2}} ≤ \frac{1}{2}e^{−A^{1/2}nσ^{2}} with an appropriate choice of the number Ā_{0}.

By the above calculation formula (7.14) yields the inequality

P ( \frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} f(ξ_{j})| ≥ An^{1/2}σ^{2}) ≤ e^{−A^{1/2}nσ^{2}}

if A > T, and the constant Ā_{0}is chosen sufficiently large.

To complete the proof of Proposition 7.3 we still show in the following Lemma 7.4 that the functions p_{l}(x^{(n)}), 1 ≤ l ≤ m, we have introduced in the above argument can be chosen as measurable functions in the space (X^{n},X^{n}). This implies that the expressions f_{p_{l}(ξ^{(n)}(ω))}(ξ_{j}(ω)) in formula (7.12) are F (ξ_{1},...,ξ_{n}) measurable random variables. Hence the formulation of (7.13) is legitime, no measurability problem arises. We shall present Lemma 7.4 together with some generalizations in Lemma 7.4A and Lemma 7.4B that we shall apply later in the proof of Propositions 15.3 and 15.4 which are multivariate versions of Proposition 7.3. We shall need these results in the proof of the multivariate version of Proposition 6.2. We have formulated them not in their most general possible form, but in the form as we shall need them.

Lemma 7.4. Let F = {f_{1},f_{2},...} be a countable and L_{2}-dense class of functions with some exponent L > 0 and parameter D ≥ 1 on a measurable space (X,X ). Fix some positive integer n, and define for all x^{(n)} = (x_{1},...,x_{n}) ∈ X^{n} the probability measure ν(x^{(n)}) = ν(x_{1},...,x_{n}) on the space (X,X ) by the formula ν(x^{(n)})(x_{j}) = \frac{1}{n}, 1 ≤ j ≤ n. For a number 0 ≤ ε ≤ 1 put m = m(ε) = [Dε^{−L}], where [·] denotes integer part. For all 0 ≤ ε ≤ 1 there exists m = m(ε) measurable functions p_{l}(x^{(n)}), 1 ≤ l ≤ m, on the measurable space (X^{n},X^{n}) with positive integer values in such a

way that inf_{1≤l≤m} ∫(f(u)−f_{p_{l}(x^{(n)})}(u))^{2}ν(x^{(n)})(du) ≤ ε^{2} for all x^{(n)} ∈ X^{n} and f ∈ F.

In the proof of Proposition 15.3 we need the following result.

Lemma 7.4A. Let F = {f_{1},f_{2},...} be a countable and L_{2}-dense class of functions with some exponent L > 0 and parameter D ≥ 1 on the k-fold product (X^{k},X^{k}) of a measurable space (X,X ) with some k ≥ 1. Fix some positive integer n, and define for all vectors x^{(n)} = (x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) ∈ X^{kn}, where x_{l}^{(j)} ∈ X for all j and l the probability measure ρ(x^{(n)}) in the space (X^{k},X^{k}) by the formula ρ(x^{(n)})(x_{l_{j}}^{(j)}, 1 ≤ j ≤ k, 1 ≤ l_{j} ≤ n) = \frac{1}{n^{k}} for all sequences (x_{l_{1}}^{(1)},...,x_{l_{k}}^{(k)}) , 1 ≤ j ≤ k,

1 ≤ l_{j} ≤ n, with coordinates of the vector x^{(n)} = (x_{l}^{(j)},1 ≤ l ≤ n, 1 ≤ j ≤ k). For all 0 ≤ ε ≤ 1 there exist m = m(ε) = [Dε^{−L}] measurable functions p_{r}(x^{(n)}), 1 ≤ r ≤ m, on the measurable space (X^{kn},X^{kn}) with positive integer values in such a way that

inf_{1≤r≤m} ∫(f(u) − f_{p_{r}(x^{(n)})}(u))^{2}ρ(x^{(n)})(du) ≤ ε^{2} for all x^{(n)} ∈ X^{kn}and f ∈ F.

In the proof of Proposition 15.4 the following result will be needed.

Lemma 7.4B. Let F = {f_{1},f_{2},...} be a countable and L_{2}-dense class of functions with some exponent L > 0 and parameter D ≥ 1 on the product space (X^{k}×Y,X^{k}×Y ) with some measurable spaces (X,X ) and (Y,Y ) and integer k ≥ 1. Fix some positive integer n, and define for all vectors x^{(n)} = (x_{l}^{(j,1)},x_{l}^{(j,−1)} , 1 ≤ l ≤ n, 1 ≤ j ≤ k) ∈ X^{2kn}, where x_{l}^{(j,±1)} ∈ X for all j and l a probability measure α(x^{(n)}) in the space (X^{k}× Y,X^{k}× Y ) in the following way. Fix some probability measure ρ in the space (Y,Y ) and two ±1 sequences ε_{1}^{(k)} = (ε_{1,1},...,ε_{k,1}) and ε_{2}^{(k)} = (ε_{1,2},...,ε_{k,2}) of length k. Define with their help first the following probability measures α_{1}(x^{(n)}) = α_{1}(x^{(n)},ε_{1}^{(k)},ε_{2}^{(k)},ρ) and α_{2}(x^{(n)}) = α_{2}(x^{(n)},ε_{1}^{(k)},ε_{2}^{(k)},ρ)

in the space (X^{k}× Y,X^{k}× Y ) for all x^{(n)} ∈ X^{2kn}. Let α_{1}(x^{(n)})({x_{l_{1}}^{(1,ε_{1,1})}} × ··· ×

\{x_{l_{k}}^{(k,ε_{k,1})}\} × B) = \frac{ρ(B)}{n^{k}} and α_{2}(x^{(n)})(\{x_{l_{1}}^{(1,ε_{1,2})}\} × ···× \{x_{l_{k}}^{(k,ε_{k,2})}\} × B) = \frac{ρ(B)}{n^{k}} with 1 ≤ l_{j} ≤ n for all 1 ≤ j ≤ k and B ∈ Y if x_{l_{j}}^{(j,ε_{j,1})} and x_{l_{j}}^{(j,ε_{j,2})} are the appropriate co-

ordinates of the vector x^{(n)} ∈ X^{2kn}. Put α(x^{(n)}) = \frac{α_{1}(x^{(n)})+α_{2}(x^{(n)})}{2} . For all 0 ≤ ε ≤ 1 there exist m = m(ε) = [Dε^{−L}] measurable functions p_{r}(x^{(n)}), 1 ≤ r ≤ m, on the measurable space (X^{2kn},X^{2kn}) with positive integer values in such a way that

inf_{1≤r≤m} ∫(f(u) − f_{p_{r}(x^{(n)})}(u))^{2}α(x^{(n)})(du) ≤ ε^{2} for all x^{(n)} ∈ X^{2kn}and f ∈ F.

Proof of Lemma 7.4. Fix some 0 < ε ≤ 1, put the number m = m(ε) introduced in the lemma, and let us list the set of all vectors (j_{1},...,j_{m}) of length m with positive integer coordinates in some way. Define for all of these vectors (j_{1},...,j_{m}) the set B(j_{1},...,j_{m}) ⊂ X^{n} in the following way. The relation x^{(n)} = (x_{1},...,x_{n}) ∈ B(j_{1},...,j_{m}) holds if and only if inf_{1≤r≤m} ∫(f(u) − f_{j_{r}} (u))^{2}dν(x^{(n)})(u) ≤ ε^{2} for all

f ∈ F. Then all sets B(j_{1},...,j_{m}) are measurable, and ⋃_{(j_{1},...,j_{m})} B(j_{1},...,j_{m}) =

X^{n} because F is an L_{2}-dense class of functions with exponent L and parameter D. Given a point x^{(n)} = (x_{1},...,x_{n}) let us choose the first vector (j_{1},...,j_{m}) = (j_{1}(x^{(n)}),...,j_{m}(x^{(n)})) in our list of vectors for which x^{(n)} ∈ B(j_{1},...,j_{m}), and define p_{l}(x^{(n)}) = j_{l}(x^{(n)}) for all 1 ≤ l ≤ m with this vector (j_{1},...,j_{m}). Then the functions p_{l}(x^{(n)}) are measurable, and the functions f_{p_{l}(x^{(n)})}, 1 ≤ l ≤ m, defined with their help together with the probability measures ν(x^{(n)}) satisfy the inequality demanded in Lemma 7.4.

The proof of Lemmas 7.4A and 7.4B is almost the same. We only have to modify the definition of the sets B(j_{1},...,j_{m}) in a natural way. The space of arguments x^{(n)} are the spaces X^{kn} and X^{2kn} in these lemmas, and we have to integrate with respect to the measures ρ(x^{(n)}) in the space X^{k} and with respect to the measures α(x^{(n)}) in the space X^{k}× Y respectively. The sets B(j_{1},...,j_{m}) are measurable also in these cases, and the rest of the proof can be applied without any change.

Formulation of the main results of this work

Former chapters of this work contain estimates about the tail distribution of normalized sums of independent, identically distributed random variables and of the supremum of appropriate classes of such random sums. They were considered together with some estimates about the tail distribution of the integral of a (deterministic) function with respect to a normalized empirical distribution and of the supremum of such integrals. This two kinds of problems are closely related, and to understand them better it is useful to investigate them together with their natural Gaussian counterpart.

In this chapter I formulate the natural multivariate versions of these results. They will be proved in the subsequent chapters. To formulate them we have to introduce some new notions. I shall also discuss some new problems whose solutions help in their proof. I finish this chapter with a short overview about the content of the remaining part of this work.

I start this chapter with the formulation of two results, Theorems 8.1 and 8.2 together with some simple consequences. They yield a sharp estimate about the tail distribution of a multiple random integral with respect to a normalized empirical distribution and about the analogous problem when the tail distribution of the supremum of such integrals is considered. These results are the natural versions of the corresponding one-variate results about the tail behaviour of an integral or of the supremum of a class of integrals with respect to a normalized empirical distribution. They can be formulated with the help of the notions introduced before, in particular with the help of the notion of multiple random integrals with respect to a normalized empirical distribution introduced in formula (4.8).

To formulate the following two results, Theorems 8.3 and 8.4 and their consequences, which are the natural multivariate versions of the results about the tail distribution of partial sums of independent random variables, and of the supremum of such sums we have to make some preparations. First we introduce the so-called U-statistics which can be considered the natural multivariate generalizations of the sum of independent and identically distributed random variables. Beside this, observe that in the one-variate case we had a good estimation about the tail distribution of sums of independent random variables only if the summands had expectation zero. We have to find the natural multivariate version of this property. Hence we define the so-called degenerate U-statistics which can be considered as the natural multivariate counterparts of sums of independent and identically distributed random variables with zero expectation. Theorems 8.3 and 8.4 contain estimates about the tail-distribution of degenerate U-statistics and of the supremum of such expressions.

In Theorems 8.5 and 8.6 I formulate the Gaussian counterparts of the above results. They deal with multiple Wiener-Itô integrals with respect to a so-called white noise. The notion of white noise and multiple Wiener–Itô integrals with respect to it and their properties needed to have a good understanding of these results will be explained in Chapter 10. Still two results are discussed in this chapter. They are Examples 8.7 and 8.8, which state that the estimates of Theorems 8.5 and 8.3 are in a certain sense sharp.

To formulate the first two results of this chapter let us consider a sequence of independent and identically distributed random variables ξ_{1},...,ξ_{n} with values in a measurable space (X,X ). Let μ denote the distribution of the random variables ξ_{j}, and introduce the empirical distribution of the sequence ξ_{1},...,ξ_{n} defined in (4.5). Given a measurable function f(x_{1},...,x_{k}) on the k-fold product space (X^{k},X^{k}) consider its integral J_{n,k}(f) with respect to the k-fold product of the normalized empirical distribution \sqrt{n}(μ_{n}− μ) defined in formula (4.8). In the definition of this integral the diagonals x_{j} = x_{l}, 1 ≤ j < l ≤ k, were omitted from the domain of integration. The following Theorem 8.1 can be considered as the multiple integral version of Bernstein’s inequality formulated in Theorem 3.1.

Theorem 8.1 (Estimate on the tail distribution of a multiple random integral with respect to a normalized empirical distribution). Let us take a measurable function f(x_{1},...,x_{k}) on the k-fold product (X^{k},X^{k}) of a measurable space (X,X ) with some k ≥ 1 together with a non-atomic probability measure μ on (X,X ) and a sequence of independent and identically distributed random variables ξ_{1},...,ξ_{n} with distribution μ on (X,X ). Let the function f satisfy the conditions

∥f∥_{∞} = sup_{x_{j}∈X, 1≤j≤k} |f(x_{1},...,x_{k})| ≤ 1,  (8.1)

and

∥f∥_{2}^{2} = ∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) ≤ σ^{2}  (8.2)

with some constant 0 < σ ≤ 1. There exist some constants C = C_{k}> 0 and α = α_{k}> 0 such that the random integral J_{n,k}(f) defined in formulas (4.5) and (4.8) satisfies the inequality

P(|k!J_{n,k}(f)| > u) ≤ C max( e^{−α(u/σ)^{2/k}} ,e^{−α(nu^{2})^{1/(k+1)}} )  (8.3)

for all u > 0. The constants C = C_{k} > 0 and α = α_{k} > 0 in formula (8.3) depend only on the parameter k.

Theorem 8.1 can be reformulated in the following equivalent form.

Theorem 8.1′. Under the conditions of Theorem 8.1

P(|k!J_{n,k}(f)| > u) ≤ Ce^{−α(u/σ)^{2/k}} for all 0 < u ≤ n^{k/2}σ^{k+1}  (8.4)

with a number σ, 0 ≤ σ ≤ 1, satisfying relation in (8.2) and some universal constants C = C_{k} > 0, α = α_{k} > 0, depending only on the multiplicity k of the integral J_{n,k}(f).

Theorem 8.1 clearly implies Theorem 8.1′, since in the case u ≤ n^{k/2}σ^{k+1} the first term is larger than the second one in the maximum at the right-hand side of formula (8.3). On the other hand, Theorem 8.1′ implies Theorem 8.1 also if u > n^{k/2}σ^{k+1}. Indeed, in this case Theorem 8.1′ can be applied with σ̄ = (un^{−k/2})^{1/(k+1)} ≥ σ if u ≤ n^{k/2}, since the condition 0 < σ̄ ≤ 1 is satisfied. This

yields that P (|k!J_{n,k}(f)| > u) ≤ C exp\{−α ( \frac{u}{σ̄})^{2/k}\} = C exp\{−α(nu^{2})^{1/(k+1)}\} if

n^{k/2} ≥ u > n^{k/2}σ^{k+1}, and relation (8.3) holds in this case. If u > 2^{k}n^{k/2}, then P(k!|J_{n,k}(f)| > u) = 0, and if n^{k/2} ≤ u < 2^{k}n^{k/2}, then

P(|k!J_{n,k}(f)| > u) ≤ P(|k!J_{n,k}(f)| > n^{k/2})
≤ C exp{−α((n · n^{k/2})^{2})^{1/(k+1)}} ≤ C exp{−2^{−k} α(nu^{2})^{1/(k+1)}} .

Hence relation (8.3) holds (with a possibly different parameter α) in these cases, too.

Theorem 8.1 or Theorem 8.1′ state that the tail distribution P(k!|J_{n,k}(f)| > u) of the k-fold random integral k!J_{n,k}(f) can be bounded similarly to the probability P(|const.ση^{k}| > u), where η is a random variable with standard normal distribution, and the number 0 ≤ σ ≤ 1 satisfies relation (8.2), provided that the level u we consider is less than n^{k/2}σ^{k+1}. As we shall see later (see Corollary 1 of Theorem 9.4), the value of the number σ^{2} in formula (8.2) is closely related to the variance of k!J_{n,k}(f). At the end of this chapter an example is given which shows that the condition u ≤ n^{k/2}σ^{k+1} is really needed in Theorem 8.1′.

The next result, Theorem 8.2, is the generalization of Theorem 4.1′for multiple random integrals with respect to a normalized empirical measure. In its formulation the notions of L_{2}-dense classes and countably approximability introduced in Chapter 4 are applied.

Theorem 8.2 (Estimate on the supremum of multiple random integrals with respect to an empirical distribution). Let us have a non-atomic probability measure μ on a measurable space (X,X ) together with a countable and L_{2}-dense class F of functions f = f(x_{1},...,x_{k}) of k variables with some parameter D ≥ 2 and exponent L ≥ 1 on the product space (X^{k},X^{k}) which satisfies the conditions

∥f∥_{∞} = sup_{x_{j}∈X, 1≤j≤k} |f(x_{1},...,x_{k})| ≤ 1,  for all f ∈ F  (8.5)

and

∥f∥_{2}^{2} = Ef^{2}(ξ_{1},...,ξ_{k}) = ∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) ≤ σ^{2}
for all f ∈ F  (8.6)

with some constant 0 < σ ≤ 1. There exist some constants C = C(k) > 0, α = α(k) > 0 and M = M(k) > 0 depending only on the parameter k such that the supremum of the random integrals k!J_{n,k}(f), f ∈ F, defined by formula (4.8) satisfies the inequality

P ( sup_{f∈F}|k!J_{n,k}(f)| ≥ u) ≤ C exp\{ −α ( \frac{u}{σ})^{2/k}\} for those numbers u

for whichnσ^{2} ≥ ( \frac{u}{σ})^{2/k} ≥ M(L^{3/2}log \frac{2}{σ} + (logD)^{3/2}),  (8.7)

where the numbers D and L agree with the parameter and exponent of the L_{2}-dense class F.

The condition about the countable cardinality of the class F can be replaced by the weaker condition that the class of random variables k!J_{n,k}(f), f ∈ F, is countably approximable.

The condition given for the number u in formula (8.7) appears in Theorem 8.2 for a similar reason as the analogous condition formulated in (4.4) in its one-variate counterpart, Theorem 4.1. The lower bound is needed, since we have a good estimate in formula (8.7) only for u ≥ E sup_{f∈F}|k!J_{n,k}(f)|. The upper bound appears,

since we have a good estimate in Theorem 8.1′ only for 0 < u ≤ n^{k/2}σ^{k+1}. If a pair of numbers (u,σ) does not satisfy condition (8.7), then we may try to get an estimate by increasing the number σ or decreasing the number u.

To formulate such a version of Theorems 8.1 and 8.2 which corresponds to the results about sums of independent random variables in the case k = 1 the following notions will be introduced.

Definition of U-statistics. Let us consider a function f = f(x_{1},...,x_{k}) on the k-th power (X^{k},X^{k}) of a space (X,X ) together with a sequence of independent and identically distributed random variables ξ_{1},...,ξ_{n}, n ≥ k, which take their values in this space (X,X ). The expression

I_{n,k}(f) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k, \\ l_{j} ≠l_{j′} ifj≠j′ \end{matrix}} f (ξ_{l_{1}},...,ξ_{l_{k}})  (8.8)

is called a U-statistic of order k with the sequence ξ_{1},...,ξ_{n}, and kernel function f.

Remark. In later calculations sometimes we shall work with U-statistics with kernel functions of the form f(x_{u_{1}},...,x_{u_{k}}) instead of f(x_{1},...,x_{k}), where {u_{1},...,u_{k}} is an arbitrary set with different elements. The U-statistic with such a kernel function will also be defined, and it equals the U-statistic with the original kernel function f defined in (8.8), i.e.

I_{n,k}(f(x_{u_{1}},...,x_{u_{k}})) = I_{n,k}(f(x_{1},...,x_{k})).  (8.9)

(Observe that if we define the function f_{π}(x_{1},...,x_{k}) = f(x_{π(1)},...,x_{π(k)}) for all permutations π of the set {1,...,k}, then I_{n,k}(f_{π}) = I_{n,k}(f), hence the above definition is legitimate.) Such a definition is natural, and it simplifies the notation in some calculations. A similar convention will be introduced about Wiener–Itô integrals in Chapter 10.

Some special U-statistics, called degenerate U-statistics, will also be introduced. They can be considered as the natural multivariate version of sums of identically distributed random variables with expectation zero. Degenerate U-statistics will be defined together with canonical kernel functions, because these two notions are closely related. For the sake of simpler notation in later discussions we shall allow general indexation of the variables in the definition of canonical functions, and we shall consider functions of the form f(x_{l_{1}},...,x_{l_{k}}) instead of f(x_{1},...,x_{k}).

Definition of degenerate U-statistics. A U-statistic I_{n,k}(f) of order k with a sequence of independent and identically distributed random variables ξ_{1},...,ξ_{n} is called degenerate if its kernel function f(x_{1},...,x_{k}) satisfies the relation

E(f(ξ_{1},...,ξ_{k})|ξ_{1} = x_{1},...,ξ_{j−1} = x_{j−1} ,ξ_{j+1} = x_{j+1} ,...,ξ_{k} = x_{k}) = 0
for all 1 ≤ j ≤ k and x_{s} ∈ X, s ≠ j.

Definition of a canonical function. A function f(x_{l_{1}},...,x_{l_{k}}) taking values in the k-fold product of a measurable space (X,X ) is called a canonical function with respect to a probability measure μ on (X,X ) if

∫ f(x_{l_{1}},...,x_{l_{j−1}} ,u,x_{l_{j+1}} ,...,x_{l_{k}})μ(du) = 0
for all 1 ≤ j ≤ k and x_{l_{s}} ∈ X, s ≠ j.  (8.10)

For the sake of more convenient notations in the subsequent part of this work we shall also speak of U-statistics of order zero. We shall write I_{n,0}(c) = c for any constant c, and I_{n,0}(c) will be called a degenerate U-statistic of order zero. A constant will be considered as a canonical function with zero arguments.

It is clear that a U-statistic I_{n,k}(f) with kernel function f and independent μdistributed random variables ξ_{1},...,ξ_{n} is degenerate if and only if its kernel function is canonical with respect to the probability measure μ. Let us also observe that

I_{n,k}(f) = I_{n,k}(Symf)  (8.11)

for all functions of k variables.

The next two results, Theorems 8.3 and 8.4, deal with degenerate U-statistics. Theorem 8.3 is the U-statistic version of Theorem 8.1, and Theorem 8.4 is the Ustatistic version of Theorem 8.2. Actually Theorem 8.3 yields a sharper estimate than Theorems 8.1, because it contains more explicit and better universal constants. I shall return to this point later.

Theorem 8.3 (Estimate on the tail distribution of a degenerate U-statistic). Let us have a measurable function f(x_{1},...,x_{k}) on the k-fold product (X^{k},X^{k}), k ≥ 1, of a measurable space (X,X ) together with a probability measure μ on (X,X ) and a sequence of independent and identically distributed random variables ξ_{1},...,ξ_{n}, n ≥ k, with distribution μ on (X,X ). Let us consider the U-statistic I_{n,k}(f) of order k with this sequence of random variables ξ_{1},...,ξ_{n}. Assume that this U-statistic is degenerate, i.e. its kernel function f(x_{1},...,x_{k}) is canonical with respect to the measure μ. Let us also assume that the function f satisfies conditions (8.1) and (8.2) with some number 0 < σ ≤ 1. Then there exist some constants A = A(k) > 0 and B = B(k) > 0 depending only on the order k of the U-statistic I_{n,k}(f) such that

P(n^{−k/2}|k!I_{n,k}(f)| > u) ≤ Aexp\{ −\frac{u^{2/k}}{2σ^{2/k}( 1 + B(un^{−k/2}σ^{−(k+1)} )^{1/k})} \} (8.12)

for all 0 ≤ u ≤ n^{k/2}σ^{k+1}.

Let us also formulate the following simple corollary of Theorem 8.3.

Corollary of Theorem 8.3. Under the conditions of Theorem 8.3 there exist some universal constants C = C(k) > 0 and α = α(k) > 0 that

P(n^{−k/2}|k!I_{n,k}(f)| > u) ≤ C exp\{ −α ( \frac{u}{σ})^{2/k}\} for all 0 ≤ u ≤ n^{k/2}σ^{k+1}.

(8.13)

The following estimate holds about the supremum of degenerate U-statistics.

Theorem 8.4 (Estimate on the supremum of degenerate U-statistics). Let us have a probability measure μ on a measurable space (X,X ) together with a countable and L_{2}-dense class F of functions f = f(x_{1},...,x_{k}) of k variables with some parameter D ≥ 2 and exponent L ≥ 1 on the product space (X^{k},X^{k}) which satisfies conditions (8.5) and (8.6) with some constant 0 < σ ≤ 1. Let us take a sequence of independent μ distributed random variables ξ_{1},...,ξ_{n}, n ≥ k, and consider the U-statistics I_{n,k}(f) with these random variables and kernel functions f ∈ F. Let us assume that all these U-statistics I_{n,k}(f), f ∈ F, are degenerate, or in an equivalent form, all functions f ∈ F are canonical with respect to the measure μ. Then there exist some constants C = C(k) > 0, α = α(k) > 0 and M = M(k) > 0 depending only on the parameter k such that the inequality

P ( sup_{f∈F}n^{−k/2}|k!I_{n,k}(f)| ≥ u) ≤ C exp\{ −α ( \frac{u}{σ})^{2/k}\} holds for those

numbers u for whichnσ^{2} ≥ ( \frac{u}{σ})^{2/k} ≥ M(L^{3/2}log \frac{2}{σ} + (logD)^{3/2}),

(8.14)

where the numbers D and L agree with the parameter and exponent of the L_{2}-dense class F.

The condition about the countable cardinality of the class F can be replaced by the weaker condition that the class of random variables n^{−k/2}I_{n,k}(f), f ∈ F, is countably approximable.

Next I formulate a Gaussian counterpart of the above results. To do this I need some notions that will be introduced in Chapter 10. In that chapter the white noise with a reference measure μ will be defined. It is an appropriate set of jointly Gaussian random variables indexed by those measurable sets A ∈ X of a measure space (X,X ,μ) with a σ-finite measure μ for which μ(A) < ∞. Its distribution depends on the measure μ which will be called the reference measure of the white noise.

In Chapter 10 it will also be shown that given a white noise μ_{W}with a non-atomic σ-additive reference measure μ on a measurable space (X,X ) and a measurable function f(x_{1},...,x_{k}) of k variables on the product space (X^{k},X^{k}) such that

∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) ≤ σ^{2} < ∞  (8.15)

a k-fold Wiener-Itô integral of the function f with respect to the white noise μ_{W}

Z_{μ,k}(f) = \frac{1}{k!} ∫ f(x_{1},...,x_{k})μ_{W}(dx_{1})...μ_{W}(dx_{k})  (8.16)

can be defined, and the main properties of this integral will be proved there. It will be seen that Wiener-Itô integrals have a similar relation to degenerate U-statistics and multiple integrals with respect to normalized empirical measures as normally distributed random variables have to partial sums of independent random variables. Hence it is useful to find the analogues of the previous results to estimates about the tail distribution of Wiener-Itô integrals. This will be done in Theorems 8.5 and 8.6.

Theorem 8.5 (Estimate on the tail distribution of a multiple Wiener–Itô integral). Let us fix a measurable space (X,X ) together with a σ-finite non-atomic measure μ on it, and let μ_{W} be a white noise with reference measure μ on (X,X ). If f(x_{1},...,x_{k}) is a measurable function on (X^{k},X^{k}) which satisfies relation (8.15) with some 0 < σ < ∞, then

P(|k!Z_{μ,k}(f)| > u) ≤ C exp\{ −\frac{1}{2} ( \frac{u}{σ})^{2/k}\}  (8.17)

for all u > 0 with some constants C = C(k) depending only on k.

Theorem 8.6 Estimate on the supremum of Wiener–Itô integrals). Let F be a countable class of functions of k variables defined on the k-fold product (X^{k},X^{k}) of a measurable space (X,X ) such that

∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) ≤ σ^{2} with some 0 < σ ≤ 1 for all f ∈ F

with some non-atomic σ-additive measure μ on (X,X ). Let us also assume that F is an L_{2}-dense class of functions in the space (X^{k},X^{k}) with respect to the measure μ^{k} with some exponent L ≥ 1 and parameter D ≥ 1, where μ^{k} is the k-fold product of the measure μ. (The classes of L_{2}-dense classes with respect to a measure were defined in Chapter 4.)

Take a white noise μ_{W} on (X,X ) with reference measure μ, and define the Wiener–Itô integrals Z_{μ,k}(f) for all f ∈ F. Fix some 0 < ε ≤ 1. The inequality

P ( sup_{f∈F}|k!Z_{μ,k}(f)| > u) ≤ CDexp\{ −\frac{1}{2} ( \frac{(1 − ε)u}{σ} ) ^{2/k}\}  (8.18)

holds for those numbers u which satisfy the inequality u ≥ ML^{k/2}σ \frac{1}{ε}(log^{k/2} \frac{2}{ε} + log^{k/2} \frac{2}{σ}). Here C = C(k) > 0, M = M(k) > 0 are some universal constants depending only on the multiplicity k of the integrals.

Remark: Theorem 8.6 is the multivariate version of Theorem 4.2 about the tail distribution of the supremum of Gaussian random variables. In Theorem 4.2 we could get good estimates for such levels u which satisfy the inequality u ≥ const.σ log^{1/2} \frac{2}{σ} with an appropriate constant, while in Theorem 8.6 we had a similar estimate under the condition u ≥ const.σ log^{k/2} \frac{2}{σ} with an appropriate constant. In Chapter 4 we presented an example which shows that the above condition on the level u in Theorem 4.2 cannot be dropped. A similar example can be given about the necessity of the analogous condition in Theorem 8.6 with the help of the subsequent Example 8.7.

Put f_{s,t}(u_{1},...,u_{k}) = ∏_{j=1}^{k} f_{s,t}^{0}(u_{j}), where f_{s,t}^{0}(u) denotes the indicator function of

the interval [s,t]. Take the class of functions

F = F_{σ} = {f_{s,t}: 0 ≤ s < t ≤ 1, t − s ≤ σ^{2/k}, s and t are rational},

and define for all functions f_{s,t} ∈ F the k-fold Wiener–Itô integral

Z(f_{s,t}) = \frac{1}{k!} ∫ f_{s,t}(u_{1},...,u_{k})W (du_{1})...W (du_{k}).

Then EZ(f_{s,t})^{2} ≤ \frac{σ^{2}}{k!} for all f_{s,t}∈ F, and it can be seen with the help of Example 8.7 similarly to the corresponding argument applied in Chapter 4 that there is some c > 0

such that P ( sup_{f_{s,t}∈F_{σ}} Z(f_{s,t}) > cσ log^{k/2} \frac{2}{σ}) → 1 as σ → 0. Beside this, it can be seen that F is an L_{2}-dense class with respect to the Lebesgue measure. This implies that the lower bound imposed on u in Theorem 8.6 cannot be dropped. I omit the details of the proof.

Formula (8.18) yields an almost as good estimate for the supremum of Wiener– Itô integrals with the choice of a small ε > 0 as formula (8.17) for a single Wiener– Itô integral. But the lower bound imposed on the number u in the estimate (8.18) depends on ε, and for a small number ε > 0 it is large.

The subsequent result presented in Example 8.7 may help to understand why Theorems 8.3 and 8.5 are sharp. Its proof and the discussion of the question about the sharpness of Theorems 8.3 and 8.5 will be postponed to Chapter 13.

Example 8.7 (A converse estimate to Theorem 8.5). Let us have a σ-finite measure μ on some measure space (X,X ) together with a white noise μ_{W} on (X,X ) with counting measure μ. Let f_{0}(x) be a real valued function on (X,X ) such that ∫ f_{0}(x)^{2}μ(dx) = 1, and take the function f(x_{1},...,x_{k}) = σf_{0}(x_{1})··· f_{0}(x_{k}) with some number σ > 0 together with the Wiener–Itô integral Z_{μ,k}(f) introduced in formula (8.16).

Then the relation ∫ f(x_{1},...,x_{k})^{2}μ(dx_{1})... μ(dx_{k}) = σ^{2} holds, and the Wiener– Itô integral Z_{μ,k}(f) satisfies the inequality

P(|k!Z_{μ,k}(f)| > u) ≥ \frac{C̄}{( \frac{u}{σ})^{1/k}+ 1} exp\{ −\frac{1}{2} ( \frac{u}{σ})^{2/k}\} for all u > 0  (8.19)

with some constant C̄> 0.

The above results show that multiple integrals with respect to a normalized empirical distribution or degenerate U-statistics satisfy some estimates similar to those about multiple Wiener–Itô integrals, but they hold under more restrictive conditions. The difference between the estimates in these problems is similar to the difference between the corresponding results in Chapter 4 whose reason was explained there. Hence this will be only briefly discussed here.

The estimates of Theorem 8.1 and 8.3 are similar to that of Theorem 8.5. Moreover, for 0 ≤ u ≤ εn^{k/2}σ^{k+1} with a small number ε > 0 Theorem 8.3 yields an almost as good estimate about degenerate U-statistics as Theorem 8.5 yields for a Wiener–Itô integral with the same kernel function f and underlying measure μ. Example 8.7 shows that the constant in the exponent of formula (8.17) cannot be improved, at least there is no possibility of an improvement if only the L_{2}-norm of the kernel function f is known. Some results discussed later indicate that neither the estimate of Theorem 8.3 can be improved. The main difference between Theorem 8.5 and the results of Theorem 8.1 or 8.3 is that in the latter case the kernel function f must satisfy not only an L_{2} but also an L_{∞} norm type condition, and the estimates of these results are formulated under the additional condition u ≤ n^{k/2}σ^{k+1}. It can be shown that the condition about the L_{∞} norm of the kernel function cannot be dropped from the conditions of these theorems, and a version of Example 3.3 will be presented in Example 8.8 which shows that in the case u ≫ n^{k/2}σ^{k+1} the lefthand side of (8.12) may satisfy only a much weaker estimate. This estimate will be given only for k = 2, but with some work it can be generalized for general indices k.

Theorems 8.2, 8.4 and 8.6 show that for the tail distribution of the supremum of a not too large class of degenerate U-statistics or multiple integrals a similar upper bound can be given as for the tail distribution of a single degenerate U-statistic or multiple integral, only the universal constants may be worse in the new estimates. However, they hold only under the additional condition that the level at which the tail distribution of the supremum is estimated is not too low. A similar phenomenon appeared already in the results of Chapter 4. Moreover, such a restriction had to be imposed in the formulation of the results here and in Chapter 4 for the same reason.

In Theorem 8.2 and 8.4 an L_{2}-dense class of kernel functions was considered, and this meant that the class of random integrals or U-statistics we consider in this result is not too large. In Theorem 8.6 a similar, but weaker condition was imposed on the class of kernel functions. They had to satisfy a similar condition, but only for the reference measure μ of the white noise appearing in the Wiener–Itô integral. A similar difference appears in the comparison of Theorems 4.1 or 4.1′with Theorem 4.2, and this difference has the same reason in the two cases.

Next I present the proof of the following Example 8.8 which is a multivariate version of Example 3.3. For the sake of simplicity I restrict my attention to the case k = 2.

Example 8.8 (A converse estimate to Theorem 8.3). Let us take a sequence of independent and identically distributed random variables ξ_{1},...,ξ_{n} with values in the plane X = R^{2} such that ξ_{j} = (η_{j,1},η_{j,2}), η_{j,1} and η_{j,2} are independent random variables with the following distributions. The distribution of η_{j,1} is defined with the help of a parameter σ^{2}, 0 < σ^{2} ≤ \frac{1}{8}, in the same way as the distribution of the random variables X_{j} in Example 3.3, i.e. η_{j,1} = η̄_{j,1} −Eη̄_{j,1} with P(η̄_{j,1} = 1) = σ̄^{2}, P(η̄_{j,1} = 0) = 1 − σ̄^{2}, where σ̄^{2} is that solution of the equation x^{2} − x + σ^{2} = 0, which is smaller than \frac{1}{2}. The distribution of the random variables η_{j,2} is given by the formula P(η_{j,2} = 1) = P(η_{j,2} = −1) = \frac{1}{2} for all 1 ≤ j ≤ n. Introduce the function f(x,y) = f((x_{1},x_{2}),(y_{1},y_{2})) = x_{1}y_{2} + x_{2}y_{1}, x = (x_{1},x_{2}) ∈ R^{2}, y = (y_{1},y_{2}) ∈ R^{2} if (x,y) is in the support of the distribution of the random vector (ξ_{1},ξ_{2}), i.e. if x_{1} and y_{1} take the values 1 − σ̄^{2} or −σ̄^{2} and x_{2} and y_{2} take the values ±1. Put f(x,y) = 0 otherwise. Define the U-statistic

I_{n,2}(f) = \frac{1}{2} ∑_{1≤j,k≤n,j≠k} f(ξ_{j},ξ_{k}) = \frac{1}{2} ∑_{1≤j,k≤n,j≠k} (η_{j,1}η_{k,2}+ η_{k,1}η_{j,2})

of order 2 with the above kernel function f and sequence of independent random variables ξ_{1},...,ξ_{n}. Then I_{n,2}(f) is a degenerate U-statistic such that |supf(x,y)| ≤ 1 and Ef^{2}(ξ_{j},ξ_{j}) = σ^{2}.

If u ≥ B_{1}nσ^{3} with some appropriate constant B_{1} > 2, B̄_{2}^{−1}n ≥ u ≥ B̄_{2}n^{−1/2} with a sufficiently large fixed number B̄_{2} > 0 and \frac{1}{4} ≥ σ^{2} ≥ \frac{1}{n^{2}}, and n is a sufficiently large number, then the estimate

P(n^{−1} I_{n,2}(f) > u) ≥ exp\{−Bn^{1/3}u^{2/3}log( \frac{u}{nσ^{3}} ) \}  (8.20)

holds with some B > 0.

Remark: In Theorem 8.3 we got the estimate P(n^{−1} I_{n,2}(f) > u) ≤ e^{−αu/σ} for the above defined degenerate U-statistic I_{n,2}(f) if 0 ≤ u ≤ nσ^{3}. In the particular case u = nσ^{3} we have the estimate P(n^{−1} I_{n,2}(f) > nσ^{3}) ≤ e^{−αnσ^{2}} . On the other hand, the above example shows that in the case u ≫ nσ^{3} we can get only a weaker estimate. It is worth looking at the estimate (8.20) with fixed parameters n and u and to observe the dependence of the upper bound on the variance σ^{2} of I_{n,2}(f). In the case σ^{2} = u^{2/3}n^{−2/3} we have the upper bound e^{−αn^{1/3}u^{2/3}} . Example 8.8 shows that in the case σ^{2} ≪ u^{2/3}n^{−2/3} we can get only a relatively small improvement of this estimate. A similar picture appears as in Example 3.3 in the case k = 1.

It is simple to check that the U-statistic introduced in the above example is degenerate because of the independence of the random variables η_{j,1} and η_{j,2} and the identity Eη_{j,1} = Eη_{j,2} = 0. Beside this, Ef(ξ_{j},ξ_{j})^{2} = σ^{2}. In the proof of the estimate (8.20) the results of Chapter 3, in particular Example 3.3 can be applied for the sequence η_{j,1}, j = 1,2,...,n. Beside this, the following result, known from the theory of large deviations will be applied. If X_{1},...,X_{n} are independent and identically distributed random variables, P(X_{1} = 1) = P(X_{1} = −1) = \frac{1}{2}, then for any number 0 ≤ α < 1 there exists some numbers C_{1} = C_{1}(α) > 0 and C_{2} = C_{2}(α) > 0 such

that P ( ∑_{j=1}^{n} X_{j} > u) ≥ C_{1}e^{−C_{2}u^{2}/n} for all 0 ≤ u ≤ αn.

Proof of Example 8.8. The inequality

P(n^{−1} I_{n,2}(f) > u)  (8.21)

≥ P ( ( ∑_{j=1}^{n} η_{j,1}) ( ∑_{j=1}^{n} η_{j,2}) > 4nu) − P ( ∑_{j=1}^{n} η_{j,1}η_{j,2} > 2nu)

holds. Because of the independence of the random variables η_{j,1} and η_{j,2} the first probability at the right-hand side of (8.21) can be bounded from below by bounding the multiplicative terms in it with v_{1} = 4n^{1/3}u^{2/3} and v_{2} = n^{2/3}u^{1/3}. The first term will be estimated by means of Example 3.3. This estimate can be applied with the choice y = v_{1}, since the relation v_{1} ≥ 4nσ^{2} holds if u ≥ B_{1}nσ^{3} with B_{1} > 1, and the remaining conditions 0 ≤ σ^{2} ≤ \frac{1}{8} and n ≥ 4v_{1} ≥ 6 also hold under the conditions of Example 8.8. The second term can be bounded with the help of the large-deviation result mentioned after the remark, since v_{2} ≤ \frac{1}{2}n if u ≤ B̄_{2}^{−1}n with a sufficiently large B̄_{2}> 0. In such a way we get the estimate

P ( ( ∑_{j=1}^{n} η_{j,1}) ( ∑_{j=1}^{n} η_{j,2}) > 4nu)

≥ P ( ∑_{j=1}^{n} η_{j,1} > v_{1}) P ( ∑_{j=1}^{n} η_{j,2} > v_{2})

≥ C exp\{ −B_{1}v_{1}log( \frac{v_{1}}{nσ^{2}} ) − B_{2} \frac{v_{2}^{2}}{n}\}

≥ C exp\{−B_{3}n^{1/3}u^{2/3}log( \frac{u}{nσ^{3}} ) \}

with appropriate constants B_{1} > 1, B_{2} > 0 and B_{3} > 0. On the other hand, by applying Bennett’s inequality, more precisely its consequence given in formula (3.4) for the sum of the random variables X_{j} = η_{j,1}η_{j,2} at level nu instead of level u we get the following upper bound for the second term at the right-hand side of (8.21).

P ( ∑_{j=1}^{n} η_{j,1}η_{j,2} > 2nu) ≤ exp\{−Knulog \frac{u}{σ^{2}} \}
≤ exp\{−2B_{4}n^{1/3}u^{2/3}log( \frac{u}{nσ^{3}} ) \} ,

since Eη_{j,1}η_{j,2} = 0, Eη_{j,1}^{2}η_{j,2}^{2} = σ^{2}, nu ≥ B_{1}n^{2}σ^{3} ≥ 2nσ^{2} because of the conditions B_{1} > 2 and nσ ≥ 1. Hence the estimate (3.4) (with parameter nu) can be applied in this case. Beside this, the constant B_{4} can be chosen sufficiently large in the last inequality if the number n or the bound B̄_{2} in Example 8.8 us chosen sufficiently large. This means that this term is negligible small. The above estimates imply the statement of Example 8.8.

Let me remark that under some mild additional restrictions the estimate (8.20) can be slightly sharpened, the term log can be replaced by log^{2/3} in the exponent of the right-hand side of (8.20). To get such an estimate some additional calculation is needed where the numbers v_{1} and v_{2} are replaced by v̄_{1} = 4n^{1/3}u^{2/3}log^{−1/3} ( \frac{u}{nσ^{3}} )

and v̄_{2}= n^{2/3}u^{1/3}log^{1/3}( \frac{u}{nσ^{3}} ).

I finish this chapter with a short overview about the remaining part of this work. In our proofs we needed some results about U-statistics, and this is the main

topic of Chapter 9. One of the results discussed there is the so-called Hoeffding decomposition of U-statistics to the linear combination of degenerate U-statistics of different order. We also needed some additional results which explain how some properties (e.g. a bound on the L_{2} and L_{∞} norm of a kernel function, the L_{2}-density property of a class F of kernel function) is inherited if we turn from the original U-statistics to the degenerate U-statistics appearing in their Hoeffding decomposition. Chapter 9 contains some results in this direction. Another important result in it is Theorem 9.4 which yields a decomposition of multiple integrals with respect to a normalized empirical distribution to the linear combination of degenerate Ustatistics. This result is very similar to the Hoeffding decomposition of U-statistics.

The main difference between them is that in the decomposition of multiple integrals much smaller coefficients appear. Theorem 9.4 makes possible to reduce the proof of Theorems 8.1 and 8.2 to the corresponding results in Theorems 8.3 and 8.4 about degenerate U-statistics.

The definition and the main properties of Wiener–Itô integrals needed in the proof of Theorems 8.5 and 8.6 are presented in Chapter 10. It also contains a result, called the diagram formula for Wiener–Itô integrals which plays an important role in our considerations. Beside this, we proved a limit theorem, where we expressed the limit of normalized degenerate U-statistics with the help of multiple Wiener–Itô integrals. This result may explain why it is natural to consider Theorem 8.5 as the natural Gaussian counterpart of Theorem 8.5, and Theorem 8.6 as the natural Gaussian counterpart of Theorem 8.6.

We could prove Bernstein’s and Bennett’s inequality by means of a good estimation of the exponential moments of the partial sums we were investigating. In the proof of their multivariate versions, in Theorems 8.3 and 8.5 this method does not work, because the exponential moments we have to bound in these cases may be infinite. On the other hand, we could prove these results by means of a good estimate on the high moments of the random variables whose tail distribution we wanted to bound. In the proof of Theorem 8.5 the moments of multiple Wiener–Itô integrals have to be bounded, and this can be done with the help of the diagram formula for Wiener–Itô integrals. In Chapter 11 and 12 we proved that there is a version of the diagram formula for degenerate U-statistics, and this enables us to estimate the moments needed in the proof of Theorem 8.3. In Chapter 13 we proved Theorems 8.3, 8.5 and a multivariate version of the Hoeffding inequality. At the end of this chapter we still discussed some results which state that in certain cases when we have some useful additional information about the behaviour of the kernel function f beside the upper bound of their L_{2} and L_{∞} norm the estimates of Theorems 8.3 or 8.5 can be improved.

Chapter 14 contains the natural multivariate versions of the results in Chapter 6. In Chapter 6 Theorem 4.2 is proved about the supremum of Gaussian random variables and in Chapter 14 its multivariate version, Theorem 8.6. Both results are proved with the help of the chaining argument. On the other hand, the chaining argument is not strong enough to prove Theorem 4.1. But as it is shown in Chapter 6, it enables us to prove a result formulated in Proposition 6.1, and to reduce the proof of Theorem 4.1 with its help to a simpler result formulated in Proposition 6.2. One of the results in Chapter 14, Proposition 14.1, is a multivariate version of Proposition 6.1. We showed that the proof of Theorem 8.4 can be reduced with its help to the proof of a result formulated in Proposition 14.2, which can be considered a multivariate version of Proposition 6.2. Chapter 14 contains still another result. It turned out that it is simpler to work with so-called decoupled U-statistics introduced in this chapter than with usual U-statistics, because they have more independence properties. In Proposition 14.2′ a version of Proposition 14.2 is formulated about degenerate U-statistics, and it is shown with the help of a result of de la Peña and Montgomery–Smith that the proof of Proposition 14.2, and thus of Theorem 8.4 can be reduced to the proof of Proposition 14.2′.

Proposition 14.2′is proved similarly to its one-variate version, Proposition 6.2. The strategy of the proof is explained in Chapter 15. The main difference between the proof of the two propositions is that since the independence properties exploited in the proof of Proposition 6.2 hold only in a weaker form in this case, we have to apply a more refined and more difficult argument. In particular, we have to apply instead of the symmetrization lemma, Lemma 7.1, a more general version of it. We presented an appropriate version of this result in Lemma 15.2. It is hard to check the conditions of Lemma 15.2 when we try to apply it in the problems arising in the proof of Proposition 14.2′. This is the reason why we had to prove Proposition 14.2′with the help of two inductive propositions, formulated in Propositions 15.3 and 15.4, while in the proof of Proposition 6.2 it was enough to prove a single result, presented in Proposition 7.3. We discuss the details of the problems and the strategy of the proof in Chapter 15. The proof of Propositions 15.3 and 15.4 is given in Chapters 16 and 17. Chapter 16 contains the symmetrization arguments needed for us, and the proof is completed with its help in Chapter 17.

Finally in Chapter 18 we give an overview of this work, and explain its relation to some similar researches. The proof of some results is given in the Appendix.

Some results about U-statistics

This chapter contains the proof of the Hoeffding decomposition theorem, an important result about U-statistics. It states that all U-statistics can be represented as a sum of degenerate U-statistics of different order. This representation can be considered as the natural multivariate version of the decomposition of a sum of independent random variable to the sum of independent random variables with expectation zero plus a constant (which can be interpreted as a random variable of zero variable). Some important properties of the Hoeffding decomposition will also be proved. In particular, it will be investigated how some properties of the kernel function of a U-statistic is inherited in the behaviour of the kernel functions of the U-statistics in its Hoeffding decomposition.

If the Hoeffding decomposition of a U-statistic is taken, then the L_{2} and L_{∞} - norms of the kernel functions appearing in the U-statistics of the Hoeffding decomposition will be bounded by means of the corresponding norm of the kernel function of the original U-statistic. It will also be shown that if we take a class of U-statistics with an L_{2}-dense class of kernel functions (and the same sequence of independent and identically distributed random variables in the definition of each Ustatistic) and consider the Hoeffding decomposition of all U-statistics in this class, then the kernel functions of the degenerate U-statistics appearing in these Hoeffding decompositions also constitute an L_{2}-dense class. Another important result of this chapter is Theorem 9.4. It yields a decomposition of a k-fold random integral with respect to a normalized empirical distribution to the linear combination of degenerate U-statistics. This result enables us to derive Theorem 8.1 from Theorem 8.3 and Theorem 8.2 from Theorem 8.4, and it is also useful in the proof of Theorems 8.3 and 8.4.

Let us first consider the Hoeffding’s decomposition. In the special case k = 1 it

states that the sum S_{n} = ∑_{j=1}^{n} ξ_{j} of independent and identically distributed random

variables can be rewritten as S_{n} = ∑_{j=1}^{n} (ξ_{j} − Eξ_{j}) + ( ∑_{j=1}^{n} Eξ_{j}) , i.e. as the sum of

76 9 Some results about U-statistics

independent random variables with zero expectation plus a constant. We introduced the convention that a constant is the kernel function of a degenerate U-statistic of order zero, and I_{n,0}(c) = c for a U-statistic of order zero. I wrote down the above trivial formula, because Hoeffding’s decomposition is actually its adaptation to a more general situation. To understand this let us first see how to adapt the above construction to the case k = 2.

In this case a sum of the form 2I_{n,2}(f) = ∑_{1≤j,k≤n,j≠k} f(ξ_{j},ξ_{k}) has to be consid-

ered. Write f(ξ_{j},ξ_{k}) = [f(ξ_{j},ξ_{k})−E(f(ξ_{j},ξ_{k})|ξ_{k})]+E(f(ξ_{j},ξ_{k})|ξ_{k}) = f_{1}(ξ_{j},ξ_{k})+ f̄_{1}(ξ_{k}) with f_{1}(ξ_{j},ξ_{k}) = f(ξ_{j},ξ_{k})−E(f(ξ_{j},ξ_{k})|ξ_{k}), and f̄_{1}(ξ_{k}) = E(f(ξ_{j},ξ_{k})|ξ_{k}) to make the conditional expectation of f_{1}(ξ_{j},ξ_{k}) with respect to ξ_{k} equal zero. Repeating this procedure for the first coordinate we define f_{2}(ξ_{j},ξ_{k}) = f_{1}(ξ_{j},ξ_{k}) − E(f_{1}(ξ_{j},ξ_{k})|ξ_{j}) and f̄_{2}(ξ_{j}) = E(f_{1}(ξ_{j},ξ_{k})|ξ_{j}). Let us also write f̄_{1}(ξ_{k}) = [f̄_{1}(ξ_{k}) − Ef̄_{1}(ξ_{k})] + Ef̄_{1}(ξ_{k}) and f̄_{2}(ξ_{j}) = [f̄_{2}(ξ_{j}) − Ef̄_{2}(ξ_{j})] + Ef̄_{2}(ξ_{j}). Simple calculation shows that 2I_{n,2}(f_{2}) is a degenerate U-statistics of order 2, and the identity 2I_{n,2}(f) = 2I_{n,2}(f_{2}) + I_{n,1}((n − 1)(f̄_{1} − Ef̄_{1})) + I_{n,1}((n − 1)((f̄_{2} − Ef̄_{2})) + n(n − 1)E(f̄_{1} + f̄_{2}) yields the decomposition of I_{n,2}(f) into a sum of degenerate Ustatistics of different orders.

Hoeffding’s decomposition can be obtained by working out the details of the above argument in the general case. But it is simpler to calculate the appropriate conditional expectations by working with the kernel functions of the U-statistics. To carry out such a program we introduce the following notations.

Let us consider the k-fold product (X^{k},X^{k},μ^{k}) of a measure space (X,X ,μ) with some probability measure μ, and define for all integrable functions f(x_{1},...,x_{k}) and indices 1 ≤ j ≤ k the projection P_{j}f of the function f to its j-th coordinate, i.e. integration of the function f with respect to its j-th coordinate.

For the sake of simpler notations in our later considerations we shall define the operator P_{j} in a slightly more general setting. Let us consider a set A = {p_{1},...,p_{s}} ⊂ {1,...,k}, put X^{A}= X_{p_{1}} ×X_{p_{2}} ×··· ×X_{p_{s}}, X^{A}= X_{p_{1}} ×X_{p_{2}} ×··· × X_{p_{s}}, μ^{A}= μ_{p_{1}} ×μ_{p_{2}} ×··· ×μ_{p_{s}}, take the product space (X^{A},X^{A},μ^{A}) and if j ∈ A, then define the operator P_{j} as mapping a function on this product space to a function on the product space (X^{A\{j}},X^{A\{j}})b by the formula

(P_{j}f)(x_{p_{1}},...,x_{p_{r−1}} ,x_{p_{r+1}} ,...,x_{p_{s}}) = ∫ f(x_{p_{1}},...,x_{p_{s}})μ(dx_{j}), if j = p_{r}. (9.1)

Let us also define the (orthogonal projection) operators Q_{j} = I −P_{j} as Q_{j}f = f −P_{j}f for all integrable functions f on the space (X^{A},X^{A},μ^{A}), and j ∈ A, i.e. put

(Q_{j}f)(x_{p_{1}},...,x_{p_{s}}) = (I − P_{j})f(x_{p_{1}},...,x_{p_{s}})
= f(x_{p_{1}},...,x_{p_{s}}) − ∫ f(x_{p_{1}},...,x_{p_{s}})μ(dx_{j}). (9.2)

In the definition (9.1) P_{j}f is a function not depending on the coordinate x_{j}, but in the definition of Q_{j} we introduce the fictive coordinate x_{j} to make the expression Q_{j}f = f − P_{j}f meaningful.

Remark. I shall use the following notation. (P_{j}f)(x_{p_{1}},...,x_{p_{r−1}} ,x_{p_{r+1}} ,...,x_{p_{s}}) will denote the value of the function P_{j}f in the point (x_{p_{1}},...,x_{p_{r−1}} ,x_{p_{r+1}} ,...,x_{p_{s}}). On the other hand, I write P_{j}f(x_{p_{1}},...,x_{p_{s}}) (without parentheses) if I want to denote the action of P_{j} to f, but it is more natural to denote also the arguments of the function f. The same notation will be applied for the operator Q_{j}.

The following result holds.

Theorem 9.1 (The Hoeffding decomposition of U-statistics). Let f(x_{1},...,x_{k}) be an integrable function on the k-fold product (X^{k},X^{k},μ^{k}) of a space (X,X ,μ) with a probability measure μ. It has a decomposition of the form

f(x_{1},...,x_{k}) = ∑_{V ⊂{1,...,k}}f_{V}(x_{j_{1}},...,x_{j_{|V |}}),  (9.3)

with f_{V}(x_{j_{1}},...,x_{j_{|V |}}) = ( ∏_{j∈{1,...,k}\V} P_{j} ∏_{j′∈V} Q_{j′}) f(x_{1},...,x_{k})

with V = {j_{1},...,j_{|V |}}, j_{1} < j_{2} < ···< j_{|V |}, for all V ⊂ {1,...,k}. Beside this, all functions f_{V}, V ⊂ {1,...,k}, defined in (9.3) are canonical with respect to the probability measure μ with |V | arguments.

Let ξ_{1},...,ξ_{n} be a sequence of independent μ distributed random variables, and consider the U-statistics I_{n,k}(f) and I_{n,|V |}(f_{V}) corresponding to the kernel functions f, f_{V} defined in (9.3) and random variables ξ_{1},...,ξ_{n}. Then

k!I_{n,k}(f) = ∑_{V ⊂{1,...,k}}(n − |V |)(n − |V | − 1)··· (n − k + 1)|V |!I_{n,|V |}(f_{V})  (9.4)

is a representation of k!I_{n,k}(f) as a sum of degenerate U-statistics, where |V | denotes the cardinality of the set V. (The product (n − |V |)(n − |V | − 1)··· (n − k + 1) is defined as 1 for V = {1,...,k}, i.e. if |V | = k.) This representation is called the Hoeffding decomposition of k!I_{n,k}(f).

Proof of Theorem 9.1. Write f = ∏_{j=1}^{k} (P_{j} + Q_{j})f. By carrying out the multipli-

cations in this identity and applying the commutativity of the operators P_{j} and Q_{j} for different indices j we get formula (9.3). To show that the functions f_{V} in formula (9.3) are canonical let us observe that this property can be rewritten in the form P_{j}f_{V} ≡ 0 (in all points (x_{s}, s ∈ V \ {j}) if j ∈ V). Since P_{j} = P_{j}^{2}, and the identity P_{j}Q_{j} = P_{j} − P_{j}^{2} = 0 holds for all j ∈ {1,...,k} this relation follows from the above mentioned commutativity of the operators P_{j} and Q_{j}, as

P_{j}f_{V} = ( ∏_{s∈{1,...,k}\V}P_{s} ∏_{s∈V \{j}}Q_{s}) P_{j}Q_{j}f = 0. By applying identity (9.3) for all

terms f(ξ_{j_{1}},...,ξ_{j_{k}}) in the sum defining the U-statistic k!I_{n,k}(f) (see formula (8.8)) and then summing them up we get relation (9.4).

In the Hoeffding decomposition we rewrote a general U-statistic in the form of a linear combination of degenerate U-statistics. In many applications of this result we still we have to know how the properties of the kernel function f of the original U-statistic are reflected in the properties of the kernel functions f_{V} of the degenerate U-statistics taking part in the Hoeffding composition. In particular, we need a good estimate on the L_{2} and L_{∞} norm of the functions f_{V} by means of the corresponding norm of the function f. Moreover, if we want to prove estimates on the tail distribution of the supremum of U-statistics I_{n,k}(f) defined with the help of an L_{2}- dense class of kernel functions F with some exponent L and parameter D, then we may need a similar estimate on the classes of kernel functions F_{V} = {f_{V}: f ∈ F } with functions f_{V}, V ∈ {1,...,k} appearing in the Hoeffding decomposition of these functions. We have to show that this class of functions is also L_{2}-dense, and we also need a good bound on the exponent and parameter of this L_{2}-dense class. In the next result such statements will be proved.

Theorem 9.2 (Some properties of the Hoeffding decomposition). Let us consider a square integrable function f(x_{1},...,x_{k}) on the k-fold product space (X^{k},X^{k},μ^{k}) and take its decomposition defined in formula (9.3). The inequalities

∫ f_{V}^{2}(x_{j}, j ∈ V) ∏_{j∈V} μ(dx_{j}) ≤ ∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k})  (9.5)

and

sup_{x_{j},j∈V} |f_{V}(x_{j}, j ∈ V)| ≤ 2^{|V |} sup_{x_{j},1≤j≤k} |f(x_{1},...,x_{k})|  (9.6)

hold for all V ⊂ {1,...,k}. In particular,

f_{∅}^{2} ≤ ∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) for V = ∅.

Let us consider an L_{2}-dense class F of functions with some parameter D ≥ 1 and exponent L ≥ 0 on the space (X^{k},X^{k}), take the decomposition (9.3) of all functions f ∈ F, and define the classes of functions F_{V}= {2^{−|V |}f_{V}: f ∈ F } for all V ⊂ {1,...,k} with the functions f_{V} taking part in this decomposition. These classes of functions F_{V} are also L_{2}-dense with the same parameter D and exponent L for all V ⊂ {1,...,k}.

Theorem 9.2 will be proved as a consequence of Proposition 9.3 presented below. To formulate it first some notations will be introduced.

Let us consider the product (Y × Z,Y × Z) of two measurable spaces (Y,Y ) and (Z,Z) together with a probability measure μ on (Z,Z) and the operator

(Pf)(y) = (P_{μ}f)(y) = ∫ f(y,z)μ(dz), y ∈ Y, z ∈ Z  (9.7)

defined for those y ∈ Y for which the above integral is finite. Let I denote the identity operator on the space of functions on Y ×Z, i.e. let (If)(y,z) = f(y,z), and introduce the operator Q = Q_{μ} = I − P = I − P_{μ} (Q_{μ}f)(y,z) = ((I − P_{μ})f)(y,z) = f(y,z) − (P_{μ}f)(y,z) = f(y,z) − ∫ f(y,z)μ(dz),

(9.8)

defined for those points (y,z) ∈ Y × Z whose first coordinate y is such that the expression (P_{μ}f)(y) is meaningful. (Here, and in the sequel a function g(y) defined on the space (Y,Y ) will be sometimes identified with the function ḡ(y,z) = g(y) on the space (Y ×Z,Y ×Z) which actually does not depend on the coordinate z.) The following result holds:

Proposition 9.3. Let us consider the direct product (Y × Z,Y × Z) of two measure spaces (Y,Y ) and (Z,Z) together with a probability measure μ on the space (Z,Z). Take the transformations P_{μ} and Q_{μ} defined in formulas (9.7) and (9.8). Given any probability measure ρ on the space (Y,Y ) consider the product measure ρ × μ on (Y × Z,Y × Z). Then the transformations P_{μ} and Q_{μ}, as maps from the space L_{2}(Y × Z,Y × Z ,μ × ρ) to L_{2}(Y,Y ,ρ) and L_{2}(Y × Z,Y × Z ,ρ × μ) respectively, have a norm less than or equal to 1, i.e.

∫ (P_{μ}f)(y)^{2}ρ(dy) ≤ ∫ f(y,z)^{2}ρ(dy)μ(dz),  (9.9)

and

∫ (Q_{μ}f)(y,z)^{2}ρ(dy)μ(dz) ≤ ∫ f(y,z)^{2}ρ(dy)μ(dz)  (9.10)

for all functions f ∈ L_{2}(Y × Z,Y × Z ,ρ × μ).

If F is an L_{2}-dense class of functions f(y,z) in the product space (Y × Z,Y × Z), with some parameter D ≥ 1 and exponent L ≥ 0, then also the classes F_{μ} = \{P_{μ}f, f ∈ F \} and G_{μ} = \{\frac{1}{2}Q_{μ}f = \frac{1}{2}(f − P_{μ}f), f ∈ F \} are L_{2}-dense classes with the same exponent L and parameter D in the spaces (Y,Y ) and (Y × Z,Y × Z) respectively.

The following corollary of Proposition 9.3 is formally more general, but it is a simple consequence of this result. Actually we shall need this corollary.

Corollary of Proposition 9.3. Let us consider the product (Y_{1}× Z × Y_{2},Y_{1}× Z × Y_{2}) of three measurable spaces (Y_{1},Y_{1}), (Z,Z) and (Y_{2},Y_{2}) with a probability measure μ on the space (Z,Z) and a probability measure ρ on Y_{1}× Y_{2},Y_{1}× Y_{2}), and define the transformations

(P_{μ}f)(y_{1},y_{2}) = ∫ f(y_{1},z,y_{2})μ(dz), y_{1} ∈ Y_{1}, z ∈ Z, y_{2} ∈ Y_{2}  (9.11)

and

(Q_{μ}f)(y_{1},z,y_{2})= ((I − P_{μ})f)(y_{1},z,y_{2}) = f(y_{1},z,y_{2}) − (P_{μ}f)(y_{1},z,y_{2}) (9.12)
= f(y_{1},z,y_{2}) − ∫ f(y_{1},z,y_{2})μ(dz), y_{1} ∈ Y_{1}, z ∈ Z, y_{2} ∈ Y_{2}

for the measurable functions f on the space Y_{1}× Z × Y_{2} integrable with respect the measure μ × ρ. Then

∫ (P_{μ}f)(y_{1},y_{2})^{2}ρ(dy_{1}, dy_{2}) ≤ ∫ f(y,z)^{2}(ρ × μ)(dy_{1}, dz, dy_{2})  (9.13)

for all probability measures ρ on (Y_{1}× Y_{2},Y_{1}× Y_{2}), where ρ × μ is the product of the probability measure ρ on (Y_{1}× Y_{2},Y_{1}× Y_{2}) and μ is a probability measure on (Z,Z). Also the inequality

∫ (Q_{μ}f)(y_{1},z,y_{2})^{2}ρ(dy_{1}, dy_{2})μ(dz) ≤ ∫ f(y_{1},z,y_{2})^{2}ρ(dy_{1}, dy_{2})μ(dz) (9.14)

holds for all functions f ∈ L_{2}(Y × Z,Y × Z ,ρ × μ).

If F is an L_{2}-dense class of functions f(y_{1},z,y_{2}) on the product space (Y_{1} × Z × Y_{2},Y_{1} × Z × Y_{2}), with some parameter D ≥ 1 and exponent L ≥ 0, then also the classes F_{μ} = \{P_{μ}f, f ∈ F \} and G_{μ} = \{\frac{1}{2}Q_{μ}f = \frac{1}{2}(f − P_{μ}f), f ∈ F \} are L_{2}- dense classes with exponent L and parameter D in the spaces (Y_{1}×Y_{2},Y_{1}×Y_{2}) and (Y_{1}× Z × Y_{2},Y_{1}× Z × Y_{2}) respectively.

This corollary is a simple consequence of Proposition 9.3 if we apply it with (Y,Y ) = (Y_{1}×Y_{2},Y_{1}×Y_{2}) and take the natural mapping f((y_{1},y_{2}),z) → f(y_{1},z,y_{2}) of a function from the space (Y ×Z,Y ×Z) to a function on (Y_{1}×Z ×Y_{2},Y_{1}×Z × Y_{2}). Beside this, we apply that measure on (Y_{1}×Z ×Y_{2},Y_{1}×Z ×Y_{2}) which is the image of the product measure ρ × μ with respect to the map induced by the above transformation on the space of measures.

Proposition 9.3, more precisely its corollary implies Theorem 9.2, since it implies that the operators P_{s}, Q_{s}, 1 ≤ s ≤ k, applied in Theorem 9.2 do not increase the L_{2}(μ) norm of a function f, and it is also clear that the norm of P_{s} is bounded by 1, the norm of Q_{s} = I − P_{s} is bounded by 2 as an operator from L_{∞} spaces to L_{∞} spaces. The corollary of Proposition 9.3 also implies that if F is an L_{2}-dense class of functions with parameter D and exponent L, then the same property holds for the classes of functions F_{P_{s}} = \{P_{s}f: f ∈ F \} and F_{Q_{s}} = \{\frac{1}{2}Q_{s}f: f ∈ F \},

1 ≤ s ≤ k. These relations together with the identity f_{V} = ( ∏_{s∈{1,...,k}\V}P_{s} ∏_{s∈V}Q_{s}) f

imply Theorem 9.2.

Proof of Proposition 9.3. The Schwarz inequality yields that

(P_{μ}f)(y)^{2} ≤ ∫ f(y,z)^{2}μ(dz) for all y ∈ Y,

and integrating this inequality with respect to the probability measure ρ(dy) we get inequality (9.9). Also the inequality

∫ (Q_{μ}f)(y,z)^{2}ρ(dy)μ(dz)= ∫ [f(y,z) − P_{μ}f(y,z)]^{2}ρ(dy)μ(dz)
≤ ∫ f(y,z)^{2}ρ(dy)μ(dz)

holds, and this is relation (9.10). This follows for instance from the observation that the functions f(y,z) − (P_{μ}f)(y,z) and (P_{μ}f)(y,z) are orthogonal in the space L_{2}(Y × Z,Y × Z ,ρ × μ).

Let us consider an arbitrary probability measure ρ on the space (Y,Y ). To prove that F_{μ} is an L_{2}-dense class with parameter D and exponent L if the same relation holds for F we have to find for all 0 < ε ≤ 1 a set {f_{1},...,f_{m}} ⊂ F_{μ}, 1 ≤ j ≤ m with m ≤ Dε^{−L} elements, such that inf_{1≤j≤m} ∫(f_{j} − f)^{2}dρ ≤ ε^{2} for all f ∈ F_{μ}. But a

similar property holds for F in the space Y ×Z with the probability measure ρ ×μ. This property together with the property of P_{μ} formulated in (9.9) imply that F_{μ} is an L_{2}-dense class.

To prove that G_{μ} is also L_{2}-dense with parameter D and exponent L under the same condition we have to find for all numbers 0 < ε ≤ 1 and probability measures ρ on Y ×Z a subset {g_{1},...,g_{m}} ⊂ G_{μ} with m ≤ Dε^{−L} elements such that inf_{1≤j≤m} ∫(g_{j} −

g)^{2}dρ ≤ ε^{2} for all g ∈ G_{μ}.

To show this let us consider the probability measure ρ̃ = \frac{1}{2}(ρ + ρ̄ × μ) on (Y × Z,Y × Z), where ρ̄ is the projection of the measure ρ to (Y,Y ), i.e. ρ̄(A) = ρ(A × Z) for all A ∈ Y , take a class of function F_{0}(ε,ρ̃) = {f_{1},...,f_{m}} ⊂ F with m ≤ Dε^{−L} elements such that inf_{1≤j≤m} ∫(f_{j} − f)^{2}dρ̃ ≤ ε^{2} for all f ∈ F,

and put \{g_{1},...,g_{m}\} = \{\frac{1}{2}Q_{μ}f_{1},..., \frac{1}{2}Q_{μ}f_{m}\}. All functions g ∈ G_{μ} can be written in the form g = \frac{1}{2}Q_{μ}f with some f ∈ F, and there exists some function f_{j} ∈ F_{0}(ε,ρ̃) such that ∫(f −f_{j})^{2}dρ̃≤ ε^{2}. Hence to complete the proof of Proposition 9.3 it is enough to show that ∫ \frac{1}{4}(Q_{μ}f − Q_{μ} f̄)^{2}dρ ≤ ∫(f − f̄)^{2}dρ̃ for all pairs f,f̄∈ F. This inequality holds, since ∫ \frac{1}{4}(Q_{μ}f − Q_{μ} f̄)^{2}dρ ≤ ∫ \frac{1}{2}(f − f̄)^{2}dρ + ∫ \frac{1}{2}(P_{μ}f − P_{μ} f̄)^{2}dρ, and ∫(P_{μ}f − P_{μ} f̄)^{2}dρ = ∫ P_{μ}(f − f̄)^{2}dρ̄ ≤ ∫(f − f̄)^{2}d(ρ̄× μ) by formula (9.9). The above relations imply that ∫ \frac{1}{4}(Q_{μ}f − Qμf̄)^{2}dρ ≤ ∫(f − f̄)^{2} \frac{1}{2}d (ρ + ρ̄× μ) = ∫(f − f̄)^{2}d ρ̃ as we have claimed.

Now we shall discuss the relation between Theorem 8.1′and Theorem 8.3 and between Theorem 8.2 and Theorem 8.4. First we show that Theorem 8.1 (or Theorem 8.1′) is equivalent to the estimate (8.13) in the corollary of Theorem 8.3 which is slightly weaker than the estimate (8.12) of Theorem 8.3. We also claim that Theorems 8.2 and 8.4 are equivalent. Both in Theorem 8.2 and in Theorem 8.4 we can restrict our attention to the case when the class of functions F is countable, since the case of countably approximable classes can be simply reduced to this situation. Let us remark that integration with respect to the measure μ_{n}− μ in the definition (4.8) of the integral J_{n,k}(f) yields some kind of normalization which is missing in the definition of the U-statistics I_{n,k}(f). This is the cause why degenerate U-statistics had to be considered in Theorems 8.3 and 8.4. The deduction of the corollary of Theorem 8.3 from Theorems 8.1′ or of Theorem 8.4 from Theorem 8.2 is fairly simple if the underlying probability measure μ is non-atomic, since in this case the identity I_{n,k}(f) = J_{n,k}(f) holds for a canonical function with respect to the measure μ. Let us remark that the non-atomic property of the measure μ is needed in this argument not only because of the conditions of Theorems 8.1′ and 8.2, but since in the proof of the above identity we need the identity ∫ f(x_{1},...,x_{k})μ(dx_{j}) ≡ 0 in the case when the domain of integration is not the whole space X but the set X \ {x_{1},...,x_{j−1} ,x_{j+1} ,...,x_{k}}.

The case of possibly atomic measures μ can be simply reduced to the case of nonatomic measures by means of the following enlargement of the space (X,X ,μ). Let us introduce the product space (X̄,X̄,μ̄) = (X,X ,μ) × ([0,1],B,λ), where B is the σ-algebra and λ is the Lebesgue measure on [0,1]. Define the function f̄((x_{1},u_{1}),...,(x_{k},u_{k})) = f(x_{1},...,x_{k}) in this enlarged space. Then I_{n,k}(f) = I_{n,k}(f̄), the measure μ̄ = μ × λ is non-atomic, and f̄is canonical with respect to μ̄ if f is canonical with respect to μ. Hence the corollary of Theorem 8.3 and Theorem 8.4 can be derived from Theorems 8.1′ and 8.2 respectively by proving them first for their counterpart in the above constructed enlarged space with the above defined functions.

Also Theorems 8.1′and 8.2 can be derived from Theorems 8.3 and 8.4 respectively, but this is a much harder problem. To do this let us observe that a random integral J_{n,k}(f) can be written as a sum of U-statistics of different order, and it can also be expressed as a sum of degenerate U-statistics if Hoeffding’s decomposition is applied for each U-statistic in this sum. Moreover, we shall show that the multiple integral of a function f of k variables with respect to a normalized empirical distribution can be decomposed to the linear combination of degenerate U-statistics with the same kernel functions f_{V} which appeared in Theorem 9.1 with relatively small coefficients. This is the content of the following Theorem 9.4. For the sake of a better understanding I shall reformulate it in a more explicit form in the special case k = 2 in Corollary 2 of Theorem 9.4 at the end of this chapter.

Theorem 9.4 (Decomposition of a multiple random integral with respect to a normalized empirical measure to a linear combination of degenerate Ustatistics). Let a non-atomic measure μ be given on a measurable space (X,X ) together with a sequence of independent, μ-distributed random variables ξ_{1},...,ξ_{n}. Take a function f(x_{1},...,x_{k}) of k variables integrable with respect to the product measure μ^{k} on the product space (X^{k},X^{k}), and consider the empirical distribution μ_{n} of the sequence ξ_{1},...,ξ_{n} introduced in (4.5) together with the k-fold random integral J_{n,k}(f) of the function f defined in (4.8). The identity

k!J_{n,k}(f) = ∑_{V ⊂{1,...,k}}C(n,k,|V |)n^{−|V |/2}|V |!I_{n,|V |}(f_{V})  (9.15)

holds with the set of (canonical) functions f_{V}(x_{j}, j ∈ V) (with respect to the measure μ) defined in formula (9.3) together with some appropriate real numbers C(n,k,p), 0 ≤ p ≤ k, where I_{n,|V |}(f_{V}) denotes the (degenerate) U-statistic of order |V | with the random variables ξ_{1},...,ξ_{n} and kernel function f_{V}. The constants C(n,k,p) in formula (9.15) satisfy the inequality |C(n,k,p)| ≤ C(k) for all n ≥ k and 0 ≤ p ≤ k with some constant C(k) < ∞ depending only on the order k of the integral J_{n,k}(f). The relations lim_{n→∞} C(n,k,p) = C(k,p) hold with some appropriate constant C(k,p) for all 1 ≤ p ≤ k, and C(n,k,k) = 1.

Remark. As the proof of Theorem 9.4 will show, the constant C(n,k,p) in formula (9.15) is a polynomial of order k − 1 of the argument n^{−1/2} with some coefficients depending on the parameters k and p. As a consequence, C(k,p) equals the constant term of this polynomial.

Theorems 8.1′ and 8.2 can be simply derived from Theorems 8.3 and 8.4 respectively with the help of Theorem 9.4. Indeed, to get Theorem 8.1′observe that formula (9.15) implies the inequality

P(|k!J_{n,k}(f)| > u) ≤ ∑_{V ⊂\{1,...,k\}}P ( n^{−|V |/2}||V |!I_{n,|V |}(f_{V})| > \frac{u}{2^{k}C(k)})  (9.16)

with a constant C(k) satisfying the inequality p!C(n,k,p) ≤ k!C(k) for all coefficients C(n,k,p), 1 ≤ p ≤ k, in (9.15). Hence Theorem 8.1′ follows from Theorem 8.3 and relations (9.5) and (9.6) in Theorem 9.2 by which the L_{2}-norm of the functions f_{V} is bounded by the L_{2}-norm of the function f and the L_{∞} -norm of f_{V} is bounded by 2^{|V |}-times the L_{∞} -norm or f. It is enough to estimate each term at the right-hand side of (9.16) by means of Theorem 8.3. It can be assumed that 2^{k}C(k) > 1. Let us first assume that also the inequality \frac{u}{2^{k}C(k)σ} ≥ 1 holds. In this case formula (8.4) in Theorem 8.1′can be obtained by means of the estimation of

each term at the right-hand side of (9.16). Observe that exp\{ −α ( \frac{u}{2^{k}C(k)σ})^{2/s}\} ≤

exp\{ −α ( \frac{u}{2^{k}C(k)σ})^{2/k}\} for all s ≤ k if \frac{u}{2^{k}C(k)σ} ≥ 1. In the other case, when

\frac{u}{2^{k}C(k)σ} ≤ 1, formula (8.4) holds again with a sufficiently large C > 0, because in this case its right-hand side of (8.4) is greater than 1.

Theorem 8.2 can be similarly derived from Theorem 8.4 by observing that relation (9.16) remains valid if |J_{n,k}(f)| is replaced by sup_{f∈F}|J_{n,k}(f)| and |I_{n,|V |}(f_{V})|

by sup_{f_{V}∈F_{V}} |I_{n,|V |}(f_{V})| in it, and we have the right to choose the constant M in for-

mula (8.7) of Theorem 8.2 sufficiently large. The only difference in the argument is that beside formulas (9.5) and (9.6) the last statement of Theorem 9.2 also has to be applied in this case. It tells that if F is an L_{2}-dense class of functions on a space (X^{k},X^{k}), then the classes of functions F_{V} = {2^{−|V |}f_{V}: f ∈ F } are also L_{2}-dense classes of functions for all V ⊂ {1,...,k} with the same exponent and parameter.

Before its proof I make some comments about the content of Theorem 9.4. The expression J_{n,k}(f) was defined as a k-fold random integral with respect to the signed measure μ_{n}− μ, where the diagonals were omitted from the domain of integration. Formula (9.15) expresses the random integral J_{n,k}(f) as a linear combination of degenerate U-statistics of different order. This is similar to the Hoeffding decomposition of the U-statistic I_{n,k}(f) to the linear combination of degenerate U-statistics defined with the same kernel functions f_{V}. The main difference between these two formulas is that in the expansion (9.15) of J_{n,k}(f) the terms I_{n,|V |}(f_{V}) appear with small coefficients C(n,k,|V |)|V |!\frac{1}{n^{|V |/2}}. As we shall see, E(C(n,k,|V |)|V |!\frac{1}{n^{|V |/2}}I_{n,V}(f_{V}))^{2} < K with a constant K < ∞ not depending on n for each set V ⊂ {1,...,k}. This can be so interpreted that the sum at the righthand side of (9.15) consists of such random variables C(n,k,|V |)|V |!n^{−|V |/2}I_{n,V}(f_{V}) which are of constant magnitude. The smallness of these coefficients is related to fact that in the definition of J_{n,k}(f) integration is taken with respect to the signed measure μ_{n}−μ instead of the empirical measure μ_{n}, which means some kind of normalization. On the other hand, these coefficients C(n,k,|V |) may have a non-zero limit as n → ∞ also for |V | < k. In particular, the expansion (9.15) may contain a constant term C(n,k,0) ≠ 0 such that even lim_{n→∞} C(n,k,0) ≠ 0. In such a case also the expected value EJ_{n,k}(f) does not equal zero. But even in such a case this expected value can be bounded by a finite number not depending on the sample size n. Next I show an example for a two-fold random integral J_{n,2}(f) such that 2EJ_{n,2}(f) = −1.

Let us choose a sequence of independent random variables ξ_{1},...,ξ_{n} with uniform distribution on the unit interval, let μ_{n} denote its empirical distribution, let f = f(x,y) denote the indicator function of the unit square, i.e. let f(x,y) = 1 if 0 ≤ x,y ≤ 1, and f(x,y) = 0 otherwise. Let us consider the random integral 2J_{n,2}(f) = n∫_{x≠y} f(x,y)(μ_{n}(dx) − dx)(μ_{n}(dy) − dy), and calculate its expected value 2EJ_{n,2}(f). By adjusting the diagonal x = y to the domain of integration and taking out the contribution obtained in this way we get that 2EJ_{n,2}(f) = nE(∫_{0}^{1}(μ_{n}(dx) − μ(dx))^{2}− n^{2}· \frac{1}{n^{2}} = −1. (The last term is the integral of the function f(x,y) on the diagonal x = y with respect to the product measure μ_{n}×μ_{n} which equals (μ_{n}− μ) × (μ_{n}− μ) on the diagonal.)

Now I turn to the proof of Theorem 9.4.

Proof of Theorem 9.4. Let us remark that for a canonical function g (with respect to the measure μ) of p variables the identity n^{−p/2} p!I_{n,p}(g) = p!J_{n,p}(g) holds. (At this point we also exploit that μ is a non-atomic measure, which implies that the identity ∫ g(x_{1},...,x_{p})μ(dx_{j}) = 0 for all 1 ≤ j ≤ p remains valid for arbitrary arguments x_{u}, 1 ≤ u ≤ p, u ≠ j, also if we omit finitely many points from the domain of integration.) This relation implies that if we calculate the (random) integral p!J_{n,p}(g) for a canonical function g we do not change the value of this integral by replacing the measures μ_{n}(dx_{j}) − μ(dx_{j}) by μ_{n}(dx_{j}) for all 1 ≤ j ≤ p. The integral we get after such a replacement equals p!n^{−1/2} I_{n,p}(g). Since all functions f_{V} appearing in formula (9.15) are canonical, the above relation between U-statistics and random integrals has the consequence that formula (9.15) can be rewritten in an equivalent form as

k!J_{n,k}(f) = ∑_{V ⊂{1,...,k}}C(n,k,|V |)|V |!J_{n,|V |}(f_{V}).  (9.17)

Here we use the convention that a constant c is a canonical function of order zero, and J_{n,0}(c) = c. We shall prove identity (9.17) by means of induction with respect to the order k of the integral k!J_{n,k}(f).

In the case k = 1 f_{{1}}(x) = f(x) − ∫ f(x)μ(dx), f_{∅} = ∫ f(x)μ(dx), and

J_{n,1}(f_{\{1\}}) = \sqrt{n}∫ (f(x) − f_{∅})(μ_{n}(dx) − μ(dx)) = J_{n,1}(f),

since ∫(μ_{n}(dx)−μ(dx)) = 0. Hence formula (9.17) holds for k = 1 with C(n,1,1) = 1 and C(n,1,0) = 0. For k = 0 relation (9.17) holds with C(n,0,0) = 1 if the convention f_{V} = f is applied for a function f of zero variables, i.e. if f is a constant function, and V = ∅. In the case k ≥ 2 we can write by taking the identity (9.3) formulated in the Hoeffding decomposition Theorem 9.1, integrating it with respect

to the product measure ∏_{j=1}^{k} (μ_{n}(dx_{j}) − μ(dx_{j})) and omitting the diagonals from the

domain of integration that

k!J_{n,k}(f) = k!J_{n,k}(f_{{1,...,k}}) + ∑_{Ṽ⊂{1,...,k},Ṽ≠{1,...,k}} k!J_{n,k}(fṼ).  (9.18)

Observe that in the case Ṽ⊂ {1,...,k}, Ṽ ≠ {1,...,k} the function fṼ has strictly less than k arguments, while the terms J_{n,k}(fṼ) at the right-hand side of (9.18) are random integrals of order k. We can rewrite these k-fold integrals as the linear combinations of random integrals of smaller multiplicity with the help of the following

Lemma 9.5. Let us take a measure space (X,X ,μ) with a non-atomic probability measure μ and an integrable function f(x_{1},...,x_{k−1}) on its k − 1-fold product, (X^{k−1},X^{k−1},μ^{k−1}), k ≥ 2. Let us also take the operator (P_{l}f)(x_{j}, j ∈ {1,...,k − 1} \ {l}) = ∫ f(x_{1},...,x_{k−1})μ(dx_{l}) for all 1 ≤ l ≤ k − 1. Let us consider the function f also as a function f(x_{1},...,x_{k}) of k variables which does not depend on its last coordinate x_{k}. The identity

k!J_{n,k}(f) = −n^{−1/2} (k − 1) · (k − 1)!J_{n,k−1}(f) − ∑_{l=1}^{k−1}(k − 2)!J_{n,k−2}(P_{l}f)  (9.19)

holds. The function P_{l}f has arguments with indices j ∈ {1,...,k − 1} \ {l}, and in the term J_{n,k−2}(P_{l}f) in (9.19) we take integration with respect to

n^{(k−2)/2} ∏_{j∈{1,...,k−1}\{l}} (dμ_{n}(x_{j}) − μ(dx_{j})).

Proof of Lemma 9.5. Formula (9.19) is equivalent to the identity

∫ ′f(x_{1},...,x_{k−1})(μ_{n}(dx_{1}) − μ(dx_{1}))...(μ_{n}(dx_{k}) − μ(dx_{k}))

= −\frac{k − 1}{n} ∫ ′f(x_{1},...,x_{k−1})∏_{s=1}^{k−1}(μ_{n}(dx_{s}) − μ(dx_{s}))

−\frac{1}{n} ∑_{l=1}^{k−1} ∫ ′[∫ f(x_{1},...,x_{k−1})μ(dx_{l})]

∏_{1≤s≤k−1,s≠l} (μ_{n}(dx_{s}) − μ(dx_{s})).

The expressions at the two sides of this identity are linear combinations of terms of the form

∫ ′f(x_{1},...,x_{k−1}) ∏_{l∈V}μ_{n}(dx_{l}) ∏_{l∈{1,...,k−1}\V} μ(dx_{l})

with V ⊂ {1,...,k−1}. A term of this form with |V | = p at the left-hand side of this identity has coefficient (−1)^{k−p} (1 − \frac{n−p}{n} ) = (−1)^{k−p} \frac{p}{n}. To see this let us calculate the integral

∫ ′f(x_{1},...,x_{k−1}) ∏_{l∈V}μ_{n}(dx_{l}) ∏_{l∈{1,...,k−1}\V} μ(dx_{l})(μ_{n}(dx_{k}) − μ(dx_{k}))

by successive integration, and integrating with respect to the variable x_{k} in the last step. Then we integrate a constant function in the last step. Beside this, since the (random) measure μ_{n} is concentrated in n points with weights \frac{1}{n}, and in the integration ∫′ we omit the diagonals from the domain of integration, we integrate with respect to a measure with total mass \frac{n−p}{n} when we are integrating with respect to μ_{n}(dx_{k}). On the other hand, the first term at the right-hand side of the identity we want to prove has coefficient (−1)^{(k−p)} \frac{k−1}{n} and the second term has coefficient (−1)^{(k−p−1)} \frac{k−1−p}{n} . Lemma 9.5 follows from these calculations.

Lemma 9.5 was proved by means of elementary calculations. One may ask how its form can be found. It may be worth observing that there are some diagram formulas that play an important role in some subsequent proofs, and they also supply the identity formulated in Lemma 9.5 together with its proof.

In these diagram formulas the product of some random integrals or U-statistics are expressed by means of the sum of appropriately defined random integrals or Ustatistics. In the subsequent part of this lecture note I discuss the diagram formula for Wiener–Itô integrals and U-statistics. I shall also mention that there is a diagram formula for the product of multiple integrals with respect to a normalized empirical distribution, and I shall indicate what its form looks like. An explicit formulation and proof of this result can be found in [35]. Lemma 9.5 can be obtained as a special case of this formula.

To get Lemma 9.5 with the help of the diagram formula take the function e(x) ≡ 1 on the space (X,X ). Then we have J_{n,1}(e) = 0 with probability one. Given a function f(x_{1},...,x_{k−1}) write up the identity J_{n,k−1}(f)J_{n,1}(e) = 0 with probability one, and rewrite its left-hand side by means of the diagram formula. The identity we get in such a way agrees with Lemma 9.5. One of the terms in this identity is k!J_{n,k}(f) which appears as the integral of the function f̄(x_{1},...,x_{k}) = f(x_{1},...,x_{k−1})e(x_{k}), and writing up all terms we get the desired formula.

Now I return to the proof of Theorem 9.4.

Completion of the proof of Theorem 9.4 with the help of Lemma 9.5. We shall prove the following slightly more general version of (9.17). If f(x_{j}, j ∈ V) is an integrable function with arguments indexed by a set V ⊂ {1,...,k}, then

k!J_{n,k}(f) = ∑_{V̄⊂V} C(n,k,|V̄|,|V |)|V̄|!J_{n,|V̄|}(fV̄)  (9.20)

with some coefficients C(n,k,p,q), 0 ≤ p ≤ q ≤ k such that |C(n,k,p,q)| ≤ C_{k} < ∞ for all arguments n and 0 ≤ p ≤ q ≤ k, the limit lim_{n→∞} C(n,k,p,q) = C(k,p,q) exists, and C(n,k,k,k) = 1.

At the left-hand side of formulas (9.20) and (9.17) the same integral J_{n,k}(f) of order k of a function f with less than or equal to k arguments is taken. (We define this integral by redefining its kernel function f as a function of k arguments by means of the introduction of some additional fictive coordinates.) At the right-hand side of these formulas the same canonical functions fV̄, V̄ ⊂ {1,...,k}, appear. They were introduced in the Hoeffding decomposition (9.3). But in (9.20) we take the integrals of the functions fV̄ only with respect to their ‘real’ coordinates with indices l ∈ V̄⊂ V. For the sake of simpler notations first we restrict our attention to the case V = {1,...,q} with some 0 ≤ q ≤ k. (Actually, it can be seen with the help of the subsequent proof that we can choose C(n,k,p,q) = C(n,k,p) with the constant C(n,k,p) appearing in (9.15) or (9.17).)

We shall prove (9.20) by means of induction with respect to k. This relation holds for k = 0, and to prove it for k = 1 we still we have to check that it also holds in the special case when f is a function of zero variable, i.e. if it is a constant, and V = ∅. But relation (9.20) holds in this case with C(n,1,0,0) = 0, since J_{n,1}(f) = 0 if f is a variable of zero arguments, i.e. if it is a constant.

We shall prove relation (9.20) for general parameter k with the help of formula (9.18), Lemma 9.5 and formula (9.3) in the Hoeffding decomposition which gives the definition of the functions fṼ appearing in (9.18). I formulate a formally more general result than relation (9.19) which follows from Lemma 9.5 if we reindex the variables of the function f considered in it. I formulate this result, because this will be applied in our calculations.

Let us take a number p ∈ {1,...,k}, k ≥ 2, and a function f(x_{j}, j ∈ {1,...,k} \ {p}), integrable with respect to the appropriate direct product of the measure μ together with the functions P_{l}(f) = P_{l}(f)(x_{j}, j ∈ {1,...,k} \ {l,p}) for all l ∈ {1,...,k} \ {p} that we get by integrating the function f with respect to the measure μ(dx_{l}). The following modified version of (9.19) holds in this case.

k!J_{n,k}(f) = −n^{−1/2} (k − 1)!(k − 1)J_{n,k−1}(f) − ∑_{l∈{1,...,k}\{p}}(k − 2)!J_{n,k−2}(P_{l}f)

(9.21)

where J_{n,k−1}(f) is the integral of the function f with respect to the measure

n^{(k−1)/2} ∏_{j∈{1,...,k}\{p}}((μ_{n}(dx_{j}) − μ(dx_{j}))

and J_{n,k−2}(P_{l}f) is the integral of the function P_{l}f with respect to the measure

n^{(k−2)/2} ∏_{j∈{1,...,k}\{p,l}}((μ_{n}(dx_{j}) − μ(dx_{j})).

(Naturally the diagonals are omitted from the domain of integration.)

First we prove (9.20) in the case V = {1,...,k}. We rewrite k!J_{n,k}(f) by means of (9.18) as a sum of random integrals of order k with kernel functions fṼ, Ṽ ⊂ {1,...,k}. We rewrite each term k!J_{n,k}(fṼ) with Ṽ⊂ {1,...,k}, Ṽ ≠ {1,...,k} in this sum (i.e. we do not consider the integral k!J_{n,k}(f_{{1,...,k}})) as a linear combination of multiple random integrals of the form J_{n,k−1}(fṼ) and J_{n,k−2}(P_{l}fṼ) of order k − 1 and k − 2 respectively with the help of identity (9.21). Then we can apply formula (9.20) for them because of our inductive hypothesis. Let us understand what kind of kernel functions appear in the integrals we get in such a way. If V̄⊂ Ṽ, then (fṼ)V̄ = fV̄ by formula (9.3). On the other hand, P_{l}fṼ = f_{Ṽ\{l}}, and in the expansion of J_{n,k}(P_{l}fṼ)) by means of (9.20) we get a linear combination of random integrals J_{n,|V̄|}(fV̄) with V̄⊂ Ṽ\{l}. By applying all these identities, summing them up, adding to them the term J_{n,k}(f_{{1,...,k}}) and applying formula (9.21) we get because of our inductive assumptions a representation k!J_{n,k}(f) = ∑_{V̄⊂V} C(n,k,V̄)|V̄|!J_{n,|V̄|}(fV̄)

(where V = {1,...,k}) of the random integral k!J_{n,k}(f) with such coefficients C(n,k,V̄) for which |C(n,k,V̄)| ≤ C(k) and the limit C(,V̄) = lim_{n→∞} C(n,k,V̄) exists. We still have to show that these coefficients can be chosen in such a way that C(n,k,V̄) = C(n,k,|V̄|), i.e. C(n,k,V̄_{1}) = C(n,k,V̄_{2}) if |V̄_{1}| = |V̄_{2}|.

Given a set Ṽ ⊂ {1,...,k}, Ṽ ≠ {1,...,k}, let us express the random integrals J_{n,k−1}(fṼ) and J_{n,k−2}(P_{l}fṼ) for all p ∈ {1,...,k} \ Ṽ in the above way, and write J_{n,k}(fṼ) and J_{n,k}(P_{l}fṼ) as the average of these sums. Working with these expressions for J_{n,k}(fṼ) and J_{n,k}(P_{l}fṼ) it can be seen that our inductive assumption also holds with such coefficients C(n,k,V̄) for which C(n,k,V̄_{1}) = C(n,k,V̄_{2}) if |V̄_{1}| = |V̄_{2}|.

In the next step let us consider the case when f = f(x_{j}, j ∈ V) with a set V = {1,...,q} such that 0 ≤ q < k. I claim that in this case the identity fṼ ≡ 0 holds for those sets Ṽ ⊂ {1,...,k} for which Ṽ∩ {q + 1,...,k} ≠ ∅, and as a consequence J_{n,k}(fṼ) = 0 with probability 1 for such sets Ṽ. First I show that relation (9.20) can be proved in the present case with the help of this relation similarly to the previous case.

In the present case formula (9.18) has the form k!J_{n,k}(f) = ∑_{Ṽ⊂V} k!J_{n,k}(fṼ), and

we can express each term k!J_{n,k}(fṼ), Ṽ⊂ V, in this sum by means of formula (9.21) by choosing fṼ as the function f and an integer p such that q + 1 ≤ p ≤ k (i.e. p ∈ {1,...,k}\V) in it. In such a way we can write k!J_{n,k}(f) as the linear combination of random integrals of the form (k − 1)!J_{n,k−1}(fṼ) and (k − 2)!J_{n,k−2}(P_{l}fṼ) = (k−2)!J_{n,k−2}(f_{Ṽ\{l}}) with some sets Ṽ⊂ V and numbers l ∈ {1,...,k}\{p}, where we took some number p such that q + 1 ≤ p ≤ k. Then we can apply relation (9.20) for parameters k − 1 and k − 2 by our inductive hypothesis, and this enables us to write J_{n,k}(f) as the linear combination of random integrals |V̄|!J_{n,|V̄|}(fV̄) with sets V̄⊂ V. Moreover, it can be seen similarly to the previous case (by writing the above identities for all p ∈ {1,...,k} \ Ṽ and taking their average) that the coefficients in this linear combination can be chosen in such a way as it was demanded in formula (9.20).

To prove that fṼ ≡ 0 if Ṽ∩ {q + 1,...,k} ≠ ∅ and f = f(x_{1},...,x_{k}) is the extension of a function f = f(x_{j}, j ∈ {1,...,q}) to X^{k} with the help of some ‘fictive’ coordinates take a number r ∈ Ṽ∩{q+1,...,k}, observe that P_{r}f = f and Q_{r}f ≡ 0 for the operators P_{r} and Q_{r} defined in (9.1) and (9.2), since r ∉ V = {1,...,q}. The definition of the function fṼ is given in formula (9.3). Observe that in the present case the operator Q_{r} and not the operator P_{r} appears in the formula defining fṼ. Hence formula (9.3) and the exchangeability of the operators P_{j} and Q_{j′} imply that fṼ ≡ 0.

Formula (9.20) in the general case simply follows from the already proved results by a reindexation of the variables of the function f. Since (9.17) is a special case of (9.20) Theorem 9.4 is proved.

Two corollaries of Theorem 9.4 will be formulated. The first one explains the content of conditions (8.2) and (8.6) in Theorems 8.1—8.4.

Corollary 1 of Theorem 9.4. If I_{n,k}(f) is a degenerate U-statistic of order k with some kernel function f, then

E ( n^{−k/2}I_{n,k}(f))^{2}

= \frac{n(n − 1)··· (n − k + 1)}{k!n^{k}} ∫ Symf^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k})

≤ \frac{1}{k!} ∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}),  (9.22)

where μ is the distribution of the random variables taking part in the definition of the U-statistic I_{n,k}(f), and Symf is the symmetrization of the function f. The k-fold multiple random integral J_{n,k}(f) with an arbitrary square integrable kernel function f satisfies the inequality

EJ_{n,k}(f)^{2} ≤ C̄(k)∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k})

with some constant C̄(k) depending only on the order k of the integral J_{n,k}(f).

Proof of Corollary 1 of Theorem 9.4. The identity

E(n^{−k/2}I_{n,k}(f))^{2} = \frac{1}{(k!)^{2}n^{k}}∑ ′Ef(ξ_{l_{1}},...,ξ_{l_{k}})f(ξ_{l′_{1}},...,ξ_{l′_{k}})  (9.23)

holds, where the prime in ∑′ means that summation is taken for such pairs of ktuples (l_{1},...,l_{k}), (l′_{1},...,l′_{k}), 1 ≤ l_{j},l′_{j} ≤ n, for which l_{j} ≠ l_{j′} and l′_{j} ≠ l′_{j′} if j ≠ j′. On the other hand, the degeneracy of the U-statistic I_{n,k}(f) implies that

Ef(ξ_{l_{1}},...,ξ_{l_{k}})f(ξ_{l′_{1}},...,ξ_{l′_{k}}) = 0

if the two sets {l_{1},...,l_{k}} and {l′_{1},...,l′_{k}} differ. This can be seen by taking such an index l_{j} from the first k-tuple which does not appear in the second one, and by observing that the conditional expectation of the product we consider equals zero by the degeneracy condition of the U-statistic under the condition that the value of all random variables except that of ξ_{l_{j}} is fixed in this product. On the other hand,

Ef(ξ_{l_{1}},...,ξ_{l_{k}})f(ξ_{l′_{1}},...,ξ_{l′_{k}})
= ∫ f(x_{1},...,x_{k})f(x_{π(1)},...,x_{π(k)})μ(dx_{1})...μ(dx_{k})

if (l′_{1},...,l′_{k}) = (π(l_{1}),...,π(l_{k})) with some (π(1),...,π(k)) ∈ Π_{k}, where Π_{k} denotes the set of all permutations of the set {1,...,k}. By summing up the above identities for all pairs (l_{1},...,l_{k}) and (l′_{1},...,l′_{k}) and by applying formula (9.23) we get the identity at the left-hand side of formula (9.22). The second relation in (9.22) is obvious.

The bound for J_{n,k}(f) follows from Theorem 9.4, formula (9.5) in Theorem 9.2 by which the L_{2}-norm of the functions f_{V} is not greater than the L_{2}-norm of the function f and the bound that formula (9.22) yields for the second moment of the degenerate U-statistics n^{−|V |/2}I_{n,|V |}(f_{V}) appearing in the expansion (9.15).

In Corollary 2 the decomposition (9.15) of a random integral J_{n,2}(f) of order 2 is described in an explicit form. This result follows from the proof of Theorem 9.4.

Corollary 2 of Theorem 9.4. Let the random integral J_{n,2}(f) satisfy the conditions of Theorem 9.4. In this case formula (9.15) can be written in the following explicit form:

2J_{n,2}(f) = \frac{2}{n}I_{n,2}(f_{\{1,2\}}) − \frac{1}{n}I_{n,1}(f_{\{1\}}) − \frac{1}{n}I_{n,1}(f_{\{2\}}) − f_{∅}

with the functions

f_{{1,2}}(x,y) = f(x,y) − ∫ f(x,y)μ(dx) − ∫ f(x,y)μ(dy)
+∫ f(x,y)μ(dx)μ(dy),
f_{{1}}(x) = ∫ f(x,y)μ(dy) − ∫ f(x,y)μ(dx)μ(dy),
f_{{2}}(y) = ∫ f(x,y)μ(dx) − ∫ f(x,y)μ(dx)μ(dy), and
f_{∅} = ∫ f(x,y)μ(dx)μ(dy).

Corollary 2 of Theorem 9.4 states that in the case k = 2 formula (9.15) holds with C(n,2,2) = 1, C(n,2,1) = −\frac{1}{\sqrt{n}} and C(n,2,0) = −1.

Multiple Wiener–Itô integrals and their properties

In this chapter I present the definition of multiple Wiener–Itô integrals and some of their most important properties needed in the proof of the results formulated in Chapter 8. Wiener–Itô integrals provide a useful tool to handle non-linear functionals of Gaussian processes. To define them first I introduce the notion of the white noise with some reference measure. Then I define the multiple Wiener–Itô integrals with respect to a white noise with some non-atomic reference measure. A most important result in the theory of multiple Wiener–Itô integrals is the so-called diagram formula presented in Theorem 10.2A. This enables us to rewrite the product of two Wiener–Itô integrals in the form of a sum of Wiener–Itô integrals. The proof of the diagram formula is given in Appendix B. This result will be generalized in Theorem 10.2 to a formula about the representation of the product of finitely many Wiener–Itô integrals as a sum of Wiener–Itô integrals. As a consequence of this result we get a formula about the expectation of products Wiener–Itô integrals. This formula will play an important role in some of our later considerations.

Another interesting result about Wiener-Itô integrals, formulated at the end of this chapter in Theorem 10.5 states that the class of random variables which can be written in the form of a sum of Wiener–Itô integrals of different order is sufficiently rich. All random variables with finite second moment which are measurable with respect to the σ-algebra generated by the (Gaussian) random variables appearing in the underlying white noise in the construction of multiple Wiener–Itô integrals can be written in such a form.

I shall also give a heuristic explanation of the diagram formula which may indicate why it has the form appearing in Theorem 10.2A. It also helps to find its analogue for (random) integrals with respect to the product of normalized empirical measures. Such a result will be useful later. A simple and useful consequence of Theorem 10.2A about the representation of the product of finitely many Wiener–Itô integrals in the form of a sum of Wiener–Itô integrals will be formulated in Theorem 10.2. This result will be also called the diagram formula. It has an important corollary about the calculation of the moments of Wiener–Itô integrals. Theorem 8.5 will be proved with the help of this corollary.

I shall give the proof of two other results about Wiener–Itô integrals in Appendix C. The first one, Theorem 10.3, is called Itô’s formula for Wiener–Itô integrals, and it explains the relation between multiple Wiener-Itô integrals and Hermite polynomials of Gaussian random variables. This result is a relatively simple consequence of the diagram formula and some basic recursive relations about Hermite polynomials.

The other result proved in Appendix C, Theorem 10.4, is a limit theorem about a sequence of appropriately normalized degenerate U-statistics. Here the limit is presented in the form of a multiple Wiener–Itô integral. This result is interesting for us, because it helps to compare Theorems 8.3 and 8.1 with their one-variate counterpart, Bernstein’s inequality. In the one-variate case Bernstein’s inequality provides a comparison between the tail distribution of sums of independent random variables and the tail of the standard normal distribution. The normal distribution appears here in a natural way as the limit in the central limit theorem.

Theorem 8.3 yields a similar result about degenerate U-statistics. The upper bound for the tail-distribution of a degenerate U-statistic given in Theorem 8.3 or in its Corollary is similar to the bound of Theorem 8.5 about the tail-distribution of a Wiener–Itô integral with the same kernel function. On the other hand, by Theorem 10.4 this Wiener–Itô integral also appears as the limit of degenerate U-statistics with the same kernel function. This shows some similarity between Theorem 8.3, and its one-variate version, the Bernstein inequality. Theorem 8.1 which is an estimate of multiple integrals with respect to a normalized empirical distribution also has a similar interpretation.

My Lecture Note [32] contains a rather detailed description of Wiener–Itô integrals. But in that work the emphasis was put on the study of a slightly different version of it. The original version of this integral introduced in [26] was only briefly discussed there, and not all details were worked out. In particular, the diagram formula needed in this work was formulated and proved only for modified Wiener–Itô integrals. I shall discuss the difference between these random integrals together with the question why a modified version of Wiener–Itô integrals was studied in [32] at the end of this chapter.

To define multiple Wiener–Itô integrals first I introduce the notion of white noise.

Definition of a white noise with some reference measure. Let us have a σ-finite measure μ on a measurable space (X,X ). A white noise with reference measure μ is a Gaussian random field μ_{W}= {μ_{W}(A): A ∈ X , μ(A) < ∞}, i.e. a set of jointly Gaussian random variables indexed by the above sets A, which satisfies the relations Eμ_{W}(A) = 0 and Eμ_{W}(A)μ_{W}(B) = μ(A ∩ B) for all A,B ∈ X such that μ(A) < ∞ and μ(B) < ∞.

I make some comments about this definition.

Remark: In the definition of a white noise sometimes also the property μ_{W}(A ∪ B) = μ_{W}(A) + μ_{W}(B) with probability 1 if A ∩ B = ∅, and μ(A) < ∞, μ(B) < ∞ is mentioned. But this condition can be omitted, because it follows from the remaining properties of the white noise. Indeed, simple calculation shows that E(μ_{W}(A ∪ B) − μ_{W}(A) − μ_{W}(B))^{2} = 0 if A ∩ B = ∅, hence μ_{W}(A ∪ B) − μ_{W}(A) − μ_{W}(B) = 0 with probability 1 in this case. It can also be observed that if some sets A_{1},...,A_{k} ∈ X , μ(A_{j}) < ∞, 1 ≤ j ≤ k, are disjoint, then the random variables μ_{W}(A_{j}), 1 ≤ j ≤ k, are independent because of the uncorrelatedness of these jointly Gaussian random variables.

It is not difficult to see that for an arbitrary reference measure μ on a space (X,X ) a white noise μ_{W} with this reference measure really exists. This follows simply from Kolmogorov’s fundamental theorem, by which if the finite dimensional distributions of a random field are defined in a consistent way, then there exists a random field with these finite dimensional distributions.

Now I turn to the definition of multiple Wiener–Itô integrals with respect to a white noise with some reference measure μ. First I introduce the class of functions whose Wiener–Itô integrals with respect to a white noise μ_{W} with a non-atomic reference measure μ will be defined.

Let us consider a measurable space (X,X ), a σ-finite, non-atomic measure μ on it and a white noise μ_{W} on (X,X ) with reference measure μ. Let us define the classes of functions H_{μ,k}, k = 1,2,..., consisting of functions of k variables on (X,X ) by the formula

H_{μ,k} = { f(x_{1},...,x_{k}): f(x_{1},...,x_{k}) is an X^{k} measurable, real valued
function on X^{k}, and∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...,μ(dx_{k}) < ∞} . (10.1)

We shall call a σ-finite measure μ on a measurable space (X,X ) non-atomic if for all sets A ∈ X such that μ(A) < ∞ and all numbers ε > 0 there is a finite

partition A = ⋃_{s=1}^{N} B_{s} of the set A with the property μ(B_{s}) < ε for all 1 ≤ s ≤ N.

There is a formally weaker definition of a non-atomic measures by which a σ-finite measure μ is non-atomic if for all measurable sets A such that 0 < μ(A) < ∞ there is a measurable set B ⊂ A with the property 0 < μ(B) < μ(A). But these two definitions of non-atomic measures are actually equivalent, although this equivalence is not trivial. I do not discuss this problem here, since it is a little bit outside from the direction of the present work. In our further considerations we shall work with the first definition of non-atomic measures.

I would also remark that non-atomic measures behave not completely so, as our first heuristic feeling would suggest. It is true that if μ is a non-atomic measure, then μ({a}) = 0 for all one-point sets {a}. But the reverse statement does not hold. There are (in some sense degenerate) measures μ for which each one-point set has zero μ measure, and which are nevertheless not non-atomic. I omit the discussion of this question.

The k-fold Wiener-Itô integrals of the functions f ∈ H_{μ,k} with respect to the white noise μ_{W} will be defined in a rather standard way. First they will be defined for some simple functions, called elementary functions, then it will be shown that the integral for these elementary functions has an L_{2} contraction property which makes possible to extend it to the class of all functions in H_{μ,k}.

Let us first introduce the following class of elementary functions H̄_{μ,k} of k variables. A function f(x_{1},...,x_{k}) on (X^{k},X^{k}) belongs to H̄_{μ,k} if there exist finitely many disjoint measurable subsets A_{1},...,A_{M}, 1 ≤ M < ∞, of the set X with finite μ-measure (i.e. A_{j} ∩ A_{j′} = ∅ if j ≠ j′, and μ(A_{j}) < ∞ for all 1 ≤ j ≤ M) such that the function f has the form

f(x_{1},...,x_{k}) = \{ \begin{matrix}c(j_{1},...,j_{k}) if (x_{1},...,x_{k}) ∈ A_{j_{1}} × ··· × A_{j_{k}} with \\ some indices (j_{1},...,j_{k}), 1 ≤ j_{s} ≤ M, 1 ≤ s ≤ k, \\ such that all numbers j_{1},...,j_{k} are different \\ 0 if (x_{1},...,x_{k}) ∉ ⋃_{(j_{1},...,j_{k}): 1≤j_{s}≤M, 1≤s≤k, and all j_{1},...,j_{k} are different.}A_{j_{1}} × ··· × A_{j_{k}} \end{matrix}

(10.2)

with some real numbers c(j_{1},...,j_{k}), 1 ≤ j_{s} ≤ M, 1 ≤ s ≤ k, defined for such arguments for which j_{1},...,j_{k} are different numbers. This means that the function f is constant on all k-dimensional rectangles A_{j_{1}} × ···× A_{j_{k}} with different, nonintersecting edges, and it equals zero on the complementary set of the union of these rectangles. The property that the support of the function f is on the union of rectangles with non-intersecting edges is sometimes interpreted so that the diagonals are omitted from the domain of integration of Wiener–Itô integrals.

The Wiener-Itô integral of an elementary function f(x_{1},...,x_{k}) of the form (10.2) with respect to a white noise μ_{W} with the (non-atomic) reference measure μ is defined by the formula

∫ f(x_{1},...,x_{k})μ_{W}(dx_{1})...μ_{W}(dx_{k})
= ∑_{1≤j_{s}≤M, 1≤s≤k all j_{1},...,j_{k} are different} c(j_{1},...,j_{k})μ_{W}(A_{j_{1}})··· μ_{W}(A_{j_{k}}).  (10.3)

(The representation of the function f in (10.2) is not unique, the sets A_{j} can be divided into smaller disjoint sets, but the Wiener–Itô integral defined in (10.3) does not depend on the representation of the function f. This can be seen with the help of the additivity property μ_{W}(A ∪ B) = μ_{W}(A) + μ_{W}(B) if A ∩ B = ∅ of the white noise μ_{W}.) The notation

Z_{μ,k}(f) = \frac{1}{k!} ∫ f(x_{1},...,x_{k})μ_{W}(dx_{1})...μ_{W}(dx_{k}),  (10.4)

will be used in the sequel, and the expression Z_{μ,k}(f) will be called the normalized Wiener–Itô integral of the function f. Such a terminology will be applied also for the Wiener–Itô integrals of all functions f ∈ H_{μ,k} to be defined later.

If f is an elementary function in H̄_{μ,k} defined in (10.2), then its normalized Wiener–Itô integral defined in (10.3) and (10.4) satisfies the relations

Ek!Z_{μ,k}(f) = 0,
E(k!Z_{μ,k}(f))^{2} =  ∑_{(j_{1},...,j_{k}): 1≤j_{s}≤M, 1≤s≤k, and all j_{1},...,j_{k} are different.} ∑_{π∈Π_{k}} c(j_{1},...,j_{k})c(j_{π(1)},...,j_{π(k)})

Eμ_{W}(A_{j_{1}})··· μ_{W}(A_{j_{k}})μ_{W}(A_{j_{π(1)}} )··· μ_{W}(A_{j_{π(k)}})
= k!∫ Symf^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k})
≤ k!∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}),  (10.5)

with Symf(x_{1},...,x_{k}) = \frac{1}{k!} ∑_{π∈Π_{k}} f(x_{π(1)},...,x_{π(k)}), where Π_{k} denotes the set of all

permutations π = {π(1),...,π(k)} of the set {1,...,k}.

The identities written down in (10.5) can be simply checked. The first relation follows from the identity Eμ_{W}(A_{j_{1}})··· μ_{W}(A_{j_{k}}) = 0 for disjoint sets A_{j_{1}},...,A_{j_{k}}, which holds, since the expectation of the product of independent random variables with zero expectation is taken. The second identity follows similarly from the identity

Eμ_{W}(A_{j_{1}})··· μ_{W}(A_{j_{k}})μ_{W}(A_{j′_{1}})··· μ_{W}(A_{j′_{k}}) = 0

if the sets of indices {j_{1},...,j_{k}} and {j′_{1},...,j′_{k}} are different,

Eμ_{W}(A_{j_{1}})··· μ_{W}(A_{j_{k}})μ_{W}(A_{j′_{1}})··· μ_{W}(A_{j′_{k}}) = μ(A_{j_{1}})··· μ(A_{j_{k}})

if {j_{1},...,j_{k}} = {j′_{1},...,j′_{k}} i.e. ifj′_{1} = j_{π(1)},...,j′_{k} = j_{π(k)} with some permutation π ∈ Π_{k},

which holds because of the facts that the μ_{W}measure of disjoint sets are independent with expectation zero, and Eμ_{W}(A)^{2} = μ(A). The remaining relations in (10.5) can be simply checked.

It is not difficult to check that

EZ_{μ,k}(f)Z_{μ,k′}(g) = 0  (10.6)

for all functions f ∈ H̄_{μ,k} and g ∈ H̄_{μ,k′} if k ≠ k′, and

Z_{μ,k}(f) = Z_{μ,k}(Symf)  (10.7)

for all functions f ∈ H̄_{μ,k} .

The definition of Wiener–Itô integrals can be extended to general functions f ∈ H_{μ,k} with the help of formula (10.5). To carry out this extension we still have to know that the class of functions H̄_{μ,k} is a dense subset of the class H_{μ,k} in the Hilbert space L_{2}(X^{k},X^{k},μ^{k}), where μ^{k} is the k-th power of the reference measure μ of the white noise μ_{W}. I briefly explain how this property of H̄_{μ,k} can be proved. The non-atomic property of the measure μ is exploited at this point.

To prove this statement it is enough to show that the indicator function of any product set A_{1}×··· ×A_{k} such that μ(A_{j}) < ∞, 1 ≤ j ≤ k, but the sets A_{1},...,A_{k} may be non-disjoint is in the L_{2}(μ^{k}) closure of H̄_{μ,k} . In the proof of this statement it will be exploited that since μ is a non-atomic measure, the sets A_{j} can be represented for all ε > 0 and 1 ≤ j ≤ k as a finite union A_{j} = ⋃_{s} B_{j,s} of disjoint sets B_{j,s} with the

property μ(B_{j,s}) < ε. By means of these relations the product A_{1}× ··· × A_{k} can be written in the form

A_{1}× ··· × A_{k} = ⋃_{s_{1},...,s_{k}}B_{1,s_{1}} × ··· × B_{k,s_{k}}  (10.8)

with some sets B_{j,s_{j}} such that μ(B_{j,s_{j}} ) < ε for all sets in this union. Moreover, we may assume, by refining the partitions of the sets A_{j} if this is necessary that any two sets B_{j,s_{j}} and B_{j′,s′_{j′}} in this representation are either disjoint, or they agree. Take such a representation of A_{1} × ··· × A_{k}, and consider the set we obtain by omitting those products B_{1,s_{1}} × ··· × B_{k,s_{k}} from the union at the right-hand side of (10.8) for which B_{i,s_{i}} = B_{j,s_{j}} for some 1 ≤ i < j ≤ k. The indicator function of the remaining set is in the class H̄_{μ,k} . Hence it is enough to show that the distance between this indicator function and the indicator function of the set A_{1} × ··· × A_{k} is less than const.ε in the L_{2}(μ^{k}) norm with some const. which may depend on the sets A_{1},...,A_{k}, but not on ε. Indeed, by letting ε tend to zero we get from this relation that the indicator function of the set A_{1}× A_{2}× ··· × A_{k} is in the closure of H̄_{μ,k} in the L_{2}(μ^{k}) norm.

Hence to prove the desired property of H̄_{μ,k} it is enough to prove the following statement. Take the representation (10.8) of A_{1}× ··· × A_{k} (which depends on ε) and fix an arbitrary pair of integers i and j such that 1 ≤ i < j ≤ k. Then the sum of the measures μ^{k}(B_{1,s_{1}} ×··· ×B_{k,s_{k}}) of those sets B_{1,s_{1}} ×··· ×B_{k,s_{k}} at the right-hand side of (10.8) for which B_{i,s_{i}} = B_{j,s_{j}} is less than const.ε. To prove this estimate observe that the μ^{k} measure of such a set can be bounded by the μ^{k−1} measure of the set we obtain by omitting the i-th term from the product defining it in the following way:

μ^{k}(B_{1,s_{1}} × ··· × B_{k,s_{k}}) ≤ εμ^{k−1}(B_{1,s_{1}} × ··· × B_{i−1,s_{i−1}} × B_{i+1,s_{i+1}} × ··· × B_{k,s_{k}}).

Let us sum up this inequality for all such sets B_{1,s_{1}} ×··· ×B_{k,s_{k}} at the right-hand side of (10.8) for which B_{i,s_{i}} = B_{j,s_{j}} . The left-hand side of the inequality we get in such a way equals the quantity we want to estimate. The expression at its right-hand side is less than ε ∏_{1≤s≤k,s≠i} μ(A_{s}), since ε-times the μ^{k−1} measure of such disjoint sets

are summed up in it which are contained in the set A_{1}×··· ×A_{i−1} ×A_{i+1} ×··· ×A_{k}. In such a way we get the estimate we wanted to prove.

Knowing that H̄_{μ,k} is a dense subset of H_{μ,k} in L_{2}(μ^{k}) norm we can finish the definition of k-fold Wiener–itô integrals in the standard way. Given any function f ∈ H_{μ,k} a sequence of functions f_{n} ∈ H̄_{μ,k} , n = 1,2,..., can be defined in such a way that

∫ |f(x_{1},...,x_{k}) − f_{n}(x_{1},...,x_{k})|^{2}μ(dx_{1})...μ(dx_{k}) → 0 as n → ∞.

By relation (10.5) the already defined Wiener–Itô integrals Z_{μ,k}(f_{n}) of the functions f_{n}, n = 1,2,..., constitute a Cauchy sequence in the space of the square integrable random variables living on the probability space, where the white noise is given. (Observe that the difference of two functions from the class H̄_{μ,k} also belongs to this class.) Hence the limit lim_{n→∞} Z_{μ,k}(f_{n}) exists in L_{2} norm, and this limit can be defined as the normalized Wiener–Itô integral Z_{μ,k}(f) of the function f. The definition of this limit does not depend on the choice of the approximating functions f_{n}, hence it is meaningful. It can be seen that relations (10.5) and (10.6) remain valid for all functions f ∈ H_{μ,k}. The following Theorem 10.1 describes the properties of multiple Wiener–Itô integrals. It contains already proved results. The only still nondiscussed part of this Theorem is Property f) of Wiener–Itô integrals. But it is easy to check this property by observing that one-fold Wiener–Itô integrals are (jointly) Gaussian, they are measurable with respect to the σ-algebra generated by the white noise μ_{W}. Beside this, the random variable μ_{W}(A) for a set A ∈ X , μ(A) < ∞, equals the (one-fold) Wiener–Itô integral of the indicator function of the set A.

Theorem 10.1 (Some properties of multiple Wiener–Itô integrals). Let a white noise μ_{W} be given with some non-atomic, σ-additive reference measure on a measurable space (X,X ). Then the k-fold Wiener–Itô integrals of all functions in the class H_{μ,k} introduced in formula (10.1) can be defined, and their normalized versions Z_{μ,k}(f) = \frac{1}{k!} ∫ f(x_{1},...,x_{k})μ_{W}(dx_{1})...μ_{W}(dx_{k}) satisfy the following relations:

a) Z_{μ,k}(αf + βg) = αZ_{μ,k}(f) + βZ_{μ,k}(g) for all f,g ∈ H_{μ,k} and real numbers α and β.

b) If A_{1},...,A_{k} are disjoint sets, μ(A_{j}) < ∞, then the function f_{A_{1},...,A_{k}}defined by the relation f_{A_{1},...,A_{k}}(x_{1},...,x_{k}) = 1 if x_{1} ∈ A_{1}, …, x_{k} ∈ A_{k}, f_{A_{1},...,A_{k}}(x_{1},...,x_{k}) = 0 otherwise, satisfies the identity

Z_{μ,k}(f_{A_{1},...,A_{k}}(x_{1},...,x_{k})) = \frac{1}{k!}μ_{W}(A_{1})··· μ_{W}(A_{k}).

c)

EZ_{μ,k}(f) = 0, and EZ_{μ,k}^{2}(f) = \frac{1}{k!}∥Symf∥_{2}^{2} ≤ \frac{1}{k!}∥f∥_{2}^{2}

for all f ∈ H_{μ,k}, where ∥f∥_{2}^{2} = ∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) is the square of the L_{2} norm of a function f ∈ H_{μ,k}.

d) Relation (10.6) holds for all functions f ∈ H_{μ,k} and g ∈ H_{μ,k′} if k ≠ k′.

e) Relation (10.7) holds for all functions f ∈ H_{μ,k}.

f) The Wiener–Itô integrals Z_{μ,1}(f) of order k = 1 are jointly Gaussian. The smallest σ-algebra with respect to which they are all measurable agrees with the σalgebra generated by the random variables μ_{W}(A), A ∈ X , μ(A) < ∞, of the white noise.

We have defined Wiener–Itô integrals of order k for all k = 1,2,.... For the sake of completeness let us introduce the class H_{μ,0} for k = 0 which consists of the real constants (functions of zero variables), and put Z_{μ,0}(c) = c. Because of relation (10.7) we could have restricted our attention to Wiener–Itô integrals with symmetric kernel functions. But at some points it was more convenient to work also with Wiener–Itô integrals of not necessarily symmetric functions.

Now I formulate the diagram formula for the product of two Wiener–Itô integrals. For this goal first I introduce some notations. Then I formulate the diagram formula with their help in Theorem 10.2A. To make this result more understandable I shall present after its formulation an example together with some pictures which may help to understand how to calculate the terms appearing in the diagram formula. A similar approach will be applied when the generalization of this result for the product of several Wiener–Itô integrals will be discussed, and also in the next chapter when a version of the diagram formula will be presented for the product of degenerate U-statistics.

To present the product of the multiple Wiener–Itô integrals of two functions f(x_{1},...,x_{k}) ∈ H_{μ,k} and g(x_{1},...,x_{l}) ∈ H_{μ,l} in the form of sums of Wiener–Itô integrals a class of diagrams Γ = Γ (k,l) will be defined. The diagrams γ ∈ Γ (k,l) have vertices (1,1),...,(1,k) and (2,1),...,(2,l), and edges ((1,j_{1}),(2,j′_{1})),…, ((1,j_{s}),(2,j′_{s})) with some 1 ≤ s ≤ min(k,l). The indices j_{1},...,j_{s} in the definition of the edges are all different, and the same relation holds for the indices j′_{1},...,j′_{s}. All diagrams γ with such properties belong to Γ (k,l). The set of vertices of the form (1,j), 1 ≤ j ≤ k, will be called the first row, and the set of vertices of the form (2,j′), 1 ≤ j′≤ l, the second row of a diagram. We demanded that edges of a diagram can connect only vertices of different rows, and at most one edge may start from each vertex of a diagram.

Given a diagram γ ∈ Γ (k,l) with the set of edges

E(γ) = {(1,j_{1}),(2,j′_{1})),...,((1,j_{s}),(2,j′_{s})}

let

V_{1}(γ) = {(1,1),...,(1,k)} \ {(1,j_{1}),...,(1,j_{s})}

and

V_{2}(γ) = {(2,1),...,(2,l)} \ {(2,j′_{1}),...,(2,j′_{s})}

denote the set of those vertices in the first and in the second row of the diagram γ respectively from which no edge starts. Put α_{γ}((1,j)) = (2,j′) if ((1,j),(2,j′)) ∈ E(γ) and α_{γ}((1,j)) = (1,j) if the diagram γ contains no edge which is of the form ((1,j),(2,j′)) ∈ E(γ). In words, the function α_{γ}(·) is defined on the vertices of the first row of the diagram γ. It replaces a vertex to the vertex it is connected to by an edge of the diagram if there is such a vertex, and it does not change those vertices from which no edge starts. Put |γ| = k + l − 2s, i.e. let |γ| equal the number of vertices in γ from which no edge starts. Given two functions f(x_{1},...,x_{k}) ∈ H_{μ,k} and g(x_{1},...,x_{l}) ∈ H_{μ,l} let us introduce their product

(f ∘ g)(x_{(1,1)},...,x_{(1,k)},x_{(2,1)},...,x_{(2,l)})
= f(x_{(1,1)},...,x_{(1,k)})g(x_{(2,1)},...,x_{(2,l)})  (10.9)

together with its transform

\overline{(f ∘ g)}_{γ}(x_{(1,j)} : (1,j) ∈ V_{1}(γ), x_{(2,j′)} : 1 ≤ j′≤ l)
= f(x_{α_{γ}((1,1))},...,x_{α_{γ}((1,k))})g(x_{(2,1)},...,x_{(2,l)}).  (10.10)

(Here the function f(x_{1},...,x_{k}) is replaced by f(x_{(1,1)},...,x_{(1,k)}) and the function g(x_{1},...,x_{l}) by g(x_{(2,1)},...,x_{(2,l)}).) With the help of the above introduced sets V_{1}(γ), V_{2}(γ) and function α_{γ}(·) let us introduce the functions F_{γ}(f,g) as

F_{γ}(f,g)(x_{(1,j)},x_{(2,j′)} : (1,j) ∈ V_{1}(γ), (2,j′) ∈ V_{2}(γ))
= ∫ \overline{(f ∘ g)}_{γ}(x_{α_{γ}((1,j))} : (1,j) ∈ V_{1}(γ), x_{(2,1)},...,x_{(2,l)})
∏_{(2,j′)∈{(2,1),...,(2,l)}\V_{2}(γ)}μ(dx_{(2,j′)})  (10.11)

for all diagrams γ ∈ Γ (k,l). In words: We take the product defined in (10.9), then if the index (1,j) of a variable x_{(1,j)} is connected with the index (2,j′) of some variable x_{(2,j′)} by an edge of the diagram γ, then we replace the variable x_{(1,j)} by x_{(2,j′)} in this product. Finally we integrate the function obtained in such a way with respect to the arguments with indices (2,j′_{1}),...,(2,j′_{s}), i.e. with those vertices of the second row of the diagram γ from which an edge starts. It is clear that F_{γ} is a function of |γ| variables. It depends on those coordinates whose indices are such vertices of γ from which no edge starts.

For the sake of simpler notations we shall also consider Wiener–Itô integrals with such kernel functions whose variables are more generally indexed. If the kfold Wiener–Itô integral with a kernel function f(x_{1},...,x_{k}) is well-defined, then we shall say that the Wiener–Itô integral with kernel function f(x_{u_{1}},...,x_{u_{k}}), where {u_{1},...,u_{k}} is an arbitrary set with k different elements, is also well defined, and it equals the Wiener–Itô integral with the original kernel function f(x_{1},...,x_{k}), i.e. we write

∫ f(x_{u_{1}},...,x_{u_{k}})μ_{W}(dx_{u_{1}})...μ_{W}(dx_{u_{k}}) = ∫ f(x_{1},...,x_{k})μ_{W}(dx_{1})...μ_{W}(dx_{k}).

(10.12)

(We have right to make such a convention since the value of a Wiener–Itô integral does not change if we permute the indices of the variables of the kernel function in an arbitrary way. This follows e.g. from (10.7).) In particular, we shall speak about the Wiener–Itô integral of the function F_{γ}(f_{1},f_{2}) defined in (10.11) without reindexing its variables x_{(1,j)} and x_{(2,j′)} ‘in the right way’. Now we can formulate the diagram formula for the product of two Wiener–Itô integrals.

Theorem 10.2A (The diagram formula for the product of two Wiener–Itô integrals). Let a non-atomic, σ-finite measure μ be given on a measurable space (X,X ) together with a white noise μ_{W} with reference measure μ, and take two functions f(x_{1},...,x_{k}) ∈ H_{μ,k} and g(x_{1},...,x_{l}) ∈ H_{μ,l}. (The classes of functions H_{μ,k} and H_{μ,l} were introduced in (10.1).) Let us consider the class of diagrams Γ (k,l) introduced above together with the functions F_{γ}(f,g), γ ∈ Γ (k,l), defined by formulas (10.9), (10.10) and (10.11) with its help. They satisfy the inequality

∥F_{γ}(f,g)∥_{2} ≤ ∥f∥_{2}∥g∥_{2} for all γ ∈ Γ (k,l),  (10.13)

where the L_{2} norm of a (generally indexed) function h(x_{u_{1}},...,x_{u_{s}}) is defined as

∥h∥_{2}^{2} = ∫ h^{2}(x_{u_{1}},...,x_{u_{s}})μ(dx_{u_{1}})...μ dx_{u_{s}}).

Beside this, the product k!Z_{μ,k}(f)l!Z_{μ,l}(g) of the Wiener–Itô integrals of the functions f and g (the notation Z_{μ,k} was introduced in (10.4)) satisfies the identity

(k!Z_{μ,k}(f))(l!Z_{μ,l}(g)) = ∑_{γ∈Γ (k,l)}|γ|!Z_{μ,|γ|}(F_{γ}(f,g))
= ∑_{γ∈Γ (k,l)}|γ|!Z_{μ,|γ|}(SymF_{γ}(f,g)).  (10.14)

The next example may help to understand how to apply the diagram formula.

Take two Wiener–Itô integrals 2!Z_{2}(f) = ∫ f(x_{1},x_{2})μ_{W}(dx_{1})μ_{W}(dx_{2}) and

3!Z_{3}(g) = ∫ g(x_{1},x_{2},x_{3})μ_{W}(dx_{1})μ_{W}(dx_{2})μ_{W}(dx_{3})

with kernel functions f(x_{1},x_{2}) and g(x_{1},x_{2},x_{3}). Let us understand how to calculate a term in the sum at the right-hand side of (10.14) which expresses the product 2!Z_{2}(f)3!Z_{3}(g) as a sum of Wiener–Itô integrals.

When we apply the diagram formula first we reindex the arguments of the functions f and g by the indices (1,1),(1,2) and (2,1),(2,2),(2,3) respectively, and take the product of these reindexed functions. We get the function

(f ∘ g)(x_{(1,1)},x_{(1,2)},x_{(2,1)},x_{(2,2)},x_{(2,3)}) = f(x_{(1,1)},x_{(1,2)})g(x_{(2,1)},x_{(2,2)},x_{(2,3)}).

We define the two rows of the diagrams we will be working with. The labels of their vertices agree with the indices of the arguments of the functions f and g. (See picture.)

The vertices of the diagrams

We consider all diagrams γ in which vertices from the first and second row are connected by edges, and from each vertex there starts zero or one edge. We define with the help of these diagrams γ some function F_{γ}(f,g) which will be the kernel functions of the Wiener–Itô integrals appearing in the diagram formula (10.14). Let us consider that diagram γ which contains one edge connecting the vertices (1,2) and (2,1).

The diagram we consider

We make a relabelling of the vertices by replacing the label of the vertices from the first row from which an edge starts with the label of the vertex with which this vertex is connected. Then we make the same reindexation with the indices of the function (f ∘ g). In the present case the diagram we take is

The reindexed version of our diagram

and we define the function

\overline{(f ∘ g)}_{γ}(x_{(1,1)},x_{(2,1)},x_{(2,2)},x_{(2,3)}) = f(x_{(1,1)},x_{(2,1)})g(x_{(2,1)},x_{(2,2)},x_{(2,3)}).

Finally we define the function F_{γ}(f,g) by integrating the function \overline{(f ∘ g)}_{γ} with respect to those variables whose indices agree with the label of a vertex from the second row of the diagram γ from which an edge starts. (In the present case this is x_{(2,1)}.)

F_{γ}(f,g)(x_{(1,1)},x_{(2,2)},x_{(2,3)}) = ∫ \overline{(f ∘ g)}_{γ}(x_{(1,1)},x_{(2,1)},x_{(2,2)},x_{(2,3)})μ(dx_{(2,1)}).

We got a function of 3 variables, and the contribution of the above diagram γ to the diagram formula (10.14) is

3!Z_{μ,3}(F_{γ}(f,g))
= ∫ F_{γ}(f,g)(x_{(1,1)},x_{(2,2)},x_{(2,3)})μ_{W}(dx_{(1,1)})μ_{W}(dx_{(2,2)})μ_{W}(dx_{(2,3)}).

In the last step some technical inconvenience appears. Originally we defined the Wiener–Itô integral of functions of the form f(x_{1},...,x_{k}), i.e. of functions whose variables have a different indexation. Generally this inconvenience is overcome in the literature by a reindexation of the variables of the kernel function F_{γ}(f,g). I chose a slightly different approach by introducing a formally more general Wiener– Itô integral in (10.12) which makes the above integral meaningful.

Theorem 10.2A will be proved in Appendix B. The following consideration yields a heuristic explanation for it. Actually it can also be considered as a sketch of proof.

In the theory of general Itô integrals when stochastic processes are integrated with respect to a Wiener processes, one of the most basic results is Itô’s formula about differentiation of functions of Itô integrals. It has a heuristic interpretation by means of the informal ‘identity’ (dW)^{2} = dt. In the case of general white noises this ‘identity’ can be generalized as (μ_{W}(dx))^{2} = μ(dx). We present a rather informal ‘proof’ of the diagram formula on the basis of this ‘identity’ and the fact that the diagonals are omitted from the domain of integration in the definition of Wiener–Itô integrals.

In this ‘proof’ we fix two numbers k ≥ 1 and l ≥ 1, and consider the product of two Wiener–Itô integrals of the functions f and g of order k and l. This product is a bilinear form of the functions f and g. Hence it is enough to check formula (10.14) for a sufficiently rich class of functions. It is enough to consider functions of the form f(x_{1},...,x_{k}) = I_{A_{1}}(x_{1})··· I_{A_{k}}(x_{k}) and g(x_{1},...,x_{l}) = I_{B_{1}}(x_{1})··· I_{B_{l}}(x_{l}) with disjoint sets A_{1},...,A_{k} and disjoint sets B_{1},...,B_{l}, where I_{A}(x) is the indicator function of a set A. (Here we have exploited that the functions f and g disappear in the diagonals.) Let us divide the sets A_{j} into the union of small disjoint sets D_{j}^{(m)}, 1 ≤ j ≤ k

with some fixed number 1 ≤ m ≤ M in such a way that μ(D_{j}^{(m)}) ≤ ε with some fixed

ε > 0, and the sets B_{j} into the union of small disjoint sets F_{j}^{(m)}, 1 ≤ j ≤ l, with some

fixed number 1 ≤ m ≤ M, in such a way that μ(F_{j}^{(m)}) ≤ ε with some fixed ε > 0.

Beside this, we also require that two sets D_{j}^{(m)} and F_{j′}^{(m′)} should be either disjoint

or they should agree. (The sets D_{j}^{(m)} are disjoint for different indices, and the same

relation holds for the sets F_{j′}^{(m′)}.)

Then the identities

k!Z_{μ,k}(f) = ∏_{j=1}^{k} ( ∑_{m=1}^{M} μ_{W}(D_{j}^{(m)}))

and

l!Z_{μ,l}(g) = ∏_{j′=1}^{l} ( ∑_{m′=1}^{M} μ_{W}(F_{j′}^{(m′)})) ,

hold, and the product of these two Wiener–Itô integrals can be written in the form of a sum by means of a term by term multiplication. Let us divide the terms of the sum we get in such a way into classes indexed by the diagrams γ ∈ Γ (k,l) in the following way: Each term in this sum is a product of the form

∏_{j=1}^{k} μ_{W}(D_{j}^{(m_{j})}) ∏_{j′=1}^{l} μ_{W}(F_{j′}^{(m′_{j})}). Let it belong to the class indexed by the diagram

γ with edges ((1,j_{1}),(2,j′_{1})),…, and ((1,j_{s}),(2,j′_{s})) if the elements in the pairs

(D_{j_{1}}^{(m_{j_{1}} )},F_{j′_{1}}^{(m_{j′_{1}} )}),…, (D_{j_{s}}^{(m_{js})},F_{j′_{s}}^{(m_{j′_{s}})}) agree, and otherwise all terms are different. Then letting ε → 0 (and taking partitions of the sets D_{j} and F_{j′} corresponding to the parameter ε) the sums of the terms in each class turn to integrals, and our calculation suggests the identity

(k!Z_{μ,k}(f))(l!Z_{μ,l}(g)) = ∑_{γ∈Γ (k,l)}Z̄_{γ}(f,g)  (10.15)

with

Z̄_{γ}(f,g) = ∫ f(x_{α_{γ}((1,1))},...,x_{α_{γ}((1,k))})g(x_{(2,1)},...,x_{(2,l)})  (10.16)
μ_{W}(dx_{α_{γ}((1,1))})...μ_{W}(dx_{α_{γ}((1,k))})μ_{W}(dx_{(2,1)})...μ_{W}(dx_{(2,l)})

with the function α_{γ}(·) introduced before formula (10.9). The indices α(1,j) of the arguments in (10.16) mean that in the case α_{γ}((1,j)) = (2,j′) the argument x_{(1,j)} has to be replaced by x_{(2,j′)}. In particular,

μ_{W}(dx_{α_{γ}((1,j))} )μ_{W}(dx_{(2,j′)}) = (μ_{W}(dx_{(2,j′)}))^{2} = μ(dx_{(2,j′)})

in this case because of the ‘identity’ (μ_{W}(dx))^{2} = μ(dx). Hence the above informal calculation yields the identity Z̄_{γ}(f,g) = |γ|!Z_{μ,|γ|}(F_{γ}(f,g)), and relations (10.15) and (10.16) imply formula (10.14).

A similar heuristic argument can be applied to get formulas for the product of integrals of normalized empirical distributions or (normalized) Poisson fields, only the starting ‘identity’ (μ_{W}(dx))^{2} = μ(dx) changes in these cases, some additional terms appear in it, which modify the final result. I return to this question in the next chapter.

It is not difficult to generalize Theorem 10.2A with the help of some additional notations to a diagram formula about the product of finitely many Wiener–Itô integrals. We shall do this in Theorem 10.2. Then to understand this result better I present an example which shows how to calculate the terms in the sum expressing the product of three Wiener–Itô integrals as a sum of Wiener–Itô integrals.

We consider the product of the Wiener–Itô integrals k_{p}!Z_{μ,k_{p}}(f_{p}), 1 ≤ p ≤ m, of m ≥ 2 functions f_{p}(x_{1},...,x_{k_{p}}) ∈ H_{μ,k_{p}}, of order k_{p} ≥ 1, 1 ≤ p ≤ m, and define a class of diagrams Γ = Γ (k_{1},...,k_{m}) in the following way.

The diagrams γ ∈ Γ = Γ (k_{1},...,k_{m}) have vertices of the form (p,r), 1 ≤ p ≤ m, 1 ≤ r ≤ k_{p}. The set of vertices {(p,r): 1 ≤ r ≤ k_{p}} with a fixed number p will be called the p-th row of the diagram γ. A diagram γ ∈ Γ = Γ (k_{1},...,k_{m}) may have some edges. All edges of a diagram connect vertices from different rows, and from each vertex there starts at most one edge. All diagrams satisfying these properties belong to Γ (k_{1},...,k_{m}). If a diagram γ contains an edge of the form ((p_{1},r_{1}),(p_{2},r_{2})) with p_{1} < p_{2}, then (p_{1},r_{1}) will be called the upper and (p_{2},r_{2}) the lower end point of this edge. Let E(γ) = {((p_{1}^{(u)},r_{1}^{(u)}),(p_{2}^{(u)},r_{2}^{(u)})), p_{1}^{(u)} < p_{2}^{(u)}, 1 ≤ u ≤ s} denote the set of all edges of a diagram γ (the number of edges in γ was denoted by s = |E(γ)|), and let us also introduce the sets V^{u}(γ) = {((p_{1}^{(u)},r_{1}^{(u)}), 1 ≤ u ≤ s}, the set of all upper end points and V^{b}(γ) = {((p_{2}^{(u)},r_{2}^{(u)}), 1 ≤ u ≤ s}, the set of all lower end points of edges in a diagram γ. Let V = V (γ) = {(p,r): 1 ≤ p ≤ m,1 ≤ r ≤ k_{p}} denote the set of all vertices of γ, and let |γ| = k_{1}+··· +k_{m}−2|E(γ)| denote the number of vertices in γ from which no edge starts. Vertices from which no edge starts will be called free vertices in the sequel. Let us also define the function α_{γ}((p,r)) for a vertex (p,r) of the diagram γ in the following way: α_{γ}((p,r)) = (p̄,r̄), if there is some pair of integers (p̄,r̄) such that ((p,r),(p̄,r̄)) ∈ E(γ) and p < p̄, i.e. (p,r) ∈ V^{u}(γ) and ((p,r),(p̄,r̄)) ∈ E(γ), and put α_{γ}((p,r)) = (p,r) for (p,r) ∈ V (γ)\V^{u}(γ). In words, the function α_{γ}(·) was defined on the set of vertices V (γ) in such a way that it replaces the label of an upper end point of an edge with the label of the lower end point of this edge, and it does not change the labels of the remaining vertices of the diagram.

With the help of the above quantities the appropriate multivariate version of the functions given in (10.9), (10.10) and (10.11) can be defined. Put

(f_{1}∘ f_{2}∘ ··· ∘ f_{m})(x_{(p,r)}, 1 ≤ p ≤ m,1 ≤ r ≤ k_{p})
= ∏_{p=1}^{m} f_{p}(x_{(p,1)},...,x_{(p,k_{p})}),  (10.17)

\overline{(f_{1}∘ f_{2}∘ ··· ∘ f_{m})} _{γ}(x_{(p,r)}, (p,r) ∈ V (γ) \ V^{u}(γ))
= ∏_{p=1}^{m} f_{p}(x_{α_{γ}((p,1))},...,x_{α_{γ}((p,k_{p}))}),  (10.18)

and

F_{γ}(f_{1},...,f_{m})(x_{(p,r)}, (p,r) ∈ V (γ) \ (V^{b}(γ) ∪ V^{u}(γ))  (10.19)
= ∫ \overline{(f_{1}∘ f_{2}∘ ··· ∘ f_{m})} _{γ}(x_{(p,r)}, (p,r) ∈ V (γ) \ V^{u}(γ))

∏_{(p,r)∈V^{b}(γ)}μ(dx_{(p,r)}).

In words, first we replace the indices 1,...,k_{p} of the function f_{p}(x_{1},...,x_{k_{p}}) by (p,1),...,(p,k_{p}), and take the product of the functions f_{p} with these reindexed variables in (10.17). Then we replace those indices of the variables in this product which agree with the index of the upper end-point of an edge in γ with the index of the lower end-points of this edge in (10.18). Finally we integrate the function obtained in such a way with respect to those variables whose indices agree with the index of a lower end-point of an edge of γ in (10.19).

With the help of the above notations the diagram formula for the product of finitely many Wiener–Itô integrals can be formulated.

Theorem 10.2 (The diagram formula for the product of finitely many Wiener– Itô integrals). Let a non-atomic, σ-finite measure μ be given on a measurable space (X,X ) together with a white noise μ_{W}with reference measure μ. Take m ≥ 2 functions f_{p}(x_{1},...,x_{k_{p}}) ∈ H_{μ,k_{p}} with some order k_{p} ≥ 1, 1 ≤ p ≤ m. Let us consider the class of diagrams Γ (k_{1},...,k_{m}) introduced above together with the functions F_{γ}(f_{1},...,f_{m}), γ ∈ Γ (k_{1},...,k_{m}), defined by formulas (10.17), (10.18) and (10.19) with its help. The L_{2}-norm of these functions satisfies the inequality

∥F_{γ}(f_{1},...,f_{m})∥_{2} ≤ ∏_{p=1}^{m} ∥f_{p}∥_{2} for all γ ∈ Γ (k_{1},...,k_{m}).  (10.20)

Beside this, the product ∏_{p=1}^{m} k_{p}!Z_{μ,k_{p}}(f_{p}) of the Wiener–Itô integrals of the functions

f_{p}, 1 ≤ p ≤ m, satisfies the identity

∏_{p=1}^{m} k_{p}!Z_{μ,k_{p}}(f_{p}) = ∑_{γ∈Γ (k_{1},...,k_{m})}|γ|!Z_{μ,|γ|}(F_{γ}(f_{1},...,f_{m}))  (10.21)
= ∑_{γ∈Γ (k_{1},...,k_{m})}|γ|!Z_{μ,|γ|}(SymF_{γ}(f_{1},...,f_{m})).

To understand the notations of the above result better let us take the product of three Wiener–Itô integrals 2!Z_{μ,2}(f_{2})4!Z_{μ,4}(f_{2})3!Z_{μ,3}(f_{3}) with kernel functions f_{1}(x_{1},x_{2}), f_{2}(x_{1},x_{2},x_{3},x_{4}) and f_{3}(x_{1},x_{2},x_{3}) and see how to calculate a term in the sum of diagram formula (10.21) which expresses this product as a sum of Wiener– Itô integrals.

Let us first define the rows of the diagrams we shall working with together with their labelling. There will be three rows with labels (1,1), (1,2), then with (2,1), (2,2), (2,3), (2,4) and finally with (3,1), (3,2), (3,3). We consider all possible diagrams which are graphs containing these vertices and edges connecting vertices from different rows with the restriction that from each vertex there can start at most one edge. We define with the help of all diagrams a function which will be the kernel-function of a Wiener–Itô integral appearing in the diagram formula (10.21). Let us consider for instance the diagram γ containing the edges ((1,1),(3,2)), ((1,2),(2,2)) and ((2,4),(3,3)), (see picture).

The diagram we consider

Let us relabel the vertices of the diagram γ by relabelling the upper vertices of each edge by the lower vertex of this edge.

The relabelled version of our diagram

We take the product of our functions with the indexation of the variables corresponding the labels of the diagrams. Then we reindex these variables corresponding to the relabelling of our diagram γ, i.e. define first the function

(f_{1}∘ f_{2}∘ f_{3})(x_{(1,1)},x_{(1,2)},x_{(2,1)},x_{(2,2)},x_{(2,3)},x_{(2,4)},x_{(3,1)},x_{(3,2)},x_{(3,3)})
= f_{1}(x_{(1,1)},x_{(1,2)})f_{2}(x_{(2,1)},x_{(2,2)},x_{(2,3)},x_{(2,4)})f_{3}(x_{(3,1)},x_{(3,2)},x_{(3,3)})

and then

\overline{(f_{1}∘ f_{2}∘ f_{3})} _{γ}(x_{(2,1)},x_{(2,2)},x_{(2,3)},x_{(3,1)},x_{(3,2)},x_{(3,3)})
= f_{1}(x_{(3,2)},x_{(2,2)})f_{2}(x_{(2,1)},x_{(2,2)},x_{(2,3)},x_{(3,3)})f_{3}(x_{(3,1)},x_{(3,2)},x_{(3,3)}).

Then we integrate the function \overline{(f_{1}∘ f_{2}∘ f_{3})} _{γ} with respect to the variables whose indices correspond to the labels of those vertices which are the lower labels of some edge. In our cases these are the indices (2,2), (3,2) and (3.3). This means that we define the function

F_{γ}(f_{1}∘ f_{2}∘ f_{3})(x_{(2,1)},x_{(2,3)},x_{(3,1)})
= ∫ \overline{(f_{1}∘ f_{2}∘ f_{3})} _{γ}(x_{(2,1)},x_{(2,2)},x_{(2,3)},x_{(3,1)},x_{(3,2)},x_{(3,3)})
μ(dx_{(2,2)})μ(dx_{(3,2)})μ(dx_{3,3}).

The function F_{γ}(f_{1},f_{2},f_{3}) is a function of three variables, and the contribution of the diagram γ to the sum at the right-hand side of (10.17) equals 3!Z_{μ,3}(F_{γ}(f_{1},f_{2},f_{3})) with the above defined kernel function F_{γ}(f_{1},f_{2},f_{3}). In the definition of this integral we apply again the convention described in (10.12).

Theorem 10.2 can be relatively simply derived from Theorem 10.2A by means of induction with respect to the number of terms whose product we consider. We still have to check that with the introduction of an appropriate notation Theorem 10.2A remains valid also in the case when the function f is a constant.

Let us also consider the case when f = c is a constant, and g ∈ H_{μ,l}. In this case we apply the convention Z_{μ,0}(c) = c, introduce the class of diagrams Γ (0,l) that consists only of one diagram γ whose first row is empty, its second row contains the vertices (2,1),...,(2,l), and it has no edge. Beside this, we define F_{γ}(c,g)(x_{(2,1)},...,x_{(2,l)}) = cg(x_{(2,1)},...,x_{(2,l)}) for this diagram γ. With such a convention Theorem 10.2A can be extended to the case of the product of two Wiener– Itô integrals of order k ≥ 0 and l ≥ 1. Theorem 10.2 can be derived from this slightly generalized result by induction with respect to the number of terms m in the product.

I explain only briefly the proof of Theorem 10.2 which is similar to the proof of Theorem 11.2 about the product of degenerate U-statistics given in Chapters 11 and 12, only some technical difficulties disappear in this case.

We can define, similarly to the corresponding definition in Chapter 11 where the diagram formula for the products of U-statistics will be formulated such a diagram γ_{pr} ∈ Γ (k_{1},...,k_{m−1}) for all γ ∈ Γ (k_{1},...,k_{m}) which is actually the restriction of the diagram γ to its first m − 1 rows. Beside this, we can define a diagram γ_{cl} ∈ Γ (|γ_{pr}|,k_{m}), where |γ_{pr}| denotes the number of free vertices of γ_{pr} in the following way. This diagram consists of two rows with |γ_{pr}| and k_{m} vertices respectively. It contains those edges of γ (after a reenumeration of the free vertices of γ_{pr} with the numbers 1,2,...,|γ_{pr}|) whose lower end points are in the m-th row of γ. It can be seen that F_{γ}(f_{1},...,f_{m}) = F_{γ_{cl}}(F_{γ_{pr}}(f_{1},...,f_{m−1}),f_{m}), and there is such a one to one correspondence (γ̄,γ̂) ↔ γ between the pairs of diagrams (γ̄,γ̂), γ̄∈ Γ (k_{1},...,k_{m−1}), γ̂∈ Γ (|γ̄|,k_{m}) and diagrams γ ∈ Γ (k_{1},...,k_{m}) for which γ̄= γ_{pr} and γ̂= γ_{cl}.

To prove the diagram formula for a product of the form ∏_{p=1}^{m} k_{p}!Z_{μ,k_{p}}(f_{p}) first we

express the product ∏_{p=1}^{m−1} k_{p}!Z_{μ,k_{p}}(f_{p}) with the help of the diagram formula by ex-

ploiting that by our inductive hypothesis it can be applied for the parameter m − 1. In such a way we can rewrite the above product as a sum of Wiener–Itô integrals with such kernel functions which can be calculated with the help of the restrictions γ_{pr} to the first m − 1 rows of the diagrams γ ∈ Γ (k_{1},...,k_{m}). Then by multiplying each term of this sum by k_{m}!Z_{μ,k_{m}}(f_{m}), calculating these products with the help of Theorem 10.2A and summing up the expressions we get in such a way we can rewrite the product at the left-hand side of (10.21) as a sum of Wiener–Itô integrals. It can be seen with the help of the properties of the diagrams γ ∈ Γ (k_{1},...,k_{m}) mentioned in the previous paragraph that the identity we get in such a way is equivalent to formula (10.21) in Theorem 10.2.

By statement c) of Theorem 10.1 all Wiener–Itô integrals of order k ≥ 1 have expectation zero. This fact together with Theorem 10.2 enable us to compute the expectation of a product of Wiener–Itô integrals. Theorem 10.2 makes possible to rewrite the product of Wiener–Itô integrals as a sum of Wiener–Itô integrals. Then its expectation can be calculated by taking the expected value of each term and summing them up. Only Wiener–Itô integrals of order zero yield a non-zero contribution to this expectation. These terms agree with the integrals of kernel functions F_{γ}(f_{1},...,f_{m}) corresponding to diagrams with no free vertices. In the next corollary I write down the result we got in this way.

Corollary of Theorem 10.2 about the expectation of a product of Wiener–Itô integrals. Let a non-atomic σ-finite measure μ be given on a measurable space (X,X ) together with a white noise μ_{W} with reference measure μ. Take m ≥ 2 functions f_{p}(x_{1},...,x_{k_{p}}) ∈ H_{μ,k_{p}}, and consider their Wiener–Itô integrals Z_{μ,k_{p}}(f_{p}), 1 ≤ p ≤ m. The expectation of the product of these random variables satisfies the identity

E ( ∏_{p=1}^{m} k_{p}!Z_{μ,k_{p}}(f_{p})) = ∑_{γ∈Γ̄(k_{1},...,k_{m})}F_{γ}(f_{1},...,f_{m}),  (10.22)

where Γ̄(k_{1},...,k_{m}) denotes the set of those diagrams γ ∈ Γ (k_{1},...,k_{m}) which have no free vertices, i.e. |γ| = 0. Such diagrams will be called closed diagrams in the sequel. (If Γ̄(k_{1},...,k_{m}) is empty, then the sum at the right-hand side of (10.22) equals zero.) The functions F_{γ}(f_{1},...,f_{m}) for γ ∈ Γ̄(k_{1},...,k_{m}) are constants, and they satisfy the inequality

|F_{γ}(f_{1},...,f_{m})| ≤ ∏_{p=1}^{m} ∥f_{p}∥_{2} for all γ ∈ Γ̄(k_{1},...,k_{m}).  (10.23)

Proof of the Corollary. Relation (10.22) is a straight consequence of formula (10.21),

part c) of Theorem 10.1 and the identity Z_{μ,0}(F_{γ}(f_{1},...,f_{m})) = F_{γ}(f_{1},...,f_{m}), if |γ| = 0. Relation (10.23) follows from (10.20).

The next result I formulate is Itô’s formula for multiple Wiener–Itô integrals. It can also be considered as a consequence of the diagram formula. It will be proved in Appendix C.

Theorem 10.3 (Itô’s formula for multiple Wiener–Itô integrals). Let a nonatomic, σ-finite measure μ be given on a measurable space (X,X ) together with a white noise μ_{W} with reference measure μ. Let us take some real valued, orthonormal functions φ_{1}(x),…, φ_{m}(x) on the measure space (X,X ,μ). Let H_{k}(u) denote the k-th Hermite polynomial with leading coefficient 1. Take the one-fold Wiener–Itô integrals η_{p} = Z_{μ,1}(φ_{p}), 1 ≤ p ≤ m, and introduce the random variables H_{k_{p}}(η_{p}),

1 ≤ p ≤ m, with some integers k_{p} ≥ 1, 1 ≤ p ≤ m. Put K_{p} = ∑_{j=1}^{p} k_{r}, 1 ≤ p ≤ m,

K_{0} = 0. Then η_{1},...,η_{m} are independent, standard normal random variables, and the identity

∏_{p=1}^{m} H_{k_{p}}(η_{p}) = K_{m}!Z_{μ,K_{m}}( ∏_{p=1}^{m} ( ∏_{j=K_{p−1} +1}^{K_{p}} φ_{p}(x_{j})) )  (10.24)

= K_{m}!Z_{μ,K_{m}}( Sym ( ∏_{p=1}^{m} ( ∏_{j=K_{p−1} +1}^{K_{p}} φ_{p}(x_{j})) ) )

holds. In particular, if φ(x) is a real valued function such that ∫ φ^{2}(x)μ(dx) = 1, then

H_{k}( ∫ φ(x)μ_{W}(dx)) = ∫ φ(x_{1})··· φ(x_{k})μ_{W}(dx_{1})...μ_{W}(dx_{k}).  (10.25)

I also formulate a limit theorem about the distribution of normalized degenerate U-statistics that will be proved in Appendix C. The limit distribution in this result is given by means of multiple Wiener–Itô integrals.

Theorem 10.4 (Limit theorem about normalized degenerate U-statistics). Let us consider a sequence of degenerate U-statistics I_{n,k}(f) of order k, n = k,k + 1,..., defined in (8.8) with the help of a sequence of independent and identically distributed random variables ξ_{1},ξ_{2},... taking values in a measurable space (X,X ) with a non-atomic distribution μ and a kernel function f(x_{1},...,x_{k}), canonical with respect to the measure μ, defined on the k-fold product (X^{k},X^{k}) of the space (X,X ) for which ∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) < ∞. Then the sequence of normalized U-statistics n^{−k/2}I_{n,k}(f) converges in distribution, as n → ∞, to the k-fold Wiener–Itô integral

Z_{μ,k}(f) = \frac{1}{k!} ∫ f(x_{1},...,x_{k})μ_{W}(dx_{1})...μ_{W}(dx_{k})

with kernel function f(x_{1},...,x_{k}) and a white noise μ_{W} with reference measure μ.

Remark. The limit behaviour of degenerate U-statistics I_{n,k}(f) with an atomic measure μ which satisfy the remaining conditions of Theorem 10.4 can be described in the following way. Take the probability space (U,U ,λ), where U = [0,1], U is the Borel σ-algebra and λ is the Lebesgue measure on it. Introduce a sequence of independent random variables η_{1},η_{2},... with uniform distribution on the interval [0,1], which is independent also of the sequence ξ_{1},ξ_{2},.... Define the product space (X̃,X̃,μ̃) = (X × U,X × U ,μ × λ) together with the function f̃(x̃_{1},...,x̃_{k}) = f̃((x_{1},u_{1}),...,(x_{k},u_{k})) = f(x_{1},...,x_{k}) with the notation x̃= (x,u) ∈ X × U, and ξ̃_{j} = (ξ_{j},η_{j}), j = 1,2,.... Then I_{n,k}(f) = I_{n,k}(f̃) (with the above defined function f̃and μ̃ distributed random variables ξ̃_{j}). Beside this, Theorem 10.4 can be applied for the degenerate U-statistics I_{n,k}(f̃), n = 1,2,....

In the next result I give an interesting representation of the Hilbert space consisting of the square integrable functions measurable with respect to a white noise μ_{W}. An isomorphism will be given with the help of Wiener–Itô integrals between this Hilbert space and the so-called Fock space to be defined below. To formulate this result first some notations will be introduced.

Let H_{μ,k}^{0} ⊂ H_{μ,k} denote the class of symmetric functions in the space H_{μ,k}, k = 0,1,2,..., i.e. f ∈ H_{μ,k} is in its subspace H_{μ,k}^{0} if and only if f(x_{1},...,x_{k}) = Symf(x_{1},...,x_{k}). Let us introduce for all k = 0,1,2,... the Hilbert space G_{k} consisting of those random variables η (on the probability space where the white noise μ_{W} is defined) which can be written in the form

η = Z_{μ,k}(f) = \frac{1}{k!} ∫ f(x_{1},...,x_{k})μ_{W}(dx_{1})...μ_{W}(dx_{k})

with some f ∈ H_{k,μ}^{0}.

It follows from part a) and c) of Theorem 10.1 that the map f → Z_{μ,k}(f) is a linear transformation of H_{μ,k}^{0} to G_{k}, and \frac{1}{k!}∥f∥_{2}^{2} = EZ_{μ,k}^{2}(f) for all f ∈ H_{μ,k}^{0}, where ∥f∥_{2} denotes the usual L_{2}-norm of the function f with respect to the k-fold power of the measure μ. By the definition of Wiener–Itô integrals the set G_{1} consists of jointly Gaussian random variables with expectation zero. The spaces H_{μ,0} and G_{0} consist of the real constants. Let us define the space Exp(H_{μ}) of infinite sequences

f = (f_{0},f_{1},...), f_{k} ∈ H_{μ,k}^{0}, k = 0,1,2,..., such that ∥f∥_{2}^{2} = ∑_{k=0}^{∞} \frac{1}{k!}∥f_{k}∥_{2}^{2} < ∞. The

space Exp(H_{μ}) with the natural addition and multiplication by a constant and the above introduced norm ∥f∥_{2} for f ∈ Exp(H_{μ}) is a Hilbert space which is called the Fock space in the literature.

Let G denote the class of random variables of the form

Z(f) = ∑_{k=0}^{∞} Z_{μ,k}(f_{k}), f = (f_{0},f_{1},f_{2},...) ∈ Exp(H_{μ}).

The next result describes the structure of the space of random variables G. It is useful for a better understanding of Wiener–Itô integrals, but it will be not used in the sequel. In its proof I shall refer to some basic measure theoretical results.

Theorem 10.5 (Isomorphism of the space of square integrable random variables measurable with respect to a white noise with a Fock space). Let a nonatomic, σ-finite measure μ be given on a measurable space (X,X ) together with a white noise μ_{W} with reference measure μ. Let us consider the class of functions H_{μ,k}^{0}, k = 0,1,2,..., and Exp(H_{μ}) together with the spaces of random variables

G_{k}, k = 0,1,2,..., and G defined above. The transformation Z: Z(f) = ∑_{k=0}^{∞} Z_{μ,k}(f_{k}),

f = (f_{0},f_{1},f_{2},...) ∈ Exp(H_{μ}), is a unitary transformation from the Hilbert space Exp(H_{μ}) to G. The Hilbert space G consists of all random variables with finite second moment, measurable with respect to the σ-algebra generated by the random variables μ_{W}(A), A ∈ X , μ(A) < ∞. This σ-algebra agrees with the σ-algebra generated by the random variables Z_{μ,1}(f_{1}), f_{1} ∈ H_{μ,1}^{0}.

Remark. For the sake of simpler notations we restrict our attention to the case when the measure space (X,X ,μ) is such that the Hilbert space of square integrable functions on this space is separable. This condition is satisfied in all interesting cases.

Proof of Theorem 10.5. Properties a) and c) in Theorem 10.1 imply that the transformation f_{k} → Z_{μ,k}(f_{k}) is a linear transformation of H_{μ,k}^{0} to G_{k}, and \frac{1}{k!}∥f_{k}∥_{2}^{2} = EZ_{μ,k}(f)^{2}. Beside this, EZ_{μ,k}(f)Z_{μ,k′}(f′_{k′}) = 0 if f_{k} ∈ H_{μ,k}^{0}, and f′_{k′}∈ H_{μ,k′}^{0} with k ≠ k′ by properties d) and c). (The latter property is needed to guarantee this relation also holds if k = 0 or k′= 0.) It follows from these relations that the map

Z: Z(f) = ∑_{k=0}^{∞} Z_{μ,k}(f_{k}), f = (f_{0},f_{1},f_{2},...) ∈ Exp(H_{μ}) is an isomorphism between

the Hilbert spaces Exp(H_{μ}) and G.

It remained to show that G contains all random variables with finite second moment, measurable with respect to the corresponding σ-algebra. Let g_{j}(u), j = 1,2,..., be an orthonormal basis in H_{μ,1}^{0} = H_{μ,1}, and introduce the random variables η_{j} = Z_{μ,1}(g_{j}), j = 1,2,.... These random variables are independent with standard normal distribution, and by Itô’s formula for Wiener–Itô integrals (Theorem 10.3) all products H_{r_{1}}(η_{j_{1}})...H_{r_{p}}(η_{j_{p}}) with r_{1}+··· +r_{p} = k are in the space G_{k}, where H_{r}(·) denotes the Hermite polynomial of order r with leading coefficient 1. We also recall the following results from the classical analysis:

a) Hermite polynomials constitute a complete orthonormal system in the L_{2}-space on the real line with respect to the standard normal distribution. (This result will be proved in Appendix C in Proposition C2.)

b) If a random variable ζ is measurable with respect to the σ-algebra generated by some random variables η_{1},η_{2},..., then there exists a Borel measurable function f(x_{1},x_{2},...) on the infinite product of the real line (R^{∞} ,B^{∞}) in such a way that ζ = f(η_{1},η_{2},...).

This means in our case that any random variable ζ measurable with respect to the σ-algebra generated by the random variables η_{j} = Z_{μ,1}(g_{j}), j = 1,2,..., can be written in the form ζ = f(η_{1},η_{2},...) with the above introduced independent, standard normal random variables η_{1},η_{2},.... If ζ has finite second moment, then the function f appearing in its representation is a function of finite L_{2}-norm in the infinite product of the real line with the infinite product of the standard normal distribution on it. Hence some classical results in analysis enable us to expand the function f with respect to products of Hermite polynomials, and this also yields the identity

ζ = ∑ c(j_{1},r_{1},...,j_{s},r_{s})H_{r_{1}}(η_{j_{1}})··· H_{r_{s}}(η_{j_{s}})

with some coefficients c(j_{1},r_{1},...,j_{s},r_{s}) such that

∑ c^{2}(j_{1},r_{1},...,j_{s},r_{s})∥H_{r_{1}}(u)∥^{2}··· ∥H_{r_{s}}(u)∥^{2} < ∞.

(Actually it is known that ∥H_{k}(u)∥^{2} = k!, but here we do not apply this fact.)

The above relations yield the desired representation of a random variable ζ with finite second moment, if it is measurable with respect to the σ-algebra generated by

the random variables in G_{1}. Indeed, the identity ζ = ∑_{k=0}^{∞} ζ_{k} holds with

ζ_{k} = ∑_{r_{1}+··· +r_{s}=k} c(j_{1},r_{1},...,j_{s},r_{s})H_{r_{1}}(η_{j_{1}})··· H_{r_{s}}(η_{j_{s}}),

and ζ_{k} ∈ G_{k} by Itô’s formula.

To complete the proof it is enough to remark that the σ-algebra generated by the random variables η_{1},η_{2},... and μ_{W}(A), A ∈ X , μ(A) < ∞ agree, as it was stated in part f) of Theorem 10.1.

The results about Wiener–Itô integrals discussed in this Chapter are useful in the study of non-linear functionals of a white noise. In my Lecture Note [32] similar problems were discussed, but in that work a slightly different version of Wiener–Itô integrals was introduced. The reason for this modification was that the solution of the problems studied in [32] demanded different methods.

In work [32] stationary Gaussian random fields were considered, and I was mainly interested in it in limit theorems for sequences of non-linear functionals on a stationary Gaussian random field. In a stationary Gaussian random field a shift operator can be introduced. This shift operator can be extended in a natural way to all random variables measurable with respect to the underlying stationary Gaussian random field. In [32] we needed a technique which helps in working with this shift operator. In an analogous case, when functions on the real line are considered, the Fourier analysis is a useful tool in the study of the shift operator. In the work [32] we tried to unify the tools of multiple Wiener–Itô integrals and Fourier analysis. This led to the definition of a slightly different version of Wiener–Itô integrals.

In the work [32] we have shown that not only the correlation function of a stationary Gaussian field can be given by means of the Fourier transform of its spectral measure, but also a random spectral measure can be constructed whose Fourier transform expresses the stationary Gaussian process itself. After the introduction of this random spectral measure a version of the multiple Wiener–Itô integral can be defined with respect to it, and all square integrable random variables, measurable with respect to the σ-algebra generated by the underlying Gaussian stationary random field can be expressed as the sum of such integrals. Moreover, such an approach enables us to apply the methods of multiple Wiener–Itô integrals and Fourier analysis simultaneously. The modified Wiener–Itô integral introduced in [32] behaves similarly to the original Wiener–Itô integral, only it has to be taken into account that the random spectral measure behaves not like a white noise, but as its ‘Fourier transform’. I omit the details. They can be found in [32].

The spaces G_{k} consisting of all k-fold Wiener–Itô integrals were introduced also in [32], and this was done for a special reason. In that work the Hilbert space of square integrable functions, measurable with respect to an underlying stationary Gaussian field was studied together with the shift operator acting on this Gaussian field. The shift operator could be extended to a unitary operator on this Hilbert space. The introduction of the subspaces G_{k} turned out to be useful, because they supplied such a decomposition of this Hilbert space which consists of orthogonal subspaces invariant with respect to the shift operator.

In the present work no shift operator was defined, and limit theorems for nonlinear functionals of a Gaussian field were not studied here. The introduction of the spaces G_{k} was useful because of a different reason. In the study of our problems we need good estimates on the 2p-th moment of random variables, measurable with respect to the underlying white noise for large integers p. As it will be shown, the high moments of the random variables in the spaces G_{k} with different indices k show an essentially different behaviour. The high moments of a random variable in G_{k} behave similarly to those of the k-th power ξ^{k} of a Gaussian random variable ξ with zero expectation. This statement will be formulated in a more explicit form in Proposition 13.1 or in its consequence, formula (13.2). A partial converse of this result will be presented in Theorem 13.6.

The diagram formula for products of degenerate U-statistics

There is a natural analogue of the diagram formula for the products of Wiener–Itô integrals both for the products of multiple integrals with respect to normalized empirical measures and for the products of degenerate U-statistics. These two results are closely related. They express the product of multiple random integrals or degenerate U-statistics as a sum of multiple random integrals or degenerate U-statistics respectively. The kernel functions of the random integrals or U-statistics appearing in this sum are defined, — similarly to the case of Wiener–Itô integrals, — by means of diagrams. This is the reason why these results are also called the diagram formula. The main difference between these diagram formulas and their version for Wiener–Itô integrals is that in the present case we have to work with a more general class of diagrams. The diagram formula for multiple integrals with respect to a normalized empirical measure will be discussed only at an informal level, while a complete proof of the analogous result about degenerate U-statistics will be given. The reason for such an approach is that the diagram formula for the product of degenerate U-statistics can be better applied in this work.

We want to prove the estimates about the tail distribution of degenerate Ustatistics and multiple integrals with respect to a normalized empirical distribution formulated in Theorems 8.3 and 8.1 with the help of good bounds on the high moments of degenerate U-statistics and multiple random integrals. In the case of degenerate U-statistics the diagram formula yields an explicit formula for these moments. We exploit that this formula expresses the product of degenerate U-statistics as a sum of degenerate U-statistics of different order. Beside this, the expected value of all degenerate U-statistics of order k ≥ 1 equals zero. Hence the expected value we are interested in equals the sum of the zero order terms appearing in the diagram formula.

The analogous problem about the moments of multiple integrals with respect to a normalized empirical measure is more difficult. The diagram formula enables us to express the moments of multiple random integrals as the sum of the expectation of such integrals of different order also in this case. But the expected value of random integrals of order k ≥ 1 with respect to a normalized empirical distribution may be non-zero. Before the proof of Theorem 9.4 we showed this in an example.

116 11 The diagram formula for products of degenerate U-statistics

First I give an informal description of the diagram formula for the product of two random integrals with respect to a normalized empirical measure. Its analogue, the diagram formula for the product of two Wiener–Itô integrals can be described in an informal way by means of formulas (10.15) and (10.16) together with the ‘identity’ (μ_{W}(dx))^{2} = μ(dx) in their interpretation. The diagram formula for the product of two multiple integrals with respect to a normalized empirical measure has a similar representation. (Observe that in the definition of the random integral J_{n,k}(·) given in formula (4.8) the diagonals are omitted from the domain of integration, similarly to the case of Wiener–Itô integrals.) In this case such a version of formulas (10.15) and (10.16) can be applied, where the random integrals Z_{μ,k} are replaced by J_{n,k}, and the white noise measure μ_{W} is replaced by the normalized empirical measure ν_{n} = \sqrt{n}(μ_{n}− μ). But the analogue of the ‘identity’ (μ_{W}(dx))^{2} = μ(dx) needed in the interpretation of these formulas has a different form. It states that (ν_{n}(dx))^{2} = μ(dx) + \frac{1}{\sqrt{n}}ν_{n}(dx). Let us ‘prove’ this new ‘identity’.

Take a small set Δ, i.e. a set Δ such that μ(Δ) is very small, write down the identity (ν_{n}(Δ))^{2} = n(μ_{n}(Δ))^{2} + n(μ(Δ))^{2} − 2nμ_{n}(Δ)μ(Δ) and observe that only a second order error is committed if the terms n(μ(Δ))^{2} and 2nμ_{n}(Δ)μ(Δ) are omitted at the right-hand side of this identity. Moreover, also a second order error is committed if n(μ_{n}(Δ))^{2} is replaced by μ_{n}(Δ), because it has second order small probability that there are at least two sample points in the small set Δ. On the other hand, n(μ_{n}(Δ))^{2} = μ_{n}(Δ) if Δ contains only zero or one sample point. The above considerations suggest that (ν_{n}(dx))^{2} = μ_{n}(dx) = μ(dx)+ \frac{1}{\sqrt{n}}[\sqrt{n}(μ_{n}(dx)−

μ(dx))] = μ(dx) + \frac{1}{\sqrt{n}}ν_{n}(dx). (This means that in the ‘identity’ expressing the

square (ν_{n}(dx))^{2} of a normalized empirical measure a correcting term \frac{1}{\sqrt{n}}ν_{n}(dx) appears. If the sample size n → ∞, then the normalized empirical measure tends to a white noise with counting measure μ, and this correcting term disappears.)

The diagram formula for the product of two multiple integrals with respect to a normalized empirical measure was proved in paper [35] with a different notation. Informally speaking, the result in this work states that the identity suggested by the above heuristic argument really holds. We remark that if the form of this identity is found, then it can be proved with the help of some algebraic calculations similarly to the proof of Lemma 9.5. We omit the proof of this result, since we shall not work with it. We shall prove instead a version of it about the product of degenerate U-statistics that we can better apply. This result is similar to the diagram formula for the products of multiple integrals with respect to a normalized empirical distribution. This similarity will be discussed in Remark 4 after Theorem 11.1.

In this chapter first I formulate the diagram formula about the product of two degenerate U-statistics in Theorem 11.1, then its generalization about the product of finitely many degenerate U-statistics in Theorem 11.2. Their proofs is postponed to the next chapter. I also present a Corollary of Theorem 11.2 about the expected value of the product of degenerate U-statistics which follows from this result and the observation that the expected value of a U-statistic of order k ≥ 1 equals zero. This result together with Lemma 11.3 which yields a bound on the L_{2}-norm of the kernel functions of the degenerate U-statistics appearing in the diagram formula will enable us to prove good estimates on the high moments of degenerate U-statistics. We can prove Theorem 8.3 about the tail distribution of degenerate U-statistics with the help of such estimates. One might try to prove the analogous result, Theorem 8.1 about the tail distribution of multiple integrals with respect to a normalized empirical distribution in a similar way with the help of the diagram formula for multiple random integrals. But that would be much harder, since the diagram formula for multiple integrals with respect to a normalized empirical distribution does not supply such a good formula for the moments of random integrals as the analogous result about degenerate U-statistics.

To describe the results of this chapter we introduce some new notions. In the formulation of the diagram formula for the product of degenerate U-statistics a more general class of diagrams has to be considered than in the case of multiple Wiener– Itô integrals. I shall define them under the name coloured diagrams. The kernel functions of the U-statistics appearing in the diagram formula will be defined with their help. First I introduce the notations needed in the formulation of the diagram formula for the product of two degenerate U-statistics, then I present this result in Theorem 11.1. After this, to understand the notations better I explain with the help of an example how to calculate a general term in this diagram formula.

A class of coloured diagrams Γ (k_{1},...,k_{m}) will be defined whose vertices will be the pairs (p,r), 1 ≤ p ≤ m, 1 ≤ r ≤ k_{p}, and the set of vertices (p,r), 1 ≤ r ≤ k_{p}, with a fixed number p will be called the p-th row of the diagram. To define the coloured diagrams of the class Γ (k_{1},...,k_{m}) first the notions of chains and coloured chains will be introduced. A sequence β = {(p_{1},r_{1}),...,(p_{s},r_{s})} with 1 ≤ p_{1} < p_{2} < ···< p_{s} ≤ m and 1 ≤ r_{u} ≤ k_{p_{u}} for all 1 ≤ u ≤ s will be called a chain. The number s of vertices (p_{u},r_{u}) in this sequence, denoted by ℓ(β), will be called the length of the chain β. Chains of length ℓ(β) = 1, i.e. chains consisting only of one vertex (p_{1},r_{1}) are also allowed. We shall define a function c(β) = ±1 which will be called the colour of the chain β, and the pair (β,c(β)) will be called a coloured chain. We shall allow arbitrary colouring c(β) = ±1 of a chain with the only restriction that a chain of length 1 can only get the colour −1, i.e. c(β) = −1 if ℓ(β) = 1.

A coloured diagram γ ∈ Γ (k_{1},...,k_{m}), is a partition of the set of vertices A(k_{1},...,k_{m}) = {(p,r): 1 ≤ p ≤ m, 1 ≤ r ≤ k_{p}} to the union of some coloured chains β ∈ γ, i.e. ⋃_{β∈γ} β = A(k_{1},...,k_{m}), and each vertex (p,r) ∈ A(k_{1},...,k_{m}) is

the element of exactly one chain β ∈ γ. Beside this, each chain β ∈ γ has a colour c_{γ}(β) = ±1. The set Γ (k_{1},...,k_{m}) consists of all coloured diagrams γ with the above properties with the only restriction that for a chain β = {(p,r)} ∈ γ of length ℓ(β) = 1 of a diagram γ ∈ Γ (k_{1},...,k_{m}) we have c_{γ}(β) = −1.

Let us define for all coloured diagrams γ ∈ Γ (k_{1},...,k_{m}) the set of open chains O(γ) = {β: β ∈ γ, c_{γ}(β) = −1} and the set of closed chains C(γ) = {β: β ∈ γ, c_{γ}(β) = 1} of this diagram γ. We shall define for all sets of bounded functions f_{p} = f_{p}(x_{1},...,x_{k_{p}}) ∈ L_{2}(X^{k_{p}},X^{k_{p}},μ^{k_{p}}), 1 ≤ p ≤ m, and diagrams γ ∈ Γ (k_{1},...,k_{m}) a bounded function F_{γ}(f_{1},...,f_{m}) = F_{γ}(f_{1},...,f_{m})(x_{1},...,x_{|O(γ)|}) with |O(γ)| variables on the product space (X^{|O(γ)|},X^{|O(γ)|},μ^{|O(γ)|}), where |O(γ)| denotes the number of open chains in the diagram γ. The arguments of the function F_{γ}(f_{1},...,f_{m}) will correspond to the open chains of the diagram γ. We will see that the function F_{γ}(f_{1},...,f_{m}) is canonical (with respect to the measure μ) if the same relation holds for the functions f_{1},...,f_{m}. In the diagram formula we shall express

the product of normalized degenerate U-statistics ∏_{p=1}^{m} n^{−k_{p}/2}k_{p}!I_{n,k_{p}}(f_{p}) as a linear

combination of the normalized degenerate U-statistics

n^{−|O(γ)|/2}|O(γ)|!I_{n,|O(γ)|}(F_{γ}(f_{1},...,f_{m})).

To define the above mentioned functions F_{γ}(f_{1},...,f_{m}) first we fix for all pairs of positive integers k_{1},k_{2} = 1,2,... and diagrams γ ∈ Γ (k_{1},k_{2}) an enumeration of the chains of γ, and beside this we also fix an enumeration of the open chains of all diagrams γ ∈ Γ (k_{1},...,k_{m}), m = 2,3,.... (For m ≥ 3 we shall need an enumeration only for the open chains.) For the sake of simpler notation we choose such an enumeration of the chains of a diagram γ for m = 2 where the chains get the labels 1,2,...,|O(γ)| + |C(γ)|, and the open chains get the first |O(γ)| labels, i.e. β(l) is an open chain if 1 ≤ l ≤ |O(γ)|, and it is a closed chain if |O(γ)| + 1 ≤ l ≤ |O(γ)| + |C(γ)|. In the case m ≥ 3 we give an enumeration only of the open chains of a diagram γ, and they will be indexed by the numbers 1 ≤ l ≤ |O(γ)|. This means that β(l) will be defined for 1 ≤ l ≤ |O(γ)|, and it is an open chain of γ.

We shall fix an enumeration of the chains of the diagrams with two rows and of the open chains of the diagrams with at least three rows at the start, and during the application of the diagram formula we shall always apply this enumeration of the chains. The subsequent definition of the functions F_{γ}(f_{1},...,f_{m}) will depend on this enumeration, but the results formulated with the help of these functions are valid for an arbitrary (previously fixed) enumeration of the chains. Hence the non-uniqueness in the definition of the functions F_{γ}(f_{1},...,f_{m}) will cause no problem.

First we formulate the diagram formula for the product of two degenerate U-statistics, i.e. we consider the case m = 2. Let us have a measurable space (X,X ) with a probability measure μ on it together with two measurable functions f_{1}(x_{1},...,x_{k_{1}}) and f_{2}(x_{1},...,x_{k_{2}}) of k_{1} and k_{2} variables on this space which are canonical with respect to the measure μ. Let ξ_{1},ξ_{2},... be a sequence of (X,X ) valued, independent and identically distributed random variables with distribution μ. We want to express the product n^{−k_{1}/2}k_{1}!I_{n,k_{1}}(f_{1})n^{−k_{2}/2}k_{2}!I_{n,k_{2}}(f_{2}) of normalized degenerate U-statistics defined with the help of the above random variables and kernel functions f_{1} and f_{2} as a sum of normalized degenerate U-statistics. For this goal we define some functions F_{γ}(f_{1},f_{2}) for all γ ∈ Γ (k_{1},k_{2}).

We shall define the function F_{γ}(f_{1},f_{2}) with the help of the previously fixed enumeration of the chains of the diagram γ. We shall introduce with the help of this enumeration also an enumeration of the vertices (1,p), (2,q), 1 ≤ p ≤ k_{1}, 1 ≤ q ≤ k_{2}, of the diagram γ. We put

α_{γ}((p,r)) = l if (p,r) ∈ β(l), p = 1,2, 1 ≤ r ≤ k_{p}.  (11.1)

Let us have two functions f_{1}(x_{1},...,x_{k_{1}}) and f_{2}(x_{1},...,x_{k_{2}}) together with a coloured diagram γ ∈ Γ (k_{1},k_{2}). We define the function F_{γ}(f_{1},f_{2}) in two steps. First we define the function

(f_{1}∘ f_{2})_{γ}(x_{1},...,x_{s(γ)})
= f_{1}(x_{α_{γ}((1,1))},...,x_{α_{γ}((1,k_{1}))})f_{2}(x_{α_{γ}((2,1))},...,x_{α_{γ}((2,k_{2}))}), (11.2)

where s(γ) = |O(γ)| + |C(γ)| is the number of chains in γ, and the indices α_{γ}(1,j) and α_{γ}(2,j′) were defined in (11.1). (In formula (11.2) the arguments of both functions f_{1} and f_{2} have different indices. But two indices α_{γ}((1,j)) and α_{γ}((2,j′)) may agree in some cases. This happens if the vertices (1,j) and (2,j′) belong to the same chain β ∈ γ of length 2.) In the second step we define the function

F_{γ}(f_{1},f_{2})(x_{1},...,x_{|O(γ)|})  (11.3)

= ( ∏_{j: β(j)∈C(γ)} P_{j} ∏_{j′: β(j′)∈O_{2}(γ)} Q_{j′}) (f_{1}∘ f_{2})_{γ}(x_{1},...,x_{|O(γ)|+|C(γ)|})

with the operators P_{j} and Q_{j′} defined in formulas (9.1) and (9.2), where C(γ) is the set of closed chains of the diagram γ, and O_{2}(γ) ⊂ O(γ) is the set of open chains of γ with length 2, i.e. O_{2}(γ) = {β: c_{γ}(β) = −1, and ℓ(β) = 2}. Let us also remark that the operators P_{j} and Q_{j′} in formula (11.3) are exchangeable, hence it is not important in what order we apply them.

Let me remark that if we applied a different enumeration of the diagrams γ ∈ Γ (k_{1},k_{2}) then we would get a different function F_{γ}(f_{1},f_{2}). This would be a reindexed version of the original function F_{γ}(f_{1},f_{2}). But the value of the U-statistic I_{n,|O(n)|}(F_{γ}(f_{1},f_{2})) does not depend on the indexation of the variables in its kernel function. Hence the identity which will be formulated in formula (11.4) of the subsequent Theorem 11.1 does not depend on the enumeration of the chains of the diagrams γ ∈ Γ (k_{1},k_{2}). Now we can formulate the following result.

Theorem 11.1 (The diagram formula for the product of two degenerate Ustatistics). Let a sequence of independent and identically distributed random variables ξ_{1},ξ_{2},... be given with some distribution μ on a measurable space (X,X ) together with two bounded, canonical functions f_{1}(x_{1},...,x_{k_{1}}) and f_{2}(x_{1},...,x_{k_{2}}) with respect to the probability measure μ on the product spaces (X^{k_{1}},X^{k_{1}}) and (X^{k_{2}},X^{k_{2}}) respectively. Let us take the class of coloured diagrams Γ (k_{1},k_{2}) introduced above together with the functions F_{γ}(f_{1},f_{2}) defined in formulas (11.1)— (11.3).

The functions F_{γ}(f_{1},f_{2}) are bounded and canonical with respect to the measure μ with |O(γ)| arguments for all coloured diagrams γ ∈ Γ , where O(γ) and C(γ) denote the set of open and closed chains of the diagram γ. The product of the normalized degenerate U-statistics n^{−k_{1}/2}k_{1}!I_{n,k_{1}}(f_{1}) and n^{−k_{2}/2}k_{2}!I_{n,k_{2}}(f_{2}), n ≥ max(k_{1},k_{2}), defined in (8.8) can be expressed as

n^{−k_{1}/2}k_{1}!I_{n,k_{1}}(f_{1}) · n^{−k_{2}/2}k_{2}!I_{n,k_{2}}(f_{2}) = ∑_{γ∈Γ (k_{1},k_{2})}^{′(n)} ∏_{j=1}^{|C(γ)|}( \frac{n − s(γ) + j}{n} )

n^{−W (γ)/2}· n^{−|O(γ)|/2}|O(γ)|!I_{n,|O(γ)|}(F_{γ}(f_{1},f_{2}))  (11.4)

with W (γ) = k_{1}+k_{2}−|O(γ)|−2|C(γ)| (we explain in Remark 1 after Theorem 11.1 that W (γ) = |O_{2}(γ)|, i.e. it equals the number of open chains with length 2) and s(γ) = |O(γ)|+|C(γ)| (which equals the number of coloured chains in γ). Here ∑^{′(n)} means that summation is taken only for such coloured diagrams γ ∈ Γ (k_{1},k_{2}) which

satisfy the inequality s(γ) ≤ n, and ∏_{j=1}^{|C(γ)|}equals 1 in the case |C(γ)| = 0. The term

I_{n,|O(γ)|}(F_{γ}(f_{1},f_{2})) can be replaced by I_{n,|O(γ)|}(SymF_{γ}(f_{1},f_{2})) in formula (11.4). Consider the L_{2}-norm of the functions F_{γ}(f_{1},f_{2})

∥F_{γ}(f_{1},f_{2})∥_{2}^{2} = ∫ F_{γ}(f_{1},f_{2})^{2}(x_{1}...,x_{|O(γ)|}) ∏_{p=1}^{|O(γ)|}μ(dx_{p}).

The inequality

∥F_{γ}(f_{1},f_{2})∥_{2} ≤ ∥f_{1}∥_{2}∥f_{2}∥_{2} if W (γ) = 0  (11.5)

holds for this norm. The condition W (γ) = 0 in formula (11.4) means that the diagram γ ∈ Γ (k_{1},k_{2}) has no chains β of length ℓ(β) = 2 with colour c_{γ}(β) = −1. For a general diagram γ ∈ Γ (k_{1},k_{2}) under the condition sup|f_{2}(x_{1},...,x_{k_{2}})| ≤ 1 the inequality

∥F_{γ}(f_{1},f_{2})∥_{2} ≤ 2^{W (γ)}∥f_{1}∥_{2}  (11.6)

holds. Inequalities (11.5) and (11.6) remain valid also in the case when f_{1} and f_{2} may be non-canonical functions.

Inequality (11.5) is actually a repetition of estimate (10.13) about the diagrams appearing in the case of Wiener–Itô integrals. Inequality (11.6) yields a weaker bound about the L_{2}-norm ∥F_{γ}(f_{1},f_{2})∥_{2} for a general diagram γ. We formulated it in a form where the functions f_{1} and f_{2} do not play a symmetrical role. This estimate depends on the L_{2}-norm of the function f_{1}, and it is assumed in it that the supremum of the function |f_{2}| is less than 1. We chose such a formulation of this inequality because it can be well generalized to the case when the product of several U-statistics is considered. The appearance of the condition about the supremum of the function |f_{2}| in the estimate (11.6) is closely related to the fact that in the estimates on the tail distribution of U-statistics, — unlike the case of Wiener–Itô integrals, — a condition is imposed not only on the L_{2}-norm of the kernel function f, but also on its L_{∞} -norm. I return to this question later.

Next I show an example which may help to understand how to apply the diagram formula for the product of two degenerate U-statistics.

Take two normalized degenerate U-statistics n^{−3/2} 3!I_{n,3}(f_{1}) and n^{−2} 4!I_{n,4}(f_{2}) with kernel functions f_{1}(x_{1},x_{2},x_{3}) and f_{2}(x_{1},x_{2},x_{3},x_{4}), and let us see how to calculate with the help of formula (11.4) a term of the sum which expresses the product 3!I_{n,3}(f_{1})4!I_{n,4}(f_{2}) as a sum of degenerate U-statistics.

Let us first understand which are the coloured diagrams we have to consider in the diagram formula (11.4), and then let us calculate the term corresponding to a coloured diagram at the right-hand side of this formula.

The coloured diagrams we have to consider have two rows with vertices labelled by (1,1), (1,2), (1,3) and (2.1), (2,2), (2,3), (2,4) respectively. The coloured diagrams are such partitions of the vertices whose elements contain from each row at most one element. The elements of these partitions which we call chains contain 1 or 2 elements. (We speak here about chains and not about graphs, because we want to apply such a terminology which also works in the more general case when we consider the diagram formula for the product of several degenerate U-statistics.) We give each chain either the colour +1 or −1. Chains consisting of only 1 vertex (chains of length 1) get the colour −1 while chains containing 2 vertices (chains of length 2) can get both colours +1 and −1. We take all coloured diagrams satisfying the above properties, and each of them yields a contribution to the sum at the right-hand side of (11.4). Let us look what kind of contribution yields the coloured diagram γ which contains a closed chain ((1,1),(2,2)) (with colour +1) and an open chain ((1,3),(2,4)) (with colour −1) of length two, and beside this it contains chains of length 1 and colour −1. They are (1,2) from the first row, and (2,1), (2,3) from the second row. (See the picture.)

The diagram with the labelling of its chains (o–o denotes open and •–• denotes closed chains)

We fix a labelling of the chains of the digram γ, and define with its help a relabelling of the vertices. We label the chains subsequently from 1 to 5 in such a way that the open chains get the smaller labels, 1,2,3 and 4. Otherwise, we choose arbitrary labelling. We have the right for it, since although the kernel function of the Ustatistic we shall define with the help of the diagram γ will depend on this labelling, but the U-statistic determined by it will not depend on it. Let us give the following labels for the chains: (1,2)–label 1, (2,1)–label 2, ((1,3),(2,4))–label 3, (2,3)– label 4, ((1,1),(2,2))–label 5. (This was an arbitrary choice.) Then we relabel the vertices contained in a chain with the label of this chain. (See the picture). (We used such a notation where the labels of the chains are put in a box, like this .)

The relabelled version of our diagram

Then we reindex the variables of the functions f_{1} and f_{2} corresponding to the new labels of the vertices in the first and second row respectively. In the present case we take the reindexed functions f_{1}(x_{5},x_{1},x_{3}) and f_{2}(x_{2},x_{5},x_{4},x_{3}). Then we define the product of these reindexed functions

(f_{1}∘ f_{2})_{γ}(x_{1},x_{2},x_{3},x_{4},x_{5}) = f_{1}(x_{5},x_{1},x_{3})f_{2}(x_{2},x_{5},x_{4},x_{3}).

Next we define the function F_{γ}(f_{1},f_{2}) introduced in (11.3) as

F_{γ}(f_{1},f_{2})(x_{1},x_{2},x_{3},x_{4}) = Q_{3}P_{5}(f_{1}∘ f_{2})_{γ}(x_{1},x_{2},x_{3},x_{4},x_{5}),

where P_{5} and Q_{3} corresponding to the closed chain with label 5 and open chain of length 2 with label 3 are the operators defined in (9.1) and (9.2) with j = 5 and j = 3 respectively. Thus

P_{5}(f_{1}∘ f_{2})_{γ}(x_{1},x_{2},x_{3},x_{4},x_{5}) = ∫ f_{1}(x_{5},x_{1},x_{3})f_{2}(x_{2},x_{5},x_{4},x_{3})μ(dx_{5}),

and

F_{γ}(f_{1},f_{2})(x_{1},x_{2},x_{3},x_{4}) = Q_{3}P_{5}(f_{1}∘ f_{2})_{γ}(x_{1},x_{2},x_{3},x_{4},x_{5})  (11.7)
= ∫ f_{1}(x_{5},x_{1},x_{3})f_{2}(x_{2},x_{5},x_{4},x_{3})μ(dx_{5})
−∫ f_{1}(x_{5},x_{1},x_{3})f_{2}(x_{2},x_{5},x_{4},x_{3})μ(dx_{3})μ(dx_{5}).

The normalized degenerate U-statistic corresponding to the diagram γ is

n^{−2} 4!I_{n,4}(F_{γ}(f_{1},f_{2})),

and the contribution of the diagram γ to the sum in the diagram formula, i.e. to the sum at the right-hand side of (11.4) is \frac{n−4}{n} · n^{−1/2} · n^{−2} 4!I_{n,4}(F_{γ}(f_{1},f_{2})). Here the factor n^{−1/2} is the term n^{−W (γ)/2}in (11.4) which is a contraction term which roughly speaking depends on the difference of the diagram γ from the ‘regular diagrams’ appearing also in the diagram formula for Wiener–Itô integrals. The factor \frac{n−4}{n} is a technical term which has no great importance. Its appearance is related to the form of the Hoeffding decomposition. In formula (9.4), expressing this relation a factor of the form (n−|V |)(n−|V |−1)··· (n−k+1) appears instead of the ‘regular term’ n^{k−|V |}, and this is the reason for the appearance of this factor.

Finally the notation ∑^{′(n)}in formula (11.4) means that the above calculated term corresponding to the diagram γ takes part in the summation only if the sample size n of the U-statistic satisfies the inequality n ≥ 5. This restriction is related to the fact that a k-fold U-statistic can be defined only if n ≥ k for the sample size. The Ustatistic with kernel function F_{γ}(f_{1},f_{2}) has order 4. Nevertheless, a slightly stronger restriction is imposed. The reason for it is that, as the proof of Theorem 11.3 will show, the U-statistic we considered here appears as a term in the Hoeffding decomposition of the U-statistic with kernel function (f_{1} ∘ f_{2})_{γ}. This is a U-statistic of order 5, and the condition n ≥ 5 comes from here.

Next we make some comments to Theorem 11.1.

Remark 1. The expression W (γ) = k_{1} + k_{2} − |O(γ)| − 2|C(γ)| appearing in formulas (11.4) and (11.5) equals |O_{2}(γ)|, i.e. it is the number of the chains β ∈ γ for which ℓ(β) = 2, and c_{γ}(β) = −1. Indeed, if W̄(γ) equals the number of chains β ∈ γ for which ℓ(β) = 1 (and as a consequence c_{γ}(β) = −1), then |O_{2}(γ)| + W̄(γ) = |O(γ)|, and 2C|(γ)| + 2|O(γ)| − W̄(γ) = k_{1} + k_{2}. (In the last identity we calculated the number of vertices in γ in two different ways.) Because of the definition of W (γ) the last identity can be rewritten as W (γ) + W̄(γ) = |O(γ)|. These relations imply the statement of this remark.

Remark 2. The term I_{n,|O(γ)|}(F_{γ}(f_{1},f_{2})) with some coloured diagram γ ∈ Γ (k_{1},k_{2}) appeared in the sum at the right-hand side of (11.4) only if the condition s(γ) ≤ n was satisfied, which means that the sample size n of the U-statistic is sufficiently large. This restriction in the summation had a technical character, which has no great importance in our investigations. It is related to the fact that a U-statistic I_{n,k}(f) was defined only if n ≥ k. As a consequence, some U-statistics disappear at the righthand side of (11.4) if the sample size n of the U-statistics is relatively small. The term I_{n,|O(γ)|}(F_{γ}(f_{1},f_{2})) appeared in (11.4) through the Hoeffding decomposition of a U-statistic with kernel function (f_{1}∘ f_{2})_{γ} defined in (11.2). This function has s(γ) arguments, and the U-statistic corresponding to it appears in our calculations only if the sample size n is not smaller than the number s(γ).

Remark 3. As I earlier mentioned the functions F_{γ}(f_{1},f_{2}) depended on the labelling of the chains β ∈ Γ (k_{1},k_{2}). This non-uniqueness in the formulation of identity (11.4) has no importance in its applications. Moreover, we can get rid of this non-uniqueness by working with symmetrical functions f_{1} and f_{2} (with functions which do not change by a permutation of their variables) and by replacing the functions F_{γ}(f_{1},f_{2}) by their symmetrizations. A similar remark holds for the general version of the diagram formula to be discussed later, where we may consider the product of several degenerate U-statistics.

Remark 4. The diagram formula formulated in Theorem 11.1 is similar to its version about the product of two multiple integrals with respect to a normalized empirical distribution. The latter result was not written up here explicitly, but its form was explained in an informal way at the beginning of this chapter. The kernel functions of the U-statistics and random integrals appearing in these formulas are indexed by the same diagrams. Their definitions are different, because in the U-statistic case we have to work with canonical functions while in the multiple integral case we have no such restriction. As a consequence we define the functions F_{γ}(f_{1},f) in this case by means of a modified version of formula (11.3), where the operators Q_{l_{p}} are omitted from the definition. The coefficients of the normalized degenerate U-statistics and random integrals in the two results are slightly different. In the multiple integral case we have to multiple with n^{−W (γ)/2} while in the U-statistic case this term is multiplied with a factor between 0 and 1. This is related to the form of the Hoeffding decomposition of U-statistics given in (9.3). The restriction in the summation ∑^{′(n)} is also related to the properties of U-statistics.

Let us turn to the formulation of the general form of the diagram formula for the product of finitely many degenerate U-statistics. After introduction of some notations we present this result in Theorem 11.2. Then we discuss an example to understand its notation better.

This result has a more complicated form than its analogue about Wiener-Itô integrals, because in the present case we cannot define the kernel functions of the U-statistics appearing in the diagram formula in a simple, direct way. We shall define them with the help of an inductive procedure. To do this first we introduce some conventions which will be useful later.

Let us recall the convention introduced after the definition of canonical degenerate U-statistics by which I_{n,0}(c) is a degenerate U-statistic of order zero, and I_{n,0}(c) = c for a constant c. If γ ∈ Γ (k_{1},k_{2}) is such a diagram for which |O(γ)| = 0, i.e. c_{γ}(β) = 1 for all chains β ∈ γ, then the expression F_{γ}(f_{1},f_{2}) defined in (11.3) is a constant, and for such a diagram γ we define the term I_{n,|O(γ)|}(F_{γ}(f_{1},f_{2})) in relation (11.4) by means of the previous convention.

We introduce another convention (similarly to the discussion of Wiener–Itô integrals in Chapter 10) which enables us to extend the validity of Theorem 11.1 to the case when k_{1} = 0, and the function f_{k_{1}} = c with a constant c. In this case Γ (k_{1},k_{2}) consists of only one diagram γ containing the chains β_{p} = {(2,p)} of length one and with colour c_{γ}({(2,p)}) = −1, 1 ≤ p ≤ k_{2}, and we define F_{γ}(f_{1},f_{2}) = cf_{2}(x_{1},...,x_{k_{2}}). Beside this, we have C(γ) = ∅, O(γ) = {(2,1),...,(2,k_{2})}, hence W (γ) = k_{1} + k_{2} − |O(γ)| − 2|C(γ)| = 0, |C(γ)| = 0. We also have s(γ) = k_{2}, thus the inequality (γ) ≤ n holds under the conditions of Theorem 11.1. Hence formula (11.4) remains valid also in the case k_{1} = 0. For the sake of completeness we introduce a listing of the (open) chains β ∈ O(γ) of the diagram(s) of the set Γ (0,k_{2}). We define β(l) = {(2,l)}, 1 ≤ l ≤ k_{2} in this case. We have introduced the above conventions because they are useful in the inductive argument we shall apply in the proof of the diagram formula for the product of degenerate U-statistics in the general case.

To formulate the diagram formula for the product of degenerate U-statistics in the general case first we define a function F_{γ}(f_{1},...,f_{m}) = F_{γ}(f_{1},...,f_{m})(x_{1},...,x_{|O(γ)|}) for each coloured diagram γ ∈ Γ (k_{1},...,k_{m}) and collection of canonical functions (canonical with respect to a probability measure μ on a measurable space (X,X )) f_{1},...,f_{m} with k_{1},…,k_{m} variables. The function F_{γ}(f_{1},...,f_{m}) we shall define has |O(γ)| arguments. It will appear as the kernel function of the degenerate U-statistic corresponding to the diagram γ at the right-hand side of the diagram formula.

The functions F_{γ}(f_{1},...,f_{m}) will be defined by induction with respect to the number m of the components in the product of degenerate U-statistics. For m = 2 we have already defined them. Let the functions F_{γ}(f_{1},...,f_{m−1}) be defined for each coloured diagram γ ∈ Γ (k_{1},...,k_{m−1}). To define F_{γ}(f_{1},...,f_{m}) for a coloured diagram γ ∈ Γ (k_{1},...,k_{m}) first we define the predecessor γ_{pr} = γ_{pr}(γ) ∈ Γ (k_{1},...,k_{m−1}) of γ. It consist of the restrictions of the chains of the diagram γ to the first m − 1 rows of this diagram together with an appropriate colouring of these restricted chains. Then we define the function F_{γ_{pr}}(f_{1},...,f_{m−1}) with |O(γ_{pr})| arguments in our inductive procedure. We shall also define a coloured diagram γ_{cl} ∈ Γ (|O(γ_{pr})|,k_{m}) of two rows, which has the heuristic content that it contains the additional information we need to reconstruct the diagram γ ∈ Γ (k_{1},...,k_{m}) from its predecessor γ_{pr}. We shall define F_{γ}(f_{1},...,f_{m}) which will be a canonical function with |O(γ)| variables with the help of the diagram γ_{cl} and the pair of functions F_{γ_{pr}}(f_{1},...,f_{m−1}) and f_{m}.

The diagram γ_{pr} ∈ Γ (k_{1},...,k_{m−1}) will consist of the chains

β_{pr} = β \ {(m,1),...,(m,k_{m})}, β ∈ γ,

i.e. we get the chain β_{pr} by dropping from β its vertex contained in the last row {(m,1),...,(m,k_{m})} of the diagram if it contains such a vertex. If we get an empty set in such a way (this happens if β consists of a single vertex of the form(m,p)) then we disregard it, i.e the empty set will be not taken as a chain of γ_{pr}. We define the colour of β_{pr} as c_{γ_{pr}}(β_{pr}) = c_{γ}(β) if β = β_{pr}, i.e. if β ∩ {(m,1),...,(m,k_{m})} = ∅, and c_{γ_{pr}}(β_{pr}) = −1 if β contains a vertex of the form (m,p), 1 ≤ p ≤ k_{m}. After the definition of the diagrams γ_{pr} ∈ Γ (k_{1},...,k_{m−1}) we can define the canonical function F_{γ_{pr}}(f_{1},...,f_{m−1}) with arguments x_{1},...,x_{|O(γ_{pr})|}by means of our inductive procedure.

We also define the diagram γ_{cl} ∈ Γ (|O(γ_{pr})|,k_{m}) for a diagram γ ∈ Γ (k_{1},...,k_{m}). We must tell which are the chains {(1,p),(2,r)}, 1 ≤ p ≤ |O(γ_{pr})|, 1 ≤ r ≤ k_{m}, of length two of the diagram γ_{cl}, and we have to define their colour. The set {(1,p),(2,r)} is a chain of length two of the diagram γ_{cl} if and only if the open chain β(p) ∈ γ_{pr} (the chain β(p) is that open chain of γ_{pr} ∈ Γ (k_{1},...,k_{m−1}) which got the label p in the enumeration of the open chains of γ_{pr}) is the restriction β_{pr} of that chain β ∈ γ for which (m,r) ∈ β. If {(1,p),(2,r)} ∈ γ_{cl}, then its colour in γ_{cl} is defined as c_{γ_{cl}}({(1,p),(2,r)}) = c_{γ}(β) with the chain β = β(p) ∈ γ, which is the chain for which (m,r) ∈ β. Those vertices (1,p) and (2,r), 1 ≤ p ≤ |O(γ_{pr})|, 1 ≤ r ≤ k_{m}, which are not contained in such a chain of length 2 will be chains of length 1 of γ_{cl} with colour −1.

Given some bounded functions f_{1},...,f_{m} of k_{p} variables, 1 ≤ p ≤ m, and a diagram γ ∈ Γ (k_{1},...,k_{m}) we shall define the function F_{γ}(f_{1},...,f_{m}) with the help of the pair of functions F_{γ_{pr}}(f_{1},...,f_{m−1}) and f_{m} and the diagram γ_{cl} ∈ Γ (|O(γ_{pr})|),k_{m}) by the formula

F_{γ}(f_{1},...,f_{m})(x_{1},x_{2},...,x_{|O(γ)|})
= F_{γ_{cl}}(F_{γ_{pr}}(f_{1},...,f_{m−1}),f_{m})(x_{1},...,x_{|O(γ_{cl})|})).  (11.8)

Here we applied formula (11.3) with the choice γ = γ_{cl} and pair of functions f_{1} = F_{γ_{pr}}(f_{1},...,f_{m−1}) and f_{2} = f_{m}. To justify the correctness of formula (11.8) we still have to show that |O(γ)| = |O(γ_{cl})|.

To prove this identity observe that the number of those open chains of γ_{cl} which contain a vertex from the first row of γ_{cl} equals the number of those open chains of β ∈ γ which have a vertex outside of the m-th row of the diagram γ. The remaining open chains of γ_{cl} contain one vertex from the second row of γ_{cl}, and they correspond to those open diagrams of γ which consist of one vertex from the m-th row of the diagram. The above observations imply the desired identity.

To formulate the general form of the diagram formula for the product of degenerate U-statistics we introduce some quantities which are the versions of the quantities W (γ), s(γ) appearing in the identity (11.4) in Theorem 11.1 in the case m > 2. Put

W (γ) = ∑_{β∈O(γ)}(ℓ(β) − 1) + ∑_{β∈C(γ)}(ℓ(β) − 2), γ ∈ Γ (k_{1},...,k_{m}),  (11.9)

where ℓ(β) denotes the length of the chain β.

To define the next quantity let us introduce some notations. We consider the chains of the form β = {(p_{1},r_{1}),...,(p_{l},r_{l})}, 1 ≤ p_{1} < p_{2} < ···< p_{l} ≤ m, with elements in the set A(k_{1},...,k_{m}) = {(p,r): 1 ≤ p ≤ m, 1 ≤ r ≤ k_{p}}, and define their upper level u(β) = p_{1}, and deepest level d(β) = p_{l}. With the help of these notions we introduce for all diagrams γ ∈ Γ (k_{1},...,k_{m}) and integers p, 1 ≤ p ≤ m, the following subsets of the diagram γ. Put B_{1}(γ,p) = {β: β ∈ γ, c_{γ}(β) = 1, d(β) = p}, and B_{2}(γ,p) = {β: β ∈ γ, c_{γ}(β) = −1, d(β) ≤ p}∪{β: β ∈ γ, u(β) ≤ p, d(β) > p}. In words, B_{1}(γ,p) consists of those chains β ∈ γ which have colour 1, all their vertices are in the first p rows of the diagram, and contain a vertex in the p-th row. The set B_{2}(γ,p) consists of those chains β ∈ γ which have either colour −1, and all their vertices are in the first p rows of the diagram, or they have (with an arbitrary colour) a vertex both in the first p rows and in the remaining rows of the diagram. Put B_{1}(γ,p) = |B_{1}(γ,p)| and B_{2}(γ,p) = |B_{2}(γ,p)|. With the help of these numbers we define

J_{n}(γ,p) = \{ \begin{matrix}∏_{j=1}^{B_{1}(γ,p)}( \frac{n−B_{1}(γ,p)−B_{2}(γ,p)+j}{n} ) if B_{1}(γ,p) ≥ 1 \\ 1 if B_{1}(γ,p) = 0 \end{matrix}  (11.10)

for all 2 ≤ p ≤ m and diagrams γ ∈ Γ (k_{1},...,k_{m}).

Theorem 11.2 will be formulated with the help of the above notations.

Theorem 11.2 (The diagram formula for the product of several degenerate Ustatistics). Let a sequence of independent and identically distributed random variables ξ_{1},ξ_{2},... be given with some distribution μ on a measurable space (X,X ) together with m ≥ 2 bounded functions f_{p}(x_{1},...,x_{k_{p}}) on the spaces (X^{k_{p}},X^{k_{p}}), 1 ≤ p ≤ m, canonical with respect to the probability measure μ. Let us consider the class of coloured diagrams Γ (k_{1},...,k_{m}) together with the functions F_{γ} = F_{γ}(f_{1},...,f_{m}), γ ∈ Γ (k_{1},...,k_{m}), defined in formulas (11.8) and the constants W (γ) and J_{n}(γ,p), 1 ≤ p ≤ m, given in formulas (11.9) and (11.10).

The functions F_{γ}(f_{1},...,f_{m}) are bounded and canonical with respect to the measure μ with |O(γ)| variables, and the product of the degenerate U-statistics I_{n,k_{p}}(f_{p}), 1 ≤ p ≤ m, n ≥ max_{1≤p≤m} k_{p}, defined in (8.8) can be written in the form

∏_{p=1}^{m} n^{−k_{p}/2}k_{p}!I_{n,k_{p}}(f_{p}) = ∑_{γ∈Γ (k_{1},...,k_{m})}^{′(n,m)}( ∏_{p=2}^{m} J_{n}(γ,p))

n^{−W (γ)/2}· n^{−|O(γ)|/2}|O(γ)|!I_{n,|O(γ)|}(F_{γ}(f_{1},...,f_{m})),  (11.11)

where ∑^{′(n,m)}means that summation is taken for those γ ∈ Γ (k_{1},...,k_{m}) which satisfy the relation B_{1}(γ,p)+B_{2}(γ,p) ≤ n for all 2 ≤ p ≤ m with the quantities B_{1}(γ,p) and B_{2}(γ,p) introduced before the definition of J_{n}(γ,p) in (11.10), and the expression W (γ) was defined in (11.9). The terms I_{n,|O(γ)|}(F_{γ}(f_{1},...,f_{m})) at the right-hand side of formula (11.11) can be replaced by I_{n,|O(γ)|}(SymF_{γ}(f_{1},...,f_{m})).

To understand better the formulation of Theorem 11.2 let us consider the following example.

Take three normalized degenerate U-statistics n^{−3/2} 3!I_{n,3}(f_{1}), n^{−2} 4!I_{n,4}(f_{2}) and n^{−3/2} 3!I_{n,3}(f_{3}) with canonical kernel functions f_{1}(x_{1},x_{2},x_{3}), f_{2}(x_{1},x_{2},x_{3},x_{4}) and f_{3}(x_{1},x_{2},x_{3}), and let us see how to calculate a term from the sum at the right-hand side of formula (11.11) which expresses the product

n^{−3/2} 3!I_{n,3}(f_{1})n^{−2} 4!I_{n,4}(f_{2})n^{−3/2} 3!I_{n,3}(f_{3})

in the form of a linear combination of degenerate U-statistics.

In this case we have to consider coloured diagrams with rows of vertices (1,1), (1,2), (1,3), then (2,1), (2,2), (2,3), (2,4), and finally (3,1), (3,2), (3,3). We have to consider all coloured diagrams with these rows, and to calculate their contribution to the sum at the right-hand side of (11.11). Let us consider for instance the diagram containing two closed chains (with colour 1) ((1,3),(2,4),(3,3)) of length 3, ((1,1),(2,2)) of length 2, an open chain (with colour −1) ((2,1),(3,1)) of length 2, and the remaining vertices (1,2), (2,3), (3,2) are chains of length 1 which are consequently open (with colour −1). (See picture.)

Our diagram γ

We want to calculate F_{γ}(f_{1},f_{2},f_{3}). For this goal first we have to determine the coloured diagrams γ_{pr} ∈ Γ (3,4) and γ_{cl} ∈ Γ (4,3) (here the first parameter 4 in the definition of the class of diagrams where γ_{cl} belongs to is the number of open chains in γ_{pr}, which is, as we will see, equals 4), and the kernel function F_{γ_{pr}}(f_{1},f_{2}). (See the picture of the diagram γ_{pr} together with a labelling of its chains and the diagram γ_{cl} to which we also attached a labelling.)

The diagram γ_{pr} corresponding to γ together with the enumeration of its open chains The diagram γ_{cl} constructed with the help of γ_{pr} and of the enumeration of its open chains

In our example γ_{pr} is a diagram with two rows (1,1), (1,2), (1,3) and (2,1), (2,2), (2,3), (2,4). It contains a closed chain ((1,1),(2,2)) and an open chain ((1,3),(2,4)) of length 2, (the latter is the restriction of a chain of length 3), and open chains of length 1, which are the vertices (1,2), (2,1), (2,3). This is the same diagram which we considered in the example after Theorem 1.1. In that example we have fixed an enumeration of the chains of this diagram. We also made the convention that the enumeration of the chains of a diagram fixed at the start cannot be modified later. Hence we have the following enumeration of the open chains of this diagram: (1,2)–label 1, (2,1)–label 2, ((1,3),(2,4))–label 3, and (2,3)–label 4.

We define the coloured diagram γ_{cl} with the help of the diagram γ_{pr} and the enumeration of its open chains. It has two rows. The vertices of the first row (1,1), (1,2), (1,3) and (1,4) correspond to the open chains of the diagram γ_{pr} with labels 1, 2, 3 and 4 respectively. The vertices of the second row, (2,1), (2,2) and (2,3) correspond to the vertices (3,1), (3,2) and (3,3) of the last row of the original diagram γ. The diagram γ_{cl} has an open chain ((1,2),(2,1)) of length two, (here the open chain (2,1) of γ_{pr} labelled by 2, is connected to the vertex (3,1) with second index 1), a closed chain of length 2 ((1,3),(2,3)) (here the open chain of γ_{pr} labelled by 3 is connected with the vertex (3,3)), and the remaining open chains of γ_{cl} of length 1 are (1,1), (1,4) (the open chains (1,2) and (2,3) of γ_{pr} with labels 1, and 4), and (2,2).

Actually we have already calculated the function F_{γ_{pr}}(f_{1},f_{2}) in formula (11.7). We can calculate similarly the function F_{γ}(f_{1},f_{2},f_{3}) = F_{γ_{cl}}(F_{γ_{pr}}(f_{1},f_{2}),f_{3}). First we fix a labelling of the chains of the diagram γ_{cl}, say (1,1)–label 1, ((1,2),(2,1))– label 2, (1,4)–label 3, (2,2)–label 4, and ((1,3),(2,3))–label 5. (I have denoted this labelling in the corresponding picture.) With such a labelling

F_{γ}(f_{1},f_{2},f_{3})(x_{1},x_{2},x_{3},x_{4}) = Q_{2}P_{5}(F_{γ_{pr}}(x_{1},x_{2},x_{5},x_{3})f_{3}(x_{2},x_{4},x_{5}))
= ∫ F_{γ_{pr}}(x_{1},x_{2},x_{5},x_{3})f_{3}(x_{2},x_{4},x_{5})μ(dx_{5})
−∫ F_{γ_{pr}}(x_{1},x_{2},x_{5},x_{3})f_{3}(x_{2},x_{4},x_{5})μ(dx_{2})μ(dx_{5}).

The normalized degenerate U-statistic corresponding to γ is

n^{−2} 4!I_{n,4}(F_{γ}(f_{1},f_{2},f_{3})),

and the term corresponding to F_{γ} in formula (11.11) is

( \frac{n − 4}{n} ) ^{2}· n^{−1} · n^{−2} 4!I_{n,4}(F_{γ}(f_{1},f_{2},f_{3}))

if n ≥ 5. In the case n ≤ 4 this term disappears.

In Theorem 11.2 the product of such degenerate U-statistics were considered whose kernel functions were bounded. This also implies that all functions F_{γ} appearing at the right-hand side of (11.11) are well-defined (i.e. the integrals appearing in their definition are convergent) and bounded. In the applications of Theorem 11.2 it is useful to have a good bound on the L_{2}-norm of the functions F_{γ}(f_{1},...,f_{m}). Such a result is formulated in the following

Lemma 11.3 (Estimate about the L_{2}-norm of the kernel functions of the Ustatistics appearing in the diagram formula). Let m functions f_{p}(x_{1},...,x_{k_{p}}), 1 ≤ p ≤ m, be given on the products (X^{k_{p}},X^{k_{p}},μ^{k_{p}}) of some measure space (X,X ,μ), 1 ≤ p ≤ m, with a probability measure μ, which satisfy inequality (8.1) (if the index k is replaced by the index k_{p} in formula (8.1)). Let us take a coloured diagram γ ∈ Γ (k_{1},...,k_{m}), and consider the function F_{γ}(f_{1},...,f_{m}) defined inductively by means of formula (11.8). The L_{2}-norm of the function F_{γ}(f_{1},...,f_{m}) (with respect to the product measure μ × ···× μ on the space where F_{γ}(f_{1},...,f_{m}) is defined) satisfies the inequality

∥F_{γ}(f_{1},...,f_{m})∥_{2} ≤ 2^{W (γ)} ∏_{p∈U(γ)}∥f_{p}∥_{2},

where W (γ) is given in (11.9), and the set U(γ) ⊂ {1,...,m} is defined as

U(γ) = {p: 1 ≤ p ≤ m, for all vertices (p,r), 1 ≤ r ≤ k_{p} the chain β ∈ γ

for which(p,r) ∈ β has the property that either u(β) = p

or d(β) = p and c_{γ}(β) = 1}.  (11.12)

(If the point (p,r) is contained in a chain β = {(p,r)} ∈ γ of length 1, then u(β) = d(β) = p, and c_{γ}(β) = −1. In this case the vertex (p,r) satisfies that condition which all vertices (p,r), 1 ≤ r ≤ k_{p}, must satisfy to guarantee the property p ∈ U(γ).)

Remark. Let us give a less formal definition of the set U(γ) in formula (11.12). It contains the indices of those rows of the diagram γ whose vertices behave in a sense nicely. This nice behaviour means the following. Each vertex is contained in a chain β of the diagram γ. We sat that a vertex has nice behaviour if it is either at the highest or the lowest level of the chain β ∈ γ containing it. Moreover, if it is at its lower level, then we also demand that β must be closed, i.e. c(β) = 1. If a vertex is contained in a chain containing no other vertex, then it is both at the higher and lower level of this chain. In this case we say that the vertex behave nicely.

The last result of this chapter is a corollary of Theorem 11.2. In this corollary we give an estimate on the expected value of a product of degenerate U-statistics. To formulate this result we introduce the following terminology. We call a (coloured) diagram γ ∈ Γ (k_{1},...,k_{m}) closed if c_{γ}(β) = 1 for all chains β ∈ γ, and denote the set of all closed diagrams by Γ̄(k_{1},...,k_{m}). Observe that F_{γ}(f_{1},...,f_{m}) is constant (a function of zero variable) if and only if γ is a closed diagram, i.e. γ ∈ Γ̄(k_{1},...,k_{m}), and

I_{n,|O(γ)|}(F_{γ}(f_{1},...,f_{m})) = I_{n,0}(F_{γ}(f_{1},...,f_{m})) = F_{γ}(f_{1},...,f_{m})

in this case. Now we formulate the following result.

Corollary of Theorem 11.2 about the expectation of a product of degenerate U-statistics. Let a finite sequence of functions f_{p}(x_{1},...,x_{k_{p}}), 1 ≤ p ≤ m, be given on the products (X^{k_{p}},X^{k_{p}}) of some measurable space (X,X ) together with a sequence of independent and identically distributed random variables with value in the space (X,X ) and some distribution μ which satisfy the conditions of Theorem 11.2.

Let us apply the notation of Theorem 11.2 together with the notion of the above introduced class of closed diagrams Γ̄(k_{1},...,k_{m}). The identity

E ( ∏_{p=1}^{m} n^{−k_{p}/2}k_{p}!I_{n,k_{p}}(f_{k_{p}}))  (11.13)

= ∑_{γ∈Γ̄(k_{1},...,k_{m})}^{′(n,m)}( ∏_{p=1}^{m} J_{n}(γ,p)) n^{−W (γ)/2}· F_{γ}(f_{1},...,f_{m})

holds. This identity has the consequence

|E ( ∏_{p=1}^{m} n^{−k_{p}/2}k_{p}!I_{n,k_{p}}(f_{k_{p}})) | ≤ ∑_{γ∈Γ̄(k_{1},...,k_{m})}n^{−W (γ)/2}|F_{γ}(f_{1},...,f_{m})|. (11.14)

Beside this, if the functions f_{p}, 1 ≤ p ≤ m, satisfy conditions (8.1) and (8.2) (with indices k_{p} instead of k in them), then the numbers |F_{γ}(f_{1},...,f_{m})| at the right-hand side of (11.14) satisfy the inequality

|F_{γ}(f_{1},...,f_{m})| ≤ 2^{W (γ)}σ^{|U(γ)|} for all γ ∈ Γ̄(k_{1},...,k_{m}).  (11.15)

In formula (11.15) the same number W (γ) and set U(γ) appear as in Lemma 11.3. The only difference is that in the present case the definition of U(γ) becomes a bit simpler, since c_{γ}(β) = 1 for all chains β ∈ γ.

Remark: We have applied a different terminology for diagrams in this chapter and in Chapter 10, where the theory of Wiener–Itô integrals was discussed. But there is a simple relation between their terminology. If we take only those diagrams considered in this chapter which contain only chains of length 1 or 2, and beside this the chains of length 1 have colour −1, and the chains of length 2 have colour 1, then we get the diagrams considered in the previous chapter. Moreover, the functions F_{γ} = F_{γ}(f_{1},...,f_{m}) are the same in the two cases. Hence formula (10.22) in the Corollary of Theorem 10.2 and formula (11.14) in the Corollary of Theorem 11.2 make possible to compare the moments of Wiener–Itô integrals and degenerate Ustatistics.

The main difference between the estimates of this chapter and those given in the Gaussian case is that formula (11.14) contains some additional terms. They are the contributions of those diagrams γ ∈ Γ̄(k_{1},...,k_{m}) which contain chains β ∈ γ with length ℓ(β) > 2. These are those diagrams γ ∈ Γ̄(k_{1},...,k_{m}) for which W (γ) ≥ 1. The estimate (11.15) given for the terms F_{γ} corresponding to such diagrams is weaker, than the estimate given for the terms F_{γ} with W (γ) = 0, since |U(γ)| < m if W (γ) ≥ 1, and |U(γ)| = m if W (γ) = 0. On the other hand, such terms have a coefficient n^{−W (γ)/2} at the right-hand side of formula (11.14). A closer study of these formulas may explain the relation between the estimates given for the tail distribution of Wiener–Itô integrals and degenerate U-statistics.

The proof of the diagram formula for U-statistics

In this chapter the results of the previous chapter will be proved. First I prove its main result, the diagram formula for the product of two degenerate U-statistics.

Proof of Theorem 11.1. In the first step of the proof the product

k_{1}!I_{n,k_{1}}(f_{1})k_{2}!I_{n,k_{2}}(f_{2})

of two degenerate U-statistics will be rewritten as a sum of not necessarily degenerate U-statistics. In this step a term by term multiplication is carried out for the product k_{1}!I_{n,k_{1}}(f_{1})k_{2}!I_{n,k_{2}}(f_{2}), and the terms of the sum obtained in such a way are put into different classes indexed by the (non-coloured) diagrams with two rows of length k_{1} and k_{2}. This step is very similar to the heuristic argument leading to formulas (10.15) and (10.16) in our explanation about the diagram formula for Wiener-Itô integrals.

In this step we consider all sets of pairs

{(l_{1},l′_{1}),...,(l_{r},l′_{r})}, 1 ≤ r ≤ min(k_{1},k_{2}),

with the following properties: 1 ≤ l_{1} < l_{2} < ···< l_{r} ≤ k_{1}, the numbers l′_{1},...,l′_{r}are all different, and 1 ≤ l′_{s}≤ k_{2}, for all 1 ≤ s ≤ r.

To a set of pairs {(l_{1},l′_{1}),...,(l_{r},l′_{r})} with the above properties let us correspond the following diagram γ̄((l_{1},l′_{1}),...,(l_{r},l′_{r})) ∈ Γ̄(k_{1},k_{2}), where Γ̄(k_{1},k_{2}) denotes the set of (non-coloured) diagrams with two rows of length k_{1} and k_{2}. The diagram γ̄((l_{1},l′_{1}),...,(l_{r},l′_{r})) has two rows, {(1,1)...,(1,k_{1})}, and {(2,1),...,(2,k_{2})}, its chains of length 2 are the sets {(1,l_{s}),(2,l′_{s})}, 1 ≤ s ≤ r, and beside this it contains the chains {(1,p)}, p ∈ {1,...,k_{1}}\{l_{1},...,l_{r}}, and {(2,p)}, p ∈ {1,...,k_{2}}\ {l′_{1},...,l′_{r}} of length 1. All (non-coloured) diagrams γ̄ ∈ Γ̄(k_{1},k_{2}) can be represented in the form γ̄ = γ̄((l_{1},l′_{1}),...,(l_{r},l′_{r})) with the help of a set of pairs {(l_{1},l′_{1}),...,(l_{r},l′_{r})}, 1 ≤ r ≤ min(k_{1},k_{2}), with the above properties in a unique way.

134 12 The proof of the diagram formula for U-statistics

To make the notation in the subsequent discussion simpler we introduce, similarly to the notation of Chapter 11, a labelling of the chains of the diagrams γ̄∈ Γ̄(k_{1},k_{2}), and then we define the labelling of the vertices of this diagram γ̄ with its help.

Let us choose the following natural labelling of the chains of a diagram. Consider the diagram γ̄= γ̄((l_{1},l′_{1}),...,(l_{r},l′_{r})) ∈ Γ̄(k_{1},k_{2}) which has s(γ̄) = k_{1}+k_{2}−r chains. The chain β ∈ γ̄containing the vertex (1,p) gets the label p, i.e. {(1,p)} = β(p) if 1 ≤ p ≤ k_{1}, and p ∉ {l_{1},...,l_{r}}, and {(1,l_{s}),(2,l′_{s})} = β(p) if p = l_{s} with some 1 ≤ s ≤ r. The remaining chains of γ̄ have the form {(2,p)} with p ∈ {1,...,k_{2}}\{l′_{1},...,l′_{r}}. Let us list the numbers p with this property in an increasing order, i.e. write {1,...,k_{2}} \ {l′_{1},...,l′_{r}} = {l̄_{1},...,l̄_{k_{2}−r} } with 1 ≤ l̄_{1} < ···< l̄_{k_{2}−r}, and define {(2,l̄_{p})} = β(k_{1}+ p) for 1 ≤ p ≤ k_{2}− r. In such a way we have labelled the chains of a diagram γ̄∈ Γ̄(k_{1},k_{2}). After this, we label its vertices (p,r) by the formula αγ̄((p,r)) = l with that label l for which (p,r) ∈ β(l). Let us also define the sets V_{1} = V_{1}(γ̄) = {1,...,k_{1}+k_{2}−r}\{l_{1},...,l_{r}} and V_{2} = V_{2}(γ̄) = {l_{1},...,l_{r}}. These sets yield the labels of the chains of length 1 and length 2 respectively, i.e. β(p) is a chain of length 1 if p ∈ V_{1}, and it is a chain of length 2 if p ∈ V_{2}.

We have defined a special labelling of the chains of the diagrams γ̄∈ Γ̄(k_{1},k_{2}), and we shall work with it during the proof. First we prove a slightly modified version of relation (11.4) with functions F_{γ}(f_{1},f_{2}) defined with the help of the above labelling of the chains, which may not satisfy all conditions we imposed for a labelling of the chains before the formulation of Theorem 11.1. Then we show that identity (11.4) remains valid with the formulation of Theorem 11.1 (i.e. with that labelling of the chains which we considered there).

Let us consider the product k_{1}!I_{n,k_{1}}(f_{1})k_{2}!I_{n,k_{2}}(f_{2}), and let us rewrite it in the form of the sum we get by carrying out a term by term multiplication in this expression. We put the terms obtained in such a way into disjoint classes indexed by the diagrams γ̄∈ Γ̄(k_{1},k_{2}) in the following way: A product

f_{1}(ξ_{j_{1}},...,ξ_{j_{k_{1}}})f_{2}(ξ_{j′_{1}},...,ξ_{j′_{k_{2}}})

belongs to the class indexed by the diagram γ̄((l_{1},l′_{1}),...,(l_{r},l′_{r})) with the parameters (l_{1},l′_{1}),...,(l_{r},l′_{r}), 1 ≤ r ≤ min(k_{1},k_{2}), where 1 ≤ l_{1} < l_{2} < ···< l_{r} ≤ k_{1}, the numbers l′_{1},...,l′_{r} are different, and 1 ≤ l′_{s} ≤ k_{2}, for all 1 ≤ s ≤ r if the indices j_{1},...,j_{k_{1}},j′_{1},...,j′_{k_{2}} in the arguments of the variables in f_{1}(·) and f_{2}(·) satisfy the relation j_{l_{s}} = j′_{l′_{s}}, 1 ≤ s ≤ r, and there is no more coincidence between the indices j_{1},...,j_{k_{1}},j′_{1},...,j′_{k_{2}}.

It is not difficult to see by applying the above partition of the terms in the product k_{1}!I_{n,k_{1}}(f_{1})k_{2}!I_{n,k_{2}}(f_{2}), and exploiting that each diagram γ̄∈ Γ̄(k_{1},k_{2}) can be represented in the form γ̄((l_{1},l′_{1}),...,(l_{r},l′_{r})) in a unique way that the identity

n^{−k_{1}/2}k_{1}!I_{n,k_{1}}(f_{1})k_{2}!n^{−k_{2}/2}I_{n,k_{2}}(f_{2})
= ∑_{γ̄∈Γ̄(k_{1},k_{2})}^{′(n)}n^{−(k_{1}+k_{2})/2} s(γ̄)!I_{n,s(γ̄)}((f_{1}∘ f_{2})γ̄)  (12.1)

holds, where the functions (f_{1} ∘ f_{2})γ̄ = (f_{1} ∘ f_{2})γ̄(x_{1},...,x_{s(γ̄)}) are defined in formula (11.2) with the help of the above introduced labelling of the chains of the diagram γ̄, and s(γ̄) = k_{1}+k_{2}−|V_{2}(γ̄)| denotes the number of chains in γ̄. (Observe that with our labelling of the chains the indices of the function (f_{1} ∘ f_{2})γ̄ are the numbers 1,...,s(γ̄).) The notation ∑^{′(n)}in (12.1) means that summation is taken only for such diagrams γ̄∈ Γ̄(k_{1},k_{2}) for which n ≥ s(γ̄). (Let me remark that although formula (11.2) was defined for coloured diagrams, the colours of the chains played no role in it.)

Relation (12.1) is not appropriate for our purposes, since the functions (f_{1}∘ f_{2})γ̄ in it may be non-canonical. To get the desired formula, Hoeffding’s decomposition will be applied for the U-statistics I_{n,s(γ̄)}((f_{1}∘f_{2})γ̄) appearing at the right-hand side of formula (12.1). This decomposition becomes slightly simpler because of some special properties of the function (f_{1}∘f_{2})γ̄which follow from the canonical property of the initial functions f_{1} and f_{2}.

To carry out this procedure let us observe that a function f(x_{1},...,x_{k}) is canonical if and only if (P_{j}f)(x_{s}, s ∈ {1,...,k}\{j}) = 0 with the operator P_{j} defined in (9.1) for all indices j and {x_{s}: 1 ≤ j]lek, s ≠ j}. Beside this, the condition that the functions f_{1} and f_{2} are canonical implies the relation P_{v}(f_{1}∘f_{2})γ̄≡ 0 if v ∈ V_{1}(γ̄) for all diagrams γ̄∈ Γ̄(k_{1},k_{2}). (The set V_{1}(γ̄) denoted the labels of the chains of length 1 in the diagram γ̄.) This relation remains valid if the function (f_{1} ∘ f_{2})γ̄ is replaced by such functions which we get by applying the product of some transformations P_{v′} and Q_{v′}, v′∈ V_{2}(γ̄), for the function (f_{1} ∘ f_{2})γ̄ with the transformations P_{v′} and Q_{v′}defined in formulas (9.1) and (9.2).

Beside this, the transformations P_{v} or Q_{v} are exchangeable with the operators P_{v′} or Q_{v′}for any pairs of indices v,v′, and P_{v}+Q_{v}= I, where I denotes the identity operator. Beside this, P_{v}Q_{v} = 0, since P_{v}Q_{v} = P_{v}− P_{v}^{2} = 0. The above relations make possible the following decomposition of the function (f_{1}∘f_{2})γ̄to the sum of canonical functions for all γ̄∈ Γ̄(k_{1},k_{2}). (In the proof of the Hoeffding decomposition a similar argument was applied.)

(f_{1}∘ f_{2})γ̄ = ∏_{v∈V_{2}(γ̄)}(P_{v}+ Q_{v})(f_{1}∘ f_{2})γ̄  (12.2)

= ∑_{A⊂V_{2}(γ̄)}( ∏_{v∈A}P_{v} ∏_{v∈V_{2}\A}Q_{v}) (f_{1}∘ f_{2})γ̄= ∑_{γ∈Γ (γ̄)} F̄_{γ}(f_{1},f_{2}),

where Γ (γ̄) denotes the set of those coloured diagrams γ ∈ Γ (k_{1},k_{2}) which contain the same chains (with colour 1 or −1) as the non-coloured diagram γ̄. Here Γγ̄ denotes the set of all such coloured coloured diagrams which have the same chains as the diagram γ̄, their chains of length 2 may have colour 1 or −1, while the colour of their chains with length 1 is −1. The function F̄_{γ}(f_{1},f_{2}) is defined for a diagram γ ∈ Γ (γ̄) in the following way.

If the colouring of the chains of a coloured diagram γ ∈ Γ (γ̄) is defined with the help of a set A ⊂ V_{2}(γ̄) by the relations c_{γ}(β(v)) = 1 if v ∈ A, c_{γ}(β(v)) = −1 if v ∈ V_{2}(γ̄) \ A, (and for the remaining chains β ∈ γ with length 1 c_{γ}(β) = −1), then

F̄_{γ}(f_{1},f_{2}) = F̄_{γ}(f_{1},f_{2})(x_{l_{1}},...,x_{l_{|O(γ)|}})
= ∏_{v∈A}P_{v} ∏_{v∈V_{2}\A}Q_{v}(f_{1}∘ f_{2})γ̄(x_{1},...,x_{s(γ̄)}).  (12.3)

Here the indices l_{1},...,l_{|O(γ)|}, l_{1} < ···< l_{|O(γ)|}, of the variables of the function F̄_{γ}(f_{1},f_{2}) are the labels of the open chains (chains with colour −1) of the diagram γ, i.e, they are the elements of the set (V_{2}(γ̄) \ A) ∪ V_{1}(γ̄). (Clearly, s(γ) = s(γ̄) for the number of chains of γ and γ̄if γ ∈ Γ (γ̄).) In such a way we have defined F̄_{γ}(f_{1},f_{2}) for each γ ∈ Γ (γ̄). The definition of this function is very similar to that of F_{γ}(f_{1},f_{2}) in formula (11.3). They differ only in the indexation of their variables. (The variables of the function F̄_{γ}(f_{1},f_{2}) have indices l_{1},...,l_{|O(γ)|}, and the set of these indices may be different of the set {1,...,|O(γ)|}. But we have defined the U-statistics with a kernel function also in this case.)

It is not difficult to check relation (12.2). We claim that it implies that a U-statistic with kernel function (f_{1}∘ f_{2})γ̄ satisfies the identity

n^{−(k_{1}+k_{2})/2} s(γ̄)!I_{n,s̄(γ̄)}((f_{1}∘ f_{2})γ̄)  (12.4)

= ∑_{γ∈Γ (γ̄)}n^{−(k_{1}+k_{2})/2} n^{|C(γ)|}J_{n}(γ)|O(γ)|!I_{n,|O(γ)|}(F̄_{γ}(f_{1},f_{2}))

with the function F̄_{γ}(f_{1},f_{2}), where C(γ) is the set of closed chains of γ, and J_{n}(γ) is defined as J_{n}(γ) = 1 if |C(γ)| = 0, and

J_{n}(γ) = ∏_{j=1}^{|C(γ)|}( \frac{n − s(γ) + j}{n} ) if |C(γ)| > 0  (12.5)

for all γ̄∈ Γ̄(k_{1},k_{2}).

Relation (12.4) follows from relation (12.2) in the same way as formula (9.4) follows from formula (9.3) in the proof of the Hoeffding decomposition. Let us understand why the coefficient n^{|C(γ)|}J_{n}(γ) appears at the right-hand side of (12.4).

This coefficient can be calculated in the following way. Let us write up the identity

n^{−(k_{1}+k_{2})/2} (f_{1}∘ f_{2})γ̄(ξ_{j_{1}},...,ξ_{j_{s(γ̄)}})

= ∑_{γ∈Γ (γ̄)}n^{−(k_{1}+k_{2})/2} F̄_{γ}(f_{1},f_{2})(ξ_{j_{l_{1}}} ,...,ξ_{j_{l_{|O(γ)|}}})

with the help of (12.2) for all sequences ξ_{j_{1}},...,ξ_{j_{s(γ̄)}}, and let us sum it up for all such sets of arguments (j_{1},...,j_{s(γ̄)}) for which all indices j_{p}, 1 ≤ p ≤ s(γ̄), are different, and 1 ≤ j_{p} ≤ n. Then we get at the left-hand side of the identity the U-statistic

n^{−(k_{1}+k_{2})/2} s(γ̄)!I_{n,s̄(γ̄)}((f_{1}∘ f_{2})γ̄) .

We still have to check that at the right-hand side of this identity we get a sum, where a term of the form n^{−(k_{1}+k_{2})/2} F̄_{γ}(f_{1},f_{2})(ξ_{j_{l_{1}}} ,...,ξ_{j_{l_{|O(γ)|}}}) appears with multiplicity

n^{|C(γ)}J_{n}(γ). Indeed, such a term appears for such vectors (j_{1},...,j_{s(γ̄)}) for which the value of |O(γ)| arguments are fixed, the remaining arguments can take arbitrary value between 1 and n with the only restriction that all coordinates must be different. (The operators P_{v}are applied for these remaining coordinates.) There are n^{|C(γ)|}J_{n}(γ) such vectors. The above observations imply identity (12.4).

Let us observe that k_{1} + k_{2} − 2|C(γ)| = |O(γ)| + W (γ) with the number W (γ) introduced in the formulation of Theorem 11.1. Hence

n^{−(k_{1}+k_{2})/2} n^{|C(γ)|}= n^{−W (γ)/2}n^{−|O(γ)|/2}.

Let us replace the left-hand side of the last identity by its right-hand side in (12.4), and let us sum up the identity we get in such a way for all γ̄∈ Γ̄(k_{1},k_{2}) such that s(γ̄) ≤ n. The identity we get in such a way together with formulas (12.1) and (12.5) imply such a version of identity (11.4) where the kernel functions F_{γ}(f_{1},f_{2}) of the U-statistics at the right-hand side of the equation are replaced by the kernel functions F̄_{γ}(f_{1},f_{2}) defined in (12.3). But we can get the function F_{γ}(f_{1},f_{2}) by reindexing the arguments of the function F̄_{γ}(f_{1},f_{2}). This can be seen by taking the original indexation of the chains of γ and looking at the indexation of the vertices it implies. On the other hand, we know that the reindexation of the variables of the kernel function does not change the value of the U-statistic. Hence I_{n,|O(γ)|}(F_{γ}(f_{1},f_{2})) = I_{n,|O(γ)|}(F̄_{γ}(f_{1},f_{2})), and identity (11.4) holds in its original form.

Clearly, I_{n,|O(γ)|}(F_{γ}(f_{1},f_{2})) = I_{n,|O(γ)|}(SymF_{γ}(f_{1},f_{2})), hence I_{n,|O(γ)|}(F_{γ}(f_{1},f_{2})) can be replaced by I_{n,|O(γ)|}(SymF_{γ}(f_{1},f_{2})) in formula (11.4). Beside this, we have shown that the functions F_{γ}(f_{1},f_{2}) are canonical, and it can be simply shown that they are bounded, if the functions f_{1} and f_{2} have this property. We still have to prove inequalities (11.5) and (11.6).

Inequality (11.5), the estimate of the L_{2}-norm of the function F_{γ}(f_{1},f_{2}) follows from the Schwarz inequality, and actually it agrees with inequality (10.13), proved at the start of Appendix B. Hence its proof is omitted here.

To prove inequality (11.6) let us introduce, similarly to formula (9.2), the operators

(Q̃_{j}h)(x_{1},...,x_{r}) = h(x_{1},...,x_{r}) + ∫ h(x_{1},...,x_{r})μ(dx_{j}), 1 ≤ j ≤ r,

in the space of functions h(x_{1},...,x_{r}) with coordinates in the space (X,X ). Observe that both the operators Q̃_{j} and the operators P_{j} defined in (9.1) are positive, i.e. they map a non-negative function to a non-negative function. Beside this, Q_{j} ≤ Q̃_{j}, and

the norms of the operators \frac{Q̃_{j}}{2} and P_{j} are bounded by 1 both in the L_{1}(μ), the L_{2}(μ) and the supremum norm.

Let us define the function

F̃_{γ}(f_{1},f_{2})(x_{1},...,x_{|O(γ)|})

= ( ∏_{j: β(j)∈C(γ)} P_{j} ∏_{j′: β(j′)∈O_{2}(γ)} Q̃_{j′}) (f_{1}∘ f_{2})_{γ}(x_{q},...,x_{|O(γ)|+|C(γ)|})

with the notation of Chapter 11. The function F̃_{γ}(f_{1},f_{2}) was defined similarly to F_{γ}(f_{1},f_{2}) defined in (11.3) with the help of (f_{1} ∘ f_{2})_{γ} only the operators Q_{j} were replaced by Q̃_{j} in its definition.

The properties of the operators P_{j} and Q̃_{j} listed above together with the condition sup|f_{2}(x_{1},...,x_{k})| ≤ 1 imply that

|F_{γ}(f_{1},f_{2})| ≤ F̃_{γ}(|f_{1}|,|f_{2}|) ≤ F̃_{γ}(|f_{1}|,1),  (12.6)

where ‘≤’ means that the function at the right-hand side is greater than or equal to the function at the left-hand side in all points, and the term 1 in (12.6) denotes the function which equals identically 1. Because of the relation (12.6) to prove relation (11.6) it is enough to show that

∥(F̃_{γ}(|f_{1}|,1)_{γ}∥_{2}

= ∥( ∏_{j: β(j)∈C(γ)} P_{j} ∏_{j′: β(j′),∈O_{2}(γ)} Q̃_{j′}) |f_{1}(x_{α_{γ}((1,1))},...,x_{α_{γ}((1,k_{1}))})|∥_{2}

≤ 2^{|O_{2}(γ)|}∥f_{1}∥_{2} = 2^{W (γ)}∥f_{1}∥_{2}.  (12.7)

But this inequality trivially holds, since the norm of all operators P_{j} in formula (12.7) is bounded by 1, the norm of all operators Q̃_{j} is bounded by 2 in the L_{2}(μ) norm, and |O_{2}(γ)| = W (γ).

Proof of Theorem 11.2. Theorem 11.2 will be proved with the help of Theorem 11.1 by induction with respect to the number m of the terms in the product of the degenerate U-statistics k_{p}!I_{n,k_{p}}(f_{p}), 1 ≤ p ≤ m. It is not difficult to check with the help of Theorem 11.1 and the recursive definition of the functions F_{γ} by applying induction with respect to m that the functions F_{γ}(f_{1},...,f_{m}) are bounded and canonical if the functions f_{1},...,f_{m} satisfy the same properties. We still have to prove the identity (11.11). This will be proved also by induction with respect to m with the help of Theorem 11.1.

For m = 2 formula (11.11) follows from Theorem 11.1, since in this case it agrees with relation (11.4). To prove this formula for m ≥ 3 first we express with the help of our inductive hypothesis the product of the first m − 1 terms in the product of degenerate U-statistics as a sum of degenerate U-statistics. Then we express the product of each term in this sum with the last U-statistic of the product as a sum of U-statistics with the help of Theorem 11.1, and sum up these identities. In such a way we express the product of m degenerate U-statistics in the form of a sum of degenerate U-statistics. We have to show that in such a way we get formula (11.11). In the proof of this statement we shall exploit that in the calculation of the product of the first m−1 U-statistics we have to work with the diagrams γ_{pr} and if we calculate the product of these terms with the m-th the U-statistic, then we calculate with the diagrams γ_{cl}.

To carry out the above program first we observe that a diagram γ ∈ Γ (k_{1},...,k_{m}) is uniquely determined by the pairs of (γ_{pr},γ_{cl}) defined with the help of γ, i.e. if γ,γ′∈ Γ (k_{1},...,k_{m}), and γ ≠ γ′, then either γ_{pr} ≠ γ′_{pr} or γ_{cl} ≠ γ′_{cl}. Hence we can identify each diagram γ ∈ Γ (k_{1},...,k_{m}) with the pair (γ_{pr},γ_{cl}) we defined with its help. Beside this, the pairs of diagrams (γ_{pr},γ_{cl}) satisfy the relations γ_{pr} ∈ Γ (k_{1},...,k_{m−1}) and γ_{cl} ∈ Γ (|O(γ_{pr})|,k_{m}). Moreover, the class of pairs of diagrams (γ_{pr},γ_{cl}), γ ∈ Γ (k_{1},...,k_{m}), have the following characterization. Take all such pairs of diagrams (γ̄,γ̂) for which γ̄∈ Γ (k_{1},...,k_{m−1}) and γ̃∈ Γ (|O(γ̄)|,k_{m}). There is a one to one correspondence between the pairs of diagrams (γ̄,γ̂) with this property and the diagrams γ ∈ Γ (k_{1},...,k_{m}) in such a way that γ̄= γ_{pr} and γ̂= γ_{cl}. (This correspondence depends on the labelling of the open chains of the diagrams γ̄∈ Γ (k_{1},...,k_{m−1}) that we have previously fixed.) It is not difficult to check the above statements, and I leave it to the reader.

Because of our inductive hypothesis we can write by applying relation (11.11) of Theorem 11.2 with parameter m − 1 the identity

∏_{p=1}^{m−1} n^{−k_{p}/2}k_{p}!I_{n,k_{p}}(f_{p}) = ∑_{γ̄∈Γ (k_{1},...,k_{m−1} )}^{′(n,m−1)}( ∏_{p=2}^{m−1} J_{n}(γ̄,p))

n^{−W (γ̄)/2}· n^{−|O(γ̄)|/2} |O(γ̄)|!I_{n,|O(γ̄)|}(Fγ̄(f_{1},...,f_{m−1})).  (12.8)

(Here we use the notations of Chapter 11.)

We get by applying the identity (11.4) of Theorem 11.1 for the product

n^{−|O(γ̄)|/2} |O(γ̄)|!I_{n,|O(γ̄)|}(Fγ̄(f_{1},...,f_{m−1})) · n^{−k_{m}/2}k_{m}!I_{n,k_{m}}(f_{m}),

and by multiplying it with ( ∏_{p=2}^{m−1} J_{n}(γ̄,p)) n^{−W (γ̄)/2} that the identity

( ∏_{p=2}^{m−1} J_{n}(γ̄,p)) n^{−W (γ̄)/2}n^{−|O(γ̄)|/2} O(γ̄)!I_{n,|O(|γ̄|}(Fγ̄(f_{1},...,f_{m−1})

·n^{−k_{m}/2}k_{m}!I_{n,k_{m}}(f_{m})
= ( ∏_{p=2}^{m−1} J_{n}(γ̄,p)) n^{−W (γ̄)/2} ∑_{γ̂∈Γ (|O(γ̄|,k_{m})}^{′(n)} ∏_{j=1}^{|C(γ̂)|}( \frac{n − s(γ̂) + j}{n} )  (12.9)
n^{−W (γ̂)/2}· n^{−|O(γ̂)|/2} |O(γ̂)|!I_{n,|O(γ̂)|}(Fγ̂(Fγ̄(f_{1},...,f_{m−1}),f_{m})).

holds for all γ̄∈ Γ (k_{1},...,k_{m−1}), where ∑_{γ̂∈Γ (|O(γ̄|),k_{m})}^{′(n)}means that summation is taken

for such diagrams γ̂∈ Γ (|O(γ̄)|,k_{m}) for which s(γ̂) = |O(γ̂)| + |C(γ̂)| ≤ n, and ∏_{j=1}^{|C(γ̂|}

equals 1, if |C(γ̂)| = 0.

We shall prove relation (11.11) for the parameter m with the help of relations (12.8) and (12.9).

Let us sum up formula (12.9) for all such diagrams γ̄∈ Γ (k_{1},...,k_{m−1}) for which B_{1}(γ̄,p) + B_{2}(γ̄,p) ≤ n for all 2 ≤ p ≤ m − 1. The numbers B_{1}(·) and B_{2}(·) in these inequalities are the numbers introduced before formula (11.10), only in this case the diagram γ is replaced by γ̄. We imposed those conditions on the terms γ̄ in this summation which appear in the conditions of the summation in ∑^{′(n,m−1)} at the right-hand side of formula (12.8) when it is applied with parameter m − 1. Hence formula (12.8) implies that the sum of the terms at the left-hand side of these identi-

ties equals ∏_{p=1}^{m} n^{−k_{p}/2}k_{p}!I_{n,k_{p}}(f_{p}), i.e. the left-hand side of (11.11) for parameter m.

To prove formula (11.11) for the parameter m it is enough to show that the sum of the right-hand side terms of the above identities equals the right-hand side of (11.11).

In the proof of this relation we shall apply the properties of the pairs of diagrams (γ_{pr},γ_{cl}) coming from a diagram γ ∈ Γ (k_{1},...,k_{m}) mentioned before. Namely, we shall exploit that there is a one to one correspondence between the diagrams γ ∈ Γ (k_{1},...,k_{m}) and pairs of diagrams (γ̄,γ̂), γ̄∈ Γ (k_{1},...,k_{m−1}), γ̂∈ Γ (|O(γ̄)|,k_{m}) in such a way that γ and the pair (γ̄,γ̂) correspond to each other if and only if γ̄= γ_{pr} and γ̂= γ_{cl}. This correspondence enables us to reformulate the statement we have to prove in the following way. Let us rewrite formula (12.9) by replacing γ̄with γ_{pr} and γ̂by γ_{cl}, with that diagram γ ∈ Γ (k_{1},...,k_{m}) for which γ̄= γ_{pr} and γ̂= γ_{cl}. It is enough to show that if we take those modified versions of (12.9) which we get by replacing the pairs (γ̄,γ̂) by the pairs (γ_{pr},γ_{cl}) with some γ ∈ Γ (k_{1},...,k_{m}) and sum up them for those γ for which B_{1}(γ_{pr},p)+B_{2}(γ_{pr},p) ≤ n for all 2 ≤ p ≤ m−1, then the sum of the right-hand side expressions in these identities equals the right-hand side of (11.11).

We shall prove the above identity with the help of the following statements to be verified later.

For all γ ∈ Γ (k_{1},...,k_{m}) the identities W (γ_{pr}) + W (γ_{cl}) = W (γ) and

∏_{p=2}^{m−1} J_{n}(γ_{pr},p) ∏_{j=1}^{|C(γ_{cl})|}( \frac{n − s(γ_{cl}) + j}{n} ) = ∏_{p=2}^{m} J_{n}(γ,p),

hold, where ∏_{j=1}^{|C(γ_{cl})|}= 1 if |C(γ_{cl})| = 0. The inequalities B_{1}(γ,p) + B_{2}(γ,p) ≤ n hold

simultaneously for all 2 ≤ p ≤ m for a diagram γ if and only if the inequalities B_{1}(γ_{pr},p)+B_{2}(γ_{pr},p) ≤ n for all 2 ≤ p ≤ m−1 and s(γ_{cl}) ≤ n hold simultaneously for this γ.

To prove the identity we claimed to hold with the help of the above relations let us first check that we sum up for the same set of γ ∈ Γ (k_{1},...,k_{m}) if we take the sum of modified versions of (12.9) for all γ such that B_{1}(γ_{pr},p)+B_{2}(γ_{pr},p) ≤ n for all 2 ≤ p ≤ m−1 and if we take the ∑^{′(n,m)}at the right-hand side of (11.11). Indeed, in the second case we have to take those diagrams γ for which B_{1}(γ,p)+B_{2}(γ,p) ≤ n for all 2 ≤ p ≤ m, while in the first case we take those diagrams γ for which B_{1}(γ_{pr},p)+B_{2}(γ_{pr},p) ≤ n for all 2 ≤ p ≤ m−1, and s(γ_{cl}) ≤ n. The last condition is contained in a slightly hidden form in the summation ∑^{′(n)}of formula (12.9). Hence the above mentioned relations imply that have to sum up for the same diagrams γ in the two cases.

Beside this, it follows from (11.8) that the same U-statistics appear for a diagram γ ∈ Γ (k_{1},...,k_{m}) in (11.11) and in the modified version of (12.9). We still have to check that they have the same coefficients in the two cases. But this holds, because the previously formulated identities imply that

n^{−(W (γ_{pr})/2}n^{−W (γ_{cl})/2}= n^{−W (γ)/2},
∏_{p=2}^{m−1} J_{n}(γ_{pr},p) ∏_{j=1}^{|C(γ_{cl})|}( \frac{n − s(γ_{cl}) + j}{n} ) = ∏_{p=2}^{m} J_{n}(γ,p)

and n^{−|O(γ_{cl})|/2}|O(γ_{cl})|! = n^{−|O(γ)|/2}|O(γ)|!, since |O(γ)| = |O(γ_{cl})|, as we have seen before.

To complete the proof of the identity it remained to check the relations we applied in the previous argument. We start with the proof of the identity W (γ_{pr}) + W (γ_{cl}) = W (γ) for the function W (·) defined in (11.9).

Let us first remark that W (γ_{cl}) = |O_{2}(γ_{cl})|, where O_{2}(γ_{cl}) is the set of open chains in γ_{cl} with length 2. Beside this if β ∈ γ is such that β ∩ {(m,1),...,(m,k)} = ∅, i.e. if the chain β contains no vertex from the last row of the diagram γ, then ℓ(β) = ℓ(β_{pr}), and c_{γ}(β) = c_{γ_{pr}}(β_{pr}). If β ∩ {(m,1),...,(m,k)} ≠ ∅, then either c_{γ}(β) = 1, ℓ(β_{pr}) = ℓ(β) − 1, and c_{γ_{pr}}(β) = −1 or c_{γ}(β) = −1 and one of the following cases appears. Either ℓ(β) = 1, and the chain β_{pr} does not exists, or ℓ(β) > 1, and ℓ(β_{pr}) = ℓ(β) − 1, c_{γ_{pr}}(β_{pr}) = −1. We get by calculating W (γ) with the help of the above relations that W (γ) = W (γ_{pr}) + |V (γ)|, where V (γ) = {β: β ∈ γ, β ∩ {(m,1),...,(m,k)} ≠ ∅, ℓ(β) > 1, c_{γ}(β) = −1}. Since |V (γ)| = |O_{2}(γ_{cl})|, the above relations imply the desired identity.

To prove the remaining relations first we observe that for each diagram γ ∈ Γ (k_{1},...,k_{m}) and number 2 ≤ p ≤ m − 1 the identities B_{1}(γ_{pr},p) = B_{1}(γ,p) and B_{2}(γ_{pr},p) = B_{2}(γ,p) hold. Beside this, |C(γ_{cl})| = B_{1}(γ,m) and |O(γ_{cl})| = B_{2}(γ,m). The identity about |C(γ_{cl})| simply follows from the definition of γ_{cl} and B_{1}(γ,m). To prove the identity about |O(γ_{cl})| observe that |O(γ_{cl})| = |O(γ)|, and |O(γ)| = B_{2}(γ,m). (Observe that in the case p = m the definition of the set B_{2}(γ,m) becomes simpler, because there is no chain β ∈ γ for which d(β) > m.)

The remaining relations can be deduced from these facts. Indeed, they imply that

J_{n}(γ_{pr},p) = J_{n}(γ,p) for all 2 ≤ p ≤ m−1. Beside this, ∏_{j=1}^{|C(γ_{cl})|}( \frac{n−s(γ_{cl})+j}{n} ) = J_{n}(γ,m)

because of the relations |C(γ_{cl})| = B_{1}(γ,m) |O(γ_{cl})| = B_{2}(γ,m), s(γ_{cl}) = |C(γ_{cl})| + |O(|γ_{cl})| and the definition of J_{n}(γ,m). Hence the identity about the product of the terms J_{n}(γ,p) holds. It can be seen similarly that the relations B_{1}(γ,p)+B_{2}(γ,p) ≤ n holds for all 2 ≤ p ≤ m − 1 if and only if B_{1}(γ_{pr},p) + B_{2}(γ_{pr},p) ≤ n for all 2 ≤ p ≤ m − 1, and B_{1}(γ,m) + B_{2}(γ,m) ≤ n if and only if s(γ_{cl}) ≤ n.

Thus we have proved identity (11.11). To complete the proof of Theorem 11.2 we still have to show that under its conditions F_{γ}(f_{1},...,f_{m}) is a bounded, canonical function. But this follows from Theorem 11.1 and relation (11.8) by a simple induction argument.

Proof of Lemma 11.3. Lemma 11.3 will be proved by induction with respect to the number m of the terms in the product of U-statistics with the help of inequalities (11.5) and (11.6). These relations imply the desired inequality for m = 2. In the case m > 2 we apply the identity (11.8) F_{γ}(f_{1},...,f_{m}) = F_{γ_{cl}}(F_{γ_{pr}}(f_{1},...,f_{m−1}),f_{m}). We have seen that W (γ) = W (γ_{pr}) + W (γ_{cl}), and it is not difficult to show that U(γ) = U(γ_{pr}) + U(γ_{cl}). Hence if U(γ_{cl}) = 0, i.e. if γ_{cl} contains a chain of length 2 with colour −1, then U(γ) = U(γ_{pr}), and an application of (11.8) and (11.6) for the diagram γ_{cl} implies Lemma 11.3 in this case.

If U(γ_{cl}) = 1, then W (γ_{cl}) = 0, U(γ) = U(γ_{pr}) + 1, W (γ) = W (γ_{pr}), and the application of (11.8) and (11.5) for the diagram γ_{cl} implies Lemma 11.3 in this case.

The corollary of Theorem 11.2 is a simple consequence of Theorem 11.2 and Lemma 11.3.

Proof of the corollary of Theorem 11.2. Observe that F_{γ} is a function of |O(γ)| arguments. Hence a coloured diagram γ ∈ Γ (k_{1},...,k_{m}) is in the class of closed diagrams, i.e. γ ∈ Γ̄(k_{1},...,k_{m}) if and only if F_{γ}(f_{1},...,f_{m}) is a constant. Thus formula (11.13) is a simple consequence of relation (11.11) and the observation that EI_{n,|O(γ)|}(F_{γ}(f_{1},...,f_{m})) = 0 if |O(γ)| ≥ 1, i.e. if γ ∉ Γ̄(k_{1},...,k_{m}), and

I_{n,|O(γ)|}(F_{γ}(f_{1},...,f_{m})) = I_{n,0}(F_{γ}(f_{1},...,f_{m})) = F_{γ}(f_{1},...,f_{m})
if γ ∈ Γ̄(k_{1},...,k_{m}).

Relations (11.14) and (11.15) follow from relation (11.13) and Lemma 11.3.

The proof of Theorems 8.3, 8.5 and Example 8.7

In this chapter we prove the estimates on the distribution of a multiple Wiener– Itô integral or degenerate U-statistic formulated in Theorems 8.5 and 8.3, and also present the proof of Example 8.7. Beside this, we prove a multivariate version of Hoeffding’s inequality (Theorem 3.4). The latter result is useful in the estimation of the supremum of a class of degenerate U-statistics. The estimate on the distribution of a multiple random integral with respect to a normalized empirical distribution given in Theorem 8.1 is omitted, because, as it was shown in Chapter 9, this result follows from the estimate of Theorem 8.3 on degenerate U-statistics. We finish this chapter with a separate part Chapter 13 B, where the results proved in this chapter are discussed together with the method of their proofs and some recent results. These new results state that in certain cases the estimates on the tail distribution of Wiener– Itô integrals and U-statistics considered in this chapter can be improved if we have some additional information on the kernel function of these Wiener–Itô integrals or U-statistics.

The proof of Theorems 8.5 and 8.3 is based on a good estimate on high moments of Wiener–Itô integrals and degenerate U-statistics. Such estimates can be proved with the help of the corollaries of Theorems 10.2 and 11.2. This approach slightly differs from the classical proof in the one-variate case. The one-variate version of the above problems is an estimate about the tail distribution of a sum of independent random variables. Such an estimate can be obtained with the help of a good bound on the moment generating function of the sum. This method does not work in the multivariate case, because, as later calculations will show, there is no good estimate on the moment-generating function of U-statistics or multiple Wiener–Itô integrals of order k ≥ 3. Actually, the moment-generating function of a Wiener–Itô integral of order k ≥ 3 is always divergent, because the tail distribution behaviour of such a random integral is similar to that of the k-th power of a Gaussian random variable. On the other hand, good bounds on the moments EZ^{2M} of a random variable Z for all positive integers M (or at least for a sufficiently rich class of parameters M) together with the application of the Markov inequality for Z^{2M} and an appropriate choice of the parameter M yield a good estimate on the tail distribution of Z.

Propositions 13.1 and 13.2 contain some estimates on the moments of Wiener–Itô integrals and degenerate U-statistics.

Proposition 13.1 (Estimate on the moments of Wiener–Itô integrals). Let us consider a function f(x_{1},...,x_{k}) of k variables on some measurable space (X,X ) which satisfies formula (8.15) with some σ-finite non-atomic measure μ. Take the k-fold Wiener–Itô integral Z_{μ,k}(f) of this function with respect to a white noise μ_{W} with reference measure μ. The inequality

E (|k!Z_{μ,k}(f)|)^{2M} ≤ 1 · 3 · 5··· (2kM − 1)σ^{2M} for all M = 1,2,...  (13.1)

holds.

By Stirling’s formula Proposition 13.1 implies that

E(|k!Z_{μ,k}(f)|)^{2M} ≤ \frac{(2kM)!}{2^{kM}(kM)!}σ^{2M} ≤ A( \frac{2}{e}) ^{kM}(kM)^{kM}σ^{2M}  (13.2)

for any A > \sqrt{2} if M ≥ M_{0} = M_{0}(A). Formula (13.2) can be considered as a simpler, better applicable version of Proposition 13.1. It can be better compared with the moment estimate on degenerate U-statistics given in formula (13.3).

Proposition 13.2 provides a similar, but weaker inequality for the moments of normalized degenerate U-statistics.

Proposition 13.2 (Estimate on the moments of degenerate U-statistics). Let us consider a degenerate U-statistic I_{n,k}(f) of order k with sample size n and with a kernel function f satisfying relations (8.1) and (8.2) with some 0 < σ^{2} ≤ 1. Fix a positive number η > 0. There exist some universal constants A < ∞ and C < ∞ such that

E ( n^{−k/2}k!I_{n,k}(f))^{2M} ≤ A(1 + C\sqrt{η})^{2kM}( \frac{2}{e}) ^{kM}(kM)^{kM}σ^{2M}

for all integers M such that 0 ≤ kM ≤ ηnσ^{2}.  (13.3)

In formula (13.3) the constant C can be chosen as C = \sqrt{2}.

Theorem 13.2 yields a good estimate on E (n^{−k/2}k!I_{n,k}(f))^{2M} with a fixed exponent 2M with the choice η = \frac{kM}{nσ^{2}}. With such a choice of the number η for-

mula (13.3) yields an estimate on the moments E (n^{−k/2}k!I_{n,k}(f))^{2M} comparable with the estimate on the corresponding Wiener–Itô integral if M ≤ nσ^{2}, while it yields a much weaker estimate if M ≫ nσ^{2}.

Now I turn to the proof of these propositions.

Proof of Proposition 13.1. Proposition 13.1 can be simply proved by means of the Corollary of Theorem 10.2 with the choice m = 2M, and f_{p} = f for all 1 ≤ p ≤ 2M. Formulas (10.22) and (10.23) yield that

E (k!Z_{μ,k}(f)^{2M}) ≤ ( ∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}))^{M}|Γ_{2M}(k)|
≤ |Γ_{2M}(k)|σ^{2M},

where |Γ_{2M}(k)| denotes the number of closed diagrams γ in the class Γ̄(\underbrace{k,...,k}_{2M times})

introduced in the corollary of Theorem 10.2. Thus to complete the proof of Proposition 13.1 it is enough to show that |Γ_{2M}(k)| ≤ 1 · 3 · 5··· (2kM − 1). But this can easily be seen with the help of the following observation. Let Γ̄_{2M}(k) denote the class of all graphs with vertices (l,j), 1 ≤ l ≤ 2M, 1 ≤ j ≤ k, such that from all vertices (l,j) exactly one edge starts, all edges connect different vertices, but edges connecting vertices (l,j) and (l,j′) with the same first coordinate l are also allowed. Let |Γ̄_{2M}(k)| denote the number of graphs in Γ̄_{2M}(k). Then clearly |Γ_{2M}(k)| ≤ |Γ̄_{2M}(k)|. On the other hand, |Γ̄_{2M}(k)| = 1 · 3 · 5··· (2kM − 1). Indeed, let us list the vertices of the graphs from Γ̄_{2M}(k) in an arbitrary way. Then the first vertex can be paired with another vertex in 2kM − 1 way, after this the first vertex from which no edge starts can be paired with 2kM − 3 vertices from which no edge starts. By following this procedure the next edge can be chosen 2kM − 5 ways, and by continuing this calculation we get the desired formula.

Proof of Proposition 13.2. Relation (13.3) will be proved by means of relations (11.14) and (11.15) in the Corollary of Theorem 11.2 with the choice m = 2M and f_{p} = f for all 1 ≤ p ≤ 2M. Let us take the class of closed coloured diagrams Γ (k,M) = Γ̄(\underbrace{k,...,k}_{2Mtimes}). This will be partitioned into subclasses Γ (k,M,r),

1 ≤ r ≤ kM, where Γ (k,M,r) contains those closed diagrams γ ∈ Γ (k,M) for which W (γ) = 2r. Let us recall that W (γ) was defined in (11.9), and in the case of closed diagrams W (γ) = ∑_{β∈γ}(ℓ(β) − 2). For a diagram γ ∈ Γ (k,M), W (γ) is an even num-

ber, since W (γ) + 2s(γ) = 2kM, i.e. W (γ) = 2r with r = kM − s, where s = s(γ) denotes the number of chains in γ.

First we prove an estimate about the cardinality of Γ (k,M,r). We claim that there exists a universal constant A < ∞ such that

|Γ (k,M,r)|≤ \binom{2kM}{2r} 1 · 3 · 5··· (2kM − 2r − 1)(kM − r)^{2r}  (13.4)

≤ A( \frac{2}{e}) ^{kM}\binom{2kM}{2r} 2^{−r} (kM)^{kM+r} for all 0 ≤ r ≤ kM

with some universal constant A < ∞.

To prove formula (13.4) let us first observe that |Γ (k,M,r)| can be bounded from above with the number of such partitions of a set with 2kM points which consists of s = kM − r sets containing at least two points. Indeed, for each γ ∈ Γ (k,M,r) the chains of the diagram γ yield a partition of the set {(p,r): 1 ≤ p ≤ 2M, 1 ≤ k ≤ r} consisting of 2r sets such that each of them contains at least two points. Moreover, the partition given in such a way determines the chains of γ, because the vertices of a chain are listed in a prescribed order. Namely, the indices of the rows which contain them follow each other in increasing order. This implies that we can correspond to each diagram γ ∈ Γ (k,M,r) a different partition of a set of 2Mk elements with the prescribed properties.

The number of the partitions with the above properties can be bounded from above in the following way. Let us calculate the number of possibilities for choosing s = kM − r disjoint subsets of cardinality two from a set of cardinality 2kM, and multiply this number with the possibility of attaching each of the remaining 2r points of the original set to one of these sets of cardinality 2.

We can choose these sets of cardinality 2 in \binom{2kM}{2r}1 · 3 · 5··· (2kM − 1) ways, since we can choose the union of these sets, which consists of 2kM − 2r points in \binom{2kM}{2kM−2r} = \binom{2kM}{2r} ways, and then we can choose the pair of the first element in 2kM − 2r − 1 ways, then the pair of the first still not chosen element in 2kM − 2r −3 ways, and continuing this procedure we get the above formula for the number of choices for these sets of cardinality 2. Then the remaining 2r points of the original set can be put in (kM − r)^{2r} ways in one of these kM − r sets of cardinality 2. The above relations imply the first inequality of formula (13.4).

To get the second inequality observe that by the Stirling formula 1·3·5··· (2kM − 2r −1) = \frac{(2kM−2r)!}{2^{kM−r}(kM−r)!} ≤ A( \frac{2}{e})^{kM−r} (kM −r)^{kM−r} with some universal constant A <

∞. Beside this, we can write (kM − r)^{kM+r} ≤ (kM)^{r}(kM − r)^{kM} = (kM)^{kM+r}(1 − \frac{r}{kM})^{kM}≤ e^{−r} (kM)^{kM+r}. These estimates imply the second inequality in (13.4).

We prove the estimate (13.3) with the help of the relations (11.14), (11.15) and (13.4). First we estimate the term n^{−W (γ)/2}|F_{γ}| for a diagram γ ∈ Γ (k,M,r) under the conditions kM ≤ ηnσ^{2} and σ^{2} ≤ 1 with the help of relation (11.15).

In this case we can write |U(γ)| ≥ 2M − W (γ) = 2M − 2r for the function U(γ) defined in (11.12). Hence by relation (11.15)

n^{−W (γ)/2}|F_{γ}| ≤ 2^{2r}n^{−r} σ^{|U(γ)|}≤ 2^{2r}(nσ^{2})^{−r} σ^{2M} ≤ η^{r}2^{2r}(kM)^{−r} σ^{2M}

for γ ∈ Γ (k,M,r) because of the conditions kM ≤ ηnσ^{2} and σ^{2} ≤ 1.

This estimate together with relation (11.14) imply that for kM ≤ ηnσ^{2}

E ( n^{−k/2}k!I_{n,k}(f_{k}))^{2M} ≤ ∑_{γ∈Γ (k,M)}n^{−W (γ)/2}· |F_{γ}|

≤ ∑_{r=0}^{kM}|Γ (k,M,r)|η^{r}2^{2r}(kM)^{−r} σ^{2M}.

Hence by formula (13.4)

E ( n^{−k/2}k!I_{n,k}(f_{k}))^{2M} ≤ A( \frac{2}{e}) ^{kM}(kM)^{kM}σ^{2M} ∑_{r=0}^{kM} \binom{2kM}{2r} ( \sqrt{2η})^{2r}

≤ A( \frac{2}{e}) ^{kM}(kM)^{kM}σ^{2M}( 1 + \sqrt{2}\sqrt{η}) ^{2kM}

if 0 ≤ kM ≤ ηnσ^{2}. Thus we have proved Proposition 13.2 with C = \sqrt{2}.

It is not difficult to prove Theorem 8.5 with the help of Proposition 13.1.

Proof of Theorem 8.5. By formula (13.2) which is a consequence of Proposition 13.1 and the Markov inequality

P (|k!Z_{μ,k}(f)| > u) ≤ \frac{E (k!Z_{μ,k}(f))^{2M}}{u^{2M}} ≤ A( \frac{2kMσ^{2/k}}{eu^{2/k}}) ^{kM}  (13.5)

with some constant A > \sqrt{2} if M ≥ M_{0} with some constant M_{0} = M_{0}(A), and M is an integer.

Put M̄= M̄(u) = \frac{1}{2k}( \frac{u}{σ})^{2/k}, and M = M(u) = [M̄], where [x] denotes the integer

part of a real number x. Choose some number u_{0} such that \frac{1}{2k}( \frac{u_{0}}{σ})^{2/k} ≥ M_{0} + 1. Then relation (13.5) can be applied with M = M(u) for u ≥ u_{0}, and this yields that

P (|k!Z_{μ,k}(f)| > u) ≤ A( \frac{2kMσ^{2/k}}{eu^{2/k}}) ^{kM}≤ e^{−kM} ≤ Ae^{k}e^{−kM̄}

= Ae^{k}exp\{ −\frac{1}{2} ( \frac{u}{σ})^{2/k}\} if u ≥ u_{0}.  (13.6)

Relation (13.6) means that relation (8.17) holds for u ≥ u_{0} with the pre-exponential coefficient Ae^{k}. Beside this u_{0} ≤ const. By enlarging this coefficient if it is needed it can be guaranteed that relation (8.17) holds for all u > 0. Theorem 8.5 is proved.

Theorem 8.3 can be proved similarly by means of Proposition 13.2. Nevertheless, the proof is technically more complicated, since in this case the optimal choice of the parameter in the Markov inequality cannot be given in such a direct form as in the proof of Theorem 8.5. In this case the Markov inequality is applied with an almost optimal choice of the parameter M.

Proof of Theorem 8.3. The Markov inequality and relation (13.3) with η = \frac{kM}{nσ^{2}} imply that

P(n^{−k/2}|k!I_{n,k}(f)| > u) ≤ \frac{E (n^{−k/2}k!I_{n,k}(f))^{2M}}{u^{2M}}  (13.7)

≤ A( \frac{1}{e} · 2kM ( 1 + C \frac{\sqrt{kM}}{\sqrt{n}σ}) ^{2}( \frac{σ}{u})^{2/k})^{kM}

for all integers M ≥ 0.

Relation (8.12) will be proved with the help of estimate (13.7) under the condition 0 ≤ \frac{u}{σ} ≤ n^{k/2}σ^{k}. To this end let us introduce the number M̄ by means of the formula

kM̄= \frac{1}{2} ( \frac{u}{σ})^{2/k} \frac{1}{1 + \frac{B}{\sqrt{n}σ} ( \frac{u}{σ})^{1/k}} = \frac{1}{2} ( \frac{u}{σ})^{2/k} \frac{1}{1 + B(un^{−k/2}σ^{−(k+1)} )^{1/k}}

with a sufficiently large number B = B(C) > 0 and M = [M̄], where [x] means the integer part of the number x.

Observe that \sqrt{kM̄}≤ ( \frac{u}{σ})^{1/k}, \frac{\sqrt{kM̄}}{\sqrt{n}σ} ≤ ( un^{−k/2}σ^{−(k+1)} )^{1/k} ≤ 1, and

( 1 + C \frac{\sqrt{kM̄}}{\sqrt{n}σ}) ^{2} ≤ 1 + B\frac{\sqrt{kM̄}}{\sqrt{n}σ} ≤ 1 + B( un^{−k/2}σ^{−(k+1)} )^{1/k}

with a sufficiently large B = B(C) > 0 if \frac{u}{σ} ≤ n^{k/2}σ^{k}. Hence

\frac{1}{e} · 2kM ( 1 + C \frac{\sqrt{kM}}{\sqrt{n}σ}) ^{2}( \frac{σ}{u})^{2/k} ≤ \frac{1}{e} · 2kM̄( 1 + C \frac{\sqrt{kM̄}}{\sqrt{n}σ}) ^{2}( \frac{σ}{u})^{2/k}

= \frac{1}{e} · \frac{( 1 + C \frac{\sqrt{kM̄}}{\sqrt{n}σ}) ^{2}}{1 + B(un^{−k/2}σ^{−(k+1)} )^{1/k}} ≤ \frac{1}{e}  (13.8)

if \frac{u}{σ} ≤ n^{k/2}σ^{k}. Inequalities (13.7) and (13.8) together yield that

P(n^{−k/2}k!|I_{n,k}(f)| > u) ≤ Ae^{−kM} ≤ Ae^{k}e^{−kM̄}

if 0 ≤ \frac{u}{σ} ≤ n^{k/2}σ^{k}. Hence the choice of the number M̄implies that inequality (8.12) holds with the pre-exponential constant Ae^{k} and the sufficiently large but fixed number B > 0. Theorem 8.3 is proved.

Remark. One would like to understand why the introduction of the quantities M̄ and M in the proof of Theorem 8.3 was a good choice. The natural choice for M would have been that number where the right-hand side expression in (13.7) takes its minimum. But we cannot calculate this number in a simple way. Hence we chose instead a sufficiently good and simple approximation for it. We get a first order approximation of this quantity if we consider the minimum of the simplified expres-

sion we get by dropping the factor ( 1 + C \frac{\sqrt{kM}}{\sqrt{n}σ}) ^{2} from the formula at the right-hand

side of (13.7). We get in such a way the approximation M_{0} = \frac{1}{2k}(\frac{u}{σ})^{2/k}, but this is a not good enough choice of the number M for our purposes. We get a better approximation by determing the place of minimum of the expression we get by replacing the number M with the number M_{0} in the factor we omitted in the previous approximation, i.e. we look for the place of minimum of

A( \frac{1}{e} · 2kM ( 1 + C \frac{\sqrt{kM_{0}}}{\sqrt{n}σ}) ^{2}( \frac{σ}{u})^{2/k})^{kM}

= A( \frac{1}{e} · 2kM ( 1 + \frac{C}{\sqrt{2n}σ} ( \frac{u}{σ})^{1/k}) ^{2}( \frac{σ}{u})^{2/k})^{kM}.

This suggests the approximation M_{1} = \frac{1}{2k}( \frac{u}{σ})^{2/k} _{\frac{1}{( 1+ \frac{C}{\sqrt{2n}σ} ( \frac{u}{σ})^{1/k}) ^{2}}} for the place of

minimum we are looking for. We can choose a similar expression for the parameter M which is almost as good as this number, but it is simpler to work with it. To find it observe that under the conditions of Theorem 8.3 we commit a small error by replacing the term (1 + \frac{C}{\sqrt{2n}σ} (\frac{u}{σ})^{1/k})^{2} in the denominator of the formula defin-

ing M_{1} by 1 + \frac{2C}{\sqrt{2n}σ} (\frac{u}{σ})^{1/k}. To see this observe that the condition \frac{u}{σ} ≤ n^{k/2}σ^{k} of

Theorem 8.3 implies that \frac{1}{\sqrt{n}σ} (\frac{u}{σ})^{1/k} ≤ 1. Moreover, in the really interesting cases this expression is very close to zero. This suggests to expand the above square, and make an approximation by omitting the quadratic term. We can try to choose the number M obtained in such a way in the proof of Theorem 8.3. Moreover, it is useful to replace the parameter C with another number with which we can work better. It turned out that we can work better if the number C is replaced with another large coefficient. This led to the introduction of the quantity kM̄= \frac{1}{2} ( \frac{u}{σ})^{2/k} _{\frac{1}{1+ \frac{B}{\sqrt{n}σ} (\frac{u}{σ})^{1/k}}}

with a sufficiently large (but fixed) number B in the proof of Theorem 8.3.

Example 8.7 is a relatively simple consequence of Itô’s formula for multiple Wiener–Itô integrals.

Proof of Example 8.7. We may restrict our attention to the case k ≥ 2. Itô’s formula for multiple Wiener-Itô integrals, more explicitly relation (10.25), implies that the random variable k!Z_{μ,k}(f) can be expressed as k!Z_{μ,k}(f) = σH_{k}(∫ f_{0}(x)μ_{W}(dx)) = σH_{k}(η), where H_{k}(x) is the k-th Hermite polynomial with leading coefficient 1, and η = ∫ f_{0}(x)μ_{W}(dx) is a standard normal random variable. Hence we get by exploiting that the coefficient of x^{k−1} in the polynomial H_{k}(x) is zero that P(k!|Z_{μ,k}(f)| > u) = P(|H_{k}(η)| ≥ \frac{u}{σ}) ≥ P (|η^{k}| − D|η^{k−2}| > \frac{u}{σ}) with a sufficiently large constant D > 0 if \frac{u}{σ} > 1. There exist such positive constants A and B for which

P ( |η^{k}| − D|η^{k−2}| > \frac{u}{σ}) ≥ P ( |η^{k}| > \frac{u}{σ} + A( \frac{u}{σ})^{(k−2)/k}) if \frac{u}{σ} > B.

Hence

P(k!|Z_{μ,k}(f)| > u) ≥ P ( |η| > ( \frac{u}{σ})^{1/k}( 1 + A( \frac{u}{σ})^{−2/k} ) )

≥ \frac{C̄exp\{−\frac{1}{2} ( \frac{u}{σ})^{2/k}\}}{( \frac{u}{σ})^{1/k}+ 1}

with an appropriate C̄> 0 if \frac{u}{σ} > B. Since P(k!|Z_{μ,k}(f)| > 0) > 0, the above inequality also holds for 0 ≤ \frac{u}{σ} ≤ B if the constant C̄> 0 is chosen sufficiently small. This means that relation (8.19) holds.

Next we prove a multivariate version of Hoeffding’s inequality. Before its formulation some notations will be introduced.

Let us fix two positive integers k and n and some real numbers a(j_{1},...,j_{k}) for all sequences of arguments {j_{1},...,j_{k}} such that 1 ≤ j_{l} ≤ n, 1 ≤ l ≤ k, and j_{l} ≠ j_{l′} if l ≠ l′.

With the help of the above real numbers a(·) and a sequence of independent random variables ε_{1},...,ε_{n}, P(ε_{j} = 1) = P(ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, the random variable

V = ∑_{(j_{1},...,j_{k}): 1≤j_{l}≤n for all 1≤l≤k, j_{l}≠j_{l′} if l≠l′} a(j_{1},...,j_{k})ε_{j_{1}} ··· ε_{j_{k}}  (13.9)

and number

S^{2} = ∑_{(j_{1},...,j_{k}): 1≤j_{l}≤n for all 1≤l≤k, j_{l}≠j_{l′} if l≠l′} a^{2}(j_{1},...,j_{k}).  (13.10)

will be introduced.

With the help of the above notations the following result can be formulated.

Theorem 13.3 (The multivariate version of Hoeffding’s inequality). The random variable V defined in formula (13.9) satisfies the inequality

P(|V | > u) ≤ C exp\{ −\frac{1}{2} ( \frac{u}{S})^{2/k}\} for all u ≥ 0  (13.11)

with the constant S defined in (13.10) and some constants C > 0 depending only on the parameter k in the expression V.

Theorem 13.3 will be proved by means of two simple lemmas. Before their formulation the random variable

Z = ∑_{(j_{1},...,j_{k}): 1≤j_{l}≤n for all 1≤l≤k, j_{l}≠j_{l′} if l≠l′} |a(j_{1},...,j_{k})|η_{j_{1}} ··· η_{j_{k}}  (13.12)

will be introduced, where η_{1},...,η_{n} are independent random variables with standard normal distribution, and the numbers a(j_{1},...,j_{k}) agree with those in formula (13.9). The following lemmas will be proved.

Lemma 13.4. The random variables V and Z introduced in (13.9) and (13.12) satisfy the inequality

EV^{2M} ≤ EZ^{2M} for all M = 1,2,....

Lemma 13.5. The random variable Z defined in formula (13.12) satisfies the inequality

EZ^{2M} ≤ 1 · 3 · 5··· (2kM − 1)S^{2M} for all M = 1,2,...  (13.13)

with the constant S defined in formula (13.10).

Proof of Lemma 13.4. We can write, by carrying out the multiplications in the expressions EV^{2M} and EZ^{2M}, by exploiting the additive and multiplicative properties of the expectation for sums and products of independent random variables together with the identities Eε_{j}^{2k+1} = 0 and Eη_{j}^{2k+1} = 0 for all k = 0,1,... that

EV^{2M} = ∑_{\begin{matrix}(j_{1},...,j_{l},m_{1},...,m_{l}): \\ 1≤j_{s}≤n, m_{s}≥1, 1≤s≤l, m_{1}+··· +m_{l}=kM \end{matrix}} A(j_{1},...,j_{l},m_{1},...,m_{l})Eε_{j_{1}}^{2m_{1}}··· Eε_{j_{l}}^{2m_{l}}(13.14)

and

EZ^{2M} = ∑_{\begin{matrix}(j_{1},...,j_{l},m_{1},...,m_{l}): \\ 1≤j_{s}≤n, m_{s}≥1, 1≤s≤l, m_{1}+··· +m_{l}=kM \end{matrix}}B(j_{1},...,j_{l},m_{1},...,m_{l})Eη_{j_{1}}^{2m_{1}}··· Eη_{j_{l}}^{2m_{l}} (13.15)

with some coefficients A(j_{1},...,j_{l},m_{1},...,m_{l}) and B(j_{1},...,j_{l},m_{1},...,m_{l}) such that

|A(j_{1},...,j_{l},m_{1},...,m_{l})| ≤ B(j_{1},...,j_{l},m_{1},...,m_{l}).  (13.16)

The coefficients A(·,·,·) and B(·,·,·) could be expressed explicitly, but we do not need such a formula. What is important for us is that A(·,·,·) can be expressed as the sum of certain terms, and B(·,·,·) as the sum of the absolute value of the same terms. Hence relation (13.16) holds. Since Eε_{j}^{2m}≤ Eη_{j}^{2m} for all parameters j and m formulas (13.14), (13.15) and (13.16) imply Lemma 13.4.

Proof of Lemma 13.5. Let us consider a white noise W (·) on the unit interval [0,1] with the Lebesgue measure λ on [0,1] as its reference measure, i.e. let us take a set of Gaussian random variables W (A) indexed by the measurable sets A ⊂ [0,1] such that EW (A) = 0, EW (A)W (B) = λ(A∩B) with the Lebesgue measure λ for all measurable subsets of the interval [0,1]. Let us introduce n orthonormal functions φ_{1}(x),...,φ_{n}(x) with respect to the Lebesgue measure on the interval [0,1], and define the random variables η_{j} = ∫ φ_{j}(x)W (dx), 0 ≤ j ≤ n. Then η_{1},...,η_{n} are independent random variables with standard normal distribution, hence we may assume that they appear in the definition of the random variable Z in formula (13.12). Beside this, the identity η_{j_{1}} ··· η_{j_{k}} = ∫ φ_{j_{1}}(x_{1})··· φ_{j_{k}}(x_{k})W (dx_{1})...W (dx_{k}) holds for all k-tuples (j_{1},...,j_{k}) such that 1 ≤ j_{s} ≤ n for all 1 ≤ s ≤ k, and the indices j_{1},…, j_{s} are different. This identity follows from Itô’s formula for multiple Wiener–Itô integrals formulated in formula (10.24) of Theorem 10.3.

Hence the random variable Z defined in (13.12) can be written in the form

Z = ∫ f(x_{1},...,x_{k})W (dx_{1})...W (dx_{k})

with the function

f(x_{1},...,x_{k}) = ∑_{(j_{1},...,j_{k}): 1≤j_{l}≤n for all 1≤l≤k, j_{l}≠j_{l′} if l≠l′} |a(j_{1},...,j_{k})|φ_{j_{1}}(x_{1})··· φ_{j_{k}}(x_{k}).

Because of the orthogonality of the functions φ_{j}(x)

S^{2} = ∫_{[0,1]^{k}}f^{2}(x_{1},...,x_{k})dx_{1}... dx_{k}.

Lemma 13.5 is a straightforward consequence of the above relations and formula (13.1) in Proposition 13.1.

Proof of Theorem 13.3. The proof of Theorem 13.3 with the help of Lemmas 13.4 and 13.5 is an almost word for word repetition of the proof of Theorem 8.5. By Lemma 13.4 inequality (13.13) remains valid if the random variable Z is replaced by the random variable V at its left-hand side. Hence the Stirling formula yields that

EV^{2M} ≤ EZ^{2M} ≤ \frac{(2kM)!}{2^{kM}(kM)!}S^{2M} ≤ C ( \frac{2}{e}) ^{kM}(kM)^{kM}S^{2M}

for any C ≥ \sqrt{2} if M ≥ M_{0}(A). As a consequence, by the Markov inequality the estimate

P(|V | > u) ≤ \frac{EV^{2M}}{u^{2M}}≤ C ( \frac{2kM}{e}( \frac{S}{u}) ^{2/k}) ^{kM}  (13.17)

holds for all C > \sqrt{2} if M ≥ M_{0}(C). Put kM̄= kM̄(u) = \frac{1}{2} ( \frac{u}{S})^{2/k} and M = M(u) = [M̄], where [x] denotes the integer part of the number x. Let us choose a threshold number u_{0} by the identity \frac{1}{2k}( \frac{u_{0}}{S})^{2/k} = M_{0}(C) + 1. Formula (13.17) can be applied with M = M(u) for u ≥ u_{0}, and it yields that

P(|V | > u) ≤ Ce^{−kM} ≤ Ce^{k}e^{−kM̄} = Ce^{k}exp\{ −\frac{1}{2} ( \frac{u}{S})^{2/k}\} if u ≥ u_{0}.

The last inequality means that relation (13.11) holds for u ≥ u_{0} if the constant C is replaced by Ce^{k} in it. With the choice of a sufficiently large constant C relation (13.11) holds for all u ≥ 0. Theorem 13.3 is proved.

13. B) A short discussion about the methods and results.

A comparison of Theorem 8.5 and Example 8.7 shows that the estimate (8.17) is sharp. At least no essential improvement of this estimate is possible which holds for all Wiener–Itô integrals with a kernel function f satisfying the conditions of Theorem 8.5. This fact also indicates that the bounds (13.1) and (13.2) on high moments of Wiener–Itô integrals are sharp. It is worth while comparing formula (13.2) with the estimate of Proposition 13.2 on moments of degenerate U-statistics.

Let us consider a normalized k-fold degenerate U-statistic n^{−k/2}k!I_{n,k}(f) with some kernel function f and a μ-distributed sample of size n. Let us compare its moments with those of a k-fold Wiener–Itô integral k!Z_{μ,k}(f) with the same kernel function f with respect to a white noise μ_{W}with reference measure μ. Let σ denote the L_{2}-norm of the kernel function f. If M ≤ εnσ^{2} with a small number ε > 0, then Proposition 13.2 (with an appropriate choice of the parameter η which is small in this case) provides an almost as good bound on the 2M-th moment of the normalized U-statistic as Proposition 13.1 does on the 2M-th moment of the corresponding Wiener–Itô integral. In the case M ≤ Cnσ^{2} with some fixed (not necessarily small) number C > 0 the 2M-th moment of the normalized U-statistic can be bounded by C(k)^{M} times the natural estimate on the 2M-th moment of the Wiener–Itô integral with some constant C(k) > 0 depending only on the number C. This can be so interpreted that in this case the estimate on the moments of the normalized U-statistic is weaker than the estimate on the moments of the Wiener–Itô integral, but they are still comparable. Finally, in the case M ≫ nσ^{2} the estimate on the 2M-th moment of the normalized U-statistic is much worse than the estimate on the 2M-th moment of the Wiener–Itô integral.

A similar picture arises if the distribution of the normalized degenerate Ustatistic

F_{n}(u) = P(n^{−k/2}|k!I_{n,k}(f)| > u)

is compared to the distribution of the Wiener–Itô integral

G(u) = P(|k!Z_{μ,k}(f)| > u).

In the case 0 ≤ u ≤ εn^{k/2}σ^{k+1} with a small ε > 0 Theorem 8.3 yields an almost as good estimate for the probability F_{n}(u) as Theorem 8.5 yields for G(u). In the case 0 ≤ u ≤ n^{k/2}σ^{k+1} these results yield similar bound for F_{n}(u) and G(u), only in the exponent of the estimate on F_{n}(u) in formula (8.12) a worse constant appears. Finally, if u ≫ n^{k/2}σ^{k+1}, then — as Example 8.8 shows, at least in the case k = 2,

— the (tail) distribution function F_{n}(u) satisfies a much worse estimate than the function G(u).

A similar picture arose in the one-variate version of this problem discussed in Chapter 3, where the normalized sums of independent random variables were investigated, and their tail-distributions were compared to that of a normally distributed random variable. To understand this similarity better it is useful to recall Theorem 10.4, i.e. the limit theorem about normalized degenerate U-statistics. Theorems 8.3 and 8.5 enable us to compare the tail behaviour of normalized degenerate U-statistics with their limit presented in the form of multiple Wiener–Itô integrals, while the one-variate versions of these results compare the distribution of sums of independent random variables with their Gaussian limit.

The proofs of the above results show that good bounds on the moments of degenerate U-statistics and multiple Wiener–Itô provide a good estimate on their distribution. To understand the behaviour of high moments of degenerate U-statistics better it is useful to have a closer look at the simplest case k = 1, when the moments of sums of independent random variables with expectation zero are considered.

Let us consider a sequence of independent and identically distributed random

variables ξ_{1},...,ξ_{n} with expectation zero, take their sum S_{n} = ∑_{j=1}^{n} ξ_{j}, and let us try

to give a good estimate on the moments ES_{n}^{2M} for all M = 1,2,.... Because of the independence of the random variables ξ_{j} and the condition Eξ_{j} = 0 the identity

ES_{n}^{2M} = ∑_{\begin{matrix}(j_{1},...,j_{s},l_{1},...,l_{s}) \\ j_{1}+··· +j_{s}=2M,j_{u}≥2, for all 1≤u≤s \\ 1≤l_{1}<l_{2}<··· <l_{s}≤n \end{matrix}}  \frac{(2M)!}{j_{1}!··· j_{s}!} Eξ_{l_{1}}^{j_{1}} ··· Eξ_{l_{s}}^{j_{s}}  (13.18)

holds. Simple combinatorial considerations suggest that the main contribution to the right-hand side of (13.18) is given by such vectors (j_{1},...,j_{M}; l_{1},...,l_{M}) for which j_{u} = 2 for all 1 ≤ u ≤ M. Their contribution is \binom{n}{M} \frac{(2M)!}{2^{M}}(Eξ_{1}^{2})^{M} ∼ n^{M} \frac{(2M)!}{2^{M}M!}(Eξ_{1}^{2})^{M}. The last asymptotic relation holds if the number n of terms in the random sum S_{n} is sufficiently large. The above considerations suggest that under not too restrictive conditions ES_{n}^{2M} ∼ (nσ^{2})^{M} \frac{(2M)!}{2^{M}M!} = Eη_{nσ^{2}}^{2M}, where σ^{2} = Eξ^{2} is the variance of the terms in the sum S_{n}, and η_{u} denotes a random variable with normal distribution with expectation zero and variance u. The question arises when the above heuristic argument gives a valid estimate.

For the sake of simplicity let us restrict our attention to the case when the absolute value of the random variables ξ_{j} is bounded by 1. Let us observe that even in this case the above heuristic argument holds only under the condition that the variance σ^{2} of the random variables ξ_{j} is not too small. Indeed, let us consider such random variables ξ_{j}, for which P(ξ_{j} = 1) = P(ξ_{j} = −1) = \frac{σ^{2}}{2} , P(ξ_{j} = 0) = 1−σ^{2}. Then these random variables ξ_{j} have variance σ^{2}, and the contribution of the terms Eξ_{j}^{2M}, 1 ≤ j ≤ n, to the sum in (13.18) equals nσ^{2}. If σ^{2} is very small, then it may

happen that nσ^{2} ≫ (nσ^{2})^{M} \frac{(2M)!}{2^{M}M!}, and the approximation given for ES_{n}^{2M} in the previous paragraph does not hold any longer. Hence the asymptotic relation for a very high moment ES_{n}^{2M} suggested by the above heuristic argument may only hold if the variance σ^{2} of the summands satisfies an appropriate lower bound.

In the proof of Proposition 13.2 a similar picture appears in a hidden way. In the calculation of the moments of a degenerate U-statistic the contribution of certain (closed) diagrams, more precisely of some integrals defined with their help, has to be estimated. Some of these diagrams (those in which all chains have length 2) appear also in the calculation of the moments of multiple Wiener–Itô integrals. In the calculation of the moments of sums of independent random variables the terms consisting of products of second moments play a similar role in the sum in formula (13.18) as the ‘nice’ diagrams consisting of chains of length 2 play in the calculation of the moments of degenerate U-statistics in formula (11.14). In nice cases the remaining diagrams (multiplied with their small coefficients in formula (11.14) do not give a greater contribution to the moments of degenerate U-statistics than these ‘nice’ diagrams, and we get an almost as good bound for the moments of a normalized degenerate U-statistic as for the moments of the corresponding multiple Wiener–Itô integral. The proof of Proposition 13.2 shows that such a situation appears under very general conditions.

Let me also remark that there is an essential difference between the tail behaviour of Wiener–Itô integrals and normalized degenerate U-statistics. A good estimate can be given on the tail distribution of Wiener–Itô integrals which depends only on the L_{2}-norm of the kernel function, while in the case of normalized degenerate Ustatistics the corresponding estimate depends not only on the L_{2}-norm but also on the L_{∞} norm of the kernel function. In Theorem 8.3 such an estimate is proved.

For k ≥ 2 the distribution of k-fold Wiener-Itô integrals are not determined by the L_{2}-norm of their kernel functions. This is an essential difference between Wiener– Itô integrals of order k ≥ 2 and k = 1. In the case k = 1 a Wiener–Itô integral is a Gaussian random variable with expectation zero, and its variance equals the square of the L_{2}-norm of its kernel function. Hence its distribution is completely determined by the L_{2}-norm of its kernel function. On the other hand, the distribution of a Wiener–Itô integral of order k ≥ 2 is not determined by its variance. Theorem 8.5 yields a ‘worst case’ estimate on the distribution of Wiener–Itô integrals if we have a bound on their variance. In the statistical problems which were the main motivation for this work we need such estimates, but it may be interesting to know what kind of estimates are known about the distribution of a multiple Wiener–Itô integral or degenerate U-statistic if we have some additional information about its kernel function. Some results will be mentioned in this direction, but most technical details will be omitted from our discussion.

H. P. Mc. Kean proved the following lower bound on the distribution of multiple Wiener–Itô integrals. (See [32] or [44].)

Theorem 13.6 (Lower bound on the tail distribution of Wiener–Itô integrals). All k-fold Wiener–Itô integrals Z_{μ,k}(f) satisfy the inequality

P(|Z_{μ,k}(f)| > u) > Ke^{−Au^{2/k}}  (13.19)

with some numbers K = K(f,μ) > 0 and A = A(f,μ) > 0.

The constant A in the exponent Au^{2/k} of formula (13.19) is always finite, but Mc. Kean’s proof yields no explicit upper bound on it. The following example shows that in certain cases if we fix the constant K in relation (13.19), then this inequality holds only with a very large constant A > 0 even if the variance of the Wiener–Itô integral equals 1.

Take a probability measure μ and a white noise μ_{W} with reference measure μ on a measurable space (X,X ), and let φ_{1},φ_{2},... be a sequence of orthonormal functions on (X,X ) with respect to this measure μ. Define for all L = 1,2,..., the function

f(x_{1},...,x_{k}) = f_{L}(x_{1},...,x_{k}) = (k!)^{1/2}L^{−1/2} ∑_{j=1}^{L} φ_{j}(x_{1})··· φ_{j}(x_{k})  (13.20)

and the Wiener–Itô integral

Z_{μ,k}(f) = Z_{μ,k}(f_{L}) = \frac{1}{k!} ∫ f_{L}(x_{1},...,x_{k})μ_{W}(dx_{1})...μ_{W}(dx_{k}).

Then EZ_{μ,k}^{2}(f) = 1, and the high moments of Z_{μ,k}(f) can be well estimated. For a large parameter L these moments are much smaller, than the bound given in Proposition 13.1. (The calculation leading to the estimation of the moments of Z_{μ,k}(f) will be omitted.) These moment estimates also imply that if the parameter L is large, then for not too large numbers u the probability P(|Z_{μ,k}(f)| > u) has a much better estimate than that given in Theorem 8.5. As a consequence, for a large number L and fixed number K relation (13.19) may hold only with a very big number A > 0.

We can expect that if we take a Gaussian random polynomial P(ξ_{1},...,ξ_{n}) whose arguments are Gaussian random variables ξ_{1},...,ξ_{n}, and which is the sum of many small almost independent terms with expectation zero, then a similar picture arises as in the case of a Wiener–Itô integral with kernel function (13.20) with a large parameter L. Such a random polynomial has an almost Gaussian distribution by the central limit theorem, and we can also expect that its not too high moments behave so as the corresponding moments of a Gaussian random variable with expectation zero and the same variance as the Gaussian random polynomial we consider. Such a bound on the moments has the consequence that the estimate on the probability of the event {ω: P(ξ_{1}(ω),...,ξ_{n}(ω)) > u} given in Theorem 8.5 can be improved if the number u is not too large. A similar picture arises if we consider Wiener– Itô integrals whose kernel function satisfies some ‘almost independence’ properties. The problem is to find the right properties under which we can get a good estimate that exploits the almost independence property of a Gaussian random polynomial or of a Wiener–Itô integral. The main result of R. Lata�a’s paper [29] can be considered as a response to this question. I describe this result below.

To formulate Lata�a’s result some new notions have to be introduced. Given a finite set A let P(A) denote the set of all its partitions. If a partition P = {B_{1},...,B_{s}} ∈ P(A) consists of s elements then we say that this partition has order s, and write |P| = s. In the special case A = {1,...,k} the notation P(A) = P_{k} will be used. Given a measurable space (X,X ) with a probability measure μ on it together with a finite set B = {b_{1},...,b_{j}} let us introduce the following notations. Take j different copies (X_{b_{r}},X_{b_{r}}) and μ_{b_{r}}, 1 ≤ r ≤ j, of this measurable space and probability measure indexed by the elements of the set B, and define their prod-

uct (X^{(B)},X^{(B)},μ^{(B)}) = ( ∏_{r=1}^{j} X_{b_{r}}, ∏_{r=1}^{j} X_{b_{r}}, ∏_{r=1}^{j} μ_{b_{r}}) . The points (x_{b_{1}},...,x_{b_{j}}) ∈ X^{(B)}

will be denoted by x^{(B)} ∈ X^{(B)} in the sequel. With the help of the above notations I introduce the quantities needed in the formulation of the following Theorem 13.7.

Let f = f(x_{1},...,x_{k}) be a function on the k-fold product (X^{k},X^{k},μ^{k}) of a measure space (X,X ,μ) with a probability measure μ. For all partitions P = {B_{1},...,B_{s}} ∈ P_{k}of the set {1,...,k} consider the functions g_{r}( x^{(B_{r})}) on the space

X^{(B_{r})}, 1 ≤ r ≤ s, and define with their help the quantities

α(P) = α(P,f,μ)

= sup_{g_{1},...,g_{s}}∫ f(x_{1},...,x_{k})g_{1}( x^{(B_{1})}) ··· g_{s}( x^{(B_{s})}) μ(dx_{1})...μ(dx_{k});

where supremum is taken for such functions

g_{1},...,g_{s}, g_{r}: X^{B_{r}} → R^{1} for which

∫ g_{r}^{2}( x^{(B_{r})}) μ^{(B_{r})}( dx^{(B_{r})}) ≤ 1 for all 1 ≤ r ≤ s,  (13.21)

and put

α_{s} = max_{P∈P_{k},|P|=s} α(P), 1 ≤ s ≤ k.  (13.22)

In Lata�a’s estimation of Wiener–Itô integrals of order k the quantities α_{s}, 1 ≤ s ≤ k, play a similar role as the number σ^{2} in Theorem 8.5. Observe that in the case |P| = 1, i.e. if P = {1,...,k} the identity α^{2}(P) = ∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) holds, which means that α_{1} = σ. The following estimate is valid for Wiener–Itô integrals of general order.

Theorem 13.7 (Lata�a’s estimate about the tail-distribution of Wiener–Itô integrals). Let a k-fold Wiener–Itô integral Z_{μ,k}(f), k ≥ 1, be defined with the help of a white noise μ_{W} with a non-atomic reference measure μ and a kernel function f of k variables such that

∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) < ∞.

There is some universal constant C(k) < ∞ depending only on the order k of the random integral such that the inequalities

E(Z_{μ,k}(f))^{2M} ≤ ( C(k) max_{1≤s≤k}(M^{s/2}α_{s}))^{2M},  (13.23)

and

P(|Z_{μ,k}(f)| > u) ≤ C(k)exp\{ −\frac{1}{C(k)} min_{1≤s≤k} ( \frac{u}{α_{s}} ) ^{2/s}\}  (13.24)

hold for all M = 1,2,... and u > 0 with the quantities α_{s}, defined in formulas (13.21) and (13.22).

Inequality (13.24) is a simple consequence of (13.23). In the special case when α_{s} ≤ M^{−(s−1)/2} for all 1 ≤ s ≤ k, t inequality (13.23) yields such an estimate on the moment EZ_{μ,k}(f)^{2M} which has the same magnitude as the 2M-th moment of a standard Gaussian random variable multiplied by a constant, and (13.24) yields a good estimate on the probability P(|Z_{μ,k}(f)| > u). Actually the result of Theorem 13.7 can be reduced to the special case when α_{s} ≤ M^{−(s−1)/2} for all 1 ≤ s ≤ k. Thus it can be interpreted so that if the quantities α_{s} of a k-fold Wiener–Itô integral are sufficiently small, then these ‘almost independence’ conditions imply that the 2M-th moment of this integral behaves similarly to a one-fold Wiener–Itô integral with the same variance.

Actually Lata�a formulated his result in a different form, and he proved a slightly weaker result. He considered Gaussian polynomials of the following form:

P(ξ_{j}^{(s)}, 1 ≤ j ≤ n, 1 ≤ s ≤ k)
= \frac{1}{k!} ∑_{(j_{1},...,j_{k}): 1≤j_{s}≤n,1≤s≤k} a(j_{1},...,j_{k})ξ_{j_{1}}^{(1)}··· ξ_{j_{k}}^{(k)},  (13.25)

where ξ_{j}^{(s)}, 1 ≤ j ≤ n and 1 ≤ s ≤ k, are independent standard normal random variables. Lata�a gave an estimate about the moments and tail-distribution of such random polynomials.

The problem about the behaviour of such random polynomials can be reformulated as a problem about the behaviour of Wiener–Itô integrals in the following way: Take a measurable space (X,X ) with a non-atomic measure μ on it. Let Z_{μ} be a white noise with reference measure μ, let us choose a set of orthogonal functions h_{j}^{(s)}(x), 1 ≤ j ≤ n, 1 ≤ s ≤ k, on the space (X,X ) with respect to the measure μ, and define the function

f(x_{1},...,x_{k}) = \frac{1}{k!} ∑_{(j_{1},...,j_{k}): 1≤j_{s}≤n,1≤s≤k} a(j_{1},...,j_{k})h_{j_{1}}^{(1)}(x_{1})··· h_{j_{k}}^{(k)}(x_{k}) (13.26)

together with the Wiener–Itô integral Z_{μ,k}(f). Since the random integrals ξ̄_{j}^{(s)} =

∫ h_{j}^{(s)}(x)Z_{μ}(dx), 1 ≤ j ≤ n, 1 ≤ s ≤ k, are independent, standard Gaussian random variables, it is not difficult to see with the help of Itô’s formula (Theorem 10.3 in this work) that the distributions of the random polynomial P(ξ_{j}^{(s)}, 1 ≤ j ≤ n, 1 ≤ s ≤ k) and Z_{μ,k}(f) agree. Here we reformulated Lata�a’s estimates about random polynomials of the form (13.25) to estimates about Wiener–Itô integrals with kernel function of the form (13.26).

These estimates are equivalent to Lata�a’s result if we restrict our attention to the special class of Wiener–Itô integrals with kernel functions of the form (13.26). But we have formulated our result for Wiener–Itô integrals with a general kernel function. Lata�a’s proof heavily exploits the special structure of the random polynomials given in (13.25), the independence of the random variables ξ_{j}^{(s)} for different parameters s in it. (It would be interesting to find a proof which does not exploit this property.) On the other hand, this result can be generalized to the case discussed in Theorem 13.7. This generalization can be proved by exploiting the theorem of de la Peña and Montgomery–Smith about the comparison of U-statistics and decoupled U-statistics (formulated in Theorem 14.3 of this work) and the properties of the Wiener–Itô integrals. I omit the details of the proof.

Lata�a also proved a converse estimate in [29] about random polynomials of Gaussian random polynomials which shows that the estimates of Theorem 13.7 are sharp. We formulate it in its original form, i.e. we restrict our attention to the case of Wiener–Itô integrals with kernel functions of the form (13.26).

Theorem 13.8 (A lower bound about the tail distribution of Wiener–Itô integrals). A random integral Z_{μ,k}(f) with a kernel function of the form (13.26) satisfies the inequalities

E(Z_{μ,k}(f))^{2M} ≥ ( C(k) max_{1≤s≤k}(M^{s/2}α_{s}))^{2M},

and

P(|Z_{μ,k}(f)| > u) ≥ \frac{1}{C(k)}exp\{ −C(k) min_{1≤s≤k} ( \frac{u}{α_{s}} ) ^{2/s}\}

for all M = 1,2,... and u > 0 with some universal constant C(k) > 0 depending only on the order k of the integral and the quantities α_{s}, defined in formula (13.21) and (13.22).

Let me finally remark that there is a counterpart of Theorem 13.7 about degenerate U-statistics. Adamczak’s paper [1] contains such a result. Here we do not discuss it, because this result is far from the main topic of this work. We only remark that some new quantities have to be introduced to formulate it. The appearance of these conditions is related to the fact that in an estimate about the tail-behaviour of a degenerate U-statistic we need a bound not only on the L_{2}-norm but also on the supremum norm of the kernel function. In a sharp estimate the bound about the supremum of the kernel function has to be replaced by a more complex system of conditions, just as the condition about the L_{2}-norm of the kernel function was replaced by a condition about the quantities α_{s}, 1 ≤ s ≤ k, defined in formulas (13.21) and (13.22) in Theorem 13.7.

Reduction of the main result in this work

The main result of this work is Theorem 8.4 or its multiple integral version Theorem 8.2. It was shown in Chapter 9 that Theorem 8.2 follows from Theorems 8.4. Hence it is enough to prove Theorem 8.4. It may be useful to study this problem together with its multiple Wiener–Itô integral version, Theorem 8.6.

Theorems 8.6 and 8.4 will be proved similarly to their one-variate versions, Theorems 4.2 and 4.1. Theorem 8.6 will be proved with the help of Theorem 8.5 about the estimation of the tail distribution of multiple Wiener–Itô integrals. A natural modification of the chaining argument applied in the proof of Theorem 4.2 works also in this case. No new difficulties arise. On the other hand, in the proof of Theorem 8.4 several new difficulties have to be overcome. I start with the proof of Theorem 8.6.

Proof of Theorem 8.6. Fix a number 0 < ε < 1, and let us list the elements of the countable set F as f_{1},f_{2},.... For all p = 0,1,2,... let us choose by exploiting the conditions of Theorem 8.6 a set of functions F_{p} = {f_{a(1,p)},...,f_{a(m_{p},p)}} ⊂ F with

m_{p} ≤ 2D2^{(2p+4)L} ε^{−L} σ^{−L} elements in such a way that inf_{1≤j≤m_{p}} ∫(f − f_{a(j,p)})^{2}dμ ≤

2^{−4p−8} ε^{2}σ^{2} for all f ∈ F, and beside this let f_{p} ∈ F_{p}. For all indices a(j,p), p = 1,2,..., 1 ≤ j ≤ m_{p}, choose a predecessor a(j′,p−1), j′= j′(j,p), 1 ≤ j′≤ m_{p−1} , in such a way that the functions f_{a(j,p)} and f_{a(j′,p−1)} satisfy the relation ∫ |f_{a(j,p)} − f_{a(j′,p−1)} |^{2}dμ ≤ ε^{2}σ^{2}2^{−4(p+1)} . Theorem 8.5 with the choice ū= ū(p) = 2^{−(p+1)} εu and σ̄ = σ̄(p) = 2^{−2p−2} εσ yields the estimates

P(A(j,p))= P ( |k!Z_{μ,k}(f_{a(j,p)} − f_{a(j′,p−1)} )| ≥ 2^{−(1+p)} εu)
≤ C exp\{ −\frac{1}{2} ( \frac{2^{p+1} u}{σ}) ^{2/k}\} ,  1 ≤ j ≤ m_{p},  (14.1)

for all p = 1,2,..., and

P(B(s))= P ( |k!Z_{μ,k}(f_{a(0,s)})| ≥ ( 1 − \frac{ε}{2}) u)

≤ C exp\{ −\frac{1}{2} ( \frac{(1 − \frac{ε}{2}) u}{σ}) ^{2/k}\} , 1 ≤ s ≤ m_{0}.  (14.2)

Since each f ∈ F is the element of at least one set F_{p}, p = 0,1,2,..., (We made a construction, where f_{p} ∈ F_{p}), the definition of the predecessor of an index a(j,p) and of the events A(j,p) and B(s) in formulas (14.1) and (14.2) together with the previous estimates imply that

P ( sup_{f∈F}|k!Z_{μ,k}(f)| ≥ u) ≤ P ( ⋃_{p=1}^{∞} ⋃_{j=1}^{m_{p}} A(j,p) ∪ ⋃_{s=1}^{m_{0}} B(s))

≤ ∑_{p=1}^{∞} ∑_{j=1}^{m_{p}} P(A(j,p)) + ∑_{s=1}^{m_{0}} P(B(s))

≤ ∑_{p=1}^{∞} 2CD2^{(2p+4)L} ε^{−L} σ^{−L} exp\{ −\frac{1}{2} ( \frac{2^{p+1} u}{σ}) ^{2/k}\}

+2^{1+4L} CDε^{−L} σ^{−L} exp\{ −\frac{1}{2} ( \frac{(1 − \frac{ε}{2}) u}{σ}) ^{2/k}\} .  (14.3)

Some calculations show that if u ≥ ML^{k/2}σ \frac{1}{ε}(log^{k/2} \frac{2}{ε} +log^{k/2} \frac{2}{σ}) with a sufficiently large constant M = M(k), then the inequalities

2^{(2p+4)L} ε^{−L} σ^{−L} exp\{ −\frac{1}{2} ( \frac{2^{p+1} u}{σ}) ^{2/k}\} ≤ 2^{−p} \{ −\frac{1}{2} ( \frac{(1 − ε)u}{σ} ) ^{2/k}\}

hold for all p = 1,2..., and

2^{4L}ε^{−L} σ^{−L} exp\{ −\frac{1}{2} ( \frac{(1 − \frac{ε}{2}) u}{σ}) ^{2/k}\} ≤ exp\{ −\frac{1}{2} ( \frac{(1 − ε) u}{σ}) ^{2/k}\} .

These inequalities together with relation (14.3) imply relation (8.18). Theorem 8.6 is proved.

The proof of Theorem 8.4 is harder. In this case the chaining argument in itself does not supply the proof, since Theorem 8.3 gives a good estimate about the distribution of a degenerate U-statistic only if it has a not too small variance. The same difficulty appeared in the proof of Theorem 4.1, and the method applied in that case will be adapted to the present situation.

A multivariate version of Proposition 6.1 will be proved in Proposition 14.1, and another result which can be considered as a multidimensional version of Proposition 6.2 will be formulated in Proposition 14.2. It will be shown that Theorem 8.4 follows from Propositions 14.1 and 14.2. Most steps of these proofs can be considered as a simple repetition of the corresponding arguments in the proof of the results in Chapter 6. Nevertheless, I wrote down them for the sake of completeness.

The result formulated in Proposition 14.1 can be proved in almost the same way as its one-variate version, Proposition 6.1. The only essential difference is that now we apply a multivariate version of Bernstein’s inequality given in the Corollary of Theorem 8.3. In the calculations of the proof of Proposition 14.1 the term (\frac{u}{σ})^{2/k} shows a behaviour similar to the term (\frac{u}{σ})^{2} in Proposition 6.1. Theorem 14.1 contains the information we can get by applying Theorem 8.3 together with the chaining argument. Its main content, inequality (14.4), yields a good estimate on the supremum of degenerate U-statistics if it is taken for an appropriate finite subclass Fσ̄ of the original class of kernel functions F. The class of kernel functions Fσ̄ is a relatively dense subclass of F in the L_{2} norm. Proposition 14.1 also provides some useful estimates on the value of the parameter σ̄ which describes how dense the class of functions Fσ̄ is in F.

Proposition 14.1. Let the k-fold power (X^{k},X^{k}) of a measurable space (X,X ) be given together with some probability measure μ on (X,X ) and a countable, L_{2}- dense class F of functions f(x_{1},...,x_{k}) of k variables with some exponent L ≥ 1 and parameter D ≥ 1 with respect to the measure μ on the product space (X^{k},X^{k}) which also has the following properties. All functions f ∈ F are canonical with respect to the measure μ, and they satisfy conditions (8.5) and (8.6) with some real number 0 < σ ≤ 1. Take a sequence of independent, μ-distributed random variables ξ_{1},...,ξ_{n}, n ≥ max(k,2), and consider the (degenerate) U-statistics I_{n,k}(f), f ∈ F, defined in formula (8.8), and fix some number Ā= Ā_{k}≥ 2^{k}.

There is a number M = M(Ā,k) such that for all numbers u > 0 for which the inequality nσ^{2} ≥ ( \frac{u}{σ})^{2/k} ≥ M(Llog \frac{2}{σ} +logD) holds, a number σ̄ = σ̄(u), 0 ≤ σ̄ ≤ σ ≤ 1, and a collection of functions Fσ̄ = F_{σ̄(u)} = {f_{1},...,f_{m}} ⊂ F with m ≤ Dσ̄^{−L} elements can be chosen in such a way that the union of the sets D_{j} = {f: f ∈

F , ∫ |f − f_{j}|^{2}dμ ≤ σ̄^{2}}, 1 ≤ j ≤ m cover the set F. i.e. F = ⋃_{j=1}^{m} D_{j}, and the

(degenerate) U-statistics I_{n,k}(f), f ∈ F_{σ̄(u)}, satisfy the inequality

P ( sup_{f∈F_{σ̄(u)}} n^{−k/2}|k!I_{n,k}(f)| ≥ \frac{u}{Ā}) ≤ 2C exp\{ −α ( \frac{u}{10Āσ} )^{2/k}\}

if nσ^{2} ≥ ( \frac{u}{σ})^{2/k} ≥ M ( Llog \frac{2}{σ} + logD)  (14.4)

with the constants α = α(k), C = C(k) appearing in formula (8.13) of the Corollary of Theorem 8.3 and the exponent L and parameter D of the L_{2}-dense class F. Beside

this, also the inequality 4( \frac{u}{Ā ; σ̄})^{2/k} ≥ nσ̄^{2} ≥ \frac{1}{64} ( \frac{u}{Āσ} )^{2/k} holds for this number σ̄ = σ̄(u). If the number u satisfies also the inequality

nσ^{2} ≥ ( \frac{u}{σ})^{2/k} ≥ M(L^{3/2}log \frac{2}{σ} + (logD)^{3/2})  (14.5)

with a sufficiently large number M = M(Ā,k), then the relation nσ̄^{2} ≥ Llogn+logD holds, too.

Proof of Proposition 14.1. Let us list the elements of the countable set F as f_{1},f_{2},.... For all p = 0,1,2,... let us choose, by exploiting the L_{2}-density property of the class F, a set F_{p} = {f_{a(1,p)},...,f_{a(m_{p},p)}} ⊂ F with m_{p} ≤ D2^{2pL}σ^{−L} elements in such a way that inf_{1≤j≤m_{p}} ∫(f − f_{a(j,p)})^{2}dμ ≤ 2^{−4p} σ^{2} for all f ∈ F.

For all indices a(j,p), p = 1,2,..., 1 ≤ j ≤ m_{p}, choose a predecessor a(j′,p − 1), j′= j′(j,p), 1 ≤ j′≤ m_{p−1} , in such a way that the functions f_{a(j,p)} and f_{a(j′,p−1)} satisfy the relation ∫ |f_{a(j,p)} − f_{a(j′,p−1)} |^{2}dμ ≤ σ^{2}2^{−4(p−1)} . Then the inequalities

∫ ( \frac{f_{a(j,p)}−f_{a(j′,p−1)}}{2} )^{2} dμ ≤ 4σ^{2}2^{−4p} and sup_{x_{j}∈X,1≤j≤k} | \frac{f_{a(j,p)}(x_{1},...,x_{k})−f_{a(j′,p−1)} (x_{1},...,x_{k})}{2} | ≤

1 hold. The Corollary of Theorem 8.3 yields that

P(A(j,p)) = P ( n^{−k/2}|k!I_{n,k}(f_{a(j,p)} − f_{a(j′,p−1)} )| ≥ \frac{2^{−(1+p)} u}{Ā})

≤ C exp\{ −α ( \frac{2^{p}u}{8Āσ} ) ^{2/k}\} if 4nσ^{2}2^{−4p} ≥ ( \frac{2^{p}u}{8Āσ} ) ^{2/k},

1 ≤ j ≤ m_{p}, p = 1,2,...,  (14.6)

and

P(B(s)) = P ( n^{−k/2}|k!I_{n,k}(f_{0,s})| ≥ \frac{u}{2Ā}) ≤ C exp\{ −α ( \frac{u}{2Āσ} )^{2/k}\} ,
1 ≤ s ≤ m_{0}, if nσ^{2} ≥ ( \frac{u}{2Āσ} )^{2/k}.  (14.7)

Introduce an integer R = R(u), R > 0, which satisfies the relations

2^{(4+2/k)(R+1)} ( \frac{u}{Āσ} )^{2/k} ≥ 2^{2+6/k} nσ^{2} ≥ 2^{(4+2/k)R} ( \frac{u}{Āσ} )^{2/k},

and define σ̄^{2} = 2^{−4R}σ^{2} and Fσ̄ = F_{R}(this is the class of functions F_{p} introduced at the start of the proof with p = R). We defined the number R, analogously to the proof of Proposition 6.1, as the largest number p for which the condition formulated in (14.6) holds. As nσ^{2} ≥ ( \frac{u}{σ})^{2/k}, and Ā≥ 2^{k} by our conditions, there exists such a positive integer R.) The cardinality m of the set Fσ̄ is clearly not greater than

Dσ̄^{−L} , and ⋃_{j=1}^{m} D_{j} = F. Beside this, the number R was chosen in such a way that

the inequalities (14.6) and (14.7) hold for 1 ≤ p ≤ R. Hence the definition of the predecessor of an index a(j,p) implies that

P ( sup_{f∈Fσ̄} n^{−k/2}k|k!I_{n,k}(f)| ≥ \frac{u}{Ā}) ≤ P ( ⋃_{p=1}^{R} ⋃_{j=1}^{m_{p}} A(j,p) ∪ ⋃_{s=1}^{m_{0}} B(s))

≤ ∑_{p=1}^{R} ∑_{j=1}^{m_{p}} P(A(j,p)) + ∑_{s=1}^{m_{0}} P(B(s))

≤ ∑_{p=1}^{∞} CD2^{2pL}σ^{−L} exp\{ −α ( \frac{2^{p}u}{8Āσ} ) ^{2/k}\} + CDσ^{−L} exp\{ −α ( \frac{u}{2Āσ} )^{2/k}\} .

If the condition ( \frac{u}{σ})^{2/k}≥ M(Llog \frac{2}{σ} +logD) holds with a sufficiently large constant M (depending on Ā), then the inequalities

D2^{2pL}σ^{−L} exp\{ −α ( \frac{2^{p}u}{8Āσ} ) ^{2/k}\} ≤ 2^{−p} exp\{ −α ( \frac{2^{p}u}{10Āσ} ) ^{2/k}\}

hold for all p = 1,2,..., and

Dσ^{−L} exp\{ −α ( \frac{u}{2Āσ} )^{2/k}\} ≤ exp\{ −α ( \frac{u}{10Āσ} )^{2/k}\} .

Hence the previous estimate implies that

P ( sup_{f∈Fσ̄} n^{−k/2}|k!I_{n,k}(f)| ≥ \frac{u}{Ā}) ≤ ∑_{p=1}^{∞} C2^{−p} exp\{ −α ( \frac{2^{p}u}{10Āσ} ) ^{2/k}\}

+C exp\{ −α ( \frac{u}{10Āσ} )^{2/k}\} ≤ 2C exp\{ −α ( \frac{u}{10Āσ} )^{2/k}\} ,

and relation (14.4) holds.

The estimates

\frac{1}{64} ( \frac{u}{Āσ} )^{2/k} ≤ 2^{−2−6/k} 2^{2R/k}( \frac{u}{Āσ} )^{2/k} = 2^{−4R} · 2^{(4+2/k)R−2−6/k} ( \frac{u}{Āσ} )^{2/k}
≤ nσ̄^{2} = 2^{−4R}nσ^{2} ≤ 2^{−4R} · 2^{(4+2/k)(R+1)−2−6/k} ( \frac{u}{Āσ} )^{2/k}
= 2^{2−4/k} · 2^{2R/k}( \frac{u}{Āσ} )^{2/k} = 2^{2−4/k} · 2^{−2R/k} ( \frac{u}{Ā ; σ̄})^{2/k} ≤ 4( \frac{u}{Ā ; σ̄})^{2/k}

hold because of the relation R ≥ 1. This means that nσ̄^{2} has the upper and lower bound formulated in Proposition 14.1. It remained to show that nσ̄^{2} ≥ Llogn+D if relation (14.5) holds.

This inequality clearly holds under the conditions of Proposition 14.1 if σ ≤ n^{−1/3} , since in this case log \frac{2}{σ} ≥ \frac{logn}{3}, and

nσ̄^{2} ≥ \frac{1}{64} ( \frac{u}{Āσ} )^{2/k} ≥ \frac{1}{64}Ā^{−2/k}M ( Llog \frac{2}{σ} + logD))
≥ \frac{1}{192}Ā^{−2/k}M(Llogn + logD) ≥ Llogn + logD

if M = M(Ā,k) is sufficiently large.

If σ ≥ n^{−1/3} , then the inequality 2^{(4+2/k)R} ( \frac{u}{Āσ} )^{2/k} ≤ 2^{2+6/k} nσ^{2} can be applied.

This implies that 2^{−4R} ≥ 2^{−4(2+6/k))/(4+2/k)} [ \frac{( \frac{u}{Āσ} ) 2/k}{nσ^{2}} ]^{4/(4+2/k)}, and

nσ̄^{2} = 2^{−4R}nσ^{2} ≥ \frac{2^{−16/3}}{Ā^{4/3}} (nσ^{2})^{1−γ} [( \frac{u}{σ})^{2/k}]^{γ} with γ = \frac{4}{4 + \frac{2}{k}} ≥ \frac{2}{3}.

The inequalities nσ^{2} ≥ n^{1/3} and nσ^{2} ≥ (\frac{u}{σ})^{2/k} ≥ M(L^{3/2}log \frac{2}{σ} + (logD)^{3/2}) ≥ \frac{M}{2}(L^{3/2} + (logD)^{3/2}) hold, (since log \frac{2}{σ} ≥ \frac{1}{2}). They yield that for sufficiently large M = M(Ā,k)

(nσ^{2})^{1−γ} [( \frac{u}{σ})^{2/k}]^{γ} ≥ (nσ^{2})^{1−γ} [( \frac{u}{σ})^{2/k}]^{2/3} = (nσ^{2})^{1/(2k+1)} [( \frac{u}{σ})^{2/k}]^{2/3},

and

nσ̄^{2} ≥ \frac{Ā^{−4/3}}{50} (nσ^{2})^{1/(2k+1)} [( \frac{u}{σ})^{2/k}]^{2/3}

≥ \frac{Ā^{−4/3}}{50} n^{1/3(2k+1)} ( \frac{M}{2}) ^{2/3}(L^{3/2}+ (logD)^{3/2})^{2/3} ≥ Llogn + logD.

A multivariate analogue of Proposition 6.2 is formulated in Proposition 14.2, and it will be shown that Propositions 14.1 and 14.2 imply Theorem 8.4.

Proposition 14.2. Let a probability measure μ be given on a measurable space (X,X ) together with a sequence of independent and μ distributed random variables ξ_{1},...,ξ_{n} and a countable L_{2}-dense class F of canonical (with respect to the measure μ) kernel functions f = f(x_{1},...,x_{k}) with some parameter D ≥ 1 and exponent L ≥ 1 on the product space (X^{k},X^{k}). Let all functions f ∈ F satisfy conditions (8.1) and (8.2) with some 0 < σ ≤ 1 such that nσ^{2} > Llogn + D. Let us consider the (degenerate) U-statistics I_{n,k}(f) with the random sequence ξ_{1},...,ξ_{n}, n ≥ max(2,k), and kernel functions f ∈ F. There exists a threshold index A_{0} = A_{0}(k) > 0 and two numbers C̄= C̄(k) > 0 and γ = γ(k) > 0 depending only on the order k of the U-statistics such that the degenerate U-statistics I_{n,k}(f), f ∈ F, satisfy the inequality

P ( sup_{f∈F}n^{−k/2}|k!I_{n,k}(f)| ≥ An^{k/2}σ^{k+1}) ≤ C̄e^{−γA^{1/2k}nσ^{2}} ifA ≥ A_{0}.  (14.8)

Proposition 14.2 yields an estimate for the tail distribution of the supremum of degenerate U-statistics at level u ≥ A_{0}n^{k/2}σ^{k+1}, i.e. in the case when Theorem 8.3 does not give a good estimate on the tail-distribution of the single degenerate Ustatistics taking part in the supremum at the left-hand side of (14.8).

Formula (8.14) will be proved by means of Proposition 14.1 with an appropriate choice of the parameter Āin it and Proposition 14.2 with the choice σ = σ̄ = σ̄(u) and the classes of functions F_{j} = \{ \frac{g−f_{j}}{2} : g ∈ D_{j}\} with the number σ̄, functions f_{j} and sets of functions D_{j}, 1 ≤ j ≤ m, introduced in Proposition 14.1. Clearly,

P ( sup_{f∈F}n^{−k/2}|k!I_{n,k}(f)| ≥ u) ≤ P ( sup_{f∈Fσ̄} n^{−k/2}|k!I_{n,k}(f)| ≥ \frac{u}{Ā})

+ ∑_{j=1}^{m} P ( sup_{g∈D_{j}}n^{−k/2} |k!I_{n,k}( \frac{f_{j} − g}{2} ) |≥ ( \frac{1}{2} − \frac{1}{2Ā}) u) ,  (14.9)

where m is the cardinality of the set of functions Fσ̄ appearing in Proposition 14.1. We shall estimate the two terms of the sum at the right-hand side of (14.9) by means of Propositions 14.1 and 14.2 with a good choice of the parameters Āand the corresponding M = M(Ā) in Proposition 14.1 together with a parameter A ≥ A_{0} in Proposition 14.2.

We shall choose the parameter A ≥ A_{0} in the application of Proposition 14.2 so that it satisfies also the relation γ A^{1/2k} ≥ 2 with the number γ appearing in relation (14.8), hence we put A = max(A_{0},(\frac{2}{γ})^{2k}). After this choice we want to define the parameter Āin Proposition 14.1 in such a way that the numbers u satisfying the conditions of Proposition 14.1 also satisfy the relation (\frac{1}{2} − \frac{1}{2Ā})u ≥ An^{k/2}σ̄^{k+1} with the already fixed number A and the number σ̄ = σ̄(u) defined in the proof of Proposition 14.1. This inequality can be rewritten in the form A^{−2/k} (\frac{1}{2} − \frac{1}{2Ā})^{2/k}(\frac{u}{σ̄})^{2/k} ≥ nσ̄^{2}. On the other hand, under the conditions of Proposition 14.1 the inequality 4(\frac{u}{Ā ; σ̄})^{2/k} ≥ nσ̄^{2} holds. Hence the desired inequality holds if A^{−2/k} (\frac{1}{2} − \frac{1}{2Ā})^{2/k} ≥ 4Ā^{−2/k}. Thus the number Ā= 2^{k+1}A + 1 is an appropriate choice.

With such a choice of Ā(together with the corresponding M = M(Ā,k)) and A we can write

P ( sup_{g∈D_{j}}n^{−k/2} |k!I_{n,k}( \frac{f_{j} − g}{2} ) |≥ ( \frac{1}{2} − \frac{1}{2Ā}) u)

≤ P ( sup_{g∈D_{j}}n^{−k/2} |k!I_{n,k}( \frac{f_{j} − g}{2} ) |≥ An^{k/2}σ̄^{k+1} ) ≤ C̄e^{−γA^{1/2k}nσ̄^{2}}

for all 1 ≤ j ≤ m. (Observe that the set of functions \frac{f_{j}−g}{2} , g ∈ D_{j}, is an L_{2}-dense class with parameter D and exponent L.) Hence Proposition 14.1 (relation (14.4) together with the inequality m ≤ Dσ̄^{−L} ) and formula (14.8) with our A ≥ A_{0} and relation (14.9) imply that

P ( sup_{f∈F}n^{−k/2}|k!I_{n,k}(f)| ≥ u)

≤ 2C exp\{ −α ( \frac{u}{10Āσ} )^{2/k}\} + C̄Dσ̄^{−L} e^{−γA^{1/2k}nσ̄^{2}}.  (14.10)

We show by repeating an argument given in Chapter 6 that Dσ̄^{−L} ≤ e^{nσ̄^{2}}. Indeed, we have to show that logD + Llog \frac{1}{σ̄} ≤ nσ̄^{2}. But, as we have seen, the relation nσ̄^{2} ≥ Llogn + logD with L ≥ 1 and D ≥ 1 implies that nσ̄^{2} ≥ logn, hence log \frac{1}{σ} ≤ logn, and logD + Llog \frac{1}{σ̄} ≤ logD + Llogn ≤ nσ̄^{2}. On the other hand, γA^{1/2k} ≥ 2 by the definition of the number A, and by the estimates of Proposi-

tion 14.1 nσ̄^{2} ≥ \frac{1}{64} ( \frac{u}{Āσ} )^{2/k}. The above relations imply that Dσ̄^{−L} e^{−γA^{1/2k}nσ̄^{2}} ≤

e^{−γA^{1/2k}nσ̄^{2}/2} ≤ exp\{−\frac{γ}{128}A^{1/2k}Ā^{−2/k}( \frac{u}{σ})^{2/k}\} . Hence relation (14.10) yields that

P ( sup_{f∈F}n^{−k/2}|k!I_{n,k}(f)| ≥ u)

≤ 2C exp\{ −\frac{α}{(10Ā)^{2}} ( \frac{u}{σ})^{2/k}\} + C̄exp\{ −\frac{γ}{128}A^{1/2k}Ā^{−2/k}( \frac{u}{σ})^{2/k}\} ,

and this estimate implies Theorem 8.4.

To complete the proof of Theorem 8.4 we have to prove Proposition 14.2. It will be proved, similarly to its one-variate version Proposition 6.2, by means of a symmetrization argument. We want to find its right formulation. It would be natural to formulate it as a result about the supremum of degenerate U-statistics. However, we shall choose a slightly different approach. There is a notion, called decoupled U-statistic. Decoupled U-statistics behave similarly to U-statistics, but it is simpler to work with them, because they have more independence properties. It turned out to be useful to introduce them and to apply a result of de la Peña and Montgomery– Smith which enables us to reduce the estimation of U-statistics to the estimation of decoupled U-statistics, and to work out the symmetrization argument for decoupled U-statistics.

Next we introduce the notion of decoupled U-statistics together with their randomized version. We also formulate a result of de la Peña and Montgomery–Smith in Theorem 14.3 which enables us to reduce Proposition 14.2 to a version of it, presented in Proposition 14.2′. It states a result similar to Proposition 14.2 about decoupled U-statistics. The proof of Proposition 14.2′is the hardest part of the problem. In Chapter 15, 16 and 17 we deal essentially with this problem. The result of de la Peña and Montgomery–Smith will be proved in Appendix D.

Definition of decoupled and randomized decoupled U-statistics. Let us have k independent copies ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence ξ_{1},...,ξ_{n} of independent and identically distributed random variables taking their values in a measurable space (X,X ) together with a measurable function f(x_{1},...,x_{k}) on the product space (X^{k},X^{k}) with values in a separable Banach space. The decoupled U-statistic Ī_{n,k}(f) determined by the random sequences ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k, and kernel function f is defined by the formula

Ī_{n,k}(f) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k, \\ l_{j} ≠l_{j′} ifj≠j′ \end{matrix}} f ( ξ_{l_{1}}^{(1)},...,ξ_{l_{k}}^{(k)}) .  (14.11)

Let us have beside the sequences of random variables ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k, and function f(x_{1},...,x_{k}) a sequence of independent random variables ε = (ε_{1},...,ε_{n}), P(ε_{l} = 1) = P(ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is independent also of the sequences of random variables ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k. The randomized decoupled U-statistic Ī_{n,k}(f,ε) (depending on the random sequences ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k, the kernel function f and the randomizing sequence ε_{1},...,ε_{n}) is defined by the formula

Ī_{n,k}^{ε}(f) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k, \\ l_{j} ≠l_{j′} ifj≠j′ \end{matrix}} ε_{l_{1}} ··· ε_{l_{k}}f ( ξ_{l_{1}}^{(1)},...,ξ_{l_{k}}^{(k)}) .  (14.12)

A decoupled or randomized decoupled U-statistics (with a real valued kernel function) will be called degenerate if its kernel function is canonical. This terminology is in full accordance with the definition of (usual) degenerate U-statistics.

A result of de la Peña and Montgomery–Smith will be formulated below. It gives an upper bound for the tail distribution of a U-statistic by means of the tail distribution of an appropriate decoupled U-statistic. It also has a generalization, where the supremum of U-statistics is bounded by the supremum of decoupled U-statistics. It enables us to reduce Proposition 14.2 to a version of it formulated Proposition 14.2′, which gives a bound on the tail distribution of the supremum of decoupled Ustatistics. It is simpler to prove this result than the original one.

Before the formulation of the theorem of de la Peña and Montgomery–Smith I make some remark about it. In this result we consider more general U-statistics with kernel functions taking values in a separable Banach space, and we compare the norm of Banach space valued U-statistics and decoupled U-statistics. (Decoupled U-statistics were defined with general Banach space valued kernel functions, and the definition of U-statistics can also be generalized to separable Banach space valued kernel functions in a natural way.) This result was formulated in such a general form for a special reason. This helped us to derive formula (14.14) of the subsequent theorem from formula (14.13). It can be exploited in the proof of formula (14.14) that the constants in the estimate (14.13) do not depend on the Banach space where the kernel function f takes its values.

Theorem 14.3 (Theorem of de la Peña and Montgomery–Smith about the comparison of U-statistics and decoupled U-statistics). Let us consider a sequence of independent and identically distributed random variables ξ_{1},...,ξ_{n} with values in a measurable space (X,X ) together with k independent copies ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k, of this sequence. Let us also have a function f(x_{1},...,x_{k}) on the k-fold product space (X^{k},X^{k}) which takes its values in a separable Banach space B. Let us take the U-statistic and decoupled U-statistic I_{n,k}(f) and Ī_{n,k}(f) with the help of the above random sequences ξ_{1},...,ξ_{n}, ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k, and kernel function f. There exist some constants C̄= C̄(k) > 0 and γ = γ(k) > 0 depending only on the order k of the U-statistic such that

P (∥k!I_{n,k}(f)∥ > u) ≤ C̄P (∥k!Ī_{n,k}(f)∥ > γu)  (14.13)

for all u > 0. Here ∥·∥ denotes the norm in the Banach space B where the function f takes its values.

More generally, if we have a countable sequence of functions f_{s}, s = 1,2,..., taking their values in the same separable Banach-space, then

P ( sup_{1≤s<∞} ∥k!I_{n,k}(f_{s})∥ > u) ≤ C̄P ( sup_{1≤s<∞} ∥k!Ī_{n,k}(f_{s})∥ > γu) .  (14.14)

Now I formulate the following version of Proposition 4.2.

Proposition 14.2′. Let a probability measure μ be given on a measurable space (X,X ) together with a sequence of independent and μ distributed random variables ξ_{1},...,ξ_{n}, n ≥ max(k,2), and a countable L_{2}-dense class F of canonical (with respect to the measure μ) kernel functions f = f(x_{1},...,x_{k}) with some parameter D ≥ 1 and exponent L ≥ 1 on the product space (X^{k},X^{k}). Let all functions f ∈ F satisfy conditions (8.1) and (8.2) with some 0 < σ ≤ 1 such that nσ^{2} > Llogn + logD. Let us take k independent copies ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k, of the random sequence ξ_{1},...,ξ_{n}, and consider the decoupled U-statistics Ī_{n,k}(f), f ∈ F, defined with their help in formula (14.11).

There exists a threshold index A_{0} = A_{0}(k) > 0 depending only on the order k of the decoupled U-statistics I_{n,k}(f), f ∈ F, such that the (degenerate) decoupled U-statistics Ī_{n,k}(f), f ∈ F, satisfy the following version of inequality (14.8):

P ( sup_{f∈F}n^{−k/2}|k!Ī_{n,k}(f)| ≥ An^{k/2}σ^{k+1}) ≤ e^{−2^{−(1/2+1/2k)} A^{1/2k}nσ^{2}} ifA ≥ A_{0}.

(14.15)

It is clear that Proposition 14.2′ and Theorem 14.3, more explicitly formula (14.14) in it, imply Proposition 14.2. Hence the proof of Theorem 8.4 was reduced to Proposition 14.2′ in this chapter. The proof of Proposition 14.2′ is based on a symmetrization argument. Its main ideas will be explained in the next chapter.

The strategy of the proof for the main result of this work

In the previous chapter the proof of Theorem 8.4 was reduced to that of Proposition 14.2′. Proposition 14.2′ is a multivariate version of Proposition 6.2, and its proof is based on similar ideas. An important step in the proof of Proposition 6.2 was a symmetrization argument in which we reduced the estimation of the proba-

bility P ( sup_{f∈F} ∑_{j=1}^{n} f(ξ_{j}) > u) to that of the probability P ( sup_{f∈F} ∑_{j=1}^{n} ε_{j}f(ξ_{j}) > \frac{u}{3}) ,

where ξ_{1},...,ξ_{n} is a sequence of independent and identically distributed random variables, and ε_{j}, 1 ≤ j ≤ n, is a sequence of independent random variables with distribution P(ε_{j} = 1) = P(ε_{j} = −1) = \frac{1}{2}, independent of the sequence ξ_{j}. We want to prove a similar symmetrization argument which helps to prove Proposition 14.2′.

The symmetrization argument applied in the proof of Proposition 6.2 was carried out in two steps. We took a copy ξ′_{1},...,ξ′_{n}of the sequence ξ_{1},...,ξ_{n}, i.e. a sequence of independent random variables which is independent also of the original sequence ξ_{1},...,ξ_{n}, and has the same distribution. In the first step we compared the tail dis-

tribution of the expression sup_{f∈F} ∑_{j=1}^{n} [f(ξ_{j}) − f(ξ′_{j})] with that of sup_{f∈F} ∑_{j=1}^{n} f(ξ_{j}) with

the help of Lemma 7.1. In the second step, in the proof of Lemma 7.2, we applied a ‘randomization argument’ which stated that the distribution of the random fields

∑_{j=1}^{n} [f(ξ_{j}) − f(ξ′_{j})] and ∑_{j=1}^{n} ε_{j}[f(ξ_{j}) − f(ξ′_{j})], f ∈ F, agree. The symmetrization ar-

gument was proved with the help of these two observations.

In the proof of Proposition 14.2′we would like to reduce the estimation of the tail distribution of the supremum of decoupled U-statistics sup_{f∈F} Ī_{n,k}(f) defined in

formula (14.11) to the estimation of the tail distribution of the supremum of the randomized decoupled U-statistics sup_{f∈F} Ī_{n,k}^{ε}(f) defined in formula (14.12) in a similar

way. To do this we have to find the multivariate version of the ‘randomization argument’ in the proof of Lemma 7.2. This will be done in the subsequent Lemma 15.1. In Lemma 7.2 this randomization argument was formulated with the help of some random variables introduced in formulas (7.4) and (7.5). We shall define their multivariate versions in formulas (15.1) and (15.2), and they will play a similar role in the formulation of Lemma 15.1.

The adaptation of the first step of the symmetrization argument of the proof of Proposition 6.2 is much harder. The proof of Proposition 6.2 was based on a symmetrization lemma formulated in Lemma 7.1. This result does not work in the present case. Hence we shall generalize it in Lemma 15.2. The proof of the symmetrization argument needed in the proof of Proposition 14.2′is difficult even with the help of this result. The hardest part of our problem appears at this point. I return to it after the formulation of Lemma 15.2.

To formulate Lemma 15.1 we introduce the following notations.

Let V_{k} = {(v(1),...,v(k)): v(j) = ±1, for all 1 ≤ j ≤ k} denote the set of all ±1 sequences of length k. Let m(v) denote the number of −1 digits in a sequence v = (v(1),...,v(k)) ∈ V_{k}. Let a (real valued) function f(x_{1},...,x_{k}) of k variables be given on a measurable space (X,X ) together with a sequence of independent and identically distributed random variables ξ_{1},...,ξ_{n} with values in the space (X,X ). Take 2k independent copies ξ_{1}^{(j,1)},...,ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)} ,...,ξ_{n}^{(j,−1)} , 1 ≤ j ≤ k, of the sequence ξ_{1},...,ξ_{n}. Let us have beside them another sequence ε = (ε_{1},...,ε_{n}), P(ε_{j} = 1) = P(ε_{j} = −1) = \frac{1}{2}, of independent random variables, also independent of all previously introduced random variables. With the help of the above quantities we introduce the random variables

Ĩ_{n,k}(f) = \frac{1}{k!} ∑_{v∈V_{k}}(−1)^{m(v)}  ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{r}≤n, r=1,...,k, \\ l_{r}≠l_{r′} if r≠r′ \end{matrix}} f ( ξ_{l_{1}}^{(1,v(1))},...,ξ_{l_{k}}^{(k,v(k))}) (15.1)

and

Ĩ_{n,k}^{ε}(f) = \frac{1}{k!} ∑_{v∈V_{k}}(−1)^{m(v)}  (15.2)

∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{r}≤n, r=1,...,k, \\ l_{r}≠l_{r′} if r≠r′ \end{matrix}} ε_{l_{1}} ··· ε_{l_{k}}f ( ξ_{l_{1}}^{(1,v(1))},...,ξ_{l_{k}}^{(k,v(k))})

The number m(v) in the above formulas denotes the number of the digits −1 in the ±1 sequence v of length k, hence it counts how many random variables ξ_{l_{j}}^{(j,1)},

1 ≤ j ≤ k, were replaced by the ‘secondary copy’ ξ_{l_{j}}^{(j,−1)} for a v ∈ V_{k} in the inner sum in formulas (15.1) or (15.2).

The following result holds.

Lemma 15.1. Let us consider a (non-empty) class of functions F of k variables f(x_{1},...,x_{k}) on the space (X^{k},X^{k}) together with the random variables Ĩ_{n,k}(f) and Ĩ_{n,k}^{ε}(f) defined in formulas (15.1) and (15.2) for all f ∈ F. The distributions of the random fields Ĩ_{n,k}(f), f ∈ F, and Ĩ_{n,k}^{ε}(f), f ∈ F, agree.

Let me recall that we say that the distribution of two random fields X(f), f ∈ F, and Y (f), f ∈ F, agree if for any finite sets {f_{1},...,f_{p}} ∈ F the distribution of the random vectors (X(f_{1}),...,X(f_{p})) and (Y (f_{1}),...,Y (f_{p})) agree.

Proof of Lemma 15.1. I even claim that for any fixed sequence

u = (u(1),...,u(n)), u(l) = ±1, 1 ≤ l ≤ n,

of length n the conditional distribution of the field Ĩ_{n,k}^{ε}(f), f ∈ F, under the condition (ε_{1},...,ε_{n}) = u = (u(1),...,u(n)) agrees with the distribution of the field of Ĩ_{n,k}(f), f ∈ F.

Indeed, the random variables Ĩ_{n,k}(f), f ∈ F, defined in (15.1) are functions of a random vector with coordinates (ξ_{l}^{(j)},ξ̄_{l}^{(j)}) = (ξ_{l}^{(j,1)},ξ_{l}^{(j,−1)} ), 1 ≤ l ≤ n, 1 ≤ j ≤ k, and the distribution of this random vector remains the same if the coordinates (ξ_{l}^{(j)},ξ̄_{l}^{(j)}) = (ξ_{l}^{(j,1)},ξ_{l}^{(j,−1)} ) are replaced by (ξ̄_{l}^{(j)},ξ_{l}^{(j)}) = (ξ_{l}^{(j,−1)} ,ξ_{l}^{(j,1)}) for such pairs of indices (l,j) for which u(l) = −1 (and the index j is arbitrary), and the coordinates (ξ_{l}^{(j)},ξ̄_{l}^{(j)}) with such pairs of indices (l,j) for which u(l) = 1 are not modified. As a consequence, the distribution of the random field Ĩ_{n,k}(f|u), f ∈ F, we get by replacing the original vector (ξ_{l}^{(j)},ξ̄_{l}^{(j)}), 1 ≤ l ≤ n, 1 ≤ j ≤ k, in the definition of the expression Ĩ_{n,k}(f) in (15.1) for all f ∈ F by this modified vector depending on u has the same distribution as the random field Ĩ_{n,k}(f), f ∈ F. On the other hand, I claim that the distribution of the random field Ĩ_{n,k}(f|u), f ∈ F, agrees with the conditional distribution of the random field Ĩ_{n,k}^{ε}(f), f ∈ F, defined in (15.2) under the condition that (ε_{1},...,ε_{n}) = u with u = (u(1),...,u(n)).

To prove the last statement let us observe that the conditional distribution of the random field Ĩ_{n,k}^{ε}(f), f ∈ F, under the condition (ε_{1},...,ε_{n}) = u is the same as the distribution of the random field we obtain by putting u(l) = ε_{l}, 1 ≤ l ≤ n, in all coordinates ε_{l} of the random variables Ĩ_{n,k}^{ε}(f). On the other hand, the random variables we get in such a way agree with the random variables appearing in the sum defining Ĩ_{n,k}(f|u), only the terms in this sum are listed in a different order. Lemma 15.1 is proved.

Next I prove the following generalized version of Lemma 7.1.

Lemma 15.2 (Generalized version of the Symmetrization Lemma). Let Z_{p} and Z̄_{p}, p = 1,2,..., be two sequences of random variables on a probability space (Ω,A ,P). Let a σ-algebra B ⊂ A be given on the probability space (Ω,A ,P) together with a B-measurable set B and two numbers α > 0 and β > 0 such that the random variables Z_{p}, p = 1,2,..., are B measurable, and the inequality

P(|Z̄_{p}| ≤ α|B)(ω) ≥ β for all p = 1,2,... if ω ∈ B  (15.3)

holds. Then

P ( sup_{1≤p<∞} |Z_{p}| > α + u) ≤ \frac{1}{β} P ( sup_{1≤p<∞} |Z_{p} − Z̄_{p}| > u) + (1 − P(B)) (15.4)

for all u > 0.

Proof of Lemma 15.2. Put τ = min{p: |Z_{p}| > α + u) if there exists such an index p ≥ 1, and put τ = 0 otherwise. Then we have, as {τ = p} ∩ B ∈ B

P(\{τ = p\} ∩ B) = ∫_{\{τ=p\}∩B} 1 · dP ≤ ∫_{\{τ=p\}∩B} \frac{1}{β} P(|Z̄_{p}| ≤ α|B)dP
= \frac{1}{β} P(\{τ = p\} ∩ \{|Z̄_{p}| ≤ α\} ∩ B)
≤ \frac{1}{β} P(\{τ = p\} ∩ \{|Z_{p} − Z̄_{p}| > u\}) for all p = 1,2,....

Hence

P ( sup_{1≤p<∞} |Z_{p}| > α + u) − (1 − P(B)) ≤ P ( { sup_{1≤p<∞} |Z_{p}| > α + u} ∩ B)

= ∑_{p=1}^{∞} P(\{τ = p\} ∩ B) ≤ \frac{1}{β} ∑_{p=1}^{∞} P(\{τ = p\} ∩ \{|Z_{p} − Z̄_{p}| > u\})

≤ \frac{1}{β} P ( sup_{1≤p<∞} |Z_{p} − Z̄_{p}| > u) .

Lemma 15.2 is proved.

To find a symmetrization argument useful in the proof of Proposition 14.2′ we

want to bound the probability P ( n^{−k/2} sup_{f∈F}|k!Ī_{n,k}(f)| > u) by

C · P ( n^{−k/2} sup_{f∈F}|k!Ĩ_{n,k}(f)| > cu) + a negligible error term

with some appropriate numbers C < ∞ and 0 < c < 1. The random variables Ī_{n,k}(f) and Ĩ_{n,k}(f) appearing in these formulas were defined in (14.11) and (15.1). (Actually we work with a slightly modified version of formula (14.11) where the random variables ξ_{l}^{(j)} are replaced by the random variables ξ_{l}^{(j,1)}.) We shall prove such an estimate with the help of Lemma 15.2. To find the random variables Z_{p} and Z̄_{p} we want to work with in Lemma 15.2 let us list the elements of the class of functions F as F = {f_{1},f_{2},...}. We shall apply Lemma 15.2 with the choice Z_{p} = n^{−k/2}k!Ī_{n,k}(f_{p}) and Z̄_{p} = n^{−k/2}k![Ī_{n,k}(f_{p}) − Ĩ_{n,k}(f_{p})], p = 1,2,..., together with the σ-algebra B = B(ξ_{l}^{(j,1)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k).

Let us observe that Z_{p} is a decoupled U-statistic depending on the random variables ξ_{l}^{(j,1)}, 1 ≤ j ≤ k, 1 ≤ l ≤ n, while Z̄_{p} is a linear combination of decoupled U-statistics whose arguments may contain not only the random variables of the form ξ_{l}^{(j,−1)} , but also the random variables of the form ξ_{l}^{(j,1)}. As a consequence, the random variables Z_{p} and Z̄_{p} are not independent. This is the reason why we cannot apply Lemma 7.2 in the proof of Proposition 14.2′.

We shall show that Lemma 15.2 with the choice of the above defined random variables Z_{p} and Z̄_{p} and the σ-algebra B may help us to prove the estimates we need in our considerations. To apply this lemma we have to show that condition (15.3) holds with an appropriate pair of numbers (α,β) and a B measurable set B of probability almost 1. To check this condition is a hard but solvable problem.

In Lemma 7.2 condition (7.1) played a role similar to condition (15.3) in Lemma 15.2. In that case we could check this condition by estimating the second moments EZ̄_{n}^{2} for all indices n. In the present case we shall estimate the supremum sup_{f_{p}∈F} E(Z̄_{p}^{2}|B) of conditional second moments. In this formula Z̄_{p} is a (complicated)

random variable depending on the function f_{p} ∈ F. The estimation of the supremum of the conditional second moments we want to work with is a hard problem, and the main difficulties of our proof appear at this point.

The conditional second moments whose supremum we want to estimate can be expressed as the integral of a random function that can be written down explicitly. In such a way we get a problem similar to the original one about the estimation of sup_{f∈F}n^{−k/2}k!Ī_{n,k}(f). It turned out that these two problems can be handled similarly.

We can work out a symmetrization argument with the help of Lemma 15.2 in both cases, and an inductive argument similar to Proposition 7.3 can be formulated and proved which supplies the results we want to prove.

We shall prove Proposition 14.2′as a consequence of two inductive propositions formulated in Propositions 15.3 and 15.4. Here we apply an approach similar to the proof of Proposition 6.2 which was done with the help of an inductive proposition formulated in Proposition 7.3. But now we have to prove two inductive propositions simultaneously, because we also have to bound the supremum of some conditional variances, and this demands special attention. To formulate the new inductive propositions first we introduce the notions of good tail behaviour for a class of decoupled U-statistics and good tail behaviour for a class of integrals of decoupled U-statistics.

Definition of good tail behaviour for a class of decoupled U-statistics. Let some measurable space (X,X ) be given together with a probability measure μ on it. Let us consider some countable class F of functions f(x_{1},...,x_{k}) on the k-fold product (X^{k},X^{k}) of the space (X,X ). Fix some positive integer n ≥ k and a positive number 0 < σ ≤ 1, and take k independent copies ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence of independent, μ-distributed random variables ξ_{1},...,ξ_{n}. Let us introduce with the help of these random variables the decoupled U-statistics Ī_{n,k}(f), f ∈ F, defined in formula (14.11). Given some real number T > 0 we say that the set of decoupled U-statistics determined by the class of functions F has a good tail behaviour at level T (with parameters n and σ^{2} which are fixed in the sequel) if

P ( sup_{f∈F}|n^{−k/2}k!Ī_{n,k}(f)| ≥ An^{k/2}σ^{k+1}) ≤ exp{−A^{1/2k}nσ^{2}} for all A > T.

(15.5)

Definition of good tail behaviour for a class of integrals of decoupled Ustatistics. Let us have a product space (X^{k}× Y,X^{k}× Y ) with some product measure μ^{k}× ρ, where (X^{k},X^{k},μ^{k}) is the k-fold product of some measurable space (X,X ,μ) with a probability measure μ, and (Y,Y ,ρ) is some other measurable space with a probability measure ρ. Fix some positive integer n ≥ k and a positive number 0 < σ ≤ 1, and consider some countable class F of functions f(x_{1},...,x_{k},y) on the product space (X^{k}×Y,X^{k}×Y ,μ^{k}×ρ). Take k independent copies ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence of independent, μ-distributed random variables ξ_{1},...,ξ_{n}. For all f ∈ F and y ∈ Y let us define the decoupled U-statistics Ī_{n,k}(f,y) = Ī_{n,k}(f_{y}) by means of these random variables ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k, the kernel function f_{y}(x_{1},...,x_{k}) = f(x_{1},...,x_{k},y) and formula (14.11). Define with the help of these U-statistics Ī_{n,k}(f,y) the random integrals

H_{n,k}(f) = ∫ [k!Ī_{n,k}(f,y)]^{2}ρ(dy), f ∈ F .  (15.6)

Choose some real number T > 0. We say that the set of random integrals H_{n,k}(f), f ∈ F, has a good tail behaviour at level T (with parameters n and σ^{2} which we fix in the sequel) if

P ( sup_{f∈F}n^{−k} H_{n,k}(f) ≥ A^{2}n^{k}σ^{2k+2}) ≤ exp{−A^{1/(2k+1)}nσ^{2}}
for all A > T.  (15.7)

Propositions 15.3 and 15.4 will be formulated with the help of the above notions.

Proposition 15.3. Let us fix a positive integer n ≥ max(k,2), a real number 0 < σ ≤ 2^{−(k+1)} , a probability measure μ on a measurable space (X,X ) together with two real numbers L ≥ 1 and D ≥ 1 such that nσ^{2} ≥ Llogn + logD. Let us consider those countable L_{2}-dense classes F of canonical kernel functions f = f(x_{1},...,x_{k}) (with respect to the measure μ) on the k-fold product space (X^{k},X^{k}) with exponent L and parameter D for which all functions f ∈ F satisfy the inequalities

sup_{x_{j}∈X,1≤j≤k} |f(x_{1},...,x_{k})| ≤ 2^{−(k+1)} and ∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) ≤ σ^{2}.

There is a real number A_{0} = A_{0}(k) > 1 such that if for all classes of functions F which satisfy the above conditions the sets of decoupled U-statistics Ī_{n,k}(f), f ∈ F, have a good tail behaviour at level T^{4/3} for some T ≥ A_{0}, then they also have a good tail behaviour at level T.

Proposition 15.4. Fix some positive integer n ≥ max(k,2), a real number 0 < σ ≤ 2^{−(k+1)} , a product space (X^{k}× Y,X^{k}× Y ) with some product measure μ^{k}× ρ, where (X^{k},X^{k},μ^{k}) is the k-fold product of some probability space (X,X ,μ), and (Y,Y ,ρ) is some other probability space together with two real numbers L ≥ 1 and D ≥ 1 such that the inequality nσ^{2} > Llogn + logD holds.

Let us consider those countable L_{2}-dense classes F consisting of canonical functions f(x_{1},...,x_{k},y) on the product space (X^{k}× Y,X^{k}× Y ) with exponent L ≥ 1 and parameter D ≥ 1 whose elements f ∈ F satisfy the inequalities

sup_{x_{j}∈X,1≤j≤k,y∈Y} |f(x_{1},...,x_{k},y)| ≤ 2^{−(k+1)}  (15.8)

and

∫ f^{2}(x_{1},...,x_{k},y)μ(dx_{1})...μ(dx_{k})ρ(dy) ≤ σ^{2} for all f ∈ F .  (15.9)

There exists some number A_{0} = A_{0}(k) > 1 such that if for all classes of functions F which satisfy the above conditions the random integrals H_{n,k}(f), f ∈ F, defined in (15.6) have a good tail behaviour at level T^{(2k+1)/2k} with some T ≥ A_{0}, then they also have a good tail behaviour at level T.

Remark: To complete the formulation of Proposition 15.4 we still have to clarify when we call a function f(x_{1},...,x_{k},y) defined on the product space (X^{k}×Y,X^{k}× Y ,μ^{k}× ρ) canonical. Here we apply a definition which slightly differs from that given in formula (8.10).

We say that a function f(x_{1},...,x_{k},y) on the product space (X^{k} × Y,X^{k} × Y ,μ^{k}× ρ) is canonical if

∫ f(x_{1},...,x_{j−1} ,u,x_{j+1} ,...,x_{k},y)μ(du) = 0
for all 1 ≤ j ≤ k, x_{s} ∈ X, s ≠ j and y ∈ Y.

In this definition we do not require the analogous identity if we integrate with respect to the variable Y with fixed arguments x_{j} ∈ X, 1 ≤ j ≤ k.

Let me also remark that the estimate (15.7) we have formulated in the definition of the property ‘good tail behaviour for a class of integrals of U-statistics’ is fairly natural. We have applied the natural normalization, and with such a normalization it is natural to expect that the tail behaviour of the distribution of sup_{f∈F}n^{−k} H_{n,k}(f)

is similar to that of const.(ση^{k}) ^{2}, where η is a standard normal random variable. Formula (15.7) expresses such a behaviour, only the power of the number A in the exponent at the right-hand side was chosen in a non-optimal way. Formula (15.5) in the formulation of the property ‘good tail behaviour for a class of decoupled U-statistics’ has a similar interpretation. It says that sup_{f∈F}|n^{−k/2}k!I_{n,k}(f)| behaves

similarly to const.σ|η^{k}| with a standard normal random variable η.

We wanted to prove the property of good tail behaviour for a class of integrals of decoupled U-statistics under appropriate, not too restrictive conditions. Let me remark that in Proposition 15.4 we have imposed beside formula (15.8) a fairly weak condition (15.9) about the L_{2}-norm of the function f. Most difficulties appear in the proof, because we did not want to impose more restrictive conditions.

It is not difficult to derive Proposition 14.2′from Proposition 15.3. Indeed, let us observe that the set of decoupled U-statistics determined by a class of functions F satisfying the conditions of Proposition 15.3 has a good tail-behaviour at level T_{0} = σ^{−(k+1)} , since under the conditions of this Proposition the probability at the lefthand side of (15.5) equals zero for A > σ^{−(k+1)} . Then we get from Proposition 15.3 by induction with respect to the number j, that this set of decoupled U-statistics has a good tail-behaviour also for all T = T_{j} = T_{0}^{(3/4)^{j}} = σ^{−(k+1)(3/4)^{j}} , j = 0,1,2,..., with such indices j for which T_{j} = σ^{−(k+1)(3/4)^{j}} ≥ A_{0}. This implies that if a class of functions F satisfies the conditions of Proposition 15.3, then the set of decoupled U-statistics determined by this class of functions has a good tail-behaviour at level T = A_{0}^{4/3}, i.e. at a level which depends only on the order k of the decoupled Ustatistics. This result implies Proposition 14.2′, only it has to be applied for the class of function F′= {2^{−(k+1)} f, f ∈ F } instead of the original class of functions F which appears in Proposition 14.2′with the same parameters σ, L and D.

Similarly to the above argument an inductive procedure yields a corollary of Proposition 15.4 formulated below. Actually, we shall need this corollary of Proposition 15.4.

Corollary of Proposition 15.4. If the class of functions F satisfies the conditions of Proposition 15.4, then there exists a constant Ā_{0}= Ā_{0}(k) > 0 depending only on k such that the class of integrals H_{n,k}(f), f ∈ F, defined in formula (15.6) have a good tail behaviour at level Ā_{0}.

Proposition 15.3 will be proved by means of a symmetrization argument which applies Lemma 15.2. The main difficulty arises when we want to check condition (15.3) with the quantities we are working with in Proposition 15.3. This difficulty can be overcome by means of Proposition 15.4, more precisely by means of its corollary. It helps us to estimate the conditional variances of the decoupled U-statistics we have to handle in the proof of Proposition 15.3. The proof of Propositions 15.3 and 15.4 apply similar arguments, and they will be proved simultaneously. The following inductive procedure will be applied in their proof. First Proposition 15.3 and then Proposition 15.4 will be proved for k = 1. If Propositions 15.3 and 15.4 are already proved for all k′< k for some number k, then first we prove Proposition 15.3 and then Proposition 15.4 for this number k.

The symmetrization arguments needed in the proof of Propositions 15.3 and 15.4 will be proved in Chapter 16. Then Propositions 15.3 and 15.4 will be proved in Chapter 17 with their help. These results imply Proposition 14.2′, hence also Theorem 8.4.

A symmetrization argument

The proof of Propositions 15.3 and 15.4 applies some ideas similar to the argument in the proof of Proposition 7.3. But here some additional technical difficulties have to be overcome. As a first step, two results formulated in Lemma 16.1A and 16.1B will be proved. They can be considered as a randomization argument with the help of Rademacher functions. They are analogous to Lemma 7.2 which was applied in the proof of Propositions 7.3. Lemma 16.1A will be applied in the proof of Proposition 15.3 and Lemma 16.1B in the proof of Proposition 15.4. In this chapter these lemmas will be proved. Their proofs will be based on some additional lemmas formulated in Lemmas 16.2A, 16.2B, 16.3A and 16.3B. By exploiting the structure of Propositions 15.3 and 15.4 we may assume when proving them for parameter k that they hold (together with their consequences) for all parameters k′< k.

Lemma 16.1A is a natural multivariate version of Lemma 7.2. Lemma 7.2 enabled us to replace the estimation of the supremum of a class of sums of independent random variables with the estimation of the supremum of the randomized version of these sums. Lemma 16.1A will enable us to reduce the proof of Proposition 15.3 to the estimation of the tail-distribution of the supremum of an appropriately defined class of randomized decoupled, degenerate U-statistics. This supremum will be estimated by means of the multi-dimensional version of Hoeffding’s inequality given in Theorem 13.3. Lemma 16.B plays a similar role in the proof of Proposition 15.4. But its application is more difficult. In this result the probability investigated in Proposition 15.4 is bounded by means of an expression depending on the supremum of some random variables W̄(f), f ∈ F, which will be defined in formula (16.7). The expressions W̄(f), f ∈ F, are rather complicated, and they are worth studying more closely. This will be done in the proof of Corollary of Lemma 16.1B which yields a more appropriate bound for the probability we want to estimate in Proposition 15.4. In the proof of Proposition 15.4 the Corollary of Lemma 16.1B will be applied instead of the original Lemma 16.1B.

The proof of Lemmas 16.1A and 16.1B is similar to that of Lemma 7.2. First we introduce k additional independent copies ξ̄_{1}^{(j)},...,ξ̄_{n}^{(j)} beside the k (independent and identically distributed) copies ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k, of the sequence ξ_{1},...,ξ_{n} and construct with their help some appropriate random sums. We shall prove in Lemmas 16.2A and 16.2B that these random sums have the same distribution as their randomized versions we shall work with in the proof of Lemmas 16.1A and 16.1B. These Lemmas formulate a natural multivariate version of an important argument in the proof of Lemma 7.2. In the proof of Lemma 7.2 we have exploited that the random sums defined in (7.4) have the same joint distribution as their randomized versions defined in (7.5). Lemmas 16.2A and 16.2B are natural multivariate versions of this statement. They enable us (similarly to the corresponding argument in the proof of Lemma 7.2) to reduce the proof of Propositions 16.1A and 16.1B to the study of some simpler questions. This will be done with the help of Lemmas 16.3A and 16.3B. In Lemma 16.3A the supremum of some conditional variances is estimated under appropriate conditions. This lemma plays a similar role in the proof of Lemma 16.1A as condition (7.1) plays in the proof of Lemma 7.1. Its result together with Lemma 15.2, which is a generalized form of the symmetrization Lemma, Lemma 7.1, enable us to prove Lemma 16.1A. Lemma 16.1B will be proved similarly, but here the conditional distribution of a more complicated expression has to be estimated. This can be done with the help of Lemma 16.3B. In Lemma 16.3B the supremum of the conditional expectation of some expressions is bounded.

The main results of this chapter are the following two lemmas.

Lemma 16.1A (Randomization argument in the proof of Proposition 15.3). Let F be a class of functions on the space (X^{k},X^{k}) which satisfies the conditions of Proposition 15.3 with some probability measure μ. Let us have k independent copies ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence of independent μ distributed random variables ξ_{1},...,ξ_{n} and a sequence of independent random variables ε = (ε_{1},...,ε_{n}), P(ε_{l} = 1) = P(ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is independent also of the random sequences ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k. Consider the decoupled U-statistics Ī_{n,k}(f), f ∈ F, defined with the help of these random variables by formula (14.11) together with their randomized version Ī_{n,k}^{ε}(f) defined in formula (14.12).

There exist some constants A_{0} = A_{0}(k) > 0 and γ = γ_{k}> 0 such that the inequality

P ( sup_{f∈F}n^{−k/2} |k!Ī_{n,k}(f)| > An^{k/2}σ^{k+1})

< 2^{k+1}P ( sup_{f∈F} |k!Ī_{n,k}^{ε}(f)| > 2^{−(k+1)} An^{k}σ^{k+1})

+2^{k}n^{k−1}e^{−γ_{k}A^{1/(2k−1)} nσ^{2}/k}  (16.1)

holds for all A ≥ A_{0}.

It may be worth remarking that the second term at the right-hand side of formula (16.1) yields a small contribution to the upper bound in this relation because of the condition nσ^{2} ≥ Llogn + logD.

To formulate Lemma 16.1B first some new quantities have to be introduced. Some of them will be used somewhat later. The quantities Ī_{n,k}^{V}(f,y) introduced in the subsequent formula (16.2) depend on the sets V ⊂ {1,...,k}, and they are the natural modifications of the inner sum terms in formula (15.1). Such expressions are needed in the formulation of the symmetrization result applied in the proof of Proposition 15.4. Their randomized versions Ī_{n,k}^{(V,ε)}(f,y), introduced in formula (16.5), correspond to the inner sum terms in formula (15.2). The integrals of these expressions will be also introduced in formulas (16.3) and (16.6).

Let us consider a class F of functions f(x_{1},...,x_{k},y) ∈ F on a space (X^{k}× Y,X^{k}× Y ,μ^{k}× ρ) which satisfies the conditions of Proposition 15.4. Let us take 2k independent copies ξ_{1}^{(j)},...,ξ_{n}^{(j)}, ξ̄_{1}^{(j)},...,ξ̄_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence of independent μ distributed random variables ξ_{1},...,ξ_{n} together with a sequence of independent random variables (ε_{1},...,ε_{n}), P(ε_{l} = 1) = P(ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is also independent of the previous random sequences. Let us introduce the notation ξ_{l}^{(j,1)} = ξ_{l}^{(j)} and ξ_{l}^{(j,−1)} = ξ̄_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k. For all subsets V ⊂ {1,...,k} of the set {1,...,k} let |V | denote the cardinality of this set, and define for all functions f(x_{1},...,x_{k},y) ∈ F and sets V ⊂ {1,...,k} the decoupled U-statistics

Ī_{n,k}^{V}(f,y) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} f ( ξ_{l_{1}}^{(1,δ_{1}(V))},...,ξ_{l_{k}}^{(k,δ_{k}(V))},y) ,  (16.2)

where δ_{j}(V) = ±1, 1 ≤ j ≤ k, is defined as δ_{j}(V) = 1 if j ∈ V, and δ_{j}(V) = −1 if j ∉ V, together with the random variables

H_{n,k}^{V}(f) = ∫ [k!Ī_{n,k}^{V}(f,y)]^{2}ρ(dy), f ∈ F .  (16.3)

We shall consider Ī_{n,k}^{V}(f,y) defined in (16.2) as a random variable with values in the space L_{2}(Y,Y ,ρ).

Put

Ī_{n,k}(f,y) = Ī_{n,k}^{{1,...,k}}(f,y), H_{n,k}(f) = H_{n,k}^{{1,...,k}}(f),  (16.4)

i.e. Ī_{n,k}(f,y) and H_{n,k}(f) are the random variables Ī_{n,k}^{V}(f,y) and H_{n,k}^{V}(f) with V = {1,...,k}, which means that these expressions are defined with the help of the random variables ξ_{l}^{(j)} = ξ_{l}^{(j,1)}, 1 ≤ j ≤ k, 1 ≤ l ≤ n.

Let us also define the ‘randomized version’ of the random variables Ī_{n,k}^{V}(f,y) and H_{n,k}^{V}(f) as

Ī_{n,k}^{(V,ε)}(f,y) = \frac{1}{k!}  ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}}ε_{l_{1}} ··· ε_{l_{k}}f ( ξ_{l_{1}}^{(1,δ_{1}(V))},...,ξ_{l_{k}}^{(k,δ_{k}(V))},y) ,

if f ∈ F ,  (16.5)

and

H_{n,k}^{(V,ε)}(f) = ∫ [k!Ī_{n,k}^{(V,ε)}(f,y)]^{2}ρ(dy), f ∈ F ,  (16.6)

where δ_{j}(V) = 1 if j ∈ V, and δ_{j}(V) = −1 if j ∈ {1,...,k} \ V. Similarly to formula (16.2), we shall consider Ī_{n,k}^{V,ε}(f,y) defined in (16.5) as a random variable with values in the space L_{2}(Y,Y ,ρ).

Let us also introduce the random variables

W̄(f) = ∫ [ ∑_{V ⊂{1,...,k}}(−1)^{(k−|V |)}k!Ī_{n,k}^{(V,ε)}(f,y)] ^{2}ρ(dy), f ∈ F .  (16.7)

With the help of the above notations Lemma 16.1B can be formulated in the following way.

Lemma 16.1B (Randomization argument in the proof of Proposition 15.4). Let F be a set of functions on (X^{k}× Y,X^{k}× Y ) which satisfies the conditions of Proposition 15.4 with some probability measure μ^{k}×ρ. Let us have 2k independent copies ξ_{1}^{(j,±1)} ,...,ξ_{n}^{(j,±1)} , 1 ≤ j ≤ k, of a sequence of independent μ distributed random variables ξ_{1},...,ξ_{n} together with a sequence of independent random variables ε_{1},...,ε_{n}, P(ε_{j} = 1) = P(ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, which is independent also of the previously considered random sequences.

Then there exist some constants A_{0} = A_{0}(k) > 0 and γ = γ_{k} such that if the integrals H_{n,k}(f), f ∈ F, determined by this class of functions F have a good tail behaviour at level T^{(2k+1)/2k} for some T ≥ A_{0}, (this property was defined in Chapter 15 in the definition of good tail behaviour for a class of integrals of decoupled U-statistics before the formulation of Propositions 15.3 and 15.4), then the inequality

P ( sup_{f∈F} |H_{n,k}(f)| > A^{2}n^{2k}σ^{2(k+1)}) < 2P ( sup_{f∈F}|W̄(f)| > \frac{A^{2}k!}{2}n^{2k}σ^{2(k+1)})
+2^{2k+1}n^{k−1}e^{−γ_{k}A^{1/2k}nσ^{2}/k}  (16.8)

holds for all A ≥ T with the random variables H_{n,k}(f) introduced in the second identity of relation (16.4) and with W̄(f) defined in formula (16.7).

A corollary of Lemma 16.1B will be formulated which can be better applied than the original lemma. Lemma 16.B is a little bit inconvenient, because the expression at the right-hand side of formula (16.8) contains a probability depending on sup_{f∈F}|W̄(f)|, and W̄(f) is a too complicated expression. Some new formulas (16.9)

and (16.10) will be introduced which enable us to rewrite W̄(f) in a slightly simpler form. These formulas yield such a corollary of Lemma 16.B which is more appropriate for our purposes. To work out the details first some diagrams will be introduced.

Let G = G (k) denote the set of all diagrams consisting of two rows, such that both rows of these diagrams are the set {1,...,k}, and these diagrams contain some edges {(j_{1},j′_{1})...,(j_{s},j′_{s})}, 0 ≤ s ≤ k, connecting a point (vertex) of the first row with a point (vertex) of the second row. The vertices j_{1},...,j_{s} which are end points of some edge in the first row are all different, and the same relation holds also for the vertices j′_{1},...,j′_{s}in the second row. Given a diagram G ∈ G let e(G) = {(j_{1},j′_{1})...,(j_{s},j′_{s})} denote the set of its edges, and let v_{1}(G) = {j_{1},...,j_{s}} be the set of those vertices in the first row and v_{2}(G) = {j′_{1},...,j′_{s}} the set of those vertices in the second row of the diagram G from which an edge of G starts.

Given a diagram G ∈ G, two sets V_{1},V_{2} ⊂ {1,...,k}, a function f defined on the space (X^{k}×,Y,X^{k}× Y ) and a probability measure ρ on (Y,Y ) we define the following random variables H_{n,k}(f|G,V_{1},V_{2}) with the help of the random variables ξ_{1}^{(j,1)},...,ξ_{n}^{(j,1)}, ξ_{1}^{(j,−1)} ,...,ξ_{n}^{(j,−1)} , 1 ≤ j ≤ k, and ε = (ε_{1},...,ε_{n}) taking part in the definition of the random variables W̄(f):

H_{n,k}(f|G,V_{1},V_{2})
= ∑_{(l_{1},...,l_{k},l′_{1},...,l′_{k}): 1≤l_{j}≤n,l_{j} ≠l_{j′} if j≠j′,1≤j,j′≤k, 1≤l′_{j}≤n,l′_{j}≠l′_{j′} if j≠j′,1≤j,j′≤k, l_{j}=l′_{j′} if (j,j′)∈e(G), l_{j} ≠l′_{j′} if (j,j′)∉e(G)} ∏_{j∈{1,...,k}\v_{1}(G)}ε_{l_{j}} ∏_{j∈{1,...,k}\v_{2}(G)}ε_{l′_{j}}

∫ f(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))},...,ξ_{l_{k}}^{(k,δ_{k}(V_{1}))},y)
f(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))},...,ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))},y)ρ(dy),  (16.9)

where δ_{j}(V_{1}) = 1 if j ∈ V_{1}, δ_{j}(V_{1}) = −1 if j ∉ V_{1}, and δ_{j}(V_{2}) = 1 if j ∈ V_{2}, δ_{j}(V_{2}) = −1 if j ∉ V_{2}. (Let us observe that if the graph G contains s edges, then the product of the ε-s in (16.9) contains 2(k − s) terms, and the number of terms in the sum (16.9) is less than n^{2k−s}.) As the Corollary of Lemma 16.1B will indicate, in the proof of Proposition 15.4 we shall need a good estimate on the tail distribution of the random variables H_{n,k}(f|G,V_{1},V_{2}) for all f ∈ F and G ∈ G, V_{1},V_{2} ⊂ {1,...,k}. Such an estimate can be obtained by means of Theorem 13.3, the multivariate version of Hoeffding’s inequality. But the estimate we get in such a way will be rewritten in a form more appropriate for our inductive procedure. This will be done in the next chapter.

The identity

W̄(f) = ∑_{G∈G ,V_{1},V_{2}⊂{1,...,k}}(−1)^{|V_{1}|+|V_{2}|}H_{n,k}(f|G,V_{1},V_{2})  (16.10)

will be proved.

To prove this identity let us write first

W̄(f) = ∑_{V_{1},V_{2}⊂{1,...,k}}(−1)^{|V_{1}|+|V_{2}|}∫ k!Ī_{n,k}^{(V_{1},ε)}(f,y)k!Ī_{n,k}^{(V_{2},ε)}(f,y)ρ(dy).

Let us express the products k!Ī_{n,k}^{(V_{1},ε)}(f,y)k!Ī_{n,k}^{(V_{2},ε)}(f,y) by means of formula (16.5). Let us rewrite this product as a sum of products of the form

∏_{j=1}^{k} ε_{l_{j}}f(··· )∏_{j=1}^{k} ε_{l′_{j}}f(··· ),

and let us define the following partition of the terms in this sum. The elements of this partition are indexed by the diagrams G ∈ G, and if we take a diagram G ∈ G with the set of edges e(G) = {(j_{1},j′_{1}),...,(j_{s},j′_{s})}, then the term of this sum determined by the indices l_{1},...,l_{k},l′_{1},...,l′_{k} belongs to the element of the partition indexed by this diagram G if and only if l_{j_{u}} = l′_{j′_{u}} for all 1 ≤ u ≤ s, and no more numbers between the indices l_{1},...,l_{k},l′_{1}...,l′_{k}may agree. Since ε_{l_{ju}}ε_{l′_{j′_{u}}} = 1 for all 1 ≤ u ≤ s and the set

of indices of the remaining random variables ε_{l_{j}} is {l_{j} : j ∈ {1,...,k} \ v_{1}(G)}, the set of indices of the remaining random variables ε_{l′_{j}} is {l′_{j}: j ∈ {1,...,k} \ v_{2}(G)},

we get by integrating the product k!Ī_{n,k}^{(V_{1},ε)}(f,y)k!Ī_{n,k}^{(V_{2},ε)}(f,y) with respect to the measure ρ that

∫ Ī_{n,k}^{(V_{1},ε)}(f,y)Ī_{n,k}^{(V_{2},ε)}(f,y)ρ(dy) = ∑_{G∈G}H_{n,k}(f|G,V_{1},V_{2})

for all V_{1},V_{2} ∈ {1,...,k}. The last two identities imply formula (16.10).

Since the number of terms in the sum of formula (16.10) is less than 2^{4k}k!, this relation implies that Lemma 16.1B has the following corollary:

Corollary of Lemma 16.1B (A simplified version of the randomization argument of Lemma 16.1B). Let a set of functions F satisfy the conditions of Proposition 15.4. Then there exist some constants A_{0} = A_{0}(k) > 0 and γ = γ_{k} > 0 such that if the integrals H_{n,k}(f), f ∈ F, determined by this class of functions F have a good tail behaviour at level T^{(2k+1)/2k} for some T ≥ A_{0}, then the inequality

P ( sup_{f∈F}|H_{n,k}(f)| > A^{2}n^{2k}σ^{2(k+1)})

≤ 2 ∑_{G∈G ,V_{1},V_{2}⊂\{1,...,k\}}P ( sup_{f∈F} |H_{n,k}(f|G,V_{1},V_{2})|> \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}} )

+2^{2k+1}n^{k−1}e^{−γ_{k}A^{1/2k}nσ^{2}/k}  (16.11)

holds for all A ≥ T with the random variables H_{n,k}(f) and H_{n,k}(f|G,V_{1},V_{2}) defined in formulas (16.4) and (16.9).

In the proof of Lemmas 16.1A and 16.1B the result of the following Lemmas 16.2A and 16.2B will be applied.

Lemma 16.2A. Let us take 2k independent copies

ξ_{1}^{(j,1)},...,ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)} ,...,ξ_{n}^{(j,−1)} , 1 ≤ j ≤ k,

of a sequence of independent μ distributed random variables ξ_{1},...,ξ_{n} together with a sequence of independent random variables (ε_{1},...,ε_{n}), P(ε_{l} = 1) = P(ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is also independent of the previous sequences.

Let F be a class of functions which satisfies the conditions of Proposition 15.3. Introduce with the help of the above random variables for all sets V ⊂ {1,...,k} and functions f ∈ F the decoupled U-statistic

Ī_{n,k}^{V}(f) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k, \\ l_{j} ≠l_{j′} ifj≠j′ \end{matrix}} f ( ξ_{l_{1}}^{(1,δ_{1}(V))},...,ξ_{l_{k}}^{(k,δ_{k}(V))})  (16.12)

and its ‘randomized version’

Ī_{n,k}^{(V,ε)}(f) = \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k, \\ l_{j} ≠l_{j′} ifj≠j′ \end{matrix}} ε_{l_{1}} ··· ε_{l_{k}}f ( ξ_{l_{1}}^{(1,δ_{1}(V))},...,ξ_{l_{k}}^{(k,δ_{k}(V))}) ,

f ∈ F ,  (16.13)

where δ_{j}(V) = ±1, and we have δ_{j}(V) = 1 if j ∈ V, and δ_{j}(V) = −1 if j ∈ {1,...,k} \ V.

Then the sets of random variables

S(f) = ∑_{V ⊂{1,...,k}}(−1)^{(k−|V |)}k!Ī_{n,k}^{V}(f), f ∈ F ,  (16.14)

and

S̄(f) = ∑_{V ⊂{1,...,k}}(−1)^{(k−|V |)}k!Ī_{n,k}^{(V,ε)}(f), f ∈ F ,  (16.15)

have the same joint distribution.

Lemma 16.2B. Let us take 2k independent copies

ξ_{1}^{(j,1)},...,ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)} ,...,ξ_{n}^{(j,−1)} , 1 ≤ j ≤ k,

of a sequence of independent, μ distributed random variables ξ_{1},...,ξ_{n} together with a sequence of independent random variables (ε_{1},...,ε_{n}), P(ε_{l} = 1) = P(ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is independent also of the previous sequences.

Let us consider a class F of functions f(x_{1},...,x_{k},y) ∈ F on a space (X^{k}× Y,X^{k}×Y ,μ^{k}×ρ) which satisfies the conditions of Proposition 15.4. For all functions f ∈ F and V ∈ {1,...,k} consider the decoupled U-statistics Ī_{n,k}^{V}(f,y) de-

fined by formula (16.2) with the help of the random variables ξ_{1}^{(j,1)},...,ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)} ,...,ξ_{n}^{(j,−1)} , and define with their help the random variables

W (f) = ∫ [ ∑_{V ⊂{1,...,k}}(−1)^{(k−|V |)}k!Ī_{n,k}^{V}(f,y)] ^{2}ρ(dy), f ∈ F .  (16.16)

Then the random vectors {W (f): f ∈ F } defined in (16.16) and {W̄(f): f ∈ F } defined in (16.7) have the same distribution.

Proof of Lemmas 16.2A and 16.2B. Lemma 16.2A actually agrees with the already proved Lemma 15.1, only the notation is different. The proof of Lemma 16.2B is also very similar to that of Lemma 15.1. It can be shown that even the following stronger statement holds. For any ±1 sequence u = (u_{1},...,u_{n}) of length n the conditional distribution of the random field W̄(f), f ∈ F, under the condition (ε_{1},...,ε_{n}) = u = (u_{1},...,u_{n}) agrees with the distribution of the random field W (f), f ∈ F.

To see this relation let us first observe that the conditional distribution of the field W̄(f) under this condition agrees with the distribution of the random field we get by replacing the random variables ε_{l} by u_{l} for all 1 ≤ l ≤ n in formulas (16.5), (16.6) and (16.7). Beside this, define the vector (ξ(u)_{l}^{(j,1)},ξ(u)_{l}^{(j,−1)} ), 1 ≤ j ≤ k, 1 ≤ l ≤ n, by the formula (ξ(u)_{l}^{(j,1)},ξ(u)_{l}^{(j,−1)} ) = (ξ_{l}^{(j,−1)} ,ξ_{l}^{(j,1)}) for those indices (j,l) for which u_{l} = −1, and (ξ(u)_{l}^{(j,1)},ξ(u)_{l}^{(j,−1)} ) = (ξ_{l}^{(j,1)},ξ_{l}^{(j,−1)} ) for which u_{l} = 1 (independently of the value of the parameter j). Then the joint distribution of the vectors (ξ(u)_{l}^{(j,1)},ξ(u)_{l}^{(j,−1)} ), 1 ≤ j ≤ k, 1 ≤ l ≤ n, and (ξ_{l}^{(j,1)},ξ_{l}^{(j,−1)} ), 1 ≤ j ≤ k, 1 ≤ l ≤ n, agree. Hence the joint distribution of the random vectors Ī_{n,k}^{V}(f,y), f ∈ F, V ⊂ {1,...,k} defined in (16.2) and of the random vectors W (f), f ∈ F, defined in (16.16) do not change if we replace in their definition the random variables ξ_{l}^{(j,1)} and ξ_{l}^{(j,−1)} by ξ(u)_{l}^{(j,1)} and ξ(u)_{l}^{(j,−1)} . But the set of random variables W (f), f ∈ F, obtained in this way agrees with the set of random variables we introduced to get a set of random variables with the same distribution as the conditional distribution of W̄(f), f ∈ F under the condition (ε_{1},...,ε_{n}) = u. (These random variables are defined as the square integral of the same sum, only the terms of this sum are listed in a different order in the two cases.) These facts imply Lemma 16.2B.

In the next step we prove the following Lemma 16.3A.

Lemma 16.3A. Let us consider a class of functions F satisfying the conditions of Proposition 15.3 with parameter k together with 2k independent copies ξ_{1}^{(j,1)},…, ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)} ,...,ξ_{n}^{(j,−1)} , 1 ≤ j ≤ k, of a sequence of independent, μ-distributed random variables ξ_{1},...,ξ_{n}. Take the random variables Ī_{n,k}^{V}(f), defined for f ∈ F and V ⊂ {1,...,k} in formula (16.12). Let

B = B(ξ_{1}^{(j,1)},...,ξ_{n}^{(j,1)}, 1 ≤ j ≤ k)

denote the σ-algebra generated by the random variables ξ_{1}^{(j,1)},...,ξ_{n}^{(j,1)} , 1 ≤ j ≤ k, i.e. by the random variables with upper indices of the form (j,1), 1 ≤ j ≤ k. There exists a number A_{0} = A_{0}(k) > 0 such that for all V ⊂ {1,...,k}, V ≠ {1,...,k}, the inequality

P ( sup_{f∈F} E ([k!Ī_{n,k}^{V}(f)]^{2}|B) > 2^{−(3k+3)} A^{2}n^{2k}σ^{2k+2}) < n^{k−1}e^{−γ_{k}A^{1/(2k−1)} nσ^{2}/k}

(16.17)

holds with a sufficiently small γ_{k} > 0 if A ≥ A_{0}.

Proof of Lemma 16.3A. Let us first consider the case V = ∅. In this case the estimate E ( (k!Ī_{n,k}^{∅}(f))^{2}| B) = E ( (k!Ī_{n,k}^{∅}(f))^{2}) ≤ k!n^{k}σ^{2} ≤ 2^{k}k!n^{2k}σ^{2k+2} holds for all f ∈ F. In the above calculation it was exploited that the functions f ∈ F are canonical, which implies certain orthogonalities, and beside this the inequality nσ^{2} ≥ \frac{1}{2} holds, because of the relation nσ^{2} ≥ Llogn + logD. The above relations imply that for V = ∅ the probability at the left-hand side of (16.17) equals zero if the number A_{0} is chosen sufficiently large. Hence inequality (16.17) holds in this case.

To avoid some complications in the notation let us first restrict our attention to sets of the form V = {1,...,u} with some 1 ≤ u < k, and prove relation (16.17) for such sets. For this goal let us introduce the random variables

Ī_{n,k}^{V}(f,l_{u+1} ,...,l_{k})

= \frac{1}{k!} ∑_{(l_{1},...,l_{u}): 1≤l_{j}≤n, j=1,...,u, l_{j} ≠l_{j′} if j≠j′for all 1≤j,j′≤k} f ( ξ_{l_{1}}^{(1,1)},...,ξ_{l_{u}}^{(u,1)},ξ_{l_{u+1}}^{(u+1,−1)} ,...,ξ_{l_{k}}^{(k,−1)})

for all f ∈ F and sequences l(u) = (l_{u+1} ,...,l_{k}) with the properties 1 ≤ l_{j} ≤ n for all u + 1 ≤ j ≤ k and l_{j} ≠ l_{j′} if j ≠ j′, i.e. let us fix the last k − u coordinates ξ_{l_{u+1}}^{(u+1,−1)} ,…, ξ_{l_{k}}^{(k,−1)} of the random variable Ī_{n,k}^{V}(f) and sum up with respect the first u coordinates. Then we can write

E (Ī_{n,k}^{V}(f)^{2}| B)  (16.18)

= E ( ( ∑_{\begin{matrix}(l_{u+1} ,...,l_{k}): 1≤l_{j}≤n j=u+1,...,k, \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} Ī_{n,k}^{V}(f,l_{u+1} ,...,l_{k})) ^{2}|B)

= ∑_{\begin{matrix}(l_{u+1} ,...,l_{k}): 1≤l_{j}≤n, j=u+1,...,k, \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} E (Ī_{n,k}^{V}(f,l_{u+1} ,...,l_{k})^{2}| B) .

The last relation follows from the identity

E (Ī_{n,k}^{V}(f,l_{u+1} ,...,l_{k})Ī_{n,k}^{V}(f,l′_{u+1} ,...,l′_{k})| B) = 0

if (l_{u+1} ,...,l_{k}) ≠ (l′_{u+1} ,...,l′_{k}), which holds, since f is a canonical function. We still exploit that the random variables ξ_{l}^{(j,1)}, 1 ≤ j ≤ u are B measurable, while the random variables ξ_{l_{j}}^{(j,−1)} , u+1 ≤ j ≤ k, are independent of the σ-algebra B. These facts enable us to calculate the above conditional expectation in a simple way.

It follows from relation (16.18) that

{ ω: sup_{f∈F} E ([k!Ī_{n,k}^{V}(f)]^{2}|B) (ω) > 2^{−(3k+3)} A^{2}n^{2k}σ^{2k+2}}  (16.19)

⊂ ⋃_{\begin{matrix}(l_{u+1} ,...,l_{k}): \\ 1≤l_{j}≤n, j=u+1,...,k. \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}}

\{ ω: sup_{f∈F} E ([k!Ī_{n,k}^{V}(f,l_{u+1} ,...,l_{k})]^{2}|B) (ω) > \frac{A^{2}n^{2k}σ^{2k+2}}{2^{(3k+3)}n^{k−u}} \} .

The probability of the events in the union at the right-hand side of (16.19) can be estimated with the help of the Corollary of Proposition 15.4 with parameter u < k instead of k. (We may assume that Proposition 15.4 holds for u < k.) I claim that this corollary yields that

P ( sup_{f∈F} E ([k!Ī_{n,k}^{V}(f,l_{u+1} ,...,l_{k})]^{2}|B) > \frac{A^{2}n^{k+u}σ^{2k+2}}{2^{(3k+3)}} )

≤ e^{−γ_{k}A^{1/(2u+1)} (n+u−k)σ^{2}}  (16.20)

with an appropriate γ_{k} > 0 for all sequences (l_{u+1} ,...,l_{k}), 1 ≤ l_{j} ≤ n, u+1 ≤ j ≤ k, such that l_{j} ≠ l_{j′} if j ≠ j′.

Let us show that if a class of functions f ∈ F satisfies the conditions of Proposition 15.3, then it also satisfies relation (16.20). For this goal introduce the space (Y,Y ,ρ) = (X^{k−u},X^{k−u},μ^{k−u}), the k − u-fold power of the measure space (X,X ,μ), and for the sake of simpler notations write y = (x_{u+1} ,...,x_{k}) for a point y ∈ Y. Let us also introduce the class of those function F̄ in the space (X^{u}× Y,X^{u}× Y ,μ^{u}× ρ) consisting of functions f̄of the form f̄(x_{1},...,x_{u},y) = f(x_{1},...,x_{k}) with y = (x_{u+1} ,...,x_{k}) and some function f(x_{1},...,x_{k}) ∈ F. If the class of function F satisfies the conditions of Proposition 15.3 (with parameter k), then the class of functions F̄ satisfies the conditions of Proposition 15.4 with parameter u < k. Hence the Corollary of Proposition 15.4 can be applied for the class of functions F̄ by our inductive hypothesis. We shall apply it for decoupled Ustatistics with the class of kernel functions F̄ and parameters n + u − k and u (instead of n and k), with the help of the independent random sequences ξ_{l}^{(j,1)}, 1 ≤ j ≤ u, l ∈ {1,...,n} \ {l_{u+1} ,...,l_{k}} of independent, μ-distributed random variables of length n+u−k, where the set of numbers {l_{u+1} ,...,l_{k}} is the set of indices appearing in formula (16.20). This means that we work with random variables ξ_{l}^{(j,1)} with index l from the set {1,...,n} \ {l_{u+1} ,...,l_{k}} instead of 1 ≤ u ≤ n + u − k. As a consequence, we shall work in the application of Proposition 5.4 with the random variables Ī_{n+u−k,u}^{l(u)} (f̄,y) and H_{n+u−k,u}^{l(u)} (f̄) to be defined below which we get by slightly modifying the definition of Ī_{n+u−k,u} (f̄,y) and H_{n+u−k,u} (f̄) by taking into account the indexation of the random variables ξ_{l}^{(j,1}.

It can be seen by means of some calculation that the conditional expecation E ( [k!Ī_{n,k}^{V}(f,l_{u+1} ,...,l_{k})]^{2}|B) we are working with can be calculated as

E ([k!Ī_{n,k}^{V}(f,l_{u+1} ,...,l_{k})]^{2}|B)
= ∫ [u!Ī_{n+u−k,u}^{l(u)} (f̄,y)]^{2}ρ(dy) = H_{n+u−k,u}^{l(u)} (f̄),  (16.21)

where the function f̄∈ F̄ is defined as f̄(x_{1},...,x_{u},y) = f(x_{1},...,x_{k}) with y = (x_{u+1} ,...,x_{k}), and the random variables Ī_{n+u−k,u}^{l(u)} (f̄,y) and H_{n+u−k,u}^{l(u)} (f̄) are defined, similarly to (16.2)–(16.4), by the formulas

Ī_{n+u−k,u}^{l(u)} (f̄,y)

= \frac{1}{u!} ∑_{\begin{matrix}(l_{1},...,l_{u}): l_{j}∈\{1,...,n\}\\{l_{u+1} ,...,l_{k}\}, j=1,...,u \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} f̄( ξ_{l_{1}}^{(1,1)},...,ξ_{l_{u}}^{(u,1)},y)

and

H_{n+u−k,u}^{l(u)} (f̄) = ∫ [u!Ī_{n+u−k,u}^{l(u)} (f̄,y)]^{2}ρ(dy), f̄∈ F̄.

The value of H_{n+u−k,u}^{l(u)} (f̄) depends on the choice of the sequence l(u), but its distribution does not depend on it. Hence we can make the following estimate with the help of the corollary of Proposition (15.4) for u < k and relation (16.21). Choose a sufficiently small γ = γ_{k} > 0. Then we have

P( sup_{f̄∈F̄}E([k!Ī_{n,k}^{V}(f,l_{u+1} ,...,l_{k})]^{2}|B) ≥ γ_{k}^{(4u+2)} A^{2}(n + u − k)^{2u}σ^{2u+2} )

= P ( sup_{f̄∈F̄}(n + u − k)^{−u} H_{n+u−k,u}^{l(u)} (f̄) ≥ γ_{k}^{(4u+2)} A^{2}(n + u − k)^{u}σ^{2u+2} )

≤ e^{−γ_{k}A^{1/(2u+1)} (n+u−k)σ^{2}} for A > A_{0}(u)γ_{k}^{−(4u+2)} .  (16.22)

It is not difficult to derive formula (16.20) from relation (16.22). It is enough to check that the level \frac{A^{2}n^{k+u}σ^{2k+2}}{2^{(3k+3)}} in the probability at the left-hand side of (16.20) can

be replaced by γ_{k}^{(4u+2)} A^{2}(n + u − k)^{2u}σ^{2u+2} if γ_{k} > 0 is chosen sufficiently small. This statement holds, since γ_{k}^{(4u+2)} A^{2}(n + u − k)^{2u}σ^{2u+2} < γ_{k}^{(4u+2)} A^{2}n^{2u}σ^{2u+2} ≤ \frac{A^{2}n^{k+u}σ^{2k+2}}{2^{(3k+3)}} if the constant γ_{k} > 0 is chosen sufficiently small, since nσ^{2} > Llogn ≤ \frac{1}{2} by the conditions of Proposition 15.3.

Relations (16.19) and (16.20) imply that

P ( sup_{f∈F} E ([k!Ī_{n,k}^{V}(f)]^{2}|B) (ω) > 2^{−(3k+3)} A^{2}n^{2k}σ^{2k+2})

≤ n^{k−u}e^{−γ_{k}A^{1/(2u+1)} (n+u−k)σ^{2}} .

Since e^{−γ_{k}A^{1/(2u+1)} (n+u−k)σ^{2}} ≤ e^{−γ_{k}A^{1/(2k−1)} nσ^{2}/k} if u ≤ k − 1, n ≥ k and A > A_{0} with a sufficiently large number A_{0}, inequality (16.17) holds for all sets V of the form V = {1,...,u}, 1 ≤ u < k.

The case of a general set V ⊂ {1,...,k}, 1 ≤ |V | < k, can be handled similarly, only the notation becomes more complicated. Moreover, the case of general sets V can be reduced to the case of sets of form we have already considered. Indeed, given some set V ⊂ {1,...,k}, 1 ≤ |V | < k, let us define a new class of function F_{V} we get by applying a rearrangement of the indices of the arguments x_{1},...,x_{k} of the functions f ∈ F in such a way that the arguments indexed by the set V are the first |V | arguments of the functions f_{V} ∈ F_{V}, and put V̄ = {1,...,|V |}. Then the class of functions F_{V} also satisfies the condition of Proposition 15.3, and we can get relation (16.17) with the set V by applying it for the set of function F_{V} and set V̄.

Now we prove Lemma 16.1A with the help of Lemma 16.2A, the generalized symmetrization lemma 15.2 and Lemma 16.3A.

Proof of Lemma 16.1A. First we show with the help of the generalized symmetrization lemma, i.e. of Lemma 15.2 and Lemma 16.3A that

P ( sup_{f∈F}n^{−k/2} |k!Ī_{n,k}(f)| > An^{k/2}σ^{k+1})  (16.23)

< 2P ( sup_{f∈F}|S(f)| > \frac{A}{2}n^{k}σ^{k+1}) + 2^{k}n^{k−1}e^{−γ_{k}A^{1/(2k−1)} nσ^{2}/k}

with the function S(f) defined in (16.14). To prove relation (16.23) introduce the random variables Z(f) = k!Ī_{n,k}^{{1,...,k}}(f) and

Z̄(f) = − ∑_{V ⊂{1,...,k},V ≠{1,...,k}}(−1)^{k−|V |}k!Ī_{n,k}^{V}(f)

for all f ∈ F, the σ-algebra B considered in Lemma 16.3A and the set

B = ⋂_{\begin{matrix}V ⊂\{1,...,k\} \\ V ≠\{1,...,k\} \end{matrix}}\{ ω: sup_{f∈F} E ([k!Ī_{n,k}^{V}(f)]^{2}|B) (ω) ≤ 2^{−(3k+3)} A^{2}n^{2k}σ^{2k+2}\} .

Observe that S(f) = Z(f) − Z̄(f), f ∈ F, B ∈ B, and by Lemma 16.3A the inequality 1 − P(B) ≤ 2^{k}n^{k−1}e^{−γ_{k}A^{1/(2k−1)} nσ^{2}/k} holds. To prove relation (16.23) apply Lemma 15.2 with the above introduced random variables Z(f) and Z̄(f), f ∈ F, (both here and in the subsequent proof of Lemma 16.1B we work with random variables Z(·) and Z̄(·) indexed by the countable set of functions f ∈ F, hence the functions f ∈ F play the role of the parameters p when Lemma 15.2 is applied) random set B and α = \frac{A}{2}n^{k}σ^{k+1}, u = \frac{A}{2}n^{k}σ^{k+1}. (At the left-hand side of (16.23) we can replace k!Ī_{n,k}(f) with Z(f), f ∈ F, because they have the same joint distribution.) It is enough to show that

P ( |Z̄(f)| > \frac{A}{2}n^{k}σ^{k+1}|B) (ω) ≤ \frac{1}{2} for all f ∈ F if ω ∈ B.  (16.24)

But

P ( k!|Ī_{n,k}^{|V |}(f)| > 2^{−(k+1)} An^{k}σ^{k+1}|B) (ω)

≤ \frac{2^{2(k+1)}E(Ī_{n,k}^{|V |}(f)^{2}|B)(ω)}{A^{2}n^{2k}σ^{2(k+1)}} ≤ 2^{−(k+1)}

for all functions f ∈ F and sets V ⊂ {1,...,k}, V ≠ {1,...,k}, if ω ∈ B by the ‘conditional Chebishev inequality’, hence relations (16.24) and (16.23) hold.

Lemma 16.1A follows from relation (16.23), Lemma 16.2A and the observation that the random variables Ī_{n,k}^{(V,ε)}(f), f ∈ F, defined in (16.13) have the same distribution for all V ⊂ {1,...,k} as the random variables Ī_{n,k}^{ε}(f), defined in formula (14.12). Hence Lemma 16.2A and the definition (16.15) of the random variables S̄(f), f ∈ F, imply the inequality

P ( sup_{f∈F}|S(f)| > \frac{A}{2}n^{k}σ^{k+1}) = P ( sup_{f∈F}|S̄(f)| > \frac{A}{2}n^{k}σ^{k+1})

≤ 2^{k}P ( sup_{f∈F} |k!Ī_{n,k}^{ε}(f)| > 2^{−(k+1)} An^{k}σ^{k+1}) .

Lemma 16.1A is proved.

Lemma 16.1B will be proved with the help of the following Lemma 16.3B, which is a version of Lemma 16.3A.

Lemma 16.3B. Let us consider a class of functions F satisfying the conditions of Proposition 15.4 together with 2k independent copies

ξ_{1}^{(j,1)},...,ξ_{n}^{(j,1)}, and ξ_{1}^{(j,−1)} ,...,ξ_{n}^{(j,−1)} , 1 ≤ j ≤ k,

of a sequence of independent, μ-distributed random variables ξ_{1},...,ξ_{n}. Take the random variables Ī_{n,k}^{V}(f,y) and H_{n,k}^{V}(f), f ∈ F, V ⊂ {1,...,k}, defined in formulas (16.2) and (16.3) with the help of these quantities. Let

B = B(ξ_{1}^{(j,1)},...,ξ_{n}^{(j,1)}, 1 ≤ j ≤ k)

denote the σ-algebra generated by the random variables ξ_{1}^{(j,1)},...,ξ_{n}^{(j,1)}, 1 ≤ j ≤ k, i.e. by those random variables which appear in the definition of the random variables Ī_{n,k}^{V}(f,y) and H_{n,k}^{V}(f) introduced in formulas (16.2) and (16.3), and have second argument 1 in their upper index.

a) There exist some numbers A_{0} = A_{0}(k) > 0 and γ = γ_{k} > 0 such that for all V ⊂ {1,...,k}, V ≠ {1,...,k}, the inequality

P ( sup_{f∈F}E(H_{n,k}^{V}(f)|B) > \frac{2^{−(4k+4)}}{(k!)^{2}} A^{(2k−1)/k}n^{2k}σ^{2k+2}) < n^{k−1}e^{−γ_{k}A^{1/2k}nσ^{2}/k}

(16.25)

holds if A ≥ A_{0}.

b) Given two subsets V_{1},V_{2} ⊂ {1,...,k} of the set {1,...,k} define the integrals (of random kernel functions)

H_{n,k}^{(V_{1},V_{2})}(f) = ∫ |k!Ī_{n,k}^{V_{1}}(f,y)k!Ī_{n,k}^{V_{2}}(f,y)|ρ(dy), f ∈ F ,  (16.26)

with the help of the functions Ī_{n,k}^{V}(f,y) defined in (16.2). There exist some numbers A_{0} = A_{0}(k) > 0 and γ = γ_{k} > 0 such that if the integrals H_{n,k}(f), f ∈ F, determined by this class of functions F have a good tail behaviour at level T^{(2k+1)/2k} for some T ≥ A_{0}, then the inequality

P ( sup_{f∈F}E(H_{n,k}^{(V_{1},V_{2})}(f)|B) > \frac{2^{−(2k+2)}}{k!} A^{2}n^{2k}σ^{2k+2}) < 2n^{k−1}e^{−γ_{k}A^{1/2k}nσ^{2}/k}

(16.27)

holds for any pairs of subsets V_{1},V_{2} ⊂ {1,...,k} with the property that at least one of them does not equal the set {1,...,k} if the number A satisfies the condition A > T.

Proof of Lemma 16.3B. Part a) of Lemma 16.3B can be proved in almost the same way as Lemma 16.3A. Hence I only briefly explain the main step of the proof. In the case V = ∅ the identity E(H_{n,k}^{V}(f)|B) = E(H_{n,k}^{V}(f)) holds, hence it is enough to show that E(H_{n,k}^{V}(f)) ≤ k!n^{k}σ^{2} ≤ 2^{k}k!n^{2k}σ^{2k+2} for all f ∈ F under the conditions of Proposition 15.4. (This relation holds, because the functions of the class F are canonical.) The case of a general set V, V ≠ ∅ and V ≠ {1,...,k}, can be reduced to the case V = {1,...,u} with some 1 ≤ u < k.

Given a set V = {1,...,u}, 1 ≤ u < k, let us define for all f ∈ F and sequences l(u) = (l_{u+1} ,...,l_{k}) with the properties 1 ≤ l_{j} ≤ n for all u + 1 ≤ j ≤ k and l_{j} ≠ l_{j′} if j ≠ j′the random variable Ī_{n,k}^{V}(f,l_{u+1} ,...,l_{k},y)

= \frac{1}{k!} ∑_{(l_{1},...,l_{u}): 1≤l_{j}≤n, j=1,...,u, l_{j} ≠l_{j′} if j≠j′for all 1≤j,j′≤k} f ( ξ_{l_{1}}^{(1,1)},...,ξ_{l_{u}}^{(u,1)},ξ_{l_{u+1}}^{(u+1,−1)} ,...,ξ_{l_{k}}^{(k,−1)},y) .

It can be shown, similarly to the proof of relation (16.18) in the proof of Proposition 16.3A that since the functions f ∈ F have the canonical property the identity

E (H̄_{n,k}^{V}(f)| B) = ∑_{\begin{matrix}(l_{u+1} ,...,l_{k}): \\ 1≤l_{j}≤n, j=u+1,...,k, \\ l_{j} ≠l_{j′}if j≠j′ \end{matrix}} ∫ E ([k!Ī_{n,k}^{V}(f,l_{u+1} ,...,l_{k},y)]^{2}|B) ρ(dy)

holds, and the proof of part a) of Lemma 16.3B can be reduced to the inequality

P ( sup_{f∈F}E ( ∫ [k!Ī_{n,k}^{V}(f,l_{u+1} ,...,l_{k},y)]^{2}ρ(dy)|B) > \frac{A^{(2k−1)/k}n^{k+u}σ^{2k+2}}{2^{(4k+4)}(k!)^{2}} )

≤ e^{−γ_{k}A^{(2k−1)/2k(2u+1)} (n+u−k)σ^{2}}

with a sufficiently small γ_{k} > 0. This inequality can be proved, similarly to relation (16.20) in the proof of Lemma 16.3A with the help of the Corollary of Proposition 15.4. Only here we have to work in the space (X^{u}×Ȳ,X^{u}×Ȳ,μ^{u}×ρ̄) where Ȳ= X^{k−u} × Y, Ȳ = X^{k−u} × Y , ρ̄ = μ^{k−u} × ρ with the class of function f̄∈ F̄ consisting of the functions f̄defined by the formula f̄(x_{1},...,x_{u},ȳ) = f(x_{1},...,x_{k},y) with some f(x_{1},...,x_{k},y) ∈ F, where ȳ= (x_{u+1} ,...,x_{k},y). Here we apply the following version of formula (16.21).

E ([k!Ī_{n,k}^{V}(f,l_{u+1} ,...,l_{k},y)]^{2}|B) = ∫ [u!Ī_{n+u−k,u}^{l(u)} (f̄,ȳ)]^{2}ρ̄(dȳ) = H_{n+u−k,u}^{l(u)} (f̄)

with the function f̄∈ F̄ for which the identity

f̄(x_{1},...,x_{u},ȳ) = f(x_{1},...,x_{k},y)

holds with ȳ= (x_{u+1} ,...,x_{k},y), and we define the random variables Ī_{n+u−k,u}^{l(u)} (f̄,ȳ)

and H_{n+u−k,u}^{l(u)} (f̄) similarly to the corresponding terms after formula (16.21), only y is replaced by ȳ, the measure ρ by ρ̄, and the presently defined functions f̄∈ F̄ are considered. I omit the details.

Part b) of Lemma 16.3B will be proved with the help of Part a) and the inequality

sup_{f∈F}E(H_{n,k}^{(V_{1},V_{2})}(f)|B) ≤ ( sup_{f∈F}E(H_{n,k}^{V_{1}}(f)|B)) ^{1/2}( sup_{f∈F}E(H_{n,k}^{V_{2}}(f)|B)) ^{1/2}.

(16.28)

To prove inequality (16.28) observe that the random variables H_{n,k}^{(V_{1},V_{2})}(f), H_{n,k}^{V_{1}}(f)

and H_{n,k}^{V_{2}}(f) can be expressed as functions of the random variables ξ_{l}^{(j,1)}, ξ_{l}^{(j,−1)} , 1 ≤ j ≤ k, 1 ≤ l ≤ n which are independent of each other, and the random variables ξ_{l}^{(j,1)} are B measurable, while the random variables ξ_{l}^{(j,−1)} are independent of this σ-algebra. Hence we can calculate the conditional expectations E(H_{n,k}^{(V_{1},V_{2})}(f)|B), E(H_{n,k}^{V_{1}}(f)|B) and E(H_{n,k}^{V_{2}}(f)|B) by putting the value of the random variables ξ^{(j,1)}(ω) in the appropriate coordinate of the functions expressing these random variables and integrating by the remaining coordinates with respect the distribution of the random variables ξ_{l}^{(j,−1)} . By writing up the above conditional expectations in such a way and applying the Schwarz inequality for them we get the inequality

E(H_{n,k}^{(V_{1},V_{2})}(f)|B) ≤ ( E(H_{n,k}^{V_{1}}(f)|B)) ^{1/2}( E(H_{n,k}^{V_{2}}(f)|B)) ^{1/2} for all f ∈ F .

It is not difficult to deduce relation (16.28) from this inequality by showing that it remains valid if we put the sup_{f∈F} expressions in it in that way as it is done in (16.28).

In the proof of Part b) of Lemma 16.3B we may assume that V_{1} ≠ {1,...,k}. Inequality (16.28) implies that

P ( sup_{f∈F}E(H_{n,k}^{(V_{1},V_{2})}(f)|B) > \frac{2^{−(2k+2)}}{k!} A^{2}n^{2k}σ^{2k+2})

≤ P ( sup_{f∈F}E(H_{n,k}^{V_{1}}(f)|B) > \frac{2^{−(4k+4)}}{(k!)^{2}} A^{(2k−1)/k}n^{2k}σ^{2k+2})

+P ( sup_{f∈F}E(H_{n,k}^{V_{2}}(f)|B) > A^{(2k+1)/k}n^{2k}σ^{2k+2})

Hence if we know that also the inequality

P ( sup_{f∈F}E(H_{n,k}^{V_{2}}(f)|B) > A^{(2k+1)/k}n^{2k}σ^{2k+2}) ≤ n^{k−1}e^{−γ_{k}A^{1/2k}nσ^{2}/k} (16.29)

holds, then we can deduce relation (16.27) from the estimate (16.25) and (16.29). Relation (16.29) follows from Part a) of Lemma 16.3B if V_{2} ≠ {1,...,k} and A ≥ 1, since in this case the level A^{(2k+1)/k}n^{2k}σ^{2k+2} can be replaced by the smaller number 2^{−(4k+2)} A^{(2k−1)/k}n^{2k}σ^{2k+2} in the probability of formula (16.29). In the case V_{2} = {1,...,k} it follows from the conditions of Part b) of Lemma 16.3B if the number γ_{k} is chosen so that γ_{k}≤ 1. Indeed, since A^{(2k+1)/2k} > T^{(2k+1)/2k}, and by the conditions of Proposition 15.4 (and as a consequence of Lemma 16.3B) inequality (15.7) holds for all Ā≥ T^{(2k+1)/2k}, we can apply this relation for the parameter A^{(2k+1)/2k}. In such a way we get inequality (16.29) also for V_{2} = {1,...,k}.

Now we turn to the proof of Lemma 16.1B.

Proof of Lemma 16.1B. By Lemma 16.2B it is enough to prove that relation (16.8) holds if the random variables W̄(f) are replaced in it by the random variables W (f) defined in formula (16.16). We shall prove this by applying the generalized form of the symmetrization lemma, Lemma 15.2, with the choice of Z(f) = H_{n,k}^{(V̄,V̄)}(f),

V̄= {1,...,k}, Z̄(f) = Z(f) − W (f), f ∈ F, B = B(ξ_{1}^{(j,1)},...,ξ_{n}^{(j,1)}; 1 ≤ j ≤ k), α = \frac{A^{2}}{2}n^{2k}σ^{2k+2}, u = \frac{A^{2}}{2}n^{2k}σ^{2k+2} and the set

B = ⋂_{\begin{matrix}(V_{1},V_{2}): V_{j}∈\{1,...,k\}, j=1,2, \\ V_{1}≠\{1,...,k\} or V_{2}≠\{1,...,k\} \end{matrix}}

\{ ω: sup_{f∈F}E(H_{n,k}^{(V_{1},V_{2})}(f)|B)(ω) ≤ \frac{2^{−(2k+2)}}{k!} A^{2}n^{2k}σ^{2k+2}\} .

By part b) of Lemma 16.3B the inequality

1 − P(B) ≤ 2^{2k+1}n^{k−1}e^{−γ_{k}A^{1/2k}nσ^{2}/k}

holds. Observe that Z(f) = H_{n,k}^{(V̄,V̄)}(f) = H_{n,k}(f) for all f ∈ F. Hence to prove Lemma 16.1B with the help of Lemma 15.2 it is enough to show that

P ( |Z̄(f)| > \frac{A^{2}}{2k!}n^{2k}σ^{2k+2} | B) (ω) ≤ \frac{1}{2} for all f ∈ F if ω ∈ B.  (16.30)

To prove this relation observe that because of the definition of the set B

E(|Z̄(f)||B)(ω)

≤ ∑_{\begin{matrix}(V_{1},V_{2}): V_{j}∈\{1,...,k\}, j=1,2, \\ V_{1}≠\{1,...,k\} or V_{2}≠\{1,...,k\} \end{matrix}}E(H_{n,k}^{(V_{1},V_{2})}(f)|B)(ω) ≤ \frac{A^{2}}{4k!}n^{2k}σ^{2k+2}

if ω ∈ B for all f ∈ F. Hence the ‘conditional Markov inequality’ implies that P ( |Z̄(f)| > \frac{A^{2}}{2k!}n^{2k}σ^{2(k+1)} | B) (ω) ≤ \frac{2k!E(|Z̄(f)||B)(ω)}{A^{2}n^{2k}σ^{2k+2}} ≤ \frac{1}{2} if ω ∈ B, and inequality (16.30) holds. Lemma 16.1B is proved.

The proof of the main result

In this chapter Propositions 15.3 and 15.4 are proved with the help of Lemmas 16A1 and 16B1. They complete the proof of Theorem 8.4, of the main result in this work.

A.) The proof of Proposition 15.3.

The proof of Proposition 15.3 is similar to that of Proposition 7.3. It applies an induction procedure with respect to the order k of the U-statistics. In the proof of Proposition 15.3 for parameter k we may assume that Propositions 15.3 and 15.4 hold for u < k. We want to give a good estimate on the expression

P ( sup_{f∈F} |k!Ī_{n,k}^{ε}(f)| > 2^{−(k+1)} An^{k}σ^{k+1})

appearing at the right-hand side of the estimate (16.1) in Lemma 16.1A. To estimate this probability we introduce (using the notation of Proposition 15.3) the functions

S_{n,k}^{2}(f)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)
= ∑_{\begin{matrix}(l_{1},...,l_{k}): \\ 1≤l_{j}≤n, j=1,...,k, \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} f^{2}( x_{l_{1}}^{(1)},...,x_{l_{k}}^{(k)}), f ∈ F ,  (17.1)

with x_{l}^{(j)} ∈ X, 1 ≤ l ≤ n, 1 ≤ j ≤ k. We define with the help of this function the following set H = H(A) ⊂ X^{kn} for all A > T similarly to formula (7.8) in the proof of Proposition 7.3:

H = H(A) = { ( x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) :
sup_{f∈F}S_{n,k}^{2}(f)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) > 2^{k}A^{4/3}n^{k}σ^{2}} .  (17.2)

First we want to show that

P({ω: (ξ_{l}^{(j)}(ω), 1 ≤ j ≤ n, 1 ≤ j ≤ k) ∈ H}) ≤ 2^{k}e^{−A^{2/3k}nσ^{2}} if A ≥ T.  (17.3)

To prove relation (17.3) we take the Hoeffding decomposition of the U-statistics with kernel functions f^{2}(x_{1},...,x_{k}), f ∈ F, given in Theorem 9.1, i.e. we write

f^{2}(x_{1},...,x_{k}) = ∑_{V ⊂{1,...,k}}f_{V}(x_{j},j ∈ V), f ∈ F ,  (17.4)

with f_{V}(x_{j},j ∈ V) = ∏_{j∉V}P_{j} ∏_{j∈V} Q_{j}f^{2}(x_{1},...,x_{k}), where P_{j} and Q_{j} are the operators

defined in formulas (9.1) and (9.2).

The functions f_{V} appearing in formula (17.4) are canonical (with respect to the measure μ), and the identity S_{n,k}^{2}(f)(ξ_{l}^{(j)} 1 ≤ l ≤ n,1 ≤ j ≤ k) = k!Ī_{n,k}(f^{2}) holds for all f ∈ F with the expression Ī_{n,k}(·) defined in (14.11). By applying the Hoeffding decomposition (17.4) for each term f^{2}(ξ_{l_{1}}^{(1)}...,ξ_{l_{k}}^{(k)}) in the expression S_{n,k}^{2}(f) we get that

P ( sup_{f∈F}S_{n,k}^{2}(f)(ξ_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) > 2^{k}A^{4/3}n^{k}σ^{2})

≤ ∑_{V ⊂{1,...,k}}P ( sup_{f∈F}n^{k−|V |}||V |!Ī_{n,|V |}(f_{V})| > A^{4/3}n^{k}σ^{2})  (17.5)

with the functions f_{V} appearing in formula (17.4). We want to give a good estimate for each term in the sum at the right-hand side in (17.5). For this goal first we show that the classes of functions {f_{V}: f ∈ F } in the expansion (17.4) satisfy the conditions of Proposition 15.3 for all V ⊂ {1,...,k}.

The functions f_{V} are canonical for all V ⊂ {1,...,k}. It follows from the conditions of Proposition 15.3 that |f^{2}(x_{1},...,x_{k})| ≤ 2^{−2(k+1)} and

∫ f^{4}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) ≤ 2^{−(k+1)} σ^{2}.

Hence relations (9.5) and (9.6) of Theorem 9.2 imply that

| sup_{x_{j}∈X,j∈V} f_{V}(x_{j},j ∈ V)| ≤ 2^{−(k+2)} ≤ 2^{−(k+1)}

and ∫ f_{V}^{2}(x_{j},j ∈ V) ∏_{j∈V} μ(dx_{j}) ≤ 2^{−(k+1)} σ^{2} ≤ σ^{2} for all V ⊂ {1,...,k}. Finally, to

check that the class of functions F_{V} = {f_{V}: f ∈ F } is L_{2}-dense with exponent L and parameter D observe that for all probability measures ρ on (X^{k},X^{k}) and pairs of functions f,g ∈ F the inequality ∫(f^{2}−g^{2})^{2}dρ ≤ 2^{−2k} ∫(f −g)^{2}dρ holds. This implies that if {f_{1},...,f_{m}}, m ≤ Dε^{−L}, is an ε-dense subset of F in the space L_{2}(X^{k},X^{k},ρ), then the set of functions {2^{k}f_{1}^{2},...,2^{k}f_{m}^{2}} is an ε-dense subset of the class of functions F′= {2^{k}f^{2}: f ∈ F }, hence F′ is also an L_{2}-dense class of functions with exponent L and parameter D. Then by Theorem 9.2 the class of functions F_{V} is also L_{2}-dense with exponent L and parameter D for all sets V ⊂ {1,...,k}.

For V = ∅, the function f_{V} is constant, the relation

f_{V} = ∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) ≤ σ^{2}

holds, and Ī_{|V |}(f_{|V |})| = f_{V} ≤ σ^{2}. Therefore the term corresponding to V = ∅ in the sum of probabilities at the right-hand side of (17.5) equals zero under the conditions of Proposition 15.3 with the choice of some A_{0} ≥ 1. I claim that the remaining terms in the sum at the right-hand side of (17.5) satisfy the inequality

P ( n^{k−|V |} sup_{f∈F}||V |!Ī_{n,|V |}(f_{V})| > A^{4/3}n^{k}σ^{2})

≤ P ( sup_{f∈F}||V |!Ī_{n,|V |}(f_{V})| > A^{4/3}n^{|V |}σ^{|V |+1}) ≤ e^{−A^{2/3k}nσ^{2}}

if 1 ≤ |V | ≤ k.  (17.6)

The first inequality in (17.6) holds, since σ^{|V |+1} ≤ σ^{2} for |V | ≥ 1, and n ≥ k ≥ |V |. The second inequality follows from the inductive hypothesis if |V | < k, since in this case the middle expression in (17.6) can be bounded with the help of Proposition 15.3 by e^{−(A^{4/3})^{1/2|V |}nσ^{2}} ≤ e^{−A^{2/3k}nσ^{2}} if A_{0} = A_{0}(k) in Proposition 15.3 is chosen sufficiently large. In the case V = {1,...,k} it follows from the inequality A ≥ T and the inductive assumption of Proposition 15.3 by which the supremum of decoupled U-statistics determined by such a class of kernel-functions which satisfies the conditions of Proposition 15.3 has a good tail behaviour at level T^{4/3}. Relations (17.5) and (17.6) together with the estimate in the case V = ∅ imply formula (17.3).

By conditioning the probability P ( |k!Ī_{n,k}^{ε}(f)| > 2^{−(k+2)} An^{k/2}σ^{k+1}) with respect

to the random variables ξ_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k we get with the help of the multivariate version of Hoeffding’s inequality (Theorem 13.3) that

P ( |k!Ī_{n,k}^{ε}(f)| > 2^{−(k+2)} An^{k}σ^{k+1} | ξ_{l}^{(j)}(ω) = x_{l}^{(j)},1 ≤ l ≤ n,1 ≤ j ≤ k)

≤ C exp\{ −\frac{1}{2} ( \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{2k+4}S_{n,k}^{2}(f)(x_{l}^{(j)},1 ≤ l ≤ n, 1 ≤ j ≤ k)} ) ^{1/k}\}

≤ Ce^{−2^{−4−4/k} A^{2/3k}nσ^{2}} for all f ∈ F

if (x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) ∉ H  (17.7)

with some appropriate constant C = C(k) > 0.

Define for all 1 ≤ j ≤ k and sets of points x_{l}^{(j)} ∈ X, 1 ≤ l ≤ n, the probability measures ρ_{j} = ρ_{j,(x_{l}^{(j)},1≤l≤n)} , 1 ≤ j ≤ k on X, uniformly distributed on the set of

points \{x_{l}^{(j)}, 1 ≤ l ≤ n\}, i.e. let ρ_{j}(x_{l}^{(j)}) = \frac{1}{n} for all 1 ≤ l ≤ n. Let us also define the product ρ = ρ(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) = ρ_{1}× ··· × ρ_{k} of these measures on the space (X^{k},X^{k}). If f is a function on (X^{k},X^{k}) such that ∫ f^{2}dρ ≤ δ^{2} with some δ > 0, then

sup_{ε_{1},...,ε_{n}}|k!Ī_{n,k}^{ε}(f)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)|

≤ n^{k}∫ |f(u_{1},...,u_{k})|ρ(du_{1},..., du_{k}) ≤ n^{k}( ∫ f^{2}dρ) ^{1/2} ≤ n^{k}δ,

u_{j} ∈ R^{k}, 1 ≤ j ≤ k, and as a consequence

sup_{ε_{1},...,ε_{n}}|k!Ī_{n,k}^{ε}(f)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)  (17.8)
−k!Ī_{n,k}^{ε}(g)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)|
≤ 2^{−(k+2)} An^{k}σ^{k+1} if ∫ (f − g)^{2}dρ ≤ (2^{−(k+2)} Aσ^{k+1})^{2},

where Ī_{n,k}^{ε}(f)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) equals the expression Ī_{n,k}^{ε}(f) defined

in (14.12) if we replace ξ_{l_{j}}^{(j)} by x_{l_{j}}^{(j)} for all 1 ≤ j ≤ k, and 1 ≤ l_{j} ≤ n in it, and

ρ is the measure ρ = ρ(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) defined above.

Remark. Similarly to the remark made in the proof of Proposition 7.3 we may restrict our attention to the case when the random variables ξ_{l}^{(j)} are non-atomic. A similar statement holds also in the proof of Proposition 15.4,

Let us fix the number δ = 2^{−(k+2)} Aσ^{k+1}, and let us list the elements of the set F as F = {f_{1},f_{2},...}. Put

m = m(δ) = max(1,Dδ^{−L}) = max(1,D(2^{(k+2)}A^{−(1)} σ^{−(k+1)} )^{L}),

and choose for all vectors x^{(n)} = (x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) ∈ X^{kn}such a sequence of positive integers p_{1}(x^{(n)}),...,p_{m}(x^{(n)})) for which

inf_{1≤l≤m} ∫ (f(u) − f_{p_{l}(x^{(n)})}(u))^{2}ρ(x^{(n)})(du) ≤ δ^{2} for all f ∈ F and x^{(n)} ∈ X^{kn}.

(Here we apply the notation ρ(x^{(n)}) = ρ(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k), which is a probability measure on X^{k} depending on x^{(n)}.) This is possible, since F is an L_{2}- dense class with exponent L and parameter D, and we can choose m = Dδ^{−L}, if δ < 1, Beside this, we can choose m = 1 if δ ≥ 1, since ∫ |f − g|^{2}dρ ≤ sup|f(x) − g(x)|^{2} ≤ 2^{−2k} ≤ 1 for all f,g ∈ F. Moreover, we have shown in Lemma 7.4A that the functions p_{l}(x^{(n)}), 1 ≤ l ≤ m, can be chosen as measurable functions of the argument x^{(n)} ∈ X^{kn}.

Let us consider the random vector ξ^{(n)}(ω) = (ξ_{l}^{(j)}(ω), 1 ≤ l ≤ n, 1 ≤ j ≤ k). By arguing similarly as we did in the proof of Proposition 7.3 we get with the help of relation (17.8) and the property of the functions f_{p_{l}(x^{(n)})}(·) constructed above that

{ ω: sup_{f∈F}|k!Ī_{n,k}^{ε}(f)(ω)| ≥ 2^{−(k+1)} An^{k}σ^{k+1}}

⊂ ⋃_{l=1}^{m} {ω: |k!Ī_{n,k}^{ε}(f_{p_{l}(ξ^{(n)}(ω))})(ω)| ≥ 2^{−(k+2)} An^{k}σ^{(k+1)}} .

The above relation and formula (17.7) imply that

P ( sup_{f∈F} |k!Ī_{n,k}^{ε}(f)(ω)| > \frac{An^{k}σ^{k+1}}{2^{(k+1)}} | ξ_{l}^{(j)}(ω) = x_{l}^{(j)},1 ≤ l ≤ n,1 ≤ j ≤ k)

≤ ∑_{l=1}^{m} P ( |k!Ī_{n,k}^{ε}(f_{p_{l}(ξ^{(n)}(ω))}(ω)| > \frac{An^{k}σ^{k+1}}{2^{k+2}} |

ξ_{l}^{(j)}(ω) = x_{l}^{(j)},1 ≤ l ≤ n,1 ≤ j ≤ k)

≤ Cm(δ)e^{−2^{−4−4/k} A^{2/3k}nσ^{2}} ≤ C(1 + D(2^{k+2}A^{−1} σ^{−(k+1)} )^{L})e^{−2^{−4−4/k} A^{2/3k}nσ^{2}}

if {x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k} ∉ H.  (17.9)

Relations (17.3) and (17.9) imply that

P ( sup_{f∈F} |k!Ī_{n,k}^{ε}(f)| > 2^{−(k+1)} An^{k}σ^{k+1})  (17.10)

≤ C(1 + D(2^{k+2}A^{−1} σ^{−(k+1)} )^{L})e^{−2^{−4−4/k} A^{2/3k}nσ^{2}} + 2^{k}e^{−A^{2/3k}nσ^{2}} if A > T.

Proposition 15.3 follows from the estimates (16.1), (17.10) and the condition nσ^{2} ≥ Llogn + logD, L,D ≥ 1, if A ≥ A_{0} with a sufficiently large number A_{0}. Indeed, in this case nσ^{2} ≥ \frac{1}{2}, (2^{k+2}A^{−1} σ^{−(k+1)} )^{L} ≤ (\frac{n^{(k+1)/2}}{(2nσ^{2})^{(k+1)/2}} )^{L} ≤ n^{L(k+1)/2} =

e^{Llogn·(k+1)/2}≤ e^{(k+1)nσ^{2}/2} , D = e^{logD} ≤ e^{nσ^{2}}, and

C(1 + D(2^{k+2}A^{−1} σ^{−(k+1)} )^{L})e^{−2^{−4−4/k} A^{2/3k}nσ^{2}} ≤ \frac{1}{3}e^{−A^{1/2k}nσ^{2}} .

The estimation of the remaining terms in the upper bound of the estimates (16.1) and (17.10) leading to the proof of relation (15.5) is simpler. We can exploit that e^{−A^{2/3k}nσ^{2}} ≪ e^{−A^{1/2k}nσ^{2}} and as n^{k−1} ≤ e^{(k−1)nσ^{2}} , hence 2^{k}e^{−A^{2/3k}nσ^{2}} ≤ \frac{1}{3}e^{−A^{1/2k}nσ^{2}} , and 2^{k}n^{k−1}e^{−γ_{k}A^{1/(2k−1)} nσ^{2}/k} ≤ 2^{k}e^{(k−1)nσ^{2}} e^{−γ_{k}A^{1/(2k−1)} nσ^{2}/k} ≪ e^{−A^{1/2k}nσ^{2}} for a large number A.

Now we turn to the proof of Proposition 15.4.

B.) The proof of Proposition 15.4.

Because of formula (16.11) in the Corollary of Lemma 16.1B to prove Proposition 15.4 i.e. inequality (15.7) it is enough to choose a sufficiently large parameter A_{0} and to show that with such a choice the random variables H_{n,k}(f|G,V_{1},V_{2}) defined in formula (16.9) satisfy the inequality

P ( sup_{f∈F} |H_{n,k}(f|G,V_{1},V_{2})|> \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}} ) ≤ 2^{k+1}e^{−A^{1/2k}nσ^{2}}

for all G ∈ G and V_{1},V_{2} ∈ {1,...,k} if A > T ≥ A_{0} (17.11)

under the conditions of Proposition 15.4.

Let us first prove formula (17.11) in the case |e(G)| = k, i.e. when all vertices of the diagram G are end-points of some edge, and the expression H_{n,k}(f|G,V_{1},V_{2}) contains no ‘symmetrizing term’ ε_{j}. In this case we apply a special argument to prove relation (17.11).

We will show with the help of the Schwarz inequality that for a diagram G such that |e(G)| = k

|H_{n,k}(f|G,V_{1},V_{2})|  (17.12)

≤ ( ∑_{\begin{matrix}(l_{1},...,l_{k}): \\ 1≤l_{j}≤n, 1≤j≤k, \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l_{1}}^{(1),δ_{1}(V_{1}))},...,ξ_{l_{k}}^{(k,δ_{k}(V_{1}))},y)ρ(dy)) ^{1/2}

( ∑_{\begin{matrix}(l_{1},...,l_{k}): \\ 1≤l_{j}≤n, 1≤j≤k, \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{2}))},...,ξ_{l_{k}}^{(k,δ_{k}(V_{2}))},y)ρ(dy)) ^{1/2}

with δ_{j}(V_{1}) = 1 if j ∈ V_{1}, δ_{j}(V_{1}) = −1 if j ∉ V_{1}, and δ_{j}(V_{2}) = 1 if j ∈ V_{2}, δ_{j}(V_{2}) = −1 if j ∉ V_{2}.

Relation (17.12) can be proved for instance by bounding first the absolute value of each integral in formula (16.9) by means of the Schwarz inequality, and then by bounding the sum appearing in such a way by means of the inequality ∑ |a_{j}b_{j}| ≤

( ∑ a_{j}^{2})^{1/2}( ∑ b_{j}^{2})^{1/2}. Observe that in the case |(e(G)| = k the summation in (16.9) is taken for such vectors (l_{1},...,l_{k},l′_{1},...,l′_{k}) for which (l′_{1},...,l′_{k}) is a permutation of the sequence (l_{1},...,l_{k}) determined by the diagram G. Hence the sum we get after applying the Schwarz inequality for each integral in (16.9) has the form ∑ a_{j}b_{j} where the set of indices j in this sum agrees with the set of vectors (l_{1},...,l_{k}) such that 1 ≤ l_{p} ≤ n for all 1 ≤ p ≤ k, and l_{p} ≠ l_{p′} if p ≠ p′.

By formula (17.12)

\{ ω: sup_{f∈F} |H_{n,k}(f|G,V_{1},V_{2})(ω)|> \frac{A^{2}n^{2k}σ^{(2(k+1)}}{2^{4k+1}} \}

⊂ \{ ω: sup_{f∈F} ∑_{\begin{matrix}(l_{1},...,l_{k}): \\ 1≤l_{j}≤n, 1≤j≤k, \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))}(ω),...,ξ_{l_{k}}^{(k,δ_{k}(V_{1}))}(ω),y)ρ(dy)

> \frac{A^{2}n^{2k}σ^{2(k+1)}k!}{2^{4k+1}}\}

∪\{ ω: sup_{f∈F} ∑_{\begin{matrix}(l_{1},...,l_{k}): \\ 1≤l_{j}≤n, 1≤j≤k, \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{2}))}(ω),...,ξ_{l_{k}}^{(k,δ_{k}(V_{2}))}(ω),y)ρ(dy)

> \frac{A^{2}n^{2k}σ^{2(k+1)}k!}{2^{4k+1}}\} ,

hence

P ( sup_{f∈F} |H_{n,k}(f|G,V_{1},V_{2})|> \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}} )

≤ 2P ( sup_{f∈F} | ∑_{\begin{matrix}(l_{1},...,l_{k}): \\ 1≤l_{j}≤n, 1≤j≤k, \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} h_{f} (ξ_{l_{1}}^{(1,1)},...,ξ_{l_{k}}^{(k,1)})| > \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}} )

= 2P ( sup_{f∈F}|k!Ī_{n,k}(h_{f})| > \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}} ) ,  (17.13)

where Ī_{n,k}(h_{f}), f ∈ F, are the decoupled U-statistics defined in (14.11) with the kernel functions h_{f} (x_{1},...,x_{k}) = ∫ f^{2}(x_{1},...,x_{k},y)ρ(dy) and the random variables ξ_{l}^{(j,1)}, 1 ≤ j ≤ k, 1 ≤ l ≤ n. (In this upper bound we could get rid of the terms δ_{j}(V_{1}) and δ_{j}(V_{2}), i.e. of the dependence of the expression H_{n,k}(f|G,V_{1},V_{2}) on the sets V_{1} and V_{2}, since the probability of the events in the previous formula do not depend on them.)

I claim that

P ( sup_{f∈F}|k!Ī_{n,k}(h_{f})| ≥ 2^{k}An^{k}σ^{2}) ≤ 2^{k}e^{−A^{1/2k}nσ^{2}} for A ≥ A_{0}  (17.14)

if the constant A_{0} = A_{0}(k) is chosen sufficiently large in Proposition 15.4. Relation (17.14) together with the relation A^{2} \frac{n^{2k}σ^{2(k+1)}}{2^{4k+1}} ≥ 2^{k}An^{k}σ^{2} (if A > A_{0} with a sufficiently large A_{0}) imply that the probability at the right-hand side of (17.13) can be bounded by 2^{k+1}e^{−A^{1/2k}nσ^{2}} , and the estimate (17.11) holds in the case |e(G)| = k.

Relation (17.14) is similar to relation (17.3) (together with the definition of the random set H in formula (17.2)), and a modification of the proof of the latter estimate yields the proof also in this case. Indeed, it follows from the conditions of Proposition 15.4 that 0 ≤ ∫ h_{f} (x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) ≤ σ^{2} for all f ∈ F, and it is not difficult to check that sup|h_{f} (x_{1},...,x_{k})| ≤ 2^{−2(k+1)} , and the class of functions H = {2^{k}h_{f} , f ∈ F } is an L_{2}-dense class with exponent L and parameter D. Hence by applying the Hoeffding decomposition of the functions h_{f}, f ∈ F, similarly to formula (17.4) we get for all V ⊂ {1,...,k} such a set of functions {h_{f})_{V}, f ∈ F }, which satisfies the conditions of Proposition 15.3. Hence a natural adaptation of the estimate given for the expression at the right-hand side of (17.5) (with the help of (17.6) and the investigation of |V |!Ī_{|V |}(f_{V}) for V = ∅) yields the proof of formula (17.14). We only have to replace S_{n,k}(f) by k!Ī_{n,k}(h_{f}), then |V |!Ī_{n,|V |}(f_{V}) by |V |!Ī_{n,|V |}((h_{f})_{V}) and the levels 2^{k}A^{4/3}n^{k}σ^{2} in (17.3) and A^{4/3}n^{k}σ^{2} in (17.5) by 2^{k}An^{k}σ^{2} and An^{k}σ^{2} respectively. Let us observe that each term of the upper bound we get in such a way can be directly bounded, since during the proof of Proposition 15.4 for parameter k we may assume that the result of Proposition 15.3 holds also for this parameter k.

In the case of a diagram G ∈ G such that e(G) < k formula (17.11) will be proved with the help of the multivariate version of Hoeffding’s inequality, Theorem 13.3. In the proof of this case an expression, analogous to S_{n,k}^{2}(f) defined in formula (17.1) will be introduced and estimated for all sets V_{1},V_{2} ⊂ {1,...,k} and diagrams G ∈ G such that |e(G)| < k. To define it first some notations will be introduced.

Let us consider the set J_{0}(G) = J_{0}(G,k,n),

J_{0}(G) = {(l_{1},...,l_{k},l′_{1},...,l′_{k}): 1 ≤ l_{j},l′_{j}≤ n, 1 ≤ j ≤ k, l_{j} ≠ l_{j′} if j ≠ j′,
l′_{j} ≠ l′_{j′} if j ≠ j′, l_{j} = l′_{j′} if (j,j′) ∈ e(G), l_{j} ≠ l′_{j′} if (j,j′) ∉ e(G)}.

The set J_{0}(G) contains those sequences (l_{1},...,l_{k},l′_{1},...,l′_{k}) which appear as indices in the summation in formula (16.9) for a fixed diagram G. We also introduce an appropriate partition of it.

For this aim let us first define the sets

M_{1}(G) = {j(1),...,j(k − |e(G)|)} = {1,...,k} \ v_{1}(G),
j(1) < ···< j(k − |e(G)|),

and

M_{2}(G) = {j̄(1),...,j̄(k − |e(G|)} = {1,...,k} \ v_{2}(G),
j̄(1) < ···< j̄(k − |e(G|),

the sets of those vertices of the first and second row of the diagram G, indexed in increasing order, from which no edge starts. Let us also introduce the set V (G) = V (G,n,k), which consists of the restriction of the vectors (l_{1},...,l_{k},l′_{1},...,l′_{k}) ∈ J_{0}(G) to the coordinates indexed by the elements of the set M_{1}(G) ∪ M_{2}(G). Formally,

V (G) = {(l_{j(1)},...,l_{j(k−|e(G)|)} ,l′_{j̄(1)},...,l′_{j̄(k−|e(G)|)}): 1 ≤ l_{j(p)},l′_{j̄(p)} ≤ n,
1 ≤ p ≤ k − |e(G)|, l_{j(p)} ≠ l_{j(p′)}, l′_{j̄(p)} ≠ l′_{j̄(p′)}
if p ≠ p′, 1 ≤ p,p′≤ k − |e(G)|,
l_{j(p)} ≠ l′_{j̄(p′)}, 1 ≤ p,p′≤ k − |e(G)|}.

The elements of V (G) are vectors with elements indexed by the set M_{1}(G)∪M_{2}(G), which take different integer values between 1 and n.

We write all vectors v = (l_{j(1)},...,l_{j(k−|e(G)|)} ,l′_{j̄(1)},...,l′_{j̄(k−|e(G)|)}) ∈ V (G) in the

form v = (v^{(1)},v^{(2)}) with^{(1)} = (l_{j(1)},...,l_{j(k−|e(G)|)} ) and v^{(2)} = (l′_{j̄(1)},...,l′_{j̄(k−|e(G)|)}),

i.e. v^{(1)} contains the first k − |e(G)| coordinates of v with indices of the set M_{1}(G), and v^{(1)} contains the last k − |e(G)| coordinates of v with indices of the set M_{2}(G). We define with their help the set E_{G}(v) which consists of those vectors ℓ = (l_{1},...,l_{k},l′_{1},...,l′_{k}) ∈ J_{0}(G) whose restrictions to the coordinates with indices in M_{1}(G) and M_{2}(G) equal v^{(1)} and v^{(2)} respectively. More explicitly, we put

E_{G}(v) = {(l_{1},...,l_{k},l′_{1},...,l′_{k}): 1 ≤ l_{j} ≤ n, 1 ≤ l′ ; j̄≤ n, for 1 ≤ j,j̄≤ k,

l_{j} ≠ l_{j′} if j ≠ j′, l′ ; j̄≠ l′ ; j̄′ if j̄≠ j̄′,

l_{j} = l′ ; j̄if (j,j̄) ∈ e(G) and l_{j} ≠ l′ ; j̄if (j,j̄) ∉ e(G), and

l_{j(r)} = v(r), l′_{j̄(r)} = v̄(r), 1 ≤ r ≤ k − |e(G)|}, for all v ∈ V (G),

where {j(1),...,j(k − |e(G)|)} = M_{1}(G), {j̄(1),...,j̄(k − |e(G)|)} = M_{2}(G), v = (v^{(1)},v^{(2)}) with v^{(1)} = (v(1),...,v(k − |e(G)|)) and v^{(2)} = (v̄(1),...,v̄(k − |e(G)|)) in the last line of this definition. Beside this, let us define

E_{G}^{1}(v) = {(l_{1},...,l_{k}): (l_{1},...,l_{k},l′_{1},...,l′_{k}) ∈ E_{G}(v)}

and

E_{G}^{2}(v) = {(l′_{1},...,l′_{k}): (l_{1}...,l_{k},l′_{1},...,l′_{k}) ∈ E_{G}(v)}.

The vectors ℓ = (l_{1},...,l_{k},l′_{1},...,l′_{k}) ∈ E_{G}(v) can be characterized in the following way. For j ∈ M_{1}(G) their coordinates l_{j} agree with the corresponding elements of v^{(1)}, for j̄∈ M_{2}(G) their coordinates l′ ; j̄ agree with the corresponding elements of

v^{(2)}. The indices of the remaining coordinates of ℓ can be partitioned into pairs (j_{s},j̄_{s′}), 1 ≤ s,s′≤ |e(G)| in such a way that (j_{s},j̄_{s′}) ∈ e(G). The identity l_{j_{s}} = l′ ; j̄_{s′} holds if (j_{s},j̄_{s′}) ∈ e(G), and if (j_{s},j̄_{s′}) ∉ e(G), then the coordinates l_{j_{s}} and l′ ; j̄_{s′} are different. Otherwise, the coordinates l_{j_{s}} and l′ ; j̄_{s′} can be freely chosen from the set {1,...,n} \ {v^{(1)},v^{(2)}}. The sets E_{G}^{1}(v) and E_{G}^{2}(v) consist of the vectors containing the first k and the second k coordinates of the vectors ℓ ∈ E_{G}(v).

The sets E_{G}(v), v ∈ V (G), constitute a partition of the set J_{0}(G), and the random variables H_{n,k}(f|G,V_{1},V_{2}) defined in (16.9) can be rewritten with their help as

H_{n,k}(f|G,V_{1},V_{2})(ω) = ∑_{v=(v^{(1)},v^{(2)})∈V (G)} ∏_{s=1}^{k−|e(G)|} ε_{l_{j(s)}}(ω) ∏_{s=1}^{k−|e(G)|} ε_{l′_{j̄(s)}}(ω)

∑_{(l_{1},...,l_{k},l′_{1}...,l′_{k})∈E_{G}(v)}∫ f(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))}(ω),...,ξ_{l_{k}}^{(k,δ_{k}(V_{1}))}(ω),y)

f(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))}(ω),...,ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))}(ω),y)ρ(dy), (17.15)

where δ_{j}(V_{1}) = 1 if j ∈ V_{1}, δ_{j}(V_{1}) = −1 if j ∉ V_{1}, and δ_{j}(V_{2}) = 1 if j ∈ V_{2}, δ_{j}(V_{2}) = −1 if j ∉ V_{2}.

Let us fix some diagram G ∈ G and sets V_{1},V_{2} ⊂ {1,...,k}. We will prove the inequality

P ( S^{2}(F |G,V_{1},V_{2}) > 2^{2k}A^{8/3}n^{2k}σ^{4}) ≤ 2^{k+1}e^{−A^{2/3k}nσ^{2}} if A ≥ A_{0} and e(G) < k

(17.16)

for the random variable

S^{2}(F |G,V_{1},V_{2}) = sup_{f∈F} ∑_{v∈V (G)}( ∑_{(l_{1},...,l_{k},l′_{1},...,l′_{k})∈E_{G}(v)}∫ f(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))},...,ξ_{l_{k}}^{(k,δ_{k}(V_{1}))},y)

f(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))},...,ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))},y)ρ(dy)) ^{2},  (17.17)

where δ_{j}(V_{1}) = 1 if j ∈ V_{1}, δ_{j}(V_{1}) = −1 if j ∉ V_{1}, and δ_{j}(V_{2}) = 1 if j ∈ V_{2}, δ_{j}(V_{2}) = −1 if j ∉ V_{2}. The random variable S^{2}(F |G,V_{1},V_{2}) defined in (17.17) plays a similar role in the proof of Proposition 15.4 as the random variable sup_{f∈F}S_{n,k}^{2}(f) with S_{n,k}^{2}(f)

defined in formula (17.1) played in the proof of Proposition 15.3.

To prove formula (17.16) let us first fix some v ∈ V (G), and let us show that the following inequality, similar to relation (17.12) holds.

( ∑_{(l_{1},...,l_{k},l′_{1},...,l′_{k})∈E_{G}(v)}∫ f(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))},...,ξ_{l_{k}}^{(k,δ_{k}(V_{1}))},y)

f(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))},...,ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))},y)ρ(dy)) ^{2}

≤ ( ∑_{(l_{1},...,l_{k})∈E_{G}^{1}(v)}∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))},...,ξ_{l_{k}}^{(k,δ_{k}(V_{1}))},y)ρ(dy))

( ∑_{(l′_{1},...,l′_{k})∈E_{G}^{2}(v)}∫ f^{2}(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))},...,ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))},y)ρ(dy))  (17.18)

for all f ∈ F and v ∈ V (G). Indeed, observe that for a vector v̄= (v̄_{1},v̄_{2}) ∈ E_{G}(v) with v̄_{1} ∈ E_{G}^{1}(v) and v̄_{2} ∈ E_{G}^{2}(v), the coordinates of the vector v̄_{1} in the set M_{1}(G) and the coordinates of the vector v̄_{2} in the set M_{2}(G) are prescribed, while the coordinates of v̄_{1} in the set v_{1}(G) are given by a permutation of the coordinates v̄_{2} in the set v_{2}(G). (The sets v_{1}(G) and v_{2}(G) were defined before the introduction of formula (16.9) as the sets of those vertices in the first and second row of the diagram G respectively from which an edge of G starts.) This permutation is determined by the diagram G. Inequality (17.18) can be proved on the basis of the above observation similarly to formula (17.12).

We shall prove with the help of formula (17.18) the following inequality.

S^{2}(F |G,V_{1},V_{2})  (17.19)

≤ sup_{f∈F} ∑_{v∈V (G)}( ∑_{(l_{1},...,l_{k})∈E_{G}^{1}(v)}∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))},...,ξ_{l_{k}}^{(k,δ_{k}(V_{1}))},y)ρ(dy))

( ∑_{(l′_{1},...,l′_{k})∈E_{G}^{2}(v)}∫ f^{2}(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))},...,ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))},y)ρ(dy))

≤ sup_{f∈F} ( ∑_{\begin{matrix}(l_{1},...,l_{k}): \\ 1≤l_{j}≤n,1≤j≤k, \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))},...,ξ_{l_{k}}^{(k,δ_{k}(V_{1}))},y)ρ(dy))

sup_{f∈F} ( ∑_{\begin{matrix}(l′_{1},...,l′_{k}): \\ 1≤l′_{j}≤n,1≤j≤k, \\ l′_{j}≠l′_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))},...,ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))},y)ρ(dy)) .

The first inequality of (17.19) is a simple consequence of formula (17.18) and the definition of the random variable S^{2}(F |G,V_{1},V_{2}). To check its second inequality let us observe that it can be reduced to the simpler relation where the expression sup_{f∈F}

is omitted at each place. The simplified inequality obtained after the omission of the expressions supcan be checked by carrying out a term by term multiplication between the products of sums appearing in (17.19). At both sides of the inequality a sum consisting of terms of the form

∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1}(V_{1}))},...,ξ_{l_{k}}^{(k,δ_{k}(V_{1}))},y)ρ(dy)
∫ f^{2}(ξ_{l′_{1}}^{(1,δ_{1}(V_{2}))},...,ξ_{l′_{k}}^{(k,δ_{k}(V_{2}))},y)ρ(dy),  (17.20)

appears. It is enough to check that if a term of this form appears in the middle term of the simplified version formula of (17.19), then it appears with multiplicity 1, and it also appears at the right-hand side of this formula. To see this, observe that each term of the form (17.20) which appears in the sum we get by carrying out the multiplications in middle term of (17.19) determines uniquely the index v = (v^{(1)},v^{(2)}) ∈ V (G) in the outer sum of the middle term in the inequality (17.19). Indeed, if the random variables defining this expression of the form (17.20) have indices ℓ = (l_{1},...,l_{k},l′_{1},...,l′_{k}), then this vector ℓ uniquely determines the vector v = (v^{(1)},v^{(2)}) ∈ V (G), since v^{(1)} must agree with the restriction of the vector l = (l_{1},...,l_{k}) to the coordinates with indices in M_{1}(G) and v^{(2)} must agree with the restriction of the vector l′= (l′_{1},...,l′_{k}) to the coordinates with indices in M_{2}(G). Beside this, by carrying out the multiplication at the right-hand side of (17.19) we get such a sum which contains all such terms of the form (17.20) which appeared in the sum expressing the middle term in inequality (17.19). The above arguments imply inequality (17.19).

Relation (17.19) implies that

P(S^{2}(F |G,V_{1},V_{2})) > 2^{2k}A^{8/3}n^{2k}σ^{4}) ≤ 2P ( sup_{f∈F}k!Ī_{n,k}(h_{f}) > 2^{k}A^{4/3}n^{k}σ^{2}) ,

where Ī_{n,k}(h_{f}), f ∈ F, are the decoupled U-statistics defined in (14.11) with the kernel functions h_{f} (x_{1},...,x_{k}) = ∫ f^{2}(x_{1},...,x_{k},y)ρ(dy) and the random variables ξ_{l}^{(j,1)}, 1 ≤ j ≤ k, 1 ≤ l ≤ n. (Here we exploited that in the last formula S^{2}(F |G,V_{1},V_{2}) is bounded by the product of two random variables whose distributions do not depend on the sets V_{1} and V_{2}.) Thus to prove inequality (17.16) it is enough to show that

2P ( sup_{f∈F}k!Ī_{n,k}(h_{f}) > 2^{k}A^{4/3}n^{k}σ^{2}) ≤ 2^{k+1}e^{−A^{2/3k}nσ^{2}} if A ≥ A_{0}.  (17.21)

Actually formula (17.21) follows from the already proven formula (17.14), only the parameter A has to be replaced by A^{4/3} in it.

With the help of relation (17.16) the proof of Proposition 15.4 can be completed similarly to Proposition 15.3. The following version of inequality (17.7) can be proved with the help of the multivariate version of Hoeffding’s inequality (Theorem 13.3) and the representation of the random variable H_{n,k}(f|G,V_{1},V_{2}) in the form (17.15).

P( |H_{n,k}(f|G,V_{1},V_{2})| > \frac{A^{2}}{2^{4k+2}} n^{2k}σ^{2(k+1)} |ξ_{l}^{j,±1} , 1 ≤ l ≤ n, 1 ≤ j ≤ k) (ω)

≤ Ce^{−2^{−(6+2/k)} A^{2/3k}nσ^{2}}  (17.22)
if S^{2}(F |G,V_{1},V_{2})(ω) ≤ 2^{2k}A^{8/3}n^{2k}σ^{4} and A ≥ A_{0}

with an appropriate constant C = C(k) > 0 for all f ∈ F and G ∈ G such that |e(G)| < k and V_{1},V_{2} ⊂ {1,...,k}. (Observe that the conditional probability estimated in (17.22) can be represented in the following way. In a point ω ∈ Ω fix the values of ξ_{l}^{(j,±1)} (ω) for all indices 1 ≤ l ≤ n and 1 ≤ j ≤ k in the random variable H_{n,k}(f|G,V_{1},V_{k}), and the conditional probability in this point ω equals the probability that the random variable, (depending on the random variables ε_{l}, 1 ≤ l ≤ n), obtained in such a way is greater than \frac{A^{2}}{2^{4k+2} k!}n^{2k}σ^{2(k+1)}.)

Indeed, in this case the conditional probability considered in (17.22) can be bounded because of the multivariate version of Hoeffding’s inequality by

C exp\{ −\frac{1}{2}( \frac{A^{4}n^{4k}σ^{4(k+1)}}{2^{8k+4}S^{2}(F |G,V_{1},V_{2})}) ^{1/2j}\} ≤ C exp\{ −\frac{1}{2}( \frac{A^{4/3}n^{2k}σ^{4k}}{2^{10k+4}}) ^{1/2j}\}

with an appropriate C = C(k) > 0, where 2j = 2k −2|e(G)|, and 0 ≤ |e(G)| ≤ k −1. Since j ≤ k, nσ^{2} ≥ \frac{1}{2}, and also \frac{A^{4/3}}{2^{10k+4}} ≥ 2 if A_{0} is chosen sufficiently large we can write in the above upper bound for the left-hand side of (17.22) j = k, and in such a way we get inequality (17.22).

The next inequality, in which we estimate sup_{f∈F}H_{n,k}(f|G,V_{1},V_{2}), is a natural ver-

sion of formula (17.9) in the proof of Proposition 15.3. We shall show that

P( sup_{f∈F}|H_{n,k}(f|G,V_{1},V_{2})| > \frac{A^{2}}{2^{4k+1}} n^{2k}σ^{2(k+1)} |ξ_{l}^{(j,±1)} , 1 ≤ l ≤ n, 1 ≤ j ≤ k) (ω)

≤ C ( 1 + D( \frac{2^{4k+3}}{A^{2}σ^{2(k+1)}} ) ^{L}) e^{−2^{−(6+2/k)} A^{2/3k}nσ^{2}}

if S^{2}(F |G,V_{1},V_{2}))(ω) ≤ 2^{2k}A^{8/3}n^{2k}σ^{4} and A ≥ A_{0}  (17.23)

for all G ∈ G such that |e(G)| < k and V_{1},V_{2} ⊂ {1,...,k}.

To prove formula (17.23) let us fix two sets V_{1},V_{2} ⊂ {1,...,k} and a diagram G such that |e(G)| < k. We shall define for all vectors x^{(n)} = (x_{l}^{(j,1)},x_{l}^{(j,−1)} , 1 ≤ l ≤ n, 1 ≤ j ≤ k) ∈ X^{2kn} some probability measure α(x^{(n)}) on the space X^{k}× Y (with the space Y which appears in the formulation of Proposition 15.4) with which we can work so as we did with the probability measures ν(x^{(n)}) and ρ(x^{(n)}) in the proof of Propositions 7.3 and 15.3.

To do this we define first for a vector x^{(n)} = (x_{l}^{(j,1)},x_{l}^{(j,−1)} , 1 ≤ l ≤ n, 1 ≤ j ≤ k) ∈ X^{2kn} and for all 1 ≤ j ≤ k two probability measures ν_{j}^{(1)} = ν_{j}^{(1)}(x^{(n)},V_{1})

and ν_{j}^{(2)} = ν_{j}^{(2)}(x^{(n)},V_{2}) in the space (X,X ) in the following way. The mea-

sures ν_{j}^{(1)}(x^{(n)},V_{1}) and ν_{j}^{(2)}(x^{(n)},V_{2}) are uniformly distributed in the set of points

x_{l}^{(j,δ_{j}(V_{1}))}, 1 ≤ l ≤ n and x_{l}^{(j,δ_{j}(V_{2}))}, 1 ≤ l ≤ n, respectively. More explicitly, we define

for all 1 ≤ j ≤ k (and sets V_{1} and V_{2}) the probability measures ν_{j}^{(1)}( {x_{l}^{(j,δ_{j}(V_{1}))}}) =

\frac{1}{n} and ν_{j}^{(2)}( \{x_{l}^{(j,δ_{j}(V_{2}))}\}) = \frac{1}{n} for all 1 ≤ l ≤ n, where δ_{j}(V_{1}) = 1 if j ∈ V_{1}, δ_{j}(V_{1}) = −1 if j ∉ V_{1}, and similarly δ_{j}(V_{2}) = 1 if j ∈ V_{2} and δ_{j}(V_{2}) = −1 if j ∉ V_{2}. Let us consider the product measures α_{1} = α_{1}(x^{(n)},V_{1}) = ν_{1}^{(1)}× ··· × ν_{k}^{(1)}× ρ and α_{2} = α_{2}(x^{(n)},V_{2}) = ν_{1}^{(2)} × ··· × ν_{k}^{(2)} × ρ on the product space (X^{k}× Y,X^{k}× Y ), where ρ is that probability measure on (Y,Y ) which appears in Proposition 15.4. With the help of the measures α_{1} and α_{2} we define the measure α = α(x^{(n)}) = α(x^{(n)},V_{1},V_{2}) = \frac{α_{1}+α_{2}}{2} in the space (X^{k}× Y,X^{k}× Y ). Let us also define the measure α̃ = α̃(x^{(n)}) = α̃(x^{(n)},V_{1},V_{2}) = ν_{1}^{(1)}× ··· ν_{k}^{(1)}× ν_{1}^{(2)}× ··· ν_{k}^{(2)}× ρ in the space (X^{2k}× Y,X^{2k}× Y ).

Define H_{n,k}(f|G,V_{1},V_{2}) as a function in the product space (X^{2kn},X^{2kn}) (with arguments x_{l}^{(j,1)} and x_{l}^{(j,−1)} , 1 ≤ j ≤ k, 1 ≤ l ≤ n) by means of formula (17.15) by

replacing the random variables ξ_{l_{j}}^{(j,δ_{j}(V_{1}))}(ω) by x_{l_{j}}^{(j,δ_{j}(V_{1}))} and the random variables

ξ_{l′_{j}}^{(j,δ_{j}(V_{2}))}(ω) by x_{l′_{j}}^{(j,δ_{j}(V_{2}))} in it for all 1 ≤ j ≤ k and 1 ≤ l_{j},l′_{j}≤ n. (We consider the value of the coefficients ε_{l_{j(s)}} and ε_{l′_{j̄(s)}} in (17.5) fixed.) With such a notation we can

write for any pairs f,g ∈ F and x^{(n)} = (x_{l}^{j,1)},x_{l}^{(j,−1)} , 1 ≤ j ≤ k, 1 ≤ l ≤ n) ∈ X^{2kn}, by exploiting the properties of the above defined measure α̃ the inequality

sup_{ε_{1},...,ε_{n}}|H_{n,k}(f|G,V_{1},V_{2})(x^{(n)}) − H_{n,k}(g|G,V_{1},V_{2})(x^{(n)})|

≤ ∑_{v=(v^{(1)},v^{(2)})∈V (G)} ∑_{(l_{1},...,l_{k},l′_{1}...,l′_{k})∈E_{G}(v)}

∫ |f(x_{l_{1}}^{(1,δ_{1}(V_{1}))},...,x_{l_{k}}^{(k,δ_{k}(V_{1}))},y)f(x_{l′_{1}}^{(1,δ_{1}(V_{2}))},...,x_{l′_{k}}^{(k,δ_{k}(V_{2}))},y)

−g(x_{l_{1}}^{(1,δ_{1}(V_{1}))},...,x_{l_{k}}^{(k,δ_{k}(V_{1}))},y)g(x_{l′_{1}}^{(1,δ_{1}(V_{2}))},...,x_{l′_{k}}^{(k,δ_{k}(V_{2}))},y)|ρ(dy)

≤ n^{2k}∫ |f(x_{1},...,x_{k},y)f(x_{k+1},...,x_{2k},y)
−g(x_{1},...,x_{k},y)g(x_{k+1},...,x_{2k},y)|α̃(dx_{1},..., dx_{2k}, dy). (17.24)

Beside this, since both sup|f(x_{1},...,x_{k},y)| ≤ 1 and sup|g(x_{1},...,x_{k},y)| ≤ 1, we have

|f(x_{1},...,x_{k},y)f(x_{k+1},...,x_{2k},y) − g(x_{1},...,x_{k},y)g(x_{k+1},...,x_{2k},y)|
≤ |f(x_{1},...,x_{k},y)||f(x_{k+1},...,x_{2k},y) − g(x_{k+1},...,x_{2k},y)|
+|g(x_{k+1},...,x_{2k})||f(x_{1},...,x_{k},y) − g(x_{1},...,x_{k},y)|
≤ |f(x_{k+1},...,x_{2k},y) − g(x_{k+1},...,x_{2k},y)|
+|f(x_{1},...,x_{k},y) − g(x_{1},...,x_{k},y)|.

It follows from this inequality, formula (17.24) and the definition of the measures α̃, α_{1}, α_{2} and α that

sup_{ε_{1},...,ε_{n}}|H_{n,k}(f|G,V_{1},V_{2})(x^{(n)}) − H_{n,k}(g|G,V_{1},V_{2})(x^{(n)})|

≤ n^{2k}∫ (|f(x_{k+1},...,x_{2k},y) − g(x_{k+1},...,x_{2k},y)|

+|f(x_{1},...,x_{k},y) − g(x_{1},...,x_{k},y)|)α̃(dx_{1},..., dx_{2k}, dy)

= n^{2k}∫ |f(x_{1},...,x_{k},y) − g(x_{1},...,x_{k},y)|

(α_{1}(dx_{1},..., dx_{k}, dy) + α_{2}(dx_{1},..., dx_{k}, dy))  (17.25)

= 2n^{2k}∫ |f(x_{1},...,x_{k},y) − g(x_{1},...,x_{k},y)|α(dx_{1},..., dx_{k}, dy)

≤ 2n^{2k}( ∫ |f(x_{1},...,x_{k},y) − g(x_{1},...,x_{k},y)|^{2}α(dx_{1},..., dx_{k}, dy))^{1/2}

with the previously defined probability measure α = α(x^{(n)}). Put δ = \frac{A^{2}σ^{2(k+1)}}{2^{4k+3}} , list the elements of F as F = {f_{1},f_{2},...}, and choose such a set of indices p_{1}(x^{(n)}),...,p_{m}(x^{(n)}) taking positive integer values with m = max(1,Dδ^{−L}) elements for which

min_{1≤l≤m} ∫ (f(u) − f_{p_{l}(x^{(n)})}(u))^{2}α(x^{(n)})(du) ≤ δ^{2} for all f ∈ F and x^{(n)} ∈ X^{2kn}.

(Here integration is taken with respect to u ∈ X^{k}× Y.)

Such a choice of the indices p_{l}(x^{(n)}), 1 ≤ l ≤ m, is possible, since F is L_{2}-dense with exponent L and parameter D. Moreover, by Lemma 7.4B we may chose the functions p_{l}(x^{(n)}), 1 ≤ l ≤ m, as measurable functions of their argument x^{(n)} ∈ X^{2kn}.

Put ξ^{(n)}(ω) = (ξ_{l}^{(j,±1)} (ω), 1 ≤ l ≤ n, 1 ≤ j ≤ k). By arguing similarly as we did in the proof of Propositions 7.3 and (15.3) we get with the help of relation (17.25) and the property of the functions f_{p_{l}(x^{(n)})}(·) constructed above that

\{ ω: sup_{f∈F}|H_{n,k}(f|G,V_{1},V_{2})(ω)| ≥ \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{(4k+1)}} \}

⊂ ⋃_{l=1}^{m} \{ ω: |H_{n,k}(f_{p_{l}(ξ^{(n)}(ω)}|G,V_{1},V_{2})(ω)(ω)| ≥ \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{(4k+2)}} \} .

Hence

P( sup_{f∈F}|H_{n,k}(f|G,V_{1},V_{2})| > \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}} |ξ_{l}^{(j,±1)} , 1 ≤ l ≤ n, 1 ≤ j ≤ k) (ω)

≤ ∑_{l=1}^{m} P( |H_{n,k}(f_{p_{l}(ξ^{(n)}(ω))}|G,V_{1},V_{2})| > \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}} |

ξ_{l}^{(j,±1)} , 1 ≤ l ≤ n, 1 ≤ j ≤ k) (ω)

for almost all ω. The last inequality together with (17.22) and the inequality m =

max(1,Dδ^{−L}) ≤ 1 + D( \frac{2^{4k+3}}{A^{2}σ^{2(k+1)}} )^{L} imply relation (17.23).

It follows from relations (17.16) and (17.23) that

P ( sup_{f∈F}|H_{n,k}(f|G,V_{1},V_{2})| > \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}} ) ≤ 2^{k+1}e^{−A^{2/3k}nσ^{2}}

+C ( 1 + D( \frac{2^{4k+3}}{A^{2}σ^{2(k+1)}} ) ^{L}) e^{−2^{−(6+2/k)} A^{2/3k}nσ^{2}} if A ≥ A_{0}

for all V_{1},V_{2} ⊂ {1,...,k} and diagram G ∈ G such that |e(G)| ≤ k −1. This inequality implies that relation (17.11) holds also in the case |e(G)| ≤ k −1 if the constants A_{0} is chosen sufficiently large in Proposition 15.4, and this completes the proof of Proposition 15.4. To prove relation (17.11) in the case |e(G)| ≤ k − 1 with the help of the last inequality it is enough to show that D(\frac{2^{4k+3}}{A^{2}σ^{2(k+1)}} )^{L} ≤ e^{const.nσ^{2}} if A > A_{0} with a sufficiently large A_{0}, since this implies that the second term at the right-hand of our last estimation is not too large.

This relation follows from the inequality nσ^{2} ≥ Llogn + logD which implies that

( \frac{2^{4k+3}}{A^{2}σ^{2(k+1)}} ) ^{L} ≤ ( \frac{n^{(k+1)}}{(2nσ^{2})^{(k+1)}} ) ^{L} ≤ n^{(k+1)L} = e^{(k+1)Llogn} ≤ e^{(k+1)nσ^{2}}

if A_{0} is sufficiently large, and D = e^{logD} ≤ e^{nσ^{2}}.

An overview of the results and a discussion of the literature

I discuss briefly the problems investigated in this work, recall some basic results related to them, and also give some references. I also write about the background of these problems which may explain the motivation for their study. I list the remarks following the subsequent chapters in this work. Chapter 1 is an introductory text, the real work starts at Chapter 2.

Chapter 2

I met the main problem considered in this work when I tried to adapt the method of proof of the central limit theorem for maximum-likelihood estimates to some more difficult questions about so-called non-parametric maximum likelihood estimate problems. The Kaplan–Meyer estimate for the empirical distribution function with the help of censored data investigated in the second chapter is an example for such problems. It is not a maximum-likelihood estimate in the classical sense, but it can be considered as a non-parametric version of it. In the estimation of the distribution function with the help of censored data we cannot apply the classical maximum likelihood method, since in this problem we have to choose our estimate from a too large class of distribution functions. The main problem is that there is no dominating measure with respect to which all candidates which may appear as our estimate have a density function. A natural way to overcome this difficulty is to choose an appropriate smaller class of distribution functions, to compare the probability of the appearance of the sample we observed with respect to all distribution functions of this class and to choose that distribution function as our estimate for which this probability takes its maximum.

The Kaplan–Meyer estimate can be found on the basis of the above principle in the following way: Let us estimate the distribution function F (x) of the censored data simultaneously together with the distribution function G(x) of the censoring data. (We have a sample of size n and know which sample elements are censored and which are censoring data.) Let us consider the class of such pairs of estimates (F_{n}(x),G_{n}(x)) of the pair (F (x),G(x)) for which the distribution function F_{n}(x) is concentrated in the censored sample points and the distribution function G_{n}(x) is concentrated in the censoring sample points; more precisely, let us also assume that if the largest sample point is a censored point, then the distribution function G_{n}(x) of the censoring data takes still another value which is larger than any sample point, and if it is a censoring point then the distribution function F_{n}(x) of the censored data takes still another value larger than any sample point. (This modification at the end of the definition is needed, since if the largest sample point is from the class of censored data, then the distribution G(x) of the censoring data in this point must be strictly less than 1, and if it is from the class of censoring data, then the value of the distribution function F (x) of the censored data must be strictly less than 1 in this point.) Let us take this class of pairs of distribution functions (F_{n}(x),G_{n}(x)), and let us choose that pair of distribution functions of this class as the (non-parametric maximum likelihood) estimate with respect to which our observation has the greatest probability.

The above extremum problem about a pair of distribution functions (F_{n}(x),G_{n}(x)) can be solved explicitly, (see [28]), and it yields the estimate of F_{n}(x) written down in formula (2.3). (The function G_{n}(x) satisfies a similar relation, only the random variables X_{j} and Y_{j} and the events δ_{j} = 1 and δ_{j} = 0 have to be replaced in it.) If we want to prove that the estimate of the distribution function we found in such a way satisfies the central limit theorem, then we can do this with the help of a good adaptation of the method applied in the study of maximum likelihood estimates. We apply an appropriate linearization procedure, and there is only one really hard part of the proof. We have to show that this linearization procedure gives a small error. This problem led to the study of a good estimate on the tail distribution of the integral of an appropriate function of two variables with respect to the product of a normalized empirical measure with itself. Moreover, as a more detailed investigation showed, we actually need the solution of a more general problem where we have to bound the tail distribution of the supremum of a class of such integrals. The main subject of this work is to solve the above problems in a more general setting, to estimate not only two-fold, but also k-fold random integrals and the supremum of such integrals for an appropriate class of kernel functions with respect to a normalized empirical distribution for all k ≥ 1.

The proof of the limit theorem for the Kaplan–Meyer estimate explained in this work applied the explicit form of this estimate. It would be interesting to find such a modification of this proof which only exploits that the Kaplan–Meyer estimate is the solution of an appropriate extremum problem. We may expect that such a proof can be generalized to a general result about the limit behaviour for a wide class of non-parametric maximum likelihood estimates. Such a consideration was behind the remark of Richard Gill I quoted at the end of Chapter 2.

A detailed proof together with a sharp estimate on the speed of convergence for the limit behaviour of the Kaplan–Meyer estimate based on the ideas presented in Chapter 2 is given in paper [40]. Paper [41] explains more about its background, and it also discusses the solution of some other non-parametric maximum likelihood problems. The results about multiple integrals with respect to a normalized empirical distribution function needed in these works were proved in [33]. These results were satisfactory for the study in [40], but they also have some drawbacks. They do not show that if the random integrals we are considering have small variances, then they satisfy better estimates. Beside this, if we consider the supremum of random integrals of an appropriate class of functions, then these results can be applied only in very special cases. Moreover, the method of proof of [33] did not allow a real generalization of these results. Hence I had to find a different approach when I tried to generalize them.

I do not know of other works where the distribution of multiple random integrals with respect to a normalized empirical distribution is studied. On the other hand, there are some works where a similar problem is investigated about the distribution of (degenerate) U-statistics. The most important results obtained in this field are contained in the book of de la Peña and Giné Decoupling, From Dependence to Independence [9]. The problems about the behaviour of degenerate U-statistics and multiple integrals with respect to a normalized empirical distribution function are closely related, but the explanation of their relation is far from trivial. The main difference between them is that integration with respect to μ_{n}−μ instead of the empirical distribution μ_{n} means of some sort of normalization, while this normalization is missing in the definition of U-statistics. I return to this question later.

Let me finish my discussion about Chapter 2 with some personal remarks. Here I investigated a special problem. But in my opinion the method applied in this chapter works well in several similar problems about the limit behaviour of a non-linear functional of independent identically distributed random variables. In the study of such problems we express the non-linear functional we are investigating as an integral with respect to the normalized empirical distribution determined by the random variables we are working with plus some negligibly small error terms. Then we have to describe the limit behaviour of the random integral we got, and this can be done with the help of some classical results of probability theory. Beside this we have to show that the remaining error terms are really small. This can be done, but at this point the results discussed in this work play a crucial role. I believe that a similar picture arises in many cases. In certain problems it may happen that the main term is not a one-fold, but a multiple integral with respect to the normalized empirical distribution. But the limit distribution of such functionals can also be described. This is the content of Theorem 10.4′proved in Appendix C.

Chapter 3

The main part of this work starts at Chapter 3. A general overview of the results without the hard technical details can be found in [36].

First the estimation of sums of independent random variables or of one-fold random integrals with respect to a normalized empirical distribution and the supremum of such expressions is investigated in Chapters 3 and 4. This question has a fairly big literature. I would mention first of all the books A course on empirical processes [13], Real Analysis and Probability [14] and Uniform Central Limit Theorems [15] of R. M. Dudley. These books contain a much more detailed description of the empirical processes than the present work together with a lot of interesting results.

In Chapter 3 I presented the proof of some classical results about the tail behaviour of sums of independent and bounded random variables with expectation zero. They are Bernstein’s and Bennett’s inequalities. Their proofs can be found at many places, e.g. in Theorem 1.3.2 of [15] and [6].) We are also interested in the question when these results give such an estimate that the central limit theorem suggests. Actually, as it is explained in Chapter 3, Bennett’s inequality gives such a bound that the Poissonian approximation of partial sums of independent random variables suggests. Bernstein’s inequality provides an estimate suggested by the central limit theorem if the variance of the sum we consider is not too small. The results in Chapter 3 explain these statements more explicitly. If the variance of the sum is too small, then Bennett’s inequality provides a slight improvement of Bernstein’s inequality. Moreover, as Example 3.3 shows, Bennett’s inequality is essentially sharp in this case. But these results are much weaker than the estimates suggested by a normal comparison.

The relative weakness of Bernstein’s and Bennett’s inequality for random sums with small variance had a deep consequence in our investigation about the supremum (of appropriate classes) of sums of independent random variables. Because of the weakness of these estimates in certain cases we had to find a new method. We could overcome the difficulty we met with the help of a symmetrization argument which is explained in Chapter 7. But to apply this method we needed another result, known under the name Hoeffding’s inequality. It yields an estimate about the tail behaviour of linear combinations of independent Rademacher functions. This result always provides such a good bound as the central limit theorem suggests. This is the reason why I discuss this inequality at the end of Chapter 3, in Theorem 3.4. It is also a classical result whose proof can be found for instance in [25].

The content of Chapter 3 can be found in the literature, e.g. in [13]. The main difference between my discussion and that of earlier works is that I put more emphasis on the investigation of the question when the estimates on the tail distribution of partial sums of independent random variables are similar to their Gaussian counterpart. I had a good reason to discuss this question in more detail. I was also interested in the estimation of the tail distribution of the supremum of partial sums of independent random variables, and in the study of this problem we have to understand when the classical methods related to Gaussian random variables can be applied and when we have to look for a new approach.

Chapter 4

Chapter 4 contains the one-variate version of our main result about the supremum of the integrals of a class F of functions with respect to a normalized empirical measure together with an equivalent statement about the tail distribution of the supremum of a class of random sums defined with the help of a sequence of independent and identically distributed random variables and a class of functions F with some nice properties. These results are formulated in Theorems 4.1 and 4.1′. They appeared in [33]. Also a Gaussian version of them is presented in Theorem 4.2 about the distribution of the supremum of a Gaussian random field with some appropriate properties. A deeper version of Theorem 4.2 is studied in paper [12]. The content of these results can be so interpreted that if we take the supremum of random integrals or of random sums determined by a nice class of functions F in the way described in Chapter 4, then the tail distribution of this supremum satisfies an almost as good estimate as the ‘worst element’ of the random variables taking part in this supremum. But such a result holds only if we consider the value of this tail distribution at a sufficiently large level, since — as some concentration inequalities imply — the supremum of these random sums are larger than the expected value of this supremum with probability almost one. I also discussed a result in Example 4.3 which shows that some rather technical conditions of Theorem 4.1 cannot be omitted.

The most important condition in Theorem 4.1 was that the class of functions F we considered in it is L_{2}-dense. This property was introduced before the formulation of Theorem 4.1. One may ask whether one can prove a better version of this result, which states a similar bound for a different, possibly larger class of functions F. It is worth mentioning that Talagrand proved results similar to Theorem 4.1 for different classes of functions F in his book [58]. These classes of functions are very different of ours, and Talagrand’s results seem to be incomparable with ours. I return to this question later in the discussion of Chapters 6 and 7, which deal with the proof of the results of Chapter 4. In the remaining part of the discussion of Chapter 4 I write about the notion of countably approximable classes of random variables and its role in the present work.

In the first formulation of our results we have imposed the condition that the class of functions F is countable, i.e. we take the supremum of countably many random variables. In the proofs this condition was heavily exploited. On the other hand, in some important applications we also need results about the supremum of a possibly non-countable set of random variables. To handle such cases I introduced the notion of countably approximable classes of random variables and proved that in the results of this work the condition about countability can be replaced by the weaker condition that the supremum of countably approximable classes is taken. R. M. Dudley worked out a different method to handle the supremum of possibly non-countably many random variables, and generally his method is applied in the literature. The relation between these two methods deserves some discussion.

To understand the problem we are discussing let us first recall that if we take a class of random variables S_{t}, t ∈ T, indexed by some index set T, then for all sets A measurable with respect to the σ-algebra generated by the random variables S_{t}, t ∈ T, there exists a countable subset T′= T′(A) ⊂ T such that the set A is measurable also with respect to the smaller σ-algebra generated by the random variable S_{t}, t ∈ T′. Beside this, if the finite dimensional distributions of the random variables S_{t}, t ∈ T, are given, then by the results of classical measure theory the probability of all events measurable with respect to the σ-algebra generated by these random variables S_{t}, t ∈ T, is also determined. But it may happen that we want to deal with such events whose probability cannot be defined in such a way. In particular, if T is

a non-countable set, then the events { ω: sup_{t∈T}S_{t}(ω) > u} are non-measurable with

respect to the above σ-algebra, and generally we cannot speak of their probabilities. To overcome this difficulty Dudley worked out a theory which enabled him to work also with outer measures. His theory is based on some rather deep results of the analysis. It can be found for instance in his book [15].

I restricted my attention to such cases when after the completion of the probability measure P we can also speak of the real (and not only outer) probabilities

P ( sup_{t∈T}S_{t} > u) . I tried to find appropriate conditions under which these probabili-

ties really exist. More explicitly, I was interested in the case when for all u > 0 there exists some set A = A_{u} measurable with respect to the σ-algebra generated by the random variables S_{t}, t ∈ T, such that the symmetric difference of the sets A_{u} and

{ ω: sup_{t∈T}S_{t}(ω) > u} is contained in a set which is measurable with respect to the

σ-algebra generated by the random variables S_{t}, t ∈ T, and it has probability zero. In

such a case the probability P ( sup_{t∈T}S_{t} > u) can be defined as P(A_{u}). This approach

led me to the definition of countable approximable classes of random variables. If this property holds, then we can speak about the probability of the event that the supremum of the random variables we are interested in is larger than some fixed value. I proved a simple but useful result in Lemma 4.4 which provides a condition for the validity of this property. In Lemma 4.5 I proved with its help that an important class of functions is countably approximable. It seems that this property can be proved for many other interesting classes of functions with the help of Lemma 4.4, but I did not investigate this question in more detail.

The problem we met here is not an abstract, technical difficulty. Indeed, the distribution of the supremum of uncountably many random variables can become different if we modify each random variable on a set of probability zero, although their finite dimensional distributions remain the same after such an operation. Hence, if we are interested in the probability of the supremum of a non-countable set of random variables with prescribed finite dimensional distributions we have to tell more explicitly which version of this set of random variables we consider. It is natural to look for such an appropriate version of the random field S_{t}, t ∈ T, whose ‘trajectories’ S_{t}(ω), t ∈ T, have nice properties for all elementary events ω ∈ Ω. Lemma 4.4 can be interpreted as a result in this spirit. The condition given for the countable approximability of a class of random variables at the end of this lemma can be considered as a smoothness type condition about the ‘trajectories’ of the random field we consider. This approach shows some analogy to some important problems in the theory of stochastic processes when a regular version of a stochastic process is considered, and the smoothness properties are investigated for the trajectories of this version.

In our problems the version of the set of random variables S_{t}, t ∈ T, we work with appears in a simple and natural way. In these problems we have finitely many random variables ξ_{1},...,ξ_{n} at the start, and all random variables S_{t}(ω), t ∈ T, we are considering can be defined individually for each ω as a function of these random variables ξ_{1}(ω),...,ξ_{n}(ω). We take the version of the random field S_{t}(ω), t ∈ T, we get in such a way and want to show that it is countably approximable. In Chapter 4 this property is proved in an important model, probably in the most important model in possible applications we are interested in. In more complicated situations when our random variables are defined not as a function of finitely many sample points, for instance in the case when we define our set of random variables by means of integrals with respect to a Gaussian random field it is harder to find the right regular version of our sets of random variables. In this case the integrals we consider are defined only with probability 1, and it demands some extra work to find their right version. But in the problems studied in this work the above sketched approach is satisfactory for our purposes, and it is simpler than that of Dudley; we do not have to follow his rather difficult technique. On the other hand, I must admit that I do not know the precise relation between the approach of this work and that of Dudley.

Chapter 5

In Chapter 4 the notion of L_{p}-dense classes, 1 ≤ p < ∞, also has been introduced. The notion of L_{2}-dense classes appeared in the formulation Theorems 4.1 and 4.1′. It can be considered as a version of the ε-entropy, discussed at many places in the literature. (See e.g. [13] or [14].) On the other hand, there seems to be no standard definition of the ε-entropy. The term of L_{2}-dense classes seemed to be the appropriate object to work with in this lecture note. To apply the results related to L_{2}-dense classes we also need some knowledge about how to check this property in concrete models. For this goal I discussed here Vapnik–Červonenkis classes, a popular and important notion of modern probability theory. Several books and papers, (see e.g. the books [15], [49], [59] and the references in them) deal with this subject. An important result in this field is Sauer’s lemma, (Lemma 5.1) which together with some other results, like Lemma 5.3 imply that several interesting classes of sets or functions are Vapnik–Červonenkis classes.

I put the proof of these results to the Appendix, partly because they can be found in the literature, partly because in this work Vapnik–Červonenkis classes play a different and less important role than at other places. Here Vapnik–Červonenkis classes are applied to show that certain classes of functions are L_{2}-dense. At this point a result of Dudley formulated in Lemma 5.2 plays an important role. It implies that a Vapnik–Červonenkis class of functions with absolute value bounded by a fixed constant is an L_{1}, and as a consequence also an L_{2}-dense class of functions. The proof of this important result which seems to be less known even among experts of this subject than it would deserve is contained in the main text. Dudley’s original result was formulated in the special case when the functions we consider are indicator functions of some sets. But its proof contains all important ideas needed in the proof of Lemma 5.2. A proof of the result in the form formulated in this work can be found in [49]. This book also contains the other results of this chapter about Vapnik–Červonenkis classes.

Chapters 6 and 7

Theorem 4.2, which is the Gaussian counterpart of Theorems 4.1 and 4.1′is proved in Chapter 6 by means of a natural and important technique, called the chaining argument. This means the application of an inductive procedure, in which an appropriate sequence of finite subsets of the original set of random variables is introduced, and a good estimate is given on the supremum of the random variables in these subsets by means of an inductive procedure. The subsets became denser subsets of the original set of the random variables at each step of this procedure. This chaining argument is a popular method in certain investigations. It is hard to say with whom to attach it. Its introduction may be connected to some works of R. M. Dudley. It is worth mentioning that Talagrand [58] worked out a sharpened version of it which yields in the study of certain problems a sharper and more useful estimate. But it seems to me that in the study of the problems of this work this improvement has a limited importance, it turns out to be useful in the study of different problems.

Theorem 4.2 can be proved by means of the chaining argument, but this method is not strong enough to supply a proof of Theorem 4.1. It provides only a weak estimate in this case, because there is no good estimate on the probability that a sum of independent random variables is greater than a prescribed value if these random variables have too small variances. As a consequence, the chaining argument supplies a much weaker estimate than the result we want to prove under the conditions of Theorem 4.1. Lemma 6.1 contains the result the chaining argument yields under these conditions. In Chapter 6 still another result, Lemma 6.2 is formulated. It can be considered as a special case of Theorem 4.1 where only the supremum of partial sums with small variances is estimated. We also show in this chapter that Lemmas 6.1 and 6.2 together imply Theorem 4.1. The proof is not difficult, despite of some non-attractive details. It has to be checked that the parameters in Lemmas 6.1 and 6.2 can be fitted to each other.

Lemma 6.2 is proved in Chapter 7. It is based on a symmetrization argument. This proof applies the ideas of a paper of Kenneth Alexander [3], and although its presentation is different from Alexander’s approach, it can be considered as a version of his proof. It may be worth mentioning that the symmetrization arguments were first applied in the theory of Vapnik–Červonenkis classes to get some useful estimates (see e.g. [49]). But it turned out that an appropriate refinement of this method supplies sharper results if we are working with L_{2}-dense classes instead of Vapnik–Červonenkis classes of functions.

A similar problem should also be mentioned at this place. M. Talagrand wrote a series of papers about concentration inequalities, (see e.g. [55] or [56]), and his research was also continued by some other authors. I would mention the works of M. Ledoux [30] and P. Massart [43]. Concentration inequalities give a bound about the difference between the supremum of a set of appropriately defined random variables and the expected value of this supremum. They express how strongly this supremum is concentrated around its expected value. Such results are closely related to Theorem 4.1, and the discussion of their relation deserves some attention. A typical concentration inequality is the following result of Talagrand [56].

Theorem 18.1 (Theorem of Talagrand). Consider n independent and identically distributed random variables ξ_{1},...,ξ_{n} with values in some measurable space (X,X ). Let F be some countable family of real-valued measurable functions

of (X,X ) such that ∥f∥_{∞} ≤ b < ∞ for every f ∈ F. Let Z = sup_{f∈F} ∑_{i=1}^{n} f(ξ_{i}) and

v = E ( sup_{f∈F} ∑_{i=1}^{n} f^{2}(ξ_{i})) . Then for every positive number x

P(Z ≥ EZ + x) ≤ K exp\{ −\frac{1}{K′}\frac{x}{b}log( 1 + \frac{xb}{v}) \}

and

P(Z ≥ EZ + x) ≤ K exp\{ −\frac{x^{2}}{2(c_{1}v + c_{2}bx)}\} ,

where K, K′, c_{1} and c_{2} are universal positive constants. Moreover, the same inequalities hold when replacing Z by −Z.

Theorem 18.1 yields, similarly to Theorem 4.1, an estimate about the distribution of the supremum for a class of sums of independent random variables. (The paper of P. Massart [43] contains a similar estimate which is better for our purposes. The main difference between these two estimates is that the bound given by Massart de-

pends on σ^{2} = sup_{f∈F} ∑_{i=1}^{n} Varf(ξ_{i}) instead of v = E ( sup_{f∈F} ∑_{i=1}^{n} f^{2}(ξ_{i})) .) Theorem 18.1

can be considered as a generalization of Bernstein’s and Bennett’s inequalities when the distribution of the supremum of partial sums (and not only the distribution of one partial sum) is estimated. A remarkable feature of this result is that it assumes no condition about the structure of the class of functions F (like the condition of L_{2}- dense property of the class F imposed in Theorem 4.1). On the other hand, the

estimates in Theorem 18.1 contain the quantity EZ = E ( sup_{f∈F} ∑_{i=1}^{n} f(ξ_{i})) . Such an

expectation of some supremum appears in all concentration inequalities. As a consequence, they are useful only if we can bound the expected value of the supremum we want to estimate. It is difficult to find a good bound on this expected value in the general case. Paper [18] provides a useful estimate on it if the expected value of the supremum of random sums is considered under the conditions of Theorem 4.1. But I preferred a direct proof of this result. Let me remark that because of the above mentioned concentration inequality the condition u ≥ const.σ log^{1/2} \frac{2}{σ} with some appropriate constant which cannot be dropped from Theorem 4.1 can be interpreted so that under the conditions of Theorem 4.1 const.σ log^{1/2} \frac{2}{σ} is an upper bound for the expected value of the supremum we investigated in this result. Example 4.3 implies that if the conditions of Theorem 4.1 are violated, then the expected value of the above supremum may be larger.

It is also worth mentioning Talagrand’s work [58] which contains several interesting results similar to Theorem 4.1. But despite their formal similarity, they are essentially different from the results of this work. This difference deserves a special discussion.

Talagrand proved in [58] by working out a more refined, better version of the chaining argument a sharp upper bound for the expected value E sup_{t∈T}ξ_{t} of the supre-

mum of countably many (jointly) Gaussian random variable with zero expectation. This result is sharp. Indeed, Talagrand proved also a lower bound for this expected value, and the quotient of his upper and lower bound is bounded by a universal constant. By applying similar arguments he also gave an upper bound for E sup_{f∈F} ∑_{k=1}^{N} f(ξ_{k}) in Proposition 2.7.2 of his book if ξ_{1},...,ξ_{N} is a sequence of inde-

pendent, identically distributed random variables with some known distribution μ, and F is a class of functions with some nice properties. Then he proved in Chapter 3 of this book some estimates with the help of this result for certain models which solved some problems that could not be solved with the help of the original version of the chaining argument.

Let me make a short comparison between our Theorem 4.1 and Talagrand’s result. Talagrand investigated in his book [58] the expected value of the supremum of partial sums, while we gave an estimate on its tail distribution. But this is not an essential difference. Talagrand’s results also give an estimate on the tail distribution of the supremum by means of concentration inequalities, and actually his proofs also provide a direct estimate for the tail distribution we are interested in without the application of these results. The main difference between the two works is that Talagrand’s method gives a sharp estimate for different classes of functions F.

Talagrand could prove sharp results in such cases when the class of functions F for which the supremum is taken consists of smooth functions. An example for such classes of functions which he thoroughly investigated is the class of Lipschitz 1 functions. In particular, in Chapter 3 of his book [58] he proved that if ξ_{1},...,ξ_{n} is a sequence of independent random variables, uniformly distributed in the unit square D = [0,1] × [0,1], and F is the class of Lipschitz 1 functions on the unit square D such that ∫_{D}f dλ = 0 for all f ∈ F, where λ denotes the Lebesgue measure on D,

then E sup_{f∈F} ∑_{l=1}^{n} f(ξ_{l}) ≤ L\sqrt{nlogn}with a universal constant L. He was interested in

this result, because it is equivalent to a theorem of Ajtai–Komlós–Tusnády [2]. (See Chapter 3 of [58] for details.) On the other hand, we can give sharp results in such cases when F consists of non-smooth functions, (see Example 5.5), and Talagrand’s method does not work in the study of such problems.

This difference in the conditions of the results in these two books is not a small technical detail. Talagrand heavily exploited in his proof that he worked with such classes of functions F from which he could select a subclass of functions of F of relatively small cardinality which is dense in F not only in the L_{2}(μ)-norm with the probability measure μ he was working with, but also in the supremum norm. He needed this property, because this enabled him to get sharp estimates on the tail distribution of the differences of functions he had to work with by means of Bernstein’s inequality. The smallness of the supremum norm of these random variables was useful, since it implied that Bernstein’s inequality provides a sharp estimate in a large domain. Talagrand needed such sharp estimates to apply (a refined version of) the chaining argument. On the other hand, we considered such classes of functions F which may have no small subclasses which are dense in F in the supremum norm.

I would characterize the difference between the results of the two works in the following way. Talagrand proved the sharpest possible estimates which can be obtained by a refinement of the chaining argument, while our main problem was to get sharp estimates also in such cases when the chaining argument does not work.

Let me remark that we could prove our results only for such classes of functions F which are L_{2}-dense. (See Theorem 4.1.) In the Gaussian counterpart of this result, in Theorem 4.2, it was enough to impose that F is an L_{2}-dense class with respect to a fixed probability measure μ. We needed the extra condition about L_{2}-dense property to prove sharp results about the tail distribution of supremum of partial sums when the chaining argument does not work.

Chapter 8

The main results of this work are presented in Chapter 8. One of them is Theorem 8.3 which is a multivariate version of Bernstein’s inequality (Theorem 3.1) about degenerate U-statistics. A weaker version of this result was first proved in a paper of Arcones and Giné in [4]. In the present form it was proved in my paper [39]. Its version about multiple integrals with respect to a normalized empirical measure formulated in Theorem 8.1 is proved in [35]. This paper contains a direct proof. On the other hand, Theorem 8.1 can be derived from Theorem 8.3 by means of Theorem 9.4 of this paper. Theorem 8.5 is the natural Gaussian counterpart of Theorem 8.3. The limit theorem about degenerate U-statistics, Theorem 10.4 (and its version about limit theorems for multiple integrals with respect to normalized empirical measures, presented in Theorem 10.4′ of Appendix C was discussed in this work to explain better the relation between degenerate U-statistics (or multiple integrals with respect to normalized empirical measures) and multiple Wiener–Itô integrals. A proof of this result based on similar ideas as that discussed here can be found in [16]. Theorem 6.6 of my lecture note [32] contains such a weaker version of Theorem 8.5 which does not take into account the variance of the random integral we are considering.

Example 8.7 is a natural supplement of Theorem 8.5. It shows that the estimate of Theorem 8.5 is sharp if only the variance of a Wiener–Itô integral is known. At the end of Chapter 13 I also mentioned the results of papers [1] and [29] without proof which also have some relation to this problem. I discussed mainly the content of [29], and explained its relation to some results discussed in this work. The proof of these papers apply a method different of those in this work. I make some comments about them in the discussion of Chapter 13.

Theorems 8.2 and 8.4 which are the natural multivariate counterparts of Theorem 4.1 and 4.1′yield an estimate about the supremum of (degenerate) U-statistics or of multiple random integrals with respect to a normalized empirical measure when the class of kernel functions in these U-statistics or random integrals satisfy some conditions. They were proved in my paper [37]. Actually I consider these theorems the hardest and most important results of this lecture note. Earlier Arcones and Giné proved a weaker version of this result in paper [5], but their work did not help in the proof of the results of this note. The proofs of the present note were based on an adaptation of Alexander’s method [3] to the multivariate case. Theorem 8.6 is the natural Gaussian counterpart of Theorems 8.2 and 8.4.

Example 8.8 in Chapter 8 shows that the condition u ≤ const.nσ^{3} imposed in Theorem 8.3 in the case k = 2 cannot be dropped. The paper of Arcones and Giné [4] contains another example explained by Talagrand to the authors of that paper which also has a similar consequence. But that example does not provide such an explicit comparison of the upper and lower bound on the probability investigated in Theorem 8.3 as Example 8.8. Similar examples could be constructed for all k ≥ 1.

Example 8.8 shows that at high levels only a very weak (and from practical point of view not really important) improvement of the estimation on the tail distribution of degenerate U-statistics is possible. But probably there exists a multivariate version of Bennett’s inequality, i.e. of Theorem 3.2 which provides such an estimate. Moreover, there is some hope to get a similar strengthened form of Theorems 8.2 and 8.4 (or of Theorem 4.2 in the one-dimensional case). This question is not investigated in the present work.

Chapter 9

Chapter 9 deals with the properties of U-statistics. Its first result, Theorem 9.1, is a classical result. It is the so-called Hoeffding decomposition of U-statistics to the sum of degenerate statistics. Its proof first appeared in the paper [24], but it can be found at many places. The explanation of this work contains some ideas similar to [54]. I tried to explain that Hoeffding’s decomposition is the natural multivariate version of the (trivial) decomposition of sums of independent random variables to sums of independent random variables with expectation zero plus the sum of the expectations of the original random variables. Moreover, even the proof of Hoeffding’s decomposition shows some similarity to this simple decomposition.

Theorem 9.2 and Proposition 9.3 can be considered as a continuation of the investigation about the Hoeffding decomposition. They tell us how some properties of the kernel function of the original U-statistic are inherited in the properties of the kernel functions of the degenerate U-statistics taking part in its Hoeffding decomposition. In several applications of Hoeffding’s decomposition we need such results.

The last result of Chapter 9, Theorem 9.4, enables us to reduce the estimation of multiple random integrals with respect to normalized empirical measures to the estimation of degenerate U-statistics. This result is a version of Hoeffding’s decomposition, where instead of U-statistics multiple integrals with respect to a normalized empirical distribution are decomposed to the sum of degenerate U-statistics. In these two decompositions the same degenerate U-statistics appear. The main difference between them is that in the decomposition of the random integrals in Theorem 9.4 the coefficients of the degenerate U-statistics are relatively small. The appearance of small coefficients in this decomposition is due to the cancellation effect caused by integration with respect to a normalized empirical measure \sqrt{n}(μ_{n}− μ). Theorem 9.4 was proved in [37]. The proof in this note is essentially different of the original proof in [37], and it is simpler.

Some remarks related to Chapters 10, 11 and 12

Theorem 8.1 can be derived from Theorem 8.3 and Theorem 8.2 from Theorem 8.4 by means of Theorem 9.4. The proof of the latter results is simpler. Chapters 10–12 contain the results needed in the proof of Theorem 8.3 and of its Gaussian counterpart Theorems 8.5 and 8.7. They are proved by means of good estimates on the high moments of degenerate U-statistics and multiple Wiener–Itô integrals. The classical proof of the one-variate counterparts of these results is based on a good estimate of the moment generating function. This method had to be replaced by the estimation of high moments, because the moment generating function of a k-fold Wiener–Itô integral is divergent for all non-zero parameters if k ≥ 3, (this is a consequence of Theorem 13.6), and this property of Wiener–Itô integrals is also reflected in the behaviour of degenerate U-statistics. On the other hand, we can give good estimates on the tail distribution of a random variable if we have good estimates on its high moments. The results of Chapters 10, 11 and 12 enable us to prove good moment estimates.

I know of two deep and interesting methods to study high moments of multiple Wiener–Itô integrals. The first of them is called Nelson’s inequality named after Edward Nelson who published it in his paper [45]. This inequality simply implies Theorem 8.5 about multiple Wiener–Itô integrals, although with worse constants. Later Leonhard Gross discovered a deep and useful generalization of this result which he published in the work Logarithmic Sobolev inequalities [21]. Gross considered in his paper a stationary Markov process X(t), t ≥ 0, and gave a good bound on the L_{p}-norm of functions of the form U_{t}(f)(x) = E(f(X(t)|X(0) = x), where the L_{p}-norm is taken with respect to the distribution of the random variable X(0). The proof of this L_{p}-norm estimate is based on the study of the infinitesimal operator of the Markov process. Gross’ results provide Nelson’s inequality, if they are applied for the Ornstein–Uhlenbeck process.

Gross’ investigation in [21] revealed very much about the behaviour of Markov processes. The book [46] is partly based on this method. Gross’ approach turned out to be very fruitful in the study of several hard problems of the probability theory and statistical physics. (See e.g [22] or [30]). It also provides a good estimate for the high moments of Wiener–Itô integrals.

There is another useful method to study Wiener–Itô integrals due to Kyoshi Itô and Roland L’vovich Dobrushin. This seemed to me more useful if we want estimate the high moments not only of Wiener–Itô integrals but also of degenerate U-statistics. I applied this method in Chapters 10, 11 and 12. I showed how we can get with its help results that enable us to prove good moment estimates both for Wiener–Itô integrals and degenerate U-statistics. The main step in this approach is the proof of a so-called diagram formula which makes possible to rewrite a product of Wiener–Itô integrals as a sum of Wiener–Itô integrals. Moreover, this result also has a natural counterpart for the products of degenerate U-statistics.

Chapter 10

In Chapter 10 I discuss a method related to Kyoshi Itô and Roland L’vovich Dobrushin. This is the theory of multiple Wiener–Itô integrals with respect to a white noise. This integral was introduced in paper [26]. It is useful, because every random variable which is measurable with respect to the σ-algebra generated by the Gaussian random variables of the underlying white noise and has finite second moment can be written as the sum of Wiener–Itô integrals of different order. Moreover, if only Wiener–Itô integrals of symmetric kernel functions are taken, then this representation is unique. Actually this result was originally proved by Norbert Wiener [60]. This representation also appeared in physics under the name Fock space. It plays an important role in quantum physics. Let me briefly explain the reason for the name white noise for the appropriate notion introduced in Chapter 10.

The notion of white noise was originally introduced at a heuristic level as the derivative of the trajectories of a Wiener process. But as these trajectories are nondifferentiable the introduction of this notion demands a better explanation. A natural way to overcome the difficulties is to consider the derivative of a trajectory of a Wiener process as a generalized random function, and to take its integral on all measurable sets. In such a way we get a collection of Gaussian random variables ξ(A) with expectation zero, indexed by the measurable sets A. These random variables have correlation function Eξ(A)ξ(B) = λ(A∩B), where λ(·) denotes the Lebesgue measure. In such a way we get a correct definition of the white noise which preserves the heuristic content of the original approach. In the definition of general white noise we allow to work with an arbitrary measure μ and not only with the Lebesgue measure λ. If we have a white noise we would like to have a tool that enables us to study not only the Gaussian random variables measurable with respect to the σ-algebra generated by the random variables of the white noise but all random variables measurable with respect to this σ-algebra. The Wiener–Itô integrals were defined with such a goal.

An important result of the theory of Wiener–Itô integrals, the so-called diagram formula, formulated in Theorem 10.2, expresses products of Wiener–Itô integrals as a sum of such integrals. This result which shows some similarity to the Feynman diagrams applied in the statistical physics was proved in [11]. Actually this paper discussed a modified version of Wiener–Itô integrals which is more appropriate to study the action of shift operators for non-linear functionals of a stationary Gaussian field. But these modified Wiener–Itô integrals can be investigated in almost the same way as the original ones. The diagram formula has a simple consequence formulated in Corollary of Theorem 10.2 of this note. It enables us to calculate the expectation of products of Wiener–Itô integrals. It yields an explicit formula for them. This result was applied in the proof of Theorem 8.5, i.e. in the estimation of the taildistribution of Wiener–Itô integrals. Itô’s formula for multiple Wiener–Itô integrals (Theorem 10.3) was proved in [26].

Actually the above results about Wiener–Itô integrals would have been sufficient for our purposes. But I also presented some other results for the sake of completeness. In particular, I discussed some results about Hermite polynomials. Wiener–Itô integrals are closely related to Hermite polynomials or to their multivariate version, to the so-called Wick polynomials. (See e.g. [32] or [42] for the definition of Wick polynomials.) Appendix C contains the most important properties of Hermite polynomials needed in the study of Wiener–Itô integrals. In particular, it contains the proof of Proposition C2 about the completeness of the Hermite polynomials in the Hilbert space of the functions square integrable with respect to the standard Gaussian distribution. This result can be found for instance in Theorem 5.2.7 of [53]. In the present proof I wanted to show that this result is closely related to the so-called moment problem, i.e. to the question when a distribution is determined by its moments uniquely. The method of proof described in this note can be applied with some refinement to prove some generalizations of Proposition C2 about the completeness of orthogonal polynomials with respect to more general weight functions.

On the other hand, I did not try to give a complete picture about Wiener–Itô integrals. The reader interested in it may consult with the book of S. Janson [27]. There are also other interesting and important topics related to Wiener–Itô integrals not discussed in this work. In some investigations of probability theory and statistical physics it is useful to study not only moments but also cumulants (called also semiinvariants in the literature) of Wiener–Itô integrals. It is also useful to study the moments and cumulants of polynomials and Wick polynomials of Gaussian random vectors. The book of Malyshev V. A. and Minlos R.A. [42] contains many interesting results about this subject.

Another interesting and popular subject not discussed in this work is the problem of limit theorems for Wiener–Itô integrals. In particular, one is interested in the question when a sequence of such random integrals satisfies the central limit theorem. The study of such problems heavily exploits the diagram formula, or more precisely its consequence about the calculation of moments and cumulants. In some works, see e.g. [46] or [48] this subject is worked out in detail. Moreover, a popular subject of recent research is the study of the speed of convergence in the central limit theorem. In such investigations the so-called Stein method turned out to be very useful. In its application the integral of sufficiently smooth test functions with respect to the distribution we are investigating are estimated together with the integral of their derivative (with respect to the same distribution). In a somewhat surprising way it turned out that if we are studying the central limit theorem for Wiener–Itô integrals with the help of the Stein method, then the role of the derivative of a function is taken by the so-called Malliavin derivative. (See [46].) So the theory of Malliavin calculus, see [47], became very important in such research. But this problem is a bit far from the main subject of this work, hence I do not go into the details.

Chapters 11 and 12

The diagram formula has a natural and useful analogue both for degenerate Ustatistics and multiple integrals with respect to a normalized empirical measure. They enable us to rewrite the product of degenerate U-statistics and multiple integrals as the sum of such expressions. Actually the proof of these results is simpler than the proof of the original diagram formula for Wiener–Itô integrals. They make possible to adapt several useful methods of the study of non-linear functionals of Gaussian random fields to the study of non-linear functionals of normalized empirical measures. But to apply them we also need some good estimate on the L_{2}-norm of the kernel functions of the random integrals or U-statistics appearing in the diagram formula. Hence we also proved such results.

A version of the diagram formula was proved for degenerate U-statistics in [39] and for multiple random integrals with respect to a normalized empirical measures in [35]. Let me remark that in the formulation of the result in the work [39] a different notation was applied than in the present note. In that paper I wanted to formulate such a version of the diagram formula for U-statistics where we work with diagrams similar to those introduced in the study of Wiener–Itô integrals. I could do this only in a somewhat artificial way. In this work I formulated the diagram formula for Ustatistics with the help of diagrams of a more general form. I introduced the notion of chains and coloured chains, and defined (coloured) diagrams with their help. The formulation of the results with the help of such more general diagrams seems to be more natural. I met some works where similar diagrams were introduced, see e.g. [48], but I did not meet works where also the coloured diagrams introduced in this work were applied. It is possible that this happened so, because I do not know the literature well enough, but this also may have a different cause.

In the work [48] the diagram formula was applied for the calculation of moments and cumulants, and if we are working only with them, then the results of this work can also be formulated with the help of so-called closed diagrams, and no coloured diagrams are needed. They are needed if we want to express the product of U-statistics as a sum of U-statistics. It may also be interesting that the results considered in [48] are based on some combinatorial arguments worked out in [50].

There are some works like [48], where diagram formulas are considered for other models too, e.g. in models where we integrate with respect to a normalized Poisson process. Nevertheless, in my opinion the results about the diagram formula for the products of Wiener–Itô integrals and in particular their modified versions for the products of integrals with respect to normalized Poisson processes, normalized empirical distribution or for the product of U-statistics did not get such an attention in the literature as they would deserve. An interesting paper in this direction is that of Surgailis [51], where a version of the diagram formula is proved for Poissonian integrals. It may be worth mentioning that the diagram formula for Poisson integrals shows a very strong similarity to the diagram formula for the product of integrals with respect to normalized empirical distributions. (Integrals with respect to normalized empirical distribution were discussed only at an informal level in this work.)

The Hermite polynomials and their multivariate versions, the Wick polynomials have their counterparts when instead of Wiener–Itô integrals we consider more general classes of random integrals. Itô’s formula creates a relation between Wiener–Itô integrals and Hermite polynomials or their multivariate versions, the Wick polynomials. The relation between Wiener–Itô integrals and Hermite polynomials has a natural counterpart in the study of other multiple random integrals. In such a way a new notion, the Appell polynomials appeared in the literature. (See e.g. [52].)

Chapter 13

Theorems 8.3, 8.5 and 8.7 were proved on the basis of the results of Chapters 10– 12 in Chapter 13. These proofs are slight modifications of those given in [39]. An earlier proof of a result similar to Theorem 8.3 based on a different method was given by Arcones and Giné in [4]. Theorem 8.3 is a slightly stronger estimate than that of Arcones and Giné. It provides at not too high levels an estimate with almost as good constants in the exponent as the corresponding estimate about Wiener–Itô integrals in Theorem 8.5. Chapter 13 also contains the proof of a multivariate version of Hoeffding’s inequality formulated in Theorem 13.3. This result is needed in the symmetrization argument applied in the proof of Theorem 8.4. A weaker version of it (an estimate with a worse constant in the exponent) which would be satisfactory for our purposes simply follows from a classical result, called Borell’s inequality, which was proved in [8]. But since the methods needed to prove this result are not discussed in this note, and I was interested in a proof which yields an estimate with the best possible constant in the exponent I chose another proof, given in [38]. It is based on the results of Chapter 10–12. Later I have learned that this estimate is contained in an implicit form also in the paper [7] of Aline Bonami.

In Part B of Chapter 13 I discussed some results related to the problems considered in this work. I would like to make some comments about the result of R. Lata�a presented in Theorem 13.7. The estimates of this result depend on such quantities which are hard to calculate. Hence they have a limited importance in the problems I had in mind when working on this lecture note. On the other hand, such results and the methods behind them may be interesting in the study of some problems of statistical physics, e.g. in the problems discussed in [57]. I would like to remark that Lata�a’s proof works only for decoupled and not for usual U-statistics. Formally, this is not a restriction, because the results of de la Peña and Montgomery–Smith (see [10]) enable us to extend their validity also for usual U-statistics. Nevertheless, the lack of a direct proof of this estimate for U-statistics disturbs me a bit, because this means for me that we do not really understand this result. I have some ideas how to get the desired proof, but it demands some time and energy to work out the details.

Chapter 14

Chapters 14–17 are devoted to the proof of Theorems 8.4 and 8.6. They are based on a similar argument as their one-variate counterparts, Theorems 4.1 and 4.2. The proof of Theorem 8.6 about the supremum of Wiener–Itô integrals is based, similarly to the proof of Theorem 4.2, on the chaining argument. In the proof of Theorem 8.4 the chaining argument yields only a weaker result formulated in Proposition 14.1 which helps to reduce Theorem 8.4 to the proof of Proposition 14.2. In the one-variate case a similar approach was applied. In that case the proof of Theorem 4.1 was reduced to that of Proposition 6.2 by means of Proposition 6.1. The next step in the proof of Theorem 8.4 has no one-variate counterpart. The notion of so-called decoupled U-statistics was introduced, and Proposition 14.2 was reduced to a similar result about decoupled U-statistics formulated in Proposition 14.2′.

The adjective ‘decoupled’ in the expression decoupled U-statistic refers to the fact that it is such a version of a U-statistic where independent copies of a sequence of independent and identically distributed random variables are put into different coordinates of the kernel function. Their study is a popular subject of some mathematical schools. In particular, the main topic of the book [9] is a comparison of the properties of U-statistics and decoupled U-statistics. A result of de la Peña and Montgomery–Smith [10] formulated in Theorem 14.3 helps in reducing some problems about U-statistics to a similar problem about decoupled U-statistics. In this lecture note the proof of Theorem 14.3 is given in Appendix D. It follows the argument of the original proof, but several steps are worked out in detail where the authors gave only a very short explanation. Paper [10] also contains some kind of converse results to Theorem 14.3, but as they are not needed in the present work, I omitted their discussion.

Decoupled U-statistics behave similarly to the original U-statistics. Beside this, some symmetrization arguments become considerably simpler if we are working with decoupled U-statistics instead of the original ones, because decoupled Ustatistics have more independence property. This can be exploited in some investigations. For example the proof of Proposition 14.2′is simpler than a direct proof of Proposition 14.2. On the other hand, Theorem 14.3 enables us to reduce the proof of Proposition 14.2 to that of Proposition 14.2′, and we have exploited this possibility. Let me finally remark that although our proofs could be simplified with the help of decoupled U-statistics, they could have been done also without it. But this would demand a much more complicated notation that would have made the proof much less transparent. Hence I have decided to introduce decoupled U-statistics and to work with them.

Chapters 15, 16 and 17

The proof of Theorem 8.4 was reduced to that of Proposition 14.2′in Chapter 14. Chapters 15–17 deal with the proof of this result. The original proof was given in my paper [37]. It is similar to that of its one-variate version, Proposition 6.2, but some additional difficulties have to be overcome. The main difficulty appears when we want to find the multivariate analogue of the symmetrization argument which could be carried out in the one-variate case by means of Lemmas 7.1 and 7.2.

In the multivariate case Lemma 7.1 is not sufficient for our purposes. So we work instead with a generalized version of this result, formulated in Lemma 15.2. The proof of Lemma 15.2 is not hard. It is a simple and natural modification of the proof of Lemma 7.1. The real difficulty arises when we want to apply it in the proof of Proposition 14.2′. When we applied the symmetrization argument Proposition 6.2 in the proof of Lemma 7.1 we have worked with two independent sequences of random variables Z_{n} and Z̄_{n}. In the analogous symmetrization argument Lemma 15.2, applied in the proof of Proposition 14.2′, we had to work with two not necessarily independent sequences of random variables Z_{p} and Z̄_{p}. This has the consequence that it is much harder to check condition (15.3) needed in the application of Lemma 15.2 than the analogous condition (7.1) in Lemma 7.1. The hardest problems in the proof of Proposition 14.2′appear at this point.

Proposition 14.2′was proved by means of an inductive procedure formulated in Proposition 15.3, which is the multivariate analogue of Proposition 7.3. A basic ingredient of both proofs was a symmetrization argument. But while this symmetrization argument could be simply carried out in the one-variate case, its adaptation to the multivariate case was a most serious problem. To overcome this difficulty another inductive statement was formulated in Proposition 15.4. Propositions 15.3 and 15.4 could be proved simultaneously by means of an appropriate inductive procedure. Their proofs were based on a refinement of the arguments in the proof of Proposition 7.3. But some new difficulties arose. In the proof of Proposition 7.3 we could simply apply Lemma 7.2, and it provided the necessary symmetrization argument. On the other hand, the verification of the corresponding symmetrization argument in the proof of Propositions 15.3 and 15.4 was much harder. Actually this was the subject of Chapter 16. After this we could prove Propositions 15.3 and 15.4 in Chapter 17 similarly to Proposition 7.3, although some additional technical difficulties arose also at this point. Here we needed the multivariate version of Hoeffding’s inequality, formulated in Theorem 13.3 and some properties of the Hoeffding decomposition of U-statistics proved in Chapter 9.

Appendix A

The proof of some results about Vapnik–Červonenkis classes

Proof of Theorem 5.1. (Sauer’s lemma). This result has several different proofs. Here I write down a relatively simple proof of P. Frankl and J. Pach which appeared in [17]. It is based on some linear algebraic arguments.

The following equivalent reformulation of Sauer’s lemma will be proved. Let us take a set S = S(n) consisting of n elements and a class E of subsets of S consisting of m subsets E_{1},...,E_{m} ⊂ S. Assume that m ≥ m_{0} + 1 with m_{0} = m_{0}(n,k) = \binom{n}{0} + \binom{n}{1} + ··· + \binom{n}{k−1}. Then there exists a set F ⊂ S of cardinality k which is shattered by the class of sets E. Actually, it is enough to show that there exists a set F of cardinality greater than or equal to k which is shattered by the class of sets E, because if a set has this property, then all of its subsets have it. This latter statement will be proved.

To prove this statement let us first list the subsets X_{0},...,X_{m_{0}} of the set S of cardinality less than or equal to k − 1, and correspond to all sets E_{i} ∈ E the vector e_{i} = (e_{i,1},...,e_{i,m_{0}}), 1 ≤ i ≤ m, with elements

e_{i,j} = \{ \begin{matrix}1 if X_{j} ⊆ E_{i} \\ 0 if X_{j} ⊈ E_{i} \end{matrix} 1 ≤ i ≤ m, and 1 ≤ j ≤ m_{0}.

Since m > m_{0}, the vectors e_{1},...,e_{m} are linearly dependent. Because of the definition of the vectors e_{i}, 1 ≤ i ≤ m, this can be expressed in the following way: There is a non-zero vector (f(E_{1}),...,f(E_{m})) such that

∑_{E_{i}: E_{i}⊇X_{j}} f(E_{i}) = 0 for all 1 ≤ j ≤ m_{0}.  (A.1)

Let F, F ⊂ S, be a minimal set with the property

∑_{E_{i}: E_{i}⊇F} f(E_{i}) = α ≠ 0.  (A.2)

Such a set F really exists, since every maximal element of the family {E_{i}: 1 ≤ i ≤ m, f(E_{i}) ≠ 0} satisfies relation (A.2). The requirement that F should be a minimal set means that if F is replaced by some H ⊂ F, H ≠ F, at the left-hand side of (A.2), then this expression equals zero. The inequality |F | ≥ k holds because of relation (A.1) and the definition of the sets X_{j}.

Introduce the quantities

Z_{F}(H) = ∑_{E_{i}: E_{i}∩F =H} f(E_{i})

for all H ⊆ F.

Then Z_{F}(F) = α, and for any set of the form H = F \ {x}, x ∈ F,

Z_{F}(H) = ∑_{E_{i}: E_{i}∩F =H} f(E_{i}) = ∑_{E_{i}: E_{i}⊇H} f(E_{i}) − ∑_{E_{i}: E_{i}⊇F} f(E_{i}) = 0 − α = −α

because of the minimality property of the set F.

Moreover, the identity

Z_{F}(H) = (−1)^{p}α for all H ⊆ F such that |H| = |F | − p, 0 ≤ p ≤ |F |.  (A.3)

holds. To show relation (A.3) observe that

Z_{F}(H) = ∑_{E_{i}: E_{i}∩F =H}f(E_{i}) = ∑_{j=0}^{p} (−1)^{j} ∑_{G: H⊂G⊂F, |G|=|H|+j} ∑_{E_{i}: E_{i}⊇G}f(E_{i})  (A.4)

for all sets H ⊂ F with cardinality |H| = |F |−p. Identity (A.4) holds, since the term

f(E_{i}) is counted at the right-hand side of (A.4) ∑_{j=0}^{l} (−1)^{j} \binom{l}{j} = (1 − 1)^{l} = 0 times if

E_{i}∩ F = G with some H ⊂ G ⊆ F with |G| = |H| + l elements, 1 ≤ l ≤ p, while in the case E_{i} ∩ F = H it is counted once. Relation (A.4) together with (A.2) and the minimality property of the set F imply relation (A.3).

It follows from relation (A.3) and the definition of the function Z_{F}(H) that for all sets H ⊆ F there exists some set E_{i} such that H = E_{i} ∩ F, i.e. F is shattered by E. Since |F | ≥ k, this implies Theorem 5.1.

Proof of Theorem 5.3. Let us fix an arbitrary set F = {x_{1},...,x_{k+1}} of the set X, and consider the set of vectors G_{k}(F) = {(g(x_{1}),...,g(x_{k+1})): g ∈ G_{k}} of the k + 1-dimensional space R^{k+1}. By the conditions of Theorem 5.3 G_{k}(F) is an at most k-dimensional subspace of R^{k+1}. Hence there exists a non-zero vector

a = (a_{1},...,a_{k+1}) such that ∑_{j=1}^{k+1} a_{j}g(x_{j}) = 0 for all g ∈ G_{k}. We may assume that

the set A = A(a) = {j: a_{j} < 0, 1 ≤ j ≤ k + 1} is non-empty, by multiplying the vector a by −1 if it is necessary.

Thus the identity

∑_{j∈A} a_{j}g(x_{j}) = ∑_{j∈{1,...,k+1}\A} (−a_{j})g(x_{j}),  for all g ∈ G_{k}  (A.5)

holds. Put B = {x_{j} : j ∈ A}. Then B ⊂ F, and F \ B ≠ {x: g(x) ≥ 0} ∩ F for all g ∈ G_{k}. Indeed, if there were some g ∈ G_{k} such that F \ B = {x: g(x) ≥ 0} ∩ F, then the left-hand side of the equation (A.5) would be strictly positive (as a_{j} < 0, g(x_{j}) < 0 if j ∈ A, and A ≠ ∅) its right-hand side would be non-positive for this g ∈ G_{k}, and this is a contradiction.

The above proved property means that D shatters no set F ⊂ X of cardinality k + 1. Hence Theorem 5.1 implies that D is a Vapnik–Červonenkis class.

Appendix B

The proof of the diagram formula for Wiener–Itô integrals

We start the proof of Theorem 10.2A (the diagram formula for the product of two Wiener–Itô integrals) with the proof of inequality (10.13). To show that this relation holds let us observe that the Cauchy inequality yields the following bound on the function F_{γ}(f,g) defined in (10.11) (with the notation introduced there):

F_{γ}^{2}(f,g)(x_{(1,j)},x_{(2,j′)}, (1,j) ∈ V_{1}(γ), (2,j′) ∈ V_{2}(γ))

≤ ∫ f^{2}(x_{α_{γ}(1,1)},...,x_{α_{γ}(1,k)}) ∏_{(2,j)∈{(2,1),...,(2,l)}\V_{2}(γ)}μ(dx_{(2,j)})

∫ g^{2}(x_{(2,1)},...,x_{(2,l)})  ∏_{(2,j)∈{(2,1),...,(2,l)}\V_{2}(γ)}μ(dx_{(2,j)}). (B.1)

The expression at the right-hand side of inequality (B.1) is the product of two functions with different arguments. The first function has arguments x_{(1,j)} with (1,j) ∈ V_{1}(γ) and the second one x_{(2,j′)} with (2,j′) ∈ V_{2}(γ). By integrating both sides of inequality (B.1) with respect to these arguments we get inequality (10.13).

Relation (10.14) will be proved first for the product of the Wiener–Itô integrals of two elementary functions. Let us consider two (elementary) functions f(x_{1},...,x_{k}) and g(x_{1},...,x_{l}) given in the following form: Let some disjoint sets A_{1},...,A_{M}, μ(A_{s}) < ∞, 1 ≤ s ≤ M, be given together with some real numbers c(s_{1},...,s_{k}) indexed with such k-tuples (s_{1},...,s_{k}), 1 ≤ s_{j} ≤ M, 1 ≤ j ≤ k, for which the numbers s_{1},...,s_{k} in a k-tuple are all different. Put f(x_{1},...,x_{k}) = c(s_{1},...,s_{k}) if (x_{1},...,x_{k}) ∈ A_{s_{1}} ×··· ×A_{s_{k}} with some vector (s_{1},...,s_{k}) with different coordinates, and and let f(x_{1},...,x_{k}) = 0 if (x_{1},...,x_{k}) is outside of these rectangles. Take similarly some disjoint sets B_{1},...,B_{M′}, μ(B_{t}) < ∞, 1 ≤ t ≤ M′, and some real numbers d(t_{1},...,t_{l}), indexed with such l-tuples (t_{1},...,t_{l}), 1 ≤ t_{j′} ≤ M′, 1 ≤ j′≤ l, for which the numbers t_{1},...,t_{l} in an l-tuple are different. Put g(x_{1},...,x_{l}) = d(t_{1},...,t_{l}) if (x_{1},...,x_{l}) ∈ B_{t_{1}}× ··· × B_{t_{l}} with edges indexed with some of the above introduced l-tuples, and let g(x_{1},...,x_{l}) = 0 otherwise.

Let us take some small number ε > 0 and rewrite the above introduced functions f(x_{1},...,x_{k}) and g(x_{1},...,x_{l}) with the help of this number ε > 0 in the following

way. Divide the sets A_{1},...,A_{M} to smaller sets A_{1}^{ε},...,A_{M(ε)}^{ε}, ⋃_{s=1}^{M(ε)}A_{s}^{ε} = ⋃_{s=1}^{M} A_{s}, in

such a way that all sets A_{1}^{ε},...,A_{M(ε)}^{ε} are disjoint, and μ(A_{s}^{ε}) ≤ ε, 1 ≤ s ≤ M(ε).

Similarly, take sets B_{1}^{ε},...,B_{M′(ε)}^{ε}, ⋃_{t=1}^{M′(ε)}B_{t}^{ε} = ⋃_{t=1}^{M′}B_{t}, in such a way that all sets

B_{1}^{ε},...,B_{M′(ε)}^{ε} are disjoint, and μ(B_{t}^{ε}) ≤ ε, 1 ≤ t ≤ M′(ε). Beside this, let us also demand that two sets A_{s}^{ε} and B_{t}^{ε}, 1 ≤ s ≤ M(ε), 1 ≤ t ≤ M′(ε), are either disjoint or they agree. Such a partition exists because of the non-atomic property of measure μ. The above defined functions f(x_{1},...,x_{k}) and g(x_{1},...,x_{l}) can be rewritten by means of these new sets A_{s}^{ε} and B_{t}^{ε}. Namely, let f(x_{1},...,x_{k}) = c^{ε}(s_{1},...,s_{k}) on the rectangles A_{s_{1}}^{ε} × ··· × A_{s_{k}}^{ε} with 1 ≤ s_{j} ≤ M(ε), 1 ≤ j ≤ k, with different indices s_{1},...,s_{k}, where c^{ε}(s_{1},...,s_{k}) = c(p_{1},...,p_{k}) with those indices (p_{1},...,p_{k}) for which A_{s_{1}}^{ε} × ···× A_{s_{k}}^{ε} ⊂ A_{p_{1}} × ···× A_{p_{k}}. The function f disappears outside of these rectangles. The function g(x_{1},...,x_{l}) can be written similarly in the form g(x_{1},...,x_{l}) = d^{ε}(t_{1},...,t_{l}) on the rectangles B_{t_{1}}^{ε}× ··· × B_{t_{l}}^{ε} with 1 ≤ t_{j′} ≤ M′(ε), 1 ≤ j′≤ l, and different indices, t_{1},...,t_{l}. Beside this, the function g disappears outside of these rectangles.

The above representation of the functions f and g through a parameter ε is useful, since it enables us to give a good asymptotic formula for the product k!Z_{μ,k}(f)l!Z_{μ,l}(g) which yields the diagram formula for the product of Wiener–Itô integrals of elementary functions with the help of a limiting procedure ε → 0.

Fix a small number ε > 0, take the representation of the functions f and g with its help, and write

k!Z_{μ,k}(f)l!Z_{μ,l}(g) = ∑_{γ∈Γ (k,l)}Z_{γ}(f,g,ε)  (B.2)

with

Z_{γ}(f,g,ε) = ∑ ^{γ}c^{ε}(s_{1},...,s_{k})d^{ε}(t_{1},...,t_{l})
μ_{W}(A_{s_{1}}^{ε})...μ_{W}(A_{s_{k}}^{ε})μ_{W}(B_{t_{1}}^{ε})...μ_{W}(B_{t_{l}}^{ε}),  (B.3)

where Γ (k,l) denotes the class of diagrams introduced before the formulation of Theorem 10.2A, and ∑^{γ} denotes summation for k + l-tuples (s_{1},...,s_{k},t_{1},...,t_{l}) such that 1 ≤ s_{j} ≤ M(ε), 1 ≤ j ≤ k, 1 ≤ t_{j′} ≤ M′(ε), 1 ≤ j′≤ l, and A_{s_{j}}^{ε} = B_{t_{j′}}^{ε} if ((1,j),(2,j′)) ∈ E(γ), i.e. if it is an edge of γ, and otherwise all sets A_{s_{j}}^{ε} and B_{t_{j′}}^{ε} are disjoint. (This sum also depends on ε.) In the case of an empty sum Z_{γ}(f,g,ε) equals zero.

We write the expression Z_{γ}(f,g,ε) for all γ ∈ Γ (k,l) in the form

Z_{γ}(f,g,ε) = Z_{γ}^{(1)}(f,g,ε) + Z_{γ}^{(2)}(f,g,ε), γ ∈ Γ (k,l),  (B.4)

Z_{γ}^{(1)}(f,g,ε) = ∑ ^{γ}c^{ε}(s_{1},...,s_{k})d^{ε}(t_{1},...,t_{l})

∏_{j: (1,j)∈V_{1}(γ)}μ_{W}(A_{s_{j}}^{ε}) ∏_{j′: (2,j′)∈V_{2}(γ)}μ_{W}(B_{t_{j′}}^{ε})

∏_{j′: (2,j′)∈{(2,1),...,(2,l)}\V_{2}(γ)}μ(B_{t_{j′}}^{ε})  (B.5)

and

Z_{γ}^{(2)}(f,g,ε) = ∑ ^{γ}c^{ε}(s_{1},...,s_{k})d^{ε}(t_{1},...,t_{l})

∏_{j: (1,j)∈V_{1}(γ)}μ_{W}(A_{s_{j}}^{ε}) ∏_{j′: (2,j′)∈V_{2}(γ)}μ_{W}(B_{t_{j′}}^{ε})

[ ∏_{j: (1,j)∈{(1,1),...,(1,k)}\V_{1}(γ)}μ_{W}(A_{s_{j}}^{ε})

∏_{j′: (2,j′)∈{(2,1),...,(2,l)}\∈V_{2}(γ)} μ_{W}(B_{t_{j′}}^{ε})

−  ∏_{j′: (2,j′)∈{(2,1),...,(2,l)}\V_{2}(γ)}μ(B_{t_{j′}}^{ε})],  (B.6)

where V_{1}(γ) and V_{2}(γ) (introduced before formula (10.9) during the preparation to the formulation of Theorem 10.2A) are the sets of vertices in the first and second row of the diagram γ from which no edge starts.

I claim that there is some constant C > 0 not depending on ε such that

E ( |γ|!Z_{μ,|γ|}(F_{γ}(f,g)) − Z_{γ}^{(1)}(f,g,ε)) ^{2} ≤ Cε for all γ ∈ Γ (k,l)  (B.7)

with the Wiener–Itô integral with the kernel function F_{γ}(f,g) defined in (10.9), (10.10) and (10.11), and

E ( Z_{γ}^{(2)}(f,g,ε))^{2} ≤ Cε for all γ ∈ Γ (k,l).  (B.8)

Relations (B.2), (B.4), (B.7) and (B.8) imply relation (10.14) if f and g are elementary functions. Indeed, (B.4), (B.7) and (B.8) imply that

lim_{ε→0} ∥|γ|!Z_{μ,|γ|}(F_{γ}(f,g)) − Z_{γ}(f,g,ε)∥ _{2} → 0 for all γ ∈ Γ (k,l),

and this relation together with (B.2) yield relation (10.14) with the help of a limiting procedure ε → 0.

To prove relation (B.7) let us introduce the function F_{γ}^{ε}(f,g)(x_{(1,j)},x_{(2,j′)}, (1,j) ∈ V_{1}(γ), (2,j′) ∈ V_{2}(γ))

= F_{γ}(f,g)(x_{(1,j)},x_{(2,j′)}, (1,j) ∈ V_{1}(γ), (2,j′) ∈ V_{2}(γ))
if x_{(1,j)} ∈ A_{s_{j}}^{ε}, for all (1,j) ∈ V_{1}(γ),
x_{(2,j′)} ∈ B_{t_{j′}}^{ε}, for all (2,j′) ∈ V_{2}(γ)), and

all sets A_{s_{j}}^{ε}, (1,j) ∈ V_{1}(γ), and B_{t_{j′}}^{ε}, (2,j′) ∈ V_{2}(γ) are different.

with the function F_{γ}(f,g) defined in (10.10) and (10.11), and put

F_{γ}^{ε}(f,g)(x_{(1,j)},x_{(2,j′)}, (1,j) ∈ V_{1}(γ), (2,j′) ∈ V_{2}(γ)) = 0 otherwise.

The function F_{γ}^{ε}(f,g) is elementary, and a comparison of its definition with relation (B.5) and the definition of the function F_{γ}(f,g) yields that

Z_{γ}^{(1)}(f,g,ε) = |γ|!Z_{μ,|γ|}(F_{γ}^{ε}(f,g)).  (B.9)

The function F_{γ}^{ε}(f,g) slightly differs from F_{γ}(f,g), since the function F_{γ}(f,g) may not disappear in such points (x_{(1,j)},x_{(2,j′)}, (1,j) ∈ V_{1}(γ), (2,j′) ∈ V_{2}(γ)) for which there is some pair (j,j′) with the property x_{(1,j)} ∈ A_{s_{j}}^{ε} and x_{(2,j′)} ∈ B_{t_{j′}}^{ε} with some sets A_{s_{j}}^{ε} and B_{t_{j′}}^{ε} such that A_{s_{j}}^{ε} = B_{t_{j′}}^{ε}, while F_{γ}^{ε}(f,g) must be zero in such points. On the other hand, in the case |γ| = max(k,l) − min(k,l), i.e. if one of the sets V_{1}(γ) or V_{2}(γ) is empty, F_{γ}(f,g) = F_{γ}^{ε}(f,g), Z_{γ}^{(1)}(f,g,ε) = |γ|!Z_{μ,|γ|}(F_{γ}(f,g)), and relation (B.7) clearly holds for such diagrams γ.

In the case |γ| = max(k,l) − min(k,l) > 0 we prove a good estimate on the measure of the set where F_{γ} ≠ F_{γ}^{ε} with respect to an appropriate power of the measure μ. Relation (B.7) will be proved with the help of this estimate and formula (B.9).

Let us define the sets A = ⋃_{s=1}^{M(ε)}A_{s}^{ε} and B = ⋃_{t=1}^{M′(ε)}B_{t}^{ε}. These sets A and B do not

depend on the parameter ε. Beside this μ(A) < ∞, and μ(B) < ∞. Define for all pairs (j_{0},j′_{0}) such that (1,j_{0}) ∈ V_{1}(γ), (2,j′_{0}) ∈ V_{2}(γ) the set

D(j_{0},j′_{0}) = {(x_{(1,j)},x_{(2,j′)}, (1,j) ∈ V_{1}(γ), (2,j′) ∈ V_{2}(γ)):

x_{(1,j_{0})} ∈ A_{s}^{ε}, x_{(2,j′_{0})} ∈ B_{t}^{ε} with some 1 ≤ s ≤ M(ε) and 1 ≤ t ≤ M′(ε)

such that A_{s}^{ε} = B_{t}^{ε}, and x_{(1,j)} ∈ A for all (1,j) ∈ V_{1}(γ),

and x_{(2,j′)} ∈ B for all (2,j′) ∈ V_{2}(γ)}.

Introduce the notation x^{γ} = (x_{(1,j)},x_{(2,j′)}), (1,j) ∈ V_{1}(γ), (2,j′) ∈ V_{2}(γ)), and consider only such vectors x^{γ} whose coordinates satisfy the conditions x_{(1,j)} ∈ A for all (1,j) ∈ V_{1}(γ) and x_{(2,j′)} ∈ B for all (2,j′) ∈ V_{2}(γ). Put

D_{γ} = {x^{γ}: F_{γ}^{ε}(f,g)(x^{γ}) ≠ F_{γ}(f,g)(x^{γ})}.

The relation D_{γ} ⊂ ⋃_{j=1}^{k} ⋃_{j′=1}^{l} D(j_{0},j′_{0}) holds, since if F_{γ}^{ε}(f,g)(x^{γ}) ≠ F_{γ}(f,g)(x^{γ})

for some vector x^{γ}, then it has some coordinates (1,j_{0}) ∈ V_{1}(γ) and (2,j′_{0}) ∈ V_{2}(γ) such that x_{(1,j_{0})} ∈ A_{s}^{ε} and x_{(1,j′_{0})} ∈ B_{t}^{ε} with some sets A_{s}^{ε} = B_{t}^{ε}, and the relation in the last line of the definition of D(j_{0},j′_{0}) must also hold for such a vector x^{γ}, since otherwise F_{γ}(f,g)(x_{γ}) = 0 = F_{γ}^{ε}(f,g)(x_{γ}).

I claim that there is some constant C_{1} such that

μ^{|V_{1}(γ)|+|V_{2}(γ)|}(D(j_{0},j′_{0})) ≤ C_{1}ε for all sets D(j_{0},j′_{0}),

where μ^{|V_{1}(γ)|+|V_{2}(γ)|}denotes the direct product of the measure μ on some copies of the original space (X,X ) indexed by (1,j) ∈ V_{1}(γ) and (2,j′) ∈ V_{2}(γ). To see this relation one has to observe that ∑_{A_{s}^{ε}=B_{t}^{ε}} μ(A_{s}^{ε})μ(B_{t}^{ε}) ≤ ∑ εμ(A_{s}^{ε}) = εμ(A). Thus the

set D(j_{0},j′_{0}) can be covered by the direct product of a set whose μ measure is not greater than εμ(A) and of a rectangle whose edges are either the set A or the set B.

The above relations imply that

μ^{|V_{1}(γ)|+|V_{2}(γ)|}(D_{γ}) ≤ C_{2}ε  (B.10)

with some constant C_{2} > 0.

Relation (B.9), estimate (B.10), the property c) formulated in Theorem 10.1 for Wiener–Itô integrals and the observation that the function F_{γ}(f,g) is bounded in supremum norm if f and g are elementary functions imply the inequality

E ( |γ|!Z_{μ,|γ|}(F_{γ}(f,g)) − Z_{γ}^{(1)}(f,g,ε)) ^{2}
= |γ!|^{2}E (Z_{μ,|γ|}(F_{γ}(f,g) − F_{γ}^{ε}(f,g))) ^{2} ≤ |γ|!∥F_{γ}(f,g) − F_{γ}^{ε}(f,g)∥_{2}^{2}
≤ Kμ^{|V_{1}(γ)|+|V_{2}(γ)|}(D_{γ}) ≤ Cε.

Hence relation (B.7) holds.

To prove relation (B.8) we rewrite E ( Z_{γ}^{(2)}(f,g,ε))^{2} in the following form:

E ( Z_{γ}^{(2)}(f,g,ε))^{2} = ∑ ^{γ}∑ ^{γ}c^{ε}(s_{1},...,s_{k})d^{ε}(t_{1},...,t_{l})c^{ε}(s̄_{1},...,s̄_{k})
d^{ε}(t̄_{1},...,t̄_{l})EU(s_{1},...,s_{k},t_{1},...,t_{l},s̄_{1},...,s̄_{k},t̄_{1},...,t̄_{l})

(B.11)

U(s_{1},...,s_{k},t_{1},...,t_{l},s̄_{1},...,s̄_{k},t̄_{1},...,t̄_{l})

= ∏_{j: (1,j)∈V_{1}(γ)}μ_{W}(A_{s_{j}}^{ε}) ∏_{j′: (2,j′)∈V_{2}(γ)}μ_{W}(B_{t_{j′}}^{ε})

∏_{j̄: (1,j̄)∈V_{1}(γ)}μ_{W}(Aε ; s̄j̄) ∏_{j̄′: (2,j̄′)∈V_{2}(γ)}μ_{W}(Bε ; t̄j̄′)

[ ∏_{j: (1,j)∈{(1,1),...,(1,k)}\V_{1}(γ)} μ_{W}(A_{s_{j}}^{ε})  ∏_{j′: (2,j′)∈{(2,1),...,(2,l)}\∈V_{2}(γ)} μ_{W}(B_{t_{j′}}^{ε})

− ∏_{j′: (2,j′)∈{(2,1),...,(2,l)}\V_{2}(γ)}μ(B_{t_{j′}}^{ε})]

[ ∏_{j̄: (1,j̄)∈{(1,1),...,(1,k)}\V_{1}(γ)} μ_{W}(Aε ; s̄j̄)  ∏_{j̄′: (2,j̄′)∈{(2,1),...,(2,l)}\∈V_{2}(γ)} μ_{W}(Bε ; t̄j̄′)

−  ∏_{j̄′: (2,j̄′)∈{(2,1),...,(2,l)}\V_{2}(γ)}μ(Bε ; t̄j̄′)].  (B.12)

The double sum ∑^{γ}∑^{γ} in (B.11) has to be understood in the following way. The first summation is taken for vectors (s_{1},...,s_{k},t_{1},...,t_{l}), and ∑^{γ} is defined in the same way as in formula (B.3). The second summation is taken for vectors (s̄_{1},...,s̄_{k},t̄_{1},...,t̄_{l}), and again the summation ∑^{γ} is taken as in (B.3), only here s̄_{j} plays the role of s_{j} and t̄_{j′}plays the role of t_{j′}.

Relation (B.8) will be proved by means of some estimates about the expectation of the above defined random variable U(·) which will be presented in the following Lemma B. To formulate this result I introduce the following Properties A and B.

Property A. A sequence s_{1},...,s_{k},t_{1},...,t_{l},s̄_{1},...,s̄_{k},t̄_{1},...,t̄_{l}, with elements 1 ≤ s_{j},s̄j̄≤ M(ε), for 1 ≤ j,j̄≤ k, and 1 ≤ t_{j},t̄j̄′≤ M′(ε) for 1 ≤ j′,j̄′ ≤ l, satisfies Property A (depending on a fixed diagram γ and number ε > 0) if the sequence of sets A_{s_{j}}^{ε}, (1,j) ∈ V_{1}(γ), B_{t_{j′}}^{ε}, (2,j′) ∈ V_{2}(γ), and the sequence of sets Aε ; s̄j̄, (1,j̄) ∈ V_{1}(γ), Bε ; t̄j̄′, (2,j̄′) ∈ V_{2}(γ), agree. (Here we say that two sequences agree if they contain the same elements in a possibly different order.)

Property B. A sequence s_{1},...,s_{k},t_{1},...,t_{l},s̄_{1},...,s̄_{k},t̄_{1},...,t̄_{l}, with elements 1 ≤ s_{j},s̄j̄≤ M(ε), for 1 ≤ j,j̄≤ k, and 1 ≤ t_{j},t̄j̄′≤ M′(ε) for 1 ≤ j′,j̄′ ≤ l, satisfies Property B (depending on a fixed diagram γ and number ε > 0) if the sequences of sets

A_{s_{j}}^{ε}, (1,j) ∈ {(1,1),...,(1,k)} \ V_{1}(γ), B_{t_{j′}}^{ε}, (2,j′) ∈ {(2,1),...,(2,l)} \ V_{2}(γ),

and

Aε ; s̄j̄,(1,j̄) ∈ {(1,1),...,(1,k)} \ V_{1}(γ), Bε ; t̄j̄′, (2,j̄′) ∈ {(2,1),...,(2,l)} \ V_{2}(γ),

have at least one common element.

(In the above definitions two sets A_{s}^{ε} and B_{t}^{ε} are identified if A_{s}^{ε} = B_{t}^{ε}.)

Now I formulate the following

Lemma B. Let us consider the function U(·) introduced in formula (B.12). Assume that its arguments s_{1},...,s_{k},t_{1},...,t_{l},s̄_{1},...,s̄_{k},t̄_{1},...,t̄_{l} are chosen in such a way that the function U(·) with these arguments appears in the double sum ∑^{γ}∑^{γ} in formula (B.11), i.e. A_{s_{j}}^{ε} = B_{t_{j′}}^{ε}if ((1,j),(2,j′)) ∈ E(γ), otherwise all sets A_{s_{j}}^{ε} and B_{t_{j′}}^{ε} are disjoint, and an analogous statement holds if the coordinates s_{1},...,s_{k},t_{1},...,t_{l} are replaced by s̄_{1},...,s̄_{k} and t̄_{1},...,t̄_{l}.

If the sequence of the arguments in U(·) does not satisfies either Property A or Property B, then

EU(s_{1},...,s_{k},t_{1},...,t_{l},s̄_{1},...,s̄_{k},t̄_{1},...,t̄_{l}) = 0.  (B.13)

If the sequence of the arguments in U(·) satisfies both Property A and Property B, then

|EU(s_{1},...,s_{k},t_{1},...,t_{l},s̄_{1},...,s̄_{k},t̄_{1},...,t̄_{l})| ≤ Cε ∏ ′μ(Aε ; s̄j̄)μ(Bε ; t̄j̄′) (B.14)

with some appropriate constant C = C(k,l) > 0 depending only on the number of variables k and l of the functions f and g. The prime in the product ∏ ′at the righthand side of (B.14) means that in this product the measure μ of those sets Aε ; s̄j̄ and Bε ; t̄j̄′ are considered, whose indices are listed among the arguments s̄j̄or t̄j̄′of U(·), and the measure μ of each such set appears exactly once. (This means that if Aε ; s̄j̄= Bε ; t̄j̄′ then one of the terms between μ(Aε ; s̄j̄) and μ(Bε ; t̄j̄′) is omitted from the product. For the sake of definitiveness let us preserve the set μ(Aε ; s̄j̄) in such a case.)

Remark. The content of Lemma B is that most terms in the double sum in formula (B.11) equal zero, and even the non-zero terms are small.

The proof of Lemma B. Let us prove first relation (B.13) in the case when Property A does not hold. It will be exploited that for disjoint sets the random variables μ_{W}(A_{s}) and μ_{W}(B_{t}) are independent, and this provides a good factorization of the expectation of certain products.

Let us carry out the multiplications in the expression U(·) defined (B.12). We get a sum consisting of 4 terms. We show that each of them has zero expectation. Indeed, if a sequence s_{1},...,s_{k},t_{1},...,t_{l},s̄_{1},...,s̄_{k},t̄_{1},...,t̄_{l}does not satisfy Property A, but it satisfies the remaining conditions of Lemma B, then each term in the sum expressing U(··· ) with these arguments is a product which contains a factor μ_{W}(A_{s_{j_{0}}}^{ε}), (1,j_{0}) ∈ V_{1}(γ) with the following property. It is independent of all those terms in this product which are in the following list: μ_{W}(A_{s_{j}}^{ε}) with some j ≠ j_{0}, 1 ≤ j ≤ k, or μ_{W}(B_{t_{j′}}^{ε}), 1 ≤ j ≤ l, or μ_{W}(Aε ; s̄j̄) with (1,j̄) ∈ V_{1}(γ), or μ_{W}(Bε ; t̄j̄′) with (2,j̄′) ∈ V_{2}(γ). We will show with the help of this property that the expectation of the terms we consider can be written in the form of a product either with a factor of the form Eμ_{W}(A_{s_{j_{0}}}^{ε}) = 0 or with a factor of the form Eμ_{W}(A_{s_{j_{0}}}^{ε})^{3} = 0. Hence this expectation equals zero.

Indeed, although the above properties do not exclude the existence of a set A_{tj̄′}^{ε}, (1,j̄′) ∈ {(1,1),...,(1,k)\V_{1}(γ) or B_{tj̄′}^{ε}, (2,j̄′) ∈ {(2,1),...,(2,l)}\V_{2}(γ) such that μ_{W}(A_{tj̄′}^{ε}) or μ_{W}(B_{tj̄′}^{ε}), is not independent of μ_{W}(A_{s_{j_{0}}}^{ε}), but this can only happen if A_{tj̄}^{ε}= B_{tj̄′}^{ε} = A_{s_{j_{0}}}^{ε}. This implies that in such a case when our term does not contain a

factor of the form Eμ_{W}(A_{s_{j_{0}}}^{ε}), then it contains a factor of the form Eμ_{W}(A_{s_{j_{0}}}^{ε})^{3} = 0. Hence EU(·) = 0 if the arguments of U(·) do not satisfy Property A.

To finish the proof of relation (B.13) it is enough consider the case when the arguments of U(·) satisfy Property A, but they do not satisfy Property B. The validity of Property A implies that the sets {A_{s_{j}}^{ε}, j ∈ V_{1}(γ)} ∪ {B_{t_{j′}}^{ε}, j′ ∈ V_{2}(γ)} and {Aε ; s̄_{j}, j ∈ V_{1}(γ)} ∪ {Bε ; t̄_{j′}, j′∈ V_{2}(γ)} agree. The conditions of Lemma B also imply that the elements of these sets are disjoint of the sets A_{s_{j}}^{ε}, B_{t_{j′}}^{ε}, Aε ; s̄j̄ and Bε ; t̄j̄′ with indices (1,j),(1,j̄) ∈ {(1,1),...,(1,k)} \ V_{1}(γ) and (2,j′),(2,j̄′) ∈ {(2,1),...,(2,l)} \ V_{2}(γ). If Property B does not hold, then we can divide the latter class of sets into two disjoint subclasses in an appropriate way. The first subclass consists of the sets A_{s_{j}}^{ε} and B_{t_{j′}}^{ε}, and the second one of the sets Aε ; s̄j̄ and Bε ; t̄j̄′

with indices such that (1,j),(1,j̄) ∈ {(1,1),...,(1,k)} \ V_{1}(γ) and (2,j′),(2,j̄′) ∈ {(2,1),...,(2,l)} \ V_{2}(γ). These facts imply that EU(·) has a factorization, which contains the term

E[ ∏_{j: (1,j)∈{(1,1),...,(1,k)}\V_{1}(γ)}μ_{W}(A_{s_{j}}^{ε})  ∏_{j′: (2,j′)∈{(2,1),...,(2,l)}\∈V_{2}(γ)} μ_{W}(B_{t_{j′}}^{ε})

− ∏_{j′: (2,j′)∈{(2,1),...,(2,l)}\V_{2}(γ)}μ(B_{s_{j′}}^{ε})] = 0,

hence relation (B.13) holds also in this case. The last expression has zero expectation, since if we take such pairs A_{s_{j}}^{ε},B_{t′_{j}}^{ε} for the sets appearing in it for which

that ((1,j),(2,j′)) ∈ E(γ), i.e. these vertices are connected with an edge of γ, then A_{s_{j}}^{ε} = B_{t′_{j}}^{ε}in a pair, and elements in different pairs are disjoint. This observation allows a factorization in the product whose expectation is taken, and then the identity Eμ_{W}(A_{s_{j}}^{ε})μ_{W}(B_{t_{j′}}^{ε}) = μ(A_{s_{j}}^{ε}) implies the desired identity.

To prove relation (B.14) if the arguments of the function U(·) satisfy both Properties A and B consider the expression (B.12) which defines U(·), carry out the term by term multiplication between the two differences at the end of this formula, take expectation for each term of the sum obtained in such a way and factorize them. Since Eμ_{W}(A)^{2} = μ(A), Eμ_{W}(A)^{4} = 3μ(A)^{2} for all sets A ∈ X , μ(A) < ∞, some calculation shows that each term can be expressed as constant times a product whose elements are those probabilities μ(Aε ; s̄j̄) and μ(Bε ; t̄j̄′) or their square which appear at the right-hand side of (B.14). Moreover, since the arguments of U(·) satisfy Property B, there will be at least one term of the form μ(A_{s}^{ε})^{2} in this product. Since μ(A_{s}^{ε})^{2} ≤ εμ(A_{s}^{ε}), these calculations provide formula (B.14). Lemma B is proved.

Relation (B.11) implies that

E ( Z_{γ}^{(2)}(f,g,ε))^{2} ≤ K ∑ ^{γ}∑ ^{γ}|EU(s_{1},...,s_{k},t_{1},...,t_{l},s̄_{1},...,s̄_{k},t̄_{1},...,t̄_{l})|(B.15)

with some appropriate K > 0. By Lemma B it is enough to sum up only for such terms U(·) in (B.15) whose arguments satisfy both Properties A and B. Moreover, each such term can be bounded by means of inequality (B.14). Let us write up the

upper bound we get on E ( Z_{γ}^{(2)}(f,g,ε))^{2} in such a way. We get a sum consisting of terms of the form μ(A_{s_{1}}^{ε})··· μ(A_{s_{p}}^{ε})μ(B_{t_{1}}^{ε})··· μ(Bε ; t̄_{q}) multiplied by constant times ε. The sets A_{s}^{ε} and B_{t}^{ε} whose measure μ appears in such a term are disjoint. Beside this 1 ≤ p ≤ k, and 1 ≤ q ≤ l.

In the above indicated estimation of E ( Z_{γ}^{(2)}(f,g,ε))^{2} with the help of formula (B.15) and Lemma B we have exploited the following fact. A term

μ(A_{s_{1}}^{ε})··· μ(A_{s_{p}}^{ε})μ(B_{t_{1}}^{ε})··· μ(Bε ; t̄_{q})

with prescribed indices s_{1},...,s_{p} and t_{1},...,t_{q} came up in the sum at the right-hand of our bound as a contribution of only finitely many expressions |EU(··· )|. Hence we get this term in the upper bound with a multiplying coefficient bounded by constant times ε.

We also have ∑_{s=1}^{M(ε)}μ(A_{s}^{ε}) + ∑_{t=1}^{M′(ε)}μ(B_{t}^{ε}) = μ(A) + μ(B) < ∞. The above relations

imply that

E ( Z_{γ}^{(2)}(f,g,ε))^{2} ≤ C_{1}ε ∑_{\begin{matrix}1≤p≤k \\ 1≤q≤l \end{matrix}} ∑_{\begin{matrix}1≤s_{l}≤M \\ 1≤l≤p \end{matrix}} ∑_{\begin{matrix}1≤t_{l}≤M′ \\ 1≤l≤q \end{matrix}} μ(A_{s_{1}}^{ε})··· μ(A_{s_{p}}^{ε})μ(B_{t_{1}}^{ε})··· μ(Bε ; t̄_{q})

≤ C_{2}ε ∑_{j=1}^{(k+l)}(μ(A) + μ(B))^{j} ≤ Cε.

Hence relation (B.8) holds.

To prove Theorem 10.2A in the general case take for all pairs of functions f ∈ H_{μ,k} and g ∈ H_{μ,l} two sequences of elementary functions f_{n}∈ H̄_{μ,k} and g_{n}∈ H̄_{μ,l} , n = 1,2,..., such that ∥f_{n}− f∥_{2} → 0 and ∥g_{n}− g∥_{2} → 0 as n → ∞. It is enough to show that

E|k!Z_{μ,k}(f)l!Z_{μ,l}(g) − k!Z_{μ,k}(f_{n})l!Z_{μ,l}(g_{n})| → 0 as n → ∞,  (B.16)

and

|γ|!E |Z_{μ,|γ|}(F_{γ}(f,g)) − Z_{μ,|γ|}(F_{γ}(f_{n},g_{n}))| → 0 as n → ∞
for all γ ∈ Γ (k,l),  (B.17)

since then a simple limiting procedure n → ∞, and the already proved part of the theorem for Wiener–Itô integrals of elementary functions imply Theorem 10.2A.

To prove relation (B.16) write with the help of Property c) in Theorem (10.1)

E|k!Z,μ,k(f)l!Z_{μ,l}(g) − k!Z_{μ,k}(f_{n})l!Z_{μ,l}(g_{n})|
≤ k!l!(E|Z_{μ,k}(f)Z_{μ,l}(g − g_{n})| + E|Z_{μ,k}(f − f_{n})Z_{μ,l}(g_{n})) |
≤ k!l!( ( EZ_{μ,k}^{2}(f))^{1/2}( EZ_{μ,l}^{2}(g − g_{n})) ^{1/2}
+( EZ_{μ,k}^{2}(f − f_{n})) ^{1/2}( EZ_{μ,l}^{2}(g_{n}))^{1/2})
≤ (k!l!)^{1/2}(∥f∥_{2}∥g − g_{n}∥_{2}+ ∥f − f_{n}∥_{2}∥g_{n}∥_{2}) .

Relation (B.16) follows from this inequality with a limiting procedure n → ∞. To prove relation (B.17) write

|γ|!E |Z_{μ,|γ|}(F_{γ}(f,g)) − Z_{μ,|γ|}(F_{γ}(f_{n},g_{n}))|

≤ |γ|!E |Z_{μ,|γ|}(F_{γ}(f,g − g_{n}))| + |γ|!E |Z_{μ,|γ|}(F_{γ}(f − f_{n},g_{n}))|

≤ |γ|!( EZ_{μ,|γ|}^{2}(F_{γ}(f,g − g_{n}))) ^{1/2}+ |γ|!( EZ_{μ,|γ|}^{2}(F_{γ}(f − f_{n},g_{n}))) ^{1/2}

≤ (|γ|!)^{1/2}(∥F_{γ}(f,g − g_{n})∥_{2}+ ∥F_{γ}(f − f_{n},g_{n})∥_{2}) ,

and observe that by relation (10.13) ∥F_{γ}(f,g − g_{n})∥_{2} ≤ ∥f∥_{2}∥g − g_{n}∥_{2}, and ∥F_{γ}(f − f_{n},g_{n})∥_{2} ≤ ∥f − f_{n}∥_{2}∥g_{n}∥_{2}. Hence

|γ|!E |Z_{μ,|γ|}(F_{γ}(f,g)) − Z_{μ,|γ|}(F_{γ}(f_{n},g_{n}))|
≤ (|γ|!)^{1/2}(∥f∥_{2}∥g − g_{n}∥_{2}+ ∥f − f_{n}∥_{2}∥g_{n}∥_{2}) .

The last inequality implies relation (B.17) with a limiting procedure n → ∞. Theorem 10.2A is proved.

Appendix C

The proof of some results about Wiener–Itô integrals

First I prove Itô’s formula about multiple Wiener–Itô integrals (Theorem 10.3). The proof is based on the diagram formula for Wiener–Itô integrals and a recursive formula about Hermite polynomials proved in Proposition C. In Proposition C2 I present the proof of another important property of Hermite polynomials. This result states that the class of all Hermite polynomials is a complete orthogonal system in an appropriate Hilbert space. It is needed in the proof of Theorem 10.5 which provides an isomorphism between a Fock space and the Hilbert space generated by Wiener– Itô integrals with respect to a white noise with an appropriate reference measure. At the end of Appendix C the proof of Theorem 10.4, a limit theorem about degenerate U-statistics is given together with a version of this result about the limit behaviour of multiple integrals with respect to a normalized empirical distribution.

Proposition C about some properties of Hermite polynomials. The functions

H_{k}(x) = (−1)^{k}e^{x^{2}/2} \frac{d^{k}}{dx^{k}}e^{−x^{2}/2}, k = 0,1,2,...  (C.1)

are the Hermite polynomials with leading coefficient 1, i.e. H_{k}(x) is a polynomial of order k with leading coefficient 1 such that

∫_{−∞}^{∞} H_{k}(x)H_{l}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = 0 ifk ≠ l.  (C.2)

Beside this,

∫_{−∞}^{∞} H_{k}^{2}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = k! for all k = 0,1,2....  (C.3)

The recursive relation

H_{k}(x) = xH_{k−1}(x) − (k − 1)H_{k−2}(x)  (C.4)

holds for all k = 1,2,....

Remark. It is more convenient to consider relation (C.4) valid also in the case k = 1. In this case H_{1}(x) = x, H_{0}(x) = 1, and relation holds with an arbitrary function H_{−1} (x).

Proof of Proposition C. It is clear from formula (C.1) that H_{k}(x) is a polynomial of order k with leading coefficient 1. Take l ≥ k, and write by means of integration by parts

∫_{−∞}^{∞} H_{k}(x)H_{l}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = ∫_{−∞}^{∞} \frac{1}{\sqrt{2π}} H_{k}(x)(−1)^{l} \frac{d^{l}}{dx^{l}}e^{−x^{2}/2} dx

= ∫_{−∞}^{∞} \frac{1}{\sqrt{2π}} \frac{d}{dx}H_{k}(x)(−1)^{l−1} \frac{d^{l−1}}{dx^{l−1}} e^{−x^{2}/2} dx.

Successive partial integration together with the identity \frac{d^{k}}{dx^{k}}H_{k}(x) = k! yield that

∫_{−∞}^{∞} H_{k}(x)H_{l}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = k!∫_{−∞}^{∞} \frac{1}{\sqrt{2π}} (−1)^{l−k} \frac{d^{l−k}}{dx^{l−k}}e^{−x^{2}/2} dx.

The last relation supplies formulas (C.2) and (C.3).

To prove relation (C.4) observe that H_{k}(x) − xH_{k−1}(x) is a polynomial of order k−2. (The term x^{k−1} is missing from this expression. Indeed, if k is an even number, then the polynomial H_{k}(x) − xH_{k−1}(x) is an even function, and it does not contain the term x^{k−1} with an odd exponent k −1. Similar argument holds if the number k is odd.) Beside this, it is orthogonal (with respect to the standard normal distribution) to all Hermite polynomials H_{l}(x) with 0 ≤ l ≤ k − 3. Hence H_{k}(x) − xH_{k−1}(x) = CH_{k−2}(x) with some constant C to be determined.

Multiply both sides of the last identity with H_{k−2}(x) and integrate them with respect to the standard normal distribution. Apply the orthogonality of the polynomials H_{k}(x) and H_{k−2}(x), and observe that the identity

∫ H_{k−1}(x)xH_{k−2}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = ∫ H_{k−1}^{2}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = (k − 1)!

holds. (In this calculation we have exploited that H_{k−1}(x) is orthogonal to H_{k−1}(x)− xH_{k−2}(x), because the order of the latter polynomial is less than k − 1.) In such a way we get the identity −(k − 1)! = C(k − 2)! for the constant C in the last identity, i.e. C = −(k − 1), and this implies relation (C.4).

Proof of Itô’s formula for multiple Wiener–Itô integrals. Let K = ∑_{p=1}^{m} k_{p}, the sum of

the order of the Hermite polynomials, denote the order of the expression in relation (10.24). Formula (10.24) clearly holds for expressions of order K = 1. It will be proved in the general case by means of induction with respect to the order K.

In the proof the functions f(x_{1}) = φ_{1}(x_{1}) and

g(x_{1},...,x_{K_{m}−1}) = ∏_{j=1}^{K_{1}−1} φ_{1}(x_{j}) · ∏_{p=2}^{m} ∏_{j=K_{p−1}}^{K_{p}−1} φ_{p}(x_{j}),

will be introduced and the product Z_{μ,1}(f)(K_{m}− 1)!Z_{μ,K_{m}−1} (g) will be calculated by means of the diagram formula. (The same notation is applied as in Theorem

10.3. In particular, K = K_{m}, and in the case K_{1} = 1 the convention ∏_{j=1}^{K_{1}−1} φ_{1}(x_{j}) =

1 is applied.) In the application of the diagram formula diagrams with two rows appear. The first row of these diagrams contains the vertex (1,1) and the second row contains the vertices (2,1),...,(2,K_{m}− 1). It is useful to divide the diagrams to three disjoint classes. The first class, Γ_{0} contains only the diagram γ_{0} without any edges. The second class Γ_{1} consists of those diagrams which have an edge of the form ((1,1),(2,j)) with some 1 ≤ j ≤ k_{1}−1, and the third class Γ_{2} is the set of those diagrams which have an edge of the form ((1,1),(2,j)) with some k_{1} ≤ j ≤ K_{m}−1. Because of the orthogonality of the functions φ_{s} for different indices s F_{γ} ≡ 0 and Z_{μ,K_{m}−2} (F_{γ}) = 0 for γ ∈ Γ_{2}. The class Γ_{1} contains k_{1}− 1 diagrams. Let us consider a diagram γ from this class with an edge ((1,1),(2,j_{0})), 1 ≤ j ≤ k_{1} − 1. We have

for such a diagram F_{γ} = ∏_{j∈{1,...,K_{1}−1}\{j_{0}}} φ_{1}(x_{(2,j)}) ∏_{p=2}^{m} ∏_{j=K_{p−1}}^{K_{p}−1} φ_{p}(x_{(2,j)}), and by our

inductive hypothesis (K_{m}− 2)!Z_{μ,K_{m}−2} (F_{γ}) = H_{k_{1}−2}(η_{1}) ∏_{p=2}^{m} H_{k_{p}}(η_{p}). Finally

K_{m}!Z_{μ,K_{m}}(F_{γ_{0}}) = K_{m}!Z_{μ,K_{m}}( ∏_{p=1}^{m} ( ∏_{j=K_{p−1} +1}^{K_{p}} φ_{p}(x_{j})) )

for the diagram γ_{0} ∈ Γ_{0} without any edge.

Our inductive hypothesis also implies the following identity for the expression we wanted to calculate with the help of the diagram formula.

Z_{μ,1}(f)(K_{m}− 1)!Z_{μ,K_{m}−1} (g) = η_{1}H_{k_{1}−1}(η_{1}) ∏_{p=2}^{m} H_{k_{p}}(η_{p}).

The above calculations together with the observation |Γ_{1}| = k_{1}−1 yield the identity

K_{m}!Z_{μ,K_{m}}( ∏_{p=1}^{m} ( ∏_{j=K_{p−1} +1}^{K_{p}} φ_{p}(x_{j})) ) = K_{m}!Z_{μ,K_{m}}(F_{γ_{0}})

= Z_{μ,1}(f)(K_{m}− 1)!Z_{μ,K_{m}−1} (g) − ∑_{γ∈Γ_{1}}(K_{m}− 2)!Z_{μ,K_{m}−2} (F_{γ})

= η_{1}H_{k_{1}−1}(η_{1}) ∏_{p=2}^{m} H_{k_{p}}(η_{p}) − (k_{1}− 1)H_{k_{1}−2}(η_{1}) ∏_{p=2}^{m} H_{k_{p}}(η_{p})

= [η_{1}H_{k_{1}−1}(η_{1}) − (k_{1}− 1)H_{k_{1}−2}(η_{1})] ∏_{p=2}^{m} H_{k_{p}}(η_{p}).  (C.5)

On the other hand, η_{1}H_{k_{1}−1}(η_{1}) − (k_{1} − 1)H_{k_{1}−2}(η_{1}) = H_{k_{1}}(η_{1}) by formula (C.4). These relations imply formula (10.24), i.e. Itô’s formula.

I present the proof of another important property of the Hermite polynomials in the following Proposition C2.

Proposition C2 on the completeness of the orthogonal system of Hermite polynomials. The Hermite polynomials H_{k}(x), k = 0,1,2,..., defined in formula (C.5) constitute a complete orthonormal system in the L_{2}-space of the functions square integrable with respect to the Gaussian measure \frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx on the real line.

Proof of Proposition C2. Let us consider the orthogonal complement of the subspace generated by the Hermite polynomials in the space of the square integrable functions with respect to the measure \frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx. It is enough to prove that this orthogonal completion contains only the identically zero function. Since the orthogonality of a function to all polynomials of the form x^{k}, k = 0,1,2,... is equivalent to the orthogonality of this function to all Hermite polynomials H_{k}(x), k = 0,1,2,..., Proposition C2 can be reformulated in the following form:

If a function g(x) on the real line is such that

∫_{−∞}^{∞} x^{k}g(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx = 0 for all k = 0,1,2,...  (C.6)

and

∫_{−∞}^{∞} g^{2}(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx < ∞,  (C.7)

then g(x) = 0 for almost all x.

Given a function g(x) on the real line whose absolute value is integrable with respect to the Gaussian measure \frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx define the (finite) measure ν_{g},

ν_{g}(A) = ∫_{A}g(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx

on the measurable sets of the real line together with its Fourier transform ν̃_{g}(t) = ∫_{−∞}^{∞} e^{itx}ν_{g}(dx). (This measure ν_{g} and its Fourier transform can be defined for all functions g satisfying relation (C.7), because their absolute value is integrable with respect to the Gaussian measure.) First I show that Proposition C2 can be reduced to the following statement: If a function g satisfies both (C.6) and (C.7) then ν̃_{g}(t) = 0 for all −∞ < t < ∞.

Indeed, if there were a function g satisfying (C.6) and (C.7) which is not identically zero, then the non-negative functions g^{+} (x) = max(0,g(x)) and g^{−} (x) = −min(0,g(x)) would be different. Then also their Fourier transform ν̃_{g^{+}} (t) and ν̃_{g^{−}} (t) would be different, since a finite measure is uniquely determined by its Fourier transform. (This statement is equivalent to an important result in probability theory, by which a probability measure on the real line is determined by its characteristic function.) But this would mean that ν̃_{g}(t) = ν̃_{g^{+}} (t) − ν̃_{g^{−}} (t) ≠ 0 for some t. Hence Proposition C2 can be reduced to the above statement.

Since |e^{itx}− 1 − (itx) − ··· − \frac{(itx)^{k}}{k!}| ≤ \frac{|tx|^{(k+1)}}{(k+1)!} for all real numbers t, x and integer k = 1,2,... we may write because of relation (C.6)

|ν̃_{g}(t)| = |∫_{−∞}^{∞} ( e^{itx}− 1 − (itx) − ··· − \frac{(itx)^{k}}{k!}) g(x)\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx|

≤ ∫_{−∞}^{∞} \frac{|t|^{(k+1)}}{(k + 1)!} |x|^{k+1}|g(x)|\frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx

for all k = 1,2,... and real number t if the function g satisfies relation (C.6). If it satisfies both relation (C.6) and (C.7), then from the last relation and the Schwarz inequality

|ν̃_{g}(t)|^{2} ≤ const. \frac{|t|^{2(k+1)}}{(k + 1)!)^{2}} ∫_{−∞}^{∞} |x|^{2(k+1)} \frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx

= const. \frac{|t|^{2(k+1)}}{(k + 1)!)^{2}} 1 · 3 · 5··· (2k + 1)

for all real number t and integer k = 1,2,.... Simple calculation shows that the righthand side of the last estimate tends to zero as k → ∞. This implies that ν̃_{g}(t) = 0 for all t, and Proposition C2 holds.

I finish Appendix C with the proof of Theorem 10.4, a limit theorem about a sequence of normalized degenerate U-statistics. It is based on an appropriate representation of the U-statistics by means of multiple random integrals which makes possible to carry out an appropriate limiting procedure.

Proof of Theorem 10.4. For all n = 1,2,..., the normalized degenerate U-statistics n^{−k/2}k!I_{n,k}(f) can be written in the form

n^{−k/2}k!I_{n,k}(f) = n^{k/2}∫ ′f(x_{1},...,x_{k})μ_{n}(dx_{1})...μ_{n}(dx_{k})  (C.8)
= n^{k/2}∫ ′f(x_{1},...,x_{k})(μ_{n}(dx_{1}) − μ(dx_{k}))...(μ_{n}(dx_{k}) − μ(dx_{1})),

where μ_{n} is the empirical distribution of the sequence ξ_{1},...,ξ_{n} defined in (4.5), and the prime in ∫′denotes that the diagonals, i.e. the points x = (x_{1},...,x_{k}) such that x_{j} = x_{j′} for some pairs of indices 1 ≤ j,j′≤ k, j ≠ j′, are omitted from the domain of integration. The second identity in relation (C.8) can be justified by means of the identity

∫ ′f(x_{1},...,x_{k})(μ_{n}(dx_{1}) − μ(dx_{1}))...(μ_{n}(dx_{k}) − μ(dx_{k})) − I_{n,k}(f)

= ∑_{V : V ∈{1,...,k},|V |≥1}(−1)^{|V |}∫ ′f(x_{1},...,x_{k})

∏_{j∈V} μ(dx_{j}) ∏_{j∈{1,...,k}\V} μ_{n}(dx_{j})) = 0.  (C.9)

This identity holds for a function f canonical with respect to a non-atomic measure μ, because each term in the sum at the right-hand side of (C.9) equals zero. Indeed, the integral of a canonical function f with respect to μ(dx_{j}) with some index j ∈ V equals zero for all fixed values x_{1},...,x_{j−1} ,x_{j+1} ,...,x_{k}. The non-atomic property of the measure μ was needed to guarantee that this integral equals zero also in the case when the diagonals are omitted from the domain of integration.

We would like to derive Theorem 10.4 from relation (C.8) by means of an appropriate limiting procedure which exploits the convergence of the random fields n^{1/2}(μ_{n}(A) − μ(A)), A ∈ X , to a Gaussian field ν(A), A ∈ X , as n → ∞. But some problems arise if we want to carry out such a program, because the fields n^{1/2}(μ_{n}− μ) converge to a non white noise type Gaussian field. The limit we get is similar to a Wiener bridge on the real line. Hence a relation between Wiener processes and Wiener bridges suggests to write the following version of formula (C.8).

Let us take a standard Gaussian random variable η, independent of the random sequence ξ_{1},ξ_{2},.... For a canonical function f the following version of (C.8) holds.

n^{−k/2}k!I_{n,k}(f) = J′_{n,k}(f)  (C.10)

with

J′_{n,k}(f) = ∫ ′ f(x_{1},...,x_{k})[\sqrt{n}(μ_{n}(dx_{1}) − μ(dx_{1})) + ημ(dx_{1})]
...[\sqrt{n}(μ_{n}(dx_{k}) − μ(dx_{k})) + ημ(dx_{k})] .  (C.11)

This relation can be seen similarly to (C.8).

The random measures n^{1/2}(μ_{n}− μ) + ημ converge to a white noise with reference measure μ. Hence Theorem 10.4 can be proved by means of formulas (C.10) and (C.11) with the help of an appropriate limiting procedure. More explicitly, I claim that the following slightly more general result holds. The expressions J′_{n,k}(f) introduced in (C.11) converge in distribution to the Wiener–Itô integral k!Z_{μ,k}(f) as n → ∞ for all functions f square integrable with respect to the product measure μ^{k}. This result also holds for non-canonical functions f. This limit theorem together with relation (C.10) imply Theorem 10.4.

The convergence of the random variables J′_{n,k}(f) defined in (C.11) to the Wiener– Itô integral k!Z_{μ,k}(f) can be easily checked for elementary functions f ∈ H̄_{μ,k} . Indeed, if A_{1},...,A_{M} are disjoint sets with μ(A_{s}) < ∞, then the multi-dimensional central limit theorem implies that the random vectors \{\sqrt{n}((μ_{n}(A_{s}) − μ(A_{s})) + ημ(A_{s}), 1 ≤ s ≤ M} converge in distribution to the random vector {(μ_{W}(A_{s}), 1 ≤ s ≤ M}, i.e. to a set of independent normal random variables ζ_{s}, Eζ_{s} = 0, 1 ≤ s ≤ M, with variance Eζ_{s}^{2} = μ(A_{s}) as n → ∞. The definition of the elementary functions given in (10.2) shows that this central limit theorem implies the demanded convergence of the sequence J′_{n,k}(f) to k!Z_{μ,k}(f) for elementary functions.

To show the convergence of the sequence J′_{n,k}(f) to k!Z_{μ,k}(f) in the general case take for any function f ∈ H_{μ,k} a sequence of elementary functions f_{N} ∈ H̄_{μ,k} such that ∥f − f_{N}∥_{2} → 0 as N → ∞. Then E(Z_{μ,k}(f) − Z_{μ,k}(f_{N}))^{2} = E(Z_{μ,k}(f − f_{N}))^{2} → 0 as N → ∞ by Property c) in Theorem 10.1. Hence the already proved part of the theorem implies that there exists some sequence of positive integers, N(n), n = 1,2,..., in such a way that N(n) → ∞, and the sequence J′_{n,k}(f_{N(n)}) converges to k!Z_{μ,k}(f) in distribution as n → ∞. Thus to complete the proof of Theorem 10.4 it is enough to show that E(J′_{n,k}(f_{N(n)}) − J′_{n,k}(f))^{2} = E(J′_{n,k}(f_{N(n)}− f))^{2} → 0 as n → ∞.

It is enough to show that

E(J′_{n,k}(f))^{2} ≤ C∥f∥_{2}^{2} for all f ∈ H_{μ,k}  (C.12)

with a constant C = C_{k} depending only on the order k of the function f and to apply inequality (C.12) for the functions f_{N(n)}− f. Relation (C.12) is a relatively simple consequence of Corollary 1 of Theorem 9.4.

Indeed,

J′_{n,k}(f) = ∑_{V ⊂{1,...,k}}η^{k−|V |}|V |!J_{n,|V |}(f_{V})

with

f_{V}(x_{j}, j ∈ V) = ∫ f(x_{1},...,x_{k}) ∏_{j′∈{1,...,k}\V} μ(dx_{j′})

and the random integral J_{n,k}(·) defined in (4.8), hence

E(J′_{n,k}(f))^{2} ≤ 2^{k} ∑_{V ⊂{1,...,k}}(|V |!)^{2}Eη^{2(k−|V |)}· EJ_{n,|V |}^{2}(f_{V}).  (C.13)

Inequality ∥f_{V}∥_{2} ≤ ∥f∥_{2} holds for all sets V ⊂ {1,...,k}, hence an application of Corollary 1 of Theorem 9.4 to all random integrals J_{n,|V |}(f) supplies (C.12).

The above proof also yields the following slight generalization of Theorem 10.4. Let us consider a finite sequence of functions f_{j} ∈ H_{μ,j}, 1 ≤ j ≤ k, canonical with respect to a non-atomic probability measure μ. The vectors {n^{−j/2} I_{n,j}(f_{j}),1 ≤ j ≤ k}, consisting of normalized degenerate U-statistics defined with the help of a sequence of independent μ-distributed random variables converge to the random vector {Z_{μ,j} (f_{j}),1 ≤ j ≤ k} in distribution as n → ∞. This result together with Theorem 9.4 imply the following limit theorem about multiple random integrals J_{n,k}(f).

Theorem 10.4′(Limit theorem about multiple random integrals with respect to a normalized empirical measure). Let a sequence of independent and identically distributed random variables ξ_{1},ξ_{2},... be given with some non-atomic distribution μ on a measurable space (X,X ) together with a function f(x_{1},...,x_{k}) on the k-fold product (X^{k},X^{k}) of the space (X,X ) such that

∫ f^{2}(x_{1},...,x_{k})μ(dx_{1})...μ(dx_{k}) < ∞.

Let us consider for all n = 1,2,... the random integrals J_{n,k}(f) of order k defined in formulas (4.5) and (4.8) with the help of the empirical distribution μ_{n} of the sequence ξ_{1},...,ξ_{n} and the function f. These random integrals J_{n,k}(f) converge in distribution, as n → ∞, to the following sum U(f) of multiple Wiener–Itô integrals:

U(f) = ∑_{V ⊂{1,...,k}}C(k,V)Z_{μ,|V |}(f_{V})

= ∑_{V ⊂\{1,...,k\}} \frac{C(k,V)}{|V |!}∫ f_{V}(x_{j}, j ∈ V) ∏_{j∈V} μ_{W}(dx_{j}),

where the functions f_{V}(x_{j}, j ∈ V), V ⊂ {1,...,k}, are those functions defined in formula (9.3) which appear in the Hoeffding decomposition of the function f(x_{1},...,x_{k}), the constants C(k,V) are the limits appearing in the limit relation lim_{n→∞} C(n,k,V) = C(k,V) satisfied by the coefficients C(n,k,V) in formula (9.15), and μ_{W} is a white noise with reference measure μ.

An essential step of the proof of Theorem 10.4 was the reduction of the case of general kernel functions to the case of elementary kernel functions. Let me make some comments about it.

It would be simple to make such a reduction if we had a good approximation of a canonical function with such elementary functions which are also canonical. But it is very hard to find such an approximation. To overcome this difficulty we reduced the proof of Theorem 10.4 to a modified version of this result where instead of a limit theorem for degenerate U-statistics a limit theorem for the random variables J′_{n,k}(f) introduced in formula (C.11) has to be proved. In the proof of such a version we could apply the approximation of a general kernel function with not necessarily canonical elementary functions. Theorem 9.4 helped us to work with such an approximation. Another natural way to overcome the above difficulty is to apply a Poissonian approximation of the normalized empirical measure. Such an approach was applied in [16] and in [34], where some generalizations of Theorem 10.4 were proved.

Appendix D

The proof of Theorem 14.3 about U-statistics and decoupled U-statistics

The proof of Theorem 14.3. It will be simpler to formulate and prove a generalized version of Theorem 14.3 where such generalized U-statistics are considered in which different kernel functions may appear in each term of the sum. More explicitly, let ℓ = ℓ(n,k) denote the set of all such sequences l = (l_{1},...,l_{k}) of integers of length k for which 1 ≤ l_{j} ≤ n, 1 ≤ j ≤ k. To define generalized U-statistics let us fix a set of functions {f_{l_{1},...,l_{k}}(x_{1},...,x_{k}), (l_{1},...,l_{k}) ∈ ℓ} which map the space (X^{k},X^{k}) to a separable Banach space B, and have the property f_{l_{1},...,l_{k}}(x_{1},...,x_{k}) ≡ 0 if l_{j} = l_{j′} for some indices j ≠ j′. (The last condition corresponds to that property of Ustatistics that the diagonals are omitted from the summation in their definition.) Let us denote this set of functions by f(ℓ), and define, similarly to the U-statistics and decoupled U-statistics the generalized U-statistics and generalized decoupled U-statistics by the formulas

I_{n,k}(f(ℓ)) = \frac{1}{k!} ∑_{(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k} f_{l_{1},...,l_{k}}(ξ_{l_{1}},...,ξ_{l_{k}})  (D.1)

and

Ī_{n,k}(f(ℓ)) = \frac{1}{k!} ∑_{(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k} f_{l_{1},...,l_{k}}( ξ_{l_{1}}^{(1)},...,ξ_{l_{k}}^{(k)})  (D.2)

(with the same independent and identically distributed random variables ξ_{l} and ξ_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k, as in the definition of the original U-statistics and decoupled U-statistics.)

The following generalization of relation (14.13) will be proved.

P (∥I_{n,k}(f(ℓ))∥ > u) ≤ A(k)P (∥Ī_{n,k}(f(ℓ))∥ > γ(k)u)  (D.3)

with some constants A(k) > 0 and γ(k) > 0 depending only on the order k of these generalized U-statistics. The sign ∥·∥ in (D.3) denotes the norm in the Banach space we are working in.

256 D The proof of Theorem 14.3 about U-statistics and decoupled U-statistics

We concentrate mainly on the proof of the generalization (D.3) of relation (14.13). Formula (14.14) is a relatively simple consequence of it. Formula (D.3) will be proved by means of an inductive procedure which works only in this more general setting. It will be derived from the following statement.

Let us take two independent copies ξ_{1}^{(1)},...,ξ_{n}^{(1)} and ξ_{1}^{(2)},...,ξ_{n}^{(2)} of our original sequence of random variables ξ_{1},...,ξ_{n}, and introduce for all sets V ⊂ {1,...,k} the function α_{V}(j), 1 ≤ j ≤ k, defined as α_{V}(j) = 1 if j ∈ V and α_{V}(j) = 2 if j ∉ V. Let us define with their help the following version of decoupled U-statistics:

I_{n,k,V}(f(ℓ)) = \frac{1}{k!} ∑_{(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k} f_{l_{1},...,l_{k}}( ξ_{l_{1}}^{(α_{V}(1))},...,ξ_{l_{k}}^{(α_{V}(k))})
for all V ⊂ {1,...,k}.  (D.4)

The following inequality will be proved: There are some constants C_{k} > 0 and D_{k} > 0 depending only on the order k of the generalized U-statistic I_{n,k}(f(ℓ)) such that for all numbers u > 0

P (∥I_{n,k}(f(ℓ))∥ > u) ≤ ∑_{V ⊂{1,...,k},1≤|V |≤k−1} C_{k}P (D_{k}∥I_{n,k,V}(f(ℓ))∥ > u) . (D.5)

Here |V | denotes the cardinality of the set V, and the condition 1 ≤ |V | ≤ k − 1 in the summation of formula (D.5) means that the sets V = ∅ and V = {1,...,k} are omitted from the summation, i.e. the terms where either α_{V}(j) = 1 or α_{V}(j) = 2 for all 1 ≤ j ≤ k are not considered. Formula (D.3) can be derived from formula (D.5) by means of an inductive argument. The hard part of the problem is to prove formula (D.5). To do this first we prove the following simple lemma.

Lemma D1. Let ξ and η be two independent and identically distributed random variables taking values in a separable Banach space B. Then

3P ( |ξ + η| > \frac{2}{3}u) ≥ P(|ξ| > u) for all u > 0.

Proof of Lemma D1. Let ξ, η and ζ be three independent, identically distributed random variables taking values in B. Then

3P ( |ξ + η| > \frac{2}{3}u) = P ( |ξ + η| > \frac{2}{3}u) + P ( |ξ + ζ| > \frac{2}{3}u)

+P ( | − (η + ζ)| > \frac{2}{3}u)

≥ P(|ξ + η + ξ + ζ − η − ζ| > 2u) = P(|ξ| > u).

To prove formula (D.5) we introduce the random variable

T_{n,k}(f(ℓ))= \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}), (s_{1},...,s_{k}): \\ 1≤l_{j}≤n,s_{j}=1 or s_{j}=2, j=1,...,k, \end{matrix}}f_{l_{1},...,l_{k}}( ξ_{l_{1}}^{(s_{1})},...,ξ_{l_{k}}^{(s_{k})})

= ∑_{V ⊂{1,...,k}}I_{n,k,V}(f(ℓ)).  (D.6)

The random variables I_{n,k}(f(ℓ)), I_{n,k,∅}(f(ℓ)) and I_{n,k,{1,...,k}}(f(ℓ)) are identically distributed, and the last two random variables are independent of each other. Hence Lemma D1 yields that

P(∥I_{n,k}(f(ℓ))∥ > u) ≤ 3P ( ∥I_{n,k,∅}(f(ℓ)) + I_{n,k,\{1,...,k\}}(f(ℓ))∥ > \frac{2}{3}u)

= 3P ( ∥T_{n,k}(f(ℓ)) − ∑_{V : V ⊂\{1,...,k\},1≤|V |≤k−1} I_{n,k,|V |}(f(ℓ))∥ > \frac{2}{3}u)

≤ 3P(3 · 2^{k−1}∥T_{n,k}(f(ℓ))∥ > u)

+ ∑_{V : V ⊂{1,...,k},1≤|V |≤k−1} 3P(3 · 2^{k−1}∥I_{n,k,|V |}(f(ℓ))∥ > u).  (D.7)

To derive relation (D.5) from relation (D.7) we need a good upper bound on the probability P(3 · 2^{k−1}∥T_{n,k}(f(ℓ))∥ > u). To get such an estimate we shall compare the tail distribution of ∥T_{n,k}(f(ℓ))∥ with that of ∥I_{n,k,V}(f(ℓ))∥ for an arbitrary set V ⊂ {1,...,k}. This will be done with the help of Lemmas D2 and D4 formulated below.

In Lemma D2 such a random variable ∥Î_{n,k,V}(f(ℓ))∥ will be constructed whose distribution agrees with that of ∥I_{n,k,V}(f(ℓ))∥. The expression Î_{n,k,V}(f(ℓ)), whose norm will be investigated will be defined in formulas (D.8) and (D.9). It is a random polynomial of some Rademacher functions ε_{1},...,ε_{n}. The coefficients of this polynomial are random variables, independent of the Rademacher functions ε_{1},...,ε_{n}. Beside this, the constant term of this polynomial equals T_{n,k}(f(ℓ)). These properties of the polynomial Î_{n,k,V}(f(ℓ)) together with Lemma D4 formulated below enable us prove such an estimate on the distribution of ∥T_{n,k}(f(ℓ))∥ that together with formula (D.7) imply relation (D.5). Let us formulate these lemmas.

Lemma D2. Let us consider a sequence of independent random variables ε_{1},...,ε_{n}, P(ε_{l} = 1) = P(ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is also independent of the random variables ξ_{1}^{(1)},...,ξ_{n}^{(1)} and ξ_{1}^{(2)},...,ξ_{n}^{(2)} appearing in the definition of the modified decoupled U-statistics I_{n,k,V}(f(ℓ)) given in formula (D.4). Let us define with their help the sequences of random variables η_{1}^{(1)},...,η_{n}^{(1)} and η_{1}^{(2)},...,η_{n}^{(2)} whose elements (η_{l}^{(1)},η_{l}^{(2)}) = (η_{l}^{(1)}(ε_{l}),η_{l}^{(2)}(ε_{l})), 1 ≤ l ≤ n, are defined by the formula

(η_{l}^{(1)}(ε_{l}),η_{l}^{(2)}(ε_{l})) = ( \frac{1 + ε_{l}}{2} ξ_{l}^{(1)}+ \frac{1 − ε_{l}}{2} ξ_{l}^{(2)}, \frac{1 − ε_{l}}{2} ξ_{l}^{(1)}+ \frac{1 + ε_{l}}{2} ξ_{l}^{(2)}) ,

i.e. let

(η_{l}^{(1)}(ε_{l}),η_{l}^{(2)}(ε_{l})) = (ξ_{l}^{(1)},ξ_{l}^{(2)}) ifε_{l} = 1,

and

(η_{l}^{(1)}(ε_{l}),η_{l}^{(2)}(ε_{l})) = (ξ_{l}^{(2)},ξ_{l}^{(1)}) ifε_{l} = −1, 1 ≤ l ≤ n.

Then the joint distribution of the pair of sequences of random variables ξ_{1}^{(1)},...,ξ_{n}^{(1)} and ξ_{1}^{(2)},...,ξ_{n}^{(2)} agrees with that of the pair of sequences η_{1}^{(1)},...,η_{n}^{(1)} and η_{1}^{(2)},...,η_{n}^{(2)}, which is also independent of the sequence ε_{1},...,ε_{n}.

Let us fix some V ⊂ {1,...,k}, and introduce the random variable

Î_{n,k,V}(f(ℓ)) = \frac{1}{k!} ∑_{(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k} f_{l_{1},...,l_{k}}( η_{l_{1}}^{(α_{V}(1))},...,η_{l_{k}}^{(α_{V}(k))}) , (D.8)

where similarly to formula (D.4) α_{V}(j) = 1 if j ∈ V, and α_{V}(j) = 2 if j ∉ V. Then the identity

2^{k}Î_{n,k,V}(f(ℓ))  (D.9)

= \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}), (s_{1},...,s_{k}): \\ 1≤l_{j}≤n, s_{j}=1 or s_{j}=2, \\ j=1,...,k, \end{matrix}} (1 + κ_{s_{1},V}^{(1)}ε_{l_{1}})··· (1 + κ_{s_{k},V}^{(k)}ε_{l_{k}})f_{l_{1},...,l_{k}}( ξ_{l_{1}}^{(s_{1})},...,ξ_{l_{k}}^{(s_{k})})

holds, where κ_{1,V}^{(j)} = 1 and κ_{2,V}^{(j)} = −1 if j ∈ V, and κ_{1,V}^{(j)} = −1 and κ_{2,V}^{(j)} = 1 if j ∉ V,

i.e. κ_{1,V}^{(j)} = 3 − 2α_{V}(j) and κ_{2,V}^{(j)} = −κ_{1,V}^{(j)}.

Before the formulation of Lemma D4 another Lemma D3 will be presented which will be applied in its proof.

Lemma D3. Let Z be a random variable taking values in a separable Banach space B with expectation zero, i.e. let Eκ(Z) = 0 for all κ ∈ B′, where B′ denotes the (Banach) space of all (bounded) linear transformations of B to the real line. Then

P(∥v + Z∥ ≥ ∥v∥) ≥ inf_{κ∈B′} \frac{(E|κ(Z)|)^{2}}{4Eκ(Z)^{2}} for all v ∈ B.

Lemma D4. Let us consider a positive integer n and a sequence of independent random variables ε_{1},...,ε_{n}, P(ε_{l} = 1) = P(ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n. Beside this, fix some positive integer k, take a separable Banach space B and choose some elements a_{s}(l_{1},...,l_{s}) of this Banach space B, 1 ≤ s ≤ k, 1 ≤ l_{j} ≤ n, l_{j} ≠ l_{j′} if j ≠ j′, 1 ≤ j,j′≤ s. With the above notations the inequality

P ( ∥v + ∑_{s=1}^{k} ∑_{\begin{matrix}(l_{1},...,l_{s}): 1≤l_{j}≤n, j=1,...,s, \\ l_{j} ≠l_{j′} ifj≠j′ \end{matrix}} a_{s}(l_{1},...,l_{s})ε_{l_{1}} ··· ε_{l_{s}} ∥ ≥ ∥v∥) ≥ c_{k} (D.10)

holds for all v ∈ B with some constant c_{k} > 0 which depends only on the parameter k. In particular, it does not depend on the norm in the separable Banach space B.

Proof of Lemma D2. Let us consider the conditional joint distribution of the sequences of random variables η_{1}^{(1)},...,η_{n}^{(1)} and η_{1}^{(2)},...,η_{n}^{(2)} under the condition that the random vector ε_{1},...,ε_{n} takes the value of some prescribed ±1 series of length n. Observe that this conditional distribution agrees with the joint distribution of the sequences ξ_{1}^{(1)},...,ξ_{n}^{(1)} and ξ_{1}^{(2)},...,ξ_{n}^{(2)} for all possible conditions. This fact implies the statement about the joint distribution of the sequences (η_{l}^{(1)},η_{l}^{(2)}), 1 ≤ l ≤ n and their independence of the sequence ε_{1},...,ε_{n}.

To prove identity (D.9) let us fix a set M ⊂ {1,...,n}, and consider the case when ε_{l} = 1 if l ∈ M and ε_{l} = −1 if l ∉ M. Put β_{V,M}(j,l) = 1 if j ∈ V and l ∈ M or j ∉ V and l ∉ M, and let β_{V,M}(j,l) = 2 otherwise. Then we have for all (l_{1},...,l_{k}), 1 ≤ l_{j} ≤ n, 1 ≤ j ≤ k, and our fixed set V

∑_{\begin{matrix}(s_{1},...,s_{k}): \\ s_{j}=1 or s_{j}=2, j=1,...,k \end{matrix}} (1 + κ_{s_{1},V}^{(1)}ε_{l_{1}})··· (1 + κ_{s_{k},V}^{(k)}ε_{l_{k}})f_{l_{1},...,l_{k}}( ξ_{l_{1}}^{(s_{1})},...,ξ_{l_{k}}^{(s_{k})})

= 2^{k}f_{l_{1},...,l_{k}}( ξ_{l_{1}}^{(β_{V,M}(1,l_{1}))},...,ξ_{l_{k}}^{(β_{V,M}(k,l_{k}))}) ,  (D.11)

since the product (1+κ_{s_{1},V}^{(1)}ε_{l_{1}})··· (1+κ_{s_{k},V}^{(k)}ε_{l_{k}}) equals either zero or 2^{k}, and it equals

2^{k} for that sequence (s_{1},...,s_{k}) for which κ_{s_{j},V}^{(j)}ε_{l_{j}} = 1 for all 1 ≤ j ≤ k, and the rela-

tion κ_{s_{j},V}^{(j)}ε_{l_{j}} = 1 is equivalent to β_{V,M}(j,l_{j}) = s_{j} for all 1 ≤ j ≤ k. (In relation (D.11) it is sufficient to consider only such products for which l_{j} ≠ l_{j′} if j ≠ j′because of the properties of the functions f_{l_{1},...,l_{k}}.)

Beside this, ξ_{l}^{β_{V,M}(l,j)} = η_{l}^{α_{V}(j)} for all 1 ≤ l ≤ n and 1 ≤ j ≤ k, and as a consequence

f_{l_{1},...,l_{k}}( ξ_{l_{1}}^{(β_{V,M}(1,l_{1}))},...,ξ_{l_{k}}^{(β_{V,M}(k,l_{k}))}) = f_{l_{1},...,l_{k}}( η_{l_{1}}^{(α_{V}(1))},...,η_{l_{k}}^{(α_{V}(k))}) .

Summing up the identities (D.11) for all 1 ≤ l_{1},...,l_{k} ≤ n and applying the last identity we get relation (D.9), since the identity obtained in such a way holds for all M ⊂ {1,...,n}.

Proof of Lemma D3. Let us first observe that if ξ is a real valued random variable with zero expectation, then P(ξ ≥ 0) ≥ \frac{(E|ξ|)^{2}}{4Eξ^{2}} since (E|ξ|)^{2} = 4(E(ξI(\{ξ ≥ 0}))^{2} ≤ 4P(ξ ≥ 0)Eξ^{2} by the Schwarz inequality, where I(A) denotes the indicator function of the set A. (In the above calculation and in the subsequent proofs I apply the convention \frac{0}{0} = 1. We need this convention if Eξ^{2} = 0. In this case we have the identities P(ξ = 0) = 1 and E|ξ| = 0, hence the above proved inequality holds in this case, too.)

Given some v ∈ B, let us choose a linear operator κ such that ∥κ∥ = 1, and κ(v) = ∥v∥. Such an operator exists by the Banach–Hahn theorem. Observe that {ω: ∥v + Z(ω)∥ ≥ ∥v∥} ⊃ {ω: κ(v + Z(ω)) ≥ κ(v)} = {ω: κ(Z(ω)) ≥ 0}. Beside this, Eκ(Z) = 0. Hence we can apply the above proved inequality for ξ = κ(Z), and it yields that P(∥v + Z∥ ≥ ∥v∥) ≥ P(κ(Z) ≥ 0) ≥ \frac{(E|κ(Z)|)^{2}}{4Eκ(Z)^{2}}. Lemma D3 is proved.

Proof of Lemma D4. Take the class of random polynomials

Y = ∑_{s=1}^{k} ∑_{\begin{matrix}(l_{1},...,l_{s}): 1≤l_{j}≤n, j=1,...,s, \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} b_{s}(l_{1},...,l_{s})ε_{l_{1}} ··· ε_{l_{s}},

where ε_{l}, 1 ≤ l ≤ n, are independent random variables with P(ε_{l} = 1) = P(ε_{l} = −1) = \frac{1}{2}, and the coefficients b_{s}(l_{1},...,l_{s}), 1 ≤ s ≤ k, are arbitrary real numbers. The proof of Lemma D4 can be reduced to the statement that there exists a constant c_{k} > 0 depending only on the order k of these polynomials such that the inequality

(E|Y |)^{2} ≥ 4c_{k}EY^{2}.  (D.12)

holds for all such polynomials Y. Indeed, consider the polynomial

Z = ∑_{s=1}^{k} ∑_{\begin{matrix}(l_{1},...,l_{s}): 1≤l_{j}≤n, j=1,...,s, \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} a_{s}(l_{1},...,l_{s})ε_{l_{1}} ··· ε_{l_{s}},

and observe that Eκ(Z) = 0 for all linear functionals κ on the space B. Hence Lemma D3 implies that the left-hand side expression in (D.10) is bounded from below by inf_{κ∈B′} \frac{(E|κ(Z)|)^{2}}{4Eκ(Z)^{2}}. On the other hand, relation (D.12) implies that inf_{κ∈B′} \frac{(E|κ(Z)|)^{2}}{4Eκ(Z)^{2}} ≥ c_{k}.

To prove relation (D.12) first we compare the moments EY^{2} and EY^{4}. Let us introduce the random variables

Y_{s} = ∑_{\begin{matrix}(l_{1},...,l_{s}): 1≤l_{j}≤n, j=1,...,s, \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} b_{s}(l_{1},...,l_{s})ε_{l_{1}} ··· ε_{l_{s}} 1 ≤ s ≤ k.

We shall show that the estimates of Chapter 13 imply that

EY_{s}^{4} ≤ 2^{4s}(EY_{s}^{2})^{2}  (D.13)

for these random variables Y_{s}.

Relation (D.13) together with the uncorrelatedness of the random variables Y_{s}, 1 ≤ s ≤ k, imply that

EY^{4}= E ( ∑_{s=1}^{k} Y_{s})^{4} ≤ k^{3} ∑_{s=1}^{k} EY_{s}^{4} ≤ k^{3}2^{4k} ∑_{s=1}^{k} (EY_{s}^{2})^{2}

≤ k^{3}2^{4k}( ∑_{s=1}^{k} EY_{s}^{2})^{2} = k^{3}2^{4k}(EY^{2})^{2}.

This estimate together with the Hölder inequality with p = 3 and q = \frac{3}{2} yield that

EY^{2} = E|Y |^{4/3}· |Y |^{2/3} ≤ (EY^{4})^{1/3}(E|Y |)^{2/3} ≤ k2^{4k/3}(EY^{2})^{2/3}(E|Y |)^{2/3},

i.e. EY^{2} ≤ k^{3}2^{4k}(E|Y |)^{2}, and relation (D.12) holds with 4c_{k} = k^{−3} 2^{−4k}. Hence to complete the proof of Lemma D4 it is enough to check relation (D.13).

In the proof of relation (D.13) we may assume that the coefficients b_{s}(l_{1},...,l_{s}) of the random variable Y_{s} are symmetric functions of the arguments l_{1},...,l_{s}, since a symmetrization of these coefficients does not change the value of Y. Put

B_{s}^{2} = ∑_{\begin{matrix}(l_{1},...,l_{s}): 1≤l_{j}≤n, j=1,...,s, \\ l_{j} ≠l_{j′} if j≠j′ \end{matrix}} b_{s}^{2}(l_{1},...,l_{s}), 1 ≤ s ≤ k.

Then

EY_{s}^{2} = s!B_{s}^{2},

and

EY_{s}^{4} ≤ 1 · 3 · 5··· (4s − 1)B_{s}^{4} = \frac{(4s)!}{2^{2s}(2s)!}B_{s}^{4}

by Lemmas 13.4 and 13.5 with the choice M = 2 and k = s. Inequality (D.13) follows from the last two relations. Indeed, to prove formula (D.13) by means of these relations it is enough to check that \frac{(4s)!}{2^{2s}(2s)!(s!)^{2}} ≤ 2^{4s}. But it is easy to check this inequality with induction with respect to s. (Actually there is a well-known inequality in the literature, known under the name Borell’s inequality, which implies inequality (D.13) with a better coefficient at the right hand side of this estimate.) We have proved Lemma D4.

Let us turn back to the estimation of the probability P(3·2^{k−1}∥T_{n,k}(f)∥ > u). Let us introduce the σ-algebra F = B(ξ_{l}^{(1)},ξ_{l}^{(2)}, 1 ≤ l ≤ n) generated by the random variables ξ_{l}^{(1)},ξ_{l}^{(2)}, 1 ≤ l ≤ n, and fix some set V ⊂ {1,...,k}. I show with the help of Lemma D4 and formula (D.9) that there exists some constant c_{k} > 0 such that the random variables T_{n,k}f(ℓ)) defined in formula (D.6) and Î_{n,k,V}(f(ℓ)) defined in formula (D.8) satisfy the inequality

P ( ∥2^{k}Î_{n,k,V}(f(ℓ))∥ > ∥T_{n,k}(f(ℓ))∥|F ) ≥ c_{k} with probability 1.  (D.14)

In the proof of (D.14) we shall exploit that in formula (D.9) 2^{k}Î_{n,k,V}(f(ℓ)) is represented by a polynomial of the Rademacher functions ε_{1},...,ε_{n} whose constant term is T_{n,k}(f(ℓ)). The coefficients of this polynomial are functions of the random variables ξ_{l}^{(1)} and ξ_{l}^{(2)}, 1 ≤ l ≤ n. The independence of these random variables from ε_{l}, 1 ≤ l ≤ n, and the definition of the σ-algebra F yield that

P ( ∥2^{k}Î_{n,k,V}(f(ℓ))∥ > ∥T_{n,k}(f(ℓ))∥|F )  (D.15)

= P_{ε_{V}}( ∥ \frac{1}{k!} ∑_{\begin{matrix}(l_{1},...,l_{k}), (s_{1},...,s_{k}): \\ 1≤l_{j}≤n,s_{j}=1 or s_{j}=2, \\ j=1,...,k, \end{matrix}}(1 + κ_{s_{1},V}^{(1)}ε_{l_{1}})··· (1 + κ_{s_{k},V}^{(k)}ε_{l_{k}})

f_{l_{1},...,l_{k}}( ξ_{l_{1}}^{(s_{1})},...,ξ_{l_{k}}^{(s_{k})}) ∥
> ∥T_{n,k}(f(ℓ))(ξ_{l}^{(j)}, 1 ≤ l ≤ n, j = 1,2)∥) ,

where P_{ε_{V}} means that the values of the random variables ξ_{l}^{(1)}, ξ_{l}^{(2)}, 1 ≤ l ≤ n, are fixed, (their value depend on the atom of the σ-algebra F we are considering) and the probability is taken with respect to the remaining random variables ε_{l}, 1 ≤ l ≤ n. At the right-hand side of (D.15) the probability of such an event is considered that the norm of a polynomial of order k of the random variables ε_{1},...,ε_{n} is larger than ∥T_{n,k}(f(ℓ))(ξ_{l}^{(j)}, 1 ≤ l ≤ n, j = 1,2)∥. Beside this, the constant term of this polynomial equals T_{n,k}(f(ℓ))(ξ_{l}^{(j)}, 1 ≤ l ≤ n, j = 1,2). Hence this probability can be bounded by means of Lemma D4, and this result yields relation (D.14).

The distributions of I_{n,k,V}(f(ℓ)) and Î_{n,k,V}(f(ℓ)) agree by the first statement of Lemma D2 and a comparison of formulas (D.4) and (D.8). Hence relation (D.14) implies that

P ( ∥2^{k}I_{n,k,V}(f(ℓ))∥ ≥ \frac{1}{3} · 2^{1−k} u) = P ( ∥2^{k}Î_{n,k,V}(f(ℓ))∥ ≥ \frac{1}{3} · 2^{1−k} u)

≥ P ( ∥2^{k}Î_{n,k,V}(f(ℓ))∥ ≥ ∥T_{n,k}(f(ℓ))∥, ∥T_{n,k}(f(ℓ))∥ ≥ \frac{1}{3} · 2^{1−k} u)

= ∫_{\{ω: ∥T_{n,k}(f(ℓ))(ω)∥≥ \frac{1}{3}·2^{1−k} u\}}P ( ∥2^{k}Î_{n,k,V}(f(ℓ))∥ > ∥T_{n,k}(f(ℓ))∥|F ) dP

≥ c_{k}P(3 · 2^{k−1}∥T_{n,k}(f(ℓ))∥ ≥ u).

The last inequality with the choice of any set V ⊂ {1,...,k}, 1 ≤ |V | ≤ k − 1, together with relation (D.7) imply formula (D.5).

We shall formulate an inductive hypothesis, and relation (D.3) will be proved together with it by means of an induction procedure with respect to the order k of the U-statistic. In the proof of this inductive procedure we shall apply the already proved relation (D.5). To formulate it some new quantities will be introduced.

Let W = W (k) denote the set of all partitions of the set {1,...,k}. Let us fix k independent copies ξ_{1}^{(j)},...,ξ_{n}^{(j)}, 1 ≤ j ≤ k, of the sequence of random variables ξ_{1},...,ξ_{n}. Given a partition W = (U_{1},...,U_{s}) ∈ W (k) let us introduce the function s_{W}(j), 1 ≤ j ≤ k, which tells for all arguments j the index of that element of the partition W which contains the point j, i.e. the value of the function s_{W}(j), 1 ≤ j ≤ k, in a point j is defined by the relation j ∈ V_{s_{W}(j)}. Let us introduce the expression

I_{n,k,W}(f(ℓ)) = \frac{1}{k!} ∑_{(l_{1},...,l_{k}): 1≤l_{j}≤n, j=1,...,k} f_{l_{1},...,l_{k}}( ξ_{l_{1}}^{(s_{W}(1))},...,ξ_{l_{k}}^{(s_{W}(k))})
for all W ∈ W (k).

An expression of the form I_{n,k,W}(f(ℓ)), W ∈ W_{k}, will be called a decoupled Ustatistic with generalized decoupling. Given a partition W = (U_{1},...,U_{s}) ∈ W_{k} let us call the number s, i.e. the number of the elements of this partition the rank both of the partition W and of the decoupled U-statistic I_{n,k,W}(f(ℓ)) with generalized decoupling.

Now I formulate the following hypothesis. For all k ≥ 2 and 2 ≤ j ≤ k there exist some constants C(k,j) > 0 and δ(k,j) > 0 such that for all W ∈ W_{k} a decoupled U-statistic I_{n,k,W}(f(ℓ)) with generalized decoupling satisfies the inequality

P(∥I_{n,k,W}(f(ℓ))∥ > u) ≤ C(k,j)P (∥Ī_{n,k}(f(ℓ))∥ > δ(k,j)u)
for all 2 ≤ j ≤ k if the rank of W equals j.  (D.16)

It will be proved by induction with respect to k that both relations (D.3) and (D.16) hold for U-statistics of order k. Let us observe that for k = 2 relation (D.3) follows from (D.5). Relation (D.16) also holds for k = 2, since in this case we have to consider only the case j = k = 2. Relation (D.16) also holds in this case with C(2,2) = 1 and δ(2,2) = 1. Hence we can start our inductive proof with k = 3. First I prove relation (D.16).

In relation (D.16) the tail-distribution of decoupled U-statistics with generalized decoupling is compared with that of the decoupled U-statistic Ī_{n,k}(f(ℓ)) introduced in (D.2). Given the order k of these U-statistics it will be proved by means of a backward induction with respect to the rank j of the decoupled U-statistics I_{n,k,W}(f(ℓ)) with generalized decoupling.

Relation (D.16) clearly holds for j = k with C(k,k) = 1 and δ(k,k) = 1. If we already know that these relations hold up to k−1, then we prove first relation (D.16) for generalized decoupling U-statistics of order k with respect to backward induction for the rank 2 ≤ j < k.

For this goal the following observation will be made. If the rank j of a partition W = (U_{1},...,U_{j}) satisfies the relation 2 ≤ j ≤ k − 1, then it contains an element with cardinality strictly less than k and strictly greater than 1. For the sake of simpler notation let us assume that the element U_{j} of this partition is such an element, and U_{j} = {t,...,k} with some 2 ≤ t ≤ k − 1. The investigation of general U-statistics of rank j, 2 ≤ j ≤ k − 1, can be reduced to this case by a reindexation of the arguments in the U-statistics if it is necessary. Let us consider the partition W̄= (U_{1},...,U_{j−1} ,{t},...,{k}) and the decoupled U-statistic I_{n,k,W̄}(f(ℓ)) with generalized decoupling corresponding to this partition W̄. It will be shown that our inductive hypothesis implies the inequality

P(∥I_{n,k,W}(f(ℓ))∥ > u) ≤ Ā(k)P (∥I_{n,k,W̄}(f(ℓ))∥ > γ̄(k)u)  (D.17)

with Ā(k) = sup_{2≤p≤k−1} A(p), γ̄(k) = inf_{2≤p≤k−1} γ(p) if the rank j of W is such that

2 ≤ j ≤ k − 1, where the constants A(p) and γ(p) agree with the corresponding coefficients in formula (D.3).

To prove relation (D.17) (where U_{j} = {t,...,k} is the last element of the partition W) let us define the σ-algebra F generated by the random variables appearing in the first t − 1 coordinates of these U-statistics, i.e. by the random variables ξ_{l_{j}}^{s_{W}(j)},1 ≤ j ≤ t −1, and 1 ≤ l_{j} ≤ n for all 1 ≤ j ≤ t −1. We have 2 ≤ t ≤ k −1. By our inductive hypothesis relation (D.3) holds for U-statistics of order p = k − t + 1, since 2 ≤ p ≤ k − 1. I claim that this implies that

P(∥I_{n,k,W}(f(ℓ))∥ > u|F) ≤ A(k − t + 1)P (∥I_{n,k,W̄}(f(ℓ))∥ > γ(k − t + 1)u|F )

(D.18)

with probability 1. Indeed, by the independence properties of the random variables ξ_{l}^{s_{W}(j)} (and ξ_{l}^{sW̄(j)}), 1 ≤ j ≤ k, 1 ≤ l ≤ n,

P(∥I_{n,k,W}(f(ℓ))∥ > u|F) = P_{ξ_{l}^{s_{W}(j)},1≤j≤t−1} (∥I_{n,k,W}(f(ℓ)∥ > u)

and

P (∥I_{n,k,W̄}(f(ℓ))∥ > γ(k − t + 1)u|F )
= P_{ξ_{l}^{s_{W}(j)},1≤j≤t−1} (∥I_{n,k,W̄}f(ℓ)∥ > γ(k − t + 1)u),

where P_{ξ_{l}^{s_{W}(j)},1≤j≤t−1} denotes that the values of the random variables ξ_{l}^{s_{W}(j)}(ω),

1 ≤ j ≤ t −1, 1 ≤ l ≤ n, are fixed, and we consider the probability that the appropriate functions of these fixed values and of the remaining random variables ξ^{s_{W}(j)} and ξ^{sW̄(j)}, t ≤ j ≤ k, satisfy the desired relation. These identities and the relation between the sets W and W̄imply that relation (D.18) is equivalent to the identity (D.3) for the generalized U-statistics of order 2 ≤ k − t + 1 ≤ k − 1 with kernel functions

f_{l_{t},...,l_{k}}(x_{t},...,x_{k})

= ∑_{(l_{1},...,l_{t−1}): 1≤l_{j}≤n, 1≤j≤t−1}f_{l_{1},...,l_{k}}(ξ_{l_{1}}^{s_{W}(1)}(ω),...,ξ_{l_{t−1}}^{s_{W}(t−1)}(ω),x_{t},...,x_{k}).

Relation (D.17) follows from inequality (D.18) if expectation is taken at both sides. As the rank of W̄is strictly greater than the rank of W, relation (D.17) together with our backward inductive assumption imply relation (D.16) for all 2 ≤ j ≤ k.

Relation (D.16) implies in particular (with the applications of partitions of order k and rank 2) that the terms in the sum at the right-hand side of (D.5) satisfy the inequality

P (D_{k}∥I_{n,k,V}(f(ℓ))∥ > u) ≤ C̄(k,j)P (∥Ī_{n,k}(f(ℓ))∥ > D̄_{k}u)

with some appropriate C̄_{k}> 0 and D̄_{k} > 0 for all V ⊂ {1,...,k}, 1 ≤ |V | ≤ k − 1. This inequality together with relation (D.5) imply that inequality (D.3) also holds for the parameter k.

In such a way we get the proof of relation (D.3) and its special case, relation (14.13). Let us prove formula (14.14) with its help first in the simpler case when the supremum of finitely many functions is taken. If M < ∞ functions f_{1},...,f_{M} are considered, then relation (14.14) for the supremum of the U-statistics and decoupled U-statistics with these kernel functions can be derived from formula (14.13) if it is applied for the function f = (f_{1},...,f_{M}) with values in the separable Banach space B_{M} which consists of the vectors (v_{1},...,v_{M}), v_{j} ∈ B, 1 ≤ j ≤ M, and the norm ∥(v_{1},...,v_{M})∥ = sup_{1≤j≤m} ∥v_{j}∥ is introduced in it. The application of formula

(14.13) with this choice yields formula (14.14) for this supremum. Let us emphasize that the constants appearing in this estimate do not depend on the number M. (We took only M < ∞ kernel functions, because with such a choice the Banach space B_{M} defined above is also separable.) Since the distribution of the random variables sup_{1≤s≤M} ∥I_{n,k}(f_{s})∥ converge to that of sup_{1≤s<∞} ∥I_{n,k}(f_{s})∥, and the distribution of the ran-

dom variables sup_{1≤s≤M} ∥Ī_{n,k}(f_{s})∥ converge to that of sup_{1≤s<∞} ∥Ī_{n,k}(f_{s})∥ as M → ∞, re-

lation (14.14) in the general case follows from its already proved special case and a limiting procedure M → ∞.

Remark. The above proved formula (D.3) can be slightly generalized. It also holds if the expressions I_{n,k}(f(ℓ)) and Ī_{n,k}(f(ℓ)) appearing in this inequality are defined in a more general way. Namely, they are the random functions introduced in formulas (D.1) and (D.2), but the sequences ξ_{1},...,ξ_{n} and their independent copies ξ_{1}^{(j)},...,ξ_{n}^{(j)} in these formulas are independent random variables which may also be non-identically distributed. Such a generalization can be proved without any essential change in the original proof.

References

1. Adamczak, R. (2006) Moment inequalities for U-statistics. Annals of Probability 34, 2288– 2314

2. Ajtai, M., Komlós, J. and Tusnády, G. (1984) On optimal matchings. Combinatorica 4 no. 4, 259–264

3. Alexander, K. (1987) The central limit theorem for empirical processes over Vapnik– Červonenkis classes. Annals of Probability 15, 178–203

4. Arcones, M. A. and Giné, E. (1993) Limit theorems for U-processes. Annals of Probability, 21, 1494–1542

5. Arcones, M. A. and Giné, E. (1994) U-processes indexed by Vapnik–Červonenkis classes of functions with application to asymptotics and bootstrap of U-statistics with estimated parameters. Stoch. Proc. Appl. 52, 17–38 6. Bennett, G. (1962) Probability inequality for the sum of independent random variables. J.

Amer. Statist. Assoc. 57, 33–45

7. Bonami, A. (1970)Étude des coefficients de Fourier des fonctions de L^{p}(G). Ann. Inst. Fourier (Grenoble) 20 335–402

8. Borell, C. (1979) On the integrability of Banach space valued Walsh polynomials. Séminaire de Probabilités XIII. Lecture Notes in Math. 721, 1–3 Springer–Verlag, Berlin

9. de la Peña, V. H. and Giné, E. (1999) Decoupling. From dependence to independence. Springer series in statistics. Probability and its application. Springer–Verlag, New York, Berlin, Heidelberg

10. de la Peña, V. H. and Montgomery–Smith, S. (1995) Decoupling inequalities for the tailprobabilities of multivariate U-statistics. Ann. Probab., 23, 806–816

11. Dobrushin, R. L. (1979) Gaussian and their subordinated fields. Annals of Probability 7, 1-28 12. Dudley, R. M. (1978) Central limit theorems for empirical measures. Annals of Probability 6,

899–929

13. Dudley, R. M. (1984) A course on empirical processes. Lecture Notes in Mathematics 1097, 1–142 Springer–Verlag, New York

14. Dudley, R. M. (1989) Real Analysis and Probability. Wadsworth & Brooks, Pacific Grove, California

15. Dudley, R. M. (1998) Uniform Central Limit Theorems. Cambridge University Press, Cambridge U.K.

16. Dynkin, E. B. and Mandelbaum, A. (1983) Symmetric statistics, Poisson processes and multiple Wiener integrals. Annals of Statistics 11, 739–745

17. Frankl, P. and Pach J. (1983) On the number of sets in null-t-design. European J. Combinatorics 4 21–23

18. Giné, E. and Guillou, A. (2001) On consistency of kernel density estimators for randomly censored data: Rates holding uniformly over adaptive intervals. Ann. Inst. Henri Poincaré PR 37 503–522

19. Giné, E., Kwapień, S, Lata�a, R. and Zinn, J. (2001) The LIL for canonical U-statistics of order 2. Annals of Probability 29 520–527

20. Giné, E., Lata�a, R. and Zinn, J. (2000) Exponential and moment inequalities for U-statistics in High dimensional probability II. Progress in Probability 47. 13–38. Birkhäuser Boston, Boston, MA.

21. Gross, L. (1975) Logarithmic Sobolev inequalities. Amer. J. Math. 97, 1061–1083

22. Guionnet, A. and Zegarlinski, B. (2003) Lectures on Logarithmic Sobolev inequalities. Lecture Notes in Mathematics 1801 1–134 2. Springer–Verlag, New York

23. Hanson, D. L. and Wright, F. T. (1971) A bound on the tail probabilities for quadratic forms in independent random variables. Ann. Math. Statist. 42 52–61

24. Hoeffding, W. (1948) A class of statistics with asymptotically normal distribution. Ann. Math. Statist. 19 293–325

25. Hoeffding, W. (1963) Probability inequalities for sums of bounded random variables. J. Amer. Math. Society 58, 13–30

26. Itô K. (1951) Multiple Wiener integral. J. Math. Soc. Japan 3. 157–164

27. Janson, S. (1997) Gaussian Hilbert Spaces. Cambridge University Press, Cambridge

28. Kaplan, E.L. and Meier P. (1958) Nonparametric estimation from incomplete data, Journal of American Statistical Association, 53, 457–481.

29. Lata�a, R. (2006) Estimates of moments and tails of Gaussian chaoses. Annals of Probability 34 2315–2331

30. Ledoux, M. (1996) On Talagrand deviation inequalities for product measures. ESAIM: Probab. Statist. 1. 63–87. Available at http://www.emath./fr/ps/.

31. Ledoux, M. (2001) The concentration of measure phenomenon. Mathematical Surveys and Monographs 89 American Mathematical Society, Providence, RI.

32. Major, P. (1981) Multiple Wiener–Itô integrals. Lecture Notes in Mathematics 849, Springer– Verlag, Berlin, Heidelberg, New York,

33. Major, P. (1988) On the tail behaviour of the distribution function of multiple stochastic integrals. Probability Theory and Related Fields, 78, 419–435 34. Major, P. (1994) Asymptotic distributions for weighted U-statistics. The Annals of Probability,

22 1514–1535

35. Major, P. (2005) An estimate about multiple stochastic integrals with respect to a normalized empirical measure. Studia Scientarum Mathematicarum Hungarica. 295–341

36. Major, P. (2005) Tail behaviour of multiple random integrals and U-statistics. Probability Reviews. 448–505

37. Major, P. (2006) An estimate on the maximum of a nice class of stochastic integrals. Probability Theory and Related Fields. 134, 489–537

38. Major, P. (2006) A multivariate generalization of Hoeffding’s inequality. Electronic Communication in Probability 2 (220–229)

39. Major, P. (2007) On a multivariate version of Bernstein’s inequality Electronic Journal of Probability 12 966–988

40. Major, P. and Rejtő, L. (1988) Strong embedding of the distribution function under random censorship. Annals of Statistics 16, 1113–1132

41. Major, P. and Rejtő, L. (1998) A note on nonparametric estimations. In the conference volume to the 65. birthday of Miklós Csörgő. 759–774

42. Malyshev, V. A. and Minlos, R. A. (991) Gibbs Random Fields. Method of cluster expansion. Kluwer, Academic Publishers, Dordrecht

43. Massart, P. (2000) About the constants in Talagrand’s concentration inequalities for empirical processes. Annals of Probability 28, 863–884

44. Mc. Kean, H. P. (1973) Wiener’s theory of non-linear noise. in Stochastic Differential Equations SIAM–AMS Proc. 6 197–209

45. Nelson, E. (1973) The free Markov field. J. Functional Analysis 12, 211–227

46. Nourdin, I. and Peccati, G. (2012) Normal approximations with Malliavin calculus: from Stein’s method to Universality. Cambridge Tracts in Mathematics, 192 Cambridge University Press, Cambridge

47. Nualart, D. (2006) Malliavin calculus and related topics of probability and Its Applications. 2. edition, Berlin, Springer–Verlag,

48. Peccati, G. and Taqqu, M. S. (2010) Wiener chaos: moments, cumulants and diagrams. Springer–Verlag, New York

49. Pollard, D. (1984) Convergence of Stochastic Processes. Springer–Verlag, New York

50. Rota, G.-C. and Wallstrom, C. (1997) Stochastic integrals: a combinatorial approach. Annals of Probability 25 (3) 1257–1283

51. Surgailis, D. (1984) On multiple Poisson stochastic integrals and associated Markov semigroups. Probab. Math. Statist. 3. no. 2 217-239

52. Surgailis, D. (2000) Long-range dependence and Appell rank. Annals of Probability 28 478– 497

53. Szegő, G. (1967) Orthogonal Polynomials. American Mathematical Society Colloquium Publications. Vol. 23

54. Takemura, A. (1983) Tensor Analysis of ANOVA decomposition. J. Amer. Statist. Assoc. 78, 894–900

55. Talagrand, M. (1994) Sharper bounds for Gaussian and empirical processes. Annals of Probability 22, 28–76

56. Talagrand, M. (1996) New concentration inequalities in product spaces. Invent. Math. 126, 505–563

57. Talagrand M. Spin Glasses: A challenge for mathematicians. Springer–Verlag, Berlin

58. Talagrand, M. (2005) The general chaining. Springer Monographs in Mathematics. Springer– Verlag, Berlin Heidelberg New York

59. Vapnik, V. N. (1995) The Nature of Statistical Learning Theory. Springer–Verlag, New York 60. Wiener, N. (1838) The homogeneous chaos.Amer. J. Math. 60 879–936

Index

B

Bennett’s inequality 15 Bernstein’s inequality 14 bound on the kernel functions in the diagram

formula for U-statistics 129

C

calculation of the expectation of a product of

degenerate U-statistics 130 calculation of the expectation of a product of

Wiener–Itô integrals 108 canonical function 65, 177 chaining argument 37, 219 closed class of functions 29 comparison of the tail distribution of

U-statistics and decoupled U-statistics (result of de la Peña and Montgomery– Smith) 170, 255

concentration inequalities 25, 220 countably approximable classes of random

variables 21, 217

D

decomposition of a multiple random integral to a linear combination of degenerate U-statistics 82

decoupled U-statistics 169 degenerate U-statistics 65 diagram formula for the product of degenerate

U-statistics 119, 126 diagram formula for Wiener–Itô integrals

99, 105, 237 E

elementary functions of k variables 94 estimate on the moments of degenerate

U-statistics 144 estimate on the moments of Wiener–Itô

integrals 144 estimate on the supremum of a class of

Gaussian random variables 23, 38 estimate on the supremum of a class of partial

sums 20, 41, 42, 54 estimate on the supremum of degenerate U-statistics 66, 166, 171

estimate on the supremum of multiple random integrals with respect to an empirical distribution 63, 83

estimate on the supremum of random integrals with respect to a normalized empirical distribution 22

estimate on the supremum of Wiener–Itô

integrals 68, 161 estimate on the tail distribution of a degenerate

U-statistic 66, 147 estimate on the tail distribution of a multiple random integral with respect to a normalized empirical distribution 62, 83

estimate on the tail distribution of a multiple Wiener–Itô integral 67, 147

F

Fock space 110

G

good tail behaviour for a class of decoupled U-statistics 175 good tail behaviour for a class of integrals of

decoupled U-statistics. 176 good tail behaviour for a class of normalized

random sums 52

H

Hermite polynomials 111, 247, 250 Hoeffding decomposition of U-statistics 77 Hoeffding’s inequality 17

I

Itô’s formula for multiple Wiener–Itô integrals 109, 248

L

L_{p}-dense classes of functions with respect to a

measure μ 23 L_{p}-dense, (in particular L_{2}-dense classes) of

functions 19 Lata�a’s estimate about the tail-distribution of

Wiener–Itô integrals 157 limit theorem about multiple random integrals

with respect to a normalized empirical measure 254

limit theorem about normalized degenerate

U-statistics 109, 251 linear Gaussian random fields 22 lower bound on the tail distribution of

Wiener–Itô integrals (result of H. P. Mc. Kean) 155

M

maximum likelihood equation 5 multiple integrals with respect to a normalized

empirical distribution 28 multivariate version of Hoeffding’s inequality N

non-atomic measure 93

P

product limit estimator (Kaplan–Meyer

method) 7, 214 properties of multiple Wiener–Itô integrals

97

R

Rademacher functions 18

randomized decoupled U-statistics 169 relation between L_{1}-dense and Vapnik– Červonenkis classes 32

S

Sauer’s lemma 31, 32, 233 shattering of a set 31 symmetrization lemma 49, 173

U

U-statistics 64

V

Vapnik-Červonenkis classes of sets and functions 31, 219

W

white noise with some reference measure μ

92, 226 Wiener–Itô integrals 93