
\beginsection 8. Formulation of the main results of this work.

Former sections of this work contain estimates about the tail
distribution of normalized sums of independent, identically
distributed random variables and about the tail distribution of
the supremum of appropriate classes of such random sums. We have
also formulated some estimates about the tail distribution of
the integral of a (deterministic) function or of the supremum of
an appropriate class of functions with respect to a normalized
empirical  distribution. These two kinds of problems are closely
related. To understand them better we have also considered their
natural Gaussian counterpart.

In this section we formulate the natural multivariate versions of
these results. They will be proved in the subsequent part of this
work. To formulate them we have to introduce some new notions. In
the subsequent sections I discuss some new problems which help to
work out some methods that enable us to prove the results of this
section. I finish this section with a short overview about the
content of the remaining part of this work. I shall also briefly
indicate why it helps us to prove the results of this section.

I start this section with the formulation of two results,
Theorems~8.1 and~8.2 together with some of their simple
consequences which yield a sharp estimate about the tail
distribution of a multiple random integral with respect to a
normalized empirical distribution and about the analogous
problem when the tail distribution of the supremum of such
integrals is considered. These results are the natural
versions of the corresponding one-variate results about the tail
behaviour of an integral or of the supremum of a class of
integrals with respect to a normalized empirical distribution.
They can be formulated with the help of the notions introduced
before, in particular with the help of the notion of multiple
random integrals with respect to a normalized empirical
distribution function introduced in formula~(4.8).

To formulate the following two results, Theorems~8.3 and~8.4 and
their consequences, which are the natural multivariate versions
of the results about the tail distribution of partial sums of
independent random variables, and of the supremum of such sums
we have to make some preparation. First we introduce the
so-called $U$-statistics which can be considered as the natural
multivariate generalizations of the sum of independent and
identically distributed random variables. Moreover, we had a good
estimation about the tail distribution of sums of independent
random variables only if the summands had expectation zero, and
we have to find the natural multivariate version of this notion.
Hence we define the so-called degenerate $U$-statistics which can
be considered as the natural multivariate counterpart of sums of
independent and identically distributed random variables with
zero expectation. Theorems~8.3 and~8.4 contain estimates about
the tail-distribution of degenerate $U$-statistics and of the
supremum of such expressions.

In Theorems~8.5 and~8.6 we formulate the Gaussian counterparts of
the above results. They deal with multiple Wiener-It\^o integrals
with respect to a so-called white noise. The notion of multiple
Wiener--It\^o integrals and their properties needed to have a good
understanding of these results will be explained in a later
section. Still two results are discussed in this section. They are
Examples~8.7 and~8.8, which state that the estimates of Theorems~8.5
and~8.3 are in a certain sense sharp.

\medskip
To formulate the first two results of this section let us consider
a sequence of independent and identically distributed random
variables $\xi_1,\dots,\xi_n$ with values in a measurable space
$(X,\Cal X)$. Let $\mu$ denote the distribution
of the random variables $\xi_j$, and introduce the empirical
distribution of the sequence $\xi_1,\dots,\xi_n$ defined in~(4.5).
Given a measurable function $f(x_1,\dots,x_k)$ on the
$k$-fold product space $(X^k,\Cal X^k)$ consider its integral
$J_{n,k}(f)$ with respect to the $k$-fold product of the normalized
empirical measure $\sqrt n(\mu_n-\mu)$ defined in formula (4.8). In
the definition of this integral the diagonals $x_j=x_l$,
$1\le j<l\le k$, were omitted from the domain of integration. The
following Theorem~8.1 can be considered as the multiple integral
version of Bernstein's inequality formulated in Theorem~3.1.

\medskip\noindent
{\bf Theorem 8.1. (Estimate on the tail distribution of a multiple
random integral with respect to a normalized empirical distribution).}
{\it Let us take a measurable function
$f(x_1,\dots,x_k)$ on the $k$-fold product $(X^k,\Cal X^k)$ of a
measurable space $(X,\Cal X)$ with some $k\ge1$ together with a
non-atomic probability measure $\mu$ on $(X,\Cal X)$ and a sequence
of independent and identically distributed random variables
$\xi_1,\dots,\xi_n$ with distribution~$\mu$ on $(X,\Cal X)$. Let the
function $f$ satisfy the conditions
$$
\|f\|_\infty=\sup_{x_j\in X,\;1\le j\le k}|f(x_1,\dots,x_k)|\le 1,
\tag8.1
$$
and
$$
\|f\|_2^2=\int f^2(x_1,\dots,x_k) \mu(\,dx_1)\dots\mu(\,dx_k)\le
\sigma^2 \tag8.2
$$
with some constant $0<\sigma\le1$. There exist some constants
$C=C_k>0$ and $\alpha=\alpha_k>0$, such that the random integral
$J_{n,k}(f)$ defined by formulas (4.5) and (4.8) satisfies the
inequality
$$
P(|J_{n,k}(f)|>u)\le C \max\(e^{-\alpha(u/\sigma)^{2/k}},
e^{-\alpha(nu^2)^{1/(k+1)}} \)  \tag8.3
$$
for all $u>0$. The constants $C=C_k>0$ and
$\alpha=\alpha_k>0$ in formula~(8.3) depend only on the parameter~$k$.}

\medskip
Theorem 8.1 can be reformulated in the following equivalent form.

\medskip\noindent
{\bf Theorem 8.1$'$.} {\it Under the conditions of Theorem 8.1
$$
P(|J_{n,k}(f)|>u)\le C e^{-\alpha(u/\sigma)^{2/k}}
\quad \text{for all } 0<u\le n^{k/2}\sigma^{k+1} \tag$8.3'$
$$
with a number $\sigma$, $0\le\sigma\le1$, satisfying relation
in~(8.2) and some universal constants $C=C_k>0$,
$\alpha=\alpha_k>0$, depending only on the multiplicity~$k$ of the
integral $J_{n,k}(f)$.}

\medskip
Theorem 8.1 clearly implies Theorem~$8.1'$, since in the case
$u\le n^{k/2}\sigma^{k+1}$ the first term is larger than the second
one in the maximum at the right-hand side of formula~(8.3). On
the other hand, Theorem~$8.1'$ implies Theorem~8.1 also if
$u>n^{k/2}\sigma^{k+1}$. Indeed, in this case Theorem~$8.1'$ can be
applied with $\bar\sigma=\(u n^{-k/2}\)^{1/(k+1)}\ge \sigma$ if
$u\le n^{k/2}$, hence also condition $0<\bar\sigma\le1$ is satisfied.
This yields that
$P\(|J_{n,k}(f)|>u\)\le C\exp\left\{-\alpha
\(\frac u{\bar\sigma}\)^{2/k}\right\}=C \exp\left\{-\alpha
(nu^2)^{1/(k+1)}\right\}$ if $n^{k/2}\ge u>n^{k/2}\sigma^{k+1}$,
and relation~(8.3) holds in this case. If $u>n^{k/2}$, then
$P(|J_{n,k}(f)|>u)=0$, and relation~(8.3) holds again.

Theorem~8.1 or Theorem~$8.1'$ state that the tail distribution
$P(|J_{n,k}(f)|>u)$ of the $k$-fold random integral $J_{n,k}(f)$ can
be bounded similarly to the probability $P(|\const\sigma\eta^k|>u)$,
where $\eta$ is a random variable with standard normal distribution
and the number $0\le\sigma\le1$ satisfies relation (8.2), provided
that the level~$u$ we consider is less than $n^{k/2}\sigma^{k+1}$.
As we shall see later (see Corollary~1 of Theorem~9.4), the value of
the number $\sigma^2$ in formula (8.2) is closely related to the
variance of $J_{n,k}(f)$. At the end of this section an example is
given which shows that the condition $u\le n^{k/2}\sigma^{k+1}$ is
really needed in Theorem~$8.1'$.

The next result, Theorem 8.2, is the generalization of Theorem~$4.1'$
for multiple random integrals with respect to a normalized empirical
measure. In its formulation the notions of $L_2$-dense classes and
countably approximability introduced in Section~4 are applied.

\medskip\noindent
{\bf Theorem 8.2. (Estimate on the supremum of multiple random
integrals with respect to an empirical distribution).}
{\it Let us have a non-atomic probability measure
$\mu$ on a measurable space $(X,\Cal X)$ together with a countable
and $L_2$-dense class $\Cal F$ of functions $f=f(x_1,\dots,x_k)$ of
$k$ variables with some parameter $D\ge1$ and exponent $L\ge1$ on
the product space $(X^k,\Cal X^k)$ which satisfies the conditions
$$
\|f\|_\infty=\supp_{x_j\in X,\;1\le j\le k}|f(x_1,\dots,x_k)|\le 1,
\qquad \text{for all } f\in \Cal F \tag8.4
$$
and
$$
\|f\|_2^2=Ef^2(\xi_1,\dots,\xi_k)=\int f^2(x_1,\dots,x_k)
\mu(\,dx_1)\dots\mu(\,dx_k)\le \sigma^2 \qquad \text{for all }
f\in \Cal F \tag8.5
$$
with some constant $0<\sigma\le1$. There exist some constants
$C=C(k)>0$, $\alpha=\alpha(k)>0$ and $M=M(k)>0$ depending only on
the parameter $k$ such that the supremum of the random integrals
$J_{n,k}(f)$, $f\in \Cal F$, defined by formula (4.8) satisfies
the inequality
$$
\aligned
P\(\supp_{f\in\Cal F}|J_{n,k}(f)|\ge u\)\le C &\exp\left\{-\alpha
\(\frac u{\sigma}\)^{2/k}\right\} \quad \text{for those numbers } u\\
\text{for which } &n\sigma^2\ge
\(\frac u\sigma\)^{2/k} \ge M(L^{3/2}\log\frac2\sigma+(\log D)^{3/2}),
\endaligned \tag8.6
$$
where the numbers $D$ and $L$ agree with the parameter and exponent
of the $L_2$-dense class~$\Cal F$.

The condition about the countable cardinality of the class $\Cal F$
can be replaced by the weaker condition that the class of random
variables $J_{n,k}(f)$, $f\in\Cal F$, is countably approximable.}

\medskip
The condition given for the number~$u$ in formula~(8.6) appears in 
Theorem~8.2 for a similar reason as the analogous condition
formulated in~(4.4) in its one-variate counterpart, Theorem~4.1.
The lower bound is needed, since we have a good estimate in 
formula~(8.6) only for $u\ge E\sup\limits_{f\in\Cal F}|J_{n,k}(f)|$.
The upper bound appears, since we have a good estimate in 
Theorem~$8.1'$ only for $0<u<n^{k/2}\sigma^{k+1}$. If a pair of
numbers $(u,\sigma)$ does not satisfy condition~(8.6), than we
may try to get an estimate by increasing the number~$\sigma$ or 
decreasing the number~$u$.

\medskip
To formulate such a version of Theorems~8.1 and~8.2 which corresponds
to the results about sums of independent random variables in the
case $k=1$  the following notions will be introduced.

\medskip\noindent
{\bf Definition of $U$-statistics.}
{\it Let us consider a function $f=f(x_1,\dots,x_k)$  on the
$k$-th power $(X^k,\Cal X^k)$ of a space $(X,\Cal X)$ together
with a sequence of independent and identically distributed random
variables $\xi_1,\dots,\xi_n$, $n\ge k$, which take their values in
this space $(X,\Cal X)$. The expression
$$
I_{n,k}(f)=\frac1{k!}\summ\Sb (l_1,\dots,l_k)\colon\;
1\le l_j\le n,\; j=1,\dots, k,\\
l_j\neq l_{j'} \text{ if } j\neq j'\endSb
f\(\xi_{l_1},\dots,\xi_{l_k}\) \tag8.7
$$
is called a $U$-statistic of order $k$ with the sequence
$\xi_1,\dots,\xi_n$, and kernel function~$f$.}

\medskip\noindent
{\it Remark.}\/ In later calculations sometimes we shall work
with $U$-statistics with kernel functions of the form
$f(x_{u_1},\dots,x_{u_k})$ instead of $f(x_1,\dots,x_k)$, where
$\{u_1,\dots,u_k\}$ is an arbitrary set with different elements.
The $U$-statistic with such a kernel function will also be defined,
and it equals the $U$-statistic  with the original kernel
function~$f$ defined in~(8.7), i.e.
$$
I_{n,k}(f(x_{u_1},\dots,x_{u_k}))=I_{n,k}(f(x_1,\dots,x_k)).
\tag$8.7'$
$$
(Observe that if we define the function
$f_\pi(x_1,\dots,x_k)=f(x_{\pi(1)},\dots,x_{\pi(k)})$ for all
permutations~$\pi$ of the set $\{1,\dots,k\}$, then
$I_{n,k}(f_\pi)=I_{n,k}(f)$, hence the above definition is legitimate.)
Such a definition is natural, and it simplifies the notation in
some calculations. A similar convention will be introduced
about Wiener--It\^o integrals in Section~10.

\medskip
Some special $U$-statistics, called degenerate $U$-statistics, will
be also introduced. They can be considered as the natural
multivariate version of sums of identically distributed random
variables with expectation zero. Degenerate $U$-statistics will be
defined together with canonical kernel functions, because these
notions are closely related.

\medskip\noindent
{\bf Definition of degenerate $U$-statistics.} {\it A $U$-statistic
$I_{n,k}(f)$ of order~$k$ with a sequence of independent and
identically distributed random variables $\xi_1,\dots,\xi_n$ is
called degenerate if its kernel function $f(x_1,\dots,x_k)$
satisfies the relation
$$
\align
&E(f(\xi_1,\dots,\xi_k)|\xi_1=x_1,\dots,\xi_{j-1}=x_{j-1},
\xi_{j+1}=x_{j+1},\dots,\xi_k=x_k)=0 \\
&\qquad\qquad \text{for all } 1\le j\le k \text { and }
x_s\in X, \; s\neq j.
\endalign
$$
}

\medskip \noindent
{\bf Definition of a canonical kernel function.} {\it A function
$f(x_1,\dots,x_k)$ taking values in the $k$-fold product of a
measurable space $(X,\Cal X)$ is called a canonical function with
respect to a probability measure $\mu$ on $(X,\Cal X)$ if
$$
\int f(x_1,\dots,x_{j-1},u,x_{j+1},\dots,x_k)\mu(\,du)=0\quad
\text{for all }1\le j\le k \text{ \ and \ } x_s\in X,\; s\neq j.
\tag8.8
$$
}

\medskip
For the sake of more convenient notations in the future we shall
speak also of $U$-statistics of order zero. We shall write
$I_{n,0}(c)=c$ for any constant $c$, and $I_{n,0}(c)$ will be
called a degenerate $U$-statistic of order zero. A constant will 
be considered as a canonical function with zero arguments.

It is clear that a $U$-statistic $I_{n,k}(f)$ with kernel 
function $f$ and independent $\mu$-distributed random variables
$\xi_1,\dots,\xi_n$ is degenerate if and only if its kernel
function is canonical with respect to the probability
measure~$\mu$. Let us also observe that
$$
I_{n,k}(f)=I_{n,k}(\text{Sym}\,f) \tag8.9
$$
for all functions of $k$ variables.

The next two results, Theorems~8.3 and~8.4, deal with degenerate
$U$-statistics. Theorem~8.3 is the $U$-statistic version of
Theorem~8.1 and Theorem~8.4 is the $U$-statistic version of
Theorem~8.2. Actually Theorem~8.3 yields a sharper estimate than
Theorems~8.1, because it contains more explicit and better
universal constants. I shall return to this point later.

\medskip\noindent
{\bf Theorem 8.3. (Estimate on the tail distribution of a
degenerate $U$-statistic).} {\it Let us have a measurable function
$f(x_1,\dots,x_k)$ on the $k$-fold product $(X^k,\Cal X^k)$,
$k\ge1$, of a measurable space $(X,\Cal X)$ together with
a probability measure $\mu$ on $(X,\Cal X)$ and a sequence of
independent and identically distributed random variables
$\xi_1,\dots,\xi_n$, $n\ge k$, with distribution~$\mu$ on
$(X,\Cal X)$. Let us consider the $U$-statistic $I_{n,k}(f)$ of
order~$k$ with this sequence of random variables $\xi_1,\dots,\xi_n$.
Assume that this $U$-statistic is degenerate, i.e. its kernel
function $f(x_1,\dots,x_k)$ is canonical with respect to the
measure~$\mu$. Let us also assume that the function $f$ satisfies
conditions~(8.1) and~(8.2) with some number $0<\sigma\le1$. Then
there exist some constants $A=A(k)>0$ and $B=B(k)>0$ depending only
on the order $k$ of the $U$-statistic $I_{n,k}(f)$ such that
$$
P(k!n^{-k/2}|I_{n,k}(f)|>u)\le A\exp\left\{-\frac{u^{2/k}}{2\sigma^{2/k}
\(1+B\(un^{-k/2}\sigma^{-(k+1)}\)^{1/k}\)}\right\} \tag8.10
$$
for all $0\le u\le n^{k/2}\sigma^{k+1}$.}

\medskip
Let us also formulate the following simple corollary of Theorem~8.3.

\medskip\noindent
{\bf Corollary of Theorem 8.3} {\it Under the conditions of Theorem~8.3
there exist some universal constants $C=C(k)>0$ and $\alpha=\alpha(k)>0$
that
$$
P(k!n^{-k/2}|I_{n,k}(f)|>u)\le C\exp\left\{-\alpha\(\frac u\sigma\)^{2/k}
\right\} \quad \text{for all } 0\le u\le n^{k/2}\sigma^{k+1}.
\tag$8.10'$
$$
}

\medskip
The following estimate holds about the supremum of degenerate
$U$-statistics.

\medskip\noindent
{\bf Theorem 8.4. (Estimate on the supremum of degenerate
$U$-statistics).} {\it Let us have a probability measure $\mu$ on
a measurable space $(X,\Cal X)$ together with a countable and
$L_2$-dense class $\Cal F$ of functions $f=f(x_1,\dots,x_k)$ of $k$
variables with some parameter $D\ge1$ and exponent~$L\ge1$ on the
product space $(X^k,\Cal X^k)$ which satisfies conditions (8.4) and
(8.5) with some constant $0<\sigma\le1$. Let us take a sequence of
independent $\mu$ distributed random variables $\xi_1,\dots,\xi_n$,
$n\ge k$, and consider the $U$ statistics $I_{n,k}(f)$ with these
random variables and kernel functions $f\in\Cal F$. Let us assume
that all these $U$-statistics $I_{n,k}(f)$, $f\in\Cal F$, are
degenerate, or in an equivalent form, all functions $f\in \Cal F$
are canonical with respect to the measure~$\mu$. Then there exist
some constants $C=C(k)>0$, $\alpha=\alpha(k)>0$ and $M=M(k)>0$
depending only on the parameter $k$ such that the inequality
$$
\aligned
P\(\supp_{f\in\Cal F}n^{-k/2}|I_{n,k}(f)|\ge u\)\le C
&\exp\left\{-\alpha \(\frac u{\sigma}\)^{2/k}\right\} \quad
\text{holds for those numbers } u \\
\text{for which } n\sigma^2\ge
&\(\frac u\sigma\)^{2/k} \ge M(L^{3/2}\log\frac2\sigma+(\log D)^{3/2}),
\endaligned \tag8.11
$$
where the numbers $D$ and $L$ agree with the parameter and
exponent of the $L_2$-dense class~$\Cal F$.

The condition about the countable cardinality of the class $\Cal F$
can be replaced by the weaker condition that the class of random
variables $n^{-k/2}I_{n,k}(f)$, $f\in\Cal F$, is countably
approximable.}

\medskip
Next I formulate a Gaussian counterpart of the above results. To do
this I need some notions that will be introduced in Section~10. In
that section the white noise with a reference measure $\mu$ will
be defined. It is an appropriate set of jointly Gaussian random
variables indexed by those measurable sets $A\in \Cal X$ of a
measure space $(X,\Cal X,\mu)$ with a $\sigma$-finite 
measure~$\mu$  for which $\mu(A)<\infty$. Its distribution depends 
on the measure~$\mu$ which will be called the reference measure of 
the white noise.

In Section~10 it will be also shown that given a white noise $\mu_W$
with a non-atomic $\sigma$-additive reference measure $\mu$ on a
measurable space $(X,\Cal X)$ and a measurable function
$f(x_1,\dots,x_k)$ of $k$ variables on the product space
$(X^k,\Cal X^k)$ such that
$$
\int f^2(x_1,\dots,x_k) \mu(\,dx_1)\dots\mu(\,dx_k)\le\sigma^2<\infty
\tag8.12
$$
a $k$-fold Wiener-It\^o integral of the function $f$ with respect
to the white noise~$\mu_W$
$$
Z_{\mu,k}(f)=\frac1{k!}\int f(x_1,\dots,x_k)
\mu_W(\,dx_1)\dots \mu_W(\,dx_k) \tag8.13
$$
can be defined, and the main properties of this integral will be
proved there. It will be seen that Wiener-It\^o integrals have a
similar relation to degenerate $U$-statistics and multiple
integrals with respect to normalized empirical measures as
normally distributed random variables have to partial sums of
independent random variables. Hence it is useful to find the
analogs of the previous estimates of this section about the
tail distribution of Wiener-It\^o integrals. The subsequent
Theorems~8.5 and~8.6 contain such results.

\medskip\noindent
{\bf Theorem 8.5. (Estimate on the tail distribution of a multiple
Wiener--It\^o integral).} {\it Let us fix a measurable space
$(X,\Cal X)$ together with a $\sigma$-finite non-atomic
measure~$\mu$ on it, and let $\mu_W$ be a white noise with reference
measure $\mu$ on $(X,\Cal X)$. If $f(x_1,\dots,x_k)$ is a measurable
function on $(X^k,\Cal X^k)$ which satisfies relation~(8.12) with
some $0<\sigma<\infty$, then
$$
P(k!|Z_{\mu,k}(f)|>u)\le C \exp\left\{-\frac12\(\frac
u\sigma\)^{2/k}\right\} \tag8.14
$$
for all $u>0$ with some constants $C=C(k)$ depending only on~$k$.}

\medskip\noindent
{\bf Theorem 8.6. (Estimate on the supremum of Wiener--It\^o
integrals).} {\it Let $\Cal F$ be a countable class of functions
of $k$ variables defined on the $k$-fold product $(X^k,\Cal X^k)$
of a measurable space $(X,\Cal X)$ such that
$$
\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots \mu(\,dx_k)\le\sigma^2
\quad \text{\rm with some } 0<\sigma\le1 \text { \rm for all }
f\in \Cal F
$$
with some non-atomic $\sigma$-additive measure~$\mu$ on $(X,\Cal X)$.
Let us also assume that $\Cal F$ is an $L_2$-dense class of functions
in the space $(X^k,\Cal X^k)$ with respect to the measure~$\mu^k$
with some exponent~$L\ge1$ and parameter~$D\ge1$, where $\mu^k$ is
the $k$-fold product of the measure~$\mu$. (The classes of
$L_2$-dense classes with respect to a measure were defined in
Section~4.)

Take a white noise $\mu_W$ on $(X,\Cal X)$ with reference measure
$\mu$, and define the Wiener--It\^o integrals $Z_{\mu,k}(f)$ for
all $f\in\Cal F$. Fix some $0<\e\le1$. The inequality
$$
P\(\sup_{f\in \Cal F} k!|Z_{\mu,k}(f)|>u\)\le CD
\exp\left\{-\frac12\(\frac{(1-\e)u}\sigma\)^{2/k}\right\}\tag8.15
$$
holds with some universal constants $C=C(k)>0$, $M=M(k)>0$
for those numbers~$u$ for which $u\ge ML^{k/2}\frac1\e
\log^{k/2}\frac2\e \cdot \sigma\log^{k/2}\frac2\sigma$.}

\medskip
Formula (8.15) yields an almost as good estimate for the supremum
of Wiener--It\^o integrals with the choice of a small $\e>0$ as
formula (8.14) for a single Wiener--It\^o integral. But the lower
bound imposed on the number $u$ in the estimate (8.15) depends
on $\e$, and for a small number $\e>0$ it is large.

The subsequent result presented in Example~8.7 may help to
understand why Theorems~8.3 and~8.5 are sharp. Its proof and
the discussion of the question about the sharpness
of Theorems~8.3 and~8.5 will be postponed to Section~13.

\medskip\noindent
{\bf Example 8.7. (A converse estimate to Theorem 8.5).} {\it Let
us have a $\sigma$-finite measure $\mu$ on some measure space
$(X,\Cal X)$ together with a white noise $\mu_W$ on $(X,\Cal X)$
with counting measure~$\mu$. Let $f_0(x)$ be a real valued function
on $(X,\Cal X)$ such that $\int f_0(x)^2\mu(\,dx)=1$, and take the
function $f(x_1,\dots,x_k)=\sigma f_0(x_1)\cdots f_0(x_k)$ with
some number $\sigma>0$ together with the Wiener--It\^o integral
$Z_{\mu,k}(f)$ introduced in formula (8.13).

Then the relation
$\int f(x_1,\dots,x_k)^2\,\mu(\,dx_1)\dots\,\mu(\,dx_k)=\sigma^2$
holds, and the Wiener--It\^o integral $Z_{\mu,k}(f)$ satisfies the
inequality
$$
P(k!|Z_{\mu,k}(f)|>u)\ge \frac{\bar C}{\(\frac u\sigma\)^{1/k}+1}
\exp\left\{-\frac12\(\frac u\sigma\)^{2/k}\right\}\quad
\text{for all } u>0 \tag8.16
$$
with some constant $\bar C>0$.}

\medskip
The above results show that multiple integrals with respect to a
normalized empirical measure or degenerate $U$-statistics satisfy
some estimates similar to those about multiple Wiener--It\^o
integrals, but they hold under more restrictive conditions. The
difference between the estimates in these problems is similar to
the difference between the corresponding results in Section~4 whose
reason was explained there. Hence they will be only briefly
discussed here. The estimates of Theorem~8.1 and~8.3 are similar to
that of Theorem~8.5. Moreover, for $0\le u\le\e n^{k/2}\sigma^{k+1}$
with a small number $\e>0$ Theorem~8.3 yields an almost as good
estimate about degenerate $U$-statistics as Theorem~8.5 yields
for a Wiener--It\^o integral with the same kernel function $f$ and
underlying measure $\mu$. Example~8.7 shows that the constant in
the exponent of formula (8.14) cannot be improved, at least there
is no possibility of an improvement if only the $L_2$-norm
of the kernel function $f$ is known. Some results discussed later
indicate that neither the estimate of Theorem~8.3 can be improved.

The main difference between Theorem~8.5 and the results of
Theorem~8.1 or~8.3 is that in the latter case the kernel
function~$f$ must satisfy not only an $L_2$ but also an $L_\infty$
norm type condition, and the estimates of these results are
formulated under the additional condition
$u\le n^{k/2}\sigma^{k+1}$. It can be shown that the condition about
the $L_\infty$ norm of the kernel function cannot be dropped from
the conditions of these theorems, and a version of Example~3.3 will
be presented in Example~8.8 which shows that in the case
$u\gg n^{k/2}\sigma^{k+1}$ the left-hand side of~(8.10) may satisfy
only a much weaker estimate. This estimate will be given only for
$k=2$, but with some work it can be generalized for general
indices~$k$.

Theorems~8.2, 8.4 and~8.6 show that for the tail distribution of the
supremum of a not too large class of degenerate $U$-statistics or
multiple integrals a similar upper bound can be given as for the tail
distribution of a single degenerate $U$-statistic or multiple integral,
only the universal constants may be worse in the new estimates.
However, they hold only under the additional condition that the level
at which the tail distribution of the supremum is estimated is not too
low. A similar phenomenon appeared already in the results of Section~4.
Moreover, such a restriction had to be imposed in the formulation of
the results here and in Section~4 for the same reason.

In Theorem~8.2 and~8.4 an $L_2$-dense class of kernel functions was
considered, and this meant that the class of random integrals or
$U$-statistics we consider in this result is not too large. In
Theorem~8.6 a similar, but weaker condition was imposed on the class
of kernel functions. They had to satisfy a similar condition, but
only for the reference measure $\mu$ of the white noise appearing in
the Wiener--It\^o integral. A similar difference appears in the
comparison of Theorems~4.1 or~$4.1'$ with Theorem~4.2, and this
difference has the same reason in the two cases.

I still present the proof of the following Example~8.8
which is a multivariate version of Example~3.3. For the sake of
simplicity I restrict my attention to the case $k=2$.

\medskip\noindent
{\bf Example 8.8. (A converse estimate to Theorem 8.3).} {\it Let us
take a sequence of independent and identically distributed random
variables $\xi_1,\dots,\xi_n$ with values in the plane $X=R^2$ such
that $\xi_j=(\eta_{j,1},\eta_{j,2})$, $\eta_{j,1}$ and $\eta_{j,2}$
are independent random variables with the following distributions.
The distribution of $\eta_{j,1}$ is defined with the help of a
parameter $\sigma^2$, $0<\sigma^2\le\frac18$, in the same way as
the distribution of the random variables $X_j$ in Example~3.3, i.e.
$\eta_{j,1}=\bar\eta_{j,1}-E\bar\eta_{j,1}$ with
$P(\bar\eta_{j,1}=1)=\bar\sigma^2$,
$P(\bar\eta_{j,1}=0)=1-\bar\sigma^2$, where $\bar\sigma^2$ is that
solution of the equation $x^2-x+\sigma^2=0$, which is smaller
than~$\frac12$. The distribution of the random variables is given
by the formula $P(\eta_{j,2}=1)=P(\eta_{j,2}=-1)=\frac12$ for all
$1\le j\le n$. Introduce the function
$f(x,y)=f((x_1,x_2),(y_1,y_2))=x_1y_2+x_2y_1$,
$x=(x_1,x_2)\in R^2$, $y=(y_1,y_2)\in R^2$ if $(x,y)$ is in the
support of the distribution of $(\xi_1,\xi_2$, i.e. if $x_1$ and
$y_1$ take the values $1-\bar\sigma^2$ or $-\bar\sigma^2$ and
$x_2$ and $y_2$ take the values $\pm1$. Put $f(x,y)=0$ otherwise.
Define the $U$-statistic
$$
I_{n,2}(f)=\frac12\sum_{1\le j,k\le n,\,j\neq k} f(\xi_j,\xi_k)
=\frac12\sum_{1\le j,k\le n,\,j\neq k}
(\eta_{j,1}\eta_{k,2}+\eta_{k,1}\eta_{j,2})
$$
of order 2 with the above kernel function $f$ and sequence of
independent random variables $\xi_1,\dots,\xi_n$. Then $I_{n,2}(f)$
is a degenerate $U$-statistic, $|\sup f(x,y)|\le 1$ and
$Ef^2(\xi_j,\xi_j)=\sigma^2$.

If $u\ge B_1n\sigma^3$ with some appropriate constant $B_1>2$,
$\bar B_2^{-1}n\ge u\ge \bar B_2 n^{-1/2}$ with a sufficiently
large fixed number $\bar B_2>0$ and
$\frac14\ge\sigma^2\ge\frac1{n^2}$, and $n$ is a sufficiently
large number, then the estimate
$$
P(n^{-1}I_{n,2}(f)>u)\ge \exp\left\{-Bn^{1/3}u^{2/3}\log
\(\frac u{n\sigma^3}\)\right\} \tag8.17
$$
holds with some $B>0$.}

\medskip\noindent
{\it Remark:}\/ In Theorem~8.3 we got the estimate
$P(n^{-1}I_{n,2}(f)>u)\le e^{-\alpha u/\sigma}$ for the above
defined degenerate $U$-statistic $I_{n,2}(f)$ if
$0\le u\le n\sigma^3$. In the particular case $u=n\sigma^3$
we have the estimate
$P(n^{-1}I_{n,2}(f)>n\sigma^3)\le e^{-\alpha n\sigma^2}$. On the
other hand, the above example shows that in the case $u\gg n\sigma^3$
we can get only a weaker estimate. It is worth looking at the
estimate~(8.17) with fixed parameters $n$ and $u$ and to observe
the dependence of the upper bound on the variance $\sigma^2$ of
$I_{n,2}(f)$. In the case $\sigma^2=u^{2/3}n^{-2/3}$ we have the
upper bound $e^{-\alpha n^{1/3}u^{2/3}}$. Example~8.8 shows
that in the case $\sigma^2\ll u^{2/3}n^{-2/3}$ we can get only a
relatively small improvement of this estimate. A similar picture
appears as in Example~3.3 in the case $k=1$.

\medskip
It is simple to check that the $U$-statistic introduced in the
above example is degenerate because of the independence of the
random variables $\eta_{j,1}$ and $\eta_{j,2}$ and the identity
$E\eta_{j,1}=E\eta_{j,2}=0$. Beside this,
$Ef(\xi_j,\xi_j)^2=\sigma^2$. In the proof of the estimate~(8.17)
the results of Section~3, in particular Example~3.3 can be applied
for the sequence $\eta_{j,1}$, $j=1,2,\dots,n$. Beside this, the
following result known from the theory of large deviations will
be applied. If $X_1,\dots,X_n$ are independent and identically
distributed random variables, $P(X_1=1)=P(X_1=-1)=\frac12$, then
for any number $0\le \alpha<1$ there exists some numbers
$C_1=C_1(\alpha)>0$ and $C_2=C_2(\alpha)>0$ such that
$P\(\summ_{j=1}^nX_j >u\)\ge C_1e^{-C_2u^2/n}$ for all
$0\le u\le \alpha n$.

\medskip\noindent
{\it Proof of Example 8.8.}\/ The inequality
$$
P(n^{-1}I_{n,2}(f)>u)\ge P\(\(\sum_{j=1}^n\eta_{j,1}\)
\(\sum_{j=1}^n\eta_{j,2}\)>4nu\)
-P\(\sum_{j=1}^n\eta_{j,1}\eta_{j,2}>2nu\) \tag8.18
$$
holds. Because of the independence of the random variables $\eta_{j,1}$
and $\eta_{j,2}$ the first probability at the right-hand side of (8.18)
can be bounded from below by bounding the multiplicative terms in it
with $v_1=4n^{1/3}u^{2/3}$ and $v_2=n^{2/3}u^{1/3}$. The first term
will be estimated by means of Example 3.3. This estimate can be
applied with the choice $y=v_1$, since the relation
$v_1\ge 4n\sigma^2$ holds if $u\ge B_1n\sigma^3$
with $B_1>1$, and the remaining conditions $0\le \sigma^2\le\frac18$
and $n\ge4v_1\ge6$ also hold under the conditions of Example~8.8.
The second term can be bounded with the help of the large-deviation result
mentioned after the remark, since $v_2\le \frac12 n$ if
$u\le \bar B_2^{-1}n$ with a sufficiently large $\bar B_2>0$.
In such a way we get the estimate
$$
\align
&P\(\(\sum_{j=1}^n\eta_{j,1}\)\(\sum_{j=1}^n\eta_{j,2}\)>4nu\)\ge
P\(\sum_{j=1}^n\eta_{j,1} >v_1\)P\(\sum_{j=1}^n\eta_{j,2}>v_2\) \\
&\qquad \ge C\exp\left\{-B_1v_1\log
\(\frac{v_1}{n\sigma^2}\)-B_2\frac{v_2^2}{n}\right\}
\ge C\exp\left\{-B_3n^{1/3}u^{2/3}\log\(\frac u{n\sigma^3}\)\right\}
\endalign
$$
with appropriate constants $B_1>1$, $B_2>0$ and $B_3>0$. On the
other hand, by applying Bennett's inequality, more precisely its
consequence given in formula (3.4) for the sum of the random
variables $X_j=\eta_{j,1}\eta_{j,2}$ at level $nu$ instead of
level~$u$ we get the following upper bound for the second term at
the right-hand side of~(8.18).
$$
\align
P\(\sum_{j=1}^n\eta_{j,1}\eta_{j,2}>2nu\)&\le
\exp\left\{-Knu\log \frac u{\sigma^2}\right\} \\
&\le \exp\left\{-2B_4n^{1/3}u^{2/3}\log\(\frac u{n\sigma^3}\)\right\},
\endalign
$$
since $E\eta_{j,1}\eta_{j,2}=0$, $E\eta^2_{j,1}\eta^2_{j,2}=\sigma^2$,
$nu\ge B_1n^2\sigma^3\ge 2n\sigma^2$ because of the conditions $B_1>2$
and $n\sigma\ge1$. Hence the estimate~(3.4) (with parameter $nu$)
can be applied in this case. Beside this, the constant $B_4$ can be
chosen sufficiently large in the last inequality if the number~$n$
or the bound~$\bar B_2$ in Example~8.8 us chosen sufficiently large.
This means that this term is negligible small. The above estimates
imply the statement of Example~8.8.

\medskip
Let me remark that under some mild additional restrictions the
estimate (8.17) can be slightly improved, the term $\log$ can be
replaced by $\log^{2/3}$ in the exponent of the right-hand side
of~(8.17). To get such an improvement some additional calculation
is needed where the numbers $v_1$ and $v_2$ are replaced by
$\bar v_1=4n^{1/3}u^{2/3}\log^{-1/3}\(\frac u{n\sigma^3}\)$ and
$\bar v_2=n^{2/3}u^{1/3}\log^{1/3}\(\frac u{n\sigma^3}\)$.

\medskip
At the end of this section I present a short overview about the
content of the remaining part of this work.

In our proofs we needed some results about $U$-statistics, and this 
is the main topic of Section~9. One of the results discussed here 
is the so-called Hoeffding decomposition of $U$-statistics to the 
linear combination of degenerate $U$-statistics of different order.
We also needed some additional  results which explain how some 
properties (e.g. a bound on the $L_2$ and $L_\infty$ norm of a 
kernel function, the $L_2$-density property of a class~$\Cal F$ of 
kernel function) is inherited if we turn from the original 
$U$-statistics to the degenerate $U$-statistics appearing in 
their Hoeffding decomposition. Section~9 contains some results 
in this direction. Another important result in it is Theorem~9.4 
which yields a decomposition of multiple integrals with respect 
to a normalized empirical distribution to the linear combination 
of degenerate $U$-statistics.  This result is very similar to the 
Hoeffding decomposition of $U$-statistics. The main difference 
between them is that in the decomposition of multiple integrals 
much smaller coefficients appear. Theorem~9.4 makes possible to 
reduce the proof of Theorems~8.1 and~8.2 to the corresponding 
results in Theorems~8.3 and~8.4 about degenerate $U$-statistics.

The definition and the main properties of Wiener--It\^o integrals
needed in the proof of Theorems~8.5 and~8.6 are presented in
Section~10. It also contains a result, called the diagram formula
for Wiener--It\^o integrals which plays an important role in our
considerations. Beside this we proved a limit theorem, where we
expressed the limit of normalized degenerate $U$-statistics with
the help of multiple Wiener--It\^o integrals. This result may
explain why it is natural to consider Theorem~8.5 as the
natural Gaussian counterpart of Theorem~8.5, and Theorem~8.6 as
the natural Gaussian counterpart of Theorem~8.6.

We could prove Bernstein's and Bennett inequality by means of a
good estimation of the exponential moments of the partial sums we
were investigating. In the proof of their multivariate versions,
in Theorems~8.3 and~8.5 this method does not work, because the
exponential moments we have to bound in these cases may be
infinite. On the other hand, we could prove these results by means
of a good estimate on the high moments of the random variables
whose tail distribution we wanted to estimate. In the proof of
Theorem~8.5 when the moments of multiple Wiener--It\^o integrals
have to be bounded this can be done with the help of the
diagram formula for Wiener--It\^o integrals. In Sections~11
and~12 we proved that there is a version of the diagram formula
for degenerate $U$-statistics, and this enables us to estimate
the moments needed in the proof of Theorem~8.3. In Section~13
we proved Theorems~8.3, 8.5 and a multivariate version of the 
Hoeffding inequality. At the end of this section we still 
discussed some results which state that in certain cases when 
we have, beside the upper bound of their $L_2$ and $L_\infty$ 
norm some additional information about the behaviour of the 
kernel function~$f$ in Theorems~8.3 or~8.5, then the estimates 
of these results can be improved.

Section~14 contains the natural multivariate versions of the
results in Section~6. In Section~6 Theorem~4.2 is proved about
the supremum of Gaussian random variables with the help of the
chaining argument. In Section~14 its multivariate version, 
Theorem~8.6 is proved with the help of the chaining argument. 
The chaining argument is not strong enough to prove Theorem~4.1. 
But as it is shown in Section~6, it enables us to prove a result 
formulated in Proposition~6.1, and to reduce the proof of 
Theorem~4.1 with its help to a simpler result formulated in 
Proposition~6.2. One of the results of Section~14, 
Proposition~14.1 is a multivariate version of Proposition~6.1.
We showed that the proof of Theorem~8.4 can be reduced with its
help to the proof of a result formulated in Proposition~14.2,
which can be considered a multivariate version of Proposition~6.2.
Section~14 contains still another result. It turned out that
it is simpler to work with so-called decoupled $U$-statistics
introduced in this section than with usual $U$-statistics,
because they have more independence properties. In
Proposition~$14.2'$ a version of Proposition~14.2 is formulated
about degenerate $U$-statistics, and it is shown with the help
of a result of de la Pe\~na and Montgomery--Smith that the proof
of Proposition~14.2, and thus of Theorem~8.4 can be reduced to
the proof of Proposition~$14.2'$.

Proposition~$14.2'$ is proved similarly to its one-variate
version, Proposition~6.2. The strategy of the proof is explained
in Section~15. The main difference between the proof of the two
propositions is that since the independence properties exploited
in the proof of Proposition~6.2 hold only in a weaker form in the
present case, we have to apply a more refined and more difficult
argument. In particular, we have to apply instead of the
symmetrization lemma, Lemma~7.1, a more general version of it,
Lemma~15.2. It is hard to check its conditions when we try to
apply it in the problems arising in the proof of 
Proposition~$14.2'$.  This is the reason why we had to prove
this result with the help of two inductive propositions, 
formulated in Proposition~15.3 and~15.4, while in the proof of 
Proposition~6.2 it was enough to prove one such result, presented 
in Proposition~7.3. We discuss the details of the problems and
the strategy of the proof in Section~15. The proof of
Propositions~15.3 and~15.4 is given in Sections~16 and~17.
Section~16 contains the symmetrization arguments needed for us,
and the proof is completed with its help in Section~17.

Finally in Section~18 we give an overview of this work, and 
explain its relation to some similar researches. The proof of 
some results is given in the Appendix.

\beginsection 9. Some results about $U$-statistics.

This section contains the proof of the Hoeffding decomposition
theorem, an important result about $U$-statistics. It states that
all $U$-statistics can be represented as a sum of degenerate
$U$-statistics of different order. This representation can be considered
as the natural multivariate version of the decomposition of a
random variable as the sum of a random variable with expectation
zero plus a constant (which can be interpreted as a random variable
of zero variable). Some important properties of the Hoeffding
decomposition will also be proved. The  properties of the kernel
function of a $U$-statistic will be compared to those of the kernel
functions of the $U$-statistics in its Hoeffding decomposition.

If the Hoeffding decomposition of a $U$-statistic is taken, then
the $L_2$ and $L_\infty$-norms of the kernel functions appearing
in the $U$-statistics of the Hoeffding decomposition will be bounded
by means of the corresponding norm of the kernel function of the
original $U$-statistic. It will be also shown that if we take a 
class of $U$-statistics with an $L_2$-dense class of kernel 
functions (and the same sequence of independent and identically 
distributed random variables in the definition of each 
$U$-statistic), and we make the Hoeffding decomposition of all
$U$-statistics in this class, then the kernel functions of the 
degenerate $U$-statistics appearing in these Hoeffding 
decompositions also constitute an $L_2$-dense class. Another 
important result of this section is Theorem~9.4. It yields a 
decomposition of a $k$-fold random integral with respect to a 
normalized empirical measure to the linear combination of 
degenerate $U$-statistics. This result makes possible to derive 
Theorem~8.1 from Theorem 8.3 and Theorem~8.2 from Theorem~8.4, 
and it is also useful in the proof of Theorems~8.3 and~8.4.

Let us first consider the Hoeffding's decomposition. In the 
special case $k=1$ it states that the sum
$S_n=\summ_{j=1}^n\xi_j$ of independent and identically 
distributed random variables can be rewritten as
$S_n=\summ_{j=1}^n(\xi_j-E\xi_j)+\(\summ_{j=1}^nE\xi_j\)$, i.e.\ 
as the sum of independent random variables with zero expectation 
plus a constant. We introduced the convention that a constant is 
the kernel function of a degenerate $U$-statistic of order zero, 
and $I_{n,0}(c)=c$ for a $U$-statistic of order zero. I wrote 
down the above trivial formula, because Hoeffding's decomposition 
is actually its adaptation to a more general situation. To 
understand this let us first see how to adapt the above
construction to the case $k=2$.

In this case a sum of the form
$2I_{n,2}(f)=\summ_{1\le j,k\le n,j\neq k} f(\xi_j,\xi_k)$
has to be considered. Write
$f(\xi_j,\xi_k)=[f(\xi_j,\xi_k)-E(f(\xi_j,\xi_k)|\xi_k)]+
E(f(\xi_j,\xi_k)|\xi_k)=f_1(\xi_j,\xi_k)+\bar f_1(\xi_k)$ with
$f_1(\xi_j,\xi_k)=f(\xi_j,\xi_k)-E(f(\xi_j,\xi_k)|\xi_k)$, and
$\bar f_1(\xi_k)=E(f(\xi_j,\xi_k)|\xi_k)$ to make the conditional
expectation of $f_1(\xi_j,\xi_k)$ with respect to $\xi_k$ equal
zero. Repeating this procedure for the first coordinate we define
$f_2(\xi_j,\xi_k)=f_1(\xi_j,\xi_k)-E(f_1(\xi_j,\xi_k)|\xi_j)$ and
$\bar f_2(\xi_j)=E(f_1(\xi_j,\xi_k)|\xi_j)$.
Let us also write $\bar f_1(\xi_k)=
[\bar f_1(\xi_k)-E\bar f_1(\xi_k)]+E\bar f_1(\xi_k)$  and
$\bar f_2(\xi_j)=[\bar f_2(\xi_j)-E\bar f_2(\xi_j)]
+E\bar f_2(\xi_j)$.
Simple calculation shows that $2I_{n,2}(f_2)$ is a degenerate
$U$-statistics of order 2, and the identity
$2I_{n,2}(f)=2I_{n,2}(f_2)+I_{n,1}((n-1)(\bar f_1-E\bar f_1))+
I_{n,1}((n-1)((\bar f_2-E\bar f_2))+n(n-1)E(\bar f_1+\bar f_2)$
yields the decomposition of $I_{n,2}(f)$ into a sum of degenerate
$U$-statistics of different orders.

Hoeffding's decomposition can be obtained by working out the details
of the above argument in the general case. But it is simpler to
calculate the appropriate conditional expectations with the help of
the kernel functions of the $U$-statistics. To carry out such a
program in the study of $U$-statistics of order~$k$ the following
notations will be introduced.

Let us consider the $k$-fold product $(X^k,\Cal X^k,\mu^k)$ of a
measure space $(X,\Cal X,\mu)$ with some probability measure $\mu$,
and define for all integrable functions $f(x_1,\dots,x_k)$ and indices
$1\le j\le k$ the projection~$P_jf$ of the function $f$ to its $j$-th
coordinate as
$$
(P_jf)(x_1,\dots,x_{j-1},x_{j+1},\dots,x_k)=\int
f(x_1,\dots,x_k)\mu(\,dx_j), \quad 1\le j\le k. \tag9.1
$$
Let us also define the operators $Q_j=I-P_j$ i.e. $Q_jf=f-P_jf$ for
all integrable functions on $f$ on the space $(X^k,\Cal X^k,\mu^k)$,
$1\le j\le k$. In the definition (9.1) $P_jf$ is a function not
depending on the coordinate $x_j$, but in the definition of $Q_j$
we introduce the fictive coordinate $x_j$ to make the expression
$Q_jf=f-P_jf$ meaningful. The following result holds.

\medskip\noindent
{\bf Theorem 9.1. (The Hoeffding decomposition).} {\it
Let $f(x_1,\dots,x_k)$ be an integrable function on the $k$-fold
product space $(X^k,\Cal X^k,\mu^k)$ of a space $(X,\Cal X,\mu)$
with a probability measure $\mu$. It has such a decomposition
$$
f=\summ_{V\subset\{1,\dots,k\}} f_V, \quad \text{with} \quad
f_V(x_j,\,j\in V)=\(\prod_{j\in\{1,\dots,k\}\setminus V}P_j
\prod_{j\in V}Q_j\)f(x_1,\dots,x_k) \tag9.2
$$
for which all functions $f_V$, $V\subset \{1,\dots,k\}$, in (9.2)
are canonical with respect to the probability measure $\mu$, and
the function $f_V$ depends on the arguments $x_j$, $j\in V$.

Let $\xi_1,\dots,\xi_n$ be a sequence of
independent $\mu$ distributed random variables, and consider the
$U$-statistics $I_{n,k}(f)$ and $I_{n,|V|}(f_V)$ corresponding to
the kernel functions $f$, $f_V$ defined in (9.2) and random variables
$\xi_1,\dots,\xi_n$. Then
$$
k!I_{n,k}(f)=\summ_{V\subset\{1,\dots,k\}}
(n-|V|)(n-|V|-1)\cdots(n-k+1)|V|! I_{n,|V|}(f_V) \tag9.3
$$
is a representation of $I_{n,k}(f)$ as a sum of degenerate
$U$-statistics, where $|V|$ denotes the cardinality of the set $V$.
(The product $(n-|V|)(n-|V|-1)\cdots(n-k+1)$ is defined as 1 for
$V=\{1,\dots,k\}$, i.e. if $|V|=k$.) This representation is called
the Hoeffding decomposition of $I_{n,k}(f)$.}

\medskip\noindent
{\it Proof of Theorem 9.1.}\/ Write $f=\prodd_{j=1}^k(P_j+Q_j)f$.
By carrying out the multiplications in this identity and applying the
commutativity of the operators $P_j$ and $Q_j$ for different indices
$j$ we get formula (9.2). To show that the functions $f_V$ in formula
(9.2) are canonical let us observe that this property can be rewritten
in the form $P_jf_V\equiv0$ (in all coordinates $x_s$, $s\in
V\setminus\{j\}$ if $j\in V$).
Since $P_j=P_j^2$, and the identity $P_jQ_j=P_j-P_j^2=0$ holds for all
$j\in\{1,\dots,k\}$ this relation follows from the above mentioned
commutativity of the operators $P_j$ and $Q_j$, as $P_jf_V=
\(\prodd_{s\in\{1,\dots,k\}\setminus V}P_s\prodd_{s\in V\setminus
\{j\}}Q_s\)P_jQ_jf=0$. By applying identity (9.2) for all terms
$f(\xi_{j_1},\dots,\xi_{j_k})$ in the sum defining the $U$-statistic
$I_{n,k}(f)$ and then summing them up we get relation (9.3).

\medskip
In the Hoeffding decomposition we rewrote a general $U$-statistic in
the form of a linear combination of degenerate $U$-statistics. In
many applications of this result we still we have to know how the
properties of the kernel function $f$ of the original $U$-statistic
are reflected in the properties of the kernel functions $f_V$ of
the degenerate $U$-statistics taking part in the Hoeffding
composition. In particular, we need a good estimate on
the $L_2$ and $L_\infty$ norm of the functions $f_V$ by means of
the corresponding norm of the function~$f$. Moreover, if we want to
prove estimates on the tail distribution of the supremum of
$U$-statistics $I_{n,k}(f)$ for a nice class of kernel functions
$f\in \Cal F$ which is an $L_2$-dense class of functions with some
exponent $L$ and parameter $D$, then we may need a similar estimate
on the class of kernel functions $f_V$, $f\in\Cal F$, with some
$V\in\{1,\dots,k\}$ appearing in the Hoeffding decomposition of
these functions. We have to show that this class of functions is
also $L_2$-dense, and we also need a good bound on the exponent and
parameter of this $L_2$-dense class. The next result formulates
such a statement.

\medskip\noindent
{\bf Theorem 9.2. (Some properties of the Hoeffding decomposition).}
{\it Let us consider a square integrable function $f(x_1,\dots,x_k)$
on the $k$-fold product space $(X^k,\Cal X^k,\mu^k)$ and take its
decomposition defined in formula (9.2). The inequalities
$$
\int f_V^2(x_j,\,j\in  V)
\prodd_{j\in V}\mu(\,dx_j)\le \int
f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k) \tag9.4
$$
and
$$
\sup_{x_j,\, j\in V} |f_V(x_j,\,j\in V)|\le2^{|V|}\sup_{x_j,\,1\le
j\le k}|f(x_1,\dots,x_k)| \tag$9.4'$
$$
hold for all $V\subset\{1,\dots,k\}$. (In particular, $f_\emptyset^2\le
\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)$ for $V=\emptyset$.)

Let us consider an $L_2$-dense class $\Cal F$ of functions with some
parameter $D\ge1$ and exponent $L\ge0$ on the space $(X^k,\Cal X^k)$, 
take the decomposition (9.2) of all functions $f\in \Cal F$, and 
define the classes of functions 
$\Cal F_V=\{2^{-|V|}f_V\colon\; f\in \Cal F\}$ for all 
$V\subset\{1,\dots,k\}$ with the functions
$f_V$ taking part in this decomposition. These classes of functions
$\Cal F_V$ are also $L_2$-dense with the same parameter~$D$ and
exponent~$L$ for all $V\subset\{1,\dots,k\}$.}

\medskip
Theorem 9.2 will be proved as a consequence of Proposition~9.3
presented below. To formulate it first some notations will be
introduced:

Let us consider the product $(Y\times Z,\Cal Y\times\Cal Z)$ of two
measurable spaces $(Y,\Cal Y)$ and $(Z,\Cal Z)$ together with a
probability measure $\mu$ on $(Z,\Cal Z)$ and the operator
$$
Pf(y)=P_\mu f(y)=\int f(y,z)\mu(\,dz),\quad y\in Y,\; z\in Z \tag9.5
$$
defined for those $y\in Y$ for which the above integral is finite.
Let $I$ denote the identity operator on the space of functions on
$Y\times Z$, i.e. let $If(y,z)=f(y,z)$, and introduce the operator
$Q=Q_\mu=I-P=I-P_\mu$
$$
Q_\mu f(y,z)=(I-P_\mu)f(y,z)=f(y,z)-P_\mu f(y,z)
=f(y,z)-\int f(y,z)\mu(\,dz), \tag9.6
$$
defined for those points $(y,z)\in Y\times Z$ whose first coordinate~$y$
is such that the expression~$P_\mu f(y)$ is meaningful.
(Here, and in the sequel a function $g(y)$ defined on the space
$(Y,\Cal Y)$ will be sometimes identified with the function
$\bar g(y,z)=g(y)$ on the space $(Y\times Z,\Cal Y\times \Cal Z)$
which actually does not depend on the coordinate $z$.) The
following result holds:

\medskip\noindent
{\bf Proposition 9.3.} {\it Let us consider the direct product
$(Y\times Z,\Cal Y\times\Cal Z)$ of two measure spaces $(Y,\Cal Y)$
and $(Z,\Cal Z)$ together with a probability measure $\mu$ on the
space $(Z,\Cal Z)$. Take the transformations $P_\mu$ and $Q_\mu$
defined in formulas (9.5) and (9.6). Given any probability measure
$\rho$ on the space $(Y,\Cal Y)$ consider the product measure
$\rho\times\mu$ on $(Y\times Z,\Cal Y\times\Cal Z)$. Then the
transformations $P_\mu$ and $Q_\mu$, as maps from the space
$L_2(Y\times Z,\Cal Y\times\Cal Z,\mu\times \rho)$ to $L_2(Y,\Cal
Y,\rho)$ and $L_2(Y\times Z,\Cal Y\times\Cal Z,\rho\times\mu)$
respectively, have a norm less than or equal to 1, i.e.
$$
\int P_\mu f(y)^2\rho(\,dy)\le\int f(y,z)^2\rho(\,dy)\mu(\,dz),
\tag9.7
$$
and
$$
\int Q_\mu f(y,z)^2\rho(\,dy)\mu(\,dz)\le\int f(y,z)^2
\rho(\,dy)\mu(\,dz) \tag9.8
$$
for all functions
$f\in L_2(Y\times Z,\Cal Y\times\Cal Z,\rho\times \mu)$.

If $\Cal F$ is an $L_2$-dense class of functions
$f(y,z)$ in the product space $(Y\times Z,\Cal Y\times\Cal Z)$,
with some parameter $D\ge1$ and exponent $L\ge0$, then also the 
classes $\Cal F_\mu=\{P_\mu f,\; f\in \Cal F\}$ and $\Cal G_\mu
=\{\frac12Q_\mu f=\frac12(f-P_\mu f),\; f\in\Cal F\}$ are 
$L_2$-dense classes with the same exponent $L$ and parameter~$D$ 
in the spaces $(Y,\Cal Y)$ and $(Y\times Z,\Cal Y\times\Cal Z)$ 
respectively.}

\medskip
The following corollary of Proposition 9.3 is formally more general,
but it is a simple consequence of this result. Actually we shall
need this corollary.

\medskip\noindent
{\bf Corollary of Proposition 9.3.} {\it Let us consider the product
$(Y_1\times Z\times Y_2,\Cal Y_1\times\Cal Z\times\Cal Y_2)$ of three
measurable spaces $(Y_1,\Cal Y_1)$, $(Z,\Cal Z)$ and $(Y_2,\Cal Y_2)$
with a probability measure $\mu$ on the space $(Z,\Cal Z)$ and a
probability measure $\rho$ on $Y_1\times Y_2,\Cal Y_1\times\Cal Y_2)$,
and define the transformations
$$
P_\mu f(y_1,y_2)=\int f(y_1,z,y_2)\mu(\,dz),\quad y_1\in Y_1,\;
z\in Z,\; y_2\in Y_2 \tag$9.5'$
$$
and
$$
\aligned
Q_\mu f(y_1,z,y_2)&=(I-P_\mu)f(y_1,z,y_2)=f(y_1,z,y_2)
-P_\mu f(y_1,z,y_2) \\
&=f(y_1,z,y_2)-\int f(y_1,z,y_2)\mu(\,dz),
\quad y_1\in Y_1,\; z\in Z, \;y_2\in Y_2
\endaligned \tag$9.6'$
$$
for the measurable functions $f$ on the space $Y_1\times Z\times Y_2$
integrable with respect the measure $\mu\times\rho$.
Then
$$
\int P_\mu f(y_1,y_2)^2\rho(\,dy_1,\,dy_2) \le\int
f(y,z)^2(\rho\times \mu)(\,dy_1,\,dz,\,dy_2), \tag$9.7'$
$$
for all probability measures $\rho$ on $(Y_1\times Y_2,\Cal
Y_1\times\Cal Y_2)$, where $\rho\times\mu$ is the product of the
probability measure $\rho$ on $(Y_1\times Y_2,\Cal Y_1\times\Cal Y_2)$
and $\mu$ on $(Z,\Cal Z)$, 
%i.e. $\rho\times\mu(\{y_1,z,y_2)\colon\;(y_1,y_2)\in A, z\in B\})
%=\rho(A)\mu(B)$ for all $A\in \Cal Y_1\times\Cal Y_2$, $B\in \Cal Z$, 
%and $\rho\times\mu$ is its unique extension as a probability measure 
%on $(Y_1\times Z \times Y_2,\Cal Y_1\times\Cal Z\times\Cal Y_2)$. 
Also the inequality
$$
\int Q_\mu f(y_1,z,y_2)^2\rho(\,dy_1,\,dy_2)\mu(\,dz)\le\int
f(y_1,z,y_2)^2\rho(\,dy_1,\,dy_2)\mu(\,dz) \tag$9.8'$
$$
holds for all functions $f\in L_2(Y\times Z,\Cal Y\times\Cal Z,
\rho\times\mu)$.

If $\Cal F$ is an $L_2$-dense class of functions $f(y_1,z,y_2)$ in
the product space $(Y_1\times Z\times Y_2,\Cal Y_1\times\Cal Z
\times Y_2)$, with some parameter $D\ge1$ and exponent $L\ge0$, 
then also the classes $\Cal F_\mu=\{P_\mu f,\; f\in \Cal F\}$ and 
$\Cal G_\mu=\{\frac12Q_\mu f=\frac12(f-P_\mu f),\; f\in\Cal F\}$ 
are $L_2$-dense classes with exponent $L$ and parameter~$D$ in the
spaces $(Y_1\times Y_2,\Cal Y_1\times \Cal Y_2)$ and $(Y_1\times
Z\times Y_2,\Cal Y_1\times\Cal Z\times\Cal Y_2)$ respectively.}

\medskip
This corollary is a simple consequence of Proposition~9.3 if we
apply it with $(Y,\Cal Y)=(Y_1\times Y_2,\Cal Y_1\times\Cal Y_2)$
and take the natural mapping $f((y_1,y_2),z)\to f(y_1,z,y_2)$ of a
function from the space $(Y\times Z,\Cal Y\times \Cal Z)$ to a
function on $(Y_1\times Z\times Y_2,\Cal Y_1\times\Cal Z\times
\Cal Y_2)$. Beside this, we apply that measure on
$(Y_1\times Z\times Y_2,\Cal Y_1\times \Cal Z\times \Cal Y_2)$
which is the image of the product measure $\rho\times\mu$ with
respect to the map induced by the above transformation on the space
of measures.

Proposition 9.3, more precisely its corollary implies Theorem 9.2,
since it implies that the operators $P_s$, $Q_s$, $1\le s\le k$,
applied in Theorem~9.2 do not increase the $L_2(\mu)$ norm of a
function $f$, and it is also clear that the norm of $P_s$ is
bounded by 1 the norm of $Q_s=I-P_s$ is bounded by 2 as an operator
from $L_\infty$ spaces to $L_\infty$ spaces. The corollary of
Proposition~9.3 also implies that if $\Cal F$ is an $L_2$-dense class
of functions with parameter $D$ and exponent~$L$, then the same
property holds for the classes of functions
$\Cal F_{P_s}=\{P_sf\colon\; f\in \Cal F\}$ and
$\Cal F_{Q_s}=\{\frac12Q_sf\colon\; f\in \Cal F\}$,
$1\le s\le k$. These relations together with the identity
$f_V=\left(\prodd\limits_{s\in\{1,\dots,k\}\setminus V}P_s
\prod\limits_{s\in V}Q_s\right)f$
imply Theorem~9.2.

\medskip\noindent
{\it Proof of Proposition 9.3.}\/ The Schwarz inequality yields that
$P_\mu(f)^2\le\int f(y,z)^2\mu(\,dz)$, and integrating this
inequality with respect to the probability measure $\rho(\,dy)$ we
get inequality~(9.7). Also the inequality
$$
\align
\int Q_\mu f(y,z)^2\rho(\,dy)\mu(\,dz)&=\int [f(y,z)-P_\mu
f(y,z)]^2\rho(\,dy)\mu(\,dz) \\
&\le\int f(y,z)^2\rho(\,dy)\mu(\,dz)
\endalign
$$
holds, and this is relation (9.8). This follows for instance from
the observation that the functions $f(y,z)-P_\mu f(y,z)$ and
$P_\mu f(y,z)$ are orthogonal in the space
$L_2(Y\times Z,\Cal Y\times\Cal Z,\rho\times\mu)$.

Let us consider an arbitrary probability measure $\rho$ on the space
$(Y,\Cal Y)$. To prove that $\Cal F_\mu$ is an $L_2$-dense class with
parameter~$D$ and exponent~$L$ if the same relation holds for
$\Cal F$ we have to find for all $0<\e\le1$ a set
$\{f_1,\dots,f_m\}\subset \Cal F_\mu$, $1\le j\le m$ with
$m\le D \e^{-L}$ elements, such that
$\inff_{1\le j\le m}\int(f_j-f)^2\,d\rho\le \e^2$ for all
$f\in \Cal F_\mu$. But a similar property holds for $\Cal F$ in
the space $Y\times Z$ with the probability measure $\rho\times\mu$.
This property together with the $L_2$ contraction property of
$P_\mu$ formulated in (9.7) imply that $\Cal F_\mu$ is an
$L_2$-dense class.

To prove that $\Cal G_\mu$ is also $L_2$-dense with parameter $D$ and
exponent $L$ under the same condition we have to find for all numbers
$0<\e\le1$ and probability measures $\rho$ on $Y\times Z$ a subset
$\{g_1,\dots,g_m\}\subset\Cal G_\mu$ with $m\le D\e^{-L}$ elements
such that $\inff_{1\le j\le m}\int (g_j-g)^2\,d\rho\le \e^2$ for all
$g\in\Cal G_\mu$.

To show this let us consider the probability measure
$\tilde\rho=\frac12(\rho+\bar\rho\times\mu)$ on $(Y\times Z,\Cal
Y\times\Cal Z)$, where $\bar\rho$ is the projection of the measure
$\rho$ to $(Y,\Cal Y)$, i.e. $\bar\rho(A)=\rho(A\times Z)$ for all
$A\in\Cal Y$, take a class of function $\Cal F_0(\e,\tilde \rho)
=\{f_1,\dots,f_m\}\subset\Cal F$ with $m\le D\e^{-L}$ elements
such that $\inff_{1\le j\le m}\int (f_j-f)^2\,d\tilde\rho\le \e^2$
for all $f\in\Cal F$, and put
$\{g_1,\dots,g_m\}=\{\frac12Q_\mu f_1,\dots,\frac12Q_\mu f_m\}$.
All functions $g\in\Cal G_\mu$ can be written in the form
$g=\frac12Q_\mu f$ with some $f\in \Cal F$, and there exists some
function $f_j\in\Cal F_0(\e,\tilde\rho)$ such that
$\int (f-f_m)^2\,d\tilde\rho\le\e^2$. Hence to complete the proof
of Proposition~9.3 it is enough to show that $\int\frac14(Q_\mu f
-Q_\mu\bar f)^2\,d\rho\le\int(f-\bar f)^2\,d\tilde\rho$ for all
pairs $f,\bar f\in\Cal F$. This inequality holds, since
$\int\frac14(Q_\mu f-Q_\mu\bar f)^2\,d\rho\le\int\frac12(f-\bar
f)^2\,d\rho+\int\frac12(P_\mu f-P_\mu\bar f)^2\,d\rho$,  and
$\int(P_\mu f-P_\mu\bar f)^2\,d\rho=\int(P_\mu f-P_\mu\bar
f)^2\,d\bar\rho\le\int(f-\bar f)^2\,d(\bar\rho\times\mu)$ by
formula 9.7. The above relations imply that $\int\frac14(Q_\mu
f-Q\mu\bar f)^2\,d\rho\le \int(f-\bar f)^2\frac12
d\,(\rho+\bar\rho\times\mu)=\int(f-\bar f)^2d\,\tilde\rho$ as we
have claimed.

\medskip
Now we shall discuss the relation between Theorem~$8.1'$ and
Theorem~8.3 and between Theorem 8.2 and Theorem 8.4. First we
show that Theorem~8.1 (or Theorem~$8.1'$) is equivalent
to the estimate~$(8.10')$ in the corollary of Theorem~8.3 which 
is slightly weaker than the estimate~(8.10) of Theorem~8.3.  We 
also claim that Theorems~8.2 and~8.4 are equivalent. Both in 
Theorem~8.2 and in Theorem~8.4 we can restrict our attention to 
the case when the class of functions $\Cal F$ is countable, 
since the case of countably approximable classes can be simply 
reduced to this situation. Let us remark that integration with 
respect to the measure $\mu_n-\mu$ in the definition~(4.8) of 
the integral $J_{n,k}(f)$ yields some kind of normalization 
which is missing in the definition of the $U$-statistics 
$I_{n,k}(f)$. This is the cause why degenerate $U$-statistics 
had to be considered in Theorems~8.3 and~8.4. The deduction of 
the corollary of Theorem~8.3 from Theorems~$8.1'$ or of 
Theorem~8.4 from Theorem~8.2 is fairly simple if the underlying 
probability measure $\mu$ is non-atomic, since in this case the 
identity $I_{n,k}(f)=J_{n,k}(f)$ holds for a canonical function 
with respect to the measure $\mu$. Let us remark that the 
non-atomic property of the measure $\mu$ is needed in this 
argument not only because of the conditions of Theorems~$8.1'$ 
and~8.2, but since in the proof of the above identity we need 
the identity $\int f(x_1,\dots,x_k)\mu(\,dx_j)\equiv0$ in the 
case when the domain of integration is not the whole space~$X$ 
but the set $X\setminus\{x_1,\dots,x_{j-1},x_{j+1},\dots,x_k\}$.

The case of possibly atomic measures $\mu$ can be simply reduced
to the case of non-atomic measures by means of the following
enlargement of the space $(X,\Cal X,\mu)$. Let us introduce the
product space $(\bar X,\bar{\Cal X},\bar\mu)=(X,\Cal X,\mu)
\times([0,1],\Cal B,\lambda)$, where $\Cal B$ is the $\sigma$-algebra
and $\lambda$ is the Lebesgue measure on $[0,1]$. Define the function
$\bar f((x_1,u_1),\dots,(x_k,u_k))=f(x_1,\dots,x_k)$ in this
enlarged space. Then $I_{n,k}(f)=I_{n,k}(\bar f)$, the measure
$\bar\mu=\mu\times\lambda$ is non-atomic, and $\bar f$ is canonical
with respect to~$\bar\mu$ if $f$ is canonical with respect to~$\mu$.
Hence the corollary of Theorem~8.3 and Theorem~8.4 can be derived
from Theorems~$8.1'$ and~8.2 respectively by proving them first for
their counterpart in the above constructed enlarged space with the
above defined functions.

Also Theorems~$8.1'$ and~8.2 can be derived from Theorems~8.3
and~8.4 respectively, but this is a much harder problem. To do
this let us observe that a random integral $J_{n,k}(f)$ can
be written as a sum of $U$-statistics of different order, and it
can also be expressed as a sum of degenerate $U$-statistics if
Hoeffding's decomposition is applied for each $U$-statistic in
this sum. We show that the coefficients of the degenerate
$U$-statistics in the above representation have relatively small
coefficients. This result is formulated in the following
Theorem~9.4. To make its content more understandable I shall 
describe it in the special case of two-fold random integrals in 
a more explicit form in Corollary~2 of Theorem~9.4 at the end of
this section.

\medskip\noindent
{\bf Theorem 9.4. (Decomposition of a multiple random integral with
respect to a normalized empirical measure to a linear combination of
degenerate $U$-statistics).} {\it Let a non-atomic measure~$\mu$ be
given on a measurable space $(X,\Cal X)$ together with a sequence of
independent, $\mu$-distributed random variables $\xi_1,\dots,\xi_n$.
Take a function $f(x_1,\dots,x_k)$ of $k$ variables on the
space $(X^k,\Cal X^k)$ such that
$$
\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)<\infty
$$
and consider the empirical distribution function $\mu_n$ of the
sequence $\xi_1,\dots,\xi_n$ introduced in (4.5) together with the
$k$-fold random integral $J_{n,k}(f)$ of the function $f$ defined in
(4.8). The identity
$$
J_{n,k}(f)=\sum_{V\subset\{1,\dots,k\}}C(n,k,V)n^{-|V|/2}
I_{n,|V|}(f_V) \tag9.9
$$
holds with the set of (canonical) functions $f_V(x_j,\;j\in V)$
(with respect to the measure $\mu$) defined in formula (9.2)
together with some real numbers $C(n,k,V)$,
$V\subset\{1,\dots,k\}$, where $I_{n,|V|}(f_V)$ denotes the
(degenerate) $U$-statistic of order $|V|$ with the
random variables $\xi_1,\dots,\xi_n$ and kernel function $f_V$.
The constants  $C(n,k,V)$ in formula (9.9) satisfy the inequality
$|C(n,k,V)|\le C(k)$ with some constant $C(k)$ depending only
on the order $k$ of the integral $J_{n,k}(f)$. The relations
$\limm_{n\to\infty}C(n,k,V)=C(k,V)$ hold with some appropriate
constant such that $0\le |C(k,V)|<\infty$, and
$C(n,k,\{1,\dots,k\})=1$ for $V=\{1,\dots,k\}$.}

\medskip\noindent
{\it Remark:}\/ Some considerations show that the coefficients
$C(n,k,V)$ in formula (9.9) depend only on the cardinality $|V|$
of the set $V$, i.e. $C(n,k,V)=C(n,k,|V|)$ can be written. This
fact will be not applied in this work.

\medskip
Theorems~$8.1'$ and~8.2 can be simply derived from Theorems~8.3
and~8.4 respectively with the help of Theorem~9.4. Indeed, to
get Theorem~$8.1'$ observe that formula~(9.9) implies the inequality
$$
P(|J_{n,k}(f)|>u)\le \sum_{V\subset\{1,\dots,k\}}
P\(n^{-|V|/2}|I_{n,|V|}(f_V)|>\frac u{2^kC(k)}\) \tag9.10
$$
with a constant $C(k)$ satisfying the inequality $C(n,k,|V|)\le
C(k)$ for all coefficients $C(n,k,|V|)$ in~(9.9). Hence
Theorem~$8.1'$ follows from Theorem~8.3 and relations~(9.4)
and~$(9.4')$ in Theorem~9.2 by which the $L_2$-norm of the
functions $f_V$ is bounded by the $L_2$-norm of the function~$f$
and the $L_\infty$-norm of $f_V$ is bounded by the $2^{|V|}$-times
the $L_\infty$-norm or $f$. It is enough to estimate each term at
the right-hand side of (9.10) by means of Theorem~8.3. It can be
assumed that $2^kC(k)>1$. Let us first assume that also the
inequality $\frac u{2^kC(k) \sigma}\ge1$ holds. In this case
formula $(8.3')$ in Theorem~$8.1'$ can be obtained by means of the
estimation of each term at the right-hand side of (9.10). Observe
that $\exp\left\{-\alpha\(\frac u{2^kC(k)\sigma}\)^{2/s}\right\}\le
\exp\left\{-\alpha\(\frac u{2^kC(k)\sigma}\)^{2/k}\right\}$ for all
$s\le k$ if $\frac u{2^kC(k)\sigma}\ge1$. In the other case, if
$\frac u{2^kC(k)\sigma}\le1$, formula~$(8.3')$ holds again with a
sufficiently large $C>0$, because in this case its right-hand side
is greater than~1.

Theorem~8.2 can be similarly derived from Theorem~8.4 by observing
that relation (9.10) remains valid if $|J_{n,k}(f)|$ is replaced
by $\supp_{f\in\Cal F}|J_{n,k}(f)|$ and $|I_{n,|V|}(f_V)|$ by
$\supp_{f_V\in\Cal F_V} |I_{n,|V|}(f_V)|$ in it, and we have the
right to choose the constant~$M$ in formula~(8.6) of Theorem~8.2
sufficiently large. The only difference in the argument is that
beside formulas~(9.4) and~$(9.4')$ the last statement of
Theorem~9.2 also has to be applied in this case. It tells that if
$\Cal F$ is an $L_2$-dense class of functions on a space
$(X^k,\Cal X^k)$, then the classes of functions
$\Cal F_V=\{2^{-|V|}f_V\colon\; f\in\Cal F\}$ are also $L_2$-dense
classes of functions for all $V\subset\{1,\dots,k\}$ with the same
exponent and parameter.

\medskip
Next I make some comments about the content of Theorem~9.4. The
expression $J_{n,k}(f)$ was defined as a $k$-fold random integral,
where the diagonals were omitted from the domain of integration. We
have integrated with respect to the signed measure $\mu_n-\mu$,
which means some kind of normalization. Thus it is not surprising
that the tail distribution behaviour of $J_{n,k}(f)$ is similar to 
that of degenerate $U$-statistics. Theorem~9.4 has such a consequence.
Formula~(9.9) expresses the random integral $J_{n,k}(f)$ as a
linear combination of degenerate $U$-statistics of different order.
This is similar to the Hoeffding decomposition of the $U$-statistic
$I_{n,k}(f)$ where the same kernel functions $f_V$ appear. But the
coefficients $C(n,k,|V|)n^{-|V|/2}$ of the terms $I_{n,|V|}(f_V)$
in the expansion (9.9) are small. On the other hand, they do not
have to disappear. In particular, the expansion (9.9) may contain
a non-zero constant term. In such a case the expected value
$EJ_{n,k}(f)$ is not equal to zero. But even in this case it can be
bounded by a finite number not depending on the sample size~$n$. 
Next I show an example for such a random integral $J_{n,2}(f)$ 
where $EJ_{n,2}(f)\neq0$.

Let us choose a sequence of independent random variables
$\xi_1,\dots,\xi_n$ with uniform distribution on the unit interval,
let $\mu_n$ denote its empirical distribution, let $f=f(x,y)$ denote
the indicator function of the unit square, i.e. let $f(x,y)=1$ if
$0\le x,y\le1$, and $f(x,y)=0$ otherwise. Let us consider the
random integral $2J_{n,2}(f)=n\int_{x\neq y} f(x,y)
(\mu_n(\,dx)-\,dx)(\mu_n(\,dy)-dy)$, and calculate its expected
value $E2J_{n,2}(f)$. By adjusting the diagonal $x=y$ to the domain
of integration and taking out the contribution obtained in this
way we get that $E2J_{n,2}(f)=nE(\int_0^1\(\mu_n(\,dx)-\mu(\,dx)\)^2
-n^2\cdot\frac1{n^2}=-1$. (The last term is the integral of the
function $f(x,y)$ on the diagonal $x=y$ with respect to the product
measure $\mu_n\times\mu_n$ which equals
$(\mu_n-\mu)\times(\mu_n-\mu)$ on the diagonal.)

Now I turn to the proof of Theorem~9.4.

\medskip\noindent
{\it Proof of Theorem 9.4.}\/ Let us first introduce the (random)
probability measures $\mu^{(l)}$, $1\le l\le n$, concentrated in the
(single) sample points $\xi_l$, i.e. let $\mu^{(l)}(A)=1$ if
$\xi_l\in A$, and $\mu^{(l)}(A)=0$ if $\xi_l\notin A$, $A\in \Cal A$,
$1\le l\le n$. Then
$\mu_n-\mu=\frac1n\summ_{l=1}^n\(\mu^{(l)}-\mu\)$, and
formula~(4.8) can be rewritten as
$$ \allowdisplaybreaks
\align
J_{n,k}(f)=\dfrac1{n^{k/2}k!}&\sum_{(l_1,\dots,l_k)\colon\, 1\le l_j\le n,
\,1\le j\le k,} \int' f(x_1,\dots,x_k)  \tag9.11  \\
&\qquad \(\mu^{(l_1)}(\,dx_1)-\mu(\,dx_1)\)\dots
\(\mu^{(l_k)}(\,dx_k)-\mu(\,dx_k)\).
\endalign
$$
To rearrange the above sum in a way more appropriate for us let us
introduce the class of all partitions $\Cal P=\Cal P_k$ of the set
$\{1,2,\dots,k\}$. For a partition $P=\{R_1,\dots,R_u\}$
$\bigcupp_{j=1}^u R_j=\{1,\dots,k\}$, $R_j\cap R_l=\emptyset$,
$1\le j<l\le u$, the sets $R_j$, $1\le j\le u$, will be called the
components of the partition~$P$. Given a sequence $(l_1,\dots,l_k)$,
$1\le l_j\le n$, $1\le j\le k$, of length $k$ let $P_H(l_1,\dots,l_k)$
denote that partition of the set $\{1,\dots,k\}$ in which two points
$s$ and $t$, $1\le s,t\le k$, belong to the same component of this
partition if and only if $l_s=l_t$. For a partition $P\in\Cal P_k$
let us define the set $\Cal H(P)=\Cal H_n(P)$ consisting of sequences
$(l_1,\dots,l_k)$ with $1\le l_j\le n$ for all $1\le j\le k$ as
$\Cal H(P)=\{(l_1,\dots,l_k)\colon\;1\le l_j\le n,\, 1\le j\le k,
P_H(l_1,\dots,l_k)=P\}$.

Let us rewrite formula (9.11) in the form
$$
\align
J_{n,k}(f)=\dfrac1{n^{k/2}k!}\sum_{P\in \Cal P} &\,\,
\sum_{(l_1,\dots, l_k)\colon\; (l_1,\dots,l_k)\in \Cal H(P)}
\int' f(x_1,\dots,x_k)   \tag9.12 \\
&\qquad \(\mu^{(l_1)}(\,dx_1)-\mu(\,dx_1)\)\dots
\(\mu^{(l_k)}(\,dx_k)-\mu(\,dx_k)\).
\endalign
$$

Let us remember that the diagonals $x_s=x_t$, $s\neq t$, were
omitted from the domain of integration in the formula defining
$J_{n,k}(f)$. This implies that if $l_s=l_t$ for some $s\neq t$,
then the measure $\mu^{(l_s)}(\,dx_s)\mu^{(l_t)}(\,dx_t)$ has zero
measure in the domain of integration. We have to understand the
cancellation effects caused because of this fact. It will be shown
that because of these cancellations the expression in formula~(9.12)
can be rewritten as a linear combination of degenerate
$U$-statistics with not too large coefficients. Beside this, it
will be seen from the calculations that the same degenerate
$U$-statistics $I_{n,|V|}(f_V)$ appear in the representation of
$J_{n,k}(f)$ as in formula (9.2). This is a natural approach, but
the detailed proof demands some rather unpleasant calculation.

Let us fix some partition $P\in\Cal P$ and investigate the integrals
in the internal sum at the right-hand side of~(9.12) corresponding
to the sequences $(l_1,\dots,l_k) \in \Cal H(P)$. For the sake of
a better understanding let us first consider such a partition $P\in
\Cal P$ which has a component of the form $\{1,\dots,s\}$ with some
$s\ge2$. The products of measures by which we have to integrate in
this case contain a part of length $s$ of the form
$\(\mu^{(l)}(dx_1)-\mu(dx_1)\)\dots \(\mu^{(l)}(dx_s)-\mu(dx_s)\)$
This part of the product measure can be rewritten in the domain of
integration as
$$
\align
&\summ_{j=1}^s (-1)^{s-1}\mu(\,dx_1)\dots\mu(\,dx_{j-1})
\mu^{(l)}(\,dx_j)\mu(\,dx_{j+1})\dots\mu(\,dx_s)
+(-1)^s\mu(dx_1)\dots\mu(dx_s)\\
&\qquad=\summ_{j=1}^s (-1)^{s-1}\mu(\,dx_1)\dots\mu(\,dx_{j-1})
(\mu^{(l)}(\,dx_j)-\mu(\,dx_l))\mu(\,dx_{j+1})\dots\mu(\,dx_s) \\
&\qquad\qquad\qquad +(-1)^{s-1}(s-1)\mu(dx_1)\dots\mu(dx_s). \tag9.13
\endalign
$$
Here we have exploited that all other terms of this product disappear
in the domain of integration which does not contain the diagonals.
Let us also observe that the term
$(-1)^{s-1}(s-1)\mu(dx_1)\dots\mu(dx_j)$ appears $n$-times if we
sum up for all $1\le l\le n$. We have assumed that $s\ge2$, since
the case $s=1$ is slightly different. In this case only the term
$\mu^{(l)}(dx_1)-\mu(\,dx_1)$ appears, i.e.\ have to put no
additional term consisting only of (deterministic) measures $\mu$.

More generally, let us fix some partition $P=\{R_1,\dots,R_u\}$,
consider the integral corresponding to a sequence
$(l_1,\dots,l_k)\in\Cal H(P)$ in the internal sum of (9.12), and
let us rewrite it as the sum of integrals with respect to product
measures with components of the form $\mu^{(l_s)}(\,dx_s)-\mu(\,dx_s)$
or $\mu(\,dx_s)$, where all measures $\mu^{(l_s)}$ appearing in a
product measure are different. Such a representation can be given,
similarly to the calculation leading to relation~(9.13), only the
notations will be more complicated. To write down what we get first
we define a set $\Cal T(P)$ whose elements are certain subsets of
$\{1,\dots,k\}$ depending on the partition $P=\{R_1,\dots,R_u\}$
together with a subset $\bar{\Cal T}(P)$ of it. The elements of
the set $\Cal T(P)$ are those sets
$\{j_1,\dots,j_{u'}\}\subset\{1,\dots,k\}$, $u'\le u$, which 
contain zero or one element from each component $R_j$, 
$1\le j\le u$, of the partition~$P$. Let 
$\bar{\Cal T}(P)\subset\Cal T(P)$ consist of those sets 
$V=\{j_1,\dots,j_{u'}\}\in \Cal T(P)$ which satisfy the following 
additional condition: If some component $R_t=\{b_t\}$, 
$1\le t\le u$, of the partition $P$ consists of only one point, 
then all sets $V\in\bar{\Cal T}(P)\subset \Cal T(P)$ contain this 
point $b_t$. With the help of the above quantities we can write in 
the case $(l_1,\dots,l_k)\in\Cal H(P)$, similarly to the 
calculation in~(9.13), that
$$ \allowdisplaybreaks
\align
&\int' f(x_1,\dots,x_k) \(\mu^{(l_1)}(\,dx_1)-\mu(\,dx_1)\)\dots
\(\mu^{(l_k)}(\,dx_k)-\mu(\,dx_k)\)  \tag9.14   \\
&\qquad=\sum_{V\in\bar{\Cal T}(P)} \alpha(V,P)\int f(x_1,\dots,x_k)
\prod_{j\in V}
\(\mu^{(l_j)}(\,dx_j)-\mu(\,dx_j)\) \prod_{j'\in\{1,\dots,k\}
\setminus V} \mu(\,dx_{j'})
\endalign
$$
with some appropriate finite constants $\alpha(V,P)$. These
constants could be calculated explicitly, but it is enough for us
to know that they depend only on the partition $P$ and the set
$V\in\bar{\Cal T}(P)$. On the other hand, it is important to observe
that a term with non-zero coefficient $\alpha(V,P)$ appears at the
right-hand side of (9.14) only for $V\in\bar{\Cal T}(P)$. This was
the reason for the introduction of the sets $\bar{\Cal T}(P)$. 
To understand relation~(9.14) observe that if $R_t=\{b_t\}$ is a 
one-point component of the  the partition $P$, then only the term
$\mu^{(l_{b_t})}(\,dx_{b_t})-\mu(\,dx_{b_t})$ appears as a 
component depending on $b_t$ in the product of measures at the
right-hand side of (9.14), and a component of the form 
$\mu(\,dx_{b_t})$ is missing. Hence a product 
$\prod\limits_{j'\in\{1,\dots,k\}\setminus V}\mu(\,dx_{j'})$ cannot 
appear at the right-hand side of (9.14) if $V\notin\bar{\Cal T}(P)$.

Let me remark that at the right-hand side of (9.14) $\int$ was
written and not $\int'$, i.e. the diagonal was not omitted from
the domain of integration. This is allowed, since the measure $\mu$
is non-atomic, and this also has the consequence that the sample
points $\xi_1,\dots,\xi_n$ are different with probability~1.

Formula (9.14) can be rewritten, by expressing its right-hand side
with the help of the random variables $\xi_l$ instead of the
measures $\mu^{(l)}$ as
$$ \allowdisplaybreaks
\align
& \int'f(x_1,\dots,x_k) \(\mu^{(l_1)}(\,dx_1)-\mu(\,dx_1)\)\dots
\(\mu^{(l_k)}(\,dx_k)-\mu(\,dx_k)\) \tag9.15    \\
&\qquad=\sum_{V\in\bar{\Cal T}(P)} \alpha(V,P)
\(\prod_{j'\in\{1,\dots,k\}\setminus V} P_{\mu,j'}
\prod_{j\in V} Q_{\mu,j}\) f(\xi_{l_j},\,j\in V)
\endalign
$$
if $(l_1,\dots,l_k)\in\Cal H(P)$. Here $Q_{\mu,j}=I-P_{\mu,j}$ is
the operator $Q_\mu$ defined in
$(9.6')$, with the choice $Y_1$ which is the product of the
first $j-1$ components of $X^k$, $Z$ is the $j$-th component and
$Y_2$ is the product of the last $k-j$ components of the product
space $X^k$. The operator $P_{\mu,j'}$ is the operator $P_\mu$
defined in $(9.5')$ with the choice of $Y_1$ as the product of the
first $j'-1$, $Z$ the $j$-th component and $Y_2$ as the product of
the last $k-j'$ components of the space $X^k$. To see why formula
(9.15) holds we have to understand that integration with respect to
$\(\mu^{(l_j)}(\,dx_j)-\mu(\,dx_j)\)$ means the application of the
operator $Q_{\mu,j}$ and then putting the value $\xi_{l_j}$ in the
argument $x_j$, while integration with respect to $\mu(\,dx_{j'})$
means the application of the operator $P_{\mu,j'}$. Beside this,
the operators $Q_{\mu,j}$ and $P_{\mu,j'}$ are exchangeable.

Fix some partition $P\in\Cal P_k$, a set $V\in\bar{\Cal T}(P)$
and sum up the expressions at the right-hand side of (9.15) with
this set~$V$ for all sequences $(l_1,\dots,l_k)\in\Cal H(P)$. We
get that
$$
\alpha(V,P) \!\!\!\!\!\! \sum_{(l_1,\dots,l_k)\in\Cal H(P)}
\(\prod_{j'\in\{1,\dots,k\}\setminus V}  \!\!\!\! P_{\mu,j'}
\prod_{j\in V} Q_{\mu,j}\) f(\xi_{l_j},\,j\in V)
=\bar\alpha(V,P,k,n)I_{n,|V|}(f_V) \tag9.16
$$
where $I_{n,|V|}(f_v)$ is the $U$-statistic of order $|V|$ with the
kernel function
$$
f_V(x_j, j\in V)=\(\prodd_{j'\in\{1,\dots,k\}\setminus V}
P_{\mu,j'} \prodd_{j\in V} Q_{\mu,j}\)f \tag9.17
$$
with that function on $f(x_1,\dots,x_k)$ which is considered in
Theorem~9.4 and some appropriate coefficients $\bar\alpha(V,P,k,n)$
at the right-hand side of~(9.16). These coefficients could be
explicitly calculated. We do not need an explicit formula
for $\bar\alpha(V,P,k,n)$, but we shall need the inequality
$|\bar\alpha(V,P,k,n)|\le D(k)n^{\beta(P,V)}$, where
$\beta(P,V)=u-|V|$ is the number of those components $R_j$,
$1\le j\le u$, of the partition $P$ for which $R_j\cap V=\emptyset$,
(here $u$ denotes the number of components in the partition~$P$), 
and the constant $D(k)<\infty$ depends only on the multiplicity~$k$
of the integral $J_{n,k}(f)$.

To show that $|\bar\alpha(V,P,k,n)|\le D(k)n^{\beta(P,V)}$ let us
observe that if we fix the coordinates $l_j$, $j\in V$, in an
arbitrary way and sum up the expression at left-hand side of~(9.16)
for the remaining indices $l_{j'}$, $j'\notin V$, then we get the
term depending on the variables $\xi_{l_j}$, $j\in V$, in the sum
defining the $U$-statistic $I_{n,|V|}(f_V)$ multiplied by
$\bar\alpha(V,P,k,n)$. Hence to get a good estimate on
$\bar\alpha(V,P,k,n)$ the number of the vectors
$(l_j,j\notin V)$ taking part in the summation at the left-hand side
of~(9.16) has to be well bounded. For this aim let us consider the
class of vectors $(l_1,\dots,l_k)\in\Cal H(P)$. Two coordinates
$l_{j'}$ and $l_{j''}$ must agree if their indices $j'$ and $j''$
belong to the same component of the partition $P$. Beside this, if
the number $j$ is contained in such a component $R_t$ of the
partition $P$ for which $R_t\cap V\neq\emptyset$, then the
coordinate $l_j$ of these vectors is fixed. Hence the value $l_{j'}$
of those non-fixed coordinates whose indices $j'$ belong to the same
component $R_t$ of the partition $P$ agree and only such components
$R_t$ have to be considered for which $R_t\cap V=\emptyset$. This
yields the upper bound $n^{\beta(P,V)}$ for the number of possible
choices of the indices $l_{j'}$, $j'\notin V$. A more careful
consideration shows that the finite limit
$$
C(k,V,P)=\limm_{n\to\infty}n^{-\beta(P,V)}
\bar\alpha(V,P,k,n), \qquad |C(k,V,P)|<\infty, \tag9.18
$$
also exists.

We get by applying relations~(9.12) and~(9.15) and summing up
relation~(9.16) first for all $V\in\bar{\Cal T}(P)$ for a partition
$P\in\Cal P_k$ and then for all $P\in \Cal P$ that the identity
$$
J_{n,k}(f)=\sum_{V\subset \{1,2,\dots,k\}} n^{-|V|/2}C(n,k,V)
\frac1{k!}\sum\Sb 1\le l_j\le n,\\ l_j\neq l_{j'} \text{ if }j\neq j'
\text{ for } j\in V\endSb f_V(\xi_{l_j},\,j\in V) \tag9.19
$$
holds with the functions $f_V(x_j,\,j\in V)$ defined in~(9.17) for
all $V\subset\{1,\dots,k\}$ and some appropriate coefficients
$C(n,k,V)$. We shall show that these coefficients satisfy the
inequality $|C(n,k,V)|\le C(k)$ with some constant $C(k)>0$.
Beside this, it is not difficult to see that the identity
$C(n,k,\{1,\dots,k\})=1$ holds. To see the estimate
$|C(n,k,V)|\le C(k)$ observe that $n^{-|V|/2}C(n,k,|V|)$
can be written as a sum of finitely many terms, (their number is
less than a number depending only on~$k$) such that all of them
can be bounded by a number of the form
$\frac{|\bar\alpha(V,P,k,n)|}{n^{k/2}k!}\le D(k)n^{-k/2+\beta(P,V)}$
with some partition $P$ and the number $\beta(P,V)$ introduced
after formula (9.16) with some $P\in\Cal P_k$ and
$V\in\bar{\Cal T}(P)$. Hence it is enough to show that
$-\frac k2+\beta(P,V)\le-\frac{|V|}2$, i.e.
$\beta(P,V)\le \frac{k-|V|}2$ if $V\in\bar{\Cal T}(P)$. This
relation clearly holds, since $\beta(P,V)$ is the number of
components of such a partition of a set with less than or equal to
$k-|V|$ elements whose components have at least 2~elements.

Relation (9.19) can be rewritten as $J_{n,k}(f)=\!\!\!\!
\summ_{V\subset \{1,2,\dots,k\}}\!\!\!\! C(n,k,V) n^{-|V|/2}
I_{n,|V|}(f_V)$, where $I_{n,|V|}(f_V)$ is the $U$-statistic with
the random variables $\xi_1,\dots,\xi_n$ and the kernel function
$f_V$ defined in (9.17). This kernel function agrees with the
function $f_V$ defined in (9.2). We have also seen that the
coefficients $C(n,k,V)$ satisfy the inequality stated
in~Theorem~9.4. Relation (9.18) together with the bound on the
terms $\beta(P,V)$ also imply that the finite limits
$\limm_{n\to\infty}C(n,k,V)=C(k,V)$ also exist. Theorem~9.4 is
proved.

\medskip
Two corollaries of Theorem~9.4 will be formulated. The first one
explains the content of conditions (8.2) and (8.5) in
Theorems~8.1---8.4.

\medskip\noindent
{\bf Corollary~1 of Theorem 9.4.}\/ {\it If $I_{n,k}(f)$
is a degenerate $U$-statistic of order $k$ with some kernel function
$f$, then
$$
\align
E\(n^{-k/2}I_{n,k}(f)\)^2&=\frac{n(n-1)\cdots(n-k+1)}{k!n^k} \int
\text{\rm Sym}\, f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)\\
&\le\frac1{k!}\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k),
\tag9.20 \endalign
$$
where $\mu$ is the distribution of the random variables taking part
in the definition of the $U$-statistic~$I_{n,k}(f)$, and
$\text{\rm Sym}\,f$ is the symmetrization of the function~$f$.
The $k$-fold multiple random integral $J_{k,n}(f)$ with an arbitrary
square integrable kernel function~$f$  satisfies the inequality
$$
EJ_{n,k}(f)^2\le\bar C(k)\int f^2(x_1,\dots,x_k)
\mu(\,dx_1)\dots\mu(\,dx_k)
$$
with some constant $\bar C(k)$ depending only on the order~$k$ of the
integral~$J_{n,k}(f)$.}

\medskip\noindent
{\it Proof of Corollary~1 of Theorem 9.4.} The identity
$$
E(n^{-k/2}I_{n,k}(f))^2=\frac1{(k!)^2n^k}\sum{\vphantom \sum}'
Ef(\xi_{l_1},\dots,\xi_{\l_k})f(\xi_{l_1'},\dots,\xi_{\l_k'}) \tag9.21
$$
holds, where the prime in $\sum'$ means that summation is taken for
such pairs of $k$-tuples $(l_1,\dots,l_k)$, $(l_1',\dots,l_k')$,
$1\le l_j,l_j'\le n$, for which $l_j\neq l_{j'}$ and
$l'_j\neq l_{j'}'$ if $j\neq j'$. The degeneracy of the
$U$-statistic $I_{n,k}(f)$ implies that
$Ef(\xi_{l_1},\dots,\xi_{\l_k})f(\xi_{l_1'},\dots,\xi_{\l_k'})=0$
if the two sets $\{l_1,\dots,\l_k\}$ and $\{l_1',\dots,\l_k'\}$
differ. This can be seen by taking such an index $l_j$ from the
first $k$-tuple which does not appear in the second one, and by
observing that the conditional expectation of the product we
consider equals zero by the degeneracy condition of the
$U$-statistic under the condition that the value of all random
variables except that of $\xi_{l_j}$ is fixed in this product.
On the other hand,
$$
Ef(\xi_{l_1},\dots,\xi_{\l_k})f(\xi_{l_1'},\dots,\xi_{\l_k'})=
\int f(x_1,\dots,x_k)f(x_{\pi(1)},\dots,x_{\pi(k)})\mu(\,dx_1)
\dots\mu(\,dx_k)
$$
if $(l_1',\dots,l_k')=(\pi(l_1),\dots,\pi(l_k))$ with some
$(\pi(1),\dots,\pi(k))\in\Pi_k$, where $\Pi_k$ denotes the set
of all permutations of the set $\{1,\dots,k\}$. By summing up
the above identities for all pairs $(l_1,\dots,l_k)$ and
$(l'_1,\dots,l'_k)$ and by applying formula~(9.21) we get the
identity at the left-hand side of formula (9.20). The second
relation in~(9.20) is obvious.

The bound for $J_{n,k}(f)$ follows from Theorem~9.4, formula (9.4)
in Theorem 9.2 by which the $L_2$-norm of the functions $f_V$ is
not greater than the $L_2$-norm of the function~$f$ and the bound
that formula~(9.20) yields for the second moment of the degenerate
$U$-statistics $n^{-|V|/2}I_{n,|V|}(f_V)$ appearing in the
expansion~(9.9).

\medskip
In Corollary~2 the decomposition (9.9) of a random integral
$J_{n,2}(f)$ of order 2 is described in an explicit form.

\medskip\noindent
{\bf Corollary 2 of Theorem 9.4.} {\it Let the random integral
$J_{n,2}(f)$ satisfy the conditions of Theorem 9.4. In this case
formula (9.9) can be written in the following explicit form:
$$
2J_{n,2}(f)=\frac2n I_{n,2}(f_{\{1,2\}})-\frac1n I_{n,1}(f_{\{1\}})
-\frac1n I_{n,1}(f_{\{2\}})-f_\emptyset  \tag$9.9'$
$$
with the functions
$$ \allowdisplaybreaks
\align
f_{\{1,2\}}(x,y)&=f(x,y)-\int f(x,y)\mu(\,dx)-
\int f(x,y)\mu(\,dy)+\int f(x,y)\mu(\,dx)\mu(\,dy),  \\
f_{\{1\}}(x)&=\int f(x,y)\mu(\,dy)-\int
f(x,y)\mu(\,dx)\mu(\,dy), \\
f_{\{2\}}(y)&=\int f(x,y)\mu(\,dx)-\int
f(x,y)\mu(\,dx)\mu(\,dy), \quad \text{and} \\
f_\emptyset&=\int f(x,y)\mu(\,dx)\mu(\,dy).
\endalign
$$
}

\beginsection 10. Multiple Wiener--It\^o integrals and their
properties.

In this section I present the definition of multiple Wiener--It\^o
integrals and some of their most important properties needed in the
proof of the results formulated in Section~8. First the notion of 
the white noise with some reference measure will be introduced, 
then multiple Wiener--It\^o integrals with respect to a white 
noise with some non-atomic reference measure will be defined. A 
most important result in the theory of multiple Wiener--It\^o 
integrals is the so-called diagram formula presented in 
Theorem~10.2A. It enables us to write the product of two 
Wiener--It\^o integrals in the form of a sum of Wiener--It\^o 
integrals. The proof of the diagram formula is given in Appendix~B.

Another interesting result about Wiener-It\^o integrals, formulated
at the end of this section in Theorem~10.5 states that the class
of random variables which can be written in the form of a sum of
Wiener--It\^o integrals of different order is sufficiently rich.
All random variables with finite second moment which are measurable
with respect to the $\sigma$-algebra generated by the (Gaussian)
random variables appearing in the underlying white noise
in the construction of multiple Wiener--It\^o integrals can
be written in such a form. This result explains why it is natural
to expect a result like the diagram formula. The product of two
Wiener--It\^o integrals is also measurable with respect to the
$\sigma$-algebra generated by the random variables in the underlying
white noise, hence if we know that such a product is square
integrable, then Theorem~10.5 implies that it can be written as the
sum of multiple Wiener--It\^o integrals of different order. The 
diagram formula is especially useful for us, because it yields an
explicit representation of a product of Wiener--It\^o integrals in 
the form of a sum of Wiener--It\^o integrals. I shall also give a 
heuristic explanation of the diagram formula which may indicate
why it has the form appearing in Theorem~10.2A. It also helps to 
find the analogue of the diagram formula for (random) integrals with 
respect to the product of normalized empirical measures. Such a 
result will be useful later.

Once the diagram formula is proved, it is not difficult to
generalize it to the product of finitely many Wiener--It\^o
integrals. This generalization, formulated in Theorem~10.2, will
also be called the diagram formula. It has an important corollary
about the calculation of the moments of Wiener--It\^o integrals.
Theorem~8.5 can be proved relatively simply by means of this
corollary.

I shall give the proof of two other results about Wiener--It\^o
integrals in Appendix~C. The first one, Theorem~10.3, is called
It\^o's formula for Wiener--It\^o integrals, and it explains the
relation between multiple Wiener-It\^o integrals and Hermite
polynomials of Gaussian random variables. This result is a
relatively simple consequence of the diagram formula and some
basic recursive relations about Hermite polynomials.

The other result proved in Appendix~C, Theorem~10.4, is a limit
theorem about a sequences of appropriately normalized degenerate
$U$-statistics. Here the limit is presented in the form of a 
multiple Wiener--It\^o integral. This result is interesting for 
us, because it helps to compare Theorems~8.3 and~8.1 with their 
one-variate counterpart, Bernstein's inequality. In the 
one-variate case Bernstein's inequality provides a comparison 
of the distribution of sums of independent random variables and 
normal distribution functions, i.e. the limit distribution in 
the central limit theorem. Theorem~8.3 yields a similar result 
about degenerate $U$-statistics. Its comparison with Theorem~8.5 
and the limit theorem proved in Appendix~C about the limit 
distribution of degenerate $U$-statistics show that degenerate 
$U$-statistics satisfy an estimate similar to Bernstein's 
inequality. The upper bound in it is similar to the estimate on 
the tail-distribution of the limit distribution of normalized 
degenerate $U$-statistics, which equals the distribution of an 
appropriate multiple Wiener--It\^o integral. Theorem~8.1 which 
is an estimate of multiple integrals with respect to a normalized 
empirical distribution also has such an interpretation.

My Lecture Note [29] contains a rather detailed description of
Wiener--It\^o integrals. But in that work the emphasis was put
on the study of a slightly different version of it. The original
version of this integral introduced in~[24] was also only briefly 
discussed there,  not all details were worked out. In particular, 
the diagram formula needed in this work was formulated and proved 
only for modified Wiener--It\^o integrals. I shall discuss the 
difference between these random integrals together with the 
question why a modified version of Wiener--It\^o integrals was 
discussed in~[29] at the end of the section.

To define multiple Wiener--It\^o integrals first the notion of
a white noise has to be introduced. This is done in the following
definition.

\medskip\noindent
{\bf Definition of a white noise with some reference measure.}
{\it Let us have a $\sigma$-finite measure $\mu$ on a measurable
space $(X,{\Cal X})$. A white noise with reference measure $\mu$ is
a Gaussian random field
$\mu_W=\{\mu_W(A)\colon\; A\in{\Cal X},\,\mu(A)<\infty\}$, i.e. a set
of jointly Gaussian random variables indexed by the above
sets~$A$, which satisfies the relations $E\mu_W(A)=0$ and
$E\mu_W(A)\mu_W(B)=\mu(A\cap B)$.}

\medskip
It is worth making some comments about this definition.

\medskip\noindent
{\it Remark:}\/ In the definition of a white noise sometimes also
the property $\mu_W(A\cup B)=\mu_W(A)+\mu_W(B)$ with probability~1
if $A\cap B=\emptyset$, and $\mu(A)<\infty$, $\mu(B)<\infty$
is mentioned. But this condition can be omitted, because it follows
from the remaining properties of the white noise. Indeed, simple
calculation shows that $E(\mu_W(A\cup B) -\mu_W(A)-\mu_W(B))^2=0$
if $A\cap B=\emptyset$, hence $\mu_W(A\cup B)-\mu_W(A)-\mu_W(B)=0$
with probability~1 in this case. It also can be observed that if some
sets $A_1,\dots,A_k\in \Cal X$, $\mu(A_j)<\infty$, $1\le j\le k$,
are disjoint, then the random variables $\mu_W(A_j)$, $1\le j\le k$,
are independent because of the uncorrelatedness of these jointly
Gaussian random variables.

\medskip
It is not difficult to see that for an arbitrary reference
measure~$\mu$ on a space $(X,\Cal X)$ a white noise $\mu_W$ with
this reference measure really exists. This follows simply from
Kolmogorov's fundamental theorem, by which if the finite dimensional
distributions of a random field are prescribed in a consistent way,
then there exists a random field with these finite
dimensional distributions.

Now I turn to the definition of multiple Wiener--It\^o integrals
with respect to a white noise with some reference measure. First I
introduce the class of functions whose Wiener--It\^o integrals
with respect to a white noise $\mu_W$ with a non-atomic reference
measure $\mu$ will be defined.

Let us consider a measurable space $(X,\Cal X)$, a non-atomic
$\sigma$-finite measure $\mu$ on it and a white noise $\mu_W$ on
$(X,\Cal X)$ with reference measure $\mu$. Let us define the
classes of functions $\Cal H_{\mu,k}$, $k=1,2,\dots$, consisting
of functions of $k$ variables on $(X,\Cal X)$ by the formula
$$
\aligned
 \Cal H_{\mu,k}&=\biggl\{f(x_1,\dots,x_k)\colon\; f(x_1,\dots,x_k)
\text{ is an $\Cal X^k$ measurable, real valued} \\
&\qquad\qquad \text{function on $X^k$, and}
\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots,\mu(\,dx_k)<\infty\biggr\}.
\endaligned \tag10.1
$$
We shall call a $\sigma$-finite measure $\mu$ on a measurable 
space $(X,\Cal X)$ non-atomic if for all sets $A\in\Cal X$ such 
that $\mu(A)<\infty$ and numbers $\e>0$ there is a finite partition 
$A=\bigcupp_{s=1}^N B_s$ of the set~$A$ with the property 
$\mu(B_s)<\e$ for all $1\le s\le N$. There is a formally weaker 
definition of a non-atomic measures by which a $\sigma$-finite 
measure~$\mu$ is non-atomic if for all measurable sets $A$ such that
$0<\mu(A)<\infty$ there exists a $B\subset A$ with the property
$0<\mu(B)<\mu(A)$. But these two definitions of non-atomic measures
are actually equivalent, although this equivalence is far from
trivial. I do not discuss this problem here, since it is a little 
bit outside from the direction of the present work. In our further 
considerations we shall work with the first definition of 
non-atomic measures.

The $k$-fold Wiener-It\^o integrals of the functions
$f\in\Cal H_{\mu,k}$ with respect to the white noise~$\mu_W$ will
be defined in a rather standard way. First they will be defined for
some simple functions, called elementary functions, then it will be
shown that the integral for this elementary functions have an $L_2$
contraction property which makes possible to extend it to the class
of functions in $\Cal H_{\mu,k}$.

Let us first introduce the following class of elementary
functions $\bar{\Cal H}_{\mu,k}$ of $k$ variables. A function
$f(x_1,\dots,x_k)$ on $(X^k,\Cal X^k)$ belongs to
$\bar{\Cal H}_{\mu,k}$ if there exist finitely many disjoint
measurable subsets $A_1,\dots,A_M$, $1\le M<\infty$, of the
set~$X$ (i.e. $A_j\cap A_{j'}=\emptyset$ if $j\neq j'$) such that
$\mu(A_j)<\infty$ for all $1\le j\le M$, and the function $f$ has
the form
$$
f(x_1,\dots,x_k)=\left\{
\aligned
&c(j_1,\dots,j_k)\quad\text{if } (x_1,\dots,x_k) \in
A_{j_1}\times\cdots \times A_{j_k} \\
&\qquad \text{with some indices } (j_1,\dots,j_k),
\quad 1\le j_s\le M,\;  1\le s\le k,\\
&\qquad \text{ such that all numbers } j_1,\dots,j_k
\text{ are different} \\
&0 \quad\text{if }(x_1,\dots,x_k)\notin\bigcupp\Sb
(j_1,\dots,j_k)\colon \; 1\le j_s\le M, \; 1\le s\le k,\\
\text{ and all } j_1,\dots,j_k\text { are different.}\endSb
A_{j_1}\times\cdots \times A_{j_k}
\endaligned \right. \tag10.2
$$
with some real numbers $c(j_1,\dots,j_k)$, $1\le j_s\le M$, $1\le
s\le k$, if all $j_1,\dots,j_k$ are different numbers. This means
that the function $f$ is constant on all $k$-dimensional
rectangles $A_{j_1}\times\dots\times A_{j_k}$ with different,
non-intersecting edges, and it equals zero on the complementary
set of the union of these rectangles. The property that the support
of the function~$f$ is on the union of rectangles with
non-intersecting edges is sometimes interpreted so that the
diagonals are omitted from the domain of integration of
Wiener--It\^o integrals.

The Wiener-It\^o integral of an elementary function
$f(x_1,\dots,x_k)$ of the form (10.2) with respect to a white
noise $\mu_W$ with the (non-atomic) reference measure $\mu$
is defined by the formula
$$
\aligned
&\int f(x_1,\dots,x_k)\mu_W(\,dx_1)\dots\mu_W(\,dx_k)\\
&\qquad=\sum\Sb 1\le j_s\le M,\;1\le s\le k \\
\text{all } j_1,\dots,j_k \text{ are different}\endSb
c(j_1,\dots,j_k) \mu_W(A_{j_1})\cdots\mu_W(A_{j_k}).
\endaligned  \tag10.3
$$
(The representation of the function $f$ in (10.2) is not unique,
the sets $A_j$ can be divided to smaller disjoint sets, but its
Wiener--It\^o integral defined in (10.3) does not depend on its
representation.) The notation
$$
Z_{\mu,k}(f)=\frac1{k!}
\int f(x_1,\dots,x_k)\mu_W(\,dx_1)\dots\mu_W(\,dx_k), \tag10.4
$$
will be used in the sequel, and the expression $Z_{\mu,k}(f)$
will be called the normalized Wiener--It\^o integral of the
function~$f$. Such a terminology will be applied also for the
Wiener--It\^o integrals of all functions $f\in\Cal H_{\mu,k}$ to
be defined later.

If $f$ is an elementary function in $\bar{\Cal H}_{\mu,k}$ defined
in (10.2), then its normalized Wiener--It\^o integral defined
in~(10.3) and~(10.4) satisfies the relations
$$
\aligned
Ek!Z_{\mu,k}(f)&=0, \\
E(k!Z_{\mu,k}(f))^2&=\sum\Sb (j_1,\dots,j_k)\colon\;
1\le j_s\le M,\; 1\le s\le k, \\
\text{and all } j_1,\dots,j_k\text{ are different.}\endSb
\sum_{\pi\in \Pi_k}
c(j_1,\dots,j_k)c(j_{\pi(1)},\dots,j_{\pi(k)})  \\
&\qquad\qquad\qquad E\mu_W(A_{j_1})\cdots\mu_W(A_{j_k})
\mu_W(A_{j_{\pi(1)}})\cdots\mu_W(A_{j_{\pi(k)}}) \\
&=k!\int \text{Sym\,} f^2(x_1,\dots,x_k)
\mu(\,dx_1)\dots\mu(\,dx_k)\\
&\le k!\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k),
\endaligned \tag10.5
$$
with $\text{Sym\,}f(x_1,\dots,x_k)=
\frac1{k!}\summ_{\pi\in\Pi_k}f(x_{\pi(1)},\dots,x_{\pi(k)})$,
where $\Pi_k$ denotes the set of all permutations
$\pi=\{\pi(1),\dots,\pi(k)\}$ of the set $\{1,\dots,k\}$.

The identities written down in (10.5) can be simply checked. The
first relation follows from the identity
$E\mu_W(A_{j_1})\cdots\mu_W(A_{j_k})=0$ for disjoint sets
$A_{j_1},\dots,A_{j_k}$, which holds, since the expectation of the
product of independent random variables with zero expectation is taken.
The second identity follows similarly from the identity
$$
\align
&E\mu_W(A_{j_1})\cdots\mu_W(A_{j_k})
\mu_W(A_{j'_1})\cdots\mu_W(A_{j'_k})=0\\
&\qquad \text{ if the sets of indices }
\{j_1,\dots,j_k\}  \text { and }
\{j'_1,\dots,j'_k\} \text{ are different,} \\
&E\mu_W(A_{j_1})\cdots\mu_W(A_{j_k})
\mu_W(A_{j'_1})\cdots\mu_W(A_{j'_k})
=\mu(A_{j_1})\cdots\mu(A_{j_k})\\
&\qquad \text{ if } \{j_1,\dots,j_k\}=
\{j'_1,\dots,j'_k\} \text{ i.e. if }
j'_1=j_{\pi(1)},\dots,j'_k=j_{\pi(k)}  \\
&\qquad \text{ with some permutation } \pi\in\Pi_k,
\endalign
$$
which holds because of the facts that the $\mu_W$ measure of
disjoint sets are independent with expectation zero, and
$E\mu_W(A)^2=\mu(A)$. The remaining relations in (10.5) can be
simply checked.

It is not difficult to check that
$$
EZ_{\mu,k}(f)Z_{\mu,k'}(g)=0  \tag10.6
$$
for all functions $f\in \bar{\Cal H}_{\mu,k}$ and
$g\in \bar{\Cal H}_{\mu,k'}$ if $k\neq k'$, and
$$
Z_{\mu,k}(f)=Z_{\mu,k}(\text{Sym}\, f) \tag10.7
$$
for all functions $f\in \bar{\Cal H}_{\mu,k}$.

The definition of Wiener--It\^o integrals can be extended to
general functions $f\in\Cal H_{\mu,k}$ with the help of the
estimate~(10.5). But to carry out this extension we still have
to know that the class of functions $\bar{\Cal H}_{\mu,k}$ is
a dense subset of the class $\Cal H_{\mu,k}$ in the Hilbert
space $L_2(X^k,\Cal X^k,\mu^k)$, where $\mu^k$ is the $k$-th power
of the reference measure $\mu$ of the white noise~$\mu_W$. I
briefly explain how this property of $\bar{\Cal H}_{\mu,k}$ can be
proved. The non-atomic property of the measure~$\mu$ is exploited
at this point.

To prove this statement it is enough to show that the indicator
function of any product set $A_1\times\cdots\times A_k$
such that $\mu(A_j)<\infty$, $1\le j\le k$, but the sets
$A_1,\dots,A_k$ may be non-disjoint is in the $L_2(\mu^k)$ closure
of $\bar{\Cal H}_{\mu,k}$. In the proof of this statement it will
be exploited that since $\mu$ is a non-atomic measure, 
the sets $A_j$ can be represented for all $\e>0$ and $1\le j\le k$ 
as a finite union $A_j=\bigcupp_s B_{j,s}$ of disjoint sets $B_{j,s}$
with the property $\mu(B_{j,s})<\e$.
%
%\plainfootnote{${}^{(1)}$} {For the sake of simplicity let us call a
%$\sigma$-finite measure $\mu$ on a measurable space $(X,\Cal X)$
%non-atomic if for all sets $A\in\Cal X$ such that $\mu(A)<\infty$ and
%numbers $\e>0$ there is a finite partition $A=\bigcupp_{s=1}^N B_s$
%of the set~$A$ with the property $\mu(B_s)<\e$ for all $1\le s\le N$.
%The original definition of a non-atomic measure is a formally weaker
%statement. It calls a measure~$\mu$ non-atomic if for all $A$ such
%that $0<\mu(A)<\infty$ there exists a $B\subset A$ with the property
%$0<\mu(B)<\mu(A)$. But the original definition of the non-atomic
%measure implies the property suggested in this footnote. (See e.g.
%Example~49 at the end of Chapter~2 in~[*].) I omit the proof of this
%non-trivial statement, because it is a little bit outside from the
%direction of the present work.}
%
By means of these relations the
product $A_1\times\cdots\times A_k$ can be written in the form
$$
A_1\times\cdots\times A_k=\bigcup_{s_1,\dots,s_k}
B_{1,s_1}\times\cdots\times B_{k,s_k} \tag10.8
$$
with some sets $B_{j,s_j}$ such that $\mu(B_{j,s_j})<\e$ for all sets
in this union. Moreover, we may assume, by refining the partitions
of the sets $A_j$ if this is necessary that any two sets $B_{j,s_j}$
and  $B_{j',s'_{j'}}$ in this representation are either disjoint, or
they agree. Take such a representation of $A_1\times\cdots\times A_k$,
and consider the set we obtain by omitting those products
$B_{1,s_1}\times\cdots\times B_{k,s_k}$ from the union at the
right-hand side of (10.8) for which $B_{i,s_i}=B_{j,s_j}$
for some $1\le i<j\le k$. The indicator function of the remaining
set is in the class $\bar{\Cal H}_{\mu,k}$. Hence it is enough to
show that the distance between this indicator function and the
indicator function of the set $A_1\times\cdots\times A_k$
is less than $\text{const.}\,\e$ in the $L_2(\mu^k)$ norm
with some $\text{const.}$ which may depend on the sets
$A_1,\dots,A_k$, but not on $\e$. Indeed, by letting $\e$ tend to
zero we get from this relation that the indicator function of the
set $A_1\times A_2\times\cdots\times A_k$ is in the closure
of $\bar{\Cal H}_{\mu,k}$ in the $L_2(\mu^k)$ norm.

Hence to prove the desired property of $\bar{\Cal H}_{\mu,k}$ it is
enough to prove the following statement. Take the representation
(10.8) of $A_1\times\cdots\times A_k$ (which depends on $\e$) and an
arbitrary pair of integers $i$ and $j$ such that $1\le i<j\le k$.
Then the sum of the measures
$\mu^k(B_{1,s_1}\times\cdots\times B_{k,s_k})$ of those sets
$B_{1,s_1}\times\cdots\times B_{k,s_k}$
at the right-hand side of (10.8) for which $B_{i,s_i}=B_{j,s_j}$ is
less than $\text{const.}\,\e$. To prove such an estimate observe that
the $\mu^k$ measure of such a set can be bounded by the $\mu^{k-1}$
measure of the set we obtain by omitting the $i$-th term from the
product defining it in the following way:
$$
\mu^k(B_{1,s_1}\times\cdots\times B_{k,s_k})\le \e
\mu^{k-1}(B_{1,s_1}\times\cdots\times B_{i-1,s_{i-1}}\times
B_{i+1,s_{i+1}}\times\cdots\times B_{k,s_k}).
$$
Let us sum up this inequality for all such sets
$B_{1,s_1}\times\cdots\times B_{k,s_k}$ at the right-hand side
of~(10.8) for which $B_{i,s_i}=B_{j,s_j}$.
The left-hand side of the inequality we get in such a way equals
the quantity we want to estimate. The expression at its right-hand
side is less than $\e\prodd_{1\le s\le k,\, s\neq i}\mu(A_s)$, since
$\e$-times the  $\mu^{k-1}$ measure of such disjoint sets are summed
up in it which are contained in the set
$A_1\times\cdots\times A_{i-1}\times A_{i+1}\times\cdots\times A_k$.
In such a way we get the estimate we wanted to prove.

Knowing that $\bar{\Cal H}_{\mu,k}$ is a dense subset of
$\Cal H_{\mu,k}$ in $L_2(\mu^k)$ norm we can finish the definition
of $k$-fold Wiener--it\^o integrals in the standard way.
Given any function $f\in\Cal H_{\mu,k}$, a sequence of functions
$f_n\in\bar{\Cal H}_{\mu,k}$, $n=1,2,\dots$, can be defined in such
a way that $\int|f(x_1,\dots,x_k)-f_n(x_1,\dots,x_k)|^2\mu(\,dx_1)
\dots\mu(\,dx_k)\to0$ as $n\to\infty$. By relation~(10.5) the
normalizations $Z_{\mu,k}(f_n)$ of the already defined Wiener--It\^o
integrals of the functions $f_n$, $n=1,2,\dots$, constitute a Cauchy
sequence in the space of square integrable random variables on the
probability space, where the white noise is given. (Observe that the
difference of two  functions from the class $\bar{\Cal H}_{\mu,k}$
also belongs to this class.) Hence the limit
$\limm_{n\to\infty}Z_{\mu,k}(f_n)$ exists in $L_2$ norm, and this
limit can be defined as the normalized Wiener--It\^o integral
$Z_{\mu,k}(f)$ of the function~$f$. The definition of this limit
does not depend on the choice of the approximating functions~$f_n$,
hence it is meaningful. It can be seen that relations~(10.5)
and~(10.6) remain valid for all functions $f\in\Cal H_{\mu,k}$.
The following Theorem~10.1 describes the properties of multiple
Wiener--It\^o integrals. It contains already proved results. The
only still non-discussed part of this Theorem is Property~f) of
Wiener--It\^o integrals. But it is easy to check this property by
observing that one-fold Wiener--It\^o integrals are (jointly)
Gaussian, they are measurable with respect to the $\sigma$-algebra
generated by the white noise $\mu_W$. Beside this,
the random variable $\mu_W(A)$ for a set $A\in \Cal X$,
$\mu(A)<\infty$, equals the (one-fold) Wiener--It\^o integral of
the indicator function of the set~$A$.

\medskip\noindent
{\bf Theorem 10.1. (Some properties of multiple Wiener--It\^o
integrals).} {\it Let a white noise $\mu_W$ be given with some
non-atomic $\sigma$-additive reference measure on a measurable
space $(X,\Cal X)$. Then the $k$-fold Wiener--It\^o integral
of all functions in the class $\Cal H_{\mu,k}$ introduced in
formula~(10.1) can be defined, and its normalized version
$Z_{\mu,k}(f)=\frac1{k!}\int f(x_1,\dots,x_k)\mu_W(\,dx_1)\dots\mu_W(dx_k)$
satisfies the following relations:

\medskip
\item{a)}
$Z_{\mu,k}(\alpha f+\beta g)=\alpha Z_{\mu,k}(f)+\beta Z_{\mu,k}(g)$
for all $f,g\in \Cal H_{\mu,k}$ and real numbers $\alpha$
and~$\beta$.
\medskip
\item{b)} If $A_1,\dots,A_k$ are disjoint sets, $\mu(A_j)<\infty$,
then the function $f_{A_1,\dots,A_k}$ defined by the relation
$f_{A_1,\dots,A_k}(x_1,\dots,x_k)=1$
if $x_1\in A_1$, \dots, $x_k\in A_k$,
$f_{A_1,\dots,A_k}(x_1,\dots,x_k)=0$  otherwise, satisfies the
identity
$$
Z_{\mu,k}(f_{A_1,\dots,A_k}(x_1,\dots,x_k))=\frac1{k!}
\mu_W(A_1)\cdots\mu_W(A_k).
$$

%\medskip
\item{c)}
$$
EZ_{\mu,k}(f)=0, \quad \text{and} \quad
EZ^2_{\mu,k}(f)=\frac1{k!}\|\text{\rm Sym}\,f\|_2^2
\le\frac1{k!}\|f\|^2_2
$$
for all $f\in\Cal H_{\mu,k}$, where $\|f\|_2^2
=\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)$ is the
square of the $L_2$ norm of a function $f\in \Cal H_{\mu,k}$.

\medskip
\item{d)} Relation (10.6) holds for all functions
$f\in \Cal H_{\mu,k}$ and $g\in \Cal H_{\mu,k'}$ if $k\neq k'$.

\medskip
\item{e)} Relation (10.7) holds for all functions
$f\in \Cal H_{\mu,k}$.

\medskip
\item{f)} The Wiener--It\^o integrals $Z_{\mu,1}(f)$ of order $k=1$
are jointly Gaussian. The smallest $\sigma$-algebra with respect to
which they are all measurable agrees with the $\sigma$-algebra
generated by the random variables $\mu_W(A)$, $A\in\Cal X$,
$\mu(A)<\infty$, of the white noise.
}

\medskip
We have defined Wiener--It\^o integrals of order~$k$ for all
$k=1,2,\dots$. For the sake of completeness let us introduce
the class $\Cal H_{\mu,0}$ for $k=0$ which consists of the real
constants (functions of zero variables), and put $Z_{\mu,0}(c)=c$.
Because of relation~(10.7) we could have restricted our attention
to Wiener--It\^o integrals with symmetric kernel functions. But it
turned out more convenient to work also with Wiener--It\^o integrals
of not necessarily symmetric functions.

Now I formulate the diagram formula for the product of two
Wiener--It\^o integrals. For this goal some notations have to be
introduced. To present the product of the multiple Wiener--It\^o
integrals of two functions $f(x_1,\dots,x_k)\in\Cal H_{\mu,k}$ and
$g(x_1,\dots,x_l)\in\Cal H_{\mu,l}$ in the form of sums of
Wiener--It\^o integrals a class of diagrams $\Gamma=\Gamma(k,l)$
will be defined. The diagrams $\gamma\in\Gamma(k,l)$ have vertices
$(1,1),\dots,(1,k)$ and $(2,1),\dots,(2,l)$, and edges
$((1,j_1),(2,j_1'))$,\dots, $((1,j_s),(2,j_s'))$ with some
$1\le s\le \min(k,l)$. The indices $j_1,\dots,j_s$ in the definition
of the edges are all different,
and the same relation holds for the indices $j'_1,\dots,j'_s$. All
such diagrams $\gamma$ belongs to $\Gamma(k,l)$. The set of vertices
of the form $(1,j)$, $1\le j\le k$, will be called the first row,
and the set of vertices of the form $(2,j')$, $1\le j'\le l$, the
second row of a diagram. We demanded that edges of a diagram can
connect only vertices of different rows, and at most one edge may
start from each vertex of a diagram.

Given a diagram $\gamma\in\Gamma(k,l)$ with the set of edges
$$
E(\gamma)=\{(1,j_1),(2,j_1')),\dots, ((1,j_s),(2,j_s')\}
$$
let $V_1(\gamma)=\{(1,1),\dots,(1,k)\}
\setminus\{(1,j_1),\dots,(1,j_s)\}$
and $V_2(\gamma)=\{(2,1),\dots,(2,l)\}\setminus
\{(2,j_1'),\dots,(2,j_s')\}$ denote the set of vertices
in the first and in the second row of the diagram $\gamma$ respectively
from which no edge starts. Put $\alpha_\gamma(1,j)=(2,j')$ if
$((1,j),(2,j'))\in E(\gamma)$ and $\alpha_\gamma(1,j)=(1,j)$ if
the diagram $\gamma$ contains no edge of the form
$((1,j),(2,j'))\in E(\gamma)$. In words, the function
$\alpha_\gamma(\cdot)$ is defined on the vertices of the first row
of the diagram $\gamma$. It replaces a vertex to the vertex it is
connected to by an edge of the diagram if there is such a vertex,
and it does not change those vertices from which no edge starts. Put
$|\gamma|=k+l-2s$, i.e. $|\gamma|$ equals the number of vertices in
$\gamma$ from which no edge starts. Given two functions
$f(x_1,\dots,x_k)\in\Cal H_{\mu,k}$ and
$g(x_1,\dots,x_l)\in\Cal H_{\mu,l}$ let us introduce their product
$$
\aligned
&F(x_{(1,1)},\dots,x_{(1,k)},x_{(2,1)},\dots,x_{(2,l)}) \\
&\qquad=F_{f,g}(x_{(1,1)},\dots,x_{(1,k)},x_{(2,1)},\dots,x_{(2,l)}) \\
&\qquad= f(x_{(1,1)},\dots,x_{(1,k)})g(x_{(2,1)},\dots,x_{(2,l)})
\endaligned \tag10.9
$$
together with its modification
$$
\aligned
&\bar F_\gamma(x_{(1,j)},\colon\; (1,j)\in V_1(\gamma),\,
x_{(2,1)}\dots,x_{(2,l)})\\
&\qquad =f(x_{\alpha_\gamma(1,1)},\dots,x_{\alpha_\gamma(1,k)})
g(x_{(2,1)},\dots,x_{(2,l)}).
\endaligned \tag10.9a
$$
(Here the function  $f(x_1,\dots,x_k)$ is replaced by
$f(x_{(1,1)},\dots,x_{(1,k)})$ and the function
$g(x_1,\dots,x_l)$ by $g(x_{(2,1)},\dots,x_{(2,l)})$.)
With the help of the above introduced sets $V_1(\gamma)$,
$V_2(\gamma)$ and function $\alpha_\gamma(\cdot)$
let us introduce the functions $F_\gamma=F_\gamma(f,g)$ as
$$ \allowdisplaybreaks
\aligned
&F_\gamma(x_{(1,j)},x_{(2,j')} \colon\; (1,j)\in V_1(\gamma), \,
(2,j')\in V_2(\gamma))\\
&\qquad =\int \bar F_\gamma(x_{\alpha_\gamma(1,j)}
\colon\; (1,j)\in V_1(\gamma),\,x_{(2,1)},\dots,x_{(2,l)}) \\
&\qquad\qquad\qquad \prod_{(2,j)\in\{(2,1),\dots,(2,l)\}\setminus
 V_2(\gamma)} \mu(\,dx_{(2,j)})
\endaligned \tag10.10
$$
for all diagrams $\gamma\in\Gamma(k,l)$. In words: We take the
product defined in (10.9), then if the index $(1,j)$ of a variable
$x_{(1,j)}$ is connected with the index $(2,j')$ of some variable
$x_{(2,j')}$ by an edge of the diagram $\gamma$, then we replace
the variable $x_{(1,j)}$ by $x_{(2,j')}$ in this product. Finally
we integrate the function obtained in such a way with respect to 
the arguments with indices $(2,j_1'),\dots,(2,j_s')$, i.e. with 
those vertices of the second row of the diagram~$\gamma$ from 
which en edge starts. It is clear that $F_\gamma$ is a function 
of $|\gamma|$ variables. It depends on those coordinates whose 
indices are such vertices of $\gamma$ from which no edge starts.

For the sake of simpler notations we shall also consider Wiener--It\^o
integrals with such kernel functions whose variables are more
generally indexed. If the $k$-fold Wiener--It\^o integral with a
kernel function $f(x_1,\dots,x_k)$ is well-defined, then we shall
say that the Wiener--It\^o integral with kernel function
$f(x_{u_1},\dots,x_{u_k})$, where $\{u_1,\dots,u_k\}$ is an
arbitrary set with $k$ different elements, is also well defined,
and it equals the Wiener--It\^o integral with the original kernel
function $f(x_1,\dots,x_k)$. (We have right to make such a
convention since the value of a Wiener--It\^o integral does not
change if we permute the indices of the variables of the kernel
function in an arbitrary way.) In particular, we shall speak
about the Wiener--It\^o integral of the function $F_\gamma$ defined
in (10.10) without reindexing its variables $x_{(1,j)}$ and
$x_{(2,j')}$ `in the right way'. Now we can formulate the diagram
formula for the product of two Wiener--It\^o integrals.

\medskip\noindent
{\bf Theorem 10.2A. (The diagram formula for the product of two
Wiener--It\^o integrals).} {\it Let a non-atomic $\sigma$-finite
measure $\mu$ be given on a measurable space
$(X,\Cal X)$ together with a white noise $\mu_W$ with reference
measure $\mu$, and take two functions
$f(x_1,\dots,x_k)\in\Cal H_{\mu,k}$ and
$g(x_1,\dots,x_l)\in\Cal H_{\mu,l}$. (The classes of functions
$\Cal H_{\mu,k}$ and $\Cal H_{\mu,l}$ were introduced in (10.1).)
Let us consider the class of diagrams $\Gamma(k,l)$ introduced above
together with the functions $F_\gamma$, $\gamma\in\Gamma(k,l)$,
defined by formulas (10.9), (10.9a) and (10.10) with its help.
They satisfy the inequality
$$
\|F_\gamma\|_2\le \|f\|_2\|g\|_2 \quad \text{for all }
\gamma\in\Gamma(k,l), \tag10.11
$$
where the $L_2$ norm of a (generally indexed) function
$h(x_{u_1},\dots,x_{u_s})$ is defined as
$$
\|h\|_2^2=\int h^2(x_{u_1},\dots,x_{u_s})
\mu(\,dx_{u_1})\dots\mu\,dx_{u_s}).
$$
Beside this, the product $Z_{\mu,k}(f)Z_{\mu,l}(g)$ of the
normalized Wiener--It\^o integrals of the functions $f$ and $g$
(the notation $Z_{\mu,k}$ was introduced in (10.4)) satisfies the
identity
$$
(k!Z_{\mu,k}(f))(l!Z_{\mu,l}(g))
=\sum_{\gamma\in \Gamma(k,l)} |\gamma|!Z_{\mu,|\gamma|}(F_\gamma)
=\sum_{\gamma\in \Gamma(k,l)} |\gamma|!Z_{\mu,|\gamma|}
(\text{\rm Sym}\,F_\gamma).
\tag10.12
$$
}

\medskip
Theorem~10.2A will be proved in Appendix~B. The
following consideration yields a heuristic explanation for it.
Actually, it can also be considered as a sketch of proof.

In the theory of general It\^o integrals when stochastic processes
are integrated with respect to a Wiener processes, one of the most
basic results is It\^o's formula about differentiation of functions
of It\^o integrals. It has a heuristic interpretation by means of
the informal `identity' $dW^2=dt$. In the case of general white
noises this `identity' can be generalized as
$\mu_W(\,dx)^2=\mu(\,dx)$. We present a rather informal `proof'
of the diagram formula on the basis of this `identity' and the fact
that the diagonals are omitted from the domain of integration in
the definition of Wiener--It\^o integrals.

In this `proof' we fix two numbers $k\ge1$ and $l\ge1$, and
consider the product of the Wiener--It\^o integrals of the
functions $f$ and $g$ of order~$k$ and~$l$. This product is a
bilinear form of the functions~$f$ and~$g$. Hence it is enough to
check formula (10.12) for a sufficiently rich class of functions.
It is enough to consider functions of the form
$f(x_1,\dots,x_k)=I_{A_1}(x_1)\cdots I_{A_k}(x_k)$ and
$g(x_1,\dots,x_l)=I_{B_1}(x_1)\cdots I_{B_l}(x_l)$ with disjoint
sets $A_1,\dots,A_k$ and disjoint sets $B_1,\dots,B_l$, where $I_A(x)$
is the indicator function of a set $A$. (Here we have exploited
that the functions $f$ and $g$ disappear in the diagonals.)
Let us divide the sets $A_j$ into the union of small disjoint sets
$D_j^{(m)}$, $1\le j\le k$ with some fixed number $1\le m\le M$ in
such a way that $\mu(D_j^{(m)})\le \e$ with some  fixed $\e>0$, and
the sets $B_j$ into the union of small disjoint sets $F_j^{(m)}$,
$1\le j\le l$, with some fixed number $1\le m\le M$, in such a way
that $\mu(F_j^{(m)})\le \e$ with some fixed $\e>0$. Beside this, we
also require that two sets $D_j^{(m)}$ and $F_{j'}^{(m')}$ should
be either disjoint or they should agree. (The sets $D_j^{(m)}$ are
disjoint for different indices, and the same relation holds for the
sets $F_{j'}^{(m')}$.)

Then the identity
$$
k!Z_{\mu,k}(f)=\prodd_{j=1}^k\(\summ_{m=1}^M\mu_W(D_j^{(m)})\)
\quad \text{and} \quad l!Z_{\mu,l}(g)
=\prodd_{j'=1}^l\(\summ_{m'=1}^M\mu_W(F_{j'}^{(m')})\),
$$
holds, and the product of these two Wiener--It\^o integrals can be
written in the form of a sum by means of a term by term
multiplication. Let us divide the terms of the sum we get in such a
way into classes indexed by the diagrams $\gamma\in\Gamma(k,l)$
in the following way: Each term in this sum is a product of the form
$\prodd_{j=1}^k\mu_W(D_j^{(m_j)})
\prodd_{j'=1}^l\mu_W(F_{j'}^{(m_j')})$. Let it belong to the
class indexed by the diagram $\gamma$ with edges
$((1,j_1),(2,j_1'))$,\dots, and $((1,j_s),(2,j'_s))$ if the elements
in the pairs $(D_{j_1}^{m_{j_1}},F_{j'_1}^{m_{j'_1}})$,\dots,
$(D_{j_s}^{m_{j_s}},F_{j'_s}^{m_{j'_s}})$ agree, and otherwise all
terms are different. Then letting $\e\to0$ (and taking partitions of
the sets $D_j$ and $F_{j'}$ corresponding to the parameter $\e$) the
sums of the terms in each class turn to integrals, and our
calculation suggests the identity
$$
(k!Z_{\mu,k}(f))(l!Z_{\mu,l}(g))
=\summ_{\gamma\in\Gamma(k,l)}\bar Z_\gamma \tag10.13
$$
with
$$
\aligned
\bar Z_\gamma&=\int
f(x_{\alpha_\gamma(1,1)},\dots,x_{\alpha_\gamma(1,k)})
g(x_{(2,1)},\dots,x_{(2,l)}) \\
&\qquad \mu_W(\,dx_{\alpha_\gamma(1,1)})\dots
\mu_W(\,dx_{\alpha_\gamma(1,k)})
\mu_W(\,dx_{(2,1)})\dots\mu_W(\,dx_{(2,l)})
\endaligned \tag10.13a
$$
with the function $\alpha_\gamma(\cdot)$ introduced before formula
(10.9). The indices $\alpha(1,j)$ of the arguments in (10.13a) mean
that in the case $\alpha_\gamma(1,j)=(2,j')$ the argument
$x_{(1,j)}$ has to be replaced by $x_{(2,j')}$. In particular,
$\mu_W(\,dx_{\alpha(1,j)})\mu_W(\,dx_{(2,j')})
=\mu_W(\,dx_{(2,j')})^2=\mu(\,dx_{(2,j')})$ in this case because
of the `identity' $\mu_W(\,dx)^2=\mu(\,dx)$. Hence the above
informal calculation yields the identity
$\bar Z_\gamma=|\gamma|!Z_{\mu,|\gamma|}(F_\gamma)$. Hence
relations~(10.13) and~(10.13a) imply formula~(10.12).

A similar heuristic argument can be applied to get formulas for the
product of integrals of normalized empirical distributions or
(normalized) Poisson fields, only the starting formula
$\mu_W(\,dx)^2=\mu(\,dx)$ changes in these cases, some additional
terms appear which modify the final result. I return to this
question in the next section.

\medskip
It is not difficult to generalize Theorem~10.2A with the help of
some additional notations to a diagram formula about the product of
finitely many Wiener--It\^o integrals.
%To do this, some additional notations have to be introduced.
Let us consider $m\ge2$ Wiener--It\^o integrals $k_p!Z_{\mu,k_p}(f_p)$,
of functions $f_p(x_1,\dots,x_{k_p})\in\Cal H_{\mu,k_p}$, of order
$k_p\ge1$, $1\le p\le m$, and define a class of diagrams
$\Gamma=\Gamma(k_1,\dots,k_m)$ in the following way.

The diagrams $\gamma\in\Gamma=\Gamma(k_1,\dots,k_m)$ have
vertices of the form $(p,r)$, $1\le p\le m$, $1\le r\le k_p$. The
set of vertices $\{(p,r)\colon\; 1\le r\le k_p\}$ with a fixed number
$p$ will be called the $p$-th row of the diagram $\gamma$. A diagram
$\gamma\in\Gamma=\Gamma(k_1,\dots,k_m)$ may have some edges. All
edges of a diagram connect vertices from different rows, and from
each vertex there starts at most one edge. All diagrams satisfying
these properties belong to $\Gamma(k_1,\dots,k_m)$. If a diagram
$\gamma$ contains an edge of the form $((p_1,r_1),(p_2,r_2))$ with
$p_1<p_2$, then $(p_1,r_1)$ will be called the upper and
$(p_2,r_2)$ the lower end point of this edge. Let $E(\gamma)=
\{((p_1^{(u)},r_1^{(u)}),(p_2^{(u)},r_2^{(u)})),\;p_1^{(u)}
<p_2^{(u)},\,1\le u\le s\}$ denote the set of all edges of a
diagram~$\gamma$ (the number of edges in $\gamma$ was denoted by
$s=|E(\gamma)|$), and let us also introduce the sets
$V^u(\gamma)=\{((p_1^{(u)},r_1^{(u)}),\,1\le u\le s\}$,
the set of all upper end points and
$V^b(\gamma)=\{((p_2^{(u)},r_2^{(u)}),\,1\le u\le s\}$,
the set of all lower end points of edges in a diagram $\gamma$.
Let $V=V(\gamma)=\{(p,r)\colon\; 1\le p\le m, 1\le r\le k_p\}$
denote the set of all vertices of $\gamma$, and let
$|\gamma|=k_1+\cdots+k_m-2|E(\gamma)|$ be equal to the number of
vertices in $\gamma$ from which no edge starts. Vertices from
which no edge starts will be called free vertices in the sequel. 
Let us also define the function $\alpha_\gamma(p,r)$ for a 
vertex $(p,r)$ of the diagram $\gamma$ in the following way:
$\alpha_\gamma(p,r)=(\bar p,\bar r)$, if there is some pair of
integers $(\bar p,\bar r)$ such that
$((p,r),(\bar p,\bar r))\in E(\gamma)$ and $p<\bar p$, i.e.
$(p,r)\in V^u(\gamma)$ and $((p,r),(\bar p,\bar r))\in E(\gamma)$,
and put $\alpha_\gamma(p,r)=(p,r)$ for
$(p,r)\in V(\gamma)\setminus V^u(\gamma)$. In words, the function
$\alpha_\gamma(\cdot)$ was defined on the set of vertices
$V(\gamma)$ in such a way that it replaces an upper end point of an
edge with the lower end point of this edge, and it does not change
the remaining vertices of the diagram.

With the help of the above quantities the appropriate multivariate
version of the functions given in (10.9), (10.9a) and (10.10) can
be defined. Put
$$
\aligned
&F(x_{(p,r)},\colon\; 1\le p\le m, 1\le r\le k_p)
=F_{f_1,\dots,f_m}(x_{(p,r)},\colon\; 1\le p\le m, 1\le r\le k_p) \\
&\qquad=\prod_{p=1}^m f_p(x_{(p,1)},\dots,x_{(p,k_p)}),
\endaligned \tag10.14
$$
$$
\bar F_\gamma(x_{(p,r)}, \colon\; (p,r)\in V(\gamma)
\setminus V^u(\gamma))=\prod_{p=1}^m
f_p(x_{\alpha_\gamma(p,1)},\dots,x_{\alpha_\gamma(p,k_p)}),
\tag$10.14a$
$$
and
$$ \allowdisplaybreaks
\align
&F_\gamma(x_{(p,r)} \colon\; (p,r)\in
V(\gamma)\setminus (V^b(\gamma)\cup V^u(\gamma))    \\
&\qquad =\int \bar F_\gamma(x_{(p,r)}, \colon\; (p,r)\in V(\gamma)
\setminus V^b(\gamma))
\prod_{(p,r)\in V^b(\gamma)} \mu(\,dx_{(p,r)}). \tag10.15
\endalign
$$
With the help of the above notations the diagram formula for the
product of finitely many Wiener--It\^o integrals can be
formulated.

\medskip\noindent
{\bf Theorem 10.2. (The diagram formula for the product of finitely
many Wiener--It\^o integrals).} {\it Let a non-atomic
$\sigma$-finite measure $\mu$ be given on a measurable
space $(X,\Cal X)$ together with a white noise $\mu_W$ with
reference measure $\mu$. Take $m\ge2$ functions
$f_p(x_1,\dots,x_{k_p})\in\Cal H_{\mu,k_p}$ with some order $k_p\ge1$,
$1\le p\le m$. Let us consider the class of diagrams
$\Gamma(k_1,\dots,k_m)$ introduced above together with the functions
$F_\gamma$, $\gamma\in\Gamma(k_1,\dots,k_m)$, defined by formulas
(10.14), (10.14a) and (10.15) with its help.
The $L_2$-norm of these functions satisfies the inequality
$$
\|F_\gamma\|_2\le \prod_{p=1}^m \|f_p\|_2 \quad \text{for all }
\gamma\in\Gamma(k_1,\dots,k_m). \tag10.16
$$
Beside this, the product $\prodd_{p=1}^m Z_{\mu,k_p}(f_p)$ of the
normalized Wiener--It\^o integrals of the functions $f_p$,
$1\le p\le m$, satisfies the identity
$$
\prod_{p=1}^m k_p!Z_{\mu,k_p}(f_p)
=\sum_{\gamma\in \Gamma(k_1,\dots,k_m)}
|\gamma|!Z_{\mu,|\gamma|}(F_\gamma)
=\sum_{\gamma\in \Gamma(k_1,\dots,k_m)} |\gamma|!Z_{\mu,|\gamma|}
(\text{\rm Sym}\,F_\gamma).
\tag10.17
$$
}

\medskip
Theorem 10.2 can be relatively simply derived from Theorem~10.2A by
means of induction with respect the number of terms whose product we
consider. We still have to check that with the introduction of an
appropriate notation Theorem~10.2A remains valid also in the case
when the function $f$ is a constant.

Let us also consider the case when $f=c$ and $g\in \Cal H_{\mu,l}$.
In this case we apply the convention $Z_{\mu,0}(c)=c$, define the
class of diagrams $\Gamma(0,l)$ that consists only of one diagram
$\gamma$ whose first row is empty, its second row contains the
vertices $(2,1),\dots,(2,l)$, and it has no edges. Beside this, we
define
$F_\gamma(x_{(2,1)},\dots,x_{(2,l)})=cg(x_{(2,1)},\dots,x_{(2,l)})$
in this case. With such a convention Theorem~10.2A can be extended
to the case of the product of two Wiener--It\^o integrals of order
$k\ge0$ and $l\ge1$. Theorem 10.2 can be derived from this slightly
generalized result by induction.

By statement c) of Theorem~10.1 all Wiener--It\^o integrals of order
$k\ge1$ have expectation zero. This fact together with Theorem~10.2
enable us to compute the expectation of a product of Wiener--It\^o
integrals. Theorem~10.2 makes possible to rewrite a product of
Wiener--It\^o integrals as a sum of Wiener--It\^o integrals. Then
its expectation can be calculated by taking the expected value
of each term and summing them up. Only constant terms yield a
non-zero contribution to this expectation. These constant terms
agree with the functions $F_\gamma$ corresponding to diagrams with
no free vertices. The next corollary writes down the result we get
in such a way.

\medskip\noindent
{\bf Corollary of Theorem 10.2 about the expectation of a product of
Wiener--It\^o integrals.} {\it Let a non-atomic $\sigma$-finite
measure $\mu$ be given on a measurable space $(X,\Cal X)$
together with a white noise~$\mu_W$ with reference measure $\mu$.
Take $m\ge2$ functions $f_p(x_1,\dots,x_{k_p})\in\Cal H_{\mu,k_p}$,
and consider their Wiener--It\^o integrals $Z_{\mu,k_p}(f_p)$,
$1\le p\le m$. The expectation of the product of these random
variables satisfies the identity
$$
E\(\prod_{p=1}^m k_p!Z_{\mu,k_p}(f_p)\)
=\sum_{\gamma\in\bar \Gamma(k_1,\dots,k_m)}F_\gamma,
\tag10.18
$$
where $\bar\Gamma(k_1,\dots,k_m)$ denotes the set of all such
diagrams $\gamma\in\Gamma(k_1,\dots,k_m)$ which have no free
vertices, i.e. $|\gamma|=0$. Such diagrams will be called closed in
the sequel. (If $\bar\Gamma(k_1,\dots,k_m)$ is empty, then
the sum at the right-hand side of (10.17) equals zero.) The
functions $F_\gamma$ for $\gamma\in\bar\Gamma(k_1,\dots,k_m)$ are
constants, and they satisfy the inequality
$$
|F_\gamma|\le \prod_{p=1}^m \|f_p\|_2 \quad \text{for all }
\gamma\in\bar\Gamma(k_1,\dots,k_m). \tag10.19
$$
}
\medskip\noindent
{\it Proof of the Corollary.}\/ Relation (10.18) is a straight
consequence of formula (10.17), part c) of Theorem~10.1 and the
identity $Z_{\mu,0}(F_\gamma)=F_\gamma$, if $|\gamma|=0$. Relation
(10.19) follows from (10.16).

\medskip
The next result I formulate, It\^o's formula for multiple
Wiener--It\^o integrals, can also be considered as a
consequence of the diagram formula. It will be proved in Appendix~C.

\medskip\noindent
{\bf Theorem 10.3. (It\^o's formula for multiple Wiener--It\^o
integrals).} {\it Let a non-atomic $\sigma$-finite
measure $\mu$ be given on a measurable space $(X,\Cal X)$ together
with a white noise~$\mu_W$ with reference measure $\mu$. Let us
take some real valued, orthonormal functions $\varphi_1(x)$,\dots,
$\varphi_m(x)$ on the measure space $(X,\Cal X,\mu)$. Let $H_k(u)$
denote the $k$-th Hermite polynomial with leading coefficient~1.
Take the one-fold Wiener--It\^o integrals
$\eta_p=Z_{\mu,1}(\varphi_p)$, $1\le p\le m$, and introduce the
random variables $H_{k_p}(\eta_p)$, $1\le p\le m$, with some
integers $k_p\ge1$, $1\le p\le m$. Put $K_p=\summ_{j=1}^p k_r$,
$1\le p\le m$, $K_0=0$. Then $\eta_1,\dots,\eta_m$ are
independent, standard normal random variables, and the identity
$$
\aligned
\prod_{p=1}^m H_{k_p}(\eta_p)&=
K_m!Z_{\mu,K_m}\(\prod_{p=1}^m \( \prod_{j=K_{p-1}+1}^{K_p}
\varphi_p(x_j)\)\)\\
&=K_m!Z_{\mu,K_m}\(\text{\rm Sym\,}
\(\prod_{p=1}^m \(\prod_{j=K_{p-1}+1}^{K_p}\varphi_p(x_j)\)\) \)\\
\endaligned \tag10.20
$$
holds. In particular, for a single real valued function $\varphi(x)$
such that $\int \varphi^2(x)\mu(\,dx)=1$
$$
H_k\(\int \varphi(x)\mu_W(\,dx)\)
=\int\varphi(x_1)\cdots\varphi(x_k)\mu_W(\,dx_1)\dots\mu_W(\,dx_k).
\tag10.21
$$
}

\medskip
I also formulate a limit theorem about the distribution of normalized
degenerate $U$-statistics. The limit distribution in this result can
be described by means of multiple Wiener--It\^o integrals. It will
be proved in Appendix~C.

\medskip\noindent
{\bf Theorem 10.4. (Limit theorem about normalized degenerate
$U$-statistics).} {\it Let us consider a sequence of degenerate
$U$-statistics $I_{n,k}(f)$ of order~$k$, \ $n=k,k+1,\dots$, defined
in~(8.7) with the help of a sequence of independent and identically
distributed random variables $\xi_1,\xi_2,\dots$ taking values in a
measurable space $(X,\Cal X)$ with a non-atomic distribution $\mu$
and a kernel function $f(x_1,\dots,x_k)$, canonical with respect
to the measure~$\mu$, defined on the $k$-fold product
$(X^k,\Cal X^k)$ of the space $(X,\Cal X)$ for which
$\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)<\infty$. Then
the sequence of normalized $U$-statistics $n^{-k/2}I_{n,k}(f)$
converges in distribution, as $n\to\infty$, to the $k$-fold
Wiener--It\^o integral
$$
Z_{\mu,k}(f)=\frac1{k!}\int
f(x_1,\dots,x_k)\mu_W(dx_1)\dots\mu_W(dx_k)
$$
with kernel function $f(x_1,\dots,x_k)$ and a white noise $\mu_W$
with reference measure~$\mu$.}

\medskip\noindent
{\it Remark.}\/ The limit behaviour of degenerate $U$-statistics
$I_{n,k}(f)$ with an atomic measure $\mu$ which satisfy the remaining
conditions of Theorem~10.4 can be described in the following way.
Take the probability space $(U,\Cal U,\lambda)$, where $U=[0,1]$,
$\Cal U$ is the Borel $\sigma$-algebra and $\lambda$ is the Lebesgue
measure on it. Introduce a sequence of independent random variables
$\eta_1,\eta_2,\dots$ with uniform distribution on the interval
$[0,1]$, which is independent also of the sequence
$\xi_1,\xi_2,\dots$. Define the product space
$(\tilde X,\tilde{\Cal X},\tilde \mu)
=(X\times U, \Cal X\times\Cal U,\mu\times\lambda)$ together with the
function $\tilde f(\tilde x_1,\dots,\tilde x_k)=\tilde
f((x_1,u_1),\dots,(x_k,u_k))=f(x_1,\dots,x_k)$ with the notation
$\tilde x=(x,u)\in X\times U$, and $\tilde \xi_j=(\xi_j,\eta_j)$,
$j=1,2,\dots$. Then $I_{n,k}(f)=I_{n,k}(\tilde f)$ (with the above
defined function $\tilde f$ and $\tilde\mu$ distributed random
variables $\tilde\xi_j$). Beside this, Theorem~10.4 can be applied
for the degenerate $U$-statistics $I_{n,k}(\tilde f)$, $n=1,2,\dots$.

\medskip
In the next result I give an interesting representation of the
Hilbert space consisting of the square integrable functions
measurable with respect to a white noise $\mu_W$. An isomorphism
will be given with the help of Wiener--It\^o integrals between this
Hilbert space and the so-called Fock space to be defined below.
To formulate this result first some notations will be introduced.

Let $\Cal H^0_{\mu,k}\subset \Cal H_{\mu,k}$ denote the class of
symmetric functions in the space $\Cal H_{\mu,k}$, $k=0,1,2,\dots$,
i.e. $f\in\Cal H_{\mu,k}$ is in its subspace $\Cal H^0_{\mu,k}$ if
and only if $f(x_1,\dots,x_k)=\text{Sym\,}f(x_1,\dots,x_k)$. Let
us introduce for all $k=0,1,2,\dots$ the Hilbert space $\Cal G_k$
consisting of those random variables $\eta$ (on the
probability space where the white noise $\mu_W$ is defined) which
can be written in the form
$$
\eta=Z_{\mu,k}(f)=\frac1{k!}\int f(x_1,\dots,x_k)\mu_W(\,dx_1)\dots
\mu_W(\,dx_k) \quad \text {with some } f\in \Cal H^0_{k,\mu}.
$$

It follows from part a) and c) of Theorem~10.1 that the map
$f\to Z_{\mu,k}(f)$ is a linear transformation of $\Cal H_{\mu,k}^0$
to $\Cal G_k$, and $\frac1{k!}\|f\|_2^2=EZ^2_{\mu,k}(f)$ for all
$f\in\Cal H_{\mu,k}^0$, where $\|f\|_2$ denotes the usual $L_2$-norm
of the function $f$ with respect to the $k$-fold power of the
measure~$\mu$. By the definition of Wiener--It\^o integrals the
set $\Cal G_1$ consists of jointly Gaussian random variables
with expectation zero. The spaces $\Cal H_{\mu,0}$ and $\Cal G_0$
consist of the real constants. Let us define the space
$\text{Exp}\,(\Cal H_\mu)$ of infinite sequences $f=(f_0,f_1,\dots)$,
$f_k\in\Cal H^0_{\mu,k}$, $k=0,1,2,\dots$, such that
$\|f\|_2^2=\summ_{k=0}^\infty\frac1{k!} \|f_k\|_2^2<\infty$. The space
$\text{Exp}\,(\Cal H_\mu)$ with the natural addition and
multiplication by a constant and the above introduced norm
$\|f\|_2$ for $f\in\text{Exp}\,(H_\mu)$ is a Hilbert space which is
called the Fock space in the literature.

Let $\Cal G$ denote the class of random variables of the form
$$
Z(f)=\summ_{k=0}^\infty Z_{\mu,k}(f_k), \quad
f=(f_0,f_1,f_2,\dots)\in\text{Exp}\,(\Cal H_\mu).
$$
The next result describes the structure of the space of random
variables $\Cal G$. It is useful for a better understanding of
Wiener--It\^o integrals, but it will be not used in the sequel.
In its proof I shall refer to some basic measure theoretical
results.

\medskip\noindent
{\bf Theorem 10.5. (Isomorphism of the space of square integrable
random variables measurable with respect to a white noise with a
Fock space).} {\it Let a non-atomic $\sigma$-finite measure $\mu$
be given on a measurable space $(X,\Cal X)$ together with a white
noise $\mu_W$ with reference measure $\mu$. Let us consider the
class of functions $\Cal H^0_{\mu,k}$, $k=0,1,2,\dots$, and
$\text{\rm Exp}\,(\Cal H_\mu)$ together with the spaces of random
variables $\Cal G_k$, $k=0,1,2,\dots$, and $\Cal G$ defined above.
The transformation $Z\colon\; Z(f)=\summ_{k=0}^\infty Z_{\mu,k}(f_k)$,
$f=(f_0,f_1,f_2,\dots)\in\text{\rm Exp}\,(\Cal H_\mu)$, is a
unitary transformation from the Hilbert spaces
$\text{\rm Exp}\,(\Cal H_\mu)$ to $\Cal G$.
The Hilbert space $\Cal G$ consists of all random variables with
finite second moment, measurable with respect to the
$\sigma$-algebra generated by the random variables $\mu_W(A)$,
$A\in\Cal X$, $\mu(A)<\infty$. This $\sigma$-algebra agrees with
the $\sigma$-algebra generated by the random variables
$Z_{\mu,1}(f_1)$, $f_1\in\Cal H_{\mu,1}^0$.}

\medskip\noindent
{\it Proof of Theorem 10.5.} Properties a) and c) in Theorem~10.1
imply that the transformation $f_k\to Z_{\mu,k}(f_k)$ is a linear
transformation of $\Cal H_{\mu,k}^0$ to $\Cal G_k$, and
$\frac1{k!}\|f_k\|^2_2=EZ_{\mu,k}(f)^2$. Beside this,
$EZ_{\mu,k}(f)Z_{\mu,k'}(f'_{k'})=0$ if $f_k\in \Cal H^0_{\mu,k}$,
and $f'_{k'}\in \Cal H^0_{\mu,k'}$ with $k\neq k'$ by properties~d)
and~c). (The latter property is needed to guarantee this relation
also holds if $k=0$ or $k'=0$.) From these relations follows that
the map $Z\colon\; Z(f)=\summ_{k=0}^\infty Z_{\mu,k}(f_k)$,
$f=(f_0,f_1,f_2,\dots)\in\text{Exp}\,(\Cal H_\mu)$ is an isomorphism
between the Hilbert spaces $\text{Exp}\,(\Cal H_\mu)$ and $\Cal G$.

It remained to show that $\Cal G$ contains all random variables with
finite second moment, measurable with respect to the corresponding
$\sigma$-algebra. Let $g_j(u)$, $j=1,2,\dots$, be an orthonormal
basis in $\Cal H_{\mu,1}^0=\Cal H_{\mu,1}$, and introduce the random
variables $\eta_j=Z_{\mu,1}(g_j)$, $j=1,2,\dots$. By It\^o's formula
for Wiener--It\^o integrals (Theorem~10.3) these random variables
are independent with standard normal distribution, and all
expressions of the form
$H_{r_1}(\eta_{j_1})\dots H_{r_p}(\eta_{j_p})$ with
$r_1+\dots+r_p=k$ are in the space $\Cal G_k$, where $H_r(\cdot)$
denotes the Hermite polynomial of order $r$ with leading
coefficient~1. To prove the desired statement by means of these
relations we still need the following results from the classical
analysis:

\medskip
\item{a)} Hermite polynomials constitute a complete orthonormal
system in the $L_2$-space on the real line with respect
to the standard normal distribution. (This result will be proved
in Section~C in Proposition~C2.)
\item{b)} If a random variable $\zeta$ is measurable
with respect to the $\sigma$-algebra generated by some random
variables $\eta_1,\eta_2,\dots$, then there exists a Borel
measurable function $f(x_1,x_2,\dots)$ on the infinite product of
the real line $(R^\infty,\Cal B^\infty)$ in such a way that
$\zeta=f(\eta_1,\eta_2,\dots)$.

\medskip
This means in our case that any random variable $\zeta$ measurable
with respect to the $\sigma$-algebra generated by the random
variables $\eta_j=Z_{\mu,1}(g_j)$, $j=1,2,\dots$, can be written in
the form $\zeta=f(\eta_1,\eta_2,\dots)$ with the above introduced
independent, standard normal random variables $\eta_1,\eta_2,\dots$.
If $\zeta$ has finite second moment, then the function $f$ appearing
in its representation is a function of finite $L_2$-norm in the 
infinite product of the real line with the infinite product of the
standard normal distribution on it. Hence some classical results 
in analysis enable us to expand the function $f$ with respect to 
products of Hermite polynomials, and this also yields the
identity
$$
\zeta=\sum c(j_1,r_1,\dots,j_s,r_s)H_{r_1}(\eta_{j_1})\cdots
H_{r_s}(\eta_{j_s})
$$
with some coefficients $c(j_1,r_1,\dots,j_s,r_s)$ such that
$$
\sum c^2(j_1,r_1,\dots,j_s,r_s)
\|H_{r_1}(u)\|^2\cdots\|H_{r_s}(u)\|^2<\infty.
$$
(Actually it is known that $\|H_k(u)\|^2=k!$, but here we do
not need this knowledge.)

The above relations yield the desired representation of a random
variable $\zeta$ with finite second moment, if it is measurable
with respect to the $\sigma$-algebra generated by the random
variables in $\Cal G_1$. Indeed, the identity
$\zeta=\summ_{k=0}^\infty \zeta_k$ holds with
$$
\zeta_k=\summ_{r_1+\cdots+r_s=k}
c(j_1,r_1,\dots,j_s,r_s)H_{r_1}(\eta_{j_1})
\cdots H_{r_s}(\eta_{j_s}),
$$
and $\zeta_k\in \Cal G_k$ by It\^o's formula.

To complete the proof it is enough to remark that the
$\sigma$-algebra generated by the random variables
$\eta_1,\eta_2,\dots$ and $\mu_W(A)$, $A\in\Cal X$, $\mu(A)<\infty$
agree, as it was stated in part~f) of Theorem~10.1.

\medskip
The results about Wiener--It\^o integrals discussed in this Section
are useful in the study of non-linear functionals of a set of
jointly Gaussian random variables defined by means of a white noise.
In my Lecture Note~[29] similar problems were discussed, but in that
work a slightly different version of Wiener--It\^o integrals was
introduced. The reason for it was that the solution of the
problems studied in~[29] demanded different methods.

In work~[29] stationary Gaussian random fields were considered, and
the main problem studied there was the description of the limit
distribution of certain sequences of non-linear functionals of such
Gaussian random fields. In a stationary Gaussian random field a 
shift operator can be introduced. The shift of all random variables
measurable with respect to the underlying stationary Gaussian random
field can be defined. In~[29] we needed a technique which helps in
working with the shift operator. Fourier analysis is a useful tool
in the study of the shift operator. In the work~[29] we tried to 
unify the tools of multiple Wiener--It\^o integrals and Fourier 
analysis. This led to the definition of a slightly different 
version of Wiener--It\^o integrals.

The idea behind this definition was the observation that not only
the correlation function of a stationary Gaussian field can be
expressed by means of the Fourier transform of its spectral
measure, but also a random spectral measure can be constructed
whose Fourier transform expresses the stationary Gaussian process
itself. After the introduction of this random spectral measure a
version of the multiple Wiener--It\^o integral can be defined
with respect to it, and all square integrable random variables,
measurable with respect to the $\sigma$-algebra generated by the
underlying Gaussian stationary random field can be expressed with
its help. Moreover, it enables us to apply the methods of multiple
Wiener--It\^o integrals and Fourier analysis simultaneously.
In~[29] such a method was worked out. The modified Wiener--It\^o
integral introduced there shows a behaviour similar to that of
the original Wiener--It\^o integral, only it has to be taken into
account that the random spectral measure behaves not like a white
noise, but as its `Fourier transform'. I omit the details. They
can be found in~[29].

The spaces $\Cal G_k$ consisting of all $k$-fold Wiener--It\^o
integrals were introduced also in~[29], and this was done for a 
special reason. In that work the Hilbert space of square integrable
functions, measurable with respect to an underlying stationary
Gaussian field was studied together with the shift operator acting
on this Gaussian field, which could be extended to a unitary 
operators on this Hilbert space. It was useful to decompose the 
Hilbert space we were working with to the direct sum of
orthogonal subspaces, invariant with respect to the shift operator.
The spaces $\Cal G_k$ were elements of such a decomposition.

In the present work no shift operator was defined, and no limit
theorem was studied for non-linear functionals of a Gaussian field.
Here the introduction of the spaces $\Cal G_k$ was useful because
of a different reason. In the study of our problems we shall need
good estimates on the $2p$-th moment of random variables,
measurable with respect to the underlying white noise for large
numbers~$p$. As it will be shown, the high moments
of the random variables in the spaces $\Cal G_k$ with different
indices~$k$ show an essentially different behaviour. For a large
number $p$ the $p$-th moment of a random variable in $\Cal G_k$
behaves similarly to that of the $k$-th power $\xi^k$ of a Gaussian
random variable $\xi$ with zero expectation. An estimate of this
type will be formulated in Proposition~13.1 or in its consequence,
in formula~(13.2) and in a partial converse of this result, in
Theorem~13.6.

