Footnotes:
Throughout this book, matrices
and vectors in abstract multidimensional space are denoted by
upright bold font. Vectors in physical three-dimensional space are
instead denoted by italic bold font.
This
problem with polynomial fitting is sometimes called Runge's
phenomenon.
Sometimes called simply "orthogonal",
the real version of "unitary".
A
highly abbreviated outline of the SVD is as follows. The
matrix
is symmetric. Therefore it has
real eigenvalues
which because of its form are
non-negative. Its eigenvectors
, satisfying
, can be arranged into an
orthonormal set in order of decreasing magnitude of
. The
eigenvectors can be considered the columns of an orthonormal matrix
, which diagonalizes
so that
is an
non-negative, diagonal matrix with diagonal values
. Since
is diagonal, the
columns of the
matrix
are orthogonal. Its
columns corresponding to
are zero and are not useful to us.
The useful columns corresponding to non-zero
(
,
say,
) can be normalized by dividing by
. Then by
appending
normalized column
-vectors that are orthogonal to
all the previous ones, we can construct a complete
orthonormal matrix
. Here
denotes the
diagonal matrix with elements
and
denotes the appended extra columns. Now consider
. The appended
make zero
contribution to this product because the lower rows of
which they multiply are always zero. The rest of the product is
. Therefore
we have constructed the singular value decomposition
.
If
, the combination
, which
arises from forming
is not an
identity matrix. Instead it has ones only at most for the first
of the
diagonal positions, and zeros thereafter. It is an
identity matrix with extra zero rows and columns padding it out to
. So the pseudo inverse is a funny kind of inverse which
works only one way.
If
is square and
nonsingular, then the pseudo
inverse is exactly the same as the (normal) inverse.
If
were a
singular square matrix, for example (and
possibly in other situations) then at least one of the original
would be zero. We usually consider the singular values (
) to be
arranged in descending order of size; so that the zero values come at
the end.
would then have an element
that is
infinite, and the formal manipulations would be
unjustified. What the pseudo-inverse does in these tricky cases is put
the value of the inverse
equal to zero instead of infinity. In
that case, once again, an incomplete identity matrix is produced, with
extra diagonal zeros at the end. And it actually doesn't
completely "work" as an inverse in either direction.
For those who know some linear algebra, what's happening is that the
pseudo-inverse projects vectors in the range of the original matrix
back into the complement of its nullspace.
See for example,
Numerical
Recipes, W H Press, B P Flannery, S A Teukolsky, and W T
Vettering, Cambridge University Press (first edition, 1989)
(henceforth cited simply as
Numerical Recipes)
Section 2.9.
by QR decomposition
This notation means the first
rows of the compound
matrix consist of
, and the next
rows are
.
This regularization is
equivalent to what is sometimes called a "smoothing spline". In
the limit of large smoothing parameter
, the function
is a straight line (zero second derivative everywhere). In the limit
of small
, it is a cubic spline interpolation through all
the values
.
This topic and many others in data fitting is
addressed for example in Siegmund Brandt (2014)
Data Analysis
Statistical and Computational Methods for Scientists and
Engineers, Fourth Ed., Springer, New York.
Of course Fourier analysis is usually approached
differently, as briefly discussed in the final chapter. We are here
just using sine as an example function, in part because it is
familiar.
There are lots of other
correct but more verbose ways of doing it.
If
is independent of
, then we should use
where
, and we then have a formula accurate to second-order
for performing simple integration
. There are fancier, higher-order, ways
to do integration, but this is by far the most straightforward.
The notation
denotes additional terms whose magnitude is
proportional to a (usually small) parameter
raised to the
power
or higher.
The proof is
rather hard work. Here's an outline. Use notation
to refer to values of the derivative
function along the exact orbit. Suppose we compose
. This is
the form corresponding to eqs. (
2.16) and (
2.17),
but using the exact
on the orbit, rather than the intermediate
estimates. By substituting from eq. (
2.11) it is
easy but tedious to show that this
is equal to
; that is, it is an accurate representation of the
-step to fourth-order. Actually it is instructive to realize that
the symmetry of the
positions,
,
, and
ensures that all even order errors in this
are
zero while the first-order error is zero because the coefficients sum
to unity. The factor of 2 difference between the center and the end
coefficients is the choice that makes the third-order error zero. In
itself this has not proved the scheme to be fourth-order accurate,
because there are non-zero differences approximately proportional to
between the
values and the
values:
,
,
and
. (Because of the order of the
-differences one can quickly see that
terms don't need to be kept.) The successive differences
such as
can be expressed in terms of
the Taylor expansion eq. (
2.12), and then the
-differences are gathered in the combination of eq. (
2.17):
. One can
then evaluate the
terms and demonstrate that they cancel to fourth
order.
A more general form of bisection in which the
interval is divided in two
unequal parts weighted by the
value of the function,
has comparable robustness, requires no additional function
evaluations per step but a couple of extra multiplications, and
converges quicker than plain bisection except in special cases. This
is sometimes called the "false-position" root-finding method. It
is generally possible to detect automatically when one of those
slowly converging special cases is encountered and take additional
measures.
A shorthand way to remember the result
for an order
derivative is that the numerator is the sum of the
neighboring
values times the coefficients of a binomial
expansion to the power
, and the denominator is
.
An alternative
approach to implementing boundary conditions is to regard the
boundary positions as outside and not operated upon by the
difference matrix. Then its indices runs only from
to
, and
it has dimensions
. For Dirichlet conditions, the
term in the difference stencils of positions
and
that
contains the boundary value is moved into the right-hand-side source
vector. Generically the matrix equation is then
|
This form keeps the matrix symmetric, which is advantageous for some
inversion algorithms.
In some
circumstances one can deliberately choose the mesh so as to put the
boundary at
. Then the implementation represented by eq. (3.15) does put the value at the correct place. This mesh choice is
appropriate if the boundary condition is known to be of purely
derivative form. It then lends itself to the alternative approach of
the previous note, excluding the boundary from
. The top
left coefficient becomes
, and
is
added to the source vector, to implement the boundary
condition. Mixed boundary conditions are not handled so easily like
this, which is why a second-order accurate form with the boundary at
has been given.
The error in the
first derivative
is approximately
,
first-order in
, but the error in
is
second-order in
giving first order accuracy.
With periodic boundary conditions the homogeneous
equation (
) is satisfied by any constant
. An
additional condition must therefore be applied to make the solution
unique. Moreover, there exists no solution of the differential
equation with continuous derivative unless
. These
requirements are reflected in the fact that the matrix of this
system is singular. If (
3.19) is solved by
pseudo-inverse, it gives the solution having zero mean:
, using as right-hand-side instead of
the quantity
.
On a structured mesh, a
finite volume method is identical to the finite difference method
provided this conservative differencing is used.
The subscript
on
reminds us this is charge density, not mass density
here.
Vector dependent variable problems are hyperbolic if the
matrix of the coefficients of their differentials is diagonizable with
real eigenvalues, as we'll see later.
Constructed with the distmesh routines from
http://persson.berkeley.edu/distmesh/
For object-oriented purists, this could
be a "method".
In effect, we linearize about an exact solution of
the equations and the
we then analyse is the (presumed small)
difference between the solution we obtain and the exact solution. In
the text we avoid encumbrance of the notation by not drawing
explicit attention to this fact.
The Nyquist limit.
Or, for example, an LU
decomposition.
See for example
Numerical Recipes section 2.6
Octave and
Matlab have a simple built-in function to do reordering, called
reshape().
and of LU decomposition and backsubstitution
We
could alternatively have used integral positions and the second
order scheme of eq.
3.16)
Removing the
(first-order) time derivative changes the classification of the equation
because it is no longer a differential equation in
at all.
Jacobi's method actually can
be used to solve a rather general matrix equation
, and is implemented as
, where
is the diagonal part of
and
is the rest. Its convergence is guaranteed
if
is
diagonally dominant, meaning that the
absolute value of the diagonal term exceeds the sum of the absolute
values of all the other coefficients of the corresponding
row.
For the matrix expert, the advancing
matrix is then explicitly "two-cyclic, and consistently ordered".
A Fourier treatment is not rigorously
justified in general, but serves to illustrate simply the most
important characteristics.
See the enrichment section
for an outline if you are
interested. G D Smith
Numerical Soution of Partial
Differential Equations, Oxford University Press, 1985, p275ff
gives an accessible detailed treatment.
If we apply the boundary
conditions, then the
eigenmode is actually
but the complex Von Neumann
analysis gives the same result.
In
fact, if
and
, then
, or
alternatively if
, and
is not far smaller
than
, then
.
the
difference between the "converged" iterative
and the
analytic
normalized to the analytic value.
If the Jacobian is position-dependent, then the
modes are only approximately uncoupled, because our analysis is
effectively assuming that
and
commute, which is not exact if
is spatially-varying. The
stability analysis is then local, and only approximate. But in any
case Von Neumann stability analysis is only approximate in
non-uniform cases.
provided the eigenvectors
are linearly independent
Although this can be proved, it
is complicated.
There is considerable subtlety
in the question "how abruptly". After all any velocity change
really takes a finite time and involves an acceleration. Why isn't
it included in
? For present purposes we'll just finesse
this question by saying that the source contains any velocity
changes that are not included in the acceleration.
(not multiple dependent
variables that have to be arranged into a vector representation
whose eigenvalue might not be real)
The transport of
radiation, of
photons, can also be treated in this
way, but their treatment has to be expressed in terms of energy (or
momentum) rather than speed, since all photons travel at the same
speed.
Also there may be other spontaneous decays or
fissions (delayed neutrons) but we'll ignore them for now.
Most reactor physics texts express the velocity
distribution differently, in terms of flux density
,
as the dependent variable. To retain the more general relevance to
solving Boltzmann's equation, we here keep
as the dependent
variable. The result is still equivalent to standard reactor physics
equations.
Although there we might have to include
self-collisions.
In practice delayed neutrons introduce a
severe complication.
Computational size reduction, retaining spatial
dependence, might call for a very small number of speed
groups. Perhaps even a single group
. Then the neutron
transport is reduced to a single diffusion equation.
And the boundary conditions are also homogeneous. Of
course "homogeneous" does not here mean uniform in space; it means
having no constant terms.
Strictly speaking,
the inverse of the eigenvalues of the matrix
.
A generalized eigenvalue is the solution
to
where both
and
are general matrices.
Actually if it were the largest eigenvalue of
, for a single group or diagonal
we could
directly use the "power method", which is eq.
9.16 with
the
-indices (
,
) exchanged. However, because we want the
smallest eigenvalue
, we must effectively invert
(which requires interior iteration) because
is never
diagonal.
e.g. SOR
See e.g.
Applied
Reactor Physics Alain Hébert, Presses internationales
Polytechnique, www.polymtl.ca, (2009). Actually many other choices
of weighting will work as well, not just
.
The zeroes in the non-diagonal matrices arise because
scatterings hardly ever move neutrons to higher energy or reduce the
energy by more than one group, and fissions give rise only to
energetic neutrons. The values given are roughly appropriate for a
pressurized water reactor.
In reactor physics
is called
the "Buckling".
Alder and Wainwright, "Phase transition for a hard
sphere system", J. Chem. Phys. 27, 1208-1209, (1957). Liquid
behavior is an important phenomenon for which Molecular Dynamics is
useful.
For fixed
, the Verlet and leap-frog
schemes are equivalent, with the identification
. The Verlet scheme,
implemented in terms of velocity as in eq. (
10.1),
requires more storage or more acceleration evaluations: because two
values of
are needed. It can be implemented in terms of
the position advance alone as
, which
requires the same storage as leap-frog.
Since the step cost for a neighboring
domain of size
is
, and the step
cost for the neighbor update is
, a formal
optimum occurs when these are equal
. So
formally, the per-step cost of this
-optimized algorithm would
be
: slightly better than
.
See
for example C K Birdsall and A B Langdon (1991)
Plasma Physics via Computer Simulation, IOP
Publishing, Bristol; or R W Hockney and J W Eastwood (1988)
Computer Simulation using Particles, Taylor and
Francis, New York
See for example the SOR estimates.
Siegmund Brandt
(2014)
Data Analysis Statistical and Computational Methods for
Scientists and Engineers, Fourth Ed., Springer, New York, gives a
much more expansive introduction to statistics and Monte Carlo
techniques.
Statisticians use the
generic word "sample" to refer to the particular result of a
single test or measurement.
Division by the factor
rather than
makes this formula an unbiassed estimate of the
distribution variance. One way to understand this is to recognize
that the number of degrees of freedom of
is
, not
. Using
is sometimes called "Bessel's
correction".
Often called the "expectation" of
Central Limit Theorem. It is not
straightforward to prove that the distribution becomes Gaussian. But
it is fairly easy to show that the variance of the sample mean is
the variance of the distribution divided by
. From the definition
of
one can immediately deduce
that
|
Take the expectation
of this quantity to obtain the variance of
the distribution of sample means:
|
The first
equality, taking the expectation inside the sum, is a simple
property of taking the expectation: the expectation of a sum is the
sum of the expectations. The second equality uses the fact that
for
because the
quantities
are statistically independent and have zero
mean. That is sufficient to yield the required result. Our estimate
for the distribution variance is
. So the unbiassed
estimate for the variance of
is
. The standard error is the
square root of this quantity.
An un-normalized
Gaussian distribution has three, including the height.
Linear interpolation is then
equivalent to representing
as a histogram. So adequate
resolution may require a fairly large number
.
Quiet
start and quasi-random selection. When starting a
particle-in-cell simulation, the initial positions of the particles
might be chosen using random numbers to decide their
location. However, they then will have density fluctuations of
various wavelengths that in plasmas may be
bigger than are
present after running the simulation for many steps. The reason for
this discrepancy is that the feedback effect of the self-consistent
electric potential tends to smooth out density fluctuations, so that
in the fully developed simulation the noise level is lower than
purely random. A simple way of saying the same thing is that
individual particles
repel others of the same type,
preventing clumping of the particles. It is therefore often
physically reasonable to start a PIC simulation with positions that
are chosen to be more evenly spaced than purely random. Indeed, for
some calculations it is advantageous (but non-physical) to start
with density fluctuations even
lower than the final level
that would be present for a steady plasma. In either case, what is
called a "Quiet Start" can be obtained by using what are called
"quasi-random" numbers [see e.g.
Numerical Recipes]
instead of (pseudo) random numbers. Quasi-random numbers are
somewhat random, but much smoother in their distribution because
each new number takes account of the already used numbers and tries
to avoid being close to them. Successive numbers are thus correlated
rather than uncorrelated. For Monte-Carlo integration, such smoother
distributions in space are also often highly appropriate and can
give lower-noise results for the same number of samples, beating the
weak,
, decrease of fractional error.
Or if we wish to do
"quiet injection" that is smoother than purely random
The
Discrete Poisson distribution. Suppose there is a very large
number
of similar, uncorrelated, events (for example radioactive
decays of atoms) waiting to occur. In a time duration far shorter
than the waiting time (e.g. the half-life) of an individual event
the probability that any one of them occurs is small, say
. Then
is the average total number of events occuring in
this time duration. The total number of events actually occuring in
a particular time sample is then an integer, and we want to find the
probability of each possible integer. Any sample consists of
choices about the individual events: yes or no. The number of yes
events is distributed as a Binomial distribution, in which the
probability of
yes events is given by the number of different
ways to choose
out of
, which is
, times
the probability of each specific arrangement of
yes events and
no events,
. The total is
|
Now recognize that the limit for large
(but constant
) of
the square bracket term is 1; while the limit of the term
is
. Therefore the
probability of obtaining
total events, of a type that are completely
uncorrelated (
),
when their average rate of occurrence is
is
This is the discrete Poisson distribution.
Or double-precision if
is very large.
The random generators I used here are both
portable. The good generator is the
RANLUX routine
described by M. Luscher,
Computer Physics Communications 79
(1994) 100, and F. James,
Computer Physics Communications
79 (1994) 111, used with the lowest luxury level. The bad generator
is the
RAN1 routine from the first (FORTRAN) edition of
Numerical Recipes (1989). It was replaced by better
routines in later editions.
For photons, the "velocity" should
be interpreted as a combination of energy (frequency or wavelength)
and propagation direction, since the particle's speed is always the
speed of light.
The velocity distribution
in discrete velocity
bins
may also be desired. It is given by summing only
those passages that occur in the velocity bin:
. It will have greater statistical
uncertainty because of fewer samples per bin.
One such book with lots of engineering examples is
Steven C Chapra and Raymond P Canale (2006) (5th Ed, or later)
"Numerical Methods for Engineers" McGraw-Hill, NY.
The volume of a
tetrahedron can be written as a
determinant using the
cartesian coordinates
of its four corners (
) as
|
These important details are discussed
in texts devoted to finite elements, such as, Thomas J R Hughes
(1987) "The Finite Element Method" Prentice Hall, Englewood
Cliffs, NJ. The classic text on the background mathematical theory
is, G Strang and G J Fix (1973, 2008) "An Analysis of the Finite
Element Method", Reissued by Wellesley-Cambridge, Press, USA.
Since
is periodic, its value at
is the same
as at
, so the range is equivalent to
.
If
is odd, then
the highest frequency is obviously
, but we'll avoid
reiterating this alternative.
is a small adjustment of the end
point to avoid it falling on a delta function.
An
extensive discussion is given in W H Press et al, "Numerical
Recipes", chapter 12. Various open source implementations are
available.
A non-zero
can be removed by reexpressing the equation in terms of
.
The present derivation draws on C P Jackson
and P C Robinson (1985) "A numerical study of various algorithms
related to the preconditioned conjugate gradient method",
International Journal for Numerical Methods in Engineering, Vol 21,
pp 1315-1338, and G Markham (1990) "Conjugate Gradient Type Methods
for Indefinite, Asymmetric, and Complex Systems" IMA Journal of
Numerical Analysis, Vol 10, pp 155-170.
This is a type of Gramm-Schmidt
basis-set construction.
Succinct descriptions of a wide
range of iterative algorithms are given by R. Barrett, M. Berry,
T. F. Chan, J. Demmel, J. Donato, J. Dongarra, V. Eijkhout, R. Pozo,
C. Romine, H. Van der Vorst (1994) "Templates for the Solution of
Linear Systems: Building Blocks for Iterative Methods" 2nd Edition,
SIAM, Philadelphia, which is available at
http://www.netlib.org/linalg/html_templates/report.html. Open
source libraries of implementations of Krylov schemes are widely
available. One of the more comprehensive is "SLATEC" at
http://www.netlib.org/slatec/.
Bi-conjugate gradient iterations are implemented using two residual
and two search direction vectors requiring just one multiplication
by
and one by
per step in this way:
1 Initialize: | choose , ,
, choose , , set . |
2 Calculate : | |
3 Update s, : | ,
, |
| |
4 Calculate : | |
5 Update s: |
and |
6 Convergence? | If not converged, increment and repeat from 2. |
The conjugate gradient scheme is the same except no barred
quantities are calculated and unbarred quantities replace them.
A compact mathematical description is given, for
example, by Stephen Jardin (2010) "Computational Methods for Plasma
Physics" CRC press, Taylor and Francis, Boca Raton. Section
3.7.2.
The
number of iterations required to converge to a given accuracy for
separated eigenvalues is approximately proportional to the square
root of the matrix condition number. The condition number is the
ratio of the maximum to the minimum eigenvalue. That is unity for
the unit matrix because all eigenvalues are 1. For a matrix
representing an isotropic finite-difference Laplacian, the
eigenvalues are the squared wave numbers of the spatial Fourier
modes. The condition number is therefore the sum of the squares of
the number of mesh points in each dimension. The number of
iterations required is therefore proportional to the mesh count of
the largest dimension. That is the same as optimized
SOR. Finding the optimum relaxation parameter for SOR is relatively
easy for a rectangular mesh on a cubical domain. Consequently
un-preconditioned Krylov methods for finite-difference equations in
simple domains are generally not much faster than SOR. Krylov methods come
into their own when the matrices are more complicated and optimizing
SOR is impossible, for example for finite element methods on unstructured
grids.
There are some problems in liquids
where sound waves are important; just a minority.
This equation arises from substituting
into the (sourceless) continuity equation, so
. If
is non-zero, so
is non-zero
even though
, then the pressure equation
acquires an additional term.
See, e.g. J H Ferziger and M Perić (2002)
"Computational Methods for Fluid Dynamics" third edition,
Springer, Berlin.
See, e.g. S Jardin (2010) op
cit.
A comprehensive introduction to limiter
methods is given in Randall J Leveque (2002) "Finite Volume Methods
for Hyperbolic Problems", Cambridge University
Press.
A large
jet aircraft may reach
.
See for example U. Piomelli
(1999) "Large-eddy simulation: achievements and challenges"
Progress in Aerospace Sciences, Vol 35, pp 335-362
or
the product of two different fluctuating quantities
See for example B E Launder, G J Reece,
and W Rodi (1975) "Progress in development of a Reynolds-stress
turbulence closure", Journal of Fluid Mechanics, Vol 68, pp
537-566.
Of course there are deeper ways to think about vector
spaces, but we here use the simplest approach.
So a column
vector can be considered a
matrix and a row vector a
matrix.
Consider recursively expanding the determinant and
subsequent cofactors, always choosing not to expand by the rows that
are identical. Eventually one ends up with all
cofactors composed
of the identical rows. They are all zero.
A left
inverse
such that
must always
be equal to a right inverse
such that
because
.
HEAD
File translated from
TEX
by
TTM,
version 4.12.