Author: 孤筝(lvbowen040427@163.com)

Optimization Problems

Mathematical Model

where

• $\vec{x}=(x_1,x_2,\cdots,x_n)^T$ is called the decision variable vector
• $f(\vec{x})$ is the objective function
• s.t. means subject to, that is, the constraints

Classification

1. By time
   1. Static problems
   2. Dynamic problems
2. By constraints
   1. Constrained problems
   2. Unconstrained problems
3. By whether the objective and constraints are linear
   1. Linear programming
   2. Nonlinear programming
4. By whether the objective and constraints are convex
   1. Convex optimization problems
   2. Non-convex optimization problems

Quadratic Form Matrix

Quadratic form:

Quadratic form matrix:

Hessian Matrix

Take a two-variable quadratic function as an example:

Feasible Solutions

• Feasible solution: a solution satisfying all constraints.
• Feasible set (admissible set, feasible region): the set of all feasible solutions.
• Optimization problem: find the point in the feasible set at which the objective function attains its maximum or minimum.
• Stationary point: if $\nabla f(x_0)=0$, then $x_0$ is called a stationary point.
• Saddle point: if $x_0$ is a stationary point but not an extremum point, then it is called a saddle point.

Convex Sets

Definition

In the plane, if the line segment joining any two points inside a figure still lies entirely inside the figure, then the figure is called a convex set.

Properties

1. The intersection of convex sets is convex.
2. A scaled convex set is still convex.
3. The sum set of convex sets, not the union, is convex.
   • If $D_1,D_2$ are convex sets, then $D_1+D_2=\{z|z=x+y,x \in D_1,y \in D_2\}$ is convex.
4. A linear combination of convex sets is convex.

Common Convex Sets

1. The empty set
2. The whole Euclidean space $\vec{R^n}$
3. A hyperplane $H=\{x \in \vec{R^n} | a_1x_1+a_2x_2+\cdots +a_nx_n=b\}$
4. A half-space $H^+=\{x \in \vec{R^n} | a_1x_1+a_2x_2+\cdots +a_nx_n \ge b\}$

Convex Combination

Let $x_i \in \vec{R^n},i=1,2,\cdots ,k$, and let $\lambda_i \ge 0,\sum^k_{i=1}\lambda_i=1$. Then $x=\sum^k_{i=1}\lambda_ix_i$ is called a convex combination of $x_1,x_2,\cdots , x_k$.
Any finite convex combination of points in a convex set still belongs to that convex set.

Extreme Points

Let $D$ be a convex set and $x \in D$. If there do not exist two distinct points $y,z \in D$ and a real number $\alpha \in (0,1)$ such that $x=\alpha y+(1-\alpha)z$, then $x$ is called an extreme point of $D$.

In plain words: for a pentagon, the extreme points are its vertices; for a semicircle, the extreme points are the two endpoints of its diameter together with the top point on the arc.

Convex Functions

Definition

Let $f(x)$ be defined on a convex set. If for any two points $x_1,x_2$ in the convex set,

then $f$ is called a convex function.
If the inequality is strict, $\lt$, then it is called a strictly convex function.

Criteria

1. If the Hessian matrix of a multivariable function is positive semidefinite, then the function is convex.
2. If the Hessian matrix is positive definite, then the function is strictly convex.
3. A multivariable linear function is convex on $\vec{R^n}$.

Convex Optimization Problems

Definition

An optimization problem in which the objective function and all constraint functions are convex functions.

• The feasible set of a convex optimization problem is convex.
• Any local optimum is also a global optimum.
• If the objective function is strictly convex, then the local optimum exists and is unique.

Linear Programming

Forms

Nonstandard Form

• Objective function: $\max z=\sum^{n}_{j=1}c_jx_j=CX$
• Coefficient matrix: $$ A= \begin{bmatrix} a_{11} & \cdots & a_{1n}\\ \vdots & \ddots & \vdots\\ a_{m1} & \cdots & a_{mn}\\ \end{bmatrix} =(P_1,P_2,\cdots,P_n) $$
• Resource vector: $b=\begin{bmatrix} b_1\\ \vdots \\ b_m\\ \end{bmatrix}$
• Decision variable vector: $X=(x_1,x_2,\cdots , x_n)^T$
• Constraints: $$ \begin{cases} \sum^{n}_{j=1}a_{ij}x_j=b_i,&i=1,2,\cdots,m\\ x_j \ge 0,& j=1,2,\cdots,n\\ \end{cases} $$ $$ \begin{cases} AX=b\\ X \ge \vec{0} \end{cases} $$

Standard Form

1. Convert a maximization problem into minimization
2. Slack variables: for $\le$ constraints, introduce slack variables to turn inequalities into equalities
3. Surplus variables: for $\ge$ constraints, introduce surplus variables to turn inequalities into equalities
4. Free variables: variables that may take arbitrary real values in practical problems, written as $x_i=x'-x''$

Basis Matrix

• Basis (basis matrix): a largest nonsingular submatrix of the coefficient matrix.
  • If the coefficient matrix $A$ is an $m \times n$ matrix with $rank(A)=m$, then any nonsingular $m \times m$ submatrix may serve as a basis matrix.
• Basic variables: the unknowns corresponding to the column vectors in the basis.
• Nonbasic variables: the unknowns that are not basic variables.
• Basic solution: the solution obtained by setting all nonbasic variables to zero.
• Nondegenerate basic solution: a basic solution in which the number of nonzero components equals the number of constraint equations. Otherwise it is a degenerate basic solution.
• Basic feasible solution: a basic solution satisfying the nonnegativity conditions in $\text{s.t.}$.
• Optimal basic feasible solution: among all basic feasible solutions, the one that gives the optimal objective value.

Properties of Linear Programming Solutions

1. The feasible set of a linear programming problem is convex.
2. If an optimal solution exists, it must be attained at a vertex of the feasible set.

Simplex Method

Reduced Costs

Each unknown corresponds to a reduced cost:

• $C^T$ is the coefficient row of the objective function
• $C^T_J$ is the row of coefficients of the basic variables in the objective function
• $P_j$ denotes the $j$th column of matrix $A$
• $c_i$ denotes the coefficient of the $i$th basic variable in the objective function
• $c_j$ denotes the coefficient of the $j$th variable in the objective function and is unrelated to $c_i$

When all reduced costs are less than or equal to zero, the current basic feasible solution is optimal.

In general, the reduced costs of basic variables are zero.

Basis Transformation

Choosing the Basis Matrix

Prefer the identity matrix as the initial basis matrix. Compute the initial basic feasible solution and the reduced costs.

Constructing the Initial Simplex Tableau

Selecting an Entering Column

If a reduced cost is greater than zero, then the corresponding column has improving potential. Choose this column as the entering column $P_j$, and the corresponding variable $x_j$ is the entering variable.

Selecting the Pivot Element

Among the positive entries $a_{ij}$ in the entering column, divide the corresponding entry of $b$ by each such element and choose the smallest ratio. The corresponding element $a_{ij}$ is the pivot.
If a reduced cost is greater than zero but all entries in that column are negative, then the linear program has no optimal solution.

Elementary Row Operations

Transform the pivot into 1 and all other entries in that column into 0.
Geometric meaning: moving to another vertex of the feasible region.

Leaving Column

Select the new basis matrix according to the updated coefficient matrix. Compared with the old basis, the replaced column is the leaving column, and its corresponding variable is the leaving variable.
Then recompute the reduced costs and form the new simplex tableau.

A New Basis Transformation

If the reduced-cost row changes and new positive reduced costs appear, choose the corresponding column as the new entering column, select a pivot, and perform elementary row operations again.

Result

When all reduced costs are less than or equal to zero, the entries of $\vec{b}$ are the values of the basic variables, while the nonbasic variables are set to 0. Together these form the optimal solution, and substituting them into the objective function gives the minimum value.

Conditions for Applying the Simplex Method

1. All elements in the nonhomogeneous term are nonnegative.
2. A feasible solution exists.
3. The sum of products of slack variables and nonbasic variables is zero.
4. The problem is a linear programming problem on a convex feasible region.
5. The feasible solution set is finite.

Artificial Variable Method

When the coefficient matrix does not contain an identity matrix, one usually introduces artificial variables to construct one artificially.

Suppose the constraints of the linear programming problem are $\sum^{n}_{j=1}a_{ij}=b_i(i=1,2,\cdots ,m)$. Add artificial variables $x_{n+1},x_{n+2},\cdots,x_{n+m}$ to each constraint and use them as the basic variables, so that they form an identity matrix and all other variables are zero. In this way we obtain an initial feasible solution $x^{(0)}=(0,0,\cdots,0,b_1,b_2,\cdots,b_m)^T$.
Starting from this point, carry out basis transformations to obtain an optimal solution without nonzero artificial variables.

If all reduced costs are negative but nonzero artificial variables still remain, then the original problem has no feasible solution.

Big-M Method

For a minimization problem, after introducing artificial variables into the constraints, assign the coefficient $M$ to the artificial variables in the objective function, where $M \in \vec{R^+}$.
To obtain the minimum objective value, one keeps carrying out basis transformations until the artificial variables become zero. For a maximization problem, $M \in \vec{R^-}$.

Degenerate Cases

If the simplex method falls into cycling while the problem does have an optimal solution, the following methods may be used to avoid cycling.

Perturbation Method

Revised Simplex Method

Duality Theory of Linear Programming

Forms of the Dual Problem

Symmetric Form

Primal problem

Dual problem

Correspondence:

• (1) The number of constraints in the primal problem equals the number of variables in its dual.
• (2) The coefficients of the primal objective function become the right-hand-side constants of the dual constraints.
• (3) If the primal objective is minimization, then the dual objective is maximization.
• (4) If the primal constraints are of type “$\ge$”, then the dual constraints are of type “$\le$”.

Asymmetric Form

Primal problem

Dual problem

General Case

If the primal problem contains a mixture of $\le,\ge,=$ constraints, first introduce slack and surplus variables so that all constraints become equalities, and then construct the dual using the asymmetric form.

Dual Simplex Method

• Simplex method: first ensure $\vec{b} \ge 0$, then iterate based on reduced costs $\le 0$.
• Dual simplex method: first ensure reduced costs $\le 0$, then iterate based on $\vec{b} \ge 0$.

Ensuring Reduced Costs $\le 0$

Choosing the Leaving Variable

If there exists a negative $b_i \lt 0$, then the row containing the smallest $\min b_i$ is chosen as the leaving row, and the corresponding variable is the leaving variable.

Choosing the Entering Variable

Divide each reduced cost by the negative coefficient in the leaving row, that is, $a_{ij} \lt 0$. The column corresponding to the smallest resulting value is chosen as the entering column, and the corresponding variable is the entering variable.

Row Operations

Use elementary row operations to transform the entering column into one that matches the basis matrix, that is, an identity column. At this point $\vec{b}$ changes as well.
Then recompute the reduced costs and ensure that they remain less than or equal to zero.

A New Basis Transformation

If there is still a negative value $b_i \lt 0$, choose the smallest $\min b_i$ and perform another basis transformation.

Result

When all $b_i \ge 0$, the vector $\vec{b}$ gives the optimal values of the basic variables, while the nonbasic variables are 0.
Substitute them into the objective function to obtain the optimal value, whether maximum or minimum.

Sensitivity Analysis

Published on 2025-05-10 at 孤筝の温暖小家, last modified on 2025-05-10

All articles on this blog are licensed under the BY-NC-SA license agreement unless otherwise stated. Please indicate the source when reprinting!

Author: 孤筝(lvbowen040427@163.com)

Preface

First Edition Preface

[[2024-09-14]] Today the makeup exam finally ended. I heard the original exam directly reused past papers. These past few days I practiced three sets of “XDU original papers” (from 2021 and two from 2023) found online. I did the 2021 paper in the morning, and in the afternoon $\frac{1}{4}$ of the questions were exact copies without any changes. I couldn’t help but laugh.

Dai Hao once said he would try his best to find the best teachers for the Qian Class. But now it seems the School of Mathematics and Statistics has no one left? Poor teaching could be excused as not focusing on education or lacking talent in teaching; but directly reusing recent past papers for exams, full of unchecked errors and omissions, made me laugh in frustration.

The exams they create have no value, and they don’t even bother to test them themselves. This is an attitude problem.

It’s fine that your university goes easy on final exams, but don’t keep fooling people with old material. You preach innovation to students, yet for yourselves, just getting by is enough. This is not the attitude for academic work, nor is it the attitude one should have for teaching.

Probability theory ends here for now. Over the past two days, I repeatedly reviewed notes, practiced problems, and corrected many errors, clarifying the knowledge structure of this course. Although the content is still relatively sparse, it should suffice as final review material. This edition will likely be the final version (probably). I’ll continue organizing Electrodynamics and Digital Signal Processing during the Mid-Autumn Festival.

Second Edition Preface

Nothing is final!!! ——Qian Xuesen

Added content on the left/right continuity of distribution functions. It seems this course is far from final…

Event Operations to Logical Operations

• $A \cup B=A+B$
• $A \cap B=A \cdot B$
• $A-B=A \bar{B}$ Event $A$ occurs and event $B$ does not occur, easily proven by Venn diagrams. $-B$ can be interpreted as $\cdot (-B)$, where $-B$ is $\bar{B}$.
• If $A \subset B$, then $A \cup B=B$, $A \cap B=A$.

After converting event operations to logical operations, most rules are shared. Using logical function operations and simplification learned in digital circuits, complex event operations can be simplified. Tips: Karnaugh maps.

Four Major Probability Formulas

Corollary

$P(A+B+C)$: Treat $A+B$ as a single event and apply the addition formula above, splitting twice to get:

Probabilities for more joint events can be derived recursively.

Complementary event: The probability that $A$ does not occur, obvious from Venn diagrams.

Non-Negativity and Normalization

Non-negativity: For any event $A$, $0 \le P(A) \le 1$. Normalization: For the total event $\Omega$, $P(\Omega)=1$.

Independence

Independence implies mutual independence.

Classical Probability Model

All elementary events have equal probability.

Eg. Coin toss, dice roll…

Classical conditional probability formula:

Bernoulli Trials (Binomial Distribution)

$n$ independent trials, each with only two outcomes: $A$ or $\bar{A}$.

$X \sim B(n,p)$

Where $p=P(A)$, $1-p=P(\bar{A})$.

Geometric Probability Model

The ratio of the length/area/volume occupied by the event to the total length/area/volume of the sample space $\Omega$. When the event’s dimension is lower than $\Omega$’s dimension, its probability is always 0. ==Warning==: A probability of 0 does not mean the event cannot occur. Eg: Randomly selecting a point inside a circle, the probability of selecting any specific point is 0, but it can still happen.

Uniform Distribution

$x \sim U(a,b)$ Approximates a linear distribution in geometric probability, with probability density:

Cumulative distribution function:

Exponential Distribution

$x \sim E(\lambda)$

Probability Density

Cumulative Distribution Function

Poisson Distribution

$X \sim \pi(\lambda)$

Normal Distribution

$x \sim N(\mu,\sigma^2)$

Probability Density

Cumulative Distribution Function

Clearly, $F(\mu)=\frac{1}{2}$, meaning $P(x \le \mu)=P(x \gt \mu)=\frac{1}{2}$.

Standard Normal Distribution

When $\mu=0,\sigma=1$, it becomes the standard normal distribution.

Corollaries

Normalization of normal distribution:

Total Probability Formula

Complete Event Group

$B_1,B_2,B_3,\cdots B_n$ form a complete event group for $\Omega$.

Total Probability Formula

Bayes’ Formula

One-Dimensional Discrete Random Variables

Probability Mass Function

Cumulative Distribution Function

One-Dimensional Continuous Random Variables

Probability Density Function

Cumulative Distribution Function

Interval Probability

$\because$ $P(x=a)=0,a \in R$ $\therefore$ The equality signs on the interval can be chosen freely.

Normalization

Two-Dimensional Discrete Random Variables

Joint Probability Mass Function

$P(X=x_i,Y=y_j)$ Create a 2D table of possible values for X and Y, filling in corresponding probabilities.

Marginal Probability Mass Function

$P(X=x_i),P(Y=y_j)$ Sum the rows/columns of the joint probability table to get $f_Y(x),f_X(y)$.

Conditional Distribution

$P(X=x_i|Y=y_j),P(Y=y_i|X=x_j)$ Divide each row/column of the joint probability table by its marginal probability. This scales the joint probabilities so each row/column sums to 1.

Independence of Two Variables

==Independence here refers to linear independence, not complete statistical independence.== Write the joint probability table as a matrix $\vec{A}$. If $\det \vec{A}=0$, X and Y are independent. Or: If the rows/columns of the joint probability table are proportional, X and Y are independent. Or: If the joint probability $\ne$ the product of marginal probabilities, i.e., $P(X=x_i,Y=y_j)\ne P(X=x_i)P(Y=y_j)$, then X and Y are not independent.

Two-Dimensional Continuous Random Variables

Joint Density Function

Normalization

Marginal Density Functions

Conditional Density

Independence

When this holds, X and Y are independent.

Distribution Function

Let $Z=X-Y$,

The distribution function $F_Z(z)=\iint_Df(x,y)dxdy$. Differentiate to get the probability density function $f_Z(z)$. ==Warning==: $F_Z(z)$ must satisfy normalization.

Expectation and Variance

Relations

When X and Y are independent, $Cov(X,Y)=0$.

Common Expectations and Variances

$(0,1)$ Distribution

$B(n,p)$ Binomial Distribution

$U(a,b)$ Uniform Distribution

$E(\lambda)$ Exponential Distribution

$P(\lambda)$ Poisson Distribution

$N(\mu,\sigma^2)$ Normal Distribution

Covariance and Correlation Coefficient

Covariance

Clearly, when $X=Y$, $Cov(X,X)=DX$.

Correlation Coefficient

Higher $|\rho|$ means stronger correlation. When $Y=X$, $X$ and $X$ are perfectly correlated, $\rho=1$. When $Y=-X$, $-X$ and $X$ are perfectly correlated, $\rho=-1$. Clearly $|\rho| \le 1$. $\rho=0$ means X and Y are uncorrelated. ==Warning==: Uncorrelated $\nRightarrow$ Independent, but Independent $\Rightarrow$ Uncorrelated.

Chebyshev’s Inequality for Probability Estimation

Central Limit Theorem

A large number of independent, identically distributed variables can be approximated by a normal distribution. If $x_1,x_2,\cdots,x_n$ are independent and identically distributed, then

Three Major Distributions

$\chi^2$ (Chi-Squared) Distribution

Upper $\alpha$ quantile $\chi^2_\alpha(n)$ Density function is in the first quadrant.

$t$ Distribution

Upper $\alpha$ quantile $t_\alpha(n)$ Density function resembles normal distribution, symmetric.

$F$ Distribution

Upper $\alpha$ quantile $F_\alpha(n_1,n_2)$ Density function is in the first quadrant.

Estimation Methods

For simple random samples that are independent and identically distributed, estimate unknown parameters.

Method of Moments

When sample size is large, approximate the sample as uniformly distributed, using sample mean to replace population mean (population moment = sample moment).

1. Calculate the expectation $EX$ (first population moment) from the given probability mass/density function.
2. Calculate the sample mean $\bar{X}$ (first sample moment) from the given sample.
3. Set $EX=\bar{X}$ and solve for $\theta_0$ as $\hat{\theta}$.

Maximum Likelihood Estimation

The estimate maximizes the probability of the observed sample. Likelihood function for the sample:

To find the maximum of $L$, take the derivative to find critical points. Since the product form is cumbersome, first take the logarithm before differentiating with respect to $\theta$.

Solve for the critical point $\theta_0$, which is the estimate $\hat{\theta}$.

Unbiasedness and Efficiency

If $E(\hat{\theta})=\theta$, then $\hat{\theta}$ is an unbiased estimator of $\theta$. If $\hat{\theta_1},\hat{\theta_2}$ are both unbiased, and $D(\hat{\theta_1}) \lt \hat{\theta_2}$, then $\hat{\theta_1}$ is more efficient than $\hat{\theta_2}$.

Interval Estimation

$X \sim N(\mu,\sigma^2)$, typically given $\bar{X}=\mu,S=\sigma$. Confidence level: $1-\alpha$, usually $\alpha=5\%$.

Confidence Interval for $\mu$

$\sigma^2$ Known

Pivotal quantity (standardized):

$\sigma^2$ Unknown

Pivotal quantity:

Confidence Interval for $\sigma^2$

Usually $\mu$ is unknown. Pivotal quantity:

Hypothesis Testing

Generally, the significance level is set at $\alpha=5\%$.

$\mu$ Test (Mean Test)

1. Hypothesis Formulation
   $H_0: \mu = \mu_0$ (null hypothesis)
   $H_1: \mu \ne \mu_0$ (alternative hypothesis)

2. Test Statistic Selection

   • When population variance $\sigma^2$ is known:
     Use $Z = \frac{\bar{X} - \mu}{\sigma / \sqrt{n}} \sim N(0,1)$ (Z-test)
   • When population variance $\sigma^2$ is unknown:
     Use $T = \frac{\bar{X} - \mu}{S / \sqrt{n}} \sim t(n-1)$ (T-test)

3. Rejection Region Determination

   • For Z-test:
     $W = (-\infty, -z_{\alpha/2}) \cup (z_{\alpha/2}, \infty)$
   • For T-test:
     $W = (-\infty, -t_{\alpha/2}(n-1)) \cup (t_{\alpha/2}(n-1), \infty)$

4. Decision Rule
   Reject $H_0$ if the computed test statistic falls within the rejection region $W$.

$\sigma^2$ Test (Variance Test)

Sample standard deviation formula:

1. Hypothesis Formulation
   $H_0: \sigma^2 = \sigma_0^2$
   $H_1: \sigma^2 \ne \sigma_0^2$

2. Test Statistic Selection
   Use $\chi^2 = \frac{(n-1)S^2}{\sigma^2} \sim \chi^2(n-1)$ (Chi-square test)

3. Rejection Region Determination
   $W = (0, \chi^2_{1-\alpha/2}(n-1)) \cup (\chi^2_{\alpha/2}(n-1), \infty)$

4. Decision Rule
   Reject $H_0$ if the test statistic falls within the rejection region $W$.

Supplementary Notes

Properties of Distribution Functions

For different types of random variables:

• Continuous random variables: The distribution function is continuous.
• Discrete random variables: The continuity of the distribution function depends on its definition.

Left-Continuous Definition

In this case:

• $F(x) = F(x^-) = F(x-0) = P(X \lt x)$
• $F(x^+) = F(x+0) = P(X \lt x) + P(X = x)$

When $P(X = x) \ne 0$, $F(x^+) \gt F(x) = F(x^-)$, making the distribution function left-continuous but not right-continuous.

Right-Continuous Definition

In this case:

• $F(x) = F(x^+) = F(x+0) = P(X \le x)$
• $F(x^-) = F(x-0) = P(X \le x) - P(X = x)$

When $P(X = x) \ne 0$, $F(x^+) = F(x) \gt F(x^-)$, making the distribution function right-continuous but not left-continuous.

Coin Toss Example

Consider a single coin toss:

• Heads (1): Probability 0.5
• Tails (0): Probability 0.5

Random variable $X$ has the distribution:

Cumulative probabilities:

Using the left-continuous definition $F(x) = P(X \lt x)$:

Here:

• $F(0^-) = F(0) = 0$
• $F(0^+) = 0.5$
• At $x=0$, there is a discontinuity point where the function is left-continuous but not right-continuous.

Published on 2024-09-10 at 孤筝の温暖小家, last modified on 2024-09-10

All articles on this blog are licensed under the BY-NC-SA license agreement unless otherwise stated. Please indicate the source when reprinting!

Author: 孤筝(lvbowen040427@163.com)

Foundations of Real Numbers Book Report

Taking advantage of this mathematical analysis assignment, I’d like to document the construction of real numbers.

The establishment of real numbers forms part of the foundational theories in mathematics, intersecting with multiple branches such as mathematical logic, set theory, and algebraic structures. Through rigorous derivation of these fundamental concepts and properties, mathematicians have constructed the real number system, providing a solid foundation for mathematical development. This process has evolved over a long history with contributions from numerous mathematicians.

1. Book Information

1.1 Mathematical Analysis

• Author: Tom A. Apostol
• Year of Publication: 1973
• Introduction: Mathematical Analysis is a classic work by Tom M. Apostol, systematically introducing foundational knowledge in mathematical analysis, including the real number system, limits, continuity, and more. With clear logic and profound insight, the author helps readers develop a deep understanding of real numbers.

1.2 Real Analysis and Functional Analysis

• Author: Kuang Jichang
• Year of Publication: 2002
• Introduction: Real Analysis and Functional Analysis is an advanced mathematics textbook by Professor Kuang Jichang, primarily covering the basic concepts, theories, and methods of real analysis and functional analysis. A distinctive feature of this book is its integration of traditional real-variable function theory, measure theory, and functional analysis into a modern foundational course using sets and mappings.

1.3 Real and Complex Analysis

• Author: Walter Rudin
• Year of Publication: 2006
• Introduction: This book is a classic in the field of analysis. Its elegant structure and practical utility are complemented by concise and brilliant examples. Both the real and complex analysis sections rigorously prove nearly all propositions presented.

1.4 Real Analysis

• Author: Halsey Royden, Patrick Fitzpatrick
• Year of Publication: 2010
• Introduction: This book has become a classic in mathematical analysis, offering students a profound theoretical foundation. The fifth edition includes significant updates, comprehensively covering measure theory, integration theory, and topics essential for modern analysts, such as metric, topological, Hilbert, and Banach spaces.

2. The Real Number System

The real number system is the cornerstone of mathematical analysis. Apostol’s book details the definition and properties of real numbers. Key features such as completeness and density make real numbers fundamental to mathematical analysis.

The construction of real numbers involves basic mathematical concepts and systematic development. The real number system provides a complete description of quantities, including integers, rational numbers, and irrational numbers.

2.1 Introduction to Rational Numbers

1. Natural Numbers: The starting point is the natural numbers: 1, 2, 3, 4, …, used for counting and ordering.

2. Integers: To address subtraction, zero and negative integers are introduced, forming the integer system: positive integers, zero, and negative integers.

3. Rational Numbers: While integers solve subtraction, division remains limited. For example, calculating $\frac{1}{3}$ or $\frac{2}{7}$ reveals numbers not expressible as integers. Rational numbers extend the integer system, ensuring any ratio of two integers belongs to this new set. The rational number system addresses gaps in the integer system, ensuring density—between any two rational numbers lies another rational number.

4. Properties of Rational Numbers: Rational numbers are closed under addition, subtraction, multiplication, and division (except by zero). These properties make rational numbers a complete number system.

2.2 Introduction to Irrational Numbers

1. Limitations of Rational Numbers: Although rational numbers represent most quantities, some numbers, like $\sqrt{2}$, cannot be expressed as a ratio of integers. No integers $a$ and $b$ satisfy $\frac{a}{b} = \sqrt{2}$.

2. Definition of Irrational Numbers: To fill this gap, irrational numbers are introduced—numbers not expressible as ratios of integers, i.e., non-rational numbers.

3. Transcendental Irrational Numbers: These are irrational numbers that cannot be roots of any algebraic equation, e.g., $e$ and $\pi$. They cannot be derived through finite algebraic operations.

2.3 Proof of Real Number Completeness

The real number system is complete: any infinite sequence on the real line has a limit. This property is crucial in mathematical analysis, especially for limits, continuity, and convergence.

Methods of Proof

Supremum Principle

Definition of Supremum:

For a set $S$ of real numbers, if a real number $M$ is an upper bound of $S$ and any number less than $M$ is not an upper bound, then $M$ is the supremum of $S$.

Example:

For $S = \{ x \in \mathbb{R} \mid 0 \lt x \lt 1 \}$, the supremum is 1.

Completeness of Real Numbers:

• Monotone Convergence Theorem: A bounded monotonic sequence converges.
• Existence of Supremum: Every non-empty bounded set of real numbers has a supremum.

Monotone Convergence Theorem

A bounded monotonic (increasing or decreasing) sequence of real numbers has a limit.

Nested Interval Theorem

For a sequence of closed intervals $[a_n, b_n]$ where each interval is contained within the previous one, there exists a real number $x$ common to all intervals.

Finite Covering Theorem (Heine-Borel Theorem)

Any open cover of a closed bounded interval has a finite subcover.

Bolzano-Weierstrass Theorem

Every bounded infinite subset of real numbers has at least one accumulation point.

Cauchy Convergence Criterion

A sequence of real numbers converges if and only if it is a Cauchy sequence.

2.4 Algebraic Structure of Real Numbers

The real number system adheres to algebraic properties:

1. Addition: Commutative, associative, identity element (0), and inverses.
2. Multiplication: Commutative, associative, identity element (1), and inverses (for non-zero elements).
3. Distributive Property: Multiplication distributes over addition.
4. Order Relation: Real numbers are ordered, with properties like transitivity and antisymmetry.

These properties make real numbers an ordered field, providing robust tools for mathematical analysis.

3. Limits and Continuity

Limits and continuity are central to mathematical analysis.

3.1 Limits of Real Numbers

Definition:

A sequence $\{a_n\}$ converges to $L$ if, for any $\varepsilon \gt 0$, there exists $N$ such that for all $n \gt N$, $|a_n - L| \lt \varepsilon$.

Example:

$\lim_{n \to \infty} \frac{1}{n} = 0$.

Properties:

• Uniqueness of limits.
• Bounded monotonic sequences converge.

3.2 Continuity of Real Numbers

Definition:

A function $f(x)$ is continuous at $x = a$ if:

1. $f(a)$ exists.
2. $\lim_{x \to a} f(x)$ exists.
3. $\lim_{x \to a} f(x) = f(a)$.

Properties:

• Sums, products, and compositions of continuous functions are continuous.

3.3 Key Theorems

Intermediate Value Theorem:

If $f$ is continuous on $[a, b]$ and $f(a) \neq f(b)$, then for any $c$ between $f(a)$ and $f(b)$, there exists $x_0 \in (a, b)$ such that $f(x_0) = c$.

Extreme Value Theorem:

A continuous function on a closed interval attains its maximum and minimum.

4. Comparative Analysis

4.1 Mathematical Analysis:

• Covers basic real analysis: construction of real numbers, continuity, limits, derivatives, and integrals.
• Emphasizes mathematical logic and set theory.
  

4.2 Real Analysis and Functional Analysis:

• Discusses advanced constructions like Dedekind cuts.
• Includes measure theory and functional analysis basics.
  

4.3 Real and Complex Analysis:

• Comprehensive coverage of real and complex analysis.
• Highlights measure theory and holomorphic functions.
  

4.4 Real Analysis:

• Focuses on the real number system’s construction and completeness.
• Explores metric spaces for deeper understanding.
  

7. Personal Reflection

Studying the construction of real numbers has expanded my mathematical perspective. Understanding the transition from rational to irrational numbers and proofs like the supremum principle has deepened my appreciation for the completeness of real numbers.

The concepts of limits and continuity reveal a profound and elegant aspect of mathematics. Limits provide tools for analyzing trends, while continuity ensures smooth transitions in functions, forming the backbone of mathematical analysis.

Published on 2023-12-27 at 孤筝の温暖小家, last modified on 2023-12-27

All articles on this blog are licensed under the BY-NC-SA license agreement unless otherwise stated. Please indicate the source when reprinting!

Author: 孤筝(lvbowen040427@163.com)

1. Residue Theorem

The Residue Theorem is a pivotal result in complex function theory, built upon the concept of residues. Its core idea states that if a function is analytic everywhere on and within a closed contour enclosing isolated singularities, the integral around the contour equals the sum of the residues at those singularities.

2. Laurent Series

A Laurent series is an expansion of a complex function into an infinite series, including both positive and negative powers. Specifically, a complex function in an annular region can be expressed as:

Here, \( c_n \) are the coefficients, and \( z_0 \) is the expansion point.

3. Higher-Order Derivative Formula

The higher-order derivative formula for complex functions resembles that of real functions but requires careful consideration in the complex plane. If a function is analytic at a point, its higher-order derivatives at that point can be obtained by term-wise differentiation of its power series.

4. Cauchy’s Integral Formula

Cauchy’s Integral Formula is a fundamental result in complex analysis, establishing a relationship between an analytic function and its integral over a contour. Specifically, if \( f(z) \) is analytic inside and on a simple closed contour \( C \), then for any point \( z_0 \) inside \( C \):

Relationships and Connections

1. Residue Theorem and Laurent Series:
   The Residue Theorem computes integrals over closed contours, while the Laurent series helps analyze the behavior of functions near singularities, facilitating residue calculation.

2. Residue Theorem and Higher-Order Derivatives:
   The Residue Theorem can derive higher-order derivatives by term-wise differentiation of the Laurent series expansion around singularities.

3. Residue Theorem and Cauchy’s Integral Formula:
   Cauchy’s Integral Formula evaluates contour integrals, while the Residue Theorem is a special case where the contour encloses finitely many isolated singularities.

1. Proof of the Residue Theorem via Laurent Series

Residue Theorem:

If \( f(z) \) is analytic everywhere inside and on a closed contour \( C \) except for isolated singularities, then:

Laurent Series Expansion:

Proof Steps:

1. Laurent Series Expansion:
   Expand \( f(z) \) around \( z_0 \):
   

   $$f(z) = \sum_{n=-\infty}^{\infty} c_n (z - z_0)^n$$

2. Integral Computation:
   Integrate term-wise over \( C \):
   

   $$\oint_C f(z) \, dz = \oint_C \left( \sum_{n=-\infty}^{\infty} c_n (z - z_0)^n \right) dz$$

3. Interchange Sum and Integral:
   By uniform convergence:
   

   $$\oint_C f(z) \, dz = \sum_{n=-\infty}^{\infty} c_n \oint_C (z - z_0)^n \, dz$$

4. Residue Extraction:
   For \( n \neq -1 \), \( \oint_C (z - z_0)^n \, dz = 0 \). Only \( n = -1 \) contributes:
   

   $$\oint_C f(z) \, dz = 2\pi i \cdot c_{-1}$$

5. Conclusion:
   The residue \( \text{Res}(f, z_0) = c_{-1} \), yielding:
   

   $$\oint_C f(z) \, dz = 2\pi i \cdot \text{Res}(f, z_0)$$

2. Deriving the Higher-Order Derivative Formula from the Residue Theorem

Residue Theorem:

Higher-Order Derivative Formula:

For \( f(z) \) analytic at \( z_0 \), the \( n \)-th derivative is:

Proof Steps:

1. Residue Theorem Setup:
   Consider \( g(z) = \frac{f(z)}{(z - z_0)^{n+1}} \). Its residue at \( z_0 \) is:
   

   $$\text{Res}(g, z_0) = \frac{f^{(n)}(z_0)}{n!}$$

2. Apply Residue Theorem to \( g(z) \):
   

   $$\oint_C \frac{f(z)}{(z - z_0)^{n+1}} \, dz = 2\pi i \cdot \text{Res}(g, z_0) = 2\pi i \cdot \frac{f^{(n)}(z_0)}{n!}$$

3. Solve for \( f^{(n)}(z_0) \):
   

   $$f^{(n)}(z_0) = \frac{n!}{2\pi i} \oint_C \frac{f(z)}{(z - z_0)^{n+1}} \, dz$$

3. Proving Cauchy’s Integral Formula via the Residue Theorem

Residue Theorem:

Cauchy’s Integral Formula:

Proof Steps:

1. Identify the Residue:
   For \( g(z) = \frac{f(z)}{z - z_0} \), the residue at \( z_0 \) is \( \text{Res}(g, z_0) = f(z_0) \).

2. Apply the Residue Theorem:
   

   $$\oint_C \frac{f(z)}{z - z_0} \, dz = 2\pi i \cdot \text{Res}(g, z_0) = 2\pi i \cdot f(z_0)$$

3. Solve for \( f(z_0) \):
   

   $$f(z_0) = \frac{1}{2\pi i} \oint_C \frac{f(z)}{z - z_0} \, dz$$

Published on 2023-12-20 at 孤筝の温暖小家, last modified on 2023-12-20

All articles on this blog are licensed under the BY-NC-SA license agreement unless otherwise stated. Please indicate the source when reprinting!

Author: 孤筝(lvbowen040427@163.com)

Complex Numbers

1. Representation of complex numbers: $$z = r\cdot e^{i\theta} = r(\cos\theta +i\cdot \sin\theta)$$
2. Elementary functions
   1. Exponential function: $e^z = e^x(\cos y + i \sin y)$
      1. $e^z$ is merely shorthand for $\exp z$ and does not imply exponentiation.
      2. $|e^z| = e^x$, $\text{Arg}(e^z) = y + 2k\pi$
   2. Logarithmic function: $\text{Ln}\,z = \ln|r| + i\,\text{Arg}\,z$
      1. The function is analytic everywhere except at the origin and the negative real axis, and $(\text{Ln}\,z)' = \frac{1}{z}$.
   3. Trigonometric functions
      1. $\cos z = \frac{e^{iz} + e^{-iz}}{2}$, $\sin z = \frac{e^{iz} - e^{-iz}}{2i}$
      2. $\text{ch}\,z = \frac{e^z + e^{-z}}{2}$, $\text{sh}\,z = \frac{e^z - e^{-z}}{2}$

Analytic Functions

1. Definition of differentiability:
   $$\lim_{\Delta z \to 0} \frac{f(z_0 + \Delta z) - f(z_0)}{\Delta z} \text{ exists, then } f(z) \text{ is differentiable at } z_0.$$
2. Definition of analyticity:
   $$f(z) \text{ is analytic at } z_0 \text{ if it is differentiable at } z_0 \text{ and in some neighborhood of } z_0.$$
   Corollary: The sum, difference, product, and quotient of analytic functions are also analytic. The composition of analytic functions is analytic.
3. Necessary and sufficient conditions for differentiability and analyticity:
   $u(x, y)$ and $v(x, y)$ are differentiable and satisfy the Cauchy-Riemann equations:
   $$\frac{\partial u}{\partial x} = \frac{\partial v}{\partial y}, \quad \frac{\partial u}{\partial y} = -\frac{\partial v}{\partial x}.$$
   If either condition fails, the function is neither differentiable nor analytic.
   Corollary:
   $$f'(z) = \frac{\partial u}{\partial x} + i \frac{\partial v}{\partial x} = \frac{1}{i} \frac{\partial u}{\partial y} + \frac{\partial v}{\partial y}.$$

Complex Integration

Key Formula

1. Cauchy-Goursat Theorem
   For a simply connected, analytic region, the integral over any closed contour is zero:
   $$\oint_C f(z) dz = 0.$$
2. Composite Contour Theorem—Extension to multiply connected regions
   Let $C$ be a simple closed curve in an analytic, multiply connected region, and $C_1, C_2, \dots, C_n$ be simple closed curves inside $C$ with the same orientation. Then:
   $$\oint_C f(z) dz = \sum_{k=1}^n \oint_{C_k} f(z) dz.$$
3. Cauchy Integral Formula—Expressing the value of a function inside a contour in terms of its boundary values
   If $f(z)$ is analytic in a region $D$ and $C$ is a positively oriented simple closed curve in $D$:
   $$2\pi i \cdot f(z_0) = \oint_C \frac{f(z)}{z - z_0} dz.$$
4. Generalized Cauchy Integral Formula—Using higher-order derivatives to compute integrals
   $$f^{(n)}(z_0) = \frac{n!}{2\pi i} \oint_C \frac{f(z)}{(z - z_0)^{n+1}} dz.$$

Series

Power Series

1. Two properties of analytic functions
   1. Analytic functions have derivatives of all orders.
   2. Every analytic function can be represented by a power series.
2. Taylor expansion:
   $$f(z) = \sum_{n=0}^\infty \frac{f^{(n)}(0)}{n!} z^n.$$
3. Methods for finding Taylor expansions [[Advanced Mathematics#Expansion of Functions into Power Series]]

Laurent Series

1. Bilateral power series
   1. The region of convergence is an annulus $R_1 \lt |z - z_0| \lt R_2$.
2. Laurent expansion:
   $$f(z) = \sum_{n=-\infty}^\infty c_n (z - z_0)^n, \quad c_n = \frac{1}{2\pi i} \oint_C \frac{f(z)}{(z - z_0)^{n+1}} dz.$$
   Corollary: When $n = -1$, $c_{-1} \cdot 2\pi i = \oint_C f(z) dz$.
3. Methods for finding Laurent expansions
   1. Compute $c_n$ directly using the definition (rarely used).
   2. Use algebraic operations or substitutions to transform the Laurent series into the form and convergence domain of a Taylor series.

Residues

Isolated Singularities

1. Definition: $f(z)$ is not analytic at $z_0$ but is analytic in some punctured neighborhood of $z_0$.
2. Classification of isolated singularities (based on negative power terms in the Laurent series)
   1. Removable singularity: No negative power terms. As $z \to z_0$, $f(z)$ approaches a finite limit.
   2. Pole: Finite number of negative power terms (if there are $m$ such terms, $z_0$ is called an $m$-th order pole). As $z \to z_0$, $f(z) \to \infty$.
   3. Essential singularity: Infinite number of negative power terms. The limit of $f(z)$ does not exist.
3. Relationship between poles and zeros
   1. Definition of zeros: For a non-zero analytic function $f(z)$, if it can be expressed as $f(z) = (z - z_0)^m \varphi(z)$, then $z_0$ is called an $m$-th order zero of $f(z)$.
      Necessary and sufficient condition: $f^{(n)}(z_0) = 0$ for $n \lt m$, and $f^{(m)}(z_0) \neq 0$.
   2. If $z_0$ is an $m$-th order zero of $f(z)$, then $z_0$ is an $m$-th order pole of $\frac{1}{f(z)}$.

Residues

1. Definition:
   $$\text{Res}[f(z), z_0] = c_{-1} = \frac{1}{2\pi i} \oint_C f(z) dz.$$
2. Rules for computing residues
   1. If $z_0$ is a simple pole of $f(z)$:
      $$\text{Res}[f(z), z_0] = \lim_{z \to z_0} (z - z_0) f(z).$$
   2. If $z_0$ is an $m$-th order pole of $f(z)$:
      $$\text{Res}[f(z), z_0] = \frac{1}{(m-1)!} \lim_{z \to z_0} \frac{d^{m-1}}{dz^{m-1}} \left( (z - z_0)^m f(z) \right).$$
   3. If $f(z) = \frac{P(z)}{Q(z)}$, where $P(z_0) \neq 0$, $Q(z_0) = 0$, and $Q'(z_0) \neq 0$:
      $$\text{Res}[f(z), z_0] = \frac{P(z_0)}{Q'(z_0)}.$$

Published on 2023-11-17 at 孤筝の温暖小家, last modified on 2023-11-17

All articles on this blog are licensed under the BY-NC-SA license agreement unless otherwise stated. Please indicate the source when reprinting!