なぜ2つのランダム変数の合計が畳み込みなのですか?


33

長い間、2つの確率変数の「合計」が畳み込みである理由を理解できませんでしたがと混合密度関数の合計はf(x)g(x)pf(x)+(1p)g(x); 畳み込みではなく算術和。「2つのランダム変数の合計」というフレーズは、googleで146,000回表示され、次のように楕円形です。RVが単一の値を生成すると考える場合、その単一の値を別のRVの単一の値に追加できます。これは、少なくとも直接ではなく、畳み込みとは関係ありません。それは2つの数値の合計です。ただし、統計のRV結果は値の集合であるため、より正確なフレーズは「2つのRVからの関連する個々の値のペアの調整された合計のセットは離散畳み込み」のようになり、...それらのRVに対応する密度関数の畳み込み。さらに単純な言語: 2 RVn-サンプルは、事実上、ベクトルの合計として加算される2つのn次元ベクトルです。

2つのランダム変数の合計が畳み込みと合計である方法の詳細を示してください。


6
私はそれが抽象的な代数的な意味で「合計」であるとは本当に信じていません。「変数の合計」を作成するとき、自然数または実数を追加するときに知っている典型的な算術演算を参照します。つまり、他の変数を「追加」して新しい変数を作成します。「変数の合計」の概念も統計の領域外に存在し、畳み込みと確率に関する表現から独立しています。だから、確かに「変数の和がある、畳み込み」間違っています。しかし、これを暗示している人はいません。その文の「is」という単語を変更する必要があります。
セクストゥス

5
これは、用語の畳み込みであるため、を「2つの関数fとgの積」(または「積」の抽象的な代数的概念としてのみ解釈と呼ぶべきではないという主張に似ていますそれらの関数のフーリエ変換の。f(x)g(x)
セクストゥスエンピリカス

16
「通知」は誤解を招くものです。ランダム変数と合計は、まったく同じ意味で意味します。「合計」は学童によって理解されます。各について、値は数値加算することで求められますおよびそれについて抽象的なことは何もありません。これらのRVにはディストリビューションがあります。分布を表す多くの方法があります。分布関数ある畳み込みののDFのおよび ; 特性関数ある製品XYω(X+Y)(ω)X(ω)Y(ω).X+YXYX+YCFの キュムラント生成関数あるそれらのCGFの。等々。X+Y
whuber

3
ランダム変数も計算にも分布が見当たりません。
whuber

8
で、私のポストの言語でstats.stackexchange.com/a/54894/919、確率変数の組二つの数字、1つの指定書かれているのそれぞれのチケットのボックスで構成さと他の これらのランダム変数の合計は、各チケットで見つかった2つの数値を加算することによって取得されます。文字通り計算は、3年生の教室に割り当てることができるタスクです。(私はこの点を強調して、操作の基本的なシンプルさと、誰もが「和」の意味を理解していることとの関連性を示すことを強調します。)のX Y (X,Y)XY.
whuber

回答:


14

ランダム変数の分布に関連付けられた畳み込み計算は、すべて総確率の法則の数学的表現です。


で、私のポストの言語で「確率変数」とはどういう意味ですか?

ランダム変数のペア(X,Y)は、それぞれがと指定された2つの数字が書き込まれたチケットのボックスで構成されます。これらのランダム変数の合計は、各チケットにある2つの数値を加算することにより取得されます。XY

そのような箱とそのチケットの写真を、ランダム変数の和の概念の明確化に投稿しました。

enter image description here

この計算は、文字通り、3年生の教室に割り当てることができるタスクです。(私はこの点を強調して、操作の基本的な単純さと、誰もが「合計」を意味することを理解するものとどれだけ強く関連しているかを示すことを強調します。)

ランダム変数の合計が数学的にどのように表現されるかは、ボックスの内容をどのように表現するかに依存します。

これらの最初の2つは、ボックスにpmf、pdf、またはmgfがない場合がありますが、常に cdf、cf、およびcgfがあるため、特別です。


コンボリューションは、確率変数の和のPMFやPDFを計算するための適切な方法である理由を確認するには、すべての3つの変数の場合を検討および、定義により、用PMF:PMFを持つでの任意の数は、合計が等しい書かれたボックス内のチケットの割合を示しますX, Y,X+YX+YzX+Yz,Pr(X+Y=z).

合計のpmfは、(互いに素なサブセットの)割合を加算する合計確率の法則に従って、書き込まれたの値に従ってチケットのセットを分解することによって検出されます より技術的には、X

ボックスの互いに素なサブセットのコレクション内で見つかったチケットの割合は、個々のサブセットの割合の合計です。

次のように適用されます。

チケットの割合、書かれたすべての可能な値の総和に等しくなければならないがチケットの割合および書かれたX+Y=zPr(X+Y=z),xX=xX+Y=z,Pr(X=x,X+Y=z).

ので、および暗示この表現は、元の変数の点で直接書き換えることができととしてX=xX+Y=zY=zx,XY

Pr(X+Y=z)=xPr(X=x,Y=zx).

それが畳み込みです。


編集

畳み込みはランダム変数の合計に関連付けられていますが、畳み込みはランダム変数自体の畳み込みではないことに注意してください!

実際、ほとんどの場合、2つのランダム変数を畳み込むことはできません。これが機能するには、それらのドメインに追加の数学的構造が必要です。この構造は、連続的なトポロジカルグループです。

詳細に入ることなく、任意の2つの関数畳み込みは抽象的に次のように見える必要があると言うだけで十分です。X,Y:GH

(XY)(g)=h,kGh+k=gX(h)Y(k).

(合計は整数である可能性があり、これが既存の変数から新しいランダム変数を生成する場合、とがいつでもが測定可能でなければなりません。ここで、トポロジーまたは測定可能性の考慮が必要になります。)XYXY

この式は2つの操作を呼び出します。 一つは、上の乗算であるが乗算値に意味を持たなければならないと 他に付加であるそれはセンスに加えなければならない追加の要素H:X(h)HY(k)H.G:G G.

ほとんどの確率アプリケーションでは、は数値(実数または複素数)のセットであり、乗算は通常のものです。 しかし、サンプル空間であるは、数学的な構造がまったくないことがよくあります。 これが、ランダム変数の畳み込みが通常定義されていない理由です。このスレッドの畳み込みに関与するオブジェクトは、ランダム変数の分布の数学的な表現です。 それらは、それらのランダム変数の共同分布が与えられると、ランダム変数の合計の分布を計算するために使用されます。HG,


参照資料

Stuart and Ord、Kendallの高度な統計理論、第1巻、第 5版、1987年、第1章、第3章、および第4章(度数分布、モーメントとキュムラント、および特性関数)。


スカラー倍算と連想代数的性質が関連していること 任意の実際の(または複合体)の数のために。良い特性の1つは、2つの密度関数の畳み込みが密度関数であることです。一方、畳み込み密度関数に制限されることはありません。畳み込みは一般に確率処理ではありません。たとえば、降雨後の湖での水の流出の処理、投与後の薬物濃度モデルなど
a(fg)=(af)g
a
Carl

@Carlそのコメントは、ランダム変数の合計について尋ねる元の質問とどのように調和していますか?せいぜい接線方向です。
whuber

過度に一般化しないようお願いします。「RVの畳み込み」は言うまでもなく「畳み込みは」で文を開始することは楕円です。ここでの私の全体の問題は、楕円記法にありました。2つの空間ベクトルのベクトル加算は、それらのベクトルが正規化されているかどうかに関係なく畳み込みです。それらが正規化されている場合、それらは確率である必要はありません、それは全体の真実であり、単なる一部ではありません。n
カール

ありがとう:あなたの質問に答えていることを強調するために、最初の文を明確にします。
whuber

RVの畳み込みには新しい追加が当てはまります。これは技術的に私が尋ねたものです。そして、おそらく私は曖昧ですが、畳み込みは常にRVのものではなく、常に密度関数のいくつかのスケール係数にそれらの密度関数を掛けたものに減らすことができます。乗法のアイデンティティ、すなわち、1
カール・

41

表記、大文字と小文字

https://en.wikipedia.org/wiki/Notation_in_probability_and_statistics

  • ランダム変数は通常、大文字のローマ字で記述されます:XYなど。
  • ランダム変数の特定の実現は、対応する小文字で書かれています。たとえば、x1x2、…、xnは確率変数X対応するサンプルで、累積確率は正式にP(X>x)記述され、確率変数と実現を区別します。

Z=X+Y z i = x i + y iを意味しますzi=xi+yixi,yi


変数の混合-> pdfの合計

https://en.wikipedia.org/wiki/Mixture_distribution

確率(たとえばZ)が異なる確率の単一の合計によって定義される場合、確率密度関数fX1fX2合計を使用します。

たとえば、ZX 1で定義される時間の小数部sおよびX 2で定義される時間小数部1 - sである場合、 PZ = z = s PX 1 = z + 1 s PX 2 = z およびf Zz = s f XX11sX2

P(Z=z)=sP(X1=z)+(1s)P(X2=z)
fZ(z)=sfX1(z)+(1s)fX2(z)

。。。。例は、6面ダイスまたは12面ダイスのいずれかのサイコロロールの選択です。片方のサイコロの時間の50〜50%を実行するとします。それから

fmixedroll(z)=0.5f6sided(z)+0.5f12sided(z)


変数の合計-> pdfの畳み込み

https://en.wikipedia.org/wiki/Convolution_of_probability_distributions

確率(Zなど)が異なる(独立した)確率の複数の和によって定義される場合、確率密度関数fX1およびfX2畳み込みを使用します。

たとえば、Z=X1+X2 (つまり合計!)で、複数の異なるペアx1,x2合計zで、それぞれ確率fX1(x1)fX2(x2)。次に、畳み込み取得

P(Z=z)=all pairs x1+x2=zP(X1=x1)P(X2=x2)

及び

fZ(z)=x1 domain of X1fX1(x1)fX2(zx1)

または連続変数の場合

fZ(z)=x1 domain of X1fX1(x1)fX2(zx1)dx1

。。。。例では、2つのダイスロールの和であるfX2(x)=fX1(x)=1/6のためx{1,2,3,4,5,6}および

fZ(z)=x{1,2,3,4,5,6} and zx{1,2,3,4,5,6}fX1(x)fX2(zx)

私は統合し、合計することを選択したノートx1 domain of X1、私はより直感的に見つけた、が、それは必要ではなく、あなたから統合することができますあなたが定義した場合fX1(x1)=0ドメイン外。

画像例

example of 'sum of variables' resulting in 'convolution of pdfs'

してみましょうZ可能X+YPz 1を知るにはP(z12dz<Z<z+12dz)z1に至るx,yすべての実現の確率を統合する必要がありますz12dz<Z=X+Y<z+12dz

So that is the integral of f(x)g(y) in the region ±12dz along the line x+y=z.


Written by StackExchangeStrike


6
@Carl it is not jargonesque. The convolution can indeed be seen as a sum of many sums. But, this is not what 'the sum of variables' refers to. It refers to such things as when we speak of a 'a sum of two dice rolls', which has a very normal meaning and interpretation in every day life (especially when we play a board game). Would you rather like to say that we take a combination of two dice rolls when we do use the algebraic sum of two dice rolls?
Sextus Empiricus

2
The probability of rolling 7 with the (single) sum of two dice is the sum of (many) probabilities for rolling 1-6, 2-5, 3-4, 4-3, 5-2, 6-1. The term sum occurs two times and in the first case, when it refers to a single summation expression, it is what the statement 'sum of two variables' refers to, as in 'sum of two dice rolls'.
Sextus Empiricus

5
Indeed, the integral replaces the sum of probabilities. But, that relates to the second use of the term sum, not the first use of the term sum. So we can still refer to the sum of two variables (which is the first use of the term). That is because the term 'sum' is not used to refer to the convolution operation or summation operation of the probabilities, but to the summation of the variables.
Sextus Empiricus

8
at least it is not jargonesque to state 'the probability density for a sum of dice rolls is defined by the convolution of the probability densities for the individual dice rolls'. The term 'a sum of dice rolls' has a very normal interpretation in every day life when there are no statisticians around with their jargon. It is in this sense (sum of dice rolls) that you need to interpret (sum of variables). This step is neither not jargonesque. People use 'sums of variables' all the time. It is only the statistician who thinks about the probabilities for these sums and starts applying convolutions
Sextus Empiricus

2
@Carl: I think you misunderstood my statement. You were saying that it is not good to call a convolution integral a sum, implying that somebody calls the convolution integral a sum. But nobody here is saying this. What was said is that a convolution integral is the pdf of the sum of certain variables. You were changing the statement to something false, and then complained that it is false.

28

Your confusion seems to arise from conflating random variables with their distributions.

To "unlearn" this confusion, it might help to take a couple of steps back, empty your mind for a moment, forget about any fancy formalisms like probability spaces and sigma-algebras (if it helps, pretend you're back in elementary school and have never heard of any of those things!) and just think about what a random variable fundamentally represents: a number whose value we're not sure about.

For example, let's say I have a six-sided die in my hand. (I really do. In fact, I have a whole bag of them.) I haven't rolled it yet, but I'm about to, and I decide to call the number that I haven't rolled yet on that die by the name "X".

What can I say about this X, without actually rolling the die and determining its value? Well, I can tell that its value won't be 7, or 1, or 12. In fact, I can tell for sure that it's going to be a whole number between 1 and 6, inclusive, because those are the only numbers marked on the die. And because I bought this bag of dice from a reputable manufacturer, I can be pretty sure that when I do roll the die and determine what number X actually is, it's equally likely to be any of those six possible values, or as close to that as I can determine.

In other words, my X is an integer-valued random variable uniformly distributed over the set {1,2,3,4,5,6}.


OK, but surely all that is obvious, so why do I keep belaboring such trivial things that you surely know already? It's because I want to make another point, which is also trivial yet, at the same time, crucially important: I can do math with this X, even if I don't know its value yet!

For example, I can decide to add one to the number X that I'll roll on the die, and call that number by the name "Q". I won't know what number this Q will be, since I don't know what X will be until I've rolled the die, but I can still say that Q will be one greater than X, or in mathematical terms, Q=X+1.

And this Q will also be a random variable, because I don't know its value yet; I just know it will be one greater than X. And because I know what values X can take, and how likely it is to take each of those values, I can also determine those things for Q. And so can you, easily enough. You won't really need any fancy formalisms or computations to figure out that Q will be a whole number between 2 and 7, and that it's equally likely (assuming that my die is as fair and well balanced as I think it is) to take any of those values.

But there's more! I could just as well decide to, say, multiply the number X that I'll roll on the die by three, and call the result R=3X. And that's another random variable, and I'm sure you can figure out its distribution, too, without having to resort to any integrals or convolutions or abstract algebra.

And if I really wanted, I could even decide to take the still-to-be-determined number X and to fold, spindle and mutilate it divide it by two, subtract one from it and square the result. And the resulting number S=(12X1)2 is yet another random variable; this time, it will be neither integer-valued nor uniformly distributed, but you can still figure out its distribution easily enough using just elementary logic and arithmetic.


OK, so I can define new random variables by plugging my unknown die roll X into various equations. So what? Well, remember when I said that I had a whole bag of dice? Let me grab another one, and call the number that I'm going to roll on that die by the name "Y".

Those two dice I grabbed from the bag are pretty much identical — if you swapped them when I wasn't looking, I wouldn't be able to tell — so I can pretty safely assume that this Y will also have the same distribution as X. But what I really want to do is roll both dice and count the total number of pips on each of them. And that total number of pips, which is also a random variable since I don't know it yet, I will call "T".

How big will this number T be? Well, if X is the number of pips I will roll on the first die, and Y is the number of pips I will roll on the second die, then T will clearly be their sum, i.e. T=X+Y. And I can tell that, since X and Y are both between one and six, T must be at least two and at most twelve. And since X and Y are both whole numbers, T clearly must be a whole number as well.


But how likely is T to take each of its possible values between two and twelve? It's definitely not equally likely to take each of them — a bit of experimentation will reveal that it's a lot harder to roll a twelve on a pair of dice than it is to roll, say, a seven.

To figure that out, let me denote the probability that I'll roll the number a on the first die (the one whose result I decided to call X) by the expression Pr[X=a]. Similarly, I'll denote the probability that I'll roll the number b on the second die by Pr[Y=b]. Of course, if my dice are perfectly fair and balanced, then Pr[X=a]=Pr[Y=b]=16 for any a and b between one and six, but we might as well consider the more general case where the dice could actually be biased, and more likely to roll some numbers than others.

Now, since the two die rolls will be independent (I'm certainly not planning on cheating and adjusting one of them based on the other!), the probability that I'll roll a on the first die and b on the second will simply be the product of those probabilities:

Pr[X=a and Y=b]=Pr[X=a]Pr[Y=b].

(Note that the formula above only holds for independent pairs of random variables; it certainly wouldn't hold if we replaced Y above with, say, Q!)

Now, there are several possible values of X and Y that could yield the same total T; for example, T=4 could arise just as well from X=1 and Y=3 as from X=2 and Y=2, or even from X=3 and Y=1. But if I had already rolled the first die, and knew the value of X, then I could say exactly what value I'd have to roll on the second die to reach any given total number of pips.

Specifically, let's say we're interested in the probability that T=c, for some number c. Now, if I know after rolling the first die that X=a, then I could only get the total T=c by rolling Y=ca on the second die. And of course, we already know, without rolling any dice at all, that the a priori probability of rolling a on the first die and ca on the second die is

Pr[X=a and Y=ca]=Pr[X=a]Pr[Y=ca].

But of course, there are several possible ways for me to reach the same total c, depending on what I end up rolling on the first die. To get the total probability Pr[T=c] of rolling c pips on the two dice, I need to add up the probabilities of all the different ways I could roll that total. For example, the total probability that I'll roll a total of 4 pips on the two dice will be:

Pr[T=4]=Pr[X=1]Pr[Y=3]+Pr[X=2]Pr[Y=2]+Pr[X=3]Pr[Y=1]+Pr[X=4]Pr[Y=0]+

Note that I went a bit too far with that sum above: certainly Y cannot possibly be 0! But mathematically that's no problem; we just need to define the probability of impossible events like Y=0 (or Y=7 or Y=1 or Y=12) as zero. And that way, we get a generic formula for the distribution of the sum of two die rolls (or, more generally, any two independent integer-valued random variables):

T=X+YPr[T=c]=aZPr[X=a]Pr[Y=ca].

And I could perfectly well stop my exposition here, without ever mentioning the word "convolution"! But of course, if you happen to know what a discrete convolution looks like, you may recognize one in the formula above. And that's one fairly advanced way of stating the elementary result derived above: the probability mass function of the sum of two integer-valued random variable is the discrete convolution of the probability mass functions of the summands.

And of course, by replacing the sum with an integral and probability mass with probability density, we get an analogous result for continuously distributed random variables, too. And by sufficiently stretching the definition of a convolution, we can even make it apply to all random variables, regardless of their distribution — although at that point the formula becomes almost a tautology, since we'll have pretty much just defined the convolution of two arbitrary probability distributions to be the distribution of the sum of two independent random variables with those distributions.

But even so, all this stuff with convolutions and distributions and PMFs and PDFs is really just a set of tools for calculating things about random variables. The fundamental objects that we're calculating things about are the random variables themselves, which really are just numbers whose values we're not sure about.

And besides, that convolution trick only works for sums of random variables, anyway. If you wanted to know, say, the distribution of U=XY or V=XY, you'd have to figure it out using elementary methods, and the result would not be a convolution.


Addendum: If you'd like a generic formula for computing the distribution of the sum / product / exponential / whatever combination of two random variables, here's one way to write one:

A=BCPr[A=a]=b,cPr[B=b and C=c][a=bc],
where stands for an arbitrary binary operation and [a=bc] is an Iverson bracket, i.e.
[a=bc]={1if a=bc, and0otherwise.

(Generalizing this formula for non-discrete random variables is left as an exercise in mostly pointless formalism. The discrete case is quite sufficient to illustrate the essential idea, with the non-discrete case just adding a bunch of irrelevant complications.)

You can check yourself that this formula indeed works e.g. for addition and that, for the special case of adding two independent random variables, it is equivalent to the "convolution" formula given earlier.

Of course, in practice, this general formula is much less useful for computation, since it involves a sum over two unbounded variables instead of just one. But unlike the single-sum formula, it works for arbitrary functions of two random variables, even non-invertible ones, and it also explicitly shows the operation instead of disguising it as its inverse (like the "convolution" formula disguises addition as subtraction).


Ps. I just rolled the dice. It turns out that X=5 and Y=6, which implies that Q=6, R=15, S=2.25, T=11, U=30 and V=15625. Now you know. ;-)


4
This should be the accepted answer! Very intuitive and clear!
Vladislavs Dovgalecs

3
@Carl: The point I'm trying to make is that the sum of the random variables is indeed a simple sum: T=X+Y. If we wish to calculate the distribution of T, then we'll need to do something more complicated, but that's a secondary issue. The random variable is not its distribution. (Indeed, a random variable is not even fully characterized by its distribution, since the (marginal) distribution alone doesn't encode information about its possible dependencies with other variables.)
Ilmari Karonen

3
@Carl: ... In any case, if you wanted to introduce a special symbol for "addition of random variables", then for consistency you should also have special symbols for "multiplication of random variables" and "division of random variables" and "exponentiation of random variables" and "logarithm of random variables" and so on. All of those operations are perfectly well defined on random variables, viewed as numbers with an uncertain value, but in all cases calculating the distribution of the result is far more involved than just doing the corresponding calculation for constants.
Ilmari Karonen

5
@Carl: The confusion goes away when you stop confusing a random variable with its distribution. Taking the distribution of a random variable is not a linear operation in any meaningful sense, so the distribution of the sum of two random variables is (usually) not the sum of their distributions. But the same is true for any nonlinear operation. Surely you're not confused by the fact that x+yx+y, so why should you be confused by the fact that Pr[X+Y=c]Pr[X=c]+Pr[Y=c]?
Ilmari Karonen

3
@Carl: Wait, what? I roll two dice, write down the results X and Y, and then calculate Z=X/Y. How is that not ordinary division? (And yes, it's still ordinary division even if I do it before I roll the dice. In that case, the values of X and Y just aren't fixed yet, and therefore neither is the value of Z.)
Ilmari Karonen

7

Actually I don't think this is quite right, unless I'm misunderstanding you.

If X and Y are independent random variables, then the sum/convolution relationship you're referring to is as follows:

p(X+Y)=p(X)p(Y)
That is, the probability density function (pdf) of the sum is equal to the convolution (denoted by the operator) of the individual pdf's of X and Y.

To see why this is, consider that for a fixed value of X=x, the sum S=X+Y follows the pdf of Y, shifted by an amount x. So if you consider all possible values of X, the distribution of S is given by replacing each point in p(X) by a copy of p(Y) centered on that point (or vice versa), and then summing over all these copies, which is exactly what a convolution is.

Formally, we can write this as:

p(S)=pY(Sx)pX(x)dx
or, equivalently:
p(S)=pX(Sy)pY(y)dy

Edit: To hopefully clear up some confusion, let me summarize some of the things I said in comments. The sum of two random variables X and Y does not refer to the sum of their distributions. It refers to the result of summing their realizations. To repeat the example I gave in the comments, suppose X and Y are the numbers thrown with a roll of two dice (X being the number thrown with one die, and Y the number thrown with the other). Then let's define S=X+Y as the total number thrown with the two dice together. For example, for a given dice roll, we might throw a 3 and a 5, and so the sum would be 8. The question now is: what does the distribution of this sum look like, and how does it relate to the individual distributions of X and Y? In this specific example, the number thrown with each die follows a (discrete) uniform distribution between [1, 6]. The sum follows a triangular distribution between [1, 12], with a peak at 7. As it turns out, this triangular distribution can be obtained by convolving the uniform distributions of X and Y, and this property actually holds for all sums of (independent) random variables.


Summing many sums is more combining than a single sum worth notating with a '+' sign. My preference would be to say that random variables combine by convolution.
Carl

6
A convolution could be called a sum of many sums, sure. But what you have to understand is that the convolution applies strictly to the PDFs of the variables that are summed. The variables themselves are not convolved. They are just added one to the other, and there is no way to construe that addition as a convolution operation (so the basic premise of your question, as it is now stated, is incorrect).
Ruben van Bergen

4
You are misunderstanding that reference. It states: The probability distribution of the sum of two or more independent random variables is the convolution of their individual distributions. It does not say that a sum of two random variables is the same as convolving those variables. It says that the distribution of the sum is the convolution of the distribution of the individual variables. A random variable and its distribution are two different things.
Ruben van Bergen

Sure, you can convolve random variables. But the sum/convolution property that is widely known and discussed in that article (and in my answer above) does not deal with convolutions of random variables. It is specifically concerned with sums of random variables, and the properties of the distribution of that sum.
Ruben van Bergen

1
("Sure, you can convolve random variables". Can you? My understanding was that because to get the distribution function of the sum of random variables you convolve the mass/density functions of each, many people talk (loosely) of convolving distributions, & some talk (wrongly) of convolving random variables. Sorry to digress, but I'm curious.)
Scortchi - Reinstate Monica

6

Start by considering the set of all possible distinct outcomes of a process or experiment. Let X be a rule (as yet unspecified) for assigning a number to any given outcome ω; let Y be too. Then S=X+Y states a new rule S for assigning a number to any given outcome: add the number you get from following rule X to the number you get from following rule Y.

We can stop there. Why shouldn't S=X+Y be called a sum?

If we go on to define a probability space, the mass (or density) function of the random variable (for that's what our rules are now) S=X+Y can be got by convolving the mass (or density) function of X with that of Y (when they're independent). Here "convolving" has its usual mathematical sense. But people often talk of convolving distributions, which is harmless; or sometimes even of convolving random variables, which apparently isn't—if it suggests reading "X+Y" as "X convoluted with Y", & therefore that the "+" in the former represents a complex operation somehow analogous to, or extending the idea of, addition rather than addition plain & simple. I hope it's clear from the exposition above, stopping where I said we could, that X+Y already makes perfect sense before probability is even brought into the picture.

In mathematical terms, random variables are functions whose co-domain is the set of real numbers & whose domain is the set of all outcomes. So the "+" in "X+Y" (or "X(ω)+Y(ω)", to show their arguments explicitly) bears exactly the same meaning as the "+" in "sin(θ)+cos(θ)". It's fine to think about how you'd sum vectors of realized values, if it aids intuition; but that oughtn't to engender confusion about the notation used for sums of random variables themselves.


[This answer merely tries to draw together succintly points made by @MartijnWeterings, @IlmariKaronen, @RubenvanBergen, & @whuber in their answers & comments. I thought it might help to come from the direction of explaining what a random variable is rather than what a convolution is. Thank you all!]


(+1) For effort. Answer too deep for me fathom. However, it did lead me to one. Please read that and let me know your thoughts.
Carl

It is the elliptic notation that confused me: Si=Xi+Yi for all i=1,2,3,...,n1,n, in other words, vector addition. If someone had said, "vector addition" rather than "addition", I would not have been scratching my head wondering what was meant, but not said.
Carl

Well, if you put realizations of X & Y into vectors, & wanted to calculate the vector of realizations of S, then you'd use vector addition. But that seems rather tangential. After all, would you feel the need to explain 'sin(θ)+cos(ϕ)' using vectors, or say that the '+' in that expression signifies vector addition?
Scortchi - Reinstate Monica

To do what? The context was discrete data, e.g., RV's, not continuous functions, e.g., PDF's or sin(θ), and sin(θ)+cos(ϕ) is an ordinary sum.
Carl

1
@Carl: (1) If a biologist models the no. eggs laid in a duck's nest as a Poisson r.v., they're not really countenancing the possibility of an infinity of eggs. If you've got a question about the role of infinite sets in Mathematics, ask it on Mathematics or Philosophy SE. (2) Though quite standard, the nomenclature can indeed mislead; hence my answer.
Scortchi - Reinstate Monica

3

In response to your "Notice", um, ... no.

Let X, Y, and Z be random variables and let Z=X+Y. Then, once you choose Z and X, you force Y=ZX. You make these two choices, in this order, when you write

P(Z=z)=P(X=x)P(Y=zx)dx.
But that's a convolution.

Notice gone. (+1) to you for caring.
Carl

2

The reason is the same that products of power functions are related to convolutions. The convolution always appears naturally, if you combine to objects which have a range (e.g. the powers of two power functions or the range of the PDFs) and where the new range appears as the sum of the original ranges.

It is easiest to see for medium values. For x+y to have medium value, either both have to have medium values, or if one has a high value, the other has to have a low value and vice versa. This matches with the form of the convolution, which has one index going from high values to low values while the other increases.

If you look at the formula for the convolution (for discrete values, just because I find it easier to see there)

(fg)(n)=kf(k)g(nk)

then you see that the sum of the parameters to the functions(nk and k) always sums exactly to n. Thus what the convolution is actually doing, it is summing all possible combinations, which have the same value.

For power functions we get

(a0+a1x1+a2x2++anxn)(b0+b1x1+b2x2++bmxm)=i=0m+nkakbikxi

which has the same pattern of combining either high exponents from the left with low exponents from the right or vice versa, to always get the same sum.

Once you see, what the convolution is actually doing here, i.e. which terms are being combined and why it must, therefore, appear in many places, the reason for convolving random variables should become quite obvious.


2

Let us prove the supposition for the continuous case, and then explain and illustrate it using histograms built up from random numbers, and the sums formed by adding ordered pairs of numbers such that the discrete convolution, and both random variables are all of length n.

From Grinstead CM, Snell JL. Introduction to probability: American Mathematical Soc.; 2012. Ch. 7, Exercise 1:

Let X and Y be independent real-valued random variables with density functions fX(x) and fY(y), respectively. Show that the density function of the sum X+Y is the convolution of the functions fX(x) and fY(y).

Let Z be the joint random variable (X,Y). Then the joint density function of Z is fX(x)fY(y), since X and Y are independent. Now compute the probability that X+Yz, by integrating the joint density function over the appropriate region in the plane. This gives the cumulative distribution function of Z.

FZ(z)=P(X+Yz)=(x,y):x+yzfX(x)fY(y)dydx
=fX(x)[yzxfY(y)dy]dx=fX(x)[FY(zx)]dx.

Now differentiate this function with respect to z to obtain the density function of z.

fZ(z)=dFZ(z)dz=fX(x)fY(zx)dx.

To appreciate what this means in practice, this was next illustrated with an example. The realization of a random number element (statistics: outcome, computer science: instance) from a distribution can be viewed as taking the inverse cumulative density function of a probability density function of a random probability. (A random probability is, computationally, a single element from a uniform distribution on the [0,1] interval.) This gives us a single value on the x-axis. Next, we generate another x-axis second random element from the inverse CDF of another, possibly different, PDF of a second, different random probability. We then have two random elements. When added, the two x-values so generated become a third element, and, notice what has happened. The two elements now become a single element of magnitude x1+x2, i.e., information has been lost. This is the context in which the "addition" is taking place; it is the addition of x-values. When multiple repetitions of this type of addition take place the resulting density of realizations (outcome density) of the sums tends toward the PDF of the convolution of the individual densities. The overall information loss results in smoothing (or density dispersion) of the convolution (or sums) compared to the constituting PDF's (or summands). Another effect is location shifting of the convolution (or sums). Note that realizations (outcomes, instances) of multiple elements afford only sparse elements populating (exemplifying) a continuous sample space.

For example, 1000 random values were created using a gamma distribution with a shape of 10/9, and a scale of 2. These were added pairwise to 1000 random values from a normal distribution with a mean of 4 and a standard deviation of 1/4. Density scaled histograms of each of the three groups of values were co-plotted (left panel below) and contrasted (right panel below) with the density functions used to generate the random data, as well as the convolution of those density functions. enter image description here

As seen in the figure, the addition of summands explanation appears to be plausible as the kernel smoothed distributions of data (red) in the left hand panel are similar to the continuous density functions and their convolution in the right hand panel.


@whuber Finally, I think I understand. The sum is of random events. Take a look at my explanation and tell me if it is clear now, please.
Carl

3
It helps to be careful with the language. Events are sets. Rarely are they even sets of numbers (that's why their elements are termed "outcomes"). Events don't add--the values of random variables do. The issue about "impressively complicated" is just a distraction. Indeed, if you want to get to the heart of the matter, make sure one of the summands in your example is a zero-mean random variable, because the mean effects an overall shift in the location. You want to understand intuitively what convolution does otherwise than shift the location.
whuber

@whuber Thanks-useful. Only in statistics is an outcome a single element of a sample space. For the rest of us an outcome is the result of an event. Smoothing AND shifting. What I show is the least confusing example of many as it reduces collision of the superimposed plots.
Carl

1
I see now how you are thinking of mixture models. You are constructing what are sometimes known as "multisets." (Usually a constructor other than brackets {,} is used in order to clarify the notation.) The idea appears to be that of an empirical distribution function: the empirical distribution of a multiset A and the empirical distribution of a multiset B give rise to the empirical distribution of their multiset union, which is the mixture of the two distributions with relative weights |A| and |B|.
whuber

1
I think I detect a potential source of confusion in these ongoing edits. Because it would take too long to explain in a comment, I have appended an edit to my answer in the hope it might help a little. Indeed, the original first line of my answer was misleading on that account, so I have fixed it, too, with apologies.
whuber

1

This question may be old, but I'd like to provide yet another perspective. It builds on a formula for a change in variable in a joint probability density. It can be found in Lecture Notes: Probability and Random Processes at KTH, 2017 Ed. (Koski, T., 2017, pp 67), which itself refers to a detailed proof in Analysens Grunder, del 2 (Neymark, M., 1970, pp 148-168):


Let a random vector X=(X1,X2,...,Xm) have the joint p.d.f. fX(x1,x2,...,xm). Define a new random vector Y=(Y1,Y2,...,Ym) by

Yi=gi(X1,X2,...,Xm),i=1,2,...,m

where gi is continuously differntiable and (g1,g2,...,gm) is invertible with the inverse

Xi=hi(Y1,Y2,...,Ym),i=1,2,...,m

Then the joint p.d.f. of Y (in the domain of invertibility) is

fY(y1,y2,...,ym)=fX(h1(x1,x2,...,xm),h2(x1,x2,...,xm),...,hm(x1,x2,...,xm))|J|

where J is the Jacobian determinant

J=|x1y1x1y2...x1ymx2y1x2y2...x2ymxmy1xmy2...xmym|


Now, let's apply this formula to obtain the joint p.d.f. of a sum of i.r.vs X1+X2:

Define the random vector X=(X1,X2) with unknown joint p.d.f. fX(x1,x2). Next, define a random vector Y=(Y1,Y2) by

Y1=g1(X1,X2)=X1+X2Y2=g2(X1,X2)=X2.

The inverse map is then

X1=h1(Y1,Y2)=Y1Y2X2=h2(Y1,Y2)=Y2.

Thus, because of this and our assumption that X1 and X2 are independent, the joint p.d.f. of Y is

fY(y1,y2)=fX(h1(y1,y2),h2(y1,y2))|J|=fX(y1y2,y2)|J|=fX1(y1y2)fX2(y2)|J|

where the Jacobian J is

J=|x1y1x1y2x2y1x2y2|=|1101|=1

To find the p.d.f. of Y1=X1+X2, we marginalize

fY1=fY(y1,y2)dy2=fX(h1(y1,y2),h2(y1,y2))|J|dy2=fX1(y1y2)fX2(y2)dy2

which is where we find your convolution :D


0

General expressions for the sums of n continuous random variables are found here:

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0216422

"Multi-stage models for the failure of complex systems, cascading disasters, and the onset of disease"

For positive random variables, the sum can be simply written in terms of a product of Laplace transforms and the inverse of their product. The method is adapted from a calculation that appeared in E.T. Jaynes "Probability Theory" textbook.


Welcome to our site. You might find the thread at stats.stackexchange.com/questions/72479, as well as the Moschopolous paper it references, to be of interest.
whuber
弊社のサイトを使用することにより、あなたは弊社のクッキーポリシーおよびプライバシーポリシーを読み、理解したものとみなされます。
Licensed under cc by-sa 3.0 with attribution required.