Discussion about Lab1

[harshraj22]

Confusion Regarding the solution

This actually calculates Pi_8, ie. if Random Variable is the state we are in (sample_space = {0,...,8}), and transition probabilities are defined by the rolling of die, to go from one outcome to other. This code seems to find the distribution of 8th Random Variable in the sequence.

rl_lab/lab1/ques4/numerical/runner.py

Line 10 in aa30495

P_8 = np.linalg.matrix_power(transition_probs, 8)

While, I guess we need to find sum(Pi_j[8] for j in range(8, inf)). ie. for all random variables greater or equal to 8, find the probability of it taking value 8.

[harshraj22]

Should we rather find something like
(Mu_0 * power(P, inf))[8]

the idea is to keep simulating till they all reach a stable state, which happens to be [0. 0. 0.75 0. 0.125 0. 0. 0. 0.125], with inf = 800

[harshraj22]

Markov Chain was wrong. Updated the solution in final zip.

[harshraj22]

Comments in the review meet:

Ques 2:

• Use lesser samples to generate Normal Distribution

Ques 3:

• Solve without area of histograms
• for part b, sample from a different distribution. Not uniform

[harshraj22]

For ques 2, we can use Box Muller Transform
For ques 3, as per discussion, consider abs(Normal) random variable, ignore the values more than pi, and rescale the rest such that they sum up to 1. Then sample from those.