In many cases of engineering modelling, the evolutionary phenomenon need to be modelled as a stochastic process, One of the classes of stochastic processes are those which are idealised based on the depth of memory they carry. In this lecture, we present the importance of Markov Chain (both homogeneous and non-homogeneous) modelling for describing the stochastic evolution of the response of engineering systems. We present the mathematical details of stochastic modelling starting from the fundamentals of probability theory.