Modelling, Analysis and Design of Stochastic Systems

Printer-friendly version
Modelling, Analysis and Design of Stochastic Systems
Course ID: 
Track course
Management Engineering

A system is characterized as “stochastic” when its evolution in time is arbitrary. To deal with such systems the first step is to understand how it operates and the purpose of studying it, in order to be able to build a model that is simple yet sufficiently true to the real. The second steps consists in carefully analyzing the model and compute the desired measures. To facilitate this special classes of stochastic processes are used, like discrete-time Markov chains, Poisson processes and continuous-time Markov chains. For each of these processes, the transient distributions, limiting distributions and cost evaluations are studied. The main part of the course deals with a particular class of stochastic system called queuing systems which is commonly used to model systems’ behavior in production and service providing systems. Typically, a queuing system consists of a stream of customers that arrive at a service facility, get served according to a given service discipline and then depart. We are interested in designing a queuing system that will help us answering questions like “How many customers are there in the queue on average?”, “How long does a typical customer spend in the queue?”, “How many customers are rejected or lost due to capacity limitations?”. Finally, an introduction to Markov Decision process is provided.

Module Contents (Syllabus):

Week 1. Probability Models Review, Introduction to Stochastic Process and Queuing Systems

Week 2. Introduction to stochastic modeling

Week 3. Discrete Time Markov Chains (DTMC)

Week 4. Continuous Time Markov Chains (CTMC)

Week 5. Exercises in  DTMC and CTMC

Week 6. Characteristics of queuing systems, PASTA, Little’s Law

Week 7. The Μ/Μ/1 model- Exercises

Week 8. The Μ/Μ/k model - Exercises

Week 9. The Μ/Μ/1/k model - Exercises

Week 10. The Μ/Μ/s/k and M/M/inf models - Exercises

Week 11. The Μ/Μ/1/k/k and M/M/s/k/k models - Exercises

Week 12. Introduction to Markov Decision Processes (MDP)

Week 13. Applications of M.D.P.

Module Objective:

The aim of the course is to provide the students the capability of modeling, analysis and design of systems the evolution of which is arbitrary. To this direction the course provides the appropriate background for understanding the behavior of a real world system and modeling its evolution using stochastic processes such as Markov processes. The course mainly focuses on queuing systems and their application is production and service systems


Class schedule: 
Wednesday 18.00-21:00
Class schedule: 
Assessment methods: 

Final Exams 100%

Recommended Reading:
Α) Principal Reference:
  1. Στοχαστικά Μοντέλα στην Επιχειρησιακή Έρευνα, Θεωρία και Εφαρμογές, Φακίνος, Δ., Εκδόσεις Συμμετρία, 2007 (κωδ. 45393) (in greek)
  2. Στοχαστικές Μέθοδοι στις Επιχειρησιακές Έρευνες, Βασιλείου, Π.Χ., Εκδόσεις Ζήτη, 2000 (κωδ. 11282) (in greek)
Β) Additional References:
  1. Ουρές Αναμονής, 2η έκδ./2008, Δ. Φακίνου, Εκδόσεις ΣΥΜΜΕΤΡΙΑ, (κωδ. 45392) (in greek)
  2. Στοχαστικές Ανελίξεις: Θεωρία και Εφαρμογές, 1η εκδ./2003, Τ.Ι. Δάρας, Π.Θ. Σύψας, Εκδόσεις ZHTH, (κωδ. 11281) (in greek)
  3. Modeling, Analysis, Design, and Control of Stochastic Systems, Kulkarni, V.G., Sprienger, 1999
  4. Introduction to Probability Models, G. Bolch, S. M. Ross, Academic Press, (10th ed.), 2009.
  5. Probability and Statistics with Reliability, Queuing, and Computer Science Applications (2nd ed.), Trivedi K. S., John Wiley & Sons, 2001