期刊文献+

Markov决策过程不确定策略特征模式 预览 被引量:2

Property Patterns of Markov Decision Process Nondeterministic Choice Scheduler
在线阅读 免费下载
分享 导出
摘要 马尔科夫决策过程可以建模具有不确定性特征的复杂系统,而在进行模型分析时需要采用策略对不确定性进行处理。首先,研究不同策略下时空有界可达概率问题,给出不确定性解决策略的定义及分类方法。其次,在时间无关策略下,证明基于确定性选取动作和随机选取动作的时空有界可达概率的一致性,并且论证了时间依赖策略相对于时间无关策略具有更好的时空有界可达概率。最后结合实例简要阐述了结论的正确性。 Markov decision process can model complex system with nondeterminism. Schedulers are required to resolve the nonderministic choices during model analysis. This paper introduced the time-and space-bounded teachability proba- bilities of markov decision process under different schedulers. Firstly, the formal definition and classification method of schedulers for nonderminism were proposed and then we proved that the reachability probabilities coincide for determini- stic and randomized schedulers under time-abstract. Also, it was proved that time-dependent scheduler generally induces probability bounds that exceed those of the corresponding time-abstract. At the end of paper, two cases were illustrated for describing the correctness of the conclusion.
作者 黄镇谨 陆阳 杨娟 方欢 HUANG Zhen-jin,LU Yang, YANG Juan, FANG Huan1 (School of Computer & Information, Heifei Univerisity of Technology, Hefei 230009 ,China)2 (Department of Computer Engineering, Guangxi University of Technology, Liuzhou 545006, China) 3 (The Anhui Provincial Key Laboratory of Mine IoT and Mine Safety Supervisory Control, Hefei 230088, China)
出处 《计算机科学》 CSCD 北大核心 2013年第4期263-266,共4页 Computer Science
基金 本文受国家自然科学基金资助项目(60873195,61070220),高等学校博士点基金资助项目(20090111110002)资助.
关键词 马尔科夫决策过程 不确定性策略 时空有界可达概率 Markov decision process Nonderministic scheduler Time-and space-bounded reachability probability
作者简介 黄镇谨(1975-),男,博士生,主要研究方向为计算机控制、形式化技术,E-mail:schzj@163.com; 陆阳(1967-),教授,博士生导师,主要研究方向为人工智能、计算机控制、传感器网络 杨娟(1983-),女,博士生,主要研究方向为人工智能、神经网络; 方欢(1982-),女,博士生,主要研究方向为Petri网、形式化方法。
  • 相关文献

参考文献14

  • 1钮俊,曾国荪,吕新荣,徐畅.随机模型检测连续时间Markov过程[J].计算机科学,2011,38(9):112-115. 被引量:2
  • 2Puterman M L. Markov Decision Processes: Discrete Stochastic Dynamic Programming[M]. NewYork: Wiley, 1994. 被引量:1
  • 3Markus B. Optimal Schedulers for Time-Bounded Reachability in CTMDPs[D]. Homburg: Saarland University, 2009. 被引量:1
  • 4Baler C, Cloth L, Haverkort B, et al. Model checking Markov chains with aciton and state labels[J]. IEEE Transactions on Software Engineering, 2007,33(4) : 209-224. 被引量:1
  • 5Cloth L, Model checking algorithms for Markov reward models [D]. Enschede, University of Twente, 2006. 被引量:1
  • 6Hermanns H. Interactive Markov Chains [M]. Heidellgg: Sp- ringer, 2002. 被引量:1
  • 7Baier C, Hermanns H, Katoen J-P, et al. Efficient computation of time-bounded reachability probabilities in uniform continuous- time Markov decision processes[J]. Theoretical Computer Scien- ce, 2005,345 (1) : 2-26. 被引量:1
  • 8Brazdil T, Forejt V, Krcal J, et al. Continuous-time stochastic games with time bounded reachability [A]//Conference on Foundations of Software Technology and Theoretical Computer Science, 2009[C]. 2009 : 61-72. 被引量:1
  • 9Lijun Z, Neuh/iul3er M R. Model checking interactive Markov chains[A]//Proceedings of TACAS, 2010[C]. 2010:53-68. 被引量:1
  • 10Neuhuer M-R, Zhang L. Time-bounded reachability probabili- ties in continuous-time Markov decision processes[A]//7th In- ternational Conference on Quantitative Evaluation of Systems, 2010[C]. 2010:209-218. 被引量:1

二级参考文献37

  • 1林闯,雷蕾.下一代互联网体系结构研究[J].计算机学报,2007,30(5):693-711. 被引量:54
  • 2Baier C,Haverkort Boudewijn R,Hermanns H,Katoen J-P.Reachability in continuous-time Markov reward decision processes//Proceedings of the Occasion of Wolfgang Thomas's 60th Birthday.Aachen Germany,2007:53-72. 被引量:1
  • 3Katoen J-P.Perspectives in probabilistic verification//Proceedings of the 2nd IFIP/IEEE International Symposium on Theoretical Aspects of Software Engineering.Nanjing,China,2008:3-10. 被引量:1
  • 4Baier C,Groβer M,Ciesinski F.Partial order reduction for probabilistic systems//Proceedings of the QEST'04.Enschede Netherlands,2004:230-239. 被引量:1
  • 5D'Argenio P R,Niebert P.Partial order reduction on concurrent probabilistic programs//Proceedings of the QEST'04.Enschede Netherlands,2004:240-249. 被引量:1
  • 6Groβer M,Norman G,Baier C,Ciesinski F,Kwiatkowska M,Parker D.On reduction criteria for probabilistic reward models//Proceedings of the FSTTCS 2006.Kolkata India,2006:309-320. 被引量:1
  • 7Cloth L,Katoen J-P,Khattri M,Pulungan R.Model checking Markov reward models with impulse rewards//Proceedings of the International Conference on Dependable Systems and Networks (DSN'05).Yokohama,Japan,2005:722-731. 被引量:1
  • 8Hermanns H.Interactive Markov chains[Ph.D.dissertation].Friedrich-Alexander University,Erlangen-Nurnberg,1998. 被引量:1
  • 9Johr S.Model checking compositional Markov systems[Ph.D.dissertation].Universitt des Saarlandes,Germany,2007. 被引量:1
  • 10Baier C,Haverkort B,Hermanns H,Katoen J-P.On the logical characterisation of performability properties//Proceedings of the ICALP 2000:Automata,Languages and Programming,Geneva Switzerland,2000:780-792. 被引量:1

共引文献5

同被引文献30

  • 1Baier C. Principles of Model Checking[M]. Massachusetts:The MIT Press,2008. 被引量:1
  • 2Baier C, Haverkort B, Hermanns H, et al. Model-checking Algorithms for Continuous Time Markov Chains [J ].IEEE Transaction on Software Engineering, 2003,29 (6) :524-541. 被引量:1
  • 3Puterman M L. Markov Decision Processes:Discrete Stochastic Dynamic Programming [ M ]. New Jeasey:John Wiley&Sons, 1994. 被引量:1
  • 4Baier C, Hermanns H, Katoen JP, et al. Efficient Computation of Time-bounded Reachability Probabilities in Unitbrm Continuous- time Markov Decision Processes [J ].Theoretical Computer Science. 2005,345 ( 1 ) :2-26. 被引量:1
  • 5Johr S. Model Checking Compositional Markov Systems[D]. Saarbrucken Germany:Saarland University,2007. 被引量:1
  • 6Baier C, Haverkort Boudewijn R, Hermanns H et al. Reachability in Continuous-time Markov reward Decision Processes [C]// Proceedings of the Occasion of Wolfgang T homas's 60th Birthday. Aachen Germany:Amsterdam University Press, 2007:53-72. 被引量:1
  • 7Ash R B , Doleans-Dade C A. Probability & Measure Theory[M ]2nd edition: San Diego:Academic Press, 2000:215-275. 被引量:1
  • 8Hermanns H, Johr S. "May We Reach It? Or must We in What Time with What Probability" [J] Measuring, Modelling and Evaluation of Computer and Communication Systems, 2008, 15 (3): 125 - 140. 被引量:1
  • 9Zhang Lijun,Neuhul3er Martin. Model Checking Interactive Markov Chains [J]. Tools and Algorithms for the Construction and Analysis of Systems, 2010, 60(15):55-68. 被引量:1
  • 10Katoen J-P, Maneesh K, Ivan S Zapreev. A Markov Reward Model Checker [C ]//Quantitative Evaluation of Systems.Los Alamitos : IEEE Computer Society Press , 2005 : 243-244. 被引量:1

引证文献2

二级引证文献1

投稿分析

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部 意见反馈