Publications

Books/Monographs

1. S.Bhatnagar, H.L.Prasad and L.A.Prashanth, Stochastic Recursive Algorithms for Optimization: Simultaneous Perturbation Methods, Lecture Notes in Control and Information Sciences Series, Vol. 434, Springer, ISBN 978-1-4471-4284-3, Edition: 2013, 302 pages.

Book Chapters

1. S.Bhatnagar, V.S.Borkar and Prashanth L.A., Adaptive feature pursuit: Online adaptation of features in reinforcement learning, Reinforcement Learning and Approximate Dynamic Programming for Feedback Control (Ed. F. Lewis and D. Liu), IEEE Press Computational Intelligence Series (to appear), 2012.

2. S.Bhatnagar, Simultaneous perturbation and finite difference methods, Wiley Encyclopedia of Operations Research and Management Science (Ed. J. Cochran), Vol. 7, pp. 4969-4991, Wiley, Hoboken, NJ, 2011.

Journal Papers

1. S.Bhatnagar, V.S.Borkar and Prabuchandran K.J., Feature search in the Grassmanian in online reinforcement learning, IEEE Journal of Selected Topics in Signal Processing, 2013 (accepted).

2. Prabuchandran K.J., S.K.Meena and S.Bhatnagar, Q-learning based energy management policies for a single sensor node with finite buffer, IEEE Wireless Communication Letters, Vol.2, Issue 1, pp.82-85, 2013 online pdf.

3. H.L.Prasad, L.A.Prashanth, S.Bhatnagar and N.Desai, Adaptive Smoothed Functional Algorithms for Optimal Staffing Levels in Service Systems, Service Science (INFORMS), 2012 (Accepted).

4. L.A.Prashanth and S.Bhatnagar, Threshold tuning using stochastic optimization for graded signal control, IEEE Transactions on Vehicular Technology, Vol. 61, No. 9, pp.3865-3880, November 2012 online pdf.

5. H.L.Prasad and S.Bhatnagar, General-Sum Stochastic Games: Verifiability Conditions for Nash Equilibria, Automatica, Vol. 48, Issue 11, pp.2923-2930, 2012 online pdf.

6. K.R.Vemu, S.Bhatnagar and N.Hemachandra, Optimal Multi-layered Congestion Based Pricing Schemes for Enhanced QoS, Computer Networks (Elsevier), Vol.56, Issue 4, pp.1249-1262, March 2012. (DOI: 10.1016/j.comnet.2011.12.004)

7. S.Bhatnagar and Lakshmanan K., An Online Actor–Critic Algorithm with Function Approximation for Constrained Markov Decision Processes, Journal of Optimization Theory and Applications (Springer), Vol. 153, No. 3, pp.688-708, 2012. (DOI: 10.1007/s10957-012-9989-5)

8. S.Bhatnagar, V.Mishra and N.Hemachandra, Stochastic algorithms for discrete parameter simulation optimization, IEEE Transactions on Automation Science and Engineering, Vol. 9, Issue 4, pp.780-793, 2011. (DOI: 10.1109/TASE.2011.2159375)

9. Karmeshu, S.Bhatnagar and V.Mishra, An optimized SDE model for slotted Aloha, IEEE Transactions on Communications, Vol. 59, No. 6, pp.1502-1508, 2011. (DOI: 10.1109/TCOMM.2011.09.090113)

10. L.A.Prashanth and S.Bhatnagar, Reinforcement learning with function approximation for traffic signal control, IEEE Transactions on Intelligent Transportation Systems, Vol. 12, No. 2, pp.412-421, 2011. (DOI: 10.1109/TITS.2010.2091408)

11. S.Bhatnagar, The Borkar-Meyn Theorem for Asynchronous Stochastic Approximations, Systems and Control Letters, Vol. 60, pp. 472-478, 2011. (DOI: 10.1016/j.sysconle.2011.04.002)

12. S.Bhatnagar and Karmeshu, Monte-Carlo Estimation of Time-Dependent Statistical Characteristics of Random Dynamical Systems, Applied Mathematical Modelling (Elsevier), Vol.35, pp.3063-3079, 2011. (DOI: 10.1016/ j.apm.2010.12.024).

13. S.Bhatnagar, N.Hemachandra and V.Mishra, Stochastic approximation algorithms for constrained optimization via simulation, ACM Transactions on Modeling and Computer Simulation, Vol. 21, Issue 3, pp:15:1-15:22, 2011.

Preprints Submitted to journals

Our recent lab technical reports can be found at the Stochastic Systems Lab page by clicking on the Research tab.

Proceedings of International Conferences

1. Prashanth L.A., Prasad H.L., N.Desai, S.Bhatnagar, Mechanisms for Hostile Agents with Capacity Constraints, Proceedings of Twelfth International Conference on Autonomous Agents and Multiagent Systems (AAMAS2013), (to appear), 2013

2. K.Laskshmanan and S.Bhatnagar, A Novel Q-learning Algorithm with Function Approximation for Constrained Markov Decision Processes, Proceedings of the Fiftieth Annual Allerton Conference on Communication, Control and Computing, UI UC, Illinois (invited paper, to appear), 2012

3. D.Ghoshdastidar, A.Dukkipati and S.Bhatnagar, q-Gaussian based smoothed functional algorithms for stochastic optimization, Proceedings of IEEE International Symposium on Information Theory (ISIT’2012), 2012.

4. Prashanth L.A., H.L.Prasad, N.Desai, S.Bhatnagar and G.Dasgupta, Stochastic optimization for adaptive labor staffing in service systems, Proceedings of 9th International Conference on Service Oriented Computing (ICSOC) (accepted), 2011.

5. Prashanth L.A. and S.Bhatnagar, Reinforcement Learning with Average Cost for Adaptive Control of Traffic Lights at Intersections, Proceedings of the 14th International IEEE Conference on Intelligent Transportation Systems (ITSC) (accepted), Washington, DC, October 5-7, 2011.

6. K.Lakshmanan and S.Bhatnagar, Smoothed functional and Quasi-Newton algorithms for routing in multi-stage queueing network with constraints, Proceedings of ICDCIT (Distributed Computing and Internet Technology, Lecture Notes in Computer Science, Vol. 65362011, pp.175-186, DOI: 10.1007978-3-642-19056-8_12), Feb.9-12, 2011, Bhubaneswar, India.