استفاده از الگوریتم یادگیری عمیق کیو جهت طراحی عاملی خودمختار برای معامله در بازار رمزارزها با تمرکز بر رفتار معامله‌گران | ||
| پژوهش های نظری و کاربردی هوش ماشینی | ||
| مقاله 4، دوره 2، شماره 2، اسفند 1403، صفحه 49-65 اصل مقاله (923.19 K) | ||
| نوع مقاله: مقاله پژوهشی | ||
| شناسه دیجیتال (DOI): 10.22034/abmir.2025.22203.1062 | ||
| نویسندگان | ||
| سید مهرداد اسلامی1؛ دکتر مهدی آقاصرام* 2؛ محمدعلی زارع چاهوکی2 | ||
| 1دانشجوی دکتری، دانشگاه یزد، دانشکده مهندسی کامپیوتر، یزد، ایران | ||
| 2دانشیار، دانشگاه یزد، دانشکده مهندسی کامپیوتر، یزد، ایران | ||
| چکیده | ||
| بازار رمزارزها، محیطی پیچیده، غیرقطعی و همراه با نوسان‌های زیادی است. ایجاد استراتژی معاملاتی در این بازار بسیار چالش‌برانگیز است. در این مقاله تأثیر رفتار معامله‌گران (معاملاتی که انجام می‌دهند) بر تغییر شرایط بازار بررسی‌شده است. عوامل بسیاری در تغییر شرایط بازار تأثیر دارند، اما درنهایت این تأثیرات، از طریق رفتار معامله‌گران به فعلیت می‌رسد. در این مقاله عاملی خودمختار جهت انجام معامله در بازار رمزارز طراحی‌شده است. عاملی که تنها با بررسی معاملات انجام‌شده تصمیم می‌گیرد. طراحی عامل مبتنی بر الگوریتم DDQN از یادگیری تقویتی است. برای آموزش عامل تمام معامله‌های انجام‌شده در صرافی رمزارز HitBTC در طول نزدیک به ۳ ماه برای ۳ جفت رمزارز، گردآوری‌شده است. نتایج پیاده‌سازی نشان می‌دهد مدل همگرا شده و در شرایط محیطی با ریسک بالا پایداری خوبی از خود نشان داده است. درنتیجه معامله‌های انجام‌شده منبع مهمی برای تصمیم‌گیری است. ترکیب این روش با روش‌های پیش‌بینی قیمت می‌تواند رویکردی جدید در طراحی عامل‌های معامله‌گر باشد. | ||
| کلیدواژه‌ها | ||
| عامل معامله‌گر هوشمند؛ رفتار معامله‌گران؛ بازار رمزارزها؛ یادگیری تقویتی عمیق؛ الگوریتم DQN | ||
| عنوان مقاله [English] | ||
| Using Deep Q Network to develop an autonomous agent for trading in the cryptocurrency market, Focusing on traders' behavior | ||
| نویسندگان [English] | ||
| Seyed Mehrdad Eslami1؛ Mehdi Agha Sarram2؛ Mohamad Ali Zare Chahoki2 | ||
| 1Phd candidate, Computer Engineering Department, Yazd University, Yazd, Iran | ||
| 2Associate Professor, Computer Engineering Department, Yazd University, Yazd, Iran | ||
| چکیده [English] | ||
| Crypto currency market is a complex, uncertain and dynamic environment with significant volatility. Developing a trading strategy in this market is highly challenging and a key area of academic research. In this article, An autonomous trading agent has been designed to analyze the effects of traders' behavior (transactions they do) on changing market conditions. Although many factors influence the market, these effects impact ultimately through traders behaviors. In this article, The agent makes decisions only by reviewing and analyzing the transactions which has been done by traders. The agent is built using DDQN reinforcement learning algorithm. To train the agent, all HitBTC`s transactions during nearly 3 months for 3 cryptocurrency pairs have been gathered. The results show that the model converges and is stable. As a result, The transactions data are important source for decision making. Combining this method with price prediction methods can be a new approach in designing trader agents. | ||
| کلیدواژه‌ها [English] | ||
| Autonomous Trading Agent, Deep Reinforcement Learning, Trades Behaviors, Deep Q Network | ||
| مراجع | ||
|
[1] T. Theate, and D. Ernst, “An application of deep reinforcement learning to algorithmic trading,” Expert Systems with Applications, vol. 173, p. 114632, Jul. 2021. [2] Y. Huang, X. Wan, L. Zhang, and X. Lu, “A novel deep reinforcement learning framework with BiLSTM-Attention networks for algorithmic trading,” Expert Systems with [3] X. Cheng, J. Zhang, Y. Zeng, and W. Xue, “MOT: A Mixture of Actors Reinforcement Learning Method by Optimal Transport for Algorithmic Trading,” Lecture Notes in Computer Science, Springer, Singapore, 2024, vol. 14648, pp. 30-42. [4] Z. Huang, N. Li, W. Mei, and W. Gong, “Algorithmic trading using combinational rule vector and deep reinforcement learning,” Applied Soft Computing, vol. 147, pp. 110802–110802, No. 2023. [5] Shavandi and M. Khedmati, “A multi-agent deep reinforcement learning framework for algorithmic trading in financial markets,” Expert Systems with Applications, p. 118124, Jul. 2022. [6] B. Hirchoua, B. Ouhbi, and B. Frikh, “Deep reinforcement learning based trading agents: Risk curiosity driven learning for financial rules-based policy,” Expert Systems with Applications, vol. 170, p. 114553, May 2021, doi: https://doi.org/10.1016/j.eswa.2020.114553. [7] M. Taghian, A. Asadi, and R. Safabakhsh, “A Reinforcement Learning Based Encoder-Decoder Framework for Learning Stock Trading Rules,” pp. 1–39, 2021, [Online]. Available: http://arxiv.org/abs/2101.03867. [8] S. Carta, A. Ferreira, A. S. Podda, D. Reforgiato Recupero, and A. Sanna, “Multi-DQN: An ensemble of Deep Q-learning agents for stock market forecasting,” Expert Systems with Applications, vol. 164, p. 113820, Feb. 2021, doi: https://doi.org/10.1016/j.eswa.2020.113820. [9] D. Kanzari and Y. Ridha Ben Said, “A complex adaptive agent modeling to predict the stock market prices,” Expert Systems with Applications, vol. 222, p. 119783, Jul. 2023, doi: https://doi.org/10.1016/j.eswa.2023.119783. [10] B. Yang, T. Liang, J. Xiong, and C. Zhong, “Deep reinforcement learning based on transformer and U-Net framework for stock trading,” Knowledge-Based Systems, vol. 262, p. 110211, Feb. 2023. [11] Z. Zhang, S. Zohren, and S. Roberts, “Deep Reinforcement Learning for Trading,” The Journal of Financial Data Science, vol. 2, no. 2, pp. 25–40, Mar. 2020, doi: https://doi.org/10.3905/jfds.2020.1.030. [12] N. majidi, M. Shamsi, and F. Marvasti, “Algorithmic Trading Using Continuous Action Space Deep Reinforcement Learning,” SSRN Electronic Journal, 2022, doi: https://doi.org/10.2139/ssrn.4276310. [13] Mahdi Massahi, and Masoud Mahootchi, “A deep Q-learning based algorithmic trading system for commodity futures markets,” Expert systems with applications, vol. 237, pp. 121711–121711, Mar. 2024. doi: https://doi.org/10.1016/j.eswa.2023.121711. [14] R. S. Sutton, Reinforcement Learning: An Introduction, Second Edition, Cambridge, Massachusetts: The Mit Press, 2018. [15] E. F. Fama, “The Behavior of Stock-Market Prices,” The Journal of Business, vol. 38, no. 1, pp. 34–105, Jan. 1965, doi: https://doi.org/10.1086/294743. [16] H. Van Hasselt, A. Guez, and D. Silver, “Deep Reinforcement Learning with Double Q-Learning,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, no. 1, Mar. 2016, doi: https://doi.org/10.1609/aaai.v30i1.10295. [17] O. B. Sezer and A. M. Ozbayoglu, “Algorithmic financial trading with deep convolutional neural networks: Time series to image conversion approach,” Applied Soft Computing, vol. 70, pp. 525–538, Sep. 2018, doi: https://doi.org/10.1016/j.asoc.2018.04.024. [18] T. Fischer and C. Krauss, “Deep learning with long short-term memory networks for financial market predictions,” European Journal of Operational Research, vol. 270, no. 2, pp. 654–669, Oct. 2018. [19] M. Wiese, R. Knobloch, R. Korn, and P. Kretschmer, “Quant GANs: deep generation of financial time series,” Quantitative Finance, vol. 20, no. 9, pp. 1419–1440, Apr. 2020. [20] J. Moody, L. Wu, Y. Liao, and M. Saffell, “Performance functions and reinforcement learning for trading systems and portfolios,” Journal of Forecasting, vol. 17, no. 56, pp. 441–470, Sep. 1998. [21] V. Mnih, K. Kavukcuoglu, D. Silver, et al., “Playing Atari with Deep Reinforcement Learning,” pp. 1–9, 2013, [Online]. Available: http://arxiv.org/abs/1312.5602. [22] D. Silver, et al., “Mastering the game of Go without human knowledge,” Nature, vol. 550, no. 7676, pp. 354–359, Oct. 2017, doi: https://doi.org/10.1038/nature24270. [23] H. Van Hasselt, “Double Q-learning,” in Conf. Neural Inf. Process Syst, NIPS, 2010, pp. 1–9. [24] Y. Deng, F. Bao, Y. Kong, Z. Ren, and Q. Dai, “Deep Direct Reinforcement Learning for Financial Signal Representation and Trading,” IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 3, pp. 653–664, Mar. 2017. doi:http://doi.org/10.1109/tnnls.2016.252240 [25] M. M. Kumbure, C. Lohrmann, P. Luukka, and J. Porras, “Machine learning techniques and data for stock market forecasting: A literature review,” Expert Systems with Applications, vol. 197, p. 116659, Jul. 2022, doi: https://doi.org/10.1016/j.eswa.2022.116659. [26] S. Fallahpour, H. Hakimian, “Paired Trading Strategy Optimization Using the Reinforcement Learning Method: Intraday Data of Tehran Stock Exchange” Financial Research Journal, vol. 21, no. 1, pp. 19-34, May 2019. [In Persian] [27] B. Sabahi, “Designing Trading Strategies Based on Deep Reinforcement Leaerning Methodes” M.S thesis, Department of Financial Management, Tehran Univ. Technol., Tehran, 2022. [In Persian] | ||
|
آمار تعداد مشاهده مقاله: 213 تعداد دریافت فایل اصل مقاله: 104 |
||