MAR126 AI-SAIL
Summary
The main inspiration behind this projects was based on how children can learn in a intuitively manner how to sail in an Optimist without understanding the details of the physics involved. Thus the question is, is it possible to somehow mimic the intuitive learning of sailing by using Reinforcement learning.
Motivation
In recent years, the use of Machine Learning has significantly increased by several industries including the maritime industry. The advantage that Machine Learning has over conventional optimization approaches is that it can be employed without the need to develop dedicated algorithms to solve complex problems. Instead, a model is built from sample data, then a decision can be made based on the training of the model. There are several approaches within Machine Learning, Reinforcement learning or RL is one of them. The very limited application of Reinforcement Learning in real time operations in the maritime industry, motivates this project to investigate how it can be used in our industry, and for what type of problems it should be used.
Goals of the project
This project is focused on two MMIP’s: Digital & Autonomous Shipping and Towards Zero Emission. It uses and contributes to key enabling technology AU / Data Science. The objective of this work is to investigate how we can use reinforcement learning in Maritime applications, and when this methodology is the best solution for the problem at hand. As a demonstration, the case of an Optimist sailing up-wind in MARIN’s Offshore basin is selected. The specific steps in this project are:
- Development of the experimental Optimist set-up.
- Development of a numerical environment consisting of a Virtual
Optimist and Offshore basin.
- Develop a python-based Reinforcement a Learning agent.
- Use the numerical environment as interface to communicate the Offshore basin and the Reinforcement Learning agent.
- Train different agents using different learning approaches.
At later stages, it is foreseen that universities and other institutes will be able to use the experimental set-up and numerical environment prepared for this project and train/tests their own agents.
Innovativeness
It is proposed to apply the artificial intelligence technique Reinforcement Learning to control the actuators installed in an Optimist sail boat. In Reinforcement Learning, an agent learns to control a dynamical system by means of interacting with it. In order to enable the agent to steer the Optimist, a boat/basin/agent unique numerical environment interface needs to be developed. Within this numerical environment, an accurate numerical representation of the Optimist and basin must be made. The majority of the initial agent training will be conducted within the numerical environment using the digital Optimist and basin. The innovative approach described above not only will allow to test different learning approaches within MARIN but will also help our partners to explore their unique learning approaches for Machine Learning. The knowledge and insight learned in this project will be a step forward in the integration of Machine Learning in the Maritime industry.
Valorisation
Tool to train numerically Reinforcement Learning agent. (Optimist and Offshore basin numerical environment). A experimental set-up to test Reinforcement learning agents. (Experimental Optimist set-up prototype)
Duration of the project
01-01-2023 / 31-03-2024