MAR 128 AI Steer: AI to combine Ship Tech and Experience for Emission Reduction

Summary of the project

In AI Sail we have seen how Reinforcement Learning captures experience without explicitly modeling the physics involved.

We are now taking on the major follow-up challenge together with the sector in the AI STEER project: can we advise a helmsman on the most energy-efficient way of sailing based on the joint and many years of experience of helmsmen.

It takes years to learn how to sail a ship and it is difficult to transfer this knowledge to new helmsmen. With new ambitions and regulations for emission reduction, shipowners cannot wait 25 years to train each new helmsman. AI STEER aims to leverage existing experience captured in onboard measured data. The data is used to develop and train AI-powered onboard digital assistants. The job of the AI STEER algorithms is to optimise a ship’s sailing on predefined segments of a voyage in view of speed and energy-efficiency, taking into account heading, speed, trim, autopilot settings, and possibly settings of other fuel-saving technology.

Years of experience is very valuable, but locked inside people’s heads. It might be tempting to try to ‘fit a model’ to available onboard measured data, but helmsmen may have made different choices in similar situations (possibly based on different requirements for the journey), introducing a lot of noise for such supervised learning approach. Another challenge is the planning of sequences of actions, which might be extracted from the data, but should not be evaluated as full sequence. Perhaps part of the sequence is near optimal and part is not. Finally, such approach would not do justice to the nature of human experience, especially personal strengths and weaknesses. Therefore, we will research and develop AI algorithms in AI STEER that are better suited to capture experience and to learn from both the strengths and weaknesses found in all available data. In the end, advice will be based on the very best human examples and is robust against common pitfalls.

Goal of the project

Our aim is to develop and train AI-assistants that can advise a helmsman on the most energy-efficient way of sailing. The assistants are based on the joint and many years of experience of helmsmen captured in onboard measured data.

Building on work done for AI Sail, we will use Reinforcement Learning (RL) to explore the data measured onboard. RL is an experience based AI, so, as with AI Sail, there is no need to make available experience explicit in terms of physics. However, we will research and develop Offline Reinforcement Learning (Offline RL) in AI STEER to primarily base the generated advice on actual human experience.

Offline RL learns from existing interaction data (e.g. onboard measured data), without the need to build its own experience through trial and error. The full experience of sequences of actions is used to train on, to correctly capture future performance related to current actions, as well as current performance. Described as a mapping, RL maps state (the current ship position, heading, speed, weather conditions, weather forecasts, target position, etc.) to action (speed change, heading change, trim). A fundamental difference between the better known supervised learning and RL is that RL does not learn the mapping from state to action directly from examples that show some action taken in some state. Instead, RL learns the utility of such action from the immediate observed performance as well as future performance. This future performance is (implicitly) evaluated with the currently best known tactics, thus isolating the contribution of current actions. This allows the algorithm to apply the best tactics for each situation in a flexible way, essentially stitching the best observed tactics from different (parts of) voyages, possibly performed by different operators, to come to the best advice.

Offline RL can lead to behavior stitching, using the best parts of different examples to arrive at even better solutions.

However, the challenge starts with defining relevant performance metrics (WP 1), such as fuel and/or time saved. The metrics can consist of competing components and the trade-off will be reflected in the advice that is derived from the data. It is important to notice that this is not an absolute measure and dependent on the preferred trade-off. Therefore it is not suited to evaluate performance found in the data (which is good), but rather to evaluate the benefit of the generated advice, again with respect to (hind-sight or alternative) trade-off settings.

Proper interpretation of data requires proper structuring and automation. To ensure efficient research and seamless future expansion of the datasets, a definition of the metadata required is needed (WP 2). Available data will be used to train Offline RL agents (WP 3). A couple of options will be used to evaluate and improve the developed

RL agents. First of all, from the RL agents an estimate of the benefit of the advice can be extracted. A numerical model for one ship and the environment will also be setup to evaluate the trained agents and to further enhance their performance with additional online learning. This environment will be based on available data for performance metrics and model test data for hydrodynamics. New data-driven approaches will be used to create the numerical environment from both data sets. Finally, a multi-fidelity model can be used for evaluation in conditions that are outside the available data, combining data from the numerical model with field data (WP 4). Evaluation of the advice by itself does not yet provide full insight in the usability as digital assistant. For this, evaluation on specific scenarios with different settings for trade-off and different ways of giving the advice will be tested and demonstrated (WP 5, 6).

Motivation

In 2023 we took up the challenge in the AI Sail project: can a computer learn to sail with an Optimist based on machine learning? The idea for ‘AI Sail’ is based on the observation that children can intuitively learn to sail in an Optimist, without understanding its details such as aerodynamics and hydrodynamics. The background of this challenge is an important one: what can artificial intelligence and machine learning contribute to a cleaner, smarter and safer maritime world? Most maritime prediction methods are based on a model-based approach: physics based models are combined in a computational model and validated in model tests and reality. With AI Sail we demonstrated the possibilities of data-driven methods, where the physics are not explicit in the model, but implicit in the data. In simple terms: if children can learn how to sail an Optimist without knowledge of aerodynamics, hydrodynamics and oceanography, an AI-algorithm should be able to learn the same. We are now taking on the major follow-up challenge together with the sector in the AI STEER project. We want to learn from human experience – captured in data measured on board – in order to advise a helmsman on the most energy-efficient way of sailing. It takes years to learn how to sail a ship and it is difficult to transfer this knowledge to new helmsmen. With new ambitions and regulations for emission reduction, shipowners cannot wait 25 years to train each new helmsman.

Innovativeness

The proposed approach enables learning from experts, while respecting that the context is not fully known (i.e. the preferred trade-offs at the time of sailing). Leveraging good quality data and a learning method that is not limited to replication of examples, learning is efficient and new tactics can even be found. This innovative approach will be a step forward in the development of AI-powered digital assistants with various

application areas: providing onboard advice on the most energy-efficient way of sailing, helping to assess the impact of risk situations and providing insight into the options for action, evaluating the reliability of onboard advice systems or optimizing ship designs. The knowledge and insight learned in this project will be a step forward in the integration of machine learning in the maritime industry.

Valorisation

Duration of the project

Startdate: 20/05/2024

End date: 31/03/2026