SenSen has recently assisted one of the world’s most technically advanced ports achieve greater compliance, safety and security for its people and facilities through the application of Live Awareness AI. At the port, which leverages human capabilities in collaboration with driverless AGVs, and cranes, and has a reputation for being a “catch-up port” where vessels that have been delayed elsewhere can work so efficiently that they leave port again on schedule, the safety credibility across the entire port relies on compliance itself being managed efficiently.

The primary objective of the engagement between SenSen and the port was to leverage computer vision and AI monitoring and analytics capabilities to support the management and enforcement of safety compliance among port workers. The port was looking for ways to detect violations such as not wearing PPE, working on a task alone when it was required to be working with someone else, or being too close to dangerous equipment.

This project posed several unique challenges to the capabilities of SenSen’s advanced Live Awareness platform:

1. Detection Accuracy: Workers appear very small in the footage captured by the PTZ (pan-tilt-zoom) cameras on the site, making it difficult for AI to detect and track them accurately.
2. Data Scarcity: Real-world data collection on site proved to be challenging due to the sporadic presence of workers within camera views. This meant that it could be difficult to gather enough training data to ensure the accuracy of the AI models.
3. Dynamic Environment: The port environment is constantly changing, with numerous camera angles and varying lighting conditions. It is always a challenge whenever AI needs to adapt to significantly changing conditions, and this was a daily occurrence at a busy port.

When brought together, these three challenges meant that a solution that was designed to promote total efficiency could go the other way. If the AI system wasn’t incredibly accurate and responsive to the conditions within the port, it ran the risk of generating excessive false alarms. Investigating all of these would overwhelm the operations team and, counter-intuitively, result in an environment that was actually less safe and compliant.

To overcome these challenges, the SenSen team were able to apply several advanced AI techniques and strategies through their Live Awareness platform, SenDISA. This was only possible thanks to the total expertise of the team in Live Awareness AI, which has contributed substantially to furthering the capabilities of the technology for future deployments:

1. Synthetic Data Generation: SenSen came up with a ground breaking and innovative approaches to generate and leverage synthetic data to complement real-world data from the customer operations to train the AI models, which subsequently allowed the AI models to have deeper training and therefore was more accurate in application.
2. Advanced Image Segmentation: SenSen developed a way of segmenting images captured by the camera so that the AI could detect tiny objects, despite the cameras themselves being of a too-low quality for that to be achieved without segmentation.
3. Depth Estimation and Geometric Calculations: SenSen spent time with developing ways for the AI to calculate distances between workers and hazardous equipment to ensure compliance with safety distances – this was made challenging because the camera views did not provide an inherently accurate depiction of distances.

Through this often creative approach to Live Awareness implementation, SenSen was able to effectively train the AI to detect and classify containers, spreaders, and workers, and to recognise whether workers were wearing PPE, while tracking the entire port from a safety perspective.

What’s more this was achieved using existing, legacy CCTV network even with the lowest-resolution cameras across the port environment. The developed and delivered solution was not only effective in flagging violations of PPE use and safe distance requirements, but it was also able to identify when workers were operating alone or in required teams. Additional smarts built into the AI meant that this monitoring was contextual, and able to identify when working alone was not a violation of any requirement to be working with another person.

The AI-powered system has been operational for almost a year, continuously monitoring and enforcing safety compliance. It has significantly reduced the number of safety violations and improved overall safety at the port. The success of the project has led to discussions about extending the contract for further phases.