Tech-enabled fish species recognition & sorting system to save Atlantic salmon
Full Solution
June 2021, phase 1 solution installation in Storelva
Berlevåg Jeger- og fiskerforening (BJFF)
This is the story on how Huawei and a local association of hunters and anglers - Berlevåg Jeger- og fiskerforening (BJFF) - developed an automated fish trap with a built-in camera system and AI-software to remove the invading foreign humpback salmon from a river in Berlevåg, a small community, on the northmost tip of Norway, neighboring the Barents Sea. By preventing the humpback salmon from swimming up the rivers, we eliminate the risk of reproduction and reduce the threat to invasive species in rivers. The Atlantic salmon populations are exposed to variety of threats but the threat from the invasive humpback salmon is the most significant. This is the story on how to save the Atlantic salmon by using technology. The humpback salmon problem is escalating drastically, and as a result, the construction design of the fish trap is prepared to be duplicated and fit into other rivers
Context
Challenges addressed
The challenge is to remove humpback salmon from Norwegian rivers to preserve the native Arctic biodiversity in an efficient way. The humpback salmon’s natural habitat is in the Pacific Ocean but has been present in Norwegian rivers since 1960s. In 2017, the presence of humpback salmon escalated quickly in rivers along the Norwegian coastal line, getting close to alarming levels. The invading foreign salmon species has been categorized as a “high risk” in threat to native biodiversity, especially for local salmon species. The sea char in Storelva is a small and hard-pressed population. If a lot of humpback salmon spawns the same area, it will be a very high risk for our local fish, potentially destroying it. The task of removing the humpback salmon from rivers has been particularly difficult as they arrive in the watercourses in packs, often among the Atlantic salmon. The removal of the humpback salmon is currently performed manually by volunteers, which is a highly inefficient.
Beneficiaries
- The Atlantic Salmon
- The Norwegian Association of Hunters and Anglers,
- Local partner organizations, NGOs and government bodies with mandate to preserve biological diversity and the environment. Local governments. Local anglers.
Location
Berlevåg, Troms, Norway
North Europe
Process
Summary of the process
The edge device will pull HD video streams from underwater cameras. The edge device processes the video streams and persist raw video clips and structured meta data to local disk space.
At regular intervals, all data persisted on the edge device will be uploaded to a remote data center. On the data center, the structured metadata will be used to train a new AI model. The AI model will be uploaded to the edge device and function as usual.
Building Blocks
Video Capturing and Fish Recognition Module
With underwater HD cameras deployed in the rivers, the video streams are fed to an edge device. The edge devices are physically close to the rivers, with the benefit of reduced latency. A highly optimized AI inference software is implemented for real-time fish recognition.
Enabling factors
By presenting the AI module with tens of thousands of labeled pictures, the algorithm quickly comes up to desired level of accuracy. As more and more species of fish passes through the system, the algorithm learns even more and becomes better and better
Lesson learned
Having the edge device close to the sorting mechanism proves to be highly efficient. Currently the species of fish is recognized by the algorithm within 8 milliseconds. Different light and background colors impact the capabilities of fish recognition. Light colored background, but not completely white proves to be best. Midnight sun has a positive effect, making the recognition able to run 24/7
Data Collection and Model Re-training
There are types of data generated by the edge device.
- Raw video clips. With limited disk space on edge device, the video clips are regularly uploaded to a data center with multi-layer backups. This type of raw data is meanful for future study and backtracing.
- Metadata from AI models. As previouly descibed, the AI software will recognize fishes appreared in the video streams. Therefore, the metadata will include image frames that actually contain fishes. For each meaningful image frames, the locations and categories of fishes will be marked. This type of metadata will enable more scientific analysis such as fish counting, fish habit research etc. More importantly, the metadata will be used to retrain the AI models used in the software. Overtime, the AI models will give more accurate recognition.
Enabling factors
In order for the solution to operate optimal, it requires a good stable internet connection. In this solution fiber was available, but the solution should work just as fine over 5G and possible even over 4G as uploading can be done without real time
Lesson learned
Having a centralized cloud storage solution is vital in order to reduce on site investments. It also enables the solution to share all previous learnings to new installations, giving them a stating point at the level of all existing solutions deployed. Each solution with contribute with additional learnings and increased quality benefitting all the others.
Automated sorting mechanism
Good local investigation of the environment, water current, water depth, angle of the river bottom, available light, power and internet connection are all factors that will impact the solution. Using durable environmentally friendly material and using companies with previous experience dealing with live fish in under water installations is helpful.
Enabling factors
In short, a mechanical open/close mechanism will sort the fish that pass the barrier and lock. Local salmon and sea trout can pass unhindered, while humpback salmon and farmed salmon are stopped and sorted out to a barrier where it can be removed and potentially processed.
Lesson learned
Installing such a solution requires good planning and skilled engineers. Working in a strong water current is demanding and requires expertise and equipment. Securing the installation to the rived bottom is vital, as you cannot have movement that can introduce small unwanted openings the fish can swim though. We have also learned that it is better to send all unrecognized fish into the tank for manual sorting, than having unwanted species getting up river to spawn.
Impacts
The impact of the project is immense for various reasons. First, the solution effectively removes humpback salmon from rivers, which contributes to maintaining the local biological diversity in general and saving the Atlantic salmon in specific. Second, the solution is scalable in a way that it can be applicable in other prone rivers in the arctic region. Third, the solution is cost-efficient compared to other manual capture solutions and liberates the local communities from tiring manual labor. In addition, the system opens for economic gains as the humpback salmon indeed is edible. A possible solution is that local actors may extract the humpback salmon from the traps to sell it on the market.
Sustainable Development Goals
SDG 3 – Good health and well-being
SDG 11 – Sustainable cities and communities
SDG 14 – Life below water
SDG 17 – Partnerships for the goals
Story
June 2021, phase 1 solution installation in Storelva
Berlevåg Jeger- og fiskerforening (BJFF)
There is every reason to be concerned about the future for the Atlantic salmon, especially in northern Norway. Last year, more humpback salmon were registered than Atlantic salmon in many of the rivers in the region that was monitored. This is problematic as it competes with other fish for both food and space, carries and spreads diseases, and potentially contaminates the local water quality. It thus threatens the local biodiversity and ecosystems in Norwegian rivers.
Atlantic salmon is endemic to the North Atlantic, and an integral part of the identity and values of people in the northernmost part of Norway. Fighting the invasive humpback salmon and retaining stocks of native species is therefore important for our culture and its practices, as well as minimizing the biological and economic damages.
For many of our small and remote communities the despair is noticeable. For years, local volunteers and river guards gather every summer, to desperately try to stop the invasive humpback salmon ascending in the rivers to spawn and threaten the survival of native species. Until now, this has been done manually in icy rivers with nets and other tools to stop the invasion.
Local communities, national authorities and key research communities are all facing distress caused by the situation. It became urgent for all stakeholders together to develop a solution that can monitor and sort all predators in the watercourses, in the estuaries and in the nearby sea areas.
In the summer of 2021, the first part of a pilot project was launched in a collaboration with Huawei, in the ‘Storelva’ river in Berlevåg municipality.
Here, the goal was to identify all anadromous fish ascending in the watercourse. With a continuous video stream from late June to September, with tens of thousands of images, the hardware and the newly developed source code were able to identify well over 90% of the salmon and humpback salmon that passed. Thus, enabling the system to recognize the river's own stock of salmon and sea char, but also register the invasive humpback salmon. A goal we achieved by a good margin.
Phase two has now been initiated to develop the latest module to be connected with sorting. In short, a mechanical open/close mechanism will sort the fish that pass the barrier and lock. Native salmon and sea trout can pass unhindered, while humpback salmon and farmed salmon are stopped and sorted out to a barrier where it can be removed and potentially processed.
