Learning to Count Small and Clustered Objects with Application to Bacterial Colonies

Abstract

Counting small and clustered objects is a challenging Computer Vision task with many realworld applications. Many researchers have attempted to apply prevalent machine learning algorithms to count objects. However, feature engineering which is a notoriously difficult part of machine learning algorithm development has yet to address the following difficulties of this task collectively: 1) small object size, 2) clustered objects, 3) expensive cost to collect and annotate data, and 4) various domain or category adaptations. This research solves these four difficulties collectively with an example application to bacterial colonies. It starts with a thorough investigation into MicrobiaNet, which is the best-performing cardinality classification method for bacterial colony counting to the best of my knowledge. Experimental results empirically prove that high image similarity across different classes is the main issue for this method to count clustered colonies accurately. Additionally, it is empirically identified that the class imbalance has a very limited impact on the counting performance. These two findings shine new light on the direction of future improvement for other researchers. Because of the limitations of the best-performing cardinality classification method for colony counting, this thesis then poses the counting task as a few-shot regression task. I adapt FamNet to particularly count small colonies and propose a new model called ACFamNet to count small and clustered colonies. ACFamNet addresses the first three aforementioned difficulties by tackling region of interest misalignment and optimising feature extraction during the feature engineering process. A real-world data set is collected for developing and evaluating ACFamNet. To address all aforementioned difficulties together, I propose ACFamNet Pro which is an advanced ACFamNet with additional multi-head attention mechanism and residual connection to count small and clustered objects. The synergy of these additional components supports the model to achieve a better counting performance and become readily generalisable to objects of a different category by dynamically weighting objects of interest, optimising gradient flow and tackling region of interest misalignment. Extensive experiments are conducted to prove ACFamNet Pro is able to tackle the aforementioned difficulties collectively.