Reinforcement Learning for Swimming in a Concentration Gradient

Gustavo C. Buscaglia, Stevens Paz, Roberto F. Ausas, Juan P. Carbajal


The coupled problem of hydrodynamics and solute transport for a microscopic swimmer is studied, with the Reynolds number set to zero and the Péclet number (Pe) ranging from 0 to 100. A suitable second order finite element method is proposed based on the FEniCS library. It is shown that errors in the temporal treatment of the geometry can destroy the dynamical properties of the system. The method is applied to compute the Sherwood number as a function of Pe in homogeneous fluids. The results agree with previously published asymptotic approximations and confirm that little gain in solute flux is achieved by swimming. A reinforcement learning framework is also introduced that considers the swimmer as a learning agent moving inside a fluid with a concentration gradient. The outcomes of the learning process show that learning locomotion is significantly easier than learning to move towards concentration maxima. Furthermore, the experiments show that the learning difficulty increases severely with the Péclet number. The results demonstrate the challenges that natural and artificial swimmers need to overcome to migrate efficiently in chemically inhomogeneous environments.

Full Text:


Asociación Argentina de Mecánica Computacional
Güemes 3450
S3000GLN Santa Fe, Argentina
Phone: 54-342-4511594 / 4511595 Int. 1006
Fax: 54-342-4511169
E-mail: amca(at)
ISSN 2591-3522