An Intelligent Robotic System for Perceptive
Pancake Batter Stirring and Precise Pouring

The uniformity of liquid batter refers to the evenness of its composition throughout, with respect to ingredients being thoroughly mixed and resolved. The goal of the uniformity check is to control the robot to stop stirring once the batter achieves uniformity. To achieve this, we measure the average torque of a push motion after $50$ rounds of quick stirring in the perceptive stiring stage, which is defined as a trial.

Fig. 3(a) shows the changes of torque throughout the stirring process when stirring five batches of batter with a water-flour ratio ranging from $1.1$ to $1.5$. The resistance torque continues to decrease over the $30$ minute stirring period, indicating that the batter viscosity keeps decreasing and never converges. This phenomenon could be attributed to the batter becoming finer as a result of constant stirring. However, humans typically assess batter uniformity after only $4$ to $5$ trials of quick stirring (approximately $160$ to $200$ seconds) as shown in Fig. 3(c). To keep our system efficient, we want to prevent stirring to the over-uniform state as shown in Fig. 3(d).

To control the robot to stop stirring at the appropriate time, we set a threshold for the changing rate of the measured torque based on human labeling. This label is obtained by the human observer’s rating on each trial’s resulting batter, with the scale shown in Fig. 3. We define the point where human observers’ rating average is $0$ to be our uniformity threshold, represented by the black crosses in Fig 3(a). We aim for the robot to stop stirring at a point that aligns closely with these identified moments. This is achieved by comparing the difference in torque between consecutive trials against this threshold, thereby determining the uniformity of the batter.

The liquid level is defined as the distance from the bottom of the bowl to the surface of the batter. We employ multiple push motions in an increasing height sequence. Fig. 4(a) illustrates the trajectory of this motion. After each pushing motion, we obtain an average torque reading. We show the relation between torque and the distance from the tip of the whisk to the bottom of the bowl, illustrated in Fig. 4(b). The slope portion corresponds to pushing in the batter while the flat portion corresponds to pushing in the air where the torque is around $0$ N.

To differentiate between a push occurring in the batter versus in the air, we set a threshold for the variation in torque measurements between successive pushes. By fitting the torque data for both batter and air pushes to linear functions, we then identify the intersection of these fitted lines as the precise point where air transitions to batter. This intersection point shows the liquid level of the batter.

Under the assumption of a uniformly mixed batter, a relationship exists between the water-flour ratio and the viscosity of the mixture. Liquids with higher viscosities offer more resistance during stirring, allowing us to develop a data-driven model to estimate the water-flour ratio based on the observed resistance torque. We collect torque data by pushing into various batches of liquid batter with different water-flour ratios and immersive depths to measure the water-flour ratio. We found that greater immersive depths and smaller water-flour ratios yield larger torques, which is consistent with expectations.

To estimate the water-flour ratio of an unknown batter, our model takes the “Torque vs. Immersion depth” curve obtained in section III-A2 as input. Then it finds the closest two fitted curves that minimize mean square error (MSE) and estimates the water-flour ratio by calculating the weighted average value as $r_{E}=(M_{1}r_{1}+M_{2}r_{2})/(M_{1}+M_{2})$. Where $r_{E}$ is the estimated water-flour ratio, $r_{i}$ is the water-flour ratio label. $M_{i}$ denotes the MSE between predicted and measured torque, computed as $M_{i}=\frac{1}{n}\sum_{j}(\hat{T_{j}}-T_{j})^{2}$, with $n$ representing the total pushing trials count, $j$ indexes each data point in the “Torque-Immersion depth” curve and $\hat{T_{j}}$ denoting predicted torque in the dataset with $T_{j}$ being measured torque.

To achieve a precisely controlled batter-pouring process, we aim to control the robot’s pouring speed with consistency and accuracy. In our method, the robot first grabs the bowl handle, and we control the rotation of the wrist joint to pour out the batter. Through experimentation, we discover that maintaining an angular speed of $0.007$ radian per second enables us to achieve a uniform flow rate across various water-flour ratios. By sticking to this speed, we can control the flow rate in a quasi-static manner.

With a constant flow rate of the batter poured out, we then control the motion speed of the robot to form an even batter stroke with well-controlled widths, which is the basic requirement to form an arbitrarily shaped pancake. We train an MLP model to predict the robot’s moving speed from the water-flour ratio of the batter and the desired stroke width.

In another more traditional case, we aimed for the robot to create a round pancake by pouring the batter from a stationary position. Here, we supervise the pouring duration to determine the pancake’s size. We train another MLP model to predict this pouring time.

This motion is designed for mixing water and flour at high speed to achieve uniformity. The motion trajectory’s radius is smaller than the bowl, thus it doesn’t make contact with it.

This motion is designed to handle the area that quick stirring motion is unable to reach. At the initial stage of stirring, flour easily accumulates in those areas. If not dispersed early in the accumulation, it will form hard-to-dissolve flour clumps.

This motion is designed to scratch the batter stick to the border of the bowl, which typically happens at the initial stage of stirring. The whisk will contact with the bowl’s edge and have small deformations. The trajectory’s height is $15$ mm above the bottom of the bowl to scrape the flour above the liquid surface.

This motion is designed to get rid of unresolved batter stuck in the space within the whisk. We command the whisk to translate back and forth within the batter at a frequency of $8$ Hz and at the distance of $5$ cm.

Our stirring process includes a preliminary stirring phase to mix the raw material into a batter mixture and a perceptive stirring phase to achieve higher batter uniformity. In the preliminary stirring phase, we execute the following series of actions over a 90-second period: quick stirring, edge scra**, fine stirring, and whisk shaking. In the perceptive stirring phase, we execute $50$ rounds of quick stirring and assess the mixture’s uniformity by conducting the push test repeatedly until the batter reaches uniform. Following this, we proceed to the liquid property estimation sub-task (Section III-A).

Another source that influences the flow rate of the batter is the initial angle. In Section III-B, we discussed how to maintain a flow rate, but the initial flow rate can vary if we change the initial angle. The goal of this module is to find an optimal initial angle so that batter with the same water-flour ratio and different liquid levels can have a constant initial flow rate. We design the spout in Fig. 6 to help regulate batter flow. We first use the liquid level to estimate a starting angle smaller than what it should be. Our system utilizes visual input to monitor the flow of the batter, with a KMeans segmentation to isolate the batter. When the batter is detected to drip from the spout, we initiate the pouring movement, as shown in Fig. 5.

To form an arbitrarily shaped pancake, we create an algorithm that can decompose a shape from a binary image into a batter-pouring trajectory for the robot. Fig. 2 shows an example of the trajectory we generate. Our algorithm contains two modes for generating trajectories of different shapes:

• •

  For an enclosed shape, our algorithm uses the shape’s edge as the initial loop and erodes the shape to generate trajectories for inner loops.

• •

  For non-enclosed lines, our algorithm skeletonizes the image to form the trajectory and then refine the shape using the Minimum Spanning Tree algorithm.

We collect training data for building the water-flour ratio estimation model and collect test data for uniformity check, liquid level, and water-flour ratio estimation. In both sets, stirring is performed by the robot. For the training set, we only collect data in the small bowl. The test set employs both the small bowl and the large bowl, demonstrating our method’s generalizability.

For the training set, we collect data from batters with a water-flour ratio varying from $1.0$ to $1.5$ in increments of $0.05$. The water quantity used for each bowl of batter is controlled around $300$ ml, along with the corresponding amount of flour by weight. The robot is programmed to execute $5$ cm push motions from a height of $3$ mm to $63$ mm above the bottom in $3$ mm increments, following the trajectory depicted in Fig 4(a), while torque data is recorded. In total, $11$ trials of batter are made and $660$ data points are collected to build our model.

For the test set, $10$ and $5$ trials of batter are prepared in small and large bowls, respectively, with water-flour ratios randomly selected between $1.0$ and $1.5$ and total weights varying from $450$ to $700$ g. For each batter trial, uniformity is assessed by human observers, followed by evaluation of the accuracy of liquid level and water-flour ratio estimations.

Since uniformity has no official definition, we rely on assessments from humans with pancake cooking experiences to evaluate uniformity. We invited $5$ individuals to evaluate the final stirring outcomes of the $15$ batter batches of the test set. They can either observe it by eyes or utilize the whisk to feel the texture of the batter and assess its consistency. After observation, they were first asked to give a rating according to a scale ranging from $-3$ to $3$, where $-3$ indicates non-uniform batter and $3$ indicates over-uniform batter. The mean uniformity score of $0.625$ with a variance of $1.1$ indicates that the stirring outcomes fall within the desired uniformity range. Results show that our stirring strategy and uniformity check also work well with a large bowl.

Table II shows the error and percentage error of liquid level estimation on our test set. We have a total of 20 test trials whose liquid levels range from $5$ to $55$ mm. Liquid level error is as low as $3.04\%$.

Table II and Fig. 7 show the error and percentage error of water-flour ratio estimation on our test set. Although our testing error in the large bowl is slightly higher, it remains in an acceptable range, demonstrating the generalization potential of our method.

To form a shape, the robot pours the batter onto a 12-inch metal pizza plate, which we believe is the best available alternative to a traditional griddle. We use the RGB camera to capture and measure the results. We collected training and testing data of the two experiments and the test data did not replicate any scenarios from the training data, but the water-flour ratios were within the same range. The liquid level also varys for every trial of data we collected.

Line Strokes Task: We collect training data by controlling the robot arm moving speed and documenting the resulting line width. Our training dataset includes trials across five distinct water-flour ratios, ranging from $1.25$ to $1.45$. The resulting line widths of all experiments ranging from $0.8$ cm to $5.9$ cm. For the test data, we use five different batters, aiming to draw lines in four widths from $1$ cm to $4$ cm, which do not exist in training data.

Round Shapes Task: We collect training data by controlling the pouring time and documenting the resulting diameters. Our training dataset includes trials with the same range of water-flour ratios. The resulting pancake diameters ranging from $2$ cm to $22.5$ cm. For the test dataset, we use five different batters to pour four varying diameters ranging from $5$ cm to $20$ cm.

We implement an MLP model with 1 hidden layer with 32 inputs and 64 outputs in PyTorch and train it using Adam optimizer for 1000 epochs and a learning rate of 0.06. The same network structure and setup are used for both experiments. Both MLP models are trained with MSE loss in a supervised manner.

Line Strokes Task: The Simple method draws the line with the same arm moving speed regardless of the water-flour ratio of the batter. It assumes all batters have the same behavior. We also do not control the flow rate and assume the speed that is needed to draw a line for all water-flour ratios is the same. Moreover, we assume the speed of different widths is proportionally increasing. We found that the average speed needed to draw a line of $1$ cm wide is $0.2$ cm/s, then we increase this speed linearly for different line widths respectively.

The Manual method draws the line by a human expert holding the bowl. We asked the human expert to try to pour different water-flour ratio batter to draw the same width as our test set. We also give the human expert a reference width in the form of a piece of paper of the desired width.

Round Shapes Task: The baseline method ignores the dynamic of the batter and only considers the target volume of pouring. So we make the robot quickly reach the final angular position, which is determined by the volume we want to pour, and hold for $30$ seconds, where the vast majority of the batter is poured and only a few drops are left in some cases.

We measured the width and the area difference of the poured batter from the desired shape. The result is shown in Fig. 8, 9 and Table III, IV. In summary, our method shows a significant improvement compared to the baseline where the batter properties are not considered.