Virtual-reality-based remote-controlled mobile robot platform

In this paper, a mobile robot system, which consists of a moving base and a built-in weapon platform, was developed. The base is controlled manually using a wireless joystick in which the remote hand weapon manipulates the built-in weapon platform. A stereo vision camera is mounted on the front plane of the built-in weapon platform. Real-time video of the battle zone is recorded by the stereo camera module and is simultaneously monitored on virtual reality glasses. The glasses are worn by the person who will control the built-in weapon. The remote hand weapon is also held by the same person, and the real-time motion directories of the hand weapon are transmitted to the main platform via user datagram protocol. The builtin weapon is fired when the remote user triggers the hand weapon. The weapon platform is locked to the target, regardless of the moving base of the mobile robot.


Introduction
The concept of 'unmanned systems' has been of increasing interest in the recent decade.Unmanned ground vehicles (UGVs) are widely used in the military industry.A majority of these platforms are equipped with independent weapon platforms.The requirement for controlling both the UGV and the weapon platforms increases the complexity of the control system.Moving platforms can be remote controlled by using a number of hardware such as joystick, keyboard and biomedical devices.On the other hand, the weapon platform control needs to be more sophisticated than the conventional remote control methods.The following issues may cause deficiencies in these methods: lack of depth information on target screening and body reflex deficiencies, being away from the battle zone.In an irregular battle zone, these deficiencies decrease the performance of the UGV; therefore, virtual reality (VR)-based weapon systems provide an efficient way to make the shooting process more realistic.Oculus VR headset is a widely used commercial product, which supplies three-dimensional image data for users [1].One can feel as if he/she is in the VR environment using the headset.Another common VR hardware is Google Cardboard [2].Cardboard is especially utilised to play VR games.In this paper, the related three-dimensional glass is a simple custom-made device.It consists of small LCD panels mounted on an empty glass case and a headset holding the glasses and the LCDs.
Remote-controlled weapon mechanisms show great diversity.The correct choice depends on the mobile platform size, actuation type, and kinematic and mechanical limits of the overall platform.SARP-1 and ATILGAN [3] are sample remote-controlled mobile weapon platforms manufactured by ASESLSAN.The Kongsberg Defence & Aerospace of Norway and the Thales Group of France manufactured the M151Protector weapon platform [4].Samsung Techwin and Korea University manufactured the Samsung SGR-A1 mobile weapon platform [5].
Control and stabilisation of the weapon platform can be achieved by applying various approaches such as proportional-integrator-derivative (PID) [6], fuzzy logic [7] and other control methods [8][9][10][11][12][13][14].PID control does not require full expert knowledge about the plant.Also, PID control is easy to implement.Although this control method requires a precisely correct mathematical model, it is low cost and satisfactory in this study.Thus, control and stabilisation of the weapon platform is performed by the PID control method.Rotational movement of the remote weapon is acquired from an inertial measurement unit (IMU).Roll-pitch-yaw rotation information is transmitted to the main weapon platform via the user datagram protocol (UDP).Once the remote hand weapon is locked to a target, the main platform also heads directly to the same point marking with a point laser.When the hand weapon is triggered, the main weapon shots the locked target simultaneously.The moving base is basically a four-wheeled mobile platform with a 2-degree-of-freedom (DOF) weapon platform.Detailed explanations are given in Section 2.

Mechanical Design and System Architecture
The mechanical body of the mobile platform consists of two main parts: mobile robot and built-in weapon module.The mobile robot is a four-wheeled skid-steering type base with 60 × 50 × 22 cm 3 dimensions.This part carries the 2-DOF weapon module and is manipulated remotely by a radio frequency band (RF-ID) joystick.The design images of these two parts are given in Figure 1.The weapon mechanism is mounted on the top plane of the mobile base with a rotational bearing and coupled with a DC motor.In addition, a bevel gear is positioned on the top plane of the mobile base; therefore, the DC motor rotates the weapon mechanism in the yaw axis.The weapon mechanism is rotated in the pitch axis with a servo motor that is coupled to the rotational arm holding the gun.The rotational plate on the mobile base can be rotated between 0°and 180°, and the servo motor bounds the second arm's rotation angle at −90° and 90°.In addition, the two axes can be controlled synchronously.The assembly drawing of the weapon module and the mobile base is shown in Figure 2.
The built-in weapon platform is equipped with three cameras.One of them is fixed on the front plane of the mobile base to track the battle zone in which the mobile platform is located.The other two cameras are utilised as a stereo vision system which is mounted on the built-in weapon arm.The stereo vision module provides a realistic vision from the battle zone; therefore, the person who commands the moving platform can feel the depth of the zone.The vision of the stereo system is transmitted to the VR glasses, and the video captured from the fixed camera is also controlled in order to track the current status of the forward direction of the robot.The stereo vision module on the weapon platform is shown in Figure 3.As clearly seen in the figure, a point laser source is placed on the gun which points to the target of the weapon platform.Once the weapon is locked to a particular target, the stability of the target tracking is achieved by the stabilisation and control module.The remote hand gun is shown in Figure 4.This is just a remote control device and is not a real gun.One can manipulate this device by holding it like a real weapon and act as if he/she was in the same place as the mobile robot platform.An IMU is mounted on the hand gun; the pitch and yaw rotation of the user is transmitted to the real platform synchronously.A lithium-polymer battery is mounted on the hand weapon as the power supply.The shooting command is acquired from the trigger button on the hand weapon.When the user pushes the trigger, the built-in weapon is fired to the current target.The inertial data and the trigger command are transferred to the main control unit (MCU) via Bluetooth communication.A brief demonstration of the system architecture is given in Figure 4.

Control Strategy
The main control problem is to keep the built-in weapon locked to a given target in two axes.In other words, it is aimed to manipulate the built-in weapon by using the remote hand weapon under dynamic conditions.The controller must give satisfactory results even when the moving base rotates within the pitch and yaw axes.Thus, the mathematical model of the weapon platform should be obtained accurately.The model of the DC motor is highly related to the interior parameters; therefore, it is necessary to have full knowledge about the actuator's parameters.In practice, the model also depends on the dynamic behaviour of the actuator.The model after loading the actuator differs from the no-load characteristics of the motor.Regarding this issue, the mathematical model of the motor is obtained by system identification method [15].SIMULINK Waijung blockset offers a powerful system identification toolbox that we used in this study.The system identification scheme is depicted in Figure 5.The output characteristic of the motor is observed under various input conditions, and the transfer function of the motor is calculated by trial-and-error method.The input signal is pulse-width modulation ratio and the output is the speed of the motor.The obtained transfer function is checked to see if it matches the physical system sufficiently by using the residual analysis block.The transfer function and physical parameter correlation are verified as the compatibility ratio is 99%.The control method implemented to the yaw and pitch axes is PID.The weapon mechanism is controlled and stabilized in real-time in two axes.The control scheme we offered is robust towards the sudden changes in the rotation of the mobile base in three-dimensional space.This shows that once the weapon is focused on a particular target point, it keeps tracking the target even when the mobile base moves in free space.
Communication between the hand weapon and the MCU is transmitted via Bluetooth, while communication between the MCU and the moving platform is handled via UDP.The MCU and the hand weapon user are assumed to be relatively close, but the moving platform is far away from the MCU.This is why the communication options differ from each other.The moving platform is equipped with a slave central processing unit, while the MCU is the master.The Bluetooth communication transmits the trigger command and IMU data, while UDP communication carries real-time video frames and pitch and yaw control commands.Another command unit is the RF-ID joystick, which is used to manipulate the wheeled mobile base manually.The PID parameters are obtained by using an auto-tune tool in MATLAB.To improve the accuracy of the plant model, the dynamic behaviour of the system is considered, while the transfer function is determined.For this purpose, Simmecanics plug-in is installed on the SolidWorks software, and the designed mechanical body is extracted in the appropriate format.Then, the dynamic model is utilized to determine the required motor torque.The dynamic model added plant is given as input to the PID tune block, as shown in Figure 6.
The advantages of PID control method are listed as follows:  PID method can be implemented only on the error signals between a reference and an output dataset. PID handles the plant as a 'blackbox' if the plant model is unknown. The implementation is relatively easy and gives satisfactory results for many applications. PID coefficients can be estimated based on the system parameters. It is feasible for many real-time applications.
The disadvantages of the implemented control method are listed as follows:  PID coefficients should be tuned in balance; otherwise, there may be undesired oscillations, overshoot, delay and steady-state error. The control performance is highly related to the model accuracy.A plant model that is not defined precisely will not resist against disturbances and uncertainties; therefore, the controller will not be robust enough. PID controllers are not robust against unexpected external disturbances such as temperature and power ripples.

Results
In this section, the control results on the yaw and pitch axes are given.The real-time experiments are implemented on the mobile robot platform developed within this study.The experimental environment is the fourth floor of the Mechatronics Engineering Department.The experimental results prove that the developed control scheme and the mobile robot platform give satisfactory results under various conditions.The developed mobile robot platform is given in Figure 7.The developed controller outputs for the yaw and pitch axes are depicted in Figures 8 and 9, respectively.The accuracy in pitch angle converges to 99%.However, the accuracy is relatively poor in pitch angle as shown in Figure 9.This is caused by gravitational and other electromagnetic effects on the inertial sensor.In Figure 10, it can be seen that the user locks a target on the wall and the built-in weapon focuses on the same point with a laser point.The PID parameters of yaw and pitch angles are determined by the Simulink auto-tune block.The coefficients are given in Figures 11 and 12, respectively.

Discussion and Conclusion
In this study, a remote-controlled mobile robot platform and built-in weapon mechanism is developed and controlled.This system can be adapted to outdoor conditions by adding a GPS module and improving the mechanical capabilities.The real-time experimental video can be viewed from the following URL: https://www.youtube.com/watch?v=jDQn5HbXJLk .

Figure 1 .
Figure 1.Mobile base robot platform (left) and the remote-controlled mobile weapon platform (right)

Figure 2 .
Figure 2. Assembly model of the mobile weapon platform

Figure 3 .
Figure 3. Demonstration of the stereo vision

Figure 4 .
Figure 4. Architecture of the remote hand weapon

Figure 5 .
Figure 5.The Waijung blockset of DC motor model identification

Figure 6 .
Figure 6.Dynamic control structure of the system