Specifically, there are three steps. Traverse the three-dimensional point cloud. Judge whether the points in the point cloud are inside the convex hulls and belong to plane set C Plane. If the points are inside the convex hulls i. These interior points will be put into the potential object point set M Object , and considered as the seed of the region growing. Starting from the seed, if the four points around the seed are inside the convex hulls, but do not belong to plane set C Plane , and the distance between two points is less than a threshold, then these four points are regarded as interior points P Interior of the object and will be put into the potential object point set M Object.
All the qualified interior points P Interior are collected and put into a potential object point set M Object. In order to avoid erroneous judgment of the points near the convex hull boundary, two-times region growing algorithm is exploited to obtain the complete object. In Figure 11 , the green dotted line represents the convex hull and the red solid line represents the object region after two-times region growing process.
Graduate Certificate in Intelligent Robotic Systems
If a point belonging to object point set M Object is on the convex hull boundary i. If any of the four points i. In addition, the points surrounding the seed are considered as new seeds as the next round judgment until there are no such points. Finally, all the potential object point sets M Object are put into the total object set O. In order to recognize objects effectively, a deep convolutional neural network CNN is designed and applied. Specifically, the architecture of our CNN is presented in Figure The network contains eight layers with weights: the first four are convolutional layers and the remaining are fully connected layers.
The neurons in the fully connected layers are linked to all neurons in the previous layer. The rectified linear units Relu is applied to every convolutional layer and fully connected layer as the activation function. After the convolutional layers, a flatten layer is employed to transformed the multidimensional feature maps into single dimensional feature maps, which can be put into the fully connected layers. Four fully connected layers have , , 64, and 4 hidden units, respectively. The output of the last fully connected layer is connected to a 4-way softmax which produces a distribution over the 4 class labels i.
After the recognition, the position information of the desired beverage container in the camera coordinate system is calculated as the mean value of position information of all the points of the actual object point set. With the coordinate transformation from the camera coordinate system to the robot coordinate system see Section 3. As mentioned at the beginning of Section 3. With the assistance of the Kinect SDK 2.
By using the coordinate transformation mentioned in Section 3. The robot manipulator has six joints and three fingers. Each finger has a controllable joint and a passive joint. When the controllable joint is grasping an object, the passive joint rotates automatically so that it can hold the object more firmly.
By using the official API, the end-effecter of the robot manipulator i. Therefore, only several separated key points in the task space are required to obtain the continuous tracking trajectories of joint space. The remaining position points in the delivering process are predefined. Moreover, the manipular state, including position and direction information, is captured and transferred to the robot controller in real time so as to perform accurate control.
The manipular state is also sent back to the decision-making layer to make sure that the task is finished. Written informed consent was obtained from each subject. In order to verify the effectiveness of the proposed ID-SIR system, two experiments are designed: one is the CNN training and the other is whole system evaluation. In order to train our CNN to recognize the desired object, a specific data set needs to be established. Without loss of generality, we task three kinds of objects i.
The data set was designed to contain 4 classes, i. Thus, 26, images in total, approximately 6, samples for each class, were gathered through a Kinect applying the region growing algorithm. The data set was then divided into training set and validation set randomly with a rate of Before training, data augmentation was implemented as generating new images with rescaling and horizontal reflections to reduce the overfitting problem. Eight volunteers were asked to attend the evaluation experiment.
You are here
The whole system evaluation process consisted of two parts: off-line training and online testing. These volunteers were all healthy subjects 19—21 years old , among which only one subject i. The EEG signal data were acquired by the following three steps. First of all, a target symbol was given randomly by the computer and displayed in the text box above the four buttons. Second, the subject was asked to pay attention to the given target symbol. Third, the buttons flashed in a random order. Each subject had to complete 40 off-line trials i. After the data acquisition, the data set was processed by the method of self-adaptive Bayesian linear discriminant analysis SA-BLDA illustrated in Section 3.
As analyzed in Section 3. From the table, we can see that the off-line training process is fast, and the accuracy is high. Table 1. During the online testing process, each subject was asked to control the robot manipulator to finish 10 times assistive drinking tasks. Evidently, two commands were required to complete each task: i grasp and deliver the desired beverage container to the month, ii put the beverage container back.
Therefore, during the online testing experiment, each subject was asked to finish 20 control commands i.
- Partner Login.
- Atlas of the Flora of the Great Plains.
- LabVIEW Robotics Module Download - National Instruments.
The snapshots of a subject experiencing one assistive drinking task are shown in Figure Snapshots of a subject completing one delivering task by using the ID-SIR system The individual agrees to publish his photo. A — H Subjects 1—8. In the second and third columns of Table 2 , the average round number M a and the corresponding average time of P signal recognition t P of each subject are presented, respectively. The fourth and fifth columns list eight average time and average accuracy when users completed 10 times drinking tasks. It is worth pointing out that a drinking task includes delivering process and returning process.
- Intelligent Robotics - Engineering Product Development (EPD)?
- Navigation Area.
- Search form!
- 30.119 Intelligent Robotics;
- Design, fabrication and control of soft robots?
In other words, the time cost of a drinking task includes time periods of P signal recognition, object recognition, object localization, and robot operating. As seen from Table 3 that the mean time of P signal recognition is 5. The average accuracy of 10 times drinking tasks controlling the robot manipulator is Table 3 shows the evaluation of the eight subjects to the proposed ID-SIR system after their experiences. These four high scores demonstrate well that the ID-SIR system is very capable and suitable for the assistive drinking tasks.
The scores of Q5 and Q6 reaching to 3. The 4. In order to highlight the advantages and effectiveness of our system, comparisons among existing BMI-based assistive robotic systems and the ID-SIR system are shown in Table 4. Table 4. As shown in Table 4 , a robotic system in Hochberg et al. Later, a female patient with tetraplegia and anarthria was assisted by the system to drink coffee from a bottle speeding more than 85 s each time with However, this system is inefficient and cause great burden on users. Users have to concentrate continually to control the robot manipulator in real time.
The robotic assistive systems in Wang et al. However, they did not consider about detection or assistive drinking problems. However, the system took almost 2 min to complete one task and the color-based classifier for recognizing a specific colorful plastic cup limited the choices for users. In order to overcome the deficits of existing systems listed in Table 4 , our ID-ARR system applies non-invasive Pbased BMI technology to complete the assistive drinking task automatically and reduce great burdens on users. It only requires users to have short time training at the beginning and concentrate only two times to give out commands during each whole drinking process.
Besides, two-times region growing algorithm and convoluted neural network are applied to recognize and locate the object, which are more effective and generalizable in practical environments. In this paper, an intention-driven semi-autonomous intelligent robotic ID-SIR system has been designed. The system is composed of a Pbased brain—computer interface BMI subsystem, a robot manipulator and an automatic-visual-inspection subsystem. It can detect a desired object and deliver it to the mouth of the user. In order to detect the intention of the user, a self-adaption Bayesian linear discriminant analysis algorithm has been exploited and performed to improve training efficiency and accuracy.
Besides, a novel two-times region growing algorithm has been proposed to obtain the complete object. The experiment results have verified the capability of the proposed ID-SIR system and the corresponding algorithms.
LabVIEW Robotics Module
Further studies will be conducted to set up the system on a mobile platform and investigate the practical performance on patients. All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Akram, F. An efficient word typing pBCI system using a modified T9 interface and random forest classifier. Bishop, C. New York: Springer-Verlag Inc. Google Scholar. Carlson, T. Brain-controlled wheelchairs: a robotic architecture. IEEE Robot. Chang, M. Methods , — Chapin, J.
Real-time control of a robot arm using simultaneously recorded neuronsin the motor cortex. Ferracuti, F. He, W. IEEE Trans. Man Cybern. Hochberg, L. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm.
Nature , — Hoffmann, U. An efficient pbased brain-computer interface for disabled subjects. Katyal, K. Kim, D. How autonomy impacts performance and satisfaction: results from a study with spinal cord injured subjects using an assistive robot. A Syst. Kim, S. Point-and-click cursor control with an intracortical neural interface system by humans with tetraplegia.
Neural Syst. Lenhardt, A. An adaptive pbased online brain-computer interface. Li, Y. Lijing, M. Mackay, D. Bayesian interpolation. Neural Comput. Onose, G. On the feasibility of using motor imagery EEG-based brain-computer interface in chronic tetraplegics for assistive robotic arm control: a clinical test and long-term post-trial follow-up. Spinal Cord 50, Prezmarcos, D.
Writing through a robot: a proof of concept for a brain-machine interface. Medical Eng. Simbolon, A. Susko, T. MIT-Skywalker: a novel gait neurorehabilitation robot for stroke and cerebral palsy. Townsend, G.
A novel pbased brain-computer interface stimulus presentation paradigm: moving beyond rows and columns. Wang, H. Wu, Q. Yu, T. Zhang, Z.
Graduate Certificate in Intelligent Robotic Systems
A flexible new technique for camera calibration. Pattern Anal. Keywords: assistive robot, neural network, semi-autonomous control, brain—machine interface, object recognition and localization. The use, distribution or reproduction in other forums is permitted, provided the original author s or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice.
No use, distribution or reproduction is permitted which does not comply with these terms. Toggle navigation. Login Register Login using. You can login by using one of your existing accounts. We will be provided with an authorization token please note: passwords are not shared with us and will sync your accounts for you. This means that you will not need to remember your user name and password in the future and you will be able to login with the account you choose to sync, with the click of a button. Forgot Password?
Suggest a Research Topic. Introduction Independent living is essential for the patients with motor deficit due to stroke, spinal cord injures, etc. Robot Control For the severely disabled patients, the less brain burden the system brought in, the better patients may feel. Object Perception Object perception is realized by embedding with the computer vision. Before ending this section, the main contributions of this paper lie as below. Edited by Bishnu Pal.
Edited by Alexander Kokorin. Edited by Theophanides Theophile. Edited by Kresimir Delac. Edited by Sergey Mikhailov. Edited by Sylvie Manguin. Edited by Felix Chan. Edited by David Pozo. Published: February 3rd DOI: Podder and Yan Yu Open access peer-reviewed 6.
Dai Open access peer-reviewed 8. Tavera Open access peer-reviewed Urbanic Open access peer-reviewed Bhanu Prasad Open access peer-reviewed Pedrosa Open access peer-reviewed Edited Volume and chapters are indexed in. Open access peer-reviewed 1. Open access peer-reviewed 2. Open access peer-reviewed 3. Open access peer-reviewed 4. Open access peer-reviewed 5. Open access peer-reviewed 6. Open access peer-reviewed 7. Open access peer-reviewed 8.