ISLAB/CAISR - User contributions [en]

On control of robots in remote workspaces using lasers

2017-10-10T07:51:01Z

BjornAstrand: Created page with "{{StudentProjectTemplate |Summary=Supervised autonomy for controlling robots at remote locations. Lasers and ultrasonic ranging to be used. |Keywords=Mechatronics, laser point..."

{{StudentProjectTemplate
|Summary=Supervised autonomy for controlling robots at remote locations. Lasers and ultrasonic ranging to be used.
|Keywords=Mechatronics, laser pointers, ultrasonic ranging, Supervised Autonomy, “man-in-the-feedback-loop”, IMU.
|TimeFrame=October 2017 to June 2018, with possible extension to September 2018
|References=In the video clip, http://mobile-robotics.com/ragvald.php , a rate gyro is used to stabilized the heading of a mini UGV prototype.

|Supervisor=Björn Åstrand,
|Level=Master
|Status=Open
}}
Objective and research questions
The goal is to build a mobile robot to test how robots can be controlled relative to objects in a remote work space. Sensors aboard the robot are laser, rate gyros/accelerometers and ultrasonic ranger. The commands of the driver to the robot should be done remotely using a navigation camera and laser pointers. The long-term goals of the project is to “understand fundamental limits” for tele-robotics. Models & tests.

Background
To illustrate the technology, consider the video sequence: http://www.mobile-robotics.com/ugv/
It shows a rate gyro stabilized 4-wheel min-ATV that runs on the surface of gravel and stones. The direction of the vehicle is determined by the driver by directing the navigation camera. Gyro-switched 90° turns and U-turns are also displayed. Second video illustrates instability of type ”phase-windup”.

The control of the robot is to be with three laser pointers on board the robot:
- a laser fix on the robot that gives the heading of the robot,
- a laser pointer where the robot indicates where it will drive with current control. This is a virtual steering wheel.
- a laser pointer that the driver uses to specify where the robot should move. Feedback to the driver is via a forward looking navigation camera.

The robot's hardware is two servo motors with wheel coders. This motor package will be the "driving front axle" for the mobile robot. Motors with wheel coders and drive electronics ready. Raspberry Pi with sensor card (SensorHat) is placed on the robot with the camera.

Project steps

WP1: Litterature Review and Basic motion – getting started
The two motors form a “wheel chair” robot. First test is to driven along an 8-shaped trajectory. Laser pointer on RC-servo to be used as a virtual steering wheel. Caster wheel for balance.

WP2: Car type of robot
The pair of motors above will be the front axis for a car type of robot.

WP3: Final tests. Reversing with a trailer behind car robot
Driver to use a laser pointer for control. Faster than manual control? Learning the geometry by driving forward. Then reversing. Applications for vehicles in mines, in forests, at construction sites etc.

Safety assurance for human in automated crane environment

2017-09-28T10:56:13Z

BjornAstrand: Created page with "{{StudentProjectTemplate |Summary=safety assurance for human in automated crane environment |Keywords=Robot perception |TimeFrame=October 2017 to June 2018, with possible exte..."

{{StudentProjectTemplate
|Summary=safety assurance for human in automated crane environment
|Keywords=Robot perception
|TimeFrame=October 2017 to June 2018, with possible extension to September 2018
|Supervisor=Björn Åstrand, Josef Bigun
|Level=Master
|Status=Open
}}
A central issue for automated systems is safety; the system must not harm humans or damage objects in the environment. Safety concerns have become more and more important as the use of automated systems has spread and advances in sensor technology, sensor integration, and object detection and avoidance have been more widely adapted.

The aim of the project is to develop algorithms and methods for safety assurance for human in automated crane environment.

SAS2

2017-09-22T14:47:26Z

BjornAstrand:

{{ResearchProjInfo
|Title=SAS2 – Situation Aware Safety Systems
|ContactInformation=Björn Åstrand
|Description=The goal with this project is to develop a safety system that better can handle the complexity in the environments where AGV systems operates.

The approach is to use 3D perception along with methods for detect, track and identify objects in the environment, such as actions of moving objects can be foreseen and concerns can be made based on objects identities (humans and other trucks/AGVs).
|ProjectResponsible=Björn Åstrand
|ProjectStart=2016/01/01
|ProjectEnd=2018/12/31
|ApplicationArea=Mechatronics
}}
{{AssignPublicationProject
|Publication=Hedenberg, Klas & Åstrand, Björn, 3D Sensors on Driverless Trucks for Detection of Overhanging Objects in the Pathway, Autonomous Industrial Vehicles : From the Laboratory to the Factory Floor., s. 41-56, 2016
}}
{{AssignProjPartner
|projectpartner=Volvo Group - Trucks Technology - Advanced Technology and Research
}}
{{AssignProjPartner
|projectpartner=Fotonic AB
}}
{{AssignProjPartner
|projectpartner=Toyota Material Handling Europe
}}
{{AssignProjPartner
|projectpartner=Kollmorgen Automation AB
}}
Research and development are done in three different areas:

Adaptive Warning fields: How to automatically construct and represent a volume of interests for safe traverse of the driverless truck? What sensors to use and how to integrate information from different sources, e.g. other sensors and maps (multi-layer)?

Tracking, Prediction, Identification and Reasoning: How to improve the performance of a safety system of a driverless truck by leveraging the system awareness by incorporating new functionalities as identification and tracking of objects/agents?

Benchmarking: How to evaluate and benchmark the performance of a safety system solution on a driverless truck for object detection and protection?

SAS2

2017-09-22T14:46:31Z

BjornAstrand: Created page with "{{ResearchProjInfo |Title=SAS2 – Situation Aware Safety Systems |ContactInformation=Björn Åstrand |Description=The goal with this project is to develop a safety system tha..."

{{ResearchProjInfo
|Title=SAS2 – Situation Aware Safety Systems
|ContactInformation=Björn Åstrand
|Description=The goal with this project is to develop a safety system that better can handle the complexity in the environments where AGV systems operates.

The approach is to use 3D perception along with methods for detect, track and identify objects in the environment, such as actions of moving objects can be foreseen and concerns can be made based on objects identities (humans and other trucks/AGVs).

|ProjectResponsible=Björn Åstrand
|ApplicationArea=Mechatronics
}}
{{AssignPublicationProject
|Publication=Hedenberg, Klas & Åstrand, Björn, 3D Sensors on Driverless Trucks for Detection of Overhanging Objects in the Pathway, Autonomous Industrial Vehicles : From the Laboratory to the Factory Floor., s. 41-56, 2016
}}
{{AssignProjPartner
|projectpartner=Volvo Group - Trucks Technology - Advanced Technology and Research
}}
{{AssignProjPartner
|projectpartner=Fotonic AB
}}
{{AssignProjPartner
|projectpartner=Toyota Material Handling Europe
}}
{{AssignProjPartner
|projectpartner=Kollmorgen Automation AB
}}
Research and development are done in three different areas:

Adaptive Warning fields: How to automatically construct and represent a volume of interests for safe traverse of the driverless truck? What sensors to use and how to integrate information from different sources, e.g. other sensors and maps (multi-layer)?

Tracking, Prediction, Identification and Reasoning: How to improve the performance of a safety system of a driverless truck by leveraging the system awareness by incorporating new functionalities as identification and tracking of objects/agents?

Benchmarking: How to evaluate and benchmark the performance of a safety system solution on a driverless truck for object detection and protection?

Adaptive warning field system

2017-09-22T14:30:03Z

BjornAstrand:

{{StudentProjectTemplate
|Summary=Adaptive warning field system
|Programme=Mobile and Autonomous Systems
|Keywords=3D perception, mapping,
|TimeFrame=January 2017 until June 2017, with possible extension until September 2017
|References=SAS2-project, http://islab.hh.se/mediawiki/SAS2
ROS - Robot Operating System, http://www.ros.org/
OpenCv - http://opencv.org/

Nemati, Hassan, Åstrand, Björn (2014). Tracking of People in Paper Mill Warehouse Using Laser Range Sensor. 2014 UKSim-AMSS 8th European Modelling Symposium, EMS 2014, Pisa, Italy, 21-23 October, 2014.

Power, P. Wayne, and Johann A. Schoonees. "Understanding background mixture models for foreground segmentation." Proceedings image and vision computing New Zealand. Vol. 2002. 2002.
|Prerequisites=Image analysis, programming skills (preferably C++ or Python)
|Supervisor=Björn Åstrand,
|Level=Master
|Status=Finished
}}
A central issue for robots and automated guided vehicles (AGVs) is safety; the robot or AGV must not harm humans or damage objects in the environment. Safety concerns have become more and more important as the use of AGVs has spread and advances in sensor technology, sensor integration, and object detection and avoidance have been more widely adapted. Today’s safety systems only work in 2D and consist of a static protection field (in size) and a speed adaptive warning field – higher speed - larger field. However, mostly the size of the warning field is hardcoded and heavily bounded to the AGV route. The setup of such system is costly and takes a long time to adjust for proper and efficient operation.

The goal with this project [as a subset of SAS2 project] is to develop a safety system based on an adaptive warning field that autonomous learns the foreground (static and dynamic obstacles) and background model (static objects). The approach is to use 3D perception along with methods that combine a method that continuously segments background from foreground model (e.g. optical flow approach or Gaussian mixture models), with a method that uses a geometric map to filter out the foreground model. A challenge is how to update/learn the geometric map.

Preferable the solutions are designed as ROS-packages (or, c++, python, matlab -code).

Resources: Facilities for data logging, cameras, depth sensor, data logging equipment, data set from warehouse and collaboration with industrial partners.

RQ: How to automatically construct and represent a volume of interests for safe traverse of the driverless truck? What sensors to use and how to integrate information from different sources, e.g. other sensors and maps? How to detect the difference between foreground (static and dynamic obstacles) or background model (static objects).

WP1: Literature review and construction of a dataset.
WP2: Develop methods for estimation of background /foreground model and an adaptive warning field system.
WP3: Evaluation of the feasibility of the system and comparison with existing systems.
WP4: [bonus] conference publication (ETFA, ECMR, TAROS)

Deliverable: an implementation and demonstration of the developed system for adaptive warning field system using data acquired in a real warehouse or mine.

Agent and object detection and classification in a warehouse setting

2017-09-22T14:29:31Z

BjornAstrand:

{{StudentProjectTemplate
|Summary=Detection, and classification of different agents (manual driven forklift trucks, other robots, humans) and objects (such as pallets) in a warehouse environment
|Keywords=Robot perception, feature extraction, machine learning, classification
|TimeFrame=October 2017 to June 2018, with possible extension to September 2018
|References=SAS2-project, http://islab.hh.se/mediawiki/SAS2

ROS - Robot Operating System, http://www.ros.org/

OpenCv - http://opencv.org/

Lalonde, Jean-Francois; Vandapel, Nicolas; Huber, Daniel; Hebert, Martial; Natural terrain classification using three-dimensional ladar data for ground robot mobility, Journal of Field Robotics, Vol. 23, No. 10, pp. 839 - 861, November, 2006

Mosberger, Rafael; Vision-based human detection from mobile machinery in industrial environments, Thesis, Örebro University, Sweden, 2016

Saarinen, Jari P.; Andreasson, Henrik; Stoyanov, Todor; Lilienthal, Achim J.; 3D normal distributions transform occupancy maps: An efficient representation for mapping in dynamic environments, The International Journal of Robotics Research, Vol 32, Issue 14, pp. 1627 – 1644, 2013

|Prerequisites=Programming skills (preferably C++ or Python)
|Supervisor=Björn Åstrand, Naveed Muhammad
|Level=Master
|Status=Open
}}
A central issue for robots and automated guided vehicles (AGVs) is safety; the robot or AGV must not harm humans or damage objects in the environment. Safety concerns have become more and more important as the use of AGVs has spread and advances in sensor technology, sensor integration, and object detection and avoidance have been more widely adapted. Today’s safety systems don’t consider the identity of different agents in close proximity to the robot and AGV.

The goal of this project [as a subset of SAS2 project] is to develop a method for detection and identification of different agents and objects (such as other AGVs, manually driven forklift trucks, humans, pallets etc.) present in a warehouse environment. The idea is to investigate if a robot, using perception data it acquires through its exteroceptive sensors such as cameras and lidars, can detect and identify different categories of agents and objects present in its environment. This includes segmentation of perception data, extraction of features, and implementation of classification techniques. Preferably the solutions are designed as ROS-packages (or, c++, python, matlab-code).

Resources: Facilities for data logging, cameras, depth sensor, data logging equipment, data set from warehouses and collaboration with industrial partners.

Research Question: How to detect and classify different agents and objects present in a warehouse environment into different categories.

WP1: Literature review. WP2: Segmentation and feature extraction from perception data.
WP3: Classification of different agents/objects using supervised and/or unsupervised learning methods.
WP4: [bonus] conference publication (ICRA, IROS, ETFA, ECMR, TAROS)

Deliverable: an implementation and demonstration of the developed system for detection and classification of agents/objects using data from warehouse environments or mines.

Agent and object detection and classification in a warehouse setting

2017-09-22T14:28:59Z

BjornAstrand: Created page with "{{StudentProjectTemplate |Summary=Detection, and classification of different agents (manual driven forklift trucks, other robots, humans) and objects (such as pallets) in a wa..."

{{StudentProjectTemplate
|Summary=Detection, and classification of different agents (manual driven forklift trucks, other robots, humans) and objects (such as pallets) in a warehouse environment
|Keywords=Robot perception, feature extraction, machine learning, classification
|TimeFrame=October 2017 to June 2018, with possible extension to September 2018
|References=SAS2-project, http://islab.hh.se/mediawiki/SAS2
ROS - Robot Operating System, http://www.ros.org/
OpenCv - http://opencv.org/
Lalonde, Jean-Francois; Vandapel, Nicolas; Huber, Daniel; Hebert, Martial; Natural terrain classification using three-dimensional ladar data for ground robot mobility, Journal of Field Robotics, Vol. 23, No. 10, pp. 839 - 861, November, 2006
Mosberger, Rafael; Vision-based human detection from mobile machinery in industrial environments, Thesis, Örebro University, Sweden, 2016
Saarinen, Jari P.; Andreasson, Henrik; Stoyanov, Todor; Lilienthal, Achim J.; 3D normal distributions transform occupancy maps: An efficient representation for mapping in dynamic environments, The International Journal of Robotics Research, Vol 32, Issue 14, pp. 1627 – 1644, 2013

|Prerequisites=Programming skills (preferably C++ or Python)
|Supervisor=Björn Åstrand, Naveed Muhammad
|Level=Master
|Status=Open
}}
A central issue for robots and automated guided vehicles (AGVs) is safety; the robot or AGV must not harm humans or damage objects in the environment. Safety concerns have become more and more important as the use of AGVs has spread and advances in sensor technology, sensor integration, and object detection and avoidance have been more widely adapted. Today’s safety systems don’t consider the identity of different agents in close proximity to the robot and AGV.

The goal of this project [as a subset of SAS2 project] is to develop a method for detection and identification of different agents and objects (such as other AGVs, manually driven forklift trucks, humans, pallets etc.) present in a warehouse environment. The idea is to investigate if a robot, using perception data it acquires through its exteroceptive sensors such as cameras and lidars, can detect and identify different categories of agents and objects present in its environment. This includes segmentation of perception data, extraction of features, and implementation of classification techniques. Preferably the solutions are designed as ROS-packages (or, c++, python, matlab-code).

Resources: Facilities for data logging, cameras, depth sensor, data logging equipment, data set from warehouses and collaboration with industrial partners.

Research Question: How to detect and classify different agents and objects present in a warehouse environment into different categories.

WP1: Literature review. WP2: Segmentation and feature extraction from perception data.
WP3: Classification of different agents/objects using supervised and/or unsupervised learning methods.
WP4: [bonus] conference publication (ICRA, IROS, ETFA, ECMR, TAROS)

Deliverable: an implementation and demonstration of the developed system for detection and classification of agents/objects using data from warehouse environments or mines.

Adaptive warning field system

2017-09-22T14:25:17Z

BjornAstrand:

{{StudentProjectTemplate
|Summary=Adaptive warning field system
|Programme=Mobile and Autonomous Systems
|Keywords=3D perception, mapping,
|TimeFrame=January 2017 until June 2017, with possible extension until September 2017
|References=SAS2-project, http://islab.hh.se/mediawiki/SAS2
ROS - Robot Operating System, http://www.ros.org/
OpenCv - http://opencv.org/

Nemati, Hassan, Åstrand, Björn (2014). Tracking of People in Paper Mill Warehouse Using Laser Range Sensor. 2014 UKSim-AMSS 8th European Modelling Symposium, EMS 2014, Pisa, Italy, 21-23 October, 2014.

Power, P. Wayne, and Johann A. Schoonees. "Understanding background mixture models for foreground segmentation." Proceedings image and vision computing New Zealand. Vol. 2002. 2002.
|Prerequisites=Image analysis, programming skills (preferably C++ or Python)
|Supervisor=Björn Åstrand,
|Level=Master
|Status=Closed
}}
A central issue for robots and automated guided vehicles (AGVs) is safety; the robot or AGV must not harm humans or damage objects in the environment. Safety concerns have become more and more important as the use of AGVs has spread and advances in sensor technology, sensor integration, and object detection and avoidance have been more widely adapted. Today’s safety systems only work in 2D and consist of a static protection field (in size) and a speed adaptive warning field – higher speed - larger field. However, mostly the size of the warning field is hardcoded and heavily bounded to the AGV route. The setup of such system is costly and takes a long time to adjust for proper and efficient operation.

The goal with this project [as a subset of SAS2 project] is to develop a safety system based on an adaptive warning field that autonomous learns the foreground (static and dynamic obstacles) and background model (static objects). The approach is to use 3D perception along with methods that combine a method that continuously segments background from foreground model (e.g. optical flow approach or Gaussian mixture models), with a method that uses a geometric map to filter out the foreground model. A challenge is how to update/learn the geometric map.

Preferable the solutions are designed as ROS-packages (or, c++, python, matlab -code).

Resources: Facilities for data logging, cameras, depth sensor, data logging equipment, data set from warehouse and collaboration with industrial partners.

RQ: How to automatically construct and represent a volume of interests for safe traverse of the driverless truck? What sensors to use and how to integrate information from different sources, e.g. other sensors and maps? How to detect the difference between foreground (static and dynamic obstacles) or background model (static objects).

WP1: Literature review and construction of a dataset.
WP2: Develop methods for estimation of background /foreground model and an adaptive warning field system.
WP3: Evaluation of the feasibility of the system and comparison with existing systems.
WP4: [bonus] conference publication (ETFA, ECMR, TAROS)

Deliverable: an implementation and demonstration of the developed system for adaptive warning field system using data acquired in a real warehouse or mine.

Data Mining In a Warehouse Inventory

2017-09-22T14:24:11Z

BjornAstrand:

{{StudentProjectTemplate
|Summary=A study of feature selection and distance measures for clustering big number of categories (>1000) and novelty detection in warehouse environment.
|Programme=Mobile and Autonomous Systems
|Keywords=object recognition, signal processing, feature selection, unsupervised clustering, large scale many class classification, data mining.
|TimeFrame= October 2017 to June 2018, with possible extension to September 2018
|References=Zeynep Akata, Florent Perronnin, Zaid Harchaoui, Cordelia Schmid. Good Practice in Large-Scale Learning for Image Classi cation. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2014, 36 (3), pp.507-520.<10.1109/TPAMI.2013.146>.<hal-00835810>

Florent Perronnin, Zeynep Akata, Zaid Harchaoui, Cordelia Schmid. Towards Good Practice in Large-Scale Learning for Image Classification. CVPR 2012 - IEEE Computer Vision and Pattern Recognition, Jun 2012, Providence (RI), United States. IEEE, pp.3482-3489, 2012,<10.1109/CVPR.2012.6248090>.<hal-00690014>

Raphael Puget, Nicolas Baskiotis, Patrick Gallinari. Sequential Dynamic Classi cation for Large Scale Multi-class Problems. Extreme Classi cation Workshop at ICML, Jul 2015, Lille,France. 2015.<hal-01207428>
|Prerequisites=Programming skills, Machine Learning, Computer Vision, Data Mining.
|Supervisor= Björn Åstrand,
|Level=Master
|Status=Open
}}
;Background
: Object recognition in problems entailing many classes is a challenging task. One example of such problems is the inventory list of warehouse. The inventory of typical warehouses often contain up to 10K different classes of objects. In this project we intend to develop inventory list maintanance method that is able to learn the number of classes of objects and train a classifier from the data. Towards this objective, we employ the background knowledge (e.g. from the Warehouse Management System - WMS) to constrain the complexity of the problem.

;Objectives
: To develop an incremental clustering algorithm, that learns new classes of object through novelty detection. The background knowledge (e.g. WMS), which is an important source of information for constraining the problem, should be exploit towards a more robust system design.

;Research Questions
: What is the optimal feature space and clustering technique for object identification in large-scale many classes? How to use background knowledge as clustering cues? How to employ novelty detection for learning new classes incrementally?

;Setup
: dataset from a real-world warehouses.

Dynamic Objects Detection and Tracking

2017-09-22T14:22:48Z

BjornAstrand:

{{StudentProjectTemplate
|Summary=Dynamic Objects Detection and Tracking in Warehouses, Using 3D Sensors.
|Programme=Mobile and Autonomous Systems
|Keywords=3D Sensor, Point Cloud, Obstacle Detection, Obstacle Tracking, Obstacle Avoidance.
|TimeFrame=October 2017 to June 2018, with possible extension to September 2018
|References=Petrovskaya, Anna, and Sebastian Thrun. "Model based vehicle detection and tracking for autonomous urban driving." Autonomous Robots 26.2-3 (2009): 123-139.

Wojke, N.; Haselich, M., "Moving vehicle detection and tracking in unstructured environments," Robotics and Automation (ICRA), 2012 IEEE International Conference on , vol., no., pp.3082,3087, 14-18 May 2012.

Moras, J.; Cherfaoui, V.; Bonnifait, P., "A lidar perception scheme for intelligent vehicle navigation," Control Automation Robotics & Vision (ICARCV), 2010 11th International Conference on , vol., no., pp.1809,1814, 7-10 Dec. 2010

Golovinskiy, Aleksey, Vladimir G. Kim, and Thomas Funkhouser. "Shape-based recognition of 3D point clouds in urban environments." Computer Vision, 2009 IEEE 12th International Conference on. IEEE, 2009.

Granstrom, K.; Lundquist, C.; Gustafsson, F.; Orguner, U., "Random Set Methods: Estimation of Multiple Extended Objects," Robotics & Automation Magazine, IEEE , vol.21, no.2, pp.73,82, June 2014

Data Association and Tracking a survey RoboEarth.

Rusu, Radu Bogdan, and Steve Cousins. "3d is here: Point cloud library (pcl)." Robotics and Automation (ICRA), 2011 IEEE International Conference on. IEEE, 2011.

Brostow, Gabriel J., et al. "Segmentation and recognition using structure from motion point clouds." Computer Vision–ECCV 2008. Springer Berlin Heidelberg, 2008. 44-57.

Drost, Bertram, et al. "Model globally, match locally: Efficient and robust 3D object recognition." Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on. IEEE, 2010.

Biasotti, S. ; Falcidieno, B. ; Giorgi, D. ; Spagnuolo, M. “Mathematical Tools for Shape Analysis and Description”, 2014, Publisher :Morgan & Claypool, Edition:1, ISBN:1627053646

Börcs, Attila, et al. "A Model-based Approach for Fast Vehicle Detection in Continuously Streamed Urban LIDAR Point Clouds." (2014).
|Prerequisites=Familiarity with filtering techniques (eg. EKF) for mobile robots localization, Image analysis, programming skills (preferably C++ or Python).
|Supervisor=Björn Åstrand, Naveed Muhammad,
|Level=Master
|Status=Open
}}
; Concise Description
: This project [as a subset of AIMS project], targets the automation of lift trucks in warehouse environments. Operating automatic guided vehicles in this particular environment is challenging due to the high expected throughput and consequently high traffic. Lift trucks are heavy vehicles operating with relatively high speed in an environment where neither the trucks, nor the humans are well protected as a regular urban traffic. This calls for a extra security measure and cautious decisions. Collision avoidance is an essential skill for mobile robots to guarantee a safe operation in a workspace shared with humans. This project focuses on detection and tracking of dynamic objects in order to avoid collision.

; Objective
: To reliably detect, segment, and track dynamic objects (eg. humans and lift trucks) from a 3D point cloud, acquired by the means of a 3D sensor mounted on a mobile robot, in a highly structured environment (warehouse).

; Research Questions
: What is the optimal sensor configuration to minimize the blind spots, data losses due to sensor deficiency, and consequently improving the detection accuracy?
: How to exploit the assumption of structured environment to improve tracking?
: How the background knowledge of agent types (humans, manually driven trucks and auto-guided trucks) and their behaviour models could improve the tracking?

; Preliminary Plan
* startup: literature review and data acquisition
* point cloud manipulation, object segmentation, scene understanding.
* filtering and tracking.
* [bonus] object recognition

;Deliverable
: An implementation and demonstration of the developed method for detection and tracking of the moving obstacle over the real data acquired in a real warehouse.

;Bonus
: conference publication (ETFA, ECMR, TAROS)

Model behaviour of agents in a warehouse setting

2017-09-22T14:21:06Z

BjornAstrand:

{{StudentProjectTemplate
|Summary=Modelling the behaviour of agents (manual driven forklift trucks, other robots, humans etc.) in a warehouse environment
|Programme=Mobile and Autonomous Systems
|Keywords=Machine learning, robot perception, modelling, simulation
|TimeFrame=October 2017 to June 2018, with possible extension to September 2018
|References=SAS2-project, http://islab.hh.se/mediawiki/SAS2

ROS - Robot Operating System, http://www.ros.org/

OpenCv - http://opencv.org/

Lidström, Kristoffer; Situation-Aware vehicles – supporting the next generation of cooperative traffic system, PhD thesis, Örebro university, 2012.

Lundström, Jens; Järpe, Eric; Verikas, Antanas; Detecting and exploring deviating behaviour of smart home residents, Expert systems with applications., 55, s. 429-440, 2016

Lidström, Kristoffer; Larsson, Tony; Act normal: using uncertainty about driver intentions as a warning criterion, 16th World Congress on Intelligent Transportation Systems (ITS WC), 21-25 September, 2009, Stockholm, Sweden

Lidström, Kristoffer; Model-based Estimation of Driver Intentions Using Particle Filtering, Proceedings of the 11th International IEEE Conference on Intelligent Transportation Systems Beijing, China, October 12-15, 2008

|Prerequisites=Programming skills (preferably C++ or Python)
|Supervisor=Björn Åstrand, Naveed Muhammad
|Level=Master
|Status=Open
}}
A central issue for robots and automated guided vehicles (AGVs) is safety; the robot or AGV must not harm humans or damage objects in the environment. Safety concerns have become more and more important as the use of AGVs has spread and advances in sensor technology, sensor integration, and object detection and avoidance have been more widely adapted. Today’s safety systems don’t consider the behaviour or the identity of different agents in close proximity to the robot and AGV.

The goal of this project [as a subset of SAS2 project] is to develop a method to model behaviour of different agents (manual driven forklift trucks, AGVs, humans) in a warehouse setting and thus use that for predicting behaviour in different scenarios. The idea is to investigate if different agents can be automatically divided into categories depending on their behaviour and how that information can be used to foresee actions of different agents.

The study also includes construction of a simulator where the validity of the developed method for behaviour modelling, is evaluated. Real data can also be used to verify the system. Preferably the solutions are designed as ROS-packages (or, c++, python, matlab-code).

Resources: Facilities for data logging, cameras, depth sensor, data logging equipment, data set from warehouses and collaboration with industrial partners.

Research Question: How to learn behaviour of different categories of agents (e.g. manual driven trucks, humans, AGVs) especially if they are only partially observed in time. How to represent behaviour of an agent?

WP1: Literature review and construction of a dataset.
WP2: Develop methods for modelling behaviour of agents in a warehouse setting.
WP3: Comparison study and development of improvements of the different systems.
WP4: [bonus] conference publication (ICRA, IROS, ETFA, ECMR, TAROS)

Deliverable: an implementation and demonstration of the developed system for modelling behaviour of agents using simulated data and data acquired in a real warehouse environments or mines.

Model behaviour of agents in a warehouse setting

2017-09-22T14:19:45Z

BjornAstrand:

{{StudentProjectTemplate
|Summary=Modelling the behaviour of agents (manual driven forklift trucks, other robots, humans etc.) in a warehouse environment
|Programme=Mobile and Autonomous Systems
|Keywords=Machine learning, robot perception, modelling, simulation
|TimeFrame=October 2017 to June 2018, with possible extension to September 2018
|References=SAS2-project, http://islab.hh.se/mediawiki/SAS2
ROS - Robot Operating System, http://www.ros.org/
OpenCv - http://opencv.org/
Lidström, Kristoffer; Situation-Aware vehicles – supporting the next generation of cooperative traffic system, PhD thesis, Örebro university, 2012.
Lundström, Jens; Järpe, Eric; Verikas, Antanas; Detecting and exploring deviating behaviour of smart home residents, Expert systems with applications., 55, s. 429-440, 2016
Lidström, Kristoffer; Larsson, Tony; Act normal: using uncertainty about driver intentions as a warning criterion, 16th World Congress on Intelligent Transportation Systems (ITS WC), 21-25 September, 2009, Stockholm, Sweden
Lidström, Kristoffer; Model-based Estimation of Driver Intentions Using Particle Filtering, Proceedings of the 11th International IEEE Conference on Intelligent Transportation Systems Beijing, China, October 12-15, 2008

|Prerequisites=Programming skills (preferably C++ or Python)
|Supervisor=Björn Åstrand, Naveed Muhammad
|Level=Master
|Status=Open
}}
A central issue for robots and automated guided vehicles (AGVs) is safety; the robot or AGV must not harm humans or damage objects in the environment. Safety concerns have become more and more important as the use of AGVs has spread and advances in sensor technology, sensor integration, and object detection and avoidance have been more widely adapted. Today’s safety systems don’t consider the behaviour or the identity of different agents in close proximity to the robot and AGV.

The goal of this project [as a subset of SAS2 project] is to develop a method to model behaviour of different agents (manual driven forklift trucks, AGVs, humans) in a warehouse setting and thus use that for predicting behaviour in different scenarios. The idea is to investigate if different agents can be automatically divided into categories depending on their behaviour and how that information can be used to foresee actions of different agents.

The study also includes construction of a simulator where the validity of the developed method for behaviour modelling, is evaluated. Real data can also be used to verify the system. Preferably the solutions are designed as ROS-packages (or, c++, python, matlab-code).

Resources: Facilities for data logging, cameras, depth sensor, data logging equipment, data set from warehouses and collaboration with industrial partners.

Research Question: How to learn behaviour of different categories of agents (e.g. manual driven trucks, humans, AGVs) especially if they are only partially observed in time. How to represent behaviour of an agent?

WP1: Literature review and construction of a dataset.
WP2: Develop methods for modelling behaviour of agents in a warehouse setting.
WP3: Comparison study and development of improvements of the different systems.
WP4: [bonus] conference publication (ICRA, IROS, ETFA, ECMR, TAROS)

Deliverable: an implementation and demonstration of the developed system for modelling behaviour of agents using simulated data and data acquired in a real warehouse environments or mines.