Showing 1–1 of 1 results for author: McVicar, J
-
Benchmarking Simulated Robotic Manipulation through a Real World Dataset
Authors:
Jack Collins,
Jessie McVicar,
David Wedlock,
Ross Brown,
David Howard,
Jürgen Leitner
Abstract:
We present a benchmark to facilitate simulated manipulation; an attempt to overcome the obstacles of physical benchmarks through the distribution of a real world, ground truth dataset. Users are given various simulated manipulation tasks with assigned protocols having the objective of replicating the real world results of a recorded dataset. The benchmark comprises of a range of metrics used to ch…
▽ More
We present a benchmark to facilitate simulated manipulation; an attempt to overcome the obstacles of physical benchmarks through the distribution of a real world, ground truth dataset. Users are given various simulated manipulation tasks with assigned protocols having the objective of replicating the real world results of a recorded dataset. The benchmark comprises of a range of metrics used to characterise the successes of submitted environments whilst providing insight into their deficiencies. We apply our benchmark to two simulation environments, PyBullet and V-Rep, and publish the results. All materials required to benchmark an environment, including protocols and the dataset, can be found at the benchmarks' website https://research.csiro.au/robotics/manipulation-benchmark/.
△ Less
Submitted 26 November, 2019; v1 submitted 4 November, 2019;
originally announced November 2019.
