-
Neural Relational Inference with Fast Modular Meta-learning
Authors:
Ferran Alet,
Erica Weng,
Tomás Lozano Pérez,
Leslie Pack Kaelbling
Abstract:
\textit{Graph neural networks} (GNNs) are effective models for many dynamical systems consisting of entities and relations. Although most GNN applications assume a single type of entity and relation, many situations involve multiple types of interactions. \textit{Relational inference} is the problem of inferring these interactions and learning the dynamics from observational data. We frame relatio…
▽ More
\textit{Graph neural networks} (GNNs) are effective models for many dynamical systems consisting of entities and relations. Although most GNN applications assume a single type of entity and relation, many situations involve multiple types of interactions. \textit{Relational inference} is the problem of inferring these interactions and learning the dynamics from observational data. We frame relational inference as a \textit{modular meta-learning} problem, where neural modules are trained to be composed in different ways to solve many tasks. This meta-learning framework allows us to implicitly encode time invariance and infer relations in context of one another rather than independently, which increases inference capacity. Framing inference as the inner-loop optimization of meta-learning leads to a model-based approach that is more data-efficient and capable of estimating the state of entities that we do not observe directly, but whose existence can be inferred from their effect on observed entities. To address the large search space of graph neural network compositions, we meta-learn a \textit{proposal function} that speeds up the inner-loop simulated annealing search within the modular meta-learning algorithm, providing two orders of magnitude increase in the size of problems that can be addressed.
△ Less
Submitted 10 October, 2023;
originally announced October 2023.
-
Impact of Orientation on the Bias of SRAM-Based PUFs
Authors:
Zain Ul Abideen,
Rui Wang,
Tiago Diadami Perez,
Geert-Jan Schrijen,
Samuel Pagliarini
Abstract:
This paper investigates the impact of memory orientation on the bias pattern of SRAM-based PUFs. We designed and fabricated a 65nm CMOS chip that contains eleven SRAM macros that exercise different memory- and chip-level parameters. At the memory level, several parameters passed to the SRAM compiler are considered, including the number of addresses, the number of words, the aspect ratio, and the c…
▽ More
This paper investigates the impact of memory orientation on the bias pattern of SRAM-based PUFs. We designed and fabricated a 65nm CMOS chip that contains eleven SRAM macros that exercise different memory- and chip-level parameters. At the memory level, several parameters passed to the SRAM compiler are considered, including the number of addresses, the number of words, the aspect ratio, and the chosen bitcell. Chip-level decisions are considered during the floorplan, including the location and rotation of each SRAM macro in the testchip. In this study, we conduct a comprehensive analysis of different memory orientations and their effect on the biasing direction. Physical measurements performed on 50 fabricated chips revealed that specific memory orientations, namely R270 and MY90, exhibit a distinct negative biasing direction compared to other orientations. Importantly, this biasing direction remains consistent regardless of memory type, column mux ratio, memory size, or the utilization of SRAMs with different bitcells. Overall, this study highlights the significance of careful physical implementation and memory orientation selection in designing SRAM-based PUFs. Our findings can guide designers in the selection of SRAM memories with properties that make for better PUFs that potentially require less error correction effort to compensate for instability.
△ Less
Submitted 13 August, 2023;
originally announced August 2023.
-
SALSy: Security-Aware Layout Synthesis
Authors:
Mohammad Eslami,
Tiago Perez,
Samuel Pagliarini
Abstract:
Integrated Circuits (ICs) are the target of diverse attacks during their lifetime. Fabrication-time attacks, such as the insertion of Hardware Trojans, can give an adversary access to privileged data and/or the means to corrupt the IC's internal computation. Post-fabrication attacks, where the end-user takes a malicious role, also attempt to obtain privileged information through means such as faul…
▽ More
Integrated Circuits (ICs) are the target of diverse attacks during their lifetime. Fabrication-time attacks, such as the insertion of Hardware Trojans, can give an adversary access to privileged data and/or the means to corrupt the IC's internal computation. Post-fabrication attacks, where the end-user takes a malicious role, also attempt to obtain privileged information through means such as fault injection and probing. Taking these threats into account and at the same time, this paper proposes a methodology for Security-Aware Layout Synthesis (SALSy), such that ICs can be designed with security in mind in the same manner as power-performance-area (PPA) metrics are considered today, a concept known as security closure. Furthermore, the trade-offs between PPA and security are considered and a chip is fabricated in a 65nm CMOS commercial technology for validation purposes - a feature not seen in previous research on security closure. Measurements on the fabricated ICs indicate that SALSy promotes a modest increase in power in order to achieve significantly improved security metrics.
△ Less
Submitted 21 August, 2023; v1 submitted 11 August, 2023;
originally announced August 2023.
-
A Pragmatic Methodology for Blind Hardware Trojan Insertion in Finalized Layouts
Authors:
Alexander Hepp,
Tiago Perez,
Samuel Pagliarini,
Georg Sigl
Abstract:
A potential vulnerability for integrated circuits (ICs) is the insertion of hardware trojans (HTs) during manufacturing. Understanding the practicability of such an attack can lead to appropriate measures for mitigating it. In this paper, we demonstrate a pragmatic framework for analyzing HT susceptibility of finalized layouts. Our framework is representative of a fabrication-time attack, where th…
▽ More
A potential vulnerability for integrated circuits (ICs) is the insertion of hardware trojans (HTs) during manufacturing. Understanding the practicability of such an attack can lead to appropriate measures for mitigating it. In this paper, we demonstrate a pragmatic framework for analyzing HT susceptibility of finalized layouts. Our framework is representative of a fabrication-time attack, where the adversary is assumed to have access only to a layout representation of the circuit. The framework inserts trojans into tapeout-ready layouts utilizing an Engineering Change Order (ECO) flow. The attacked security nodes are blindly searched utilizing reverse-engineering techniques. For our experimental investigation, we utilized three crypto-cores (AES-128, SHA-256, and RSA) and a microcontroller (RISC-V) as targets. We explored 96 combinations of triggers, payloads and targets for our framework. Our findings demonstrate that even in high-density designs, the covert insertion of sophisticated trojans is possible. All this while maintaining the original target logic, with minimal impact on power and performance. Furthermore, from our exploration, we conclude that it is too naive to only utilize placement resources as a metric for HT vulnerability. This work highlights that the HT insertion success is a complex function of the placement, routing resources, the position of the attacked nodes, and further design-specific characteristics. As a result, our framework goes beyond just an attack, we present the most advanced analysis tool to assess the vulnerability of HT insertion into finalized layouts.
△ Less
Submitted 19 August, 2022;
originally announced August 2022.
-
A Security-aware and LUT-based CAD Flow for the Physical Synthesis of eASICs
Authors:
Zain UlAbideen,
Tiago Diadami Perez,
Mayler Martins,
Samuel Pagliarini
Abstract:
Numerous threats are associated with the globalized integrated circuit (IC) supply chain, such as piracy, reverse engineering, overproduction, and malicious logic insertion. Many obfuscation approaches have been proposed to mitigate these threats by preventing an adversary from fully understanding the IC (or parts of it). The use of reconfigurable elements inside an IC is a known obfuscation techn…
▽ More
Numerous threats are associated with the globalized integrated circuit (IC) supply chain, such as piracy, reverse engineering, overproduction, and malicious logic insertion. Many obfuscation approaches have been proposed to mitigate these threats by preventing an adversary from fully understanding the IC (or parts of it). The use of reconfigurable elements inside an IC is a known obfuscation technique, either as a coarse grain reconfigurable block (i.e., eFPGA) or as a fine grain element (i.e., FPGA-like look-up tables). This paper presents a security-aware CAD flow that is LUT-based yet still compatible with the standard cell based physical synthesis flow. More precisely, our CAD flow explores the FPGA-ASIC design space and produces heavily obfuscated designs where only small portions of the logic resemble an ASIC. Therefore, we term this specialized solution an "embedded ASIC" (eASIC). Nevertheless, even for heavily LUT-dominated designs, our proposed decomposition and pin swap** algorithms allow for performance gains that enable performance levels that only ASICs would otherwise achieve. On the security side, we have developed novel template-based attacks and also applied existing attacks, both oracle-free and oracle-based. Our security analysis revealed that the obfuscation rate for an SHA-256 study case should be at least 45% for withstanding traditional attacks and at least 80% for withstanding template-based attacks. When the 80\% obfuscated SHA-256 design is physically implemented, it achieves a remarkable frequency of 368MHz in a 65nm commercial technology, whereas its FPGA implementation (in a superior technology) achieves only 77MHz.
△ Less
Submitted 12 July, 2022;
originally announced July 2022.
-
Hardware Trojan Insertion in Finalized Layouts: From Methodology to a Silicon Demonstration
Authors:
Tiago Perez,
Samuel Pagliarini
Abstract:
Owning a high-end semiconductor foundry is a luxury very few companies can afford. Thus, fabless design companies outsource integrated circuit fabrication to third parties. Within foundries, rogue elements may gain access to the customer's layout and perform malicious acts, including the insertion of a hardware trojan (HT). Many works focus on the structure/effects of a HT, while very few have dem…
▽ More
Owning a high-end semiconductor foundry is a luxury very few companies can afford. Thus, fabless design companies outsource integrated circuit fabrication to third parties. Within foundries, rogue elements may gain access to the customer's layout and perform malicious acts, including the insertion of a hardware trojan (HT). Many works focus on the structure/effects of a HT, while very few have demonstrated the viability of their HTs in silicon. Even fewer disclose how HTs are inserted or the time required for this activity. Our work details, for the first time, how effortlessly a HT can be inserted into a finalized layout by presenting an insertion framework based on the engineering change order flow. For validation, we have built an ASIC prototype in 65nm CMOS technology comprising of four trojaned cryptocores. A side-channel HT is inserted in each core with the intent of leaking the cryptokey over a power channel. Moreover, we have determined that the entire attack can be mounted in a little over one hour. We also show that the attack was successful for all tested samples. Finally, our measurements demonstrate the robustness of our SCT against skews in the manufacturing process.
△ Less
Submitted 30 August, 2022; v1 submitted 6 December, 2021;
originally announced December 2021.
-
G-GPU: A Fully-Automated Generator of GPU-like ASIC Accelerators
Authors:
Tiago Diadami Perez,
Márcio M. Gonçalves,
José Rodrigo Azambuja,
Leonardo Gobatto,
Marcelo Brandalero,
Samuel Pagliarini
Abstract:
Modern Systems on Chip (SoC), almost as a rule, require accelerators for achieving energy efficiency and high performance for specific tasks that are not necessarily well suited for execution in standard processing units. Considering the broad range of applications and necessity for specialization, the design of SoCs has thus become expressively more challenging. In this paper, we put forward the…
▽ More
Modern Systems on Chip (SoC), almost as a rule, require accelerators for achieving energy efficiency and high performance for specific tasks that are not necessarily well suited for execution in standard processing units. Considering the broad range of applications and necessity for specialization, the design of SoCs has thus become expressively more challenging. In this paper, we put forward the concept of G-GPU, a general-purpose GPU-like accelerator that is not application-specific but still gives benefits in energy efficiency and throughput. Furthermore, we have identified an existing gap for these accelerators in ASIC, for which no known automated generation platform/tool exists. Our solution, called GPUPlanner, is an open-source generator of accelerators, from RTL to GDSII, that addresses this gap. Our analysis results show that our automatically generated G-GPU designs are remarkably efficient when compared against the popular CPU architecture RISC-V, presenting speed-ups of up to 223 times in raw performance and up to 11 times when the metric is performance derated by area. These results are achieved by executing a design space exploration of the GPU-like accelerators, where the memory hierarchy is broken in a smart fashion and the logic is pipelined on demand. Finally, tapeout-ready layouts of the G-GPU in 65nm CMOS are presented.
△ Less
Submitted 6 December, 2021; v1 submitted 11 November, 2021;
originally announced November 2021.
-
From FPGAs to Obfuscated eASICs: Design and Security Trade-offs
Authors:
Zain Ul Abideen,
Tiago Diadami Perez,
Samuel Pagliarini
Abstract:
Threats associated with the untrusted fabrication of integrated circuits (ICs) are numerous: piracy, overproduction, reverse engineering, hardware trojans, etc. The use of reconfigurable elements (i.e., look-up tables as in FPGAs) is a known obfuscation technique. In the extreme case, when the circuit is entirely implemented as an FPGA, no information is revealed to the adversary but at a high cos…
▽ More
Threats associated with the untrusted fabrication of integrated circuits (ICs) are numerous: piracy, overproduction, reverse engineering, hardware trojans, etc. The use of reconfigurable elements (i.e., look-up tables as in FPGAs) is a known obfuscation technique. In the extreme case, when the circuit is entirely implemented as an FPGA, no information is revealed to the adversary but at a high cost in area, power, and performance. In the opposite extreme, when the same circuit is implemented as an ASIC, best-in-class performance is obtained but security is compromised. This paper investigates an intermediate solution between these two. Our results are supported by a custom CAD tool that explores this FPGA-ASIC design space and enables a standard-cell based physical synthesis flow that is flexible and compatible with current design practices. Layouts are presented for obfuscated circuits in a 65nm commercial technology, demonstrating the attained obfuscation both graphically and quantitatively. Furthermore, our security analysis revealed that for truly hiding the circuit's intent (not only portions of its structure), the obfuscated design also has to chiefly resemble an FPGA: only some small amount of logic can be made static for an adversary to remain unaware of what the circuit does.
△ Less
Submitted 13 October, 2021; v1 submitted 11 October, 2021;
originally announced October 2021.
-
Side-Channel Trojan Insertion -- a Practical Foundry-Side Attack via ECO
Authors:
Tiago Perez,
Malik Imran,
Pablo Vaz,
Samuel Pagliarini
Abstract:
Design companies often outsource their integrated circuit (IC) fabrication to third parties where ICs are susceptible to malicious acts such as the insertion of a side-channel hardware trojan horse (SCT). In this paper, we present a framework for designing and inserting an SCT based on an engineering change order (ECO) flow, which makes it the first to disclose how effortlessly a trojan can be ins…
▽ More
Design companies often outsource their integrated circuit (IC) fabrication to third parties where ICs are susceptible to malicious acts such as the insertion of a side-channel hardware trojan horse (SCT). In this paper, we present a framework for designing and inserting an SCT based on an engineering change order (ECO) flow, which makes it the first to disclose how effortlessly a trojan can be inserted into an IC. The trojan is designed with the goal of leaking multiple bits per power signature reading. Our findings and results show that a rogue element within a foundry has, today, all means necessary for performing a foundry-side attack via ECO.
△ Less
Submitted 29 March, 2021; v1 submitted 1 February, 2021;
originally announced February 2021.
-
A Heteroscedastic Likelihood Model for Two-frame Optical Flow
Authors:
Timothy Farnworth,
Christopher Renton,
Reuben Strydom,
Adrian Wills,
Tristan Perez
Abstract:
Machine vision is an important sensing technology used in mobile robotic systems. Advancing the autonomy of such systems requires accurate characterisation of sensor uncertainty. Vision includes intrinsic uncertainty due to the camera sensor and extrinsic uncertainty due to environmental lighting and texture, which propagate through the image processing algorithms used to produce visual measuremen…
▽ More
Machine vision is an important sensing technology used in mobile robotic systems. Advancing the autonomy of such systems requires accurate characterisation of sensor uncertainty. Vision includes intrinsic uncertainty due to the camera sensor and extrinsic uncertainty due to environmental lighting and texture, which propagate through the image processing algorithms used to produce visual measurements. To faithfully characterise visual measurements, we must take into account these uncertainties.
In this paper, we propose a new class of likelihood functions that characterises the uncertainty of the error distribution of two-frame optical flow that enables a heteroscedastic dependence on texture. We employ the proposed class to characterise the Farneback and Lucas Kanade optical flow algorithms and achieve close agreement with their respective empirical error distributions over a wide range of texture in a simulated environment. The utility of the proposed likelihood model is demonstrated in a visual odometry ego-motion study, which results in performance competitive with contemporary methods. The development of an empirically congruent likelihood model advances the requisite tool-set for vision-based Bayesian inference and enables sensor data fusion with GPS, LiDAR and IMU to advance robust autonomous navigation.
△ Less
Submitted 31 December, 2020; v1 submitted 14 October, 2020;
originally announced October 2020.
-
A Survey on Split Manufacturing: Attacks, Defenses, and Challenges
Authors:
Tiago D. Perez,
Samuel Pagliarini
Abstract:
In today's integrated circuit (IC) ecosystem, owning a foundry is not economically viable, and therefore most IC design houses are now working under a fabless business model. In order to overcome security concerns associated with the outsorcing of IC fabrication, the Split Manufacturing technique was proposed. In Split Manufacturing, the Front End of Line (FEOL) layers (transistors and lower metal…
▽ More
In today's integrated circuit (IC) ecosystem, owning a foundry is not economically viable, and therefore most IC design houses are now working under a fabless business model. In order to overcome security concerns associated with the outsorcing of IC fabrication, the Split Manufacturing technique was proposed. In Split Manufacturing, the Front End of Line (FEOL) layers (transistors and lower metal layers) are fabricated at an untrusted high-end foundry, while the Back End of Line (BEOL) layers (higher metal layers) are manufactured at a trusted low-end foundry. This approach hides the BEOL connections from the untrusted foundry, thus preventing overproduction and piracy threats. However, many works demonstrate that BEOL connections can be derived by exploiting layout characteristics that are introduced by heuristics employed in typical floorplanning, placement, and routing algorithms. Since straightforward Split Manufacturing may not afford a desirable security level, many authors propose defense techniques to be used along with Split Manufacturing. In our survey, we present a detailed overview of the technique, the many types of attacks towards Split Manufacturing, as well as possible defense techniques described in the literature. For the attacks, we present a concise discussion on the different threat models and assumptions, while for the defenses we classify the studies into three categories: proximity perturbation, wire lifting, and layout obfuscation. The main outcome of our survey is to highlight the discrepancy between many studies -- some claim netlists can be reconstructed with near perfect precision, while others claim marginal success in retrieving BEOL connections. Finally, we also discuss future trends and challenges inherent to Split Manufacturing, including the fundamental difficulty of evaluating the efficiency of the technique.
△ Less
Submitted 12 October, 2020; v1 submitted 8 June, 2020;
originally announced June 2020.
-
A Sweet Pepper Harvesting Robot for Protected Crop** Environments
Authors:
Chris Lehnert,
Chris McCool,
Inkyu Sa,
Tristan Perez
Abstract:
Using robots to harvest sweet peppers in protected crop** environments has remained unsolved despite considerable effort by the research community over several decades. In this paper, we present the robotic harvester, Harvey, designed for sweet peppers in protected crop** environments that achieved a 76.5% success rate (within a modified scenario) which improves upon our prior work which achie…
▽ More
Using robots to harvest sweet peppers in protected crop** environments has remained unsolved despite considerable effort by the research community over several decades. In this paper, we present the robotic harvester, Harvey, designed for sweet peppers in protected crop** environments that achieved a 76.5% success rate (within a modified scenario) which improves upon our prior work which achieved 58% and related sweet pepper harvesting work which achieved 33\%. This improvement was primarily achieved through the introduction of a novel peduncle segmentation system using an efficient deep convolutional neural network, in conjunction with 3D post-filtering to detect the critical cutting location. We benchmark the peduncle segmentation against prior art demonstrating a considerable improvement in performance with an F_1 score of 0.564 compared to 0.302. The robotic harvester uses a perception pipeline to detect a target sweet pepper and an appropriate grasp and cutting pose used to determine the trajectory of a multi-modal harvesting tool to grasp the sweet pepper and cut it from the plant. A novel decoupling mechanism enables the grip** and cutting operations to be performed independently. We perform an in-depth analysis of the full robotic harvesting system to highlight bottlenecks and failure points that future work could address.
△ Less
Submitted 28 October, 2018;
originally announced October 2018.
-
A Rapidly Deployable Classification System using Visual Data for the Application of Precision Weed Management
Authors:
David Hall,
Feras Dayoub,
Tristan Perez,
Chris McCool
Abstract:
In this work we demonstrate a rapidly deployable weed classification system that uses visual data to enable autonomous precision weeding without making prior assumptions about which weed species are present in a given field. Previous work in this area relies on having prior knowledge of the weed species present in the field. This assumption cannot always hold true for every field, and thus limits…
▽ More
In this work we demonstrate a rapidly deployable weed classification system that uses visual data to enable autonomous precision weeding without making prior assumptions about which weed species are present in a given field. Previous work in this area relies on having prior knowledge of the weed species present in the field. This assumption cannot always hold true for every field, and thus limits the use of weed classification systems based on this assumption. In this work, we obviate this assumption and introduce a rapidly deployable approach able to operate on any field without any weed species assumptions prior to deployment. We present a three stage pipeline for the implementation of our weed classification system consisting of initial field surveillance, offline processing and selective labelling, and automated precision weeding. The key characteristic of our approach is the combination of plant clustering and selective labelling which is what enables our system to operate without prior weed species knowledge. Testing using field data we are able to label 12.3 times fewer images than traditional full labelling whilst reducing classification accuracy by only 14%.
△ Less
Submitted 26 April, 2018; v1 submitted 25 January, 2018;
originally announced January 2018.
-
Fruit Quantity and Quality Estimation using a Robotic Vision System
Authors:
M. Halstead,
C. McCool,
S. Denman,
T. Perez,
C. Fookes
Abstract:
Accurate localisation of crop remains highly challenging in unstructured environments such as farms. Many of the developed systems still rely on the use of hand selected features for crop identification and often neglect the estimation of crop quantity and quality, which is key to assigning labor during farming processes. To alleviate these limitations we present a robotic vision system that can a…
▽ More
Accurate localisation of crop remains highly challenging in unstructured environments such as farms. Many of the developed systems still rely on the use of hand selected features for crop identification and often neglect the estimation of crop quantity and quality, which is key to assigning labor during farming processes. To alleviate these limitations we present a robotic vision system that can accurately estimate the quantity and quality of sweet pepper (Capsicum annuum L), a key horticultural crop. This system consists of three parts: detection, quality estimation, and tracking. Efficient detection is achieved using the FasterRCNN framework. Quality is then estimated in the same framework by learning a parallel layer which we show experimentally results in superior performance than treating quality as extra classes in the traditional Faster-RCNN framework. Evaluation of these two techniques outlines the improved performance of the parallel layer, where we achieve an F1 score of 77.3 for the parallel technique yet only 72.5 for the best scoring (red) of the multi-class implementation. To track the crop we present a tracking via detection approach, which uses the FasterRCNN with parallel layers, that is also a vision-only solution. This approach is cheap to implement as it only requires a camera and in experiments across 2 days we show that our proposed system can accurately estimate the number of sweet pepper present, within 4.1% of the ground truth.
△ Less
Submitted 17 January, 2018;
originally announced January 2018.
-
In-Field Peduncle Detection of Sweet Peppers for Robotic Harvesting: a comparative study
Authors:
Chris Lehnert,
Chris McCool,
Tristan Perez
Abstract:
Robotic harvesting of crops has the potential to disrupt current agricultural practices. A key element to enabling robotic harvesting is to safely remove the crop from the plant which often involves locating and cutting the peduncle, the part of the crop that attaches it to the main stem of the plant.
In this paper we present a comparative study of two methods for performing peduncle detection.…
▽ More
Robotic harvesting of crops has the potential to disrupt current agricultural practices. A key element to enabling robotic harvesting is to safely remove the crop from the plant which often involves locating and cutting the peduncle, the part of the crop that attaches it to the main stem of the plant.
In this paper we present a comparative study of two methods for performing peduncle detection. The first method is based on classic colour and geometric features obtained from the scene with a support vector machine classifier, referred to as PFH-SVM. The second method is an efficient deep neural network approach, MiniInception, that is able to be deployed on a robotic platform. In both cases we employ a secondary filtering process that enforces reasonable assumptions about the crop structure, such as the proximity of the peduncle to the crop. Our tests are conducted on Harvey, a sweet pepper harvesting robot, and is evaluated in a greenhouse using two varieties of sweet pepper, Ducati and Mercuno. We demonstrate that the MiniInception method achieves impressive accuracy and considerably outperforms the PFH-SVM approach achieving an F1 score of 0.564 and 0.302 respectively.
△ Less
Submitted 29 September, 2017;
originally announced September 2017.
-
Lessons Learnt from Field Trials of a Robotic Sweet Pepper Harvester
Authors:
Christopher Lehnert,
Christopher McCool,
Tristan Perez
Abstract:
In this paper, we present the lessons learnt during the development of a new robotic harvester (Harvey) that can autonomously harvest sweet pepper (capsicum) in protected crop** environments. Robotic harvesting offers an attractive potential solution to reducing labour costs while enabling more regular and selective harvesting, optimising crop quality, scheduling and therefore profit. Our approa…
▽ More
In this paper, we present the lessons learnt during the development of a new robotic harvester (Harvey) that can autonomously harvest sweet pepper (capsicum) in protected crop** environments. Robotic harvesting offers an attractive potential solution to reducing labour costs while enabling more regular and selective harvesting, optimising crop quality, scheduling and therefore profit. Our approach combines effective vision algorithms with a novel end-effector design to enable successful harvesting of sweet peppers. We demonstrate a simple and effective vision-based algorithm for crop detection, a grasp selection method, and a novel end-effector design for harvesting. To reduce the complexity of motion planning and to minimise occlusions we focus on picking sweet peppers in a protected crop** environment where plants are grown on planar trellis structures. Initial field trials in protected crop** environments, with two cultivars, demonstrate the efficacy of this approach. The results show that the robot harvester can successfully detect, grasp, and detach crop from the plant within a real protected crop** system. The novel contributions of this work have resulted in significant and encouraging improvements in sweet pepper picking success rates compared with the state-of-the-art. Future work will look at detecting sweet pepper peduncles and improving the total harvesting cycle time for each sweet pepper. The methods presented in this paper provide steps towards the goal of fully autonomous and reliable crop picking systems that will revolutionise the horticulture industry by reducing labour costs, maximising the quality of produce, and ultimately improving the sustainability of farming enterprises.
△ Less
Submitted 19 June, 2017;
originally announced June 2017.
-
Autonomous Sweet Pepper Harvesting for Protected Crop** Systems
Authors:
Chris Lehnert,
Andrew English,
Chris McCool,
Adam Tow,
Tristan Perez
Abstract:
In this letter, we present a new robotic harvester (Harvey) that can autonomously harvest sweet pepper in protected crop** environments. Our approach combines effective vision algorithms with a novel end-effector design to enable successful harvesting of sweet peppers. Initial field trials in protected crop** environments, with two cultivar, demonstrate the efficacy of this approach achieving…
▽ More
In this letter, we present a new robotic harvester (Harvey) that can autonomously harvest sweet pepper in protected crop** environments. Our approach combines effective vision algorithms with a novel end-effector design to enable successful harvesting of sweet peppers. Initial field trials in protected crop** environments, with two cultivar, demonstrate the efficacy of this approach achieving a 46% success rate for unmodified crop, and 58% for modified crop. Furthermore, for the more favourable cultivar we were also able to detach 90% of sweet peppers, indicating that improvements in the gras** success rate would result in greatly improved harvesting performance.
△ Less
Submitted 6 June, 2017;
originally announced June 2017.
-
Peduncle Detection of Sweet Pepper for Autonomous Crop Harvesting - Combined Colour and 3D Information
Authors:
Inkyu Sa,
Chris Lehnert,
Andrew English,
Chris McCool,
Feras Dayoub,
Ben Upcroft,
Tristan Perez
Abstract:
This paper presents a 3D visual detection method for the challenging task of detecting peduncles of sweet peppers (Capsicum annuum) in the field. Cutting the peduncle cleanly is one of the most difficult stages of the harvesting process, where the peduncle is the part of the crop that attaches it to the main stem of the plant. Accurate peduncle detection in 3D space is therefore a vital step in re…
▽ More
This paper presents a 3D visual detection method for the challenging task of detecting peduncles of sweet peppers (Capsicum annuum) in the field. Cutting the peduncle cleanly is one of the most difficult stages of the harvesting process, where the peduncle is the part of the crop that attaches it to the main stem of the plant. Accurate peduncle detection in 3D space is therefore a vital step in reliable autonomous harvesting of sweet peppers, as this can lead to precise cutting while avoiding damage to the surrounding plant. This paper makes use of both colour and geometry information acquired from an RGB-D sensor and utilises a supervised-learning approach for the peduncle detection task. The performance of the proposed method is demonstrated and evaluated using qualitative and quantitative results (the Area-Under-the-Curve (AUC) of the detection precision-recall curve). We are able to achieve an AUC of 0.71 for peduncle detection on field-grown sweet peppers. We release a set of manually annotated 3D sweet pepper and peduncle images to assist the research community in performing further research on this topic.
△ Less
Submitted 30 January, 2017;
originally announced January 2017.
-
Collective intelligence: aggregation of information from neighbors in a guessing game
Authors:
Toni Pérez,
Jordi Zamora,
Víctor M. Eguíluz
Abstract:
Complex systems show the capacity to aggregate information and to display coordinated activity. In the case of social systems the interaction of different individuals leads to the emergence of norms, trends in political positions, opinions, cultural traits, and even scientific progress. Examples of collective behavior can be observed in activities like the Wikipedia and Linux, where individuals ag…
▽ More
Complex systems show the capacity to aggregate information and to display coordinated activity. In the case of social systems the interaction of different individuals leads to the emergence of norms, trends in political positions, opinions, cultural traits, and even scientific progress. Examples of collective behavior can be observed in activities like the Wikipedia and Linux, where individuals aggregate their knowledge for the benefit of the community, and citizen science, where the potential of collectives to solve complex problems is exploited. Here, we conducted an online experiment to investigate the performance of a collective when solving a guessing problem in which each actor is endowed with partial information and placed as the nodes of an interaction network. We measure the performance of the collective in terms of the temporal evolution of the accuracy, finding no statistical difference in the performance for two classes of networks, regular lattices and random networks. We also determine that a Bayesian description captures the behavior pattern the individuals follow in aggregating information from neighbors to make decisions. In comparison with other simple decision models, the strategy followed by the players reveals a suboptimal performance of the collective. Our contribution provides the basis for the micro-macro connection between individual based descriptions and collective phenomena.
△ Less
Submitted 29 March, 2016;
originally announced April 2016.
-
Persistence in voting behavior: stronghold dynamics in elections
Authors:
Toni Pérez,
Juan Fernández-Gracia,
Jose J. Ramasco,
Víctor M. Eguíluz
Abstract:
Influence among individuals is at the core of collective social phenomena such as the dissemination of ideas, beliefs or behaviors, social learning and the diffusion of innovations. Different mechanisms have been proposed to implement inter-agent influence in social models from the voter model, to majority rules, to the Granoveter model. Here we advance in this direction by confronting the recentl…
▽ More
Influence among individuals is at the core of collective social phenomena such as the dissemination of ideas, beliefs or behaviors, social learning and the diffusion of innovations. Different mechanisms have been proposed to implement inter-agent influence in social models from the voter model, to majority rules, to the Granoveter model. Here we advance in this direction by confronting the recently introduced Social Influence and Recurrent Mobility (SIRM) model, that reproduces generic features of vote-shares at different geographical levels, with data in the US presidential elections. Our approach incorporates spatial and population diversity as inputs for the opinion dynamics while individuals' mobility provides a proxy for social context, and peer imitation accounts for social influence. The model captures the observed stationary background fluctuations in the vote-shares across counties. We study the so-called political strongholds, i.e., locations where the votes-shares for a party are systematically higher than average. A quantitative definition of a stronghold by means of persistence in time of fluctuations in the voting spatial distribution is introduced, and results from the US Presidential Elections during the period 1980-2012 are analyzed within this framework. We compare electoral results with simulations obtained with the SIRM model finding a good agreement both in terms of the number and the location of strongholds. The strongholds duration is also systematically characterized in the SIRM model. The results compare well with the electoral results data revealing an exponential decay in the persistence of the strongholds with time.
△ Less
Submitted 23 March, 2015;
originally announced March 2015.
-
Structural and functional networks in complex systems with delay
Authors:
Víctor M. Eguíluz,
Toni Pérez,
Javier Borge-Holtoefer,
Alex Arenas
Abstract:
Functional networks of complex systems are obtained from the analysis of the temporal activity of their components, and are often used to infer their unknown underlying connectivity. We obtain the equations relating topology and function in a system of diffusively delay-coupled elements in complex networks. We solve exactly the resulting equations in motifs (directed structures of three nodes), an…
▽ More
Functional networks of complex systems are obtained from the analysis of the temporal activity of their components, and are often used to infer their unknown underlying connectivity. We obtain the equations relating topology and function in a system of diffusively delay-coupled elements in complex networks. We solve exactly the resulting equations in motifs (directed structures of three nodes), and in directed networks. The mean-field solution for directed uncorrelated networks shows that the clusterization of the activity is dominated by the in-degree of the nodes, and that the locking frequency decreases with increasing average degree. We find that the exponent of a power law degree distribution of the structural topology, b, is related to the exponent of the associated functional network as a =1/(2-b), for b < 2.
△ Less
Submitted 23 May, 2011; v1 submitted 7 February, 2011;
originally announced February 2011.
