-
LLMEasyQuant -- An Easy to Use Toolkit for LLM Quantization
Authors:
Dong Liu,
Meng Jiang,
Kaiser Pister
Abstract:
Currently, there are many quantization methods appeared for LLM quantization, yet few are user-friendly and easy to be deployed locally. Packages like TensorRT and Quantohave many underlying structures and self-invoking internal functions, which are not conducive to developers' personalized development and learning for deployment. Therefore, we develop LLMEasyQuant, it is a package aiming to for e…
▽ More
Currently, there are many quantization methods appeared for LLM quantization, yet few are user-friendly and easy to be deployed locally. Packages like TensorRT and Quantohave many underlying structures and self-invoking internal functions, which are not conducive to developers' personalized development and learning for deployment. Therefore, we develop LLMEasyQuant, it is a package aiming to for easy quantization deployment which is user-friendly and suitable for beginners' learning.
△ Less
Submitted 28 June, 2024;
originally announced June 2024.
-
PromptSet: A Programmer's Prompting Dataset
Authors:
Kaiser Pister,
Dhruba Jyoti Paul,
Patrick Brophy,
Ishan Joshi
Abstract:
The rise of capabilities expressed by large language models has been quickly followed by the integration of the same complex systems into application level logic. Algorithms, programs, systems, and companies are built around structured prompting to black box models where the majority of the design and implementation lies in capturing and quantifying the `agent mode'. The standard way to shape a cl…
▽ More
The rise of capabilities expressed by large language models has been quickly followed by the integration of the same complex systems into application level logic. Algorithms, programs, systems, and companies are built around structured prompting to black box models where the majority of the design and implementation lies in capturing and quantifying the `agent mode'. The standard way to shape a closed language model is to prime it for a specific task with a tailored prompt, often initially handwritten by a human. The textual prompts co-evolve with the codebase, taking shape over the course of project life as artifacts which must be reviewed and maintained, just as the traditional code files might be. Unlike traditional code, we find that prompts do not receive effective static testing and linting to prevent runtime issues. In this work, we present a novel dataset called PromptSet, with more than 61,000 unique developer prompts used in open source Python programs. We perform analysis on this dataset and introduce the notion of a static linter for prompts. Released with this publication is a HuggingFace dataset and a Github repository to recreate collection and processing efforts, both under the name \texttt{pisterlabs/promptset}.
△ Less
Submitted 26 February, 2024;
originally announced February 2024.
-
Investigating Compounding Prediction Errors in Learned Dynamics Models
Authors:
Nathan Lambert,
Kristofer Pister,
Roberto Calandra
Abstract:
Accurately predicting the consequences of agents' actions is a key prerequisite for planning in robotic control. Model-based reinforcement learning (MBRL) is one paradigm which relies on the iterative learning and prediction of state-action transitions to solve a task. Deep MBRL has become a popular candidate, using a neural network to learn a dynamics model that predicts with each pass from high-…
▽ More
Accurately predicting the consequences of agents' actions is a key prerequisite for planning in robotic control. Model-based reinforcement learning (MBRL) is one paradigm which relies on the iterative learning and prediction of state-action transitions to solve a task. Deep MBRL has become a popular candidate, using a neural network to learn a dynamics model that predicts with each pass from high-dimensional states to actions. These "one-step" predictions are known to become inaccurate over longer horizons of composed prediction - called the compounding error problem. Given the prevalence of the compounding error problem in MBRL and related fields of data-driven control, we set out to understand the properties of and conditions causing these long-horizon errors. In this paper, we explore the effects of subcomponents of a control problem on long term prediction error: including choosing a system, collecting data, and training a model. These detailed quantitative studies on simulated and real-world data show that the underlying dynamics of a system are the strongest factor determining the shape and magnitude of prediction error. Given a clearer understanding of compounding prediction error, researchers can implement new types of models beyond "one-step" that are more useful for control.
△ Less
Submitted 17 March, 2022;
originally announced March 2022.
-
BotNet: A Simulator for Studying the Effects of Accurate Communication Models on Multi-agent and Swarm Control
Authors:
Mark Selden,
Jason Zhou,
Felipe Campos,
Nathan Lambert,
Daniel Drew,
Kristofer S. J. Pister
Abstract:
Decentralized control in multi-robot systems is dependent on accurate and reliable communication between agents. Important communication factors, such as latency and packet delivery ratio, are strong functions of the number of agents in the network. Findings from studies of mobile and high node-count radio-frequency (RF) mesh networks have only been transferred to the domain of multi-robot systems…
▽ More
Decentralized control in multi-robot systems is dependent on accurate and reliable communication between agents. Important communication factors, such as latency and packet delivery ratio, are strong functions of the number of agents in the network. Findings from studies of mobile and high node-count radio-frequency (RF) mesh networks have only been transferred to the domain of multi-robot systems to a limited extent, and typical multi-agent robotic simulators often depend on simple propagation models that do not reflect the behavior of realistic RF networks. In this paper, we present a new open source swarm robotics simulator, BotNet, with an embedded standards-compliant time-synchronized channel hop** (6TiSCH) RF mesh network simulator. Using this simulator we show how more accurate communications models can limit even simple multi-robot control tasks such as flocking and formation control, with agent counts ranging from 10 up to 2500 agents. The experimental results are used to motivate changes to the inter-robot communication propagation models and other networking components currently used in practice in order to bridge the sim-to-real gap.
△ Less
Submitted 31 August, 2021;
originally announced August 2021.
-
Learning Accurate Long-term Dynamics for Model-based Reinforcement Learning
Authors:
Nathan O. Lambert,
Albert Wilcox,
Howard Zhang,
Kristofer S. J. Pister,
Roberto Calandra
Abstract:
Accurately predicting the dynamics of robotic systems is crucial for model-based control and reinforcement learning. The most common way to estimate dynamics is by fitting a one-step ahead prediction model and using it to recursively propagate the predicted state distribution over long horizons. Unfortunately, this approach is known to compound even small prediction errors, making long-term predic…
▽ More
Accurately predicting the dynamics of robotic systems is crucial for model-based control and reinforcement learning. The most common way to estimate dynamics is by fitting a one-step ahead prediction model and using it to recursively propagate the predicted state distribution over long horizons. Unfortunately, this approach is known to compound even small prediction errors, making long-term predictions inaccurate. In this paper, we propose a new parametrization to supervised learning on state-action data to stably predict at longer horizons -- that we call a trajectory-based model. This trajectory-based model takes an initial state, a future time index, and control parameters as inputs, and directly predicts the state at the future time index. Experimental results in simulated and real-world robotic tasks show that trajectory-based models yield significantly more accurate long term predictions, improved sample efficiency, and the ability to predict task reward. With these improved prediction properties, we conclude with a demonstration of methods for using the trajectory-based model for control.
△ Less
Submitted 31 August, 2021; v1 submitted 16 December, 2020;
originally announced December 2020.
-
Nonholonomic Yaw Control of an Underactuated Flying Robot with Model-based Reinforcement Learning
Authors:
Nathan Lambert,
Craig Schindler,
Daniel Drew,
Kristofer Pister
Abstract:
Nonholonomic control is a candidate to control nonlinear systems with path-dependant states. We investigate an underactuated flying micro-aerial-vehicle, the ionocraft, that requires nonholonomic control in the yaw-direction for complete attitude control. Deploying an analytical control law involves substantial engineering design and is sensitive to inaccuracy in the system model. With specific as…
▽ More
Nonholonomic control is a candidate to control nonlinear systems with path-dependant states. We investigate an underactuated flying micro-aerial-vehicle, the ionocraft, that requires nonholonomic control in the yaw-direction for complete attitude control. Deploying an analytical control law involves substantial engineering design and is sensitive to inaccuracy in the system model. With specific assumptions on assembly and system dynamics, we derive a Lie bracket for yaw control of the ionocraft. As a comparison to the significant engineering effort required for an analytic control law, we implement a data-driven model-based reinforcement learning yaw controller in a simulated flight task. We demonstrate that a simple model-based reinforcement learning framework can match the derived Lie bracket control (in yaw rate and chosen actions) in a few minutes of flight data, without a pre-defined dynamics function. This paper shows that learning-based approaches are useful as a tool for synthesis of nonlinear control laws previously only addressable through expert-based design.
△ Less
Submitted 12 January, 2021; v1 submitted 2 September, 2020;
originally announced September 2020.
-
Learning for Microrobot Exploration: Model-based Locomotion, Sparse-robust Navigation, and Low-power Deep Classification
Authors:
Nathan O. Lambert,
Farhan Toddywala,
Brian Liao,
Eric Zhu,
Lydia Lee,
Kristofer S. J. Pister
Abstract:
Building intelligent autonomous systems at any scale is challenging. The sensing and computation constraints of a microrobot platform make the problems harder. We present improvements to learning-based methods for on-board learning of locomotion, classification, and navigation of microrobots. We show how simulated locomotion can be achieved with model-based reinforcement learning via on-board sens…
▽ More
Building intelligent autonomous systems at any scale is challenging. The sensing and computation constraints of a microrobot platform make the problems harder. We present improvements to learning-based methods for on-board learning of locomotion, classification, and navigation of microrobots. We show how simulated locomotion can be achieved with model-based reinforcement learning via on-board sensor data distilled into control. Next, we introduce a sparse, linear detector and a Dynamic Thresholding method to FAST Visual Odometry for improved navigation in the noisy regime of mm scale imagery. We end with a new image classifier capable of classification with fewer than one million multiply-and-accumulate (MAC) operations by combining fast downsampling, efficient layer structures and hard activation functions. These are promising steps toward using state-of-the-art algorithms in the power-limited world of edge-intelligence and microrobots.
△ Less
Submitted 27 April, 2020;
originally announced April 2020.
-
Data-efficient Learning of Morphology and Controller for a Microrobot
Authors:
Thomas Liao,
Grant Wang,
Brian Yang,
Rene Lee,
Kristofer Pister,
Sergey Levine,
Roberto Calandra
Abstract:
Robot design is often a slow and difficult process requiring the iterative construction and testing of prototypes, with the goal of sequentially optimizing the design. For most robots, this process is further complicated by the need, when validating the capabilities of the hardware to solve the desired task, to already have an appropriate controller, which is in turn designed and tuned for the spe…
▽ More
Robot design is often a slow and difficult process requiring the iterative construction and testing of prototypes, with the goal of sequentially optimizing the design. For most robots, this process is further complicated by the need, when validating the capabilities of the hardware to solve the desired task, to already have an appropriate controller, which is in turn designed and tuned for the specific hardware. In this paper, we propose a novel approach, HPC-BBO, to efficiently and automatically design hardware configurations, and evaluate them by also automatically tuning the corresponding controller. HPC-BBO is based on a hierarchical Bayesian optimization process which iteratively optimizes morphology configurations (based on the performance of the previous designs during the controller learning process) and subsequently learns the corresponding controllers (exploiting the knowledge collected from optimizing for previous morphologies). Moreover, HPC-BBO can select a "batch" of multiple morphology designs at once, thus parallelizing hardware validation and reducing the number of time-consuming production cycles. We validate HPC-BBO on the design of the morphology and controller for a simulated 6-legged microrobot. Experimental results show that HPC-BBO outperforms multiple competitive baselines, and yields a $360\%$ reduction in production cycles over standard Bayesian optimization, thus reducing the hypothetical manufacturing time of our microrobot from 21 to 4 months.
△ Less
Submitted 3 May, 2019;
originally announced May 2019.
-
Experimental Clock Calibration\\on a Crystal-Free Mote-on-a-Chip
Authors:
Ioana Suciu,
Filip Maksimovic,
David Burnett,
Osama Khan,
Brad Wheeler,
Arvind Sundararajan,
Thomas Watteyne,
Xavier Vilajosana,
Kris Pister
Abstract:
The elimination of the off-chip frequency reference, typically a crystal oscillator, would bring important benefits in terms of size, price and energy efficiency to IEEE802.15.4 compliant radios and systems-on-chip. The stability of on-chip oscillators is orders of magnitude worse than that of a crystal. It is known that as the temperature changes, they can drift more than 50 ppm/°C. This paper pr…
▽ More
The elimination of the off-chip frequency reference, typically a crystal oscillator, would bring important benefits in terms of size, price and energy efficiency to IEEE802.15.4 compliant radios and systems-on-chip. The stability of on-chip oscillators is orders of magnitude worse than that of a crystal. It is known that as the temperature changes, they can drift more than 50 ppm/°C. This paper presents the result of an extensive experimental study. First, we propose mechanisms for crystal-free radios to be able to track an IEEE802.15.4 join proxy, calibrate the on-chip oscillators and maintain calibration against temperature changes. Then, we implement the resulting algorithms on a crystal-free platform and present the results of an experimental validation. We show that our approach is able to track a crystal-based IEEE802.15.4-compliant join proxy and maintain the requested radio frequency stability of +/-40 ppm, even when subject to temperature variation of 2°C/min.
△ Less
Submitted 17 April, 2019;
originally announced April 2019.
-
Low Level Control of a Quadrotor with Deep Model-Based Reinforcement Learning
Authors:
Nathan O. Lambert,
Daniel S. Drew,
Joseph Yaconelli,
Roberto Calandra,
Sergey Levine,
Kristofer S. J. Pister
Abstract:
Designing effective low-level robot controllers often entail platform-specific implementations that require manual heuristic parameter tuning, significant system knowledge, or long design times. With the rising number of robotic and mechatronic systems deployed across areas ranging from industrial automation to intelligent toys, the need for a general approach to generating low-level controllers i…
▽ More
Designing effective low-level robot controllers often entail platform-specific implementations that require manual heuristic parameter tuning, significant system knowledge, or long design times. With the rising number of robotic and mechatronic systems deployed across areas ranging from industrial automation to intelligent toys, the need for a general approach to generating low-level controllers is increasing. To address the challenge of rapidly generating low-level controllers, we argue for using model-based reinforcement learning (MBRL) trained on relatively small amounts of automatically generated (i.e., without system simulation) data. In this paper, we explore the capabilities of MBRL on a Crazyflie centimeter-scale quadrotor with rapid dynamics to predict and control at <50Hz. To our knowledge, this is the first use of MBRL for controlled hover of a quadrotor using only on-board sensors, direct motor input signals, and no initial dynamics knowledge. Our controller leverages rapid simulation of a neural network forward dynamics model on a GPU-enabled base station, which then transmits the best current action to the quadrotor firmware via radio. In our experiments, the quadrotor achieved hovering capability of up to 6 seconds with 3 minutes of experimental training data.
△ Less
Submitted 19 July, 2019; v1 submitted 11 January, 2019;
originally announced January 2019.
-
Decentralized Control of a Hexapod Robot Using a Wireless Time Synchronized Network
Authors:
James Fang,
Dinesh Parimi,
Arjun Dhindsa,
Craig B. Schindler,
Kristofer S. J. Pister
Abstract:
Robots and control systems rely upon precise timing of sensors and actuators in order to operate intelligently. We present a functioning hexapod robot that walks with a dual tripod gait; each tripod is actuated using its own local controller running on a separate wireless node. We compare and report the results of operating the robot using two different decentralized control schemes. With the firs…
▽ More
Robots and control systems rely upon precise timing of sensors and actuators in order to operate intelligently. We present a functioning hexapod robot that walks with a dual tripod gait; each tripod is actuated using its own local controller running on a separate wireless node. We compare and report the results of operating the robot using two different decentralized control schemes. With the first scheme, each controller relies on its own local clock to generate control signals for the tripod it controls. With the second scheme, each controller relies on a variable that is local to itself but that is necessarily the same across controllers as a by-product of their host nodes being part of a time synchronized IEEE802.15.4e network. The gait synchronization error (time difference between what both controllers believe is the start of the gait period) grows linearly when the controllers use their local clocks, but remains bounded to within 112 microseconds when the controllers use their nodes' time synchronized local variable.
△ Less
Submitted 23 August, 2018;
originally announced August 2018.
-
Learning Flexible and Reusable Locomotion Primitives for a Microrobot
Authors:
Brian Yang,
Grant Wang,
Roberto Calandra,
Daniel Contreras,
Sergey Levine,
Kristofer Pister
Abstract:
The design of gaits for robot locomotion can be a daunting process which requires significant expert knowledge and engineering. This process is even more challenging for robots that do not have an accurate physical model, such as compliant or micro-scale robots. Data-driven gait optimization provides an automated alternative to analytical gait design. In this paper, we propose a novel approach to…
▽ More
The design of gaits for robot locomotion can be a daunting process which requires significant expert knowledge and engineering. This process is even more challenging for robots that do not have an accurate physical model, such as compliant or micro-scale robots. Data-driven gait optimization provides an automated alternative to analytical gait design. In this paper, we propose a novel approach to efficiently learn a wide range of locomotion tasks with walking robots. This approach formalizes locomotion as a contextual policy search task to collect data, and subsequently uses that data to learn multi-objective locomotion primitives that can be used for planning. As a proof-of-concept we consider a simulated hexapod modeled after a recently developed microrobot, and we thoroughly evaluate the performance of this microrobot on different tasks and gaits. Our results validate the proposed controller and learning scheme on single and multi-objective locomotion tasks. Moreover, the experimental simulations show that without any prior knowledge about the robot used (e.g., dynamics model), our approach is capable of learning locomotion primitives within 250 trials and subsequently using them to successfully navigate through a maze.
△ Less
Submitted 28 February, 2018;
originally announced March 2018.
-
New Directions: Wireless Robotic Materials
Authors:
Nikolaus Correll,
Prabal Dutta,
Richard Han,
Kristofer Pister
Abstract:
We describe opportunities and challenges with wireless robotic materials. Robotic materials are multi-functional composites that tightly integrate sensing, actuation, computation and communication to create smart composites that can sense their environment and change their physical properties in an arbitrary programmable manner. Computation and communication in such materials are based on miniatur…
▽ More
We describe opportunities and challenges with wireless robotic materials. Robotic materials are multi-functional composites that tightly integrate sensing, actuation, computation and communication to create smart composites that can sense their environment and change their physical properties in an arbitrary programmable manner. Computation and communication in such materials are based on miniature, possibly wireless, devices that are scattered in the material and interface with sensors and actuators inside the material. Whereas routing and processing of information within the material build upon results from the field of sensor networks, robotic materials are pushing the limits of sensor networks in both size (down to the order of microns) and numbers of devices (up to the order of millions). In order to solve the algorithmic and systems challenges of such an approach, which will involve not only computer scientists, but also roboticists, chemists and material scientists, the community requires a common platform - much like the "Mote" that bootstrapped the widespread adoption of the field of sensor networks - that is small, provides ample of computation, is equipped with basic networking functionalities, and preferably can be powered wirelessly.
△ Less
Submitted 15 August, 2017;
originally announced August 2017.
