-
ASCENT: Amplifying Power Side-Channel Resilience via Learning & Monte-Carlo Tree Search
Authors:
Jitendra Bhandari,
Animesh Basak Chowdhury,
Mohammed Nabeel,
Ozgur Sinanoglu,
Siddharth Garg,
Ramesh Karri,
Johann Knechtel
Abstract:
Power side-channel (PSC) analysis is pivotal for securing cryptographic hardware. Prior art focused on securing gate-level netlists obtained as-is from chip design automation, neglecting all the complexities and potential side-effects for security arising from the design automation process. That is, automation traditionally prioritizes power, performance, and area (PPA), sidelining security. We pr…
▽ More
Power side-channel (PSC) analysis is pivotal for securing cryptographic hardware. Prior art focused on securing gate-level netlists obtained as-is from chip design automation, neglecting all the complexities and potential side-effects for security arising from the design automation process. That is, automation traditionally prioritizes power, performance, and area (PPA), sidelining security. We propose a "security-first" approach, refining the logic synthesis stage to enhance the overall resilience of PSC countermeasures. We introduce ASCENT, a learning-and-search-based framework that (i) drastically reduces the time for post-design PSC evaluation and (ii) explores the security-vs-PPA design space. Thus, ASCENT enables an efficient exploration of a large number of candidate netlists, leading to an improvement in PSC resilience compared to regular PPA-optimized netlists. ASCENT is up to 120x faster than traditional PSC analysis and yields a 3.11x improvement for PSC resilience of state-of-the-art PSC countermeasures
△ Less
Submitted 1 July, 2024; v1 submitted 27 June, 2024;
originally announced June 2024.
-
LLM-Aided Testbench Generation and Bug Detection for Finite-State Machines
Authors:
Jitendra Bhandari,
Johann Knechtel,
Ramesh Narayanaswamy,
Siddharth Garg,
Ramesh Karri
Abstract:
This work investigates the potential of tailoring Large Language Models (LLMs), specifically GPT3.5 and GPT4, for the domain of chip testing. A key aspect of chip design is functional testing, which relies on testbenches to evaluate the functionality and coverage of Register-Transfer Level (RTL) designs. We aim to enhance testbench generation by incorporating feedback from commercial-grade Electro…
▽ More
This work investigates the potential of tailoring Large Language Models (LLMs), specifically GPT3.5 and GPT4, for the domain of chip testing. A key aspect of chip design is functional testing, which relies on testbenches to evaluate the functionality and coverage of Register-Transfer Level (RTL) designs. We aim to enhance testbench generation by incorporating feedback from commercial-grade Electronic Design Automation (EDA) tools into LLMs. Through iterative feedback from these tools, we refine the testbenches to achieve improved test coverage. Our case studies present promising results, demonstrating that this approach can effectively enhance test coverage. By integrating EDA tool feedback, the generated testbenches become more accurate in identifying potential issues in the RTL design. Furthermore, we extended our study to use this enhanced test coverage framework for detecting bugs in the RTL implementations
△ Less
Submitted 24 June, 2024;
originally announced June 2024.
-
Lightweight Masking Against Static Power Side-Channel Attacks
Authors:
Jitendra Bhandari,
Mohammed Nabeel,
Likhitha Mankali,
Ozgur Sinanoglu,
Ramesh Karri,
Johann Knechtel
Abstract:
This paper presents a novel defense strategy against static power side-channel attacks (PSCAs), a critical threat to cryptographic security. Our method is based on (1) carefully tuning high-Vth versus low-Vth cell selection during synthesis, accounting for both security and timing impact, and (2), at runtime, randomly switching the operation between these cells. This approach serves to significant…
▽ More
This paper presents a novel defense strategy against static power side-channel attacks (PSCAs), a critical threat to cryptographic security. Our method is based on (1) carefully tuning high-Vth versus low-Vth cell selection during synthesis, accounting for both security and timing impact, and (2), at runtime, randomly switching the operation between these cells. This approach serves to significantly obscure static power patterns, which are at the heart of static PSCAs. Our experimental results on a commercial 28nm node show a drastic increase in the effort required for a successful attack, namely up to 96 times more traces. When compared to prior countermeasures, ours incurs little cost, making it a lightweight defense.
△ Less
Submitted 5 February, 2024;
originally announced February 2024.
-
Pearl: A Production-ready Reinforcement Learning Agent
Authors:
Zheqing Zhu,
Rodrigo de Salvo Braz,
Jalaj Bhandari,
Daniel Jiang,
Yi Wan,
Yonathan Efroni,
Liyuan Wang,
Ruiyang Xu,
Hongbo Guo,
Alex Nikulkov,
Dmytro Korenkevych,
Urun Dogan,
Frank Cheng,
Zheng Wu,
Wanqiao Xu
Abstract:
Reinforcement Learning (RL) offers a versatile framework for achieving long-term goals. Its generality allows us to formalize a wide range of problems that real-world intelligent systems encounter, such as dealing with delayed rewards, handling partial observability, addressing the exploration and exploitation dilemma, utilizing offline data to improve online performance, and ensuring safety const…
▽ More
Reinforcement Learning (RL) offers a versatile framework for achieving long-term goals. Its generality allows us to formalize a wide range of problems that real-world intelligent systems encounter, such as dealing with delayed rewards, handling partial observability, addressing the exploration and exploitation dilemma, utilizing offline data to improve online performance, and ensuring safety constraints are met. Despite considerable progress made by the RL research community in addressing these issues, existing open-source RL libraries tend to focus on a narrow portion of the RL solution pipeline, leaving other aspects largely unattended. This paper introduces Pearl, a Production-ready RL agent software package explicitly designed to embrace these challenges in a modular fashion. In addition to presenting preliminary benchmark results, this paper highlights Pearl's industry adoptions to demonstrate its readiness for production usage. Pearl is open sourced on Github at github.com/facebookresearch/pearl and its official website is located at pearlagent.github.io.
△ Less
Submitted 6 December, 2023;
originally announced December 2023.
-
Optimizing Long-term Value for Auction-Based Recommender Systems via On-Policy Reinforcement Learning
Authors:
Ruiyang Xu,
Jalaj Bhandari,
Dmytro Korenkevych,
Fan Liu,
Yuchen He,
Alex Nikulkov,
Zheqing Zhu
Abstract:
Auction-based recommender systems are prevalent in online advertising platforms, but they are typically optimized to allocate recommendation slots based on immediate expected return metrics, neglecting the downstream effects of recommendations on user behavior. In this study, we employ reinforcement learning to optimize for long-term return metrics in an auction-based recommender system. Utilizing…
▽ More
Auction-based recommender systems are prevalent in online advertising platforms, but they are typically optimized to allocate recommendation slots based on immediate expected return metrics, neglecting the downstream effects of recommendations on user behavior. In this study, we employ reinforcement learning to optimize for long-term return metrics in an auction-based recommender system. Utilizing temporal difference learning, a fundamental reinforcement learning algorithm, we implement an one-step policy improvement approach that biases the system towards recommendations with higher long-term user engagement metrics. This optimizes value over long horizons while maintaining compatibility with the auction framework. Our approach is grounded in dynamic programming ideas which show that our method provably improves upon the existing auction-based base policy. Through an online A/B test conducted on an auction-based recommender system which handles billions of impressions and users daily, we empirically establish that our proposed method outperforms the current production system in terms of long-term user engagement metrics.
△ Less
Submitted 30 July, 2023; v1 submitted 23 May, 2023;
originally announced May 2023.
-
ALICE: An Automatic Design Flow for eFPGA Redaction
Authors:
Chiara Muscari Tomajoli,
Luca Collini,
Jitendra Bhandari,
Abdul Khader Thalakkattu Moosa,
Benjamin Tan,
Xifan Tang,
Pierre-Emmanuel Gaillardon,
Ramesh Karri,
Christian Pilato
Abstract:
Fabricating an integrated circuit is becoming unaffordable for many semiconductor design houses. Outsourcing the fabrication to a third-party foundry requires methods to protect the intellectual property of the hardware designs. Designers can rely on embedded reconfigurable devices to completely hide the real functionality of selected design portions unless the configuration string (bitstream) is…
▽ More
Fabricating an integrated circuit is becoming unaffordable for many semiconductor design houses. Outsourcing the fabrication to a third-party foundry requires methods to protect the intellectual property of the hardware designs. Designers can rely on embedded reconfigurable devices to completely hide the real functionality of selected design portions unless the configuration string (bitstream) is provided. However, selecting such portions and creating the corresponding reconfigurable fabrics are still open problems. We propose ALICE, a design flow that addresses the EDA challenges of this problem. ALICE partitions the RTL modules between one or more reconfigurable fabrics and the rest of the circuit, automating the generation of the corresponding redacted design.
△ Less
Submitted 15 May, 2022;
originally announced May 2022.
-
Not All Fabrics Are Created Equal: Exploring eFPGA Parameters For IP Redaction
Authors:
Jitendra Bhandari,
Abdul Khader Thalakkattu Moosa,
Benjamin Tan,
Christian Pilato,
Ganesh Gore,
Xifan Tang,
Scott Temple,
Pierre-Emmanuel Gaillardo,
Ramesh Karri
Abstract:
Semiconductor design houses rely on third-party foundries to manufacture their integrated circuits (IC). While this trend allows them to tackle fabrication costs, it introduces security concerns as external (and potentially malicious) parties can access critical parts of the designs and steal or modify the Intellectual Property (IP). Embedded FPGA (eFPGA) redaction is a promising technique to prot…
▽ More
Semiconductor design houses rely on third-party foundries to manufacture their integrated circuits (IC). While this trend allows them to tackle fabrication costs, it introduces security concerns as external (and potentially malicious) parties can access critical parts of the designs and steal or modify the Intellectual Property (IP). Embedded FPGA (eFPGA) redaction is a promising technique to protect critical IPs of an ASIC by \textit{redacting} (i.e., removing) critical parts and map** them onto a custom reconfigurable fabric. Only trusted parties will receive the correct bitstream to restore the redacted functionality. While previous studies imply that using an eFPGA is a sufficient condition to provide security against IP threats like reverse-engineering, whether this truly holds for all eFPGA architectures is unclear, thus motivating the study in this paper. We examine the security of eFPGA fabrics generated by varying different FPGA design parameters. We characterize the power, performance, and area (PPA) characteristics and evaluate each fabric's resistance to SAT-based bitstream recovery. Our results encourage designers to work with custom eFPGA fabrics rather than off-the-shelf commercial FPGAs and reveals that only considering a redaction fabric's bitstream size is inadequate for gauging security.
△ Less
Submitted 7 November, 2021;
originally announced November 2021.
-
Exploring eFPGA-based Redaction for IP Protection
Authors:
Jitendra Bhandari,
Abdul Khader Thalakkattu Moosa,
Benjamin Tan,
Christian Pilato,
Ganesh Gore,
Xifan Tang,
Scott Temple,
Pierre-Emmanuel Gaillardon,
Ramesh Karri
Abstract:
Recently, eFPGA-based redaction has been proposed as a promising solution for hiding parts of a digital design from untrusted entities, where legitimate end-users can restore functionality by loading the withheld bitstream after fabrication. However, when deciding which parts of a design to redact, there are a number of practical issues that designers need to consider, including area and timing ov…
▽ More
Recently, eFPGA-based redaction has been proposed as a promising solution for hiding parts of a digital design from untrusted entities, where legitimate end-users can restore functionality by loading the withheld bitstream after fabrication. However, when deciding which parts of a design to redact, there are a number of practical issues that designers need to consider, including area and timing overheads, as well as security factors. Adapting an open-source FPGA fabric generation flow, we perform a case study to explore the trade-offs when redacting different modules of open-source intellectual property blocks (IPs) and explore how different parts of an eFPGA contribute to the security. We provide new insights into the feasibility and challenges of using eFPGA-based redaction as a security solution.
△ Less
Submitted 25 October, 2021;
originally announced October 2021.
-
On Linear Convergence of Policy Gradient Methods for Finite MDPs
Authors:
Jalaj Bhandari,
Daniel Russo
Abstract:
We revisit the finite time analysis of policy gradient methods in the one of the simplest settings: finite state and action MDPs with a policy class consisting of all stochastic policies and with exact gradient evaluations. There has been some recent work viewing this setting as an instance of smooth non-linear optimization problems and showing sub-linear convergence rates with small step-sizes. H…
▽ More
We revisit the finite time analysis of policy gradient methods in the one of the simplest settings: finite state and action MDPs with a policy class consisting of all stochastic policies and with exact gradient evaluations. There has been some recent work viewing this setting as an instance of smooth non-linear optimization problems and showing sub-linear convergence rates with small step-sizes. Here, we take a different perspective based on connections with policy iteration and show that many variants of policy gradient methods succeed with large step-sizes and attain a linear rate of convergence.
△ Less
Submitted 13 December, 2021; v1 submitted 21 July, 2020;
originally announced July 2020.
-
Global Optimality Guarantees For Policy Gradient Methods
Authors:
Jalaj Bhandari,
Daniel Russo
Abstract:
Policy gradients methods apply to complex, poorly understood, control problems by performing stochastic gradient descent over a parameterized class of polices. Unfortunately, even for simple control problems solvable by standard dynamic programming techniques, policy gradient algorithms face non-convex optimization problems and are widely understood to converge only to a stationary point. This wor…
▽ More
Policy gradients methods apply to complex, poorly understood, control problems by performing stochastic gradient descent over a parameterized class of polices. Unfortunately, even for simple control problems solvable by standard dynamic programming techniques, policy gradient algorithms face non-convex optimization problems and are widely understood to converge only to a stationary point. This work identifies structural properties -- shared by several classic control problems -- that ensure the policy gradient objective function has no suboptimal stationary points despite being non-convex. When these conditions are strengthened, this objective satisfies a Polyak-lojasiewicz (gradient dominance) condition that yields convergence rates. We also provide bounds on the optimality gap of any stationary point when some of these conditions are relaxed.
△ Less
Submitted 19 June, 2022; v1 submitted 4 June, 2019;
originally announced June 2019.
-
A Finite Time Analysis of Temporal Difference Learning With Linear Function Approximation
Authors:
Jalaj Bhandari,
Daniel Russo,
Raghav Singal
Abstract:
Temporal difference learning (TD) is a simple iterative algorithm used to estimate the value function corresponding to a given policy in a Markov decision process. Although TD is one of the most widely used algorithms in reinforcement learning, its theoretical analysis has proved challenging and few guarantees on its statistical efficiency are available. In this work, we provide a simple and expli…
▽ More
Temporal difference learning (TD) is a simple iterative algorithm used to estimate the value function corresponding to a given policy in a Markov decision process. Although TD is one of the most widely used algorithms in reinforcement learning, its theoretical analysis has proved challenging and few guarantees on its statistical efficiency are available. In this work, we provide a simple and explicit finite time analysis of temporal difference learning with linear function approximation. Except for a few key insights, our analysis mirrors standard techniques for analyzing stochastic gradient descent algorithms, and therefore inherits the simplicity and elegance of that literature. Final sections of the paper show how all of our main results extend to the study of TD learning with eligibility traces, known as TD($λ$), and to Q-learning applied in high-dimensional optimal stop** problems.
△ Less
Submitted 6 November, 2018; v1 submitted 6 June, 2018;
originally announced June 2018.
