-
SWAM: Revisiting Swap and OOMK for Improving Application Responsiveness on Mobile Devices
Authors:
Geunsik Lim,
Donghyun Kang,
MyungJoo Ham,
Young Ik Eom
Abstract:
Existing memory reclamation policies on mobile devices may be no longer valid because they have negative effects on the response time of running applications. In this paper, we propose SWAM, a new integrated memory management technique that complements the shortcomings of both the swap** and killing mechanism in mobile devices and improves the application responsiveness. SWAM consists of (1) Ada…
▽ More
Existing memory reclamation policies on mobile devices may be no longer valid because they have negative effects on the response time of running applications. In this paper, we propose SWAM, a new integrated memory management technique that complements the shortcomings of both the swap** and killing mechanism in mobile devices and improves the application responsiveness. SWAM consists of (1) Adaptive Swap that performs swap** adaptively into memory or storage device while managing the swap space dynamically, (2) OOM Cleaner that reclaims shared object pages in the swap space to secure available memory and storage space, and (3) EOOM Killer that terminates processes in the worst case while prioritizing the lowest initialization cost applications as victim processes first. Experimental results demonstrate that SWAM significantly reduces the number of applications killed by OOMK (6.5x lower), and improves application launch time (36% faster) and response time (41% faster), compared to the conventional schemes.
△ Less
Submitted 14 June, 2023;
originally announced June 2023.
-
A New Frontier of AI: On-Device AI Training and Personalization
Authors:
Ji Joong Moon,
Hyun Suk Lee,
Jiho Chu,
Donghak Park,
Seungbaek Hong,
Hyungjun Seo,
Donghyeon Jeong,
Sungsik Kong,
MyungJoo Ham
Abstract:
Modern consumer electronic devices have started executing deep learning-based intelligence services on devices, not cloud servers, to keep personal data on devices and to reduce network and cloud costs. We find such a trend as the opportunity to personalize intelligence services by updating neural networks with user data without exposing the data out of devices: on-device training. However, the li…
▽ More
Modern consumer electronic devices have started executing deep learning-based intelligence services on devices, not cloud servers, to keep personal data on devices and to reduce network and cloud costs. We find such a trend as the opportunity to personalize intelligence services by updating neural networks with user data without exposing the data out of devices: on-device training. However, the limited resources of devices incurs significant difficulties. We propose a light-weight on-device training framework, NNTrainer, which provides highly memory-efficient neural network training techniques and proactive swap** based on fine-grained execution order analysis for neural networks. Moreover, its optimizations do not sacrifice accuracy and are transparent to training algorithms; thus, prior algorithmic studies may be implemented on top of NNTrainer. The evaluations show that NNTrainer can reduce memory consumption down to 1/20 (saving 95%!) and effectively personalizes intelligence services on devices. NNTrainer is cross-platform and practical open-source software, which is being deployed to millions of mobile devices.
△ Less
Submitted 4 January, 2024; v1 submitted 9 June, 2022;
originally announced June 2022.
-
Toward Among-Device AI from On-Device AI with Stream Pipelines
Authors:
MyungJoo Ham,
Sangjung Woo,
Jaeyun Jung,
Wook Song,
Gichan Jang,
Yongjoo Ahn,
Hyoung Joo Ahn
Abstract:
Modern consumer electronic devices often provide intelligence services with deep neural networks. We have started migrating the computing locations of intelligence services from cloud servers (traditional AI systems) to the corresponding devices (on-device AI systems). On-device AI systems generally have the advantages of preserving privacy, removing network latency, and saving cloud costs. With t…
▽ More
Modern consumer electronic devices often provide intelligence services with deep neural networks. We have started migrating the computing locations of intelligence services from cloud servers (traditional AI systems) to the corresponding devices (on-device AI systems). On-device AI systems generally have the advantages of preserving privacy, removing network latency, and saving cloud costs. With the emergent of on-device AI systems having relatively low computing power, the inconsistent and varying hardware resources and capabilities pose difficulties. Authors' affiliation has started applying a stream pipeline framework, NNStreamer, for on-device AI systems, saving developmental costs and hardware resources and improving performance. We want to expand the types of devices and applications with on-device AI services products of both the affiliation and second/third parties. We also want to make each AI service atomic, re-deployable, and shared among connected devices of arbitrary vendors; we now have yet another requirement introduced as it always has been. The new requirement of "among-device AI" includes connectivity between AI pipelines so that they may share computing resources and hardware capabilities across a wide range of devices regardless of vendors and manufacturers. We propose extensions of the stream pipeline framework, NNStreamer, for on-device AI so that NNStreamer may provide among-device AI capability. This work is a Linux Foundation (LF AI and Data) open source project accepting contributions from the general public.
△ Less
Submitted 16 January, 2022;
originally announced January 2022.
-
BB: Booting Booster for Consumer Electronics with Modern OS
Authors:
Geunsik Lim,
MyungJoo Ham
Abstract:
Unconventional computing platforms have spread widely and rapidly following smart phones and tablets: consumer electronics such as smart TVs and digital cameras. For such devices, fast booting is a critical requirement; waiting tens of seconds for a TV or a camera to boot up is not acceptable, unlike a PC or smart phone. Moreover, the software platforms of these devices have become as rich as conv…
▽ More
Unconventional computing platforms have spread widely and rapidly following smart phones and tablets: consumer electronics such as smart TVs and digital cameras. For such devices, fast booting is a critical requirement; waiting tens of seconds for a TV or a camera to boot up is not acceptable, unlike a PC or smart phone. Moreover, the software platforms of these devices have become as rich as conventional computing devices to provide comparable services. As a result, the booting procedure to start every required OS service, hardware component, and application, the quantity of which is ever increasing, may take unbearable time for most consumers. To accelerate booting, this paper introduces \textit{Booting Booster} (BB), which is used in all 2015 Samsung Smart TV models, and which runs the Linux-based Tizen OS. BB addresses the init scheme of Linux, which launches initial user-space OS services and applications and manages the life cycles of all user processes, by identifying and isolating booting-critical tasks, deferring non-critical tasks, and enabling execution of more tasks in parallel. BB has been successfully deployed in Samsung Smart TV 2015 models achieving a cold boot in 3.5 s (compared to 8.1 s with full commercial-grade optimizations without BB) without the need for suspend-to-RAM or hibernation. After this successful deployment, we have released the source code via http://opensource.samsung.com, and BB will be included in the open-source OS, Tizen (http://tizen.org).
△ Less
Submitted 21 January, 2021;
originally announced January 2021.
-
TAOS-CI: Lightweight & Modular Continuous Integration System for Edge Computing
Authors:
Geunsik Lim,
MyungJoo Ham,
Jijoong Moon,
Wook Song,
Sangjung Woo,
Sewon Oh
Abstract:
With the proliferation of IoT and edge devices, we are observing a lot of consumer electronics becoming yet another IoT and edge devices. Unlike traditional smart devices, such as smart phones, consumer electronics, in general, have significant diversities with fewer number of devices per product model. With such high diversities, the proliferation of edge devices requires frequent and seamless up…
▽ More
With the proliferation of IoT and edge devices, we are observing a lot of consumer electronics becoming yet another IoT and edge devices. Unlike traditional smart devices, such as smart phones, consumer electronics, in general, have significant diversities with fewer number of devices per product model. With such high diversities, the proliferation of edge devices requires frequent and seamless updates of consumer electronics, which makes the manufacturers prone to regressions because the manufacturers have less resource per an instance of software release; i.e., they need to repeat releases by the number of product models times the number of updates. Continuous Integration (CI) systems can help prevent regression bugs from actively develo** software packages including the frequently updated device software platforms. The proposed CI system provides a portable and modular software platform automatically inspecting potential issues of incoming changes with the enabled modules: code format and style, performance regressions, static checks on the source code, build and packaging tests, and dynamic checks with the built binary before deploying a platform image on the IoT and edge devices. Besides, our proposed approach is lightweight enough to be hosted in normal desktop computers even for dozens of developers. As a result, it can be easily applied to a lot of various source code repositories. Evaluation results demonstrate that the proposed method drastically improves plug-ins execution time and memory consumption, compared with methods in previous studies.
△ Less
Submitted 21 January, 2021;
originally announced January 2021.
-
LightSys: Lightweight and Efficient CI System for Improving Integration Speed of Software
Authors:
Geunsik Lim,
MyungJoo Ham,
Jijoong Moon,
Wook Song
Abstract:
The complexity and size increase of software has extended the delay for developers as they wait for code analysis and code merge. With the larger and more complex software, more developers nowadays are develo** software with large source code repositories. The tendency for software platforms to immediately update software packages with feature updates and bug-fixes is a significant obstacle. Con…
▽ More
The complexity and size increase of software has extended the delay for developers as they wait for code analysis and code merge. With the larger and more complex software, more developers nowadays are develo** software with large source code repositories. The tendency for software platforms to immediately update software packages with feature updates and bug-fixes is a significant obstacle. Continuous integration systems may help prevent software flaws during the active development of software packages, even when they are deployed and updated frequently. Herein, we present a portable and modular code review automation system that inspects incoming code changes such as code format and style, performance regression, static analysis, build and deployment tests, and dynamic analysis before merging and changing code. The proposed mechanisms are sufficiently lightweight to be hosted on a regular desktop computer even for numerous developers. The resulting reduced costs allow developers to apply the proposed mechanism to many source code repositories. Experimental results demonstrate that the proposed mechanism drastically reduces overheads and improves usability compared with conventional mechanisms: execution time (6x faster), CPU usage (40% lower), memory consumption (1/180), and no out-of-memory occurrence.
△ Less
Submitted 29 January, 2021; v1 submitted 19 January, 2021;
originally announced January 2021.
-
NNStreamer: Efficient and Agile Development of On-Device AI Systems
Authors:
MyungJoo Ham,
Jijoong Moon,
Geunsik Lim,
Jaeyun Jung,
Hyoungjoo Ahn,
Wook Song,
Sangjung Woo,
Parichay Kapoor,
Dongju Chae,
Gichan Jang,
Yongjoo Ahn,
Jihoon Lee
Abstract:
We propose NNStreamer, a software system that handles neural networks as filters of stream pipelines, applying the stream processing paradigm to deep neural network applications. A new trend with the wide-spread of deep neural network applications is on-device AI. It is to process neural networks on mobile devices or edge/IoT devices instead of cloud servers. Emerging privacy issues, data transmis…
▽ More
We propose NNStreamer, a software system that handles neural networks as filters of stream pipelines, applying the stream processing paradigm to deep neural network applications. A new trend with the wide-spread of deep neural network applications is on-device AI. It is to process neural networks on mobile devices or edge/IoT devices instead of cloud servers. Emerging privacy issues, data transmission costs, and operational costs signify the need for on-device AI, especially if we deploy a massive number of devices. NNStreamer efficiently handles neural networks with complex data stream pipelines on devices, significantly improving the overall performance with minimal efforts. Besides, NNStreamer simplifies implementations and allows reusing off-the-shelf media filters directly, which reduces developmental costs significantly. We are already deploying NNStreamer for a wide range of products and platforms, including the Galaxy series and various consumer electronic devices. The experimental results suggest a reduction in developmental costs and enhanced performance of pipeline architectures and NNStreamer. It is an open-source project incubated by Linux Foundation AI, available to the public and applicable to various hardware and software platforms.
△ Less
Submitted 15 January, 2021;
originally announced January 2021.
-
NNStreamer: Stream Processing Paradigm for Neural Networks, Toward Efficient Development and Execution of On-Device AI Applications
Authors:
MyungJoo Ham,
Ji Joong Moon,
Geunsik Lim,
Wook Song,
Jaeyun Jung,
Hyoungjoo Ahn,
Sangjung Woo,
Youngchul Cho,
**hyuck Park,
Sewon Oh,
Hong-Seok Kim
Abstract:
We propose nnstreamer, a software system that handles neural networks as filters of stream pipelines, applying the stream processing paradigm to neural network applications. A new trend with the wide-spread of deep neural network applications is on-device AI; i.e., processing neural networks directly on mobile devices or edge/IoT devices instead of cloud servers. Emerging privacy issues, data tran…
▽ More
We propose nnstreamer, a software system that handles neural networks as filters of stream pipelines, applying the stream processing paradigm to neural network applications. A new trend with the wide-spread of deep neural network applications is on-device AI; i.e., processing neural networks directly on mobile devices or edge/IoT devices instead of cloud servers. Emerging privacy issues, data transmission costs, and operational costs signifies the need for on-device AI especially when a huge number of devices with real-time data processing are deployed. Nnstreamer efficiently handles neural networks with complex data stream pipelines on devices, improving the overall performance significantly with minimal efforts. Besides, nnstreamer simplifies the neural network pipeline implementations and allows reusing off-shelf multimedia stream filters directly; thus it reduces the developmental costs significantly. Nnstreamer is already being deployed with a product releasing soon and is open source software applicable to a wide range of hardware architectures and software platforms.
△ Less
Submitted 12 January, 2019;
originally announced January 2019.
-
A Boundary Approximation Algorithm for Distributed Sensor Networks
Authors:
Michael I. Ham,
Marko A. Rodriguez
Abstract:
We present an algorithm for boundary approximation in locally-linked sensor networks that communicate with a remote monitoring station. Delaunay triangulations and Voronoi diagrams are used to generate a sensor communication network and define boundary segments between sensors, respectively. The proposed algorithm reduces remote station communication by approximating boundaries via a decentraliz…
▽ More
We present an algorithm for boundary approximation in locally-linked sensor networks that communicate with a remote monitoring station. Delaunay triangulations and Voronoi diagrams are used to generate a sensor communication network and define boundary segments between sensors, respectively. The proposed algorithm reduces remote station communication by approximating boundaries via a decentralized computation executed within the sensor network. Moreover, the algorithm identifies boundaries based on differences between neighboring sensor readings, and not absolute sensor values. An analysis of the bandwidth consumption of the algorithm is presented and compared to two naive approaches. The proposed algorithm reduces the amount of remote communication (compared to the naive approaches) and becomes increasingly useful in networks with more nodes.
△ Less
Submitted 21 January, 2009;
originally announced January 2009.
