Showing 1–12 of 12 results for author: Je, H

Constellation Dataset: Benchmarking High-Altitude Object Detection for an Urban Intersection

Authors: Mehmet Kerem Turkcan, Sanjeev Narasimhan, Chengbo Zang, Gyung Hyun Je, Bo Yu, Mahshid Ghasemi, Javad Ghaderi, Gil Zussman, Zoran Kostic

Abstract: We introduce Constellation, a dataset of 13K images suitable for research on detection of objects in dense urban streetscapes observed from high-elevation cameras, collected for a variety of temporal conditions. The dataset addresses the need for curated data to explore problems in small object detection exemplified by the limited pixel footprint of pedestrians observed tens of meters from above.… ▽ More We introduce Constellation, a dataset of 13K images suitable for research on detection of objects in dense urban streetscapes observed from high-elevation cameras, collected for a variety of temporal conditions. The dataset addresses the need for curated data to explore problems in small object detection exemplified by the limited pixel footprint of pedestrians observed tens of meters from above. It enables the testing of object detection models for variations in lighting, building shadows, weather, and scene dynamics. We evaluate contemporary object detection architectures on the dataset, observing that state-of-the-art methods have lower performance in detecting small pedestrians compared to vehicles, corresponding to a 10% difference in average precision (AP). Using structurally similar datasets for pretraining the models results in an increase of 1.8% mean AP (mAP). We further find that incorporating domain-specific data augmentations helps improve model performance. Using pseudo-labeled data, obtained from inference outcomes of the best-performing models, improves the performance of the models. Finally, comparing the models trained using the data collected in two different time intervals, we find a performance drift in models due to the changes in intersection conditions over time. The best-performing model achieves a pedestrian AP of 92.0% with 11.5 ms inference time on NVIDIA A100 GPUs, and an mAP of 95.4%. △ Less

Submitted 25 April, 2024; originally announced April 2024.

Fully Quantized Always-on Face Detector Considering Mobile Image Sensors

Authors: Haechang Lee, Wongi Jeong, Dongil Ryu, Hyunwoo Je, Albert No, Kijeong Kim, Se Young Chun

Abstract: Despite significant research on lightweight deep neural networks (DNNs) designed for edge devices, the current face detectors do not fully meet the requirements for "intelligent" CMOS image sensors (iCISs) integrated with embedded DNNs. These sensors are essential in various practical applications, such as energy-efficient mobile phones and surveillance systems with always-on capabilities. One not… ▽ More Despite significant research on lightweight deep neural networks (DNNs) designed for edge devices, the current face detectors do not fully meet the requirements for "intelligent" CMOS image sensors (iCISs) integrated with embedded DNNs. These sensors are essential in various practical applications, such as energy-efficient mobile phones and surveillance systems with always-on capabilities. One noteworthy limitation is the absence of suitable face detectors for the always-on scenario, a crucial aspect of image sensor-level applications. These detectors must operate directly with sensor RAW data before the image signal processor (ISP) takes over. This gap poses a significant challenge in achieving optimal performance in such scenarios. Further research and development are necessary to bridge this gap and fully leverage the potential of iCIS applications. In this study, we aim to bridge the gap by exploring extremely low-bit lightweight face detectors, focusing on the always-on face detection scenario for mobile image sensor applications. To achieve this, our proposed model utilizes sensor-aware synthetic RAW inputs, simulating always-on face detection processed "before" the ISP chain. Our approach employs ternary (-1, 0, 1) weights for potential implementations in image sensors, resulting in a relatively simple network architecture with shallow layers and extremely low-bitwidth. Our method demonstrates reasonable face detection performance and excellent efficiency in simulation studies, offering promising possibilities for practical always-on face detectors in real-world applications. △ Less

Submitted 2 November, 2023; originally announced November 2023.

Comments: Accepted to ICCV 2023 Workshop on Low-Bit Quantized Neural Networks (LBQNN), Oral

Efficient Unified Demosaicing for Bayer and Non-Bayer Patterned Image Sensors

Authors: Haechang Lee, Dongwon Park, Wongi Jeong, Kijeong Kim, Hyunwoo Je, Dongil Ryu, Se Young Chun

Abstract: As the physical size of recent CMOS image sensors (CIS) gets smaller, the latest mobile cameras are adopting unique non-Bayer color filter array (CFA) patterns (e.g., Quad, Nona, QxQ), which consist of homogeneous color units with adjacent pixels. These non-Bayer sensors are superior to conventional Bayer CFA thanks to their changeable pixel-bin sizes for different light conditions but may introdu… ▽ More As the physical size of recent CMOS image sensors (CIS) gets smaller, the latest mobile cameras are adopting unique non-Bayer color filter array (CFA) patterns (e.g., Quad, Nona, QxQ), which consist of homogeneous color units with adjacent pixels. These non-Bayer sensors are superior to conventional Bayer CFA thanks to their changeable pixel-bin sizes for different light conditions but may introduce visual artifacts during demosaicing due to their inherent pixel pattern structures and sensor hardware characteristics. Previous demosaicing methods have primarily focused on Bayer CFA, necessitating distinct reconstruction methods for non-Bayer patterned CIS with various CFA modes under different lighting conditions. In this work, we propose an efficient unified demosaicing method that can be applied to both conventional Bayer RAW and various non-Bayer CFAs' RAW data in different operation modes. Our Knowledge Learning-based demosaicing model for Adaptive Patterns, namely KLAP, utilizes CFA-adaptive filters for only 1% key filters in the network for each CFA, but still manages to effectively demosaic all the CFAs, yielding comparable performance to the large-scale models. Furthermore, by employing meta-learning during inference (KLAP-M), our model is able to eliminate unknown sensor-generic artifacts in real RAW data, effectively bridging the gap between synthetic images and real sensor RAW. Our KLAP and KLAP-M methods achieved state-of-the-art demosaicing performance in both synthetic and real RAW data of Bayer and non-Bayer CFAs. △ Less

Submitted 20 July, 2023; originally announced July 2023.

PyNET-QxQ: An Efficient PyNET Variant for QxQ Bayer Pattern Demosaicing in CMOS Image Sensors

Authors: Minhyeok Cho, Haechang Lee, Hyunwoo Je, Kijeong Kim, Dongil Ryu, Albert No

Abstract: Deep learning-based image signal processor (ISP) models for mobile cameras can generate high-quality images that rival those of professional DSLR cameras. However, their computational demands often make them unsuitable for mobile settings. Additionally, modern mobile cameras employ non-Bayer color filter arrays (CFA) such as Quad Bayer, Nona Bayer, and QxQ Bayer to enhance image quality, yet most… ▽ More Deep learning-based image signal processor (ISP) models for mobile cameras can generate high-quality images that rival those of professional DSLR cameras. However, their computational demands often make them unsuitable for mobile settings. Additionally, modern mobile cameras employ non-Bayer color filter arrays (CFA) such as Quad Bayer, Nona Bayer, and QxQ Bayer to enhance image quality, yet most existing deep learning-based ISP (or demosaicing) models focus primarily on standard Bayer CFAs. In this study, we present PyNET-QxQ, a lightweight demosaicing model specifically designed for QxQ Bayer CFA patterns, which is derived from the original PyNET. We also propose a knowledge distillation method called progressive distillation to train the reduced network more effectively. Consequently, PyNET-QxQ contains less than 2.5% of the parameters of the original PyNET while preserving its performance. Experiments using QxQ images captured by a proto type QxQ camera sensor show that PyNET-QxQ outperforms existing conventional algorithms in terms of texture and edge reconstruction, despite its significantly reduced parameter count. △ Less

Submitted 5 May, 2023; v1 submitted 8 March, 2022; originally announced March 2022.

Comments: Accepted by IEEE Access

ShortcutFusion: From Tensorflow to FPGA-based accelerator with reuse-aware memory allocation for shortcut data

Authors: Duy Thanh Nguyen, Hyeonseung Je, Tuan Nghia Nguyen, Soojung Ryu, Kyujoong Lee, Hyuk-Jae Lee

Abstract: Residual block is a very common component in recent state-of-the art CNNs such as EfficientNet or EfficientDet. Shortcut data accounts for nearly 40% of feature-maps access in ResNet152 [8]. Most of the previous DNN compilers, accelerators ignore the shortcut data optimization. This paper presents ShortcutFusion, an optimization tool for FPGA-based accelerator with a reuse-aware static memory allo… ▽ More Residual block is a very common component in recent state-of-the art CNNs such as EfficientNet or EfficientDet. Shortcut data accounts for nearly 40% of feature-maps access in ResNet152 [8]. Most of the previous DNN compilers, accelerators ignore the shortcut data optimization. This paper presents ShortcutFusion, an optimization tool for FPGA-based accelerator with a reuse-aware static memory allocation for shortcut data, to maximize on-chip data reuse given resource constraints. From TensorFlow DNN models, the proposed design generates instruction sets for a group of nodes which uses an optimized data reuse for each residual block. The accelerator design implemented on the Xilinx KCU1500 FPGA card 2.8x faster and 9.9x more power efficient than NVIDIA RTX 2080 Ti for 256x256 input size. . Compared to the result from baseline, in which the weights, inputs, and outputs are accessed from the off-chip memory exactly once per each layer, ShortcutFusion reduces the DRAM access by 47.8-84.8% for RetinaNet, Yolov3, ResNet152, and EfficientNet. Given a similar buffer size to ShortcutMining [8], which also mine the shortcut data in hardware, the proposed work reduces off-chip access for feature-maps 5.27x while accessing weight from off-chip memory exactly once. △ Less

Submitted 13 February, 2022; v1 submitted 15 June, 2021; originally announced June 2021.

Comments: Accepted for publication in IEEE Transaction on Circuits and Systems I: Regular Papers

Drop Impact on Hot Plates: Contact times, Lift-off and the Lamella Rupture

Authors: Sang-Hyeon Lee, Kirsten Harth, Maaike Rump, Minwoo Kim, Detlef Lohse, Kamel Fezzaa, Jung Ho Je

Abstract: When a liquid drop impacts on a heated substrate, it can remain deposited, or violently boil in contact, or lift off with or without ever touching the surface. The latter is known as the Leidenfrost effect. The duration and area of the liquid--substrate contact is highly relevant for the heat transfer, as well as other effects such as corrosion. However, most experimental studies rely on side view… ▽ More When a liquid drop impacts on a heated substrate, it can remain deposited, or violently boil in contact, or lift off with or without ever touching the surface. The latter is known as the Leidenfrost effect. The duration and area of the liquid--substrate contact is highly relevant for the heat transfer, as well as other effects such as corrosion. However, most experimental studies rely on side view imaging to determine contact times, and those are often mixed with the time until the drop lifts off from the substrate. Here, we develop and validate a reliable method of contact time determination using high-speed X-ray and Total Internal Reflection measurements. We exemplarily compare contact and lift-off times on flat silicon and sapphire substrates. We show that drops can rebound even without formation of a complete vapor layer, with a wide range of lift-off times. On sapphire, we find a local minimum of lift-off times much shorter than by capillary rebound in the comparatively low-temperature regime of transition boiling / thermal atomization. We elucidate the underlying mechanism related to spontaneous rupture of the lamella and receding of the contact area. △ Less

Submitted 14 July, 2020; v1 submitted 16 March, 2020; originally announced March 2020.

Comments: S. H. Lee and K. Harth contributed equally to this work

Journal ref: Soft Matter 16 pp. 7935-7949 (2020)

Chiral nematic self-assembly of minimally surface damaged chitin nanofibrils and its load bearing functions

Authors: Dongyeop X. Oh, Yun Jeong Cha, Hoang-Linh Nguyen, Hwa Heon Je, Yong Seok Jho, Dong Soo Hwang, Dong Ki Yoon

Abstract: Chitin is one of the most abundant biomaterials in nature, with 10^10 tons produced annually as hierarchically organized nanofibril fillers to reinforce the exoskeletons of arthropods. This green and cheap biomaterial has attracted great attention due to its potential application to reinforce biomedical materials. Despite that, its practical use is limited since the extraction of chitin nanofibril… ▽ More Chitin is one of the most abundant biomaterials in nature, with 10^10 tons produced annually as hierarchically organized nanofibril fillers to reinforce the exoskeletons of arthropods. This green and cheap biomaterial has attracted great attention due to its potential application to reinforce biomedical materials. Despite that, its practical use is limited since the extraction of chitin nanofibrils requires surface modification involving harsh chemical treatments, leading to difficulties in reproducing their natural prototypal hierarchical structure, i.e. chiral nematic phase. Here, we develop a chemical etching-free approach using calcium ions, called natural way, to disintegrate the chitin nanofibrils while kee** the essential moiety for the self-assembly, ultimately resulting in the reproduction of chitins natural chiral structure in a polymeric matrix. This chiral chitin nanostructure exceptionally toughens the composite. Our resultant chiral nematic phase of chitin materials can contribute to the understanding and use of the reinforcing strategy in nature. △ Less

Submitted 24 March, 2016; originally announced March 2016.

Comments: 29 pages, 4 figures

Journal ref: Sci. Rep., 2016, 6, 23245

"Dust, Ice, and Gas In Time" (DIGIT) Herschel Observations of GSS30-IRS1 in Ophiuchus

Authors: Hyerin Je, Jeong-Eun Lee, Seokho Lee, Joel D. Green, Neal J. Evans II

Abstract: As a part of the "Dust, Ice, and Gas In Time" (DIGIT) key program on Herschel, we observed GSS30-IRS1, a Class I protostar located in Ophiuchus (d = 120 pc), with Herschel/Photodetector Array Camera and Spectrometer (PACS). More than 70 lines were detected within a wavelength range from 50 micron to 200 micron, including CO, H2O, OH, and two atomic [O I] lines at 63 and 145 micron. The [C II] line… ▽ More As a part of the "Dust, Ice, and Gas In Time" (DIGIT) key program on Herschel, we observed GSS30-IRS1, a Class I protostar located in Ophiuchus (d = 120 pc), with Herschel/Photodetector Array Camera and Spectrometer (PACS). More than 70 lines were detected within a wavelength range from 50 micron to 200 micron, including CO, H2O, OH, and two atomic [O I] lines at 63 and 145 micron. The [C II] line, known as a tracer of externally heated gas by the interstellar radiation field, is also detected at 158 micron. All lines, except [O I] and [C II], are detected only at the central spaxel of 9.4" X 9.4". The [O I] emissions are extended along a NE-SW orientation, and the [C II] line is detected over all spaxels, indicative of external PDR. The total [C II] intensity around GSS30 reveals that the far-ultraviolet radiation field is in the range of 3 to 20 G0, where G0 is in units of the Habing Field, 1.6 X 10^{-3} erg cm^{-2} s^{-1}. This enhanced external radiation field heats the envelope of GSS30-IRS1, causing the continuum emission to be extended, unlike the molecular emission. The best-fit continuum model of GSS30-IRS1 with the physical structure including flared disk, envelope, and outflow shows that the internal luminosity is 10 Lsun, and the region is externally heated by a radiation field enhanced by a factor of 130 compared to the standard local interstellar radiation field. △ Less

Submitted 20 January, 2015; originally announced January 2015.

Comments: 63 pages, 25 figures, accepted for publication in ApJS

Shedding new light on the mystery of wetting on soft solids

Authors: Su Ji Park, Byung Mook Weon, Ji San Lee, Junho Lee, **kyung Kim, Jung Ho Je

Abstract: One of the most questionable issues in wetting is the vertical force balance that is excluded in Young's law. On soft deformable solids, such as biotic materials and synthetic polymers, the vertical force of liquid leads to a microscopic protrusion of the contact line, i.e. a "wetting ridge". The wetting principle that determines the geometry of the ridge-tip is at the heart of the issues over the… ▽ More One of the most questionable issues in wetting is the vertical force balance that is excluded in Young's law. On soft deformable solids, such as biotic materials and synthetic polymers, the vertical force of liquid leads to a microscopic protrusion of the contact line, i.e. a "wetting ridge". The wetting principle that determines the geometry of the ridge-tip is at the heart of the issues over the past half century. Here, we reveal a universal wetting principle by directly visualizing ridge-tips with high spatio-temporal resolution using x-ray microscopy. We find that the tip-geometry is asymmetric and bent, and invariant during ridge growth or by surface softness. This singular geometry is explained by linking the macroscopic and microscopic contact angles to Young's and Neumann's laws, respectively. Our dual-scale model would be applicable to a general framework in wetting and give new clues to various issues in cell-substrate interaction and elasto-capillary problems. △ Less

Submitted 28 December, 2013; originally announced December 2013.

Comments: 5 pages, 4 figures, with supplementary information (1 figure, 2 tables), submitted to Nature Communications

Patterning droplets with durotaxis

Authors: Robert W. Style, Yonglu Che, Su Ji Park, Byung Mook Weon, Jung Ho Je, Callen Hyland, Guy K. German, Michael Power, Larry A. Wilen, John S. Wettlaufer, Eric R. Dufresne

Abstract: Numerous cell-types have shown a remarkable ability to detect and move along gradients in stiffness of an underlying substrate -- a process known as durotaxis. The mechanisms underlying durotaxis are still unresolved, but generally believed to involve active sensing and locomotion. Here, we show that simple liquid droplets also undergo durotaxis. By modulating substrate stiffness, we obtain fine c… ▽ More Numerous cell-types have shown a remarkable ability to detect and move along gradients in stiffness of an underlying substrate -- a process known as durotaxis. The mechanisms underlying durotaxis are still unresolved, but generally believed to involve active sensing and locomotion. Here, we show that simple liquid droplets also undergo durotaxis. By modulating substrate stiffness, we obtain fine control of droplet position on soft, flat substrates. Unlike other control mechanisms, droplet durotaxis works without imposing chemical, thermal, electrical or topographical gradients. This enables a new approach to large-scale droplet patterning and is potentially useful for many applications, such as microfluidics, thermal control and microfabrication. △ Less

Submitted 23 May, 2013; v1 submitted 15 April, 2013; originally announced April 2013.

Journal ref: Proc. Nat. Acad. Sci. 110, 12541 (2013)

Mathematical link of evolving aging and complexity

Authors: Byung Mook Weon, Jung Ho Je

Abstract: Aging is a fundamental aspect of living systems that undergo a progressive deterioration of physiological function with age and an increase of vulnerability to disease and death. Living systems, known as complex systems, require complexity in interactions among molecules, cells, organs, and individuals or regulatory mechanisms to perform a variety of activities for survival. On this basis, aging c… ▽ More Aging is a fundamental aspect of living systems that undergo a progressive deterioration of physiological function with age and an increase of vulnerability to disease and death. Living systems, known as complex systems, require complexity in interactions among molecules, cells, organs, and individuals or regulatory mechanisms to perform a variety of activities for survival. On this basis, aging can be understood in terms of a progressive loss of complexity with age; this suggests that complexity in living systems would evolve with age. In general, aging dynamics is mathematically depicted by a survival function, which monotonically changes from 1 to 0 with age. It would be then useful to find an adequate survival function to link aging dynamics and complexity evolution. Here we describe a flexible survival function, which is derived from the stretched exponential function by adopting an age-dependent exponent. We note that the exponent is associated with evolving complexity, i.e., a fractal-like scaling in cumulative mortality. The survival function well depicts a general feature in survival curves; healthy populations show a tendency to evolve towards rectangular-like survival curves, as examples in humans or laboratory animals. This tendency suggests that both aging and complexity would evolve towards healthy survival in living systems. Our function to link aging with complexity may contribute to better understanding of biological aging in terms of complexity evolution. △ Less

Submitted 12 November, 2010; originally announced November 2010.

Comments: 14 pages, 5 figures, submitted to PLoS ONE

Predicting Human Lifespan Limits

Authors: Byung Mook Weon, Jung Ho Je

Abstract: Recent discoveries show steady improvements in life expectancy during modern decades. Does this support that humans continue to live longer in future? We recently put forward the maximum survival tendency, as found in survival curves of industrialized countries, which is described by extended Weibull model with age-dependent stretched exponent. The maximum survival tendency suggests that human s… ▽ More Recent discoveries show steady improvements in life expectancy during modern decades. Does this support that humans continue to live longer in future? We recently put forward the maximum survival tendency, as found in survival curves of industrialized countries, which is described by extended Weibull model with age-dependent stretched exponent. The maximum survival tendency suggests that human survival dynamics may possess its intrinsic limit, beyond which survival is inevitably forbidden. Based on such tendency, we develop the model and explore the patterns in the maximum lifespan limits from industrialized countries during recent three decades. This analysis strategy is simple and useful to interpret the complicated human survival dynamics. △ Less

Submitted 27 August, 2009; v1 submitted 24 August, 2009; originally announced August 2009.

Comments: 11 pages, 3 figures, 2 tables; Natural Science (in press)