-
Building a temperature forecasting model for the city with the regression neural network (RNN)
Authors:
Nguyen Phuc Tran,
Duy Thanh Tran,
Thi Thuy Nga Duong
Abstract:
In recent years, a study by environmental organizations in the world and Vietnam shows that weather change is quite complex. global warming has become a serious problem in the modern world, which is a concern for scientists. last century, it was difficult to forecast the weather due to missing weather monitoring stations and technological limitations. this made it hard to collect data for building…
▽ More
In recent years, a study by environmental organizations in the world and Vietnam shows that weather change is quite complex. global warming has become a serious problem in the modern world, which is a concern for scientists. last century, it was difficult to forecast the weather due to missing weather monitoring stations and technological limitations. this made it hard to collect data for building predictive models to make accurate simulations. in Vietnam, research on weather forecast models is a recent development, having only begun around 2000. along with advancements in computer science, mathematical models are being built and applied with machine learning techniques to create more accurate and reliable predictive models. this article will summarize the research and solutions for applying recurrent neural networks to forecast urban temperatures.
△ Less
Submitted 27 May, 2024;
originally announced May 2024.
-
Ant Colony Optimization for Cooperative Inspection Path Planning Using Multiple Unmanned Aerial Vehicles
Authors:
Duy Nam Bui,
Thuy Ngan Duong,
Manh Duong Phung
Abstract:
This paper presents a new swarm intelligence-based approach to deal with the cooperative path planning problem of unmanned aerial vehicles (UAVs), which is essential for the automatic inspection of infrastructure. The approach uses a 3D model of the structure to generate viewpoints for the UAVs. The calculation of the viewpoints considers the constraints related to the UAV formation model, camera…
▽ More
This paper presents a new swarm intelligence-based approach to deal with the cooperative path planning problem of unmanned aerial vehicles (UAVs), which is essential for the automatic inspection of infrastructure. The approach uses a 3D model of the structure to generate viewpoints for the UAVs. The calculation of the viewpoints considers the constraints related to the UAV formation model, camera parameters, and requirements for data post-processing. The viewpoints are then used as input to formulate the path planning as an extended traveling salesman problem and the definition of a new cost function. Ant colony optimization is finally used to solve the problem to yield optimal inspection paths. Experiments with 3D models of real structures have been conducted to evaluate the performance of the proposed approach. The results show that our system is not only capable of generating feasible inspection paths for UAVs but also reducing the path length by 29.47\% for complex structures when compared with another heuristic approach. The source code of the algorithm can be found at https://github.com/duynamrcv/aco_3d_ipp.
△ Less
Submitted 13 February, 2024;
originally announced February 2024.
-
Infinitesimal CR automorphisms and stability groups of nonminimal infinite type models in $\mathbb C^2$
Authors:
Van Thu Ninh,
Thi Ngoc Oanh Duong,
Van Hoang Pham,
Hyeseon Kim
Abstract:
We determine infinitesimal $\mathrm{CR}$ automorphisms and stability groups of real hypersurfaces in $\mathbb C^2$ in the case when the hypersurface is nonminimal and of infinite type at the reference point.
We determine infinitesimal $\mathrm{CR}$ automorphisms and stability groups of real hypersurfaces in $\mathbb C^2$ in the case when the hypersurface is nonminimal and of infinite type at the reference point.
△ Less
Submitted 20 April, 2020; v1 submitted 30 August, 2019;
originally announced August 2019.
-
GraphMP: I/O-Efficient Big Graph Analytics on a Single Commodity Machine
Authors:
Peng Sun,
Yonggang Wen,
Ta Nguyen Binh Duong,
Xiaokui Xiao
Abstract:
Recent studies showed that single-machine graph processing systems can be as highly competitive as cluster-based approaches on large-scale problems. While several out-of-core graph processing systems and computation models have been proposed, the high disk I/O overhead could significantly reduce performance in many practical cases. In this paper, we propose GraphMP to tackle big graph analytics on…
▽ More
Recent studies showed that single-machine graph processing systems can be as highly competitive as cluster-based approaches on large-scale problems. While several out-of-core graph processing systems and computation models have been proposed, the high disk I/O overhead could significantly reduce performance in many practical cases. In this paper, we propose GraphMP to tackle big graph analytics on a single machine. GraphMP achieves low disk I/O overhead with three techniques. First, we design a vertex-centric sliding window (VSW) computation model to avoid reading and writing vertices on disk. Second, we propose a selective scheduling method to skip loading and processing unnecessary edge shards on disk. Third, we use a compressed edge cache mechanism to fully utilize the available memory of a machine to reduce the amount of disk accesses for edges. Extensive evaluations have shown that GraphMP could outperform existing single-machine out-of-core systems such as GraphChi, X-Stream and GridGraph by up to 51, and can be as highly competitive as distributed graph engines like Pregel+, PowerGraph and Chaos.
△ Less
Submitted 18 February, 2019; v1 submitted 9 October, 2018;
originally announced October 2018.
-
GraphMP: An Efficient Semi-External-Memory Big Graph Processing System on a Single Machine
Authors:
Peng Sun,
Yonggang Wen,
Ta Nguyen Binh Duong,
Xiaokui Xiao
Abstract:
Recent studies showed that single-machine graph processing systems can be as highly competitive as cluster-based approaches on large-scale problems. While several out-of-core graph processing systems and computation models have been proposed, the high disk I/O overhead could significantly reduce performance in many practical cases. In this paper, we propose GraphMP to tackle big graph analytics on…
▽ More
Recent studies showed that single-machine graph processing systems can be as highly competitive as cluster-based approaches on large-scale problems. While several out-of-core graph processing systems and computation models have been proposed, the high disk I/O overhead could significantly reduce performance in many practical cases. In this paper, we propose GraphMP to tackle big graph analytics on a single machine. GraphMP achieves low disk I/O overhead with three techniques. First, we design a vertex-centric sliding window (VSW) computation model to avoid reading and writing vertices on disk. Second, we propose a selective scheduling method to skip loading and processing unnecessary edge shards on disk. Third, we use a compressed edge cache mechanism to fully utilize the available memory of a machine to reduce the amount of disk accesses for edges. Extensive evaluations have shown that GraphMP could outperform state-of-the-art systems such as GraphChi, X-Stream and GridGraph by 31.6x, 54.5x and 23.1x respectively, when running popular graph applications on a billion-vertex graph.
△ Less
Submitted 9 July, 2017;
originally announced July 2017.
-
GraphH: High Performance Big Graph Analytics in Small Clusters
Authors:
Peng Sun,
Yonggang Wen,
Ta Nguyen Binh Duong,
Xiaokui Xiao
Abstract:
It is common for real-world applications to analyze big graphs using distributed graph processing systems. Popular in-memory systems require an enormous amount of resources to handle big graphs. While several out-of-core approaches have been proposed for processing big graphs on disk, the high disk I/O overhead could significantly reduce performance. In this paper, we propose GraphH to enable high…
▽ More
It is common for real-world applications to analyze big graphs using distributed graph processing systems. Popular in-memory systems require an enormous amount of resources to handle big graphs. While several out-of-core approaches have been proposed for processing big graphs on disk, the high disk I/O overhead could significantly reduce performance. In this paper, we propose GraphH to enable high-performance big graph analytics in small clusters. Specifically, we design a two-stage graph partition scheme to evenly divide the input graph into partitions, and propose a GAB (Gather-Apply-Broadcast) computation model to make each worker process a partition in memory at a time. We use an edge cache mechanism to reduce the disk I/O overhead, and design a hybrid strategy to improve the communication performance. GraphH can efficiently process big graphs in small clusters or even a single commodity server. Extensive evaluations have shown that GraphH could be up to 7.8x faster compared to popular in-memory systems, such as Pregel+ and PowerGraph when processing generic graphs, and more than 100x faster than recently proposed out-of-core systems, such as GraphD and Chaos when processing big graphs.
△ Less
Submitted 7 August, 2017; v1 submitted 16 May, 2017;
originally announced May 2017.
-
Towards Distributed Machine Learning in Shared Clusters: A Dynamically-Partitioned Approach
Authors:
Peng Sun,
Yonggang Wen,
Ta Nguyen Binh Duong,
Shengen Yan
Abstract:
Many cluster management systems (CMSs) have been proposed to share a single cluster with multiple distributed computing systems. However, none of the existing approaches can handle distributed machine learning (ML) workloads given the following criteria: high resource utilization, fair resource allocation and low sharing overhead. To solve this problem, we propose a new CMS named Dorm, incorporati…
▽ More
Many cluster management systems (CMSs) have been proposed to share a single cluster with multiple distributed computing systems. However, none of the existing approaches can handle distributed machine learning (ML) workloads given the following criteria: high resource utilization, fair resource allocation and low sharing overhead. To solve this problem, we propose a new CMS named Dorm, incorporating a dynamically-partitioned cluster management mechanism and an utilization-fairness optimizer. Specifically, Dorm uses the container-based virtualization technique to partition a cluster, runs one application per partition, and can dynamically resize each partition at application runtime for resource efficiency and fairness. Each application directly launches its tasks on the assigned partition without petitioning for resources frequently, so Dorm imposes flat sharing overhead. Extensive performance evaluations showed that Dorm could simultaneously increase the resource utilization by a factor of up to 2.32, reduce the fairness loss by a factor of up to 1.52, and speed up popular distributed ML applications by a factor of up to 2.72, compared to existing approaches. Dorm's sharing overhead is less than 5% in most cases.
△ Less
Submitted 21 April, 2017;
originally announced April 2017.
-
MetaFlow: a Scalable Metadata Lookup Service for Distributed File Systems in Data Centers
Authors:
Peng Sun,
Yonggang Wen,
Ta Nguyen Binh Duong,
Haiyong Xie
Abstract:
In large-scale distributed file systems, efficient meta- data operations are critical since most file operations have to interact with metadata servers first. In existing distributed hash table (DHT) based metadata management systems, the lookup service could be a performance bottleneck due to its significant CPU overhead. Our investigations showed that the lookup service could reduce system throu…
▽ More
In large-scale distributed file systems, efficient meta- data operations are critical since most file operations have to interact with metadata servers first. In existing distributed hash table (DHT) based metadata management systems, the lookup service could be a performance bottleneck due to its significant CPU overhead. Our investigations showed that the lookup service could reduce system throughput by up to 70%, and increase system latency by a factor of up to 8 compared to ideal scenarios. In this paper, we present MetaFlow, a scalable metadata lookup service utilizing software-defined networking (SDN) techniques to distribute lookup workload over network components. MetaFlow tackles the lookup bottleneck problem by leveraging B-tree, which is constructed over the physical topology, to manage flow tables for SDN-enabled switches. Therefore, metadata requests can be forwarded to appropriate servers using only switches. Extensive performance evaluations in both simulations and testbed showed that MetaFlow increases system throughput by a factor of up to 3.2, and reduce system latency by a factor of up to 5 compared to DHT-based systems. We also deployed MetaFlow in a distributed file system, and demonstrated significant performance improvement.
△ Less
Submitted 10 November, 2016; v1 submitted 4 November, 2016;
originally announced November 2016.
-
The block cipher NSABC (public domain)
Authors:
Alice Nguyenova-Stepanikova,
Tran Ngoc Duong
Abstract:
We introduce NSABC/w -- Nice-Structured Algebraic Block Cipher using w-bit word arithmetic, a 4w-bit analogous of Skipjack [NSA98] with 5w-bit key. The Skipjack's internal 4-round Feistel structure is replaced with a w-bit, 2-round cascade of a binary operation (x,z)\mapsto(x\boxdot z)\lll(w/2) that permutes a text word x under control of a key word z. The operation \boxdot, similarly to the multi…
▽ More
We introduce NSABC/w -- Nice-Structured Algebraic Block Cipher using w-bit word arithmetic, a 4w-bit analogous of Skipjack [NSA98] with 5w-bit key. The Skipjack's internal 4-round Feistel structure is replaced with a w-bit, 2-round cascade of a binary operation (x,z)\mapsto(x\boxdot z)\lll(w/2) that permutes a text word x under control of a key word z. The operation \boxdot, similarly to the multiplication in IDEA [LM91, LMM91], bases on an algebraic group over w-bit words, so it is also capable of decrypting by means of the inverse element of z in the group. The cipher utilizes a secret 4w-bit tweak -- an easily changeable parameter with unique value for each block encrypted under the same key [LRW02] -- that is derived from the block index and an additional 4w -bit key. A software implementation for w=64 takes circa 9 clock cycles per byte on x86-64 processors.
△ Less
Submitted 17 May, 2011;
originally announced May 2011.
