-
The ImmersaDesk3 -- Experiences With A Flat Panel Display for Virtual Reality
Authors:
Dave Pape,
Josephine Anstey,
Mike Bogucki,
Greg Dawe,
Tom DeFanti,
Andy Johnson,
Dan Sandin
Abstract:
In this paper we discuss the design and implementation of a plasma display panel for a wide field of view desktop virtual reality environment. Present commercial plasma displays are not designed with virtual reality in mind, leading to several problems in generating stereo imagery and obtaining good tracking information. Although we developed solutions for a number of these problems, the limitatio…
▽ More
In this paper we discuss the design and implementation of a plasma display panel for a wide field of view desktop virtual reality environment. Present commercial plasma displays are not designed with virtual reality in mind, leading to several problems in generating stereo imagery and obtaining good tracking information. Although we developed solutions for a number of these problems, the limitations of the system preclude its current use in practical applications, and point to issues that must be resolved for flat panel displays to be useful for VR.
△ Less
Submitted 16 November, 2023;
originally announced November 2023.
-
Towards a Dynamic Composability Approach for using Heterogeneous Systems in Remote Sensing
Authors:
Ilkay Altintas,
Ismael Perez,
Dmitry Mishin,
Adrien Trouillaud,
Christopher Irving,
John Graham,
Mahidhar Tatineni,
Thomas DeFanti,
Shawn Strande,
Larry Smarr,
Michael L. Norman
Abstract:
Influenced by the advances in data and computing, the scientific practice increasingly involves machine learning and artificial intelligence driven methods which requires specialized capabilities at the system-, science- and service-level in addition to the conventional large-capacity supercomputing approaches. The latest distributed architectures built around the composability of data-centric app…
▽ More
Influenced by the advances in data and computing, the scientific practice increasingly involves machine learning and artificial intelligence driven methods which requires specialized capabilities at the system-, science- and service-level in addition to the conventional large-capacity supercomputing approaches. The latest distributed architectures built around the composability of data-centric applications led to the emergence of a new ecosystem for container coordination and integration. However, there is still a divide between the application development pipelines of existing supercomputing environments, and these new dynamic environments that disaggregate fluid resource pools through accessible, portable and re-programmable interfaces. New approaches for dynamic composability of heterogeneous systems are needed to further advance the data-driven scientific practice for the purpose of more efficient computing and usable tools for specific scientific domains. In this paper, we present a novel approach for using composable systems in the intersection between scientific computing, artificial intelligence (AI), and remote sensing domain. We describe the architecture of a first working example of a composable infrastructure that federates Expanse, an NSF-funded supercomputer, with Nautilus, a Kubernetes-based GPU geo-distributed cluster. We also summarize a case study in wildfire modeling, that demonstrates the application of this new infrastructure in scientific workflows: a composed system that bridges the insights from edge sensing, AI and computing capabilities with a physics-driven simulation.
△ Less
Submitted 13 November, 2022;
originally announced November 2022.
-
Auto-scaling HTCondor pools using Kubernetes compute resources
Authors:
Igor Sfiligoi,
Thomas DeFanti,
Frank Würthwein
Abstract:
HTCondor has been very successful in managing globally distributed, pleasantly parallel scientific workloads, especially as part of the Open Science Grid. HTCondor system design makes it ideal for integrating compute resources provisioned from anywhere, but it has very limited native support for autonomously provisioning resources managed by other solutions. This work presents a solution that allo…
▽ More
HTCondor has been very successful in managing globally distributed, pleasantly parallel scientific workloads, especially as part of the Open Science Grid. HTCondor system design makes it ideal for integrating compute resources provisioned from anywhere, but it has very limited native support for autonomously provisioning resources managed by other solutions. This work presents a solution that allows for autonomous, demand-driven provisioning of Kubernetes-managed resources. A high-level overview of the employed architectures is presented, paired with the description of the setups used in both on-prem and Cloud deployments in support of several Open Science Grid communities. The experience suggests that the described solution should be generally suitable for contributing Kubernetes-based resources to existing HTCondor pools.
△ Less
Submitted 2 May, 2022;
originally announced May 2022.
-
HTCondor data movement at 100 Gbps
Authors:
Igor Sfiligoi,
Frank Würthwein,
Thomas DeFanti,
John Graham
Abstract:
HTCondor is a major workload management system used in distributed high throughput computing (dHTC) environments, e.g., the Open Science Grid. One of the distinguishing features of HTCondor is the native support for data movement, allowing it to operate without a shared filesystem. Coupling data handling and compute scheduling is both convenient for users and allows for significant infrastructure…
▽ More
HTCondor is a major workload management system used in distributed high throughput computing (dHTC) environments, e.g., the Open Science Grid. One of the distinguishing features of HTCondor is the native support for data movement, allowing it to operate without a shared filesystem. Coupling data handling and compute scheduling is both convenient for users and allows for significant infrastructure flexibility but does introduce some limitations. The default HTCondor data transfer mechanism routes both the input and output data through the submission node, making it a potential bottleneck. In this document we show that by using a node equipped with a 100 Gbps network interface (NIC) HTCondor can serve data at up to 90 Gbps, which is sufficient for most current use cases, as it would saturate the border network links of most research universities at the time of writing.
△ Less
Submitted 8 July, 2021;
originally announced July 2021.
-
Workflow-Driven Distributed Machine Learning in CHASE-CI: A Cognitive Hardware and Software Ecosystem Community Infrastructure
Authors:
Ilkay Altintas,
Kyle Marcus,
Isaac Nealey,
Scott L. Sellars,
John Graham,
Dima Mishin,
Joel Polizzi,
Daniel Crawl,
Thomas DeFanti,
Larry Smarr
Abstract:
The advances in data, computing and networking over the last two decades led to a shift in many application domains that includes machine learning on big data as a part of the scientific process, requiring new capabilities for integrated and distributed hardware and software infrastructure. This paper contributes a workflow-driven approach for dynamic data-driven application development on top of…
▽ More
The advances in data, computing and networking over the last two decades led to a shift in many application domains that includes machine learning on big data as a part of the scientific process, requiring new capabilities for integrated and distributed hardware and software infrastructure. This paper contributes a workflow-driven approach for dynamic data-driven application development on top of a new kind of networked Cyberinfrastructure called CHASE-CI. In particular, we present: 1) The architecture for CHASE-CI, a network of distributed fast GPU appliances for machine learning and storage managed through Kubernetes on the high-speed (10-100Gbps) Pacific Research Platform (PRP); 2) A machine learning software containerization approach and libraries required for turning such a network into a distributed computer for big data analysis; 3) An atmospheric science case study that can only be made scalable with an infrastructure like CHASE-CI; 4) Capabilities for virtual cluster management for data communication and analysis in a dynamically scalable fashion, and visualization across the network in specialized visualization facilities in near real-time; and, 5) A step-by-step workflow and performance measurement approach that enables taking advantage of the dynamic architecture of the CHASE-CI network and container management infrastructure.
△ Less
Submitted 25 February, 2019;
originally announced March 2019.
