-
Design choices for productive, secure, data-intensive research at scale in the cloud
Authors:
Diego Arenas,
Jon Atkins,
Claire Austin,
David Beavan,
Alvaro Cabrejas Egea,
Steven Carlysle-Davies,
Ian Carter,
Rob Clarke,
James Cunningham,
Tom Doel,
Oliver Forrest,
Evelina Gabasova,
James Geddes,
James Hetherington,
Radka Jersakova,
Franz Kiraly,
Catherine Lawrence,
Jules Manser,
Martin T. O'Reilly,
James Robinson,
Helen Sherwood-Taylor,
Serena Tierney,
Catalina A. Vallejos,
Sebastian Vollmer,
Kirstie Whitaker
Abstract:
We present a policy and process framework for secure environments for productive data science research projects at scale, by combining prevailing data security threat and risk profiles into five sensitivity tiers, and, at each tier, specifying recommended policies for data classification, data ingress, software ingress, data egress, user access, user device control, and analysis environments. By p…
▽ More
We present a policy and process framework for secure environments for productive data science research projects at scale, by combining prevailing data security threat and risk profiles into five sensitivity tiers, and, at each tier, specifying recommended policies for data classification, data ingress, software ingress, data egress, user access, user device control, and analysis environments. By presenting design patterns for security choices for each tier, and using software defined infrastructure so that a different, independent, secure research environment can be instantiated for each project appropriate to its classification, we hope to maximise researcher productivity and minimise risk, allowing research organisations to operate with confidence.
△ Less
Submitted 15 September, 2019; v1 submitted 23 August, 2019;
originally announced August 2019.
-
Collaborative Low-Rank Subspace Clustering
Authors:
Stephen Tierney,
Yi Guo,
Junbin Gao
Abstract:
In this paper we present Collaborative Low-Rank Subspace Clustering. Given multiple observations of a phenomenon we learn a unified representation matrix. This unified matrix incorporates the features from all the observations, thus increasing the discriminative power compared with learning the representation matrix on each observation separately. Experimental evaluation shows that our method outp…
▽ More
In this paper we present Collaborative Low-Rank Subspace Clustering. Given multiple observations of a phenomenon we learn a unified representation matrix. This unified matrix incorporates the features from all the observations, thus increasing the discriminative power compared with learning the representation matrix on each observation separately. Experimental evaluation shows that our method outperforms subspace clustering on separate observations and the state of the art collaborative learning algorithm.
△ Less
Submitted 12 April, 2017;
originally announced April 2017.
-
Tractable Clustering of Data on the Curve Manifold
Authors:
Stephen Tierney,
Junbin Gao,
Yi Guo,
Zheng Zhang
Abstract:
In machine learning it is common to interpret each data point as a vector in Euclidean space. However the data may actually be functional i.e.\ each data point is a function of some variable such as time and the function is discretely sampled. The naive treatment of functional data as traditional multivariate data can lead to poor performance since the algorithms are ignoring the correlation in th…
▽ More
In machine learning it is common to interpret each data point as a vector in Euclidean space. However the data may actually be functional i.e.\ each data point is a function of some variable such as time and the function is discretely sampled. The naive treatment of functional data as traditional multivariate data can lead to poor performance since the algorithms are ignoring the correlation in the curvature of each function. In this paper we propose a tractable method to cluster functional data or curves by adapting the Euclidean Low-Rank Representation (LRR) to the curve manifold. Experimental evaluation on synthetic and real data reveals that this method massively outperforms prior clustering methods in both speed and accuracy when clustering functional data.
△ Less
Submitted 12 April, 2017;
originally announced April 2017.
-
Efficient Sparse Subspace Clustering by Nearest Neighbour Filtering
Authors:
Stephen Tierney,
Yi Guo,
Junbin Gao
Abstract:
Sparse Subspace Clustering (SSC) has been used extensively for subspace identification tasks due to its theoretical guarantees and relative ease of implementation. However SSC has quadratic computation and memory requirements with respect to the number of input data points. This burden has prohibited SSCs use for all but the smallest datasets. To overcome this we propose a new method, k-SSC, that…
▽ More
Sparse Subspace Clustering (SSC) has been used extensively for subspace identification tasks due to its theoretical guarantees and relative ease of implementation. However SSC has quadratic computation and memory requirements with respect to the number of input data points. This burden has prohibited SSCs use for all but the smallest datasets. To overcome this we propose a new method, k-SSC, that screens out a large number of data points to both reduce SSC to linear memory and computational requirements. We provide theoretical analysis for the bounds of success for k-SSC. Our experiments show that k-SSC exceeds theoretical expectations and outperforms existing SSC approximations by maintaining the classification performance of SSC. Furthermore in the spirit of reproducible research we have publicly released the source code for k-SSC
△ Less
Submitted 12 April, 2017;
originally announced April 2017.
-
Low-Rank Representation over the Manifold of Curves
Authors:
Stephen Tierney,
Junbin Gao,
Yi Guo,
Zhengwu Zhang
Abstract:
In machine learning it is common to interpret each data point as a vector in Euclidean space. However the data may actually be functional i.e.\ each data point is a function of some variable such as time and the function is discretely sampled. The naive treatment of functional data as traditional multivariate data can lead to poor performance since the algorithms are ignoring the correlation in th…
▽ More
In machine learning it is common to interpret each data point as a vector in Euclidean space. However the data may actually be functional i.e.\ each data point is a function of some variable such as time and the function is discretely sampled. The naive treatment of functional data as traditional multivariate data can lead to poor performance since the algorithms are ignoring the correlation in the curvature of each function. In this paper we propose a method to analyse subspace structure of the functional data by using the state of the art Low-Rank Representation (LRR). Experimental evaluation on synthetic and real data reveals that this method massively outperforms conventional LRR in tasks concerning functional data.
△ Less
Submitted 6 January, 2016; v1 submitted 4 January, 2016;
originally announced January 2016.
-
Segmentation of Subspaces in Sequential Data
Authors:
Stephen Tierney,
Yi Guo,
Junbin Gao
Abstract:
We propose Ordered Subspace Clustering (OSC) to segment data drawn from a sequentially ordered union of subspaces. Similar to Sparse Subspace Clustering (SSC) we formulate the problem as one of finding a sparse representation but include an additional penalty term to take care of sequential data. We test our method on data drawn from infrared hyper spectral, video and motion capture data. Experime…
▽ More
We propose Ordered Subspace Clustering (OSC) to segment data drawn from a sequentially ordered union of subspaces. Similar to Sparse Subspace Clustering (SSC) we formulate the problem as one of finding a sparse representation but include an additional penalty term to take care of sequential data. We test our method on data drawn from infrared hyper spectral, video and motion capture data. Experiments show that our method, OSC, outperforms the state of the art methods: Spatial Subspace Clustering (SpatSC), Low-Rank Representation (LRR) and SSC.
△ Less
Submitted 15 April, 2015;
originally announced April 2015.