Careers | Phone Book | A - Z Index

Sifting Through a Trillion Electrons

CRDhpTrillion.jpg

SDM's Surendra Byna and colleagues from Berkeley Lab’s Computational Research Division teamed up with researchers to develop novel software strategies for storing, mining, and analyzing massive datasets generated by a state-of-the-art plasma physics code called VPIC. » Read More

Catching Turbulence in the Solar Wind

bz-322-full.jpg

Massive datasets plus modelling, visualization and analytics allow researchers to "see" the unseen: the turbulence in solar winds. » Read More

Arie Shoshani Earns Lifetime Achievement Award

Arie award

More than 25 years ago, Arie Shoshani realized that researchers were facing significant challenges in organizing, managing and analyzing their scientific data. He set out to develop computer applications to help them better meet the challenges and created the Scientific Data Management Group in the process. » Read More

The Scientific Data Management (SDM) group develops technologies and tools for efficient data access and storage management of massive scientific datasets. We are currently developing storage resource management tools, data querying technologies, in situ feature extraction algorithms, along with software platforms for exascale data. The group also works closely with application scientists to address their data processing challenges. These tools and application development activities are backed by active research efforts on novel algorithms for emerging hardware platforms.

Group Leader: John Wu

»Visit the Scientific Data Management (SDM) site.

SDM Publications

Enhancing IoT Anomaly Detection Performance for Federated Learning

December 17, 2020

Botnets Detection Using Recurrent Variational Autoencoder

December 7, 2020

SDN for End-to-end Networked Science at the Exascale

December 1, 2020

Enhancing IoT Anomaly Detection Performance for Federated Learning

November 18, 2020

Towards HPC I/O performance prediction through large-scale log analysis

June 24, 2020

Transfer Learning Approach for Botnet Detection based on Recurrent Variational Autoencoder

June 23, 2020

Evaluation of Deep Learning Models for Network PerformancePrediction for Scientific Facilities

June 23, 2020

HPC Workload Characterization using Feature Selection and Clustering

June 23, 2020

Feature Selection and Tree-based Classification for Wireless Intrusion Detection

June 23, 2020

A Deep Deterministic Policy Gradient Based Network Scheduler For Deadline-Driven Data Transfers

June 22, 2020

DASSA: Parallel DAS Data Storage and Analysis for Subsurface Event Detection

May 15, 2020

Predicting Resource Requirement in Intermediate Palomar Transient Factory Workflow

May 11, 2020

BBOS: Efficient HPC Storage Management via Burst Buffer Over-Subscription

May 11, 2020

ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems

February 2, 2020

Life Course as a Contextual System to Investigate the Effects of Life Events, Gender, and Generation on Travel Mode Use

January 12, 2020

Analysis in the Data Path of an Object-centric Data Management System

December 18, 2019

Tuning Object-centric Data Management Systems for Large Scale Scientific Applications

December 18, 2019

Exploring Metadata Search Essentials for Scientific Data Management

December 17, 2019

Spatiotemporal Real-Time Anomaly Detection for Supercomputing Systems

December 10, 2019

Identifying Time Series Similarity in Large-Scale Earth System Datasets

December 10, 2019

More from SDM Publications »