Careers | Phone Book | A - Z Index

Sifting Through a Trillion Electrons

CRDhpTrillion.jpg

SDM's Surendra Byna and colleagues from Berkeley Lab’s Computational Research Division teamed up with researchers to develop novel software strategies for storing, mining, and analyzing massive datasets generated by a state-of-the-art plasma physics code called VPIC. » Read More

Catching Turbulence in the Solar Wind

bz-322-full.jpg

Massive datasets plus modelling, visualization and analytics allow researchers to "see" the unseen: the turbulence in solar winds. » Read More

Arie Shoshani Earns Lifetime Achievement Award

Arie award

More than 25 years ago, Arie Shoshani realized that researchers were facing significant challenges in organizing, managing and analyzing their scientific data. He set out to develop computer applications to help them better meet the challenges and created the Scientific Data Management Group in the process. » Read More

The Scientific Data Management (SDM) group develops technologies and tools for efficient data access and storage management of massive scientific datasets. We are currently developing storage resource management tools, data querying technologies, in situ feature extraction algorithms, along with software platforms for exascale data. The group also works closely with application scientists to address their data processing challenges. These tools and application development activities are backed by active research efforts on novel algorithms for emerging hardware platforms.

Group Leader: John Wu

»Visit the Scientific Data Management (SDM) site.

SDM Publications

Automated Variable Selection for Network Anomaly Detection

July 1, 2021

Analyzing scientific data sharing patterns with in-network data caching

June 21, 2021

Access Patterns of Disk Cache for Large Scientific Archive

June 21, 2021

GPU-based Classification for Wireless Intrusion Detection

June 21, 2021

A Biologically Inspired Network for Learning from Streaming Data

June 21, 2021

Adaptive Stochastic Gradient Descent for Deep Learning on Heterogeneous CPU+GPU Architectures

May 17, 2021

Improving Botnet Detection with Recurrent Neural Network and Transfer Learning

April 26, 2021

Network Traffic Performance Analysis and Anomaly Detection using Supervised Machine Learning

March 31, 2021

An empirical study of I/O separation for burst buffers in HPC systems

February 1, 2021

Clustering Life Course to Understand the Heterogeneous Effects of Life Events, Gender and Generation on Habitual Travel Modes

December 19, 2020

Enhancing IoT Anomaly Detection Performance for Federated Learning

December 17, 2020

Effective Missing Value Imputation Methods for Building Monitoring Data

December 10, 2020

Combining Ambient Noise and Distributed Acoustic Sensing (DAS) Deployed on Dark Fiber Networks for High-resolution Imaging at the Basin Scale

December 9, 2020

Deep Learning for Surface Wave Identification in Distributed Acoustic Sensing Data

December 8, 2020

Botnets Detection Using Recurrent Variational Autoencoder

December 7, 2020

SDN for End-to-end Networked Science at the Exascale

December 1, 2020

Deep Reinforcement Learning based Control for two-dimensional Coherent Combining

December 1, 2020

Artificial Neural Networks Applied to Stabilization of 81-beam Coherent Combining

December 1, 2020

Enhancing IoT Anomaly Detection Performance for Federated Learning

November 18, 2020

The Superfacility project: automated pipelines for experiments and HPC

November 18, 2020

More from SDM Publications »