Skip to navigation Skip to content
Careers | Phone Book | A - Z Index

Publications

Deborah Agarwal

2023

Hector G. Martin, Tijana Radivojevic, Jeremy Zucker, Kristofer Bouchard, Jess Sustarich, Sean Peisert, Dan Arnold, Nathan Hillson, Gyorgy Babnigg, Jose M. Marti, Christopher J. Mungall, Gregg T. Beckham, Lucas Waldburger, James Carothers, ShivShankar Sundaram, Deb Agarwal, Blake A. Simmons, Tyler Backman, Deepanwita Banerjee, Deepti Tanjore, Lavanya Ramakrishnan, Anup Singh, "Perspectives for Self-Driving Labs in Synthetic Biology", Current Opinion in Biotechnology, February 2023, doi: 10.1016/j.copbio.2022.102881

2022

MB Simmonds, WJ Riley, DA Agarwal, X Chen, S Cholia, R Crystal-Ornelas, ET Coon, D Dwivedi, VC Hendrix, M Huang, A Jan, Z Kakalia, J Kumar, CD Koven, L Li, M Melara, L Ramakrishnan, DM Ricciuto, AP Walker, W Zhi, Q Zhu, C Varadharajan, Guidelines for Publicly Archiving Terrestrial Model Data to Enhance Usability, Intercomparison, and Synthesis, Data Science Journal, 2022, doi: 10.5334/dsj-2022-003

C Varadharajan, VC Hendrix, DS Christianson, M Burrus, C Wong, SS Hubbard, DA Agarwal, BASIN-3D: A brokering framework to integrate diverse environmental data, Computers and Geosciences, 2022, doi: 10.1016/j.cageo.2021.105024

B Faybishenko, R Versteeg, G Pastorello, D Dwivedi, C Varadharajan, D Agarwal, Challenging problems of quality assurance and quality control (QA/QC) of meteorological time series data, Stochastic Environmental Research and Risk Assessment, Pages: 1049--1062 2022, doi: 10.1007/s00477-021-02106-w

F Molz, B Faybishenko, D Agarwal, A broad exploration of nonlinear dynamics in microbial systems motivated by chemostat experiments producing deterministic chaos., 2022,

2021

C Varadharajan, Z Kakalia, E Alper, EL Brodie, M Burrus, RWH Carroll, D Christianson, W Dong, V Hendrix, M Henderson, S Hubbard, D Johnson, R Versteeg, KH Williams, DA Agarwal, The Colorado East River Community Observatory Data Collection, Hydrological Processes 35(6), 2021, doi: 10.22541/au.161962485.54378235/v1

D. A. Agarwal, J. Damerow, C. Varadharajan, D. S. Christianson, G. Z. Pastorello, Y.-W. Cheah, L. Ramakrishnan, "Balancing the needs of consumers and producers for scientific data collections", Ecological Informatics, 2021, 62:101251, doi: 10.1016/j.ecoinf.2021.101251

J Müller, B Faybishenko, D Agarwal, S Bailey, C Jiang, Y Ryu, C Tull, L Ramakrishnan, Assessing data change in scientific datasets, Concurrency and Computation: Practice and Experience, 2021, doi: 10.1002/cpe.6245

SL Brantley, T Wen, DA Agarwal, JG Catalano, PA Schroeder, K Lehnert, C Varadharajan, J Pett-Ridge, M Engle, AM Castronova, RP Hooper, X Ma, L Jin, K McHenry, E Aronson, AR Shaughnessy, LA Derry, J Richardson, J Bales, EM Pierce, The future low-temperature geochemical data-scape as envisioned by the U.S. geochemical community, Computers and Geosciences, 2021, doi: 10.1016/j.cageo.2021.104933

JE Damerow, C Varadharajan, K Boye, EL Brodie, M Burrus, KD Chadwick, R Crystal-Ornelas, H Elbashandy, RJ Eloy Alves, KS Ely, AE Goldman, T Haberman, V Hendrix, Z Kakalia, KM Kemner, AB Kersting, N Merino, F O Brien, Z Perzan, E Robles, P Sorensen, JC Stegen, RL Walls, P Weisenhorn, M Zavarin, D Agarwal, Sample identifiers and metadata to support data management and reuse in multidisciplinary ecosystem sciences, Data Science Journal, 2021, doi: 10.5334/dsj-2021-011

2020

G. Z. Pastorello, C. Trotta, E. Canfora, H. Chu, D. Christianson, Y.-W. Cheah, C. Poindexter, J. Chen, A. Elbashandy, M. Humphrey, P. Isaac, D. Polidori, M. Reichstein, A. Ribeca, C. van Ingen, N. Vuichard, L. Zhang, B. Amiro, C. Ammann, M. A. Arain, J. Ardö, T. Arkebauer, S. K. Arndt, N. Arriga, M. Aubinet, M. Aurela, D. Baldocchi, A. Barr, E. Beamesderfer, L. B. Marchesini, O. Bergeron, J. Beringer, C. Bernhofer, D. Berveiller, D. Billesbach, T. A. Black, P. D. Blanken, G. Bohrer, J. Boike, P. V. Bol stad, D. Bonal, J.-M. Bonnefond, D. R. Bowling, R. Bracho, J. Brodeur, C. Brümmer, N. Buchmann, B. Burban, S. P. Burns, P. Buysse, P. Cale, M. Cavagna, P. Cellier, S. Chen, I. Chini, T. R. Chris tensen, J. Cleverly, A. Collalti, C. Consalvo, B. D. Cook, D. Cook, C. Coursolle, E. Cremonese, P. S. Curtis, E. D’Andrea, H. da Rocha, X. Dai, K. J. Davis, B. D. Cinti, A. de Grandcourt, A. D. Ligne, R. C. D. Oliveira, N. Delpierre, A. R. Desai, C. M. D. Bella, P. di Tommasi, H. Dolman, F. Domingo, G. Dong, S. Dore, P. Duce, E. Dufrêne, A. Dunn, J. Dušek, D. Eamus, U. Eichelmann, H. A. M. ElKhidir, W. Eugster, C. M. Ewenz, B. Ewers, D. Famulari, S. Fares, I. Feigenwinter, A. Feitz, R. Fensholt, G. Fil ippa, M. Fischer, J. Frank, M. Galvagno, M. Gharun, D. Gianelle, B. Gielen, B. Gioli, A. Gitelson, I. Goded, M. Goeckede, A. H. Goldstein, C. M. Gough, M. L. Goulden, A. Graf, A. Griebel, C. Gruening, T. Grünwald, A. Hammerle, S. Han, X. Han, B. U. Hansen, C. Hanson, J. Hatakka, Y. He, M. Hehn, B. Heinesch, N. Hinko-Najera, L. Hörtnagl, L. Hutley, A. Ibrom, H. Ikawa, M. Jackowicz-Korczynski, D. Janouš, W. Jans, R. Jassal, S. Jiang, T. Kato, M. Khomik, J. Klatt, A. Knohl, S. Knox, H. Kobayashi, G. Koerber, O. Kolle, Y. Kosugi, A. Kotani, A. Kowalski, B. Kruijt, J. Kurbatova, W. L. Kutsch, H. Kwon, S. Launiainen, T. Laurila, B. Law, R. Leuning, Y. Li, M. Liddell, J.-M. Limousin, M. Lion, A. J. Liska, A. Lohila, A. López-Ballesteros, E. López-Blanco, B. Loubet, D. Loustau, A. Lucas-Moffat, J. Lüers, S. Ma, C. Macfarlane, V. Magliulo, R. Maier, I. Mammarella, G. Manca, B. Marcolla, H. A. Margolis, S. Mar ras, W. Massman, M. Mastepanov, R. Matamala, J. H. Matthes, F. Mazzenga, H. McCaughey, I. McHugh, A. M. S. McMillan, L. Merbold, W. Meyer, T. Meyers, S. D. Miller, S. Minerbi, U. Moderow, R. K. Monson, L. Montagnani, C. E. Moore, E. Moors, V. Moreaux, C. Moureaux, J. W. Munger, T. Nakai, J. Neirynck, Z. Nesic, G. Nicolini, A. Noormets, M. Northwood, M. Nosetto, Y. Nouvellon, K. Novick, W. Oechel, J. E. Olesen, J.-M. Ourcival, S. A. Papuga, F.-J. Parmentier, E. Paul-Limoges, M. Pavelka, M. Peichl, E. Pendall, R. P. Phillips, K. Pilegaard, N. Pirk, G. Posse, T. Powell, H. Prasse, S. M. Prober, S. Ram bal, U. Rannik, N. Raz-Yaseef, D. Reed, V. R. de Dios, N. Restrepo-Coupe, B. R. Reverter, M. Roland, S. Sabbatini, T. Sachs, S. R. Saleska, E. P. S.-C. nete, Z. M. Sanchez-Mejia, H. P. Schmid, M. Schmidt, K. Schneider, F. Schrader, I. Schroder, R. L. Scott, P. Sedlák, P. Serrano-Ortíz, C. Shao, P. Shi, I. Shironya, L. Siebicke, L. Šigut, R. Silberstein, C. Sirca, D. Spano, R. Steinbrecher, R. M. Stevens, C. Sturtevant, A. Suyker, T. Tagesson, S. Takanashi, Y. Tang, N. Tapper, J. Thom, F. Tiedemann, M. Tomassucci, J.-P. Tuovinen, S. Urbanski, R. Valentini, M. van der Molen, E. van Gorsel, K. van Huissteden, A. Varlagin, J. Verfaillie, T. Vesala, C. Vincke, D. Vitale, N. Vygodskaya, J. P. Walker, E. Walter-Shea, H. Wang, R. Weber, S. Westermann, C. Wille, S. Wofsy, G. Wohlfahrt, S. Wolf, W. Woodgate, Y. Li, R. Zampedri, J. Zhang, G. Zhou, D. Zona, D. Agarwal, S. Biraud, M. Torn, D. Papale, "The FLUXNET2015 dataset and the ONEFlux processing pipeline for eddy covariance data", Scientific Data, 2020, 7:225, doi: 10.1038/s41597-020-0534-3

2019

P. Linton, W. Melodia, A. Lazar, D. Agarwal, L. Bianchi, D. Ghoshal, K. Wu, G. Pastorello, L. Ramakrishnan, "Identifying Time Series Similarity in Large-Scale Earth System Datasets", The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC19), 2019,

C Varadharajan, S Cholia, C Snavely, V Hendrix, C Procopiou, D Swantek, W Riley, D Agarwal, "Launching an Accessible Archive of Environmental Data", Eos, January 1, 2019, 100, doi: 10.1029/2019eo111263

Payton A Linton, William M Melodia, Alina Lazar, Deborah Agarwal, Ludovico Bianchi, Devarshi Ghoshal, Kesheng Wu, Gilberto Pastorello, Lavanya Ramakrishnan, "Identifying Time Series Similarity in Large-Scale Earth System Datasets", 2019,

Payton Linton, William Melodia, Alina Lazar, Deborah Agarwal, Ludovico Bianchi, Devarshi Ghoshal, Gilberto Pastorello, Lavanya Ramakrishnan, Kesheng Wu, Understanding Data Similarity in Large-Scale Scientific Datasets, 2019 IEEE International Conference on Big Data (Big Data), Pages: 4525--4531 2019,

C Varadharajan, B Faybishenko, A Henderson, M Henderson, VC Hendrix, SS Hubbard, Z Kakalia, A Newman, B Potter, H Steltzer, R Versteeg, DA Agarwal, KH Williams, C Wilmer, Y Wu, W Brown, M Burrus, RWH Carroll, DS Christianson, B Dafflon, D Dwivedi, BJ Enquist, Challenges in Building an End-to-End System for Acquisition, Management, and Integration of Diverse Data from Sensor Networks in Watersheds: Lessons from a Mountainous Community Observatory in East River, Colorado, IEEE Access, Pages: 182796--18 2019, doi: 10.1109/ACCESS.2019.2957793

2018

B Faybishenko, F Molz, D Agarwal, "Nonlinear dynamics simulations of microbial ecological processes: Model, diagnostic parameters of deterministic chaos, and sensitivity analysis", Springer Proceedings in Mathematics and Statistics, ( 2018) Pages: 437--465 doi: 10.1007/978-3-030-02825-1_19

DC Miller, JD Siirola, D Agarwal, AP Burgard, A Lee, JC Eslick, B Nicholson, C Laird, LT Biegler, D Bhattacharyya, NV Sahinidis, IE Grossmann, CE Gounaris, D Gunter, "Next Generation Multi-Scale Process Systems Engineering Framework", Computer Aided Chemical Engineering, 2018, 44:2209--2214, doi: 10.1016/B978-0-444-64241-7.50363-3

D Ghoshal, L Ramakrishnan, D Agarwal, "Dac-Man: Data Change Management for Scientific Datasets on HPC Systems", SC ’18, Piscataway, NJ, USA, IEEE Press, 2018, 72:1--72:1,

2017

Gilberto Z. Pastorello, Dario Papale, Housen Chu, Carlo Trotta, Deb A. Agarwal, Eleonora Canfora, Dennis D. Baldocchi, M. S. Torn, "A new data set to keep a sharper eye on land-air exchanges", Eos, 2017, 98:28-32, doi: 10.1029/2017EO071597

Danielle S. Christianson, Charuleka Varadharajan, Bradley Christoffersen, Matteo Detto, Faybishenko, Bruno O. Gimenez, Val C. Hendrix, Kolby J. Jardine, Robinson Negron-Juarez, Z. Pastorello, Thomas L. Powell, Megha Sandesh, Jeffrey M. Warren, Brett T. Wolfe, Jeffrey Q. Chambers, Lara M. Kueppers, Nathan G. McDowell, Deborah A. Agarwal, "A metadata reporting framework (FRAMES) for synthesis of ecohydrological observations", Ecological Informatics, 2017, 42:148-158, doi: 10.1016/j.ecoinf.2017.06.002

DA Agarwal, C Varadharajan, S Cholia, C Snavely, VC Hendrix, D Gunter, WJ Riley, M jones, AE budden, D Vieglas, Environmental System Science Data Infrastructure for a Virtual Ecosystem (ESS-DIVE)-A New US DOE Data Archive, 2017,

D Guyon, AC Orgerie, C Morin, D Agarwal, "How Much Energy Can Green HPC Cloud Users Save?", Proceedings - 2017 25th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, PDP 2017, 2017, 416--420, doi: 10.1109/PDP.2017.62

YW Cheah, J Boverhof, A Elbashandy, D Agarwal, J Leek, T Epperly, J Eslick, D Miller, "Data management and simulation support accelerating carbon capture through computing", Proceedings of the 2016 IEEE 12th International Conference on e-Science, e-Science 2016, 2017, 389--398, doi: 10.1109/eScience.2016.7870924

Gilberto Z. Pastorello, Dan K. Gunter, Housen Chu, Danielle S. Christianson, Carlo Trotta, Eleonora Canfora, Boris Faybishenko, You-Wei Cheah, Norm Beekwilder, Stephen W. Chan, Sigrid Dengel, Trevor Keenan, Fianna O Brien, Abderahman Elbashandy, Cristina M. Poindexter, Marty Humphrey, Dario Papale, Deb A. Agarwal, "Hunting Data Rogues at Scale: Data Quality Control for Observational Data in Research Infrastructures", Proceedings of the 13th IEEE International Conference on e-Science (e-Science 2017), Auckland, New Zealand, 2017, doi: 10.1109/eScience.2017.64

2016

Deborah A Agarwal, Boris Faybishenko, Vicky L Freedman, Harinarayan Krishnan, Gary Kushner, Carina Lansing, Ellen Porter, Alexandru Romosan, Arie Shoshani, Haruko Wainwright, others, "A science data gateway for environmental management", Concurrency and Computation: Practice and Experience, 2016, 28:1994--2004,

DC Miller, D Agarwal, D Bhattacharyya, J Boverhof, YW Cheah, Y Chen, J Eslick, J Leek, J Ma, P Mahapatra, B Ng, NV Sahinidis, C Tong, SE Zitney, "Innovative computational tools and models for the design, optimization and control of carbon capture processes", Computer Aided Chemical Engineering, 2016, 38:2391--2396, doi: 10.1016/B978-0-444-63428-3.50403-3

2014

V Hendrix, L Ramakrishnan, Y Ryu, C Van Ingen, KR Jackson, D Agarwal, "CAMP: Community access MODIS pipeline", Future Generation Computer Systems, 2014, 36:418--429, doi: 10.1016/j.future.2013.09.023

Gilberto Z. Pastorello, Deb A. Agarwal, Taghrid Samak, Dario Papale, Trotta, Alessio Ribeca, Cristina M. Poindexter, Boris Faybishenko, Dan K. Gunter, Rachel Hollowgrass, Eleonora Canfora, "Observational data patterns for time series data quality assessment", Proceedings of the 10th IEEE International Conference on e-Science (e-Science 2014), Guaruja, Brazil, 2014, doi: 10.1109/eScience.2014.45

DC Miller, M Syamlal, DS Mebane, C Storlie, D Bhattacharyya, NV Sahinidis, D Agarwal, C Tong, SE Zitney, A Sarkar, X Sun, S Sundaresan, E Ryan, D Engel, C Dale, "Carbon capture simulation initiative: A case study in multiscale modeling and new challenges", Annual Review of Chemical and Biomolecular Engineering, 2014, 5:301--323, doi: 10.1146/annurev-chembioeng-060713-040321

Lavanya Ramakrishnan, Sarah S. Poon, Val C. Hendrix, Dan K. Gunter, Gilberto Z. Pastorello, Deb A. Agarwal, "Experiences with User-Centered Design for the Tigres Workflow API", Proceedings of the 10th IEEE International Conference on e-Science (e-Science 2014), Guaruja, Brazil, 2014, doi: 10.1109/eScience.2014.56

JR Balderrama, M Simonin, L Ramakrishnan, V Hendrix, C Morin, D Agarwal, C Tedeschi, "Combining workflow templates with a shared space-based execution model", Proceedings of WORKS 2014: The 9th Workshop on Workflows in Support of Large-Scale Science - held in conjunction with SC 2014: The International Conference for High Performance Computing, Networking, Storage and Analysis, 2014, 50--58, doi: 10.1109/WORKS.2014.14

2012

Karen L. Schuchardt, Deborah A. Agarwal, Stefan A. Finsterle, Carl W. Gable, Ian Gorton, Luke J. Gosink, Elizabeth H. Keating, Carina S. Lansing, Joerg Meyer, William A.M. Moeglein, George S.H. Pau, Ellen A. Porter, Sumit Purohit, Mark L. Rockhold, Arie Shoshani, and Chandrika Sivaramakrishnan, Akuna, "Integrated Toolsets Supporting Advanced Subsurface Flow and Transport Simulations for Environmental Management", XIX International Conference on Computational Methods in Water Resources (CMWR 2012), University of Illinois at Urbana-Champaign, June 2012,

Karen L. Schuchardt, Deborah A. Agarwal, Stefan A. Finsterle, Carl W. Gable, Ian Gorton, Luke J. Gosink, Elizabeth H. Keating, Carina S. Lansing, Joerg Meyer, William A.M. Moeglein, George S.H. Pau, Ellen A. Porter, Sumit Purohit, Mark L. Rockhold, Arie Shoshani, Chandrika Sivaramakrishnan, "Akuna-Integrated Toolsets Supporting Advanced Subsurface Flow and Transport Simulations for Environmental Management", XIX International Conference on Computational Methods in Water Resources (CMWR 2012), University of Illinois at Urbana-Champaign, June 17-22, 2012, 2012,

D Baldocchi, M Reichstein, D Papale, L Koteen, R Vargas, D Agarwal, R Cook, The role of trace gas flux networks in the biogeosciences, Eos, Pages: 217--218 2012, doi: 10.1029/2012EO230001

V Hendrix, J Li, K Jackson, L Ramakrishnan, Y Ryu, K Beattie, C Morin, D Skinner, C van Ingen, D Agarwal, "Community Access to MODIS Satellite Reprojection and Reduction Pipeline and Data Sets", AGU Fall Meeting, 2012,

2011

D Agarwal, YW Cheah, D Fay, J Fay, D Guo, T Hey, M Humphrey, K Jackson, Jie Li, C Poulain, Y Ryu, C Van Ingen, "Data-intensive science: The Terapixel and MODISAzure projects", International Journal of High Performance Computing Applications, 2011, 25:304--316, doi: 10.1177/1094342011414746

2010

D. Agarwal, M. Humphrey,N., Beekwilder, K. Jackson, M. Goode, and C. van Ingen, "A Data Centered Collaboration Portal to Support Global Carbon-Flux Analysis", Concurrency and Computation: Practice and Experience - Successes in Furthering Scientific Discovery, June 2010, LBNL 1827E,

Y Simmhan, E Soroush, C Van Ingen, D Agarwal, L Ramakrishnan, "BReW: Blackbox resource selection for e-Science workflows", 2010 5th Workshop on Workflows in Support of Large-Scale Science, WORKS 2010, 2010, doi: 10.1109/WORKS.2010.5671857

L Ramakrishnan, C Guok, K Jackson, E Kissel, DM Swany, D Agarwal, "On-demand overlay networks for large scientific data transfers", CCGrid 2010 - 10th IEEE/ACM International Conference on Cluster, Cloud, and Grid Computing, 2010, 359--367, doi: 10.1109/CCGRID.2010.82

M Bishop, J Cummins, S Peisert, A Singh, B Bhumiratana, D Agarwal, D Frincke, M Hogarth, "Relationships and data sanitization: A study in scarlet", Proceedings New Security Paradigms Workshop, January 2010, 151--163, doi: 10.1145/1900546.1900567

DA Agarwal, M Humphrey, NF Beekwilder, KR Jackson, MM Goode, C Van Ingen, "A data-centered collaboration portal to support global carbon-flux analysis", Concurrency Computation Practice and Experience, 2010, 22:2323--2334, doi: 10.1002/cpe.1600

J Li, M Humphrey, D Agarwal, K Jackson, C Van Ingen, Y Ryu, EScience in the cloud: A MODIS satellite data reprojection and reduction pipeline in the Windows Azure platform, Proceedings of the 2010 IEEE International Symposium on Parallel and Distributed Processing, IPDPS 2010, 2010, doi: 10.1109/IPDPS.2010.5470418

J Li, M Humphrey, YW Cheah, Y Ryu, D Agarwal, K Jackson, C Van Ingen, Fault tolerance and scaling in e-Science cloud applications: Observations from the continuing development of MODISAzure, Proceedings - 2010 6th IEEE International Conference on e-Science, eScience 2010, Pages: 246--253 2010, doi: 10.1109/eScience.2010.47

2009

DA Agarwal, DA Tran, MD Yarvis, Guest editorial, International Journal of Pervasive Computing and Communications, Pages: 18 2009, doi: 10.1108/ijpcc.2009.36105daa.001

M Humphrey, D Agarwal, C Van Ingen, Fluxdata.org: Publication and curation of shared scientific climate and earth sciences data, e-Science 2009 - 5th IEEE International Conference on e-Science, Pages: 118--125 2009, doi: 10.1109/e-Science.2009.25

2008

William T.C. Kramer, John M. Shalf, E. Wes Bethel, D. Agarwal, Michael Banda, John Hules, Juan C. Meza, Leonid Oliker, Horst Simon, David Skinner, Francesca Verdier, Howard Walter, Michael Wehner, and Katherine Yelick, "HPC in 2016: A View Point from NERSC", Proceedings of the Cray User Group Conference, Helsinki, Finland, 2008,

M Humphrey, N Beekwilder, D Agarwal, D Baldocchi, M Goode, C Ingen, D Papale, M Reichstein, M Rodriguez, R Vargas, N Li, J Gupchup, Y Ryu, Creating and Accessing the Global Fluxnet Data Set, 2008,

2007

PV Sundareshwar, R Murtugudde, G Srinivasan, S Singh, KJ Ramesh, R Ramesh, SB Verma, D Agarwal, D Baldocchi, CK Baru, KK Baruah, GR Chowdhury, VK Dadhwal, CBS Dutt, J Fuentes, PK Gupta, WW Hargrove, M Howard, CS Jha, S Lal, WK Michener, AP Mitra, JT Morris, RR Myneni, M Naja, R Nemani, R Purvaja, S Raha, SK Santhana Vanan, M Sharma, A Subramaniam, R Sukumar, RR Twilley, PR Zimmerman, Environment: Environmental monitoring network for India, Science, Pages: 204--205 2007, doi: 10.1126/science.1137417

2004

WTC Kramer, A Shoshani, DA Agarwal, BR Draney, G Jin, GF Butler, JA Hules, Deep scientific computing requires deep data, IBM Journal of Research and Development, Pages: 209--232 2004, doi: 10.1147/rd.482.0209

2003

S Peggs, T Satogata, D Agarwal, D Rice, Remote operations in A global accelerator network, Proceedings of the IEEE Particle Accelerator Conference, Pages: 278--282 2003,

1998

DA Agarwal, LE Moser, PM Melliar-Smith, RK Budhia, The Totem Multiple-Ring Ordering and Topology Maintenance Protocol, ACM Transactions on Computer Systems, Pages: 93--132 1998, doi: 10.1145/279227.279228

1995

Y Amir, LE Moser, PM Melliar-Smith, DA Agarwal, P Ciarfella, "The Totem Single-Ring Ordering and Membership Protocol", ACM Transactions on Computer Systems (TOCS), 1995, 13:311--342, doi: 10.1145/210223.210224

E. Wes Bethel

2024

Jan Balewski, Mercy G Amankwah, Roel Van Beeumen, E Wes Bethel, Talita Perciano, Daan Camps, "Quantum-parallel vectorized data encodings and computations on trapped-ion and transmon QPUs", Journal, February 10, 2024, 14, doi: https://doi.org/10.1038/s41598-024-53720-x

2023

GM Wallace, Z Bai, N Bertelli, EW Bethel, T Perciano, S Shiraiwa, JC Wright, "Towards Fast, Accurate Predictions of RF Simulations via Data-driven Modeling: Forward and Lateral Models", Conference, AIP Publishing, August 1, 2023, 2984, doi: https://doi.org/10.1063/5.0162422

2022

Gregory Wallace, Zhe Bai, Robbie Sadre, Talita Perciano, Nicola Bertelli, Syun'ichi Shiraiwa, Wes Bethel, John Wright, "Towards fast and accurate predictions of radio frequency power deposition and current profile via data-driven modelling: applications to lower hybrid current drive", Journal of Plasma Physics, August 18, 2022, 88:895880401, doi: 10.1017/S0022377822000708

M. G. Amankwah, D. Camps, E. W. Bethel, R. Van Beeumen, T. Perciano, "Quantum pixel representations and compression for N-dimensional images", Nature Scientific Reports, May 11, 2022, 12:7712, doi: 10.1038/s41598-022-11024-y

S. Zhang, R. Sadre, B. A. Legg, H. Pyles, T. Perciano, E. W. Bethel, D. Baker, O. Rübel, J. J. D. Yoreo, "Rotational dynamics and transition mechanisms of surface-adsorbed proteins", Proceedings of the National Academy of Sciences, April 11, 2022, 119:e202024211, doi: 10.1073/pnas.2020242119

E. Wes Bethel, Burlen Loring, Utkarsh Ayachit, P. N. Duque, Nicola Ferrier, Joseph Insley, Junmin Gu, Kress, Patrick O’Leary, Dave Pugmire, Silvio Rizzi, Thompson, Will Usher, Gunther H. Weber, Brad Whitlock, Wolf, Kesheng Wu, "Proximity Portability and In Transit, M-to-N Data Partitioning and Movement in SENSEI", In Situ Visualization for Computational Science, ( 2022) doi: 10.1007/978-3-030-81627-8_20

E. Wes Bethel, Burlen Loring, Utkarsh Ayatchit, David Camp, P. N. Duque, Nicola Ferrier, Joseph Insley, Junmin Gu, Kress, Patrick O’Leary, David Pugmire, Silvio Rizzi, Thompson, Gunther H. Weber, Brad Whitlock, Matthew Wolf, Kesheng Wu, "The SENSEI Generic In Situ Interface: Tool and Processing Portability at Scale", In Situ Visualization for Computational Science, ( 2022) doi: 10.1007/978-3-030-81627-8_13

2021

E. W. Bethel, C. Heinemann, and T. Perciano, "Performance Tradeoffs in Shared-memory Platform Portable Implementations of a Stencil Kernel", Eurographics Symposium on Parallel Graphics and Visualization, June 14, 2021,

2020

Stefano Marchesini, Anuradha Trivedi, Pablo Enfedaque, Talita Perciano, Dilworth Parkinson, "Sparse Matrix-Based HPC Tomography", Computational Science -- ICCS 2020, Cham, Springer International Publishing, 2020, 248--261, doi: 10.1007/978-3-030-50371-0_18

E. Wes Bethel, David Camp, Talita Perciano, Colleen Heinemann, "Performance Analysis of Traditional and Data-Parallel Primitive Implementations of Visualization and Analysis Kernels", Berkeley, CA, USA, 94720, 2020,

H. Childs, S. Ahern, J. Ahrens, A. C. Bauer, J. Bennett, E. W. Bethel, P.-T. Bremer, E. Brugger, J. Cottam, M. Dorier, S. Dutta, J. Favre, T. Fogal, S. Frey, C. Garth, B. Geveci, W. F. Godoy, C. D. Hansen, C. Harrison, B. Hentschel, J. Insley, C. Johnson, S. Klasky, A. Knoll, J. Kress, M. Larsen, J. Lofstead, K.-L. Ma, P. Malakar, J. Meredith, K. Moreland, P. Navratil, P. O Leary, M. Parashar, V. Pascucci, J. Patchett, T. Peterka, S. Petruzza, N. Podhorszki, D. Pugmire, M. Rasquin, S. Rizzi, D. H. Rogers, S. Sane, F. Sauer, R. Sisneros, H.-W. Shen, W. Usher, R. Vickery, V. Vishwanath, I. Wald, R. Wang, G. H. Weber, B. Whitlock, M. Wolf, H. Yu, S. B. Ziegler, "A Terminology for In Situ Visualization and Analysis Systems", International Journal of High Performance Computing Applications, 2020, 34:676--691, doi: 10.1177/1094342020935991

2019

Junmin Gu, Burlen Loring, Kesheng Wu, E Wes Bethel, "HDF5 as a vehicle for in transit data movement", Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization, 2019, 39--43,

2018

B Loring, A Myers, D Camp, EW Bethel, "Python-based in situ analysis and visualization", Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization - ISAV 18, ACM Press, 2018, doi: 10.1145/3281464.3281465

B Lessley, T Perciano, C Heinemann, D Camp, H Childs, EW Bethel, "DPP-PMRF: Rethinking Optimization for a Probabilistic Graphical Model Using Data-Parallel Primitives", The 8th IEEE Symposium on Large Data Analysis and Visualization - LDAV 2018, 2018,

C Heinemann, T Perciano, D Ushizima, EW Bethel, "Distributed memory parallel Markov random fields using graph partitioning", Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017, 2018, 2018-Jan:3332--3341, doi: 10.1109/BigData.2017.8258318

2017

E. Wes Bethel, In Situ Processing Overview and Relevance to the HPC Community, SIAM Conference on Computational Science and Engineering, MS74: In Situ Methods and Infrastructures: Faster Insight Through Smarter Computing, 2017,

U Ayachit, B Whitlock, M Wolf, B Loring, B Geveci, D Lonie, EW Bethel, "The SENSEI generic in situ interface", Proceedings of ISAV 2016: 2nd Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis, 2017, 40--44, doi: 10.1109/ISAV.2016.13

B Lessley, T Perciano, M Mathai, H Childs, EW Bethel, "Maximal clique enumeration with data-parallel primitives", 2017 IEEE 7th Symposium on Large Data Analysis and Visualization, LDAV 2017, 2017, 2017-Dec:16--25, doi: 10.1109/LDAV.2017.8231847

T Perciano, D Ushizima, H Krishnan, D Parkinson, N Larson, DM Pelt, W Bethel, F Zok, J Sethian, "Insight into 3D micro-CT data: Exploring segmentation algorithms through performance metrics", Journal of Synchrotron Radiation, 2017, 24:1065--1077, doi: 10.1107/S1600577517010955

E Bethel, zorder-lib: Library API for Z-Order Memory Layout:, 2017,

EW Bethel, Towards a data-centric research and development roadmap for large-scale science user facilities, Proceedings - 13th IEEE International Conference on eScience, eScience 2017, Pages: 462--464 2017, doi: 10.1109/eScience.2017.72

2016

Andrew C. Bauer, Kenneth E. Jansen, E. Wes Bethel, Utkarsh Ayachit, Michel Rasquin, Benjamin Matthews, Steve Jordan, "In Situ Analysis and Visualization at Scale with PHASTA and ParaView Catalyst on Mira and Theta", SC16 Scientific Visualization Showcase, 2016,

E. Wes Bethel, Martin Greenwald, Kerstin Kleese Dam, Manish Parashar, Stefan M. Wild, H. Steven Wiley, "2016 IEEE 12th International Conference on e-Science", 2016 IEEE 12th International Conference on e-Science, Baltimore, MD, USA, 2016, 213--222,

Hoa Nguyen, D\ aith\ i Stone, E. Wes Bethel, "Statistical Projections for Multi-dimensional Visual Data Exploration", 6th IEEE Symposium on Large Data Analysis and Visualization, 2016,

moderator E. Wes Bethel (organizer, Hank Childs, Ken Moreland, Dave Pugmire, Matt Larsen, Matthieu Dorier, In Situ Efforts and Challenges in Large Data Analysis and Visualization, IEEE Symposium on Large Data Analysis and Visualization (LDAV), 2016,

Utkarsh Ayachit, Andrew Bauer, Earl PN Duque, Greg Eisenhauer, Nicola Ferrier, Junmin Gu, Kenneth E Jansen, Burlen Loring, Zarija Lukic, Suresh Menon, others, "Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures", SC 16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2016, 921--932, LBNL 1007264,

T Perciano, DM Ushizima, EW Bethel, YD Mizrahi, D Parkinson, JA Sethian, "Reduced-complexity image segmentation under parallel Markov Random Field formulation using graph partitioning", Proceedings - International Conference on Image Processing, ICIP, 2016, 2016-Aug:1259--1263, doi: 10.1109/ICIP.2016.7532560

O Rübel, B Loring, JL Vay, DP Grote, R Lehe, S Bulanov, H Vincenti, EW Bethel, "WarpIV: In Situ Visualization and Analysis of Ion Accelerator Simulations", IEEE Computer Graphics and Applications, 2016, 36:22--35, doi: 10.1109/MCG.2016.62

DM Ushizima, HA Bale, EW Bethel, P Ercius, BA Helms, H Krishnan, LT Grinberg, M Haranczyk, AA Macdowell, K Odziomek, DY Parkinson, T Perciano, RO Ritchie, C Yang, "IDEAL: Images Across Domains, Experiments, Algorithms and Learning", JOM, 2016, 68:2963--2972, doi: 10.1007/s11837-016-2098-4

2013

David Camp, Hari Krishnan, David Pugmire, Christoph Garth, Ian Johnson, E. Wes Bethel, Kenneth I. Joy, and Hank Childs., "GPU Acceleration of Particle Advection Workloads in a Parallel, Distributed Memory Setting", Proceedings of Eurographics Symposium on Parallel Graphics and Visualization (EGPGV), May 5, 2013,

Hank Childs, Berk Geveci, William J. Schroeder, Jeremy S. Meredith, Kenneth Moreland, Christopher Sewell, Torsten Kuhlen, E.Wes Bethel, "Research Challenges for Visualization Software", IEEE Computer, May 1, 2013, 46:34-43, LBNL 6239E,

E Wes Bethel, Prabhat Prabhat, Suren Byna, Oliver R\ ubel, K John Wu, Michael Wehner, "Why high performance visual data analytics is both relevant and difficult", Visualization and Data Analysis 2013, January 2013, 8654:86540B, LBNL LBNL-6063E,

Kesheng Wu, E Bethel, Ming Gu, David Leinweber, Oliver R\ ubel, "A big data approach to analyzing market volatility", Algorithmic Finance, 2013, 2:241--267, LBNL LBNL-6382E,

Understanding the microstructure of the financial market requires the processing of a vast amount of data related to individual trades, and sometimes even multiple levels of quotes. Analyzing such a large volume of data requires tremendous computing power that is not easily available to financial academics and regulators. Fortunately, public funded High Performance Computing (HPC) power is widely available at the National Laboratories in the US. In this paper we demonstrate that the HPC resource and the techniques for data-intensive sciences can be used to greatly accelerate the computation of an early warning indicator called Volume-synchronized Probability of Informed trading (VPIN). The test data used in this study contains five and a half year's worth of trading data for about 100 most liquid futures contracts, includes about 3 billion trades, and takes 140GB as text files. By using (1) a more efficient file format for storing the trading records, (2) more effective data structures and algorithms, and (3) parallelizing the computations, we are able to explore 16,000 different ways of computing VPIN in less than 20 hours on a 32-core IBM DataPlex machine. Our test demonstrates that a modest computer is sufficient to monitor a vast number of trading activities in real-time -- an ability that could be valuable to regulators.

Our test results also confirm that VPIN is a strong predictor of liquidity-induced volatility. With appropriate parameter choices, the false positive rates are about 7% averaged over all the futures contracts in the test data set. More specifically, when VPIN values rise above a threshold (CDF > 0.99), the volatility in the subsequent time windows is higher than the average in 93% of the cases.

Kesheng Wu, Wes Bethel, Ming Gu, David, Oliver R\ ubel, Testing VPIN on Big Data, Available at SSRN 2318259, 2013,

O Rübel, A Greiner, S Cholia, K Louie, EW Bethel, TR Northen, BP Bowen, "OpenMSI: A high-performance web-based platform for mass spectrometry imaging", Analytical Chemistry, 2013, 85:10354--103, doi: 10.1021/ac402540a

E Wes Bethel, Prabhat Prabhat, Suren Byna, R\ ubel, K John Wu, Michael Wehner, Why high performance visual data analytics is both and difficult, IS\&T/SPIE Electronic Imaging, Pages: 86540B-865 2013, doi: 10.1117/12.2010980

DN Williams, T Bremer, C Doutriaux, J Patchett, S Williams, G Shipman, R Miller, DR Pugmire, B Smith, C Steed, EW Bethel, H Childs, H Krishnan, P Prabhat, M Wehner, CT Silva, E Santos, D Koop, T Ellqvist, J Poco, B Geveci, A Chaudhary, A Bauer, A Pletzer, D Kindig, GL Potter, TP Maxwell, Ultrascale visualization of climate data, Computer, Pages: 68--76 2013, doi: 10.1109/MC.2013.119

2012

E. Wes Bethel and Mark Howison, "Multi-core and Many-core Shared-memory Parallel Raycasting Volume Rendering Optimization and Tuning", International Journal of High Performance Computing Applications, 2012, LBNL 5362E,

Hank Childs, Eric Brugger, Brad Whitlock, Jeremy Meredith, Sean Ahern, David Pugmire, Kathleen Biagas, Mark Miller, Cyrus Harrison, Gunther H. Weber, Hari Krishnan, Thomas Fogal, Allen Sanderson, Christoph Garth, E. Wes Bethel, David Camp, Oliver Rubel, Marc Durant, Jean M. Favre, Paul Navratil, "VisIt: An End-User Tool For Visualizing and Analyzing Very Large Data", High Performance Visualization---Enabling Extreme-Scale Scientific Insight, ( October 2012) Pages: 357--372

E. Wes Bethel, David Camp, Hank Childs, Christoph Garth, Mark Howison, Kenneth I. Joy, David Pugmire, "Hybrid Parallelism", High Performance Visualization---Enabling Extreme-Scale Scientific Insight, ( October 2012) Pages: 261--290

Hank Childs, David Pugmire, Sean Ahern, Brad Whitlock, Mark Howison, Prabhat, Gunther Weber, E. Wes Bethel, "Visualization at Extreme Scale Concurrency", High Performance Visualization---Enabling Extreme-Scale Scientific Insight, ( October 2012) Pages: 291--306

E. Wes Bethel, Hank Childs, Charles Hansen (editors), High Performance Visualization---Enabling Extreme-Scale Scientific Insight, Chapman & Hall, CRC Computational Science, (CRC Press/Francis--Taylor Group: October 2012)

Oliver Rübel, Cameron, G. R. Geddes, Min Chen, Estelle Cormier-Michel, and E. Wes Bethel, "Query-driven Analysis of Plasma-based Particel Acceleration Data", Poster Abstracts of IEEE VisWeek, October 2012,

Mehmet Balman, Eric Pouyoul, Yushu Yao, E. Wes Bethel, Burlen Loring, Prabhat, John Shalf, Alex Sim, and Brian L. Tierney, "Experiences with 100G Network Applications", In Proceedings of the Fifth international Workshop on Data-intensive Distributed Computing, in conjunction with ACM High Performance Distributing Computing (HPDC) Conference, 2012, Delft, Netherlands, June 2012, LBNL 5603E, doi: 10.1145/2286996.2287004

100Gbps networking has finally arrived, and many research and educational in- stitutions have begun to deploy 100Gbps routers and services. ESnet and Internet2 worked together to make 100Gbps networks available to researchers at the Super- computing 2011 conference in Seattle Washington. In this paper, we describe two of the first applications to take advantage of this network. We demonstrate a visu- alization application that enables remotely located scientists to gain insights from large datasets. We also demonstrate climate data movement and analysis over the 100Gbps network. We describe a number of application design issues and host tuning strategies necessary for enabling applications to scale to 100Gbps rates. 

E. Wes Bethel, David Camp, Hank Childs, Mark Howison, Hari Krishnan, Burlen Loring, Joerg Meyer, Prabhat, Oliver Ruebel, Daniela Ushizima, Gunther Weber, "Towards Exascale: High Performance Visualization and Analytics – Project Status Report. Technical Report", DOE Exascale Research Conference, April 2012,

Ushizima, D.M., Weber, G., Morozov, D., Bethel, W., Sethian, J.A., "Algorithms for Microstructure Description applied to Microtomography", Carbon Cycle 2.0 Symposium, February 10, 2012,

Surendra Byna, Jerry Chou, Oliver Rubel, Homa Karimabadi, William S Daughter, Vadim Roytershteyn, E Wes Bethel, Mark Howison, Ke-Jou Hsu, Kuan-Wu Lin, others, "Parallel I/O, analysis, and visualization of a trillion particle simulation", SC 12: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, January 2012, 1--12,

Oliver R\ ubel, Surendra Byna, Kesheng Wu, Fuyu Li, Michael Wehner, Wes Bethel, others, "Teca: A parallel toolkit for extreme climate analysis", Procedia Computer Science, Elsevier, January 2012, 9:866--876, LBNL 5352E,

We present TECA, a parallel toolkit for detecting extreme events in large climate datasets. Modern climate datasets expose parallelism across a number of dimensions: spatial locations, timesteps and ensemble members. We design TECA to exploit these modes of parallelism and demonstrate a prototype implementation for detecting and tracking three classes of extreme events: tropical cyclones, extra-tropical cyclones and atmospheric rivers. We process a modern TB-sized CAM5 simulation dataset with TECA, and demonstrate good runtime performance for the three case studies.

E. W. Bethel, Surendra Byna, Jerry Chou, Cormier-Michel, Cameron G. R. Geddes, Howison, Fuyu Li, Prabhat, Ji Qiang, R\ ubel, Rob D. Ryne, Michael Wehner, Wu, "Big Data Analysis and Visualization: What Do LINACS Tropical Storms Have In Common?", 11th International Computational Accelerator Physics ICAP 2012, Germany, 2012,

EW Bethel, S. Byna, J. Chou, E., CGR Geddes, M. Howison, F. Li J. Q. Prabhat, O. R\ ubel, RD Ryne and, Big Data Analysis and Visualization: What Do LINACS Tropical Storms Have In Common?, 11th International Computational Accelerator Physics ICAP 2012, 2012,

O. R\ ubel, S. Byna, K. Wu, F. Li, M., W. Bethel, others, "TECA: A Parallel Toolkit for Extreme Climate", Procedia Computer Science, Elsevier, 2012, 9:866--876, doi: 10.1016/j.procs.2012.04.093

E. W. Bethel and D. Leinweber and O. Rubel and K. Wu, "Federal Market Information Technology in the Post Flash Crash Era: Roles of Supercomputing", The Journal of Trading, 2012, 7:9-24, LBNL 5263E, doi: 10.3905/jot.2012.7.2.009

E. Wes Bethel, David Leinweber, Oliver Rübel Kesheng Wu, Federal Market Information Technology in the Crash Era: Roles for Supercomputing, The Journal of Trading, Pages: 9--25 2012, doi: 10.3905/jot.2012.7.2.009

E. Wes Bethel, "Exploration of Optimization Options for Increasing Performance of a GPU Implementation of a Three-dimensional Bilateral Filter", 2012, LBNL 5406E,

O. Rübel, S.V.E. Keränen, M.D. Biggin, D.W. Knowles, G.H. Weber, H. Hagen, B. Hamann, and E.W. Bethel, "Linking Advanced Visualization and MATLAB for the Analysis of 3D Gene Expression Data", Mathematics and Visualization, Visualization in Medicine and Life Sciences II, Progress and New Challenges, edited by L. Linsen and B. Hamann and H. Hagen and H.-C. Hege, (Springer Verlag: 2012) Pages: 267-285, LBNL 4891E,

Mark Howison, E. Wes Bethel, Hank Childs, "Hybrid Parallelism for Volume Rendering on Large, Multi- and Many-core Systems", IEEE Transactions on Visualization and Computer Graphics, January 2012, 18:17-29, LBNL 4370E,

Surendra Byna, Jerry Chou, Oliver R\ ubel, , Homa Karimabadi, William S. Daughton, Roytershteyn, E. Wes Bethel, Mark Howison Ke-Jou Hsu, Kuan-Wu Lin, Arie Shoshani, Uselton, Kesheng Wu, Parallel I/O, Analysis, and Visualization of a Particle Simulation, Proceedings of SuperComputing 2012, 2012,

J Meyer, EW Bethel, JL Horsman, SS Hubbard, H Krishnan, A Romosan, EH Keating, L Monroe, R Strelitz, P Moore, G Taylor, B Torkian, TC Johnson, I Gorton, Visual data analysis as an integral part of environmental management, IEEE Transactions on Visualization and Computer Graphics, Pages: 2088--2094 2012, doi: 10.1109/TVCG.2012.278

H Krishnan, J Meyer, A Romosan, H Childs, EW Bethel, Enabling advanced environmental management via remote and distributed visual data exploration and analysis, Computing and Visualization in Science, Pages: 123--133 2012, doi: 10.1007/s00791-013-0204-5

M Balman, E Pouyoul, Y Yao, EW Bethel, B Loring, Prabhat, J Shalf, A Sim, BL Tierney, "Experiences with 100Gbps network applications", DIDC 12 - 5th International Workshop on Data-Intensive Distributed Computing, 2012, 33--42, doi: 10.1145/2286996.2287004

D Ushizima, D Morozov, GH Weber, AGC Bianchi, JA Sethian, EW Bethel, "Augmented topological descriptors of pore networks for material science", IEEE Transactions on Visualization and Computer Graphics, 2012, 18:2041--2050, LBNL 5964E, doi: 10.1109/TVCG.2012.200

2011

E. Wes Bethel, David Leinweber, Oliver Rübel, Kesheng Wu, "Federal Market Information Technology in the Post Flash Crash Era: Roles of Supercomputing", Workshop on High Performance Computational Finance at SC11, Seattle, WA, USA, November 2011, LBNL 5263E,

Ushizima, D.M., Parkinson, D., Nico, P., Ajo-Franklin, J., Macdowell, A., Kocar, B., Bethel E.W, Sethian J.A., "Statistical segmentation and porosity quantification of 3D x-ray microtomography", XXXIV Applications of Digital Image Processing: Proceeding of SPIE 2011, San Diego, CA, USA, August 2011,

Ushizima, D.M., Weber, G.H., Ajo-Franklin, J., Kim, Y., Macdowell, A., Morozov, D., Nico, P., Parkinson, D., Trebotich, D., Wan, J., and Bethel E.W., "Analysis and visualization for multiscale control of geologic CO2", Journal of Physics: Conference Series, Proceedings of SciDAC 2011, July 2011, LBNL Denver, CO, USA,

Jerry Chou, Mark Howison, Brian Austin, Kesheng Wu, Ji Qiang, E Wes Bethel, Arie Shoshani, Oliver R\ ubel, Rob D Ryne, "Parallel index and query for large scale data analysis", Proceedings of 2011 international conference for high performance computing, networking, storage and analysis, 2011, 1--11, LBNL 5317E,

J. Chou, K. Wu, O. R\ ubel, M. Howison, Qiang, Prabhat, B. Austin, E. W. Bethel, D. Ryne, A. Shoshani, "Parallel Index and Query for Large Scale Data", SC11, 2011, doi: 10.1145/2063384.2063424

M Prabhat, S Byna, C Paciorek, G Weber, K Wu, T Yopes, MF Wehner, G Ostrouchov, D Pugmire, R Strelitz, others, "Pattern Detection and Extreme Value Analysis on Large Climate Data", AGUFM, Pages: IN41C--03 January 2011,

Mark Howison, Mike McGreevy, Bruce Palmer, Oliver Ruebel, Kesheng Wu, "ExaHDF5: An I/O Platform for Exascale Data Models, Analysis and Performance", 2011,

Luke J. Gosink, Christoph Garth, John C. Anderson, E. Wes Bethel, and Kenneth I. Joy, "An Application of Multivariate Statistical Analysis for Query-Driven Visualization", IEEE Transactions on Visualization and Computer Graphics, 2011, 11:264-275, LBNL 3536E, doi: 10.1109/TVCG.2010.80

E. Wes Bethel, John van Rosendale, Dale Southard, Kelly Gaither, Hank Childs, Eric Brugger, Sean Ahern, "Visualization at Supercomputing Centers: The Tale of Little Big Iron and the Three Skinny Guys", IEEE Computer Graphics and Applications, January 2011, 31:90-95, LBNL 4180E,

Maciej Haranczyk, Richard L. Martin, Prabhat, James A. Sethian & E. Wes Bethel, "Computational Approaches for the High-Throughput Analysis of Porous materials for Energy related applications", Scientific Discovery through Advanced Computing 2011, 2011,

Prabhat, S. Byna, C. Paciorek, G. Weber, Wu, T. Yopes, M. Wehner, W. Collins, G., R. Strelitz, E. W. Bethel, Pattern Detection and Extreme Value Analysis on Large Data, DOE/BER Climate and Earth System Modeling PI Meeting, 2011,

E. W. Bethel, D. Leinweber, O. Rübel, K., Federal Market Information Technology in the Post Crash Era: Roles for Supercomputing, WHPCF, Pages: 23--30 2011, doi: 10.1145/2088256.2088267

Prabhat, Quincey Koziol, Karen Schuchardt, E. Wes Bethel, Jerry Chuo, Mark Howison, Mike, Bruce Palmer, Oliver Ruebel, Kesheng, ExaHDF5: An I/O Platform for Exascale Data Analysis and Performance, SciDAC 2011, 2011,

2010

Mark Howison, Andreas Adelmann, E. Wes Bethel, Achim Gsell, Benedikt Oswald, Prabhat, "H5hut: A High-Performance I/O Library for Particle-Based Simulations", Proceedings of 2010 Workshop on Interfaces and Abstractions for Scientific Data Storage (IASDS10), Heraklion, Crete, Greece, September 2010, LBNL 4021E,

M. Howison, E. W. Bethel, and H. Childs, "Hybrid Parallelism for Volume Rendering on Large, Multi-core Systems", Journal of Physics: Conference Series, Proceedings of SciDAC 2010, Chattanooga, TN, USA, July 2010, LBNL 4024E,

M. Howison, E. W. Bethel, and H. Childs, "Hybrid Parallelism for Volume Rendering on Large, Multi-core Systems", Proceedings of Astronum 2010, San Diego, CA, USA, June 2010, LBNL 4024E,

Mark Howison, E. Wes Bethel, Hank Childs, "MPI-hybrid Parallelism for Volume Rendering on Large, Multi-core Systems", In Proceedings of the Eurographics Symposium on Parallel Graphics and Visualization (EGPGV), Norköping, Sweden, May 2010, LBNL 3297E,

O. Rübel, G. H. Weber, M-Y Huang, E. W. Bethel, M. D. Biggin, C. C. Fowlkes, C. Luengo Hendriks, S. V. E. Keränen, M. Eisen, D. Knowles, J. Malik, H. Hagen and B. Hamann, "Integrating Data Clustering and Visualization for the Analysis of 3D Gene Expression Data", IEEE Transactions on Computational Biology and Bioinformatics, March 2010, 7:64-79, LBNL 382E,

Daniela Ushizima, Cameron Geddes, Estelle Cormier-Michel, E. Wes Bethel, Janet Jacobsen, Prabhat, Oliver Rubel, Gunther Weber, Bernard Hamann, Peter Messmer, Hans Hagen, "Automated detection and analysis of particle beams in laser-plasma accelerator simulations", Machine Learning, edited by Yagang Zhang, (In-Teh: February 2010) Pages: 367-389, LBNL 3845E,

Oliver R\ ubel, Sean Ahern, E Wes Bethel, Mark D Biggin, Hank Childs, Estelle Cormier-Michel, Angela DePace, Michael B Eisen, Charless C Fowlkes, Cameron GR Geddes, others, "Coupling visualization and data analysis for knowledge discovery from multi-dimensional scientific data", Procedia computer science, Elsevier, January 2010, 1:1757--1764, LBNL 3669E,

Gunther Weber, "Recent advances in visit: Amr streamlines and query-driven visualization", 2010,

Hank Childs, David Pugmire, Sean Ahern, Brad Whitlock, Mark Howison, Prabhat, Gunther Weber, E. Wes Bethel, "Extreme Scaling of Production Visualization Software on Diverse Architectures", IEEE Computer Graphics and Applications, January 2010, 30:22--31, LBNL 3403E, doi: 10.1109/MCG.2010.51

E. Wes Bethel, "Using wesBench to Study the Rendering Performance of Graphics Processing Units", 2010, LBNL 3025E,

Oliver Rübel, Sean Ahern, E. Wes Bethel, D. Biggin, Hank Childs, Estelle, Angela DePace, Michael B. Eisen Charless C. Fowlkes, Cameron G. R. Geddes, Hagen, Bernd Hamann, Min-Yu Huang, Soile E. Keränen, David W. Knowles, Cris L. Hendriks, Jitendra Malik, Jeremy Meredith Peter Messmer, Prabhat, Daniela Ushizima, H. Weber, Kesheng Wu, "Coupling visualization and data analysis for knowledge from multi-dimensional scientific data", Procedia Computer Science, 2010, 1:1751--1758, doi: 10.1016/j.procs.2010.04.197

2009

E. Wes Bethel, Hank Childs, Ajith Mascarenhas, Valerio Pascucci, and Prabhat, "Scientific Data Management Challenges in High Performance Visual Data Analysis", Scientific Data Management: Challenges, Existing Technology, and Deployment, edited by Arie Shoshani and Doron Rotem, (Chapman & Hall CRC Press: December 2009) LBNL 1449E,

E. W. Bethel, C. Johnson, S. Ahern, J. Bell, P.-T. Bremer, H. Childs, E. Cormier-Michel, M. Day, E. Deines, T. Fogal, C. Garth, C. G. R. Geddes, H. Hagen, B. Hamann, C. Hansen, J. Jacobsen, K. Joy, J. Kruger, J. Meredith, P. Messmer, G. Ostrouchov, V. Pascucci, K. Potter, Prabhat, D. Pugmire, O. Rubel, A. Sanderson, C. Silva, D. Ushizima, G. Weber, B. Whitlock, K. Wu, "Occam's Razor and Petascale Visual Data Analysis", SciDAC 2009, J. of Physics: Conference Series, San Diego, California, July 2009, LBNL 2210E,

Luke Gosink, Kesheng Wu, E. Wes Bethel, John D. Owens, Kenneth I. Joy, "Data Parallel Bin-based Indexing for Answering Queries on Multi-core Architecture", Proceedings of the 21st International Conference on Scientific and Statistical Database Management (SSDBM), June 2009, 5566:110-129, LBNL 2211E,

Shoaib Kamil, Cy Chan, Samuel Williams, Leonid Oliker, John Shalf, Mark Howison, E. Wes Bethel, Prabhat, "A Generalized Framework for Auto-tuning Stencil Computations", BEST PAPER AWARD - Cray User Group Conference (CUG), Atlanta, GA, May 4, 2009, LBNL 2078E,

Best Paper Award

K Wu, S Ahern, EW Bethel, J Chen, H Childs, C Geddes, J Gu, H Hagen, B Hamann, J Lauret, others, "FastBit: Interactively Searching Massive Data", Proc. of SciDAC 2009, 2009, LBNL 2164E,

Oliver R\ ubel, Cameron GR Geddes, Estelle Cormier-Michel, Kesheng Wu, Gunther H Weber, Daniela M Ushizima, Peter Messmer, Hans Hagen, Bernd Hamann, Wes Bethel, others, "Automatic beam path analysis of laser wakefield particle acceleration data", Computational Science \& Discovery, January 2009, 2:015005, LBNL 2734E,

C. G. R. Geddes, E Cormier-Michel, E. H. Esarey, C. B. Schroeder, J.-L. Vay, W. P. Leemans, D. L.. Bruhwiler, J. R. Cary, B. Cowan, M. Durant, P. Hamill, P. Messmer, P. Mullowney, C. Nieter, K. Paul, S. Shasharina, S. Veitzer, G. Weber, O. Rübel, D. Ushizima, Prabhat, E. W.Bethel, K. Wu, Large Fields for Smaller Facility Sources, SciDAC Review, Pages: 13-21, 2009,

E Bethel, "Modern Scientific Visualization is More than Just Pretty Pictures", January 2009, LBNL 1450E,

Luke J Gosink, Kesheng Wu, E Wes Bethel, John D Owens, Kenneth I Joy, "Data parallel bin-based indexing for answering queries on multi-core architectures", International Conference on Scientific and Statistical Database Management, 2009, 110--129,

 

 

E Wes Bethel, Chris Johnson, Sean Ahern, John Bell, Peer-Timo Bremer, Hank Childs, Estelle Cormier-Michel, Marc Day, Eduard Deines, Tom Fogal, others, "Occam s razor and petascale visual data analysis", Journal of Physics: Conference Series, 2009, 180:012084,

C. Garth, E. Deines, K. Joy, E. W. Bethel, H. Childs, G. Weber, S. Ahern, D. Pugmire, A. Sanderson, C. Johnson, Twists and Turns: Vector Field Visual Data Analysis for Petascale Computational Science, SciDAC Review, Pages: 10-21, 2009,

E. Wes Bethel, "High Performance, Three-Dimensional Bilateral Filtering", 2009, LBNL 1601E,

Luke J. Gosink, Kesheng Wu, E. Wes Bethel, D. Owens, Kenneth I. Joy, Data Parallel Bin-Based Indexing for Answering Queries Multi-core Architectures, Lecture Notes in Computer Science, Pages: 110--129 2009, doi: 10.1007/978-3-642-02279-1_9

Oliver R\ ubel, Cameron G R Geddes, Estelle, Kesheng Wu, Prabhat, Gunther H, Daniela M Ushizima, Peter Messmer, Hans, Bernd Hamann, Wes Bethel, "Automatic beam path analysis of laser wakefield acceleration data", Computational Science \& Discovery, 2009, 2:015005,

2008

O. Rübel, Prabhat, K. Wu, H. Childs, J. Meredith, C.G.R. Geddes, E. Cormier-Michel, S. Ahern, G.H. Weber, P. Messmer, H. Hagen, B. Hamann and E.W. Bethel, "High Performance Multivariate Visual Data Exploration for Extemely Large Data", Supercomputing (SC), Austin, Texas, USA, November 2008, LBNL 716E,

O. Rübel, Prabhat, K. Wu, H. Childs, J. Meredith, C.G.R. Geddes, E. Cormier-Michel, S. Ahern, G.H. Weber, P. Messmer, H. Hagen, B. Hamann and E.W. Bethel, "Application of High-performance Visual Analysis Methods to Laser Wakefield Particle Acceleration Data", IEEE Visualization 2008, October 2008,

B. Paul, S. Ahern, E. W. Bethel, E. Brugger, R. Cook, J. Daniel, K. Lewis, J. Owen, and D. Southard, "Chromium Renderserver: Scalable and Open Remote Rendering Infrastructure", IEEE Transactions on Visualization and Computer Graphics, May 2008, 14:627-639, LBNL 63693,

G.H. Weber, V. Beckner, H. Childs, T. Ligocki, M. Miller, B. van Straalen, E.W. Bethel, "Visualization of Scalar Adaptive Mesh Refinement Data", Numerical Modeling of Space Plasma Flows: Astronum-2007 (Astronomical Society of the Pacific Conference Series), April 2008, 385:309-320, LBNL 220E,

J. Chen, I. Yoon and E. W. Bethel, "Interactive, Internet Delivery of Visualization via Structured, Prerendered Multiresolution Imagery.", IEEE Transactions on Visualization and Computer Graphics, March 2008, 14:302-312, LBNL 62252,

William T.C. Kramer, John M. Shalf, E. Wes Bethel, D. Agarwal, Michael Banda, John Hules, Juan C. Meza, Leonid Oliker, Horst Simon, David Skinner, Francesca Verdier, Howard Walter, Michael Wehner, and Katherine Yelick, "HPC in 2016: A View Point from NERSC", Proceedings of the Cray User Group Conference, Helsinki, Finland, 2008,

Luke J Gosink, "Bin-hash indexing: A parallel method for fast query processing", 2008, LBNL 729E,

E. Wes Bethel, Oliver Rübel, Prabhat, Wu, Gunther H. Weber, Valerio Pascucci Hank Childs, Ajith Mascarenhas, Jeremy, Sean Ahern, "Modern Scientific Visualization is More than Just Pictures", Numerical Modeling of Space Plasma Flows: (Astronomical Society of the Pacific Series), St. Thomas, USVI, 2008, 301--317,

Luke J. Gosink, Kesheng Wu, E. Wes Bethel, D. Owens, Kenneth I. Joy, Bin-Hash Indexing: A Parallel Method For Fast Processing, 2008,

Oliver R\ ubel, Prabhat, Kesheng Wu, Hank, Jeremy Meredith, Cameron G. R. Geddes, Cormier-Michel, Sean Ahern, Gunther H., Peter Messmer, Hans Hagen, Bernd Hamann E. Wes Bethel, Application of High-performance Visual Analysis to Laser Wakefield Particle Acceleration Data, IEEE Visualization 2008, 2008,

Oliver R\ ubel, Prabhat, Kesheng Wu, Hank, Jeremy Meredith, Cameron G. R. Geddes, Cormier-Michel, Sean Ahern, Gunther H., Peter Messmer, Hans Hagen, Bernd Hamann E. Wes Bethel, High Performance Multivariate Visual Data Exploration Extemely Large Data, SuperComputing 2008 (SC08), Pages: 51 2008,

Daniela Ushizima, Oliver Rübel, Prabhat, Gunther Weber, E. Wes Bethel, Cecilia Aragon, Cameron Geddes, Estelle Cormier-Michel, Bernd Hamann, Peter Messmer, Hans Hagen, "Automated Analysis for Detecting Beams in Laser Wakefield Simulations", 2008 Seventh International Conference on Machine Learning and Applications, Proceedings of IEEE ICMLA'08, 2008, 382-387, LBNL 960E,

Luke J. Gosink, John C. Anderson, E. Wes Bethel, Kenneth I. Joy, "Query-Driven Visualization of Time-Varying Adaptive Mesh Refinement Data", IEEE Transactions on Visualization and Computer Graphics, 2008, 14:1715-1722, LBNL 803E,

E. Wes Bethel, Chris Johnson, Charles Hansen, Claudio Silva, Steven Parker, Allen Sanderson, Lee Myers, Martin Cole, Xavier Tricoche, Sean Ahern, George Ostrouchov, Dave Pugmire, Jamison Daniel, Jeremy Meredith, Valerio Pascucci, Hank Childs, Peer-Timo Bremer, Ajith Mascarenhas, Ken Joy, Bernd Hamann, Christoph Garth, Cecilia Aragon, Gunther Weber, and Prabhat, Seeing the Unseeable, SciDAC Review, Pages: 24-33, 2008,

O. Rübel, G. H. Weber, M-Y Huang, E. W. Bethel, S. V. E. Keränen, C. C. Fowlkes, C. L. Luengo Hendriks, A. H. DePace, L. Simirenko, M. B. Eisen, M. D. Biggin, H. Hagen, J. Malik, D. W. Knowles and B. Hamann, "PointCloudXplore 2: Visual Exploration of 3D Gene Expression", Visualization of Large and Unstructured Data Sets, edited by C. Garth, H. Hagen, M. Hering-Bertram, (Gesellschaft fuer Informatik (GI): 2008) LBNL 249E,

2006

K. Wu, K. Stockinger, A. Shoshani, Wes, "FastBit--Helps Finding the Proverbial Needle in a", 2006, LBNL LBNL-PUB/963,

Luke Gosink, John Shalf, Kurt Stockinger, Wu, Wes Bethel, "HDF5-FastQuery: Accelerating Complex Queries on Datasets using Fast Bitmap Indices", SSDBM 2006, Vienna, Austria, July 2006, IEEE Computer Society Press., 2006, 149--158,

Luke Gosink, John Shalf, Kurt Stockinger, Kesheng Wu, Wes Bethel, HDF5-FastQuery: Accelerating complex queries on HDF datasets using fast bitmap indices, 18th International Conference on Scientific and Statistical Database Management (SSDBM 06), Pages: 149--158 2006,

E. Wes Bethel, Scott Campbell, Eli Dart, Kurt Stockinger, Kesheng Wu, Accelerating Network Traffic Analysis Using Visualization, Symposium on Visual Analytics Science and Technology Baltimore, Maryland, USA, October 31 - November 2006, Pages: 115--122 2006,

E. Wes Bethel, Scott Campbell, Eli Dart, John Shalf, Kurt Stockinger, Kesheng Wu, High Performance Visualization using Query-Driven and Analytics, 2006,

Kurt Stockinger, E. Wes Bethel, Scott Campbell, Eli Dart, Kesheng Wu, Detecting distributed scans using high-performance visualization, SC 06, Pages: 82 2006,

2005

E. Wes Bethel, Scott Campbell, Eli Dart, Lee, Steven A. Smith, Kurt Stockinger, Tierney, Kesheng Wu, "Interactive Analysis of Large Network Data Collections Query-Driven Visualization", 2005,

Kurt Stockinger, John Shalf, Kesheng Wu, E Wes Bethel, "Query-driven visualization of large data sets", VIS 05. IEEE Visualization, 2005., 2005, 167--174,

Kurt Stockinger, John Shalf, Wes Bethel, Kesheng Wu, DEX: Increasing the Capability of Scientific Data Analysis by Using Efficient Bitmap Indices to Accelerate Scientific Visualization, SSDBM, Pages: 35-44 2005,

Kurt Stockinger, Kesheng Wu, Scott Campbell, Lau, Mike Fisk, Eugene Gavrilov, Alex, Christopher E. Davis, Rick Olinger, Rob, Jim Prewett, Paul Weber, Thomas P., E. Wes Bethel, Steve Smith, Network Traffic Analysis With Query Driven, SC 2005, 2005,

Horst D. Simon, William T.C. Kramer, David H. Bailey, Michael J. Banda, E. Wes Bethel, Jonathan T. Carter, James M. Craw, William J. Fortney, John A. Hules, Nancy L. Meyer, Juan C. Meza, Esmond G. Ng, Lynn E. Rippe, William C. Saphir, Francesca Verdier, Howard A. Walter, Katherine A. Yelick, "Science-Driven Computing: NERSC's Plan for 2006-2010", 2005,

2003

Gunther H. Weber, Martin Öhler, Oliver Kreylos, John Shalf, Wes Bethel, Bernd Hamann, Gerik Scheuermann, "Parallel Cell Projection Rendering of Adaptive Mesh Refinement Data", IEEE Symposium on Parallel and Large-Data Visualization and Graphics, 2003, 51-60,

E. Wes Bethel, Greg Abram, John Shalf, Randall Frank, Jim Ahrens, Steve Parker, N. Samatova, Mark Miller, Interoperability of Visualization Software and Data Models is NOT an Achievable Goal, IEEE Visualization, Pages: 607-610 2003,

T. J. Jankun-Kelly, Kreylos, Ma, Hamann, I. Joy, John Shalf, E. Wes Bethel, Deploying Web-Based Visual Exploration Tools on the Grid, IEEE Computer Graphics and Applications, Pages: 40-50 2003,

E. Wes Bethel, John Shalf, Grid-Distributed Visualizations Using Connectionless Protocols, IEEE Computer Graphics and Applications, Pages: 51-59 2003,

John Shalf, E. Wes Bethel, The Grid and Future Visualization System Architectures, IEEE Computer Graphics and Applications, Pages: 6-9 2003,

Joshua Boverhof

2019

Reinhard Gentz, Sean Peisert, Joshua Boverhof, Daniel Gunter, "SPARCS: Stream-Processing Architecture applied in Real-time Cyber-physical Security", Proceedings of the 15th IEEE International Conference on e-Science (eScience), San Diego, CA, IEEE, September 2019, doi: 10.1109/eScience.2019.00028

2017

YW Cheah, J Boverhof, A Elbashandy, D Agarwal, J Leek, T Epperly, J Eslick, D Miller, "Data management and simulation support accelerating carbon capture through computing", Proceedings of the 2016 IEEE 12th International Conference on e-Science, e-Science 2016, 2017, 389--398, doi: 10.1109/eScience.2016.7870924

S Peisert, R Gentz, J Boverhof, C McParland, S Engle, A Elbashandy, D Gunter, "LBNL Open Power Data", January 2017, doi: 10.21990/C21599

2016

DC Miller, D Agarwal, D Bhattacharyya, J Boverhof, YW Cheah, Y Chen, J Eslick, J Leek, J Ma, P Mahapatra, B Ng, NV Sahinidis, C Tong, SE Zitney, "Innovative computational tools and models for the design, optimization and control of carbon capture processes", Computer Aided Chemical Engineering, 2016, 38:2391--2396, doi: 10.1016/B978-0-444-63428-3.50403-3

2010

S Cholia, D Skinner, J Boverhof, "NEWT: A RESTful service for building High: Performance Computing web applications", 2010 Gateway Computing Environments Workshop, GCE 2010, 2010, doi: 10.1109/GCE.2010.5676125

2009

K Chard, W Tan, J Boverhof, R Madduri, I Foster, Wrap Scientific Applications as WSRE Grid Services using gRAVI, 2009 IEEE INTERNATIONAL CONFERENCE ON WEB SERVICES, VOLS 1 AND 2, Pages: 83--+ 2009, doi: 10.1109/ICWS.2009.110

2008

A Baranovski, K Beattie, S Bharathi, J Boverhof, J Bresnahan, A Chervenak, I Foster, T Freeman, D Gunter, K Keahey, C Kesselman, R Kettimuthu, N Leroy, M Link, M Livny, R Madduri, G Oleynik, L Pearlman, R Schuler, B Tierney, Enabling petascale science: Data management, troubleshooting, and scalable science services, Journal of Physics: Conference Series, 2008, doi: 10.1088/1742-6596/125/1/012068

2005

M Humphrey, G Wasson, J Gawor, J Bester, S Lang, I Foster, S Pickles, M McKeown, K Jackson, J Boverhof, M Rodriguez, L Berkeley, S Meder, IEEE, State and events for web services: A comparison of five WS-resource framework and WS-notification implementations, 14th IEEE International Symposium on High Performance Distributed Computing, Proceedings, Pages: 3--13 2005, doi: 10.1109/HPDC.2005.1520928

1999

L Smale, J Boverhof, The suprachiasmatic nucleus and intergeniculate leaflet of Arvicanthis niloticus, a diurnal murid rodent from east Africa, JOURNAL OF COMPARATIVE NEUROLOGY, Pages: 190--208 1999, doi: 10.1002/(SICI)1096-9861(19990111)403:2<190::AID-CNE4>3.0.CO;2-K

Surendra Byna

2024

D.K. Sung, Y. Son, A. Sim, K. Wu, S. Byna, H. Tang, H. Eom, C. Kim, S. Kim, "A2FL: Autonomous and Adaptive File Layout in HPC through Real-time Access Pattern Analysis", 38th IEEE International Parallel & Distributed Processing Symposium (IPDPS2024), 2024,

Neeraj Rajesh, Keith Bateman, Jean Luca Bez, Suren Byna, Anthony Kougkas, Xian-He Sun, "TunIO: An AI-powered Framework for Optimizing HPC I/O", 38th IEEE International Parallel & Distributed Processing Symposium, San Fransicso, CA, US, May 27, 2024,

Hammad Ather, Jean Luca Bez, Yankun Xia, Suren Byna, "Drilling Down I/O Bottlenecks with Cross-layer I/O Profile Exploration", 38th IEEE International Parallel & Distributed Processing Symposium, San Francisco, CA, USA, May 27, 2024,

Jean Luca Bez, Houjun Tang, Scot Breitenfeld, Huihuo Zheng, Wei-Keng Liao, Kaiyuan Hou, Zanhua Huang, Suren Byna, "h5bench: Exploring HDF5 Access Patterns Performance in Pre-Exascale Platforms", Concurrency and Computation: Practice and Experience (CCPE), January 31, 2024,

2023

Jean Luca Bez, Suren Byna, Shadi Ibrahim, "I/O Access Patterns in HPC Applications: A 360-Degree Survey", ACM Computing Surveys, September 15, 2023, 56, doi: 10.1145/3611007

Bin Dong, Jean Luca Bez, Suren Byna, "AIIO: Using Artificial Intelligence for Job-Level and Automatic I/O Performance Bottleneck Diagnosis.", In Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing (HPDC ’23), June 16, 2023,

Hammad Ather, Jean Luca Bez, Boyana Norris, Suren Byna, "Illuminating the I/O Optimization Path of Scientific Applications", High Performance Computing: 38th International Conference, ISC High Performance 2023, Hamburg, Germany, May 21–25, 2023, Proceedings, Hamburg, Germany, Springer-Verlag, May 21, 2023, 22–41, doi: https://doi.org/10.1007/978-3-031-32041-5_2

The existing parallel I/O stack is complex and difficult to tune due to the interdependencies among multiple factors that impact the performance of data movement between storage and compute systems. When performance is slower than expected, end-users, developers, and system administrators rely on I/O profiling and tracing information to pinpoint the root causes of inefficiencies. Despite having numerous tools that collect I/O metrics on production systems, it is not obvious where the I/O bottlenecks are (unless one is an I/O expert), their root causes, and what to do to solve them. Hence, there is a gap between the currently available metrics, the issues they represent, and the application of optimizations that would mitigate performance slowdowns. An I/O specialist often checks for common problems before diving into the specifics of each application and workload. Streamlining such analysis, investigation, and recommendations could close this gap without requiring a specialist to intervene in every case. In this paper, we propose a novel interactive, user-oriented visualization, and analysis framework, called Drishti. This framework helps users to pinpoint various root causes of I/O performance problems and to provide a set of actionable recommendations for improving performance based on the observed characteristics of an application. We evaluate the applicability and correctness of Drishti using four use cases from distinct science domains and demonstrate its value to end-users, developers, and system administrators when seeking to improve an application’s I/O performance.

Hammad Ather, Jean Luca Bez, Boyana Norris, Suren Byna, "Illuminating the I/O Optimization Path of Scientific Applications", International Conference on High Performance Computing (ISC'23), Springer Nature Switzerland, May 10, 2023, 22-41, doi: https://doi.org/10.1007/978-3-031-32041-5_2

S. Kim, A. Sim, K. Wu, S. Byna, Y. Son, H. Eom, "Design and Implementation of I/O Performance Prediction Scheme on HPC Systems through Large-scale Log Analysis", Journal of Big Data, 2023, 10(65), doi: 10.1186/s40537-023-00741-4

2022

Jean Luca Bez, Hammad Ather, Suren Byna, "Drishti: Guiding End-Users in the I/O Optimization Journey", PDSW 2022, held in conjunction with SC22, 2022,

Sunggon Kim, Alex Sim, Kesheng Wu, Suren Byna, Yongseok Son, "Design and implementation of dynamic I/O control scheme for large scale distributed file systems", Cluster Computing, 2022, 25(6):1--16, doi: 10.1007/s10586-022-03640-0

Jean Luca Bez, Ahmad Maroof Karimi, Arnab K. Paul, Bing Xie, Suren Byna, Philip Carns, Sarp Oral, Feiyi Wang, Jesse Hanley, "Access Patterns and Performance Behaviors of Multi-layer Supercomputer I/O Subsystems under Production Load", 31st International ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC '22), Association for Computing Machinery, June 27, 2022, 43–55, doi: 10.1145/3502181.3531461

D. Bard, C. Snavely, L. Gerhardt, J. Lee, B. Totzke, K. Antypas, W. Arndt, J. Blaschke, S. Byna, R. Cheema, S. Cholia, M. Day, B. Enders, A. Gaur, A. Greiner, T. Groves, M. Kiran, Q. Koziol, T. Lehman, K. Rowland, C. Samuel, A. Selvarajan, A. Sim, D. Skinner, L. Stephey, R. Thomas, G. Torok, "LBNL Superfacility Project Report", Lawrence Berkeley National Laboratory, 2022, doi: 10.48550/arXiv.2206.11992

Jean Luca Bez, Suren Byna, Understanding I/O Behavior with Interactive Darshan Log Analysis, Exascale Computing Project (ECP) Community Days BoF, 2022,

Houjun Tang, Quincey Koziol, John Ravi, and Suren Byna,, "Transparent Asynchronous Parallel I/O using Background Threads", IEEE Transactions on Parallel and Distributed Systems, April 4, 2022, 33, doi: 10.1109/TPDS.2021.3090322

2021

Qiao Kang, Scot Breitenfeld, Kaiyuan Hou, Wei-keng Liao, Robert Ross, and Suren Byna,, "Optimizing Performance of Parallel I/O Accesses to Non-contiguous Blocks in Multiple Array Variables", IEEE BigData 2021 conference, December 19, 2021,

J. Bang, C. Kim, K. Wu, A. Sim, S. Byna, H. Sung, H. Eom, "An In-Depth I/O Pattern Analysis in HPC Systems", IEEE International Conference on High Performance Computing, Data & Analytics (HiPC2021), 2021, doi: 10.1109/HiPC53243.2021.00056

Houjun Tang, Bing Xie, Suren Byna, Phillip Carns, Quincey Koziol, Sudarsun Kannan, Jay Lofstead, and Sarp Oral,, "SCTuner: An Auto-tuner Addressing Dynamic I/O Needs on Supercomputer I/O Sub-systems", 6th International Parallel Data Systems Workshop (PDSW) 2021, held in conjunction with SC21, November 21, 2021,

Cong Xu, Suparna Bhattacharya, Martin Foltin, Suren Byna, and Paolo Faraboschi, "Data-Aware Storage Tiering for Deep Learning", 6th International Parallel Data Systems Workshop (PDSW) 2021, held in conjunction with SC21, November 21, 2021,

Wei Zhang, Suren Byna, Hyogi Sim, Sangkeun Lee, Sudharshan Vazhkudai, and Yong Chen,, "Exploiting User Activeness for Data Retention in HPC Systems", International Conference for High Performance Computing, Networking, Storage and Analysis (SC '21), November 21, 2021, doi: https://doi.org/10.1145/3458817.3476201

Bo Fang, Daoce Wang, Sian Jin, Quincey Koziol, Zhao Zhang, Qiang Guan, Suren Byna, Sriram Krishnamoorthy, and Dingwen Tao,, "Characterizing Impacts of Storage Faults on HPC Applications: A Methodology and Insights", IEEE Cluster 2021, September 1, 2021,

Suren Byna, Houjun Tang, and Quincey Koziol,, Automatic and Transparent Scientific Data Management with Object Abstractions, PASC 2021, in a Minisymposium on "Data Movement Orchestration on HPC Systems", July 31, 2021,

Bing Xie, Houjun Tang, Suren Byna, Jesse Hanley, Quincey Koziol, Tonglin Li, Sarp Oral,, "Battle of the Defaults: Extracting Performance Characteristics of HDF5 under Production Load", CCGrid 2021, May 31, 2021,

Tonglin Li, Suren Byna, Quincey Koziol, Houjun Tang, Jean Luca Bez, Qiao Kang, "h5bench: HDF5 I/O Kernel Suite for Exercising HPC I/O Patterns", Cray User Group (CUG) 2021, January 1, 2021,

Jean Luca Bez, Houjun Tang, Bing Xie, David Williams-Young, Rob Latham, Rob Ross, Sarp Oral, Suren Byna, "I/O Bottleneck Detection and Tuning: Connecting the Dots using Interactive Log Analysis", 2021 IEEE/ACM Sixth International Parallel Data Systems Workshop (PDSW), January 1, 2021, 15-22, doi: 10.1109/PDSW54622.2021.00008

2020

D. Bard, C. Snavely, L. Gerhardt, J. Lee, B. Totzke, K. Antypas, S. Byna, R. Cheema, S. Cholia, M. Day, B. Enders, A. Gaur, A. Greiner, T. Groves, M. Kiran, Q. Koziol, K. Rowland, C. Samuel, A. Selvarajan, A. Sim, D. Skinner, R. Thomas, G. Torok, The Superfacility project: automated pipelines for experiments and HPC, International Conference for High Performance Computing, Networking, Storage, and Analysis (SC20), State of the Practice (SOP), 2020,

B. Enders, D. Bard, C. Snavely, L. Gerhardt, J. Lee, B. Totzke, K. Antypas, S. Byna, R. Cheema, S. Cholia, M. Day, A. Gaur, A. Greiner, T. Groves, M. Kiran, Q. Koziol, K. Rowland, C. Samuel, A. Selvarajan, A. Sim, D. Skinner, R. Thomas, G. Torok, "Cross-facility science with the Superfacility Project at LBNL", 2nd Workshop on Large-scale Experiment-in-the-Loop Computing (XLOOP 2020), in conjunction with the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 20), 2020, doi: 10.1109/XLOOP51963.2020.00006

Bin Dong, Ver\ onica Rodr\ \iguez Tribaldos, Xin Xing, Suren Byna, Jonathan Ajo-Franklin, Kesheng Wu, "DASSA: Parallel DAS Data Storage and Analysis for Subsurface Event Detection", 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS), July 14, 2020, 254--263,

Sunggon Kim, Alex Sim, Kesheng Wu, Suren Byna, Yongseok Son, Hyeonsang Eom, "Towards hpc i/o performance prediction through large-scale log analysis", Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing (HPDC 2020), 2020, 77--88, doi: 10.1145/3369583.3392678

Jiwoo Bang, Chungyong Kim, Kesheng Wu, Alex Sim, Suren Byna, Sunggon Kim, Hyeonsang Eom, "HPC Workload Characterization Using Feature Selection and Clustering", ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA 2020), in conjunction with The 29th ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC 2020), 2020, 33--40, doi: 10.1145/3391812.3396270

Houjun Tang, Suren Byna, Bin Dong, Quincey Koziol, "Parallel Query Service for Object-centric Data Management Systems", 2020 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), IEEE, May 18, 2020, 406-415,

Suren Byna, M. Scot Breitenfeld, Bin Dong, Quincey Koziol, Elena Pourmal, Dana Robinson, Jerome Soumagne, Houjun Tang, Venkatram Vishwanath, and Richard Warren, "ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems", Journal of Computer Science and Technology 2020, 35(1): 145-160, February 2, 2020, doi: 10.1007/s11390-020-9822-9

2019

Richard Warren, Jerome Soumagne, Jingqing Mu, Houjun Tang, Suren Byna, Bin Dong, Quincey Koziol, "Analysis in the Data Path of an Object-centric Data Management System", 26th IEEE International Conference on High Performance Computing, Data, and Analytics (HiPC 2019), December 18, 2019,

Houjun Tang, Suren Byna, Stephen Bailey, Zarija Lukic, Jialin Liu, Quincey Koziol, Bin Dong, "Tuning Object-centric Data Management Systems for Large Scale Scientific Applications", 26th IEEE International Conference on High Performance Computing, Data, and Analytics (HiPC 2019), December 18, 2019,

Wei Zhang, Suren Byna, Chenxu Niu, Yong Chen, "Exploring Metadata Search Essentials for Scientific Data Management", 26th IEEE International Conference on High Performance Computing, Data, and Analytics (HiPC 2019), December 17, 2019,

Tirthak Patel, Suren Byna, Glenn K. Lockwood, Devesh Tiwari, "Revisiting I/O Behavior in Large-Scale Storage Systems: The Expected and the Unexpected", Supercomputing 2019 (SC19), November 24, 2019, doi: 10.1145/3295500.3356183

Donghe Kang, Oliver Rübel, Suren Byna, Spyros Blanas, "Comparison of Array Management Library Performance - A Neuroscience Use Case", SC19 Poster, November 20, 2019,

Wei Zhang, Suren Byna, Houjun Tang, Brody Williams, Yong Chen, "MIQS: Metadata Indexing and erying Service for Self-Describing File Formats", The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC19), November 19, 2019,

Houjun Tang, Quincey Koziol, Suren Byna, John Mainzer, Tonglin Li, "Enabling Transparent Asynchronous I/O using Background Threads", 2019 IEEE/ACM Fourth International Parallel Data Systems Workshop (PDSW 2019), November 19, 2019, doi: DOI 10.1109/PDSW49588.2019.00006

Megha Agarwal, Divyansh Singhvi, Preeti Malakar, Suren Byna, "Active Learning-based Automatic Tuning and Prediction of Parallel I/O Performance", 2019 IEEE/ACM Fourth International Parallel Data Systems Workshop (PDSW), November 19, 2019, doi: DOI 10.1109/PDSW49588.2019.00007

Glenn K. Lockwood, Shane Snyder, Suren Byna, Philip Carns, Nicholas J. Wright, "Understanding Data Motion in the Modern HPC Data Center", 2019 IEEE/ACM Fourth International Parallel Data Systems Workshop (PDSW), November 19, 2019, doi: DOI 10.1109/PDSW49588.2019.00012

S. Kim, A. Sim, K. Wu, S. Byna, T. Wang, Y. Son, H. Eom, "DCA-IO: A Dynamic I/O Control Scheme for Parallel and Distributed File System", 19th Annual IEEE/ACM International Symposium in Cluster, Cloud, and Grid Computing (CCGrid 2019), 2019, doi: 10.1109/CCGRID.2019.00049

Teng Wang, Suren Byna, Glenn Lockwood, Philip Carns, Shane Snyder, Sunggon Kim, Nicholas Wright, "A Zoom-in Analysis of I/O Logs to Detect Root Causes of I/O Performance Bottlenecks", IEEE/ACM CCGrid 2019, May 14, 2019,

Tonglin Li, Quincey Koziol, Houjun Tang, Jialin Liu, Suren Byna, "I/O Performance Analysis of Science Applications Using HDF5 File-level Provenance", Cray User Group (CUG) 2019, May 10, 2019,

Jingqing Mu, Jerome Soumagne, Suren Byna, Quincey Koziol, Houjun Tang, Richard Warren, "Interfacing HDF5 with A Scalable Object-centric Storage System on Hierarchical Storage", Cray User Group (CUG) 2019, May 7, 2019,

Babak Behzad, Suren Byna, Prabhat, and Marc Snir, "Optimizing I/O Performance of HPC Applications with Autotuning", ACM Transactions on Parallel Computing (TOPC), February 28, 2019,

Beytullah Yildiz, Kesheng Wu, Suren Byna, Arie Shoshani, "Parallel membership queries on very large scientific data sets using bitmap indexes", Concurrency and Computation: Practice and Experience, January 1, 2019, 31:e5157,

Many scientific applications produce very large amounts of data as advances in hardware fuel computing and experimental facilities. Managing and analyzing massive quantities of scientific data is challenging as data are often stored in specific formatted files, such as HDF5 and NetCDF, which do not offer appropriate search capabilities. In this research, we investigated a special class of search capability, called membership query, to identify whether queried elements of a set are members of an attribute. Attributes that naturally have classification values appear frequently in scientific domains such as category and object type as well as in daily life such as zip code and occupation. Because classification attribute values are discrete and require random data access, performing a membership query on a large scientific data set creates challenges. We applied bitmap indexing and parallelization to membership queries to overcome these challenges. Bitmap indexing provides high performance not only for low cardinality attributes but also for high cardinality attributes, such as floating‐point variables, electric charge, or momentum in a particle physics data set, due to compression algorithms such as Word‐Aligned Hybrid. We conducted experiments, in a highly parallelized environment, on data obtained from a particle accelerator model and a synthetic data set.

Bin Dong, Kesheng Wu, Suren Byna, Houjun Tang, "SLOPE: Structural Locality-Aware Programming Model for Composing Array Data Analysis", International Conference on High Performance Computing, January 1, 2019, 61--80,

Bin Dong, Patrick Kilian, Xiaocan Li, Fan Guo, Suren Byna, Kesheng Wu, "Terabyte-scale Particle Data Analysis: An ArrayUDF Case Study", Proceedings of the 31st International Conference on Scientific and Statistical Database Management, January 1, 2019, 202--205,

2018

Suren Byna, Quincey Koziol, Venkatram Vishwanath, Jerome Soumagne, Houjun Tang, Kimmy Mu, Richard Warren, François Tessier, Bin Dong, Teng Wang, and Jialin Liu, Proactive Data Containers (PDC): An object-centric data store for large-scale computing systems, AGU Fall Meeting, December 13, 2018,

Glenn Lockwood, Shane Snyder, Teng Wang, Suren Byna, Phil Carns, and Nicholas Wright, "A Year in the Life of a Parallel File System", International Conference for High Performance Computing, Networking, and Storage (SC'18), IEEE / ACM, November 15, 2018,

Fahim Chowdhury, Jialin Liu, Quincey Koziol, Thorsten Kurth, Steven Farrell, Suren Byna, Prabhat, Weikuan Yu,, Initial Characterization of I/O in Large-Scale Deep Learning Applications, 3rd Joint International Workshop on Parallel Data Storage and Data Intensive Scalable Computing Systems (PDSW-DISCS), November 13, 2018,

Jialin Liu, Quincey Koziol, Gregory Butler, Neil Fortner, Mohamad Chaarawi, Houjun Tang, Suren Byna, Glenn Lockwood, Ravi Cheema, Kristy Kallback-Rose, Damian Hazen, Prabhat, "Evaluation of HPC Application I/O on Object Storage Systems", 3rd Joint International Workshop on Parallel Data Storage and Data Intensive Scalable Computing Systems (PDSW-DISCS), November 12, 2018,

Wei Zhang, Houjun Tang, Suren Byna, Yong Chen, "DART: Distributed Adaptive Radix Tree for Efficient Affix-based Keyword Search on HPC Systems", Proceedings of the 27th International Conference on Parallel Architectures and Compilation Techniques, November 1, 2018, 24,

Kimmy Mu, Jerome Soumagne, Houjun Tang, Suren Byna, Quincey Koziol, Richard Warren, "A Server-managed Transparent Object Storage Abstraction for HPC", 2018 IEEE International Conference on Cluster Computing (CLUSTER), September 10, 2018,

Teng Wang, Suren Byna, Glenn Lockwood, Nicholas Wright, Phil Carns, and Shane Snyder,, "IOMiner: Large-scale Analytics Framework for Gaining Knowledge from I/O Logs", IEEE Cluster 2018, September 10, 2018,

Teng Wang, Suren Byna, Bin Dong, and Houjun Tang, "UniviStor: Integrated Hierarchical and Distributed Storage for HPC", IEEE Cluster 2018., September 1, 2018,

Houjun Tang, Suren Byna, Francois Tessier, Teng Wang, Bin Dong, Jingqing Mu, Quincey Koziol, Jerome Soumagne, Venkatram Vishwanath, Jialin Liu, and Richard Warren, "Toward Scalable and Asynchronous Object-centric Data Management for HPC", 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) 2018, May 1, 2018,

Haoyuan Xing, Sofoklis Floratos, Spyros Blanas, Suren Byna, Prabhat, Kesheng Wu, and Paul Brown,, "ArrayBridge: Interweaving declarative array processing with imperative high-performance computing", 34th IEEE International Conference on Data Engineering (ICDE) 2018, April 17, 2018,

Bharti Wadhwa, Suren Byna, Ali R. Butt, "Toward Transparent Data Management in Multi-layer Storage Hierarchy for HPC Systems", IEEE International Conference on Cloud Engineering 2018 (IC2E 2018), April 17, 2018,

Haoyuan Xing, Sofoklis Floratos, Spyros Blanas, Suren Byna, M Prabhat, Kesheng Wu, Paul Brown, "ArrayBridge: Interweaving declarative array processing in SciDB with imperative HDF5-based programs", 2018 IEEE 34th International Conference on Data Engineering (ICDE), 2018, 977--988,

Bin Dong, Teng Wang, Houjun Tang, Quincey Koziol, Kesheng Wu, Suren Byna, "ARCHIE: Data analysis acceleration with array caching in hierarchical storage", 2018 IEEE International Conference on Big Data (Big Data), January 1, 2018, 211--220,

Kesheng Wu, Surendra Byna, Bin Dong, others, VPIC IO utilities, 2018,

Kesheng Wu, Bin Dong, Surendra Byna, "Scientific Data Services Framework for Plasma Physics", APS, 2018, 2018:BM10--006,

2017

Glenn Lockwood, Shane Snyder, Wucherl Yoo, Kevin Harms, Zachary Nault, Suren Byna, Philip Carns, Nicholas Wright, "UMAMI: A Recipe for Generating Meaningful Metrics through Holistic I/O Performance Analysis", 2nd Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems (PDSW-DISCS), 2017 (Held in conjunction with SC17), November 14, 2017,

Houjun Tang, Suren Byna, Bin Dong, Jialin Liu, and Quincey Koziol, "SoMeta: Scalable Object-centric Metadata Management for High Performance Computing", IEEE Cluster 2017, September 5, 2017,

Suren Byna, Mohamad Chaarawi, Quincey Koziol, John Mainzer, and Frank Willmore,, "Tuning HDF5 subfiling performance on parallel file systems", Cray User Group (CUG) meeting 2017, May 10, 2017,

Cong Xu, Shane Snyder, Omkar Kulkarni, Vishwanath Venkatesan, Philip Carns, Suren Byna, Robert Sisneros, and Kalyana Chadalavada,, "DXT: Darshan eXtended Tracing", Cray User Group (CUG) meeting 2017, May 10, 2017,

Jialin Liu, Quincey Koziol, Houjun Tang, François Tessier, Wahid Bhimji, Brandon Cook, Brian Austin, Suren Byna, Bhupender Thakur, Glenn Lockwood, Jack Deslippe, Prabhat, "Understanding the I/O Performance Gap Between Cori KNL and Haswell", Cray User Group Conference 2017 (CUG 2017), May 1, 2017,

Bin Dong, Kesheng Wu, Surendra Byna, Jialin Liu, Weijie Zhao, Florin Rusu, "ArrayUDF: User-defined scientific data analysis on arrays", Proceedings of the 26th International Symposium on High-Performance Parallel and Distributed Computing, January 1, 2017, 53--64,

2016

M. Bryson, S. Byna (Advisor), A. Sim (Advisor), K. Wu (Advisor), "The Search for Missing Parallel IO Performance on the Cori Supercomputer", International Conference for High Performance Computing, Networking, Storage and Analysis (SC’16), ACM Student Research Competition (SRC), 2016,

Bin Dong, Surendra Byna, Kesheng Wu, "SDS-Sort: Scalable Dynamic Skew-aware Parallel", HPDC 16, New York, NY, USA, ACM, 2016, 57--68, doi: 10.1145/2907294.2907300

Md. Mostofa Ali Patwary, Nadathur Rajagopalan Satish, Narayanan Sundaram, Jialin Liu, Peter Sadowski, Evan Racah, Suren Byna, Craig Tull, Wahid Bhimji, Prabhat, and Pradeep Dubey,, "PANDA: Extreme Scale Parallel K-Nearest Neighbor on Distributed Architectures", 30th IEEE International Parallel & Distributed Processing Symposium (IPDPS) 2016, Chicago, May 23, 2016,

Wahid Bhimji, Debbie Bard, Melissa Romanus, David Paul, Andrey Ovsyannikov, Brian Friesen, Matt Bryson, Joaquin Correa, Glenn K. Lockwood, Vakho Tsulaia, Suren Byna, Steve Farrell, Doga Gursoy, Chris Daley, Vince Beckner, Brian Van Straalen, Nicholas Wright, Katie Antypas, Prabhat,, "Accelerating Science with the NERSC Burst Buffer Early User Program", Cray User Group (CUG) 2016, May 10, 2016,

Cong Xu, Suren Byna, Vishwanath Venkatesan, Robert Sisneros, Omkar Kulkarni, Mohamad Chaarawi, and Kalyana Chadalavada, "LIOProf: Exposing Lustre File System Behavior for I/O Middleware", Cray User Group (CUG) 2016, May 10, 2016,

Dharshi Devendran, Suren Byna, Bin Dong, Brian van Straalen, Hans Johansen, Noel Keen, and Nagiza Samatova,, "Collective I/O Optimizations for Adaptive Mesh Refinement Data Writes on Lustre File System", Cray User Group (CUG) 2016, May 10, 2016,

Harinarayan Krishnan, Burlen Loring, Suren Byna, Michael F. Wehner, Travis A. O'Brien, Prabhat, Chris Paciorek, and Daithi Stone, "Enabling End-to-End Climate Science Workflows in High Performance Computing Environments", The AMS (American Meteorological Society) 96th Annual Meeting, January 6, 2016,

Burlen Loring, Suren Byna, Prabhat, Junmin Gu, Hari Krishnan, Michael Wehner, and Oliver Ruebel, "TECA an Extreme Event Detection and Climate Analysis Package for High Performance Computing", The AMS (American Meteorological Society) 96th Annual Meeting, January 6, 2016,

Houjun Tang, Suren Byna, Steve Harenberg, Xiaocheng Zou, Wenzhao Zhang, Kesheng Wu, Bin Dong, Oliver Rubel, Kristofer Bouchard, Scott Klasky, others, "Usage pattern-driven dynamic data layout reorganization", 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), January 1, 2016, 356--365,

Wenzhao Zhang, Houjun Tang, Steve Harenberg, Surendra Byna, Xiaocheng Zou, Dharshi Devendran, Daniel F Martin, Kesheng Wu, Bin Dong, Scott Klasky, others, "Amrzone: A runtime amr data sharing framework for scientific applications", 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), January 1, 2016, 116--125,

Bin Dong, Surendra Byna, Kesheng Wu, "Sds-sort: Scalable dynamic skew-aware parallel sorting", Proceedings of the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing, January 1, 2016, 57--68,

Xiaocheng Zou, David A Boyuka II, Dhara Desai, Daniel F Martin, Suren Byna, Kesheng Wu, "AMR-aware in situ indexing and scalable querying", Proceedings of the 24th High Performance Computing Symposium, January 1, 2016, 26,

Bin Dong, Suren Byna, Kesheng Wu, Hans Johansen, Jeffrey N Johnson, Noel Keen, others, "Data elevator: Low-contention data movement in hierarchical storage system", 2016 IEEE 23rd international conference on high performance computing (HiPC), January 1, 2016, 152--161,

Houjun Tang, Suren Byna, Steve Harenberg, Wenzhao Zhang, Xiaocheng Zou, Daniel F Martin, Bin Dong, Dharshi Devendran, Kesheng Wu, David Trebotich, others, "In situ storage layout optimization for amr spatio-temporal read accesses", 2016 45th International Conference on Parallel Processing (ICPP), January 1, 2016, 406--415,

Wenzhao Zhang, Houjun Tang, Stephen Ranshous, Surendra Byna, Daniel F Mart\ \in, Kesheng Wu, Bin Dong, Scott Klasky, Nagiza F Samatova, "Exploring memory hierarchy and network topology for runtime AMR data sharing across scientific applications", 2016 IEEE International Conference on Big Data (Big Data), January 1, 2016, 1359--1366,

2015

Hari Krishnan, Suren Byna, Michael Wehner, Junmin Gu, Travis O'Brien, Burlen Loring, Daithi Stone, William Collins, Prabhat, Yunjie Liu, Jeffrey Johnson, and Christopher Paciorek, "Enabling Efficient Climate Science Workflows in High Performance Computing Environments", AGU Fall Meeting, 2015, December 13, 2015,

Soyoung Jeon, Prabhat, Suren Byna, Junmin Gu, William Collins, and Michael Wehner,, "Characterization of extreme precipitation within atmospheric river events over California", Advances in Statistical Climatology, Meteorology and Oceanography (ASCMO), November 21, 2015, 1:45-57, doi: 10.5194/ascmo-1-45-2015

Md. Mostofa Ali Patwary, Suren Byna, Nadathur Rajagopalan Satish, Narayanan Sundaram, Zarija Lukic, Vadim Roytershteyn, Michael J. Anderson, Yushu Yao, Mr Prabhat, and Pradeep Dubey, "BD-CATS: Big Data Clustering at Trillion Particle Scale", Supercomputing 2015 (SC15), Supercomputing 2015 (SC15), November 17, 2015,

Babak Behzad, Suren Byna, Prabhat and Marc Snir, "Pattern-driven Parallel I/O Tuning", 10th Parallel Data Storage Workshop (PDSW) 2015, held in conjunction with SC15, 10th Parallel Data Storage Workshop (PDSW) 2015, to be held in conjunction with SC15, November 16, 2015,

Shane Snyder, Philip Carns, Robert Latham, Misbah Mubarak, Chris Carothers, Babak Behzad, Huong Vu Thanh Luu, Suren Byna, and Prabhat, "Techniques for Modeling Large-scale HPC I/O Workloads", the 6th International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS15), in conjunction with SC15, the 6th International Workshop on Performance Modeling, Benchmarking and Simulation of High Performa, November 15, 2015,

Jinoh Kim, Bin Dong, Suren Byna, and Kesheng Wu, "Security for the Scientific Data Service Framework", 2nd International Workshop on Privacy and Security of Big Data (PSBD 2015), in conjunction with IEEE BigData 2015, 2015,

Prabhat, Suren Byna, Venkat Vishwanath, Eli Dart, Michael Wehner, and William Collins,, "TECA: Petscale Pattern Recognition for Climate Science", 16th International Conference on Computer Analysis of Images and Patterns (CAIP) 2015, 2015,

Babak Behzad, Suren Byna, Stefan Wild, Prabhat and Marc Snir, "Dynamic Model-driven Parallel I/O Performance Tuning", IEEE Cluster 2015, 2015,

Xiaocheng (Chris) Zou, Suren Byna, Hans Johansen, Daniel Martin, Nagiza F. Samatova, Arie Shoshani, John Wu, "Six-fold Speedup of Ice Calving Detection Achieved by AMR-aware Parallel Connected Component Labeling", SciDAC PI Meeting, July 2015, 2015,

H. Luu, M. Winslett, W. Gropp, R. Ross, P. Carns, K. Harms, Prabhat, S. Byna, Y. Yao,, "A Multi-platform Study of I/O Behavior on Petascale Supercomputers", The 24th ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC) 2015, 2015,

Xiaocheng Zou, Kesheng Wu, David A. Boyuka, Daniel F. Martin, Suren Byna, Houjun, Kushal Bansal, Terry J. Ligocki, Hans Johansen, and Nagiza F. Samatova, "Parallel In Situ Detection of Connected Components Adaptive Mesh Refinement Data", Proceedings of the Cluster, Cloud and Grid Computing (CCGrid) 2015, 2015,

Suren Byna, Robert Sisneros, Kalyana Chadalavada, Quincey Koziol, "Tuning Parallel I/O on Blue Waters for Writing 10 Trillion Particles", Cray User Group (CUG) meeting 2015, 2015,

Suren Byna, Brian Austin, "Evaluation of Parallel I/O Performance and Energy Consumption with Frequency Scaling on Cray XC30", Cray User Group (CUG) meeting 2015, 2015,

Bin Dong, Surendra Byna, Kesheng Wu, "Heavy-tailed distribution of parallel I/O system response time", Proceedings of the 10th Parallel Data Storage Workshop, 2015, 37--42,

Bin Dong, Surendra Byna, Kesheng Wu, "Spatially clustered join on heterogeneous scientific data sets", 2015 IEEE International Conference on Big Data (Big Data), 2015, 371--380,

2014

Soyoung Jeon, Christopher Paciorek, Prabhat, Surendra Byna, William Collins, Michael Wehner, "Uncertainty Quantification for Characterizing Spatial Tail Dependence under Statistical Framework", AGU, Fall Meeting 2014, 2014,

Babak Behzad, Surendra Byna, Stefan M. Wild, Mr. Prabhat, Marc Snir, "Improving Parallel I/O Autotuning with Performance Modeling", ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC 2014), New York, NY, USA, ACM, 2014, 253--256, doi: 10.1145/2600212.2600708

Spyros Blanas, Kesheng Wu, Surendra Byna, Bin Dong, Arie Shoshani, "Parallel Data Analysis Directly on Scientific File Formats", SIGMOD 14, 2014, 385--396, doi: 10.1145/2588555.2612185

Spyros Blanas, Kesheng Wu, Surendra Byna, Bin Dong, Arie Shoshani, "Parallel data analysis directly on scientific file formats", Proceedings of the 2014 ACM SIGMOD international conference on Management of data, January 1, 2014, 385--396,

M Scot Breitenfeld, Kalyana Chadalavada, Robert Sisneros, Surendra Byna, Quincey Koziol, Neil Fortner, Prabhat, Venkat Vishwanath, "Recent Progress in Tuning Performance of Large-scale I/O with Parallel HDF5", The 9th Parallel Data Storage Workshop (PDSW) held in conjunction with SC14, 2014,

Hsuan-Te Chiu, Jerry Chou, Venkat Vishwanath, Surendra Byna, Kesheng Wu, "Simplifying index file structure to improve I/O performance of parallel indexing", 2014 20th IEEE International Conference on Parallel and Distributed Systems (ICPADS), 2014, 576--583,

Ted Habermann, Andrew Collette, Steve Vincena, Jay Jay Billings, Matt Gerring, Konrad Hinsen, Werner Benger, Filipe RNC Maia, Suren Byna, Pierre de Buyl, "The Hierarchical Data Format (HDF): A Foundation for Sustainable Data and Software", 2nd Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE2), in conjunction with Supercomputing 2014 (SC14), 2014,

Jialin Liu, Surendra Byna, Bin Dong, Kesheng Wu, Yong Chen, "Model-driven data layout selection for improving read performance", 2014 IEEE International Parallel \& Distributed Processing Symposium Workshops, 2014, 1708--1716,

Bin Dong, Surendra Byna, Kesheng Wu, "Parallel query evaluation as a Scientific Data Service", 2014 IEEE International Conference on Cluster Computing (CLUSTER), January 1, 2014, 194--202,

Jialin Liu, S. Byna, Bin Dong, Kesheng Wu, Chen, "Model-Driven Data Layout Selection for Improving Read", Parallel Distributed Processing Symposium Workshops 2014 IEEE International, 2014, 1708--1716, doi: 10.1109/IPDPSW.2014.190

Bin Dong, S. Byna, Kesheng Wu, Parallel query evaluation as a Scientific Data, Cluster Computing (CLUSTER), 2014 IEEE International on, Pages: 194--202 2014, doi: 10.1109/CLUSTER.2014.6968765

Hsuan-Te Chiu, Jerry Chou, Venkat Vishwanath, Byna, Kesheng Wu, Simplifying Index File Structure to Improve I/O of Parallel Indexing, The 20th IEEE International Conference on Parallel and Systems (ICPADS 2014), 2014,

2013

Babak Behzad, Huong Vu Thanh Luu, Joseph Huchette, Surendra Byna, Prabhat, Ruth Aydt, Quincey Koziol, and Marc Snir, "Taming parallel I/O complexity with auto-tuning", In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (SC '13), 2013,

Babak Behzad, Joseph Huchette, Huong Vu Thanh Luu, Ruth Aydt, Surendra Byna, Yushu Yao, Quincey Koziol, and Prabhat, "A framework for auto-tuning HDF5 applications", Proceedings of the 22nd international symposium on High-performance parallel and distributed computing (HPDC), 2013,

E Wes Bethel, Prabhat Prabhat, Suren Byna, Oliver R\ ubel, K John Wu, Michael Wehner, "Why high performance visual data analytics is both relevant and difficult", Visualization and Data Analysis 2013, January 2013, 8654:86540B, LBNL LBNL-6063E,

Bin Dong, Surendra Byna, Kesheng Wu, "SDS: a framework for scientific data services", Proceedings of the 8th Parallel Data Storage Workshop, January 1, 2013, 27--32,

Bin Dong, Surendra Byna, Kesheng Wu, "Expediting scientific data analysis with reorganization of data", 2013 IEEE International Conference on Cluster Computing (CLUSTER), January 1, 2013, 1--8,

Kuan-Wu Lin, Surendra Byna, Jerry Chou, Wu, "Optimizing FastQuery performance on Lustre file", Proceedings of the 25th International Conference on and Statistical Database Management, 2013, 29,

Bin Dong, S. Byna, Kesheng Wu, Expediting scientific data analysis with of data, Cluster Computing (CLUSTER), 2013 IEEE International on, Pages: 1--8 2013, doi: 10.1109/CLUSTER.2013.6702675

2012

Babak Behzad, Joey Huchette, Huong Luu, Ruth Aydt, Quincey Koziol, Prabhat, Suren Byna, Mohamad Chaarawi, Yushu Yao, "Auto-Tuning of Parallel IO Parameters for HDF5 Applications", Proceedings of the 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, 2012,

Y. Yin, S. Byna, H. Song, X.-H. Sun, and R. Thakur, "Boosting Application-Specific Parallel I/O Optimization Using IOSIG", IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), Ottowa, Canada, May 13, 2012,

Surendra Byna, Jerry Chou, Oliver Rubel, Homa Karimabadi, William S Daughter, Vadim Roytershteyn, E Wes Bethel, Mark Howison, Ke-Jou Hsu, Kuan-Wu Lin, others, "Parallel I/O, analysis, and visualization of a trillion particle simulation", SC 12: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, January 2012, 1--12,

Oliver R\ ubel, Surendra Byna, Kesheng Wu, Fuyu Li, Michael Wehner, Wes Bethel, others, "Teca: A parallel toolkit for extreme climate analysis", Procedia Computer Science, Elsevier, January 2012, 9:866--876, LBNL 5352E,

We present TECA, a parallel toolkit for detecting extreme events in large climate datasets. Modern climate datasets expose parallelism across a number of dimensions: spatial locations, timesteps and ensemble members. We design TECA to exploit these modes of parallelism and demonstrate a prototype implementation for detecting and tracking three classes of extreme events: tropical cyclones, extra-tropical cyclones and atmospheric rivers. We process a modern TB-sized CAM5 simulation dataset with TECA, and demonstrate good runtime performance for the three case studies.

E. W. Bethel, Surendra Byna, Jerry Chou, Cormier-Michel, Cameron G. R. Geddes, Howison, Fuyu Li, Prabhat, Ji Qiang, R\ ubel, Rob D. Ryne, Michael Wehner, Wu, "Big Data Analysis and Visualization: What Do LINACS Tropical Storms Have In Common?", 11th International Computational Accelerator Physics ICAP 2012, Germany, 2012,

EW Bethel, S. Byna, J. Chou, E., CGR Geddes, M. Howison, F. Li J. Q. Prabhat, O. R\ ubel, RD Ryne and, Big Data Analysis and Visualization: What Do LINACS Tropical Storms Have In Common?, 11th International Computational Accelerator Physics ICAP 2012, 2012,

O. R\ ubel, S. Byna, K. Wu, F. Li, M., W. Bethel, others, "TECA: A Parallel Toolkit for Extreme Climate", Procedia Computer Science, Elsevier, 2012, 9:866--876, doi: 10.1016/j.procs.2012.04.093

2011

Mehmet Balman, Suredra Byna, "Open Problems in network-aware data management in exa-scale computing and terabit networking era", In Proceedings of the First international Workshop on Network-Aware Data Management, in conjunction with ACM/IEEE international Conference For High Performance Computing, Networking, Storage and Analysis, 2011, Seattle, WA, November 11, 2011, LBNL 6176E, doi: http://dx.doi.org/10.1145/2110217.2110229

Accessing and managing large amounts of data is a great challenge in collaborative computing environments where resources and users are geographically distributed. Recent advances in network technology led to next-generation high- performance networks, allowing high-bandwidth connectiv- ity. Efficient use of the network infrastructure is necessary in order to address the increasing data and compute require- ments of large-scale applications. We discuss several open problems, evaluate emerging trends, and articulate our per- spectives in network-aware data management. 

Kesheng Wu, Surendra Byna, Doron Rotem, Arie, "Scientific Data Services -- A High-Performance I/O with Array Semantics", HPCDB, IEEE, 2011, doi: 10.11v45/2125636.2125640

Surendra Byna, Michael F Wehner, Kesheng John Wu, "Detecting atmospheric rivers in large climate datasets", Proceedings of the 2nd international workshop on Petascal data analytics: challenges and opportunities, 2011, 7--14,

Extreme precipitation events on the western coast of North America are often traced to an unusual weather phenomenon known as atmospheric rivers. Although these storms may provide a significant fraction of the total water to the highly managed western US hydrological system, the resulting intense weather poses severe risks to the human and natural infrastructure through severe flooding and wind damage. To aid the understanding of this phenomenon, we have developed an efficient detection algorithm suitable for analyzing large amounts of data. In addition to detecting actual events in the recent observed historical record, this detection algorithm can be applied to global climate model output providing a new model validation methodology. Comparing the statistical behavior of simulated atmospheric river events in models to observations will enhance confidence in projections of future extreme storms. Our detection algorithm is based on a thresholding condition on the total column integrated water vapor established by Ralph et al. (2004) followed by a connected component labeling procedure to group the mesh points into connected regions in space. We develop an efficient parallel implementation of the algorithm and demonstrate good weak and strong scaling. We process a 30-year simulation output on 10,000 cores in under 3 seconds.

M Prabhat, S Byna, C Paciorek, G Weber, K Wu, T Yopes, MF Wehner, G Ostrouchov, D Pugmire, R Strelitz, others, "Pattern Detection and Extreme Value Analysis on Large Climate Data", AGUFM, Pages: IN41C--03 January 2011,

1969

Md Kamal Hossain Chowdhury, Houjun Tang, Jean Luca Bez, Purushotham V. Bangalore, Suren Byna, "Efficient Asynchronous I/O with Request Merging", 2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), St. Petersburg, FL, USA, IEEE, December 31, 1969, 628-636, doi: 10.1109/IPDPSW59300.2023.00107

David Camp

2022

E. Wes Bethel, Burlen Loring, Utkarsh Ayatchit, David Camp, P. N. Duque, Nicola Ferrier, Joseph Insley, Junmin Gu, Kress, Patrick O’Leary, David Pugmire, Silvio Rizzi, Thompson, Gunther H. Weber, Brad Whitlock, Matthew Wolf, Kesheng Wu, "The SENSEI Generic In Situ Interface: Tool and Processing Portability at Scale", In Situ Visualization for Computational Science, ( 2022) doi: 10.1007/978-3-030-81627-8_13

2018

B Loring, A Myers, D Camp, EW Bethel, "Python-based in situ analysis and visualization", Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization - ISAV 18, ACM Press, 2018, doi: 10.1145/3281464.3281465

2013

David Camp, Hari Krishnan, David Pugmire, Christoph Garth, Ian Johnson, E. Wes Bethel, Kenneth I. Joy, and Hank Childs., "GPU Acceleration of Particle Advection Workloads in a Parallel, Distributed Memory Setting", Proceedings of Eurographics Symposium on Parallel Graphics and Visualization (EGPGV), May 5, 2013,

2012

Hank Childs, Eric Brugger, Brad Whitlock, Jeremy Meredith, Sean Ahern, David Pugmire, Kathleen Biagas, Mark Miller, Cyrus Harrison, Gunther H. Weber, Hari Krishnan, Thomas Fogal, Allen Sanderson, Christoph Garth, E. Wes Bethel, David Camp, Oliver Rubel, Marc Durant, Jean M. Favre, Paul Navratil, "VisIt: An End-User Tool For Visualizing and Analyzing Very Large Data", High Performance Visualization---Enabling Extreme-Scale Scientific Insight, ( October 2012) Pages: 357--372

E. Wes Bethel, David Camp, Hank Childs, Christoph Garth, Mark Howison, Kenneth I. Joy, David Pugmire, "Hybrid Parallelism", High Performance Visualization---Enabling Extreme-Scale Scientific Insight, ( October 2012) Pages: 261--290

David Camp, Hank Childs, Christoph Garth, David Pugmire, Kenneth I. Joy, "Parallel Stream Surface Computation for Large Data Sets", Proceedings of IEEE Symposium on Large Data Analysis and Visualization (LDAV), October 2012, 39--47, LBNL 5776E,

E. Wes Bethel, David Camp, Hank Childs, Mark Howison, Hari Krishnan, Burlen Loring, Joerg Meyer, Prabhat, Oliver Ruebel, Daniela Ushizima, Gunther Weber, "Towards Exascale: High Performance Visualization and Analytics – Project Status Report. Technical Report", DOE Exascale Research Conference, April 2012,

2011

David Camp, Christoph Garth, Hank Childs, Dave Pugmire, Kenneth I. Joy, "Streamline Integration Using MPI-Hybrid Parallelism on a Large Multicore Architecture", IEEE Transactions on Visualization and Computer Graphics, November 2011, 17:1702-1713, LBNL 4563E, doi: http://doi.ieeecomputersociety.org/10.1109/TVCG.2010.259

David Camp, Hank Childs, Amit Chourasia, Christoph Garth, Kenneth Joy, "Evaluating the Benefits of An Extended Memory Hierarchy for Parallel Streamline Algorithms", Proceedings IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV) 2011, Providence, RI, USA, IEEE Press, October 2011, 57-64, LBNL 5503E,

You-Wei Cheah

2021

D. A. Agarwal, J. Damerow, C. Varadharajan, D. S. Christianson, G. Z. Pastorello, Y.-W. Cheah, L. Ramakrishnan, "Balancing the needs of consumers and producers for scientific data collections", Ecological Informatics, 2021, 62:101251, doi: 10.1016/j.ecoinf.2021.101251

2020

G. Z. Pastorello, C. Trotta, E. Canfora, H. Chu, D. Christianson, Y.-W. Cheah, C. Poindexter, J. Chen, A. Elbashandy, M. Humphrey, P. Isaac, D. Polidori, M. Reichstein, A. Ribeca, C. van Ingen, N. Vuichard, L. Zhang, B. Amiro, C. Ammann, M. A. Arain, J. Ardö, T. Arkebauer, S. K. Arndt, N. Arriga, M. Aubinet, M. Aurela, D. Baldocchi, A. Barr, E. Beamesderfer, L. B. Marchesini, O. Bergeron, J. Beringer, C. Bernhofer, D. Berveiller, D. Billesbach, T. A. Black, P. D. Blanken, G. Bohrer, J. Boike, P. V. Bol stad, D. Bonal, J.-M. Bonnefond, D. R. Bowling, R. Bracho, J. Brodeur, C. Brümmer, N. Buchmann, B. Burban, S. P. Burns, P. Buysse, P. Cale, M. Cavagna, P. Cellier, S. Chen, I. Chini, T. R. Chris tensen, J. Cleverly, A. Collalti, C. Consalvo, B. D. Cook, D. Cook, C. Coursolle, E. Cremonese, P. S. Curtis, E. D’Andrea, H. da Rocha, X. Dai, K. J. Davis, B. D. Cinti, A. de Grandcourt, A. D. Ligne, R. C. D. Oliveira, N. Delpierre, A. R. Desai, C. M. D. Bella, P. di Tommasi, H. Dolman, F. Domingo, G. Dong, S. Dore, P. Duce, E. Dufrêne, A. Dunn, J. Dušek, D. Eamus, U. Eichelmann, H. A. M. ElKhidir, W. Eugster, C. M. Ewenz, B. Ewers, D. Famulari, S. Fares, I. Feigenwinter, A. Feitz, R. Fensholt, G. Fil ippa, M. Fischer, J. Frank, M. Galvagno, M. Gharun, D. Gianelle, B. Gielen, B. Gioli, A. Gitelson, I. Goded, M. Goeckede, A. H. Goldstein, C. M. Gough, M. L. Goulden, A. Graf, A. Griebel, C. Gruening, T. Grünwald, A. Hammerle, S. Han, X. Han, B. U. Hansen, C. Hanson, J. Hatakka, Y. He, M. Hehn, B. Heinesch, N. Hinko-Najera, L. Hörtnagl, L. Hutley, A. Ibrom, H. Ikawa, M. Jackowicz-Korczynski, D. Janouš, W. Jans, R. Jassal, S. Jiang, T. Kato, M. Khomik, J. Klatt, A. Knohl, S. Knox, H. Kobayashi, G. Koerber, O. Kolle, Y. Kosugi, A. Kotani, A. Kowalski, B. Kruijt, J. Kurbatova, W. L. Kutsch, H. Kwon, S. Launiainen, T. Laurila, B. Law, R. Leuning, Y. Li, M. Liddell, J.-M. Limousin, M. Lion, A. J. Liska, A. Lohila, A. López-Ballesteros, E. López-Blanco, B. Loubet, D. Loustau, A. Lucas-Moffat, J. Lüers, S. Ma, C. Macfarlane, V. Magliulo, R. Maier, I. Mammarella, G. Manca, B. Marcolla, H. A. Margolis, S. Mar ras, W. Massman, M. Mastepanov, R. Matamala, J. H. Matthes, F. Mazzenga, H. McCaughey, I. McHugh, A. M. S. McMillan, L. Merbold, W. Meyer, T. Meyers, S. D. Miller, S. Minerbi, U. Moderow, R. K. Monson, L. Montagnani, C. E. Moore, E. Moors, V. Moreaux, C. Moureaux, J. W. Munger, T. Nakai, J. Neirynck, Z. Nesic, G. Nicolini, A. Noormets, M. Northwood, M. Nosetto, Y. Nouvellon, K. Novick, W. Oechel, J. E. Olesen, J.-M. Ourcival, S. A. Papuga, F.-J. Parmentier, E. Paul-Limoges, M. Pavelka, M. Peichl, E. Pendall, R. P. Phillips, K. Pilegaard, N. Pirk, G. Posse, T. Powell, H. Prasse, S. M. Prober, S. Ram bal, U. Rannik, N. Raz-Yaseef, D. Reed, V. R. de Dios, N. Restrepo-Coupe, B. R. Reverter, M. Roland, S. Sabbatini, T. Sachs, S. R. Saleska, E. P. S.-C. nete, Z. M. Sanchez-Mejia, H. P. Schmid, M. Schmidt, K. Schneider, F. Schrader, I. Schroder, R. L. Scott, P. Sedlák, P. Serrano-Ortíz, C. Shao, P. Shi, I. Shironya, L. Siebicke, L. Šigut, R. Silberstein, C. Sirca, D. Spano, R. Steinbrecher, R. M. Stevens, C. Sturtevant, A. Suyker, T. Tagesson, S. Takanashi, Y. Tang, N. Tapper, J. Thom, F. Tiedemann, M. Tomassucci, J.-P. Tuovinen, S. Urbanski, R. Valentini, M. van der Molen, E. van Gorsel, K. van Huissteden, A. Varlagin, J. Verfaillie, T. Vesala, C. Vincke, D. Vitale, N. Vygodskaya, J. P. Walker, E. Walter-Shea, H. Wang, R. Weber, S. Westermann, C. Wille, S. Wofsy, G. Wohlfahrt, S. Wolf, W. Woodgate, Y. Li, R. Zampedri, J. Zhang, G. Zhou, D. Zona, D. Agarwal, S. Biraud, M. Torn, D. Papale, "The FLUXNET2015 dataset and the ONEFlux processing pipeline for eddy covariance data", Scientific Data, 2020, 7:225, doi: 10.1038/s41597-020-0534-3

2018

You-Wei Cheah, Danielle Svehla Christianson, Housen Chu, Gilberto Pastorello, Fianna O’Brien, Yeongshnn Ong, Catharine van Ingen, Margaret Torn, Deb Agarwal, AmeriFlux BADM: Implementing lessons from 12 years of long-tail data management into next generation earth science systems (IN34A-03), 2018 AGU Fall Meeting, Washington, D.C., December 12, 2018,

Cheah You-Wei, Drew Paine, Devarshi Ghoshal, Lavanya Ramakrishnan, Bringing Data Science to Qualitative Analysis, 2018 IEEE 14th International Conference on e-Science, Pages: 325-326 2018, doi: 10.1109/eScience.2018.00076

2017

YW Cheah, J Boverhof, A Elbashandy, D Agarwal, J Leek, T Epperly, J Eslick, D Miller, "Data management and simulation support accelerating carbon capture through computing", Proceedings of the 2016 IEEE 12th International Conference on e-Science, e-Science 2016, 2017, 389--398, doi: 10.1109/eScience.2016.7870924

Gilberto Z. Pastorello, Dan K. Gunter, Housen Chu, Danielle S. Christianson, Carlo Trotta, Eleonora Canfora, Boris Faybishenko, You-Wei Cheah, Norm Beekwilder, Stephen W. Chan, Sigrid Dengel, Trevor Keenan, Fianna O Brien, Abderahman Elbashandy, Cristina M. Poindexter, Marty Humphrey, Dario Papale, Deb A. Agarwal, "Hunting Data Rogues at Scale: Data Quality Control for Observational Data in Research Infrastructures", Proceedings of the 13th IEEE International Conference on e-Science (e-Science 2017), Auckland, New Zealand, 2017, doi: 10.1109/eScience.2017.64

2016

DC Miller, D Agarwal, D Bhattacharyya, J Boverhof, YW Cheah, Y Chen, J Eslick, J Leek, J Ma, P Mahapatra, B Ng, NV Sahinidis, C Tong, SE Zitney, "Innovative computational tools and models for the design, optimization and control of carbon capture processes", Computer Aided Chemical Engineering, 2016, 38:2391--2396, doi: 10.1016/B978-0-444-63428-3.50403-3

2013

You-Wei Cheah, Richard Canon, Beth Plale, Lavanya Ramakrishnan, "Milieu: Provenance Collection and Query Framework for High Performance Computing Systems", IEEE Big Data Congress, 2013,

2011

D Agarwal, YW Cheah, D Fay, J Fay, D Guo, T Hey, M Humphrey, K Jackson, Jie Li, C Poulain, Y Ryu, C Van Ingen, "Data-intensive science: The Terapixel and MODISAzure projects", International Journal of High Performance Computing Applications, 2011, 25:304--316, doi: 10.1177/1094342011414746

2010

J Li, M Humphrey, YW Cheah, Y Ryu, D Agarwal, K Jackson, C Van Ingen, Fault tolerance and scaling in e-Science cloud applications: Observations from the continuing development of MODISAzure, Proceedings - 2010 6th IEEE International Conference on e-Science, eScience 2010, Pages: 246--253 2010, doi: 10.1109/eScience.2010.47

Shreyas Cholia

2022

D. Bard, C. Snavely, L. Gerhardt, J. Lee, B. Totzke, K. Antypas, W. Arndt, J. Blaschke, S. Byna, R. Cheema, S. Cholia, M. Day, B. Enders, A. Gaur, A. Greiner, T. Groves, M. Kiran, Q. Koziol, T. Lehman, K. Rowland, C. Samuel, A. Selvarajan, A. Sim, D. Skinner, L. Stephey, R. Thomas, G. Torok, "LBNL Superfacility Project Report", Lawrence Berkeley National Laboratory, 2022, doi: 10.48550/arXiv.2206.11992

MB Simmonds, WJ Riley, DA Agarwal, X Chen, S Cholia, R Crystal-Ornelas, ET Coon, D Dwivedi, VC Hendrix, M Huang, A Jan, Z Kakalia, J Kumar, CD Koven, L Li, M Melara, L Ramakrishnan, DM Ricciuto, AP Walker, W Zhi, Q Zhu, C Varadharajan, Guidelines for Publicly Archiving Terrestrial Model Data to Enhance Usability, Intercomparison, and Synthesis, Data Science Journal, 2022, doi: 10.5334/dsj-2022-003

2020

D. Bard, C. Snavely, L. Gerhardt, J. Lee, B. Totzke, K. Antypas, S. Byna, R. Cheema, S. Cholia, M. Day, B. Enders, A. Gaur, A. Greiner, T. Groves, M. Kiran, Q. Koziol, K. Rowland, C. Samuel, A. Selvarajan, A. Sim, D. Skinner, R. Thomas, G. Torok, The Superfacility project: automated pipelines for experiments and HPC, International Conference for High Performance Computing, Networking, Storage, and Analysis (SC20), State of the Practice (SOP), 2020,

B. Enders, D. Bard, C. Snavely, L. Gerhardt, J. Lee, B. Totzke, K. Antypas, S. Byna, R. Cheema, S. Cholia, M. Day, A. Gaur, A. Greiner, T. Groves, M. Kiran, Q. Koziol, K. Rowland, C. Samuel, A. Selvarajan, A. Sim, D. Skinner, R. Thomas, G. Torok, "Cross-facility science with the Superfacility Project at LBNL", 2nd Workshop on Large-scale Experiment-in-the-Loop Computing (XLOOP 2020), in conjunction with the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 20), 2020, doi: 10.1109/XLOOP51963.2020.00006

2019

C Varadharajan, S Cholia, C Snavely, V Hendrix, C Procopiou, D Swantek, W Riley, D Agarwal, "Launching an Accessible Archive of Environmental Data", Eos, January 1, 2019, 100, doi: 10.1029/2019eo111263

2018

Wahid Bhimji, Steven Farrell, Oliver Evans, Matthew Henderson, Shreyas Cholia, Aaron Vose, Mr Prabhat, Rollin Thomas, Richard Shane Canon, "Interactive HPC Deep Learning with Jupyter Notebooks", Supercomputing 2018, Dallas, TX, November 2018,

Shreyas Cholia, Matthew Henderson, Oliver Evans, Fernando Perez, "Kale: A System for Enabling Human-in-the-loop Interactivity in HPC Workflows", Science Gateways 2018, figshare, September 26, 2018, doi: 10.6084/m9.figshare.7067075.v3

S. Farrell, A. Vose, O. Evans, M. Henderson, S. Cholia, W. Bhimji, R. Thomas, S.
Canon, and Prabhat,,
"Interactive Distributed Deep Learning with Jupyter Notebooks", ISC Workshop on Interactive High-Performance Computing, June 28, 2018,

Dáithí A Stone, Mark D Risser, Oliver M Angélil, Michael F Wehner, Shreyas Cholia, Noel Keen, Harinarayan Krishnan, Travis A O Brien, William D Collins, "A basis set for exploration of sensitivity to prescribed ocean conditions for estimating human contributions to extreme weather in CAM5. 1-1degree", Weather and climate extremes, 2018, 19:10--19,

Anubhav Jain, Joseph Montoya, Shyam Dwaraknath, Nils ER Zimmermann, John Dagdelen, Matthew Horton, Patrick Huck, Donny Winston, Shreyas Cholia, Shyue Ping Ong, others, "The Materials Project: Accelerating Materials Design Through Theory-Driven Data and Tools", Handbook of Materials Modeling: Methods: Theory and Modeling, (Springer: 2018) Pages: 1--34

2017

Shreyas Cholia, Matthew Henderson, Oliver Evans, Demo: Extending Jupyter to Support Interactive High Performance Computing, Science Gateways 2017, October 2017, doi: 10.6084/m9.figshare.5501137.v1

DA Agarwal, C Varadharajan, S Cholia, C Snavely, VC Hendrix, D Gunter, WJ Riley, M jones, AE budden, D Vieglas, Environmental System Science Data Infrastructure for a Virtual Ecosystem (ESS-DIVE)-A New US DOE Data Archive, 2017,

Rollin Thomas, Shane Canon, Shreyas Cholia, Lisa Gerhardt, Evan Racah, "Toward Interactive Supercomputing at NERSC with Jupyter", Cray User Group (CUG) Conference Proceedings, 2017,

2016

Patrick Huck, Dan Gunter, Shreyas Cholia, Donald Winston, AT N Diaye, Kristin Persson, "User applications driven by the community contribution framework MPContribs in the Materials Project", Concurrency and Computation: Practice and Experience, 2016, 28:1982--1993,

A Greiner, E Racah, R Canon, J Liu, Y Liu, D Bard, L Gerhardt, R Thomas, S Cholia, J Porter, W Bhimji, Q Koziol, Prabhat, "Data-Intensive Supercomputing for Science", 2016,

2015

Stone, D., H. Shiogama, P. Wolski, O. Angélil, S. Cholias, N. Christidis, A. Dittus, C. Folland, A. King, J. Kinter, H. Krishnan, S.-K. Min, M. Wehner, "The C20C+ Detection and Attribution Project", Fall Meeting of the American Geophysical Union, 2015,

P Huck, D Gunter, S Cholia, D Winston, A N Diaye, KA Persson, "User Applications Driven by the Community Contribution Framework MPContribs in the Materials Project.", CoRR, 2015, abs/1510,

SP Ong, S Cholia, A Jain, M Brafman, D Gunter, G Ceder, KA Persson, "The Materials Application Programming Interface (API): A simple, flexible and efficient API for materials data based on REpresentational State Transfer (REST) principles", Computational Materials Science, 2015, 97:209--215, doi: 10.1016/j.commatsci.2014.10.037

S Cholia, T Sun, "The NEWT platform: An extensible plugin framework for creating ReSTful HPC APIs", Proceedings of GCE 2014: 9th Gateway Computing Environments Workshop, held in conjunction with SC 2014: The International Conference for High Performance Computing, Networking, Storage and Analysis, 2015, 17--20, doi: 10.1109/GCE.2014.14

R Madduri, A Rodriguez, T Uram, K Heitmann, T Malik, S Sehrish, R Chard, S Cholia, M Paterno, J Kowalkowski, S Habib, "PDACS: A Portal for Data Analysis Services for Cosmological Simulations", Computing in Science and Engineering, 2015, 17:18--26, doi: 10.1109/MCSE.2015.87

NJ Wright, SS Dosanjh, AK Andrews, KB Antypas, B Draney, RS Canon, S Cholia, CS Daley, KM Fagnan, RA Gerber, L Gerhardt, L Pezzaglia, A Prabhat, KH Schafer, J Srinivasan, "Cori: A pre-exascale supercomputer for big data and HPC applications", Advances in Parallel Computing, 2015, 26:82--100, doi: 10.3233/978-1-61499-583-8-82

R Chard, S Sehrish, A Rodriguez, R Madduri, TD Uram, M Paterno, K Heitmann, S Cholia, J Kowalkowski, S Habib, "PDACS-A portal for data analysis services for cosmological simulations", Proceedings of GCE 2014: 9th Gateway Computing Environments Workshop, held in conjunction with SC 2014: The International Conference for High Performance Computing, Networking, Storage and Analysis, 2015, 30--33, doi: 10.1109/GCE.2014.7

2014

Dáithí Stone, Michael Wehner, Shreyas Cholia, Harinarayan Krishnan, Piotr Wolski, Mark Tadross, Chris Folland, Nikos Christidis, Hideo Shiogama, "The C20C+ Detection and Attribution Project", Integrated Climate Modeling Principal Investigator Meeting 2014, 2014,

R Cowles, C Jackson, V Welch, S Cholia, "A model for identity management in future scientific collaboratories", Proceedings of Science, 2014, 23-28-Ma,

2013

A Jain, SP Ong, G Hautier, W Chen, WD Richards, S Dacek, S Cholia, D Gunter, D Skinner, G Ceder, KA Persson, "Commentary: The materials project: A materials genome approach to accelerating materials innovation", APL Materials, 2013, 1, doi: 10.1063/1.4812323

SP Ong, WD Richards, A Jain, G Hautier, M Kocher, S Cholia, D Gunter, VL Chevrier, KA Persson, G Ceder, "Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis", Computational Materials Science, 2013, 68:314--319, doi: 10.1016/j.commatsci.2012.10.028

O Rübel, A Greiner, S Cholia, K Louie, EW Bethel, TR Northen, BP Bowen, "OpenMSI: A high-performance web-based platform for mass spectrometry imaging", Analytical Chemistry, 2013, 85:10354--103, doi: 10.1021/ac402540a

M Di Pierro, J Hetrick, S Cholia, J Simone, C Deta, "The new Gauge Connection at NERSC", Proceedings of Science, 2013, 29-July-,

2012

D Gunter, S Cholia, A Jain, M Kocher, K Persson, L Ramakrishnan, SP Ong, G Ceder, "Community accessible datastore of high-throughput calculations: Experiences from the materials project", Proceedings - 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, SCC 2012, 2012, 1244--1251, doi: 10.1109/SC.Companion.2012.150

A Jain, G Hautier, SP Ong, C Moore, B Kang, H Chen, X Ma, JC Kim, M Kocher, D Gunter, S Cholia, A Greiner, DH Bailey, D Skinner, K Persson, G Ceder, "Materials Project: A public materials database and its application to lithium ion battery cathode design", ABSTRACTS OF PAPERS OF THE AMERICAN CHEMICAL SOCIETY, 2012, 243,

2011

MD Pierro, J Hetrick, S Cholia, D Skinner, "Making QCD lattice data accessible and organized through advanced web interfaces", Proceedings of Science, 2011, 139,

2010

Lavanya Ramakrishnan, Keith Jackson, Shane Canon, Shreyas Cholia, John Shalf, "Defining Future Platform Requirements for e-Science Cloud (Position paper)", ACM Symposium on Cloud Computing 2010 (ACM SOCC 2010), Indianapolis, Indiana, 2010,

S Cholia, D Skinner, J Boverhof, "NEWT: A RESTful service for building High: Performance Computing web applications", 2010 Gateway Computing Environments Workshop, GCE 2010, 2010, doi: 10.1109/GCE.2010.5676125

KR Jackson, L Ramakrishnan, K Muriki, S Canon, S Cholia, J Shalf, HJ Wasserman, NJ Wright, "Performance analysis of high performance computing applications on the Amazon Web Services cloud", Proceedings - 2nd IEEE International Conference on Cloud Computing Technology and Science, CloudCom 2010, 2010, 159--168, doi: 10.1109/CloudCom.2010.69

2009

M Pala, S Cholia, SA Rea, SW Smith, "Interoperable PKI data distribution in computational grids", International Journal of Grid and High Performance Computing, 2009, 1:56--73, doi: 10.4018/jghpc.2009040105

2008

S Cholia, RJ Porter, "Publication and protection of sensitive site information in a grid infrastructure", Proceedings CCGRID 2008 - 8th IEEE International Symposium on Cluster Computing and the Grid, 2008, 639--644, doi: 10.1109/CCGRID.2008.86

M Pala, S Cholia, SA Rea, SW Smith, "Extending PKI interoperability in computational grids", Proceedings CCGRID 2008 - 8th IEEE International Symposium on Cluster Computing and the Grid, 2008, 645--650, doi: 10.1109/CCGRID.2008.70

Danielle Svehla Christianson

2022

C Varadharajan, VC Hendrix, DS Christianson, M Burrus, C Wong, SS Hubbard, DA Agarwal, BASIN-3D: A brokering framework to integrate diverse environmental data, Computers and Geosciences, 2022, doi: 10.1016/j.cageo.2021.105024

C Varadharajan, AP Appling, B Arora, DS Christianson, VC Hendrix, V Kumar, AR Lima, J Müller, S Oliver, M Ombadi, T Perciano, JM Sadler, H Weierbach, JD Willard, Z Xu, J Zwart, "Can machine learning accelerate process understanding and decision-relevant predictions of river water quality?", Hydrological Processes, January 1, 2022, 36, doi: 10.1002/hyp.14565

H Weierbach, AR Lima, JD Willard, VC Hendrix, DS Christianson, M Lubich, C Varadharajan, Stream Temperature Predictions for River Basin Management in the Pacific Northwest and Mid-Atlantic Regions Using Machine Learning, Water (Switzerland), 2022, doi: 10.3390/w14071032

2021

C Varadharajan, Z Kakalia, E Alper, EL Brodie, M Burrus, RWH Carroll, D Christianson, W Dong, V Hendrix, M Henderson, S Hubbard, D Johnson, R Versteeg, KH Williams, DA Agarwal, The Colorado East River Community Observatory Data Collection, Hydrological Processes 35(6), 2021, doi: 10.22541/au.161962485.54378235/v1

D. A. Agarwal, J. Damerow, C. Varadharajan, D. S. Christianson, G. Z. Pastorello, Y.-W. Cheah, L. Ramakrishnan, "Balancing the needs of consumers and producers for scientific data collections", Ecological Informatics, 2021, 62:101251, doi: 10.1016/j.ecoinf.2021.101251

2020

G. Z. Pastorello, C. Trotta, E. Canfora, H. Chu, D. Christianson, Y.-W. Cheah, C. Poindexter, J. Chen, A. Elbashandy, M. Humphrey, P. Isaac, D. Polidori, M. Reichstein, A. Ribeca, C. van Ingen, N. Vuichard, L. Zhang, B. Amiro, C. Ammann, M. A. Arain, J. Ardö, T. Arkebauer, S. K. Arndt, N. Arriga, M. Aubinet, M. Aurela, D. Baldocchi, A. Barr, E. Beamesderfer, L. B. Marchesini, O. Bergeron, J. Beringer, C. Bernhofer, D. Berveiller, D. Billesbach, T. A. Black, P. D. Blanken, G. Bohrer, J. Boike, P. V. Bol stad, D. Bonal, J.-M. Bonnefond, D. R. Bowling, R. Bracho, J. Brodeur, C. Brümmer, N. Buchmann, B. Burban, S. P. Burns, P. Buysse, P. Cale, M. Cavagna, P. Cellier, S. Chen, I. Chini, T. R. Chris tensen, J. Cleverly, A. Collalti, C. Consalvo, B. D. Cook, D. Cook, C. Coursolle, E. Cremonese, P. S. Curtis, E. D’Andrea, H. da Rocha, X. Dai, K. J. Davis, B. D. Cinti, A. de Grandcourt, A. D. Ligne, R. C. D. Oliveira, N. Delpierre, A. R. Desai, C. M. D. Bella, P. di Tommasi, H. Dolman, F. Domingo, G. Dong, S. Dore, P. Duce, E. Dufrêne, A. Dunn, J. Dušek, D. Eamus, U. Eichelmann, H. A. M. ElKhidir, W. Eugster, C. M. Ewenz, B. Ewers, D. Famulari, S. Fares, I. Feigenwinter, A. Feitz, R. Fensholt, G. Fil ippa, M. Fischer, J. Frank, M. Galvagno, M. Gharun, D. Gianelle, B. Gielen, B. Gioli, A. Gitelson, I. Goded, M. Goeckede, A. H. Goldstein, C. M. Gough, M. L. Goulden, A. Graf, A. Griebel, C. Gruening, T. Grünwald, A. Hammerle, S. Han, X. Han, B. U. Hansen, C. Hanson, J. Hatakka, Y. He, M. Hehn, B. Heinesch, N. Hinko-Najera, L. Hörtnagl, L. Hutley, A. Ibrom, H. Ikawa, M. Jackowicz-Korczynski, D. Janouš, W. Jans, R. Jassal, S. Jiang, T. Kato, M. Khomik, J. Klatt, A. Knohl, S. Knox, H. Kobayashi, G. Koerber, O. Kolle, Y. Kosugi, A. Kotani, A. Kowalski, B. Kruijt, J. Kurbatova, W. L. Kutsch, H. Kwon, S. Launiainen, T. Laurila, B. Law, R. Leuning, Y. Li, M. Liddell, J.-M. Limousin, M. Lion, A. J. Liska, A. Lohila, A. López-Ballesteros, E. López-Blanco, B. Loubet, D. Loustau, A. Lucas-Moffat, J. Lüers, S. Ma, C. Macfarlane, V. Magliulo, R. Maier, I. Mammarella, G. Manca, B. Marcolla, H. A. Margolis, S. Mar ras, W. Massman, M. Mastepanov, R. Matamala, J. H. Matthes, F. Mazzenga, H. McCaughey, I. McHugh, A. M. S. McMillan, L. Merbold, W. Meyer, T. Meyers, S. D. Miller, S. Minerbi, U. Moderow, R. K. Monson, L. Montagnani, C. E. Moore, E. Moors, V. Moreaux, C. Moureaux, J. W. Munger, T. Nakai, J. Neirynck, Z. Nesic, G. Nicolini, A. Noormets, M. Northwood, M. Nosetto, Y. Nouvellon, K. Novick, W. Oechel, J. E. Olesen, J.-M. Ourcival, S. A. Papuga, F.-J. Parmentier, E. Paul-Limoges, M. Pavelka, M. Peichl, E. Pendall, R. P. Phillips, K. Pilegaard, N. Pirk, G. Posse, T. Powell, H. Prasse, S. M. Prober, S. Ram bal, U. Rannik, N. Raz-Yaseef, D. Reed, V. R. de Dios, N. Restrepo-Coupe, B. R. Reverter, M. Roland, S. Sabbatini, T. Sachs, S. R. Saleska, E. P. S.-C. nete, Z. M. Sanchez-Mejia, H. P. Schmid, M. Schmidt, K. Schneider, F. Schrader, I. Schroder, R. L. Scott, P. Sedlák, P. Serrano-Ortíz, C. Shao, P. Shi, I. Shironya, L. Siebicke, L. Šigut, R. Silberstein, C. Sirca, D. Spano, R. Steinbrecher, R. M. Stevens, C. Sturtevant, A. Suyker, T. Tagesson, S. Takanashi, Y. Tang, N. Tapper, J. Thom, F. Tiedemann, M. Tomassucci, J.-P. Tuovinen, S. Urbanski, R. Valentini, M. van der Molen, E. van Gorsel, K. van Huissteden, A. Varlagin, J. Verfaillie, T. Vesala, C. Vincke, D. Vitale, N. Vygodskaya, J. P. Walker, E. Walter-Shea, H. Wang, R. Weber, S. Westermann, C. Wille, S. Wofsy, G. Wohlfahrt, S. Wolf, W. Woodgate, Y. Li, R. Zampedri, J. Zhang, G. Zhou, D. Zona, D. Agarwal, S. Biraud, M. Torn, D. Papale, "The FLUXNET2015 dataset and the ONEFlux processing pipeline for eddy covariance data", Scientific Data, 2020, 7:225, doi: 10.1038/s41597-020-0534-3

2019

C Varadharajan, B Faybishenko, A Henderson, M Henderson, VC Hendrix, SS Hubbard, Z Kakalia, A Newman, B Potter, H Steltzer, R Versteeg, DA Agarwal, KH Williams, C Wilmer, Y Wu, W Brown, M Burrus, RWH Carroll, DS Christianson, B Dafflon, D Dwivedi, BJ Enquist, Challenges in Building an End-to-End System for Acquisition, Management, and Integration of Diverse Data from Sensor Networks in Watersheds: Lessons from a Mountainous Community Observatory in East River, Colorado, IEEE Access, Pages: 182796--18 2019, doi: 10.1109/ACCESS.2019.2957793

2017

Danielle S. Christianson, Charuleka Varadharajan, Bradley Christoffersen, Matteo Detto, Faybishenko, Bruno O. Gimenez, Val C. Hendrix, Kolby J. Jardine, Robinson Negron-Juarez, Z. Pastorello, Thomas L. Powell, Megha Sandesh, Jeffrey M. Warren, Brett T. Wolfe, Jeffrey Q. Chambers, Lara M. Kueppers, Nathan G. McDowell, Deborah A. Agarwal, "A metadata reporting framework (FRAMES) for synthesis of ecohydrological observations", Ecological Informatics, 2017, 42:148-158, doi: 10.1016/j.ecoinf.2017.06.002

Gilberto Z. Pastorello, Dan K. Gunter, Housen Chu, Danielle S. Christianson, Carlo Trotta, Eleonora Canfora, Boris Faybishenko, You-Wei Cheah, Norm Beekwilder, Stephen W. Chan, Sigrid Dengel, Trevor Keenan, Fianna O Brien, Abderahman Elbashandy, Cristina M. Poindexter, Marty Humphrey, Dario Papale, Deb A. Agarwal, "Hunting Data Rogues at Scale: Data Quality Control for Observational Data in Research Infrastructures", Proceedings of the 13th IEEE International Conference on e-Science (e-Science 2017), Auckland, New Zealand, 2017, doi: 10.1109/eScience.2017.64

2016

Danielle S Christianson, Cari G Kaufman, "Effects of sample design and landscape features on a measure of environmental heterogeneity", Methods in Ecology and Evolution, 2016, 7:770--782,

2011

ZM Subin, WJ Riley, J Jin, DS Christianson, MS Torn, LM Kueppers, "Ecosystem feedbacks to climate change in California: development, Testing, and analysis using a coupled regional atmosphere and land surface model (WRF3--CLM3. 5)", Earth Interactions, 2011, 15:1--38,

Bin Dong

2023

Bin Dong, Jean Luca Bez, Suren Byna, "AIIO: Using Artificial Intelligence for Job-Level and Automatic I/O Performance Bottleneck Diagnosis.", In Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing (HPDC ’23), June 16, 2023,

2022

Jonathan Ajo‐Franklin, Verónica Rodríguez Tribaldos, Avinash Nayak, Feng Cheng, Robert Mellors, Benxin Chi, Todd Wood, Michelle Robertson, Cody Rotermund, Eric Matzel, Dennise C. Templeton, Christina Morency, Kesheng Wu, Bin Dong, Patrick Dobson;, "The Imperial Valley Dark Fiber Project: Toward Seismic Studies Using DAS and Telecom Infrastructure for Geothermal Applications", Seismological Research Letters, June 24, 2022,

Bin Dong, Alex Popescu, Veronica Rodriguez Tribaldos, Suren Byna, Jonathan Ajo-Franklin, Kesheng Wu, "Real-time and post-hoc compression for data from Distributed Acoustic Sensing", Computers \& Geosciences, June 24, 2022, 105181,

Runzhou Han, Suren Byna, Houjun Tang, Bin Dong, and Mai Zheng,, "PROV-IO: An I/O-Centric Provenance Framework for Scientific Data on HPC Systems", HPDC 2022, June 23, 2022,

John Wu, Bin Dong, Alex Sim, Automating Data Management Through Unified Runtime Systems, DOE ASCR Workshop on the Management and Storage of Scientific Data, 2022, doi: 10.2172/1843500

Bin Dong, Kesheng Wu, Suren Byna, User-Defined Tensor Data Analysis, SpringerBrief, (January 1, 2022)

Screen Shot 2022 06 24 at 1.24.03 PM

2020

Jonathan Blair Ajo-Franklin, Ver\ onica Rodr\ \iguez Tribaldos, Avinash Nayak, Nathaniel J Lindsey, Feng Cheng, Benxin Chi, Bin Dong, Kesheng Wu, Inder Monga, Distributed Acoustic Sensing (DAS) at the Plot to Basin Scale: Connecting Near-Surface Sensing and Seismology with a Common Observational Tool, AGU Fall Meeting 2020, 2020,

Bin Dong, Ver\ onica Rodr\ \iguez Tribaldos, Xin Xing, Suren Byna, Jonathan Ajo-Franklin, Kesheng Wu, "DASSA: Parallel DAS Data Storage and Analysis for Subsurface Event Detection", 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS), July 14, 2020, 254--263,

Houjun Tang, Suren Byna, Bin Dong, Quincey Koziol, "Parallel Query Service for Object-centric Data Management Systems", 2020 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), IEEE, May 18, 2020, 406-415,

Suren Byna, M. Scot Breitenfeld, Bin Dong, Quincey Koziol, Elena Pourmal, Dana Robinson, Jerome Soumagne, Houjun Tang, Venkatram Vishwanath, and Richard Warren, "ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems", Journal of Computer Science and Technology 2020, 35(1): 145-160, February 2, 2020, doi: 10.1007/s11390-020-9822-9

2019

Richard Warren, Jerome Soumagne, Jingqing Mu, Houjun Tang, Suren Byna, Bin Dong, Quincey Koziol, "Analysis in the Data Path of an Object-centric Data Management System", 26th IEEE International Conference on High Performance Computing, Data, and Analytics (HiPC 2019), December 18, 2019,

Houjun Tang, Suren Byna, Stephen Bailey, Zarija Lukic, Jialin Liu, Quincey Koziol, Bin Dong, "Tuning Object-centric Data Management Systems for Large Scale Scientific Applications", 26th IEEE International Conference on High Performance Computing, Data, and Analytics (HiPC 2019), December 18, 2019,

Bin Dong, Kesheng Wu, Suren Byna, Houjun Tang, "SLOPE: Structural Locality-Aware Programming Model for Composing Array Data Analysis", International Conference on High Performance Computing, January 1, 2019, 61--80,

Bin Dong, Patrick Kilian, Xiaocan Li, Fan Guo, Suren Byna, Kesheng Wu, "Terabyte-scale Particle Data Analysis: An ArrayUDF Case Study", Proceedings of the 31st International Conference on Scientific and Statistical Database Management, January 1, 2019, 202--205,

2018

Suren Byna, Quincey Koziol, Venkatram Vishwanath, Jerome Soumagne, Houjun Tang, Kimmy Mu, Richard Warren, François Tessier, Bin Dong, Teng Wang, and Jialin Liu, Proactive Data Containers (PDC): An object-centric data store for large-scale computing systems, AGU Fall Meeting, December 13, 2018,

Teng Wang, Suren Byna, Bin Dong, and Houjun Tang, "UniviStor: Integrated Hierarchical and Distributed Storage for HPC", IEEE Cluster 2018., September 1, 2018,

Houjun Tang, Suren Byna, Francois Tessier, Teng Wang, Bin Dong, Jingqing Mu, Quincey Koziol, Jerome Soumagne, Venkatram Vishwanath, Jialin Liu, and Richard Warren, "Toward Scalable and Asynchronous Object-centric Data Management for HPC", 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) 2018, May 1, 2018,

Bin Dong, Teng Wang, Houjun Tang, Quincey Koziol, Kesheng Wu, Suren Byna, "ARCHIE: Data analysis acceleration with array caching in hierarchical storage", 2018 IEEE International Conference on Big Data (Big Data), January 1, 2018, 211--220,

Kesheng Wu, Surendra Byna, Bin Dong, others, VPIC IO utilities, 2018,

Kesheng Wu, Bin Dong, Surendra Byna, "Scientific Data Services Framework for Plasma Physics", APS, 2018, 2018:BM10--006,

Weijie Zhao, Florin Rusu, Bin Dong, Kesheng Wu, Anna YQ Ho, Peter Nugent, "Distributed caching for processing raw arrays", Proceedings of the 30th International Conference on Scientific and Statistical Database Management, 2018, 1--12,

Xin Xing, Bin Dong, Jonathan Ajo-Franklin, Kesheng Wu, "Automated Parallel Data Processing Engine with Application to Large-Scale Feature Extraction", 2018 IEEE/ACM Machine Learning in HPC Environments (MLHPC), January 1, 2018, 37--46,

2017

Houjun Tang, Suren Byna, Bin Dong, Jialin Liu, and Quincey Koziol, "SoMeta: Scalable Object-centric Metadata Management for High Performance Computing", IEEE Cluster 2017, September 5, 2017,

Bin Dong, Kesheng Wu, Surendra Byna, Jialin Liu, Weijie Zhao, Florin Rusu, "ArrayUDF: User-defined scientific data analysis on arrays", Proceedings of the 26th International Symposium on High-Performance Parallel and Distributed Computing, January 1, 2017, 53--64,

Weijie Zhao, Florin Rusu, Bin Dong, Kesheng Wu, Peter Nugent, "Incremental view maintenance over array data", Proceedings of the 2017 ACM International Conference on Management of Data, January 1, 2017, 139--154,

Tzuhsien Wu, Jerry Chou, Shyng Hao, Bin Dong, Scott Klasky, Kesheng Wu, "Optimizing the query performance of block index through data analysis and I/O modeling", Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, January 1, 2017, 1--10,

2016

Bin Dong, Surendra Byna, Kesheng Wu, "SDS-Sort: Scalable Dynamic Skew-aware Parallel", HPDC 16, New York, NY, USA, ACM, 2016, 57--68, doi: 10.1145/2907294.2907300

Houjun Tang, Suren Byna, Steve Harenberg, Xiaocheng Zou, Wenzhao Zhang, Kesheng Wu, Bin Dong, Oliver Rubel, Kristofer Bouchard, Scott Klasky, others, "Usage pattern-driven dynamic data layout reorganization", 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), January 1, 2016, 356--365,

Wenzhao Zhang, Houjun Tang, Steve Harenberg, Surendra Byna, Xiaocheng Zou, Dharshi Devendran, Daniel F Martin, Kesheng Wu, Bin Dong, Scott Klasky, others, "Amrzone: A runtime amr data sharing framework for scientific applications", 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), January 1, 2016, 116--125,

Bin Dong, Surendra Byna, Kesheng Wu, "Sds-sort: Scalable dynamic skew-aware parallel sorting", Proceedings of the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing, January 1, 2016, 57--68,

Xiaocheng Zou, David A Boyuka II, Dhara Desai, Daniel F Martin, Suren Byna, Kesheng Wu, "AMR-aware in situ indexing and scalable querying", Proceedings of the 24th High Performance Computing Symposium, January 1, 2016, 26,

Bin Dong, Suren Byna, Kesheng Wu, Hans Johansen, Jeffrey N Johnson, Noel Keen, others, "Data elevator: Low-contention data movement in hierarchical storage system", 2016 IEEE 23rd international conference on high performance computing (HiPC), January 1, 2016, 152--161,

Houjun Tang, Suren Byna, Steve Harenberg, Wenzhao Zhang, Xiaocheng Zou, Daniel F Martin, Bin Dong, Dharshi Devendran, Kesheng Wu, David Trebotich, others, "In situ storage layout optimization for amr spatio-temporal read accesses", 2016 45th International Conference on Parallel Processing (ICPP), January 1, 2016, 406--415,

Wenzhao Zhang, Houjun Tang, Stephen Ranshous, Surendra Byna, Daniel F Mart\ \in, Kesheng Wu, Bin Dong, Scott Klasky, Nagiza F Samatova, "Exploring memory hierarchy and network topology for runtime AMR data sharing across scientific applications", 2016 IEEE International Conference on Big Data (Big Data), January 1, 2016, 1359--1366,

Tzuhsien Wu, Hao Shyng, Jerry Chou, Bin Dong, Kesheng Wu, "Indexing blocks to reduce space and time requirements for searching large data files", 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), January 1, 2016, 398--402,

Weijie Zhao, Florin Rusu, Bin Dong, Kesheng Wu, "Similarity Join over Array Data", SIGMOD, January 1, 2016, 2007--2022,

2015

Jinoh Kim, Bin Dong, Suren Byna, and Kesheng Wu, "Security for the Scientific Data Service Framework", 2nd International Workshop on Privacy and Security of Big Data (PSBD 2015), in conjunction with IEEE BigData 2015, 2015,

Bin Dong, Surendra Byna, Kesheng Wu, "Heavy-tailed distribution of parallel I/O system response time", Proceedings of the 10th Parallel Data Storage Workshop, 2015, 37--42,

Bin Dong, Surendra Byna, Kesheng Wu, "Spatially clustered join on heterogeneous scientific data sets", 2015 IEEE International Conference on Big Data (Big Data), 2015, 371--380,

2014

Bin Dong, Xiuqiao Li, Limin Xiao, Li Ruan, "Towards minimizing disk I/O contention: A partitioned file assignment approach", Future Generation Computer Systems, Volume 37, July 2014, Pages 178-190, 2014,

Spyros Blanas, Kesheng Wu, Surendra Byna, Bin Dong, Arie Shoshani, "Parallel Data Analysis Directly on Scientific File", SIGMOD 14, 2014, 385--396, doi: 10.1145/2588555.2612185

Spyros Blanas, Kesheng Wu, Surendra Byna, Bin Dong, Arie Shoshani, "Parallel data analysis directly on scientific file formats", Proceedings of the 2014 ACM SIGMOD international conference on Management of data, January 1, 2014, 385--396,

Bin Dong, Surendra Byna, Kesheng Wu, "Parallel query evaluation as a Scientific Data Service", 2014 IEEE International Conference on Cluster Computing (CLUSTER), January 1, 2014, 194--202,

Jialin Liu, S. Byna, Bin Dong, Kesheng Wu, Chen, "Model-Driven Data Layout Selection for Improving Read", Parallel Distributed Processing Symposium Workshops 2014 IEEE International, 2014, 1708--1716, doi: 10.1109/IPDPSW.2014.190

Bin Dong, S. Byna, Kesheng Wu, Parallel query evaluation as a Scientific Data, Cluster Computing (CLUSTER), 2014 IEEE International on, Pages: 194--202 2014, doi: 10.1109/CLUSTER.2014.6968765

2013

Bin Dong, Surendra Byna, Kesheng Wu, "SDS: a framework for scientific data services", Proceedings of the 8th Parallel Data Storage Workshop, January 1, 2013, 27--32,

Bin Dong, Surendra Byna, Kesheng Wu, "Expediting scientific data analysis with reorganization of data", 2013 IEEE International Conference on Cluster Computing (CLUSTER), January 1, 2013, 1--8,

Bin Dong, S. Byna, Kesheng Wu, Expediting scientific data analysis with of data, Cluster Computing (CLUSTER), 2013 IEEE International on, Pages: 1--8 2013, doi: 10.1109/CLUSTER.2013.6702675

2012

Bin Dong, Xiuqiao Li, Qimeng Wu, Limin Xiao, Li Ruan, "A dynamic and adaptive load balancing strategy for parallel file system with large-scale I/O servers", Journal of Parallel and Distributed Computing (JPDC), Volume 72, Issue 10, October 2012, Pages 1254-1268, 2012,

Bin Dong, Xiuqiao Li, Limin Xiao, Li Ruan, "A New File-Specific Stripe Size Selection Method for Highly Concurrent Data Access", The 13th ACM/IEEE International Conference on Grid Computing (Grid 2012), 2012, 2012,

Abdelrahman Elbashandy

2021

Devarshi Ghoshal, Drew Paine, Gilberto Pastorello, Abdelrahman Elbashandy, Dan Gunter, Oluwamayowa Amusat, Lavanya Ramakrishnan, "Experiences with Reproducibility: Case Studies from Scientific Workflows", (P-RECS'21) Proceedings of the 4th International Workshop on Practical Reproducible Evaluation of Computer Systems, ACM, June 21, 2021, doi: 10.1145/3456287.3465478

Reproducible research is becoming essential for science to ensure transparency and for building trust. Additionally, reproducibility provides the cornerstone for sharing of methodology that can improve efficiency. Although several tools and studies focus on computational reproducibility, we need a better understanding about the gaps, issues, and challenges for enabling reproducibility of scientific results beyond the computational stages of a scientific pipeline. In this paper, we present five different case studies that highlight the reproducibility needs and challenges under various system and environmental conditions. Through the case studies, we present our experiences in reproducing different types of data and methods that exist in an experimental or analysis pipeline. We examine the human aspects of reproducibility while highlighting the things that worked, that did not work, and that could have worked better for each of the cases. Our experiences capture a wide range of scenarios and are applicable to a much broader audience who aim to integrate reproducibility in their everyday pipelines.

2020

G. Z. Pastorello, C. Trotta, E. Canfora, H. Chu, D. Christianson, Y.-W. Cheah, C. Poindexter, J. Chen, A. Elbashandy, M. Humphrey, P. Isaac, D. Polidori, M. Reichstein, A. Ribeca, C. van Ingen, N. Vuichard, L. Zhang, B. Amiro, C. Ammann, M. A. Arain, J. Ardö, T. Arkebauer, S. K. Arndt, N. Arriga, M. Aubinet, M. Aurela, D. Baldocchi, A. Barr, E. Beamesderfer, L. B. Marchesini, O. Bergeron, J. Beringer, C. Bernhofer, D. Berveiller, D. Billesbach, T. A. Black, P. D. Blanken, G. Bohrer, J. Boike, P. V. Bol stad, D. Bonal, J.-M. Bonnefond, D. R. Bowling, R. Bracho, J. Brodeur, C. Brümmer, N. Buchmann, B. Burban, S. P. Burns, P. Buysse, P. Cale, M. Cavagna, P. Cellier, S. Chen, I. Chini, T. R. Chris tensen, J. Cleverly, A. Collalti, C. Consalvo, B. D. Cook, D. Cook, C. Coursolle, E. Cremonese, P. S. Curtis, E. D’Andrea, H. da Rocha, X. Dai, K. J. Davis, B. D. Cinti, A. de Grandcourt, A. D. Ligne, R. C. D. Oliveira, N. Delpierre, A. R. Desai, C. M. D. Bella, P. di Tommasi, H. Dolman, F. Domingo, G. Dong, S. Dore, P. Duce, E. Dufrêne, A. Dunn, J. Dušek, D. Eamus, U. Eichelmann, H. A. M. ElKhidir, W. Eugster, C. M. Ewenz, B. Ewers, D. Famulari, S. Fares, I. Feigenwinter, A. Feitz, R. Fensholt, G. Fil ippa, M. Fischer, J. Frank, M. Galvagno, M. Gharun, D. Gianelle, B. Gielen, B. Gioli, A. Gitelson, I. Goded, M. Goeckede, A. H. Goldstein, C. M. Gough, M. L. Goulden, A. Graf, A. Griebel, C. Gruening, T. Grünwald, A. Hammerle, S. Han, X. Han, B. U. Hansen, C. Hanson, J. Hatakka, Y. He, M. Hehn, B. Heinesch, N. Hinko-Najera, L. Hörtnagl, L. Hutley, A. Ibrom, H. Ikawa, M. Jackowicz-Korczynski, D. Janouš, W. Jans, R. Jassal, S. Jiang, T. Kato, M. Khomik, J. Klatt, A. Knohl, S. Knox, H. Kobayashi, G. Koerber, O. Kolle, Y. Kosugi, A. Kotani, A. Kowalski, B. Kruijt, J. Kurbatova, W. L. Kutsch, H. Kwon, S. Launiainen, T. Laurila, B. Law, R. Leuning, Y. Li, M. Liddell, J.-M. Limousin, M. Lion, A. J. Liska, A. Lohila, A. López-Ballesteros, E. López-Blanco, B. Loubet, D. Loustau, A. Lucas-Moffat, J. Lüers, S. Ma, C. Macfarlane, V. Magliulo, R. Maier, I. Mammarella, G. Manca, B. Marcolla, H. A. Margolis, S. Mar ras, W. Massman, M. Mastepanov, R. Matamala, J. H. Matthes, F. Mazzenga, H. McCaughey, I. McHugh, A. M. S. McMillan, L. Merbold, W. Meyer, T. Meyers, S. D. Miller, S. Minerbi, U. Moderow, R. K. Monson, L. Montagnani, C. E. Moore, E. Moors, V. Moreaux, C. Moureaux, J. W. Munger, T. Nakai, J. Neirynck, Z. Nesic, G. Nicolini, A. Noormets, M. Northwood, M. Nosetto, Y. Nouvellon, K. Novick, W. Oechel, J. E. Olesen, J.-M. Ourcival, S. A. Papuga, F.-J. Parmentier, E. Paul-Limoges, M. Pavelka, M. Peichl, E. Pendall, R. P. Phillips, K. Pilegaard, N. Pirk, G. Posse, T. Powell, H. Prasse, S. M. Prober, S. Ram bal, U. Rannik, N. Raz-Yaseef, D. Reed, V. R. de Dios, N. Restrepo-Coupe, B. R. Reverter, M. Roland, S. Sabbatini, T. Sachs, S. R. Saleska, E. P. S.-C. nete, Z. M. Sanchez-Mejia, H. P. Schmid, M. Schmidt, K. Schneider, F. Schrader, I. Schroder, R. L. Scott, P. Sedlák, P. Serrano-Ortíz, C. Shao, P. Shi, I. Shironya, L. Siebicke, L. Šigut, R. Silberstein, C. Sirca, D. Spano, R. Steinbrecher, R. M. Stevens, C. Sturtevant, A. Suyker, T. Tagesson, S. Takanashi, Y. Tang, N. Tapper, J. Thom, F. Tiedemann, M. Tomassucci, J.-P. Tuovinen, S. Urbanski, R. Valentini, M. van der Molen, E. van Gorsel, K. van Huissteden, A. Varlagin, J. Verfaillie, T. Vesala, C. Vincke, D. Vitale, N. Vygodskaya, J. P. Walker, E. Walter-Shea, H. Wang, R. Weber, S. Westermann, C. Wille, S. Wofsy, G. Wohlfahrt, S. Wolf, W. Woodgate, Y. Li, R. Zampedri, J. Zhang, G. Zhou, D. Zona, D. Agarwal, S. Biraud, M. Torn, D. Papale, "The FLUXNET2015 dataset and the ONEFlux processing pipeline for eddy covariance data", Scientific Data, 2020, 7:225, doi: 10.1038/s41597-020-0534-3

2017

YW Cheah, J Boverhof, A Elbashandy, D Agarwal, J Leek, T Epperly, J Eslick, D Miller, "Data management and simulation support accelerating carbon capture through computing", Proceedings of the 2016 IEEE 12th International Conference on e-Science, e-Science 2016, 2017, 389--398, doi: 10.1109/eScience.2016.7870924

Gilberto Z. Pastorello, Dan K. Gunter, Housen Chu, Danielle S. Christianson, Carlo Trotta, Eleonora Canfora, Boris Faybishenko, You-Wei Cheah, Norm Beekwilder, Stephen W. Chan, Sigrid Dengel, Trevor Keenan, Fianna O Brien, Abderahman Elbashandy, Cristina M. Poindexter, Marty Humphrey, Dario Papale, Deb A. Agarwal, "Hunting Data Rogues at Scale: Data Quality Control for Observational Data in Research Infrastructures", Proceedings of the 13th IEEE International Conference on e-Science (e-Science 2017), Auckland, New Zealand, 2017, doi: 10.1109/eScience.2017.64

S Peisert, R Gentz, J Boverhof, C McParland, S Engle, A Elbashandy, D Gunter, "LBNL Open Power Data", January 2017, doi: 10.21990/C21599

Hamdy Elgammal

2017

GH Weber, MS Bandstra, DH Chivers, HH Elgammal, V Hendrix, J Kua, JS Maltz, K Muriki, Y Ong, K Song, MJ Quinlan, L Ramakrishnan, BJ Quiter, "Web-based visual data exploration for improved radiological source detection", Concurrency Computation, 2017, 29, doi: 10.1002/cpe.4203

Reinhard Gentz

2020

Chris Lawson, Jose Manuel Martí, Tijana Radivojevic, Sai Vamshi R. Jonnalagadda, Reinhard Gentz, Nathan J. Hillson, Sean Peisert, Joonhoon Kim, Blake A. Simons, Christopher J. Petzold, Steven W. Singer, Aindrila Mukhopadhyay, Deepti Tanjore, Josh Dunn, Héctor García Martín,, "Machine Learning for Metabolic Engineering: A Review", Metabolic Engineering, November 19, 2020,

2019

Reinhard Gentz, Sean Peisert, "An Examination and Survey of Random Bit Flips and Scientific Computing", Trusted CI Report, December 20, 2019,

Mahdi Jamei, Raksha Ramakrishna, Teklemariam Tesfay, Reinhard Gentz, Ciaran Roberts, Anna Scaglione, Sean Peisert, "Phasor Measurement Units Optimal Placement and Performance Limits for Fault Localization", IEEE Journal on Selected Areas in Communications (J-SAC), Special Issue on Communications and Data Analytics in Smart Grid, November 6, 2019, 38(1):180-192, doi: 10.1109/jsac.2019.2951971

Thomas W. Edgar, Aditya Ashok, Garret E. Seppala, K.M. Arthur-Durrett, M. Engels, Reinhard Gentz, Sean Peisert, "An Automated Disruption-Tolerant Key Management Framework for Critical Systems", Journal of Information Warfare, October 8, 2019, 18(4):85-124, doi: https://www.jinfowar.com/journal/volume-18-issue-4/automated-disruption-tolerant-device-authentication-key-management-framework-critical-systems

Reinhard Gentz, Sean Peisert, Joshua Boverhof, Daniel Gunter, "SPARCS: Stream-Processing Architecture applied in Real-time Cyber-physical Security", Proceedings of the 15th IEEE International Conference on e-Science (eScience), San Diego, CA, IEEE, September 2019, doi: 10.1109/eScience.2019.00028

Reinhard Gentz, Héctor García Martin, Edward Baidoo, Sean Peisert, "Workflow Automation in Liquid Chromatography Mass Spectrometry", Proceedings of the 15th IEEE International Conference on e-Science (eScience), San Diego, CA, IEEE, September 2019, doi: 10.1109/eScience.2019.00095

Melissa Stockman, Dipankar Dwivedi, Reinhard Gentz, Sean Peisert, "Detecting Programmable Logic Controller Code Using Machine Learning", International Journal of Critical Infrastructure Protection, September 2019, vol. 26,, doi: 10.1016/j.ijcip.2019.100306

Ciaran Roberts, Anna Scaglione, Mahdi Jamei, Reinhard Gentz, Sean Peisert, Emma M. Stewart, Chuck McParland, Alex McEachern, Daniel Arnold, "Learning Behavior of Distribution System Discrete Control Devices for Cyber-Physical Security", IEEE Transaction on Smart Grid, August 1, 2019, 11(1):749-761, doi: 0.1109/TSG.2019.2936016

2018

Sean Peisert, Ciaran Roberts, Anna Scaglione, Mahdi Jamei, Reinhard Gentz, Charles McParland, Alex McEachren, Galen Rasche, Aaron Snyder, "Supporting Cyber Security of Power Distribution Systems by Detecting Differences Between Real-time Micro-Synchrophasor Measurements and Cyber-Reported SCADA - Final Report", October 15, 2018,

2017

S Peisert, R Gentz, J Boverhof, C McParland, S Engle, A Elbashandy, D Gunter, "LBNL Open Power Data", January 2017, doi: 10.21990/C21599

L. Ferrari, A. Scaglione, R. Gentz, Y. W. P. Hong, "Convergence Results on Pulse Coupled Oscillator Protocols in Locally Connected Networks", IEEE/ACM Transactions on Networking, 2017, 25:1004-1019, doi: 10.1109/TNET.2016.2611379

2016

R. Gentz, A. Scaglione, L. Ferrari, Y. W. P. Hong, "PulseSS: A Pulse-Coupled Synchronization and Scheduling Protocol for Clustered Wireless Sensor Networks", IEEE Internet of Things Journal, 2016, 3:1222-1234, doi: 10.1109/JIOT.2016.2576923

R. Gentz, S. X. Wu, H. T. Wai, A. Scaglione, A. Leshem, "Data Injection Attacks in Randomized Gossiping", IEEE Transactions on Signal and Information Processing over Networks, 2016, 2:523-538, doi: 10.1109/TSIPN.2016.2614898

2015

Georgia Koutsandria, Reinhard Gentz, Mahdi Jamei, Anna Scaglione, Sean Peisert, Chuck McParland, "A real-time testbed environment for cyber-physical security on the power grid", Proceedings of the First ACM Workshop on Cyber-Physical Systems-Security and/or PrivaCy, January 1, 2015, 67--78, doi: 10.1145/2808705.2808707

Reinhard Gentz, Anna Scaglione, Yao-Win Hong, Lorenzo Ferrari, "PulseSS: A Microcontroller Implementation of Pulse-Coupled Scheduling and Synchronization Protocol for Cluster-BasedWireless Sensor Networks", 2015,

R. Gentz, H. T. Wai, A. Scaglione, A. Leshem, "Detection of data injection attacks in decentralized learning", 2015 49th Asilomar Conference on Signals, Systems and Computers, 2015, 350-354, doi: 10.1109/ACSSC.2015.7421145

2014

Lorenzo Ferrari, Reinhard Gentz, Anna Scaglione, Masood Parvania, "The Pulse Coupled Phasor Measurement Units", IEEE Smartgridcomm, 2014,

Devarshi Ghoshal

2021

Devarshi Ghoshal, Ludovico Bianchi, Abdelilah Essiari, Drew Paine, Sarah Poon, Michael Beach, Alpha N'Diaye, Patrick Huck, Lavanya Ramakrishnan, "Science Capsule: Towards Sharing and Reproducibility of Scientific Workflows", 2021 IEEE Workshop on Workflows in Support of Large-Scale Science (WORKS), November 15, 2021, doi: 10.1109/WORKS54523.2021.00014

Workflows are increasingly processing large volumes of data from scientific instruments, experiments and sensors. These workflows often consist of complex data processing and analysis steps that might include a diverse ecosystem of tools and also often involve human-in-the-loop steps. Sharing and reproducing these workflows with collaborators and the larger community is critical but hard to do without the entire context of the workflow including user notes and execution environment. In this paper, we describe Science Capsule, which is a framework to capture, share, and reproduce scientific workflows. Science Capsule captures, manages and represents both computational and human elements of a workflow. It automatically captures and processes events associated with the execution and data life cycle of workflows, and lets users add other types and forms of scientific artifacts. Science Capsule also allows users to create `workflow snapshots' that keep track of the different versions of a workflow and their lineage, allowing scientists to incrementally share and extend workflows between users. Our results show that Science Capsule is capable of processing and organizing events in near real-time for high-throughput experimental and data analysis workflows without incurring any significant performance overheads.

Devarshi Ghoshal, Drew Paine, Gilberto Pastorello, Abdelrahman Elbashandy, Dan Gunter, Oluwamayowa Amusat, Lavanya Ramakrishnan, "Experiences with Reproducibility: Case Studies from Scientific Workflows", (P-RECS'21) Proceedings of the 4th International Workshop on Practical Reproducible Evaluation of Computer Systems, ACM, June 21, 2021, doi: 10.1145/3456287.3465478

Reproducible research is becoming essential for science to ensure transparency and for building trust. Additionally, reproducibility provides the cornerstone for sharing of methodology that can improve efficiency. Although several tools and studies focus on computational reproducibility, we need a better understanding about the gaps, issues, and challenges for enabling reproducibility of scientific results beyond the computational stages of a scientific pipeline. In this paper, we present five different case studies that highlight the reproducibility needs and challenges under various system and environmental conditions. Through the case studies, we present our experiences in reproducing different types of data and methods that exist in an experimental or analysis pipeline. We examine the human aspects of reproducibility while highlighting the things that worked, that did not work, and that could have worked better for each of the cases. Our experiences capture a wide range of scenarios and are applicable to a much broader audience who aim to integrate reproducibility in their everyday pipelines.

Devarshi Ghoshal, Ludovico Bianchi, Abdelilah Essiari, Michael Beach, Drew Paine, Lavanya Ramakrishnan, "Science Capsule - Capturing the Data Life Cycle", Journal of Open Source Software, 2021, 6:2484, doi: 10.21105/joss.02484

2020

Drew Paine, Devarshi Ghoshal, Lavanya Ramakrishnan, "Experiences with a Flexible User Research Process to Build Data Change Tools", Journal of Open Research Software, September 1, 2020, doi: 10.5334/jors.284

Scientific software development processes are understood to be distinct from commercial software development practices due to uncertain and evolving states of scientific knowledge. Sustaining these software products is a recognized challenge, but under-examined is the usability and usefulness of such tools to their scientific end users. User research is a well-established set of techniques (e.g., interviews, mockups, usability tests) applied in commercial software projects to develop foundational, generative, and evaluative insights about products and the people who use them. Currently these approaches are not commonly applied and discussed in scientific software development work. The use of user research techniques in scientific environments can be challenging due to the nascent, fluid problem spaces of scientific work, varying scope of projects and their user communities, and funding/economic constraints on projects.

In this paper, we reflect on our experiences undertaking a multi-method user research process in the Deduce project. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists make decisions around data change. There is a lack of common terminology since the concept of systematically measuring and managing data change is under explored in scientific environments. To bridge this gap we conducted user research that focuses on user practices, needs, and motivations to help us design and develop metrics and tools for data change. This paper contributes reflections and the lessons we have learned from our experiences. We offer key takeaways for scientific software project teams to effectively and flexibly incorporate similar processes into their projects.

Drew Paine, Devarshi Ghoshal, Lavanya Ramakrishnan, "Investigating Scientific Data Change with User Research Methods", August 20, 2020, LBNL LBNL-2001347,

Scientific datasets are continually expanding and changing due to fluctuations with instruments, quality assessment and quality control processes, and modifications to software pipelines. Datasets include minimal information about these changes or their effects requiring scientists manually assess modifications through a number of labor intensive and ad-hoc steps. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists systematically identify and make decisions around data changes. Currently, there is a lack of understanding, and common practices, for identifying and evaluating changes in datasets since systematically measuring and managing data change is under explored in scientific work. We are conducting user research to address this need by exploring scientist's conceptualizations, behaviors, needs, and motivations when dealing with changing datasets. Our user research utilizes multiple methods to produce foundational, generative insights and evaluate research products produced by our team. In this paper, we detail our user research process and outline our findings about data change that emerge from our studies. Our work illustrates how scientific software teams can push beyond just usability testing user interfaces or tools to better probe the underlying ideas they are developing solutions to address.

2019

P. Linton, W. Melodia, A. Lazar, D. Agarwal, L. Bianchi, D. Ghoshal, K. Wu, G. Pastorello, L. Ramakrishnan, "Identifying Time Series Similarity in Large-Scale Earth System Datasets", The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC19), 2019,

Payton A Linton, William M Melodia, Alina Lazar, Deborah Agarwal, Ludovico Bianchi, Devarshi Ghoshal, Kesheng Wu, Gilberto Pastorello, Lavanya Ramakrishnan, "Identifying Time Series Similarity in Large-Scale Earth System Datasets", 2019,

2018

Cheah You-Wei, Drew Paine, Devarshi Ghoshal, Lavanya Ramakrishnan, Bringing Data Science to Qualitative Analysis, 2018 IEEE 14th International Conference on e-Science, Pages: 325-326 2018, doi: 10.1109/eScience.2018.00076

Devarshi Ghoshal, "Deduce: Managing Data Change Pipelines", Conference on Data Analysis (CoDA 2018), 2018,

D Ghoshal, L Ramakrishnan, D Agarwal, "Dac-Man: Data Change Management for Scientific Datasets on HPC Systems", SC ’18, Piscataway, NJ, USA, IEEE Press, 2018, 72:1--72:1,

S Swaid, M Maat, H Krishnan, D Ghoshal, L Ramakrishnan, "Usability heuristic evaluation of scientific data analysis and visualization tools", Advances in Intelligent Systems and Computing, 2018, 607:471--482, doi: 10.1007/978-3-319-60492-3_45

2017

Devarshi Ghoshal, Lavanya Ramakrishnan, "MaDaTS: Managing Data on Tiered Storage for Scientific Workflows", Proceedings of the 26th International Symposium on High-Performance Parallel and Distributed Computing (HPDC '17), ACM, 2017, 41--52, doi: 10.1145/3078597.3078611

D Ghoshal, V Hendrix, W Fox, S Balasubhramanian, L Ramakrishnan, "FRIEDA: Flexible Robust Intelligent Elastic Data Management Framework", The Journal of Open Source Software, 2017, 2:164--164, doi: 10.21105/joss.00164

W Fox, D Ghoshal, A Souza, GP Rodrigo, L Ramakrishnan, "E-HPC: A library for elastic resource management in HPC environments", Proceedings of WORKS 2017: 12th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2017: The International Conference for High Performance Computing, Networking, Storage and Analysis, 2017, doi: 10.1145/3150994.3150996

2016

V Hendrix, J Fox, D Ghoshal, L Ramakrishnan, "Tigres Workflow Library: Supporting Scientific Pipelines on HPC Systems", Proceedings - 2016 16th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2016, 2016, 146--155, doi: 10.1109/CCGrid.2016.54

CS Daley, D Ghoshal, GK Lockwood, S Dosanjh, L Ramakrishnan, NJ Wright, "Performance characterization of scientific workflows for the optimal use of Burst Buffers", CEUR Workshop Proceedings, 2016, 1800:69--73,

2014

L Ramakrishnan, D Ghoshal, V Hendrix, E Feller, P Mantha, C Morin, "Storage and Data Life Cycle Management in Cloud Environments with FRIEDA.", Cloud Computing for Data-Intensive Applications, (Springer: 2014) Pages: 357--378

2013

Devarshi Ghoshal, Lavanya Ramakrishnan, "FRIEDA: Flexible Robust Intelligent Elastic Data Management in Cloud Environments", 2012 SC Companion: High Performance Computing, Networking, Storage and Analysis (SCC), IEEE, 2013, 1096--1105, doi: 10.1109/SC.Companion.2012.132

Lavanya Ramakrishnan, Adam Scovel, Iwona Sakrejda, Susan Coghlan, Shane Canon, Anping Liu, Devarshi Ghoshal, Krishna Muriki, Nicholas J. Wright, "Magellan - A Testbed to Explore Cloud Computing for Science", On the Road to Exascale Computing: Contemporary Architectures in High Performance Computing, (Chapman & Hall/CRC Press: 2013)

2011

Devarshi Ghoshal, Richard Shane Canon, Lavanya Ramakrishnan, "I/O Performance of Virtualized Cloud Environments", Proceedings of the Second International Workshop on Data Intensive Computing in the Clouds (DataCloud-SC '11), ACM, 2011, 71--80, doi: 10.1145/2087522.2087535

Anna Giannakou

2021

Ayaz Akram, Anna Giannakou, Venkatesh Akella, Jason Lowe-Power, Sean Peisert, "Performance Analysis of Scientific Computing Workloads on General Purpose TEEs", Proceedings of the 35th IEEE International Parallel & Distributed Processing Symposium (IPDPS), IEEE, May 2021, doi: 10.1109/IPDPS49936.2021.00115

2020

Ayaz Akram, Anna Giannakou, Venkatesh Akella, Jason Lowe-Power, Sean Peisert, "Performance Analysis of Scientific Computing Workloads on Trusted Execution Environments", arXiv preprint arXiv:2010.13216, October 25, 2020,

Anna Giannakou, Dipankar Dwivedi, Sean Peisert, "A Machine Learning Approach for Packet Loss Prediction in ScienceFlows", Future Generation Computer Systems, January 2020, 102:190-197, doi: 10.1016/j.future.2019.07.053

2018

Anna Giannakou, Daniel Gunter, Sean Peisert, "Flowzilla: A Methodology for Detecting Data Transfer Anomalies in Research Networks", Workshop on Innovating the Network for Data-Intensive Science (INDIS), November 11, 2018, doi: 10.1109/INDIS.2018.00004

Monte Goode

2012

Taghrid Samak, Dan Gunter, Monte Goode, Ewa Deelman, Fabio Silva, Karan Vahi, "Failure Analysis of Distributed Scientific Workflows Executing in the Cloud", 8th International Conference on Network and Service Management (CNSM 2012), 2012,

2011

Taghrid Samak, Dan Gunter, Monte Goode, Ewa Deelman, Gaurang Mehta, Fabio Silva, Karan Vahi, "Failure Prediction and Localization in Large Scientific Workflows", The Sixth Workshop on Workflows in Support of Large-Scale Science (WORKS11), 2011,

Dan Gunter, Taghrid Samak, Ewa Deelman, Christopher H. Brooks, Monte Goode, Gideon Juve, Gaurang Mehta, Priscilla Moraes, Fabio Silva, Martin Swany, Karan Vahi, "Online Workflow Management and Performance Analysis with STAMPEDE", 7th International Conference on Network and Service Management (CNSM 2011), Paris, France, 2011,

2010

D. Agarwal, M. Humphrey,N., Beekwilder, K. Jackson, M. Goode, and C. van Ingen, "A Data Centered Collaboration Portal to Support Global Carbon-Flux Analysis", Concurrency and Computation: Practice and Experience - Successes in Furthering Scientific Discovery, June 2010, LBNL 1827E,

DA Agarwal, M Humphrey, NF Beekwilder, KR Jackson, MM Goode, C Van Ingen, "A data-centered collaboration portal to support global carbon-flux analysis", Concurrency Computation Practice and Experience, 2010, 22:2323--2334, doi: 10.1002/cpe.1600

2008

M Humphrey, N Beekwilder, D Agarwal, D Baldocchi, M Goode, C Ingen, D Papale, M Reichstein, M Rodriguez, R Vargas, N Li, J Gupchup, Y Ryu, Creating and Accessing the Global Fluxnet Data Set, 2008,

Junmin Gu

2022

Lipeng Wan, Axel Huebl, Junmin Gu, Franz Poeschel, Ana Gainaru, Ruonan Wang, Jieyang Chen, Xin Liang, Dmitry Ganyushin, Todd Munson, Ian Foster, Jean-Luc Vay, Norbert Podhorszki, Kesheng Wu, Scott Klasky, "Improving I/O Performance for Exascale Applications Through Online Data Layout Reorganization", IEEE Transactions on Parallel and Distributed Systems, 2022, 33:878-890, doi: 10.1109/TPDS.2021.3100784

E. Wes Bethel, Burlen Loring, Utkarsh Ayachit, P. N. Duque, Nicola Ferrier, Joseph Insley, Junmin Gu, Kress, Patrick O’Leary, Dave Pugmire, Silvio Rizzi, Thompson, Will Usher, Gunther H. Weber, Brad Whitlock, Wolf, Kesheng Wu, "Proximity Portability and In Transit, M-to-N Data Partitioning and Movement in SENSEI", In Situ Visualization for Computational Science, ( 2022) doi: 10.1007/978-3-030-81627-8_20

E. Wes Bethel, Burlen Loring, Utkarsh Ayatchit, David Camp, P. N. Duque, Nicola Ferrier, Joseph Insley, Junmin Gu, Kress, Patrick O’Leary, David Pugmire, Silvio Rizzi, Thompson, Gunther H. Weber, Brad Whitlock, Matthew Wolf, Kesheng Wu, "The SENSEI Generic In Situ Interface: Tool and Processing Portability at Scale", In Situ Visualization for Computational Science, ( 2022) doi: 10.1007/978-3-030-81627-8_13

2021

Franz Poeschel, Juncheng E, William F. Godoy, Norbert Podhorszki, Scott Klasky, Greg Eisenhauer, Philip E. Davis, Lipeng Wan, Ana Gainaru, Junmin Gu, Fabian Koller, René Widera, Michael Bussmann, Axel Huebl, "Transitioning from file-based HPC workflows to streaming data pipelines with openPMD and ADIOS2", Smoky Mountains Computational Sciences and Engineering Conference (SMC2021), 2021,

2020

William F.Godoy, Norbert Podhorszki, Ruonan Wang, Chuck Atkins, Greg Eisenhauer, Junmin Gu,Philip Davis,J ong Choi, Kai Germaschewski, Kevin Huck, Axel Huebl, Mark Kim, James Kress, Tahsin Kurc, Qing Liu, Jeremy Logan, Kshitij Mehta, George Ostrouchov, Manish Parashar, Franz Poeschel, David Pugmire, Eric Suchyta, KeichiTakahashi, NickThompson, Seiji Tsutsumi, Lipeng Wan, Matthew Wolf, Kesheng Wu, Scott Klasky, "ADIOS 2: The Adaptable Input Output System. A framework for high-performance data management", SoftwareX, 2020, 12,

2019

Junmin Gu, Burlen Loring, Kesheng Wu, E. Wes Bethel, "HDF5 as a vehicle for in transit data movement", The Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV'19), 2019, doi: 10.1145/3364228.3364237

Junmin Gu, Burlen Loring, Kesheng Wu, E Wes Bethel, "HDF5 as a vehicle for in transit data movement", Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization, 2019, 39--43,

2018

Junmin Gu, Scott Klasky, Norbert Podhorszki, Ji Qiang, Kesheng Wu, "Querying large scientific data sets with adaptable IO system ADIOS", Asian Conference on Supercomputing Frontiers, 2018, 51--69,

2016

Burlen Loring, Suren Byna, Prabhat, Junmin Gu, Hari Krishnan, Michael Wehner, and Oliver Ruebel, "TECA an Extreme Event Detection and Climate Analysis Package for High Performance Computing", The AMS (American Meteorological Society) 96th Annual Meeting, January 6, 2016,

David Pugmire, James Kress, Jong Choi, Scott Klasky, Tahsin Kurc, Randy Michael Churchill, Matthew Wolf, Greg Eisenhower, Hank Childs, Kesheng Wu, others, "Visualization and analysis for near-real-time decision making in distributed workflows", 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2016, 1007--1013,

Utkarsh Ayachit, Andrew Bauer, Earl PN Duque, Greg Eisenhauer, Nicola Ferrier, Junmin Gu, Kenneth E Jansen, Burlen Loring, Zarija Lukic, Suresh Menon, others, "Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures", SC 16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2016, 921--932, LBNL 1007264,

D. Pugmire, J. Kress, J. Choi, S. Klasky, Kurc, R. M. Churchill, M. Wolf, G., H. Childs, K. Wu, A. Sim, J. Gu, J. Low, "Visualization and Analysis for Near-Real-Time Decision in Distributed Workflows", 2016 IEEE International Parallel and Distributed Symposium Workshops (IPDPSW), 2016, 1007--1013, doi: 10.1109/IPDPSW.2016.175

2014

A. L. Chervenak, A. Sim, J. Gu, R. Schuler, N. Hirpathak, "Adaptation and Policy-Based Resource Allocation for Efficient Bulk Data Transfers in High Performance Computing Environments", 4th International Workshop on Network-aware Data Management (NDM'14), 2014,

A. L. Chervenak, A. Sim, J. Gu, R. Schuler, N. Hirpathak, "Efficient Data Staging Using Performance-Based Adaptation and Policy-Based Resource Allocation", 22nd Euromicro International Conference on Parallel, Distributed and Network-based Processing, 2014,

2012

Junmin Gu, David Smith, Ann L. Chervenak, Alex Sim, "Adaptive Data Transfers that Utilize Policies for Resource Sharing", The 2nd International Workshop on Network-Aware Data Management Workshop (NDM2012), 2012,

D. Yu, D. Katramatos, A. Shoshani, A. Sim, J. Gu, V. Natarajan, "StorNet: Integrating Storage Resource Management with Dynamic Network Provisioning for Automated Data Transfer", International Committee for Future Accelerators (ICFA) Standing Committee on Inter-Regional Connectivity (SCIC) 2012 Report: Networking for High Energy Physics, 2012,

2011

J. Gu, D. Katramatos, X. Liu, V. Natarajan, A. Shoshani, A. Sim, D. Yu, S. Bradley, S. McKee, "StorNet: Integrated Dynamic Storage and Network Resource Provisioning and Management for Automated Data Transfers", Journal of Physics: Conf. Ser., 2011, 331, doi: 10.1088/1742- 6596/331/1/012002

G. Garzoglio, J. Bester, K. Chadwick, D. Dykstra, D. Groep, J. Gu, T. Hesselroth, O. Koeroo, T. Levshina, S. Martin, M. Salle, N. Sharma, A. Sim, S. Timm, A. Verstegen, "Adoption of a SAML-XACML Profile for Authorization Interoperability across Grid Middleware in OSG and EGEE", Journal of Physics: Conf. Ser., 2011, 331, doi: 10.1088/1742-6596/331/6/062011

Junmin Gu, Dimitrios Katramatos, Xin Liu, Vijaya Natarajan, Arie Shoshani, Alex Sim, Dantong Yu, Scott Bradley, Shawn McKee, "StorNet: Co-Scheduling of End-to-End Bandwidth Reservation on Storage and Network Systems for High Performance Data Transfers", IEEE INFOCOM HSN 2011, 2011,

Dean N. Williams, Ian T. Foster, Don E. Middleton, Rachana Ananthakrishnan, Neill Miller, Mehmet Balman, Junmin Gu, Vijaya Natarajan, Arie Shoshani, Alex Sim, Gavin Bell, Robert Drach, Michael Ganzberger, Jim Ahrens, Phil Jones, Daniel Crichton, Luca Cinquini, David Brown, Danielle Harper, Nathan Hook, Eric Nienhouse, Gary Strand, Hannah Wilcox, Nathan Wilhelmi, Stephan Zednik, Steve Hankin, Roland Schweitzer, John Harney, Ross Miller, Galen Shipman, Feiyi Wang, Peter Fox, Patrick West, Stephan Zednik, Ann Chervenak, Craig Ward, "Earth System Grid Center for Enabling Technologies (ESG-CET): A Data Infrastructure for Data-Intensive Climate Research", SciDAC Conference, 2011,

2009

M. Riedel, E. Laure, Th. Soddemann, L. Field, J. P. Navarro, J. Casey, M. Litmaath, J. Ph. Baud, B. Koblitz, C. Catlett, D. Skow, C. Zheng, P. M. Papadopoulos, M. Katz, N. Sharma, O. Smirnova, B. Kónya, P. Arzberger, F. Würthwein, A. S. Rana, T. Martin, M. Wan, V. Welch, T. Rimovsky, S. Newhouse, A. Vanni, Y. Tanaka, Y. Tanimura, T. Ikegami, D. Abramson, C. Enticott, G. Jenkins, R. Pordes, N. Sharma, S. Timm, N. Sharma, G. Moont, M. Aggarwal, D. Colling, O. van der Aa, A. Sim, V. Natarajan, A. Shoshani, J. Gu, S. Chen, G. Galang, R. Zappi, L. Magnoni, V. Ciaschini, M. Pace, V. Venturi, M. Marzolla, P. Andreetto, B. Cowles, S. Wang, Y. Saeki, H. Sato, S. Matsuoka, P. Uthayopas, S. Sriprayoonsakul, O. Koeroo, M. Viljoen, L. Pearlman, S. Pickles, David Wallom, G. Moloney, J. Lauret, J. Marsteller, P. Sheldon, S. Pathak, S. De Witt, J. Mencák, J. Jensen, M. Hodges, D. Ross, S. Phatanapherom, G. Netzer, A. R. Gregersen, M. Jones, S. Chen, P. Kacsuk, A. Streit, D. Mallmann, F. Wolf, T. Lippert, Th. Delaitre, E. Huedo, N. Geddes, "Interoperation of world-wide production e-Science infrastructures", Concurrency and Computation: Practice and Experience, 2009, 21(8):961-990,

Arie Shoshani, Flavia Donno, Junmin Gu, Jason Hick, Maarten Litmaath, Alex Sim, "Dynamic Storage Management", Scientific Data Management: Challenges, Technology, and Deployment, edited by Arie Shoshani, Doron Rotem, (Chapman & Hall/CRC Computational Science: 2009)

K Wu, S Ahern, EW Bethel, J Chen, H Childs, C Geddes, J Gu, H Hagen, B Hamann, J Lauret, others, "FastBit: Interactively Searching Massive Data", Proc. of SciDAC 2009, 2009, LBNL 2164E,

2008

P. Jakl, J. Lauret, A. Hanushevsky, A. Shoshani, A. Sim, J. Gu, "Grid data access on widely distributed worker nodes using scalla and SRM", Journal of Physics: Conf. Ser., 2008, 119, doi: 10.1088/1742-6596/119/7/072019

Alex Sim, Arie Shoshani (Editors), Paolo Badino, Olof Barring, Jean‐Philippe Baud, Ezio Corso, Shaun De Witt, Flavia Donno, Junmin Gu, Michael Haddox‐Schatz, Bryan Hess, Jens Jensen, Andy Kowalski, Maarten Litmaath, Luca Magnoni, Timur Perelmutov, Don Petravick, Chip Watson, The Storage Resource Manager Interface Specification Version 2.2, Open Grid Forum, Document in Full Recommendation, GFD.129, 2008,

2007

L. Abadie, P. Badino, J. Baud, E. Corso, M. Crawford, S. De Witt, F. Donno, A. Forti, P. Fuhrmann,
G. Grosdidier, J. Gu , J. Jensen, S. Lemaitre, M. Litmaath, D. Litvinsev, G. Lo Presti, L. Magnoni, T. Mkrtchan, A. Moibenko, V. Natarajan, G. Oleynik, T. Perelmutov, D. Petravick, A. Shoshani, A. Sim, M. Sponza, R. Zappi,
"Storage Resource Managers: Recent International Experience on Requirements and Multiple Co-Operating Implementations", the 24th IEEE Conference on Mass Storage Systems and Technologies, 2007,

F. Donno, L. Abadie, P. Badino, J. Baud, E. Corso, M. Crawford, S. De Witt, A. Forti, P. Fuhrmann, G. Grosdidier, J. Gu , J. Jensen, S. Lemaitre, M. Litmaath, D. Litvinsev, G. Lo Presti, L. Magnoni, T. Mkrtchan, A. Moibenko, V. Natarajan, G. Oleynik, T. Perelmutov, D. Petravick, A. Shoshani, A. Sim, M. Sponza, R. Zappi, "Storage Resource Manager version 2.2: design, implementation, and testing experience", Journal of Physics: Conf. Ser., 2007, 119, doi: 10.1088/1742-6596/119/6/062028

2005

Kesheng Wu, Junmin Gu, Jerome Lauret, Arthur Poskanzer, Arie Shoshani, Alexander Sim, Zhang, "Grid Collector: Facilitating Efficient Selective from Data Grids", International Supercomputer Conference 2005, 2005,

2004

Alex Sim, Junmin Gu, Arie Shoshani, Vijaya Natarajan, "DataMover: Robust Terabytes-Scale Multi-file Replication over Wide-Area Networks", the 16th International Conference on Scientific and Statistical Database Management (SSDBM 2004), 2004,

2003

Arie Shoshani, Alexander Sim, Junmin Gu, "Storage Resource Managers: Essential Components for the Grid", Grid Resource Management: State of the Art and Future Trends, edited by Jarek Nabrzyski, Jennifer M. Schopf, Jan Weglarz, (Kluwer Academic Publishers: 2003)

A. Sim, J. Gu, A. Shoshani, E. Hjort, D. Olson, "Experience with Deploying Storage Resource Managers to Achieve Robust File Replication", Computing in High Energy Physics, 2003,

Arie Shoshani, Alex Sim, Junmin Gu, Storage Resource Managers: Essential Components for Grid Applications, Globus World, 2003,

Kesheng Wu, Wei-Ming Zhang, Alexander Sim, Gu, Arie Shoshani, "Grid Collector: An Event Catalog With Automated File", Proceedings of IEEE Nuclear Science Symposium 2003, 2003, doi: 10.1109/NSSMIC.2003.1351830

Kesheng Wu, Wei-Ming Zlang, Alexander Sim, Junmin Gu, Arie Shoshani, "Grid collector: An event catalog with automated file management", 2003 IEEE Nuclear Science Symposium. Conference Record (IEEE Cat. No. 03CH37515), 2003, LBNL 55563,

2002

A. Shoshani, A. Sim, J. Gu, "Storage Resource Managers: Middleware components for Grid Storage", the 19th IEEE Symposium on Mass Storage Systems, 2002,

Daniel Gunter

2023

Mohammed A. Alhussaini, Zachary M. Binger, Bianca M. Souza-Chaves, Oluwamayowa O. Amusat, Jangho Park, Timothy V. Bartholomew, Dan Gunter, Andrea Achilli, "Analysis of backwash settings to maximize net water production in an engineering-scale ultrafiltration system for water reuse", Journal of Water Process Engineering, 2023, 53, doi: 10.1016/j.jwpe.2023.103761

2022

Andrew Adams, Emily K. Adams, Dan Gunter, Ryan Kiser, Mark Krenz, Sean Peisert, John Zage, "Roadmap for Securing Operational Technology in NSF Scientific Research", Trusted CI Report, November 16, 2022, doi: 10.5281/zenodo.7327987

Emily K. Adams, Daniel Gunter, Ryan Kiser, Mark Krenz, Sean Peisert, Susan Sons, John Zage, "Findings of the 2022 Trusted CI Study on the Security of Operational Technology in NSF Scientific Research", Trusted CI Report, July 15, 2022, doi: doi.org/10.5281/zenodo.6828675

2021

Dan Gunter, Oluwamayowa Amusat, Tim Bartholomew, Markus Drouven, "Santa Barbara Desalination Digital Twin Technical Report", LBNL Technical Report, 2021, LBNL LBNL-2001437,

Devarshi Ghoshal, Drew Paine, Gilberto Pastorello, Abdelrahman Elbashandy, Dan Gunter, Oluwamayowa Amusat, Lavanya Ramakrishnan, "Experiences with Reproducibility: Case Studies from Scientific Workflows", (P-RECS'21) Proceedings of the 4th International Workshop on Practical Reproducible Evaluation of Computer Systems, ACM, June 21, 2021, doi: 10.1145/3456287.3465478

Reproducible research is becoming essential for science to ensure transparency and for building trust. Additionally, reproducibility provides the cornerstone for sharing of methodology that can improve efficiency. Although several tools and studies focus on computational reproducibility, we need a better understanding about the gaps, issues, and challenges for enabling reproducibility of scientific results beyond the computational stages of a scientific pipeline. In this paper, we present five different case studies that highlight the reproducibility needs and challenges under various system and environmental conditions. Through the case studies, we present our experiences in reproducing different types of data and methods that exist in an experimental or analysis pipeline. We examine the human aspects of reproducibility while highlighting the things that worked, that did not work, and that could have worked better for each of the cases. Our experiences capture a wide range of scenarios and are applicable to a much broader audience who aim to integrate reproducibility in their everyday pipelines.

2019

Reinhard Gentz, Sean Peisert, Joshua Boverhof, Daniel Gunter, "SPARCS: Stream-Processing Architecture applied in Real-time Cyber-physical Security", Proceedings of the 15th IEEE International Conference on e-Science (eScience), San Diego, CA, IEEE, September 2019, doi: 10.1109/eScience.2019.00028

2018

Anna Giannakou, Daniel Gunter, Sean Peisert, "Flowzilla: A Methodology for Detecting Data Transfer Anomalies in Research Networks", Workshop on Innovating the Network for Data-Intensive Science (INDIS), November 11, 2018, doi: 10.1109/INDIS.2018.00004

AP Arkin,RW Cottingham,CS Henry,NL Harris,RL Stevens,S Maslov,P Dehal,D Ware,F Perez,S Canon,MW Sneddon,ML Henderson,WJ Riehl,D Murphy-Olson,SY Chan,RT Kamimura,S Kumari,MM Drake,TS Brettin,EM Glass,D Chivian,D Gunter,DJ Weston,BH Allen,J Baumohl,AA Best,B Bowen,SE Brenner,CC Bun,JM Chandonia,JM Chia,R Colasanti,N Conrad,JJ Davis,BH Davison,M Dejongh,S Devoid,E Dietrich,I Dubchak,JN Edirisinghe,G Fang,JP Faria,PM Frybarger,W Gerlach,M Gerstein,A Greiner,J Gurtowski,HL Haun,F He,R Jain,MP Joachimiak,KP Keegan,S Kondo,V Kumar,ML Land,F Meyer,M Mills,PS Novichkov,T Oh,GJ Olsen,R Olson,B Parrello,S Pasternak,E Pearson,SS Poon,GA Price,S Ramakrishnan,P Ranjan,PC Ronald,MC Schatz,SMD Seaver,M Shukla,RA Sutormin,MH Syed,J Thomason,NL Tintle,D Wang,F Xia,H Yoo,S Yoo,D Yu, "KBase: The United States department of energy systems biology knowledgebase", Nature Biotechnology, July 2018, 36:566--569, doi: 10.1038/nbt.4163

DC Miller, JD Siirola, D Agarwal, AP Burgard, A Lee, JC Eslick, B Nicholson, C Laird, LT Biegler, D Bhattacharyya, NV Sahinidis, IE Grossmann, CE Gounaris, D Gunter, "Next Generation Multi-Scale Process Systems Engineering Framework", Computer Aided Chemical Engineering, 2018, 44:2209--2214, doi: 10.1016/B978-0-444-64241-7.50363-3

2017

DA Agarwal, C Varadharajan, S Cholia, C Snavely, VC Hendrix, D Gunter, WJ Riley, M jones, AE budden, D Vieglas, Environmental System Science Data Infrastructure for a Virtual Ecosystem (ESS-DIVE)-A New US DOE Data Archive, 2017,

Gilberto Z. Pastorello, Dan K. Gunter, Housen Chu, Danielle S. Christianson, Carlo Trotta, Eleonora Canfora, Boris Faybishenko, You-Wei Cheah, Norm Beekwilder, Stephen W. Chan, Sigrid Dengel, Trevor Keenan, Fianna O Brien, Abderahman Elbashandy, Cristina M. Poindexter, Marty Humphrey, Dario Papale, Deb A. Agarwal, "Hunting Data Rogues at Scale: Data Quality Control for Observational Data in Research Infrastructures", Proceedings of the 13th IEEE International Conference on e-Science (e-Science 2017), Auckland, New Zealand, 2017, doi: 10.1109/eScience.2017.64

S Peisert, R Gentz, J Boverhof, C McParland, S Engle, A Elbashandy, D Gunter, "LBNL Open Power Data", January 2017, doi: 10.21990/C21599

L Ramakrishnan, D Gunter, "Ten principles for creating usable software for science", Proceedings - 13th IEEE International Conference on eScience, eScience 2017, 2017, 210--218, doi: 10.1109/eScience.2017.34

2016

Patrick Huck, Dan Gunter, Shreyas Cholia, Donald Winston, AT N Diaye, Kristin Persson, "User applications driven by the community contribution framework MPContribs in the Materials Project", Concurrency and Computation: Practice and Experience, 2016, 28:1982--1993,

2015

P Huck, D Gunter, S Cholia, D Winston, A N Diaye, KA Persson, "User Applications Driven by the Community Contribution Framework MPContribs in the Materials Project.", CoRR, 2015, abs/1510,

SP Ong, S Cholia, A Jain, M Brafman, D Gunter, G Ceder, KA Persson, "The Materials Application Programming Interface (API): A simple, flexible and efficient API for materials data based on REpresentational State Transfer (REST) principles", Computational Materials Science, 2015, 97:209--215, doi: 10.1016/j.commatsci.2014.10.037

2014

Gilberto Z. Pastorello, Deb A. Agarwal, Taghrid Samak, Dario Papale, Trotta, Alessio Ribeca, Cristina M. Poindexter, Boris Faybishenko, Dan K. Gunter, Rachel Hollowgrass, Eleonora Canfora, "Observational data patterns for time series data quality assessment", Proceedings of the 10th IEEE International Conference on e-Science (e-Science 2014), Guaruja, Brazil, 2014, doi: 10.1109/eScience.2014.45

Lavanya Ramakrishnan, Sarah S. Poon, Val C. Hendrix, Dan K. Gunter, Gilberto Z. Pastorello, Deb A. Agarwal, "Experiences with User-Centered Design for the Tigres Workflow API", Proceedings of the 10th IEEE International Conference on e-Science (e-Science 2014), Guaruja, Brazil, 2014, doi: 10.1109/eScience.2014.56

2013

Elif Dede, Madhusudhan Govindaraju, Daniel Gunter, Richard Canon, Lavanya Ramakrishnan, "Semi-Structured Data Analysis using MongoDB and MapReduce: A Performance Evaluation", Proceedings of the 4th international workshop on Scientific cloud computing, 2013,

A Jain, SP Ong, G Hautier, W Chen, WD Richards, S Dacek, S Cholia, D Gunter, D Skinner, G Ceder, KA Persson, "Commentary: The materials project: A materials genome approach to accelerating materials innovation", APL Materials, 2013, 1, doi: 10.1063/1.4812323

SP Ong, WD Richards, A Jain, G Hautier, M Kocher, S Cholia, D Gunter, VL Chevrier, KA Persson, G Ceder, "Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis", Computational Materials Science, 2013, 68:314--319, doi: 10.1016/j.commatsci.2012.10.028

2012

Taghrid Samak, Dan Gunter, Monte Goode, Ewa Deelman, Fabio Silva, Karan Vahi, "Failure Analysis of Distributed Scientific Workflows Executing in the Cloud", 8th International Conference on Network and Service Management (CNSM 2012), 2012,

Karan Vahi, Ian Harvey, Taghrid Samak, Dan Gunter, Kieran Evans, David Rogers, Ian Taylor, Monte Goode, Fabio Silva, Eddie Al-Shakarchi, Gaurang Mehta, Andrew Jones, Ewa Deelman, "A General Approach to Real-time Workflow Monitoring", The Seventh Workshop on Workflows in Support of Large-Scale Science (WORKS12), 2012,

D Gunter, S Cholia, A Jain, M Kocher, K Persson, L Ramakrishnan, SP Ong, G Ceder, "Community accessible datastore of high-throughput calculations: Experiences from the materials project", Proceedings - 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, SCC 2012, 2012, 1244--1251, doi: 10.1109/SC.Companion.2012.150

Dan Gunter, Raj Kettimuthu, Ezra Kissel, Martin Swany, Jun Yi, Jason Zurawski, "Exploiting Network Parallelism for Improving Data Transfer Performance", SC12 Companion, 2012,

T Samak, D Gunter, V Hendrix, "Scalable analysis of network measurements with Hadoop and Pig", Proceedings of the 2012 IEEE Network Operations and Management Symposium, NOMS 2012, 2012, 1254--1259, doi: 10.1109/NOMS.2012.6212060

Ezra Kissel, Ahmed El-Hassany, Guilherme Fernandes, Martin Swany, Dan Gunter, Taghrid Samak, Jennifer M. Schopf, "Scalable Integrated Performance Analysis of Multi-Gigabit Networks", Fifth International Workshop on Distributed Autonomous Network Management Systems 2012 (DANMS 12), 2012,

Elif Dede, Zacharia Fadika, Jessica Hartog, Modhusudhan Govindaraju, Lavanya Ramakrishnan, Daniel Gunter, Richard Shane Canon, "MARISSA: MApReduce Implementation for Streaming Science Applications", IEEE eScience Conference, 2012,

A Jain, G Hautier, SP Ong, C Moore, B Kang, H Chen, X Ma, JC Kim, M Kocher, D Gunter, S Cholia, A Greiner, DH Bailey, D Skinner, K Persson, G Ceder, "Materials Project: A public materials database and its application to lithium ion battery cathode design", ABSTRACTS OF PAPERS OF THE AMERICAN CHEMICAL SOCIETY, 2012, 243,

2011

Taghrid Samak, Dan Gunter, Monte Goode, Ewa Deelman, Gaurang Mehta, Fabio Silva, Karan Vahi, "Failure Prediction and Localization in Large Scientific Workflows", The Sixth Workshop on Workflows in Support of Large-Scale Science (WORKS11), 2011,

Dan Gunter, Taghrid Samak, Ewa Deelman, Christopher H. Brooks, Monte Goode, Gideon Juve, Gaurang Mehta, Priscilla Moraes, Fabio Silva, Martin Swany, Karan Vahi, "Online Workflow Management and Performance Analysis with STAMPEDE", 7th International Conference on Network and Service Management (CNSM 2011), Paris, France, 2011,

Taghrid Samak, Dan Gunter, Ewa Deelman, Gideon Juve, Gaurang Mehta, Fabio Silva, Karan Vahi, "Online Fault and Anomaly Detection for Large-Scale Scientific Workflows", 13th IEEE Conference on High Performance Computing and Communications (HPCC-2011), 2011,

Elif Dede, Madhusudan Govindaraju, Daniel Gunter, Lavanya Ramakrishnan, "Riding the Elephant: Managing Ensembles with Hadoop", 4th Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS), 2011,

2010

A. Sim, D. Gunter, V. Natarajan, A. Shoshani, D. Williams, J. Long, J. Hick, J. Lee, E. Dart, "Efficient Bulk Data Replication for the Earth System Grid", Data Driven E-science: Use Cases and Successful Applications of Distributed Computing Infrastructures (ISGC 2010), (Springer-Verlag New York Inc: 2010) Pages: 435

Scott Callaghan, Ewa Deelman, Dan Gunter, Gideon Juve, Philip Maechling, Christopher Brooks, Karan Vahi, Kevin Milner, Robert Graves, Edward Field, David Okaya, Thomas Jordan, "Scaling up Workflow-based Applications", Journal of Computer and System Sciences, 2010, 76:428--446,

Raj Kettimuthu, Alex Sim, Dan Gunter, Bill Allcock, Peer T. Bremer, John Bresnahan, Andrew Cherry, Lisa Childers, Eli Dart, Ian Foster, Kevin Harms, Jason Hick, Jason Lee, Michael Link, Jeff Long, Keith Miller, Vijaya Natarajan, Valerio Pascucci, Ken Raffenetti, David Ressman, Dean Williams, Loren Wilson, Linda Winkler, "Lessons learned from moving earth system grid data sets over a 20 Gbps wide-area network", HPDC 10, New York, NY, USA, ACM, 2010, 316--319, doi: 10.1145/1851476.1851519

2007

Shoaib Kamil, Pinar, Gunter, Lijewski, Oliker, John Shalf, "Reconfigurable hybrid interconnection for static and dynamic scientific applications", Conf. Computing Frontiers, 2007, 183-194, LBNL 60060,

Dan Gunter, Brian L. Tierney, Aaron Brown, Martin Swany, John Bresnahan, Jennifer M. Schopf, "Log summarization and anomaly detection for troubleshooting distributed systems", Grid Computing, 2007 8th IEEE/ACM International Conference on, Washington, DC, USA, IEEE Computer Society, 2007, 226--234, doi: 10.1109/GRID.2007.4354137

Matthew Henderson

2021

C Varadharajan, Z Kakalia, E Alper, EL Brodie, M Burrus, RWH Carroll, D Christianson, W Dong, V Hendrix, M Henderson, S Hubbard, D Johnson, R Versteeg, KH Williams, DA Agarwal, The Colorado East River Community Observatory Data Collection, Hydrological Processes 35(6), 2021, doi: 10.22541/au.161962485.54378235/v1

2019

C Varadharajan, B Faybishenko, A Henderson, M Henderson, VC Hendrix, SS Hubbard, Z Kakalia, A Newman, B Potter, H Steltzer, R Versteeg, DA Agarwal, KH Williams, C Wilmer, Y Wu, W Brown, M Burrus, RWH Carroll, DS Christianson, B Dafflon, D Dwivedi, BJ Enquist, Challenges in Building an End-to-End System for Acquisition, Management, and Integration of Diverse Data from Sensor Networks in Watersheds: Lessons from a Mountainous Community Observatory in East River, Colorado, IEEE Access, Pages: 182796--18 2019, doi: 10.1109/ACCESS.2019.2957793

2018

Wahid Bhimji, Steven Farrell, Oliver Evans, Matthew Henderson, Shreyas Cholia, Aaron Vose, Mr Prabhat, Rollin Thomas, Richard Shane Canon, "Interactive HPC Deep Learning with Jupyter Notebooks", Supercomputing 2018, Dallas, TX, November 2018,

Shreyas Cholia, Matthew Henderson, Oliver Evans, Fernando Perez, "Kale: A System for Enabling Human-in-the-loop Interactivity in HPC Workflows", Science Gateways 2018, figshare, September 26, 2018, doi: 10.6084/m9.figshare.7067075.v3

AP Arkin,RW Cottingham,CS Henry,NL Harris,RL Stevens,S Maslov,P Dehal,D Ware,F Perez,S Canon,MW Sneddon,ML Henderson,WJ Riehl,D Murphy-Olson,SY Chan,RT Kamimura,S Kumari,MM Drake,TS Brettin,EM Glass,D Chivian,D Gunter,DJ Weston,BH Allen,J Baumohl,AA Best,B Bowen,SE Brenner,CC Bun,JM Chandonia,JM Chia,R Colasanti,N Conrad,JJ Davis,BH Davison,M Dejongh,S Devoid,E Dietrich,I Dubchak,JN Edirisinghe,G Fang,JP Faria,PM Frybarger,W Gerlach,M Gerstein,A Greiner,J Gurtowski,HL Haun,F He,R Jain,MP Joachimiak,KP Keegan,S Kondo,V Kumar,ML Land,F Meyer,M Mills,PS Novichkov,T Oh,GJ Olsen,R Olson,B Parrello,S Pasternak,E Pearson,SS Poon,GA Price,S Ramakrishnan,P Ranjan,PC Ronald,MC Schatz,SMD Seaver,M Shukla,RA Sutormin,MH Syed,J Thomason,NL Tintle,D Wang,F Xia,H Yoo,S Yoo,D Yu, "KBase: The United States department of energy systems biology knowledgebase", Nature Biotechnology, July 2018, 36:566--569, doi: 10.1038/nbt.4163

S. Farrell, A. Vose, O. Evans, M. Henderson, S. Cholia, W. Bhimji, R. Thomas, S.
Canon, and Prabhat,,
"Interactive Distributed Deep Learning with Jupyter Notebooks", ISC Workshop on Interactive High-Performance Computing, June 28, 2018,

GP Rodrigo, M Henderson, GH Weber, C Ophus, K Antypas, L Ramakrishnan, "ScienceSearch: Enabling Search through Automatic Metadata Generation", 2018 IEEE 14th International Conference on e-Science (e-Science), IEEE, 2018, doi: 10.1109/escience.2018.00025

2017

Shreyas Cholia, Matthew Henderson, Oliver Evans, Demo: Extending Jupyter to Support Interactive High Performance Computing, Science Gateways 2017, October 2017, doi: 10.6084/m9.figshare.5501137.v1

Valerie Hendrix

2022

MB Simmonds, WJ Riley, DA Agarwal, X Chen, S Cholia, R Crystal-Ornelas, ET Coon, D Dwivedi, VC Hendrix, M Huang, A Jan, Z Kakalia, J Kumar, CD Koven, L Li, M Melara, L Ramakrishnan, DM Ricciuto, AP Walker, W Zhi, Q Zhu, C Varadharajan, Guidelines for Publicly Archiving Terrestrial Model Data to Enhance Usability, Intercomparison, and Synthesis, Data Science Journal, 2022, doi: 10.5334/dsj-2022-003

C Varadharajan, VC Hendrix, DS Christianson, M Burrus, C Wong, SS Hubbard, DA Agarwal, BASIN-3D: A brokering framework to integrate diverse environmental data, Computers and Geosciences, 2022, doi: 10.1016/j.cageo.2021.105024

C Varadharajan, AP Appling, B Arora, DS Christianson, VC Hendrix, V Kumar, AR Lima, J Müller, S Oliver, M Ombadi, T Perciano, JM Sadler, H Weierbach, JD Willard, Z Xu, J Zwart, "Can machine learning accelerate process understanding and decision-relevant predictions of river water quality?", Hydrological Processes, January 1, 2022, 36, doi: 10.1002/hyp.14565

H Weierbach, AR Lima, JD Willard, VC Hendrix, DS Christianson, M Lubich, C Varadharajan, Stream Temperature Predictions for River Basin Management in the Pacific Northwest and Mid-Atlantic Regions Using Machine Learning, Water (Switzerland), 2022, doi: 10.3390/w14071032

2021

C Varadharajan, Z Kakalia, E Alper, EL Brodie, M Burrus, RWH Carroll, D Christianson, W Dong, V Hendrix, M Henderson, S Hubbard, D Johnson, R Versteeg, KH Williams, DA Agarwal, The Colorado East River Community Observatory Data Collection, Hydrological Processes 35(6), 2021, doi: 10.22541/au.161962485.54378235/v1

JE Damerow, C Varadharajan, K Boye, EL Brodie, M Burrus, KD Chadwick, R Crystal-Ornelas, H Elbashandy, RJ Eloy Alves, KS Ely, AE Goldman, T Haberman, V Hendrix, Z Kakalia, KM Kemner, AB Kersting, N Merino, F O Brien, Z Perzan, E Robles, P Sorensen, JC Stegen, RL Walls, P Weisenhorn, M Zavarin, D Agarwal, Sample identifiers and metadata to support data management and reuse in multidisciplinary ecosystem sciences, Data Science Journal, 2021, doi: 10.5334/dsj-2021-011

R Crystal-Ornelas, C Varadharajan, B Bond-Lamberty, K Boye, M Burrus, S Cholia, M Crow, J Damerow, R Devarakonda, KS Ely, A Goldman, S Heinz, V Hendrix, Z Kakalia, SC Pennington, E Robles, A Rogers, M Simmonds, T Velliquette, H Weierbach, P Weisenhorn, JN Welch, DA Agarwal, A Guide to Using GitHub for Developing and Versioning Data Standards and Reporting Formats, Earth and Space Science, 2021, doi: 10.1029/2021EA001797

2019

C Varadharajan, S Cholia, C Snavely, V Hendrix, C Procopiou, D Swantek, W Riley, D Agarwal, "Launching an Accessible Archive of Environmental Data", Eos, January 1, 2019, 100, doi: 10.1029/2019eo111263

C Varadharajan, B Faybishenko, A Henderson, M Henderson, VC Hendrix, SS Hubbard, Z Kakalia, A Newman, B Potter, H Steltzer, R Versteeg, DA Agarwal, KH Williams, C Wilmer, Y Wu, W Brown, M Burrus, RWH Carroll, DS Christianson, B Dafflon, D Dwivedi, BJ Enquist, Challenges in Building an End-to-End System for Acquisition, Management, and Integration of Diverse Data from Sensor Networks in Watersheds: Lessons from a Mountainous Community Observatory in East River, Colorado, IEEE Access, Pages: 182796--18 2019, doi: 10.1109/ACCESS.2019.2957793

2018

EM Stewart, P Top, M Chertkov, D Deka, S Backhaus, A Lokhov, C Roberts, V Hendrix, S Peisert, A Florita, TJ King, MJ Reno, "Integrated multi-scale data analytics and machine learning for the distribution grid", 2017 IEEE International Conference on Smart Grid Communications, SmartGridComm 2017, 2018, 2018-Jan:423--429, doi: 10.1109/SmartGridComm.2017.8340693

2017

Danielle S. Christianson, Charuleka Varadharajan, Bradley Christoffersen, Matteo Detto, Faybishenko, Bruno O. Gimenez, Val C. Hendrix, Kolby J. Jardine, Robinson Negron-Juarez, Z. Pastorello, Thomas L. Powell, Megha Sandesh, Jeffrey M. Warren, Brett T. Wolfe, Jeffrey Q. Chambers, Lara M. Kueppers, Nathan G. McDowell, Deborah A. Agarwal, "A metadata reporting framework (FRAMES) for synthesis of ecohydrological observations", Ecological Informatics, 2017, 42:148-158, doi: 10.1016/j.ecoinf.2017.06.002

DA Agarwal, C Varadharajan, S Cholia, C Snavely, VC Hendrix, D Gunter, WJ Riley, M jones, AE budden, D Vieglas, Environmental System Science Data Infrastructure for a Virtual Ecosystem (ESS-DIVE)-A New US DOE Data Archive, 2017,

GH Weber, MS Bandstra, DH Chivers, HH Elgammal, V Hendrix, J Kua, JS Maltz, K Muriki, Y Ong, K Song, MJ Quinlan, L Ramakrishnan, BJ Quiter, "Web-based visual data exploration for improved radiological source detection", Concurrency Computation, 2017, 29, doi: 10.1002/cpe.4203

D Ghoshal, V Hendrix, W Fox, S Balasubhramanian, L Ramakrishnan, "FRIEDA: Flexible Robust Intelligent Elastic Data Management Framework", The Journal of Open Source Software, 2017, 2:164--164, doi: 10.21105/joss.00164

E Stewart, V Hendrix, M Chertkov, D Deka, "Integrated Multi-Scale Data Analytics and Machine Learning for the Distribution Grid and Building-to-Grid Interface", 2017,

2016

V Hendrix, J Fox, D Ghoshal, L Ramakrishnan, "Tigres Workflow Library: Supporting Scientific Pipelines on HPC Systems", Proceedings - 2016 16th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2016, 2016, 146--155, doi: 10.1109/CCGrid.2016.54

M Verma, JB Fisher, K Mallick, Y Ryu, H Kobayashi, A Guillaume, G Moore, L Ramakrishnan, V Hendrix, S Wolf, M Sikka, G Kiely, G Wohlfahrt, B Gielen, O Roupsard, P Toscano, A Arain, A Cescatti, "Global surface net-radiation at 5 km from MODIS Terra", Remote Sensing, 2016, 8, doi: 10.3390/rs8090739

2014

V Hendrix, L Ramakrishnan, Y Ryu, C Van Ingen, KR Jackson, D Agarwal, "CAMP: Community access MODIS pipeline", Future Generation Computer Systems, 2014, 36:418--429, doi: 10.1016/j.future.2013.09.023

Lavanya Ramakrishnan, Sarah S. Poon, Val C. Hendrix, Dan K. Gunter, Gilberto Z. Pastorello, Deb A. Agarwal, "Experiences with User-Centered Design for the Tigres Workflow API", Proceedings of the 10th IEEE International Conference on e-Science (e-Science 2014), Guaruja, Brazil, 2014, doi: 10.1109/eScience.2014.56

JR Balderrama, M Simonin, L Ramakrishnan, V Hendrix, C Morin, D Agarwal, C Tedeschi, "Combining workflow templates with a shared space-based execution model", Proceedings of WORKS 2014: The 9th Workshop on Workflows in Support of Large-Scale Science - held in conjunction with SC 2014: The International Conference for High Performance Computing, Networking, Storage and Analysis, 2014, 50--58, doi: 10.1109/WORKS.2014.14

S Panitkin, FB Megino, JC Bejar, D Benjamin, AD Girolamo, I Gable, V Hendrix, J Hover, K Kucharczyk, RM Llamas, P Love, H Ohman, M Paterson, R Sobie, R Taylor, R Walker, A Zaytsev, "ATLAS cloud R\&amp;D", Journal of Physics: Conference Series, 2014, 513, doi: 10.1088/1742-6596/513/6/062037

H Öhman, S Panitkin, V Hendrix, "Using Puppet to contextualize computing resources for ATLAS analysis on Google compute engine", Journal of Physics: Conference Series, 2014, 513, doi: 10.1088/1742-6596/513/3/032073

L Ramakrishnan, D Ghoshal, V Hendrix, E Feller, P Mantha, C Morin, "Storage and Data Life Cycle Management in Cloud Environments with FRIEDA.", Cloud Computing for Data-Intensive Applications, (Springer: 2014) Pages: 357--378

2013

E. Masanet, A. Shehabi, L. Ramakrishnan, J. Liang, X. Ma, B. Walker, V. Hendrix, P Mantha, "The Energy Efficiency Potential of Cloud-Based Software: A U.S.Case Study", 2013, LBNL 6298E,

2012

V Hendrix, J Li, K Jackson, L Ramakrishnan, Y Ryu, K Beattie, C Morin, D Skinner, C van Ingen, D Agarwal, "Community Access to MODIS Satellite Reprojection and Reduction Pipeline and Data Sets", AGU Fall Meeting, 2012,

T Samak, D Gunter, V Hendrix, "Scalable analysis of network measurements with Hadoop and Pig", Proceedings of the 2012 IEEE Network Operations and Management Symposium, NOMS 2012, 2012, 1254--1259, doi: 10.1109/NOMS.2012.6212060

F Harald Barreiro Megino, D Benjamin, K De, I Gable, V Hendrix, S Panitkin, M Paterson, A De Silva, D Van Der Ster, R Taylor, RA Vitillo, R Walker, "Exploiting virtualization and cloud computing in ATLAS", Journal of Physics: Conference Series, 2012, 396, doi: 10.1088/1742-6596/396/3/032011

V Hendrix, D Benjamin, Y Yao, "Scientific cluster deployment and recovery - Using puppet to simplify cluster management", Journal of Physics: Conference Series, 2012, 396, doi: 10.1088/1742-6596/396/4/042027

Mariam Kiran

2022

D. Bard, C. Snavely, L. Gerhardt, J. Lee, B. Totzke, K. Antypas, W. Arndt, J. Blaschke, S. Byna, R. Cheema, S. Cholia, M. Day, B. Enders, A. Gaur, A. Greiner, T. Groves, M. Kiran, Q. Koziol, T. Lehman, K. Rowland, C. Samuel, A. Selvarajan, A. Sim, D. Skinner, L. Stephey, R. Thomas, G. Torok, "LBNL Superfacility Project Report", Lawrence Berkeley National Laboratory, 2022, doi: 10.48550/arXiv.2206.11992

Qiang Du, Dan Wang, Tong Zhou, Antonio Gilardi, Mariam Kiran, Bashir Mohammed, Derun Li, and Russell Wilcox, "Experimental beam combining stabilization using machine learning trained while phases drift", Advanced Solid State Lasers 2022, © 2022 Optica Publishing Group, June 1, 2022, Vol. 30,:pp. 12639-, doi: https://doi.org/10.1364/OE.450255

Sugeerth Murugesan, Mariam Kiran, Bernd Hamann, Gunther H. Weber, "Netostat: Analyzing Dynamic Flow Patterns in High-Speed Networks", Cluster Computing, 2022, doi: 10.1007/s10586-022-03543-0

2021

Shen Sheng, Mariam Kiran, Bashir Mohammed, "DynamicDeepFlow: An Approach for Identifying Changes in Network Traffic Flow Using Unsupervised Clustering", (BEST PAPER) 4th International Conference on Machine Learning for Networking (MLN'2021), December 6, 2021,

V. Dumont, C. Garner, A. Trivedi, C. Jones, V. Ganapati, J. Mueller, T. Perciano, M. Kiran, and M. Day, "HYPPO: A Surrogate-Based Multi-Level Parallelism Tool for Hyperparameter Optimization", 2021 IEEE/ACM Workshop on Machine Learning in High Performance Computing Environments (MLHPC), November 15, 2021,

Bashir Mohammed, Mariam Kiran, Bjoern Enders, "NetGraf: An End-to-End Learning Network Monitoring Service", 2021 IEEE Workshop on Innovating the Network for Data-Intensive Science (INDIS), November 15, 2021, doi: 10.1109/INDIS54524.2021.00007

B Mohammed, M Kiran; N Krishnaswamy; Keshang, Wu, "Predicting WAN Traffic Volumes using Fourier and Multivariate SARIMA Approach", International Journal of Big Data Intelligence, November 3, 2021, doi: 10.1504/IJBDI.2021.118742

M Kiran, B Mohammed, Q Du, D Wang, S Shen, R Wilcox, "Controlling Laser Beam Combining via an Active Reinforcement Learning Algorithm", Advanced Solid State Lasers 2021, Washington, DC United States, October 4, 2021,

Meriam Gay Bautista, Zhi Jackie Yao, Anastasiia Butko, Mariam Kiran, Mekena Metcalf, "Towards Automated Superconducting Circuit Calibration using Deep Reinforcement Learning", 2021 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), Tampa, FL, USA, IEEE, August 23, 2021, pp. 462-46, doi: 10.1109/ISVLSI51109.2021.00091

2020

Bashir Mohammed, Mariam Kiran, Dan Wang, Qiang Du, Russell Wilcox, "Deep Reinforcement Learning based Control for two-dimensional Coherent Combining", Laser Applications Conference, pp. JTu5A-7. Optical Society of America, 2020., OSA Publishing, December 1, 2020,

Dan Wang, Qiang Du, Tong Zhou, Bashir Mohammed, Mariam Kiran, Derun Li, Russell Wilcox, "Artificial Neural Networks Applied to Stabilization of 81-beam Coherent Combining", Advanced Solid State Lasers, Optical Society of America, December 1, 2020,

Divneet Kaur, Bashir Mohammed, Mariam Kiran, "NetGraf: A Collaborative Network Monitoring Stack for Network Experimental Testbeds", December 1, 2020,

D. Bard, C. Snavely, L. Gerhardt, J. Lee, B. Totzke, K. Antypas, S. Byna, R. Cheema, S. Cholia, M. Day, B. Enders, A. Gaur, A. Greiner, T. Groves, M. Kiran, Q. Koziol, K. Rowland, C. Samuel, A. Selvarajan, A. Sim, D. Skinner, R. Thomas, G. Torok, The Superfacility project: automated pipelines for experiments and HPC, International Conference for High Performance Computing, Networking, Storage, and Analysis (SC20), State of the Practice (SOP), 2020,

B. Enders, D. Bard, C. Snavely, L. Gerhardt, J. Lee, B. Totzke, K. Antypas, S. Byna, R. Cheema, S. Cholia, M. Day, A. Gaur, A. Greiner, T. Groves, M. Kiran, Q. Koziol, K. Rowland, C. Samuel, A. Selvarajan, A. Sim, D. Skinner, R. Thomas, G. Torok, "Cross-facility science with the Superfacility Project at LBNL", 2nd Workshop on Large-scale Experiment-in-the-Loop Computing (XLOOP 2020), in conjunction with the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 20), 2020, doi: 10.1109/XLOOP51963.2020.00006

N Krishnaswamy; M Kiran; B Mohammed; Singh Kunal, "Data-driven Learning to Predict WAN Network Traffic.", SNTA '20: Proceedings of the 3rd International Workshop on Systems and Network Telemetry and Analytics, November 3, 2020, 11-18, doi: 10.1145/3391812.3396268

T Mallick, M Kiran, B Mohammed, Prasanna Balaprakash, "Dynamic Graph Neural Network for Traffic Forecasting in Wide Area Networks.", Machine Learning Big Data 2020, IEEE, November 2, 2020, 1-10, doi: 10.1109/BigData50022.2020.9512748

M Hocine, M Kiran, A Mercian, and B Mohammed, "Using Machine Learning for Intent-based provisioning in High-Speed Science Networks.", SNTA '20: Proceedings of the 3rd International Workshop on Systems and Network Telemetry and Analytics, November 2, 2020, 27-30, doi: 10.1145/3391812.3396269

2019

M Kiran, B Mohammed and N. Krishnaswamy, "DeepRoute: Herding Elephant and Mice Flows with Reinforcement Learning", 2nd IFIP International Conference on Machine Learning for Networking (MLN'2019), December 2, 2019, doi: 10.1007/978-3-030-45778-5_20

B Mohammed, M Kiran, N Krishnaswamy, "DeepRoute on Chameleon: Experimenting with Large-scale Reinforcement Learning and SDN on Chameleon Testbed", IEEE 27th International Conference on Network Protocols (ICNP), IEEE, November 14, 2019, 1-2, doi: 10.1109/ICNP.2019.8888090

George Papadimitriou, Mariam Kiran, Cong Wang, Anirban Mandal, Ewa Deelman, "Training Classifiers to Identify TCP Signatures in Scientific Workflows", INDIS, SC19, November 14, 2019,

B Mohammed, N Krishnaswamy, M Kiran, "Multivariate Time-Series Prediction for Traffic in Large WAN Topology", ACM/IEEE Symposium on Architectures for Networking and Communications, August 1, 2019, doi: 10.1109/ANCS.2019.8901870

M Kiran, A Chhabra, "Understanding flows in high-speed scientific networks: A Netflow data study", Future Generation Computer Systems, February 1, 2019, 94:72-79,

Mariam Kiran, Anshuman Chhabra, "Understanding flows in high-speed scientific networks: A Netflow data study", Future Generation Computer Science, 2019,

2018

F Alali, N Hanford, E Pouyoul, R Kettimuthu, M Kiran, B Mack-Crane, B Tierney, Y Kumar, D Ghosal, "Calibers: A bandwidth calendaring paradigm for science workflows", Future Generation Computer Systems, 2018, 89:736--745, doi: 10.1016/j.future.2018.07.030

A Bennaceur, A Cano, L Georgieva, M Kiran, M Salama, P Yadav, "Issues in gender diversity and equality in the UK", Proceedings - International Conference on Software Engineering, 2018, 5--9, doi: 10.1145/3195570.3195571

B Mohammed, B Modu, KM Maiyama, H Ugail, I Awan, M Kiran, "Failure Analysis Modelling in an Infrastructure as a Service (Iaas) Environment", Electronic Notes in Theoretical Computer Science, 2018, 340:41--54, doi: 10.1016/j.entcs.2018.09.004

M Gribaudo, M Iacono, M Kiran, "A performance modeling framework for lambda architecture based applications", Future Generation Computer Systems, 2018, 86:1032--1041, doi: 10.1016/j.future.2017.07.033

M Kiran, E Pouyoul, A Mercian, B Tierney, C Guok, I Monga, "Enabling intent to configure scientific networks for high performance demands", Future Generation Computer Systems, 2018, 79:205--214, doi: 10.1016/j.future.2017.04.020

2017

B Mohammed, S Moyo, KM Maiyama, S Kinteh, ANMK Al-Shaidy, MA Kamala, M Kiran, "Technical Report on Deploying a highly secured OpenStack Cloud Infrastructure using BradStack as a Case Study", 2017,

A Mercian, M Kiran, E Pouyoul, B Tierney, I Monga, "INDIRA: Application Intent network assistant to configure SDN-based high performance scientific networks", Optics InfoBase Conference Papers, 2017, Part F40, doi: 10.1364/OFC.2017.Tu3L.6

M Usman, A Iqbal, M Kiran, "A bandwidth friendly architecture for Cloud Gaming", International Conference on Information Networking, 2017, 179--184, doi: 10.1109/ICOIN.2017.7899500

KM Maiyama, D Kouvatsos, B Mohammed, M Kiran, MA Kamala, "Performance modelling and analysis of an OpenStack IaaS cloud computing platform", Proceedings - 2017 IEEE 5th International Conference on Future Internet of Things and Cloud, FiCloud 2017, 2017, 2017-Jan:198--205, doi: 10.1109/FiCloud.2017.54

T Eftonova, M Kiran, M Stannett, "Long-term Macroeconomic Dynamics of Competition in the Russian Economy using Agent- based Modelling.", IJSDA, 2017, 6:1--20,

B Mohammed, M Kiran, KM Maiyama, MM Kamala, IU Awan, "Failover strategy for fault tolerance in cloud computing environment", Journal of Software - Practice and Experience, 2017, 47:1243--1274, doi: https://onlinelibrary.wiley.com/doi/abs/10.1002/spe.2491

2016

B Mohammed, M Kiran, I-U Awan, KM Maiyama, "An Integrated Virtualized Strategy for Fault Tolerance in Cloud Computing Environment", 2016 Intl IEEE Conferences on Ubiquitous Intelligence & Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People, and Smart World Congress (UIC/ATC/ScalCom/CBDCom/IoP/SmartWorld), 2016, 542--549, doi: 10.1109/UIC-ATC-ScalCom-CBDCom-IoP-SmartWorld.2016.0094

B Mohammed, M Kiran, IU Awan, KM Maiyama, "Optimising fault tolerance in real-time cloud computing IaaS environment", Proceedings - 2016 IEEE 4th International Conference on Future Internet of Things and Cloud, FiCloud 2016, 2016, 363--370, doi: 10.1109/FiCloud.2016.58

M Kiran, A Simons, "Testing Software Services in Cloud Ecosystems", International Journal of Cloud Applications and Computing, 2016, 6:42--58, doi: 10.4018/ijcac.2016010103

2015

B Mohammed, M Kiran, "Analysis of Cloud Test Beds Using OpenSource Solutions", Proceedings - 2015 International Conference on Future Internet of Things and Cloud, FiCloud 2015 and 2015 International Conference on Open and Big Data, OBD 2015, 2015, 195--203, doi: 10.1109/FiCloud.2015.106

2014

Harinarayan Krishnan

2018

S Swaid, M Maat, H Krishnan, D Ghoshal, L Ramakrishnan, "Usability heuristic evaluation of scientific data analysis and visualization tools", Advances in Intelligent Systems and Computing, 2018, 607:471--482, doi: 10.1007/978-3-319-60492-3_45

Dáithí A Stone, Mark D Risser, Oliver M Angélil, Michael F Wehner, Shreyas Cholia, Noel Keen, Harinarayan Krishnan, Travis A O Brien, William D Collins, "A basis set for exploration of sensitivity to prescribed ocean conditions for estimating human contributions to extreme weather in CAM5. 1-1degree", Weather and climate extremes, 2018, 19:10--19,

P Enfedaque, H Chang, H Krishnan, S Marchesini, "GPU-based implementation of ptycho-ADMM for high performance x-ray imaging", Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2018, 10860 LN:540--553, doi: 10.1007/978-3-319-93698-7_41

RJ Pandolfi, DB Allan, E Arenholz, L Barroso-Luque, SI Campbell, TA Caswell, A Blair, F De Carlo, S Fackler, AP Fournier, G Freychet, M Fukuto, D Gürsoy, Z Jiang, H Krishnan, D Kumar, RJ Kline, R Li, C Liman, S Marchesini, A Mehta, AT N Diaye, DY Parkinson, H Parks, LA Pellouchoud, T Perciano, F Ren, S Sahoo, J Strzalka, D Sunday, CJ Tassone, D Ushizima, S Venkatakrishnan, KG Yager, P Zwart, JA Sethian, A Hexemer, "Xi-cam: a versatile interface for data visualization and analysis", Journal of Synchrotron Radiation, 2018, 25:1261--1270, doi: 10.1107/S1600577518005787

C Li, C Michel, L Seland Graff, I Bethke, G Zappa, TJ Bracegirdle, E Fischer, BJ Harvey, T Iversen, MP King, H Krishnan, L Lierhammer, D Mitchell, J Scinocca, H Shiogama, DA Stone, JJ Wettstein, Midlatitude atmospheric circulation responses under 1.5 and 2.0g°C warming and implications for regional impacts, Earth System Dynamics, Pages: 359--382 2018, doi: 10.5194/esd-9-359-2018

M Wehner, D Stone, D Mitchell, H Shiogama, E Fischer, LS Graff, VV Kharin, L Lierhammer, B Sanderson, H Krishnan, Changes in extremely hot days under stabilized 1.5 and 2.0 °c global warming scenarios as simulated by the HAPPI multi-model ensemble, Earth System Dynamics, Pages: 299--311 2018, doi: 10.5194/esd-9-299-2018

MF Wehner, KA Reed, B Loring, D Stone, H Krishnan, "Changes in tropical cyclones under stabilized 1.5 and 2.0°C global warming scenarios as simulated by the Community Atmospheric Model under the HAPPI protocols", Earth System Dynamics, 2018, 9:187--195, doi: 10.5194/esd-9-187-2018

JJ Billings, AR Bennett, J Deyton, K Gammeltoft, J Graham, D Gorin, H Krishnan, M Li, AJ McCaskey, T Patterson, R Smith, GR Watson, A Wojtowicz, The eclipse integrated computational environment, SoftwareX, Pages: 234--244 2018, doi: 10.1016/j.softx.2018.07.004

M Wehner, D Stone, H Shiogama, P Wolski, A Ciavarella, N Christidis, H Krishnan, Early 21st century anthropogenic changes in extremely hot days as simulated by the C20C+ detection and attribution multi-model ensemble, Weather and Climate Extremes, Pages: 1--8 2018, doi: 10.1016/j.wace.2018.03.001

DA Stone, MD Risser, OM Angélil, MF Wehner, S Cholia, N Keen, H Krishnan, TA O Brien, WD Collins, A basis set for exploration of sensitivity to prescribed ocean conditions for estimating human contributions to extreme weather in CAM5.1-1degree, Weather and Climate Extremes, Pages: 10--19 2018, doi: 10.1016/j.wace.2017.12.003

YS Yu, M Farmand, C Kim, Y Liu, CP Grey, FC Strobridge, T Tyliszczak, R Celestre, P Denes, J Joseph, H Krishnan, FRNC Maia, ALD Kilcoyne, S Marchesini, TPC Leite, T Warwick, H Padmore, J Cabana, DA Shapiro, Three-dimensional localization of nanoscale battery reactions using soft X-ray tomography, Nature Communications, 2018, doi: 10.1038/s41467-018-03401-x

DA Shapiro, R Celestre, B Enders, J Joseph, H Krishnan, MA Marcus, K Nowrouzi, H Padmore, J Park, A Warwick, Y-S Yu, The COSMIC Imaging Beamline at the Advanced Light Source: a new facility for spectro-microscopy of nano-materials, Microscopy and Microanalysis, Pages: 8--11 2018, doi: 10.1017/s1431927618012485

B Enders, K Nowrouzi, H Krishnan, S Marchesini, J Park, Y-S Yu, DA Shapiro, Dataflow at the COSMIC Beamline - Stream Processing and Supercomputing, Microscopy and Microanalysis, Pages: 58--59 2018, doi: 10.1017/s1431927618012710

2017

Stone, D. A., H. Krishnan, R. Lance, S. Sippel, and M. F. Wehner, "The First and Second Hackathons of the International CLIVAR C20C+ Detection and Attribution Project", CLIVAR Exchanges, 2017,

Timmermans, B., D. Stone, M. Wehner, and H. Krishnan, "Impact of tropical cyclones on modeled wind-wave climate", Geophysical Research Letters, 2017, 44:1393-1401, doi: 10.1002/2016GL071681

T Perciano, D Ushizima, H Krishnan, D Parkinson, N Larson, DM Pelt, W Bethel, F Zok, J Sethian, "Insight into 3D micro-CT data: Exploring segmentation algorithms through performance metrics", Journal of Synchrotron Radiation, 2017, 24:1065--1077, doi: 10.1107/S1600577517010955

O Angélil, D Stone, M Wehner, CJ Paciorek, H Krishnan, W Collins, "An independent assessment of anthropogenic attribution statements for recent extreme temperature and rainfall events", Journal of Climate, 2017, 30:5--16, doi: 10.1175/JCLI-D-16-0077.1

Benedikt J Daurer, Hari Krishnan, Talita Perciano, Filipe RNC Maia, David A Shapiro, James A Sethian, Stefano Marchesini, "Nanosurveyor: a framework for real-time data processing", Advanced structural and chemical imaging, 2017, 3:7,

T Perciano, D Ushizima, H Krishnan, D Parkinson, J Sethian, "FibriPy: A software environment for fiber analysis from 3D micro-computed tomography data", Advanced Materials - TechConnect Briefs 2017, 2017, 1:25--28,

DY Parkinson, DM Pelt, T Perciano, D Ushizima, H Krishnan, HS Barnard, AA MacDowell, J Sethian, "Machine learning for micro-tomography", Proceedings of SPIE - The International Society for Optical Engineering, 2017, 10391, doi: 10.1117/12.2274731

B Timmermans, D Stone, M Wehner, H Krishnan, Impact of tropical cyclones on modeled extreme wind-wave climate, Geophysical Research Letters, Pages: 1393--1401 2017, doi: 10.1002/2016GL071681

BJ Daurer, H Krishnan, T Perciano, FRNC Maia, DA Shapiro, JA Sethian, S Marchesini, Nanosurveyor: a framework for real-time data processing., Advanced structural and chemical imaging, Pages: 7 2017, doi: 10.1186/s40679-017-0039-0

M Farmand, R Celestre, P Denes, ALD Kilcoyne, S Marchesini, H Padmore, T Tyliszczak, T Warwick, X Shi, J Lee, YS Yu, J Cabana, J Joseph, H Krishnan, T Perciano, FRNC Maia, DA Shapiro, "Near-edge X-ray refraction fine structure microscopy", Applied Physics Letters, 2017, 110, doi: 10.1063/1.4975377

2016

M. Wehner, D. Stone, H. Krishnan, K. AchutaRao, F. Castillo, "The deadly combination of heat and humidity in India and Pakistan in summer 201", Bulletin of the American Meteorological Society, 2016, 97:S81-S86, doi: 10.1175/BAMS-D-16-0145.2

Harinarayan Krishnan, Burlen Loring, Suren Byna, Michael F. Wehner, Travis A. O'Brien, Prabhat, Chris Paciorek, and Daithi Stone, "Enabling End-to-End Climate Science Workflows in High Performance Computing Environments", The AMS (American Meteorological Society) 96th Annual Meeting, January 6, 2016,

Burlen Loring, Suren Byna, Prabhat, Junmin Gu, Hari Krishnan, Michael Wehner, and Oliver Ruebel, "TECA an Extreme Event Detection and Climate Analysis Package for High Performance Computing", The AMS (American Meteorological Society) 96th Annual Meeting, January 6, 2016,

Deborah A Agarwal, Boris Faybishenko, Vicky L Freedman, Harinarayan Krishnan, Gary Kushner, Carina Lansing, Ellen Porter, Alexandru Romosan, Arie Shoshani, Haruko Wainwright, others, "A science data gateway for environmental management", Concurrency and Computation: Practice and Experience, 2016, 28:1994--2004,

DM Ushizima, HA Bale, EW Bethel, P Ercius, BA Helms, H Krishnan, LT Grinberg, M Haranczyk, AA Macdowell, K Odziomek, DY Parkinson, T Perciano, RO Ritchie, C Yang, "IDEAL: Images Across Domains, Experiments, Algorithms and Learning", JOM, 2016, 68:2963--2972, doi: 10.1007/s11837-016-2098-4

SV Venkatakrishnan, KA Mohan, K Beattie, J Correa, E Dart, JR Deslippe, A Hexemer, H Krishnan, AA MacDowell, S Marchesini, SJ Patton, T Perciano, JA Sethian, R Stromsness, BL Tierney, CE Tull, D Ushizima, DY Parkinson, "Making advanced scientific algorithms and big scientific data management more accessible", IS and T International Symposium on Electronic Imaging Science and Technology, 2016, doi: 10.2352/ISSN.2470-1173.2016.19.COIMG-155

DY Parkinson, K Beattie, X Chen, J Correa, E Dart, BJ Daurer, JR Deslippe, A Hexemer, H Krishnan, AA Macdowell, FRNC Maia, S Marchesini, HA Padmore, SJ Patton, T Perciano, JA Sethian, D Shapiro, R Stromsness, N Tamura, BL Tierney, CE Tull, D Ushizima, "Real-time data-intensive computing", AIP Conference Proceedings, 2016, 1741, doi: 10.1063/1.4952921

S Marchesini, H Krishnan, BJ Daurer, DA Shapiro, T Perciano, JA Sethian, FRNC Maia, "SHARP: A distributed GPU-based ptychographic solver", Journal of Applied Crystallography, 2016, 49:1245--1252, doi: 10.1107/S1600576716008074

M Wehner, D Stone, H Krishnan, K Achutarao, F Castillo, 16. The deadly combination of heat and humidity in India and Pakistan in summer 2015, Bulletin of the American Meteorological Society, Pages: S81--S86 2016, doi: 10.1175/BAMS-D-16-0145.1

TA O Brien, WD Collins, K Kashinath, O Rübel, S Byna, J Gu, H Krishnan, PA Ullrich, Resolution dependence of precipitation statistical fidelity in hindcast simulations, Journal of Advances in Modeling Earth Systems, Pages: 976--990 2016, doi: 10.1002/2016MS000671

2015

Stone, D., H. Shiogama, P. Wolski, O. Angélil, S. Cholias, N. Christidis, A. Dittus, C. Folland, A. King, J. Kinter, H. Krishnan, S.-K. Min, M. Wehner, "The C20C+ Detection and Attribution Project", Fall Meeting of the American Geophysical Union, 2015,

Hari Krishnan, Suren Byna, Michael Wehner, Junmin Gu, Travis O'Brien, Burlen Loring, Daithi Stone, William Collins, Prabhat, Yunjie Liu, Jeffrey Johnson, and Christopher Paciorek, "Enabling Efficient Climate Science Workflows in High Performance Computing Environments", AGU Fall Meeting, 2015, December 13, 2015,

J Donatelli, M Haranczyk, A Hexemer, H Krishnan, X Li, L Lin, F Maia, S Marchesini, D Parkinson, T Perciano, D Shapiro, D Ushizima, C Yang, JA Sethian, "CAMERA: The Center for Advanced Mathematics for Energy Research Applications", Synchrotron Radiation News, 2015, 28:4--9, doi: 10.1080/08940886.2015.1013413

Y Li, S Meyer, J Lim, SC Lee, WE Gent, S Marchesini, H Krishnan, T Tyliszczak, D Shapiro, ALD Kilcoyne, WC Chueh, Effects of Particle Size, Electronic Connectivity, and Incoherent Nanoscale Domains on the Sequence of Lithiation in LiFePO<inf>4</inf> Porous Electrodes, Advanced Materials, Pages: 6591--6597 2015, doi: 10.1002/adma.201502276

Yiyang Li, Sophie Meyer, Jongwoo Lim, Sang Chul Lee, William E Gent, Stefano Marchesini, Harinarayan Krishnan, Tolek Tyliszczak, David Shapiro, Arthur L David Kilcoyne, others, Effects of Particle Size, Electronic Connectivity, and Incoherent Nanoscale Domains on the Sequence of Lithiation in LiFePO4 Porous Electrodes, Advanced Materials, Pages: 6591--6597 2015,

Yiyang Li, Sophie Meyer, Jongwoo Lim, Sang Chul Lee, William E Gent, Stefano Marchesini, Harinarayan Krishnan, Tolek Tyliszczak, David Shapiro, Arthur L David Kilcoyne, others, Electrode Lithiation: Effects of Particle Size, Electronic Connectivity, and Incoherent Nanoscale Domains on the Sequence of Lithiation in LiFePO4 Porous Electrodes (Adv. Mater. 42/2015), Advanced Materials, Pages: 6590--6590 2015,

2014

Dáithí Stone, Michael Wehner, Shreyas Cholia, Harinarayan Krishnan, Piotr Wolski, Mark Tadross, Chris Folland, Nikos Christidis, Hideo Shiogama, "The C20C+ Detection and Attribution Project", Integrated Climate Modeling Principal Investigator Meeting 2014, 2014,

D Ushizima, T Perciano, H Krishnan, B Loring, H Bale, D Parkinson, J Sethian, "Structure recognition from high resolution images of ceramic composites", Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014, 2014, 683--691, doi: 10.1109/BigData.2014.7004292

2013

David Camp, Hari Krishnan, David Pugmire, Christoph Garth, Ian Johnson, E. Wes Bethel, Kenneth I. Joy, and Hank Childs., "GPU Acceleration of Particle Advection Workloads in a Parallel, Distributed Memory Setting", Proceedings of Eurographics Symposium on Parallel Graphics and Visualization (EGPGV), May 5, 2013,

DN Williams, T Bremer, C Doutriaux, J Patchett, S Williams, G Shipman, R Miller, DR Pugmire, B Smith, C Steed, EW Bethel, H Childs, H Krishnan, P Prabhat, M Wehner, CT Silva, E Santos, D Koop, T Ellqvist, J Poco, B Geveci, A Chaudhary, A Bauer, A Pletzer, D Kindig, GL Potter, TP Maxwell, Ultrascale visualization of climate data, Computer, Pages: 68--76 2013, doi: 10.1109/MC.2013.119

2012

Hank Childs, Eric Brugger, Brad Whitlock, Jeremy Meredith, Sean Ahern, David Pugmire, Kathleen Biagas, Mark Miller, Cyrus Harrison, Gunther H. Weber, Hari Krishnan, Thomas Fogal, Allen Sanderson, Christoph Garth, E. Wes Bethel, David Camp, Oliver Rubel, Marc Durant, Jean M. Favre, Paul Navratil, "VisIt: An End-User Tool For Visualizing and Analyzing Very Large Data", High Performance Visualization---Enabling Extreme-Scale Scientific Insight, ( October 2012) Pages: 357--372

Tamay M. Ozgokmen, Andrew C. Poje, Paul F. Fischer, Hank Childs, Harinarayan Krishnan, Christoph Garth, Angelique C. Haza, Edward Ryan, "On multi-scale dispersion under the influence of surface mixed layer instabilities", Ocean Modelling, October 2012, 56:16-30,

E. Wes Bethel, David Camp, Hank Childs, Mark Howison, Hari Krishnan, Burlen Loring, Joerg Meyer, Prabhat, Oliver Ruebel, Daniela Ushizima, Gunther Weber, "Towards Exascale: High Performance Visualization and Analytics – Project Status Report. Technical Report", DOE Exascale Research Conference, April 2012,

J Meyer, EW Bethel, JL Horsman, SS Hubbard, H Krishnan, A Romosan, EH Keating, L Monroe, R Strelitz, P Moore, G Taylor, B Torkian, TC Johnson, I Gorton, Visual data analysis as an integral part of environmental management, IEEE Transactions on Visualization and Computer Graphics, Pages: 2088--2094 2012, doi: 10.1109/TVCG.2012.278

H Krishnan, J Meyer, A Romosan, H Childs, EW Bethel, Enabling advanced environmental management via remote and distributed visual data exploration and analysis, Computing and Visualization in Science, Pages: 123--133 2012, doi: 10.1007/s00791-013-0204-5

TM Özgökmen, AC Poje, PF Fischer, H Childs, H Krishnan, C Garth, AC Haza, E Ryan, On multi-scale dispersion under the influence of surface mixed layer instabilities and deep flows, Ocean Modelling, Pages: 16--30 2012, doi: 10.1016/j.ocemod.2012.07.004

H Krishnan, C Garth, J Gühring, MA Gülsün, A Greiser, KI Joy, Analysis of time-dependent flow-sensitive PC-MRI data, IEEE Transactions on Visualization and Computer Graphics, Pages: 966--977 2012, doi: 10.1109/TVCG.2011.80

2009

H Krishnan, C Garth, KI Joy, Time and streak surfaces for flow visualization in large time-varying data sets, IEEE Transactions on Visualization and Computer Graphics, Pages: 1267--1274 2009, doi: 10.1109/TVCG.2009.190

2008

C Garth, H Krishnan, X Tricoche, T Bobach, KI Joy, Generation of accurate integral surfaces in time-dependent vector fields, IEEE Transactions on Visualization and Computer Graphics, Pages: 1404--1411 2008, doi: 10.1109/TVCG.2008.133

2007

AR Fuller, H Krishnan, K Mahrous, B Hamann, KI Joy, Real-time procedural volumetric fire, Proceedings - I3D 2007, ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, Pages: 175--180 2007, doi: 10.1145/1230100.1230131

Burlen Loring

2022

E. Wes Bethel, Burlen Loring, Utkarsh Ayachit, P. N. Duque, Nicola Ferrier, Joseph Insley, Junmin Gu, Kress, Patrick O’Leary, Dave Pugmire, Silvio Rizzi, Thompson, Will Usher, Gunther H. Weber, Brad Whitlock, Wolf, Kesheng Wu, "Proximity Portability and In Transit, M-to-N Data Partitioning and Movement in SENSEI", In Situ Visualization for Computational Science, ( 2022) doi: 10.1007/978-3-030-81627-8_20

E. Wes Bethel, Burlen Loring, Utkarsh Ayatchit, David Camp, P. N. Duque, Nicola Ferrier, Joseph Insley, Junmin Gu, Kress, Patrick O’Leary, David Pugmire, Silvio Rizzi, Thompson, Gunther H. Weber, Brad Whitlock, Matthew Wolf, Kesheng Wu, "The SENSEI Generic In Situ Interface: Tool and Processing Portability at Scale", In Situ Visualization for Computational Science, ( 2022) doi: 10.1007/978-3-030-81627-8_13

2019

Junmin Gu, Burlen Loring, Kesheng Wu, E Wes Bethel, "HDF5 as a vehicle for in transit data movement", Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization, 2019, 39--43,

2018

B Loring, A Myers, D Camp, EW Bethel, "Python-based in situ analysis and visualization", Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization - ISAV 18, ACM Press, 2018, doi: 10.1145/3281464.3281465

MF Wehner, KA Reed, B Loring, D Stone, H Krishnan, "Changes in tropical cyclones under stabilized 1.5 and 2.0°C global warming scenarios as simulated by the Community Atmospheric Model under the HAPPI protocols", Earth System Dynamics, 2018, 9:187--195, doi: 10.5194/esd-9-187-2018

2017

U Ayachit, B Whitlock, M Wolf, B Loring, B Geveci, D Lonie, EW Bethel, "The SENSEI generic in situ interface", Proceedings of ISAV 2016: 2nd Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis, 2017, 40--44, doi: 10.1109/ISAV.2016.13

2016

Harinarayan Krishnan, Burlen Loring, Suren Byna, Michael F. Wehner, Travis A. O'Brien, Prabhat, Chris Paciorek, and Daithi Stone, "Enabling End-to-End Climate Science Workflows in High Performance Computing Environments", The AMS (American Meteorological Society) 96th Annual Meeting, January 6, 2016,

Burlen Loring, Suren Byna, Prabhat, Junmin Gu, Hari Krishnan, Michael Wehner, and Oliver Ruebel, "TECA an Extreme Event Detection and Climate Analysis Package for High Performance Computing", The AMS (American Meteorological Society) 96th Annual Meeting, January 6, 2016,

Utkarsh Ayachit, Andrew Bauer, Earl PN Duque, Greg Eisenhauer, Nicola Ferrier, Junmin Gu, Kenneth E Jansen, Burlen Loring, Zarija Lukic, Suresh Menon, others, "Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures", SC 16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2016, 921--932, LBNL 1007264,

O Rübel, B Loring, JL Vay, DP Grote, R Lehe, S Bulanov, H Vincenti, EW Bethel, "WarpIV: In Situ Visualization and Analysis of Ion Accelerator Simulations", IEEE Computer Graphics and Applications, 2016, 36:22--35, doi: 10.1109/MCG.2016.62

M Alegro, E Amaro, B Loring, H Heinsen, E Alho, L Zollei, D Ushizima, LT Grinberg, "Multimodal Whole Brain Registration: MRI and High Resolution Histology", IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2016, 634--642, doi: 10.1109/CVPRW.2016.85

2015

Hari Krishnan, Suren Byna, Michael Wehner, Junmin Gu, Travis O'Brien, Burlen Loring, Daithi Stone, William Collins, Prabhat, Yunjie Liu, Jeffrey Johnson, and Christopher Paciorek, "Enabling Efficient Climate Science Workflows in High Performance Computing Environments", AGU Fall Meeting, 2015, December 13, 2015,

B Loring, H Karimabadi, V Rortershteyn, "A Screen Space GPGPU Surface LIC Algorithm for Distributed Memory Data Parallel Sort Last Rendering Infrastructures", NUMERICAL MODELING OF SPACE PLASMA FLOWS: ASTRONUM-2014, 2015, 498:231--239,

2014

D Ushizima, T Perciano, H Krishnan, B Loring, H Bale, D Parkinson, J Sethian, "Structure recognition from high resolution images of ceramic composites", Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014, 2014, 683--691, doi: 10.1109/BigData.2014.7004292

H Karimabadi, V Roytershteyn, HX Vu, YA Omelchenko, J Scudder, W Daughton, A Dimmock, K Nykyri, M Wan, D Sibeck, M Tatineni, A Majumdar, B Loring, B Geveci, "The link between shocks, turbulence, and magnetic reconnection in collisionless plasmas", Physics of Plasmas, 2014, 21, doi: 10.1063/1.4882875

W Daughton, TKM Nakamura, H Karimabadi, V Roytershteyn, B Loring, "Computing the reconnection rate in turbulent kinetic layers by using electron mixing to identify topology", Physics of Plasmas, 2014, 21, doi: 10.1063/1.4875730

2013

H Karimabadi, B Loring, P O leary, A Majumdar, M Tatineni, B Geveci, "In-situ visualization for global hybrid simulations", ACM International Conference Proceeding Series, 2013, doi: 10.1145/2484762.2484822

H Karimabadi, V Roytershteyn, M Wan, WH Matthaeus, W Daughton, P Wu, M Shay, B Loring, J Borovsky, E Leonardis, SC Chapman, TKM Nakamura, "Coherent structures, intermittent turbulence, and dissipation in high-temperature plasmas", Physics of Plasmas, 2013, 20, doi: 10.1063/1.4773205

2012

Mehmet Balman, Eric Pouyoul, Yushu Yao, E. Wes Bethel, Burlen Loring, Prabhat, John Shalf, Alex Sim, and Brian L. Tierney, "Experiences with 100G Network Applications", In Proceedings of the Fifth international Workshop on Data-intensive Distributed Computing, in conjunction with ACM High Performance Distributing Computing (HPDC) Conference, 2012, Delft, Netherlands, June 2012, LBNL 5603E, doi: 10.1145/2286996.2287004

100Gbps networking has finally arrived, and many research and educational in- stitutions have begun to deploy 100Gbps routers and services. ESnet and Internet2 worked together to make 100Gbps networks available to researchers at the Super- computing 2011 conference in Seattle Washington. In this paper, we describe two of the first applications to take advantage of this network. We demonstrate a visu- alization application that enables remotely located scientists to gain insights from large datasets. We also demonstrate climate data movement and analysis over the 100Gbps network. We describe a number of application design issues and host tuning strategies necessary for enabling applications to scale to 100Gbps rates. 

E. Wes Bethel, David Camp, Hank Childs, Mark Howison, Hari Krishnan, Burlen Loring, Joerg Meyer, Prabhat, Oliver Ruebel, Daniela Ushizima, Gunther Weber, "Towards Exascale: High Performance Visualization and Analytics – Project Status Report. Technical Report", DOE Exascale Research Conference, April 2012,

M Balman, E Pouyoul, Y Yao, EW Bethel, B Loring, Prabhat, J Shalf, A Sim, BL Tierney, "Experiences with 100Gbps network applications", DIDC 12 - 5th International Workshop on Data-Intensive Distributed Computing, 2012, 33--42, doi: 10.1145/2286996.2287004

M Wan, WH Matthaeus, H Karimabadi, V Roytershteyn, M Shay, P Wu, W Daughton, B Loring, SC Chapman, "Intermittent dissipation at kinetic scales in collisionless plasma turbulence", Physical Review Letters, 2012, 109, doi: 10.1103/PhysRevLett.109.195001

2011

H Karimabadi, B Loring, HX Vu, Y Omelchenko, M Tatineni, A Majumdar, U Ayachit, B Geveci, "Petascale Global Kinetic Simulations of The Magnetosphere and Visualization Strategies for Analysis of Very Large Multi-Variate Data Sets", NUMERICAL MODELING OF SPACE PLASMA FLOWS - ASTRONUM 2010, 2011, 444:281--+,

2010

H Karimabadi, J Dorelli, HX Vu, B Loring, Y Omelchenko, Is Quadrupole Structure of Out-of-Plane Magnetic Field Evidence for Hall Reconnection?, MODERN CHALLENGES IN NONLINEAR PLASMA PHYSICS: A FESTSCHRIFT HONORING THE CAREER OF DENNIS PAPADOPOULOS, Pages: 137--+ 2010,

Dmitriy Morozov

2018

K Beketayev, D Yeliussizov, D Morozov, GH Weber, B Hamann, "Measuring the Error in Approximating the Sub-Level Set Topology of Sampled Scalar Data", International Journal of Computational Geometry and Applications, 2018, 28:57--77, doi: 10.1142/S0218195918500036

Erich Lohrmann, Zarija Lukic, Dmitriy Morozov, Juliane Mueller, "Programmable In Situ System for Iterative Workflows", Lecture Notes in Computer Science, Cham, Springer International Publishing, January 1, 2018, 10773:122--131, doi: 10.1007/978-3-319-77398-8\_7

P Koanantakool, A Ali, A Azad, A Buluç, D Morozov, L Oliker, KA Yelick, S-Y Oh, "Communication-Avoiding Optimization Methods for Distributed Massive-Scale Sparse Inverse Covariance Estimation.", Proceedings of Machine Learning Research, PMLR, 2018, 84:1376--1386,

Andrey Babichev, Dmitriy Morozov, Yuri Dabaghian, "Robust spatial memory maps encoded by networks with transient", PLoS computational biology, 2018, 14:e1006433, doi: https://doi.org/10.1371/journal.pcbi.1006433

2017

P Oesterling, C Heine, GH Weber, D Morozov, G Scheuermann, "Computing and visualizing time-varying merge trees for high-dimensional data", Mathematics and Visualization, ( 2017) Pages: 87--101 doi: 10.1007/978-3-319-44684-4_5

Michael Kerber, Dmitriy Morozov, Arnur Nigmetov, "Geometry Helps to Compare Persistence Diagrams", Journal of Experimental Algorithmics, 2017, 22:1.4:1--1.4, doi: 10.1145/3064175

Dmitriy Smirnov, Dmitriy Morozov, Triplet Merge Trees, 2017,

Herbert Edelsbrunner, Dmitriy Morozov, "Persistent Homology", Handbook of Discrete and Computational Geometry, 3rd ed., (CRC Press: 2017)

2016

Brian Friesen, Ann Almgren, Zarija Lukić, Gunther Weber, Dmitriy Morozov, Vincent Beckner, Marcus Day, "In situ and in-transit analysis of cosmological simulations", Computational Astrophysics and Cosmology, 2016, 3 (4):1-18,

Utkarsh Ayachit, Andrew Bauer, Earl PN Duque, Greg Eisenhauer, Nicola Ferrier, Junmin Gu, Kenneth E Jansen, Burlen Loring, Zarija Lukic, Suresh Menon, others, "Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures", SC 16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2016, 921--932, LBNL 1007264,

P Koanantakool, A Azad, A Buluc, D Morozov, SY Oh, L Oliker, K Yelick, "Communication-Avoiding Parallel Sparse-Dense Matrix-Matrix Multiplication", Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016, January 2016, 842--853, doi: 10.1109/IPDPS.2016.117

Dmitriy Morozov, Tom Peterka, "Block-parallel data analysis with DIY2", Large Data Analysis and Visualization (LDAV), 2016 IEE Symposium on, 2016, 29--36,

D Morozov, Z Lukić, "Master of puppets: Cooperative multitasking for in situ processing", HPDC 2016 - Proceedings of the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing, 2016, 285--288, doi: 10.1145/2907294.2907301

Dmitriy Morozov, Tom Peterka, "Efficient Delaunay tessellation through K-D tree decomposition", SC 16, Piscataway, NJ, USA, IEEE Press, 2016, 62,

Gunnar Carlsson, Vin de Silva, Sara Verovsek, Dmitriy Morozov, Parametrized Homology via Zigzag Persistence, 2016,

2015

R Lewis, D Morozov, "Parallel computation of persistent homology using the blowup complex", Annual ACM Symposium on Parallelism in Algorithms and Architectures, 2015, 2015-Jun:323--331, doi: 10.1145/2755573.2755587

2014

K Beketayev, D Yeliussizov, D Morozov, GH Weber, B Hamann, "Measuring the distance between merge trees", Mathematics and Visualization, ( 2014) Pages: 151--165 doi: 10.1007/978-3-319-04099-8_10

Dmitriy Morozov, Gunther H Weber, "Distributed Contour Trees", Topological Methods in Data Analysis and Visualization III, (Springer International Publishing: 2014) Pages: 89--102 doi: 10.1007/978-3-319-04099-8\_6

T Peterka, D Morozov, C Phillips, "High-Performance Computation of Distributed-Memory Parallel 3D Voronoi and Delaunay Tessellation", International Conference for High Performance Computing, Networking, Storage and Analysis, SC, 2014, 2015-Jan:997--1007, doi: 10.1109/SC.2014.86

2013

Dmitriy Morozov, Gunther Weber, "Distributed Merge Trees", PPoPP 13, New York, NY, USA, ACM, 2013, 93--102, doi: 10.1145/2442516.2442526

Leonidas Guibas, Dmitriy Morozov, Quentin M\ erigot, "Witnessed k-Distance", Discrete \& computational geometry, 2013, 49:22--45, doi: 10.1007/s00454-012-9465-x

Herbert Edelsbrunner, Dmitriy Morozov, "Persistent Homology: Theory and Practice", European Congress of Mathematics Krakow, 2 -- 7 July, 2012, Zuerich, Switzerland, European Mathematical Society Publishing House, 2013, 31--50, doi: 10.4171/120-1/3

Paul Bendich, Herbert Edelsbrunner, Dmitriy Morozov, Amit, "Homology and robustness of level and interlevel sets", Homology, Homotopy and Applications, 2013, 15:51--72,

2012

D Ushizima, D Morozov, GH Weber, AGC Bianchi, JA Sethian, EW Bethel, "Augmented topological descriptors of pore networks for material science", IEEE Transactions on Visualization and Computer Graphics, 2012, 18:2041--2050, LBNL 5964E, doi: 10.1109/TVCG.2012.200

K Beketayev, GH Weber, D Morozov, A Abzhanov, B Hamann, "Geometry-preserving topological landscapes", Proceedings - WASA 2012: Workshop at SIGGRAPH Asia 2012, 2012, 155--160, doi: 10.1145/2425296.2425324

M Aanjaneya, F Chazal, D Chen, M Glisse, L Guibas, D Morozov, "Metric graph reconstruction from noisy data", International Journal of Computational Geometry and Applications, 2012, 22:305--325, doi: 10.1142/S0218195912600072

Fianna O'Brien

2018

You-Wei Cheah, Danielle Svehla Christianson, Housen Chu, Gilberto Pastorello, Fianna O’Brien, Yeongshnn Ong, Catharine van Ingen, Margaret Torn, Deb Agarwal, AmeriFlux BADM: Implementing lessons from 12 years of long-tail data management into next generation earth science systems (IN34A-03), 2018 AGU Fall Meeting, Washington, D.C., December 12, 2018,

2017

Gilberto Z. Pastorello, Dan K. Gunter, Housen Chu, Danielle S. Christianson, Carlo Trotta, Eleonora Canfora, Boris Faybishenko, You-Wei Cheah, Norm Beekwilder, Stephen W. Chan, Sigrid Dengel, Trevor Keenan, Fianna O Brien, Abderahman Elbashandy, Cristina M. Poindexter, Marty Humphrey, Dario Papale, Deb A. Agarwal, "Hunting Data Rogues at Scale: Data Quality Control for Observational Data in Research Infrastructures", Proceedings of the 13th IEEE International Conference on e-Science (e-Science 2017), Auckland, New Zealand, 2017, doi: 10.1109/eScience.2017.64

Sang-Yun Oh

2018

P Koanantakool, A Ali, A Azad, A Buluç, D Morozov, L Oliker, KA Yelick, S-Y Oh, "Communication-Avoiding Optimization Methods for Distributed Massive-Scale Sparse Inverse Covariance Estimation.", Proceedings of Machine Learning Research, PMLR, 2018, 84:1376--1386,

2016

P Koanantakool, A Azad, A Buluc, D Morozov, SY Oh, L Oliker, K Yelick, "Communication-Avoiding Parallel Sparse-Dense Matrix-Matrix Multiplication", Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016, January 2016, 842--853, doi: 10.1109/IPDPS.2016.117

2015

Sang-Yun Oh, Bala Rajaratnam, Joong-Ho Won, "On the Solution Path of Regularized Covariance Estimators", (submitted), 2015,

2014

Sang-Yun Oh, Onkar Dalal, Kshitij Khare, Bala Rajaratnam, "Optimization Methods for Sparse Pseudo-Likelihood Graphical Model Selection", Neural Information Processing Systems, 2014,

Kshitij Khare, Sang-Yun Oh, Bala Rajaratnam, "A convex pseudo-likelihood framework for high dimensional partial correlation estimation with convergence guarantees", Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2014,

Schizophrenia Working Group of the Psychiatric Genomics Consortium, "Biological insights from 108 schizophrenia-associated genetic loci", July 24, 2014, 511:421-427, doi: doi:10.1038/nature13595

2012

Douglas F Levinson, Jianxin Shi, Kai Wang, Sang Oh, Brien Riley, Ann E Pulver, Dieter B Wildenauer, Claudine Laurent, Bryan J Mowry, Pablo V Gejman, Michael J Owen, Kenneth S Kendler, Gerald Nestadt, Sibylle G Schwab, Jacques Mallet, Deborah Nertney, Alan R Sanders, Nigel M Williams, Brandon Wormley, Virginia K Lasseter, Margot Albus, Stephanie Godard-Bauché, Madeline Alexander, Jubao Duan, Michael C O’Donovan, Dermot Walsh, Anthony O’Neill, George N Papadimitriou, Dimitris Dikeos, Wolfgang Maier, Bernard Lerer, Dominique Campion, David Cohen, Maurice Jay, Ayman Fanous, Peter Eichhammer, Jeremy M Silverman, Nadine Norton, Nancy Zhang, Hakon Hakonarson, Cynthia Gao, Ami Citri, Mark Hansen, Stephan Ripke, Frank Dudbridge, Peter A Holmans, "Genome-wide association study of multiplex schizophrenia pedigrees", The American Journal of Psychiatry, September 1, 2012, doi: doi:10.1176/appi.ajp.2012.11091423

2011

Douglas F Levinson, Jubao Duan, Sang Oh, Kai Wang, Alan R Sanders, Jianxin Shi, Nancy Zhang, Bryan J Mowry, Ann Olincy, Farooq Amin, C Robert Cloninger, Jeremy M Silverman, Nancy G Buccola, William F Byerley, Donald W Black, Kenneth S Kendler, Robert Freedman, Frank Dudbridge, Itsik Pe'er, Hakon Hakonarson, Sarah E Bergen, Ayman H Fanous, Peter A Holmans, Pablo V Gejman, "Copy number variants in schizophrenia: confirmation of five previous findings and new evidence for 3q29 microdeletions and VIPR2 duplications", The American Journal of Psychiatry, March 1, 2011, doi: doi:10.1176/appi.ajp.2010.10060876

2005

Bryan W. Reutter, Sang Oh, Grant T. Gullberg, Ronald H. Huesman, "Improved quantitation of dynamic SPECT via fully 4-D joint estimation of compartmental models and blood input function directly from projections", Nuclear Science Symposium Conference Record, 2005 IEEE, October 23, 2005, doi: 10.1109/NSSMIC.2005.1596802

2002

R. Stompor, A. Balbi, J.D. Borrill, P.G. Ferreira, S. Hanany, A.H. Jaffe, A.T. Lee, S. Oh, B. Rabii, P.L. Richards, G.F. Smoot, C.D. Winant, J.-H.P. Wu, "Making maps of the cosmic microwave background: The MAXIMA example", Physical Review D, January 2002, 65:022003, doi: 10.1103/PhysRevD.65.022003

2001

A.H. Jaffe, P.A.R. Ade, A. Balbi, J.J. Bock, J.R. Bond, J. Borrill, A. Boscaleri, K. Coble, B.P. Crill, P. De Bernardis, P. Farese, P.G. Ferreira, K. Ganga, M. Giacometti, S. Hanany, E. Hivon, V.V. Hristov, A. Iacoangeli, A.E. Lange, A.T. Lee, L. Martinis, S. Masi, P.D. Mauskopf, A. Melchiorri, T. Montroy, C.B. Netterfield, S. Oh, E. Pascale, F. Piacentini, D. Pogosyan, S. Prunet, B. Rabii, S. Rao, P.L. Richards, G. Romeo, J.E. Ruhl, F. Scaramuzzi, D. Sforna, G.F. Smoot, R. Stompor, C.D. Winant, J.H.P. Wu, "Cosmology from MAXIMA-1, BOOMERANG, and COBE DMR cosmic microwave background observations", Physical Review Letters, January 2001, 86:3475-3479, doi: 10.1103/PhysRevLett.86.3475

A. Balbi, P. Ade, J. Bock, J. Borrill, A., P. De Bernardis, P. G. Ferreira, S., V. Hristov, A. H. Jaffe, A. T. Lee, S., E. Pascale, B. Rabii, P. L. Richards, G. F., R. Stompor, C. D. Winant, J. H. P. Wu, "Erratum: Constraints on Cosmological Parameters from MAXIMA-1", Astrophysical Journal Letters, January 2001, 558:L145-L145, doi: 10.1086/323608

J.H.P. Wu, A. Balbi, J. Borrill, P.G. Ferreira, S. Hanany, A.H. Jaffe, A.T. Lee, S. Oh, B. Rabii, P.L. Richards, G.F. Smoot, R. Stompor, C.D. Winant, "Asymmetric beams in cosmic microwave background anisotropy experiments", Astrophysical Journal, Supplement Series, January 2001, 132:1-17, doi: 10.1086/318947

2000

S. Hanany, P. Ade, A. Balbi, J. Bock, J., A. Boscaleri, P. de Bernardis, P. G., V. V. Hristov, A. H. Jaffe, A. E., A. T. Lee, P. D. Mauskopf, C. B. Netterfield, S., E. Pascale, B. Rabii, P. L. Richards, G. F., R. Stompor, C. D. Winant, J. H. P. Wu, "MAXIMA-1: A Measurement of the Cosmic Microwave Background Anisotropy on Angular Scales of 10 -5\deg", Astrophysical Journal Letters, December 2000, 545:L5-L9, doi: 10.1086/317322

A. Balbi, P. Ade, J. Bock, J. Borrill, A. Boscaleri, P. De Bernardis, P.G. Ferreira, S. Hanany, V. Hristov, A.H. Jaffe, A.T. Lee, S. Oh, E. Pascale, B. Rabii, P.L. Richards, G.F. Smoot, R. Stompor, C.D. Winant, J.H.P. Wu, "Constraints on cosmological parameters from MAXIMA-1", Astrophysical Journal, January 2000, 545:L1-L4, doi: 10.1086/317323

1969

Bala Rajaratnam, Sang-Yun Oh, Michael T. Tsiang, Richard A. Olshen, "Successive Standardization: Application to Case-Control Studies", Topics in Applied Statistics, ( December 31, 1969) doi: 10.1007/978-1-4614-7846-1_19

Drew Paine

2021

Devarshi Ghoshal, Ludovico Bianchi, Abdelilah Essiari, Drew Paine, Sarah Poon, Michael Beach, Alpha N'Diaye, Patrick Huck, Lavanya Ramakrishnan, "Science Capsule: Towards Sharing and Reproducibility of Scientific Workflows", 2021 IEEE Workshop on Workflows in Support of Large-Scale Science (WORKS), November 15, 2021, doi: 10.1109/WORKS54523.2021.00014

Workflows are increasingly processing large volumes of data from scientific instruments, experiments and sensors. These workflows often consist of complex data processing and analysis steps that might include a diverse ecosystem of tools and also often involve human-in-the-loop steps. Sharing and reproducing these workflows with collaborators and the larger community is critical but hard to do without the entire context of the workflow including user notes and execution environment. In this paper, we describe Science Capsule, which is a framework to capture, share, and reproduce scientific workflows. Science Capsule captures, manages and represents both computational and human elements of a workflow. It automatically captures and processes events associated with the execution and data life cycle of workflows, and lets users add other types and forms of scientific artifacts. Science Capsule also allows users to create `workflow snapshots' that keep track of the different versions of a workflow and their lineage, allowing scientists to incrementally share and extend workflows between users. Our results show that Science Capsule is capable of processing and organizing events in near real-time for high-throughput experimental and data analysis workflows without incurring any significant performance overheads.

Drew Paine, Sarah Poon, Lavanya Ramakrishnan, "Investigating User Experiences with Data Abstractions on High Performance Computing Systems", June 29, 2021, LBNL LBNL-2001374,

Scientific exploration generates expanding volumes of data that commonly require High Performance Computing (HPC) systems to facilitate research. HPC systems are complex ecosystems of hardware and software that frequently are not user friendly. The Usable Data Abstractions (UDA) project set out to build usable software for scientific workflows in HPC environments by undertaking multiple rounds of qualitative user research. Qualitative research investigates how individuals accomplish their work and our interview-based study surfaced a variety of insights about the experiences of working in and with HPC ecosystems. This report examines multiple facets to the experiences of scientists and developers using and supporting HPC systems. We discuss how stakeholders grasp the design and configuration of these systems, the impacts of abstraction layers on their ability to successfully do work, and the varied perceptions of time that shape this work. Examining the adoption of the Cori HPC at NERSC we explore the anticipations and lived experiences of users interacting with this system's novel storage feature, the Burst Buffer. We present lessons learned from across these insights to illustrate just some of the challenges HPC facilities and their stakeholders need to account for when procuring and supporting these essential scientific resources to ensure their usability and utility to a variety of scientific practices.

Devarshi Ghoshal, Drew Paine, Gilberto Pastorello, Abdelrahman Elbashandy, Dan Gunter, Oluwamayowa Amusat, Lavanya Ramakrishnan, "Experiences with Reproducibility: Case Studies from Scientific Workflows", (P-RECS'21) Proceedings of the 4th International Workshop on Practical Reproducible Evaluation of Computer Systems, ACM, June 21, 2021, doi: 10.1145/3456287.3465478

Reproducible research is becoming essential for science to ensure transparency and for building trust. Additionally, reproducibility provides the cornerstone for sharing of methodology that can improve efficiency. Although several tools and studies focus on computational reproducibility, we need a better understanding about the gaps, issues, and challenges for enabling reproducibility of scientific results beyond the computational stages of a scientific pipeline. In this paper, we present five different case studies that highlight the reproducibility needs and challenges under various system and environmental conditions. Through the case studies, we present our experiences in reproducing different types of data and methods that exist in an experimental or analysis pipeline. We examine the human aspects of reproducibility while highlighting the things that worked, that did not work, and that could have worked better for each of the cases. Our experiences capture a wide range of scenarios and are applicable to a much broader audience who aim to integrate reproducibility in their everyday pipelines.

Devarshi Ghoshal, Ludovico Bianchi, Abdelilah Essiari, Michael Beach, Drew Paine, Lavanya Ramakrishnan, "Science Capsule - Capturing the Data Life Cycle", Journal of Open Source Software, 2021, 6:2484, doi: 10.21105/joss.02484

Marco Pritoni, Drew Paine, Gabriel Fierro, Cory Mosiman, Michael Poplawski, Joel Bender, Jessica Granderson, "Metadata Schemas and Ontologies for Building Energy Applications: A Critical Review and Use Case Analysis", Energies, April 6, 2021, doi: 10.3390/en14072024

Digital and intelligent buildings are critical to realizing efficient building energy operations and a smart grid. With the increasing digitalization of processes throughout the life cycle of buildings, data exchanged between stakeholders and between building systems have grown significantly. However, a lack of semantic interoperability between data in different systems is still prevalent and hinders the development of energy-oriented applications that can be reused across buildings, limiting the scalability of innovative solutions. Addressing this challenge, our review paper systematically reviews metadata schemas and ontologies that are at the foundation of semantic interoperability necessary to move toward improved building energy operations. The review finds 40 schemas that span different phases of the building life cycle, most of which cover commercial building operations and, in particular, control and monitoring systems. The paper’s deeper review and analysis of five popular schemas identify several gaps in their ability to fully facilitate the work of a building modeler attempting to support three use cases: energy audits, automated fault detection and diagnosis, and optimal control. Our findings demonstrate that building modelers focused on energy use cases will find it difficult, labor intensive, and costly to create, sustain, and use semantic models with existing ontologies. This underscores the significant work still to be done to enable interoperable, usable, and maintainable building models. We make three recommendations for future work by the building modeling and energy communities: a centralized repository with a search engine for relevant schemas, the development of more use cases, and better harmonization and standardization of schemas in collaboration with industry to facilitate their adoption by stakeholders addressing varied energy-focused use cases.

2020

Drew Paine, Lavanya Ramakrishnan, "Understanding Interactive and Reproducible Computing With Jupyter Tools at Facilities", LBNL Technical Report, October 31, 2020, LBNL LBNL-2001355,

Increasingly Jupyter tools are being adopted and incorporated into High Performance Computing (HPC) and scientific user facilities. Adopting Jupyter tools enables more interactive and reproducible computational work at facilities across data life cycles. As the volume, variety, and scope of data grow, scientists need to be able to analyze and share results in user friendly ways. Human-centered research highlights design challenges around computational notebooks, and our qualitative user study shifts focus to better characterize how Jupyter tools are being used in HPC and science user facilities today. We conducted twenty-nine interviews, and obtained 103 survey responses from NERSC Jupyter users, to better understand the increasing role of interactive computing tools in DOE sponsored scientific work. We examine a range of issues that emerge using and supporting Jupyter in HPC ecosystems, including: how Jupyter is being used by scientists in HPC and user facility ecosystems; how facilities are purposefully supporting Jupyter in their ecosystems; feedback NERSC users have about the facility’s deployment, and, discuss features NERSC indicated would be helpful. We offer a variety of takeaways for staff supporting Jupyter at facilities, Project Jupyter and related open source communities, and funding agencies supporting interactive computing work.

Drew Paine, Devarshi Ghoshal, Lavanya Ramakrishnan, "Experiences with a Flexible User Research Process to Build Data Change Tools", Journal of Open Research Software, September 1, 2020, doi: 10.5334/jors.284

Scientific software development processes are understood to be distinct from commercial software development practices due to uncertain and evolving states of scientific knowledge. Sustaining these software products is a recognized challenge, but under-examined is the usability and usefulness of such tools to their scientific end users. User research is a well-established set of techniques (e.g., interviews, mockups, usability tests) applied in commercial software projects to develop foundational, generative, and evaluative insights about products and the people who use them. Currently these approaches are not commonly applied and discussed in scientific software development work. The use of user research techniques in scientific environments can be challenging due to the nascent, fluid problem spaces of scientific work, varying scope of projects and their user communities, and funding/economic constraints on projects.

In this paper, we reflect on our experiences undertaking a multi-method user research process in the Deduce project. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists make decisions around data change. There is a lack of common terminology since the concept of systematically measuring and managing data change is under explored in scientific environments. To bridge this gap we conducted user research that focuses on user practices, needs, and motivations to help us design and develop metrics and tools for data change. This paper contributes reflections and the lessons we have learned from our experiences. We offer key takeaways for scientific software project teams to effectively and flexibly incorporate similar processes into their projects.

Drew Paine, Devarshi Ghoshal, Lavanya Ramakrishnan, "Investigating Scientific Data Change with User Research Methods", August 20, 2020, LBNL LBNL-2001347,

Scientific datasets are continually expanding and changing due to fluctuations with instruments, quality assessment and quality control processes, and modifications to software pipelines. Datasets include minimal information about these changes or their effects requiring scientists manually assess modifications through a number of labor intensive and ad-hoc steps. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists systematically identify and make decisions around data changes. Currently, there is a lack of understanding, and common practices, for identifying and evaluating changes in datasets since systematically measuring and managing data change is under explored in scientific work. We are conducting user research to address this need by exploring scientist's conceptualizations, behaviors, needs, and motivations when dealing with changing datasets. Our user research utilizes multiple methods to produce foundational, generative insights and evaluate research products produced by our team. In this paper, we detail our user research process and outline our findings about data change that emerge from our studies. Our work illustrates how scientific software teams can push beyond just usability testing user interfaces or tools to better probe the underlying ideas they are developing solutions to address.

Christine T. Wolf, Drew Paine, "Sensemaking Practices in the Everyday Work of AI/ML Software Engineering", IEEE/ACM 42nd International Conference on Software Engineering Workshops (ICSEW’20), ACM, April 5, 2020, doi: 10.1145/3387940.3391496

Drew Paine, Charlotte P. Lee, "Coordinative Entities: Forms of Organizing in Data Intensive Science", Journal of Computer Supported Cooperative Work, February 11, 2020, doi: 10.1007/s10606-020-09372-2

2019

Christine T. Wolf, Julia Bullard, Stacy Wood, Amelia Acker, Drew Paine, Charlotte P. Lee, "Mapping the “How” of Collaborative Action: Research Methods for Studying Contemporary Sociotechnical Processes", CSCW '19: Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing, November 10, 2019, doi: 10.1145/3311957.3359441

Process has been a concern since the beginning of CSCW. Developments in sociotechnical landscapes raise new challenges for studying processes (e.g., massive online communities bringing together vast crowds; Big Data technologies connecting many through the flow of data). This re-opens questions about how we study, document, conceptualize, and design to support processes in complex, contemporary sociotechnical systems. This one-day workshop will bring together researchers to discuss the CSCW community’s unique focus and methodological toolkit for studying process and workflow; provide a collaborative space for the improvement and extension of research projects within this space; and catalyze a network of scholars with expertise and interest in addressing challenging methodological questions around studying process in contemporary, sociotechnical systems.

Drew Paine, Lavanya Ramakrishnan, "Surfacing Data Change in Scientific Work", iConference 2019, Springer Verlag, March 19, 2019, 15-26, doi: 10.1007/978-3-030-15742-5_2

2018

Cheah You-Wei, Drew Paine, Devarshi Ghoshal, Lavanya Ramakrishnan, Bringing Data Science to Qualitative Analysis, 2018 IEEE 14th International Conference on e-Science, Pages: 325-326 2018, doi: 10.1109/eScience.2018.00076

David P. Randall, Drew Paine, Charlotte P. Lee, "Educational Outreach & Stakeholder Role Evolution in a Cyberinfrastructure Project", 2018 IEEE 14th International Conference on e-Science, IEEE Computer Society, 2018, 201-211, doi: 10.1109/eScience.2018.00035

Gilberto Pastorello

2022

B Faybishenko, R Versteeg, G Pastorello, D Dwivedi, C Varadharajan, D Agarwal, Challenging problems of quality assurance and quality control (QA/QC) of meteorological time series data, Stochastic Environmental Research and Risk Assessment, Pages: 1049--1062 2022, doi: 10.1007/s00477-021-02106-w

2021

Devarshi Ghoshal, Drew Paine, Gilberto Pastorello, Abdelrahman Elbashandy, Dan Gunter, Oluwamayowa Amusat, Lavanya Ramakrishnan, "Experiences with Reproducibility: Case Studies from Scientific Workflows", (P-RECS'21) Proceedings of the 4th International Workshop on Practical Reproducible Evaluation of Computer Systems, ACM, June 21, 2021, doi: 10.1145/3456287.3465478

Reproducible research is becoming essential for science to ensure transparency and for building trust. Additionally, reproducibility provides the cornerstone for sharing of methodology that can improve efficiency. Although several tools and studies focus on computational reproducibility, we need a better understanding about the gaps, issues, and challenges for enabling reproducibility of scientific results beyond the computational stages of a scientific pipeline. In this paper, we present five different case studies that highlight the reproducibility needs and challenges under various system and environmental conditions. Through the case studies, we present our experiences in reproducing different types of data and methods that exist in an experimental or analysis pipeline. We examine the human aspects of reproducibility while highlighting the things that worked, that did not work, and that could have worked better for each of the cases. Our experiences capture a wide range of scenarios and are applicable to a much broader audience who aim to integrate reproducibility in their everyday pipelines.

D. A. Agarwal, J. Damerow, C. Varadharajan, D. S. Christianson, G. Z. Pastorello, Y.-W. Cheah, L. Ramakrishnan, "Balancing the needs of consumers and producers for scientific data collections", Ecological Informatics, 2021, 62:101251, doi: 10.1016/j.ecoinf.2021.101251

2020

G. Z. Pastorello, C. Trotta, E. Canfora, H. Chu, D. Christianson, Y.-W. Cheah, C. Poindexter, J. Chen, A. Elbashandy, M. Humphrey, P. Isaac, D. Polidori, M. Reichstein, A. Ribeca, C. van Ingen, N. Vuichard, L. Zhang, B. Amiro, C. Ammann, M. A. Arain, J. Ardö, T. Arkebauer, S. K. Arndt, N. Arriga, M. Aubinet, M. Aurela, D. Baldocchi, A. Barr, E. Beamesderfer, L. B. Marchesini, O. Bergeron, J. Beringer, C. Bernhofer, D. Berveiller, D. Billesbach, T. A. Black, P. D. Blanken, G. Bohrer, J. Boike, P. V. Bol stad, D. Bonal, J.-M. Bonnefond, D. R. Bowling, R. Bracho, J. Brodeur, C. Brümmer, N. Buchmann, B. Burban, S. P. Burns, P. Buysse, P. Cale, M. Cavagna, P. Cellier, S. Chen, I. Chini, T. R. Chris tensen, J. Cleverly, A. Collalti, C. Consalvo, B. D. Cook, D. Cook, C. Coursolle, E. Cremonese, P. S. Curtis, E. D’Andrea, H. da Rocha, X. Dai, K. J. Davis, B. D. Cinti, A. de Grandcourt, A. D. Ligne, R. C. D. Oliveira, N. Delpierre, A. R. Desai, C. M. D. Bella, P. di Tommasi, H. Dolman, F. Domingo, G. Dong, S. Dore, P. Duce, E. Dufrêne, A. Dunn, J. Dušek, D. Eamus, U. Eichelmann, H. A. M. ElKhidir, W. Eugster, C. M. Ewenz, B. Ewers, D. Famulari, S. Fares, I. Feigenwinter, A. Feitz, R. Fensholt, G. Fil ippa, M. Fischer, J. Frank, M. Galvagno, M. Gharun, D. Gianelle, B. Gielen, B. Gioli, A. Gitelson, I. Goded, M. Goeckede, A. H. Goldstein, C. M. Gough, M. L. Goulden, A. Graf, A. Griebel, C. Gruening, T. Grünwald, A. Hammerle, S. Han, X. Han, B. U. Hansen, C. Hanson, J. Hatakka, Y. He, M. Hehn, B. Heinesch, N. Hinko-Najera, L. Hörtnagl, L. Hutley, A. Ibrom, H. Ikawa, M. Jackowicz-Korczynski, D. Janouš, W. Jans, R. Jassal, S. Jiang, T. Kato, M. Khomik, J. Klatt, A. Knohl, S. Knox, H. Kobayashi, G. Koerber, O. Kolle, Y. Kosugi, A. Kotani, A. Kowalski, B. Kruijt, J. Kurbatova, W. L. Kutsch, H. Kwon, S. Launiainen, T. Laurila, B. Law, R. Leuning, Y. Li, M. Liddell, J.-M. Limousin, M. Lion, A. J. Liska, A. Lohila, A. López-Ballesteros, E. López-Blanco, B. Loubet, D. Loustau, A. Lucas-Moffat, J. Lüers, S. Ma, C. Macfarlane, V. Magliulo, R. Maier, I. Mammarella, G. Manca, B. Marcolla, H. A. Margolis, S. Mar ras, W. Massman, M. Mastepanov, R. Matamala, J. H. Matthes, F. Mazzenga, H. McCaughey, I. McHugh, A. M. S. McMillan, L. Merbold, W. Meyer, T. Meyers, S. D. Miller, S. Minerbi, U. Moderow, R. K. Monson, L. Montagnani, C. E. Moore, E. Moors, V. Moreaux, C. Moureaux, J. W. Munger, T. Nakai, J. Neirynck, Z. Nesic, G. Nicolini, A. Noormets, M. Northwood, M. Nosetto, Y. Nouvellon, K. Novick, W. Oechel, J. E. Olesen, J.-M. Ourcival, S. A. Papuga, F.-J. Parmentier, E. Paul-Limoges, M. Pavelka, M. Peichl, E. Pendall, R. P. Phillips, K. Pilegaard, N. Pirk, G. Posse, T. Powell, H. Prasse, S. M. Prober, S. Ram bal, U. Rannik, N. Raz-Yaseef, D. Reed, V. R. de Dios, N. Restrepo-Coupe, B. R. Reverter, M. Roland, S. Sabbatini, T. Sachs, S. R. Saleska, E. P. S.-C. nete, Z. M. Sanchez-Mejia, H. P. Schmid, M. Schmidt, K. Schneider, F. Schrader, I. Schroder, R. L. Scott, P. Sedlák, P. Serrano-Ortíz, C. Shao, P. Shi, I. Shironya, L. Siebicke, L. Šigut, R. Silberstein, C. Sirca, D. Spano, R. Steinbrecher, R. M. Stevens, C. Sturtevant, A. Suyker, T. Tagesson, S. Takanashi, Y. Tang, N. Tapper, J. Thom, F. Tiedemann, M. Tomassucci, J.-P. Tuovinen, S. Urbanski, R. Valentini, M. van der Molen, E. van Gorsel, K. van Huissteden, A. Varlagin, J. Verfaillie, T. Vesala, C. Vincke, D. Vitale, N. Vygodskaya, J. P. Walker, E. Walter-Shea, H. Wang, R. Weber, S. Westermann, C. Wille, S. Wofsy, G. Wohlfahrt, S. Wolf, W. Woodgate, Y. Li, R. Zampedri, J. Zhang, G. Zhou, D. Zona, D. Agarwal, S. Biraud, M. Torn, D. Papale, "The FLUXNET2015 dataset and the ONEFlux processing pipeline for eddy covariance data", Scientific Data, 2020, 7:225, doi: 10.1038/s41597-020-0534-3

2019

P. Linton, W. Melodia, A. Lazar, D. Agarwal, L. Bianchi, D. Ghoshal, K. Wu, G. Pastorello, L. Ramakrishnan, "Identifying Time Series Similarity in Large-Scale Earth System Datasets", The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC19), 2019,

Payton A Linton, William M Melodia, Alina Lazar, Deborah Agarwal, Ludovico Bianchi, Devarshi Ghoshal, Kesheng Wu, Gilberto Pastorello, Lavanya Ramakrishnan, "Identifying Time Series Similarity in Large-Scale Earth System Datasets", 2019,

Payton Linton, William Melodia, Alina Lazar, Deborah Agarwal, Ludovico Bianchi, Devarshi Ghoshal, Gilberto Pastorello, Lavanya Ramakrishnan, Kesheng Wu, Understanding Data Similarity in Large-Scale Scientific Datasets, 2019 IEEE International Conference on Big Data (Big Data), Pages: 4525--4531 2019,

2017

Gilberto Z. Pastorello, Dario Papale, Housen Chu, Carlo Trotta, Deb A. Agarwal, Eleonora Canfora, Dennis D. Baldocchi, M. S. Torn, "A new data set to keep a sharper eye on land-air exchanges", Eos, 2017, 98:28-32, doi: 10.1029/2017EO071597

Danielle S. Christianson, Charuleka Varadharajan, Bradley Christoffersen, Matteo Detto, Faybishenko, Bruno O. Gimenez, Val C. Hendrix, Kolby J. Jardine, Robinson Negron-Juarez, Z. Pastorello, Thomas L. Powell, Megha Sandesh, Jeffrey M. Warren, Brett T. Wolfe, Jeffrey Q. Chambers, Lara M. Kueppers, Nathan G. McDowell, Deborah A. Agarwal, "A metadata reporting framework (FRAMES) for synthesis of ecohydrological observations", Ecological Informatics, 2017, 42:148-158, doi: 10.1016/j.ecoinf.2017.06.002

Gilberto Z. Pastorello, Dan K. Gunter, Housen Chu, Danielle S. Christianson, Carlo Trotta, Eleonora Canfora, Boris Faybishenko, You-Wei Cheah, Norm Beekwilder, Stephen W. Chan, Sigrid Dengel, Trevor Keenan, Fianna O Brien, Abderahman Elbashandy, Cristina M. Poindexter, Marty Humphrey, Dario Papale, Deb A. Agarwal, "Hunting Data Rogues at Scale: Data Quality Control for Observational Data in Research Infrastructures", Proceedings of the 13th IEEE International Conference on e-Science (e-Science 2017), Auckland, New Zealand, 2017, doi: 10.1109/eScience.2017.64

Gilberto Z. Pastorello, "Generation of Uniform Data Products for AmeriFlux and FLUXNET", The practice of reproducible research: case studies and lessons from the data-intensive sciences, (University of California Press: 2017) Pages: 305-309

2016

Craig A. Emmerton, Vincent L. St. Louis, Elyn R. Humphreys, John A. Gamon, Joel D. Barker, Gilberto Z. Pastorello, "Net ecosystem exchange of CO2 with rapidly changing high Arctic landscapes", Global Change Biology, 2016, 22:1185-1200, doi: 10.1111/gcb.13064

Ran Wang, John A. Gamon, Craig A. Emmerton, Li Haitao, Enrica Nestola, Gilberto Z. Pastorello, Olaf Menzer, "Integrated Analysis of Productivity and Biodiversity in a Southern Alberta Prairie", Remote Sensing, 2016, 8:2014, doi: 10.3390/rs8030214

2014

Gilberto Z. Pastorello, Deb A. Agarwal, Taghrid Samak, Dario Papale, Trotta, Alessio Ribeca, Cristina M. Poindexter, Boris Faybishenko, Dan K. Gunter, Rachel Hollowgrass, Eleonora Canfora, "Observational data patterns for time series data quality assessment", Proceedings of the 10th IEEE International Conference on e-Science (e-Science 2014), Guaruja, Brazil, 2014, doi: 10.1109/eScience.2014.45

Lavanya Ramakrishnan, Sarah S. Poon, Val C. Hendrix, Dan K. Gunter, Gilberto Z. Pastorello, Deb A. Agarwal, "Experiences with User-Centered Design for the Tigres Workflow API", Proceedings of the 10th IEEE International Conference on e-Science (e-Science 2014), Guaruja, Brazil, 2014, doi: 10.1109/eScience.2014.56

Angela Harris, John A. Gamon, Gilberto Z. Pastorello, Christopher Y. S. Wong, "Retrieval of the photochemical reflectance index for assessing xanthophyll cycle activity: a comparison of near-surface optical sensors", Biogeosciences, 2014, 11:6277-6292, doi: 10.5194/bg-11-6277-2014

2011

Gilberto Z. Pastorello, G. Arturo Sanchez-Azofeifa, Mario A. Nascimento, "Enviro-Net: from networks of ground-based sensor systems to a Web platform for sensor data management", Sensors, 2011, 11:6454-6479, doi: 10.3390/s110606454

Gilberto Z. Pastorello, G. Arturo Sanchez-Azofeifa, Mario A. Nascimento, "Enviro-Net: A Network of Ground-based Sensors for Tropical Dry Forests in the Americas", Proceedings of the 34th International Symposium on Remote Sensing of Environment (ISRSE 2011), Sydney, Australia, 2011, 4p,

2010

Gilberto Z. Pastorello, Jaudete Daltio, Claudia M. B. Medeiros, "A Mechanism for Propagation of Semantic Annotations of Multimedia Content", Journal of Multimedia, 2010, 5:332-342, doi: 10.4304/jmm.5.4.332-342

John A. Gamon, Craig Coburn, Lawrence B. Flanagan, Karl F. Huemmrich, Kiddle, G. Arturo Sanchez-Azofeifa, Donnette R. Thayer, Loris Vescovo, Damiano Gianelli, Daniel A. Sims, Abdullah Faiz Rahman, Gilberto Z. Pastorello, "SpecNet Revisited: Bridging Flux and Remote Sensing Communities", Canadian Journal of Remote Sensing, 2010, 36:S376-S390, doi: 10.5589/m10-067

Roger Curry, Cameron Kiddle, Rob Simmonds, Gilberto Z. Pastorello, "An On-line Collaborative Data Management System", Proceedings of the 6th Gateway Computing Environments Workshop (GCE 2010), New Orleans, LA, USA, 2010, doi: 10.1109/GCE.2010.5676120

2009

Gilberto Z. Pastorello, Rodrigo D. A. Senra, Claudia M. B. Medeiros, "A standards-based framework to foster geospatial data and process interoperability", Journal of the Brazilian Computer Society, 2009, 15:13-26, doi: 10.1007/BF03192574

2008

Gilberto Z. Pastorello, Rodrigo D. A. Senra, Claudia M. B. Medeiros, "Bridging the gap between geospatial resource providers and model developers", Proceedings of the 16th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (GIS 2008), Irvine, CA, USA, 2008, 379-382, doi: 10.1145/1463434.1463489

Gilberto Z. Pastorello, Luiz C. Gomes Jr, Claudia M. B. Medeiros, Andre Santanche, "Sensor Data Publication on the Web for Scientific Applications", Proceedings of the 4th International Conference on Web Information Systems and Technologies (WEBIST 2008), Funchal, Madeira, Portugal, 2008, 137-142, doi: 10.5220/0001515301370142

Gilberto Z. Pastorello, Claudia M. B. Medeiros, Andre Santanche, "Accessing and Processing Sensing Data", Proceedings of the 11th IEEE International Conference on Computational Science and Engineering (CSE 2008), Sao Paulo, Brazil, 2008, 353--360, doi: 10.1109/CSE.2008.23

Gilberto Z. Pastorello, Jaudete Daltio, Claudia M. B. Medeiros, "Multimedia Semantic Annotation Propagation", Proceedings of the 1st IEEE International Workshop on Data Semantics for Multimedia Systems and Applications (DSMSA 2008), 10th IEEE International Symposium on Multimedia (ISM 2008), Berkeley, CA, USA, 2008, 509--514, doi: 10.1109/ISM.2008.77

2007

Andre Santanche, Claudia M. B. Medeiros, Gilberto Z. Pastorello, "User-author centered multimedia building blocks", Multimedia Systems Journal, 2007, 12:403-421, doi: 10.1007/s00530-006-0050-0

Gilberto Z. Pastorello, C. M. B. Medeiros, Andre Santanche, "Applying Scientific Workflows to Manage Sensor Data", Proceedings of the 1st Brazilian e-Science Workshop (BreSci 2007), 22nd Brazilian Symposium on Databases (SBBD 2007), Joao Pessoa, Brazil, 2007, 9-18,

2005

Claudia M. B. Medeiros, Jose Perez-Alcazar, Luciano Digiampietri, Gilberto Z. Pastorello, Andre Santanche, Ricardo S. Torres, Edmundo Madeira, Evandro Bacarin, "WOODSS and the Web: Annotating and Reusing Scientific Workflows", SIGMOD Record, 2005, 34:18-23, doi: 10.1145/1084805.1084810

Sean Peisert

2023

Tong Wu, Anna Scaglione, Adrian Petru Surani, Daniel Arnold, Sean Peisert, "Network-Constrained Reinforcement Learning for Optimal EV Charging Control", Proceedings of the IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm), October 2023,

Robert Currie, Sean Peisert, Anna Scaglione, Aram Shumavon, Nikhil Ravi, "Data Privacy for the Grid: Toward a Data Privacy Standard for Inverter-Based and Distributed Energy Resources", IEEE Power & Energy Magazine, October 1, 2023,

Jim Basney, Sean Peisert, Scott Russell, Kelli Shute, Bart Miller, Kathy Benninger, "A Vision for Securing NSF's Essential Scientific Cyberinfrastructure - Trusted CI Five-Year Strategic Plan (2024-2029)", Trusted CI Report, August 1, 2023, doi: 10.5281/zenodo.8193607

Sachin Kadam, Anna Scaglione, Nikhil Ravi, Sean Peisert, Brent Lunghino, Aram Shumavon, "Optimum Noise Mechanism for Differentially Private Queries in Discrete Finite Sets", Proceedings of the 2023 IEEE International Conference on Smart Applications, Communications and Networking (SmartNets), Istanbul, Turkey, July 25, 2023,

Raksha Ramakrishna, Anna Scaglione, Tong Wu, Nikhil Ravi, Sean Peisert, "Differential Privacy for Class-based Data: A Practical Gaussian Mechanism", June 23, 2023, doi: 10.1109/TIFS.2023.3289128

Nikhil Ravi, Anna Scaglione, Julieta Giraldez, Parth Pradhan, Chuck Moran, Sean Peisert, "Solar Photovoltaic Systems Metadata Inference and Differentially Private Publication", arXiv preprint arXiv:2304.03749, April 7, 2023, doi: 10.48550/arXiv.2304.03749

George Cybenko, Carl Landwehr, Shari Lawrence Pfleeger, Sean Peisert, A 20th Anniversary Episode Chat With S&P Editors, IEEE Security & Privacy, Pages: 9-16 April 2023, doi: 10.1109/MSEC.2023.3239179

Sean Peisert, "The First 20 Years of IEEE Security & Privacy [From the Editors]", IEEE Security & Privacy, April 1, 2023, 21(2):4-6, doi: 10.1109/MSEC.2023.3236420

Hector G. Martin, Tijana Radivojevic, Jeremy Zucker, Kristofer Bouchard, Jess Sustarich, Sean Peisert, Dan Arnold, Nathan Hillson, Gyorgy Babnigg, Jose M. Marti, Christopher J. Mungall, Gregg T. Beckham, Lucas Waldburger, James Carothers, ShivShankar Sundaram, Deb Agarwal, Blake A. Simmons, Tyler Backman, Deepanwita Banerjee, Deepti Tanjore, Lavanya Ramakrishnan, Anup Singh, "Perspectives for Self-Driving Labs in Synthetic Biology", Current Opinion in Biotechnology, February 2023, doi: 10.1016/j.copbio.2022.102881

2022

Ammar Haydari, Chen-Nee Chuah, Michael Zhang, Jane Macfarlane, Sean Peisert, "Differentially Private Map Matching for Mobility Trajectories", Proceedings of the 2022 Annual Computer Security Applications Conference (ACSAC), Austin, TX, ACM, December 2022, doi: 0.1145/3564625.3567974

Andrew Adams, Emily K. Adams, Dan Gunter, Ryan Kiser, Mark Krenz, Sean Peisert, John Zage, "Roadmap for Securing Operational Technology in NSF Scientific Research", Trusted CI Report, November 16, 2022, doi: 10.5281/zenodo.7327987

Ayaz Akram, Venkatesh Akella, Sean Peisert, Jason Lowe-Power, "SoK: Limitations of Confidential Computing via TEEs for High-Performance Compute Systems", Proceedings of the 2022 IEEE International Symposium on Secure and Private Execution Environment Design (SEED), September 2022,

Yize Chen, Yuanyuan Shi, Daniel Arnold, Sean Peisert, "SAVER: Safe Learning-Based Controller for Real-Time Voltage Regulation", Proceedings of the 2022 IEEE Power Engineering Society (PES) General Meeting, Denver, CO, July 2022,

Emily K. Adams, Daniel Gunter, Ryan Kiser, Mark Krenz, Sean Peisert, Susan Sons, John Zage, "Findings of the 2022 Trusted CI Study on the Security of Operational Technology in NSF Scientific Research", Trusted CI Report, July 15, 2022, doi: doi.org/10.5281/zenodo.6828675

Daniel Arnold, Sy-Toan Ngo, Ciaran Roberts, Yize Chen, Anna Scaglione, Sean Peisert, "Adam-based Augmented Random Search for Control Policies for Distributed Energy Resource Cyber Attack Mitigation", Proceedings of the 2022 American Control Conference (ACC), June 2022,

Sean Peisert, Unsafe at Any Clock Speed: the Insecurity of Computer System Design, Implementation, and Operation [From the Editors], IEEE Security & Privacy, Pages: 4-9 January 2022, doi: 10.0.4.85/MSEC.2021.3127086

2021

James R. Clavin, Yue Huang, Xin Wang, Pradeep M. Prakash, Sisi Duan, Jianwu Wang, Sean Peisert, "A Framework for Evaluating BFT", Proceedings of the IEEE International Conference on Parallel and Distributed Systems (ICPADS), IEEE, December 2021,

Andrew Adams, Kay Avila, Elisa Heymann, Mark Krenz, Jason R. Lee, Barton Miller, Sean Peisert, "Guide to Securing Scientific Software", Trusted CI Report, December 14, 2021, doi: 10.5281/zenodo.5777646

Ammar Haydari, Michael Zhang, Chen-Nee Chuah, Jane Macfarlane, Sean Peisert, Adaptive Differential Privacy Mechanism for Aggregated Mobility Dataset, arXiv preprint arXiv:2112.08487, December 10, 2021,

Yize Chen, Yuanyuan Shi, Daniel Arnold, Sean Peisert, SAVER: Safe Learning-Based Controller for Real-Time Voltage Regulation, arXiv preprint arXiv:2111.15152,, November 30, 2021,

Luca Pion-Tonachini, Kristofer Bouchard, Hector Garcia Martin, Sean Peisert, W. Bradley Holtz, Anil Aswani, Dipankar Dwivedi, Haruko Wainwright, Ghanshyam Pilania, Benjamin Nachman, Babetta L. Marrone, Nicola Falco, Prabhat, Daniel Arnold, Alejandro Wolf-Yadlin, Sarah Powers, Sharlee Climer, Quinn Jackson, Ty Carlson, Michael Sohn, Petrus Zwart, Neeraj Kumar, Amy Justice, Claire Tomlin, Daniel Jacobson, Gos Micklem, Georgios V. Gkoutos, Peter J. Bickel, Jean-Baptiste Cazier, Juliane Müller, Bobbie-Jo Webb-Robertson, Rick Stevens, Mark Anderson, Ken Kreutz-Delgado, Michael W. Mahoney, James B. Brown,, Learning from Learning Machines: a New Generation of AI Technology to Meet the Needs of Science, arXiv preprint arXiv:2111.13786, November 27, 2021,

Nikhil Ravi, Anna Scaglione, Sachin Kadam, Reinhard Gentz, Sean Peisert, Brent Lunghino, Emmanuel Levijarvi, Aram Shumavon, Differentially Private K-means Clustering Applied to Meter Data Analysis and Synthesis, arXiv preprint arXiv:2112.03801, November 23, 2021,

Sachin Kadam, Anna Scaglione, Nikhil Ravi, Sean Peisert, Brent Lunghino, Aram Shumavon, Optimum Noise Mechanism for Differentially Private Queries in Discrete Finite Sets, arXiv preprint arXiv:2111.11661, November 23, 2021,

Nikhil Ravi, Anna Scaglione, Sean Peisert, Colored Noise Mechanism for Differentially Private Clustering, arXiv preprint arXiv:2111.07850, November 15, 2021,

Yize Chen, Daniel Arnold, Yuanyuan Shi, Sean Peisert, Understanding the Safety Requirements for Learning-based Power Systems Operations, arXiv preprint arXiv:2110.04983, October 11, 2021,

Andrew Adams, Kay Avila, Elisa Heymann, Mark Krenz, Jason R. Lee, Barton Miller, Sean Peisert, "The State of the Scientific Software World: Findings of the 2021 Trusted CI Software Assurance Annual Challenge Interviews", Trusted CI Report, September 29, 2021,

Ayaz Akram, Venkatesh Akella, Sean Peisert, Jason Lowe-Power,, "Enabling Design Space Exploration for RISC-V Secure Compute Environments", Proceedings of the Fifth Workshop on Computer Architecture Research with RISC-V (CARRV), (co-located with ISCA 2021), June 17, 2021,

Ciaran Roberts, Sy-Toan Ngo, Alexandre Milesi, Anna Scaglione, Sean Peisert, Daniel Arnold, "Deep Reinforcement Learning for Mitigating Cyber-Physical DER Voltage Unbalance Attacks”", Proceedings of the 2021 American Control Conference (ACC), May 2021, doi: 10.23919/ACC50511.2021.9482815

Ayaz Akram, Anna Giannakou, Venkatesh Akella, Jason Lowe-Power, Sean Peisert, "Performance Analysis of Scientific Computing Workloads on General Purpose TEEs", Proceedings of the 35th IEEE International Parallel & Distributed Processing Symposium (IPDPS), IEEE, May 2021, doi: 10.1109/IPDPS49936.2021.00115

Sean Peisert, "Trustworthy Scientific Computing", Communications of the ACM (CACM), May 2021, doi: 10.1145/3457191

Sean Peisert, Bruce Schneier, Hamed Okhravi, Fabio Massacci, Terry Benzel, Carl Landwehr, Mohammad Mannan, Jelena Mirkovic, Atul Prakash, James Bret Michael, "Perspectives on the SolarWinds Incident", IEEE Security & Privacy, April 2021, 7-13, doi: 10.1109/MSEC.2021.3051235

Fabio Massacci, Trent Jaeger, Sean Peisert, "SolarWinds and the Challenges of Patching: Can We Ever Stop Dancing With the Devil?", IEEE Security & Privacy, April 2021, 14-19, doi: 10.1109/MSEC.2021.3050433

2020

Chris Lawson, Jose Manuel Martí, Tijana Radivojevic, Sai Vamshi R. Jonnalagadda, Reinhard Gentz, Nathan J. Hillson, Sean Peisert, Joonhoon Kim, Blake A. Simons, Christopher J. Petzold, Steven W. Singer, Aindrila Mukhopadhyay, Deepti Tanjore, Josh Dunn, Héctor García Martín,, "Machine Learning for Metabolic Engineering: A Review", Metabolic Engineering, November 19, 2020,

Ignacio Losada Carreño, Raksha Ramakrishna, Anna Scaglione, Daniel Arnold, Ciaran Roberts, Sy-Toan Ngo, Sean Peisert, David Pinney, "SODA: An Irradiance-Based Tool to Generate Sub-Minute Solar Power Stochastic Time Series", Proceedings of the IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm), IEEE, November 2020,

Ciaran Roberts Sy-Toan Ngo, Alexandre Milesi, Sean Peisert, Daniel Arnold, Shammya Saha, Anna Scaglione, Nathan Johnson, Anton Kocheturov, Dmitriy Fradkin, "Deep Reinforcement Learning for DER Cyber-Attack Mitigation", Proceedings of the IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm), IEEE, November 2020,

Ayaz Akram, Anna Giannakou, Venkatesh Akella, Jason Lowe-Power, Sean Peisert, "Performance Analysis of Scientific Computing Workloads on Trusted Execution Environments", arXiv preprint arXiv:2010.13216, October 25, 2020,

Bogdan Copos, Sean Peisert, "Catch Me If You Can: Using Power Analysis to Identify HPC Activity", arXiv:2005.03135 [cs.CR], May 6, 2020,

Ross Gegan, Christina Mao, Dipak Ghosal, Matt Bishop, Sean Peisert, "Anomaly Detection for Science DMZ Using System Performance Data", Proceedings of the 2020 IEEE International Conference on Computing, Networking and Communications (ICNC 2020), Big Island, HI, February 2020, doi: 10.1109/ICNC47757.2020.9049695

Anna Giannakou, Dipankar Dwivedi, Sean Peisert, "A Machine Learning Approach for Packet Loss Prediction in ScienceFlows", Future Generation Computer Systems, January 2020, 102:190-197, doi: 10.1016/j.future.2019.07.053

2019

Reinhard Gentz, Sean Peisert, "An Examination and Survey of Random Bit Flips and Scientific Computing", Trusted CI Report, December 20, 2019,

Amir Teshome Wonjiga, Louis Rilling, Christine Morin, Sean Peisert, "Blockchain as a Trusted Component in Cloud SLA Verification", Proceedings of the International Workshop on Cloud, IoT and Fog Security (CIFS), co-located with the 12th IEEE/ACM International Conference on Utility and Cloud Computing (UCC), Auckland, New Zealand, December 2019, 93-100, doi: 10.1145/3368235.3368872

Mahdi Jamei, Raksha Ramakrishna, Teklemariam Tesfay, Reinhard Gentz, Ciaran Roberts, Anna Scaglione, Sean Peisert, "Phasor Measurement Units Optimal Placement and Performance Limits for Fault Localization", IEEE Journal on Selected Areas in Communications (J-SAC), Special Issue on Communications and Data Analytics in Smart Grid, November 6, 2019, 38(1):180-192, doi: 10.1109/jsac.2019.2951971

Thomas W. Edgar, Aditya Ashok, Garret E. Seppala, K.M. Arthur-Durrett, M. Engels, Reinhard Gentz, Sean Peisert, "An Automated Disruption-Tolerant Key Management Framework for Critical Systems", Journal of Information Warfare, October 8, 2019, 18(4):85-124, doi: https://www.jinfowar.com/journal/volume-18-issue-4/automated-disruption-tolerant-device-authentication-key-management-framework-critical-systems

Reinhard Gentz, Sean Peisert, Joshua Boverhof, Daniel Gunter, "SPARCS: Stream-Processing Architecture applied in Real-time Cyber-physical Security", Proceedings of the 15th IEEE International Conference on e-Science (eScience), San Diego, CA, IEEE, September 2019, doi: 10.1109/eScience.2019.00028

Reinhard Gentz, Héctor García Martin, Edward Baidoo, Sean Peisert, "Workflow Automation in Liquid Chromatography Mass Spectrometry", Proceedings of the 15th IEEE International Conference on e-Science (eScience), San Diego, CA, IEEE, September 2019, doi: 10.1109/eScience.2019.00095

Melissa Stockman, Dipankar Dwivedi, Reinhard Gentz, Sean Peisert, "Detecting Programmable Logic Controller Code Using Machine Learning", International Journal of Critical Infrastructure Protection, September 2019, vol. 26,, doi: 10.1016/j.ijcip.2019.100306

Ciaran Roberts, Anna Scaglione, Mahdi Jamei, Reinhard Gentz, Sean Peisert, Emma M. Stewart, Chuck McParland, Alex McEachern, Daniel Arnold, "Learning Behavior of Distribution System Discrete Control Devices for Cyber-Physical Security", IEEE Transaction on Smart Grid, August 1, 2019, 11(1):749-761, doi: 0.1109/TSG.2019.2936016

Andrew Adams, Kay Avila, Jim Basney, Dana Brunson, Robert Cowles, Jeannette Dopheide, Terry Fleury, Elisa Heymann, Florence Hudson, Craig Jackson, Ryan Kiser, Mark Krenz, Jim Marsteller, Barton P. Miller, Sean Peisert, Scott Russell, Susan Sons, Von Welch, John Zage, "Trusted CI Experiences in Cybersecurity and Service to Open Science", Proceedings of the Conference on Practice and Experience in Advanced Research Computing (PEARC), ACM, July 2019, doi: 10.1145/3332186.3340601

Sean Peisert, Brooks Evans, Michael Liang, Barclay Osborn, David Rusting, David Thurston, Security Without Moats and Walls: Zero-Trust Networking for Enhancing Security in R&E Environments, CENIC Annual Conference, March 19, 2019,

Sean Peisert, Experiences in Building a Mission-Driven Security R&D Program for Science and Energy, Computer Science Colloquium Seminar, University of California, Davis, February 7, 2019,

Sean Peisert, Daniel Arnold, Using Physics to Improve Cybersecurity for the Distribution Grid and Distributed Energy Resources, Naval Postgraduate School, February 5, 2019,

Sean Peisert, Building a Mission-Driven, Applied Cybersecurity R&D Program from Scratch, VISA Research, January 23, 2019,

2018

Anna Giannakou, Daniel Gunter, Sean Peisert, "Flowzilla: A Methodology for Detecting Data Transfer Anomalies in Research Networks", Workshop on Innovating the Network for Data-Intensive Science (INDIS), November 11, 2018, doi: 10.1109/INDIS.2018.00004

Sean Peisert, Usable Computer Security and Privacy to Enable and Encourage Data Sharing for Scientific Research, National Academies of Sciences, Engineering, and Medicine Committee on Science, Engineering, Medicine, and Public Policy (COSEMPUP) Meeting, November 8, 2018,

Mahdi Jamei, Anna Scaglione, Sean Peisert, "Low-Resolution Fault Localization Using Phasor Measurement Units with Community Detection", Proceedings of the 2018 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm), Allborg, Denmark, IEEE, October 29, 2018, doi: 10.1109/SmartGridComm.2018.8587461

Sean Peisert, Ciaran Roberts, Anna Scaglione, Mahdi Jamei, Reinhard Gentz, Charles McParland, Alex McEachren, Galen Rasche, Aaron Snyder, "Supporting Cyber Security of Power Distribution Systems by Detecting Differences Between Real-time Micro-Synchrophasor Measurements and Cyber-Reported SCADA - Final Report", October 15, 2018,

Sean Peisert, Security Concerns of an NRP, Second National Research Platform (NRP) Workshop, August 6, 2018,

"A Holistic Approach to Distribution Grid Intrusion Detection Systems", Ciaran Roberts, Anna Scaglione Sean Peisert,, EnergyCentral, July 18, 2018,

Sean Peisert, Cyber Security Challenges and Opportunities in High-Performance Computing Environments, International Supercomputing Conference, June 26, 2018,

Sean Peisert, Keynote: Cybersecurity for HPC Systems: State of the Art and Looking to the Future, High-Performance Computing Security Workshop, National Institute of Standards and Technology (NIST), March 28, 2018,

Sean Peisert, Eli Dart, William K. Barnett, James Cuff, Robert L. Grossman, Edward Balas, Ari Berman, Anurag Shankar, Brian Tierney, "The Medical Science DMZ: An Network Design Pattern for Data-Intensive Medical Science", Journal of the American Medical Informatics Association (JAMIA), March 2018, 25(3):267-274, doi: 10.1093/jamia/ocx104

Sean Peisert, Ciaran Roberts, Cyber Security of Power Distribution Systems Using Micro-Synchrophasor Measurements and Cyber-Reported SCADA, EPRI Power Delivery & Utilization Winter 2018 Program Advisory & Sector Council Meeting, February 7, 2018,

EM Stewart, P Top, M Chertkov, D Deka, S Backhaus, A Lokhov, C Roberts, V Hendrix, S Peisert, A Florita, TJ King, MJ Reno, "Integrated multi-scale data analytics and machine learning for the distribution grid", 2017 IEEE International Conference on Smart Grid Communications, SmartGridComm 2017, 2018, 2018-Jan:423--429, doi: 10.1109/SmartGridComm.2017.8340693

Terry Benzel and Sean Peisert, Selected Papers from the 2017 IEEE Symposium on Security and Privacy [Guest editors' introduction], IEEE Security & Privacy, Pages: 10-11 January 2018, doi: 10.1109/MSP.2018.1331038

2017

David Hatchell, Patrick Miller, Michael Coleman, Sean Peisert, Cybersecurity for the Electricity Grid", Bits & Watts Annual Conference, November 6, 2017,

Sean Peisert, Security in High Performance Computing Environments, Computing Sciences/NERSC Security Seminar, October 5, 2017,

Sean Peisert, Matt Bishop, Ed Talbot,, "A Model of Owner Controlled, Full-Provenance, Non-Persistent, High-Availability Information Sharing", Proceedings of the 2017 New Security Paradigms Workshop (NSPW), Santa Cruz, CA, October 2017, 80-89, doi: 10.1145/3171533.3171536

Sean Peisert, Security and Privacy in Data-Intensive, High-Performance Computing Contexts, Berkeley Institute for Data Science (BIDS), October 2, 2017,

Jonathan Ganz, Sean Peisert, "ASLR: How Robust is the Randomness", Proceedings of the IEEE Secure Development Conference (SecDev), Cambridge, MA, IEEE Computer Society, September 24, 2017, doi: 10.1109/SecDev.2017.19

Mahdi Jamei, Anna Scaglione, Ciaran Roberts, Emma Stewart, Sean Peisert, Chuck McParland, Alex McEachern, "Anomaly Detection Using μPMU Measurements in Distribution Grids", IEEE Transactions on Power Systems, 2017, doi: 10.1109/TPWRS.2017.2764882

Sean Peisert, "Security in High-Performance Computing Environments", Communications of the ACM (CACM), September 2017, 60(9):72-80, doi: 10.1145/3096742

Mahdi Jamei, Anna Scaglione, Ciaran Roberts, Alex McEachern, Emma Stewart, Sean Peisert, Chuck McParland, "Online Thevenin Parameter Tracking Using Synchrophasor Data", Proceedings of the 2017 IEEE Power Engineering Society (PES) General Meeting (GM), Chicago, IL, IEEE, July 2017, doi: 10.1109/PESGM.2017.8273818

Alberto Gonzalez, Jason Leigh, Sean Peisert, Brian Tierney, Edward Balas, Predrag Radulovic, Jennifer M. Schopf, "Big Data and Analysis of Data Transfers for International Research Networks Using NetSage", Proceedings of IEEE BigData Congress 2017, Honolulu, Hawaii, June 2017, doi: 10.1109/BigDataCongress.2017.51

Sean Peisert, Mike Corn, Dewight Kramer, David Rusting, Tye Stallard, The Role of the WAN and the Community to Improve Security, 2017 UC Information Security Symposium,, June 21, 2017,

Galen Rasche, Jenna Goodward, Sheeraz Haji, Gabriel Paun, Sean Peisert, Managing Energy: Role of Data and Security, Prospect Silicon Valley 2017 Innovation and Impact Symposium, June 14, 2017,

Sean Peisert, Greg Bell, Anita Nikolich, Von Welch, Cybersecurity: New Directions for Research and Education - Your own safety is at stake when your neighbor's wall is ablaze. (—Horace), CENIC Annual Conference — The Right Connection ¦ CENIC 2.0, March 22, 2017,

Leon J. Osterweil, Matt Bishop, Heather M. Conboy, Huong Phan, Borislava I. Simidchieva, George S. Avrunin, Lori A. Clarke, Sean Peisert, "A Comprehensive Framework for Using Iterative Analysis to Improve Human-Intensive Process Security: An Election Example", ACM Transactions on Privacy and Security (TOPS), 2017, 20(2), doi: https://doi.org/10.1145/3041041

Sean Peisert, Von Welch, Andrew Adams, Michael Dopheide, Susan Sons, RuthAnne Bevier, Rich LeDuc, Pascal Meunier, Stephen Schwab, and Karen Stocks, Ilkay Altintas, James Cuff, Reagan Moore, Warren Raquel, "Open Science Cyber Risk Profile", February 10, 2017, doi: 2022/21259

Richard LeDuc, Sean Peisert, Karen Stocks, Von Welch, Open Science Cyber Risk Profile (OSCRP), National Science Foundation Cybersecurity Center of Excellence (CCoE) Webinar Series, January 23, 2017,

Mahdi Jamei, Anna Scaglione, Ciaran Roberts, Emma Stewart, Sean Peisert, Chuck McParland, Alex McEachern, "Automated Anomaly Detection in Distribution Grids Using µPMU Measurements", Proceedings of the 50th Hawaii International Conference on System Sciences (HICSS), Electric Energy Systems Track, Resilient Networks Minitrack, IEEE, January 2017, doi: http://hdl.handle.net/10125/41543

S Peisert, R Gentz, J Boverhof, C McParland, S Engle, A Elbashandy, D Gunter, "LBNL Open Power Data", January 2017, doi: 10.21990/C21599

2016

Sean Peisert, Overcoming Security and Privacy Challenges in Computing and Networking in Medical Research Environments, Department of Public Health Sciences, University of California, Davis School of Medicine,, December 14, 2016,

Sean Peisert (moderator), Jill Gemmill, Michael Sinatra, Von Welch, National Cybersecurity Panel, NSF Campus Cyberinfrastructure/ESCC/The Quilt Colocated Meeting, October 20, 2016,

Lee Beausoleil, David Lombard, Angelos Keromytis, Sean Peisert, Panel: HPC Monitoring, NSCI: High-Performance Computing Security Workshop, September 30, 2016,

Sean Peisert, Security Expert on Why HPC Matters - Cybersecurity for HPC Systems: Challenges and Opportunities, NSCI: High-Performance Computing Security Workshop, September 29, 2016,

Mahdi Jamei, Emma Stewart, Sean Peisert, Anna Scaglione, Chuck McParland, Ciaran Roberts, Alex McEachern, "Micro Synchrophasor-Based Intrusion Detection in Automated Distribution Systems: Towards Critical Infrastructure Security", IEEE Internet Computing, September 2016, 20(5):18-27, doi: 10.1109/MIC.2016.102

Jonathan Ganz, Matt Bishop, and Sean Peisert, "Security Analysis of Scantegrity, an Electronic Voting System", University of California, Davis, Department of Computer Science Technical Report, June 2016,

Alberto Gonzalez, Jason Leigh, Sean Peisert, Brian Tierney, Andrew Lee, Jennifer M. Schopf, "NetSage: Open Privacy-Aware Network Measurement, Analysis, And Visualization Service", Proceedings of TNC16 Networking Conference, Prague, Czech Republic, June 2016,

Sean Peisert, William K. Barnett, Eli Dart, James Cuff, Robert L. Grossman, Edward Balas, Ari Berman, Anurag Shankar, Brian Tierney, "The Medical Science DMZ", Journal of the American Medical Informatics Association (JAMIA), May 2, 2016, 23(6):1199-1201, doi: 10.1093/jamia/ocw032

Sean Peisert, CENIC 2016 Conference Panel: Security in R&E Networks and Campus Environments, 2016 CENIC Annual Conference, March 22, 2016,

Sean Peisert, Computer Security & the Electric Power Grid, 15th Annual ON*VECTOR Photonics Workshop, March 1, 2016,

2015

Jason Adams, Monica Lieng, Brooks Kuhn, Edward Guo, Edik Simonian, Sean Peisert, JP Delplanque, Nick Anderson, "Automated Mechanical Ventilator Waveform Analysis of Patient-Ventilator Asynchrony", CHEST Journal, Pages: 175A October 2015, doi: 10.1378/chest.2281731

PURPOSE: Mechanical ventilation is a life-saving intervention but is associated with adverse effects including ventilator-induced lung injury (VILI). Patient-ventilator asynchrony (PVA) is thought to contribute to VILI, but the study of PVA has been hampered by limited access to the high frequency, large volume data streams produced by modern ventilators and a lack of robust analytics. To address these limitations, we developed an automated pipeline for breath-by-breath analysis of ventilator waveform data.

METHODS: Simulated pressure and flow time series data representing normal breaths and common forms of PVA were generated on PB840 ventilators, collected unobtrusively using small, customized wireless peripheral devices, and transmitted to a networked server for storage and analysis. Two critical care physicians reviewed all waveforms to generate gold standards. Rule-based algorithms were developed to quantify inspiratory and expiratory tidal volumes (TV) and identify PVA subtypes including double trigger and delayed termination asynchrony. Data were split randomly into derivation and validation sets. Algorithm performance was compared to ventilator reported values and clinician annotation.

RESULTS: The mean difference between algorithm-determined and ventilator-reported TVs was 3.1% (99% CI ± 1.36%). Algorithm agreement with clinician annotation was excellent for double trigger PVA and moderate for delayed termination PVA, with Kappa statistics of 0.85 and 0.58, respectively. In the validation data set (n = 492 breaths), double trigger asynchrony was detected with an overall accuracy of 94.1%, sensitivity of 100%, and specificity of 92.8%.

CONCLUSIONS: A pipeline combining wireless ventilator data acquisition and rule-based analytic algorithms informed by the principles of bedside ventilator waveform analysis allows for automated, quantitative breath-by-breath analysis of patient-ventilator interactions.

CLINICAL IMPLICATIONS: We have recently deployed this system in the medical intensive care unit of the UC Davis Medical Center, which will enable further development of mechanical ventilation analytics. We have begun to explore the use of supervised machine learning and dynamic time series modeling to improve the classification of other common types of PVA and of clinical phenotypes associated with respiratory failure. This system will help to better define the epidemiology and clinical impact of PVA and other forms of off-target mechanical ventilation, and may lead to improved decision support and patient outcomes.

Daniel Chung, Matt Bishop, and Sean Peisert, "Distributed Helios - Mitigating Denial of Service Attacks in Online Voting", University of California, Davis, Department of Computer Science Technical Report, October 16, 2015,

Adrian Chavez, William M.S. Stout, and Sean Peisert, "Techniques for the Dynamic Randomization of Network Attributes", Proceedings of the 49th Annual International Carnahan Conference on Security Technology, Taipei, Taiwan, Republic of China, IEEE Press, September 2015, doi: 10.1109/CCST.2015.7389661

Sisi Duan, Jingtao Sun, Sean Peisert, "Towards a Self-Adaptive Middleware for Building Reliable Publish/Subscribe Systems", Proceedings of the 8th International Conference on Internet and Distributed Computing Systems (IDCS), Berkshire, United Kingdom, Springer, September 2015, 157-168, doi: 10.1007/978-3-319-23237-9_14

Sean Peisert, et al., "ASCR Cybersecurity for Scientific Computing Integrity - Research Pathways and Ideas", U.S. Department of Energy Office of Science report, September 2015, LBNL 191105, doi: 10.2172/1236181

Sean Peisert, Security Research Using Cyber-Physical Systems, IT Security Symposium, June 16, 2015,

Sean Peisert, Models of Secure and Private Information Sharing, University of California, San Diego School of Medicine, Division of Biomedical Informatics Seminar Series, April 10, 2015,

Sean Peisert, et al., "ASCR Cybersecurity for Scientific Computing Integrity", U.S. Department of Energy Office of Science report, February 27, 2015, LBNL 6953E, doi: 10.2172/1223021

Georgia Koutsandria, Reinhard Gentz, Mahdi Jamei, Anna Scaglione, Sean Peisert, Chuck McParland, "A real-time testbed environment for cyber-physical security on the power grid", Proceedings of the First ACM Workshop on Cyber-Physical Systems-Security and/or PrivaCy, January 1, 2015, 67--78, doi: 10.1145/2808705.2808707

2014

Sisi Duan, Hein Meling, Sean Peisert, Haibin Zhang,, "BChain: Byzantine Replication with High Throughput and Embedded Reconfiguration", Proceedings of the 18th International Conference on Principles of Distributed Systems (OPODIS), Cortina, Italy, Springer, December 2014, 91-106, doi: 10.1007/978-3-319-14472-6_7

Chuck McParland, Sean Peisert, Anna Scaglione, "Monitoring Security of Networked Control Systems: It's the Physics", IEEE Security and Privacy, November 2014, 12(6):32-39, doi: 10.1109/MSP.2014.122

Sean Peisert, Jonathan Margulies, Closing the Gap on Securing Energy Sector Control Systems [Guest editors' introduction], IEEE Security & Privacy, Pages: 13-14 November 2014, doi: 10.1109/MSP.2014.110

Sean Peisert, Jonathan Margulies, Eric Byres, Paul Dorey, Dale Peterson, Zach Tudor, "Control System Security from the Front Lines (Roundtable)", IEEE Security and Privacy, November 2014, 12(6):55-58, doi: 10.1109/MSP.2014.112

Sisi Duan, Karl Levitt, Hein Meling, Sean Peisert, Haibin Zhang, "Byzantine Fault Tolerance from Intrusion Detection", Proceedings of the 33rd IEEE International Symposium on Reliable Distributed Systems (SRDS), Nara, Japan, October 2014, 253-264, doi: 10.1109/SRDS.2014.28

Sean Peisert, Security for Computational Infrastructure for Financial Technology, DataLead 2014: Leading the Way in Big Data, Haas School of Business, UC Berkeley, September 30, 2014,

Sean Peisert, Jonathan Margulies, David M. Nicol, Himanshu Khurana, Chris Sawall,, "Designed-in Security for Cyber-Physical Systems (Roundtable)", IEEE Security and Privacy, September 2014, 12(5):9-12, doi: 10.1109/MSP.2014.90

Tiancheng Chang, Sisi Duan, Hein Meling, Sean Peisert, Haibin Zhang, "P2S: A Fault-Tolerant Publish/Subscribe Infrastructure", Proceedings of the 8th ACM International Conference on Distributed Event Based Systems (DEBS), Mumbai, India, ACM Press, May 2014, 189-197, doi: 10.1145/2611286.2611305

Matt Bishop, Heather Conboy, Huong Phan, Borislava I. Simidchieva, George Avrunin, Lori Clarke, Lee Osterweil, Sean Peisert,, "Insider Detection by Process Analysis", Proceedings of the 2014 Workshop on Research for Insider Threat (WRIT), IEEE Computer Society Security and Privacy Workshops, San Jose, CA, IEEE Computer Society, May 18, 2014, doi: 10.1109/SPW.2014.40

Sean Peisert, Challenges in Insider Threat Research, Workshop on Research for Insider Threat (WRIT), IEEE Security and Privacy Workshops (SPW), May 18, 2014,

Peter G. Neumann, Sean Peisert, Marvin Schaefer, "The IEEE Symposium on Security and Privacy, in Retrospect", IEEE Security and Privacy, May 2014, 12(3):15-17, doi: 10.1109/MSP.2014.59

Sisi Duan, Sean Peisert, and Karl Levitt, "hBFT: Speculative Byzantine Fault Tolerance With Minimum Cost", IEEE Transactions on Dependable and Secure Computing (TDSC), March 19, 2014, 12(1):58-70, doi: 10.1109/TDSC.2014.2312331

Masood Parvania, Georgia Koutsandria, Vishak Muthukumary, Sean Peisert, Chuck McParland, Anna Scaglione, "Hybrid Control Network Intrusion Detection Systems for Automated Power Distribution Systems", Proc. of the 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), January 1, 2014, 774--779, doi: 10.1109/DSN.2014.81

Georgia Koutsandria, Vishak Muthukumar, Masood Parvania, Sean Peisert, Chuck McParland, Anna Scaglione, "A hybrid network IDS for protective digital relays in the power transmission grid", Smart Grid Communications (SmartGridComm), 2014 IEEE International Conference on, January 2014, 908--913, doi: 10.1109/SmartGridComm.2014.7007764

2013

Lizzie Coles-Kemp, Carrie Gates, Dieter Gollmann, Sean Peisert, Christian Probst, "Organizational Processes for Supporting Sustainable Security", Report from Dagstuhl Seminar 120501, November 4, 2013, doi: 10.4230/DagRep.2.12.37

Sean Peisert, Ed Talbot, Tom Kroeger, "Principles of Authentication", Proceedings of the 2013 New Security Paradigms Workshop (NSPW), Banff, Canada, ACM, September 2013, 47-56, doi: 10.1145/2535813.2535819

Sean Peisert and Steven Templeton, "The Hive Mind: Applying a Distributed Security Sensor Network to GENI- GENI Spiral 2 Final Project Report", UC Davis Technical Report, September 4, 2013,

Sean Peisert, Cyber Resilience Metrics, First International Symposium on Resilient Cyber Systems, Resilience Week 2013, August 13, 2013,

Sean Peisert, Health Informatics Minute: Aligning Organizational and Employee Computer Security Goals for Health Informatics, Seventh Annual Health Informatics Graduate Program Conference, March 22, 2013,

Sean Whalen, Sean Peisert, Matt Bishop, "Multiclass Classification of Distributed Memory Parallel Computations", Pattern Recognition Letters (PRL), February 2013, 34(3):322-329, doi: 10.1016/j.patrec.2012.10.007

Sean Peisert, Matt Bishop, "Dynamic, Flexible, and Optimistic Access Control", Working Paper, January 2013,

2012

Xiao Li, Zhifang Wang, Vishak Muthukumar, Anna Scaglione, Chuck McParland, Sean Peisert, "Networked Loads in the Distribution Grid", Proceedings of the 2012 APSIPA Annual Summit and Conference, Hollywood, CA, December 3, 2012,

Sean Peisert, Institute for Information Infrastructure Protection (I3P), 10th Anniversary Event, The National Press Club, October 10, 2012,

Sean Peisert, Ed Talbot, Matt Bishop, "Turtles All the Way Down: A Clean-Slate, Ground-Up, First-Principles Approach to Secure Systems", Proceedings of the 2012 New Security Paradigms Workshop (NSPW), ACM, September 2012, doi: 10.1145/2413296.2413299

Matt Bishop, Sean Peisert, "Security and Elections", IEEE Security & Privacy, September 2012, 10(5):64-67, doi: 10.1109/MSP.2012.127

Sean Whalen, Sophie Engle, Sean Peisert, Matt Bishop, "Network-Theoretic Classification of Parallel Computation Patterns", International Journal of High Performance Computing Applications (IJHPCA), May 2012, 26(2):159-169, doi: 10.1177/1094342012436618

2011

"The Hive Mind Project - Digital Ants for Intrusion Detection", Sean Peisert, DETERlab Testbed Quarterly Newsletter, 2011,

Sean Whalen, Sean Peisert, Matt Bishop, "Network-Theoretic Classification of Parallel Computation Patterns", Proceedings of the First International Workshop on Characterizing Applications for Heterogeneous Exascale Systems (CACHES), Tucson, AZ, IEEE Computer Society, June 4, 2011,

2010

Matt Bishop, Sophie Engle, Deborah A. Frincke, Carrie Gates, Frank L. Greitzer, Sean Peisert, Sean Whalen, "A Risk Management Approach to the "Insider Threat"", Insider Threats in Cyber Security, "Advances in Information Security" Series, edited by Christian W. Probst, Jeffrey Hunker, Matt Bishop, (Springer: September 2010) Pages: 115-138 doi: 10.1007/978-1-4419-7133-3_6

Borislava I. Simidchieva, Sophie J. Engle, Michael Clifford, Alicia Clay Jones, Sean Peisert, Matt Bishop, Lori A. Clarke, Leon J. Osterweil,, "Modeling Faults to Improve Election Process Robustness", Proceedings of the 2010 Electronic Voting Technology Workshop/ Workshop on Trustworthy Elections (EVT/WOTE), Washinton, D.C., USENIX, August 2010,

Sean Peisert, "Fingerprinting Communication and Computation on HPC Machines", Lawrence Berkeley National Laboratory Technical Report, June 2010, LBNL LBNL-3483E,

Peter G. Neumann, Matt Bishop, Sean Peisert, Marv Schaefer, "Reflections on the 30th Anniversary of the IEEE Symposium on Security and Privacy", Proceedings of the 31st IEEE Symposium on Security and Privacy, Oakland/Berkeley, CA, IEEE Computer Society, May 2010, 3-13, doi: 10.1109/SP.2010.43

Sean Peisert, Matt Bishop, Keith Marzullo, "What Do Firewalls Protect? An Empirical Study of Firewalls, Vulnerabilities, and Attacks", UC Davis CS Technical Report CSE-2010-8, March 2010,

M Bishop, J Cummins, S Peisert, A Singh, B Bhumiratana, D Agarwal, D Frincke, M Hogarth, "Relationships and data sanitization: A study in scarlet", Proceedings New Security Paradigms Workshop, January 2010, 151--163, doi: 10.1145/1900546.1900567

2009

Sean Peisert, Matt Bishop, Laura Corriss, Steven J. Greenwald, "Quis Custodiet ipsos Custodes? A New Paradigm for Analyzing Security Paradigms", Proceedings of the 2009 New Security Paradigms Workshop (NSPW), The Queen's College, Oxford, United Kingdom, ACM, September 2009, 133-144, doi: 10.1145/1719030.1719041

Matt Bishop, Sean Peisert, Candice Hoke, Mark Graff, David Jefferson, "E-Voting and Forensics: Prying Open the Black Box", Proceedings of the 2009 Electronic Voting Technology Workshop/Workshop on Trustworthy Elections (EVT/WOTE), Montreal, Canada, USENIX, August 2009,

2008

Matt Bishop, Mark Graff, Candice Hoke, David Jefferson, Sean Peisert, "Resolving the Unexpected in Elections: Election Officials' Options", October 8, 2008,

Sean Peisert, Keynote Address: Computer Forensics In Forensis, Third International IEEE Workshop on Systematic Approaches to Digital Forensic Engineering (IEEE/SADFE-2008) (held in conjunction with the 2008 IEEE Symposium on Security and Privacy), May 22, 2008,

Sean Peisert, Matt Bishop, Keith Marzullo, "Computer Forensics In Forensis", ACM Operating Systems Review (OSR), April 2008, 42:112-122, doi: 10.1109/TDSC.2007.1003

2007

Sean Peisert, Matt Bishop, "I'm a Scientist, Not a Philosopher!", IEEE Security and Privacy, July 2007, 5(4):48-51, doi: 10.1109/MSP.2007.84

Sean Peisert, Matt Bishop, Sidney Karin, Keith Marzullo,, "Analysis of Computer Intrusions Using Sequences of Function Calls", IEEE Transactions on Dependable and Secure Computing (TDSC), April 2007, 4(2):137-150, doi: 10.1109/TDSC.2007.1003

Sean Peisert, Matt Bishop, Sidney Karin, Keith Marzullo,, "Toward Models for Forensic Analysis", Proceedings of the Second International Workshop on Systematic Approaches to Digital Forensic Engineering (SADFE), IEEE, April 1, 2007, 3-15, doi: 10.1109/SADFE.2007.23

A Model of Forensic Analysis Using Goal-Oriented Logging, Sean P. Peisert, Ph.D. Dissertation, Dept. of Computer Science and Engineering, University of California, San Diego, March 2007,

2006

Matt Bishop, Sean Peisert, "Your Security Policy is What???", UC Davis CS Technical Report CSE-2006-20, March 2006,

2005

Sean Peisert, Matt Bishop, Sidney Karin, Keith Marzullo,, "Principles-Driven Forensic Analysis", Proceedings of the 2005 New Security Paradigms Workshop (NSPW), Lake Arrowhead, CA, ACM, September 1, 2005, 85-93, doi: 10.1145/1146269.1146291

Sean Peisert, "Forensics for System Administrators", ;login:, August 2005, 30(4):34-42,

Talita Perciano

2024

Jan Balewski, Mercy G Amankwah, Roel Van Beeumen, E Wes Bethel, Talita Perciano, Daan Camps, "Quantum-parallel vectorized data encodings and computations on trapped-ion and transmon QPUs", Journal, February 10, 2024, 14, doi: https://doi.org/10.1038/s41598-024-53720-x

Zhe Bai, Abdelilah Essiari, Talita Perciano, Kristofer E Bouchard, "AutoCT: Automated CT registration, segmentation, and quantification", Software X, January 5, 2024, 26, doi: https://doi.org/10.1016/j.softx.2024.101673

2023

E Wes Bethel, Mercy G Amankwah, Jan Balewski, Roel Van Beeumen, Daan Camps, Daniel Huang, Talita Perciano, "Quantum computing and visualization: A disruptive technological change ahead", Journal, November 6, 2023, 43, doi: https://doi.org/10.1109/MCG.2023.3316932

GM Wallace, Z Bai, N Bertelli, EW Bethel, T Perciano, S Shiraiwa, JC Wright, "Towards Fast, Accurate Predictions of RF Simulations via Data-driven Modeling: Forward and Lateral Models", Conference, AIP Publishing, August 1, 2023, 2984, doi: https://doi.org/10.1063/5.0162422

2022

Gregory Wallace, Zhe Bai, Robbie Sadre, Talita Perciano, Nicola Bertelli, Syun'ichi Shiraiwa, Wes Bethel, John Wright, "Towards fast and accurate predictions of radio frequency power deposition and current profile via data-driven modelling: applications to lower hybrid current drive", Journal of Plasma Physics, August 18, 2022, 88:895880401, doi: 10.1017/S0022377822000708

M. G. Amankwah, D. Camps, E. W. Bethel, R. Van Beeumen, T. Perciano, "Quantum pixel representations and compression for N-dimensional images", Nature Scientific Reports, May 11, 2022, 12:7712, doi: 10.1038/s41598-022-11024-y

S. Zhang, R. Sadre, B. A. Legg, H. Pyles, T. Perciano, E. W. Bethel, D. Baker, O. Rübel, J. J. D. Yoreo, "Rotational dynamics and transition mechanisms of surface-adsorbed proteins", Proceedings of the National Academy of Sciences, April 11, 2022, 119:e202024211, doi: 10.1073/pnas.2020242119

M. Avaylon, R. Sadre, Z. Bai, T. Perciano, "Adaptable Deep Learning and Probabilistic Graphical Model System for Semantic Segmentation", Advances in Artificial Intelligence and Machine Learnin, March 31, 2022, 2:288--302, doi: 10.54364/AAIML.2022.1119

C Varadharajan, AP Appling, B Arora, DS Christianson, VC Hendrix, V Kumar, AR Lima, J Müller, S Oliver, M Ombadi, T Perciano, JM Sadler, H Weierbach, JD Willard, Z Xu, J Zwart, "Can machine learning accelerate process understanding and decision-relevant predictions of river water quality?", Hydrological Processes, January 1, 2022, 36, doi: 10.1002/hyp.14565

2021

V. Dumont, C. Garner, A. Trivedi, C. Jones, V. Ganapati, J. Mueller, T. Perciano, M. Kiran, and M. Day, "HYPPO: A Surrogate-Based Multi-Level Parallelism Tool for Hyperparameter Optimization", 2021 IEEE/ACM Workshop on Machine Learning in High Performance Computing Environments (MLHPC), November 15, 2021,

E. W. Bethel, C. Heinemann, and T. Perciano, "Performance Tradeoffs in Shared-memory Platform Portable Implementations of a Stencil Kernel", Eurographics Symposium on Parallel Graphics and Visualization, June 14, 2021,

2020

Li Zhou, Chao Yang, Weiguo Gao, Talita Perciano, Karen M. Davies, Nicholas K. Sauter, "Subcellular structure segmentation from cryo-electron tomograms via machine learning", PLOS Journal of Computational Biology, April 2, 2020, submitte, doi: doi: https://doi.org/10.1101/2020.04.09.034025

Stefano Marchesini, Anuradha Trivedi, Pablo Enfedaque, Talita Perciano, Dilworth Parkinson, "Sparse Matrix-Based HPC Tomography", Computational Science -- ICCS 2020, Cham, Springer International Publishing, 2020, 248--261, doi: 10.1007/978-3-030-50371-0_18

E. Wes Bethel, David Camp, Talita Perciano, Colleen Heinemann, "Performance Analysis of Traditional and Data-Parallel Primitive Implementations of Visualization and Analysis Kernels", Berkeley, CA, USA, 94720, 2020,

H Chang, J J Donatelli, P Enfedaque, G Freychet, M Haranczyk, A Hexemer, Z Hu, O Jain, H Krishnan, D Kumar, X Li, L Lin, M MacNeil, S Marchesini, X Mo, M Noack, K Pande, R Pandolfi, D Parkinson, D M Pelt, T Perciano, D A Shapiro, D Ushizima, C Yang, P H Zwart, J A Sethian, "Building Mathematics, Algorithms, and Software for Experimental Facilities", Handbook on Big Data and Machine Learning in the Physical Sciences, ( 2020) Pages: 189--240 doi: 10.1142/9789811204579_0012

Talita Perciano, Colleen Heinemann, David Camp, Brenton Lessley, E Wes Bethel, "Shared-Memory Parallel Probabilistic Graphical Modeling Optimization: Comparison of Threads, OpenMP, and Data-Parallel Primitives", High Performance Computing, Cham, Springer International Publishing, 2020, 127--145, doi: 10.1007/978-3-030-50743-5_7

2018

B Lessley, T Perciano, C Heinemann, D Camp, H Childs, EW Bethel, "DPP-PMRF: Rethinking Optimization for a Probabilistic Graphical Model Using Data-Parallel Primitives", The 8th IEEE Symposium on Large Data Analysis and Visualization - LDAV 2018, 2018,

C Heinemann, T Perciano, D Ushizima, EW Bethel, "Distributed memory parallel Markov random fields using graph partitioning", Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017, 2018, 2018-Jan:3332--3341, doi: 10.1109/BigData.2017.8258318

RJ Pandolfi, DB Allan, E Arenholz, L Barroso-Luque, SI Campbell, TA Caswell, A Blair, F De Carlo, S Fackler, AP Fournier, G Freychet, M Fukuto, D Gürsoy, Z Jiang, H Krishnan, D Kumar, RJ Kline, R Li, C Liman, S Marchesini, A Mehta, AT N Diaye, DY Parkinson, H Parks, LA Pellouchoud, T Perciano, F Ren, S Sahoo, J Strzalka, D Sunday, CJ Tassone, D Ushizima, S Venkatakrishnan, KG Yager, P Zwart, JA Sethian, A Hexemer, "Xi-cam: a versatile interface for data visualization and analysis", Journal of Synchrotron Radiation, 2018, 25:1261--1270, doi: 10.1107/S1600577518005787

2017

B Lessley, T Perciano, M Mathai, H Childs, EW Bethel, "Maximal clique enumeration with data-parallel primitives", 2017 IEEE 7th Symposium on Large Data Analysis and Visualization, LDAV 2017, 2017, 2017-Dec:16--25, doi: 10.1109/LDAV.2017.8231847

T Perciano, D Ushizima, H Krishnan, D Parkinson, N Larson, DM Pelt, W Bethel, F Zok, J Sethian, "Insight into 3D micro-CT data: Exploring segmentation algorithms through performance metrics", Journal of Synchrotron Radiation, 2017, 24:1065--1077, doi: 10.1107/S1600577517010955

Benedikt J Daurer, Hari Krishnan, Talita Perciano, Filipe RNC Maia, David A Shapiro, James A Sethian, Stefano Marchesini, "Nanosurveyor: a framework for real-time data processing", Advanced structural and chemical imaging, 2017, 3:7,

T Perciano, D Ushizima, H Krishnan, D Parkinson, J Sethian, "FibriPy: A software environment for fiber analysis from 3D micro-computed tomography data", Advanced Materials - TechConnect Briefs 2017, 2017, 1:25--28,

DY Parkinson, DM Pelt, T Perciano, D Ushizima, H Krishnan, HS Barnard, AA MacDowell, J Sethian, "Machine learning for micro-tomography", Proceedings of SPIE - The International Society for Optical Engineering, 2017, 10391, doi: 10.1117/12.2274731

M Farmand, R Celestre, P Denes, ALD Kilcoyne, S Marchesini, H Padmore, T Tyliszczak, T Warwick, X Shi, J Lee, YS Yu, J Cabana, J Joseph, H Krishnan, T Perciano, FRNC Maia, DA Shapiro, "Near-edge X-ray refraction fine structure microscopy", Applied Physics Letters, 2017, 110, doi: 10.1063/1.4975377

2016

T Perciano, DM Ushizima, EW Bethel, YD Mizrahi, D Parkinson, JA Sethian, "Reduced-complexity image segmentation under parallel Markov Random Field formulation using graph partitioning", Proceedings - International Conference on Image Processing, ICIP, 2016, 2016-Aug:1259--1263, doi: 10.1109/ICIP.2016.7532560

DM Ushizima, HA Bale, EW Bethel, P Ercius, BA Helms, H Krishnan, LT Grinberg, M Haranczyk, AA Macdowell, K Odziomek, DY Parkinson, T Perciano, RO Ritchie, C Yang, "IDEAL: Images Across Domains, Experiments, Algorithms and Learning", JOM, 2016, 68:2963--2972, doi: 10.1007/s11837-016-2098-4

SV Venkatakrishnan, KA Mohan, K Beattie, J Correa, E Dart, JR Deslippe, A Hexemer, H Krishnan, AA MacDowell, S Marchesini, SJ Patton, T Perciano, JA Sethian, R Stromsness, BL Tierney, CE Tull, D Ushizima, DY Parkinson, "Making advanced scientific algorithms and big scientific data management more accessible", IS and T International Symposium on Electronic Imaging Science and Technology, 2016, doi: 10.2352/ISSN.2470-1173.2016.19.COIMG-155

DY Parkinson, K Beattie, X Chen, J Correa, E Dart, BJ Daurer, JR Deslippe, A Hexemer, H Krishnan, AA Macdowell, FRNC Maia, S Marchesini, HA Padmore, SJ Patton, T Perciano, JA Sethian, D Shapiro, R Stromsness, N Tamura, BL Tierney, CE Tull, D Ushizima, "Real-time data-intensive computing", AIP Conference Proceedings, 2016, 1741, doi: 10.1063/1.4952921

S Marchesini, H Krishnan, BJ Daurer, DA Shapiro, T Perciano, JA Sethian, FRNC Maia, "SHARP: A distributed GPU-based ptychographic solver", Journal of Applied Crystallography, 2016, 49:1245--1252, doi: 10.1107/S1600576716008074

T Perciano, F Tupin, R Hirata, RM Cesar, "A two-level Markov random field for road network extraction and its application with optical, SAR, and multitemporal data", International Journal of Remote Sensing, 2016, 37:3584--3610, doi: 10.1080/01431161.2016.1201227

2015

J Donatelli, M Haranczyk, A Hexemer, H Krishnan, X Li, L Lin, F Maia, S Marchesini, D Parkinson, T Perciano, D Shapiro, D Ushizima, C Yang, JA Sethian, "CAMERA: The Center for Advanced Mathematics for Energy Research Applications", Synchrotron Radiation News, 2015, 28:4--9, doi: 10.1080/08940886.2015.1013413

AW Wills, DJ Michalak, P Ercius, ER Rosenberg, T Perciano, D Ushizima, R Runser, BA Helms, "Block Copolymer Packing Limits and Interfacial Reconfigurability in the Assembly of Periodic Mesoporous Organosilicas", Advanced Functional Materials, 2015, 25:4120--4128, doi: 10.1002/adfm.201501059

D Ushizima, T Perciano, D Parkinson, "Fast detection of material deformation through structural dissimilarity", Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015, 2015, 2775--2781, doi: 10.1109/BigData.2015.7364080

2014

D Ushizima, T Perciano, H Krishnan, B Loring, H Bale, D Parkinson, J Sethian, "Structure recognition from high resolution images of ceramic composites", Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014, 2014, 683--691, doi: 10.1109/BigData.2014.7004292

2013

T Perciano, MW Urban, NDA Mascarenhas, M Fatemi, AC Frery, GT Silva, "Deconvolution of vibroacoustic images using a simulation model based on a three dimensional point spread function", Ultrasonics, 2013, 53:36--44, doi: 10.1016/j.ultras.2012.03.011

Lavanya Ramakrishnan

2023

Hector G. Martin, Tijana Radivojevic, Jeremy Zucker, Kristofer Bouchard, Jess Sustarich, Sean Peisert, Dan Arnold, Nathan Hillson, Gyorgy Babnigg, Jose M. Marti, Christopher J. Mungall, Gregg T. Beckham, Lucas Waldburger, James Carothers, ShivShankar Sundaram, Deb Agarwal, Blake A. Simmons, Tyler Backman, Deepanwita Banerjee, Deepti Tanjore, Lavanya Ramakrishnan, Anup Singh, "Perspectives for Self-Driving Labs in Synthetic Biology", Current Opinion in Biotechnology, February 2023, doi: 10.1016/j.copbio.2022.102881

2022

MB Simmonds, WJ Riley, DA Agarwal, X Chen, S Cholia, R Crystal-Ornelas, ET Coon, D Dwivedi, VC Hendrix, M Huang, A Jan, Z Kakalia, J Kumar, CD Koven, L Li, M Melara, L Ramakrishnan, DM Ricciuto, AP Walker, W Zhi, Q Zhu, C Varadharajan, Guidelines for Publicly Archiving Terrestrial Model Data to Enhance Usability, Intercomparison, and Synthesis, Data Science Journal, 2022, doi: 10.5334/dsj-2022-003

2021

Devarshi Ghoshal, Ludovico Bianchi, Abdelilah Essiari, Drew Paine, Sarah Poon, Michael Beach, Alpha N'Diaye, Patrick Huck, Lavanya Ramakrishnan, "Science Capsule: Towards Sharing and Reproducibility of Scientific Workflows", 2021 IEEE Workshop on Workflows in Support of Large-Scale Science (WORKS), November 15, 2021, doi: 10.1109/WORKS54523.2021.00014

Workflows are increasingly processing large volumes of data from scientific instruments, experiments and sensors. These workflows often consist of complex data processing and analysis steps that might include a diverse ecosystem of tools and also often involve human-in-the-loop steps. Sharing and reproducing these workflows with collaborators and the larger community is critical but hard to do without the entire context of the workflow including user notes and execution environment. In this paper, we describe Science Capsule, which is a framework to capture, share, and reproduce scientific workflows. Science Capsule captures, manages and represents both computational and human elements of a workflow. It automatically captures and processes events associated with the execution and data life cycle of workflows, and lets users add other types and forms of scientific artifacts. Science Capsule also allows users to create `workflow snapshots' that keep track of the different versions of a workflow and their lineage, allowing scientists to incrementally share and extend workflows between users. Our results show that Science Capsule is capable of processing and organizing events in near real-time for high-throughput experimental and data analysis workflows without incurring any significant performance overheads.

Drew Paine, Sarah Poon, Lavanya Ramakrishnan, "Investigating User Experiences with Data Abstractions on High Performance Computing Systems", June 29, 2021, LBNL LBNL-2001374,

Scientific exploration generates expanding volumes of data that commonly require High Performance Computing (HPC) systems to facilitate research. HPC systems are complex ecosystems of hardware and software that frequently are not user friendly. The Usable Data Abstractions (UDA) project set out to build usable software for scientific workflows in HPC environments by undertaking multiple rounds of qualitative user research. Qualitative research investigates how individuals accomplish their work and our interview-based study surfaced a variety of insights about the experiences of working in and with HPC ecosystems. This report examines multiple facets to the experiences of scientists and developers using and supporting HPC systems. We discuss how stakeholders grasp the design and configuration of these systems, the impacts of abstraction layers on their ability to successfully do work, and the varied perceptions of time that shape this work. Examining the adoption of the Cori HPC at NERSC we explore the anticipations and lived experiences of users interacting with this system's novel storage feature, the Burst Buffer. We present lessons learned from across these insights to illustrate just some of the challenges HPC facilities and their stakeholders need to account for when procuring and supporting these essential scientific resources to ensure their usability and utility to a variety of scientific practices.

Devarshi Ghoshal, Drew Paine, Gilberto Pastorello, Abdelrahman Elbashandy, Dan Gunter, Oluwamayowa Amusat, Lavanya Ramakrishnan, "Experiences with Reproducibility: Case Studies from Scientific Workflows", (P-RECS'21) Proceedings of the 4th International Workshop on Practical Reproducible Evaluation of Computer Systems, ACM, June 21, 2021, doi: 10.1145/3456287.3465478

Reproducible research is becoming essential for science to ensure transparency and for building trust. Additionally, reproducibility provides the cornerstone for sharing of methodology that can improve efficiency. Although several tools and studies focus on computational reproducibility, we need a better understanding about the gaps, issues, and challenges for enabling reproducibility of scientific results beyond the computational stages of a scientific pipeline. In this paper, we present five different case studies that highlight the reproducibility needs and challenges under various system and environmental conditions. Through the case studies, we present our experiences in reproducing different types of data and methods that exist in an experimental or analysis pipeline. We examine the human aspects of reproducibility while highlighting the things that worked, that did not work, and that could have worked better for each of the cases. Our experiences capture a wide range of scenarios and are applicable to a much broader audience who aim to integrate reproducibility in their everyday pipelines.

Devarshi Ghoshal, Ludovico Bianchi, Abdelilah Essiari, Michael Beach, Drew Paine, Lavanya Ramakrishnan, "Science Capsule - Capturing the Data Life Cycle", Journal of Open Source Software, 2021, 6:2484, doi: 10.21105/joss.02484

D. A. Agarwal, J. Damerow, C. Varadharajan, D. S. Christianson, G. Z. Pastorello, Y.-W. Cheah, L. Ramakrishnan, "Balancing the needs of consumers and producers for scientific data collections", Ecological Informatics, 2021, 62:101251, doi: 10.1016/j.ecoinf.2021.101251

J Müller, B Faybishenko, D Agarwal, S Bailey, C Jiang, Y Ryu, C Tull, L Ramakrishnan, Assessing data change in scientific datasets, Concurrency and Computation: Practice and Experience, 2021, doi: 10.1002/cpe.6245

2020

Drew Paine, Lavanya Ramakrishnan, "Understanding Interactive and Reproducible Computing With Jupyter Tools at Facilities", LBNL Technical Report, October 31, 2020, LBNL LBNL-2001355,

Increasingly Jupyter tools are being adopted and incorporated into High Performance Computing (HPC) and scientific user facilities. Adopting Jupyter tools enables more interactive and reproducible computational work at facilities across data life cycles. As the volume, variety, and scope of data grow, scientists need to be able to analyze and share results in user friendly ways. Human-centered research highlights design challenges around computational notebooks, and our qualitative user study shifts focus to better characterize how Jupyter tools are being used in HPC and science user facilities today. We conducted twenty-nine interviews, and obtained 103 survey responses from NERSC Jupyter users, to better understand the increasing role of interactive computing tools in DOE sponsored scientific work. We examine a range of issues that emerge using and supporting Jupyter in HPC ecosystems, including: how Jupyter is being used by scientists in HPC and user facility ecosystems; how facilities are purposefully supporting Jupyter in their ecosystems; feedback NERSC users have about the facility’s deployment, and, discuss features NERSC indicated would be helpful. We offer a variety of takeaways for staff supporting Jupyter at facilities, Project Jupyter and related open source communities, and funding agencies supporting interactive computing work.

Drew Paine, Devarshi Ghoshal, Lavanya Ramakrishnan, "Experiences with a Flexible User Research Process to Build Data Change Tools", Journal of Open Research Software, September 1, 2020, doi: 10.5334/jors.284

Scientific software development processes are understood to be distinct from commercial software development practices due to uncertain and evolving states of scientific knowledge. Sustaining these software products is a recognized challenge, but under-examined is the usability and usefulness of such tools to their scientific end users. User research is a well-established set of techniques (e.g., interviews, mockups, usability tests) applied in commercial software projects to develop foundational, generative, and evaluative insights about products and the people who use them. Currently these approaches are not commonly applied and discussed in scientific software development work. The use of user research techniques in scientific environments can be challenging due to the nascent, fluid problem spaces of scientific work, varying scope of projects and their user communities, and funding/economic constraints on projects.

In this paper, we reflect on our experiences undertaking a multi-method user research process in the Deduce project. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists make decisions around data change. There is a lack of common terminology since the concept of systematically measuring and managing data change is under explored in scientific environments. To bridge this gap we conducted user research that focuses on user practices, needs, and motivations to help us design and develop metrics and tools for data change. This paper contributes reflections and the lessons we have learned from our experiences. We offer key takeaways for scientific software project teams to effectively and flexibly incorporate similar processes into their projects.

Drew Paine, Devarshi Ghoshal, Lavanya Ramakrishnan, "Investigating Scientific Data Change with User Research Methods", August 20, 2020, LBNL LBNL-2001347,

Scientific datasets are continually expanding and changing due to fluctuations with instruments, quality assessment and quality control processes, and modifications to software pipelines. Datasets include minimal information about these changes or their effects requiring scientists manually assess modifications through a number of labor intensive and ad-hoc steps. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists systematically identify and make decisions around data changes. Currently, there is a lack of understanding, and common practices, for identifying and evaluating changes in datasets since systematically measuring and managing data change is under explored in scientific work. We are conducting user research to address this need by exploring scientist's conceptualizations, behaviors, needs, and motivations when dealing with changing datasets. Our user research utilizes multiple methods to produce foundational, generative insights and evaluate research products produced by our team. In this paper, we detail our user research process and outline our findings about data change that emerge from our studies. Our work illustrates how scientific software teams can push beyond just usability testing user interfaces or tools to better probe the underlying ideas they are developing solutions to address.

2019

P. Linton, W. Melodia, A. Lazar, D. Agarwal, L. Bianchi, D. Ghoshal, K. Wu, G. Pastorello, L. Ramakrishnan, "Identifying Time Series Similarity in Large-Scale Earth System Datasets", The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC19), 2019,

Drew Paine, Lavanya Ramakrishnan, "Surfacing Data Change in Scientific Work", iConference 2019, Springer Verlag, March 19, 2019, 15-26, doi: 10.1007/978-3-030-15742-5_2

Payton A Linton, William M Melodia, Alina Lazar, Deborah Agarwal, Ludovico Bianchi, Devarshi Ghoshal, Kesheng Wu, Gilberto Pastorello, Lavanya Ramakrishnan, "Identifying Time Series Similarity in Large-Scale Earth System Datasets", 2019,

Payton Linton, William Melodia, Alina Lazar, Deborah Agarwal, Ludovico Bianchi, Devarshi Ghoshal, Gilberto Pastorello, Lavanya Ramakrishnan, Kesheng Wu, Understanding Data Similarity in Large-Scale Scientific Datasets, 2019 IEEE International Conference on Big Data (Big Data), Pages: 4525--4531 2019,

2018

Cheah You-Wei, Drew Paine, Devarshi Ghoshal, Lavanya Ramakrishnan, Bringing Data Science to Qualitative Analysis, 2018 IEEE 14th International Conference on e-Science, Pages: 325-326 2018, doi: 10.1109/eScience.2018.00076

D Ghoshal, L Ramakrishnan, D Agarwal, "Dac-Man: Data Change Management for Scientific Datasets on HPC Systems", SC ’18, Piscataway, NJ, USA, IEEE Press, 2018, 72:1--72:1,

GP Rodrigo, M Henderson, GH Weber, C Ophus, K Antypas, L Ramakrishnan, "ScienceSearch: Enabling Search through Automatic Metadata Generation", 2018 IEEE 14th International Conference on e-Science (e-Science), IEEE, 2018, doi: 10.1109/escience.2018.00025

Gunther H. Weber, Colin Ophus, Lavanya Ramakrishnan, "Automated Labeling of Electron Microscopy Images Using Deep Learning", Proc. IEEE/ACM Machine Learning in HPC Environments (MLHPC), 2018, 26--36, doi: 10.1109/MLHPC.2018.8638633

S Swaid, M Maat, H Krishnan, D Ghoshal, L Ramakrishnan, "Usability heuristic evaluation of scientific data analysis and visualization tools", Advances in Intelligent Systems and Computing, 2018, 607:471--482, doi: 10.1007/978-3-319-60492-3_45

E Deelman, T Peterka, I Altintas, CD Carothers, KK van Dam, K Moreland, M Parashar, L Ramakrishnan, M Taufer, J Vetter, "The future of scientific workflows", International Journal of High Performance Computing Applications, 2018, 32:159--175, doi: 10.1177/1094342017704893

GP Rodrigo, PO Östberg, E Elmroth, K Antypas, R Gerber, L Ramakrishnan, "Towards understanding HPC users and systems: A NERSC case study", Journal of Parallel and Distributed Computing, 2018, 111:206--221, doi: 10.1016/j.jpdc.2017.09.002

GP Rodrigo, E Elmroth, P-O Ostberg, L Ramakrishnan, "ScSF: A Scheduling Simulation Framework", Job Scheduling Strategies for Parallel Processing, Cham, Springer International Publishing, 2018, 152--173,

2017

Devarshi Ghoshal, Lavanya Ramakrishnan, "MaDaTS: Managing Data on Tiered Storage for Scientific Workflows", Proceedings of the 26th International Symposium on High-Performance Parallel and Distributed Computing (HPDC '17), ACM, 2017, 41--52, doi: 10.1145/3078597.3078611

L Ramakrishnan, D Gunter, "Ten principles for creating usable software for science", Proceedings - 13th IEEE International Conference on eScience, eScience 2017, 2017, 210--218, doi: 10.1109/eScience.2017.34

GH Weber, MS Bandstra, DH Chivers, HH Elgammal, V Hendrix, J Kua, JS Maltz, K Muriki, Y Ong, K Song, MJ Quinlan, L Ramakrishnan, BJ Quiter, "Web-based visual data exploration for improved radiological source detection", Concurrency Computation, 2017, 29, doi: 10.1002/cpe.4203

D Ghoshal, V Hendrix, W Fox, S Balasubhramanian, L Ramakrishnan, "FRIEDA: Flexible Robust Intelligent Elastic Data Management Framework", The Journal of Open Source Software, 2017, 2:164--164, doi: 10.21105/joss.00164

W Fox, D Ghoshal, A Souza, GP Rodrigo, L Ramakrishnan, "E-HPC: A library for elastic resource management in HPC environments", Proceedings of WORKS 2017: 12th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2017: The International Conference for High Performance Computing, Networking, Storage and Analysis, 2017, doi: 10.1145/3150994.3150996

GP Rodrigo, E Elmroth, P-O Östberg, L Ramakrishnan, "Enabling Workflow-Aware Scheduling on HPC Systems", Proceedings of the 26th International Symposium on High-Performance Parallel and Distributed Computing - HPDC 17, ACM Press, 2017, doi: 10.1145/3078597.3078604

2016

V Hendrix, J Fox, D Ghoshal, L Ramakrishnan, "Tigres Workflow Library: Supporting Scientific Pipelines on HPC Systems", Proceedings - 2016 16th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2016, 2016, 146--155, doi: 10.1109/CCGrid.2016.54

M Verma, JB Fisher, K Mallick, Y Ryu, H Kobayashi, A Guillaume, G Moore, L Ramakrishnan, V Hendrix, S Wolf, M Sikka, G Kiely, G Wohlfahrt, B Gielen, O Roupsard, P Toscano, A Arain, A Cescatti, "Global surface net-radiation at 5 km from MODIS Terra", Remote Sensing, 2016, 8, doi: 10.3390/rs8090739

CS Daley, D Ghoshal, GK Lockwood, S Dosanjh, L Ramakrishnan, NJ Wright, "Performance characterization of scientific workflows for the optimal use of Burst Buffers", CEUR Workshop Proceedings, 2016, 1800:69--73,

NC Chen, SS Poon, L Ramakrishnan, CR Aragon, "Considering time in designing Large-Scale systems for scientific computing", Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW, 2016, 27:1535--1547, doi: 10.1145/2818048.2819988

GP Rodrigo, P-O Ostberg, E Elmroth, K Antypas, R Gerber, L Ramakrishnan, IEEE, "Towards Understanding Job Heterogeneity in HPC: A NERSC Case Study", 2016 16TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND GRID COMPUTING (CCGRID), 2016, 521--526, doi: 10.1109/CCGrid.2016.32

E Dede, B Sendir, P Kuzlu, J Weachock, M Govindaraju, L Ramakrishnan, "Processing Cassandra Datasets with Hadoop-Streaming Based Approaches", IEEE Transactions on Services Computing, 2016, 9:46--58, doi: 10.1109/tsc.2015.2444838

2015

GP Rodrigo Álvarez, P-O Östberg, E Elmroth, L Ramakrishnan, A2L2, Proceedings of the 8th International Workshop on Virtualization Technologies in Distributed Computing - VTDC 15, 2015, doi: 10.1145/2755979.2755983

C Daley, L Ramakrishnan, S Dosanjh, N Wright, "Analyses of Scientific Workflows for Effective Use of Future Architectures", The 6th International Workshop on Big Data Analytics: Challenges, and Opportunities (BDAC-15), 2015 at SC, 2015,

E Feller, L Ramakrishnan, C Morin, "Performance and energy efficiency of big data applications in cloud environments: A Hadoop case study", Journal of Parallel and Distributed Computing, 2015, 79-80:80--89, doi: 10.1016/j.jpdc.2015.01.001

GP Rodrigo Álvarez, P-O Östberg, E Elmroth, K Antypas, R Gerber, L Ramakrishnan, "HPC System Lifetime Story: Workload Characterization and Evolutionary Analyses on NERSC Systems", Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing, 2015, 57--60,

H Sim, Y Kim, SS Vazhkudai, D Tiwari, A Anwar, AR Butt, L Ramakrishnan, ACM, "AnalyzeThis: An Analysis Workflow-Aware Storage System", PROCEEDINGS OF SC15: THE INTERNATIONAL CONFERENCE FOR HIGH PERFORMANCE COMPUTING, NETWORKING, STORAGE AND ANALYSIS, 2015, doi: 10.1145/2807591.2807622

2014

V Hendrix, L Ramakrishnan, Y Ryu, C Van Ingen, KR Jackson, D Agarwal, "CAMP: Community access MODIS pipeline", Future Generation Computer Systems, 2014, 36:418--429, doi: 10.1016/j.future.2013.09.023

Lavanya Ramakrishnan, Sarah S. Poon, Val C. Hendrix, Dan K. Gunter, Gilberto Z. Pastorello, Deb A. Agarwal, "Experiences with User-Centered Design for the Tigres Workflow API", Proceedings of the 10th IEEE International Conference on e-Science (e-Science 2014), Guaruja, Brazil, 2014, doi: 10.1109/eScience.2014.56

JR Balderrama, M Simonin, L Ramakrishnan, V Hendrix, C Morin, D Agarwal, C Tedeschi, "Combining workflow templates with a shared space-based execution model", Proceedings of WORKS 2014: The 9th Workshop on Workflows in Support of Large-Scale Science - held in conjunction with SC 2014: The International Conference for High Performance Computing, Networking, Storage and Analysis, 2014, 50--58, doi: 10.1109/WORKS.2014.14

L Ramakrishnan, D Ghoshal, V Hendrix, E Feller, P Mantha, C Morin, "Storage and Data Life Cycle Management in Cloud Environments with FRIEDA.", Cloud Computing for Data-Intensive Applications, (Springer: 2014) Pages: 357--378

Elif Dede, Zacharia Fadika, Madhusudhan Govindaraju, Lavanya Ramakrishnan, "Benchmarking MapReduce Implementations Under Different Application Scenarios", Future Generation Computer Systems, 2014,

Elif Dede, Zacharia Fadika, Madhusudhan Govindaraju, Lavanya Ramakrishnan, "MARIANE: Using MApReduce In HPC Environments", Future Generation Computer Systems, 2014,

Elif Dede, Bedri Sendir, Pinar Kuzlu, Madhusudhan Govindaraju, Lavanya Ramakrishnan, "A Processing Pipeline for Cassandra Datasets Based on Hadoop Streaming", IEEE International Congress on Big Data, 2014,

Tonglin Hawk, Ioan Raicu, Lavanya Ramakrishnan, "Scalable State Management for Scientific Applications in the Cloud", IEEE International Congress on Big Data, 2014,

2013

Devarshi Ghoshal, Lavanya Ramakrishnan, "FRIEDA: Flexible Robust Intelligent Elastic Data Management in Cloud Environments", 2012 SC Companion: High Performance Computing, Networking, Storage and Analysis (SCC), IEEE, 2013, 1096--1105, doi: 10.1109/SC.Companion.2012.132

Elif Dede, Madhusudhan Govindaraju, Daniel Gunter, Richard Canon, Lavanya Ramakrishnan, "Semi-Structured Data Analysis using MongoDB and MapReduce: A Performance Evaluation", Proceedings of the 4th international workshop on Scientific cloud computing, 2013,

E. Masanet, A. Shehabi, L. Ramakrishnan, J. Liang, X. Ma, B. Walker, V. Hendrix, P Mantha, "The Energy Efficiency Potential of Cloud-Based Software: A U.S.Case Study", 2013, LBNL 6298E,

Lavanya Ramakrishnan, Adam Scovel, Iwona Sakrejda, Susan Coghlan, Shane Canon, Anping Liu, Devarshi Ghoshal, Krishna Muriki, Nicholas J. Wright, "Magellan - A Testbed to Explore Cloud Computing for Science", On the Road to Exascale Computing: Contemporary Architectures in High Performance Computing, (Chapman & Hall/CRC Press: 2013)

Eugen Feller, Lavanya Ramakrishnan, Christine Morin, "On the Performance and Energy Efficiency of Hadoop Deployment Models", The IEEE International Conference on Big Data 2013 (IEEE BigData 2013), Santa Clara, U.S.A, 2013,

Lavanya Ramakrishnan, Iwona Sakrejda, Richard Shane Canon and Nicholas Wright, "CAMP", On the Road to Exascale Computing: Contemporary Architectures in High Performance Computing, (Chapman & Hall/CRC Press: 2013)

You-Wei Cheah, Richard Canon, Beth Plale, Lavanya Ramakrishnan, "Milieu: Provenance Collection and Query Framework for High Performance Computing Systems", IEEE Big Data Congress, 2013,

2012

Zacharia Fadika, Madhusudhan Govindaraju, Shane Canon, Lavanya Ramakrishnan, "Evaluting Hadoop for Data-Intensive Scientific Operations", IEEE Cloud Computing, 2012,

V Hendrix, J Li, K Jackson, L Ramakrishnan, Y Ryu, K Beattie, C Morin, D Skinner, C van Ingen, D Agarwal, "Community Access to MODIS Satellite Reprojection and Reduction Pipeline and Data Sets", AGU Fall Meeting, 2012,

D Gunter, S Cholia, A Jain, M Kocher, K Persson, L Ramakrishnan, SP Ong, G Ceder, "Community accessible datastore of high-throughput calculations: Experiences from the materials project", Proceedings - 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, SCC 2012, 2012, 1244--1251, doi: 10.1109/SC.Companion.2012.150

Elif Dede, Zacharia Fadika, Jessica Hartog, Modhusudhan Govindaraju, Lavanya Ramakrishnan, Daniel Gunter, Richard Shane Canon, "MARISSA: MApReduce Implementation for Streaming Science Applications", IEEE eScience Conference, 2012,

Lavanya Ramakrishnan, Richard Shane Canon, Krishna Muriki, Iwona Sakrejda, Nicholas J. Wright, "Evaluating Interconnect and Virtualization Performance for High Performance Computing", Special Issue of ACM Performance Evaluation Review, 2012, 40(2),

2011

Devarshi Ghoshal, Richard Shane Canon, Lavanya Ramakrishnan, "I/O Performance of Virtualized Cloud Environments", Proceedings of the Second International Workshop on Data Intensive Computing in the Clouds (DataCloud-SC '11), ACM, 2011, 71--80, doi: 10.1145/2087522.2087535

Elif Dede, Madhusudan Govindaraju, Daniel Gunter, Lavanya Ramakrishnan, "Riding the Elephant: Managing Ensembles with Hadoop", 4th Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS), 2011,

Zacharia Fadika, Elif Dede, Madhusudhan Govindaraju, Lavanya Ramakrishnan, "Benchmarking MapReduce Implementations for Application Usage Scenarios", Grid 2011: 12th IEEE/ACM International Conference on Grid Computing, Lyon Conference Centre, France, IEEE Computer Society, 2011, 1-8, doi: http://grid2011.mnm-team.org/?page_id138

Zacharia Fadika, Elif Dede, Madhusudhan Govindaraju, Lavanya Ramakrishnan, "MARIANE: MApReduce Implementation Adapted for HPC Environments", Grid 2011: 12th IEEE/ACM International Conference on Grid Computing, Lyon Conference Centre, France, IEEE Computer Society, 2011, 1-8, doi: http://grid2011.mnm-team.org/?page_id138

You-Wei Cheah, Beth Plale, Joey Kendall-Morwick, David Leake, Lavanya Ramakrishnan, "A Noisy 10GB Provenance Database", Second International Workshop on Traceability and Compliance of Semi-Structured Processes (TC4SP2011), 2011,

Lavanya Ramakrishnan, Piotr T. Zbiegel, Scott Campbell, Rick Bradshaw, Richard Shane Canon, Susan Coghlan, Iwona Sakrejda, Narayan Desai, Tina Declerck, Anping Liu, "Magellan: Experiences from a Science Cloud", ScienceCloud 11, New York, NY, USA, ACM, 2011, 49--58, doi: http://doi.acm.org/10.1145/1996109.1996119

Lavanya Ramakrishnan, Dennis Gannon, Jeffrey Chase, Daniel Nurmi, Rich Wolski, "Deadline-Sensitive Workflow Orchestration Without Explicit Resource Control", Journal of Parallel and Distributed Computing, 2011,

Keith R. Jackson, Krishna Muriki, Lavanya Ramakrishnan, Karl J. Runge, Rollin C. Thomas, "Performance and cost analysis of the Supernova factory on the Amazon AWS cloud", Sci. Program., 2011, 19:107--119,

2010

Keith Jackson, Lavanya Ramakrishnan, Rollin Thomas, Karl J. Runge, "Seeking Supernovae in the Clouds: A Performance Study", 1st Workshop on Scientific Cloud Computing, co-located with ACM HPDC 2010 (High Performance Distributed Computing), Chicago, IL, 2010,

Lavanya Ramakrishnan, Keith Jackson, Shane Canon, Shreyas Cholia, John Shalf, "Defining Future Platform Requirements for e-Science Cloud (Position paper)", ACM Symposium on Cloud Computing 2010 (ACM SOCC 2010), Indianapolis, Indiana, 2010,

Lavanya Ramakrishnan, Dennis Gannon, Beth Plale, "WORKEM: Representing and Emulating Distributed Scientific Workflow Execution State", 10th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2010), Melbourne Australia, 2010,

Y Simmhan, E Soroush, C Van Ingen, D Agarwal, L Ramakrishnan, "BReW: Blackbox resource selection for e-Science workflows", 2010 5th Workshop on Workflows in Support of Large-Scale Science, WORKS 2010, 2010, doi: 10.1109/WORKS.2010.5671857

L Ramakrishnan, C Guok, K Jackson, E Kissel, DM Swany, D Agarwal, "On-demand overlay networks for large scientific data transfers", CCGrid 2010 - 10th IEEE/ACM International Conference on Cluster, Cloud, and Grid Computing, 2010, 359--367, doi: 10.1109/CCGRID.2010.82

Lavanya Ramakrishnan, Beth Plale, "A Multi-Dimensional Classification Model for Scientific Workflow Characteristics", Workshop on Workflow Approaches to New Data-centric Science, Indianapolis, Indiana, 2010,

Yogesh Simmhan, Lavanya Ramakrishnan, "Comparison of Resource Platform Selection Approaches for Scientific Workflows", 1st Workshop on Scientific Cloud Computing, co-located with ACM HPDC 2010 (High Performance Distributed Computing), Chicago, Illinois, 2010,

KR Jackson, L Ramakrishnan, K Muriki, S Canon, S Cholia, J Shalf, HJ Wasserman, NJ Wright, "Performance analysis of high performance computing applications on the Amazon Web Services cloud", Proceedings - 2nd IEEE International Conference on Cloud Computing Technology and Science, CloudCom 2010, 2010, 159--168, doi: 10.1109/CloudCom.2010.69

2009

Lavanya Ramakrishnan, Daniel Nurmi, Anirban Mandal, Charles Koelbel, Dennis Gannon, T. Mark Huang, Yang-Suk Kee, Graziano Oberteli, Kiran Thyagaraja, Rich Wolski, Asim Yarkhan, Dmitrii Zagorodnov, "VGrADS: Enabling e-Science Workflows on Grids and Clouds with Fault Tolerance", Proceedings of the ACM/IEEE SC2009 Conference on High Performance Computing, Networking, Storage and Analysis, Portland, Oregon, Portland, Oregon, 2009,

Lavanya Ramakrishnan, Daniel A. Reed, "Predictable Quality of Service Atop Degradable Distributed Systems", Journal of Cluster Computing, 2009,

2008

Lavanya Ramakrishnan, Daniel A. Reed, "Performability Modeling for Scheduling and Fault Tolerance Strategies for Scientific Workflows", ACM/IEEE International Symposium on High Performance Distributed Computing, Boston Massachusetts, 2008,

Robert J. Fowler, Todd Gamblin, Gopi Kandaswamy, Anirban Mandal, Allan K. Porterfield, Lavanya Ramakrishnan, Daniel A. Reed, "Challenges of Scale: When All Computing Becomes Grid Computing", High Performance Computing and Grids in Action, (IOS Press: 2008)

2007

Lavanya Ramakrishnan, Yogesh Simmhan, Beth Plale, "Realization of Dynamically Adaptive Weather Analysis and Forecasting in LEAD", In Dynamic Data Driven Applications Systems Workshop (DDDAS) in conjunction with ICCS (Invited), Beijing, China, 2007,

2006

Lavanya Ramakrishnan, Laura Grit, Adriana Iamnitchi, David Irwin, Aydan Yumerefendi, Jeff Chase, "Toward a Doctrine of Containment: Grid Hosting with Adaptive Resource Control", Proceedings of the ACM/IEEE SC2006 Conference on High Performance Computing, Networking, Storage and Analysis, Tampa, Florida, Tampa, Florida, 2006,

Lavanya Ramakrishnan, Mark S.C Reed, Jeffrey L. Tilson, Daniel A. Reed, "Grid Portals for Bioinformatics", Second International Workshop on Grid Computing Environments (GCE), Held in conjunction with ACM/IEEE Conference for High Performance Computing, Networking, Storage and Analysis, Tampa, Florida, 2006,

Lavanya Ramakrishnan, Brian O. Blanton, Howard M. Lander, Richard A. Luettich, Jr, Daniel A. Reed, Steven R. Thorpe, "Real-time Storm Surge Ensemble Modeling in a Grid Environment", Second International Workshop on Grid Computing Environments (GCE), Held in conjunction ACM/IEEE Conference for High Performance Computing, Networking, Storage and Analysis, Tampa, Florida, 2006,

2005

Kelvin K. Droegemeier, Dennis Gannon, Daniel Reed, Beth Plale, Jay Alameda, Tom Baltzer, Keith Brewster, Richard Clark, Ben Domenico, Sara Graves, Everette Joseph, Donald Murray, Rahul Ramachandran, Mohan Ramamurthy, Lavanya Ramakrishnan, John A. Rushing, Daniel Weber, Robert Wilhelmson, Anne Wilson, Ming Xue, Sepideh Yalda, "Service-Oriented Environments for Dynamically Interacting with Mesoscale Weather", Computing in Science and Engg., 2005, 7:12--29, doi: http://dx.doi.org/10.1109/MCSE.2005.124

2004

Lavanya Ramakrishnan, "Securing Next-Generation Grids", IT Professional., 2004, 6:34-39,

2003

Timothy J. Smith, Lavanya Ramakrishnan, "Joint Policy Management and Auditing in Virtual Organizations.", Grid Computing, Phoenix, Arizona, 2003, 117-124,

Dennis Gannon, Rachana Ananthakrishnan, Sriram, Madhusudhan Govindaraju, Lavanya, Aleksander Slominski, "Grid Web Services and Application Factories", Grid Computing: Making the Global Infrastructure a Reality, (Wiley: 2003)

2002

Lavanya Ramakrishnan, Helen Nell Rehn, Jay, Rachana Ananthakrishnan, Madhusudhan, Aleksander Slominski, Kay Connelly Von Welch, Dennis Gannon, Randall Bramley, Hampton, "An Authorization Framework for a Grid Based Common Architecture", Proceedings of the 3rd International Workshop on Grid Baltimore, Maryland, Springer Press, 2002, 169--180,

Dennis Gannon, Randall Bramley, Geoffrey Fox, Shava Smallen, Al Rossi, Rachana Ananthakrishnan, Felipe Bertrand, Ken Chiu, Matt Farrellee, Madhu Govindaraju, Sriram Krishnan, Lavanya Ramakrishnan, Yogesh Simmhan, Alek Slominski, Yu Ma, Caroline Olariu, Nicolas Rey-Cenvaz, "Programming the Grid: Distributed Software Components, P2P and Grid Web Services for Scientific Applications", Journal of Cluster Computing, 2002, 5:325--336,

Oliver Rübel

2022

S. Zhang, R. Sadre, B. A. Legg, H. Pyles, T. Perciano, E. W. Bethel, D. Baker, O. Rübel, J. J. D. Yoreo, "Rotational dynamics and transition mechanisms of surface-adsorbed proteins", Proceedings of the National Academy of Sciences, April 11, 2022, 119:e202024211, doi: 10.1073/pnas.2020242119

2021

Hamish A. Carr, Gunther H. Weber, Christopher M. Sewell, Oliver R\ ubel, Patricia Fasel, James P. Ahrens, "Scalable Contour Tree Computation by Data Parallel Peak Pruning", Transactions on Visualization and Computer Graphics, 2021, 27:2437--2454, doi: 10.1109/TVCG.2019.2948616

Hamish Carr, Oliver Rübel, Gunther H. Weber, James Ahrens, "Optimization and Augmentation for Data Parallel Contour Trees", IEEE Transactions on Visualization and Computer Graphics, 2021, doi: 10.1109/TVCG.2021.3064385

2020

Petar Hristov, Gunther H. Weber, Hamish A. Carr, Oliver R\ ubel, James P. Ahrens, "Data Parallel Hypersweeps for In Situ Topological Analysis", Proceedings of the 10th IEEE Symposium on Large Data Analysis and Visualization (LDAV), 2020, 12--21, doi: 10.1109/LDAV51489.2020.00008

2019

Donghe Kang, Oliver Rübel, Suren Byna, Spyros Blanas, "Comparison of Array Management Library Performance - A Neuroscience Use Case", SC19 Poster, November 20, 2019,

Oliver Rübel, Andrew Tritt, Benjamin Dichter, Thomas Braun, Nicholas Cain, Nathan Clack, Thomas J. Davidson, Max Dougherty, Jean-Christophe Fillion-Robin, Nile Graddis, Michael Grauer, Justin T. Kiggins, Lawrence Niu, Doruk Ozturk, William Schroeder, Ivan Soltesz, Friedrich T. Sommer, Karel Svoboda, Lydia Ng, Loren M. Frank, Kristofer Bouchard, "NWB:N 2.0: An Accessible Data Standard for Neurophysiology", bioRxiv, January 17, 2019, doi: https://doi.org/10.1101/523035

2018

O. Erbilgin, O. Rübel, K. B. Louie, M. Trinh, M. de Raad, T. Wildish, D. W. Udwary, C. A. Hoover,, "MAGI: A Bayesian-like method for metabolite, annotation, and gene integration", bioRxiv, December 19, 2018, doi: 10.1101/204362

K. E. Bouchard, J.B. Aimone, M. Chun, T. Dean, M. Denker, M. Diesmann, D. Donofrio, L.M. Frank, N. Kasthuri, C. Koch, O. Rübel, H. Simon, F. T. Sommer, Prabhat, "International Neuroscience Initiatives Through the Lens of High-Performance Computing", IEEE Computer, April 12, 2018, 51(4):50-59, doi: doi 10.1109/MC.2018.2141039

O. Rübel, B. P. Bowen, "BASTet: Shareable and reproducible analysis and visualization of mass spectrometry imaging data via OpenMSI", IEEE Transactions on Visualization and Computer Graphics, January 12, 2018, 24,no. 1, doi: 10.1109/TVCG.2017.2744479

2017

M. de Raad, T. de Rond, O. Rübel, J. D. Keasling, T. R. Northen, B. P. Bowen, "OpenMSI Arrayed Analysis Toolkit: Analyzing Spatially Defined Samples Using Mass Spectrometry Imaging", ACS Analytical Chemistry, May 3, 2017, doi: 10.1021/acs.analchem.6b05004

2016

O. Rübel, M.Dougherty, Prabhat, P. Denes, D. Conant, E. F. Chang, and K. Bouchard, "Methods for Specifying Scienti c Data Standards and Modeling Relationships with Applications to Neuroscience," Frontiers in Neuroinformatics", Frontiers in Neuroinformatics, November 4, 2016, 10, doi: 10.3389/fninf.2016.00048

K. E. Bouchard, J, B. Aimone, M. Chun, T. Dean, M. Denker, M. Diesmann, D. D. Donofrio, L. M. Frank, N. Kasthuri, C. Koch, O. Rübel, H. D. Simon, F. T. Sommer, Prabhat, "High-Performance Computing in Neuroscience for Data-Driven Discovery, Integration, and Dissemination", Neuron, November 2, 2016, 92(3):628-631, doi: http://dx.doi.org/10.1016/j.neuron. 2016.10.035

Burlen Loring, Suren Byna, Prabhat, Junmin Gu, Hari Krishnan, Michael Wehner, and Oliver Ruebel, "TECA an Extreme Event Detection and Climate Analysis Package for High Performance Computing", The AMS (American Meteorological Society) 96th Annual Meeting, January 6, 2016,

O Rübel, B Loring, JL Vay, DP Grote, R Lehe, S Bulanov, H Vincenti, EW Bethel, "WarpIV: In Situ Visualization and Analysis of Ion Accelerator Simulations", IEEE Computer Graphics and Applications, 2016, 36:22--35, doi: 10.1109/MCG.2016.62

TA O Brien, WD Collins, K Kashinath, O Rübel, S Byna, J Gu, H Krishnan, PA Ullrich, Resolution dependence of precipitation statistical fidelity in hindcast simulations, Journal of Advances in Modeling Earth Systems, Pages: 976--990 2016, doi: 10.1002/2016MS000671

2015

J. Yang, O. Rübel, M. Mahoney, Prabhat, B.P. Bowen, "Identifying important ions and positions in mass spectrometry imaging data using CUR matrix decompositions", ACS Analytical Chemistry, March 2015, doi: 10.1021/ac5040264

2013

E Wes Bethel, Prabhat Prabhat, Suren Byna, Oliver R\ ubel, K John Wu, Michael Wehner, "Why high performance visual data analytics is both relevant and difficult", Visualization and Data Analysis 2013, January 2013, 8654:86540B, LBNL LBNL-6063E,

Kesheng Wu, E Bethel, Ming Gu, David Leinweber, Oliver R\ ubel, "A big data approach to analyzing market volatility", Algorithmic Finance, 2013, 2:241--267, LBNL LBNL-6382E,

Understanding the microstructure of the financial market requires the processing of a vast amount of data related to individual trades, and sometimes even multiple levels of quotes. Analyzing such a large volume of data requires tremendous computing power that is not easily available to financial academics and regulators. Fortunately, public funded High Performance Computing (HPC) power is widely available at the National Laboratories in the US. In this paper we demonstrate that the HPC resource and the techniques for data-intensive sciences can be used to greatly accelerate the computation of an early warning indicator called Volume-synchronized Probability of Informed trading (VPIN). The test data used in this study contains five and a half year's worth of trading data for about 100 most liquid futures contracts, includes about 3 billion trades, and takes 140GB as text files. By using (1) a more efficient file format for storing the trading records, (2) more effective data structures and algorithms, and (3) parallelizing the computations, we are able to explore 16,000 different ways of computing VPIN in less than 20 hours on a 32-core IBM DataPlex machine. Our test demonstrates that a modest computer is sufficient to monitor a vast number of trading activities in real-time -- an ability that could be valuable to regulators.

Our test results also confirm that VPIN is a strong predictor of liquidity-induced volatility. With appropriate parameter choices, the false positive rates are about 7% averaged over all the futures contracts in the test data set. More specifically, when VPIN values rise above a threshold (CDF > 0.99), the volatility in the subsequent time windows is higher than the average in 93% of the cases.

Kesheng Wu, Wes Bethel, Ming Gu, David, Oliver R\ ubel, Testing VPIN on Big Data, Available at SSRN 2318259, 2013,

O Rübel, A Greiner, S Cholia, K Louie, EW Bethel, TR Northen, BP Bowen, "OpenMSI: A high-performance web-based platform for mass spectrometry imaging", Analytical Chemistry, 2013, 85:10354--103, doi: 10.1021/ac402540a

2012

Hank Childs, Eric Brugger, Brad Whitlock, Jeremy Meredith, Sean Ahern, David Pugmire, Kathleen Biagas, Mark Miller, Cyrus Harrison, Gunther H. Weber, Hari Krishnan, Thomas Fogal, Allen Sanderson, Christoph Garth, E. Wes Bethel, David Camp, Oliver Rubel, Marc Durant, Jean M. Favre, Paul Navratil, "VisIt: An End-User Tool For Visualizing and Analyzing Very Large Data", High Performance Visualization---Enabling Extreme-Scale Scientific Insight, ( October 2012) Pages: 357--372

Oliver Rübel, Cameron, G. R. Geddes, Min Chen, Estelle Cormier-Michel, and E. Wes Bethel, "Query-driven Analysis of Plasma-based Particel Acceleration Data", Poster Abstracts of IEEE VisWeek, October 2012,

E. Wes Bethel, David Camp, Hank Childs, Mark Howison, Hari Krishnan, Burlen Loring, Joerg Meyer, Prabhat, Oliver Ruebel, Daniela Ushizima, Gunther Weber, "Towards Exascale: High Performance Visualization and Analytics – Project Status Report. Technical Report", DOE Exascale Research Conference, April 2012,

Jihan Kim, Richard L. Martin, Oliver Rübel, Maciej Haranczyk & Berend Smit, "High-throughput Characterization of Porous Materials Using Graphics Processing Units", Journal of Chemical Theory and Computation, March 16, 2012, 8:1684–1693, LBNL 5409E, doi: 10.1021/ct200787v

We have developed a high-throughput graphics processing unit (GPU) code that can characterize a large database of crystalline porous materials. In our algorithm, the GPU is utilized to accelerate energy grid calculations, where the grid values represent interactions (i.e., Lennard-Jones + Coulomb potentials) between gas molecules (i.e., CH4 and CO2) and materials’ framework atoms. Using a parallel flood fill central processing unit (CPU) algorithm, inaccessible regions inside the framework structures are identified and blocked, based on their energy profiles. Finally, we compute the Henry coefficients and heats of adsorption through statistical Widom insertion Monte Carlo moves in the domain restricted to the accessible space. The code offers significant speedup over a single core CPU code and allows us to characterize a set of porous materials at least an order of magnitude larger than those considered in earlier studies. For structures selected from such a prescreening algorithm, full adsorption isotherms can be calculated by conducting multiple Grand Canonical Monte Carlo (GCMC) simulations concurrently within the GPU.

Soile V.E. Keränen, Oliver Rübel, David W. Knowles and Mark D. Biggin, "Computational modeling of cis-regulatory modules from 3D exprression data in Drosophila blastoderm atlas", Drosophila Genetics, March 2012,

Surendra Byna, Jerry Chou, Oliver Rubel, Homa Karimabadi, William S Daughter, Vadim Roytershteyn, E Wes Bethel, Mark Howison, Ke-Jou Hsu, Kuan-Wu Lin, others, "Parallel I/O, analysis, and visualization of a trillion particle simulation", SC 12: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, January 2012, 1--12,

Oliver R\ ubel, Surendra Byna, Kesheng Wu, Fuyu Li, Michael Wehner, Wes Bethel, others, "Teca: A parallel toolkit for extreme climate analysis", Procedia Computer Science, Elsevier, January 2012, 9:866--876, LBNL 5352E,

We present TECA, a parallel toolkit for detecting extreme events in large climate datasets. Modern climate datasets expose parallelism across a number of dimensions: spatial locations, timesteps and ensemble members. We design TECA to exploit these modes of parallelism and demonstrate a prototype implementation for detecting and tracking three classes of extreme events: tropical cyclones, extra-tropical cyclones and atmospheric rivers. We process a modern TB-sized CAM5 simulation dataset with TECA, and demonstrate good runtime performance for the three case studies.

E. W. Bethel, Surendra Byna, Jerry Chou, Cormier-Michel, Cameron G. R. Geddes, Howison, Fuyu Li, Prabhat, Ji Qiang, R\ ubel, Rob D. Ryne, Michael Wehner, Wu, "Big Data Analysis and Visualization: What Do LINACS Tropical Storms Have In Common?", 11th International Computational Accelerator Physics ICAP 2012, Germany, 2012,

Allen R Sanderson, Brad Whitlock, H Childs, GH Weber, K Wu, others, "A system for query based analysis and visualization", January 2012, LBNL 5507E,

E. W. Bethel and D. Leinweber and O. Rubel and K. Wu, "Federal Market Information Technology in the Post Flash Crash Era: Roles of Supercomputing", The Journal of Trading, 2012, 7:9-24, LBNL 5263E, doi: 10.3905/jot.2012.7.2.009

E. Wes Bethel, David Leinweber, Oliver Rübel Kesheng Wu, Federal Market Information Technology in the Crash Era: Roles for Supercomputing, The Journal of Trading, Pages: 9--25 2012, doi: 10.3905/jot.2012.7.2.009

O. Rübel, S.V.E. Keränen, M.D. Biggin, D.W. Knowles, G.H. Weber, H. Hagen, B. Hamann, and E.W. Bethel, "Linking Advanced Visualization and MATLAB for the Analysis of 3D Gene Expression Data", Mathematics and Visualization, Visualization in Medicine and Life Sciences II, Progress and New Challenges, edited by L. Linsen and B. Hamann and H. Hagen and H.-C. Hege, (Springer Verlag: 2012) Pages: 267-285, LBNL 4891E,

Samuel Gerber, Oliver Rübel, Peer-Timo Bremer, Valerio Pascucci and Ross T. Whitaker, "Morse-Smale Regression", Journal of Computational and Graphical Statistics, January 2012, doi: 10.1080/10618600.2012.657132

  • Download File: MSR.pdf (pdf: 292 KB)

2011

E. Wes Bethel, David Leinweber, Oliver Rübel, Kesheng Wu, "Federal Market Information Technology in the Post Flash Crash Era: Roles of Supercomputing", Workshop on High Performance Computational Finance at SC11, Seattle, WA, USA, November 2011, LBNL 5263E,

R. Ryne, B. Austin, J. Byrd, J. Corlett, E. Esarey, C. G. R. Geddes, W. Leemans, X. Li, Prabhat, J. Qiang, O. Rübel, J.-L. Vay, M. Venturini, K. Wu, B. Carlsten, D. Higdon and N. Yampolsky, "High Performance Computing in Accelerator Science: Past Successes, Future Challenges", Workshop on Data and Communications in Basic Energy Sciences: Creating a Pathway for Scientific Discovery, October 2011,

Jerry Chou, Mark Howison, Brian Austin, Kesheng Wu, Ji Qiang, E Wes Bethel, Arie Shoshani, Oliver R\ ubel, Rob D Ryne, "Parallel index and query for large scale data analysis", Proceedings of 2011 international conference for high performance computing, networking, storage and analysis, 2011, 1--11, LBNL 5317E,

J. Chou, K. Wu, O. R\ ubel, M. Howison, Qiang, Prabhat, B. Austin, E. W. Bethel, D. Ryne, A. Shoshani, "Parallel Index and Query for Large Scale Data", SC11, 2011, doi: 10.1145/2063384.2063424

Mark Howison, Mike McGreevy, Bruce Palmer, Oliver Ruebel, Kesheng Wu, "ExaHDF5: An I/O Platform for Exascale Data Models, Analysis and Performance", 2011,

E. W. Bethel, D. Leinweber, O. Rübel, K., Federal Market Information Technology in the Post Crash Era: Roles for Supercomputing, WHPCF, Pages: 23--30 2011, doi: 10.1145/2088256.2088267

2010

O. Rübel, Linking Automated Data Analysis and Visualization with Applications in Developmental Biology and High-energy Physics, Schriftenreihe Informatik, (Der Dekan (hrsg), Fachbereich Informatik, Technische Universität Kaiserslautert: December 2010) LBNL 3155E,

O. Rübel, G. H. Weber, M-Y Huang, E. W. Bethel, M. D. Biggin, C. C. Fowlkes, C. Luengo Hendriks, S. V. E. Keränen, M. Eisen, D. Knowles, J. Malik, H. Hagen and B. Hamann, "Integrating Data Clustering and Visualization for the Analysis of 3D Gene Expression Data", IEEE Transactions on Computational Biology and Bioinformatics, March 2010, 7:64-79, LBNL 382E,

Daniela Ushizima, Cameron Geddes, Estelle Cormier-Michel, E. Wes Bethel, Janet Jacobsen, Prabhat, Oliver Rubel, Gunther Weber, Bernard Hamann, Peter Messmer, Hans Hagen, "Automated detection and analysis of particle beams in laser-plasma accelerator simulations", Machine Learning, edited by Yagang Zhang, (In-Teh: February 2010) Pages: 367-389, LBNL 3845E,

Oliver R\ ubel, Sean Ahern, E Wes Bethel, Mark D Biggin, Hank Childs, Estelle Cormier-Michel, Angela DePace, Michael B Eisen, Charless C Fowlkes, Cameron GR Geddes, others, "Coupling visualization and data analysis for knowledge discovery from multi-dimensional scientific data", Procedia computer science, Elsevier, January 2010, 1:1757--1764, LBNL 3669E,

Gunther Weber, "Recent advances in visit: Amr streamlines and query-driven visualization", 2010,

Oliver Rübel, Sean Ahern, E. Wes Bethel, D. Biggin, Hank Childs, Estelle, Angela DePace, Michael B. Eisen Charless C. Fowlkes, Cameron G. R. Geddes, Hagen, Bernd Hamann, Min-Yu Huang, Soile E. Keränen, David W. Knowles, Cris L. Hendriks, Jitendra Malik, Jeremy Meredith Peter Messmer, Prabhat, Daniela Ushizima, H. Weber, Kesheng Wu, "Coupling visualization and data analysis for knowledge from multi-dimensional scientific data", Procedia Computer Science, 2010, 1:1751--1758, doi: 10.1016/j.procs.2010.04.197

2009

E. W. Bethel, C. Johnson, S. Ahern, J. Bell, P.-T. Bremer, H. Childs, E. Cormier-Michel, M. Day, E. Deines, T. Fogal, C. Garth, C. G. R. Geddes, H. Hagen, B. Hamann, C. Hansen, J. Jacobsen, K. Joy, J. Kruger, J. Meredith, P. Messmer, G. Ostrouchov, V. Pascucci, K. Potter, Prabhat, D. Pugmire, O. Rubel, A. Sanderson, C. Silva, D. Ushizima, G. Weber, B. Whitlock, K. Wu, "Occam's Razor and Petascale Visual Data Analysis", SciDAC 2009, J. of Physics: Conference Series, San Diego, California, July 2009, LBNL 2210E,

K Wu, S Ahern, EW Bethel, J Chen, H Childs, C Geddes, J Gu, H Hagen, B Hamann, J Lauret, others, "FastBit: Interactively Searching Massive Data", Proc. of SciDAC 2009, 2009, LBNL 2164E,

Oliver R\ ubel, Cameron GR Geddes, Estelle Cormier-Michel, Kesheng Wu, Gunther H Weber, Daniela M Ushizima, Peter Messmer, Hans Hagen, Bernd Hamann, Wes Bethel, others, "Automatic beam path analysis of laser wakefield particle acceleration data", Computational Science \& Discovery, January 2009, 2:015005, LBNL 2734E,

C. G. R. Geddes, E Cormier-Michel, E. H. Esarey, C. B. Schroeder, J.-L. Vay, W. P. Leemans, D. L.. Bruhwiler, J. R. Cary, B. Cowan, M. Durant, P. Hamill, P. Messmer, P. Mullowney, C. Nieter, K. Paul, S. Shasharina, S. Veitzer, G. Weber, O. Rübel, D. Ushizima, Prabhat, E. W.Bethel, K. Wu, Large Fields for Smaller Facility Sources, SciDAC Review, Pages: 13-21, 2009,

E Bethel, "Modern Scientific Visualization is More than Just Pretty Pictures", January 2009, LBNL 1450E,

E Wes Bethel, Chris Johnson, Sean Ahern, John Bell, Peer-Timo Bremer, Hank Childs, Estelle Cormier-Michel, Marc Day, Eduard Deines, Tom Fogal, others, "Occam s razor and petascale visual data analysis", Journal of Physics: Conference Series, 2009, 180:012084,

Oliver R\ ubel, Cameron G R Geddes, Estelle, Kesheng Wu, Prabhat, Gunther H, Daniela M Ushizima, Peter Messmer, Hans, Bernd Hamann, Wes Bethel, "Automatic beam path analysis of laser wakefield acceleration data", Computational Science \& Discovery, 2009, 2:015005,

G. H. Weber, O. Rübel, M.-Y. Huang, A. H. DePace, C. C. Fowlkes, S. V. E. Keränen, C. L. Luengo Hendriks, H. Hagen, D. W. Knowles, J. Malik, M. D. Biggin and B. Hamann, "Visual exploration of three-dimensional gene expression using physical views and linked abstract views", IEEE Transactions on Computational Biology and Bioinformatics, 2009, 6:296-309, LBNL 63776, doi: 10.1109/TCBB.2007.70249

2008

O. Rübel, Prabhat, K. Wu, H. Childs, J. Meredith, C.G.R. Geddes, E. Cormier-Michel, S. Ahern, G.H. Weber, P. Messmer, H. Hagen, B. Hamann and E.W. Bethel, "High Performance Multivariate Visual Data Exploration for Extemely Large Data", Supercomputing (SC), Austin, Texas, USA, November 2008, LBNL 716E,

O. Rübel, Prabhat, K. Wu, H. Childs, J. Meredith, C.G.R. Geddes, E. Cormier-Michel, S. Ahern, G.H. Weber, P. Messmer, H. Hagen, B. Hamann and E.W. Bethel, "Application of High-performance Visual Analysis Methods to Laser Wakefield Particle Acceleration Data", IEEE Visualization 2008, October 2008,

C. C. Fowlkes, C. L. Luengo Hendriks, S. V. E. Keränen, G. H. Weber, O. Rübel, M.-Y. Huang, S. Chatoor, A. H. DePace, L. Simirenko, C. Henriquez, A. Beaton, R. Weiszmann, S. Celniker, B. Hamann, D. W. Knowles, M. D. Biggin, M. B. Eisen, J. Malik, "A Quantitative Spatio-temporal Atlas of Gene Expression in the Drosophila Blastoderm", Cell, April 18, 2008, 133:364-374,

E. Wes Bethel, Oliver Rübel, Prabhat, Wu, Gunther H. Weber, Valerio Pascucci Hank Childs, Ajith Mascarenhas, Jeremy, Sean Ahern, "Modern Scientific Visualization is More than Just Pictures", Numerical Modeling of Space Plasma Flows: (Astronomical Society of the Pacific Series), St. Thomas, USVI, 2008, 301--317,

Oliver R\ ubel, Prabhat, Kesheng Wu, Hank, Jeremy Meredith, Cameron G. R. Geddes, Cormier-Michel, Sean Ahern, Gunther H., Peter Messmer, Hans Hagen, Bernd Hamann E. Wes Bethel, Application of High-performance Visual Analysis to Laser Wakefield Particle Acceleration Data, IEEE Visualization 2008, 2008,

Oliver R\ ubel, Prabhat, Kesheng Wu, Hank, Jeremy Meredith, Cameron G. R. Geddes, Cormier-Michel, Sean Ahern, Gunther H., Peter Messmer, Hans Hagen, Bernd Hamann E. Wes Bethel, High Performance Multivariate Visual Data Exploration Extemely Large Data, SuperComputing 2008 (SC08), Pages: 51 2008,

Daniela Ushizima, Oliver Rübel, Prabhat, Gunther Weber, E. Wes Bethel, Cecilia Aragon, Cameron Geddes, Estelle Cormier-Michel, Bernd Hamann, Peter Messmer, Hans Hagen, "Automated Analysis for Detecting Beams in Laser Wakefield Simulations", 2008 Seventh International Conference on Machine Learning and Applications, Proceedings of IEEE ICMLA'08, 2008, 382-387, LBNL 960E,

O. Rübel, G. H. Weber, M-Y Huang, E. W. Bethel, S. V. E. Keränen, C. C. Fowlkes, C. L. Luengo Hendriks, A. H. DePace, L. Simirenko, M. B. Eisen, M. D. Biggin, H. Hagen, J. Malik, D. W. Knowles and B. Hamann, "PointCloudXplore 2: Visual Exploration of 3D Gene Expression", Visualization of Large and Unstructured Data Sets, edited by C. Garth, H. Hagen, M. Hering-Bertram, (Gesellschaft fuer Informatik (GI): 2008) LBNL 249E,

M.-Y. Huang, O. Rübel, G.H. Weber, C.L. Luengo Hendriks, M.D. Biggin, H. Hagen, B. Hamann, "Segmenting Gene Expression Patterns of Early-stage Drosophila Embryos.", Mathematical Methods for Visualization in Medicine and Life Sciences, edited by L. Linsen, H. Hagen, B. Hamann, (Springer-Verlag: January 2008) Pages: 313--327, LBNL 62450,

2006

Cris L. Luengo Hendriks, Soile V. E. Keränen, C. Fowlkes, Lisa Simirenko, Gunther H. Weber, H. DePace, Clara Henriquez, David W. Kaszuba, Hamann, Michael B. Eisen, Jitendra Malik, Damir Sudar, D. Biggin, David W. Knowles, "Three-dimensional Morphology and Gene Expression in the Drosophila Blastoderm at Cellular Resolution I: Data Acquisition Pipeline", Genome Biology, 2006, 7:R123, doi: 10.1186/gb-2006-7-12-r123

Arie Shoshani

2019

Beytullah Yildiz, Kesheng Wu, Suren Byna, Arie Shoshani, "Parallel membership queries on very large scientific data sets using bitmap indexes", Concurrency and Computation: Practice and Experience, January 1, 2019, 31:e5157,

Many scientific applications produce very large amounts of data as advances in hardware fuel computing and experimental facilities. Managing and analyzing massive quantities of scientific data is challenging as data are often stored in specific formatted files, such as HDF5 and NetCDF, which do not offer appropriate search capabilities. In this research, we investigated a special class of search capability, called membership query, to identify whether queried elements of a set are members of an attribute. Attributes that naturally have classification values appear frequently in scientific domains such as category and object type as well as in daily life such as zip code and occupation. Because classification attribute values are discrete and require random data access, performing a membership query on a large scientific data set creates challenges. We applied bitmap indexing and parallelization to membership queries to overcome these challenges. Bitmap indexing provides high performance not only for low cardinality attributes but also for high cardinality attributes, such as floating‐point variables, electric charge, or momentum in a particle physics data set, due to compression algorithms such as Word‐Aligned Hybrid. We conducted experiments, in a highly parallelized environment, on data obtained from a particle accelerator model and a synthetic data set.

2016

Deborah A Agarwal, Boris Faybishenko, Vicky L Freedman, Harinarayan Krishnan, Gary Kushner, Carina Lansing, Ellen Porter, Alexandru Romosan, Arie Shoshani, Haruko Wainwright, others, "A science data gateway for environmental management", Concurrency and Computation: Practice and Experience, 2016, 28:1994--2004,

2015

Xiaocheng (Chris) Zou, Suren Byna, Hans Johansen, Daniel Martin, Nagiza F. Samatova, Arie Shoshani, John Wu, "Six-fold Speedup of Ice Calving Detection Achieved by AMR-aware Parallel Connected Component Labeling", SciDAC PI Meeting, July 2015, 2015,

Elaheh Pourabbas, Arie Shoshani, "The Composite Data Model: A Unified Approach for Combining and Querying Multiple Data Models", IEEE Trans. Knowl. Data Eng, 2015, 27(5):1424-1437,

2014

Spyros Blanas, Kesheng Wu, Surendra Byna, Bin Dong, Arie Shoshani, "Parallel Data Analysis Directly on Scientific File", SIGMOD 14, 2014, 385--396, doi: 10.1145/2588555.2612185

US Patent 8,705,342 B2. “Co-scheduling of network resource provisioning and host-to-host bandwidth reservation on high-performance network and storage systems”, D. Yu, D. Katramatos, A. Sim, and A. Shoshani, Apr. 22, 2014, LBNL IB-3152, BNL BSA 11-02.

Spyros Blanas, Kesheng Wu, Surendra Byna, Bin Dong, Arie Shoshani, "Parallel Data Analysis Directly on Scientific File Formats", SIGMOD 14, 2014, 385--396, doi: 10.1145/2588555.2612185

DP Schissel, Gheni Abla, SM Flanagan, M Greenwald, X Lee, A Romosan, A Shoshani, J Stillerman, J Wright, "Automated metadata, provenance cataloging and navigable interfaces: Ensuring the usefulness of extreme-scale data", Fusion Engineering and Design, North-Holland, 2014,

John C Wright, Martin Greenwald, Joshua Stillerman, Gheni Abla, Bobby Chanthavong, Sean Flanagan, David Schissel, Xia Lee, Alex Romosan, Arie Shoshani, The MPO API: A tool for recording scientific workflows, Fusion Engineering and Design, 2014,

Qian Sun, Fan Zhang, Tong Jin, Hoang Bui, Kesheng Wu, Arie Shoshani, Hemanth Kolla, Scott Klasky, Jacqueline Chen, Manish Parashar, "Scalable run-time data indexing and querying for scientific simulations", Big Data Analytics: Challenges and Opportunities (BDAC-14) Workshop at Supercomputing Conference, 2014,

Spyros Blanas, Kesheng Wu, Surendra Byna, Bin Dong, Arie Shoshani, "Parallel data analysis directly on scientific file formats", Proceedings of the 2014 ACM SIGMOD international conference on Management of data, January 1, 2014, 385--396,

2013

Alex Romosan, Arie Shoshani, Kesheng Wu, Victor Markowitz, Kostas Mavrommatis, "Accelerating gene context analysis using bitmaps", Proceedings of the 25th International Conference on Scientific and Statistical Database Management, 2013, 1--12, LBNL 6397E,

2012

Karen L. Schuchardt, Deborah A. Agarwal, Stefan A. Finsterle, Carl W. Gable, Ian Gorton, Luke J. Gosink, Elizabeth H. Keating, Carina S. Lansing, Joerg Meyer, William A.M. Moeglein, George S.H. Pau, Ellen A. Porter, Sumit Purohit, Mark L. Rockhold, Arie Shoshani, and Chandrika Sivaramakrishnan, Akuna, "Integrated Toolsets Supporting Advanced Subsurface Flow and Transport Simulations for Environmental Management", XIX International Conference on Computational Methods in Water Resources (CMWR 2012), University of Illinois at Urbana-Champaign, June 2012,

D. Yu, D. Katramatos, A. Shoshani, A. Sim, J. Gu, V. Natarajan, "StorNet: Integrating Storage Resource Management with Dynamic Network Provisioning for Automated Data Transfer", International Committee for Future Accelerators (ICFA) Standing Committee on Inter-Regional Connectivity (SCIC) 2012 Report: Networking for High Energy Physics, 2012,

Karen L. Schuchardt, Deborah A. Agarwal, Stefan A. Finsterle, Carl W. Gable, Ian Gorton, Luke J. Gosink, Elizabeth H. Keating, Carina S. Lansing, Joerg Meyer, William A.M. Moeglein, George S.H. Pau, Ellen A. Porter, Sumit Purohit, Mark L. Rockhold, Arie Shoshani, Chandrika Sivaramakrishnan, "Akuna-Integrated Toolsets Supporting Advanced Subsurface Flow and Transport Simulations for Environmental Management", XIX International Conference on Computational Methods in Water Resources (CMWR 2012), University of Illinois at Urbana-Champaign, June 17-22, 2012, 2012,

Surendra Byna, Jerry Chou, Oliver Rubel, Homa Karimabadi, William S Daughter, Vadim Roytershteyn, E Wes Bethel, Mark Howison, Ke-Jou Hsu, Kuan-Wu Lin, others, "Parallel I/O, analysis, and visualization of a trillion particle simulation", SC 12: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, January 2012, 1--12,

Elaheh Pourabbas, Arie Shoshani, Kesheng Wu, "Minimizing index size by reordering rows and columns", International Conference on Scientific and Statistical Database Management, January 2012, 467--484,

Benson Ma, Arie Shoshani, Alex Sim, Kesheng, Yong-Ik Byun, Jaegyoon Hahm, Min-Su Shin, "Efficient Attribute-Based Data Access in Astronomy", The 2nd International Workshop on Network-Aware Data Workshop (NDM2012), 2012, 562--571,

G. F. Lofstead, Q. Liu, J. Logan, Y. Tian, Abbasi, N. Podhorszki, J. Y. Choi, S., R. Tchoua, R. A. Oldfield, others, "Hello ADIOS: The Challenges and Lessons of Leadership Class I/O Frameworks", 2012,

2011

J. Gu, D. Katramatos, X. Liu, V. Natarajan, A. Shoshani, A. Sim, D. Yu, S. Bradley, S. McKee, "StorNet: Integrated Dynamic Storage and Network Resource Provisioning and Management for Automated Data Transfers", Journal of Physics: Conf. Ser., 2011, 331, doi: 10.1088/1742- 6596/331/1/012002

A. Shoshani, I. Altintas, J. Chen, G. Chin, A. Choudhary, D. Crawl, T. Critchlow, K. Gao, B. Grimm, H. Iyer, C. Kamath, A. Khan, S. Klasky, S. Koehler, S. Lang, R. Latham, J. W. Li, W. Liao, J. Ligon, Q. Liu, B. Ludaescher, P. Mouallem, M. Nagappan, N. Podhorszki, R. Ross, D. Rotem, N. Samatova, C. Silva, A. Sim, R. Tchoua, R. Thakur, M. Vouk, K. Wu, W. Yu, "The Scientific Data Management Center: Available Technologies and Highlights", SciDAC Conference, 2011,

Junmin Gu, Dimitrios Katramatos, Xin Liu, Vijaya Natarajan, Arie Shoshani, Alex Sim, Dantong Yu, Scott Bradley, Shawn McKee, "StorNet: Co-Scheduling of End-to-End Bandwidth Reservation on Storage and Network Systems for High Performance Data Transfers", IEEE INFOCOM HSN 2011, 2011,

Jerry Chou, Mark Howison, Brian Austin, Kesheng Wu, Ji Qiang, E Wes Bethel, Arie Shoshani, Oliver R\ ubel, Rob D Ryne, "Parallel index and query for large scale data analysis", Proceedings of 2011 international conference for high performance computing, networking, storage and analysis, 2011, 1--11, LBNL 5317E,

Kesheng Wu, Rishi R Sinha, Chad Jones, Stephane Ethier, Scott Klasky, Kwan-Liu Ma, Arie Shoshani, Marianne Winslett, "Finding regions of interest on toroidal meshes", Computational Science \& Discovery, 2011, 4:015003,

Kesheng Wu, Surendra Byna, Doron Rotem, Arie, "Scientific Data Services -- A High-Performance I/O with Array Semantics", HPCDB, IEEE, 2011, doi: 10.11v45/2125636.2125640

J. Chou, K. Wu, O. R\ ubel, M. Howison, Qiang, Prabhat, B. Austin, E. W. Bethel, D. Ryne, A. Shoshani, "Parallel Index and Query for Large Scale Data", SC11, 2011, doi: 10.1145/2063384.2063424

Jinoh Kim, Hasan Abbasi, Luis Chac\ on, Docan, Scott Klasky, Qing Liu, Norbert, Arie Shoshani, Kesheng Wu, "Parallel In Situ Indexing for Data-intensive", LDAV, 2011, 65--72, doi: 10.1109/LDAV.2011.6092319

Dean N. Williams, Ian T. Foster, Don E. Middleton, Rachana Ananthakrishnan, Neill Miller, Mehmet Balman, Junmin Gu, Vijaya Natarajan, Arie Shoshani, Alex Sim, Gavin Bell, Robert Drach, Michael Ganzberger, Jim Ahrens, Phil Jones, Daniel Crichton, Luca Cinquini, David Brown, Danielle Harper, Nathan Hook, Eric Nienhouse, Gary Strand, Hannah Wilcox, Nathan Wilhelmi, Stephan Zednik, Steve Hankin, Roland Schweitzer, John Harney, Ross Miller, Galen Shipman, Feiyi Wang, Peter Fox, Patrick West, Stephan Zednik, Ann Chervenak, Craig Ward, "Earth System Grid Center for Enabling Technologies (ESG-CET): A Data Infrastructure for Data-Intensive Climate Research", SciDAC Conference, 2011,

2010

Alex Sim, Mehmet Balman, Dean N. Williams, Arie Shoshani, Vijaya Natarajan, "Adaptive Transfer Adjustment in Efficient Bulk Data Transfer Management for Climate Datasets", The 22nd IASTED International Conference on Parallel and Distributed Computing and System, Marina Del Rey, CA, November 20, 2010, LBNL 3985E,

Many scientific applications and experiments, such as high energy and nuclear physics, astrophysics, climate observation and modeling, combustion, nano-scale material sciences, and computational biology, generate extreme volumes of data with a large number of files. These data sources are distributed among national and international data repositories, and are shared by large numbers of geographically distributed scientists. A large portion of the data is frequently accessed, and a large volume of data is moved from one place to another for analysis and storage. A challenging issue in such efforts is the limited network capacity for moving large datasets. A tool that addresses this challenge is the Bulk Data Mover (BDM), a data transfer management tool used in the Earth System Grid (ESG) community. It has been managing massive dataset transfers efficiently in the environment where the network bandwidth is limited. Adaptive transfer adjustment was studied to enhance the BDM to handle significant end-to-end performance changes in the dynamic network environments as well as to control the data transfers for the desired transfer performance. We describe the results from our hands-on data transfer management experience in the climate research community. We study a practical transfer estimation model and state our initial results from the adaptive transfer adjustment methodology. 

Mehmet Balman, Evangelos Chaniotakis, Arie Shoshani, Alex Sim, "A Flexible Reservation Algorithm for Advance Network Provisioning", ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, New Orleans, LA, November 2010 (SC'10)., New Orleans, LA, IEEE Computer Society Washington, DC, USA ISBN: 978-1-4244-7559-, November 14, 2010, LBNL 4017E, doi: http://dx.doi.org/10.1109/SC.2010.4

Many scientific applications need support from a communication infrastructure that provides predictable performance, which requires effective algorithms for bandwidth reservations. Network reservation sys- tems such as ESnet’s OSCARS, establish guaranteed bandwidth of secure virtual circuits for a certain bandwidth and length of time. However, users currently cannot inquire about bandwidth availability, nor have alternative suggestions when reservation requests fail. In general, the number of reservation options is exponential with the number of nodes n, and current reservation commitments. We present a novel approach for path finding in time-dependent networks taking advantage of user-provided parameters of total volume and time constraints, which produces options for earliest completion and shortest duration. The theoretical complexity is only O(n2r2) in the worst-case, where r is the number of reservations in the desired time interval. We have implemented our algorithm and developed efficient methodologies for incorporation into network reservation frameworks. Performance measurements confirm the theoretical predictions. 

M. Balman, E. Chaniotakis, A. Shoshani, A. Sim, "A New Approach in Advance Network Reservation and Provisioning for High-Performance Scientific Data Transfers", 2010, LBNL 4091E,

Julian Cummings, Jay Lofstead, Karsten Schwan, Alexander Sim, Arie Shoshani, Ciprian Docan, Manish Parashar, Scott Klasky, Norbert Podhorszki, Roselyne Barreto, "EFFIS: An End-to-end Framework for Fusion Integrated Simulation", 18th Euromicro Conference on Parallel, Distributed and Network-based Processing, 2010,

A. Sim, D. Gunter, V. Natarajan, A. Shoshani, D. Williams, J. Long, J. Hick, J. Lee, E. Dart, "Efficient Bulk Data Replication for the Earth System Grid", Data Driven E-science: Use Cases and Successful Applications of Distributed Computing Infrastructures (ISGC 2010), (Springer-Verlag New York Inc: 2010) Pages: 435

Kesheng Wu, Arie Shoshani, Kurt Stockinger, "Analyses of multi-level and multi-component compressed indexes", ACM Transactions on Database Systems, ACM, 2010, 35:1--52, doi: 10.1145/1670243.1670245

E. Pourabbas, A. Shoshani, "Improving Estimation Accuracy of Aggregate Queries on Data Cubes", Data & Knowledge Engineering 69 (2010), January 1, 2010, 69:50-72,

2009

Scientific Data Management: Challenges, Technology, and Deployment, edited by Arie Shoshani and Doron Rotem, (Chapman & Hall/CRC Computational Science: December 2009)

A. Sim, A. Shoshani, F. Donno, J. Jensen, Storage Resource Manager Interface Specification V2.2 Implementations Experience Report, Open Grid Forum, GFD.154, 2009,

D. N. Williams, R. Ananthakrishnan, D. E. Bernholdt, S. Bharathi, D. Brown, M. Chen, A. L. Chervenak, L. Cinquini, R. Drach, I. T. Foster, P. Fox, D. Fraser, J. Garcia, S. Hankin, P. Jones, D. E. Middleton, J. Schwidder, R. Schweitzer, R. Schuler, A. Shoshani, F. Siebenlist, A. Sim, W. G. Strand, M. Su, N. Wilhelmi, "The Earth System Grid: Enabling Access to Multimodel Climate Simulation Data", American Meteorological Society, 2009, 90(2):195-205,

M. Riedel, E. Laure, Th. Soddemann, L. Field, J. P. Navarro, J. Casey, M. Litmaath, J. Ph. Baud, B. Koblitz, C. Catlett, D. Skow, C. Zheng, P. M. Papadopoulos, M. Katz, N. Sharma, O. Smirnova, B. Kónya, P. Arzberger, F. Würthwein, A. S. Rana, T. Martin, M. Wan, V. Welch, T. Rimovsky, S. Newhouse, A. Vanni, Y. Tanaka, Y. Tanimura, T. Ikegami, D. Abramson, C. Enticott, G. Jenkins, R. Pordes, N. Sharma, S. Timm, N. Sharma, G. Moont, M. Aggarwal, D. Colling, O. van der Aa, A. Sim, V. Natarajan, A. Shoshani, J. Gu, S. Chen, G. Galang, R. Zappi, L. Magnoni, V. Ciaschini, M. Pace, V. Venturi, M. Marzolla, P. Andreetto, B. Cowles, S. Wang, Y. Saeki, H. Sato, S. Matsuoka, P. Uthayopas, S. Sriprayoonsakul, O. Koeroo, M. Viljoen, L. Pearlman, S. Pickles, David Wallom, G. Moloney, J. Lauret, J. Marsteller, P. Sheldon, S. Pathak, S. De Witt, J. Mencák, J. Jensen, M. Hodges, D. Ross, S. Phatanapherom, G. Netzer, A. R. Gregersen, M. Jones, S. Chen, P. Kacsuk, A. Streit, D. Mallmann, F. Wolf, T. Lippert, Th. Delaitre, E. Huedo, N. Geddes, "Interoperation of world-wide production e-Science infrastructures", Concurrency and Computation: Practice and Experience, 2009, 21(8):961-990,

Arie Shoshani, Flavia Donno, Junmin Gu, Jason Hick, Maarten Litmaath, Alex Sim, "Dynamic Storage Management", Scientific Data Management: Challenges, Technology, and Deployment, edited by Arie Shoshani, Doron Rotem, (Chapman & Hall/CRC Computational Science: 2009)

John Shalf and Jason Hick (Arie Shoshani and Doron Rotem), "Storage Technology Fundamentals", Scientific Data Management: Challenges, Technology, and Deployment, Volume . Chapman & Hall/CRC, 2009,

K Wu, S Ahern, EW Bethel, J Chen, H Childs, C Geddes, J Gu, H Hagen, B Hamann, J Lauret, others, "FastBit: Interactively Searching Massive Data", Proc. of SciDAC 2009, 2009, LBNL 2164E,

2008

P. Jakl, J. Lauret, A. Hanushevsky, A. Shoshani, A. Sim, J. Gu, "Grid data access on widely distributed worker nodes using scalla and SRM", Journal of Physics: Conf. Ser., 2008, 119, doi: 10.1088/1742-6596/119/7/072019

D. N. Williams, R. Ananthakrishnan, D. E. Bernholdt, S. Bharathi, D. Brown, M. Chen, A. L. Chervenak, L. Cinquini, R. Drach, I. T. Foster, P. Fox, S. Hankin, V. E. Henson, P. Jones, D. E. Middleton, J. Schwidder, R. Schweitzer, R. Schuler, A Shoshani, F. Siebenlist, A. Sim, W. G. Strand, N. Wilhelmi, M. Su, "Data Management and Analysis for the Earth System Grid", SciDAC Conference, 2008,

Alex Sim, Arie Shoshani (Editors), Paolo Badino, Olof Barring, Jean‐Philippe Baud, Ezio Corso, Shaun De Witt, Flavia Donno, Junmin Gu, Michael Haddox‐Schatz, Bryan Hess, Jens Jensen, Andy Kowalski, Maarten Litmaath, Luca Magnoni, Timur Perelmutov, Don Petravick, Chip Watson, The Storage Resource Manager Interface Specification Version 2.2, Open Grid Forum, Document in Full Recommendation, GFD.129, 2008,

C S Chang, S Klasky, J Cummings, R. Samtaney, A Shoshani, L Sugiyama, D Keyes, S Ku, G Park, S Parker, N Podhorszki, H. Strauss, H Abbasi, M Adams, R Barreto, G Bateman, K Bennett, Y Chen, E D’Azevedo, C Docan, S Ethier, E Feibush, L Greengard, T Hahm, F Hinton, C Jin, A. Khan, A Kritz, P Krsti, T Lao, W Lee, Z Lin, J Lofstead, P Mouallem, M Nagappan, A Pankin, M Parashar, M Pindzola, C Reinhold, D Schultz, K Schwan, D. Silver, A Sim, D Stotler, M Vouk, M Wolf, H Weitzner, P Worley, Y Xiao, E Yoon, D Zorin, "Toward a first- principles integrated simulation of tokamak edge plasmas", Journal of Physics: Conf. Ser., 2008, 125, doi: 10.1088/1742-6596/125/1/012042

R Ananthakrishnan, D E Bernholdt, S Bharathi, D Brown, M Chen, A L Chervenak, L Cinquini, R Drach, I T Foster, P Fox, D Fraser, K Halliday, S Hankin, P Jones, C Kesselman, D E Middleton, J Schwidder, R Schweitzer, R Schuler, A Shoshani, F Siebenlist, A Sim, W G Strand, N Wilhelmi, M Su, D N Williams, "Building a global federation system for climate change research: the earth system grid center for enabling technologies (ESG-CET)", Journal of Physics: Conf. Ser., 2008, 78, doi: 10.1088/1742-6596/78/1/012050

Kurt Stockinger, John Cieslewicz, Kesheng Wu, Rotem, Arie Shoshani, "Using Bitmap Indexing Technology for Combined and Text Queries", Annals of Information Systems, (Springer: 2008) Pages: 1--23

Rishi Rakesh Sinha, Marianne Winslett, Kesheng, Kurt Stockinger, Arie Shoshani, "Adaptive Bitmap Indexes for Space-Constrained", ICDE 2008, 2008, 1418--1420,

Kesheng Wu, Kurt Stockinger, Arie Shoshani, "Breaking the curse of cardinality on bitmap indexes", International Conference on Scientific and Statistical Database Management, 2008, 348--365,

Meiyappan Nagappan, Mladen A. Vouk, Kesheng Wu Alex Sim, Arie Shoshani, "Efficient Operational Profiling of Systems Using Arrays on Execution Logs", ISSRE, 2008, 313--314, doi: 10.1109/ISSRE.2008.45

2007

L. Abadie, P. Badino, J. Baud, E. Corso, M. Crawford, S. De Witt, F. Donno, A. Forti, P. Fuhrmann,
G. Grosdidier, J. Gu , J. Jensen, S. Lemaitre, M. Litmaath, D. Litvinsev, G. Lo Presti, L. Magnoni, T. Mkrtchan, A. Moibenko, V. Natarajan, G. Oleynik, T. Perelmutov, D. Petravick, A. Shoshani, A. Sim, M. Sponza, R. Zappi,
"Storage Resource Managers: Recent International Experience on Requirements and Multiple Co-Operating Implementations", the 24th IEEE Conference on Mass Storage Systems and Technologies, 2007,

F. Donno, L. Abadie, P. Badino, J. Baud, E. Corso, M. Crawford, S. De Witt, A. Forti, P. Fuhrmann, G. Grosdidier, J. Gu , J. Jensen, S. Lemaitre, M. Litmaath, D. Litvinsev, G. Lo Presti, L. Magnoni, T. Mkrtchan, A. Moibenko, V. Natarajan, G. Oleynik, T. Perelmutov, D. Petravick, A. Shoshani, A. Sim, M. Sponza, R. Zappi, "Storage Resource Manager version 2.2: design, implementation, and testing experience", Journal of Physics: Conf. Ser., 2007, 119, doi: 10.1088/1742-6596/119/6/062028

Elaheh Pourabbas, Arie Shoshani, "Efficient Estimation of Joint Queries from Multiple OLAP Databases", ACM Transactions on Database Systems (TODS), March 1, 2007, Volume 3,

Kesheng Wu, Kurt Stockinger, Arie Shoshani, Performance of Multi-Level and Multi-Component Bitmap Indexes, 2007, doi: 10.1145/1670243.1670245

Frederick Reiss, Kurt Stockinger, Kesheng Wu, Shoshani, Joseph M. Hellerstein, "Enabling Real-Time Querying of Live and Historical Data", SSDBM 2007, 2007,

2006

Elaheh Pourabbas, Arie Shoshani, "The Composite OLAP-Object Data Model: Removing an Unnecessary Barrier", International Conference on Scientific and Statistical Database Management (SSDBM) 2006, July 3, 2006, 291-300,

A. Shoshani, A. Sim, K. Stockinger, "RRS: Replica Registration Service for Data Grids", Lecture Notes in Computer Science, edited by Jean-Marc Pierson, (Springer-Verlag GmbH Publisher: 2006) Pages: 100-112

D. E. Middleton, D. E. Bernholdt, D. Brown, M. Chen, A. L. Chervenak, L. Cinquini, R. Drach, P. Fox, P. Jones, C. Kesselman, I. T. Foster, V. Nefedova, A. Shoshani, A. Sim, W. G. Strand, D. Williams, "Enabling worldwide access to climate simulation data: the earth system grid (ESG)", SciDAV Conference, 2006,

P. Jakl, J. Lauret, A. Hanushevky, A. Shoshani, A. Sim, "From rootd to Xrootd, from physical to logical files: experience on accessing and managing distributed data", Computing in High Energy Physics (CHEP), 2006,

E. Hjort, L. Hajdu, J. Lauret, D. Olson, A. Sim, A. Shoshani, "Data and Computational Grid Coupling in RHIC/STAR – An Analysis Scenario using SRM Technology", Computing in High Energy Physics (CHEP), 2006,

Kesheng Wu, Ekow J Otoo, Arie Shoshani, "Optimizing bitmap indices with efficient compression", ACM Transactions on Database Systems (TODS), 2006, 31:1--38,

K. Wu, K. Stockinger, A. Shoshani, Wes, "FastBit--Helps Finding the Proverbial Needle in a", 2006, LBNL LBNL-PUB/963,

F. Reiss, K. Stockinger, K. Wu, A. Shoshani J. M. Hellerstein, "Efficient analysis of live and historical streaming and its application to cybersecurity", 2006,

2005

D. Bernholdt, S. Bharathi, D. Brown, K. Chanchio, M. Chen, A. Chervenak, L. Cinquini, B. Zrach, I. Foster, P. Fox, J. Garcia, C. Kesselman, R. Markel, D. Middleton, V. Nefedova, L. Pouchard, A. Shoshani, A. Sim, G. Strand, D. Williams, "The Earth System Grid: Supporting the Next Generation of Climate Modeling Research", IEEE, 2005, 93(3):485-495,

A. Shoshani, A. Sim, K. Stockinger, "RRS: Replica Registration Service for Data Grids", International Workshop on Data Management in Grids, 2005,

Arie Shoshani, Alex Sim, Kurt Stockinger, "Replica Registration Service Functional Interface Specification 1.0", 2005, LBNL 57520,

Kesheng Wu, Junmin Gu, Jerome Lauret, Arthur Poskanzer, Arie Shoshani, Alexander Sim, Zhang, "Grid Collector: Facilitating Efficient Selective from Data Grids", International Supercomputer Conference 2005, 2005,

2004

Eric Hjort, Doug Olson, Jerome Lauret, Arie Shoshani, Alex Sim, "Production mode Data- Replication framework in STAR using the HRM Grid middleware", Computing in High Energy Physics, 2004,

Alex Sim, Junmin Gu, Arie Shoshani, Vijaya Natarajan, "DataMover: Robust Terabytes-Scale Multi-file Replication over Wide-Area Networks", the 16th International Conference on Scientific and Statistical Database Management (SSDBM 2004), 2004,

K. Wu, A. Shoshani, E. J. Otoo, Word aligned bitmap compression method, data and apparatus, US Patent 6,831,575, 2004,

Kesheng Wu, Ekow J Otoo, Arie Shoshani, "An efficient compression scheme for bitmap indices", 2004,

Kesheng Wu, Wei-Ming Zhang, Victor, Jerome Lauret, Arie Shoshani, "The Grid Collector: Using an Event Catalog to Speed up Analysis in Distributed Environment", Proceedings of Computing in High Energy and Nuclear (CHEP) 2004, 2004,

Kurt Stockinger, Kesheng Wu, Arie Shoshani, Evaluation Strategies for Bitmap Indices with, International Conference on Database and Expert Applications (DEXA 2004), Zaragoza, Spain, 2004,

2003

Elaheh Pourabbas, Arie Shoshani, "Answering Joint Queries from Multiple Aggregate OLAP Databases", Data Warehousing and Knowledge Discovery, 5th International Conference, DaWaK 2003, September 3, 2003, 24-34,

Arie Shoshani, Alexander Sim, Junmin Gu, "Storage Resource Managers: Essential Components for the Grid", Grid Resource Management: State of the Art and Future Trends, edited by Jarek Nabrzyski, Jennifer M. Schopf, Jan Weglarz, (Kluwer Academic Publishers: 2003)

Ann L. Chervenak, Ewa Deelman, Carl Kesselman, William E. Allcock, Ian T. Foster, Veronika Nefedova, Jason Lee, Alex Sim, Arie Shoshani, Bob Drach, Dean Williams, Don Middleton, "High-performance remote access to climate simulation data: a challenge problem for data grid technologies", Parallel Computing, 2003, 29(10):1335-1356,

A. Sim, J. Gu, A. Shoshani, E. Hjort, D. Olson, "Experience with Deploying Storage Resource Managers to Achieve Robust File Replication", Computing in High Energy Physics, 2003,

D. Yu, J. Lauret, A. Shoshani, D. Oldon, E. Hjort, A. Sim, "The Design of High Performance Data Replication in the Grid Environment for the STAR Collaboration", Computing in High Energy Physics, 2003,

L. Pouchard, L. Cinquini, B. Drach, D. Middleton, D. Bernholdt, K. Chanchio, I. Foster, V. Nefedova, D. Brown, P. Fox, J. Garcia, G. Strand, D. Williams, A. Chervenak, C. Kesselman, A. Shoshani, A. Sim, "An Ontology for Scientific Information in a Grid Environment: the Earth System Grid", the Symposium on Cluster Computing and the Grid (CCGrid), 2003,

Arie Shoshani, Alex Sim, Junmin Gu, Storage Resource Managers: Essential Components for Grid Applications, Globus World, 2003,

Kesheng Wu, Wei-Ming Zhang, Alexander Sim, Gu, Arie Shoshani, "Grid Collector: An Event Catalog With Automated File", Proceedings of IEEE Nuclear Science Symposium 2003, 2003, doi: 10.1109/NSSMIC.2003.1351830

Kesheng Wu, Wei-Ming Zlang, Alexander Sim, Junmin Gu, Arie Shoshani, "Grid collector: An event catalog with automated file management", 2003 IEEE Nuclear Science Symposium. Conference Record (IEEE Cat. No. 03CH37515), 2003, LBNL 55563,

2002

Elaheh Pourabbas, Arie Shoshani, "Joint Queries Estimation from Multiple OLAP Databases", International Conference on Scientific and Statistical Database Management, 2002 (SSDBM’02), July 24, 2002,

A. Shoshani, A. Sim, J. Gu, "Storage Resource Managers: Middleware components for Grid Storage", the 19th IEEE Symposium on Mass Storage Systems, 2002,

Kesheng Wu, Ekow Otoo, Arie Shoshani, Compressing Bitmap Indexes for Faster Search, Proceedings of SSDBM 02, Pages: 99--108 2002,

Kurt Stockinger, Kesheng Wu, Arie Shoshani, Strategies for processing ad hoc queries on large data, Proceedings of DOLAP 02, Pages: 72--79 2002,

2001

B. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Lee, A. Sim, A. Shoshani, B. Drach, D. Williams, "High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies", Super Computing 2001, 2001,

A. Sim, H. Nordberg, L.M. Bernardo, A. Shoshani, D. Rotem, "Experience with using CORBA to implement a file caching coordination system", Concurrency and Computation: Practice and Experience, 2001, 13:1-15,

D. Olson, E. Hjort, J. Lauret, M. Messer, A. Shoshani, A. Sim, "Non-shared Disk Cluster - A Fault Tolerant, Commodity Approach to Hi-Bandwidth Data Analysis", Computing in High Energy Physics, 2001,

L Bernardo, H Nordberg, D Olson, A Shoshani, A Sim, A Vaniachine, D Zimmerman, B Gibbard, R Porter, T Wenaus, others, "New capabilities in the HENP grand challenge storage access system and its application at RHIC", Computer physics communications, 2001, 140:179--188,

Kesheng Wu, Ekow J Otoo, Arie Shoshani, A performance comparison of bitmap indexes, Proceedings of the tenth international conference on Information and knowledge management, Pages: 559--561 2001,

2000

A. Shoshani, A. Sim, L.M. Bernerdo, H. Nordberg, "Coordinating Simultaneous Caching of File Bundles from Tertiary Storage", International Conference on Scientific and Statistical Database Management (SSDBM), 2000,

L. M. Bernardo, B. Gibbard, D. Malon, H. Nordberg, D. Olson, R. Porter, A. Shoshani, A. Sim, A. Vaniachine, T. Wenaus, K. Wu, D. Zimmerman, "New Capabilities in the HENP Grand Challenge Storage Access System and its Application at RHIC", Computing in High Energy Physics, 2000,

L. M. Bernardo, A. Shoshani, A. Sim, H. Nordberg, "Access Coordination Of Tertiary Storage For High Energy Physics Applications", the 17th IEEE Symposium on Mass Storage Systems, 2000,

A. Sim, A. Shoshani, HRM: Hierarchical Resource Manager, Globus World, 2000,

A. Sim, A. Shoshani, L. M. Bernardo, H. Nordberg, A Storage Access Coordination System for Perabyte Scale Scientific Data, IONA World, 2000,

1999

A. Sim, H. Nordberg, L. M. Bernardo, A. Shoshani, D. Rotem, "Storage Access Coordination Using CORBA", Distributed Objects and Application, 1999, 168-175,

A. Shoshani, L.M. Bernardo, H. Nordberg, D. Rotem and A. Sim, "Multidimensional Indexing and Query Coordination for Tertiary Storage Management", International Conference on Scientific and Statistical Database Management, 1999, 214-225,

1998

L.M. Bernardo, D. Rotem, A. Shoshani, H. Nordberg, A. Sim, "Using Access Patterns to Partition Large Datasets on Tertiary Storage in Order to Minimize Retrieval Costs", 1998, LBNL 41504,

A. Shoshani, L.M. Bernardo, H. Nordberg, D. Rotem, A. Sim, "Storage Management for High Energy Physics Applications", Computing in High Energy Physics, 1998,

Alex Sim

2024

D.K. Sung, Y. Son, A. Sim, K. Wu, S. Byna, H. Tang, H. Eom, C. Kim, S. Kim, "A2FL: Autonomous and Adaptive File Layout in HPC through Real-time Access Pattern Analysis", 38th IEEE International Parallel & Distributed Processing Symposium (IPDPS2024), 2024,

L. Zhou, Q. Lin, K. Chowdhury, S. Masood, A. Eichenberger, H. Min, A. Sim, J. Wang, Y. Wang, K. Wu, B. Yuan, J. Zou, "Serving Deep Learning Model in Relational Databases", 27th International Conference on Extending Database Technology (EDBT2024), 2024,

R. Frehner, K. Wu, A. Sim, J. Kim, K. Stockinger, "Detecting Anomalies in Time Series Using Kernel Density Approaches", IEEE Access, 2024, doi: 10.1109/ACCESS.2024.3371891

2023

A, Sharma, X. Li, H. Guan, G. Sun, L. Zhang, L. Wang, K. Wu, L. Cao, E. Zhu, A. Sim, T. Wu, J. Zou, "Automatic Data Transformation Using Large Language Model – An Experimental Study on Building Energy Data", IEEE International Conference on Big Data (BigData), 2023,

C. M. Oguchi, D. Ghosal, A. Sim, K. Wu, "Counterfactual Analysis: A Case Study on Impact of External Events on Building Energy Consumption", International Workshop on Big Data Analytics for Sustainability (BDA4S), 2023,

A. Sim, E. Kissel, D. Hazen, C. Guok, "Experiences in deploying in-network data caches", 26th International Conference on Computing in High Energy & Nuclear Physics (CHEP2023), 2023,

J. Bellavita, C. Sim, K. Wu, A. Sim, S. Yoo, H. Ito, V. Garonne, E. Lancon, "Understanding Data Access Patterns for dCache System", 26th International Conference on Computing in High Energy & Nuclear Physics (CHEP2023), 2023,

C. Sim, K. Wu, A. Sim, I. Monga, C. Guok, D. Hazen, F. Würthwein, D. Davila, H. Newman, J. Balcas, "Predicting Resource Utilization Trends with Southern California Petabyte Scale Cache", 26th International Conference on Computing in High Energy & Nuclear Physics (CHEP2023), 2023,

J. W. Chung, A. Sim, B. Quiter, Y. Wu, W. Zhao, K. Wu, "Preparing Spectral Data for Machine Learning: A Study of Geological Classification from Aerial Surveys", Machine Learning and the Physical Sciences Workshop (ML4PS), 2023,

R. Monga, A. Sim (advisor), K. Wu (advisor), "Comparative Study of the Cache Utilization Trends for Regional Scientific Data Caches", ACM/IEEE The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC’23), ACM Student Research Competition (SRC), First place winner, 2023,

H-C. Yang, L. Jin, A. Lazar, A. Todd-Blick, A. Sim, K. Wu, Q. Chen, C. A. Spurlock, "Gender Gaps in Mode Usage, Vehicle Ownership, and Spatial Mobility When Entering Parenthood: A Life Course Perspective", Systems, 2023, 11(6):314, doi: 10.3390/systems11060314

R. Shao, A. Sim, K. Wu, J. Kim, "Leveraging History to Predict Abnormal Transfers in Distributed Workflows", Sensors, 2023, 23(12):5485, doi: 10.3390/s23125485

Z. Deng, A. Sim, K. Wu, C. Guok, I. Monga, F. Andrijauskas, F. Wuerthwein, D. Weitzel, "Analyzing Transatlantic Network Traffic Patterns with Scientific Data Caches", 6th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA 2023), 2023, doi: 10.1145/3589012.3594897

C. Guok, E. Kissel, A. Sim, ESnet's In-Network Caching Pilot, The Network Conference 2023 (TNC'23), 2023,

C. Sim, K. Wu, A. Sim, I. Monga, C. Guok, F. Wurthwein, D. Davila, H. Newman, J. Balcas, Predicting Resource Usage Trends with Southern California Petabyte Scale Cache, 26th International Conference on Computing in High Energy & Nuclear Physics (CHEP 2023), 2023,

J. Bellavita, C. Sim, K. Wu, A. Sim, S. Yoo, H. Ito, V. Garonne, E. Lancon, Understanding Data Access Patterns for dCache System, 26th International Conference on Computing in High Energy & Nuclear Physics (CHEP 2023), 2023,

E. Kissel, A. Sim, C. Guok, Experiences in deploying in-network data caches, 26th International Conference on Computing in High Energy & Nuclear Physics (CHEP 2023), 2023,

S. Kim, A. Sim, K. Wu, S. Byna, Y. Son, H. Eom, "Design and Implementation of I/O Performance Prediction Scheme on HPC Systems through Large-scale Log Analysis", Journal of Big Data, 2023, 10(65), doi: 10.1186/s40537-023-00741-4

C. Sim, K. Wu, A. Sim, I. Monga, C. Guok, F. Wurthwein, D. Davila, H. Newman, J. Balcas, "Effectiveness and predictability of in-network storage cache for Scientific Workflows", International Conference on Computing, Networking and Communication (ICNC 2023), 2023, doi: 10.1109/ICNC57223.2023.10074058

J. Wang, K. Wu, A. Sim, S. Hwangbo, "Locating Partial Discharges in Power Transformers with Convolutional Iterative Filtering", Sensors, 2023, 23, doi: 10.3390/s23041789

H-C. Yang, L. Jin, A. Lazar, A. Todd-Blick, A. Sim, K. Wu, Q. Chen, C. A. Spurlock, Gender Gaps in Mode Usage, Vehicle Ownership, and Spatial Mobility When Entering Parenthood: A Life Course Perspective, Transportation Research Board 102nd Annual Meeting,, 2023,

J. Bang, A. Sim, G. Lockwood, H. Eom, H. Sung, "Design and Implementation of Burst Buffer Over-Subscription Scheme for HPC Storage Systems", IEEE Access, 2023, doi: 10.1109/ACCESS.2022.3233829

2022

Julian Bellavita, Alex Sim (advisor), John Wu (advisor), "Predicting Scientific Dataset Popularity Using dCache Logs", ACM/IEEE The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC’22), ACM Student Research Competition (SRC), Second place winner, 2022,

Poster (PDF)

The dCache installation is a storage management system that acts as a disk cache for high-energy physics (HEP) data. Storagespace on dCache is limited relative to persistent storage devices, therefore, a heuristic is needed to determine what data should be kept in the cache. A good cache policy would keep frequently accessed data in the cache, but this requires knowledge of future dataset popularity. We present methods for forecasting the number of times a dataset stored on dCache will be accessed in the future. We present a deep neural network that can predict future dataset accesses accurately, reporting a final normalized loss of 4.6e-8. We present a set of algorithms that can forecast future dataset accesses given an access sequence. Included are two novel algorithms, Backup Predictor and Last N Successors, that outperform other file prediction algorithms. Findings suggest that it is possible to anticipate dataset popularity in advance.

C. Sim, C. Guok (advisor), A. Sim (advisor), K. Wu (advisor), "Data Throughput Performance Trends of Regional Scientific Data Cache", ACM/IEEE The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC’22), ACM Student Research Competition (SRC), 2022,

Sunggon Kim, Alex Sim, Kesheng Wu, Suren Byna, Yongseok Son, "Design and implementation of dynamic I/O control scheme for large scale distributed file systems", Cluster Computing, 2022, 25(6):1--16, doi: 10.1007/s10586-022-03640-0

L. Jin, A. Lazar, C. Brown, V. Garikapati, B. Sun, S. Ravulaparthy, Q. Chen, A. Sim, K. Wu, T. Wenzel, T. Ho, C. A. Spurlock, "What Makes You Hold onto That Old Car? Joint Insights from Machine Learning and Multinomial Logit on Vehicle-level Transaction Decisions", Frontiers in Future Transportation, Connected Mobility and Automation, 2022, 3:894654, doi: 10.3389/ffutr.2022.894654

R. Shao, J. Kim A. Sim, K. Wu, "Predicting Slow Connections in Scientific Computing", 5th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA) 2022, in conjunction with The 31st ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2022, doi: 10.1145/3526064.3534112

J. Bellavita, A. Sim, K. Wu, I. Monga, C. Guok, F. Würthwein, D. Davila, "Studying Scientific Data Lifecycle in On-demand Distributed Storage Caches", 5th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA) 2022, in conjunction with The 31st ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2022, doi: 10.1145/3526064.3534111

R. Han, A. Sim, K. Wu, I. Monga, C. Guok, F. Würthwein, D. Davila, J. Balcas, H. Newman, "Access Trends of In-network Cache for Scientific Data", 5th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA), in conjunction with The 31st ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2022, doi: 10.1145/3526064.3534110

J. Kim, M. Cafaro, J. Chou, A. Sim, "SNTA’22: The 5th Workshop on Systems and Network Telemetry and Analytics", In the proceedings of The 31st ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC'22), 2022, doi: 10.1145/3502181.3535108

D. Bard, C. Snavely, L. Gerhardt, J. Lee, B. Totzke, K. Antypas, W. Arndt, J. Blaschke, S. Byna, R. Cheema, S. Cholia, M. Day, B. Enders, A. Gaur, A. Greiner, T. Groves, M. Kiran, Q. Koziol, T. Lehman, K. Rowland, C. Samuel, A. Selvarajan, A. Sim, D. Skinner, L. Stephey, R. Thomas, G. Torok, "LBNL Superfacility Project Report", Lawrence Berkeley National Laboratory, 2022, doi: 10.48550/arXiv.2206.11992

Yujing Ma, Florin Rusu, Kesheng Wu, Alexander Sim, 2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Pages: 1088--1097 2022, doi: 10.1109/IPDPSW55747.2022.00177

K. Wang, S. Lee, J. Balewski, A. Sim, P. Nugent, A. Agrawal, A. Choudhary, K. Wu, W-K. Liao, "Using Multi-resolution Data to Accelerate Neural Network Training in Scientific Applications", 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGrid 2022), 2022, doi: 10.1109/CCGrid54584.2022.00050

B. Weinger, J. Kim, A. Sim, M. Nakashima, N. Moustafa, K. Wu, "Enhancing IoT Anomaly Detection Performance for Federated Learning", Digital Communications and Networks, Special Issue on Edge Computation and Intelligence, 2022, doi: 10.1016/j.dcan.2022.02.007

John Wu, Bin Dong, Alex Sim, Automating Data Management Through Unified Runtime Systems, DOE ASCR Workshop on the Management and Storage of Scientific Data, 2022, doi: 10.2172/1843500

John Wu, Ben Brown, Paolo Calafiura, Quincey Koziol, Dongeun Lee, Alex Sim, Devesh Tiwari, Support for In-Flight Data Analyses in Scientific Workflows, DOE ASCR Workshop on the Management and Storage of Scientific Data, 2022, doi: 10.2172/1843500

A. Pereira, A. Sim, K. Wu, S. Yoo, H. Ito, "Data access pattern analysis for dCache storage system", International Conference on High Performance Computing in Asia-Pacific Region (HPC Asia 2022), 2022,

Ling Jin, Alina Lazar, Caitlin Brown, Bingrong Sun, Venu Garikapati, Srinath Ravulaparthy, Qianmiao Chen, Alexander Sim, Kesheng Wu, Tin Ho, Thomas Wenzel, C. Anna Spurlock, What Makes You Hold on to That Old Car? Joint Insights from Machine Learning and Multinomial Logit on Vehicle-level Transaction Decisions, Transportation Research Board 101st Annual Meeting, 2022,

2021

J. Bang, C. Kim, K. Wu, A. Sim, S. Byna, H. Sung, H. Eom, "An In-Depth I/O Pattern Analysis in HPC Systems", IEEE International Conference on High Performance Computing, Data & Analytics (HiPC2021), 2021, doi: 10.1109/HiPC53243.2021.00056

S. Lee, Q. Kang, K. Wang, J. Balewski, A. Sim, A. Agrawal, A. Choudhary, P. Nugent, K. Wu, W-K. Liao, "Asynchronous I/O Strategy for Large-Scale Deep Learning Applications", IEEE International Conference on High Performance Computing, Data & Analytics (HiPC2021), 2021, doi: 10.1109/HiPC53243.2021.00046

A. Lazar, L. Jin, C. Brown, C. A. Spurlock, A. Sim, K. Wu, "Performance of the Gold Standard and Machine Learning in Predicting Vehicle Transactions", the 3rd International Workshop on Big Data Tools, Methods, and Use Cases for Innovative Scientific Discovery (BTSD 2021), 2021, doi: 10.1109/BigData52589.2021.9671286

J. Cheung, A. Sim, J. Kim, K. Wu, "Performance Prediction of Large Data Transfers", ACM/IEEE The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC21), ACM Student Research Competition (SRC), 2021,

A. Syal, A. Lazar, J. Kim, A. Sim, K. Wu, "Network traffic performance analysis from passive measurements using gradient boosting machine learning", International Journal of Big Data Intelligence, 2021, 8:13-30, doi: 10.1504/IJBDI.2021.118741

Y. Ma, F. Rusu, K. Wu, A. Sim, Adaptive Elastic Training for Sparse Deep Learning on Heterogeneous Multi-GPU Servers, arXiv preprint arXiv:2110.07029, 2021,

E. Copps, A. Sim (Advisor), K. Wu (Advisor), "Analyzing scientific data sharing patterns with in-network data caching", ACM Richard Tapia Celebration of Diversity in Computing (TAPIA 2021), ACM Student Research Competition (SRC), 2021,

M. Nakashima, A. Sim, Y. Kim, J. Kim, J. Kim, "Automated Feature Selection for Anomaly Detection in Network Traffic Data", ACM Transactions on Management Information Systems (TMIS), 2021, 12:1-28, doi: 10.1145/3446636

E. Copps, H. Zhang, A. Sim, K. Wu, I. Monga, C. Guok, F. Würthwein, D. Davila, E. Fajardo, "Analyzing scientific data sharing patterns with in-network data caching", 4th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA 2021), 2021, doi: 10.1145/3452411.3464441

Y. Wang, K. Wu, A. Sim, S. Yoo, S. Misawa, "Access Patterns of Disk Cache for Large Scientific Archive", 4th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA 2021), 2021, doi: 10.1145/3452411.3464444

A. Lazar, A. Sim, K. Wu, "GPU-based Classification for Wireless Intrusion Detection", 4th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA 2021), 2021, doi: 10.1145/3452411.3464445

Y. Ma, F. Ruso, A. Sim, K. Wu, "Adaptive Stochastic Gradient Descent for Deep Learning on Heterogeneous CPU+GPU Architectures", Heterogeneity in Computing Workshop (HCW 2021), in conjunction with the 35th IEEE International Parallel & Distributed Processing Symposium (IPDPS), 2021, doi: 10.1109/IPDPSW52791.2021.00012

J. Kim, A. Sim, J. Kim, K, Wu, J. Hahm, Improving Botnet Detection with Recurrent Neural Network and Transfer Learning, arXiv preprint arXiv:2104.12602, 2021,

2020

Ling Jin, Alina Lazar, James Sears, Annika Todd, Alex Sim, Kesheng Wu, Hung-Chai Yang, C. Anna Spurlock, "Clustering Life Course to Understand the Heterogeneous Effects of Life Events, Gender and Generation on Habitual Travel Modes", IEEE Access, 2020, 1-17, doi: 10.1109/ACCESS.2020.3032328

B. Weinger, J. Kim, A. Sim, M. Nakashima, N. Moustafa, K. Wu, "Enhancing IoT Anomaly Detection Performance for Federated Learning", The 16th IEEE International Conference on Mobility, Sensing and Networking (IEEE MSN 2020), 2020, doi: 10.1109/MSN50589.2020.00045

B. Cho, T. Dayrit, Y. Gao, Z. Wang, T. Hong, A. Sim, K. Wu, "Effective Missing Value Imputation Methods for Building Monitoring Data", The 2nd International Workshop on Big Data Tools, Methods, and Use Cases for Innovative Scientific Discovery (BTSD 2020) in conjunction with IEEE International Conference on Big Data (IEEE BigData 2020), 2020, doi: 10.1109/BigData50022.2020.9378230

J. Kim, A. Sim, J. Kim, K. Wu, "Botnets Detection Using Recurrent Variational Autoencoder", IEEE Global Communications Conference (Globecom 2020), 2020, doi: 10.1109/GLOBECOM42002.2020.9348169

I. Monga, C. Guok, J. MacAuley, A. Sim, H. Newman, J. Balcas, P. DeMar, L. Winkler, T. Lehman, X. Yang, "SDN for End-to-end Networked Science at the Exascale", Future Generation Computer Systems, 2020, doi: 10.1016/j.future.2020.04.018

D. Bard, C. Snavely, L. Gerhardt, J. Lee, B. Totzke, K. Antypas, S. Byna, R. Cheema, S. Cholia, M. Day, B. Enders, A. Gaur, A. Greiner, T. Groves, M. Kiran, Q. Koziol, K. Rowland, C. Samuel, A. Selvarajan, A. Sim, D. Skinner, R. Thomas, G. Torok, The Superfacility project: automated pipelines for experiments and HPC, International Conference for High Performance Computing, Networking, Storage, and Analysis (SC20), State of the Practice (SOP), 2020,

B. Enders, D. Bard, C. Snavely, L. Gerhardt, J. Lee, B. Totzke, K. Antypas, S. Byna, R. Cheema, S. Cholia, M. Day, A. Gaur, A. Greiner, T. Groves, M. Kiran, Q. Koziol, K. Rowland, C. Samuel, A. Selvarajan, A. Sim, D. Skinner, R. Thomas, G. Torok, "Cross-facility science with the Superfacility Project at LBNL", 2nd Workshop on Large-scale Experiment-in-the-Loop Computing (XLOOP 2020), in conjunction with the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 20), 2020, doi: 10.1109/XLOOP51963.2020.00006

Brett Weinger, Alex Sim (Advisor), John Wu (Advisor), Jinoh Kim (Advisor), "Enhancing IoT Anomaly Detection Performance for Federated Learning", International Conference for High Performance Computing, Networking, Storage and Analysis (SC’20), ACM Student Research Competition (SRC), 2020,

A. Sim, Statistical Pattern Detection with Locally Exchangeable Measures, International Conference on Advanced Communications and Computation (INFOCOMP 2020), 2020,

C. A. Spurlock, A. Gopal, J. Auld, P. Leiby, C. Sheppard, T. Wenzel, S. Belal, A. Duvall, A. Enam, S. Fujita, A. Henao, L. Jin, E. Kontou, A. Lazar, Z. Needell, C. Rames, T. Rashidi, J. Sears, A. Sim, M. Stinson, M. Taylor, A. Todd-Blick, O. Verbas, V. Walker, J. Ward, G. Wong-Parodi, K. Wu, H.-C. Yang, "SMART Mobility, Mobility Decision Science Capstone Report", Vehicle Technologies Office (VTO), Office of Energy Efficiency and Renewable Energy (EERE), US Department of Energy, 2020,

Sunggon Kim, Alex Sim, Kesheng Wu, Suren Byna, Yongseok Son, Hyeonsang Eom, "Towards hpc i/o performance prediction through large-scale log analysis", Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing (HPDC 2020), 2020, 77--88, doi: 10.1145/3369583.3392678

Gaurav R Ghosal, Dipak Ghosal, Alex Sim, Aditya V Thakur, Kesheng Wu, "A Deep Deterministic Policy Gradient Based Network Scheduler For Deadline-Driven Data Transfers", Proceedings of International Federation for Information Processing (IFIP) Networking Conference (NETWORKING 2020), 2020, 253--261,

Jiwoo Bang, Chungyong Kim, Kesheng Wu, Alex Sim, Suren Byna, Sunggon Kim, Hyeonsang Eom, "HPC Workload Characterization Using Feature Selection and Clustering", ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA 2020), in conjunction with The 29th ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC 2020), 2020, 33--40, doi: 10.1145/3391812.3396270

Jeeyung Kim, Alex Sim, Jinoh Kim, Kesheng Wu, Jaegyoon Hahm, "Transfer Learning Approach for Botnet Detection Based on Recurrent Variational Autoencoder", ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA 2020), in conjunction with The 29th ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC 2020), 2020, 41--47, doi: 10.1145/3391812.3396273

M. Nakashima, A. Sim, J. Kim, "Evaluation of Deep Learning Models for Network PerformancePrediction for Scientific Facilities", the 3rd ACM International Workshop on System and Network Telemetry and Analysis (SNTA) 2020, in conjunction with The 29th ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2020, doi: 10.1145/3391812.3396272

S. Bhandari, A. K. Kukreja, A. Lazar, A. Sim, K. Wu, "Feature Selection and Tree-based Classification for Wireless Intrusion Detection", the 3rd ACM International Workshop on System and Network Telemetry and Analysis (SNTA) 2020, in conjunction with The 29th ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2020, doi: 10.1145/3391812.3396274

Qiao Kang, Alex Sim, Peter Nugent, Sunwoo Lee, Wei-keng Liao, Ankit Agrawal, Alok Choudhary, Kesheng Wu, "Predicting Resource Requirement in Intermediate Palomar Transient Factory Workflow", 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID 2020), 2020, 619--628, doi: 10.1109/CCGrid49817.2020.00-31

H. Sung, J. Bang, C. Kim, H. Kim, A. Sim, G. K. Lockwood, H. Eom, "BBOS: Efficient HPC Storage Management via Burst Buffer Over-Subscription", the 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGrid 2020), 2020, doi: 10.1109/CCGrid49817.2020.00-79

L. Jin, A. Lazar, J. Sears, A. Todd, A. Sim, K. Wu, C. A. Spurlock, "Life Course as a Contextual System to Investigate the Effects of Life Events, Gender, and Generation on Travel Mode Use", Transportation Research Board (TRB) 99th Annual Meeting, 2020,

Jeeyung Kim, Alex Sim, Jinoh Kim, Kesheng Wu, Botnet Detection Using Recurrent Variational Autoencoder, arXiv preprint arXiv:2004.00234, 2020,

2019

A. Lazar, A. Ballow, L. Jin, C. A. Spurlock, A. Sim, K. Wu, "Machine Learning for Prediction of Mid to LongTerm Habitual Transportation Mode Use", International Workshop on Big Data Tools, Methods, and Use Cases for Innovative Scientific Discovery (BTSD), in conjunction with the IEEE International Conference on Big Data (Big Data), 2019, doi: 10.1109/BigData47090.2019.9006411

L. Jin, A. Lazar, J. Sears, A. Todd, A. Sim, K. Wu, C. A. Spurlock, Life course as a contextual system to investigate the effects of life events, gender and generation on travel mode usage, The Behavior, Energy & Climate Change Conference (BECC), 2019,

J. Balcas, H. Newman, M. Spiropulu, X. Yang, T. Lehman, I. Monga, C. Guok, J. MacAuley, A. Sim, P. Demar, "SDN for End-to-End Networking at Exascale", the 24th International Conference on Computing in High Energy and Nuclear Physics (CHEP2019), 2019,

Alexandra Ballow, Alina Lazar (Advisor), Alex Sim (Advisor), Kesheng Wu (Advisor), "Handling Missing Values in Joint Sequence Analysis", ACM Richard Tapia Celebration of Diversity in Computing (TAPIA 2019), ACM Student Research Competition (SRC), First place winner, Pages: 19 2019,

J. Choi, A. Sim, Data reduction methods, systems and devices, U.S. Patent No. 10,366,078, 2019,

U.S. Patent No. 10,366,078, “DATA REDUCTION METHODS, SYSTEMS, AND DEVICES”, LBNL IB2013-133.

S. Kim, A. Sim, K. Wu, S. Byna, T. Wang, Y. Son, H. Eom, "DCA-IO: A Dynamic I/O Control Scheme for Parallel and Distributed File System", 19th Annual IEEE/ACM International Symposium in Cluster, Cloud, and Grid Computing (CCGrid 2019), 2019, doi: 10.1109/CCGRID.2019.00049

J. Kim, A. Sim, B. Tierney, S. Suh, I. Kim, "Multivariate Network Traffic Analysis using Clustered Patterns", Journal of Computing, April 2019, 101(4):339-361, doi: 10.1007/s00607-018-0619-4

J. Kim, A. Sim, "A new approach to multivariate network traffic analysis", Journal of Computer Science and Technology, 2019, 34(2):388–402, doi: 10.1007/s11390-019-1915-y

Alexandra Ballow, Alina Lazar, Alex Sim, Kesheng Wu, "Joint Sequence Analysis Challenges: How to Handle Missing Values and Mixed Variable Types", SIAM Conference on Computational Science and Engineering (CSE19), 2019,

Tyler Leibengood, Alina Lazar, Alex Sim, Kesheng Wu, "Network Traffic Performance Prediction with Multivariate Clusters in Time Windows", SIAM Conference on Computational Science and Engineering (CSE19), 2019,

Olivia Del Guercio, Rafael Orozco, Alex Sim, Kesheng Wu, "Multidimensional Compression with Pattern Matching", 2019 Data Compression Conference (DCC), Pages: 567--567 2019,

Alina Lazar, Ling Jin, C Anna Spurlock, Kesheng Wu, Alex Sim, Annika Todd, "Evaluating the effects of missing values and mixed data types on social sequence clustering using t-SNE visualization", Journal of Data and Information Quality (JDIQ), 2019, 11:1--22,

Sambit Shukla, Dipak Ghosal, Kesheng Wu, Alex Sim, Matthew Farrens, "Co-optimizing Latency and Energy for IoT services using HMP servers in Fog Clusters", 2019 Fourth International Conference on Fog and Mobile Edge Computing (FMEC), 2019, 121--128,

Hanul Sung, Jiwoo Bang, Alexander Sim, Kesheng Wu, Hyeonsang Eom, "Understanding Parallel I/O Performance Trends Under Various HPC Configurations", Proceedings of the ACM Workshop on Systems and Network Telemetry and Analytics, 2019, 29--36,

Mengtian Jin, Youkow Homma, Alex Sim, Wilko Kroeger, Kesheng Wu, "Performance prediction for data transfers in LCLS workflow", Proceedings of the ACM Workshop on Systems and Network Telemetry and Analytics, 2019, 37--44,

Olivia Del Guercio, Rafael Orozco, Alex Sim, Kesheng Wu, "Similarity-based Compression with Multidimensional Pattern Matching", Proceedings of the ACM Workshop on Systems and Network Telemetry and Analytics, 2019, 19--24,

Astha Syal, Alina Lazar, Jinoh Kim, Alex Sim, Kesheng Wu, "Automatic detection of network traffic anomalies and changes", Proceedings of the ACM Workshop on Systems and Network Telemetry and Analytics, 2019, 3--10,

Dipak Ghosal, Sambit Shukla, Alex Sim, Aditya V Thakur, Kesheng Wu, "A Reinforcement Learning Based Network Scheduler For Deadline-Driven Data Transfers", 2019 IEEE Global Communications Conference (GLOBECOM), 2019, 1--6,

Qiao Kang, Ankit Agrawal, Alok Choudhary, Alex Sim, Kesheng Wu, Rajkumar Kettimuthu, Peter H Beckman, Zhengchun Liu, Wei-keng Liao, "Spatiotemporal Real-Time Anomaly Detection for Supercomputing Systems", 2019 IEEE International Conference on Big Data (Big Data), 2019, 4381--4389,

Burak Cetin, Alina Lazar, Jinoh Kim, Alex Sim, Kesheng Wu, "Federated Wireless Network Intrusion Detection", 2019 IEEE International Conference on Big Data (Big Data), Pages: 6004--6006 2019,

Kesheng Wu, Alex Sim, Jonathan Wang, Seongwook Hwangbo, Methods, systems, and devices for accurate signal timing of power component events, 2019,

US Patent app no. 20190138371, “Methods, systems, and devices for accurate signal timing of power component events”

2018

Kade Gibson, Dongeun Lee, Jaesik Choi, Alex Sim, "Dynamic Online Performance Optimization in Streaming Data Compression", IEEE International Conference on Big Data (Big Data 2018), 2018, doi: 10.1109/bigdata.2018.8621867

Karen Tu, Alex Sim (Advisor), John Wu (Advisor), "Identification of Network Data Transfer Bottlenecks in HPC Systems", International Conference for High Performance Computing, Networking, Storage and Analysis (SC’18), ACM Student Research Competition (SRC), 2018,

I. Monga, C. Guok, J. MacAuley, A. Sim, H. Newman, J. Balcas, P. DeMar, L. Winkler, T. Lehman, X. Yang, "SDN for End-to-end Networked Science at the Exascale (SENSE)", Innovate the Network for Data-Intensive Science Workshop (INDIS 2018), in conjunction with the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC'18), 2018, doi: 10.1109/INDIS.2018.00007

J. Kim, J. Choi, A. Sim, "Spatio-temporal Analysis of HPC I/O and Connection Data", International Workshop on Scalable Network Traffic Analytics (SNTA 2018), 2018, in conjunction with the 38th IEEE International Conference on Distributed Computing Systems (ICDCS 2018), 2018, doi: 10.1109/icdcs.2018.00176

Taehoon Kim, Jaesik Choi, Dongeun Lee, Alex Sim, C Anna Spurlock, Annika Todd, Kesheng Wu, "Predicting baseline for analysis of electricity pricing", International Journal of Big Data Intelligence, 2018, 5:3--20,

Hongyuan Zhan, Gabriel Gomes, Xiaoye S Li, Kamesh Madduri, Alex Sim, Kesheng Wu, "Consensus ensemble system for traffic flow prediction", IEEE Transactions on Intelligent Transportation Systems, 2018, 19:3903--3914,

Cecilia Dao, Xinyu Liu, Alex Sim, Craig Tull, Kesheng Wu, "Modeling data transfers: change point and anomaly detection", 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), 2018, 1589--1594,

Rajkumar Kettimuthu, Zhengchun Liu, Ian Foster, Peter H Beckman, Alex Sim, Kesheng Wu, Wei-keng Liao, Qiao Kang, Ankit Agrawal, Alok Choudhary, "Towards autonomic science infrastructure: architecture, limitations, and open issues", Proceedings of the 1st International Workshop on Autonomous Infrastructure for Science, 2018, 1--9,

Mengying Yang, Xinyu Liu, Wilko Kroeger, Alex Sim, Kesheng Wu, "Identifying anomalous file transfer events in LCLS workflow", Proceedings of the 1st International Workshop on Autonomous Infrastructure for Science, 2018, 1--4,

Sowmya Balasubramanian, Dipak Ghosal, Kamala Narayanan Balasubramanian Sharath, Eric Pouyoul, Alex Sim, Kesheng Wu, Brian Tierney, "Auto-tuned publisher in a pub/sub system: Design and performance evaluation", 2018 IEEE International Conference on Autonomic Computing (ICAC), 2018, 21--30,

Jonathan Wang, Kesheng Wu, Alex Sim, Seongwook Hwangbo, "Feature Engineering and Classification Models for Partial Discharge in Power Transformers", Mij, 2018, 1001:60,

Tal Shachaf, Alexander Sim, Kesheng Wu, Wilko Kroeger, "Detecting Anomalies in the LCLS Workflow", 2018 IEEE International Conference on Big Data (Big Data), 2018, 3256--3260,

Alina Lazar, Kesheng Wu, Alex Sim, "Predicting Network Traffic Using TCP Anomalies", 2018 IEEE International Conference on Big Data (Big Data), Pages: 5369--5371 2018,

2017

Jinoh Kim, Alex Sim, "A New Approach to Online, Multivariate Network Traffic Analysis", 2nd Workshop on Network Security Analytics and Automation (NSAA), in conjunction with the 26th International Conference on Computer Communications and Networks (ICCCN 2017), 2017, doi: 10.1109/ICCCN.2017.8038520

J. Kim, A. Sim, S.C. Suh, I. Kim, "An Approach to Online Network Monitoring Using Clustered Patterns", International Conference on Computing, Networking and Communications (ICNC 2017), 2017, doi: 10.1109/ICCNC.2017.7876207

J. Kim, W. Yoo, A. Sim, S.C. Suh, I. Kim, "A Lightweight Network Anomaly Detection Technique", International Workshop on Computing, Networking and Communications (CNC 2017), 2017, doi: 10.1109/ICCNC.2017.7876251

Ling Jin, Doris Lee, Alex Sim, Sam Borgeson, Kesheng Wu, C Anna Spurlock, Annika Todd, "Comparison of clustering techniques for residential energy behavior using smart meter data", 2017,

Dongeun Lee, Alex Sim, Jaesik Choi, Kesheng Wu, "Expanding statistical similarity based data reduction to capture diverse patterns", 2017 Data Compression Conference (DCC), Pages: 445--445 2017,

Jonathan Wang, Wucherl Yoo, Alex Sim, Peter Nugent, Kesheng Wu, "Parallel variable selection for effective performance prediction", 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), 2017, 208--217,

Dongeun Lee, Alex Sim, Jaesik Choi, Kesheng Wu, "Improving statistical similarity based data reduction for non-stationary data", Proceedings of the 29th International Conference on Scientific and Statistical Database Management, 2017, 1--6,

Updated experiment version: https://sdm.lbl.gov/oapapers/ssdbm17-lee-upd.pdf
Original version: http://dl.acm.org/citation.cfm?doid=3085504.3085583

Kesheng Wu, Dongeun Lee, Alex Sim, Jaesik Choi, "Statistical data reduction for streaming data", 2017 New York Scientific Data Summit (NYSDS), 2017, 1--6,

Jonathan Wang, Kesheng Wu, Alex Sim, Seongwook Hwangbo, "Convolutional Filtering for Accurate Signal Timing from Noisy Streaming Data", 2017 IEEE 15th Intl Conf on Dependable, Autonomic and Secure Computing, 15th Intl Conf on Pervasive Intelligence and Computing, 3rd Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech, 2017, 941--948,

Jonathan Wang, Kesheng Wu, Alex Sim, Seongwook Hwangbo, "Feature Engineering and Classification Models for Partial Discharge Events in Power Transformers", Proceedings of the Fourth IEEE/ACM International Conference on Big Data Computing, Applications and Technologies, Pages: 269--270 2017,

Alina Lazar, Ling Jin, C Anna Spurlock, Kesheng Wu, Alex Sim, "Data quality challenges with missing values and mixed types in joint sequence analysis", 2017 IEEE International Conference on Big Data (Big Data), 2017, 2620--2627,

Peter Harrington, Wucherl Yoo, Alexander Sim, Kesheng Wu, "Diagnosing parallel I/O bottlenecks in HPC applications", International Conference for High Performance Computing Networking Storage and Analysis (SCI7) ACM Student Research Competition (SRC), 2017,

Jonathan Wang, Kesheng Wu, Alex Sim, Seongwook Hwangbo, "Accurate signal timing from high frequency streaming data", 2017 IEEE International Conference on Big Data (Big Data), Pages: 4852--4854 2017,

2016

Sam Fries, Sasha Ames, Alex Sim, Dean Williams, "HPSS Connections to ESGF: BASEJumper", 2016 Earth System Grid Federation (ESGF) Conference, 2016,

M. Bryson, S. Byna (Advisor), A. Sim (Advisor), K. Wu (Advisor), "The Search for Missing Parallel IO Performance on the Cori Supercomputer", International Conference for High Performance Computing, Networking, Storage and Analysis (SC’16), ACM Student Research Competition (SRC), 2016,

M. Bae, W. Yoo (Advisor), A. Sim (Advisor), K. Wu (Advisor), "Discovering Energy Resource Usage Patterns on Scientific Clusters", International Conference for High Performance Computing, Networking, Storage and Analysis (SC’16), ACM Student Research Competition (SRC), Third place winner, 2016, 2016,

Jonathan Wang, Wucherl Yoo, Alex Sim, K John Wu, "Analysis of Variable Selection Methods on Scientific Cluster Measurement Data", 2016,

David Pugmire, James Kress, Jong Choi, Scott Klasky, Tahsin Kurc, Randy Michael Churchill, Matthew Wolf, Greg Eisenhower, Hank Childs, Kesheng Wu, others, "Visualization and analysis for near-real-time decision making in distributed workflows", 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2016, 1007--1013,

D. Pugmire, J. Kress, J. Choi, S. Klasky, Kurc, R. M. Churchill, M. Wolf, G., H. Childs, K. Wu, A. Sim, J. Gu, J. Low, "Visualization and Analysis for Near-Real-Time Decision in Distributed Workflows", 2016 IEEE International Parallel and Distributed Symposium Workshops (IPDPSW), 2016, 1007--1013, doi: 10.1109/IPDPSW.2016.175

Wucherl Yoo, Michelle Koo, Yi Cao, Alex Sim, Peter Nugent, Kesheng Wu, "Performance Analysis Tool for HPC and Big Data Applications on Scientific Clusters", Conquering Big Data with High Performance Computing, (Springer, Cham: 2016) Pages: 139--161

Dongeun Lee, Alex Sim, Jaesik Choi, Kesheng Wu, "Novel data reduction based on statistical similarity", Proceedings of the 28th International Conference on Scientific and Statistical Database Management, 2016, 1--12,

Wucherl Yoo, Alex Sim, Kesheng Wu, "Machine learning based job status prediction in scientific clusters", 2016 SAI Computing Conference (SAI), 2016, 44--53,

Lingfei Wu, Kesheng John Wu, Alex Sim, Michael Churchill, Jong Y Choi, Andreas Stathopoulos, Choong-Seock Chang, Scott Klasky, "Towards real-time detection and tracking of spatio-temporal features: Blob-filaments in fusion plasma", IEEE Transactions on Big Data, 2016, 2:262--275,

2015

S. Fries, A. Sim, "HPSS connections to ESGF", Earth System Grid Federation Conference, (ESGF 2015), 2015,

M. Koo, W. Yoo (advisor), A. Sim (advisor), "I/O Performance Analysis Framework on Measurement Data from Scientific Clusters", International Conference for High Performance Computing, Networking, Storage and Analysis (SC’15), ACM Student Research Competition (SRC), 2015, 2015,

J. Kim, A. Sim, "Peeking Network States with Clustered Patterns", 2015, LBNL 1003744,

K. Hu, J. Choi, A. Sim, J. Jiang, "Best Predictive Generalized Linear Mixed Model with Predictive Lasso for High-Speed Network Data Analysis", International Journal of Statistics and Probability, 2015,

S. Shannigrahi, A. J. Barczyk, C. Papadopoulos, A. Sim, I. Monga, H. Newman, K. Wu, E. Yeh, "Named Data Networking in Climate Research and HEP Applications", 21st International Conference on Computing in High Energy and Nuclear Physics (CHEP2015), 2015,

W. Yoo, A. Sim, "Network Bandwidth Utilization Forecast Model on High Bandwidth Networks", IEEE International Conference on Computing, Networking and Communications (ICNC’15), 2015, doi: 10.1109/ICCNC.2015.7069393

David H Bailey, Stephanie Ger, Marcos L\ opez de Prado, Alexander Sim, "Statistical overfitting and backtest performance", Risk-Based and Factor Investing, 2015,

http://ssrn.com/abstract=2507040

Wucherl Yoo, Michelle Koo, Yi Cao, Alex Sim, Peter Nugent, Kesheng Wu, "Patha: Performance analysis tool for hpc applications", 2015 IEEE 34th International Performance Computing and Communications Conference (IPCCC), 2015, 1--8,

Taehoon Kim, Dongeun Lee, Jaesik Choi, Anna Spurlock, Alex Sim, Annika Todd, Kesheng Wu, "Extracting baseline electricity usage using gradient tree boosting", 2015 IEEE International Conference on Smart City/SocialCom/SustainCom (SmartCity), 2015, 734--741,

L. Wu, K. Wu, A. Sim, M. Churchill, J. Y. Choi, A. Stathopoulos, C.S. Chang, S. Klasky, "Towards Real-Time Detection and Tracking of Blob-Filaments in Fusion Plasma Big Data", WM-CS-2015-01, Department of Computer Science, College of William and Mary, 2015, doi: 10.48550/arXiv.1505.03532

W. Yoo, M. Koo, Y. Cao, A. Sim, P. Nugent, K. Wu, PATHA: Performance Analysis Tool for HPC, 2015 IEEE 34th International Performance Computing and Conference (IPCCC), Pages: 1--8 2015, doi: 10.1109/PCCC.2015.7410313

Taehoon Kim, Dongeun Lee, Jaesik Choi, C. Anna Spurlock, Alex Sim, Annika Todd, Kesheng Wu, "Extracting Baseline Electricity Usage with Gradient Boosting", International Conference on Big Intelligence and Computing (DataCom 2015), 2015, doi: 10.1109/SmartCity.2015.156

2014

W. Yoo, A. Sim, "Efficient Changing Pattern Detection on High Bandwidth Network Measurements", 7th International Conference on Grid and Distributed Computing, 2014,

A. L. Chervenak, A. Sim, J. Gu, R. Schuler, N. Hirpathak, "Adaptation and Policy-Based Resource Allocation for Efficient Bulk Data Transfers in High Performance Computing Environments", 4th International Workshop on Network-aware Data Management (NDM'14), 2014,

John Wu, Alex Sim, Lingfei Wu, Abraham Frankl, Scott Klasky, Jong Y Choi, CS Chang, Michael Churchill, "Exercising ICEE Framework with Fusion Blob Detection", DOE/ASCR NGNS PI meeting, 2014,

US Patent 8,705,342 B2. “Co-scheduling of network resource provisioning and host-to-host bandwidth reservation on high-performance network and storage systems”, D. Yu, D. Katramatos, A. Sim, and A. Shoshani, Apr. 22, 2014, LBNL IB-3152, BNL BSA 11-02.

A. L. Chervenak, A. Sim, J. Gu, R. Schuler, N. Hirpathak, "Efficient Data Staging Using Performance-Based Adaptation and Policy-Based Resource Allocation", 22nd Euromicro International Conference on Parallel, Distributed and Network-based Processing, 2014,

Lingfei Wu, Kesheng Wu, Alex Sim, Michael Churchill, Jong Y Choi, Andreas Stathopoulos, CS Chang, Scott Klasky, "High-performance outlier detection algorithm for finding blob-filaments in plasma", Proc. of 5rd International Workshop on Big Data Analytics: Challenges and Opportunites (BDAC-14), held in conjunction with ACM/IEEE SC14, 2014,

Lingfei Wu, Kesheng Wu, Alex Sim, Andreas Stathopoulos, "Real-time outlier detection algorithm for finding blob-filaments in plasma", ACM/IEEE SC14 ACM SRC Poster, 2014,

David H. Bailey, Stephanie Ger, Marcos L\ opez Prado, Alexander Sim, Kesheng Wu, "Statistical Overfitting and Backtest Performance", http://ssrn.com/abstract2507040, ( January 1, 2014)

ISBN 978-1-78548-008-9

L. Wu, K. Wu, A. Sim, M. Churchill, J. Y. Choi, A. Stathopoulos, CS Chang, S. Klasky, "High-Performance Outlier Detection Algorithm for Blob-Filaments in Plasma", 5th International Workshop on Big Data Analytics: and Opportunities (BDAC 14), 2014,

2013

J. Choi, K. Hu, A. Sim, "Relational Dynamic Bayesian Networks with Locally Exchangeable Measures", 2013, LBNL 6341E,

K. Hu, J. Choi, J. Jiang, A. Sim, "Best Predictive GLMM using LASSO with Application on High- Speed Network", 2013, LBNL 6327E,

K. Hu, A. Sim, D. Antoniades, C. Dovrolis, "Estimating and Forecasting Network Traffic Performance based on Statistical Patterns Observed in SNMP data", the 9th International Conference on Machine Learning and Data Mining (MLDM2013), 2013,

D. Antoniades, K. Hu, A. Sim, C. Dovrolis, "What SNMP data can tell us about Edge-to-Edge network performance", Passive and Active Measurement Conference (PAM2013), 2013,

K. Hu, A. Sim, D. Antoniades, C. Dovrolis, Statistical Prediction Models for Network Traffic Performance, the APAN 35 conference and the Winter 2013 ESCC/Internet2 Joint Techs meeting (TIP2013), 2013,

Jong Y Choi, Kesheng Wu, Jacky C Wu, Alex Sim, Qing G Liu, Matthew Wolf, C Chang, Scott Klasky, "Icee: Wide-area in transit data processing framework for near real-time scientific applications", 4th SC Workshop on Petascale (Big) Data Analytics: Challenges and Opportunities in conjunction with SC13, 2013, 11,

2012

Junmin Gu, David Smith, Ann L. Chervenak, Alex Sim, "Adaptive Data Transfers that Utilize Policies for Resource Sharing", The 2nd International Workshop on Network-Aware Data Management Workshop (NDM2012), 2012,

Mehmet Balman, Eric Pouyoul, Yushu Yao, E. Wes Bethel, Burlen Loring, Prabhat, John Shalf, Alex Sim, and Brian L. Tierney, "Experiences with 100G Network Applications", In Proceedings of the Fifth international Workshop on Data-intensive Distributed Computing, in conjunction with ACM High Performance Distributing Computing (HPDC) Conference, 2012, Delft, Netherlands, June 2012, LBNL 5603E, doi: 10.1145/2286996.2287004

100Gbps networking has finally arrived, and many research and educational in- stitutions have begun to deploy 100Gbps routers and services. ESnet and Internet2 worked together to make 100Gbps networks available to researchers at the Super- computing 2011 conference in Seattle Washington. In this paper, we describe two of the first applications to take advantage of this network. We demonstrate a visu- alization application that enables remotely located scientists to gain insights from large datasets. We also demonstrate climate data movement and analysis over the 100Gbps network. We describe a number of application design issues and host tuning strategies necessary for enabling applications to scale to 100Gbps rates. 

M. Balman, A. Sim, "Scaling the Earth System Grid to 100Gbps Networks", 2012, LBNL 5794E,

D. Yu, D. Katramatos, A. Shoshani, A. Sim, J. Gu, V. Natarajan, "StorNet: Integrating Storage Resource Management with Dynamic Network Provisioning for Automated Data Transfer", International Committee for Future Accelerators (ICFA) Standing Committee on Inter-Regional Connectivity (SCIC) 2012 Report: Networking for High Energy Physics, 2012,

Benson Ma, Arie Shoshani, Alex Sim, Kesheng, Yong-Ik Byun, Jaegyoon Hahm, Min-Su Shin, "Efficient Attribute-Based Data Access in Astronomy", The 2nd International Workshop on Network-Aware Data Workshop (NDM2012), 2012, 562--571,

2011

J. Gu, D. Katramatos, X. Liu, V. Natarajan, A. Shoshani, A. Sim, D. Yu, S. Bradley, S. McKee, "StorNet: Integrated Dynamic Storage and Network Resource Provisioning and Management for Automated Data Transfers", Journal of Physics: Conf. Ser., 2011, 331, doi: 10.1088/1742- 6596/331/1/012002

A. Shoshani, I. Altintas, J. Chen, G. Chin, A. Choudhary, D. Crawl, T. Critchlow, K. Gao, B. Grimm, H. Iyer, C. Kamath, A. Khan, S. Klasky, S. Koehler, S. Lang, R. Latham, J. W. Li, W. Liao, J. Ligon, Q. Liu, B. Ludaescher, P. Mouallem, M. Nagappan, N. Podhorszki, R. Ross, D. Rotem, N. Samatova, C. Silva, A. Sim, R. Tchoua, R. Thakur, M. Vouk, K. Wu, W. Yu, "The Scientific Data Management Center: Available Technologies and Highlights", SciDAC Conference, 2011,

G. Garzoglio, J. Bester, K. Chadwick, D. Dykstra, D. Groep, J. Gu, T. Hesselroth, O. Koeroo, T. Levshina, S. Martin, M. Salle, N. Sharma, A. Sim, S. Timm, A. Verstegen, "Adoption of a SAML-XACML Profile for Authorization Interoperability across Grid Middleware in OSG and EGEE", Journal of Physics: Conf. Ser., 2011, 331, doi: 10.1088/1742-6596/331/6/062011

Junmin Gu, Dimitrios Katramatos, Xin Liu, Vijaya Natarajan, Arie Shoshani, Alex Sim, Dantong Yu, Scott Bradley, Shawn McKee, "StorNet: Co-Scheduling of End-to-End Bandwidth Reservation on Storage and Network Systems for High Performance Data Transfers", IEEE INFOCOM HSN 2011, 2011,

Dean N. Williams, Ian T. Foster, Don E. Middleton, Rachana Ananthakrishnan, Neill Miller, Mehmet Balman, Junmin Gu, Vijaya Natarajan, Arie Shoshani, Alex Sim, Gavin Bell, Robert Drach, Michael Ganzberger, Jim Ahrens, Phil Jones, Daniel Crichton, Luca Cinquini, David Brown, Danielle Harper, Nathan Hook, Eric Nienhouse, Gary Strand, Hannah Wilcox, Nathan Wilhelmi, Stephan Zednik, Steve Hankin, Roland Schweitzer, John Harney, Ross Miller, Galen Shipman, Feiyi Wang, Peter Fox, Patrick West, Stephan Zednik, Ann Chervenak, Craig Ward, "Earth System Grid Center for Enabling Technologies (ESG-CET): A Data Infrastructure for Data-Intensive Climate Research", SciDAC Conference, 2011,

2010

Alex Sim, Mehmet Balman, Dean N. Williams, Arie Shoshani, Vijaya Natarajan, "Adaptive Transfer Adjustment in Efficient Bulk Data Transfer Management for Climate Datasets", The 22nd IASTED International Conference on Parallel and Distributed Computing and System, Marina Del Rey, CA, November 20, 2010, LBNL 3985E,

Many scientific applications and experiments, such as high energy and nuclear physics, astrophysics, climate observation and modeling, combustion, nano-scale material sciences, and computational biology, generate extreme volumes of data with a large number of files. These data sources are distributed among national and international data repositories, and are shared by large numbers of geographically distributed scientists. A large portion of the data is frequently accessed, and a large volume of data is moved from one place to another for analysis and storage. A challenging issue in such efforts is the limited network capacity for moving large datasets. A tool that addresses this challenge is the Bulk Data Mover (BDM), a data transfer management tool used in the Earth System Grid (ESG) community. It has been managing massive dataset transfers efficiently in the environment where the network bandwidth is limited. Adaptive transfer adjustment was studied to enhance the BDM to handle significant end-to-end performance changes in the dynamic network environments as well as to control the data transfers for the desired transfer performance. We describe the results from our hands-on data transfer management experience in the climate research community. We study a practical transfer estimation model and state our initial results from the adaptive transfer adjustment methodology. 

Mehmet Balman, Evangelos Chaniotakis, Arie Shoshani, Alex Sim, "A Flexible Reservation Algorithm for Advance Network Provisioning", ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, New Orleans, LA, November 2010 (SC'10)., New Orleans, LA, IEEE Computer Society Washington, DC, USA ISBN: 978-1-4244-7559-, November 14, 2010, LBNL 4017E, doi: http://dx.doi.org/10.1109/SC.2010.4

Many scientific applications need support from a communication infrastructure that provides predictable performance, which requires effective algorithms for bandwidth reservations. Network reservation sys- tems such as ESnet’s OSCARS, establish guaranteed bandwidth of secure virtual circuits for a certain bandwidth and length of time. However, users currently cannot inquire about bandwidth availability, nor have alternative suggestions when reservation requests fail. In general, the number of reservation options is exponential with the number of nodes n, and current reservation commitments. We present a novel approach for path finding in time-dependent networks taking advantage of user-provided parameters of total volume and time constraints, which produces options for earliest completion and shortest duration. The theoretical complexity is only O(n2r2) in the worst-case, where r is the number of reservations in the desired time interval. We have implemented our algorithm and developed efficient methodologies for incorporation into network reservation frameworks. Performance measurements confirm the theoretical predictions. 

D. Hasenkamp, A. Sim, M. Wehner, K. Wu, "Finding Tropical Cyclones on Clouds", Supercomputing 2010, ACM SRC 3rd place, 2010,

M. Balman, E. Chaniotakis, A. Shoshani, A. Sim, "A New Approach in Advance Network Reservation and Provisioning for High-Performance Scientific Data Transfers", 2010, LBNL 4091E,

Julian Cummings, Jay Lofstead, Karsten Schwan, Alexander Sim, Arie Shoshani, Ciprian Docan, Manish Parashar, Scott Klasky, Norbert Podhorszki, Roselyne Barreto, "EFFIS: An End-to-end Framework for Fusion Integrated Simulation", 18th Euromicro Conference on Parallel, Distributed and Network-based Processing, 2010,

G. Attebury, A. Baranovski, K. Bloom, B. Bockelman, D. Kcira, J. Letts, T. Levshina, C. Lundestedt, T. Martin, W. Maier, H. Pi, A. Rana, I. Sfiligoi, A. Sim, M. Thomas, F. Wuerthwein, "Roadmap for Applying Hadoop Distributed File System in Scientific Grid Computing", International Symposium on Grid Computing (ISGC), 2010,

A. Sim, D. Gunter, V. Natarajan, A. Shoshani, D. Williams, J. Long, J. Hick, J. Lee, E. Dart, "Efficient Bulk Data Replication for the Earth System Grid", Data Driven E-science: Use Cases and Successful Applications of Distributed Computing Infrastructures (ISGC 2010), (Springer-Verlag New York Inc: 2010) Pages: 435

Raj Kettimuthu, Alex Sim, Dan Gunter, Bill Allcock, Peer T. Bremer, John Bresnahan, Andrew Cherry, Lisa Childers, Eli Dart, Ian Foster, Kevin Harms, Jason Hick, Jason Lee, Michael Link, Jeff Long, Keith Miller, Vijaya Natarajan, Valerio Pascucci, Ken Raffenetti, David Ressman, Dean Williams, Loren Wilson, Linda Winkler, "Lessons learned from moving earth system grid data sets over a 20 Gbps wide-area network", HPDC 10, New York, NY, USA, ACM, 2010, 316--319, doi: 10.1145/1851476.1851519

Daren Hasenkamp, Alexander Sim, Michael Wehner, Kesheng Wu, "Finding tropical cyclones on a cloud computing cluster: Using parallel virtualization for large-scale climate simulation analysis", 2010 IEEE Second International Conference on Cloud Computing Technology and Science, 2010, 201--208, LBNL 4218E,

 

 

2009

A. Sim, A. Shoshani, F. Donno, J. Jensen, Storage Resource Manager Interface Specification V2.2 Implementations Experience Report, Open Grid Forum, GFD.154, 2009,

D. N. Williams, R. Ananthakrishnan, D. E. Bernholdt, S. Bharathi, D. Brown, M. Chen, A. L. Chervenak, L. Cinquini, R. Drach, I. T. Foster, P. Fox, D. Fraser, J. Garcia, S. Hankin, P. Jones, D. E. Middleton, J. Schwidder, R. Schweitzer, R. Schuler, A. Shoshani, F. Siebenlist, A. Sim, W. G. Strand, M. Su, N. Wilhelmi, "The Earth System Grid: Enabling Access to Multimodel Climate Simulation Data", American Meteorological Society, 2009, 90(2):195-205,

G. Attebury, A. Baranovski, K. Bloom, B. Bockelman, D. Kcira, J. Letts, T. Levshina, C. Lundestedt, T. Martin, W. Maier, H. Pi, A. Rana, I. Sfiligoi, A. Sim, M. Thomas, F. Wuerthwein, "Hadoop Distributed File System for the Grid", IEEE Nuclear Science Symposium, 2009,

M. Riedel, E. Laure, Th. Soddemann, L. Field, J. P. Navarro, J. Casey, M. Litmaath, J. Ph. Baud, B. Koblitz, C. Catlett, D. Skow, C. Zheng, P. M. Papadopoulos, M. Katz, N. Sharma, O. Smirnova, B. Kónya, P. Arzberger, F. Würthwein, A. S. Rana, T. Martin, M. Wan, V. Welch, T. Rimovsky, S. Newhouse, A. Vanni, Y. Tanaka, Y. Tanimura, T. Ikegami, D. Abramson, C. Enticott, G. Jenkins, R. Pordes, N. Sharma, S. Timm, N. Sharma, G. Moont, M. Aggarwal, D. Colling, O. van der Aa, A. Sim, V. Natarajan, A. Shoshani, J. Gu, S. Chen, G. Galang, R. Zappi, L. Magnoni, V. Ciaschini, M. Pace, V. Venturi, M. Marzolla, P. Andreetto, B. Cowles, S. Wang, Y. Saeki, H. Sato, S. Matsuoka, P. Uthayopas, S. Sriprayoonsakul, O. Koeroo, M. Viljoen, L. Pearlman, S. Pickles, David Wallom, G. Moloney, J. Lauret, J. Marsteller, P. Sheldon, S. Pathak, S. De Witt, J. Mencák, J. Jensen, M. Hodges, D. Ross, S. Phatanapherom, G. Netzer, A. R. Gregersen, M. Jones, S. Chen, P. Kacsuk, A. Streit, D. Mallmann, F. Wolf, T. Lippert, Th. Delaitre, E. Huedo, N. Geddes, "Interoperation of world-wide production e-Science infrastructures", Concurrency and Computation: Practice and Experience, 2009, 21(8):961-990,

Arie Shoshani, Flavia Donno, Junmin Gu, Jason Hick, Maarten Litmaath, Alex Sim, "Dynamic Storage Management", Scientific Data Management: Challenges, Technology, and Deployment, edited by Arie Shoshani, Doron Rotem, (Chapman & Hall/CRC Computational Science: 2009)

J. Jensen, R. Downing, D. Ross, A. Sim, "Practical Grid Storage Interoperation", Journal of Grid Computing, 2009, 7:3, doi: 10.1007/s10723-009-9127-2

K Wu, S Ahern, EW Bethel, J Chen, H Childs, C Geddes, J Gu, H Hagen, B Hamann, J Lauret, others, "FastBit: Interactively Searching Massive Data", Proc. of SciDAC 2009, 2009, LBNL 2164E,

2008

P. Jakl, J. Lauret, A. Hanushevsky, A. Shoshani, A. Sim, J. Gu, "Grid data access on widely distributed worker nodes using scalla and SRM", Journal of Physics: Conf. Ser., 2008, 119, doi: 10.1088/1742-6596/119/7/072019

D. N. Williams, R. Ananthakrishnan, D. E. Bernholdt, S. Bharathi, D. Brown, M. Chen, A. L. Chervenak, L. Cinquini, R. Drach, I. T. Foster, P. Fox, S. Hankin, V. E. Henson, P. Jones, D. E. Middleton, J. Schwidder, R. Schweitzer, R. Schuler, A Shoshani, F. Siebenlist, A. Sim, W. G. Strand, N. Wilhelmi, M. Su, "Data Management and Analysis for the Earth System Grid", SciDAC Conference, 2008,

Alex Sim, Arie Shoshani (Editors), Paolo Badino, Olof Barring, Jean‐Philippe Baud, Ezio Corso, Shaun De Witt, Flavia Donno, Junmin Gu, Michael Haddox‐Schatz, Bryan Hess, Jens Jensen, Andy Kowalski, Maarten Litmaath, Luca Magnoni, Timur Perelmutov, Don Petravick, Chip Watson, The Storage Resource Manager Interface Specification Version 2.2, Open Grid Forum, Document in Full Recommendation, GFD.129, 2008,

C S Chang, S Klasky, J Cummings, R. Samtaney, A Shoshani, L Sugiyama, D Keyes, S Ku, G Park, S Parker, N Podhorszki, H. Strauss, H Abbasi, M Adams, R Barreto, G Bateman, K Bennett, Y Chen, E D’Azevedo, C Docan, S Ethier, E Feibush, L Greengard, T Hahm, F Hinton, C Jin, A. Khan, A Kritz, P Krsti, T Lao, W Lee, Z Lin, J Lofstead, P Mouallem, M Nagappan, A Pankin, M Parashar, M Pindzola, C Reinhold, D Schultz, K Schwan, D. Silver, A Sim, D Stotler, M Vouk, M Wolf, H Weitzner, P Worley, Y Xiao, E Yoon, D Zorin, "Toward a first- principles integrated simulation of tokamak edge plasmas", Journal of Physics: Conf. Ser., 2008, 125, doi: 10.1088/1742-6596/125/1/012042

R Ananthakrishnan, D E Bernholdt, S Bharathi, D Brown, M Chen, A L Chervenak, L Cinquini, R Drach, I T Foster, P Fox, D Fraser, K Halliday, S Hankin, P Jones, C Kesselman, D E Middleton, J Schwidder, R Schweitzer, R Schuler, A Shoshani, F Siebenlist, A Sim, W G Strand, N Wilhelmi, M Su, D N Williams, "Building a global federation system for climate change research: the earth system grid center for enabling technologies (ESG-CET)", Journal of Physics: Conf. Ser., 2008, 78, doi: 10.1088/1742-6596/78/1/012050

W. Betts, L. Didenko, T. Freeman, P. Jakl, L. Hajdu, E. Hjort, K. Keahey, J. Lauret, D. Olson, A. Rose, I. Sakrejda, A. Sim, "STAR Grid Activities, OSG and Beyond", International Symposium on Grid Computing (ISGC), 2008,

Meiyappan Nagappan, Mladen A. Vouk, Kesheng Wu Alex Sim, Arie Shoshani, "Efficient Operational Profiling of Systems Using Arrays on Execution Logs", ISSRE, 2008, 313--314, doi: 10.1109/ISSRE.2008.45

2007

L. Abadie, P. Badino, J. Baud, E. Corso, M. Crawford, S. De Witt, F. Donno, A. Forti, P. Fuhrmann,
G. Grosdidier, J. Gu , J. Jensen, S. Lemaitre, M. Litmaath, D. Litvinsev, G. Lo Presti, L. Magnoni, T. Mkrtchan, A. Moibenko, V. Natarajan, G. Oleynik, T. Perelmutov, D. Petravick, A. Shoshani, A. Sim, M. Sponza, R. Zappi,
"Storage Resource Managers: Recent International Experience on Requirements and Multiple Co-Operating Implementations", the 24th IEEE Conference on Mass Storage Systems and Technologies, 2007,

F. Donno, L. Abadie, P. Badino, J. Baud, E. Corso, M. Crawford, S. De Witt, A. Forti, P. Fuhrmann, G. Grosdidier, J. Gu , J. Jensen, S. Lemaitre, M. Litmaath, D. Litvinsev, G. Lo Presti, L. Magnoni, T. Mkrtchan, A. Moibenko, V. Natarajan, G. Oleynik, T. Perelmutov, D. Petravick, A. Shoshani, A. Sim, M. Sponza, R. Zappi, "Storage Resource Manager version 2.2: design, implementation, and testing experience", Journal of Physics: Conf. Ser., 2007, 119, doi: 10.1088/1742-6596/119/6/062028

2006

A. Shoshani, A. Sim, K. Stockinger, "RRS: Replica Registration Service for Data Grids", Lecture Notes in Computer Science, edited by Jean-Marc Pierson, (Springer-Verlag GmbH Publisher: 2006) Pages: 100-112

D. E. Middleton, D. E. Bernholdt, D. Brown, M. Chen, A. L. Chervenak, L. Cinquini, R. Drach, P. Fox, P. Jones, C. Kesselman, I. T. Foster, V. Nefedova, A. Shoshani, A. Sim, W. G. Strand, D. Williams, "Enabling worldwide access to climate simulation data: the earth system grid (ESG)", SciDAV Conference, 2006,

P. Jakl, J. Lauret, A. Hanushevky, A. Shoshani, A. Sim, "From rootd to Xrootd, from physical to logical files: experience on accessing and managing distributed data", Computing in High Energy Physics (CHEP), 2006,

E. Hjort, L. Hajdu, J. Lauret, D. Olson, A. Sim, A. Shoshani, "Data and Computational Grid Coupling in RHIC/STAR – An Analysis Scenario using SRM Technology", Computing in High Energy Physics (CHEP), 2006,

2005

D. Bernholdt, S. Bharathi, D. Brown, K. Chanchio, M. Chen, A. Chervenak, L. Cinquini, B. Zrach, I. Foster, P. Fox, J. Garcia, C. Kesselman, R. Markel, D. Middleton, V. Nefedova, L. Pouchard, A. Shoshani, A. Sim, G. Strand, D. Williams, "The Earth System Grid: Supporting the Next Generation of Climate Modeling Research", IEEE, 2005, 93(3):485-495,

A. Shoshani, A. Sim, K. Stockinger, "RRS: Replica Registration Service for Data Grids", International Workshop on Data Management in Grids, 2005,

Arie Shoshani, Alex Sim, Kurt Stockinger, "Replica Registration Service Functional Interface Specification 1.0", 2005, LBNL 57520,

Kesheng Wu, Junmin Gu, Jerome Lauret, Arthur Poskanzer, Arie Shoshani, Alexander Sim, Zhang, "Grid Collector: Facilitating Efficient Selective from Data Grids", International Supercomputer Conference 2005, 2005,

2004

Eric Hjort, Doug Olson, Jerome Lauret, Arie Shoshani, Alex Sim, "Production mode Data- Replication framework in STAR using the HRM Grid middleware", Computing in High Energy Physics, 2004,

Alex Sim, Junmin Gu, Arie Shoshani, Vijaya Natarajan, "DataMover: Robust Terabytes-Scale Multi-file Replication over Wide-Area Networks", the 16th International Conference on Scientific and Statistical Database Management (SSDBM 2004), 2004,

2003

Arie Shoshani, Alexander Sim, Junmin Gu, "Storage Resource Managers: Essential Components for the Grid", Grid Resource Management: State of the Art and Future Trends, edited by Jarek Nabrzyski, Jennifer M. Schopf, Jan Weglarz, (Kluwer Academic Publishers: 2003)

Ann L. Chervenak, Ewa Deelman, Carl Kesselman, William E. Allcock, Ian T. Foster, Veronika Nefedova, Jason Lee, Alex Sim, Arie Shoshani, Bob Drach, Dean Williams, Don Middleton, "High-performance remote access to climate simulation data: a challenge problem for data grid technologies", Parallel Computing, 2003, 29(10):1335-1356,

A. Sim, J. Gu, A. Shoshani, E. Hjort, D. Olson, "Experience with Deploying Storage Resource Managers to Achieve Robust File Replication", Computing in High Energy Physics, 2003,

D. Yu, J. Lauret, A. Shoshani, D. Oldon, E. Hjort, A. Sim, "The Design of High Performance Data Replication in the Grid Environment for the STAR Collaboration", Computing in High Energy Physics, 2003,

L. Pouchard, L. Cinquini, B. Drach, D. Middleton, D. Bernholdt, K. Chanchio, I. Foster, V. Nefedova, D. Brown, P. Fox, J. Garcia, G. Strand, D. Williams, A. Chervenak, C. Kesselman, A. Shoshani, A. Sim, "An Ontology for Scientific Information in a Grid Environment: the Earth System Grid", the Symposium on Cluster Computing and the Grid (CCGrid), 2003,

Arie Shoshani, Alex Sim, Junmin Gu, Storage Resource Managers: Essential Components for Grid Applications, Globus World, 2003,

Kesheng Wu, Wei-Ming Zhang, Alexander Sim, Gu, Arie Shoshani, "Grid Collector: An Event Catalog With Automated File", Proceedings of IEEE Nuclear Science Symposium 2003, 2003, doi: 10.1109/NSSMIC.2003.1351830

Kesheng Wu, Wei-Ming Zlang, Alexander Sim, Junmin Gu, Arie Shoshani, "Grid collector: An event catalog with automated file management", 2003 IEEE Nuclear Science Symposium. Conference Record (IEEE Cat. No. 03CH37515), 2003, LBNL 55563,

2002

A. Shoshani, A. Sim, J. Gu, "Storage Resource Managers: Middleware components for Grid Storage", the 19th IEEE Symposium on Mass Storage Systems, 2002,

2001

B. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Lee, A. Sim, A. Shoshani, B. Drach, D. Williams, "High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies", Super Computing 2001, 2001,

A. Sim, H. Nordberg, L.M. Bernardo, A. Shoshani, D. Rotem, "Experience with using CORBA to implement a file caching coordination system", Concurrency and Computation: Practice and Experience, 2001, 13:1-15,

D. Olson, E. Hjort, J. Lauret, M. Messer, A. Shoshani, A. Sim, "Non-shared Disk Cluster - A Fault Tolerant, Commodity Approach to Hi-Bandwidth Data Analysis", Computing in High Energy Physics, 2001,

E. Hjort, D. Olson, A. Sim, J. Yang, J. Lauret, M. Messer, "Data Grid Services in STAR, Initial Deployment: Site-to-Site File Replication", Computing in High Energy Physics, 2001,

L Bernardo, H Nordberg, D Olson, A Shoshani, A Sim, A Vaniachine, D Zimmerman, B Gibbard, R Porter, T Wenaus, others, "New capabilities in the HENP grand challenge storage access system and its application at RHIC", Computer physics communications, 2001, 140:179--188,

L. Bernardo, H. Nordberg, D. Olson, A. Sim, A. Vaniachine, D. Zimmerman, B. Gibbard, R. Porter, T. Wenaus, D., "New capabilities in the HENP Grand Challenge Storage System and its application at RHIC", Computer Physics Communications, 2001, 140:179--188,

2000

A. Shoshani, A. Sim, L.M. Bernerdo, H. Nordberg, "Coordinating Simultaneous Caching of File Bundles from Tertiary Storage", International Conference on Scientific and Statistical Database Management (SSDBM), 2000,

L. M. Bernardo, B. Gibbard, D. Malon, H. Nordberg, D. Olson, R. Porter, A. Shoshani, A. Sim, A. Vaniachine, T. Wenaus, K. Wu, D. Zimmerman, "New Capabilities in the HENP Grand Challenge Storage Access System and its Application at RHIC", Computing in High Energy Physics, 2000,

L. M. Bernardo, A. Shoshani, A. Sim, H. Nordberg, "Access Coordination Of Tertiary Storage For High Energy Physics Applications", the 17th IEEE Symposium on Mass Storage Systems, 2000,

A. Sim, A. Shoshani, HRM: Hierarchical Resource Manager, Globus World, 2000,

A. Sim, A. Shoshani, L. M. Bernardo, H. Nordberg, A Storage Access Coordination System for Perabyte Scale Scientific Data, IONA World, 2000,

1999

A. Sim, H. Nordberg, L. M. Bernardo, A. Shoshani, D. Rotem, "Storage Access Coordination Using CORBA", Distributed Objects and Application, 1999, 168-175,

A. Shoshani, L.M. Bernardo, H. Nordberg, D. Rotem and A. Sim, "Multidimensional Indexing and Query Coordination for Tertiary Storage Management", International Conference on Scientific and Statistical Database Management, 1999, 214-225,

1998

L.M. Bernardo, D. Rotem, A. Shoshani, H. Nordberg, A. Sim, "Using Access Patterns to Partition Large Datasets on Tertiary Storage in Order to Minimize Retrieval Costs", 1998, LBNL 41504,

A. Shoshani, L.M. Bernardo, H. Nordberg, D. Rotem, A. Sim, "Storage Management for High Energy Physics Applications", Computing in High Energy Physics, 1998,

1996

A. Sim, B. Parvin, P. Keagy, "Invariant Representation and Classification of Fruits from X-ray Images", International Journal of Imaging Systems and Technology, 1996, 7:231-237,

1995

A. Sim, B. Parvin, P. Keagy, "Invariant Representation and Hierarchical Network for Inspection of Nuts from X-ray Images", IEEE International Conference on Neural Networks, 1995, II:738-743,

A. Sim, B. Parvin, P. Keagy, "Machine Vision Inspection of Insect Infested Pistachio Nuts from X-ray Images", Vision Interface, 1995, 17-22,

Houjun Tang

2024

D.K. Sung, Y. Son, A. Sim, K. Wu, S. Byna, H. Tang, H. Eom, C. Kim, S. Kim, "A2FL: Autonomous and Adaptive File Layout in HPC through Real-time Access Pattern Analysis", 38th IEEE International Parallel & Distributed Processing Symposium (IPDPS2024), 2024,

Jean Luca Bez, Houjun Tang, Scot Breitenfeld, Huihuo Zheng, Wei-Keng Liao, Kaiyuan Hou, Zanhua Huang, Suren Byna, "h5bench: Exploring HDF5 Access Patterns Performance in Pre-Exascale Platforms", Concurrency and Computation: Practice and Experience (CCPE), January 31, 2024,

2022

Houjun Tang, Quincey Koziol, John Ravi, and Suren Byna,, "Transparent Asynchronous Parallel I/O using Background Threads", IEEE Transactions on Parallel and Distributed Systems, April 4, 2022, 33, doi: 10.1109/TPDS.2021.3090322

2021

Houjun Tang, Bing Xie, Suren Byna, Phillip Carns, Quincey Koziol, Sudarsun Kannan, Jay Lofstead, and Sarp Oral,, "SCTuner: An Auto-tuner Addressing Dynamic I/O Needs on Supercomputer I/O Sub-systems", 6th International Parallel Data Systems Workshop (PDSW) 2021, held in conjunction with SC21, November 21, 2021,

Cong Xu, Suparna Bhattacharya, Martin Foltin, Suren Byna, and Paolo Faraboschi, "Data-Aware Storage Tiering for Deep Learning", 6th International Parallel Data Systems Workshop (PDSW) 2021, held in conjunction with SC21, November 21, 2021,

Suren Byna, Houjun Tang, and Quincey Koziol,, Automatic and Transparent Scientific Data Management with Object Abstractions, PASC 2021, in a Minisymposium on "Data Movement Orchestration on HPC Systems", July 31, 2021,

Bing Xie, Houjun Tang, Suren Byna, Jesse Hanley, Quincey Koziol, Tonglin Li, Sarp Oral,, "Battle of the Defaults: Extracting Performance Characteristics of HDF5 under Production Load", CCGrid 2021, May 31, 2021,

David McCallen, Houjun Tang, Suiwen Wu, Eric Eckert, Junfei Huang, N Anders Petersson, "Coupling of regional geophysics and local soil-structure models in the EQSIM fault-to-structure earthquake simulation framework", The International Journal of High Performance Computing Applications, May 25, 2021, doi: 10.1177/10943420211019118

David McCallen, Anders Petersson, Arthur Rodgers, Arben Pitarka, Mamun Miah, Floriana Petrone, Bjorn Sjogreen, Norman Abrahamson, Houjun Tang, "EQSIM—A multidisciplinary framework for fault-to-structure earthquake simulations on exascale computers part I: Computational models and workflow", Earthquake Spectra, May 1, 2021, 37:707-735, doi: 10.1177/8755293020970982

Tonglin Li, Suren Byna, Quincey Koziol, Houjun Tang, Jean Luca Bez, Qiao Kang, "h5bench: HDF5 I/O Kernel Suite for Exercising HPC I/O Patterns", Cray User Group (CUG) 2021, January 1, 2021,

Jean Luca Bez, Houjun Tang, Bing Xie, David Williams-Young, Rob Latham, Rob Ross, Sarp Oral, Suren Byna, "I/O Bottleneck Detection and Tuning: Connecting the Dots using Interactive Log Analysis", 2021 IEEE/ACM Sixth International Parallel Data Systems Workshop (PDSW), January 1, 2021, 15-22, doi: 10.1109/PDSW54622.2021.00008

2020

Houjun Tang, Suren Byna, Bin Dong, Quincey Koziol, "Parallel Query Service for Object-centric Data Management Systems", 2020 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), IEEE, May 18, 2020, 406-415,

Suren Byna, M. Scot Breitenfeld, Bin Dong, Quincey Koziol, Elena Pourmal, Dana Robinson, Jerome Soumagne, Houjun Tang, Venkatram Vishwanath, and Richard Warren, "ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems", Journal of Computer Science and Technology 2020, 35(1): 145-160, February 2, 2020, doi: 10.1007/s11390-020-9822-9

2019

Richard Warren, Jerome Soumagne, Jingqing Mu, Houjun Tang, Suren Byna, Bin Dong, Quincey Koziol, "Analysis in the Data Path of an Object-centric Data Management System", 26th IEEE International Conference on High Performance Computing, Data, and Analytics (HiPC 2019), December 18, 2019,

Houjun Tang, Suren Byna, Stephen Bailey, Zarija Lukic, Jialin Liu, Quincey Koziol, Bin Dong, "Tuning Object-centric Data Management Systems for Large Scale Scientific Applications", 26th IEEE International Conference on High Performance Computing, Data, and Analytics (HiPC 2019), December 18, 2019,

Wei Zhang, Suren Byna, Houjun Tang, Brody Williams, Yong Chen, "MIQS: Metadata Indexing and erying Service for Self-Describing File Formats", The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC19), November 19, 2019,

Houjun Tang, Quincey Koziol, Suren Byna, John Mainzer, Tonglin Li, "Enabling Transparent Asynchronous I/O using Background Threads", 2019 IEEE/ACM Fourth International Parallel Data Systems Workshop (PDSW 2019), November 19, 2019, doi: DOI 10.1109/PDSW49588.2019.00006

Tonglin Li, Quincey Koziol, Houjun Tang, Jialin Liu, Suren Byna, "I/O Performance Analysis of Science Applications Using HDF5 File-level Provenance", Cray User Group (CUG) 2019, May 10, 2019,

Jingqing Mu, Jerome Soumagne, Suren Byna, Quincey Koziol, Houjun Tang, Richard Warren, "Interfacing HDF5 with A Scalable Object-centric Storage System on Hierarchical Storage", Cray User Group (CUG) 2019, May 7, 2019,

Bin Dong, Kesheng Wu, Suren Byna, Houjun Tang, "SLOPE: Structural Locality-Aware Programming Model for Composing Array Data Analysis", International Conference on High Performance Computing, January 1, 2019, 61--80,

2018

Suren Byna, Quincey Koziol, Venkatram Vishwanath, Jerome Soumagne, Houjun Tang, Kimmy Mu, Richard Warren, François Tessier, Bin Dong, Teng Wang, and Jialin Liu, Proactive Data Containers (PDC): An object-centric data store for large-scale computing systems, AGU Fall Meeting, December 13, 2018,

Jialin Liu, Quincey Koziol, Gregory Butler, Neil Fortner, Mohamad Chaarawi, Houjun Tang, Suren Byna, Glenn Lockwood, Ravi Cheema, Kristy Kallback-Rose, Damian Hazen, Prabhat, "Evaluation of HPC Application I/O on Object Storage Systems", 3rd Joint International Workshop on Parallel Data Storage and Data Intensive Scalable Computing Systems (PDSW-DISCS), November 12, 2018,

Wei Zhang, Houjun Tang, Suren Byna, Yong Chen, "DART: Distributed Adaptive Radix Tree for Efficient Affix-based Keyword Search on HPC Systems", Proceedings of the 27th International Conference on Parallel Architectures and Compilation Techniques, November 1, 2018, 24,

Kimmy Mu, Jerome Soumagne, Houjun Tang, Suren Byna, Quincey Koziol, Richard Warren, "A Server-managed Transparent Object Storage Abstraction for HPC", 2018 IEEE International Conference on Cluster Computing (CLUSTER), September 10, 2018,

Teng Wang, Suren Byna, Bin Dong, and Houjun Tang, "UniviStor: Integrated Hierarchical and Distributed Storage for HPC", IEEE Cluster 2018., September 1, 2018,

Houjun Tang, Suren Byna, Francois Tessier, Teng Wang, Bin Dong, Jingqing Mu, Quincey Koziol, Jerome Soumagne, Venkatram Vishwanath, Jialin Liu, and Richard Warren, "Toward Scalable and Asynchronous Object-centric Data Management for HPC", 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) 2018, May 1, 2018,

Bin Dong, Teng Wang, Houjun Tang, Quincey Koziol, Kesheng Wu, Suren Byna, "ARCHIE: Data analysis acceleration with array caching in hierarchical storage", 2018 IEEE International Conference on Big Data (Big Data), January 1, 2018, 211--220,

2017

Houjun Tang, Suren Byna, Bin Dong, Jialin Liu, and Quincey Koziol, "SoMeta: Scalable Object-centric Metadata Management for High Performance Computing", IEEE Cluster 2017, September 5, 2017,

Jialin Liu, Quincey Koziol, Houjun Tang, François Tessier, Wahid Bhimji, Brandon Cook, Brian Austin, Suren Byna, Bhupender Thakur, Glenn Lockwood, Jack Deslippe, Prabhat, "Understanding the I/O Performance Gap Between Cori KNL and Haswell", Cray User Group Conference 2017 (CUG 2017), May 1, 2017,

2016

Wenzhao Zhang, Houjun Tang, Xiaocheng Zou, Steven Harenberg, Qing Liu, Scott Klasky, Nagiza F Samatova, "Exploring Memory Hierarchy to Improve Scientific Data Read Performance", 2015 IEEE International Conference on Cluster Computing, 2016, 66--69,

Houjun Tang, Suren Byna, Steve Harenberg, Xiaocheng Zou, Wenzhao Zhang, Kesheng Wu, Bin Dong, Oliver Rubel, Kristofer Bouchard, Scott Klasky, others, "Usage pattern-driven dynamic data layout reorganization", 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), January 1, 2016, 356--365,

Wenzhao Zhang, Houjun Tang, Steve Harenberg, Surendra Byna, Xiaocheng Zou, Dharshi Devendran, Daniel F Martin, Kesheng Wu, Bin Dong, Scott Klasky, others, "Amrzone: A runtime amr data sharing framework for scientific applications", 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), January 1, 2016, 116--125,

Xiaocheng Zou, David A Boyuka II, Dhara Desai, Daniel F Martin, Suren Byna, Kesheng Wu, "AMR-aware in situ indexing and scalable querying", Proceedings of the 24th High Performance Computing Symposium, January 1, 2016, 26,

Houjun Tang, Suren Byna, Steve Harenberg, Wenzhao Zhang, Xiaocheng Zou, Daniel F Martin, Bin Dong, Dharshi Devendran, Kesheng Wu, David Trebotich, others, "In situ storage layout optimization for amr spatio-temporal read accesses", 2016 45th International Conference on Parallel Processing (ICPP), January 1, 2016, 406--415,

Wenzhao Zhang, Houjun Tang, Stephen Ranshous, Surendra Byna, Daniel F Mart\ \in, Kesheng Wu, Bin Dong, Scott Klasky, Nagiza F Samatova, "Exploring memory hierarchy and network topology for runtime AMR data sharing across scientific applications", 2016 IEEE International Conference on Big Data (Big Data), January 1, 2016, 1359--1366,

2015

Xiaocheng Zou, Kesheng Wu, David A. Boyuka, Daniel F. Martin, Suren Byna, Houjun, Kushal Bansal, Terry J. Ligocki, Hans Johansen, and Nagiza F. Samatova, "Parallel In Situ Detection of Connected Components Adaptive Mesh Refinement Data", Proceedings of the Cluster, Cloud and Grid Computing (CCGrid) 2015, 2015,

David A Boyuka II, Houjun Tang, Kushal Bansal, Xiaocheng Zou, Scott Klasky, Nagiza F Samatova, "The hyperdyadic index and generalized indexing and query with PIQUE", Proceedings of the 27th International Conference on Scientific and Statistical Database Management, 2015, 20,

2014

John Jenkins, Xiaocheng Zou, Houjun Tang, Dries Kimpe, Robert Ross, Nagiza F Samatova, "Radar: Runtime asymmetric data-access driven scientific data replication", International Supercomputing Conference, 2014, 296--313,

Houjun Tang, Xiaocheng Zou, John Jenkins, David A Boyuka II, Stephen Ranshous, Dries Kimpe, Scott Klasky, Nagiza F Samatova, "Improving read performance with online access pattern analysis and prefetching", European Conference on Parallel Processing, 2014, 246--257,

Xiaocheng Zou, Sriram Lakshminarasimhan, David A Boyuka II, Stephen Ranshous, Houjun Tang, Scott Klasky, Nagiza F Samatova, "Fast set intersection through run-time bitmap construction over pfordelta-compressed indexes", European Conference on Parallel Processing, 2014, 668--679,

2013

Eric R Schendel, Steve Harenberg, Houjun Tang, Venkatram Vishwanath, Michael E Papka, Nagiza F Samatova, "A generic high-performance method for deinterleaving scientific data", European Conference on Parallel Processing, 2013, 571--582,

1969

Md Kamal Hossain Chowdhury, Houjun Tang, Jean Luca Bez, Purushotham V. Bangalore, Suren Byna, "Efficient Asynchronous I/O with Request Merging", 2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), St. Petersburg, FL, USA, IEEE, December 31, 1969, 628-636, doi: 10.1109/IPDPSW59300.2023.00107

Daniela Ushizima

2018

T. Ke, A. S. Brewster, S. X. Yu, D. Ushizima, C. Yang, N. K. Sauter, "A convolutional neural network-based screening tool for X-ray serial crystallography", JOURNAL OF SYNCHROTRON RADIATION, April 24, 2018, 25:665-670, doi: 10.1107/S1600577518004873

2016

T Perciano, DM Ushizima, EW Bethel, YD Mizrahi, D Parkinson, JA Sethian, "Reduced-complexity image segmentation under parallel Markov Random Field formulation using graph partitioning", Proceedings - International Conference on Image Processing, ICIP, 2016, 2016-Aug:1259--1263, doi: 10.1109/ICIP.2016.7532560

DM Ushizima, HA Bale, EW Bethel, P Ercius, BA Helms, H Krishnan, LT Grinberg, M Haranczyk, AA Macdowell, K Odziomek, DY Parkinson, T Perciano, RO Ritchie, C Yang, "IDEAL: Images Across Domains, Experiments, Algorithms and Learning", JOM, 2016, 68:2963--2972, doi: 10.1007/s11837-016-2098-4

2012

E. Wes Bethel, David Camp, Hank Childs, Mark Howison, Hari Krishnan, Burlen Loring, Joerg Meyer, Prabhat, Oliver Ruebel, Daniela Ushizima, Gunther Weber, "Towards Exascale: High Performance Visualization and Analytics – Project Status Report. Technical Report", DOE Exascale Research Conference, April 2012,

Ushizima, D.M., Weber, G., Morozov, D., Bethel, W., Sethian, J.A., "Algorithms for Microstructure Description applied to Microtomography", Carbon Cycle 2.0 Symposium, February 10, 2012,

D Ushizima, D Morozov, GH Weber, AGC Bianchi, JA Sethian, EW Bethel, "Augmented topological descriptors of pore networks for material science", IEEE Transactions on Visualization and Computer Graphics, 2012, 18:2041--2050, LBNL 5964E, doi: 10.1109/TVCG.2012.200

2011

Uselton, A., Ushizima, D.M., "I/O Workload Analysis with Server-side Data Collection", Supercomputing (SC), November 13, 2011,

Ushizima, D.M., Parkinson, D., Nico, P., Ajo-Franklin, J., Macdowell, A., Kocar, B., Bethel E.W, Sethian J.A., "Statistical segmentation and porosity quantification of 3D x-ray microtomography", XXXIV Applications of Digital Image Processing: Proceeding of SPIE 2011, San Diego, CA, USA, August 2011,

Ushizima, D.M., Weber, G.H., Ajo-Franklin, J., Kim, Y., Macdowell, A., Morozov, D., Nico, P., Parkinson, D., Trebotich, D., Wan, J., and Bethel E.W., "Analysis and visualization for multiscale control of geologic CO2", Journal of Physics: Conference Series, Proceedings of SciDAC 2011, July 2011, LBNL Denver, CO, USA,

2010

D. M. Ushizima, F. N. S. Medeiros, J. Cuadros, C. I. O. Martins, "Vessel Network Detection Using Contour Evolution and Color Components", Proceedings of the 32nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Buenos Aires, Argentina, PubMed-MedLine, September 2010, 3129-3132,

C. Rycroft, D. M. Ushizima, R. Saye, C. M. Ghajar, J. A. Sethian, "Building a physical cell simulation and comparing with confocal microscopy", Bay Area Physical Sciences - Oncology Center (NCI) Meeting 2010, UCSF Medical Sciences, September 2010,

A.Uselton, K.Antypas, D.M.Ushizima, J.Sukharev, "File System Monitoring as a Window Into User I/O Requirements", CUG-2010, May 2010,

D. M. Ushizima, F. Medeiros, "Retinopathy diagnosis from ocular fundus image analysis, Modeling and Analysis of Biomedical Image", SIAM Conference on Imaging Science (IS10), Chicago, Il, April 2010,

D. M. Ushizima and J. Cuadros, "Ocular fundus for retinopathy characterization", Bay Area Vision Meeting, February 5, 2010,

Daniela Ushizima, Cameron Geddes, Estelle Cormier-Michel, E. Wes Bethel, Janet Jacobsen, Prabhat, Oliver Rubel, Gunther Weber, Bernard Hamann, Peter Messmer, Hans Hagen, "Automated detection and analysis of particle beams in laser-plasma accelerator simulations", Machine Learning, edited by Yagang Zhang, (In-Teh: February 2010) Pages: 367-389, LBNL 3845E,

Oliver R\ ubel, Sean Ahern, E Wes Bethel, Mark D Biggin, Hank Childs, Estelle Cormier-Michel, Angela DePace, Michael B Eisen, Charless C Fowlkes, Cameron GR Geddes, others, "Coupling visualization and data analysis for knowledge discovery from multi-dimensional scientific data", Procedia computer science, Elsevier, January 2010, 1:1757--1764, LBNL 3669E,

Oliver Rübel, Sean Ahern, E. Wes Bethel, D. Biggin, Hank Childs, Estelle, Angela DePace, Michael B. Eisen Charless C. Fowlkes, Cameron G. R. Geddes, Hagen, Bernd Hamann, Min-Yu Huang, Soile E. Keränen, David W. Knowles, Cris L. Hendriks, Jitendra Malik, Jeremy Meredith Peter Messmer, Prabhat, Daniela Ushizima, H. Weber, Kesheng Wu, "Coupling visualization and data analysis for knowledge from multi-dimensional scientific data", Procedia Computer Science, 2010, 1:1751--1758, doi: 10.1016/j.procs.2010.04.197

E. A. Carvalho, D. M. Ushizima, F. N. S. Medeiros, C. I. O. Martins, R. C. P. Marques, I. N. S. Oliveira, "SAR imagery segmentation by statistical region growing and hierarchical merging", Digital Signal Processing, 2010, 20:1365-1378, doi: 10.1016/j.dsp.2009.10.014

Gladeston C. Leite, Daniela M. Ushizima, Fátima N. S. Medeiros, Gilson G. De Lima, "Wavelet Analysis for Wind Fields Estimation", Sensors, 2010, 10:5994--6016, doi: 10.3390/s100605994

2009

E. W. Bethel, C. Johnson, S. Ahern, J. Bell, P.-T. Bremer, H. Childs, E. Cormier-Michel, M. Day, E. Deines, T. Fogal, C. Garth, C. G. R. Geddes, H. Hagen, B. Hamann, C. Hansen, J. Jacobsen, K. Joy, J. Kruger, J. Meredith, P. Messmer, G. Ostrouchov, V. Pascucci, K. Potter, Prabhat, D. Pugmire, O. Rubel, A. Sanderson, C. Silva, D. Ushizima, G. Weber, B. Whitlock, K. Wu, "Occam's Razor and Petascale Visual Data Analysis", SciDAC 2009, J. of Physics: Conference Series, San Diego, California, July 2009, LBNL 2210E,

Oliver R\ ubel, Cameron GR Geddes, Estelle Cormier-Michel, Kesheng Wu, Gunther H Weber, Daniela M Ushizima, Peter Messmer, Hans Hagen, Bernd Hamann, Wes Bethel, others, "Automatic beam path analysis of laser wakefield particle acceleration data", Computational Science \& Discovery, January 2009, 2:015005, LBNL 2734E,

C. G. R. Geddes, E Cormier-Michel, E. H. Esarey, C. B. Schroeder, J.-L. Vay, W. P. Leemans, D. L.. Bruhwiler, J. R. Cary, B. Cowan, M. Durant, P. Hamill, P. Messmer, P. Mullowney, C. Nieter, K. Paul, S. Shasharina, S. Veitzer, G. Weber, O. Rübel, D. Ushizima, Prabhat, E. W.Bethel, K. Wu, Large Fields for Smaller Facility Sources, SciDAC Review, Pages: 13-21, 2009,

Oliver R\ ubel, Cameron G R Geddes, Estelle, Kesheng Wu, Prabhat, Gunther H, Daniela M Ushizima, Peter Messmer, Hans, Bernd Hamann, Wes Bethel, "Automatic beam path analysis of laser wakefield acceleration data", Computational Science \& Discovery, 2009, 2:015005,

C. I. O. Martins, D. M. Ushizima, F. N. S. Medeiros, F. N. Bezerra, R. C. P. Marques, N. D. A. Mascarenhas, "Iterative Self-dual Reconstruction on Radar Image Recovery", Proc. of IEEE Workshop on Applications of Computer Vision, Snowbird, Utah, 2009, 37-42, LBNL 3846E,

Regis C. P. Marques, Fatima N. S. Medeiros, Daniela M. Ushizima, "Target Dectection on SAR Images Using Level Set Methods", IEEE Transactions on Systems, Man and Cybernetics, 2009, 39:214-222, LBNL 958E, doi: 10.1109/TSMCC.2008.2006685

2008

Daniela Ushizima, Oliver Rübel, Prabhat, Gunther Weber, E. Wes Bethel, Cecilia Aragon, Cameron Geddes, Estelle Cormier-Michel, Bernd Hamann, Peter Messmer, Hans Hagen, "Automated Analysis for Detecting Beams in Laser Wakefield Simulations", 2008 Seventh International Conference on Machine Learning and Applications, Proceedings of IEEE ICMLA'08, 2008, 382-387, LBNL 960E,

C.I.O. Martins, R.M.S. Veras, G.L.B. Ramalho, F.N.S. Medeiros, D. M. Ushizima, "Automatic Microaneurysm Detection and Characterization Through Digital Color Fundus Images", Brazilian Artificial Intelligence Community Conference, Tenth Brazilian Symposium on Neural Networks, Proceedings of IEEE SBRN'2008, 2008,

Gunther H. Weber

2022

E. Wes Bethel, Burlen Loring, Utkarsh Ayachit, P. N. Duque, Nicola Ferrier, Joseph Insley, Junmin Gu, Kress, Patrick O’Leary, Dave Pugmire, Silvio Rizzi, Thompson, Will Usher, Gunther H. Weber, Brad Whitlock, Wolf, Kesheng Wu, "Proximity Portability and In Transit, M-to-N Data Partitioning and Movement in SENSEI", In Situ Visualization for Computational Science, ( 2022) doi: 10.1007/978-3-030-81627-8_20

E. Wes Bethel, Burlen Loring, Utkarsh Ayatchit, David Camp, P. N. Duque, Nicola Ferrier, Joseph Insley, Junmin Gu, Kress, Patrick O’Leary, David Pugmire, Silvio Rizzi, Thompson, Gunther H. Weber, Brad Whitlock, Matthew Wolf, Kesheng Wu, "The SENSEI Generic In Situ Interface: Tool and Processing Portability at Scale", In Situ Visualization for Computational Science, ( 2022) doi: 10.1007/978-3-030-81627-8_13

Sugeerth Murugesan, Mariam Kiran, Bernd Hamann, Gunther H. Weber, "Netostat: Analyzing Dynamic Flow Patterns in High-Speed Networks", Cluster Computing, 2022, doi: 10.1007/s10586-022-03543-0

2021

Hamish A. Carr, Gunther H. Weber, Christopher M. Sewell, Oliver R\ ubel, Patricia Fasel, James P. Ahrens, "Scalable Contour Tree Computation by Data Parallel Peak Pruning", Transactions on Visualization and Computer Graphics, 2021, 27:2437--2454, doi: 10.1109/TVCG.2019.2948616

Hamish Carr, Oliver Rübel, Gunther H. Weber, James Ahrens, "Optimization and Augmentation for Data Parallel Contour Trees", IEEE Transactions on Visualization and Computer Graphics, 2021, doi: 10.1109/TVCG.2021.3064385

Robbie Sadre, Colin Ophus, Anstasiia Butko, Gunther H Weber, "Deep Learning Segmentation of Complex Features in Atomic-Resolution Phase Contrast Transmission Electron Microscopy Images", Microscopy and Microanalysis, 2021, doi: 10.1017/S1431927621000167

Jan-Tobias Sohns, Gunther H. Weber, Christoph Garth, "Distributed Task-Parallel Topology-Controlled Volume Rendering", Topological Methods in Data Analysis and Visualization VI: Theory, Algorithms, and Applications, (Springer International Publishing: 2021) Pages: 55-69 doi: 10.1007/978-3-030-83500-2_4

2020

H. Childs, S. Ahern, J. Ahrens, A. C. Bauer, J. Bennett, E. W. Bethel, P.-T. Bremer, E. Brugger, J. Cottam, M. Dorier, S. Dutta, J. Favre, T. Fogal, S. Frey, C. Garth, B. Geveci, W. F. Godoy, C. D. Hansen, C. Harrison, B. Hentschel, J. Insley, C. Johnson, S. Klasky, A. Knoll, J. Kress, M. Larsen, J. Lofstead, K.-L. Ma, P. Malakar, J. Meredith, K. Moreland, P. Navratil, P. O Leary, M. Parashar, V. Pascucci, J. Patchett, T. Peterka, S. Petruzza, N. Podhorszki, D. Pugmire, M. Rasquin, S. Rizzi, D. H. Rogers, S. Sane, F. Sauer, R. Sisneros, H.-W. Shen, W. Usher, R. Vickery, V. Vishwanath, I. Wald, R. Wang, G. H. Weber, B. Whitlock, M. Wolf, H. Yu, S. B. Ziegler, "A Terminology for In Situ Visualization and Analysis Systems", International Journal of High Performance Computing Applications, 2020, 34:676--691, doi: 10.1177/1094342020935991

Petar Hristov, Gunther H. Weber, Hamish A. Carr, Oliver R\ ubel, James P. Ahrens, "Data Parallel Hypersweeps for In Situ Topological Analysis", Proceedings of the 10th IEEE Symposium on Large Data Analysis and Visualization (LDAV), 2020, 12--21, doi: 10.1109/LDAV51489.2020.00008

Jonas Lukasczyk, Christoph Garth, Gunther H. Weber, Tim Biedert, Ross Maciejewski, Heike Leitte, "Dynamic Nested Tracking Graphs", IEEE Transactions on Visualization and Computer Graphics (Proceedings IEEE VIS 2019), 2020, 26:249--258, doi: 10.1109/TVCG.2019.2934368

Sugeerth Murugesan, Kristofer Bouchard, Jesse Brown, Mariam Kiran, Dan Lurie, Bernd Hamann, Gunther H. Weber, "State-based Network Similarity Visualization", Information Visualization, 2020, 19:96--113, doi: 10.1177/1473871619882019

Anna-Pia Lohfink, Florian Wetzels, Jonas Lukasczyk, Gunther H. Weber, Christoph Garth, "Fuzzy Contour Trees: Alignment and Joint Layout of Multiple Contour Trees", Computer Graphics Forum (Special Issue, Proceedings Eurographics/IEEE Symposium on Visualization), 2020, 39:343--355, doi: 10.1111/cgf.13985

Hamish A. Carr, Julien Tierney, Gunther H. Weber, "Pathological and Test Cases For Reeb Analysis", Topological Methods in Data Analysis and Visualization V: Theory, Algorithms, and Applications", (Springer International Publishing: 2020) Pages: 103--120 doi: 10.1007/978-3-030-43036-8_7

2018

GP Rodrigo, M Henderson, GH Weber, C Ophus, K Antypas, L Ramakrishnan, "ScienceSearch: Enabling Search through Automatic Metadata Generation", 2018 IEEE 14th International Conference on e-Science (e-Science), IEEE, 2018, doi: 10.1109/escience.2018.00025

Gunther H. Weber, Colin Ophus, Lavanya Ramakrishnan, "Automated Labeling of Electron Microscopy Images Using Deep Learning", Proc. IEEE/ACM Machine Learning in HPC Environments (MLHPC), 2018, 26--36, doi: 10.1109/MLHPC.2018.8638633

Tom Liebmann, Gunther H. Weber, Gerik Scheuermann, "Hierarchical Correlation Clustering in Multiple 2D Scalar Fields", Computer Graphics Forum (Special Issue, Proceedings Symposium on Visualization), 2018, 37, doi: 10.1111/cgf.13396

K Beketayev, D Yeliussizov, D Morozov, GH Weber, B Hamann, "Measuring the Error in Approximating the Sub-Level Set Topology of Sampled Scalar Data", International Journal of Computational Geometry and Applications, 2018, 28:57--77, doi: 10.1142/S0218195918500036

2017

Sugeerth Murugesan, Kristofer Bouchard, Jesse A. Brown, Bernd Hamann, William W. Seeley, Andrew Trujillo, Gunther H. Weber, "Brain Modulyzer: Interactive Visual Analysis of Functional Brain Connectivity", IEEE Transactions on Computational Biology and Bioinformatics, 2017, 14(4):805-818, LBNL 1005732, doi: 10.1109/TCBB.2016.2564970

Sugeerth Murugesan, Kristofer Bouchard, Edward Chang, Dougherty, Bernd Hamann, Gunther H. Weber, "Multi-scale Visual Analysis of Time-varying Electrocorticography Data Clustering of Brain Regions", BMC Bioinformatics, 2017, 18:236, doi: 10.1186/s12859-017-1633-9

Jonas Lukasczyk, Ross Maciejewski, Gunther H. Weber, Garth, Heike Leitte, "Nested Tracking Graphs", Computer Graphics Forum (Special Issue, Proceedings Symposium on Visualization), 2017, 36 (3):12--22, doi: 10.1111/cgf.13164

Gunther H. Weber, Sheelagh Carpendale, David Ebert, Brian Fisher Hans Hagen, Ben Shneiderman, Anders Ynnerman, "Apply or Die: On the Role and Assessment of Application Papers in", IEEE Computer Graphics \& Applications, 2017, 37 (3):96--104, doi: 10.1109/MCG.2017.51

GH Weber, MS Bandstra, DH Chivers, HH Elgammal, V Hendrix, J Kua, JS Maltz, K Muriki, Y Ong, K Song, MJ Quinlan, L Ramakrishnan, BJ Quiter, "Web-based visual data exploration for improved radiological source detection", Concurrency Computation, 2017, 29, doi: 10.1002/cpe.4203

P Oesterling, C Heine, GH Weber, D Morozov, G Scheuermann, "Computing and visualizing time-varying merge trees for high-dimensional data", Mathematics and Visualization, ( 2017) Pages: 87--101 doi: 10.1007/978-3-319-44684-4_5

2016

Brian Friesen, Ann Almgren, Zarija Lukić, Gunther Weber, Dmitriy Morozov, Vincent Beckner, Marcus Day, "In situ and in-transit analysis of cosmological simulations", Computational Astrophysics and Cosmology, 2016, 3 (4):1-18,

Wahid Bhimji, Debbie Bard, Melissa Romanus, David Paul, Andrey Ovsyannikov, Brian Friesen, Matt Bryson, Joaquin Correa, Glenn K. Lockwood, Vakho Tsulaia, Suren Byna, Steve Farrell, Doga Gursoy, Chris Daley, Vince Beckner, Brian Van Straalen, Nicholas Wright, Katie Antypas, Prabhat,, "Accelerating Science with the NERSC Burst Buffer Early User Program", Cray User Group (CUG) 2016, May 10, 2016,

Utkarsh Ayachit, Andrew Bauer, Earl PN Duque, Greg Eisenhauer, Nicola Ferrier, Junmin Gu, Kenneth E Jansen, Burlen Loring, Zarija Lukic, Suresh Menon, others, "Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures", SC 16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2016, 921--932, LBNL 1007264,

Andrey Ovsyannikov, Melissa Romanus, Brian Van Straalen, Gunther H. Weber, David Trebotich, "Scientific Workflows at DataWarp-Speed: Accelerated Data-Intensive Science using NERSC s Burst Buffer", Proceedings of the 1st Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems, IEEE Press, 2016, 1--6, doi: 10.1109/PDSW-DISCS.2016.005

Hamish A. Carr, Gunther H. Weber, Christopher M. Sewell, James P. Ahrens, "Parallel Peak Pruning for Scalable SMP Contour Tree Computation", Proceedings of the 6th IEEE Symposium on Large Data Analysis and Visualization (LDAV), 2016, 75--84, doi: 10.1109/LDAV.2016.7874312

Sugeerth Murugesan, Kristofer Bouchard, Edward Chang, Dougherty, Bernd Hamann, Gunther H. Weber, "Hierarchical Spatio-temporal Visual Analysis of Cluster Evolution in Data", Proceedings of the 7th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, New York, NY, USA, ACM, 2016, 630--639, doi: 10.1145/2975167.2985688

2014

Gunther H. Weber, Hans Johansen, Daniel T. Graves, Terry J. Ligocki, "Simulating Urban Environments for Energy Analysis", Proceedings Visualization in Environmental Sciences (EnvirVis), 2014, LBNL 6652E,

Gunther H. Weber, Helwig Hauser, "Interactive Visual Exploration and Analysis", Mathematics and Visualization, (Springer-Verlag: 2014) Pages: 161--174, LBNL 6655E,

Patrick Oesterling, Christian Heine, Gunther H. Weber, Gerik Scheuermann, "A Topology-Based Approach to Visualize the Thematic Composition of Document Collections", Theory and Applications of Natural Language Processing, (Springer International Publishing: 2014) Pages: 63-85 doi: 10.1007/978-3-319-12655-5_4

K Beketayev, D Yeliussizov, D Morozov, GH Weber, B Hamann, "Measuring the distance between merge trees", Mathematics and Visualization, ( 2014) Pages: 151--165 doi: 10.1007/978-3-319-04099-8_10

Dmitriy Morozov, Gunther H Weber, "Distributed Contour Trees", Topological Methods in Data Analysis and Visualization III, (Springer International Publishing: 2014) Pages: 89--102 doi: 10.1007/978-3-319-04099-8\_6

2013

Patrick Oesterling, Christian Heine, Gunther H. Weber, Gerik Scheuermann, "Visualizing global structure of nD point clouds as topological 1D height to enable supervised local data analysis", IEEE Transactions on Visualization and Computer Graphics, 2013, 19:514-526, LBNL 5694E,

Gunther H. Weber, Hank Childs, Jeremy S. Meredith, "Recent Advances in VisIt: Parallel Crack-free Isosurface Extraction", Numerical Modeling of Space Plasma Flows: Astronum-2012 (Astronomical Society of the Pacific Conference Series), 2013,

Dmitriy Morozov, Gunther Weber, "Distributed Merge Trees", PPoPP 13, New York, NY, USA, ACM, 2013, 93--102, doi: 10.1145/2442516.2442526

2012

Hank Childs, Eric Brugger, Brad Whitlock, Jeremy Meredith, Sean Ahern, David Pugmire, Kathleen Biagas, Mark Miller, Cyrus Harrison, Gunther H. Weber, Hari Krishnan, Thomas Fogal, Allen Sanderson, Christoph Garth, E. Wes Bethel, David Camp, Oliver Rubel, Marc Durant, Jean M. Favre, Paul Navratil, "VisIt: An End-User Tool For Visualizing and Analyzing Very Large Data", High Performance Visualization---Enabling Extreme-Scale Scientific Insight, ( October 2012) Pages: 357--372

Hank Childs, David Pugmire, Sean Ahern, Brad Whitlock, Mark Howison, Prabhat, Gunther Weber, E. Wes Bethel, "Visualization at Extreme Scale Concurrency", High Performance Visualization---Enabling Extreme-Scale Scientific Insight, ( October 2012) Pages: 291--306

Gunther H. Weber, Hank Childs, Jeremy S. Meredith, "Efficient Parallel Extraction of Crack-free Isosurfaces from Adaptive Mesh Refinement (AMR) Data", Proceedings of IEEE Symposium on Large Data Analysis and Visualization (LDAV), October 2012, 31--38, LBNL 5799E,

E. Wes Bethel, David Camp, Hank Childs, Mark Howison, Hari Krishnan, Burlen Loring, Joerg Meyer, Prabhat, Oliver Ruebel, Daniela Ushizima, Gunther Weber, "Towards Exascale: High Performance Visualization and Analytics – Project Status Report. Technical Report", DOE Exascale Research Conference, April 2012,

Gunther H. Weber, Kenes Beketayev, Peer-Timo Bremer, Bernd Hamann, Maciej Haranczyk, Mario Hlawitschka, Valerio Pascucci, "Comprehensible Presentation of Topological Information", Status report for DOE Exascale Research Conference, April 2012, LBNL 5693E,

Gunther H. Weber, Dmitriy Morozov, Kenes Beketayev, John Bell, Peer-Timo Bremer, Marc Day, Bernd Hamann, Christian Heine, Maciej Haranczyk, Mario Hlawitschka, Valerio Pascucci, Patrick Oesterling, Gerik Scheuermann, "Topology-based Visualization and Analysis of High-dimensional Data and Time-varying Data at the Extreme Scale", DOE Exascale Research Conference, April 2012,

Gunther H. Weber, Peer-Timo Bremer, "In-situ Analysis: Challenges and Opportunities", Position paper for DOE Exascale Research Conference, April 2012, LBNL 5692E,

Ushizima, D.M., Weber, G., Morozov, D., Bethel, W., Sethian, J.A., "Algorithms for Microstructure Description applied to Microtomography", Carbon Cycle 2.0 Symposium, February 10, 2012,

Allen R Sanderson, Brad Whitlock, H Childs, GH Weber, K Wu, others, "A system for query based analysis and visualization", January 2012, LBNL 5507E,

O. Rübel, S.V.E. Keränen, M.D. Biggin, D.W. Knowles, G.H. Weber, H. Hagen, B. Hamann, and E.W. Bethel, "Linking Advanced Visualization and MATLAB for the Analysis of 3D Gene Expression Data", Mathematics and Visualization, Visualization in Medicine and Life Sciences II, Progress and New Challenges, edited by L. Linsen and B. Hamann and H. Hagen and H.-C. Hege, (Springer Verlag: 2012) Pages: 267-285, LBNL 4891E,

D Ushizima, D Morozov, GH Weber, AGC Bianchi, JA Sethian, EW Bethel, "Augmented topological descriptors of pore networks for material science", IEEE Transactions on Visualization and Computer Graphics, 2012, 18:2041--2050, LBNL 5964E, doi: 10.1109/TVCG.2012.200

K Beketayev, GH Weber, D Morozov, A Abzhanov, B Hamann, "Geometry-preserving topological landscapes", Proceedings - WASA 2012: Workshop at SIGGRAPH Asia 2012, 2012, 155--160, doi: 10.1145/2425296.2425324

Dogan Demir, Kenes Beketayev, Gunther H. Weber, Peer-Timo Bremer, Valerio Pascucci, Bernd Hamann, "Topology Exploration with Hierarchical Landscapes", Proceedings of the Workshop at SIGGRAPH Asia 2012, New York, NY, USA, ACM, 2012, 147--154, doi: 10.1145/2425296.2425323

2011

Huang, M.-Y., Mackey, L., Keraenen, S.V.E., Weber, G.H., Jordan, M.I., Knowles, D.W., Biggin, M.D. and Hamann, B., "Visually Relating Gene Expression and in vivo DNA Binding Data", Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine 2011 (IEEE BIBM 2011), Los Alamitos, California, IEEE Computer Society Press, November 2011, 586-589, LBNL 5423E,

Ushizima, D.M., Weber, G.H., Ajo-Franklin, J., Kim, Y., Macdowell, A., Morozov, D., Nico, P., Parkinson, D., Trebotich, D., Wan, J., and Bethel E.W., "Analysis and visualization for multiscale control of geologic CO2", Journal of Physics: Conference Series, Proceedings of SciDAC 2011, July 2011, LBNL Denver, CO, USA,

Weber, G.H., Bremer, P.T., Gyulassy A., and Pascucci, "Topology-based Feature Definition and Analysis", Numerical Modeling of Space Plasma Flows, San Diego, CA, USA, Astronomical Society of the Pacific, June 2011, 444:292-297, LBNL 5020E,

Beketayev, K., Weber, G.H., Haranczyk, M., Bremer, P.-T., Hlawitschka, M., and Hamann, B., "Topology-based Visualization of Transformation Pathways in Complex Chemical Systems", Computer Graphics Forum (Special Issue, Proc. Eurographics / IEEE Symposium on Visualization), June 2011, 663-672, LBNL 5242E,

M Prabhat, S Byna, C Paciorek, G Weber, K Wu, T Yopes, MF Wehner, G Ostrouchov, D Pugmire, R Strelitz, others, "Pattern Detection and Extreme Value Analysis on Large Climate Data", AGUFM, Pages: IN41C--03 January 2011,

Prabhat, S. Byna, C. Paciorek, G. Weber, Wu, T. Yopes, M. Wehner, W. Collins, G., R. Strelitz, E. W. Bethel, Pattern Detection and Extreme Value Analysis on Large Data, DOE/BER Climate and Earth System Modeling PI Meeting, 2011,

MacCarthy, B., Carr, H., Weber, G.H., "Topological Galleries: A High Level User Interface for Topology Controlled Volume Rendering", 2011, LBNL 5019E,

Weber, G.H., Bremer, P.-T. and Pascucci, V., "Topological Cacti: Visualizing Contour-based Statistics", Topological Methods in Data Analysis and Visualization II, (Springer Verlag: 2011) Pages: 63-76, LBNL 5018E,

Deines E., Weber, G.H., Garth, C., Van Straalen, B. Borovikov, S., Martin, D.F., and Joy, K.I., "On the computation of integral curves in adaptive mesh refinement vector fields", Proceedings of Dagstuhl Seminar on Scientific Visualization 2009, Schloss Dagstuhl, 2011, 2:73-91, LBNL 4972E,

  • Download File: 7.pdf (pdf: 799 KB)

Bremer, P.-T., Weber, G.H., Tierny, J., Pascucci, V., Day, M.S., and Bell, J.B., "Interactive Exploration and Analysis of Large Scale Turbulent Combustion Using Topology-based Data Segmentation", IEEE Transactions on Visualization and Computer Graphics, 2011, 17(9):1307-1324, LBNL 5921E, doi: 10.1109/TVCG.2010.253

G Weber, PT Bremer, M Day, J Bell, V Pascucci, "Feature tracking using Reeb graphs", Mathematics and Visualization, ( 2011) Pages: 241--253, LBNL 4226E, doi: 10.1007/978-3-642-15014-2_20

2010

Min-Yu Huang, Gunther H. Weber, Xiao-Yong Li, Mark D. Biggin, and Bernd Hamann, "Quantitative Visualization of ChIP-chip Data by Using Linked Views", Proceedings IEEE International Conference on Bioinformatics and Biomedicine 2010 (IEEE BIBM 2010) Workshops, Workshop on Integrative Data Analysis in Systems Biology (IDASB), Los Alamitos, California, IEEE Computer Society Press, December 8, 2010, 195-200, LBNL 4491E,

Patrick Oesterling, Gerik Scheuermann, Sven Teresniak, Gerhard Heyer, Steffen Koch, Thomas Ertl, Gunther H. Weber, "Two-stage Framework for a Topology-Based Projection and Visualization of Classified Document Collections", Proceedings IEEE Symposium on Visual Analytics Science and Technology (IEEE VAST), Salt Lake City, Utah, USA, October 2010, LBNL 4074E,

O. Rübel, G. H. Weber, M-Y Huang, E. W. Bethel, M. D. Biggin, C. C. Fowlkes, C. Luengo Hendriks, S. V. E. Keränen, M. Eisen, D. Knowles, J. Malik, H. Hagen and B. Hamann, "Integrating Data Clustering and Visualization for the Analysis of 3D Gene Expression Data", IEEE Transactions on Computational Biology and Bioinformatics, March 2010, 7:64-79, LBNL 382E,

Daniela Ushizima, Cameron Geddes, Estelle Cormier-Michel, E. Wes Bethel, Janet Jacobsen, Prabhat, Oliver Rubel, Gunther Weber, Bernard Hamann, Peter Messmer, Hans Hagen, "Automated detection and analysis of particle beams in laser-plasma accelerator simulations", Machine Learning, edited by Yagang Zhang, (In-Teh: February 2010) Pages: 367-389, LBNL 3845E,

Oliver R\ ubel, Sean Ahern, E Wes Bethel, Mark D Biggin, Hank Childs, Estelle Cormier-Michel, Angela DePace, Michael B Eisen, Charless C Fowlkes, Cameron GR Geddes, others, "Coupling visualization and data analysis for knowledge discovery from multi-dimensional scientific data", Procedia computer science, Elsevier, January 2010, 1:1757--1764, LBNL 3669E,

Gunther Weber, "Recent advances in visit: Amr streamlines and query-driven visualization", 2010,

Hank Childs, David Pugmire, Sean Ahern, Brad Whitlock, Mark Howison, Prabhat, Gunther Weber, E. Wes Bethel, "Extreme Scaling of Production Visualization Software on Diverse Architectures", IEEE Computer Graphics and Applications, January 2010, 30:22--31, LBNL 3403E, doi: 10.1109/MCG.2010.51

Oliver Rübel, Sean Ahern, E. Wes Bethel, D. Biggin, Hank Childs, Estelle, Angela DePace, Michael B. Eisen Charless C. Fowlkes, Cameron G. R. Geddes, Hagen, Bernd Hamann, Min-Yu Huang, Soile E. Keränen, David W. Knowles, Cris L. Hendriks, Jitendra Malik, Jeremy Meredith Peter Messmer, Prabhat, Daniela Ushizima, H. Weber, Kesheng Wu, "Coupling visualization and data analysis for knowledge from multi-dimensional scientific data", Procedia Computer Science, 2010, 1:1751--1758, doi: 10.1016/j.procs.2010.04.197

PT Bremer, GH Weber, V Pascucci, M Day, JB Bell, "Analyzing and tracking burning structures in lean premixed hydrogen flames", IEEE Transactions on Visualization and Computer Graphics, 2010, 16:248--260, LBNL 2276E, doi: 10.1109/TVCG.2009.69

2009

D. Pugmire, H. Childs, C. Garth, S. Ahern, G.H. Weber, "Scalable computation of streamlines on very large datasets", Proc. Supercomputing, Portland, OR, USA, November 2009, LBNL 3264E,

E. W. Bethel, C. Johnson, S. Ahern, J. Bell, P.-T. Bremer, H. Childs, E. Cormier-Michel, M. Day, E. Deines, T. Fogal, C. Garth, C. G. R. Geddes, H. Hagen, B. Hamann, C. Hansen, J. Jacobsen, K. Joy, J. Kruger, J. Meredith, P. Messmer, G. Ostrouchov, V. Pascucci, K. Potter, Prabhat, D. Pugmire, O. Rubel, A. Sanderson, C. Silva, D. Ushizima, G. Weber, B. Whitlock, K. Wu, "Occam's Razor and Petascale Visual Data Analysis", SciDAC 2009, J. of Physics: Conference Series, San Diego, California, July 2009, LBNL 2210E,

N. Shah, N., S. E. Dillard, G.H. Weber, B. Hamann, "Volume visualization of multiple alignment of large genomic DNA", Mathematical Foundations of Scientific Visualization, Computer Graphics, and Massive Data Exploration, edited by Torsten Moller and Bernd Hamann and Robert Russell, (Springer-Verlag: July 2009) Pages: 325-342, LBNL 63126,

M. Hlawitschka, G.H. Weber, A. Anwander, O.T. Carmichael, B. Hamann and G. Scheuermann, "Interactive Volume Rendering of Diffusion Tensor Data", Visualization and Processing of Tensor Fields: Advances and Perspectives, (Springer-Verlag: April 2009) Pages: 161-176, LBNL 2286E,

S. E. Dillard, V. Natarjan, G. H. Weber, V. Pascucci and B. Hamann, "Topology-guided Tessellation of Quadratic Elements", International Journal of Computational Geometry & Applications (IJCGA), April 2009, 19:195-211, LBNL 63771,

K Wu, S Ahern, EW Bethel, J Chen, H Childs, C Geddes, J Gu, H Hagen, B Hamann, J Lauret, others, "FastBit: Interactively Searching Massive Data", Proc. of SciDAC 2009, 2009, LBNL 2164E,

Oliver R\ ubel, Cameron GR Geddes, Estelle Cormier-Michel, Kesheng Wu, Gunther H Weber, Daniela M Ushizima, Peter Messmer, Hans Hagen, Bernd Hamann, Wes Bethel, others, "Automatic beam path analysis of laser wakefield particle acceleration data", Computational Science \& Discovery, January 2009, 2:015005, LBNL 2734E,

C. G. R. Geddes, E Cormier-Michel, E. H. Esarey, C. B. Schroeder, J.-L. Vay, W. P. Leemans, D. L.. Bruhwiler, J. R. Cary, B. Cowan, M. Durant, P. Hamill, P. Messmer, P. Mullowney, C. Nieter, K. Paul, S. Shasharina, S. Veitzer, G. Weber, O. Rübel, D. Ushizima, Prabhat, E. W.Bethel, K. Wu, Large Fields for Smaller Facility Sources, SciDAC Review, Pages: 13-21, 2009,

E Bethel, "Modern Scientific Visualization is More than Just Pretty Pictures", January 2009, LBNL 1450E,

E Wes Bethel, Chris Johnson, Sean Ahern, John Bell, Peer-Timo Bremer, Hank Childs, Estelle Cormier-Michel, Marc Day, Eduard Deines, Tom Fogal, others, "Occam s razor and petascale visual data analysis", Journal of Physics: Conference Series, 2009, 180:012084,

C. Garth, E. Deines, K. Joy, E. W. Bethel, H. Childs, G. Weber, S. Ahern, D. Pugmire, A. Sanderson, C. Johnson, Twists and Turns: Vector Field Visual Data Analysis for Petascale Computational Science, SciDAC Review, Pages: 10-21, 2009,

Oliver R\ ubel, Cameron G R Geddes, Estelle, Kesheng Wu, Prabhat, Gunther H, Daniela M Ushizima, Peter Messmer, Hans, Bernd Hamann, Wes Bethel, "Automatic beam path analysis of laser wakefield acceleration data", Computational Science \& Discovery, 2009, 2:015005,

G. H. Weber, O. Rübel, M.-Y. Huang, A. H. DePace, C. C. Fowlkes, S. V. E. Keränen, C. L. Luengo Hendriks, H. Hagen, D. W. Knowles, J. Malik, M. D. Biggin and B. Hamann, "Visual exploration of three-dimensional gene expression using physical views and linked abstract views", IEEE Transactions on Computational Biology and Bioinformatics, 2009, 6:296-309, LBNL 63776, doi: 10.1109/TCBB.2007.70249

PT Bremer, GH Weber, J Tierny, V Pascucci, MS Day, JB Bell, "A topological framework for the interactive exploration of large scale turbulent combustion", e-Science 2009 - 5th IEEE International Conference on e-Science, January 2009, 247--254, LBNL 3183E, doi: 10.1109/e-Science.2009.42

2008

O. Rübel, Prabhat, K. Wu, H. Childs, J. Meredith, C.G.R. Geddes, E. Cormier-Michel, S. Ahern, G.H. Weber, P. Messmer, H. Hagen, B. Hamann and E.W. Bethel, "High Performance Multivariate Visual Data Exploration for Extemely Large Data", Supercomputing (SC), Austin, Texas, USA, November 2008, LBNL 716E,

O. Rübel, Prabhat, K. Wu, H. Childs, J. Meredith, C.G.R. Geddes, E. Cormier-Michel, S. Ahern, G.H. Weber, P. Messmer, H. Hagen, B. Hamann and E.W. Bethel, "Application of High-performance Visual Analysis Methods to Laser Wakefield Particle Acceleration Data", IEEE Visualization 2008, October 2008,

C. C. Fowlkes, C. L. Luengo Hendriks, S. V. E. Keränen, G. H. Weber, O. Rübel, M.-Y. Huang, S. Chatoor, A. H. DePace, L. Simirenko, C. Henriquez, A. Beaton, R. Weiszmann, S. Celniker, B. Hamann, D. W. Knowles, M. D. Biggin, M. B. Eisen, J. Malik, "A Quantitative Spatio-temporal Atlas of Gene Expression in the Drosophila Blastoderm", Cell, April 18, 2008, 133:364-374,

G.H. Weber, V. Beckner, H. Childs, T. Ligocki, M. Miller, B. van Straalen, E.W. Bethel, "Visualization of Scalar Adaptive Mesh Refinement Data", Numerical Modeling of Space Plasma Flows: Astronum-2007 (Astronomical Society of the Pacific Conference Series), April 2008, 385:309-320, LBNL 220E,

E. Wes Bethel, Oliver Rübel, Prabhat, Wu, Gunther H. Weber, Valerio Pascucci Hank Childs, Ajith Mascarenhas, Jeremy, Sean Ahern, "Modern Scientific Visualization is More than Just Pictures", Numerical Modeling of Space Plasma Flows: (Astronomical Society of the Pacific Series), St. Thomas, USVI, 2008, 301--317,

Oliver R\ ubel, Prabhat, Kesheng Wu, Hank, Jeremy Meredith, Cameron G. R. Geddes, Cormier-Michel, Sean Ahern, Gunther H., Peter Messmer, Hans Hagen, Bernd Hamann E. Wes Bethel, High Performance Multivariate Visual Data Exploration Extemely Large Data, SuperComputing 2008 (SC08), Pages: 51 2008,

Daniela Ushizima, Oliver Rübel, Prabhat, Gunther Weber, E. Wes Bethel, Cecilia Aragon, Cameron Geddes, Estelle Cormier-Michel, Bernd Hamann, Peter Messmer, Hans Hagen, "Automated Analysis for Detecting Beams in Laser Wakefield Simulations", 2008 Seventh International Conference on Machine Learning and Applications, Proceedings of IEEE ICMLA'08, 2008, 382-387, LBNL 960E,

E. Wes Bethel, Chris Johnson, Charles Hansen, Claudio Silva, Steven Parker, Allen Sanderson, Lee Myers, Martin Cole, Xavier Tricoche, Sean Ahern, George Ostrouchov, Dave Pugmire, Jamison Daniel, Jeremy Meredith, Valerio Pascucci, Hank Childs, Peer-Timo Bremer, Ajith Mascarenhas, Ken Joy, Bernd Hamann, Christoph Garth, Cecilia Aragon, Gunther Weber, and Prabhat, Seeing the Unseeable, SciDAC Review, Pages: 24-33, 2008,

O. Rübel, G. H. Weber, M-Y Huang, E. W. Bethel, S. V. E. Keränen, C. C. Fowlkes, C. L. Luengo Hendriks, A. H. DePace, L. Simirenko, M. B. Eisen, M. D. Biggin, H. Hagen, J. Malik, D. W. Knowles and B. Hamann, "PointCloudXplore 2: Visual Exploration of 3D Gene Expression", Visualization of Large and Unstructured Data Sets, edited by C. Garth, H. Hagen, M. Hering-Bertram, (Gesellschaft fuer Informatik (GI): 2008) LBNL 249E,

M.-Y. Huang, O. Rübel, G.H. Weber, C.L. Luengo Hendriks, M.D. Biggin, H. Hagen, B. Hamann, "Segmenting Gene Expression Patterns of Early-stage Drosophila Embryos.", Mathematical Methods for Visualization in Medicine and Life Sciences, edited by L. Linsen, H. Hagen, B. Hamann, (Springer-Verlag: January 2008) Pages: 313--327, LBNL 62450,

2007

Shengyin Gu, Iain Anderson, Victor Kunin, Michael Cipriano, Minovitsky, Gunther H. Weber, Nina Amenta, Bernd Hamann Inna Dubchak, "TreeQ-VISTA: An Interactive Tree Visualization Tool with Functional Query Capabilities", Bioinformatics, 2007, 23:764--766, doi: 10.1093/bioinformatics/btl643

Gunther H. Weber, Scott E. Dillard, Hamish Carr, Pascucci, Bernd Hamann, "Topology-Controlled Volume Rendering", IEEE Transactions on Visualization and Computer Graphics, 2007, 13:330--341, doi: 10.1109/TVCG.2007.47

Oliver G. Staadt, Vijay Natarjan, Gunther H. Weber, F. Wiley, B. Hamann, "Interactive Processing and Visualization of Image Data for Biomedical and Science Applications", BMC Cell Biology, 2007, 8:S10, doi: 10.1186/1471-2121-8-S1-S10

Gunther H. Weber, Peer-Timo Bremer, Valerio Pascucci, "Topological Landscapes: A Terrain Metaphor for Scientific Data", IEEE Transactions on Visualization and Computer Graphics Issue: Proceedings of IEEE Visualization 2007), 2007, 13:1416--1423, doi: 10.1109/TVCG.2007.70601

2006

Cris L. Luengo Hendriks, Soile V. E. Keränen, C. Fowlkes, Lisa Simirenko, Gunther H. Weber, H. DePace, Clara Henriquez, David W. Kaszuba, Hamann, Michael B. Eisen, Jitendra Malik, Damir Sudar, D. Biggin, David W. Knowles, "Three-dimensional Morphology and Gene Expression in the Drosophila Blastoderm at Cellular Resolution I: Data Acquisition Pipeline", Genome Biology, 2006, 7:R123, doi: 10.1186/gb-2006-7-12-r123

2003

Gunther H. Weber, Martin Öhler, Oliver Kreylos, John Shalf, Wes Bethel, Bernd Hamann, Gerik Scheuermann, "Parallel Cell Projection Rendering of Adaptive Mesh Refinement Data", IEEE Symposium on Parallel and Large-Data Visualization and Graphics, 2003, 51-60,

2001

Gunther H. Weber, Oliver Kreylos, Terry J. Ligocki, Jonh Shalf, Hans Hagen, Bernd Hamann, Ken I. Joy, Kwan-Liu Ma, "High-quality Volume Rendering of Adaptive Mesh Refinement Data", VMV, 2001, 121-128,

Kesheng Wu

2024

D.K. Sung, Y. Son, A. Sim, K. Wu, S. Byna, H. Tang, H. Eom, C. Kim, S. Kim, "A2FL: Autonomous and Adaptive File Layout in HPC through Real-time Access Pattern Analysis", 38th IEEE International Parallel & Distributed Processing Symposium (IPDPS2024), 2024,

L. Zhou, Q. Lin, K. Chowdhury, S. Masood, A. Eichenberger, H. Min, A. Sim, J. Wang, Y. Wang, K. Wu, B. Yuan, J. Zou, "Serving Deep Learning Model in Relational Databases", 27th International Conference on Extending Database Technology (EDBT2024), 2024,

R. Frehner, K. Wu, A. Sim, J. Kim, K. Stockinger, "Detecting Anomalies in Time Series Using Kernel Density Approaches", IEEE Access, 2024, doi: 10.1109/ACCESS.2024.3371891

2023

A, Sharma, X. Li, H. Guan, G. Sun, L. Zhang, L. Wang, K. Wu, L. Cao, E. Zhu, A. Sim, T. Wu, J. Zou, "Automatic Data Transformation Using Large Language Model – An Experimental Study on Building Energy Data", IEEE International Conference on Big Data (BigData), 2023,

C. M. Oguchi, D. Ghosal, A. Sim, K. Wu, "Counterfactual Analysis: A Case Study on Impact of External Events on Building Energy Consumption", International Workshop on Big Data Analytics for Sustainability (BDA4S), 2023,

J. Bellavita, C. Sim, K. Wu, A. Sim, S. Yoo, H. Ito, V. Garonne, E. Lancon, "Understanding Data Access Patterns for dCache System", 26th International Conference on Computing in High Energy & Nuclear Physics (CHEP2023), 2023,

C. Sim, K. Wu, A. Sim, I. Monga, C. Guok, D. Hazen, F. Würthwein, D. Davila, H. Newman, J. Balcas, "Predicting Resource Utilization Trends with Southern California Petabyte Scale Cache", 26th International Conference on Computing in High Energy & Nuclear Physics (CHEP2023), 2023,

J. W. Chung, A. Sim, B. Quiter, Y. Wu, W. Zhao, K. Wu, "Preparing Spectral Data for Machine Learning: A Study of Geological Classification from Aerial Surveys", Machine Learning and the Physical Sciences Workshop (ML4PS), 2023,

R. Monga, A. Sim (advisor), K. Wu (advisor), "Comparative Study of the Cache Utilization Trends for Regional Scientific Data Caches", ACM/IEEE The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC’23), ACM Student Research Competition (SRC), First place winner, 2023,

H-C. Yang, L. Jin, A. Lazar, A. Todd-Blick, A. Sim, K. Wu, Q. Chen, C. A. Spurlock, "Gender Gaps in Mode Usage, Vehicle Ownership, and Spatial Mobility When Entering Parenthood: A Life Course Perspective", Systems, 2023, 11(6):314, doi: 10.3390/systems11060314

R. Shao, A. Sim, K. Wu, J. Kim, "Leveraging History to Predict Abnormal Transfers in Distributed Workflows", Sensors, 2023, 23(12):5485, doi: 10.3390/s23125485

Z. Deng, A. Sim, K. Wu, C. Guok, I. Monga, F. Andrijauskas, F. Wuerthwein, D. Weitzel, "Analyzing Transatlantic Network Traffic Patterns with Scientific Data Caches", 6th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA 2023), 2023, doi: 10.1145/3589012.3594897

C. Sim, K. Wu, A. Sim, I. Monga, C. Guok, F. Wurthwein, D. Davila, H. Newman, J. Balcas, Predicting Resource Usage Trends with Southern California Petabyte Scale Cache, 26th International Conference on Computing in High Energy & Nuclear Physics (CHEP 2023), 2023,

J. Bellavita, C. Sim, K. Wu, A. Sim, S. Yoo, H. Ito, V. Garonne, E. Lancon, Understanding Data Access Patterns for dCache System, 26th International Conference on Computing in High Energy & Nuclear Physics (CHEP 2023), 2023,

S. Kim, A. Sim, K. Wu, S. Byna, Y. Son, H. Eom, "Design and Implementation of I/O Performance Prediction Scheme on HPC Systems through Large-scale Log Analysis", Journal of Big Data, 2023, 10(65), doi: 10.1186/s40537-023-00741-4

C. Sim, K. Wu, A. Sim, I. Monga, C. Guok, F. Wurthwein, D. Davila, H. Newman, J. Balcas, "Effectiveness and predictability of in-network storage cache for Scientific Workflows", International Conference on Computing, Networking and Communication (ICNC 2023), 2023, doi: 10.1109/ICNC57223.2023.10074058

J. Wang, K. Wu, A. Sim, S. Hwangbo, "Locating Partial Discharges in Power Transformers with Convolutional Iterative Filtering", Sensors, 2023, 23, doi: 10.3390/s23041789

H-C. Yang, L. Jin, A. Lazar, A. Todd-Blick, A. Sim, K. Wu, Q. Chen, C. A. Spurlock, Gender Gaps in Mode Usage, Vehicle Ownership, and Spatial Mobility When Entering Parenthood: A Life Course Perspective, Transportation Research Board 102nd Annual Meeting,, 2023,

2022

Julian Bellavita, Alex Sim (advisor), John Wu (advisor), "Predicting Scientific Dataset Popularity Using dCache Logs", ACM/IEEE The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC’22), ACM Student Research Competition (SRC), Second place winner, 2022,

Poster (PDF)

The dCache installation is a storage management system that acts as a disk cache for high-energy physics (HEP) data. Storagespace on dCache is limited relative to persistent storage devices, therefore, a heuristic is needed to determine what data should be kept in the cache. A good cache policy would keep frequently accessed data in the cache, but this requires knowledge of future dataset popularity. We present methods for forecasting the number of times a dataset stored on dCache will be accessed in the future. We present a deep neural network that can predict future dataset accesses accurately, reporting a final normalized loss of 4.6e-8. We present a set of algorithms that can forecast future dataset accesses given an access sequence. Included are two novel algorithms, Backup Predictor and Last N Successors, that outperform other file prediction algorithms. Findings suggest that it is possible to anticipate dataset popularity in advance.

C. Sim, C. Guok (advisor), A. Sim (advisor), K. Wu (advisor), "Data Throughput Performance Trends of Regional Scientific Data Cache", ACM/IEEE The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC’22), ACM Student Research Competition (SRC), 2022,

Sunggon Kim, Alex Sim, Kesheng Wu, Suren Byna, Yongseok Son, "Design and implementation of dynamic I/O control scheme for large scale distributed file systems", Cluster Computing, 2022, 25(6):1--16, doi: 10.1007/s10586-022-03640-0

L. Jin, A. Lazar, C. Brown, V. Garikapati, B. Sun, S. Ravulaparthy, Q. Chen, A. Sim, K. Wu, T. Wenzel, T. Ho, C. A. Spurlock, "What Makes You Hold onto That Old Car? Joint Insights from Machine Learning and Multinomial Logit on Vehicle-level Transaction Decisions", Frontiers in Future Transportation, Connected Mobility and Automation, 2022, 3:894654, doi: 10.3389/ffutr.2022.894654

R. Shao, J. Kim A. Sim, K. Wu, "Predicting Slow Connections in Scientific Computing", 5th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA) 2022, in conjunction with The 31st ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2022, doi: 10.1145/3526064.3534112

J. Bellavita, A. Sim, K. Wu, I. Monga, C. Guok, F. Würthwein, D. Davila, "Studying Scientific Data Lifecycle in On-demand Distributed Storage Caches", 5th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA) 2022, in conjunction with The 31st ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2022, doi: 10.1145/3526064.3534111

R. Han, A. Sim, K. Wu, I. Monga, C. Guok, F. Würthwein, D. Davila, J. Balcas, H. Newman, "Access Trends of In-network Cache for Scientific Data", 5th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA), in conjunction with The 31st ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2022, doi: 10.1145/3526064.3534110

Yujing Ma, Florin Rusu, Kesheng Wu, Alexander Sim, 2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Pages: 1088--1097 2022, doi: 10.1109/IPDPSW55747.2022.00177

K. Wang, S. Lee, J. Balewski, A. Sim, P. Nugent, A. Agrawal, A. Choudhary, K. Wu, W-K. Liao, "Using Multi-resolution Data to Accelerate Neural Network Training in Scientific Applications", 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGrid 2022), 2022, doi: 10.1109/CCGrid54584.2022.00050

B. Weinger, J. Kim, A. Sim, M. Nakashima, N. Moustafa, K. Wu, "Enhancing IoT Anomaly Detection Performance for Federated Learning", Digital Communications and Networks, Special Issue on Edge Computation and Intelligence, 2022, doi: 10.1016/j.dcan.2022.02.007

Lipeng Wan, Axel Huebl, Junmin Gu, Franz Poeschel, Ana Gainaru, Ruonan Wang, Jieyang Chen, Xin Liang, Dmitry Ganyushin, Todd Munson, Ian Foster, Jean-Luc Vay, Norbert Podhorszki, Kesheng Wu, Scott Klasky, "Improving I/O Performance for Exascale Applications Through Online Data Layout Reorganization", IEEE Transactions on Parallel and Distributed Systems, 2022, 33:878-890, doi: 10.1109/TPDS.2021.3100784

John Wu, Bin Dong, Alex Sim, Automating Data Management Through Unified Runtime Systems, DOE ASCR Workshop on the Management and Storage of Scientific Data, 2022, doi: 10.2172/1843500

John Wu, Ben Brown, Paolo Calafiura, Quincey Koziol, Dongeun Lee, Alex Sim, Devesh Tiwari, Support for In-Flight Data Analyses in Scientific Workflows, DOE ASCR Workshop on the Management and Storage of Scientific Data, 2022, doi: 10.2172/1843500

A. Pereira, A. Sim, K. Wu, S. Yoo, H. Ito, "Data access pattern analysis for dCache storage system", International Conference on High Performance Computing in Asia-Pacific Region (HPC Asia 2022), 2022,

Ling Jin, Alina Lazar, Caitlin Brown, Bingrong Sun, Venu Garikapati, Srinath Ravulaparthy, Qianmiao Chen, Alexander Sim, Kesheng Wu, Tin Ho, Thomas Wenzel, C. Anna Spurlock, What Makes You Hold on to That Old Car? Joint Insights from Machine Learning and Multinomial Logit on Vehicle-level Transaction Decisions, Transportation Research Board 101st Annual Meeting, 2022,

E. Wes Bethel, Burlen Loring, Utkarsh Ayachit, P. N. Duque, Nicola Ferrier, Joseph Insley, Junmin Gu, Kress, Patrick O’Leary, Dave Pugmire, Silvio Rizzi, Thompson, Will Usher, Gunther H. Weber, Brad Whitlock, Wolf, Kesheng Wu, "Proximity Portability and In Transit, M-to-N Data Partitioning and Movement in SENSEI", In Situ Visualization for Computational Science, ( 2022) doi: 10.1007/978-3-030-81627-8_20

E. Wes Bethel, Burlen Loring, Utkarsh Ayatchit, David Camp, P. N. Duque, Nicola Ferrier, Joseph Insley, Junmin Gu, Kress, Patrick O’Leary, David Pugmire, Silvio Rizzi, Thompson, Gunther H. Weber, Brad Whitlock, Matthew Wolf, Kesheng Wu, "The SENSEI Generic In Situ Interface: Tool and Processing Portability at Scale", In Situ Visualization for Computational Science, ( 2022) doi: 10.1007/978-3-030-81627-8_13

2021

J. Bang, C. Kim, K. Wu, A. Sim, S. Byna, H. Sung, H. Eom, "An In-Depth I/O Pattern Analysis in HPC Systems", IEEE International Conference on High Performance Computing, Data & Analytics (HiPC2021), 2021, doi: 10.1109/HiPC53243.2021.00056

S. Lee, Q. Kang, K. Wang, J. Balewski, A. Sim, A. Agrawal, A. Choudhary, P. Nugent, K. Wu, W-K. Liao, "Asynchronous I/O Strategy for Large-Scale Deep Learning Applications", IEEE International Conference on High Performance Computing, Data & Analytics (HiPC2021), 2021, doi: 10.1109/HiPC53243.2021.00046

A. Lazar, L. Jin, C. Brown, C. A. Spurlock, A. Sim, K. Wu, "Performance of the Gold Standard and Machine Learning in Predicting Vehicle Transactions", the 3rd International Workshop on Big Data Tools, Methods, and Use Cases for Innovative Scientific Discovery (BTSD 2021), 2021, doi: 10.1109/BigData52589.2021.9671286

J. Cheung, A. Sim, J. Kim, K. Wu, "Performance Prediction of Large Data Transfers", ACM/IEEE The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC21), ACM Student Research Competition (SRC), 2021,

B Mohammed, M Kiran; N Krishnaswamy; Keshang, Wu, "Predicting WAN Traffic Volumes using Fourier and Multivariate SARIMA Approach", International Journal of Big Data Intelligence, November 3, 2021, doi: 10.1504/IJBDI.2021.118742

A. Syal, A. Lazar, J. Kim, A. Sim, K. Wu, "Network traffic performance analysis from passive measurements using gradient boosting machine learning", International Journal of Big Data Intelligence, 2021, 8:13-30, doi: 10.1504/IJBDI.2021.118741

Y. Ma, F. Rusu, K. Wu, A. Sim, Adaptive Elastic Training for Sparse Deep Learning on Heterogeneous Multi-GPU Servers, arXiv preprint arXiv:2110.07029, 2021,

E. Copps, A. Sim (Advisor), K. Wu (Advisor), "Analyzing scientific data sharing patterns with in-network data caching", ACM Richard Tapia Celebration of Diversity in Computing (TAPIA 2021), ACM Student Research Competition (SRC), 2021,

E. Copps, H. Zhang, A. Sim, K. Wu, I. Monga, C. Guok, F. Würthwein, D. Davila, E. Fajardo, "Analyzing scientific data sharing patterns with in-network data caching", 4th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA 2021), 2021, doi: 10.1145/3452411.3464441

Y. Wang, K. Wu, A. Sim, S. Yoo, S. Misawa, "Access Patterns of Disk Cache for Large Scientific Archive", 4th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA 2021), 2021, doi: 10.1145/3452411.3464444

A. Lazar, A. Sim, K. Wu, "GPU-based Classification for Wireless Intrusion Detection", 4th ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA 2021), 2021, doi: 10.1145/3452411.3464445

Y. Ma, F. Ruso, A. Sim, K. Wu, "Adaptive Stochastic Gradient Descent for Deep Learning on Heterogeneous CPU+GPU Architectures", Heterogeneity in Computing Workshop (HCW 2021), in conjunction with the 35th IEEE International Parallel & Distributed Processing Symposium (IPDPS), 2021, doi: 10.1109/IPDPSW52791.2021.00012

J. Kim, A. Sim, J. Kim, K, Wu, J. Hahm, Improving Botnet Detection with Recurrent Neural Network and Transfer Learning, arXiv preprint arXiv:2104.12602, 2021,

Donghun Koo, Jaehwan Lee, Jialin Liu, Eun-Kyu Byun, Jae-Hyuck Kwak, Glenn K Lockwood, Soonwook Hwang, Katie Antypas, Kesheng Wu, Hyeonsang Eom, "An empirical study of I/O separation for burst buffers in HPC systems", Journal of Parallel and Distributed Computing, 2021, 148:96-108, doi: 10.1016/j.jpdc.2020.10.007

2020

Ling Jin, Alina Lazar, James Sears, Annika Todd, Alex Sim, Kesheng Wu, Hung-Chai Yang, C. Anna Spurlock, "Clustering Life Course to Understand the Heterogeneous Effects of Life Events, Gender and Generation on Habitual Travel Modes", IEEE Access, 2020, 1-17, doi: 10.1109/ACCESS.2020.3032328

B. Weinger, J. Kim, A. Sim, M. Nakashima, N. Moustafa, K. Wu, "Enhancing IoT Anomaly Detection Performance for Federated Learning", The 16th IEEE International Conference on Mobility, Sensing and Networking (IEEE MSN 2020), 2020, doi: 10.1109/MSN50589.2020.00045

B. Cho, T. Dayrit, Y. Gao, Z. Wang, T. Hong, A. Sim, K. Wu, "Effective Missing Value Imputation Methods for Building Monitoring Data", The 2nd International Workshop on Big Data Tools, Methods, and Use Cases for Innovative Scientific Discovery (BTSD 2020) in conjunction with IEEE International Conference on Big Data (IEEE BigData 2020), 2020, doi: 10.1109/BigData50022.2020.9378230

Veronica Rodr\iguez Tribaldos, Nathaniel J Lindsey, Shan Dou, Craig Ulrich, Michelle Robertson, Bin Dong, Vincent Dumont, Kesheng Wu, Inder Monga, Chris Tracy, others, Combining Ambient Noise and Distributed Acoustic Sensing (DAS) Deployed on Dark Fiber Networks for High-resolution Imaging at the Basin Scale, AGU Fall Meeting 2020, 2020,

V. Dumont, V. Rodriguez Tribaldos, J. Ajo-Franklin, K. Wu, "Deep Learning for Surface Wave Identification in Distributed Acoustic Sensing Data", IEEE BigData 2020, December 8, 2020,

J. Kim, A. Sim, J. Kim, K. Wu, "Botnets Detection Using Recurrent Variational Autoencoder", IEEE Global Communications Conference (Globecom 2020), 2020, doi: 10.1109/GLOBECOM42002.2020.9348169

William F.Godoy, Norbert Podhorszki, Ruonan Wang, Chuck Atkins, Greg Eisenhauer, Junmin Gu,Philip Davis,J ong Choi, Kai Germaschewski, Kevin Huck, Axel Huebl, Mark Kim, James Kress, Tahsin Kurc, Qing Liu, Jeremy Logan, Kshitij Mehta, George Ostrouchov, Manish Parashar, Franz Poeschel, David Pugmire, Eric Suchyta, KeichiTakahashi, NickThompson, Seiji Tsutsumi, Lipeng Wan, Matthew Wolf, Kesheng Wu, Scott Klasky, "ADIOS 2: The Adaptable Input Output System. A framework for high-performance data management", SoftwareX, 2020, 12,

Brett Weinger, Alex Sim (Advisor), John Wu (Advisor), Jinoh Kim (Advisor), "Enhancing IoT Anomaly Detection Performance for Federated Learning", International Conference for High Performance Computing, Networking, Storage and Analysis (SC’20), ACM Student Research Competition (SRC), 2020,

Jonathan Blair Ajo-Franklin, Ver\ onica Rodr\ \iguez Tribaldos, Avinash Nayak, Nathaniel J Lindsey, Feng Cheng, Benxin Chi, Bin Dong, Kesheng Wu, Inder Monga, Distributed Acoustic Sensing (DAS) at the Plot to Basin Scale: Connecting Near-Surface Sensing and Seismology with a Common Observational Tool, AGU Fall Meeting 2020, 2020,

V. Dumont, V. Rodriguez Tribaldos, J. Ajo-Franklin, K. Wu, "Deep Learning on Real Geophysical Data: A Case Study for Distributed Acoustic Sensing Research", NeurIPS "Machine Learning and the Physical Sciences" workshop, 2020,

C. A. Spurlock, A. Gopal, J. Auld, P. Leiby, C. Sheppard, T. Wenzel, S. Belal, A. Duvall, A. Enam, S. Fujita, A. Henao, L. Jin, E. Kontou, A. Lazar, Z. Needell, C. Rames, T. Rashidi, J. Sears, A. Sim, M. Stinson, M. Taylor, A. Todd-Blick, O. Verbas, V. Walker, J. Ward, G. Wong-Parodi, K. Wu, H.-C. Yang, "SMART Mobility, Mobility Decision Science Capstone Report", Vehicle Technologies Office (VTO), Office of Energy Efficiency and Renewable Energy (EERE), US Department of Energy, 2020,

Bin Dong, Ver\ onica Rodr\ \iguez Tribaldos, Xin Xing, Suren Byna, Jonathan Ajo-Franklin, Kesheng Wu, "DASSA: Parallel DAS Data Storage and Analysis for Subsurface Event Detection", 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS), July 14, 2020, 254--263,

Sunggon Kim, Alex Sim, Kesheng Wu, Suren Byna, Yongseok Son, Hyeonsang Eom, "Towards hpc i/o performance prediction through large-scale log analysis", Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing (HPDC 2020), 2020, 77--88, doi: 10.1145/3369583.3392678

Gaurav R Ghosal, Dipak Ghosal, Alex Sim, Aditya V Thakur, Kesheng Wu, "A Deep Deterministic Policy Gradient Based Network Scheduler For Deadline-Driven Data Transfers", Proceedings of International Federation for Information Processing (IFIP) Networking Conference (NETWORKING 2020), 2020, 253--261,

Jiwoo Bang, Chungyong Kim, Kesheng Wu, Alex Sim, Suren Byna, Sunggon Kim, Hyeonsang Eom, "HPC Workload Characterization Using Feature Selection and Clustering", ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA 2020), in conjunction with The 29th ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC 2020), 2020, 33--40, doi: 10.1145/3391812.3396270

Jeeyung Kim, Alex Sim, Jinoh Kim, Kesheng Wu, Jaegyoon Hahm, "Transfer Learning Approach for Botnet Detection Based on Recurrent Variational Autoencoder", ACM International Workshop on ​System and Network Telemetry and Analysis (SNTA 2020), in conjunction with The 29th ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC 2020), 2020, 41--47, doi: 10.1145/3391812.3396273

S. Bhandari, A. K. Kukreja, A. Lazar, A. Sim, K. Wu, "Feature Selection and Tree-based Classification for Wireless Intrusion Detection", the 3rd ACM International Workshop on System and Network Telemetry and Analysis (SNTA) 2020, in conjunction with The 29th ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2020, doi: 10.1145/3391812.3396274

Qiao Kang, Alex Sim, Peter Nugent, Sunwoo Lee, Wei-keng Liao, Ankit Agrawal, Alok Choudhary, Kesheng Wu, "Predicting Resource Requirement in Intermediate Palomar Transient Factory Workflow", 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID 2020), 2020, 619--628, doi: 10.1109/CCGrid49817.2020.00-31

L. Jin, A. Lazar, J. Sears, A. Todd, A. Sim, K. Wu, C. A. Spurlock, "Life Course as a Contextual System to Investigate the Effects of Life Events, Gender, and Generation on Travel Mode Use", Transportation Research Board (TRB) 99th Annual Meeting, 2020,

Jeeyung Kim, Alex Sim, Jinoh Kim, Kesheng Wu, Botnet Detection Using Recurrent Variational Autoencoder, arXiv preprint arXiv:2004.00234, 2020,

2019

A. Lazar, A. Ballow, L. Jin, C. A. Spurlock, A. Sim, K. Wu, "Machine Learning for Prediction of Mid to LongTerm Habitual Transportation Mode Use", International Workshop on Big Data Tools, Methods, and Use Cases for Innovative Scientific Discovery (BTSD), in conjunction with the IEEE International Conference on Big Data (Big Data), 2019, doi: 10.1109/BigData47090.2019.9006411

Junmin Gu, Burlen Loring, Kesheng Wu, E. Wes Bethel, "HDF5 as a vehicle for in transit data movement", The Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV'19), 2019, doi: 10.1145/3364228.3364237

P. Linton, W. Melodia, A. Lazar, D. Agarwal, L. Bianchi, D. Ghoshal, K. Wu, G. Pastorello, L. Ramakrishnan, "Identifying Time Series Similarity in Large-Scale Earth System Datasets", The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC19), 2019,

L. Jin, A. Lazar, J. Sears, A. Todd, A. Sim, K. Wu, C. A. Spurlock, Life course as a contextual system to investigate the effects of life events, gender and generation on travel mode usage, The Behavior, Energy & Climate Change Conference (BECC), 2019,

Alexandra Ballow, Alina Lazar (Advisor), Alex Sim (Advisor), Kesheng Wu (Advisor), "Handling Missing Values in Joint Sequence Analysis", ACM Richard Tapia Celebration of Diversity in Computing (TAPIA 2019), ACM Student Research Competition (SRC), First place winner, Pages: 19 2019,

Antoine Bambade, Kesheng Wu, "An Assessment of the Prediction Quality of VPIN", Advanced Analytics and Artificial Intelligence Applications, (IntechOpen: 2019)

S. Kim, A. Sim, K. Wu, S. Byna, T. Wang, Y. Son, H. Eom, "DCA-IO: A Dynamic I/O Control Scheme for Parallel and Distributed File System", 19th Annual IEEE/ACM International Symposium in Cluster, Cloud, and Grid Computing (CCGrid 2019), 2019, doi: 10.1109/CCGRID.2019.00049

Alexandra Ballow, Alina Lazar, Alex Sim, Kesheng Wu, "Joint Sequence Analysis Challenges: How to Handle Missing Values and Mixed Variable Types", SIAM Conference on Computational Science and Engineering (CSE19), 2019,

Tyler Leibengood, Alina Lazar, Alex Sim, Kesheng Wu, "Network Traffic Performance Prediction with Multivariate Clusters in Time Windows", SIAM Conference on Computational Science and Engineering (CSE19), 2019,

Payton A Linton, William M Melodia, Alina Lazar, Deborah Agarwal, Ludovico Bianchi, Devarshi Ghoshal, Kesheng Wu, Gilberto Pastorello, Lavanya Ramakrishnan, "Identifying Time Series Similarity in Large-Scale Earth System Datasets", 2019,

Payton Linton, William Melodia, Alina Lazar, Deborah Agarwal, Ludovico Bianchi, Devarshi Ghoshal, Gilberto Pastorello, Lavanya Ramakrishnan, Kesheng Wu, Understanding Data Similarity in Large-Scale Scientific Datasets, 2019 IEEE International Conference on Big Data (Big Data), Pages: 4525--4531 2019,

Beytullah Yildiz, Kesheng Wu, Suren Byna, Arie Shoshani, "Parallel membership queries on very large scientific data sets using bitmap indexes", Concurrency and Computation: Practice and Experience, January 1, 2019, 31:e5157,

Many scientific applications produce very large amounts of data as advances in hardware fuel computing and experimental facilities. Managing and analyzing massive quantities of scientific data is challenging as data are often stored in specific formatted files, such as HDF5 and NetCDF, which do not offer appropriate search capabilities. In this research, we investigated a special class of search capability, called membership query, to identify whether queried elements of a set are members of an attribute. Attributes that naturally have classification values appear frequently in scientific domains such as category and object type as well as in daily life such as zip code and occupation. Because classification attribute values are discrete and require random data access, performing a membership query on a large scientific data set creates challenges. We applied bitmap indexing and parallelization to membership queries to overcome these challenges. Bitmap indexing provides high performance not only for low cardinality attributes but also for high cardinality attributes, such as floating‐point variables, electric charge, or momentum in a particle physics data set, due to compression algorithms such as Word‐Aligned Hybrid. We conducted experiments, in a highly parallelized environment, on data obtained from a particle accelerator model and a synthetic data set.

Bin Dong, Kesheng Wu, Suren Byna, Houjun Tang, "SLOPE: Structural Locality-Aware Programming Model for Composing Array Data Analysis", International Conference on High Performance Computing, January 1, 2019, 61--80,

Bin Dong, Patrick Kilian, Xiaocan Li, Fan Guo, Suren Byna, Kesheng Wu, "Terabyte-scale Particle Data Analysis: An ArrayUDF Case Study", Proceedings of the 31st International Conference on Scientific and Statistical Database Management, January 1, 2019, 202--205,

Junmin Gu, Burlen Loring, Kesheng Wu, E Wes Bethel, "HDF5 as a vehicle for in transit data movement", Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization, 2019, 39--43,

Olivia Del Guercio, Rafael Orozco, Alex Sim, Kesheng Wu, "Multidimensional Compression with Pattern Matching", 2019 Data Compression Conference (DCC), Pages: 567--567 2019,

Alina Lazar, Ling Jin, C Anna Spurlock, Kesheng Wu, Alex Sim, Annika Todd, "Evaluating the effects of missing values and mixed data types on social sequence clustering using t-SNE visualization", Journal of Data and Information Quality (JDIQ), 2019, 11:1--22,

Sambit Shukla, Dipak Ghosal, Kesheng Wu, Alex Sim, Matthew Farrens, "Co-optimizing Latency and Energy for IoT services using HMP servers in Fog Clusters", 2019 Fourth International Conference on Fog and Mobile Edge Computing (FMEC), 2019, 121--128,

Hanul Sung, Jiwoo Bang, Alexander Sim, Kesheng Wu, Hyeonsang Eom, "Understanding Parallel I/O Performance Trends Under Various HPC Configurations", Proceedings of the ACM Workshop on Systems and Network Telemetry and Analytics, 2019, 29--36,

Mengtian Jin, Youkow Homma, Alex Sim, Wilko Kroeger, Kesheng Wu, "Performance prediction for data transfers in LCLS workflow", Proceedings of the ACM Workshop on Systems and Network Telemetry and Analytics, 2019, 37--44,

Olivia Del Guercio, Rafael Orozco, Alex Sim, Kesheng Wu, "Similarity-based Compression with Multidimensional Pattern Matching", Proceedings of the ACM Workshop on Systems and Network Telemetry and Analytics, 2019, 19--24,

Astha Syal, Alina Lazar, Jinoh Kim, Alex Sim, Kesheng Wu, "Automatic detection of network traffic anomalies and changes", Proceedings of the ACM Workshop on Systems and Network Telemetry and Analytics, 2019, 3--10,

Dipak Ghosal, Sambit Shukla, Alex Sim, Aditya V Thakur, Kesheng Wu, "A Reinforcement Learning Based Network Scheduler For Deadline-Driven Data Transfers", 2019 IEEE Global Communications Conference (GLOBECOM), 2019, 1--6,

Qiao Kang, Ankit Agrawal, Alok Choudhary, Alex Sim, Kesheng Wu, Rajkumar Kettimuthu, Peter H Beckman, Zhengchun Liu, Wei-keng Liao, "Spatiotemporal Real-Time Anomaly Detection for Supercomputing Systems", 2019 IEEE International Conference on Big Data (Big Data), 2019, 4381--4389,

Burak Cetin, Alina Lazar, Jinoh Kim, Alex Sim, Kesheng Wu, "Federated Wireless Network Intrusion Detection", 2019 IEEE International Conference on Big Data (Big Data), Pages: 6004--6006 2019,

Kesheng Wu, Alex Sim, Jonathan Wang, Seongwook Hwangbo, Methods, systems, and devices for accurate signal timing of power component events, 2019,

US Patent app no. 20190138371, “Methods, systems, and devices for accurate signal timing of power component events”

Jongbeen Han, Heemin Kim, Hyeonsang Eom, Jonathan Coignard, Kesheng Wu, Yongseok Son, "Enabling SQL-Query Processing for Ethereum-based Blockchain Systems", Proceedings of the 9th International Conference on Web Intelligence, Mining and Semantics, 2019, 1--7,

Jung Heon Song, Marcos L\ opez de Prado, Horst D Simon, Kesheng Wu, Extracting Signals from High-Frequency Trading with Digital Signal Processing Tools, The Journal of Financial Data Science, Pages: 124--138 2019,

Devarshi Ghoshal, Kesheng Wu, Eric Pouyoul, Erich Strohmaier, "Analysis and Prediction of Data Transfer Throughput for Data-Intensive Workloads", 2019 IEEE International Conference on Big Data (Big Data), 2019, 3648--3657,

2018

Karen Tu, Alex Sim (Advisor), John Wu (Advisor), "Identification of Network Data Transfer Bottlenecks in HPC Systems", International Conference for High Performance Computing, Networking, Storage and Analysis (SC’18), ACM Student Research Competition (SRC), 2018,

Hongyuan Zhan, Gabriel Gomes, Xiaoye S Li, Kamesh Madduri, Kesheng Wu, "Efficient Online Hyperparameter Optimization for Kernel Ridge Regression with Applications to Traffic Time Series Prediction", arXiv preprint arXiv:1811.00620, 2018,

Weijie Zhao, Florin Rusu, Kesheng Wu, Peter Nugent, "Automatic identification and classification of Palomar Transient Factory astrophysical objects in GLADE", International Journal of Computational Science and Engineering, 2018, 16:337--349,

Weijie Zhao, Florin Rusu, Bin Dong, Kesheng Wu, Anna YQ Ho, Peter Nugent, "Distributed Caching for Complex Querying of Raw Arrays", SSDBM, 2018,

Haoyuan Xing, Sofoklis Floratos, Spyros Blanas, Suren Byna, Prabhat, Kesheng Wu, and Paul Brown,, "ArrayBridge: Interweaving declarative array processing with imperative high-performance computing", 34th IEEE International Conference on Data Engineering (ICDE) 2018, April 17, 2018,

Haoyuan Xing, Sofoklis Floratos, Spyros Blanas, Suren Byna, M Prabhat, Kesheng Wu, Paul Brown, "ArrayBridge: Interweaving declarative array processing in SciDB with imperative HDF5-based programs", 2018 IEEE 34th International Conference on Data Engineering (ICDE), 2018, 977--988,

Bin Dong, Teng Wang, Houjun Tang, Quincey Koziol, Kesheng Wu, Suren Byna, "ARCHIE: Data analysis acceleration with array caching in hierarchical storage", 2018 IEEE International Conference on Big Data (Big Data), January 1, 2018, 211--220,

Kesheng Wu, Surendra Byna, Bin Dong, others, VPIC IO utilities, 2018,

Kesheng Wu, Bin Dong, Surendra Byna, "Scientific Data Services Framework for Plasma Physics", APS, 2018, 2018:BM10--006,

Junmin Gu, Scott Klasky, Norbert Podhorszki, Ji Qiang, Kesheng Wu, "Querying large scientific data sets with adaptable IO system ADIOS", Asian Conference on Supercomputing Frontiers, 2018, 51--69,

Taehoon Kim, Jaesik Choi, Dongeun Lee, Alex Sim, C Anna Spurlock, Annika Todd, Kesheng Wu, "Predicting baseline for analysis of electricity pricing", International Journal of Big Data Intelligence, 2018, 5:3--20,

Hongyuan Zhan, Gabriel Gomes, Xiaoye S Li, Kamesh Madduri, Alex Sim, Kesheng Wu, "Consensus ensemble system for traffic flow prediction", IEEE Transactions on Intelligent Transportation Systems, 2018, 19:3903--3914,

Cecilia Dao, Xinyu Liu, Alex Sim, Craig Tull, Kesheng Wu, "Modeling data transfers: change point and anomaly detection", 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), 2018, 1589--1594,

Rajkumar Kettimuthu, Zhengchun Liu, Ian Foster, Peter H Beckman, Alex Sim, Kesheng Wu, Wei-keng Liao, Qiao Kang, Ankit Agrawal, Alok Choudhary, "Towards autonomic science infrastructure: architecture, limitations, and open issues", Proceedings of the 1st International Workshop on Autonomous Infrastructure for Science, 2018, 1--9,

Mengying Yang, Xinyu Liu, Wilko Kroeger, Alex Sim, Kesheng Wu, "Identifying anomalous file transfer events in LCLS workflow", Proceedings of the 1st International Workshop on Autonomous Infrastructure for Science, 2018, 1--4,

Sowmya Balasubramanian, Dipak Ghosal, Kamala Narayanan Balasubramanian Sharath, Eric Pouyoul, Alex Sim, Kesheng Wu, Brian Tierney, "Auto-tuned publisher in a pub/sub system: Design and performance evaluation", 2018 IEEE International Conference on Autonomic Computing (ICAC), 2018, 21--30,

Jonathan Wang, Kesheng Wu, Alex Sim, Seongwook Hwangbo, "Feature Engineering and Classification Models for Partial Discharge in Power Transformers", Mij, 2018, 1001:60,

Tal Shachaf, Alexander Sim, Kesheng Wu, Wilko Kroeger, "Detecting Anomalies in the LCLS Workflow", 2018 IEEE International Conference on Big Data (Big Data), 2018, 3256--3260,

Alina Lazar, Kesheng Wu, Alex Sim, "Predicting Network Traffic Using TCP Anomalies", 2018 IEEE International Conference on Big Data (Big Data), Pages: 5369--5371 2018,

Kesheng Wu, Horst D Simon, "High-Performance Computational Intelligence and Forecasting Technologies", 2018,

Weijie Zhao, Florin Rusu, Bin Dong, Kesheng Wu, Anna YQ Ho, Peter Nugent, "Distributed caching for processing raw arrays", Proceedings of the 30th International Conference on Scientific and Statistical Database Management, 2018, 1--12,

Xin Xing, Bin Dong, Jonathan Ajo-Franklin, Kesheng Wu, "Automated Parallel Data Processing Engine with Application to Large-Scale Feature Extraction", 2018 IEEE/ACM Machine Learning in HPC Environments (MLHPC), January 1, 2018, 37--46,

Hongyuan Zhan, Gabriel Gomes, Xiaoye S Li, Kamesh Madduri, Kesheng Wu, "Efficient online hyperparameter learning for traffic flow prediction", 2018 21st International Conference on Intelligent Transportation Systems (ITSC), 2018, 164--169,

2017

Shashanka Ubaru, Kesheng Wu, Kristofer E. Bouchard, "UoI-NMF Cluster: A Robust Nonnegative Matrix Factorization Algorithm for Improved Parts-Based Decomposition and Reconstruction of Noisy Data", the 16th IEEE International Conference on Machine Learning and Applications (ICMLA 2017), 2017, 241-248, doi: 10.1109/ICMLA.2017.0-152

Bin Dong, Kesheng Wu, Surendra Byna, Jialin Liu, Weijie Zhao, Florin Rusu, "ArrayUDF: User-defined scientific data analysis on arrays", Proceedings of the 26th International Symposium on High-Performance Parallel and Distributed Computing, January 1, 2017, 53--64,

Ling Jin, Doris Lee, Alex Sim, Sam Borgeson, Kesheng Wu, C Anna Spurlock, Annika Todd, "Comparison of clustering techniques for residential energy behavior using smart meter data", 2017,

Dongeun Lee, Alex Sim, Jaesik Choi, Kesheng Wu, "Expanding statistical similarity based data reduction to capture diverse patterns", 2017 Data Compression Conference (DCC), Pages: 445--445 2017,

Jonathan Wang, Wucherl Yoo, Alex Sim, Peter Nugent, Kesheng Wu, "Parallel variable selection for effective performance prediction", 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), 2017, 208--217,

Dongeun Lee, Alex Sim, Jaesik Choi, Kesheng Wu, "Improving statistical similarity based data reduction for non-stationary data", Proceedings of the 29th International Conference on Scientific and Statistical Database Management, 2017, 1--6,

Updated experiment version: https://sdm.lbl.gov/oapapers/ssdbm17-lee-upd.pdf
Original version: http://dl.acm.org/citation.cfm?doid=3085504.3085583

Kesheng Wu, Dongeun Lee, Alex Sim, Jaesik Choi, "Statistical data reduction for streaming data", 2017 New York Scientific Data Summit (NYSDS), 2017, 1--6,

Jonathan Wang, Kesheng Wu, Alex Sim, Seongwook Hwangbo, "Convolutional Filtering for Accurate Signal Timing from Noisy Streaming Data", 2017 IEEE 15th Intl Conf on Dependable, Autonomic and Secure Computing, 15th Intl Conf on Pervasive Intelligence and Computing, 3rd Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech, 2017, 941--948,

Jonathan Wang, Kesheng Wu, Alex Sim, Seongwook Hwangbo, "Feature Engineering and Classification Models for Partial Discharge Events in Power Transformers", Proceedings of the Fourth IEEE/ACM International Conference on Big Data Computing, Applications and Technologies, Pages: 269--270 2017,

Alina Lazar, Ling Jin, C Anna Spurlock, Kesheng Wu, Alex Sim, "Data quality challenges with missing values and mixed types in joint sequence analysis", 2017 IEEE International Conference on Big Data (Big Data), 2017, 2620--2627,

Peter Harrington, Wucherl Yoo, Alexander Sim, Kesheng Wu, "Diagnosing parallel I/O bottlenecks in HPC applications", International Conference for High Performance Computing Networking Storage and Analysis (SCI7) ACM Student Research Competition (SRC), 2017,

Jonathan Wang, Kesheng Wu, Alex Sim, Seongwook Hwangbo, "Accurate signal timing from high frequency streaming data", 2017 IEEE International Conference on Big Data (Big Data), Pages: 4852--4854 2017,

Weijie Zhao, Florin Rusu, Bin Dong, Kesheng Wu, Peter Nugent, "Incremental view maintenance over array data", Proceedings of the 2017 ACM International Conference on Management of Data, January 1, 2017, 139--154,

Tzuhsien Wu, Jerry Chou, Shyng Hao, Bin Dong, Scott Klasky, Kesheng Wu, "Optimizing the query performance of block index through data analysis and I/O modeling", Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, January 1, 2017, 1--10,

2016

M. Bryson, S. Byna (Advisor), A. Sim (Advisor), K. Wu (Advisor), "The Search for Missing Parallel IO Performance on the Cori Supercomputer", International Conference for High Performance Computing, Networking, Storage and Analysis (SC’16), ACM Student Research Competition (SRC), 2016,

M. Bae, W. Yoo (Advisor), A. Sim (Advisor), K. Wu (Advisor), "Discovering Energy Resource Usage Patterns on Scientific Clusters", International Conference for High Performance Computing, Networking, Storage and Analysis (SC’16), ACM Student Research Competition (SRC), Third place winner, 2016, 2016,

Jonathan Wang, Wucherl Yoo, Alex Sim, K John Wu, "Analysis of Variable Selection Methods on Scientific Cluster Measurement Data", 2016,

Bin Dong, Surendra Byna, Kesheng Wu, "SDS-Sort: Scalable Dynamic Skew-aware Parallel", HPDC 16, New York, NY, USA, ACM, 2016, 57--68, doi: 10.1145/2907294.2907300

Deborah A Agarwal, Boris Faybishenko, Vicky L Freedman, Harinarayan Krishnan, Gary Kushner, Carina Lansing, Ellen Porter, Alexandru Romosan, Arie Shoshani, Haruko Wainwright, others, "A science data gateway for environmental management", Concurrency and Computation: Practice and Experience, 2016, 28:1994--2004,

Houjun Tang, Suren Byna, Steve Harenberg, Xiaocheng Zou, Wenzhao Zhang, Kesheng Wu, Bin Dong, Oliver Rubel, Kristofer Bouchard, Scott Klasky, others, "Usage pattern-driven dynamic data layout reorganization", 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), January 1, 2016, 356--365,

Wenzhao Zhang, Houjun Tang, Steve Harenberg, Surendra Byna, Xiaocheng Zou, Dharshi Devendran, Daniel F Martin, Kesheng Wu, Bin Dong, Scott Klasky, others, "Amrzone: A runtime amr data sharing framework for scientific applications", 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), January 1, 2016, 116--125,

Bin Dong, Surendra Byna, Kesheng Wu, "Sds-sort: Scalable dynamic skew-aware parallel sorting", Proceedings of the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing, January 1, 2016, 57--68,

Xiaocheng Zou, David A Boyuka II, Dhara Desai, Daniel F Martin, Suren Byna, Kesheng Wu, "AMR-aware in situ indexing and scalable querying", Proceedings of the 24th High Performance Computing Symposium, January 1, 2016, 26,

Bin Dong, Suren Byna, Kesheng Wu, Hans Johansen, Jeffrey N Johnson, Noel Keen, others, "Data elevator: Low-contention data movement in hierarchical storage system", 2016 IEEE 23rd international conference on high performance computing (HiPC), January 1, 2016, 152--161,

Houjun Tang, Suren Byna, Steve Harenberg, Wenzhao Zhang, Xiaocheng Zou, Daniel F Martin, Bin Dong, Dharshi Devendran, Kesheng Wu, David Trebotich, others, "In situ storage layout optimization for amr spatio-temporal read accesses", 2016 45th International Conference on Parallel Processing (ICPP), January 1, 2016, 406--415,

Wenzhao Zhang, Houjun Tang, Stephen Ranshous, Surendra Byna, Daniel F Mart\ \in, Kesheng Wu, Bin Dong, Scott Klasky, Nagiza F Samatova, "Exploring memory hierarchy and network topology for runtime AMR data sharing across scientific applications", 2016 IEEE International Conference on Big Data (Big Data), January 1, 2016, 1359--1366,

David Pugmire, James Kress, Jong Choi, Scott Klasky, Tahsin Kurc, Randy Michael Churchill, Matthew Wolf, Greg Eisenhower, Hank Childs, Kesheng Wu, others, "Visualization and analysis for near-real-time decision making in distributed workflows", 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2016, 1007--1013,

Utkarsh Ayachit, Andrew Bauer, Earl PN Duque, Greg Eisenhauer, Nicola Ferrier, Junmin Gu, Kenneth E Jansen, Burlen Loring, Zarija Lukic, Suresh Menon, others, "Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures", SC 16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2016, 921--932, LBNL 1007264,

D. Pugmire, J. Kress, J. Choi, S. Klasky, Kurc, R. M. Churchill, M. Wolf, G., H. Childs, K. Wu, A. Sim, J. Gu, J. Low, "Visualization and Analysis for Near-Real-Time Decision in Distributed Workflows", 2016 IEEE International Parallel and Distributed Symposium Workshops (IPDPSW), 2016, 1007--1013, doi: 10.1109/IPDPSW.2016.175

Wucherl Yoo, Michelle Koo, Yi Cao, Alex Sim, Peter Nugent, Kesheng Wu, "Performance Analysis Tool for HPC and Big Data Applications on Scientific Clusters", Conquering Big Data with High Performance Computing, (Springer, Cham: 2016) Pages: 139--161

Dongeun Lee, Alex Sim, Jaesik Choi, Kesheng Wu, "Novel data reduction based on statistical similarity", Proceedings of the 28th International Conference on Scientific and Statistical Database Management, 2016, 1--12,

Wucherl Yoo, Alex Sim, Kesheng Wu, "Machine learning based job status prediction in scientific clusters", 2016 SAI Computing Conference (SAI), 2016, 44--53,

Lingfei Wu, Kesheng John Wu, Alex Sim, Michael Churchill, Jong Y Choi, Andreas Stathopoulos, Choong-Seock Chang, Scott Klasky, "Towards real-time detection and tracking of spatio-temporal features: Blob-filaments in fusion plasma", IEEE Transactions on Big Data, 2016, 2:262--275,

Tzuhsien Wu, Hao Shyng, Jerry Chou, Bin Dong, Kesheng Wu, "Indexing blocks to reduce space and time requirements for searching large data files", 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), January 1, 2016, 398--402,

Weijie Zhao, Florin Rusu, Bin Dong, Kesheng Wu, "Similarity Join over Array Data", SIGMOD, January 1, 2016, 2007--2022,

Kesheng Wu, Elizabeth N Coviello, SM Flanagan Martin Greenwald, Xia Lee, Alex Romosan, P Schissel, Arie Shoshani, Josh Stillerman John Wright, MPO: A System to Document and Analyze Distributed Workflows, International Provenance and Annotation Workshop, Pages: 166--170 2016, doi: 10.1007/978-3-319-40593-3_14

2015

Jinoh Kim, Bin Dong, Suren Byna, and Kesheng Wu, "Security for the Scientific Data Service Framework", 2nd International Workshop on Privacy and Security of Big Data (PSBD 2015), in conjunction with IEEE BigData 2015, 2015,

Xiaocheng (Chris) Zou, Suren Byna, Hans Johansen, Daniel Martin, Nagiza F. Samatova, Arie Shoshani, John Wu, "Six-fold Speedup of Ice Calving Detection Achieved by AMR-aware Parallel Connected Component Labeling", SciDAC PI Meeting, July 2015, 2015,

Xiaocheng Zou, Kesheng Wu, David A. Boyuka, Daniel F. Martin, Suren Byna, Houjun, Kushal Bansal, Terry J. Ligocki, Hans Johansen, and Nagiza F. Samatova, "Parallel In Situ Detection of Connected Components Adaptive Mesh Refinement Data", Proceedings of the Cluster, Cloud and Grid Computing (CCGrid) 2015, 2015,

S. Shannigrahi, A. J. Barczyk, C. Papadopoulos, A. Sim, I. Monga, H. Newman, K. Wu, E. Yeh, "Named Data Networking in Climate Research and HEP Applications", 21st International Conference on Computing in High Energy and Nuclear Physics (CHEP2015), 2015,

Bin Dong, Surendra Byna, Kesheng Wu, "Heavy-tailed distribution of parallel I/O system response time", Proceedings of the 10th Parallel Data Storage Workshop, 2015, 37--42,

Bin Dong, Surendra Byna, Kesheng Wu, "Spatially clustered join on heterogeneous scientific data sets", 2015 IEEE International Conference on Big Data (Big Data), 2015, 371--380,

David H Bailey, Stephanie Ger, Marcos L\ opez de Prado, Alexander Sim, "Statistical overfitting and backtest performance", Risk-Based and Factor Investing, 2015,

http://ssrn.com/abstract=2507040

Wucherl Yoo, Michelle Koo, Yi Cao, Alex Sim, Peter Nugent, Kesheng Wu, "Patha: Performance analysis tool for hpc applications", 2015 IEEE 34th International Performance Computing and Communications Conference (IPCCC), 2015, 1--8,

Taehoon Kim, Dongeun Lee, Jaesik Choi, Anna Spurlock, Alex Sim, Annika Todd, Kesheng Wu, "Extracting baseline electricity usage using gradient tree boosting", 2015 IEEE International Conference on Smart City/SocialCom/SustainCom (SmartCity), 2015, 734--741,

L. Wu, K. Wu, A. Sim, M. Churchill, J. Y. Choi, A. Stathopoulos, C.S. Chang, S. Klasky, "Towards Real-Time Detection and Tracking of Blob-Filaments in Fusion Plasma Big Data", WM-CS-2015-01, Department of Computer Science, College of William and Mary, 2015, doi: 10.48550/arXiv.1505.03532

W. Yoo, M. Koo, Y. Cao, A. Sim, P. Nugent, K. Wu, PATHA: Performance Analysis Tool for HPC, 2015 IEEE 34th International Performance Computing and Conference (IPCCC), Pages: 1--8 2015, doi: 10.1109/PCCC.2015.7410313

Taehoon Kim, Dongeun Lee, Jaesik Choi, C. Anna Spurlock, Alex Sim, Annika Todd, Kesheng Wu, "Extracting Baseline Electricity Usage with Gradient Boosting", International Conference on Big Intelligence and Computing (DataCom 2015), 2015, doi: 10.1109/SmartCity.2015.156

Jung Heon Song, Marcos Lopez de Prado, Horst D, Kesheng Wu, Understanding Natural Gas Futures Trading Through Data, Available at SSRN 2657224, 2015,

Gili Rosenberg, Poya Haghnegahdar, Phil Goddard Peter Carr, Kesheng Wu, Marcos L\ opez de, Solving the optimal trading trajectory problem using a annealer, Proceedings of the 8th Workshop on High Performance Finance, Pages: 7 2015,

2014

John Wu, Alex Sim, Lingfei Wu, Abraham Frankl, Scott Klasky, Jong Y Choi, CS Chang, Michael Churchill, "Exercising ICEE Framework with Fusion Blob Detection", DOE/ASCR NGNS PI meeting, 2014,

Spyros Blanas, Kesheng Wu, Surendra Byna, Bin Dong, Arie Shoshani, "Parallel Data Analysis Directly on Scientific File", SIGMOD 14, 2014, 385--396, doi: 10.1145/2588555.2612185

Spyros Blanas, Kesheng Wu, Surendra Byna, Bin Dong, Arie Shoshani, "Parallel Data Analysis Directly on Scientific File Formats", SIGMOD 14, 2014, 385--396, doi: 10.1145/2588555.2612185

Qian Sun, Fan Zhang, Tong Jin, Hoang Bui, Kesheng Wu, Arie Shoshani, Hemanth Kolla, Scott Klasky, Jacqueline Chen, Manish Parashar, "Scalable run-time data indexing and querying for scientific simulations", Big Data Analytics: Challenges and Opportunities (BDAC-14) Workshop at Supercomputing Conference, 2014,

Spyros Blanas, Kesheng Wu, Surendra Byna, Bin Dong, Arie Shoshani, "Parallel data analysis directly on scientific file formats", Proceedings of the 2014 ACM SIGMOD international conference on Management of data, January 1, 2014, 385--396,

Hsuan-Te Chiu, Jerry Chou, Venkat Vishwanath, Surendra Byna, Kesheng Wu, "Simplifying index file structure to improve I/O performance of parallel indexing", 2014 20th IEEE International Conference on Parallel and Distributed Systems (ICPADS), 2014, 576--583,

Bin Dong, Surendra Byna, Kesheng Wu, "Parallel query evaluation as a Scientific Data Service", 2014 IEEE International Conference on Cluster Computing (CLUSTER), January 1, 2014, 194--202,

Jialin Liu, S. Byna, Bin Dong, Kesheng Wu, Chen, "Model-Driven Data Layout Selection for Improving Read", Parallel Distributed Processing Symposium Workshops 2014 IEEE International, 2014, 1708--1716, doi: 10.1109/IPDPSW.2014.190

Bin Dong, S. Byna, Kesheng Wu, Parallel query evaluation as a Scientific Data, Cluster Computing (CLUSTER), 2014 IEEE International on, Pages: 194--202 2014, doi: 10.1109/CLUSTER.2014.6968765

Hsuan-Te Chiu, Jerry Chou, Venkat Vishwanath, Byna, Kesheng Wu, Simplifying Index File Structure to Improve I/O of Parallel Indexing, The 20th IEEE International Conference on Parallel and Systems (ICPADS 2014), 2014,

Lingfei Wu, Kesheng Wu, Alex Sim, Michael Churchill, Jong Y Choi, Andreas Stathopoulos, CS Chang, Scott Klasky, "High-performance outlier detection algorithm for finding blob-filaments in plasma", Proc. of 5rd International Workshop on Big Data Analytics: Challenges and Opportunites (BDAC-14), held in conjunction with ACM/IEEE SC14, 2014,

Lingfei Wu, Kesheng Wu, Alex Sim, Andreas Stathopoulos, "Real-time outlier detection algorithm for finding blob-filaments in plasma", ACM/IEEE SC14 ACM SRC Poster, 2014,

David H. Bailey, Stephanie Ger, Marcos L\ opez Prado, Alexander Sim, Kesheng Wu, "Statistical Overfitting and Backtest Performance", http://ssrn.com/abstract2507040, ( January 1, 2014)

ISBN 978-1-78548-008-9

L. Wu, K. Wu, A. Sim, M. Churchill, J. Y. Choi, A. Stathopoulos, CS Chang, S. Klasky, "High-Performance Outlier Detection Algorithm for Blob-Filaments in Plasma", 5th International Workshop on Big Data Analytics: and Opportunities (BDAC 14), 2014,

Jung Heon Song, Marcos L\ opez de Prado, Horst Simon, Kesheng Wu, "Exploring Irregular Time Series Through Non-uniform Fourier Transform", WHPCF 14, Piscataway, NJ, USA, IEEE Press, 2014, 37--44, doi: 10.1109/WHPCF.2014.8

F. Rusu, P. Nugent, K. Wu, "Implementing the Palomar Transient Factory Real-Time Pipeline in GLADE: Results and", Lecture Notes in Computer Science, ( 2014) Pages: 53--66

Jung Heon Song, Kesheng Wu, Horst D Simon, "Parameter Analysis of the VPIN (Volume synchronized of Informed Trading) Metric", Quantitative Financial Risk Management: Theory and, 2014,

2013

William Gu, Jaesik Choi, Ming Gu, Horst Simon, Kesheng Wu, "Fast Change Point Detection for Electricity Market Analysis", IEEE International Conference on Big Data, 2013, LBNL LBNL-6388E, doi: 10.1109/BigData.2013.6691733

Alex Romosan, Arie Shoshani, Kesheng Wu, Victor Markowitz, Kostas Mavrommatis, "Accelerating gene context analysis using bitmaps", Proceedings of the 25th International Conference on Scientific and Statistical Database Management, 2013, 1--12, LBNL 6397E,

E Wes Bethel, Prabhat Prabhat, Suren Byna, Oliver R\ ubel, K John Wu, Michael Wehner, "Why high performance visual data analytics is both relevant and difficult", Visualization and Data Analysis 2013, January 2013, 8654:86540B, LBNL LBNL-6063E,

Bin Dong, Surendra Byna, Kesheng Wu, "SDS: a framework for scientific data services", Proceedings of the 8th Parallel Data Storage Workshop, January 1, 2013, 27--32,

Bin Dong, Surendra Byna, Kesheng Wu, "Expediting scientific data analysis with reorganization of data", 2013 IEEE International Conference on Cluster Computing (CLUSTER), January 1, 2013, 1--8,

Kuan-Wu Lin, Surendra Byna, Jerry Chou, Wu, "Optimizing FastQuery performance on Lustre file", Proceedings of the 25th International Conference on and Statistical Database Management, 2013, 29,

Bin Dong, S. Byna, Kesheng Wu, Expediting scientific data analysis with of data, Cluster Computing (CLUSTER), 2013 IEEE International on, Pages: 1--8 2013, doi: 10.1109/CLUSTER.2013.6702675

Jong Y Choi, Kesheng Wu, Jacky C Wu, Alex Sim, Qing G Liu, Matthew Wolf, C Chang, Scott Klasky, "Icee: Wide-area in transit data processing framework for near real-time scientific applications", 4th SC Workshop on Petascale (Big) Data Analytics: Challenges and Opportunities in conjunction with SC13, 2013, 11,

Kesheng Wu, E Bethel, Ming Gu, David Leinweber, Oliver R\ ubel, "A big data approach to analyzing market volatility", Algorithmic Finance, 2013, 2:241--267, LBNL LBNL-6382E,

Understanding the microstructure of the financial market requires the processing of a vast amount of data related to individual trades, and sometimes even multiple levels of quotes. Analyzing such a large volume of data requires tremendous computing power that is not easily available to financial academics and regulators. Fortunately, public funded High Performance Computing (HPC) power is widely available at the National Laboratories in the US. In this paper we demonstrate that the HPC resource and the techniques for data-intensive sciences can be used to greatly accelerate the computation of an early warning indicator called Volume-synchronized Probability of Informed trading (VPIN). The test data used in this study contains five and a half year's worth of trading data for about 100 most liquid futures contracts, includes about 3 billion trades, and takes 140GB as text files. By using (1) a more efficient file format for storing the trading records, (2) more effective data structures and algorithms, and (3) parallelizing the computations, we are able to explore 16,000 different ways of computing VPIN in less than 20 hours on a 32-core IBM DataPlex machine. Our test demonstrates that a modest computer is sufficient to monitor a vast number of trading activities in real-time -- an ability that could be valuable to regulators.

Our test results also confirm that VPIN is a strong predictor of liquidity-induced volatility. With appropriate parameter choices, the false positive rates are about 7% averaged over all the futures contracts in the test data set. More specifically, when VPIN values rise above a threshold (CDF > 0.99), the volatility in the subsequent time windows is higher than the average in 93% of the cases.

Kesheng Wu, Wes Bethel, Ming Gu, David, Oliver R\ ubel, Testing VPIN on Big Data, Available at SSRN 2318259, 2013,

W. Gu, J. Choi, M. Gu, H. D. Simon, K., "Fast Change Point Detection for electricity market", 2013 IEEE International Conference on Big Data, 2013, 50--57, doi: 10.1109/BigData.2013.6691733

Jong Y. Choi, Kesheng Wu, Jacky C. Wu, Alex, Qing G. Liu, Matthew Wolf, CS Chang, Klasky, ICEE: Wide-area In Transit Data Processing Framework Near Real-Time Scientific Applications, PDAC workshop, SC13, 2013,

2012

Surendra Byna, Jerry Chou, Oliver Rubel, Homa Karimabadi, William S Daughter, Vadim Roytershteyn, E Wes Bethel, Mark Howison, Ke-Jou Hsu, Kuan-Wu Lin, others, "Parallel I/O, analysis, and visualization of a trillion particle simulation", SC 12: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, January 2012, 1--12,

Elaheh Pourabbas, Arie Shoshani, Kesheng Wu, "Minimizing index size by reordering rows and columns", International Conference on Scientific and Statistical Database Management, January 2012, 467--484,

Benson Ma, Arie Shoshani, Alex Sim, Kesheng, Yong-Ik Byun, Jaegyoon Hahm, Min-Su Shin, "Efficient Attribute-Based Data Access in Astronomy", The 2nd International Workshop on Network-Aware Data Workshop (NDM2012), 2012, 562--571,

G. F. Lofstead, Q. Liu, J. Logan, Y. Tian, Abbasi, N. Podhorszki, J. Y. Choi, S., R. Tchoua, R. A. Oldfield, others, "Hello ADIOS: The Challenges and Lessons of Leadership Class I/O Frameworks", 2012,

Oliver R\ ubel, Surendra Byna, Kesheng Wu, Fuyu Li, Michael Wehner, Wes Bethel, others, "Teca: A parallel toolkit for extreme climate analysis", Procedia Computer Science, Elsevier, January 2012, 9:866--876, LBNL 5352E,

We present TECA, a parallel toolkit for detecting extreme events in large climate datasets. Modern climate datasets expose parallelism across a number of dimensions: spatial locations, timesteps and ensemble members. We design TECA to exploit these modes of parallelism and demonstrate a prototype implementation for detecting and tracking three classes of extreme events: tropical cyclones, extra-tropical cyclones and atmospheric rivers. We process a modern TB-sized CAM5 simulation dataset with TECA, and demonstrate good runtime performance for the three case studies.

E. W. Bethel, Surendra Byna, Jerry Chou, Cormier-Michel, Cameron G. R. Geddes, Howison, Fuyu Li, Prabhat, Ji Qiang, R\ ubel, Rob D. Ryne, Michael Wehner, Wu, "Big Data Analysis and Visualization: What Do LINACS Tropical Storms Have In Common?", 11th International Computational Accelerator Physics ICAP 2012, Germany, 2012,

Allen R Sanderson, Brad Whitlock, H Childs, GH Weber, K Wu, others, "A system for query based analysis and visualization", January 2012, LBNL 5507E,

E. W. Bethel and D. Leinweber and O. Rubel and K. Wu, "Federal Market Information Technology in the Post Flash Crash Era: Roles of Supercomputing", The Journal of Trading, 2012, 7:9-24, LBNL 5263E, doi: 10.3905/jot.2012.7.2.009

Ichitaro Yamazaki, Kesheng Wu, "A Communication-Avoiding Thick-Restart Lanczos Method a Distributed-Memory System", Lecture Notes in Computer Science, 2012, 7155:345--354, doi: 10.1007/978-3-642-29737-3_39

E. Wes Bethel, David Leinweber, Oliver Rübel Kesheng Wu, Federal Market Information Technology in the Crash Era: Roles for Supercomputing, The Journal of Trading, Pages: 9--25 2012, doi: 10.3905/jot.2012.7.2.009

2011

E. Wes Bethel, David Leinweber, Oliver Rübel, Kesheng Wu, "Federal Market Information Technology in the Post Flash Crash Era: Roles of Supercomputing", Workshop on High Performance Computational Finance at SC11, Seattle, WA, USA, November 2011, LBNL 5263E,

R. Ryne, B. Austin, J. Byrd, J. Corlett, E. Esarey, C. G. R. Geddes, W. Leemans, X. Li, Prabhat, J. Qiang, O. Rübel, J.-L. Vay, M. Venturini, K. Wu, B. Carlsten, D. Higdon and N. Yampolsky, "High Performance Computing in Accelerator Science: Past Successes, Future Challenges", Workshop on Data and Communications in Basic Energy Sciences: Creating a Pathway for Scientific Discovery, October 2011,

A. Shoshani, I. Altintas, J. Chen, G. Chin, A. Choudhary, D. Crawl, T. Critchlow, K. Gao, B. Grimm, H. Iyer, C. Kamath, A. Khan, S. Klasky, S. Koehler, S. Lang, R. Latham, J. W. Li, W. Liao, J. Ligon, Q. Liu, B. Ludaescher, P. Mouallem, M. Nagappan, N. Podhorszki, R. Ross, D. Rotem, N. Samatova, C. Silva, A. Sim, R. Tchoua, R. Thakur, M. Vouk, K. Wu, W. Yu, "The Scientific Data Management Center: Available Technologies and Highlights", SciDAC Conference, 2011,

Jerry Chou, Mark Howison, Brian Austin, Kesheng Wu, Ji Qiang, E Wes Bethel, Arie Shoshani, Oliver R\ ubel, Rob D Ryne, "Parallel index and query for large scale data analysis", Proceedings of 2011 international conference for high performance computing, networking, storage and analysis, 2011, 1--11, LBNL 5317E,

Kesheng Wu, Rishi R Sinha, Chad Jones, Stephane Ethier, Scott Klasky, Kwan-Liu Ma, Arie Shoshani, Marianne Winslett, "Finding regions of interest on toroidal meshes", Computational Science \& Discovery, 2011, 4:015003,

Kesheng Wu, Surendra Byna, Doron Rotem, Arie, "Scientific Data Services -- A High-Performance I/O with Array Semantics", HPCDB, IEEE, 2011, doi: 10.11v45/2125636.2125640

J. Chou, K. Wu, O. R\ ubel, M. Howison, Qiang, Prabhat, B. Austin, E. W. Bethel, D. Ryne, A. Shoshani, "Parallel Index and Query for Large Scale Data", SC11, 2011, doi: 10.1145/2063384.2063424

Jinoh Kim, Hasan Abbasi, Luis Chac\ on, Docan, Scott Klasky, Qing Liu, Norbert, Arie Shoshani, Kesheng Wu, "Parallel In Situ Indexing for Data-intensive", LDAV, 2011, 65--72, doi: 10.1109/LDAV.2011.6092319

Surendra Byna, Michael F Wehner, Kesheng John Wu, "Detecting atmospheric rivers in large climate datasets", Proceedings of the 2nd international workshop on Petascal data analytics: challenges and opportunities, 2011, 7--14,

Extreme precipitation events on the western coast of North America are often traced to an unusual weather phenomenon known as atmospheric rivers. Although these storms may provide a significant fraction of the total water to the highly managed western US hydrological system, the resulting intense weather poses severe risks to the human and natural infrastructure through severe flooding and wind damage. To aid the understanding of this phenomenon, we have developed an efficient detection algorithm suitable for analyzing large amounts of data. In addition to detecting actual events in the recent observed historical record, this detection algorithm can be applied to global climate model output providing a new model validation methodology. Comparing the statistical behavior of simulated atmospheric river events in models to observations will enhance confidence in projections of future extreme storms. Our detection algorithm is based on a thresholding condition on the total column integrated water vapor established by Ralph et al. (2004) followed by a connected component labeling procedure to group the mesh points into connected regions in space. We develop an efficient parallel implementation of the algorithm and demonstrate good weak and strong scaling. We process a 30-year simulation output on 10,000 cores in under 3 seconds.

M Prabhat, S Byna, C Paciorek, G Weber, K Wu, T Yopes, MF Wehner, G Ostrouchov, D Pugmire, R Strelitz, others, "Pattern Detection and Extreme Value Analysis on Large Climate Data", AGUFM, Pages: IN41C--03 January 2011,

Jerry Chou, Kesheng Wu, Prabhat, "FastQuery: A General Indexing and Querying System Scientific Data", SSDBM, 2011, 573--574, doi: 10.1007/978-3-642-22351-8_42

Jerry Chou, Kesheng Wu, Prabhat, "FastQuery: A Parallel Indexing System for Data", IASDS, IEEE, 2011, doi: 10.1109/CLUSTER.2011.86

Jerry Chou, Kesheng Wu, others, "Fastquery: A parallel indexing system for scientific data", 2011 IEEE International Conference on Cluster Computing, 2011, 455--464,

Kamesh Madduri, Kesheng Wu, "Massive-Scale RDF Processing Using Compressed Bitmap", SSDBM, Springer, 2011, 470--479, doi: 10.1007/978-3-642-22351-8_30

Weikuan Yu, Kesheng Wu, Wei-Shinn Ku, Cong Xu Juan Gao, BMF: Bitmapped Mass Fingerprinting for Fast Protein, CLUSTER, 2011, doi: 10.1109/CLUSTER.2011.11

2010

D. Hasenkamp, A. Sim, M. Wehner, K. Wu, "Finding Tropical Cyclones on Clouds", Supercomputing 2010, ACM SRC 3rd place, 2010,

Kesheng Wu, Arie Shoshani, Kurt Stockinger, "Analyses of multi-level and multi-component compressed indexes", ACM Transactions on Database Systems, ACM, 2010, 35:1--52, doi: 10.1145/1670243.1670245

Daren Hasenkamp, Alexander Sim, Michael Wehner, Kesheng Wu, "Finding tropical cyclones on a cloud computing cluster: Using parallel virtualization for large-scale climate simulation analysis", 2010 IEEE Second International Conference on Cloud Computing Technology and Science, 2010, 201--208, LBNL 4218E,

 

 

Oliver R\ ubel, Sean Ahern, E Wes Bethel, Mark D Biggin, Hank Childs, Estelle Cormier-Michel, Angela DePace, Michael B Eisen, Charless C Fowlkes, Cameron GR Geddes, others, "Coupling visualization and data analysis for knowledge discovery from multi-dimensional scientific data", Procedia computer science, Elsevier, January 2010, 1:1757--1764, LBNL 3669E,

Gunther Weber, "Recent advances in visit: Amr streamlines and query-driven visualization", 2010,

Kesheng Wu, Kamesh Madduri, Shane Canon, "Multi-level bitmap indexes for flash memory storage", Proceedings of the Fourteenth International Database Engineering \& Applications Symposium, 2010, 114--116,

Ichitaro Yamazaki, Zhaojun Bai, Horst D. Simon Lin-Wang Wang, Kesheng Wu, "Adaptive Projection Subspace Dimension for the Lanczos Method", ACM Transactions on Mathematical Software, 2010, 37, doi: 10.1145/1824801.1824805

2009

E. W. Bethel, C. Johnson, S. Ahern, J. Bell, P.-T. Bremer, H. Childs, E. Cormier-Michel, M. Day, E. Deines, T. Fogal, C. Garth, C. G. R. Geddes, H. Hagen, B. Hamann, C. Hansen, J. Jacobsen, K. Joy, J. Kruger, J. Meredith, P. Messmer, G. Ostrouchov, V. Pascucci, K. Potter, Prabhat, D. Pugmire, O. Rubel, A. Sanderson, C. Silva, D. Ushizima, G. Weber, B. Whitlock, K. Wu, "Occam's Razor and Petascale Visual Data Analysis", SciDAC 2009, J. of Physics: Conference Series, San Diego, California, July 2009, LBNL 2210E,

Luke Gosink, Kesheng Wu, E. Wes Bethel, John D. Owens, Kenneth I. Joy, "Data Parallel Bin-based Indexing for Answering Queries on Multi-core Architecture", Proceedings of the 21st International Conference on Scientific and Statistical Database Management (SSDBM), June 2009, 5566:110-129, LBNL 2211E,

K Wu, S Ahern, EW Bethel, J Chen, H Childs, C Geddes, J Gu, H Hagen, B Hamann, J Lauret, others, "FastBit: Interactively Searching Massive Data", Proc. of SciDAC 2009, 2009, LBNL 2164E,

Oliver R\ ubel, Cameron GR Geddes, Estelle Cormier-Michel, Kesheng Wu, Gunther H Weber, Daniela M Ushizima, Peter Messmer, Hans Hagen, Bernd Hamann, Wes Bethel, others, "Automatic beam path analysis of laser wakefield particle acceleration data", Computational Science \& Discovery, January 2009, 2:015005, LBNL 2734E,

C. G. R. Geddes, E Cormier-Michel, E. H. Esarey, C. B. Schroeder, J.-L. Vay, W. P. Leemans, D. L.. Bruhwiler, J. R. Cary, B. Cowan, M. Durant, P. Hamill, P. Messmer, P. Mullowney, C. Nieter, K. Paul, S. Shasharina, S. Veitzer, G. Weber, O. Rübel, D. Ushizima, Prabhat, E. W.Bethel, K. Wu, Large Fields for Smaller Facility Sources, SciDAC Review, Pages: 13-21, 2009,

E Bethel, "Modern Scientific Visualization is More than Just Pretty Pictures", January 2009, LBNL 1450E,

Luke J Gosink, Kesheng Wu, E Wes Bethel, John D Owens, Kenneth I Joy, "Data parallel bin-based indexing for answering queries on multi-core architectures", International Conference on Scientific and Statistical Database Management, 2009, 110--129,

 

 

Lifeng He, Yuyan Chao, Kenji Suzuki, Kesheng Wu, "Fast connected-component labeling", Pattern recognition, 2009, 42:1977--1987,

Meiyappan Nagappan, Kesheng Wu, Mladen A Vouk, "Efficiently extracting operational profiles from execution logs using suffix arrays", 2009 20th International Symposium on Software Reliability Engineering, January 1, 2009, 41--50,

An important software reliability engineering tool is operational profiles. In this paper we propose a cost effective automated approach for creating second generation operational profiles using execution logs of a software product. Our algorithm parses the execution logs into sequences of events and produces an ordered list of all possible subsequences by constructing a suffix array of the events. The difficulty in using execution logs is that the amount of data that needs to be analyzed is often extremely large (more than a million records per day in many applications). Our approach is very efficient. We show that our approach requires O(N) in space and time to discover all possible patterns in N events. We discuss a practical implementation of the algorithm in the context of the logs from a large cloud computing system.

Kesheng Wu, Ekow Otoo, Kenji Suzuki, "Optimizing two-pass connected-component labeling", Pattern Analysis \& Applications, 2009, 12:117--135,

E Wes Bethel, Chris Johnson, Sean Ahern, John Bell, Peer-Timo Bremer, Hank Childs, Estelle Cormier-Michel, Marc Day, Eduard Deines, Tom Fogal, others, "Occam s razor and petascale visual data analysis", Journal of Physics: Conference Series, 2009, 180:012084,

Ekow Otoo, Kesheng Wu, Accelerating queries on very large datasets, 2009,

Meiyappan Nagappan, Kesheng Wu, Mladen A. Vouk, Efficiently Extracting Operational Profiles from Logs Using Suffix Arrays, ISSRE, Pages: 41--50 2009, doi: 10.1109/ISSRE.2009.23

Kamesh Madduri, Kesheng Wu, Efficient joins with compressed bitmap indexes, Proceedings of the 18th ACM conference on Information and knowledge management, Pages: 1017--1026 2009,

2008

O. Rübel, Prabhat, K. Wu, H. Childs, J. Meredith, C.G.R. Geddes, E. Cormier-Michel, S. Ahern, G.H. Weber, P. Messmer, H. Hagen, B. Hamann and E.W. Bethel, "High Performance Multivariate Visual Data Exploration for Extemely Large Data", Supercomputing (SC), Austin, Texas, USA, November 2008, LBNL 716E,

O. Rübel, Prabhat, K. Wu, H. Childs, J. Meredith, C.G.R. Geddes, E. Cormier-Michel, S. Ahern, G.H. Weber, P. Messmer, H. Hagen, B. Hamann and E.W. Bethel, "Application of High-performance Visual Analysis Methods to Laser Wakefield Particle Acceleration Data", IEEE Visualization 2008, October 2008,

Kurt Stockinger, John Cieslewicz, Kesheng Wu, Rotem, Arie Shoshani, "Using Bitmap Indexing Technology for Combined and Text Queries", Annals of Information Systems, (Springer: 2008) Pages: 1--23

Rishi Rakesh Sinha, Marianne Winslett, Kesheng, Kurt Stockinger, Arie Shoshani, "Adaptive Bitmap Indexes for Space-Constrained", ICDE 2008, 2008, 1418--1420,

Kesheng Wu, Kurt Stockinger, Arie Shoshani, "Breaking the curse of cardinality on bitmap indexes", International Conference on Scientific and Statistical Database Management, 2008, 348--365,

Meiyappan Nagappan, Mladen A. Vouk, Kesheng Wu Alex Sim, Arie Shoshani, "Efficient Operational Profiling of Systems Using Arrays on Execution Logs", ISSRE, 2008, 313--314, doi: 10.1109/ISSRE.2008.45

Luke J Gosink, "Bin-hash indexing: A parallel method for fast query processing", 2008, LBNL 729E,

I. Yamazaki, K. Wu, H. Simon, "nu-TRLan User Guide version 1.0", 2008, LBNL 1288E,

E. Wes Bethel, Oliver Rübel, Prabhat, Wu, Gunther H. Weber, Valerio Pascucci Hank Childs, Ajith Mascarenhas, Jeremy, Sean Ahern, "Modern Scientific Visualization is More than Just Pictures", Numerical Modeling of Space Plasma Flows: (Astronomical Society of the Pacific Series), St. Thomas, USVI, 2008, 301--317,

Luke J. Gosink, Kesheng Wu, E. Wes Bethel, D. Owens, Kenneth I. Joy, Bin-Hash Indexing: A Parallel Method For Fast Processing, 2008,

Oliver R\ ubel, Prabhat, Kesheng Wu, Hank, Jeremy Meredith, Cameron G. R. Geddes, Cormier-Michel, Sean Ahern, Gunther H., Peter Messmer, Hans Hagen, Bernd Hamann E. Wes Bethel, Application of High-performance Visual Analysis to Laser Wakefield Particle Acceleration Data, IEEE Visualization 2008, 2008,

Oliver R\ ubel, Prabhat, Kesheng Wu, Hank, Jeremy Meredith, Cameron G. R. Geddes, Cormier-Michel, Sean Ahern, Gunther H., Peter Messmer, Hans Hagen, Bernd Hamann E. Wes Bethel, High Performance Multivariate Visual Data Exploration Extemely Large Data, SuperComputing 2008 (SC08), Pages: 51 2008,

2007

Kesheng Wu, Kurt Stockinger, Arie Shoshani, Performance of Multi-Level and Multi-Component Bitmap Indexes, 2007, doi: 10.1145/1670243.1670245

Frederick Reiss, Kurt Stockinger, Kesheng Wu, Shoshani, Joseph M. Hellerstein, "Enabling Real-Time Querying of Live and Historical Data", SSDBM 2007, 2007,

Kesheng Wu, "Fastbit reference manual", 2007, LBNL LBNL PUB/3192,

Kurt Stockinger, Kesheng Wu, "Bitmap indices for data warehouses", Data Warehouses and OLAP: Concepts, Architectures and Solutions, (IGI Global: 2007) Pages: 157--178

Elizabeth O Neil, Patrick O Neil, Kesheng Wu, Bitmap Index Design Choices and Their Performance, IDEAS 2007, Pages: 72--84 2007,

2006

Kesheng Wu, Ekow J Otoo, Arie Shoshani, "Optimizing bitmap indices with efficient compression", ACM Transactions on Database Systems (TODS), 2006, 31:1--38,

K. Wu, K. Stockinger, A. Shoshani, Wes, "FastBit--Helps Finding the Proverbial Needle in a", 2006, LBNL LBNL-PUB/963,

F. Reiss, K. Stockinger, K. Wu, A. Shoshani J. M. Hellerstein, "Efficient analysis of live and historical streaming and its application to cybersecurity", 2006,

Kurt Stockinger, Kesheng Wu, Rene Brun, Canal, "Bitmap indices for fast end-user physics analysis in", Nuclear Instruments and Methods in Physics Research A: Accelerators, Spectrometers, Detectors and Equipment, 2006, 559:99--102,

Luke Gosink, John Shalf, Kurt Stockinger, Wu, Wes Bethel, "HDF5-FastQuery: Accelerating Complex Queries on Datasets using Fast Bitmap Indices", SSDBM 2006, Vienna, Austria, July 2006, IEEE Computer Society Press., 2006, 149--158,

Luke Gosink, John Shalf, Kurt Stockinger, Kesheng Wu, Wes Bethel, HDF5-FastQuery: Accelerating complex queries on HDF datasets using fast bitmap indices, 18th International Conference on Scientific and Statistical Database Management (SSDBM 06), Pages: 149--158 2006,

E. Wes Bethel, Scott Campbell, Eli Dart, Kurt Stockinger, Kesheng Wu, Accelerating Network Traffic Analysis Using Visualization, Symposium on Visual Analytics Science and Technology Baltimore, Maryland, USA, October 31 - November 2006, Pages: 115--122 2006,

E. Wes Bethel, Scott Campbell, Eli Dart, John Shalf, Kurt Stockinger, Kesheng Wu, High Performance Visualization using Query-Driven and Analytics, 2006,

Kurt Stockinger, E. Wes Bethel, Scott Campbell, Eli Dart, Kesheng Wu, Detecting distributed scans using high-performance visualization, SC 06, Pages: 82 2006,

Doron Rotem, Kurt Stockinger, Kesheng Wu, Minimizing I/O Costs of Multi-Dimensional Queries Bitmap Indices, SSDBM 2006, Vienna, Austria, July 2006, 2006,

2005

Kesheng Wu, Junmin Gu, Jerome Lauret, Arthur Poskanzer, Arie Shoshani, Alexander Sim, Zhang, "Grid Collector: Facilitating Efficient Selective from Data Grids", International Supercomputer Conference 2005, 2005,

K. Wu, E. Otoo, "A simpler proof of the average case complexity of with path compression", 2005,

Kesheng Wu, "FastBit: an efficient indexing technology for data-intensive science", Journal of Physics: Conference Series, IOP Publishing, 2005, 16:556--560, LBNL LBNL-2164E, doi: 10.1088/1742-6596/16/1/077

Kesheng Wu, Ekow Otoo, Arie Shoshani, "Optimizing connected component labeling algorithms", Medical Imaging 2005: Image Processing, 2005, 5747:1965--1976,

Kesheng Wu, Ekow Otoo, Kenji Suzuki, "Two Strategies to Speed up Connected Component Algorithms", 2005,

E. Wes Bethel, Scott Campbell, Eli Dart, Lee, Steven A. Smith, Kurt Stockinger, Tierney, Kesheng Wu, "Interactive Analysis of Large Network Data Collections Query-Driven Visualization", 2005,

Kurt Stockinger, John Shalf, Kesheng Wu, E Wes Bethel, "Query-driven visualization of large data sets", VIS 05. IEEE Visualization, 2005., 2005, 167--174,

Kurt Stockinger, John Shalf, Wes Bethel, Kesheng Wu, DEX: Increasing the Capability of Scientific Data Analysis by Using Efficient Bitmap Indices to Accelerate Scientific Visualization, SSDBM, Pages: 35-44 2005,

Kurt Stockinger, Kesheng Wu, Scott Campbell, Lau, Mike Fisk, Eugene Gavrilov, Alex, Christopher E. Davis, Rick Olinger, Rob, Jim Prewett, Paul Weber, Thomas P., E. Wes Bethel, Steve Smith, Network Traffic Analysis With Query Driven, SC 2005, 2005,

Doron Rotem, Kurt Stockinger, Kesheng Wu, Optimizing I/O Costs of Multi-dimensional Queries Bitmap Indices., DEXA, Pages: 220--229 2005,

2004

K. Wu, A. Shoshani, E. J. Otoo, Word aligned bitmap compression method, data and apparatus, US Patent 6,831,575, 2004,

Kesheng Wu, Ekow J Otoo, Arie Shoshani, "An efficient compression scheme for bitmap indices", 2004,

Kesheng Wu, Wei-Ming Zhang, Victor, Jerome Lauret, Arie Shoshani, "The Grid Collector: Using an Event Catalog to Speed up Analysis in Distributed Environment", Proceedings of Computing in High Energy and Nuclear (CHEP) 2004, 2004,

Kurt Stockinger, Kesheng Wu, Arie Shoshani, Evaluation Strategies for Bitmap Indices with, International Conference on Database and Expert Applications (DEXA 2004), Zaragoza, Spain, 2004,

2003

Kesheng Wu, Wei-Ming Zhang, Alexander Sim, Gu, Arie Shoshani, "Grid Collector: An Event Catalog With Automated File", Proceedings of IEEE Nuclear Science Symposium 2003, 2003, doi: 10.1109/NSSMIC.2003.1351830

Kesheng Wu, Wei-Ming Zlang, Alexander Sim, Junmin Gu, Arie Shoshani, "Grid collector: An event catalog with automated file management", 2003 IEEE Nuclear Science Symposium. Conference Record (IEEE Cat. No. 03CH37515), 2003, LBNL 55563,

2002

Kesheng Wu, Ekow Otoo, Arie Shoshani, Compressing Bitmap Indexes for Faster Search, Proceedings of SSDBM 02, Pages: 99--108 2002,

Kurt Stockinger, Kesheng Wu, Arie Shoshani, Strategies for processing ad hoc queries on large data, Proceedings of DOLAP 02, Pages: 72--79 2002,

Andreas Stathopoulos, Kesheng Wu, "A Block Orthogonalization Procedure with Constant Rquirements", SIAM Journal on Scientific Computing, 2002, 23:2165--2182,

2001

L Bernardo, H Nordberg, D Olson, A Shoshani, A Sim, A Vaniachine, D Zimmerman, B Gibbard, R Porter, T Wenaus, others, "New capabilities in the HENP grand challenge storage access system and its application at RHIC", Computer physics communications, 2001, 140:179--188,

Kesheng Wu, Ekow J Otoo, Arie Shoshani, A performance comparison of bitmap indexes, Proceedings of the tenth international conference on Information and knowledge management, Pages: 559--561 2001,

2000

L. M. Bernardo, B. Gibbard, D. Malon, H. Nordberg, D. Olson, R. Porter, A. Shoshani, A. Sim, A. Vaniachine, T. Wenaus, K. Wu, D. Zimmerman, "New Capabilities in the HENP Grand Challenge Storage Access System and its Application at RHIC", Computing in High Energy Physics, 2000,

Kesheng Wu, Horst Simon, "Thick-restart Lanczos method for large symmetric problems", SIAM J. Matrix Anal. Appl., 2000, 22:602--616,

1999

Kesheng Wu, Horst Simon, "A Parallel Lanczos method for symmetric generalized problems", Computing and Visualization in Science, 1999, 2:37--46,

Kesheng Wu, Horst D Simon, "Dynamic restarting schemes for eigenvalue problems", 1999,

Kesheng Wu, Horst Simon, "An Evaluation of Parallel Shift-and-Invert Lanczos", Proceedings of The 1999 International Conference on and Distributed Processing Techniques and Las Vegas, Nevada, June 28 - July 1, 1999, 2913--19,

Kesheng Wu, Horst Simon, "Parallel Efficiency of the Lanczos method for problems", Berkeley, CA, 1999,

Kesheng Wu, Andrew Canning, Horst D. Simon, Wang, "Thick-Restart Lanczos method for electronic calculations", Journal of Computational Physics, 1999, 154:156--173,

Kesheng Wu, Horst Simon, TRLAN user guide, 1999,

K Wu, A Canning, HD Simon, LW Wang, "Thick-Restart Lanczos Method for Electronic Structure Calculations", Journal of Computational Physics, 1999, 154:156--173,

1998

Kesheng Wu, Andrew Canning, Horst D. Simon, "A new Lanczos method for electronic structure", Proceedings of ACM/IEEE SC98 Conference, November 1998, in Orlando, FL, New York, NY, IEEE, 1998,

Kesheng Wu, Yousef Saad, Andreas Stathopoulos, Inexact Newton Preconditioning Techniques for Problems, Electronic Transactions on Numerical Analysis, Pages: 202--214 1998,

Kesheng Wu, Horst D. Simon, Thick-restart Lanczos method for symmetric problems, Lecture Notes in Computer Science, Pages: 43--55 1998,

1993

Kesheng Wu, Robert Savit, William Brock, "Statistical tests for deterministic effects in broad time series", Physica D, 1993, 69:172--188, doi: 10.1016/0167-2789(93)90188-7

1992

Kesheng Wu, Stability of midpoint methods on second order ODEs, 1992,

Wucherl Yoo

2017

J. Kim, W. Yoo, A. Sim, S.C. Suh, I. Kim, "A Lightweight Network Anomaly Detection Technique", International Workshop on Computing, Networking and Communications (CNC 2017), 2017, doi: 10.1109/ICCNC.2017.7876251

Jonathan Wang, Wucherl Yoo, Alex Sim, Peter Nugent, Kesheng Wu, "Parallel variable selection for effective performance prediction", 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), 2017, 208--217,

2016

M. Bae, W. Yoo (Advisor), A. Sim (Advisor), K. Wu (Advisor), "Discovering Energy Resource Usage Patterns on Scientific Clusters", International Conference for High Performance Computing, Networking, Storage and Analysis (SC’16), ACM Student Research Competition (SRC), Third place winner, 2016, 2016,

Jonathan Wang, Wucherl Yoo, Alex Sim, K John Wu, "Analysis of Variable Selection Methods on Scientific Cluster Measurement Data", 2016,

Wucherl Yoo, Michelle Koo, Yi Cao, Alex Sim, Peter Nugent, Kesheng Wu, "Performance Analysis Tool for HPC and Big Data Applications on Scientific Clusters", Conquering Big Data with High Performance Computing, (Springer, Cham: 2016) Pages: 139--161

Wucherl Yoo, Alex Sim, Kesheng Wu, "Machine learning based job status prediction in scientific clusters", 2016 SAI Computing Conference (SAI), 2016, 44--53,

2015

M. Koo, W. Yoo (advisor), A. Sim (advisor), "I/O Performance Analysis Framework on Measurement Data from Scientific Clusters", International Conference for High Performance Computing, Networking, Storage and Analysis (SC’15), ACM Student Research Competition (SRC), 2015, 2015,

W. Yoo, A. Sim, "Network Bandwidth Utilization Forecast Model on High Bandwidth Networks", IEEE International Conference on Computing, Networking and Communications (ICNC’15), 2015, doi: 10.1109/ICCNC.2015.7069393

Wucherl Yoo, Michelle Koo, Yi Cao, Alex Sim, Peter Nugent, Kesheng Wu, "Patha: Performance analysis tool for hpc applications", 2015 IEEE 34th International Performance Computing and Communications Conference (IPCCC), 2015, 1--8,

W. Yoo, M. Koo, Y. Cao, A. Sim, P. Nugent, K. Wu, PATHA: Performance Analysis Tool for HPC, 2015 IEEE 34th International Performance Computing and Conference (IPCCC), Pages: 1--8 2015, doi: 10.1109/PCCC.2015.7410313

2014

W. Yoo, A. Sim, "Efficient Changing Pattern Detection on High Bandwidth Network Measurements", 7th International Conference on Grid and Distributed Computing, 2014,

2013

M. Montanari, E. Chan, K. Larson, W. Yoo, R. H. Campbell, "Distributed security policy conformance", Computers & Security, March 31, 2013,

2012

W. Yoo, K. Larson, L. Baugh, S. Kim, R. H. Campbell, "ADP: automated diagnosis of performance pathologies using hardware events", SIGMETRICS '12: Proc. of the 12th ACM SIGMETRICS/PERFORMANCE joint international conference on Measurement and Modeling of Computer Systems, 2012,

2011

W. Yoo, K. Larson, L. Baugh, S. Kim, W. Ahn, R. H. Campbell, "Automated Fingerprinting of Performance Pathologies Using Performance Monitoring Units (PMUs)", HotPar'11: Proc. of USENIX Workshop on Hot topics in parallelism., May 26, 2011,

M. Montanari, E. Chan, K. Larson, W. Yoo, R. H. Campbell, "Distributed security policy conformance", Future Challenges in Security and Privacy for Academia and Industry, January 1, 2011,

2010

W. Yoo, S. Shi, W. J. Jeon, K. Nahrstedt, R. H. Campbell, "Real-time parallel remote rendering for mobile devices using graphics processing units", ICME '10: IEEE International Conference onMultimedia and Expo, July 19, 2010,