On Evolving Network Security for a Research and Enterprise Network

SC15 security technology racks with help from Dell,
Gigamon, Reservoir Labs, Splunk, and Verisign.

With more than 12,000 researchers, students, professionals, and vendors attending the conference on high performance computing, networking, storage and analysis conference, better known as SC, SC’s internet, SCinet, has to be at the forefront of network security research and implementations to stay ahead of the game on viruses, trojans, denial of service and other forms of compromise.

As the backbone of data communications for SC, SCinet supports high-bandwidth demos, HPC workshops, classrooms, and thousands of conference attendees accessing the network at any given point. With network traffic from the exhibit floor, wireless and external network traffic, there are many opportunities for the network to be compromised. SCinet’s Network Security team works to develop novel strategies to protect the integrity of the network.

“The Network Security team has transitioned from using signatures to detect single instances of malicious activity in the network to performing behavioral analytics, which track the overall patterns of security threats,” says Jeff Boote, Technical Staff at Sandia National Laboratories and co-lead for the SCinet Network Security team. “We started using these security algorithms to detect malicious hosts on the network about 2-3 years ago to be more proactive and evolve with the security trends.”

Resevoir Labs staff on SCinet. 

One reason for major changes in the current security strategy is the dramatic increase in network traffic at the conference. Carrie Gates, Chief Security Scientist at Dell Research and Co-lead for the Network Security team, estimates that the amount of traffic that they monitor is two to three orders of magnitude greater than the monitored traffic three years ago.

“We went from 2 10-gigabit taps (20-gigabit aggregate) in 2011, to 31 taps with 580-gigabit aggregate in 2015. With an expected 5000 simultaneous hosts on the Wi-Fi alone, on an open network that is only up for one to two weeks, the Network Security team must find novel ways to prevent malicious activity on the network,” says Gates.

Working with vendors and partners, the SCinet team integrates the technologies to monitor for harmful network traffic, detect and prevent compromised systems, and mitigate the effects of compromised systems.

Over the last three years, SCinet has used Gigamon to help monitor and secure its network operation centers and Internet access gateway. Gigamon acts as a type of mirror, allowing the Network Security team to peer into network traffic and observe a copy of the activity going through the gateway.  This traffic is being analyzed by technologies such as the Dell Firewall Sandwich, which provides high-speed deep packet inspection for malicious security events.

SCinet also serves as a research testbed for the network security.

“Back in 2000, the intrusion detection system, Bro, was being developed by Berkeley Lab and ICSI. SCinet was used as a place to test their technology on a network that is both production and research oriented,” says Gates. “It’s a balance here. SCinet provides the protection grade security of a full production network, while at same time working with technology experts who want to test new features. We’re able to provide both on SCinet. We provide production level network security while allowing for research innovation.”

One SCinet partner, Reservoir Labs, began as a SCinet Sandbox (now called Network Research Exhibition) project and is now an integral part of security architecture. Reservoir uses Bro technology under the covers to both log and analyze activity on the network.  Like Bro, Reservoir used SCinet to test the high-speed analysis features of their product while also providing production-level stability and results.

While SCinet has done behavior-based analysis of security data within a research context for more than a decade, these techniques are becoming ever more popular in day-to-day network operations. To fulfill that role in 2015, SCinet is using Splunk to aggregate the security events from Gigamon, Reservoir, and Dell SonicWALL, along with intelligence data from Verisign’s iDefense, to look for suspicious behaviors in network traffic.

This year, SCinet has over $5,000,000 in vendor loaned technology for network security.

The SC15 SCinet Network Operating Center brining in 1.62 Terabits per second
of network bandwidth.

We want to thank all the vendors and researchers over the years for engaging with SCinet and allowing us to explore the boundaries of security technologies. We look forward to seeing you at SC16!

SC15 Breaks Exhibits and Attendance Records While in Austin, Texas

Noted science communicator and award-winning actor Alan Alda gives the SC15 Keynote Address in Austin, TX, where record-breaking attendance reached approximately  13,000 people.

 Media Contact: Brian Ban (773) 454-7423 or BrianBan@SC15Austin.com

Austin, TX - SC15, the 27th anniversary conference of high performance computing (HPC), networking, storage and analysis, celebrated the contributions of researchers and scientists - from those just starting their careers to those whose contributions have made lasting impacts.

The SC15 Exhibit Hall broke records in Austin, TX. (Click photo to enlarge.)

The conference drew a record-breaking 12,914+ registered attendees (as of data collected on 11/20) and featured a technical program spanning six days. The exhibit hall featured 343 exhibitors from industry, academia and research organizations from around the world.

Crowds rushed to the SC15 Exhibit Hall between engaging Technical Sessions.

“In a world of ever-growing, complex problems, high performance computing has a plethora of opportunities to impact lives across multiple fields,” said Jackie Kern, SC15 General Chair from the University of Illinois at Champaign-Urbana.

SC15 found its home in Austin, TX - pictured here at dawn.

She continued, “No longer is high performance computing reserved for only the biggest corporations in the world. It is now trickling down to smaller, medium-sized firms, which is only increasing its impact. This conference brings together the best research and industry minds in the world for one packed week of sharing ideas, networking, and education.”

SC15 General Chair Jackie Kern being interviewed by Austin-CBS affiliate KEYE in the Exhibit Hall.

According to Kern, the SC15 Exhibition was sold out and the industry-focused exhibit space was the largest to date in the history of the conference. The 217 industry-focused exhibits also marks a new SC15 record. In addition to record-breaking exhibit space, there were a total of 43 first-time exhibitors and 113 international exhibitors from 23 countries (outside the US).  In all, there was a total of approx. 137,000 net square feet of exhibit space and 343 industry and research organizations

SCinet - once again, the world's fastest conference computer network.

During the conference, Austin also became the hub for the world’s fastest conference computer network - SCinet (SC15’s custom network) which made 1.62 (16 100G + 2 10G) Terabits of bandwidth available to exhibitors and attendees.  The network featured 84 miles of fiber deployed throughout the convention center and $18 million in loaned equipment.  It was all made possible by 130 volunteers representing global organizations spanning academia, government and industry.

Noted science communicator and award-winning actor Alan Alda opened the SC15 Technical Program with a keynote speech focusing on the role of science in our society and the intersection of science and computing to a full house of approximately 3,000 people in Austin, Texas on November 17th.

Jeffrey S. Vetter, the SC15 Technical Program Chair, takes the stage.

The Technical Program again offered the highest quality in original HPC research. The SC workshops set a new record with more than 2,000 attendees. There were 10 Best Paper Finalists and 5 Gordon Bell Finalists. These submissions represent the best of the best in a wide variety of research topics in HPC.

Congrats to the 2015 Gordon Bell Prize Award winners. The team received their award at SC15 for using innovative algorithms & implicit solvers to realistically simulate current conditions of the Earth’s interior for the first time. Their work could herald a major step toward more accurately predicting earthquakes and volcanic eruptions.

Overall Stats on Tech Program Tracks:

•    75 Birds-of-a-Feathers
•    78 Papers
•    12 Panels
•    135 Posters  
•    41 Tutorials
•    42 Workshops
•    14 Invited Speakers
•    14 Doctoral Showcase Presentations
•    10 Emerging Technology Presentations

Intel executive Diane Bryant addresses the crowd at the HPC Matters Plenary.

For the second year, SC featured an opening “HPC Matters” plenary that this year was led by Diane Bryant, senior vice president and general manager of Intel’s Data Center Group, who discussed how next-generation supercomputers are transforming HPC.  She further identified exciting opportunities for advancing scientific research and discovery to deliver far-reaching impacts on society.

As part of this Plenary, Intel announced a $300,000 scholarship per-year for five years for women/minorities with STEM undergraduate degrees to fund fellowships for those pursuing PhD's degrees in computational and data science.  This commitment is in honor of Intel's third CEO Andrew Grove. 

The entire Plenary is available online and available by clicking here.

SC16 will be held next November 13th-18th in Salt Lake City, Utah.  For more details, click here.

About SC15
SC15, sponsored by ACM (Association for Computing Machinery) and IEEE Computer Society offers a complete technical education program and exhibition to showcase the many ways high performance computing, networking, storage and analysis lead to advances in scientific discovery, research, education and commerce. This premier international conference includes a globally attended technical program, workshops, tutorials, a world class exhibit area, demonstrations and opportunities for hands-on learning. For more information on SC15, please visit http://www.sc15.supercomputing.org/, or contact communications@info.supercomputing.org for more information.

About ACM
ACM, the Association for Computing Machinery www.acm.org, is the world’s largest educational and scientific computing society, uniting computing educators, researchers and professionals to inspire dialogue, share resources and address the field’s challenges. ACM strengthens the computing profession’s collective voice through strong leadership, promotion of the highest standards, and recognition of technical excellence. ACM supports the professional growth of its members by providing opportunities for life-long learning, career development, and professional.

Connecting Live in Austin with One of the Mosts Powerful Networks in the World

The SCinet network, SC’s Supercomputing Internet, is now live! On November 14, the Austin Convention Center became home to the fastest and most innovative computer network in the world built by volunteers, delivering more than 1.6 terabits per second of network bandwidth to the SC conference (SC15).

SCinet gives the SC conference attendees a unique chance to showcase and discover the latest research in HPC. By building the fastest, most innovative operational network possible every year, SCinet enables data-intensive research and live-use of high performing hardware to run multi-gigabit demonstrations, requiring a fast and robust infrastructure.

SC15 SCinet Contributors (click to enlarge).

“This network is unrivaled with regards to its capabilities and the broad-reaching influence, both nationally and internationally, to support demonstrations and experiments that could not be done easily in any other place. It’s a one-of-a-kind environment where research meets production,” says Davey Wheeler, SCinet Chair and Senior Network Engineer from the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign (UIUC). “As the SCinet Chair leading the development of this network, this year is a culmination of 17 years of experience working on SCinet from year to year. It is humbling and honoring to be able to work aside these colleagues and see the tremendous talent, dedication, and creativity of the volunteers.”

SCinet is built by a team of expert volunteers from around the world, taking one year to design the network, three weeks to set it up, four days to operate it, and twenty-four hours to tear it down. Over 100 engineers from industry, academia and government institutions came together to build this network, using over $22 million in loaned equipment and over 89 miles of newly installed fiber optic cables.

“Having been the SCinet Chair for SC07 in Reno, I am intimately familiar with the incredible amount of planning and work that goes into creating what will be the most powerful network. Over 130 SCinet volunteers from more than 15 countries have worked energetically for the past year to provide wired and wireless access to our conference attendees, and the platform for our exhibitors to showcase bandwidth-driven HPC and cloud computing applications. SCinet continues to be a crucial part of SC and I am extremely grateful for their hard work,” says Jackie Kern, Director of IT Shared Services at UIUC and SC15 Conference Chair.

For SC15, SCinet has connected multiple 100 gigabit circuits, bringing an unprecedented 1.62 terabits per second of bandwidth to the Austin Convention Center. Lonestar Education and Research Network (LEARN) leads this effort in collaboration with leading national and international research networks and commodity providers. LEARN and SCinet supports the HPC community by providing multiple 100 gigabit waves and complementary capabilities throughout the SC15 conference events. 

In addition to the massive external capacity SCinet brings to the convention center, the network is also supporting research initiatives through a half-day workshop, Innovating the Network for Data-Intensive Science (INDIS); and the Network Research Exhibition (NRE). SCinet organizes the INDIS workshop to discuss technical papers and show floor demonstrations dedicated to high performance networking technologies, innovations, protocols, hardware, and much more.

Further, SCinet is providing the wireless connectivity for more than 11,000 expected conference attendees throughout the conference areas. The SCinet team built the SC15 wireless network using 339 wireless access points to support more than 4,000 simultaneous users on the conference wifi. The wireless network will include support for eduroam (education roaming) service, which allows users (researchers, teachers, students, and staff) from participating institutions to securely access the protected wireless network using their home organization’s login credentials.

SCinet is the result of the hard work and significant contributions of many government, research, education and corporate collaborators who have volunteered time, equipment and expertise to ensure SC15’s success. This year, SCinet continued the Contributors Program and we would like to give a special thank you to all SCinet contributors and volunteers!

Platinum contributors: CenturyLink, Ciena, ESnet, Infinera, Internet2, Juniper Networks, and Lonestar Education and Research Network (LEARN).

Gold contributors: Brocade, Cisco, and Gigamon.

Silver contributors: Alcatel-Lucent, Arista, Austin Convention Center, Dell, Greater Austin Area Telecommunications Network (GAATN), inMon, IXIA, Reservoir Labs, Spirent, and The University of Texas System.

Bronze contributors: CABLExpress, Commscope, Splunk, Verisign, and Viavi Solutions.

Volunteers from the following organizations supporting the development and deployment of SCinet: Alcatel-Lucent, Army Research Laboratory, CABLExpress Corporation, CENIC, CenturyLink, Ciena, Cisco, Clemson University, DFN-Verein, DataDirect Networks, Dell Research, Energy Sciences Network (ESnet), Florida LambdaRail (FLR), Freelance, Georgia Institute of Technology, Idaho State University, InMon Corporation, Indiana University, Indiana University GlobalNOC, Indiana University of Pennsylvania, Infinera, Internet Consulting of Vermont, Internet2, JDSU, Lawrence Livermore National Laboratory, Lonestar Education and Research Network (LEARN), Louisiana Optical Network Initiative (LONI), Louisiana State University, the National Center for Supercomputing Applications, National Oceanic and Atmospheric Administration (NOAA), Pennsylvania State University, Purdue University, REANNZ, Radware, Reservoir Labs, SURFnet, Sandia National Laboratories, The University of Texas at Austin, University Corporation for Atmospheric Research, University of Amsterdam, University of California, San Diego, University of Colorado Boulder, University of Heidelberg, University of Illinois at Urbana-Champaign, University of Michigan, University of Oklahoma, University of Pittsburgh, University of Tennessee, Knoxville, University of Texas, University of Wisconsin-Madison, Utah Education Network, and Virginia Polytechnic Institute and State University.

If you’re interested in volunteering for SCinet or participating in the Contributor’s Program for SC16, please email scinet@info.supercomputing.org.

SC15's SCinet “Network Research Exhibition” Demonstrates Innovations on High-Capacity Networks, Driving Science Discovery

Look for these signs to find the NRE 2015 demos.

The 2015 edition of SC15’s SCinet Network Research Exhibition, a forum to showcase new and cutting-edge networking technologies, will host the Second Annual Workshop on Innovating the Network for Data Intensive Science (INDIS) on Monday, November 18.

This year’s workshop will feature new emerging technologies, such as Software Defined Networking (SDN), Science DMZs, network function virtualization (NFV), software-defined exchanges (SDX), Openflow, and many more services used in computing and storage.

These technologies are changing how SCinet, universities, supercomputing centers, data centers, networks and clouds operate. During this workshop further discussions will ensue on more exotic technologies around the corner with wide-area Infiniband, Named Data Networking and Intent-based Networking.

“The influence of these technologies on the scientific process has been tremendous,” says Brian Tierney, Staff Scientist at Berkeley Lab and co-chair of NRE. “Using Science DMZs gives scientists access to a wealth of data unconstrained by geography and time to transfer the data. Supercomputing centers are now optimized around massive data pipes and storage. The worldwide high-speed optical networks and exchanges are specifically architected to support this new paradigm of data intensive science.”

SCinet is deeply interested and involved in working with these technologies, which work to support and meet various supercomputer and data application requirements. Therefore SCinet looks to demonstrate the newest developments at SC every year. The INDIS workshop will host a panel session on the influence of these developments on SCinet’s core technology, thus demonstrating the proof of concepts for inclusion in future production networks.

“Technologies like SDN, NFV and Science DMZs will make their way into SCinet as these are adopted by the HPC community. SCinet is ideally positioned to endeavor and dry-run the new emerging technologies while supporting cutting-edge high performance computing and data processing applications on the SC exhibition floor,” said Professor Dr. Cees de Laat, this year’s co-chair of NRE.

The goal of NRE 2015 is to highlight network innovations from the experiments running over SCinet, the SC conference’s dedicated high performance research and production network. SCinet serves as the platform for exhibitors to demonstrate the advanced computing resources of their home institutions and elsewhere by supporting a wide variety of bandwidth-driven applications, including supercomputing and cloud computing.

At the same time, the SCinet team is working to improve the platform and drive innovation in the arena of interconnecting networks. The INDIS workshop, organized by the SCinet team, brings together those developments from the different booths and the show floor network.

The program for INDIS can be found by clicking here.

Descriptions of the NRE demos are available here.

Editorial Note: Mary Hester was instrumental in this article.

Simplifying the World’s Most Powerful Computer Network with SDN

For the first time, SC’s research and production network (SCinet) will be using the emerging technology software defined networking (SDN) to manage and simplify the operations for a portion of the SC conference’s show floor network.

SCinet is the research and production network that serves as the backbone of data communications for the annual SC Conference. By using SDN in the SCinet network, the group of network engineers deploying SCinet will be able to transfer the task of configuring individual network switching devices to a single piece of software, removing human error from the process of setting up connections within the network.

Nick Buraglio, SDN Project Lead

“Take the last three problems or errors that have occurred on a network of any notable size,” says Nick Buraglio, network engineer at the U.S. Department of Energy’s Energy Sciences Network (ESnet) and lead of SCinet’s SDN project, “it’s almost always a configuration problem—some kind of human error that caused those issues.”

As the pilot year for this SDN project, half of SCinet’s circa one hundred, one-gigabit booth connections will be SDN configured. This technology will simplify managing these network connections and will hopefully reduce the time engineers spend troubleshooting configuration and provisioning errors.

“Previously SCinet had to create its own configurations for devices which required a unique configuration template for each device along a path. Using SDN, we can take all the idiosyncrasies of various devices into account and configure these devices using a single piece of software,” said J.P. Velders, a network engineer from the University of Amsterdam and co-chair of SCinet’s routing team.

Although still an emerging technology, SCinet anticipates that SDN will increase efficiencies. This network provides a unique platform for running high performance computing (HPC) applications and demonstrations at the SC conference, which is in its 27th year. SCinet, built by more than 100 volunteers from academia, industry and government, has been providing the network connectivity and platform for HPC research at SC since 1991.

From left: SCinet volunteers J.P. Velders, Paul Wefel, Conan Moore and Davey Wheeler (SCinet Chair)

“SDN allows the participating vendors to showcase the programmability of their equipment in a production environment.  In addition to the enhancements to the SCinet architecture itself, the use of SDN helps promote the practical usability of the technology to conference participants and the network industry as a whole, “ says Conan Moore, a network engineer from the University of Colorado Boulder and co-chair of SCinet’s routing team.

To implement SDN, SCinet is partnering with Brocade, a network technology company. Brocade is providing both the routing and switching equipment, and the Controller software, which uses OpenFlow, the feature that enables SCinet to deploy SDN across multiple devices.

The SCinet team used OpenFlow in 2014 with Big Switch Networks technology for cybersecurity applications.  “Last year we monitored network traffic for security events using SDN to control the security monitoring infrastructure,” Buraglio said.

While SDN will be isolated to part of the SCinet network this year, Velders anticipates that the technology will span the entire network by 2016.

“Our goal is to scale up the use of SDN to the entire network which will save time when having to configure several hundreds of booth connections and their associated circuits. It will allow us to become more agile and more flexible to accommodate changes and requests,” Velders said.

Brief Q&A with SC15 SCinet Chair David Wheeler

The following is a brief Q&A with David Wheeler, SC15 SCinet Chair from the National Center for

David Wheeler, SC15 SCinet Chair

Supercomputing Applications (NCSA) at the University of Illinois where he serves as a Senior Network Engineer.

Describe your day job:
One of the responsibilities of our team is to architect the HPC system networking.  We evaluate current and future capabilities to ensure high-performing network performance for our HPC environment.  We also collaborate with the University of Illinois to support our local researchers and serve as members of the ICCN (Inter Campus Communications Network), which provides network services around the state of Illinois for the three main University of Illinois campuses.

How/why did you become involved in SC?
I started working for NCSA in 1998 and part of my hiring package was to participate in building SCinet for SC98 in Orlando.  Little did I know what that would fully entail, but with that opportunity, I was exposed to the extent of work required to bring the network together.  My first job that year was to help lay the fiber in the exhibition hall, which catered to my detailed experimentalist nature.  In following years, I’ve also lead the SCinet Wireless Team, IT Team, and eventually the SCinet management team.

What makes SCinet unique or special?
SCinet is the fastest network in the world created by volunteers. We build it to be the fastest network in order to support all the demos, applications, and cutting-edge technology that is displayed every year during SC.  Additionally, the “fastest” network speed grows rapidly from year to year. In 2013, SCinet brought in 885 Gigabits per second of bandwidth for SC13, and in 2014, SCinet brought in 1.3 Terabits per seconds for SC14. (That’s 1.3 trillion bits of data per second—more than 100,000 times faster than an average home network speed.)

Celebrating its 25th anniversary, SCinet is also a research and education network, or R&E network. This means that in designing and deploying this network and technology, our highest priority is serving our community needs (HPC and SC’s needs). It is a mission-driven network supported purely by volunteer efforts and contributions. It is more than a “customer-client service” relationship, it’s about what can we do to push the limits. How can we build and supply the network for our community? How can we be innovative in providing a high-performance network?

SCinet is also a unique forum where volunteers come from industry, academia, and government share practices and expertise as well as try out the latest research and technologies.

What are the biggest challenges facing SCinet this year?
Building SCinet is an amazing opportunity for the SCinet team but does provide its challenges—there’s no manual for this kind of work. However that is also part of the fun. SCinet is able to test and operate the newest technologies that a lot of the enterprise networks will not see for years.

SCinet is always pushing the limits of network innovation and operations. This year SCinet is provisioning production Software-Defined Networking (SDN) circuits to the exhibit floor booths, providing smarter, dynamically manageable connections. We are very excited to see the turn out for this project.

Describe the planning required to pull off SCinet:
SCinet has an amazing amount of dedicated volunteers and vendors every year who work very hard to make SCinet come to life for the SC conference. We have more than 15 teams that comprise all of SCinet and more than 10 operational teams focusing on all aspects of setting up a network from power to cybersecurity to laying the fiber cables.

It takes a full year to plan SCinet, one week to set it up, one week to operate it, and 24-hours to tear it down. The planning starts in force for the next year the same day as tear down for the previous year. In addition to the exploratory planning that started two years before SC15, we started aggressively planning for SC15 on November 21, 2014—the very last day of SC14.

Describe the technical hardware involved (type/length of cable, computing power, etc):
SCinet will install 80-90 miles of fiber within the Austin Convention Center.  With SCinet supporting more than 10,000 attendees and 330 booths, we expect to need 15 tons of cooling and around 30KW of power at 480V.  We will again supply more than 1 Terabit of bandwidth, donated from vendors and neighboring research and education networks. 

How many volunteers and hours go into SCinet?
SCinet has more than 120 volunteers from academia, government, and industry from over 50 different countries from year to year. Counting the hours is much more difficult though. SCinet’s volunteers are driven and dedicated. Often times we will pull many late nights and do whatever we can to make sure things get done in support of the exhibitors, demos, and attendees and I can’t really put an hour amount on this effort—just a huge amount of gratitude for everyone’s hard work.

How does someone get involved in SCinet for future conferences?
We are always looking for new volunteers. We have information about joining SCinet on the SC15 website. But mainly sending an email with a statement of interest is what it comes down to.

Most of the volunteers are supported by their various organizations that contribute volunteer hours and/or equipment for SCinet. We are greatly appreciative of this and try to widely recognize our community and vendors for contributions through the SCinet Contributor program.

How do you balance heading up this impressive task with your day job and personal life?
That is truly a challenge in taking on this opportunity. I am always thankful for the family I’m blessed with and keep them as a priority.  Additionally I strive to be a model for my children and teach them that hard work is rewarding. Being a volunteer on SCinet and chairing the design of this network is certainly challenging. It is also motivating to see where efforts in my day job overlap and are enhanced by my involvement in SC and SCinet—I would not be able to serve in this role if I didn’t have such great institutional support.

I also have to thank the previous SCinet chairs, including Bill Kramer, for their guidance. One main takeaway was to get at least one thing done for SCinet every day.

I’ve been encircled by many wonderful colleagues and friends who are striving to see SCinet succeed in providing the most capable network set up in a short amount of time.

In order to say that you had a successful SC15, what do you need to happen?
SCinet really relies on its volunteers and vendor contributors to make everything happen and to have a successful year. We would not be able to do the amazing amount of work in deploying this network without everyone’s efforts and contributions. If we continue to have the momentum that we have every year, this will be a successful show and we will continue to push the boundaries and expectations for high-performance networking as always.