Latest Architecture Minutes from Bill

Bettina Kauth, DFN

Tony Wildish, Princeton

Lars Fischer, NORDUNet

Bruno Hoeft, KIT

Roy Hockett, UMich

Artur Barczyk, Caltech

Ian Gable, University of Victoria/HEPnet

John MacAuley, ESnet

Shawn McKee, U. Mich.

Magnus Bergroth, NORDUNet

Bob Fischer, NORDUNet

Joe Mambretti, StarLight


And many others that I missed. Please send me your names if you want to be listed in the minutes


1       Sites and initial P2P connection experiments

The call was devoted to discussing Artur’s presentation of a proposed framework for CMS P2P site experiments.

Shawn McKee will report for ATLAS after discussions at an ATLAS meeting this week.

The elements of ARTUR’s proposal are:

·      Use AutoGOLE as a core WAN P2P service

·      Define the Service Termination Points (STPs) for each site

·      There are several possibilities for the STPs

1.     Termination could be on a system inside the site boundary

-     this implies that the site has an NSI Connection Service implemented with a Network Resource Manager that manages the network infrastructure to provide the service

-       a NSI Network Services Agent that acts as a Provider Agent to accept requests from a NSA Requester Agent and interacts with the NRM to set up the circuit

2.     Termination could be on a fixed VLAN at the site boundary that proxies for – connects to – the application system

-     The site’s upstream provider (NREN, RON, etc.) must be capable of setting up a P2P circuit to this STP

-     Site infrastructure for the experiment is mostly limited to maintaining the VLAN – application system mapping

3.     Termination could be on a router that routes for, e.g., a site subnet that has the target resources

-     In this case the STP is like a site VLAN, but the data-plane connection issues are considerable (See Magnus Bergroth comments, below)

4.     Termination could be at an NSI-enabled Exchange Point

-     This implies that there is a static VLAN from the end system to the site boundary, through the upstream provider to an NSI-enabled Exchange Point

-     Site infrastructure for the experiment is mostly limited to maintaining the VLAN – application system mapping

-     Upstream provider infrastructure is mostly limited to maintaining the VLAN – site mapping

·      Once the STPs are connected (typically at L2) a data-plane connection needs to be made

  • The issues here seem to revolve around the issues of assigning IP addresses to the STPs

-     Magnus Bergroth’s email comments characterize the issues:

If the p2p layer2 circuit can be delivered directly to FDT Transfer nodes, then these nodes knows that they are going to be directly connected to each other as they, or the PhEDEx node, made the request to be connected. In that case they only need to find a set of ip addresses that do not interfere with any other resources at any of the sites. Preferable would the site requesting the p2p circuit also assign a set of unique ip-addresses that would be used for the communication over the p2p link.

NDGF is built up with multiple smaller sites that are connected with routers. I that case the p2p circuit would be terminated on one of the central routers, and as I understood from the call that would be one way to connect.

But I can't find a good way to configure the routers interface connected to the p2p service. The router needs to know what ip-address is on the other side and what to reach over that link. There is today to my knowledge no way to share layer3 information with NSI, that could be used.

If the p2p link is going to be used for a long time, like months, then the information could be configured statically on the router. But without and dynamic protocol or link detection there is a big potential that traffic will be sent out in a blackhole, when the p2p circuit is not configured.

I am very interested if someone has a any thoughts on how you could solve the problem on dynamically connect unknown routers together in different ASs. If it was in my domain I could just use unnumbered links and run OSPF that would form adjacency when the links comes up.

·      Accessing the NSI control plane from the user application

  • A PhEDEx site agent capable of negotiating with an NSI User Agent / Aggregrator is in development

-     Tony Wildish estimates 2/2015 for availability

  • The PhEDEx server (FDT) will use the P2P circuit if available, otherwise it fails over to the General Internet connection


·      All agreed that developing and deploying a monitoring mechanism for the P2P circuits was critical

2       Use of AutoGOLE for LHCONE P2P experiments

Gerben van Malenstein will send out his proposal for the timing of an AutoGOLE / LHCONE experiment.



Appendix 1    The P2P experiment participants as currently understood

Project leads:

Harvey Newman <>

Phil DeMar Fermilab <>

Michael Ernst Brookhaven <>

Shawn McKee UMich <>

Kim Kramaric <>

Tangui Coulouarn GEANT <>

Inder Monga ESnet <>

Dale Finkelson <>

Joe J Mambretti <>

Gerben van Malenstein <>


Edoardo Martelli <>

Roy Hockett UMich <>

Magnus Bergroth <>

Gerd Behrmann NeIC <>

Ramiro Voicu USLHCNet <>

Chin Guok ESnet <>

John Bigrow

Azher Mughal

Bettina Kauth <>

Submitted by David Foster on Mon, 10/27/2014 - 08:59