3. Distributed deployment example

Deploying FRED on multiple servers brings at least two advantages:

• increased performance,

• access control on the network level.

Deploying on multiple physical servers is not the only distributed solution, deploying on virtual servers or separating tasks on the process level is also possible.

Nodes overview

Nodes in this document represent execution environments.

We work with the following nodes:

• EPP node – EPP service

• ADMIN node – web admin service

• WEB node – public web services: Unix WHOIS, Web WHOIS, RDAP, Domain Browser [1]

• HM node – zone management

• APP node – application servers, CLI admin tools, pgbouncer, CORBA naming service

• DB node – the main FRED database

• LOGDB node – the logger database

• MOJEID node [1] – MojeID service and database

[1] (1,2)

MojeID and Domain Browser are CZ-specific extensions, and they are not described further.

Note

Hardware parameters background

The hardware parameters described further are minimum requirements for a Registry of about 1 million domains and with this approximate traffic:

• EPP:

  • write operations: ~ 3,5 million / month (they increase the size of both databases)

  • read-only operations: ~ 30 million / month (they increase the size of logdb)

• WHOIS: 15 million operations / month (they increase the size of logdb)

WHOIS includes both Unix and Web variant, and RDAP, too.

Tip

Redundancy

This text does not describe redundancy options in detail, but here is a quick tip:

• database replication is a standard technique to protect data,

• the whole system can be replicated in several instances on different localities, which can substitute one another when one instance fails or during a system upgrade.

3.1. Network

Network rules are described per node in the following sections, but here is an overview of logical connections in the network (a single instance of the system).

Network – Logical topology

Restricted network access means that servers should be accessed only from IP addresses allowed on a firewall.

Unrestricted network access means that servers can be accessed from any IP address, but only necessary ports should be open for access as illustrated in the network rules for each node.

The port numbers mentioned in the network rules are settings resulting from the default installation.

3.2. EPP node

Services: EPP service

Packages:

• libapache2-mod-corba

• libapache2-mod-eppd

Hardware parameters (see the background):

• CPU: @2.0 GHz, 10 cores

• Memory: 16 GB–32 GB

• Storage: 200 GB

Network:

• access to EPP (tcp, port 700) permitted only from particular IP addresses (or ranges) declared by registrars

Network rules for CORBA clients on the EPP node

Service	Description		Server	Protocol /Port	Service	Description
apache2 mod-eppd	registrar interface/epp service	→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2224	fred-rifd	FRED registrar interface daemon
		→	corba	tcp/2226	fred-logd	FRED logging daemon

3.3. ADMIN node

Services: WebAdmin service

Packages:

• fred-common

• fred-idl

• fred-pyfco

• fred-pylogger

• fred-webadmin

Hardware parameters (see the background):

• CPU: @2.0 GHz, 10 cores

• Memory: 16 GB–32 GB

• Storage: 200 GB

Network:

• access to HTTPS (tcp, port 443) permitted only from the private network of the Registry

Network rules for CORBA clients on the ADMIN node

Service	Description		Server	Protocol /Port	Service	Description
webadmin/daphne	web based registry administration	→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2222	fred-adifd	FRED administration interface daemon
		→	corba	tcp/2228	fred-msgd	FRED messaging daemon
		→	corba	tcp/2234	fred-rsifd	FRED registry record statement daemon
		→	corba	tcp/2226	fred-logd	FRED logging daemon
		→	corba	tcp/2225	fred-pyfred@mailer	FRED pyfred service – mailer module
		→	corba	tcp/2232	fred-pyfred@filemanager	FRED pyfred service – filemanager module

3.4. WEB node

Services: Unix WHOIS, Web WHOIS, RDAP

Packages:

• fred-idl

• fred-pyfco

• fred-pylogger

• fred-rdap

• fred-webwhois

• libapache2-mod-corba

• libapache2-mod-whoisd

Hardware parameters (see the background):

• CPU: @2.0 GHz, 10 cores

• Memory: 16 GB–32 GB

• Storage: 200 GB

Network:

• access to HTTPS (tcp, port 443) permitted from anyone

• access to WHOIS (tcp, port 43) permitted from anyone

Network rules for CORBA clients on the WEB node

Service	Description		Server	Protocol /Port	Service	Description
apache2 mod-whoisd	unix whois service (rfc 3912)	→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2223	fred-pifd	FRED public interface daemon
		→	corba	tcp/2226	fred-logd	FRED logging daemon
nginx	web whois service	→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2223	fred-pifd	FRED public interface daemon
		→	corba	tcp/2234	fred-rsifd	FRED registry record statement daemon
		→	corba	tcp/2226	fred-logd	FRED logging daemon
nginx	rdap service	→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2223	fred-pifd	FRED public interface daemon
		→	corba	tcp/2226	fred-logd	FRED logging daemon

3.5. HM node

Hidden master for the DNS infrastructure.

Services: zone file generation, zone signing, notifying DNS servers

Packages:

• fred-idl

• pyfred-genzone

• python-pyfred

Hardware parameters (see the background):

• CPU: @2.0 GHz, 10 cores

• Memory: 16 GB–32 GB

• Storage: 200 GB

Network:

• access to IXFR (tcp, port 53) permitted only from DNS servers

Network rules for CORBA clients on the HM node

Service	Description		Server	Protocol /Port	Service	Description
genzone-client	zone file generator	→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2231	fred-pyfred@genzone	FRED pyfred service – genzone module

3.6. APP node

Services:

• CORBA naming service (omninames) as a virtual server “corba”,

• backend application servers,

• CLI administration tools,

• pgbouncer – prepares and recycles database connections to decrease overhead costs

Packages:

• cdnskey-scanner

• fred-akm

• fred-common

• fred-doc2pdf

• fred-idl

• fred-logger-maintenance

• fred-server: fred-adifd, fred-akmd, fred-logd, fred-pifd, fred-rifd, fred-rsifd

• fred-transproc

• python-pyfred, fred-pyfred, pyfred-filemanager

Hardware parameters (see the background):

• CPU: @2.0 GHz, 10 cores

• Memory: 16 GB–32 GB

• Storage: 400 GB

  Note

  Consider that the storage will contain files managed by the FRED File Manager.

Network:

• only internal access from the private network of the Registry

Network rules for CORBA clients on the APP node

Service	Description		Server	Protocol /Port	Service	Description
fred-akm	fred-akm	→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2233	fred-akmd	FRED AKM interface daemon
		→	corba	tcp/2225	fred-pyfred@mailer	FRED pyfred service - mailer module
fred-admin	fred-admin	→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2224	fred-rifd	FRED registrar interface daemon
		→	corba	tcp/2232	fred-pyfred@filemanager	FRED pyfred service – filemanager module
		→	corba	tcp/2225	fred-pyfred@mailer	FRED pyfred service – mailer module

Network rules for CORBA servers on the APP node

Service	Description		Server	Protocol /Port	Service	Description
fred-logd	FRED logging daemon	→	localhost	tcp/5432	pgbouncer	connection pooler for PostgreSQL
fred-rifd	FRED registrar interface daemon	→	localhost	tcp/5432	pgbouncer	connection pooler for PostgreSQL
		→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2225	fred-pyfred@mailer	FRED pyfred service – mailer module
		→	corba	tcp/2229	fred-pyfred@techcheck	FRED pyfred service – techcheck module
fred-akmd	FRED AKM interface daemon	→	localhost	tcp/5432	pgbouncer	connection pooler for PostgreSQL
		→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2226	fred-logd	FRED logging daemon daemon
fred-adifd	FRED administration interface daemon	→	localhost	tcp/5432	pgbouncer	connection pooler for PostgreSQL
		→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2226	fred-logd	FRED logging daemon daemon
		→	corba	tcp/2225	fred-pyfred@mailer	FRED pyfred service – mailer module
fred-msgd	FRED messaging daemon	→	localhost	tcp/5432	pgbouncer	connection pooler for PostgreSQL
		→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2232	fred-pyfred@filemanager	FRED pyfred service – filemanager module
fred-pifd	FRED public interface daemon	→	localhost	tcp/5432	pgbouncer	connection pooler for PostgreSQL
		→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2226	fred-logd	FRED logging daemon daemon
		→	corba	tcp/2225	fred-pyfred@mailer	FRED pyfred service – mailer module
		→	corba	tcp/2232	fred-pyfred@filemanager	FRED pyfred service – filemanager module
fred-rsifd	FRED	→	localhost	tcp/5432	pgbouncer	connection pooler for PostgreSQL
		→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2225	fred-pyfred@mailer	FRED pyfred service – mailer module
		→	corba	tcp/2232	fred-pyfred@filemanager	FRED pyfred service – filemanager module
fred-pyfred@genzone	FRED pyfred service – genzone module	→	localhost	tcp/5432	pgbouncer	connection pooler for PostgreSQL
fred-pyfred@mailer	FRED pyfred service – mailer module	→	localhost	tcp/5432	pgbouncer	connection pooler for PostgreSQL
		→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2232	fred-pyfred@filemanager	FRED pyfred service – filemanager module
fred-pyfred@filemanager	FRED pyfred service – filemanager module	→	localhost	tcp/5432	pgbouncer):	connection pooler for PostgreSQL
fred-pyfred@techcheck	FRED pyfred service – techcheck module	→	localhost	tcp/5432	pgbouncer	connection pooler for PostgreSQL
		→	corba	tcp/2809	omninames	OmniORB Interoperable Naming Service
		→	corba	tcp/2225	fred-pyfred@mailer	FRED pyfred service – mailer module

3.7. Database nodes

Database is separated into two nodes:

• DB – the main database freddb – data of all domains, contacts, registrars, history etc.

• LOGDB – the audit log (logger) database logdb – logging of all user transactions

We have the logger database separately due to high workload.

Packages:

• fred-db

Hardware parameters (see the background) – DB:

• CPU: 2x @2.0 GHz, at least 10 cores per CPU

• Memory: 32 GB–64 GB

  Note

  Consider that, ideally, this whole database should fit into the memory, which is possible only till a certain number of objects though. See also Storage considerations.

• Storage: 400 GB

  Note

  Consider:

  • storage size can be even smaller depending on the size of the database, which depends on the number of objects in the db and registrars’ behaviour (growth of object history),

  • the size of the database after 5-year operation of a registry of 1 million domains can be about 30 GB,

  • extra space for garbage accumulation (before vacuuming), temporary dumps during migrations, and other db maintenance costs.

Hardware parameters (see the background) – LOGDB:

• CPU: 2x @2.0 GHz, at least 10 cores per CPU

• Memory: 32 GB–64 GB

  Note

  Consider this the lowest requirement. This amount of memory might be filled quite soon.

• Storage

  Note

  Consider:

  • how many months of logs are necessary to be kept in the database (the last year? the two last years?) and how much logs can be kept in backups,

  • growth rate of the log records (according to the traffic estimation as described above): EPP ~ 135 GB / month, WHOIS ~ 30 GB / month.

Network:

• accessed only by the backend server(s) from the APP node

3.8. Component deployment diagram

Diagram of FRED components deployed on multiple nodes