AR-Cluster Server
Version 6: 1 January 2012
I have been consulting and working in developing solutions for several large clients in the area including one of the largest eCommerce web sites in the world. Over the last few years I have learned a lot about building enterprise level solutions and industry best practices. This learning has been applied to this release of AR-Cluster. Do not expect it to work like older version as it’s all new and from the ground up.
The key to selecting a cluster is performance. With the addition of CW Skimmer spots and the Reverse Beacon Network the demands on the DX-Cluster system have increased by orders of magnitude. Existing cluster software has demonstrated scalability issues and software authors have reduced features and/or imposed spot filters just so they can keep up. The approach in AR-Cluster version 6 was to build a multi-threaded application from ground up that was optimized for the high spot volumes. Users can get 100% of the raw spotting data or apply their own filters as needed.
DXers and contesters need to know their cluster application keep up with present day loads and scale to future spot volumes. How fast is it? You do the math, in AR-Cluster, performance metrics are calculated in real time and logged in the app log. Here is a sample app log with execution timing, in seconds. Ex: [0.00081]].
1. Leverages the latest technologies in software development
2. Optimized for fast startup, up and running in 4 seconds taking user connections and DX spots
3. Works with skimmer spots
4. Scalable for future skimmer spot loads. Will run 1 million DX spots per hour @ less that 20 percent CPU
5. New multi-threaded telnet server tested to 2000 telnet login’s
6. All new advanced filtering with special filters for skimmer spots
7. Can be used as a spot aggregator for mixing/filtering skimmer and legacy DX spots for contest stations
8. Multi-threaded - makes use of multi-core processors making things operate much better
9. New spot processing logic that offers huge performance increases
10. Physical databases are automatically taken off-line for maintenance and verified, purged and compacted each night
11. New node to node protocol
12. New logging features allow configuration and pager/email alerts and also display timing of processing and stack trace details in case of a issue
13. Developed using a pluggable architecture allowing that allows expansion by other third party
developers
14. Configuration is maintained in simple XML files
15. New filters allow you to filter by anything in the spot including many new fields added by the software
16. New SH/QSL command that pulls content on-line from the www.qslinfo.de database
17. Third party content is updated automatically each night: LOTW, Cluster list, Master call and CTY
18. The US/VE callsign databases are automatically downloaded, processed and updated every week
19. New graphical plotting features
20. New spot tracking capabilities to reduce cluster abuse
21. Uses a new CTY file version that providing better zone resolution
22. Increased accuracy on Lat/Lon position
23. New contest specific filters
24.
Includes a new AR-Cluster client app for the cluster users.
The quickest way to get up to speed is to view the on-line YouTube videos. Several of the features have changes as the product has matured but it’s a good start to see the operation and setup. This is also a good opportunity to get a feel for the new dockable window feature.
AR-Cluster Server - Setup and Configuration
AR-Cluster Server Release V - Jan 31
AR-Cluster Server Callsign Database Processing - The latest AR-Cluster code downloaded and processes the US/FCC and VE callbooks automatically each week with no sysop intervention.
Installation is done with a standard windows installer. The latest ARC6 Server install can be found at: ftp://ftp.ab5k.net/ArCluster/Version6/LatestArcServer/
The files
are zipped and the password protected.
Run the setup.exe and follow the instructions. Note the installation folder path. When done, copy your license file into the
installation folder path.
If you want to support connections from outside you will need to open up the following port on your firewall:
1.
Port 23: Telnet server port – for new users
defaults to no skimmer spots
2.
Port 7373:
Telnet server port – for new users defaults to skimmer spots
3.
Port 3605: Port where ARC4 active nodes
connect
4.
Port 3606: Port where ARC4 passive nodes
connect
5.
Port
3607: Port where ARC6 nodes connect
6.
Port
3608: Port where ARC6 Client applications connect
7.
Port
9000 : Where PCxx nodes connect
The
first time the ArcServer application runs, a message
box will display indicating missing FCC/VE callsign databases.
The
application will automatically connect to the appropriate web resources and
down, process and place the databases on-line.
The process will take about 5 minutes.
The FCC/VE callsign databases allow for
filtering by states, callsign lookups and extended
information about spots such as Name, ARRL Section, and County. As soon as the callsign
databases are on-line, callsign lookups can be done
using the Tools>Callbook Lookup menu:
The configuration is done using Tools>Setup menu. The configuration is saved into a XML file located at: \Cfg\ArcServerCfg.xml file.
You can open the file and see the same information as displayed in the Config>Setup menu. If you are comfortable in editing XML you can also make configurations changes in the xml file.
While editing if you mess up the file generating non validating XML, you can simply delete the cfg file and restart the app and a stock factory cfg file will be created for you.
When using the Config>Setup editor it’s recommended that you do not use symbols like “>” or “<”. If you use them and ever edit the XML file they will look a bit odd like “>”.
The Telnet server configuration for port 23 looks like:
I/O
connections are configured using the Tools>Setup menu.
There are
two types of I/O connections available:
Server and Client. The Client
I/O allows you to connect to other nodes or skimmer devices. Sample connections are provided to the
Reverse Beacon Network, popular skimmers, IRC channel, ARC6 node connections,
ARC4 Active Node connection, ARC4 Passive Node connection, and other Telnet
connections. Client I/O options are:
|
Field |
Description |
|
ConnectAs |
The call used when you are pulling spots from a connection. You can leave this blank on ARC4 and ARC6
connects as it’s pulled from the license file. |
|
ConnectTo |
Is the call we are connecting to. On the node ARC6 node to node connection
it can left empty as it will fill in during the connect process |
|
Description |
Is for the sysop. It’s the
text that shows on the screen to describe the connection. It can be left blank. |
|
Enabled |
Enables or disabled the connection |
|
IpAddress |
The IP address or URL of the
connection. Ex: 64.128.19.154 or ab5k.net |
|
Password |
Optional password user to access
password protected skimmers |
|
Port |
Port for the connection |
|
Type |
The type of the connection. Note
you should never make more than one Arc4Active connection active. Setting up more than one active connection
will cause loops in the network. |
The server
IO allows other users or nodes to connect to you. Sample connections are provided to setup a
Telnet Server(s), ARC6 Server, ARC4 Active Server, ARC4 Passive Server, and a PCxx Server. Server I/O options
are:
|
Field |
Description |
|
DefaultAnnFilter |
The default filter applied when a new
user logs into the node |
|
DefaultDxFilter |
The default filter applied when a new
user logs into the node |
|
DefaultWxFilter |
The default filter applied when a new
user logs into the node |
|
Description |
Is for the sysop. It’s the
text that shows on the screen to describe the connection. It can be left blank. |
|
Enabled |
Enables or disabled the connection |
|
IntroLoginFile |
Login file that’s played as a user logs
into the node (Location = Cfg/Login) |
|
LoginErrorMessage |
Recommend you use – Please enter a valid
callsign |
|
Port |
Port for the connection |
|
Type |
The type of the connection. |
|
WelcomeLoginFile |
Welcome login file that’s played after
the user has logged into the node
(Location = Cfg/Login) |
Sample I/O connections are included. Most are active and all you have to do is to enable them. The samples can be user or modified as needed for I/O connections.
ArcServer comes configured with a telnet server for user connections setup on port 23. To view the factory configured setup select the Config>Setup menu option. The following dialog will launch.
Expand the IoDevices>Server>List and press the “…” button. On the left side of the screen you can select a Server provider and the current settings will present on the right side of the screen. The details for the telnet server running at port 23 are:
The sysop can configure the telnet server controlling the port as well as messages to the user. See section on Connecting via Telnet to test telnet connections.
You can test a telnet connection by selecting the menu option Connects>MyTelnetServers>23 or you can start a DOS window and type “Telnet localhost”. You should get the login screen as shown below.
Enter a call and press the enter key.
ArcServer can also be used as a spot aggregator to collect spots from various sources and feed a contest station or normal cluster connections. In this case you can make multiple outbound spot connections to gather spots from around the world. In this example we will connect to the K8ND AR-Cluster using a telnet client.
On your ARC6 node select Config>Setup. Expand IoDevices, expand client and select the client list. The configuration dialog will launch. Select Add and fill out the configuration on the right side of the dialog. Fill out the settings as follows:
Connect as: The call you want to connect as (my case AB5K-1)
Connect To: K8ND
Description: Telnet to K8ND
Enabled: True
IpAddress: 70.62.162.18
Password: leave blank
Port: 7300
Type: TelnetClient
Exit by clicking OK and Save. See sample configuration below:
You can restart the app or select Connects>Outbound Disabled and check the K8ND configuration. The connection should display in the Outbound Connections window.
The telnet connection to K8ND will pull in and process DX, ANN, WX and WWV spots. No spots will be sent back to K8ND as suggested by the one way green arrow icon in the treeview.
You can also test by connecting into the telnet server, see section Connecting via Telnet where you should see DX, ANN, WX and WWV spots.
ArcServer can also be used to pull in skimmer spots from the Reverse Beacon Network, RBN, site. To activate just bring up the Disabled Client Configurations and select a connection. The REV1BCN is the recommended connections as it operates using AR-Cluster 6 software. A sample configuration dialog is displayed below.
The REV2BCN site outputs spots in a modified format in that there are extra characters in the comment field. Thus it requires a different parser and a type of TelnetExtComment. You can also test by connecting into the telnet server, see section Connecting via Telnet where you should see skimmer spots. Sample below:
The log file contains detailed information about internal processing. In addition the log file contains timing information on processing spots. In addition it contains information as to errors in processing spot data. The log file also lists the class and method executing when the logging function occurs. This can provide valuable information in case an issue is encountered.
The File>App Log(Docked) menu will launch the BareTail
utility that is used for viewing logs. BareTail will automatically scroll the log file or it will
also allow you to scroll backwards in the logs.
There is also a nice highlighting feature that you can use to highlight
errors and key words.
Here is a sample log with some points highlighted in red numbers from one
to five. The numbers in the brackets are
internal timing code execution. For
example the DX spot received at 03:20:55 took [0.00147] seconds to process.
1. ArcServer started and loaded the CTY cache.
2. A connection to the Rev Bcn network was made
3. DX spots are being processed – four seconds after starting
Log activities fall into categories of:
FATAL – Something very bad happened
ERROR – Something faulted that may or not be bad
WARN – Something happened that you may want to be aware of
INFO – General information
DEBUG – Lines added to aid in debug of the app
If you have issues, please send an edited log (plus/minus 3
minutes) around the time when the issue occurred. The log generally contains
valuable information that points exactly to the area of the code where the
exception occurred.
The sysop can control the threshold on the events that are logged into the log file. While running in real-time, if you open the \Cfg\Libs\ArcServerLog4net.xml file, toward the top of the file you will see a line that looks like:
That line sets the minimum level of events that are logged to the file. For example if you open it and change it from DEBUG to INFO and save the file while monitoring the log file, you will see the number of events logged drop.
The logs are saved in the Logs folder.
The logging is totally configurable. You can set the logging to display as much
info as you want or dial it down to just see real errors. You can even send email alerts on errors. Logging options are setup in the file: Cfg\Libs\log4net.xml
Part of the configuration is done using macros. For example the macro {NODECALL} substitutes to the callsign set in the license file for the node. Here is a list of current macros that can be used in node configuration:
{CRLF} Carriage return and linefeed
{NODECALL} Node callsign {USERNAME} User’s name {USERCALL} User’s call
The main configuration files are setup where they are automatically pulled from the web every night with no intervention by the sysop. The automatic download can be turned off if needed in the configuration file.
There are provisions for local sysop
extensions add additional content for some features like:
The
content configuration can be accomplished while the application is running by
simply editing the appropriate file.
Configuration files are located in the \Cfg\Setup
folder. As soon as the file is saved, ArcServer will automatically detect the change to the file, reload the new
settings and log the reload in the application log.
The
application will automatically take the databases off-line and purges and compacts them each day.
The cluster will continue to run uninterrupted. There is a setting in the configuration (ScheduledMaintenanceHour) that sets the hour when system
maintenance will be done. The factory
default is 08:00Z which is a good time for nodes in North America. System maintenance is done on a low priority
background thread. During system
maintenance content files are also downloaded and updated. All of this is done while the cluster is
running.
On Monday
evenings the system maintenance also includes downloading, processing and
building the US/FCC and VE callbooks databases.
Here is a
typical system maintenance example extracted from the log file. In this case the total time for system
maintenance was 9 seconds.
Bare Tail is installed for monitering of application log files. Bare Tail allows the monitering of multiple
files and it also includes features to highlight lines containing key
text. The highlight feature can be used to
highlight key events in the log like errors or calls that you need to
track. Here is a sample highlight
configuration that highlights key log events in red:
A “Health” window is located on the right
side of the screen. The icon on the
window indicates an aggregated view of the health of the node. In this sample below, the red-cross icon
indicates an issue. The window is normally
hidden and looks like:
If you hover over the Health tab, it will
expand and show the details of the health of the node like:
In this case the health details indicate we have
a good flow of spots but the connected nodes and user fall below the sysop
thresholds set in the Tools>Setup>Health-Monitor.
The monitoring and filtering of spots was previously embedded in the main AR-Cluster application. In the new version 6 code, the monitoring and filtering of spots has been moved into a brand new application called AR-Cluster Client. AR-Cluster Client is a Windows based application that runs under the .Net 4 framework. The AR-Cluster Client application brings a whole new concept to cluster filters allowing filtering on the user’s desktop with simple visual filters that can be adjusted in real-time. The application also includes new docking features that allow the user to drag and dock various windows on his screen.
The AR-Cluster Client application can be given to your users so they can also have access to a rich client application for spot display and filtering. AR-Cluster Client is documented in a separate manual. The AR-Cluster Client app can be downloaded from: ftp://ftp.ab5k.net/ArCluster/Version6/LatestArcClient/
The AR-Cluster Client app can be ran and docked inside AR-Cluster Server by doing a Connect>Arc-Client App.
AR-Cluster version 6 is multi-threaded and
is perfect for processing high volume skimmer spots. The following display shows AR-Cluster
running on a 3 GHz 3- year old Dell quad core box running at a spot rate of
just over 1 million spots per hour. The
CPU utilization is under 20 percent. The
spot rate graph shown on the left is built into AR-Cluster.
A connections window
is available on the left side of the app that details the local node
connections. The window can kept pinned
where it’s always visible or if the pin is released, the window is available is
you hover the mouse over it. The connection icons in the tree-view window
contain small arrows that indicate the direction of the connection (outbound or
inbound). Right clicking on the
tree-view will display a secondary menu with options for Configuration,
Connection and QRZ lookup.
A sample tree-view window is displayed
below:
You may see duplicate calls in the
connection tree-view window. This is
normal as AR-Cluster allows multiple connections with the same callsign.
A new diagnostic/monitor feature was added
to log RX and TX traffic. To activate
I/O traffic logging Rt-Click on the
Connects tree-view select the Connection option and check the Log I/O
Traffic menu option. You can log more
than one I/O connection but the log will get very busy so it’s generally best
to only monitor one I/O connection. To
view the log select the Files>I/O Traffic Log options.
1. View>LocalTelnet menu does not launch - If you have a 64-bit machine and have a issue where the View>LocalTelnet menu does not launch the telnet client, copy the TELNET.EXE from the Window/System 32 folder to the AR-Cluster folder. Technical details at: http://social.msdn.microsoft.com/Forums/en-US/netfxbcl/thread/2ecbd899-711d-4cd0-9285-87d136c95cff
2. Missing Telnet Client - In the newer operating systems you have to enable the telnet client. In WIN7 its Control Panel>Programs and Features>Turn Windows features on or off. This last pick is in the right-hand column.
3. Issue installing to Server 2008, Win7 and Vista (de N4AC) – Make sure you have the files “unblocked” and that the executables are run as Admin. SysInternals has a nifty utility that will let you delete the file streams (which cause the “block” since the files came from another computer). Here’s the URL... http://technet.microsoft.com/en-us/sysinternals/bb897440
Syntax: Streams –s –d <directory name>
You can use a file name if you want to delete the stream from just one file.
I put the streams file into my Windows\System32 directory so that it will be accessible anywhere.
Once I did that, v6 starts up and runs on WinServer 2008 R2.
That utility will work on Vista and Win7, too.
1.
How are
node connections configured?
Everyone that connects to port 3607 is an ARC6 node. All connections are allowed. This keeps the sysop from having to do configuration work like maintaining node connection files. The IpLockout.dat file can be used to control abuse.
2. How can I tell what ports are being used on my machine? From a command prompt, type NETSTAT -NA - That will show you what ports are in use by the OS and any programs.
3.
I’m
having issues running on a 64-bit operating system and NTFS. This issue has been noted on Windows Server 2008 and a Windows 7
Home Premium setup. The distributed
files include streams files. You need
to download this NT SysInternals utility:
http://technet.microsoft.com/en-us/sysinternals/bb897440
Stick it in c:\windows\system32 or somewhere in your path. Then run it like this against where you
installed ARC6. I had to do it for both the Client and the Server:
c:\streams -s -d c:\where you installed\arcserver
path