X-PUSH Monitoring Server

This service allows operators to monitor and manage X-PUSH through console and script.

How to use monitoring server

Two methods are provided to monitor the operation of the X-PUSH server.

• console (telnet)

• script (batch, shell)

How to use console (telnet)

Connect to telnet

In the command prompt (cmd), type telnet X-PUSH server IP monitoring port (default port: 50003).

Login to telnet

Log in by referring to the id and pw in $XPUSH_HOME/conf/user.properties.

[telnet] show status

• show status : This command retrieves and displays the current server status.

[telnet] show clients

• show clients : This command displays all client information (userId, ip, projectId, topicType, topicId) currently connected to the server.

[telnet] show client [userID]

• show client [userID] : This command displays the client information that matches the entered userID among clients currently connected to the server.

[telnet] show client (userID, projectID, topicType, topicId)

• show client (Enter key) : This command displays detailed search results based on four conditions (userId, projectId, topicType, topicId) among clients currently connected to the server.

[telnet] save clients

• save clients : This command saves all client information (userId, ip, projectId, topicType, topicId) currently connected to the server as a json file.

-> The clientInfo folder is created in $XPUSH_HOME and the json file with the name including the creation time is created inside the folder.

[telnet] help

• help : This command is used when you need help with a command.

How to use script (batch, shell)

• Windows : Run batch script

• Linux : Run shell script

[script] show_status

• show_status : This script is to view the current server status. Go to $XPUSH_HOME/bin and run show_status.bat/sh. When running, the required parameters are as follows.

Windows : show_status.bat X-PUSH Server IP Monitoring port (Default port: 50003) id pw

ex) show_status.bat localhost 50003 tobesoft xpush

Linux : ./show_status.sh X-PUSH Server IP Monitoring port (Default port: 50003) id pw

ex) ./show_status.sh localhost 50003 tobesoft xpush

The following is an example of running.

[script] show_clients

• show_clients : This script displays all client information (userId, ip, projectId, topicType, topicId) currently connected to the server. Go to $XPUSH_HOME/bin and run show_clients.bat/sh. When running, the required parameters are as follows.

Windows : show_clients.bat X-PUSH Server IP Monitoring port (Default port: 50003) id pw

ex) show_clients.bat localhost 50003 tobesoft xpush

Linux : ./show_clients.sh X-PUSH Server IP Monitoring port (Default port: 50003) id pw

ex) ./show_clients.sh localhost 50003 tobesoft xpush

The following is an example of running.

[script] show_client (userID)

• show_client (userID) : This script displays the client information that matches the entered userID among clients currently connected to the server. Go to $XPUSH_HOME/bin and run show_client.bat/sh. When running, the required parameters are as follows.

Windows : show_client.bat X-PUSH Server IP Monitoring port (Default port: 50003) id pw userID

ex) show_client.bat localhost 50003 tobesoft xpush user01

Linux : ./show_client.sh X-PUSH Server IP Monitoring port (Default port: 50003) id pw userID

ex) ./show_client.sh localhost 50003 tobesoft xpush user01

The following is an example of running.

[script] save_clients

• save_clients : This script saves all client information (userId, ip, projectId, topicType, topicId) currently connected to the server as a json file. Go to $XPUSH_HOME/bin and run save_clients.bat/sh. When running, the required parameters are as follows.

Windows : save_clients.bat X-PUSH Server IP Monitoring port (Default port: 50003) id pw

ex) save_clients.bat localhost 50003 tobesoft xpush

Linux : ./save_clients.sh X-PUSH Server IP Monitoring port (Default port: 50003) id pw

ex) ./save_clients.sh localhost 50003 tobesoft xpush

The following is an example of running.

-> The clientInfo folder is created in $XPUSH_HOME and the json file with the name including the creation time is created inside the folder.

Detailed Item Description (show_status)

.module Main X-PUSH Server

RUNTIME

MEMORY

QUEUE

The queue value is a statistic collected at 0.1-second intervals. A total of 10,000 are stored. The most recent 1,000 seconds (approx. 16 minutes) of stats are provided.

Client Servant Container

Container-related delays and tps-related values represent the most recent 10,000 message cases. When individual messages are transmitted to containers and processed, each delay is collected and the respective statistics are calculated.

The items provided per container are as follows:

Socket Push Message Provider

Per Servant

Each message provider's IP address, port, connected accounts, and the number of messages provided are displayed.

Provider Message Queue

Repository Service

QUEUE

It means the number of messages currently waiting / queue size.

Message Process Time(milisec)

It means the time required to process a message by each Queue (Provide, Response, Request, Topic) and the status of the operating thread that processes the corresponding message.

DB Connection Pool

Notification Message Queue

Notification Result Update Queue

Apns Notification Message Queue

FCM Notification Message Queue

ApnsNotifier Service

Message Process Time(milisec)

It means the message processing time of the ApnsQueue’s operating thread and Provider sending data to Apns, and the status of the operating thread that processes the corresponding message.

FcmNotifier Service

Message Process Time(milisec)

It means the message processing time of FcmQueue’s operating thread and Provider sending data to FCM, and the status of the operating thread that processes the corresponding message.

Etc Message Queue

JMX

JConsole Connection

Windows environment setup

1. Run %JAVA_HOME%\bin\jconsole

2. Enter the IP address and port in the [Remote] tab, and click the [Connect] button.

X-PUSH is bound to 0.0.0.0 and run for both Linux and Windows. For Windows, however, the routing information is registered in the routing table under its own IP address for access via the corresponding address. This is not possible for Linux, because the same information is not registered in the routing table by default.

Method 1

Assign the actual IP address instead of "0.0.0.0" under the "ServerBindAddress" section in xpush_conf.xml.

<service name="Publisher">
	...
	<attribute name="ServerBindAddress">192.168.0.1</attribute>
	...
</service>

Method 2

Assign the IP address to the "/etc/hosts" file.

192.168.0.1 localhost.localdomain localhost
::1 localhost6.localdomain localhost6

Item

X-PUSH MBean

ConnectionManagerService

NodeManagerService

PublisherService

RepositoryService

System

Container

Notification

ApnsNotifierService

GcmNotifierService

Provider

Identifying Resource Shortages

Resource shortages for server operation can be identified through the values from show status.

Memory Shortage

Normal operation is not possible when the free heap memory is low. If the amount of free heap memory is less than 10% of the total available, the memory must be expanded, and the settings must be changed. Modify the -Xms and -Xmx values in the "startup.bat / startup.sh" for the memory settings.

startup.bat
set JAVA_OPTS=-Xms1024m –Xmx1024m –server

startup.sh
JAVA_OPTS=”-Xms1024m –Xmx1024m –server $JAVA_OPTS”

Connection Resource Shortage

The maximum number of clients that can connect to an X-PUSH Server is determined by the container number set in "xpush_config.xml" and the number of servants per container.

The default values at the time of deployment are 20 Servant Containers and 50 Servants per container. A total of 1,000 clients are able to connect. (20 containers x 50 servants) Additional client connections are no longer allowed when the Total Client Count reaches the maximum number of connections. In such cases, the settings need to be modified.

CPU/Network Resource Shortage

A Waiting Ratio value greater than 30% signifies that messages are not immediately delivered and are waiting in standby. This indicates a CPU or network resource shortage. In this case, the Average Messages in Queue value is usually large, and the Average Tps is at a low level. CPU and network usage should be monitored using external tools, and the necessary resources should be expanded.

Lack of Database Connection

When DB usage increases, there is no DB Connection to process other messages, resulting in messages to be accumulated in Queue. At this time, the Max Connection of DB can be checked to increase the value in maxActive in X-PUSH xpush_conf.xml

[show status Symptom] 
MODULE : Repository Service
          QUEUE
                    Provide Message Queue Size = 100 / 100
                    Response Message Queue Size = 500 / 20000
                    Request Message Queue Size = 1000 / -1
                    Topic Message Queue Size = 0 / -1
          Message Process Time(milisec)
                    Provide           # : 0 / 0        Avg : 0      Min : 0      Max : 0
                    Response        # : 0 / 0        Avg : 0      Min : 0      Max : 0
                    Request          # : 20 / 20     Avg : 0      Min : 0      Max : 0
                    Topic             # : 0 / 0        Avg : 0      Min : 0      Max : 0

If all 20 Connections are used in Request, messages are accumulated in Queue as other Queues cannot be processed.

<service name="DbcpService">
	...
	<attribute name="maxActive">50</attribute>
	...
</service>