Installation

System Requirements

  1. Supported Operating Systems

    • Red Hat Linux 7.X
    • Red Hat Linux 8.X
    • Centos 7.X
    • Oracle Linux 8.X - Unbreakable Enterprise Kernel (UEK)
    • Centos Stream 7.X, 8.X
    • AlmaLinux 8.X
    • RockyLinux 8.X
  2. Supported Web Browsers

    • Google Chrome
    • Mozilla Firefox
    • Opera
    • Microsoft Edge
  3. Network communication

    From To Port Protocol Description
    Siem Agent Siem service 1514 TCP (default) Agent connection service
    1514 UDP (optional) Agent connection service (disabled by default)
    1515 TCP Agent enrollment service
    Siem service 1516 TCP Siem cluster daemon
    Source **** UDP (default) Siem Syslog collector (disabled by default)
    **** TCP (optional) Siem Syslog collector (disabled by default)
    Siem service 55000 TCP Siem server RESTful API
    Every ELS component Data Node 9200 TCP License verification through License Service
    Integration source 9200 TCP Data Node API
    Other cluster nodes 9300 TCP Data Node transport
    User browser Logserver GUI 5601 TCP Default GUI
    5602 TCP Admin console
    5603 TCP Wiki GUI
    GUI License Service 9000 TCP Manage files, services and pipelines.

Installation method

The ITRS Log Analytics installer is delivered as:

  • RPM package itrs-log-analytics-data-node and itrs-log-analytics-client-node,
  • “install.sh” installation script

Interactive installation using “install.sh”

The ITRS Log Analytics comes with simple installation script called install.sh. It is designed to facilitate the installation and deployment process of our product. After running (execute) the script, it will detect supported distribution and by default it will ask incl. about the components we want to install. The script is located in the "install"directory.

The installation process:

  • unpack the archive containing the installer tar xjf itrs-log-analytics-${product-version}.x.x86_64.tar.bz2
  • unpack the archive containing the SIEM installer (only in SIEM plan) tar xjf itrs-log-analytics-siem-plan-${product-version}.x.x86_64.tar.bz2
  • copy license to installation directory cp es_*.* install/
  • go to the installation directory (you can run install.sh script from any location)
  • run installation script with interactive install command ./install.sh -i

During interactive installation you will be ask about following tasks:

  • install & configure Network Probe with custom ITRS Log Analytics Configuration - like Beats, Syslog, Blacklist, Netflow, Winrm, Logtrail, OP5, etc;
  • install the ITRS Log Analytics Client Node, as well as the other client-node dependencies;
  • install the ITRS Log Analytics Data Node, as well as the other data-node dependencies;
  • load the ITRS Log Analytics custom dashboards, alerts and configs;

Non-interactive installation mode using “install.sh”

With the help of an install script, installation is possible without questions that require user interaction, which can be helpful with automatic deployment. In this case, you should provide options which components (data, client node) should be installed.

Example:

./install.sh -n -d - will install only data node components.

./install.sh -n -c -d - will install both - data and client node components.

Check cluster/indices status and Data Node version

Invoke curl command to check the status of Data Node:

    curl -s -u $CREDENTIAL localhost:9200/_cluster/health?pretty

    {
      "cluster_name" : "logserver",
      "status" : "green",
      "timed_out" : false,
      "number_of_nodes" : 1,
      "number_of_data_nodes" : 1,
      "active_primary_shards" : 25,
      "active_shards" : 25,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 0,
      "delayed_unassigned_shards" : 0,
      "number_of_pending_tasks" : 0,
      "number_of_in_flight_fetch" : 0,
      "task_max_waiting_in_queue_millis" : 0,
      "active_shards_percent_as_number" : 100.0
    }
    curl -s -u $CREDENTIAL localhost:9200

    {
      "name" : "node-1",
      "cluster_name" : "logserver",
      "cluster_uuid" : "igrASEDRRamyQgy-zJRSfg",
      "version" : {
        "number" : "7.3.2",
        "build_flavor" : "oss",
        "build_type" : "rpm",
        "build_hash" : "1c1faf1",
        "build_date" : "2019-09-06T14:40:30.409026Z",
        "build_snapshot" : false,
        "lucene_version" : "8.1.0",
        "minimum_wire_compatibility_version" : "6.8.0",
        "minimum_index_compatibility_version" : "6.0.0-beta1"
      },
      "tagline" : "You Know, for Search"
    }

If everything went correctly, we should see 100% allocated shards in cluster health.

Generating basic system information report

The install.sh script also contains functions for collecting basic information about the system environment - such information can be helpful in the support process or troubleshooting. Note that you can redirect output (STDOUT) to external file.

Example:

./install.sh -s > system_report.txt

“install.sh” command list

Run install.sh --help to see information about builtin commands and options.

Usage: install.sh {COMMAND} {OPTIONS}

COMMAND is one of:
    -i|install                  Run ITRS Log Analytics installation wizard.
    -n|noninteractive           Run ITRS Log Analytics installation in non interactive mode.
    -u|upgrade                  Update ITRS Log Analytics packages.
    -s|systeminfo               Get basic system information report.

OPTIONS if one of:
    -v|--verbose                Run commands with verbose flag.
    -d|--data                   Select data node installation for   non interactive mode.
    -c|--client                 Select client node installation for non interactive mode.

Post installation steps

  • configure Data Node cluster settings

    vi /etc/logserver/logserver.yml
    
    • add all IPs of Data Nodes in the following directive:

      discovery.seed_hosts: [ "172.10.0.1:9300", "172.10.0.2:9300" ]
      
  • start Data Node service with systemctl start command

  • Example:

    systemctl start logserver
    
  • start Network Probe service with systemctl start command

  • start Cerebro service with systemctl start command

  • start Logserver GUI service with systemctl start command

  • Example: ```bash systemctl start logserver-gui

    
    
  • start Alert service with systemctl start command

  • Example:

    systemctl start alert
    
  • start Skimmer service with systemctl start command

  • Example:

    systemctl start skimmer
    
  • Example agent configuration files and additional documentation can be found in the Agents directory:

    • filebeat
    • winlogbeat
    • op5 naemon logs
    • op5 perf_data
  • For blacklist creation, you can use crontab or Logserver GUI scheduler, but the most preferable method is Network Probe input. Instructions to set it up can be found at logserver-probe/lists/README.md

  • It is recomended to make small backup of system indices - copy “configuration-backup.sh” script from Agents directory to desired location, and change backupPath= to desired location. Then set up a crontab:

    0 1 * * * /path/to/script/configuration-backup.sh
    
  • Redirect Logserver GUI port 5601/TCP to 443/TCP

    firewall-cmd --zone=public --add-masquerade --permanent
    firewall-cmd --zone=public --add-forward-port=port=443:proto=tcp:toport=5601 --permanent
    firewall-cmd --reload
    

    # NOTE: Logserver GUI on 443 tcp port without redirection needs additional permissions:

    setcap 'CAP_NET_BIND_SERVICE=+eip' /usr/share/logserver-gui/node/bin/node
    
  • Cookie TTL and Cookie Keep Alive - for better work comfort, you can set two new variables in the Logserver GUI configuration .yml file /etc/logserver-gui/:

    login.cookiettl: 10
    login.cookieKeepAlive: true
    

    CookieTTL is the value in minutes of the cookie’s lifetime. The cookieKeepAlive renews this time with every valid query made by browser clicks.

    After saving changes in the configuration file, you must restart the logserver GUI service with systemctl restart command

Scheduling bad IP lists update

Requirements:

  • Make sure you have Logserver 7.0 or newer.
  • Enter your credentials into scripts: misp_threat_lists.sh

To update bad reputation lists and to create .blacklists index, you have to run misp_threat_lists.sh script (best is to put in schedule).

  1. This can be done in cron (host with Network Probe installed) in /etc/crontab:

    0 2 * * * user /etc/logserver-probe/lists/bin/misp_threat_lists.sh
    
  2. Or with Logserver GUI Scheduller app (only if Network Probe is running on the same host).

    • Prepare script path:
    /bin/ln -sfn /etc/logserver-probe/lists/bin /opt/ai/bin/lists
    chown user:group /etc/logserver-probe/lists/
    chmod g+w /etc/logserver-probe/lists/
    
    • Log in to Logserver GUI and go to Scheduler app. Set it up with below options and push “Submit” button:
    Name:           MispThreatList
    Cron pattern:   0 2 * * *
    Command:        lists/misp_threat_lists.sh
    Category:       network-probe
    
  3. After a couple of minutes check for blacklists index:

    curl -sS -u logserver:logserver -XGET '127.0.0.1:9200/_cat/indices/.blacklists?s=index&v'
    
    health status index       uuid                   pri rep docs.count docs.deleted store.size pri.store.size
    green  open   .blacklists Mld2Qe2bSRuk2VyKm-KoGg   1   0      76549            0      4.7mb          4.7mb
    

Web Application Firewall requriments

The ITRS Log Analytics GUI requires the following request parameters to be allowed in WAF:

  • URI Length: 2048 characters,
  • Cookie Number In Request: 16,
  • Header Number In Request: 50,
  • Request Header Name Length: 1024 characters,
  • Request Header Value Length: 4096 characters,
  • URL Parameter Name Length: 1024 characters,
  • URL Parameter Value Length: 4096 characters,
  • Request Header Length: 8192 bytes,
  • Request Body Length: 67108864 bytes.

Docker support

To get system cluster up and running in Docker, you can use Docker Compose.

Sample a docker-compose.yml file:

version: '7.1.0'
services:
  itrs-log-analytics-client-node:
    image: docker.emca.pl/itrs-log-analytics-client-node:7.1.0
    container_name: itrs-log-analytics-client-node
    environment:
      - node.name=itrs-log-analytics-client-node
      - cluster.name=logserver
      - discovery.seed_hosts=itrs-log-analytics-client-node,itrs-log-analytics-data-node,itrs-log-analytics-collector-node
      - cluster.initial_master_nodes=itrs-log-analytics-client-node,itrs-log-analytics-data-node,itrs-log-analytics-collector-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/logserver/data
    ports:
      - 9200:9200
    networks:
      - logserver
  itrs-log-analytics-data-node:
    image: docker.emca.pl/itrs-log-analytics-client-node:7.1.0
    container_name: itrs-log-analytics-data-node
    environment:
      - node.name=itrs-log-analytics-data-node
      - cluster.name=logserver
      - discovery.seed_hosts=itrs-log-analytics-client-node,itrs-log-analytics-data-node,itrs-log-analytics-collector-node
      - cluster.initial_master_nodes=itrs-log-analytics-client-node,itrs-log-analytics-data-node,itrs-log-analytics-collector-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data02:/usr/share/logserver/data
    networks:
      - logserver
  itrs-log-analytics-collector-node:
    image: docker.emca.pl/itrs-log-analytics-collector-node:7.1.0
    container_name: itrs-log-analytics-collector-node
    environment:
      - node.name=itrs-log-analytics-collector-node
      - cluster.name=logserver
      - discovery.seed_hosts=itrs-log-analytics-client-node,itrs-log-analytics-data-node,itrs-log-analytics-collector-node
      - cluster.initial_master_nodes=itrs-log-analytics-client-node,itrs-log-analytics-data-node,itrs-log-analytics-collector-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data03:/usr/share/logserver/data
    networks:
      - logserver

volumes:
  data01:
    driver: local
  data02:
    driver: local
  data03:
    driver: local

networks:
  logserver:
    driver: bridge