Getting Rock on RHEL

#43

Hmm… I may need to see if I can get my hands on a RHEL ISO and try the install. Seems like something under the hood is not working in the deploy script. Have you tried running the deploy one more time after being able to manually install those packages?

#44

I will try that again in a few minutes. Heres how to get a RHEL iso to test on:
https://developers.redhat.com/products/rhel/download/

You have to create a RHEL account
I’m running RHEL 7.6

If you want to know exactly what i’ve done before you started helping today let me know and i’ll message you my “cut-sheet”

#45

Thanks for the link, I have an account and it is on my todo list to build RockNSM against RHEL anyway so I will just bump this since I have noticed you have been trying to get this working for almost a week.

#46

Ok, So I got it working. Here are the steps I had to take to get this working on redhat.

  1. Subscribe to RHEL to enable repos.
  2. Install EPEL and dependancies.

sudo rpm -ivh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum install python2-markupsafe git ansible

  1. Clone the ROCK repo.

git clone https://github.com/rocknsm/rock.git

  1. Copy all the components to the correct locations

mkdir -p /usr/share/rock/
mkdir -p /etc/rocknsm
mkdir -p /srv/rocknsm/support
cp -R /home/admin/rock/playbooks/ /usr/share/rock/.
cp -R /home/admin/rock/roles/ /usr/share/rock/.
cp /home/admin/rock/roles/etc/hosts.ini /etc/rocknsm/.

  1. Remove the generate defaults from the site yml.
    vi /usr/share/rock/playbooks/site.yml
#!/usr/bin/env ansible-playbook
- import_playbook: deploy-rock.yml
  1. Generate RockNSM defaults

/home/admin/rock/bin/generate_defaults.sh

  1. Edit the config to use online and ignore local repos
    vi /etc/rocknsm/config.yml
... bunch of lines ...

rock_online_install: True <---- this one

... bunch of lines ...

rock_disable_offline_repo: True <----- this one

... bunch of lines ...
  1. Deploy rocknsm

/home/admin/rock/bin/deploy_rock.sh

  1. This will fail on the CentOS Repo’s, probably a smoother way in the future to get around this.
  2. Edit the Centos repos so that the $releasever is 7
    vi /etc/yum.repos.d/CentOS-Base.repo
[base]
enabled = 1
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
mirrorlist = http://mirrorlist.centos.org/?release=7&arch=$basearch&repo=os&infra=$infra
name = CentOS-$releasever - Base

[updates]
enabled = 1
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
mirrorlist = http://mirrorlist.centos.org/?release=7&arch=$basearch&repo=updates&infra=$infra
name = CentOS-$releasever - Updates

[extras]
enabled = 1
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
mirrorlist = http://mirrorlist.centos.org/?release=7&arch=$basearch&repo=extras&infra=$infra
name = CentOS-$releasever - Extras
  1. I then cleaned up my YUM and cached it which allowed me to bypass having to muck with the playbook.

yum clean all
yum makecache fast

  1. Run deploy again which will successfully install this time.

/home/admin/rock/bin/deploy_rock.sh

  1. ???
  2. Profit

$ sudo rockctl status
ZOOKEEPER:
Active: active (running) since Tue 2019-03-12 03:05:52 UTC; 48min agoi
KAFKA:
Active: active (running) since Tue 2019-03-12 03:06:21 UTC; 47min agoi
BRO:
Active: active (running) since Tue 2019-03-12 03:07:54 UTC; 46min agoi
SURICATA:
Active: active (running) since Tue 2019-03-12 03:10:10 UTC; 43min agoi
FILEBEAT:
Active: active (running) since Tue 2019-03-12 03:11:18 UTC; 42min agoi
ELASTICSEARCH:
Active: active (running) since Tue 2019-03-12 03:24:52 UTC; 29min agoi
LOGSTASH:
Active: active (running) since Tue 2019-03-12 03:27:08 UTC; 26min agoi
KIBANA:
Active: active (running) since Tue 2019-03-12 03:29:28 UTC; 24min agoi
STENOGRAPHER:
Active: active (exited) since Tue 2019-03-12 03:06:42 UTC; 47min agoi
FSF:
Active: active (running) since Tue 2019-03-12 03:11:18 UTC; 42min agoi
DOCKET:
Active: active (running) since Tue 2019-03-12 03:23:25 UTC; 30min agoi
STENOGRAPHER@ENS34:
Active: active (running) since Tue 2019-03-12 03:06:42 UTC; 47min agoi

1 Like
#48

I am testing this now on my box to see if I have any issues. From a hypothetical standpoint how would you replicate this on an air gapped network if you could stand up your own repos?

#49

@Nick it’s possible to override the URLs for the repos so that you can point them to your own mirror. We’re also working on a RHEL ISO version, but we can’t freely distribute it at the moment. We’d like to get approval to do that, but that’s in progress. What we can do is make it easier for others to build their own ISO.

#50

@spartan782 It appears that everything is working. The ELK stack itself failed to to start based on rockctl status but I’m ok with that as I am going to be sending a copy of data to Splunk. I can’t thank you enough for your help and the same goes to @koelslaw

@dcode I sent you a message.

I will test this a few more times on other systems just to make sure everything seems to be working. Should be done in the next day or so and will post any new errors/issues.

#51

Rock on RHEL

Disable FIPS to allow Deployment on all components

If STIG enabled FIPS then here is how to disable it to allow deployment

Remove the dracut-fips* packages

sudo yum remove dracut-fips\*

Backup existing FIPS initramfs

sudo mv -v /boot/initramfs-$(uname -r).img{,.FIPS-bak}

Run dracut to rebuild the initramfs

sudo dracut

Run Grubby

sudo grubby --update-kernel=ALL --remove-args=fips=1

Carefully up date the grub config file setting fips=0

sudo vi /etc/default/grub

Reboot the VM

sudo reboot

Log back in…

Confirm that fips is disabled by

sysctl crypto.fips_enabled

if it returns 0 then it has been properly disabled

Deployment of Rock across All Machines

Mount the rocknsm2-4.iso to /mnt

Copy the folders form the mounted device to /srv/rocknsm

Create the a place for your pet ROCK to live

sudo mkdir -p /usr/share/rock

Git Clone or Copy the rocknsm repo to the /usr/share/rock directory

sudo git clone https://github.com/rocknsm/rock.git

Install Ansible to coordinate the installation of the Sensor

sudo yum install ansible

Ensure the latest version of markupsafe is installed also

sudo yum install python2-markupsafe

Copy the hosts.ini file that so ansible knows where to deploy things using sudo cp /usr/share/rock/etc/rocknsm/hosts.ini /etc/rocknsm/.

NOTE: Most of the Rock configuration is now automated and can be called from anywhere on the os.

[admin@sensor ~]$ sudo rock help
Usage: /sbin/rock COMMAND [options]
Commands:
setup               Launch TUI to configure this host for deployment
tui                 Alias for setup
ssh-config          Configure hosts in inventory to use key-based auth (multinode)
deploy              Deploy selected ROCK components
deploy-offline      Same as deploy --offline (Default ISO behavior)
deploy-online       Same as deploy --online
stop                Stop all ROCK services
start               Start all ROCK services
restart             Restart all ROCK services
status              Report status for all ROCK services
genconfig           Generate default configuration based on current system
destroy             Destroy all ROCK data: indexes, logs, PCAP, i.e. EVERYTHING
                      NOTE: Will not remove any services, just the data

Options:
--config, -c <config_yaml>         Specify full path to configuration overrides
--extra, -e <ansible variables>    Set additional variables as key=value or YAML/JSON passed to ansible-playbook
--help, -h                         Show this usage information
--inventory, -i <inventory_path>   Specify path to Ansible inventory file
--limit <host>                     Specify host to run plays
--list-hosts                       Outputs a list of matching hosts; does not execute anything else
--list-tags                        List all available tags
--list-tasks                       List all tasks that would be executed
--offline, -o                      Deploy ROCK using only local repos (Default ISO behavior)
--online, -O                       Deploy ROCK using online repos
--playbook, -p <playbook_path>     Specify path to Ansible playbook file
--skip-tags <tags>                 Only run plays and tasks whose tags do not match these values
--tags, -t <tags>                  Only run plays and tasks tagged with these values
--verbose, -v                      Increase verbosity of ansible-playbook

Change Directory into usr/share/rock/bin if you are not already there

Run sudo ./rock ssh-config to setup ssh on all the host you will use for the deployment. It uses the host from the previously created hosts.ini if you have a multinode deployment

Run sudo ./rock genconfig to generate config file. Unless you are doing something really off the beaten path of a normal deployment you should not need to edit this file.

Ensure you are in the /usr/share/rock/bin/ directory.

Fire off the installation

sudo ./rock deploy-offline

Ensure the following ports on the firewall are open if you need them to be

  • 9300 TCP - Node coordination (I am sure elastic has abetter name for this)
  • 9200 TCP - Elasticsearch
  • 5601 TCP - Kibna
  • 22 TCP - SSH Access
  • 9092 TCP - Kafka
sudo firewall-cmd --add-port=9300/tcp --permanent

Reload the firewall config

sudo firewall-cmd --reload

Check the Suricata threads per interface. This is so Suricata doesn’t compete with bro for cpu threads

%YAML 1.1
---
default-rule-path: "/var/lib/suricata/rules"
rule-files:
  - suricata.rules

af-packet:
  - interface: em3
    threads: 4 <---------
    cluster-id: 98
    cluster-type: cluster_flow
    defrag: yes
    use-mmap: yes
    mmap-locked: yes
    #rollover: yes
    tpacket-v3: yes
    use-emergency-flush: yes
default-log-dir: /data/suricata
#52

Thank you guys so much for this. I will work on testing it when I get a chance. Hopefully if all goes well I can work on getting this setup and working. Maybe even in a cluster Whoot Whoot!